i18nizer 0.2.0 → 0.4.0-b

package/README.md

@@ -51,7 +51,80 @@ Keys are stored inside:
 
 ---
 
-## ⚡ Usage
+## ⚡ Quick Start
+
+### Initialize Your Project
+
+#### Interactive Mode (Recommended)
+
+```bash
+i18nizer start
+```
+
+This launches an **interactive setup** that will:
+- 🔍 Auto-detect your framework (Next.js or React)
+- 🔍 Auto-detect your i18n library (next-intl, react-i18next, i18next)
+- ❓ Ask you to confirm or change the detected settings
+- ✅ Create `i18nizer.config.yml` with optimal defaults
+- 📁 Set up `.i18nizer/` directory for caching and project data
+- 📂 Create `messages/` directory for translation files
+
+#### Non-Interactive Mode (CI/Automation)
+
+```bash
+i18nizer start --yes
+```
+
+Auto-detects and uses default values without prompts.
+
+#### Manual Configuration
+
+Specify framework and i18n library explicitly:
+
+```bash
+# Next.js with next-intl
+i18nizer start --framework nextjs --i18n next-intl
+
+# React with react-i18next
+i18nizer start --framework react --i18n react-i18next
+
+# Custom setup
+i18nizer start --framework custom --i18n custom
+```
+
+**Available options:**
+- `--framework`: `nextjs`, `react`, `custom`
+- `--i18n`: `next-intl`, `react-i18next`, `i18next`, `custom`
+- `--yes`, `-y`: Skip interactive prompts
+- `--force`, `-f`: Re-initialize existing project
+
+### Translate Your Components
+
+**Translate a single file:**
+
+```bash
+i18nizer translate src/components/Login.tsx --locales en,es,fr
+```
+
+**Translate all components in your project:**
+
+```bash
+i18nizer translate --all --locales en,es,fr
+```
+
+**Preview changes without modifying files:**
+
+```bash
+i18nizer translate <file> --dry-run
+```
+
+**Show generated JSON output:**
+
+```bash
+i18nizer translate <file> --show-json
+```
+
+### Legacy Command (Still Supported)
 
 ```bash
 i18nizer extract <file-path> --locales en,es,fr --provider openai
@@ -102,7 +175,7 @@ export function Login() {
 
 ---
 
-### Generated JSON (`.i18nizer/messages/en/Login.json`)
+### Generated JSON (`messages/en/Login.json`)
 
 ```json
 {
@@ -116,10 +189,30 @@ export function Login() {
 
 ---
 
-## 📂 Output Structure
+## 📂 Project Structure
+
+When initialized with `i18nizer start`:
+
+```
+your-project/
+├─ i18nizer.config.yml       # Configuration file
+├─ .i18nizer/
+│  ├─ cache/
+│  │  └─ translations.json   # Translation cache
+│  └─ ...
+└─ messages/                 # Translation files (configurable path)
+   ├─ en/
+   │  └─ Login.json
+   ├─ es/
+   │  └─ Login.json
+   └─ fr/
+      └─ Login.json
+```
+
+Legacy standalone mode (without `i18nizer start`):
 
 ```
-.i18nizer/
+[HOME]/.i18nizer/
 ├─ api-keys.json
 ├─ tsconfig.json
 └─ messages/
@@ -135,29 +228,87 @@ export function Login() {
 
 ## ✨ Features
 
+### Phase 1 (Current)
+
+- **Project-level integration** with `i18nizer start` and `i18nizer translate`
+- **Configuration system** with `i18nizer.config.yml`
+- **Framework presets** (Next.js + next-intl, React + react-i18next)
+- **Intelligent caching** to avoid redundant AI translation requests
+- **String deduplication** with deterministic key reuse
+- **Configurable behavior** (allowed functions, props, member functions)
+- **Dry-run mode** to preview changes
+- **JSON output preview** with `--show-json`
+- Project-wide or single-file translation
 - Works with **JSX & TSX**
 - Rewrites components automatically (`t("key")`)
 - Always generates **English camelCase keys**
 - Supports **any number of locales**
 - Isolated TypeScript parsing (no project tsconfig required)
-- Friendly logs, and errors
+- Friendly logs with colors and spinners
+
+### Supported Extraction Cases
+
+- **JSX text children**: `<div>Hello</div>`
+- **JSX attributes**: `placeholder`, `title`, `alt`, `aria-label`, `aria-placeholder`, `label`, `text`, `tooltip`, `helperText`
+- **String literals**: `placeholder="Enter name"`
+- **Curly-braced strings**: `placeholder={"Enter name"}`
+- **Template literals**: `` placeholder={`Enter name`} ``
+- **Template literals with placeholders**: `` <p>{`Hello ${name}`}</p> ``
+- **Ternary operators**: `placeholder={condition ? "Text A" : "Text B"}`
+- **Logical AND**: `{condition && "Visible text"}`
+- **Logical OR**: `{condition || "Fallback text"}`
+- **Nested expressions**: Complex combinations of the above
+
+### Filtering & Quality
+
+- **Skips non-translatable content**: Single symbols (`*`, `|`), punctuation-only strings (`...`), whitespace
+- **Deterministic key generation**: Same input always produces the same key
+- **Stable JSON output**: Alphabetically sorted keys, consistent formatting
+- **2-space indentation**: Clean and diff-friendly JSON files
 
 ---
 
 ## 🔮 Roadmap
 
-- Configurable output directory
-- Framework support (Vue, Svelte)
-- i18n library presets (`next-intl`, `react-i18next`)
-- Watch mode
-- Non-AI fallback mode
+### ✅ Phase 0: Foundation & Reliability (Complete)
+- Stable extraction and replacement
+- Deterministic key generation
+- Comprehensive test coverage
+- JSON output quality
+
+### ✅ Phase 1: Project Integration (Complete)
+- `i18nizer start` command for project initialization
+- `i18nizer translate` command with `--all` flag
+- Configuration system with YAML
+- Framework detection and presets
+- Intelligent caching system
+- Cross-file string deduplication
+- Configurable behavior (allowed props, functions, etc.)
+- Dry-run and JSON preview modes
+
+### 🚧 Phase 2: Advanced Features (Planned)
+- [ ] Watch mode for continuous translation
+- [ ] Non-AI fallback mode
+- [ ] Framework support (Vue, Svelte)
+- [ ] Additional i18n library presets
+- [ ] Pluralization support
+- [ ] Context-aware translations
+- [ ] Translation memory and glossary
+
+---
+
+## ⚠️ Current Limitations
+
+- AI-generated keys may vary between runs (deterministic fallback available)
+- Only supports React JSX/TSX (no Vue, Svelte yet)
+- Does not handle runtime-only string generation
 
 ---
 
 ## ⚠️ Notes
 
 - API keys are **never committed**
-- JSON files are stored per project in `.i18nizer/`
+- JSON files are stored in `/{HOME}/.i18nizer/`
 - Designed for incremental adoption
 
 ---

package/bin/dev.js

@@ -1,5 +1,5 @@
-#!/usr/bin/env -S node --loader ts-node/esm --disable-warning=ExperimentalWarning
-
-import {execute} from '@oclif/core'
-
-await execute({development: true, dir: import.meta.url})
+#!/usr/bin/env -S node --loader ts-node/esm --disable-warning=ExperimentalWarning
+
+import {execute} from '@oclif/core'
+
+await execute({development: true, dir: import.meta.url})

package/dist/commands/start.js

@@ -0,0 +1,154 @@
+import { Command, Flags } from "@oclif/core";
+import chalk from "chalk";
+import inquirer from "inquirer";
+import ora from "ora";
+import { detectFramework, detectI18nLibrary, generateConfig, getMessagesDir, getProjectDir, isProjectInitialized, normalizeI18nLibrary, writeConfig, } from "../core/config/config-manager.js";
+export default class Start extends Command {
+    static description = "🚀 Initialize i18nizer in your project with framework presets";
+    static flags = {
+        force: Flags.boolean({
+            char: "f",
+            default: false,
+            description: "Force re-initialization even if config exists",
+        }),
+        framework: Flags.string({
+            description: "Framework (nextjs, react, custom)",
+            options: ["nextjs", "react", "custom"],
+        }),
+        i18n: Flags.string({
+            description: "i18n library (next-intl, react-i18next, i18next, custom)",
+            options: ["next-intl", "react-i18next", "i18next", "custom"],
+        }),
+        preset: Flags.string({
+            char: "p",
+            deprecated: {
+                message: "Use --framework instead",
+                version: "0.4.0",
+            },
+            description: "Framework preset (deprecated, use --framework)",
+            options: ["nextjs", "react", "custom"],
+        }),
+        yes: Flags.boolean({
+            char: "y",
+            default: false,
+            description: "Skip interactive prompts and use detected/default values",
+        }),
+    };
+    async run() {
+        const { flags } = await this.parse(Start);
+        const cwd = process.cwd();
+        // Check if already initialized
+        if (isProjectInitialized(cwd) && !flags.force) {
+            this.log(chalk.yellow("⚠️  Project is already initialized!"));
+            this.log(chalk.cyan("💡 Use"), chalk.bold("--force"), chalk.cyan("to re-initialize"));
+            return;
+        }
+        let framework;
+        let i18nLibrary;
+        // Handle deprecated --preset flag
+        if (flags.preset) {
+            framework = flags.preset;
+        }
+        else if (flags.framework) {
+            framework = flags.framework;
+        }
+        else {
+            // Detect framework and i18n library
+            const detectedFramework = detectFramework(cwd);
+            const detectedI18n = detectI18nLibrary(cwd);
+            if (flags.yes) {
+                // Non-interactive mode: use detected values
+                framework = detectedFramework;
+                i18nLibrary = detectedI18n ?? undefined;
+                this.log(chalk.cyan("🔍 Auto-detected:"));
+                this.log(`  Framework: ${chalk.bold(framework)}`);
+                if (i18nLibrary) {
+                    this.log(`  i18n Library: ${chalk.bold(i18nLibrary)}`);
+                }
+            }
+            else {
+                // Interactive mode: ask user
+                const answers = await inquirer.prompt([
+                    {
+                        choices: [
+                            { name: "Next.js", value: "nextjs" },
+                            { name: "React", value: "react" },
+                            { name: "Custom", value: "custom" },
+                        ],
+                        default: detectedFramework,
+                        message: "What framework are you using?",
+                        name: "framework",
+                        type: "list",
+                    },
+                    {
+                        choices: [
+                            { name: "next-intl", value: "next-intl" },
+                            { name: "react-i18next", value: "react-i18next" },
+                            { name: "i18next", value: "i18next" },
+                            { name: "Custom / None", value: "custom" },
+                        ],
+                        default: detectedI18n ?? "custom",
+                        message: "Which i18n library are you using?",
+                        name: "i18nLibrary",
+                        type: "list",
+                    },
+                ]);
+                framework = answers.framework;
+                i18nLibrary = normalizeI18nLibrary(answers.i18nLibrary);
+            }
+        }
+        // Override with CLI flag if provided
+        if (flags.i18n) {
+            i18nLibrary = normalizeI18nLibrary(flags.i18n);
+        }
+        const spinner = ora("Initializing i18nizer...").start();
+        try {
+            // Generate config
+            spinner.text = "Generating configuration...";
+            const config = generateConfig(framework, i18nLibrary);
+            // Create project directory
+            const projectDir = getProjectDir(cwd);
+            spinner.succeed(`✅ Created ${chalk.cyan(".i18nizer/")} directory`);
+            // Write config file
+            spinner.start("Writing configuration file...");
+            writeConfig(cwd, config);
+            let presetDescription = framework;
+            if (i18nLibrary) {
+                presetDescription += ` + ${i18nLibrary}`;
+            }
+            spinner.succeed(`✅ Created ${chalk.cyan("i18nizer.config.yml")} with ${chalk.bold(presetDescription)} preset`);
+            // Create messages directory
+            spinner.start("Setting up messages directory...");
+            const messagesDir = getMessagesDir(cwd, config);
+            spinner.succeed(`✅ Created ${chalk.cyan(config.messages.path + "/")} directory`);
+            // Summary
+            this.log("");
+            this.log(chalk.green("🎉 i18nizer initialized successfully!"));
+            this.log("");
+            this.log(chalk.bold("📋 Configuration:"));
+            this.log(`  Framework: ${chalk.cyan(config.framework)}`);
+            if (config.i18nLibrary) {
+                this.log(`  i18n Library: ${chalk.cyan(config.i18nLibrary)}`);
+            }
+            this.log(`  i18n Function: ${chalk.cyan(config.i18n.function)}`);
+            this.log(`  Import: ${chalk.cyan(config.i18n.import.named)} from "${chalk.cyan(config.i18n.import.source)}"`);
+            this.log(`  Messages Path: ${chalk.cyan(config.messages.path)}`);
+            this.log(`  Default Locale: ${chalk.cyan(config.messages.defaultLocale)}`);
+            this.log("");
+            this.log(chalk.bold("🚀 Next steps:"));
+            this.log(`  1. Review and customize ${chalk.cyan("i18nizer.config.yml")}`);
+            this.log(`  2. Run ${chalk.cyan("i18nizer translate --all")} to translate all components`);
+            this.log(`  3. Or translate a single file: ${chalk.cyan("i18nizer translate <file>")}`);
+            this.log("");
+        }
+        catch (error) {
+            spinner.fail("❌ Initialization failed");
+            if (error instanceof Error) {
+                this.error(error.message);
+            }
+            else {
+                this.error("An unknown error occurred");
+            }
+        }
+    }
+}

package/dist/commands/translate.js

@@ -0,0 +1,249 @@
+import { Args, Command, Flags } from "@oclif/core";
+import chalk from "chalk";
+import path from "node:path";
+import ora from "ora";
+import { generateTranslations } from "../core/ai/client.js";
+import { buildPrompt } from "../core/ai/promt.js";
+import { extractTexts } from "../core/ast/extract-text.js";
+import { insertUseTranslations } from "../core/ast/insert-user-translations.js";
+import { parseFile } from "../core/ast/parse-file.js";
+import { replaceTempKeysWithT } from "../core/ast/replace-text-with-text.js";
+import { TranslationCache } from "../core/cache/translation-cache.js";
+import { detectFramework, generateConfig, getMessagesDir, getProjectDir, isProjectInitialized, loadConfig, } from "../core/config/config-manager.js";
+import { Deduplicator } from "../core/deduplication/deduplicator.js";
+import { parseAiJson } from "../core/i18n/parse-ai-json.js";
+import { saveSourceFile } from "../core/i18n/sace-source-file.js";
+import { writeLocaleFiles } from "../core/i18n/write-files.js";
+import { findProjectComponents } from "../core/scanner/file-scanner.js";
+const VALID_PROVIDERS = ["gemini", "huggingface", "openai"];
+export default class Translate extends Command {
+    static args = {
+        file: Args.string({
+            description: "Path to a specific TSX/JSX file (optional, use --all for project-level)",
+        }),
+    };
+    static description = "🌍 Extract and translate strings from components (project-level or standalone)";
+    static flags = {
+        all: Flags.boolean({
+            char: "a",
+            default: false,
+            description: "Translate all components in the project",
+        }),
+        "dry-run": Flags.boolean({
+            char: "d",
+            default: false,
+            description: "Preview changes without writing files",
+        }),
+        locales: Flags.string({
+            char: "l",
+            description: "Locales to generate (comma-separated, e.g., en,es,fr)",
+        }),
+        provider: Flags.string({
+            char: "p",
+            description: "AI provider (gemini | huggingface | openai)",
+        }),
+        "show-json": Flags.boolean({
+            char: "j",
+            default: false,
+            description: "Display generated translation JSON output",
+        }),
+    };
+    async run() {
+        const { args, flags } = await this.parse(Translate);
+        const cwd = process.cwd();
+        // Validate input
+        if (!flags.all && !args.file) {
+            this.error("❌ Please specify a file or use --all to translate all components");
+        }
+        if (flags.all && args.file) {
+            this.error("❌ Cannot specify both --all and a file path");
+        }
+        // Load or generate config
+        let config;
+        const isInitialized = isProjectInitialized(cwd);
+        if (isInitialized) {
+            config = loadConfig(cwd);
+            this.log(chalk.cyan("📋 Using project configuration"));
+        }
+        else {
+            // Standalone mode: use defaults
+            const framework = detectFramework(cwd);
+            config = generateConfig(framework);
+            this.log(chalk.yellow("⚠️  Project not initialized, using defaults"));
+            this.log(chalk.cyan("💡 Run"), chalk.bold("i18nizer start"), chalk.cyan("to initialize the project"));
+        }
+        // Override config with flags if provided
+        const locales = flags.locales
+            ? flags.locales.split(",")
+            : [config.messages.defaultLocale, "es"]; // Default fallback
+        let provider = "huggingface";
+        if (flags.provider) {
+            const p = flags.provider.toLowerCase();
+            if (!VALID_PROVIDERS.includes(p)) {
+                this.error(`❌ Invalid provider: ${flags.provider}. Valid options: ${VALID_PROVIDERS.join(", ")}`);
+            }
+            provider = p;
+        }
+        // Get files to process
+        const filesToProcess = flags.all
+            ? findProjectComponents(cwd)
+            : [path.resolve(cwd, args.file)];
+        if (filesToProcess.length === 0) {
+            this.log(chalk.yellow("⚠️  No component files found"));
+            return;
+        }
+        this.log(chalk.cyan("📂 Files to process:"), chalk.bold(filesToProcess.length));
+        this.log(chalk.cyan("🌐 Locales:"), locales.join(", "));
+        this.log(chalk.cyan("🤖 Provider:"), provider);
+        if (flags["dry-run"]) {
+            this.log(chalk.yellow("\n🔍 DRY RUN MODE - No files will be modified\n"));
+        }
+        // Initialize cache and deduplicator
+        const projectDir = getProjectDir(cwd);
+        const cache = new TranslationCache(projectDir);
+        const deduplicator = new Deduplicator(cache);
+        let totalExtracted = 0;
+        let totalReused = 0;
+        let totalCached = 0;
+        // Process each file
+        for (const filePath of filesToProcess) {
+            const componentName = path.basename(filePath).replace(/\.(tsx|jsx)$/, "");
+            const spinner = ora(`Processing ${componentName}...`).start();
+            try {
+                // Parse and extract
+                const sourceFile = parseFile(filePath);
+                const texts = extractTexts(sourceFile, {
+                    allowedFunctions: config.behavior.allowedFunctions,
+                    allowedMemberFunctions: config.behavior.allowedMemberFunctions,
+                    allowedProps: config.behavior.allowedProps,
+                });
+                if (texts.length === 0) {
+                    spinner.info(`⏭️  ${componentName}: No translatable texts found`);
+                    continue;
+                }
+                totalExtracted += texts.length;
+                // Deduplicate and assign keys
+                const mappedTexts = texts.map((t) => {
+                    const result = deduplicator.deduplicate(t.text, componentName, config.behavior.detectDuplicates);
+                    if (result.isReused)
+                        totalReused++;
+                    if (result.isCached)
+                        totalCached++;
+                    return {
+                        key: result.key,
+                        node: t.node,
+                        placeholders: t.placeholders,
+                        tempKey: t.tempKey,
+                        text: t.text,
+                        isCached: result.isCached,
+                    };
+                });
+                // Build translations JSON
+                const i18nJson = {};
+                for (const mapped of mappedTexts) {
+                    i18nJson[mapped.key] = {};
+                    if (mapped.isCached) {
+                        // Use cached translations
+                        const cached = cache.get(mapped.text);
+                        for (const locale of locales) {
+                            i18nJson[mapped.key][locale] =
+                                cached.locales[locale] ?? mapped.text;
+                        }
+                    }
+                    else {
+                        // Will need AI translation
+                        for (const locale of locales) {
+                            i18nJson[mapped.key][locale] = ""; // Placeholder
+                        }
+                    }
+                }
+                // Get AI translations for non-cached texts
+                const textsNeedingTranslation = mappedTexts.filter((t) => !t.isCached);
+                if (textsNeedingTranslation.length > 0) {
+                    spinner.text = `${componentName}: Generating translations with ${provider}...`;
+                    const prompt = buildPrompt({
+                        componentName,
+                        locales,
+                        texts: textsNeedingTranslation.map((t) => ({
+                            tempKey: t.tempKey,
+                            text: t.text,
+                        })),
+                    });
+                    const raw = await generateTranslations(prompt, provider);
+                    if (!raw)
+                        throw new Error("AI did not return any data");
+                    const aiJson = parseAiJson(raw);
+                    const namespace = aiJson[componentName];
+                    // Merge AI translations
+                    for (const mapped of textsNeedingTranslation) {
+                        const aiTranslations = namespace[mapped.tempKey];
+                        if (aiTranslations) {
+                            const aiKey = aiTranslations.key;
+                            for (const locale of locales) {
+                                i18nJson[mapped.key][locale] = aiTranslations[locale] ?? mapped.text;
+                            }
+                            // Update cache
+                            cache.set({
+                                componentName,
+                                key: mapped.key,
+                                locales: i18nJson[mapped.key],
+                                text: mapped.text,
+                            });
+                        }
+                    }
+                }
+                // Show JSON if requested
+                if (flags["show-json"]) {
+                    this.log("\n" + chalk.cyan(`${componentName} JSON:`));
+                    this.log(JSON.stringify({ [componentName]: i18nJson }, null, 2));
+                    this.log("");
+                }
+                // Write files (unless dry-run)
+                if (!flags["dry-run"]) {
+                    // Write locale files
+                    if (isInitialized) {
+                        const messagesDir = getMessagesDir(cwd, config);
+                        writeLocaleFiles(componentName, { [componentName]: i18nJson }, locales, messagesDir);
+                    }
+                    else {
+                        // Standalone mode: use home directory
+                        writeLocaleFiles(componentName, { [componentName]: i18nJson }, locales);
+                    }
+                    // Rewrite component
+                    insertUseTranslations(sourceFile, componentName);
+                    replaceTempKeysWithT(mappedTexts.map((m) => ({
+                        key: m.key,
+                        node: m.node,
+                        placeholders: m.placeholders,
+                        tempKey: m.tempKey,
+                    })), {
+                        allowedFunctions: config.behavior.allowedFunctions,
+                        allowedMemberFunctions: config.behavior.allowedMemberFunctions,
+                        allowedProps: config.behavior.allowedProps,
+                    });
+                    saveSourceFile(sourceFile);
+                }
+                spinner.succeed(`✅ ${componentName}: ${texts.length} strings (${totalReused} reused, ${totalCached} cached)`);
+            }
+            catch (error) {
+                spinner.fail(`❌ ${componentName}: Failed`);
+                if (error instanceof Error) {
+                    this.error(error.message);
+                }
+            }
+        }
+        // Save cache
+        if (!flags["dry-run"]) {
+            cache.save();
+        }
+        // Summary
+        this.log("");
+        this.log(chalk.green("🎉 Translation complete!"));
+        this.log(chalk.cyan("📊 Summary:"));
+        this.log(`  Files processed: ${chalk.bold(filesToProcess.length)}`);
+        this.log(`  Strings extracted: ${chalk.bold(totalExtracted)}`);
+        this.log(`  Keys reused: ${chalk.bold(totalReused)}`);
+        this.log(`  Cached translations: ${chalk.bold(totalCached)}`);
+        this.log("");
+    }
+}