@shipi18n/cli 1.0.0 → 1.1.0

package/README.md

@@ -1,9 +1,24 @@
 # @shipi18n/cli
 
+[![npm version](https://img.shields.io/npm/v/@shipi18n/cli)](https://www.npmjs.com/package/@shipi18n/cli)
+[![npm downloads](https://img.shields.io/npm/dw/@shipi18n/cli)](https://www.npmjs.com/package/@shipi18n/cli)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![GitHub last commit](https://img.shields.io/github/last-commit/Shipi18n/shipi18n-cli)](https://github.com/Shipi18n/shipi18n-cli)
+[![CI](https://github.com/Shipi18n/shipi18n-cli/actions/workflows/ci.yml/badge.svg)](https://github.com/Shipi18n/shipi18n-cli/actions)
+[![codecov](https://codecov.io/gh/Shipi18n/shipi18n-cli/branch/main/graph/badge.svg)](https://codecov.io/gh/Shipi18n/shipi18n-cli)
+
 Command-line tool for translating locale files with [Shipi18n](https://shipi18n.com).
 
 > **🚀 Translate JSON files in seconds** - One command, multiple languages!
 
+## Why Shipi18n CLI?
+
+- **Stop copy-pasting into Google Translate** - One command translates to 100+ languages
+- **Placeholders stay intact** - `{name}`, `{{count}}`, `%s` are preserved automatically
+- **Works with your existing setup** - Drop into any React, Vue, Next.js, i18next project
+- **90-day Translation Memory** - Same content? Cached. No extra cost.
+- **Free tier included** - 100 keys, 3 languages, no credit card required
+
 ## Features
 
 - ✅ **Translate JSON files** to 100+ languages with one command
@@ -82,6 +97,8 @@ shipi18n translate <input> [options]
 - `-o, --output <dir>` - Output directory (default: `./locales`)
 - `--api-key <key>` - API key (overrides config)
 - `--preserve-placeholders` - Preserve placeholders (default: `true`)
+- `--no-fallback` - Disable fallback to source for missing translations
+- `--no-regional-fallback` - Disable regional fallback (e.g., pt-BR → pt)
 
 **Examples:**
 
@@ -97,6 +114,48 @@ shipi18n translate ja.json --source ja --target en,es
 
 # Use inline API key
 shipi18n translate en.json --target es --api-key sk_live_...
+
+# Translate with regional variants (pt-BR will fallback to pt if needed)
+shipi18n translate en.json --target es,pt-BR,zh-TW
+
+# Disable fallback (strict mode - fail if translation missing)
+shipi18n translate en.json --target es --no-fallback
+```
+
+### Fallback Behavior
+
+By default, the CLI handles missing translations gracefully:
+
+| Scenario | Default Behavior |
+|----------|-----------------|
+| Missing translation for a language | Uses source content (English) |
+| Missing regional variant (pt-BR) | Falls back to base language (pt), then source |
+| Missing translation for a key | Fills from source content |
+
+**Fallback output example:**
+```
+✓ Translated to 3 languages!
+✓ Saved: ./locales/es.json
+✓ Saved: ./locales/pt-BR.json
+✓ Saved: ./locales/zh-TW.json
+
+Fallback information:
+  • pt-BR → pt (regional fallback)
+  • zh-TW → en (source fallback)
+  • es: 2 keys used fallback
+    - checkout.terms
+    - checkout.privacy
+
+✨ Successfully translated 3 files!
+```
+
+**Disable fallback:**
+```bash
+# Strict mode - no fallback to source
+shipi18n translate en.json --target es --no-fallback
+
+# Disable regional fallback only (pt-BR won't fall back to pt)
+shipi18n translate en.json --target pt-BR --no-regional-fallback
 ```
 
 ### Keys Management
@@ -368,18 +427,40 @@ npm link
 shipi18n --help
 ```
 
-## License
+## Documentation & Resources
 
-MIT
+📚 **Full Documentation:** [shipi18n.com/integrations/cli](https://shipi18n.com/integrations/cli)
 
-## Links
+| Resource | Link |
+|----------|------|
+| **Getting Started** | [shipi18n.com](https://shipi18n.com) |
+| **API Reference** | [shipi18n.com/api](https://shipi18n.com/api) |
+| **i18next Best Practices** | [shipi18n.com/integrations/react](https://shipi18n.com/integrations/react) |
+| **Blog & Tutorials** | [shipi18n.com/blog](https://shipi18n.com/blog) |
 
-- [Shipi18n Website](https://shipi18n.com)
-- [Documentation](https://shipi18n.com/docs)
-- [API Reference](https://shipi18n.com/docs/api)
-- [GitHub](https://github.com/Shipi18n/shipi18n-cli)
-- [Support](https://github.com/Shipi18n/shipi18n-cli/issues)
+## Related Packages
+
+| Package | Description |
+|---------|-------------|
+| [@shipi18n/api](https://www.npmjs.com/package/@shipi18n/api) | Node.js SDK for programmatic use |
+| [vite-plugin-shipi18n](https://www.npmjs.com/package/vite-plugin-shipi18n) | Vite plugin for build-time translation |
+| [i18next-shipi18n-backend](https://www.npmjs.com/package/i18next-shipi18n-backend) | i18next backend for dynamic loading |
+| [shipi18n-github-action](https://github.com/marketplace/actions/shipi18n-auto-translate) | GitHub Action for CI/CD |
+
+## Examples
+
+- [Node.js Example](https://github.com/Shipi18n/shipi18n-nodejs-example) - Basic usage examples
+- [Vue Example](https://github.com/Shipi18n/shipi18n-vue-example) - Vue 3 + vue-i18n integration
+
+## License
+
+MIT
 
 ---
 
-Built with ❤️ by [Shipi18n](https://shipi18n.com) - Smart translation API for developers
+<p align="center">
+  <a href="https://shipi18n.com">shipi18n.com</a> ·
+  <a href="https://github.com/Shipi18n">GitHub</a> ·
+  <a href="https://shipi18n.com/pricing">Pricing</a>
+</p>
+

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@shipi18n/cli",
-  "version": "1.0.0",
+  "version": "1.1.0",
   "description": "Command-line tool for translating locale files with Shipi18n",
   "main": "src/index.js",
   "type": "module",
@@ -19,14 +19,24 @@
     "dev": "node bin/shipi18n.js",
     "build": "echo 'No build step needed for now'",
     "test": "NODE_OPTIONS='--experimental-vm-modules' jest",
-    "test:watch": "NODE_OPTIONS='--experimental-vm-modules' jest --watch"
+    "test:watch": "NODE_OPTIONS='--experimental-vm-modules' jest --watch",
+    "test:coverage": "NODE_OPTIONS='--experimental-vm-modules' jest --coverage"
   },
   "keywords": [
-    "translation",
     "i18n",
-    "internationalization",
+    "translation",
     "localization",
+    "internationalization",
+    "locale",
+    "translate",
     "cli",
+    "i18next",
+    "react-intl",
+    "json-translation",
+    "locale-files",
+    "multilingual",
+    "l10n",
+    "ai-translation",
     "shipi18n"
   ],
   "author": "Shipi18n",

package/src/commands/translate.js

@@ -5,6 +5,73 @@ import { Shipi18nAPI } from '../lib/api.js';
 import { getConfig } from '../lib/config.js';
 import { logger, formatError } from '../utils/logger.js';
 
+/**
+ * Flatten a nested object into dot-notation keys
+ */
+function flattenObject(obj, prefix = '') {
+  const result = {};
+  for (const [key, value] of Object.entries(obj)) {
+    const newKey = prefix ? `${prefix}.${key}` : key;
+    if (value && typeof value === 'object' && !Array.isArray(value)) {
+      Object.assign(result, flattenObject(value, newKey));
+    } else {
+      result[newKey] = value;
+    }
+  }
+  return result;
+}
+
+/**
+ * Unflatten dot-notation keys back into nested object
+ */
+function unflattenObject(obj) {
+  const result = {};
+  for (const [key, value] of Object.entries(obj)) {
+    const keys = key.split('.');
+    let current = result;
+    for (let i = 0; i < keys.length - 1; i++) {
+      if (!current[keys[i]]) {
+        current[keys[i]] = {};
+      }
+      current = current[keys[i]];
+    }
+    current[keys[keys.length - 1]] = value;
+  }
+  return result;
+}
+
+/**
+ * Deep merge two objects
+ */
+function deepMerge(target, source) {
+  const result = { ...target };
+  for (const [key, value] of Object.entries(source)) {
+    if (value && typeof value === 'object' && !Array.isArray(value)) {
+      result[key] = deepMerge(result[key] || {}, value);
+    } else {
+      result[key] = value;
+    }
+  }
+  return result;
+}
+
+/**
+ * Find keys that exist in source but not in target
+ */
+function findMissingKeys(sourceJson, targetJson) {
+  const sourceFlat = flattenObject(sourceJson);
+  const targetFlat = flattenObject(targetJson);
+
+  const missingKeys = {};
+  for (const [key, value] of Object.entries(sourceFlat)) {
+    if (!(key in targetFlat)) {
+      missingKeys[key] = value;
+    }
+  }
+
+  return unflattenObject(missingKeys);
+}
+
 export function translateCommand(program) {
   program
     .command('translate <input>')
@@ -14,6 +81,9 @@ export function translateCommand(program) {
     .option('-o, --output <dir>', 'Output directory', './locales')
     .option('--api-key <key>', 'API key (overrides config)')
     .option('--preserve-placeholders', 'Preserve placeholders like {name}, {{value}}, etc.', true)
+    .option('--no-fallback', 'Disable fallback to source language for missing translations')
+    .option('--no-regional-fallback', 'Disable regional fallback (e.g., pt-BR -> pt)')
+    .option('-i, --incremental', 'Only translate new/missing keys (skip existing translations)')
     .action(async (input, options) => {
       const spinner = logger.spinner('Translating...');
 
@@ -51,38 +121,148 @@ export function translateCommand(program) {
         // Parse target languages
         const targetLanguages = options.target.split(',').map(lang => lang.trim());
         const sourceLanguage = options.source;
+        const outputDir = options.output;
+        const inputFileName = parse(input).name;
+
+        // Incremental mode: load existing translations and find missing keys
+        let jsonToTranslate = json;
+        const existingTranslations = {};
+        let incrementalStats = { total: 0, existing: 0, toTranslate: 0 };
+
+        if (options.incremental) {
+          spinner.text = 'Checking existing translations...';
+
+          const sourceKeyCount = Object.keys(flattenObject(json)).length;
+          incrementalStats.total = sourceKeyCount;
 
-        spinner.text = `Translating to ${targetLanguages.length} language${targetLanguages.length > 1 ? 's' : ''}...`;
+          // Load existing translations for each target language
+          for (const lang of targetLanguages) {
+            const targetFile = join(outputDir, lang, `${inputFileName}.json`);
+            const altTargetFile = join(outputDir, `${lang}.json`);
 
-        // Translate
+            let existingFile = null;
+            if (existsSync(targetFile)) {
+              existingFile = targetFile;
+            } else if (existsSync(altTargetFile)) {
+              existingFile = altTargetFile;
+            }
+
+            if (existingFile) {
+              try {
+                const existingContent = readFileSync(existingFile, 'utf8');
+                existingTranslations[lang] = JSON.parse(existingContent);
+              } catch (e) {
+                logger.warn(`Could not parse ${existingFile}, will re-translate`);
+              }
+            }
+          }
+
+          // Find keys that need translation (missing from ANY target language)
+          const allMissingKeys = {};
+          for (const lang of targetLanguages) {
+            const existing = existingTranslations[lang] || {};
+            const missing = findMissingKeys(json, existing);
+            const missingFlat = flattenObject(missing);
+
+            for (const [key, value] of Object.entries(missingFlat)) {
+              if (!(key in allMissingKeys)) {
+                allMissingKeys[key] = value;
+              }
+            }
+          }
+
+          const missingKeyCount = Object.keys(allMissingKeys).length;
+          incrementalStats.existing = sourceKeyCount - missingKeyCount;
+          incrementalStats.toTranslate = missingKeyCount;
+
+          if (missingKeyCount === 0) {
+            spinner.succeed(chalk.green('All translations up to date!'));
+            logger.log('');
+            logger.log(chalk.gray(`   ${sourceKeyCount} key${sourceKeyCount !== 1 ? 's' : ''} already translated`));
+            return;
+          }
+
+          jsonToTranslate = unflattenObject(allMissingKeys);
+          spinner.text = `Translating ${missingKeyCount} new key${missingKeyCount !== 1 ? 's' : ''} to ${targetLanguages.length} language${targetLanguages.length > 1 ? 's' : ''}...`;
+          logger.log('');
+          logger.info(`Incremental mode: ${chalk.cyan(missingKeyCount)} new key${missingKeyCount !== 1 ? 's' : ''} to translate (${incrementalStats.existing} already exist)`);
+        } else {
+          spinner.text = `Translating to ${targetLanguages.length} language${targetLanguages.length > 1 ? 's' : ''}...`;
+        }
+
+        // Translate with fallback support
         const api = new Shipi18nAPI(apiKey);
         const translations = await api.translateJSON({
-          json,
+          json: jsonToTranslate,
           sourceLanguage,
           targetLanguages,
           preservePlaceholders: options.preservePlaceholders,
+          fallback: {
+            fallbackToSource: options.fallback !== false,
+            regionalFallback: options.regionalFallback !== false,
+          },
         });
 
-        spinner.succeed(chalk.green(`Translated to ${targetLanguages.length} language${targetLanguages.length > 1 ? 's' : ''}!`));
+        const keyCount = Object.keys(flattenObject(jsonToTranslate)).length;
+        spinner.succeed(chalk.green(`Translated ${keyCount} key${keyCount !== 1 ? 's' : ''} to ${targetLanguages.length} language${targetLanguages.length > 1 ? 's' : ''}!`));
 
         // Save translated files
-        const outputDir = options.output;
         if (!existsSync(outputDir)) {
           mkdirSync(outputDir, { recursive: true });
         }
 
         let savedCount = 0;
         for (const [langCode, content] of Object.entries(translations)) {
-          if (langCode === 'warnings') continue;
+          if (langCode === 'warnings' || langCode === 'fallbackInfo' || langCode === 'namespaceInfo') continue;
+
+          // In incremental mode, merge with existing translations
+          let finalContent = content;
+          if (options.incremental && existingTranslations[langCode]) {
+            finalContent = deepMerge(existingTranslations[langCode], content);
+          }
 
           const outputFile = join(outputDir, `${langCode}.json`);
-          writeFileSync(outputFile, JSON.stringify(content, null, 2), 'utf8');
-          logger.success(`Saved: ${chalk.cyan(outputFile)}`);
+          writeFileSync(outputFile, JSON.stringify(finalContent, null, 2), 'utf8');
+          logger.success(`Saved: ${chalk.cyan(outputFile)}${options.incremental ? chalk.gray(' (merged)') : ''}`);
           savedCount++;
         }
 
+        // Show fallback info if any fallbacks were used
+        if (translations.fallbackInfo && translations.fallbackInfo.used) {
+          const fallbackInfo = translations.fallbackInfo;
+          logger.log('');
+          logger.info('Fallback information:');
+
+          // Regional fallbacks
+          if (Object.keys(fallbackInfo.regionalFallbacks).length > 0) {
+            for (const [lang, baseLang] of Object.entries(fallbackInfo.regionalFallbacks)) {
+              logger.log(`  ${chalk.blue('•')} ${lang} → ${baseLang} ${chalk.gray('(regional fallback)')}`);
+            }
+          }
+
+          // Languages that fell back to source
+          if (fallbackInfo.languagesFallbackToSource.length > 0) {
+            for (const lang of fallbackInfo.languagesFallbackToSource) {
+              logger.log(`  ${chalk.yellow('•')} ${lang} → ${sourceLanguage} ${chalk.gray('(source fallback)')}`);
+            }
+          }
+
+          // Keys that used fallback
+          if (Object.keys(fallbackInfo.keysFallback).length > 0) {
+            for (const [lang, keys] of Object.entries(fallbackInfo.keysFallback)) {
+              logger.log(`  ${chalk.yellow('•')} ${lang}: ${keys.length} key${keys.length > 1 ? 's' : ''} used fallback`);
+              if (keys.length <= 5) {
+                keys.forEach(key => {
+                  logger.log(`    ${chalk.gray('- ' + key)}`);
+                });
+              }
+            }
+          }
+        }
+
         // Show warnings if any
         if (translations.warnings && translations.warnings.length > 0) {
+          logger.log('');
           logger.warn('Warnings:');
           translations.warnings.forEach(warning => {
             logger.log(`  ${chalk.yellow('•')} ${warning.message}`);

package/src/lib/api.js

@@ -14,14 +14,39 @@ export class Shipi18nAPI {
 
   /**
    * Translate JSON file
+   * @param {Object} options
+   * @param {Object|string} options.json - JSON content to translate
+   * @param {string} options.sourceLanguage - Source language code
+   * @param {string[]} options.targetLanguages - Target language codes
+   * @param {boolean} options.preservePlaceholders - Preserve placeholders
+   * @param {Object} options.fallback - Fallback options
+   * @param {boolean} options.fallback.fallbackToSource - Use source content when translation missing (default: true)
+   * @param {boolean} options.fallback.regionalFallback - Enable pt-BR -> pt fallback (default: true)
+   * @param {string} options.fallback.fallbackLanguage - Custom fallback language
    */
-  async translateJSON({ json, sourceLanguage = 'en', targetLanguages, preservePlaceholders = true }) {
+  async translateJSON({
+    json,
+    sourceLanguage = 'en',
+    targetLanguages,
+    preservePlaceholders = true,
+    fallback = {}
+  }) {
     if (!this.apiKey) {
       throw new Error('API key is required. Set SHIPI18N_API_KEY or run: shipi18n config set apiKey YOUR_KEY');
     }
 
+    const {
+      fallbackToSource = true,
+      regionalFallback = true,
+      fallbackLanguage,
+    } = fallback;
+
+    const sourceContent = typeof json === 'string' ? JSON.parse(json) : json;
     const jsonString = typeof json === 'string' ? json : JSON.stringify(json);
 
+    // Process regional languages - add base languages for fallback
+    const { processedTargets, regionalMap } = this.processRegionalLanguages(targetLanguages, regionalFallback);
+
     const response = await fetch(`${this.baseUrl}/api/translate`, {
       method: 'POST',
       headers: {
@@ -32,7 +57,7 @@ export class Shipi18nAPI {
         inputMethod: 'text',
         text: jsonString,
         sourceLanguage,
-        targetLanguages: JSON.stringify(targetLanguages),
+        targetLanguages: JSON.stringify(processedTargets),
         preservePlaceholders: String(preservePlaceholders),
       }),
     });
@@ -50,8 +75,8 @@ export class Shipi18nAPI {
     // Parse JSON strings back to objects
     const parsed = {};
     for (const [lang, jsonStr] of Object.entries(result)) {
-      if (lang === 'warnings') {
-        parsed.warnings = jsonStr;
+      if (lang === 'warnings' || lang === 'namespaceInfo') {
+        parsed[lang] = jsonStr;
         continue;
       }
       try {
@@ -61,7 +86,177 @@ export class Shipi18nAPI {
       }
     }
 
-    return parsed;
+    // Apply fallback logic
+    return this.applyFallbacks(
+      parsed,
+      sourceContent,
+      targetLanguages,
+      sourceLanguage,
+      fallbackToSource,
+      regionalFallback,
+      fallbackLanguage,
+      regionalMap
+    );
+  }
+
+  /**
+   * Process regional language codes for fallback support
+   */
+  processRegionalLanguages(targetLanguages, regionalFallback) {
+    const regionalMap = {};
+    const processedTargets = [];
+    const baseLanguagesAdded = new Set();
+
+    for (const lang of targetLanguages) {
+      if (lang.includes('-') && regionalFallback) {
+        const baseLang = lang.split('-')[0];
+        regionalMap[lang] = baseLang;
+
+        if (!baseLanguagesAdded.has(baseLang) && !targetLanguages.includes(baseLang)) {
+          processedTargets.push(baseLang);
+          baseLanguagesAdded.add(baseLang);
+        }
+      }
+
+      if (!processedTargets.includes(lang)) {
+        processedTargets.push(lang);
+      }
+    }
+
+    return { processedTargets, regionalMap };
+  }
+
+  /**
+   * Apply fallback logic to translation results
+   */
+  applyFallbacks(result, sourceContent, targetLanguages, sourceLanguage, fallbackToSource, regionalFallback, fallbackLanguage, regionalMap) {
+    const fallbackInfo = {
+      used: false,
+      languagesFallbackToSource: [],
+      regionalFallbacks: {},
+      keysFallback: {},
+    };
+
+    for (const lang of targetLanguages) {
+      const translation = result[lang];
+
+      // Case 1: Entire language missing
+      if (!translation || Object.keys(translation).length === 0) {
+        // Try regional fallback first
+        if (regionalFallback && regionalMap[lang]) {
+          const baseLang = regionalMap[lang];
+          const baseTranslation = result[baseLang];
+
+          if (baseTranslation && Object.keys(baseTranslation).length > 0) {
+            result[lang] = { ...baseTranslation };
+            fallbackInfo.used = true;
+            fallbackInfo.regionalFallbacks[lang] = baseLang;
+            continue;
+          }
+        }
+
+        // Fall back to source
+        if (fallbackToSource) {
+          result[lang] = { ...sourceContent };
+          fallbackInfo.used = true;
+          fallbackInfo.languagesFallbackToSource.push(lang);
+        }
+        continue;
+      }
+
+      // Case 2: Check for missing keys
+      if (fallbackToSource && typeof translation === 'object') {
+        const missingKeys = this.findMissingKeys(sourceContent, translation);
+
+        if (missingKeys.length > 0) {
+          fallbackInfo.used = true;
+          fallbackInfo.keysFallback[lang] = missingKeys;
+
+          for (const key of missingKeys) {
+            const fallbackValue = this.getNestedValue(sourceContent, key);
+
+            // Try regional fallback first
+            if (regionalFallback && regionalMap[lang]) {
+              const baseLang = regionalMap[lang];
+              const baseTranslation = result[baseLang];
+              const baseValue = baseTranslation ? this.getNestedValue(baseTranslation, key) : undefined;
+
+              if (baseValue !== undefined) {
+                this.setNestedValue(translation, key, baseValue);
+                continue;
+              }
+            }
+
+            if (fallbackValue !== undefined) {
+              this.setNestedValue(translation, key, fallbackValue);
+            }
+          }
+        }
+      }
+    }
+
+    if (fallbackInfo.used) {
+      result.fallbackInfo = fallbackInfo;
+    }
+
+    return result;
+  }
+
+  /**
+   * Find missing keys in translation
+   */
+  findMissingKeys(source, translation, prefix = '') {
+    const missing = [];
+
+    for (const key of Object.keys(source)) {
+      const fullKey = prefix ? `${prefix}.${key}` : key;
+      const sourceValue = source[key];
+      const translationValue = translation[key];
+
+      if (translationValue === undefined || translationValue === null || translationValue === '') {
+        missing.push(fullKey);
+      } else if (
+        typeof sourceValue === 'object' &&
+        sourceValue !== null &&
+        !Array.isArray(sourceValue) &&
+        typeof translationValue === 'object' &&
+        translationValue !== null
+      ) {
+        missing.push(...this.findMissingKeys(sourceValue, translationValue, fullKey));
+      }
+    }
+
+    return missing;
+  }
+
+  /**
+   * Get nested value from object using dot notation
+   */
+  getNestedValue(obj, path) {
+    return path.split('.').reduce((current, key) => {
+      if (current && typeof current === 'object' && key in current) {
+        return current[key];
+      }
+      return undefined;
+    }, obj);
+  }
+
+  /**
+   * Set nested value in object using dot notation
+   */
+  setNestedValue(obj, path, value) {
+    const keys = path.split('.');
+    let current = obj;
+
+    for (let i = 0; i < keys.length - 1; i++) {
+      const key = keys[i];
+      if (!(key in current) || typeof current[key] !== 'object') {
+        current[key] = {};
+      }
+      current = current[key];
+    }
+
+    current[keys[keys.length - 1]] = value;
   }
 
   /**