anylang-dev 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,231 @@
+# anylang-dev
+
+`anylang-dev` is a small bring-your-own-key website translation CLI. It scans your source code for explicit translation calls and writes JSON locale files.
+
+```js
+const title = $tr("home.title", "This would get translated");
+const untouched = "This stays as it is";
+```
+
+It works in JSX and TSX when the text is wrapped in a JavaScript expression:
+
+```tsx
+export function Hero() {
+  const $tr = useAnyLang(language);
+
+  return (
+    <section>
+      <h1>{$tr("home.title", "Welcome back")}</h1>
+      <button aria-label={$tr("actions.saveChanges", "Save changes")}>
+        {$tr("actions.save", "Save")}
+      </button>
+      <p>This plain JSX text stays as it is.</p>
+    </section>
+  );
+}
+```
+
+By default, `anylang` scans `.js`, `.jsx`, `.ts`, `.tsx`, `.vue`, and `.html` files under `src`.
+
+## Language selector
+
+`anylang` does not require a built-in selector. Build any selector UI you want and pass the selected locale to `setLanguage`.
+
+```tsx
+const [language, setSelectedLanguage] = useState<LanguageCode>("en");
+const $tr = useAnyLang(language);
+
+function handleLanguageChange(nextLanguage: LanguageCode) {
+  setLanguage(nextLanguage);
+  setSelectedLanguage(nextLanguage);
+}
+
+return (
+  <select
+    value={language}
+    onChange={(event) => handleLanguageChange(event.target.value as LanguageCode)}
+  >
+    <option value="en">English</option>
+    <option value="hi">हिन्दी</option>
+    <option value="ja">日本語</option>
+  </select>
+);
+```
+
+Use `$tr("key", "source text")` anywhere in the same render tree:
+
+```tsx
+<h1>{$tr("hero.title", "Translate your website with anylang")}</h1>
+```
+
+## Quick start
+
+```bash
+npm link
+anylang init
+anylang scan
+```
+
+`anylang scan` creates locale files without calling a translation provider. To translate for real with Gemini, add your own API key to `.env` in the project where you run `anylang`:
+
+```env
+GEMINI_API_KEY=your-provider-key
+```
+
+Then run:
+
+```bash
+anylang translate
+```
+
+## Config
+
+`anylang init` creates:
+
+```json
+{
+  "sourceLocale": "en",
+  "targetLocales": ["hi"],
+  "include": ["src/**/*.{js,jsx,ts,tsx,vue,html}"],
+  "exclude": ["node_modules", ".git", "dist", "build", ".next"],
+  "outDir": "locales",
+  "runtime": {
+    "output": "src/anylang.generated.ts",
+    "importFrom": "anylang-dev/runtime"
+  },
+  "functionName": "$tr",
+  "provider": {
+    "name": "gemini",
+    "model": "gemini-2.5-flash"
+  }
+}
+```
+
+The provider is intentionally BYOK. `anylang` does not include a platform key, proxy requests, track usage, or store billing data. It automatically loads `.env` from the current project before calling the provider.
+
+## Providers
+
+Choose a provider by setting `provider.name`. Each provider reads its standard API key from `.env`.
+
+| Provider | `provider.name` | `.env` key |
+| --- | --- | --- |
+| Gemini | `gemini` | `GEMINI_API_KEY` |
+| OpenAI | `openai` | `OPENAI_API_KEY` |
+| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` |
+| Cohere | `cohere` | `COHERE_API_KEY` |
+| Mistral | `mistral` | `MISTRAL_API_KEY` |
+| DeepSeek | `deepseek` | `DEEPSEEK_API_KEY` |
+| Groq | `groq` | `GROQ_API_KEY` |
+| OpenRouter | `openrouter` | `OPENROUTER_API_KEY` |
+| Perplexity | `perplexity` | `PERPLEXITY_API_KEY` |
+| xAI | `xai` | `XAI_API_KEY` |
+| Together AI | `together` | `TOGETHER_API_KEY` |
+| Fireworks AI | `fireworks` | `FIREWORKS_API_KEY` |
+| Custom OpenAI-compatible | `openai-compatible` | `ANYLANG_API_KEY` |
+
+Example:
+
+```json
+{
+  "provider": {
+    "name": "anthropic",
+    "model": "claude-3-5-haiku-latest"
+  }
+}
+```
+
+For custom OpenAI-compatible gateways, provide `baseUrl` and `model`:
+
+```json
+{
+  "provider": {
+    "name": "openai-compatible",
+    "baseUrl": "https://your-gateway.example.com/v1",
+    "model": "your-model"
+  }
+}
+```
+
+## Output
+
+Scanning or translating creates:
+
+```text
+locales/
+  en.json
+  hi.json
+  anylang.lock.json
+src/
+  anylang.generated.ts
+```
+
+The lock file stores SHA-256 fingerprints so unchanged strings are skipped on later runs.
+
+## Workflow
+
+1. Wrap source text in your app:
+
+```tsx
+<h1>{$tr("hero.title", "Translate your website with anylang")}</h1>
+```
+
+2. Scan the project:
+
+```bash
+anylang scan
+```
+
+This writes keyed source entries to `locales/en.json` and creates placeholder entries in each target locale.
+It also generates `src/anylang.generated.ts`, which imports all locale JSON files and exports runtime helpers.
+
+3. Translate with Gemini:
+
+```env
+GEMINI_API_KEY=your-gemini-api-key
+```
+
+```bash
+anylang translate
+```
+
+This scans again, sends missing or changed target entries to Gemini, and writes the translated text into files like `locales/hi.json`.
+
+Source locale output:
+
+```json
+{
+  "hero.title": {
+    "text": "Translate your website with anylang",
+    "variables": []
+  }
+}
+```
+
+Target locale output:
+
+```json
+{
+  "hero.title": {
+    "source": "Translate your website with anylang",
+    "text": "anylang से अपनी वेबसाइट का अनुवाद करें",
+    "variables": []
+  }
+}
+```
+
+On later runs, `anylang translate` compares `targetEntry.source` against the current source text. If they differ, it retranslates that key. If they match, it keeps the existing translation and skips the AI call.
+
+## Runtime
+
+Import the generated runtime file in your app:
+
+```tsx
+import {
+  languages,
+  setLanguage,
+  useAnyLang,
+  type LanguageCode
+} from "./anylang.generated";
+```
+
+You do not manually import `en.json`, `hi.json`, `ja.json`, etc. The generated file does that for you based on `sourceLocale` and `targetLocales`.

package/bin/anylang.js

@@ -0,0 +1,8 @@
+#!/usr/bin/env node
+
+import { runCli } from "../src/cli.js";
+
+runCli(process.argv.slice(2)).catch((error) => {
+  console.error(error instanceof Error ? error.message : String(error));
+  process.exitCode = 1;
+});

package/package.json

@@ -0,0 +1,41 @@
+{
+  "name": "anylang-dev",
+  "version": "0.1.0",
+  "description": "Bring-your-own-key website translation JSON generator.",
+  "type": "module",
+  "files": [
+    "bin",
+    "src",
+    "README.md",
+    "LICENSE"
+  ],
+  "bin": {
+    "anylang": "bin/anylang.js"
+  },
+  "exports": {
+    ".": "./src/runtime.js",
+    "./runtime": {
+      "types": "./src/runtime.d.ts",
+      "default": "./src/runtime.js"
+    }
+  },
+  "scripts": {
+    "test": "node --test"
+  },
+  "engines": {
+    "node": ">=20"
+  },
+  "keywords": [
+    "i18n",
+    "translation",
+    "localization",
+    "gemini",
+    "react",
+    "tsx"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+ssh://git@github.com/akshaywritescode/anylang-dev.git"
+  },
+  "license": "MIT"
+}

package/src/cli.js

@@ -0,0 +1,107 @@
+import { mkdir, writeFile } from "node:fs/promises";
+import path from "node:path";
+import { DEFAULT_CONFIG, loadConfig } from "./config.js";
+import { extractProjectStrings } from "./extract.js";
+import { runPipeline } from "./pipeline.js";
+
+const help = `anylang
+
+Commands:
+  init              Create anylang.config.json
+  extract           Print discovered $tr(...) strings
+  scan              Scan source files and prepare locale JSON files
+  translate         Scan source files, translate missing entries, and update JSON files
+  run               Alias for translate
+
+Options:
+  --config <path>   Config path (default: anylang.config.json)
+  --dry-run         Do not call a translation provider
+`;
+
+export async function runCli(argv) {
+  const { command, options } = parseArgs(argv);
+
+  if (!command || command === "help" || command === "--help" || command === "-h") {
+    console.log(help);
+    return;
+  }
+
+  if (command === "init") {
+    await initConfig(options.config);
+    return;
+  }
+
+  if (command === "extract") {
+    const config = await loadConfig(options.config);
+    const result = await extractProjectStrings(config);
+    for (const item of result.items) {
+      console.log(`${item.value}\t${path.relative(process.cwd(), item.file)}:${item.line}:${item.column}`);
+    }
+    console.log(`Found ${result.items.length} translatable string${result.items.length === 1 ? "" : "s"}.`);
+    return;
+  }
+
+  if (command === "scan") {
+    const config = await loadConfig(options.config);
+    const summary = await runPipeline(config, { dryRun: true });
+    console.log(`Scanned ${summary.sourceCount} source string${summary.sourceCount === 1 ? "" : "s"}.`);
+    console.log(`Updated ${summary.localeCount} locale file${summary.localeCount === 1 ? "" : "s"} in ${summary.outDir}.`);
+    console.log("No provider calls were made.");
+    return;
+  }
+
+  if (command === "run" || command === "translate") {
+    const config = await loadConfig(options.config);
+    const summary = await runPipeline(config, { dryRun: options.dryRun });
+    console.log(`Extracted ${summary.sourceCount} source string${summary.sourceCount === 1 ? "" : "s"}.`);
+    console.log(`Updated ${summary.localeCount} locale file${summary.localeCount === 1 ? "" : "s"} in ${summary.outDir}.`);
+    if (summary.translatedCount > 0) {
+      console.log(`Translated ${summary.translatedCount} new/changed entr${summary.translatedCount === 1 ? "y" : "ies"}.`);
+    }
+    if (summary.skippedTranslationCount > 0) {
+      console.log(`Skipped ${summary.skippedTranslationCount} entr${summary.skippedTranslationCount === 1 ? "y" : "ies"} without provider calls.`);
+    }
+    return;
+  }
+
+  throw new Error(`Unknown command: ${command}\n\n${help}`);
+}
+
+function parseArgs(argv) {
+  if (argv.length === 0 || argv[0] === "--help" || argv[0] === "-h") {
+    return { command: argv[0], options: { config: "anylang.config.json", dryRun: false } };
+  }
+
+  const options = {
+    config: "anylang.config.json",
+    dryRun: false
+  };
+  let command;
+
+  for (let index = 0; index < argv.length; index += 1) {
+    const arg = argv[index];
+    if (!command && !arg.startsWith("--")) {
+      command = arg;
+      continue;
+    }
+    if (arg === "--config") {
+      options.config = argv[index + 1];
+      index += 1;
+      continue;
+    }
+    if (arg === "--dry-run") {
+      options.dryRun = true;
+      continue;
+    }
+    throw new Error(`Unknown option: ${arg}`);
+  }
+
+  return { command, options };
+}
+
+async function initConfig(configPath = "anylang.config.json") {
+  const resolved = path.resolve(configPath);
+  await mkdir(path.dirname(resolved), { recursive: true });
+  await writeFile(resolved, `${JSON.stringify(DEFAULT_CONFIG, null, 2)}\n`, { flag: "wx" });
+  console.log(`Created ${path.relative(process.cwd(), resolved)}`);
+}

package/src/config.js

@@ -0,0 +1,51 @@
+import { readFile } from "node:fs/promises";
+import path from "node:path";
+
+export const DEFAULT_CONFIG = {
+  sourceLocale: "en",
+  targetLocales: ["hi"],
+  include: ["src/**/*.{js,jsx,ts,tsx,vue,html}"],
+  exclude: ["node_modules", ".git", "dist", "build", ".next"],
+  outDir: "locales",
+  runtime: {
+    output: "src/anylang.generated.ts",
+    importFrom: "anylang-dev/runtime"
+  },
+  functionName: "$tr",
+  provider: {
+    name: "gemini",
+    baseUrl: "https://generativelanguage.googleapis.com/v1beta",
+    model: "gemini-2.5-flash"
+  }
+};
+
+export async function loadConfig(configPath = "anylang.config.json") {
+  const resolved = path.resolve(configPath);
+  let raw;
+  try {
+    raw = await readFile(resolved, "utf8");
+  } catch (error) {
+    if (error && error.code === "ENOENT") {
+      throw new Error(`Missing config file: ${configPath}. Run "anylang init" first.`);
+    }
+    throw error;
+  }
+
+  const parsed = JSON.parse(raw);
+  const config = {
+    ...DEFAULT_CONFIG,
+    ...parsed,
+    provider: {
+      ...DEFAULT_CONFIG.provider,
+      ...(parsed.provider || {})
+    }
+  };
+
+  if (!config.sourceLocale) throw new Error("Config must include sourceLocale.");
+  if (!Array.isArray(config.targetLocales)) throw new Error("Config targetLocales must be an array.");
+  if (!Array.isArray(config.include)) throw new Error("Config include must be an array.");
+  if (!config.outDir) throw new Error("Config must include outDir.");
+  if (!config.functionName) throw new Error("Config must include functionName.");
+
+  return config;
+}

package/src/env.js

@@ -0,0 +1,32 @@
+import { existsSync, readFileSync } from "node:fs";
+import path from "node:path";
+
+export function loadDotEnv(cwd = process.cwd()) {
+  const envPath = path.join(cwd, ".env");
+  if (!existsSync(envPath)) return;
+
+  const contents = readFileSync(envPath, "utf8");
+  for (const line of contents.split(/\r?\n/)) {
+    const trimmed = line.trim();
+    if (!trimmed || trimmed.startsWith("#")) continue;
+
+    const separatorIndex = trimmed.indexOf("=");
+    if (separatorIndex === -1) continue;
+
+    const key = trimmed.slice(0, separatorIndex).trim();
+    const rawValue = trimmed.slice(separatorIndex + 1).trim();
+    if (!key || process.env[key] !== undefined) continue;
+
+    process.env[key] = unquote(rawValue);
+  }
+}
+
+function unquote(value) {
+  if (
+    (value.startsWith("\"") && value.endsWith("\"")) ||
+    (value.startsWith("'") && value.endsWith("'"))
+  ) {
+    return value.slice(1, -1);
+  }
+  return value;
+}