shallok 0.1.1 → 0.1.3

package/skills/react-localization-and-translation-i18n/references/translation.md

@@ -1,11 +1,37 @@
 # Translation Engine & Agent-Native Translation Reference
 
 ## Agent-Native Translation (Primary Workflow)
-When using Shallok via an AI coding agent (e.g., in Claude, Cursor, Copilot, Antigravity):
+When using Shallok via an AI coding agent (e.g., in Antigravity, Claude, Cursor, Copilot):
 - **No external API keys or LLM provider subscriptions are needed.**
 - The AI agent itself generates accurate, natural translations for each target locale.
-- The AI agent inspects `messages/<locale>.json` or runs `localize status --json` to identify entries with `status: "missing"` or `status: "stale"`.
-- The AI agent writes the translated strings into `messages/<locale>.json` (or uses `localize entry set <locale> <key> "<translated_text>"`).
+- **Always verify context first**: Before translating any key, check that all keys in `localization_base/<source_locale>.json` have proper semantic context. If any key lacks context or has generic AST context, generate rich context before translating.
+- **Always translate from `localization_base/<source_locale>.json` and its context**: Never translate raw isolated string fragments. Combine the canonical source `content` and its semantic `context` to generate culturally authentic, grammatically correct translations.
+- Inspect `messages/<locale>.json` or run `shallok status --json` to identify entries with `status: "missing"` or `status: "stale"`.
+- Write the translated strings into `messages/<locale>.json` (or use `shallok entry set <locale> <key> "<translated_text>"`).
+
+## Core Rules for Translations
+
+### 1. Pre-Flight Verification & Persistence
+- Before making changes to a new or existing repo, verify `shallok` is installed and check that `localization.json` and `localization_base/` exist. If not, initialize with `shallok init`.
+- **NEVER delete `localization.json` or `localization_base/`**. They hold the single source of truth, content hashes, version counters, and context.
+
+### 2. Normalized Dot-Paths for Keys
+- All keys must use normalized dot notation (e.g., `"checkout.pay_now"`, `"auth.login.title"`).
+- Never use nested object structures. If an existing codebase uses nested objects for keys, ask the user to convert/flatten them to normalized dot-path keys.
+
+### 3. Standardized `messages/` Folder
+- All translations must reside in `messages/` (`messages/<locale>.json` and `messages/runtime/<locale>.json`).
+- If an existing project uses another directory (such as `locales/`, `src/locales/`, `translations/`), ask the user to switch to `messages/` and migrate existing files.
+
+### 4. Context Verification Before Translation
+- Check all keys in `localization_base/<source_locale>.json` have meaningful semantic context.
+- If any key lacks context, generate clear context describing the screen, element, and user action before initiating translation.
+
+### 5. Always Translate from Source Locale Catalog + Context
+- Read both `content` and `context` from `localization_base/<source_locale>.json`.
+- Preserve all interpolation placeholders (`{name}`, `{count}`, `{amount}`) exactly as formatted.
+
+---
 
 ### Translation Schema
 Each entry in `messages/<locale>.json` maintains:
@@ -35,7 +61,7 @@ The AI agent must strictly preserve all interpolation placeholders in target tra
 - Kannada: `"ಮರಳಿ ಸ್ವಾಗತ, {userName}! ನಿಮ್ಮ ಕಾರ್ಟ್‌ನಲ್ಲಿ {itemCount} ಐಟಂಗಳು ಇವೆ."`
 - Hindi: `"वापसी पर स्वागत है, {userName}! आपके कार्ट में {itemCount} आइटम हैं।"`
 
-If a translation drops a placeholder or introduces an invalid placeholder, `localize validate` will fail with an error.
+If a translation drops a placeholder or introduces an invalid placeholder, `shallok validate` will fail with an error.
 
 ---
 

package/skills/react-localization-and-translation-i18n/references/versioning.md

@@ -1,8 +1,22 @@
 # Per-Key Versioning & Context Reference
 
+## Strict Catalog Persistence: NEVER Delete `localization.json` or `localization_base/`
+> [!CAUTION]
+> `localization.json` and `localization_base/` are the permanent foundation of the localization system:
+> - `localization.json` stores project settings, locales, paths, and key strategies.
+> - `localization_base/` stores canonical source strings, per-key version counters (`key_version`), cryptographic SHA-256 hashes (`content_hash`), occurrence positions, and enriched semantic `context`.
+> - **Under NO circumstances should you EVER delete `localization.json` or `localization_base/`**. Deleting them resets version counters, destroys human- and AI-enriched context, breaks incremental translation, and corrupts catalog synchronization.
+
+---
+
 ## Canonical Source Catalog (`localization_base/`)
 The canonical catalog is the single source of truth for all active and obsolete keys, including their cryptographic content hashes and version counters.
 
+### Key Format: Normalized Dot-Separated Paths Only
+- All keys must use normalized dot-separated string paths (e.g. `"checkout.pay_now"`, `"auth.login.title"`).
+- **Never use nested object-based keys**.
+- If an existing project uses object-based keys, ask the user to convert and flatten them to normalized dot-paths before modifying catalogs.
+
 ### Structure of a Canonical Entry
 ```json
 {
@@ -26,12 +40,14 @@ The canonical catalog is the single source of truth for all active and obsolete
 }
 ```
 
-## Context Enrichment by AI Agents
-1. When strings are extracted via `localize extract`, the CLI provides baseline AST context (e.g. `JSX text in <button> (in App <button>)`).
-2. The AI Agent analyzes the source code, component hierarchy, and user flow, and enriches the `context` with semantic clarity:
-   - CLI method: `localize context set <key> "<semantic_description>"`
+## Context Enrichment & Verification Before Translation
+1. When strings are extracted via `shallok extract`, the CLI provides baseline AST context (e.g. `JSX text in <button> (in App <button>)`).
+2. **Mandatory Check Before Translation**: Before generating any translations, check that all active keys in `localization_base/<source_locale>.json` have proper, high-quality semantic context.
+3. If any key lacks context or has only generic AST descriptions, analyze the source code, component hierarchy, and user flow, and enrich the `context` with semantic clarity:
+   - CLI method: `shallok context set <key> "<semantic_description>"`
    - Direct edit: Update `"context"` in `localization_base/<source_locale>.json`.
-3. **Preservation Guarantee**:
+4. **Always Translate from Source Locale + Context**: Translations must be derived directly from `localization_base/<source_locale>.json` using BOTH `content` and `context`.
+5. **Preservation Guarantee**:
    Shallok's reconciliation algorithm (`reconcileCanonicalCatalog`) checks if a key already has an enriched context. Unless the source code introduces an explicit inline comment (`{/* context: ... */}`), the AI agent's enriched context is **strictly preserved** across all future extractions and will not be overwritten by generic AST node descriptions.
 
 ## Reconciliation & Versioning Rules
@@ -54,4 +70,4 @@ In `messages/<locale>.json`, each entry records:
   }
 }
 ```
-A translation is **stale** when `entry.source_version < canonicalEntry.key_version`. Stale entries retain their existing translated value for reference until the AI agent updates them.
+A translation is **stale** when `entry.source_version < canonicalEntry.key_version`. Stale entries retain their existing translated value for reference until the AI agent updates them from `localization_base/<source_locale>.json` and its enriched `context`.

package/skills/localize-app/SKILL.md

@@ -1,80 +0,0 @@
----
-name: localize-app
-description: Instructions and workflows for AI coding agents to inspect, initialize, extract, version, enrich context, translate, and maintain localization in web applications without requiring external translation API keys.
----
-
-# Localize App Skill (Agent-Native Workflow)
-
-Use this skill when asked to add, configure, update, or audit localization in a project (e.g., *"Add localization to this application"*, *"Add Kannada and Hindi"*, *"Translate new strings"*, *"Update stale translations"*).
-
-## Two Ways Developers Use Shallok
-
-1. **Via CLI (`npx shallok init`)**: The developer runs `npx shallok init` in their project. Shallok initializes the catalogs, installs `shallok` into `devDependencies`, and adds this skill directly into `.agents/skills/` and `skills/`.
-2. **Direct Skill Installation**: The developer installs or links this skill into their AI coding agent and directly asks: *"Add localization to this application"* or *"Translate into Kannada and Hindi"*.
-
----
-
-## Agent Step 0: Verify & Install Shallok CLI
-
-**Before executing any localization commands, you (the AI Agent) MUST verify that `shallok` is available:**
-
-1. Check `package.json` for `"shallok"` in `devDependencies` or run:
-   ```bash
-   npx shallok --version
-   ```
-2. **If `shallok` is NOT installed, install it immediately** using the project's package manager:
-   - If `pnpm-lock.yaml` exists: `pnpm add -D shallok`
-   - If `yarn.lock` exists: `yarn add -D shallok`
-   - If `bun.lockb` / `bun.lock` exists: `bun add -d shallok`
-   - Otherwise: `npm install -D shallok`
-   *(Alternatively, run commands directly via `npx shallok <command>`)*
-
----
-
-## Core Architecture & Agent Role
-
-Shallok pairs a deterministic AST CLI (`shallok`) with **the AI Agent as the translator and context enricher**:
-
-1. **The CLI (`shallok`)** handles deterministic tasks:
-   - AST extraction and safe code transformations (`shallok extract`).
-   - Per-key versioning and SHA-256 hash reconciliation (`localization_base/`).
-   - Runtime message compilation (`shallok build`).
-   - Placeholder and catalog integrity checks (`shallok validate`).
-
-2. **The AI Agent (You)** handles semantic intelligence:
-   - **No external API keys or LLM provider keys needed!** You, as the AI agent, generate the translations directly.
-   - **Context Enrichment**: You analyze the UI components, user flows, and screen layouts to provide rich, meaningful semantic `context` in `localization_base/` for each key.
-   - **Culturally Appropriate Translations**: You translate the strings into configured target locales while strictly preserving interpolation tokens (e.g., `{count}`, `{name}`).
-
----
-
-## 6-Phase Agent Workflow
-
-### Phase A: Analyze
-Execute read-only repository inspection before making changes:
-```bash
-shallok analyze --json
-```
-
-### Phase B: Plan and Review
-Present a concise plan to the developer (framework, router, locales, scaffolding).
-
-### Phase C: Initialize
-```bash
-shallok init --source en --locales kn,hi --framework <name> --yes
-```
-
-### Phase D: Extract & Enrich Context
-1. Preview proposed extraction: `shallok extract --dry-run --json`
-2. Extract strings: `shallok extract`
-3. Enrich Context: `shallok context set <key> "<semantic_description>"`
-
-### Phase E: Translate (AI-Driven)
-1. Check status: `shallok status --json`
-2. Generate Natural Translations preserving `{tokens}`.
-3. Update catalog directly or via: `shallok entry set <locale> <key> "<translated_text>"`
-
-### Phase F: Apply, Build & Validate
-1. Transform source components: `shallok extract --apply`
-2. Compile runtime message bundles: `shallok build`
-3. Validate catalog integrity: `shallok validate`

package/skills/localize-app/references/extraction.md

@@ -1,29 +0,0 @@
-# AST Extraction Reference
-
-## Extraction Strategy
-Shallok uses Babel AST parsing (`@babel/parser` and `@babel/traverse`) with full JSX and TypeScript support to safely inspect and transform source files without regular expressions.
-
-## What is Extracted
-1. **JSX Text Nodes**:
-   - `<h1>Welcome</h1>` → `{t('home.welcome')}`
-   - `<button>Pay now</button>` → `<button>{t('checkout.pay_now')}</button>`
-2. **User-Facing Attributes**:
-   - `placeholder="Enter your email"`
-   - `aria-label="Submit login form"`
-   - `title="Cart summary"`
-   - `alt="Product thumbnail"`
-   - `label="Password"`
-3. **Conditional UI Expressions**:
-   - `{isMember ? 'VIP Member' : 'Guest User'}`
-4. **Template Literals with Variables**:
-   - `Hello, ${name}!` → `Hello, {name}!` with variable preservation.
-
-## What is Explicitly Ignored
-- Technical attributes: `className`, `id`, `name`, `key`, `ref`, `type`, `href`, `src`, `data-testid`, `role`.
-- Console and logger calls: `console.log(...)`, `console.error(...)`, `logger.debug(...)`.
-- URLs, file paths, hex colors, CSS units.
-- Non-user-facing strings, code constants, and symbols.
-- Already localized calls: `t(...)`, `i18n.t(...)`.
-
-## Dry-Run Mode
-Always use `localize extract --dry-run` to preview keys and occurrences before committing changes to source files.

package/skills/localize-app/references/frameworks.md

@@ -1,25 +0,0 @@
-# Framework Adapters Reference
-
-## Supported Frameworks
-
-### 1. React (`react-i18next`)
-- **Initialization**: Generates `src/i18n.ts` configuring `i18next` with `initReactI18next`.
-- **Runtime Messages**: Generates clean key-value message files in `messages/<locale>.json`.
-- **AST Transformation**:
-  - Adds `import { useTranslation } from 'react-i18next';`
-  - Injects `const { t } = useTranslation();` in components
-  - Replaces text with `{t('key')}`
-
-### 2. Next.js (`next-intl`)
-- **App Router Support**:
-  - Generates `src/i18n/routing.ts` defining `locales` and `defaultLocale`.
-  - Generates `src/i18n/request.ts` using `getRequestConfig`.
-- **Runtime Messages**: Generates runtime message files in `messages/<locale>.json`.
-- **AST Transformation**:
-  - Adds `import { useTranslations } from 'next-intl';`
-  - Injects `const t = useTranslations();` in components
-  - Replaces text with `{t('key')}`
-
-### 3. Vue (`vue-i18n`)
-- **Initialization**: Generates `src/i18n.ts` using `createI18n`.
-- **Runtime Messages**: Generates nested runtime message files in `messages/<locale>.json`.

package/skills/localize-app/references/translation.md

@@ -1,45 +0,0 @@
-# Translation Engine & Agent-Native Translation Reference
-
-## Agent-Native Translation (Primary Workflow)
-When using Shallok via an AI coding agent (e.g., in Claude, Cursor, Copilot, Antigravity):
-- **No external API keys or LLM provider subscriptions are needed.**
-- The AI agent itself generates accurate, natural translations for each target locale.
-- The AI agent inspects `messages/<locale>.json` or runs `localize status --json` to identify entries with `status: "missing"` or `status: "stale"`.
-- The AI agent writes the translated strings into `messages/<locale>.json` (or uses `localize entry set <locale> <key> "<translated_text>"`).
-
-### Translation Schema
-Each entry in `messages/<locale>.json` maintains:
-```json
-{
-  "checkout.pay_now": {
-    "value": "ಈಗ ಪಾವತಿಸಿ",
-    "source_version": 1,
-    "translated_at": "2026-10-19T14:00:00.000Z",
-    "status": "translated"
-  }
-}
-```
-
-### Supported Statuses
-- `missing`: Key exists in canonical catalog but has no translation.
-- `stale`: Key source version was incremented; old translation preserved for reference.
-- `translated`: Translated and matching the current canonical `source_version`.
-- `review_required`: Flagged for human review.
-- `approved`: Formally approved translation.
-
----
-
-## Placeholder Preservation Rules
-The AI agent must strictly preserve all interpolation placeholders in target translations:
-- Source: `"Welcome back, {userName}! You have {itemCount} items in your cart."`
-- Kannada: `"ಮರಳಿ ಸ್ವಾಗತ, {userName}! ನಿಮ್ಮ ಕಾರ್ಟ್‌ನಲ್ಲಿ {itemCount} ಐಟಂಗಳು ಇವೆ."`
-- Hindi: `"वापसी पर स्वागत है, {userName}! आपके कार्ट में {itemCount} आइटम हैं।"`
-
-If a translation drops a placeholder or introduces an invalid placeholder, `localize validate` will fail with an error.
-
----
-
-## CLI-Based Translation (Optional / Fallback)
-For automated pipelines or scripts running outside an interactive AI agent:
-1. **Mock Provider (`mock`)**: Default provider with built-in dictionary for local development and CI testing.
-2. **LLM Provider (`llm`)**: Calls OpenAI, Gemini, or Anthropic HTTP APIs when `TRANSLATION_API_KEY` is provided.

package/skills/localize-app/references/versioning.md

@@ -1,57 +0,0 @@
-# Per-Key Versioning & Context Reference
-
-## Canonical Source Catalog (`localization_base/`)
-The canonical catalog is the single source of truth for all active and obsolete keys, including their cryptographic content hashes and version counters.
-
-### Structure of a Canonical Entry
-```json
-{
-  "checkout.pay_now": {
-    "key": "checkout.pay_now",
-    "content": "Pay now",
-    "context": "Primary action button on checkout screen to trigger payment modal",
-    "relative_file_path": "src/components/checkout.tsx",
-    "occurrences": [
-      {
-        "relative_file_path": "src/components/checkout.tsx",
-        "line": 42,
-        "column": 12
-      }
-    ],
-    "last_updated_at": "2026-10-19T10:00:00.000Z",
-    "key_version": 1,
-    "content_hash": "sha256:...",
-    "status": "active"
-  }
-}
-```
-
-## Context Enrichment by AI Agents
-1. When strings are extracted via `localize extract`, the CLI provides baseline AST context (e.g. `JSX text in <button> (in App <button>)`).
-2. The AI Agent analyzes the source code, component hierarchy, and user flow, and enriches the `context` with semantic clarity:
-   - CLI method: `localize context set <key> "<semantic_description>"`
-   - Direct edit: Update `"context"` in `localization_base/<source_locale>.json`.
-3. **Preservation Guarantee**:
-   Shallok's reconciliation algorithm (`reconcileCanonicalCatalog`) checks if a key already has an enriched context. Unless the source code introduces an explicit inline comment (`{/* context: ... */}`), the AI agent's enriched context is **strictly preserved** across all future extractions and will not be overwritten by generic AST node descriptions.
-
-## Reconciliation & Versioning Rules
-1. **New Key**: Initialized at `key_version = 1`, `status = "active"`.
-2. **Unchanged Content & Context**: `key_version` and `last_updated_at` remain unchanged.
-3. **Modified Source Content**: `key_version` increments by 1; `last_updated_at` is updated.
-4. **File Move / Occurrence Change**: Path and occurrence lines are updated; `key_version` does NOT increment.
-5. **Key Removed from Source**: Marked `status = "obsolete"`; preserved in catalog for historical retention.
-6. **Key Reintroduced**: Reactivated as `"active"`.
-
-## Stale Detection in Locale Catalogs
-In `messages/<locale>.json`, each entry records:
-```json
-{
-  "checkout.pay_now": {
-    "value": "ಈಗ ಪಾವತಿಸಿ",
-    "source_version": 1,
-    "translated_at": "2026-10-19T10:05:00.000Z",
-    "status": "translated"
-  }
-}
-```
-A translation is **stale** when `entry.source_version < canonicalEntry.key_version`. Stale entries retain their existing translated value for reference until the AI agent updates them.