shallok 0.1.1 → 0.1.3

package/skills/localization-and-translation-i18n/SKILL.md

@@ -14,20 +14,95 @@ Use this skill when asked to add, configure, update, or audit localization in **
 
 ---
 
-## Agent Step 0: Verify & Install Shallok CLI
+## Agent Step 0: Mandatory Pre-Flight Checklist (Run Before ANY Code or Config Changes)
 
-**Before executing any localization commands, you (the AI Agent) MUST verify that `shallok` is available:**
+**Before making ANY changes to a new repo or an existing repo, you (the AI Agent) MUST execute the following checks in order:**
 
-1. Check `package.json` for `"shallok"` in `devDependencies` or run:
-   ```bash
-   npx shallok --version
-   ```
-2. **If `shallok` is NOT installed, install it immediately** using the project's package manager:
-   - If `pnpm-lock.yaml` exists: `pnpm add -D shallok`
-   - If `yarn.lock` exists: `yarn add -D shallok`
-   - If `bun.lockb` / `bun.lock` exists: `bun add -d shallok`
-   - Otherwise: `npm install -D shallok`
-   *(Alternatively, run commands directly via `npx shallok <command>`)*
+### 1. Check if Shallok is Installed
+- Inspect `package.json` for `"shallok"` or `"@shallok/cli"` in `devDependencies` (or `dependencies`), or run:
+  ```bash
+  npx shallok --version
+  ```
+- **If `shallok` is NOT installed, install it immediately into `devDependencies`:**
+  - If `pnpm-lock.yaml` exists: `pnpm add -D shallok`
+  - If `yarn.lock` exists: `yarn add -D shallok`
+  - If `bun.lockb` / `bun.lock` exists: `bun add -d shallok`
+  - Otherwise: `npm install -D shallok`
+
+### 2. Check if `localization.json` and `localization_base/` Exist
+- Verify that `localization.json` exists in the project root.
+- Verify that the `localization_base/` directory exists with `<source_locale>.json` and `metadata.json`.
+- **If either does NOT exist, create them using Shallok:**
+  ```bash
+  shallok init --source en --locales kn,hi --framework <name> --yes
+  ```
+
+### 3. NEVER Delete `localization.json` or `localization_base/`
+> [!CAUTION]
+> **CRITICAL PERSISTENCE RULE: NEVER DELETE `localization.json` OR `localization_base/`!**
+> - `localization.json` is the single source-of-truth project manifest.
+> - `localization_base/` contains the canonical catalog (`<source_locale>.json`) with per-key version numbers (`key_version`), cryptographic content hashes (`sha256:`), source code occurrences, and enriched semantic context, alongside `metadata.json`.
+> - **Under NO circumstances should you EVER delete `localization.json` or `localization_base/`**. Deleting them destroys the versioning history, wipes out human- and AI-enriched context, resets version counters, breaks incremental translation, and corrupts catalog synchronization.
+
+---
+
+## Core Architectural & Formatting Rules
+
+### Rule 1: Always Use Normalized Paths for Keys (Never Object-Based Paths)
+- **Always use normalized dot-separated key paths**: e.g., `"auth.login.title"`, `"checkout.pay_now"`, `"settings.profile.change_password"`.
+- **Never use nested object-based paths**: Do not structure keys or catalogs as nested JSON objects (e.g., `{ "auth": { "login": { "title": "..." } } }`). All keys in `localization_base/<source_locale>.json`, `messages/<locale>.json`, and runtime bundles MUST be flat normalized string paths.
+- **If an existing project is using object-based keys**:
+  - **STOP and ask the user** for confirmation to flatten them into normalized dot-separated key paths before proceeding with extraction or catalog edits.
+  - Explain that normalized dot-paths are required for deterministic AST extraction, per-key versioning, and collision-free reconciliation.
+  - Upon user approval, migrate and flatten all nested keys to normalized dot-paths.
+
+### Rule 2: Always Use the `messages/` Directory Structure (Ask User & Migrate If Different)
+- **Standard translations folder**: Shallok strictly standardizes on `messages/` for authoritative locale catalogs (`messages/<locale>.json`) and compiled runtime bundles (`messages/runtime/<locale>.json`).
+- **If the project uses any other folder structure** (e.g., `locales/`, `src/locales/`, `translations/`, `lang/`, `src/i18n/locales/`):
+  - **STOP and ask the user** for approval to switch to the standardized `messages/` directory structure.
+  - Upon user approval, **migrate** all existing translation files into `messages/`, update framework configuration and import paths, reconcile existing strings into `localization_base/`, and clean up the old directory.
+
+### Rule 3: Check Context for All Keys in `localization_base/{source_locale}` Before Translating
+- **Before generating any translations for any target locale**:
+  - Inspect all active keys in `localization_base/<source_locale>.json` (e.g., `localization_base/en.json`).
+  - Verify that **every key has meaningful, accurate semantic `context`**.
+  - If any key lacks context, has an empty context (`""`), or contains only generic AST descriptions (e.g., `"JSX text in button"`), you MUST inspect the component/screen and **generate rich semantic context for all such keys BEFORE generating translations**.
+  - Enrich context via:
+    ```bash
+    shallok context set <key> "<semantic_description>"
+    ```
+    or directly edit the `"context"` field in `localization_base/<source_locale>.json`.
+
+### Rule 4: Always Generate Translations from `localization_base/{source_locale}` and Its Context
+- **Never translate raw strings in isolation**:
+  - All translations MUST be generated directly from `localization_base/<source_locale>.json`, reading BOTH the source string (`content`) and its semantic `context`.
+  - The `context` provides critical disambiguation (grammatical gender, button vs label vs heading, formality, technical terminology).
+  - Strictly preserve all interpolation tokens (`{userName}`, `{count}`, `{amount}`) exactly as formatted in the source `content`.
+
+### Rule 5: Migrate Legacy Base Locale References to Specified Structure
+- **If the application already has a localization setup**:
+  - Inspect how the base/source locale (`source_locale`, e.g., `en`) is loaded.
+  - If any configuration or code file (e.g., `i18n.ts`, `src/i18n.ts`, `src/i18n/request.ts`, `app/_layout.tsx`, `main.tsx`) is importing or referencing a legacy file other than our specified structure (e.g., `src/locales/en.json`, `locales/en.json`, `public/locales/en.json`):
+    1. **Reconcile legacy base strings** into `localization_base/<source_locale>.json` and `messages/<source_locale>.json`.
+    2. **Update all code imports and references** across the project to point to the standardized compiled runtime bundle (`messages/runtime/<locale>.json`) or the CDN loader.
+    3. Remove or deprecate the legacy base locale file so that Shallok's AST extraction does NOT falsely extract the legacy JSON file as source code.
+
+### Rule 6: Translation Loading Strategy: Local for Static/SSR vs. CDN for Client/Mobile
+Choose the loading strategy based on application architecture:
+- **Static & SSR Applications (Astro, Next.js, and similar static/SSR apps)**:
+  - **Recommend keeping translations LOCAL** in `messages/runtime/<locale>.json`.
+  - Load directly from the local bundle or filesystem during server rendering or static build.
+  - **Benefits**: Zero network latency, instant First Contentful Paint (FCP), seamless Server Component hydration, and zero remote network failure risks.
+- **Client & Mobile Applications (Expo, React Native, React SPA / Vite)**:
+  - **Recommend loading translations from a CDN**:
+    - For mobile apps and client SPAs, suggest uploading compiled `messages/runtime/<locale>.json` to a CDN (e.g. Cloudflare R2, AWS S3/CloudFront, Supabase Storage, Vercel Blob).
+    - **Why?** Enables instant Over-the-Air (OTA) translation and typo fixes without requiring App Store / Google Play review cycles or full client web redeployments.
+    - **Scaffold CDN Environment Variables**:
+      - For Expo: `EXPO_PUBLIC_TRANSLATIONS_CDN_URL=https://cdn.yourdomain.com/locales`
+      - For React / Vite: `VITE_TRANSLATIONS_CDN_URL=https://cdn.yourdomain.com/locales`
+    - **Ask the user to set/update these ENV variables** with their actual CDN endpoint.
+    - **Resilient Runtime with Local Fallback**: The client loader fetches from CDN with caching (e.g., `AsyncStorage` / `localStorage`), while always maintaining bundled `messages/runtime/<locale>.json` as an offline fallback.
+    - **Upload Script**: Provide an upload script (e.g. `scripts/upload-locales.mjs`) to upload `messages/runtime/` bundles to the CDN.
 
 ---
 
@@ -56,6 +131,10 @@ Inspect repository architecture without modifying files:
 shallok analyze --json
 ```
 - Examine detected framework, router, programming language, and source paths (`src/`).
+- Check existing translation catalogs and folders (`locales/`, `src/locales/`, `messages/`).
+- Check if existing base locale is referencing legacy files (e.g., `src/locales/en.json`, `locales/en.json`).
+- Check if existing keys use object-based nesting or normalized dot-paths.
+- Determine if the project is Static/SSR (Next.js, Astro) or Client/Mobile (Expo, React Native, React SPA).
 - Determine candidate source files and target locales from the user request.
 - **Never edit files during Phase A.**
 
@@ -64,18 +143,27 @@ Present a concise plan to the developer:
 1. **Detected Framework**: Framework name and router type.
 2. **Runtime Localization Solution**: Suggested runtime library (e.g. `next-intl`, `react-i18next`, `expo-localization`).
 3. **Locales**: Source locale (e.g. `en`) and target locales (e.g. `kn` Kannada, `hi` Hindi).
-4. **Planned Scaffolding**:
+4. **Directory & Key Architecture Check**:
+   - If non-standard locale folders or legacy base locale references exist: Propose migrating to `messages/` and updating all import references to `messages/runtime/<locale>.json`.
+   - If object-based keys exist: Propose flattening to normalized dot-path keys and request user confirmation.
+5. **Loading Strategy Recommendation**:
+   - For **Astro / Next.js**: Recommend local bundle loading from `messages/runtime/`.
+   - For **Expo / React Native / React SPA**: Recommend CDN loading with `EXPO_PUBLIC_TRANSLATIONS_CDN_URL` or `VITE_TRANSLATIONS_CDN_URL`, with local fallback, and ask the user to configure the CDN ENV.
+6. **Planned Scaffolding**:
    - `localization.json` (Project manifest)
-   - `localization_base/<source_locale>.json` & `metadata.json` (Canonical catalog & versions)
-   - `messages/<locale>.json` (Locale catalogs)
+   - `localization_base/<source_locale>.json` & `metadata.json` (Canonical catalog & versions — **never delete**)
+   - `messages/<locale>.json` (Authoritative locale catalogs)
    - Framework adapter configuration
-5. Confirm with the user before proceeding (or proceed if `--yes` / user requested).
+7. Confirm with the user before proceeding (or proceed if `--yes` / user requested).
 
 ### Phase C: Initialize
 Initialize project configuration and catalogs:
 ```bash
 shallok init --source en --locales kn,hi --framework <name> --yes
 ```
+1. If migrating from an existing non-standard folder (e.g., `src/locales/en.json`), copy/migrate existing translations into `messages/` and reconcile into `localization_base/`.
+2. **Update all code references**: Replace legacy imports like `import en from './locales/en.json'` with `messages/runtime/` or CDN loader.
+3. If using CDN loading (Expo / React Native / React), create/update `.env` / `.env.example` with the CDN URL variable and prompt the user to configure it.
 
 ### Phase D: Extract & Enrich Context (AI-Driven)
 1. **Preview proposed extraction**:
@@ -86,32 +174,32 @@ shallok init --source en --locales kn,hi --framework <name> --yes
    ```bash
    shallok extract
    ```
-3. **Enrich Context (Performed by AI Agent)**:
+3. **Mandatory Context Check & Enrichment (Performed by AI Agent)**:
    - Inspect `localization_base/<source_locale>.json`.
-   - While the CLI records initial AST element locations, you analyze the actual screen and component purpose.
-   - Enrich `context` with semantic clarity (e.g. *"Primary CTA button on checkout page to trigger payment modal"*).
-   - Use the CLI:
+   - **Verify that every key has proper semantic context**.
+   - If any key lacks context or has only generic AST descriptions, analyze the screen and component purpose and enrich `context` BEFORE proceeding to translation:
      ```bash
      shallok context set <key> "<semantic_description>"
      ```
      Or edit `"context"` directly in `localization_base/<source_locale>.json`.
    - **Preservation Guarantee**: Shallok's reconciliation algorithm strictly preserves your enriched context across all future extractions.
 
-### Phase E: Translate (AI-Driven)
+### Phase E: Translate (AI-Driven from `localization_base/{source_locale}` + Context)
 **No external API keys or LLM provider subscriptions are required.**
 You, as the AI coding agent, generate the translations directly:
 
-1. **Identify entries needing translation**:
+1. **Verify Context First**: Ensure all keys in `localization_base/<source_locale>.json` have proper semantic context.
+2. **Identify entries needing translation**:
    ```bash
    shallok status --json
    ```
    Inspect `messages/<locale>.json` for keys with `status: "missing"` or `status: "stale"`.
 
-2. **Generate Translations**:
-   - Translate each entry into natural, culturally accurate target language using the source `content` and your enriched `context`.
+3. **Generate Translations from `localization_base/{source_locale}` and Its Context**:
+   - Translate each entry into natural, culturally accurate target language reading the source `content` AND its semantic `context` from `localization_base/<source_locale>.json`.
    - **Rule**: Strictly preserve all interpolation placeholders (`{name}`, `{count}`, `{price}`).
 
-3. **Update Target Locale Catalogs**:
+4. **Update Target Locale Catalogs**:
    Update `messages/<locale>.json` directly or using the CLI:
    ```bash
    shallok entry set <locale> <key> "<translated_text>"
@@ -133,7 +221,7 @@ You, as the AI coding agent, generate the translations directly:
    ```bash
    shallok extract --apply
    ```
-   Replaces hardcoded strings with `{t('key')}` and injects necessary hooks.
+   Replaces hardcoded strings with `{t('key')}` using normalized dot-separated key paths and injects necessary hooks.
 
 2. **Build runtime message bundles**:
    ```bash
@@ -161,10 +249,10 @@ You, as the AI coding agent, generate the translations directly:
   ```bash
   shallok locale add <locale>
   ```
-  Translate active keys into `messages/<locale>.json`, then run `shallok build && shallok validate`.
+  Check context in `localization_base/<source_locale>.json`, translate active keys into `messages/<locale>.json` using source content and context, then run `shallok build && shallok validate`.
 
 - **Source wording changed**:
-  Run `shallok extract`. Only modified keys increment version (`key_version`); only dependent entries are flagged `stale`. Translate the changed string, then `shallok build`.
+  Run `shallok extract`. Only modified keys increment version (`key_version`); only dependent entries are flagged `stale`. Check and update context if needed, translate the changed string from `localization_base/<source_locale>.json` and context, then `shallok build`.
 
 - **Rename a key**:
   ```bash

package/skills/localization-and-translation-i18n/references/extraction.md

@@ -3,6 +3,11 @@
 ## Extraction Strategy
 Shallok uses Babel AST parsing (`@babel/parser` and `@babel/traverse`) with full JSX and TypeScript support to safely inspect and transform source files without regular expressions.
 
+## Normalized Dot-Paths for Keys (Never Object-Based)
+- **Always use normalized dot-separated key paths**: e.g., `"checkout.pay_now"`, `"auth.login.title"`, `"dashboard.metrics.active_users"`.
+- **Never use object-based or nested object paths**: Catalogs must never store nested object trees.
+- **If an existing project uses object-based keys**: Ask the user for confirmation to flatten them to normalized dot-path keys before running extraction or migrating catalogs.
+
 ## What is Extracted
 1. **JSX Text Nodes**:
    - `<h1>Welcome</h1>` → `{t('home.welcome')}`
@@ -26,4 +31,4 @@ Shallok uses Babel AST parsing (`@babel/parser` and `@babel/traverse`) with full
 - Already localized calls: `t(...)`, `i18n.t(...)`.
 
 ## Dry-Run Mode
-Always use `localize extract --dry-run` to preview keys and occurrences before committing changes to source files.
+Always use `shallok extract --dry-run` to preview keys and occurrences before committing changes to source files.

package/skills/localization-and-translation-i18n/references/frameworks.md

@@ -1,39 +1,58 @@
 # Framework Adapters Reference
 
-## Supported Frameworks
+## Loading Strategies: Local vs. CDN
+
+| Framework Type | Recommended Strategy | Primary Rationale | Configuration & ENV |
+| :--- | :--- | :--- | :--- |
+| **Astro** (`astro:i18n`) | **Local** (`messages/runtime/`) | Static builds (SSG) & SSR; zero network hops, zero build dependency on external services. | Bundled in `src/i18n/utils.ts` |
+| **Next.js** (`next-intl`) | **Local** (`messages/runtime/`) | Server Components & SSR; instant First Contentful Paint, zero latency in Node/Edge. | Bundled in `src/i18n/request.ts` |
+| **Expo / React Native** | **CDN-Backed** (+ Local Fallback) | Instant Over-The-Air (OTA) translation & typo fixes without waiting for App Store review. | `EXPO_PUBLIC_TRANSLATIONS_CDN_URL` + AsyncStorage caching + local fallback |
+| **React SPA** (Vite / CRA) | **CDN-Backed** (+ Local Fallback) | Instant copy updates without rebuilding web bundle. | `VITE_TRANSLATIONS_CDN_URL` + localStorage caching + local fallback |
+
+---
+
+## Supported Frameworks & Adapters
 
 ### 1. React (`react-i18next`)
 - **Initialization**: Generates `src/i18n.ts` configuring `i18next` with `initReactI18next`.
 - **Runtime Messages**: Generates clean key-value message files in `messages/runtime/<locale>.json`.
+- **CDN Loading**: Suggest `VITE_TRANSLATIONS_CDN_URL` in `.env` with local `messages/runtime/` fallback and `scripts/upload-locales.mjs`.
+- **Legacy Migration**: If `src/main.tsx` or `src/i18n.ts` imported `src/locales/en.json`, migrate to `messages/runtime/` or CDN loader.
 - **AST Transformation**:
   - Adds `import { useTranslation } from 'react-i18next';`
   - Injects `const { t } = useTranslation();` in components
-  - Replaces text with `{t('key')}`
+  - Replaces text with `{t('key')}` using normalized dot-separated key paths.
 
 ### 2. Next.js (`next-intl`)
 - **App Router Support**:
   - Generates `src/i18n/routing.ts` defining `locales` and `defaultLocale`.
-  - Generates `src/i18n/request.ts` using `getRequestConfig`.
+  - Generates `src/i18n/request.ts` using `getRequestConfig` loading local `messages/runtime/<locale>.json`.
 - **Runtime Messages**: Generates runtime message files in `messages/runtime/<locale>.json`.
+- **Loading Strategy**: Local loading recommended for Server Components and SSR.
+- **Legacy Migration**: If `src/i18n/request.ts` or `middleware.ts` imported old locale JSONs, update imports to `messages/runtime/<locale>.json`.
 - **AST Transformation**:
   - Adds `import { useTranslations } from 'next-intl';`
   - Injects `const t = useTranslations();` in components
-  - Replaces text with `{t('key')}`
+  - Replaces text with `{t('key')}` using normalized dot-separated key paths.
 
 ### 3. Expo / React Native (`expo-localization` + `react-i18next`)
 - **Initialization**: Generates `i18n.ts` using `getLocales` from `expo-localization` and `i18next`.
 - **Runtime Messages**: Generates runtime message files in `messages/runtime/<locale>.json`.
+- **CDN Loading**: Suggest `EXPO_PUBLIC_TRANSLATIONS_CDN_URL` in `.env` with `AsyncStorage` caching and local `messages/runtime/` fallback.
+- **Legacy Migration**: If `app/_layout.tsx` or `src/i18n.ts` imported `src/locales/en.json`, migrate to `messages/runtime/` or CDN loader.
 - **AST Transformation**:
   - Adds `import { useTranslation } from 'react-i18next';`
   - Injects `const { t } = useTranslation();` in mobile components
-  - Replaces text inside `<Text>` and attributes like `placeholder` with `{t('key')}`
+  - Replaces text inside `<Text>` and attributes like `placeholder` with `{t('key')}` using normalized dot-separated key paths.
 
 ### 4. Astro (`astro:i18n`)
-- **Initialization**: Generates `src/i18n/utils.ts` defining `locales`, `defaultLocale`, and `useTranslations`.
+- **Initialization**: Generates `src/i18n/utils.ts` defining `locales`, `defaultLocale`, and `useTranslations` loading local `messages/runtime/<locale>.json`.
 - **Runtime Messages**: Generates runtime message files in `messages/runtime/<locale>.json`.
+- **Loading Strategy**: Local loading recommended for fast static SSG and hybrid SSR.
+- **Legacy Migration**: If `src/i18n/utils.ts` or `.astro` files imported old locale JSONs, update imports to `messages/runtime/<locale>.json`.
 - **Component Usage**:
   - In Astro components frontmatter: `const t = await useTranslations(locale);`
-  - Renders `{t('key')}` in templates and islands.
+  - Renders `{t('key')}` in templates and islands using normalized dot-separated key paths.
 
 ### 5. Vue (`vue-i18n`)
 - **Initialization**: Generates `src/i18n.ts` using `createI18n`.

package/skills/localization-and-translation-i18n/references/translation.md

@@ -1,11 +1,37 @@
 # Translation Engine & Agent-Native Translation Reference
 
 ## Agent-Native Translation (Primary Workflow)
-When using Shallok via an AI coding agent (e.g., in Claude, Cursor, Copilot, Antigravity):
+When using Shallok via an AI coding agent (e.g., in Antigravity, Claude, Cursor, Copilot):
 - **No external API keys or LLM provider subscriptions are needed.**
 - The AI agent itself generates accurate, natural translations for each target locale.
-- The AI agent inspects `messages/<locale>.json` or runs `localize status --json` to identify entries with `status: "missing"` or `status: "stale"`.
-- The AI agent writes the translated strings into `messages/<locale>.json` (or uses `localize entry set <locale> <key> "<translated_text>"`).
+- **Always verify context first**: Before translating any key, check that all keys in `localization_base/<source_locale>.json` have proper semantic context. If any key lacks context or has generic AST context, generate rich context before translating.
+- **Always translate from `localization_base/<source_locale>.json` and its context**: Never translate raw isolated string fragments. Combine the canonical source `content` and its semantic `context` to generate culturally authentic, grammatically correct translations.
+- Inspect `messages/<locale>.json` or run `shallok status --json` to identify entries with `status: "missing"` or `status: "stale"`.
+- Write the translated strings into `messages/<locale>.json` (or use `shallok entry set <locale> <key> "<translated_text>"`).
+
+## Core Rules for Translations
+
+### 1. Pre-Flight Verification & Persistence
+- Before making changes to a new or existing repo, verify `shallok` is installed and check that `localization.json` and `localization_base/` exist. If not, initialize with `shallok init`.
+- **NEVER delete `localization.json` or `localization_base/`**. They hold the single source of truth, content hashes, version counters, and context.
+
+### 2. Normalized Dot-Paths for Keys
+- All keys must use normalized dot notation (e.g., `"checkout.pay_now"`, `"auth.login.title"`).
+- Never use nested object structures. If an existing codebase uses nested objects for keys, ask the user to convert/flatten them to normalized dot-path keys.
+
+### 3. Standardized `messages/` Folder
+- All translations must reside in `messages/` (`messages/<locale>.json` and `messages/runtime/<locale>.json`).
+- If an existing project uses another directory (such as `locales/`, `src/locales/`, `translations/`), ask the user to switch to `messages/` and migrate existing files.
+
+### 4. Context Verification Before Translation
+- Check all keys in `localization_base/<source_locale>.json` have meaningful semantic context.
+- If any key lacks context, generate clear context describing the screen, element, and user action before initiating translation.
+
+### 5. Always Translate from Source Locale Catalog + Context
+- Read both `content` and `context` from `localization_base/<source_locale>.json`.
+- Preserve all interpolation placeholders (`{name}`, `{count}`, `{amount}`) exactly as formatted.
+
+---
 
 ### Translation Schema
 Each entry in `messages/<locale>.json` maintains:
@@ -35,7 +61,7 @@ The AI agent must strictly preserve all interpolation placeholders in target tra
 - Kannada: `"ಮರಳಿ ಸ್ವಾಗತ, {userName}! ನಿಮ್ಮ ಕಾರ್ಟ್‌ನಲ್ಲಿ {itemCount} ಐಟಂಗಳು ಇವೆ."`
 - Hindi: `"वापसी पर स्वागत है, {userName}! आपके कार्ट में {itemCount} आइटम हैं।"`
 
-If a translation drops a placeholder or introduces an invalid placeholder, `localize validate` will fail with an error.
+If a translation drops a placeholder or introduces an invalid placeholder, `shallok validate` will fail with an error.
 
 ---
 

package/skills/localization-and-translation-i18n/references/versioning.md

@@ -1,8 +1,22 @@
 # Per-Key Versioning & Context Reference
 
+## Strict Catalog Persistence: NEVER Delete `localization.json` or `localization_base/`
+> [!CAUTION]
+> `localization.json` and `localization_base/` are the permanent foundation of the localization system:
+> - `localization.json` stores project settings, locales, paths, and key strategies.
+> - `localization_base/` stores canonical source strings, per-key version counters (`key_version`), cryptographic SHA-256 hashes (`content_hash`), occurrence positions, and enriched semantic `context`.
+> - **Under NO circumstances should you EVER delete `localization.json` or `localization_base/`**. Deleting them resets version counters, destroys human- and AI-enriched context, breaks incremental translation, and corrupts catalog synchronization.
+
+---
+
 ## Canonical Source Catalog (`localization_base/`)
 The canonical catalog is the single source of truth for all active and obsolete keys, including their cryptographic content hashes and version counters.
 
+### Key Format: Normalized Dot-Separated Paths Only
+- All keys must use normalized dot-separated string paths (e.g. `"checkout.pay_now"`, `"auth.login.title"`).
+- **Never use nested object-based keys**.
+- If an existing project uses object-based keys, ask the user to convert and flatten them to normalized dot-paths before modifying catalogs.
+
 ### Structure of a Canonical Entry
 ```json
 {
@@ -26,12 +40,14 @@ The canonical catalog is the single source of truth for all active and obsolete
 }
 ```
 
-## Context Enrichment by AI Agents
-1. When strings are extracted via `localize extract`, the CLI provides baseline AST context (e.g. `JSX text in <button> (in App <button>)`).
-2. The AI Agent analyzes the source code, component hierarchy, and user flow, and enriches the `context` with semantic clarity:
-   - CLI method: `localize context set <key> "<semantic_description>"`
+## Context Enrichment & Verification Before Translation
+1. When strings are extracted via `shallok extract`, the CLI provides baseline AST context (e.g. `JSX text in <button> (in App <button>)`).
+2. **Mandatory Check Before Translation**: Before generating any translations, check that all active keys in `localization_base/<source_locale>.json` have proper, high-quality semantic context.
+3. If any key lacks context or has only generic AST descriptions, analyze the source code, component hierarchy, and user flow, and enrich the `context` with semantic clarity:
+   - CLI method: `shallok context set <key> "<semantic_description>"`
    - Direct edit: Update `"context"` in `localization_base/<source_locale>.json`.
-3. **Preservation Guarantee**:
+4. **Always Translate from Source Locale + Context**: Translations must be derived directly from `localization_base/<source_locale>.json` using BOTH `content` and `context`.
+5. **Preservation Guarantee**:
    Shallok's reconciliation algorithm (`reconcileCanonicalCatalog`) checks if a key already has an enriched context. Unless the source code introduces an explicit inline comment (`{/* context: ... */}`), the AI agent's enriched context is **strictly preserved** across all future extractions and will not be overwritten by generic AST node descriptions.
 
 ## Reconciliation & Versioning Rules
@@ -54,4 +70,4 @@ In `messages/<locale>.json`, each entry records:
   }
 }
 ```
-A translation is **stale** when `entry.source_version < canonicalEntry.key_version`. Stale entries retain their existing translated value for reference until the AI agent updates them.
+A translation is **stale** when `entry.source_version < canonicalEntry.key_version`. Stale entries retain their existing translated value for reference until the AI agent updates them from `localization_base/<source_locale>.json` and its enriched `context`.

package/skills/nextjs-localization-and-translation-i18n/SKILL.md

@@ -14,20 +14,86 @@ Use this skill when asked to add, configure, update, or audit localization in **
 
 ---
 
-## Agent Step 0: Verify & Install Shallok CLI
+## Agent Step 0: Mandatory Pre-Flight Checklist (Run Before ANY Code or Config Changes)
+
+**Before making ANY changes to a new repo or an existing repo, you (the AI Agent) MUST execute the following checks in order:**
+
+### 1. Check if Shallok is Installed
+- Inspect `package.json` for `"shallok"` or `"@shallok/cli"` in `devDependencies` (or `dependencies`), or run:
+  ```bash
+  npx shallok --version
+  ```
+- **If `shallok` is NOT installed, install it immediately into `devDependencies`:**
+  - If `pnpm-lock.yaml` exists: `pnpm add -D shallok`
+  - If `yarn.lock` exists: `yarn add -D shallok`
+  - If `bun.lockb` / `bun.lock` exists: `bun add -d shallok`
+  - Otherwise: `npm install -D shallok`
+
+### 2. Check if `localization.json` and `localization_base/` Exist
+- Verify that `localization.json` exists in the project root.
+- Verify that the `localization_base/` directory exists with `<source_locale>.json` and `metadata.json`.
+- **If either does NOT exist, create them using Shallok:**
+  ```bash
+  shallok init --source en --locales kn,hi --framework nextjs --yes
+  ```
+
+### 3. NEVER Delete `localization.json` or `localization_base/`
+> [!CAUTION]
+> **CRITICAL PERSISTENCE RULE: NEVER DELETE `localization.json` OR `localization_base/`!**
+> - `localization.json` is the single source-of-truth project manifest.
+> - `localization_base/` contains the canonical catalog (`<source_locale>.json`) with per-key version numbers (`key_version`), cryptographic content hashes (`sha256:`), source code occurrences, and enriched semantic context, alongside `metadata.json`.
+> - **Under NO circumstances should you EVER delete `localization.json` or `localization_base/`**. Deleting them destroys the versioning history, wipes out human- and AI-enriched context, resets version counters, breaks incremental translation, and corrupts catalog synchronization.
 
-**Before executing any localization commands, you (the AI Agent) MUST verify that `shallok` is available:**
+---
 
-1. Check `package.json` for `"shallok"` in `devDependencies` or run:
-   ```bash
-   npx shallok --version
-   ```
-2. **If `shallok` is NOT installed, install it immediately** using the project's package manager:
-   - If `pnpm-lock.yaml` exists: `pnpm add -D shallok`
-   - If `yarn.lock` exists: `yarn add -D shallok`
-   - If `bun.lockb` / `bun.lock` exists: `bun add -d shallok`
-   - Otherwise: `npm install -D shallok`
-   *(Alternatively, run commands directly via `npx shallok <command>`)*
+## Core Architectural & Formatting Rules
+
+### Rule 1: Always Use Normalized Paths for Keys (Never Object-Based Paths)
+- **Always use normalized dot-separated key paths**: e.g., `"auth.login.title"`, `"checkout.pay_now"`, `"settings.profile.change_password"`.
+- **Never use nested object-based paths**: Do not structure keys or catalogs as nested JSON objects (e.g., `{ "auth": { "login": { "title": "..." } } }`). All keys in `localization_base/<source_locale>.json`, `messages/<locale>.json`, and runtime bundles MUST be flat normalized string paths.
+- **If an existing project is using object-based keys**:
+  - **STOP and ask the user** for confirmation to flatten them into normalized dot-separated key paths before proceeding with extraction or catalog edits.
+  - Explain that normalized dot-paths are required for deterministic AST extraction, per-key versioning, and collision-free reconciliation.
+  - Upon user approval, migrate and flatten all nested keys to normalized dot-paths.
+
+### Rule 2: Always Use the `messages/` Directory Structure (Ask User & Migrate If Different)
+- **Standard translations folder**: Shallok strictly standardizes on `messages/` for authoritative locale catalogs (`messages/<locale>.json`) and compiled runtime bundles (`messages/runtime/<locale>.json`).
+- **If the project uses any other folder structure** (e.g., `locales/`, `src/locales/`, `translations/`, `lang/`, `src/i18n/locales/`):
+  - **STOP and ask the user** for approval to switch to the standardized `messages/` directory structure.
+  - Upon user approval, **migrate** all existing translation files into `messages/`, update framework configuration and import paths, reconcile existing strings into `localization_base/`, and clean up the old directory.
+
+### Rule 3: Check Context for All Keys in `localization_base/{source_locale}` Before Translating
+- **Before generating any translations for any target locale**:
+  - Inspect all active keys in `localization_base/<source_locale>.json` (e.g., `localization_base/en.json`).
+  - Verify that **every key has meaningful, accurate semantic `context`**.
+  - If any key lacks context, has an empty context (`""`), or contains only generic AST descriptions (e.g., `"JSX text in button"`), you MUST inspect the component/screen and **generate rich semantic context for all such keys BEFORE generating translations**.
+  - Enrich context via:
+    ```bash
+    shallok context set <key> "<semantic_description>"
+    ```
+    or directly edit the `"context"` field in `localization_base/<source_locale>.json`.
+
+### Rule 4: Always Generate Translations from `localization_base/{source_locale}` and Its Context
+- **Never translate raw strings in isolation**:
+  - All translations MUST be generated directly from `localization_base/<source_locale>.json`, reading BOTH the source string (`content`) and its semantic `context`.
+  - The `context` provides critical disambiguation (grammatical gender, button vs label vs heading, formality, technical terminology).
+  - Strictly preserve all interpolation tokens (`{userName}`, `{count}`, `{amount}`) exactly as formatted in the source `content`.
+
+### Rule 5: Migrate Legacy Base Locale References to Specified Structure
+- **If the Next.js app already has an existing localization setup**:
+  - Check how the base/source locale (`source_locale`, e.g., `en`) is imported and loaded.
+  - If any file (e.g. `src/i18n/request.ts`, `src/i18n.ts`, `middleware.ts`, `app/[locale]/layout.tsx`) is importing a legacy file (such as `src/locales/en.json`, `locales/en.json`, or arbitrary JSON files):
+    1. **Reconcile strings** into `localization_base/<source_locale>.json` and `messages/<source_locale>.json`.
+    2. **Update all code imports and request loaders** to point to the standardized compiled runtime bundle (`messages/runtime/<locale>.json`).
+    3. Remove or deprecate the legacy base locale file so that Shallok does NOT extract it as source code (preventing false occurrences like `"relative_file_path": "src/locales/en.json"`).
+
+### Rule 6: Next.js Architecture: Keep Translations Local for Instant Server Rendering
+- **Recommend keeping translations LOCAL in `messages/runtime/<locale>.json`**:
+  - Next.js Server Components, Server-Side Rendering (SSR), and Static Site Generation (SSG) execute in Node.js or Edge runtime.
+  - Loading compiled message bundles directly from `messages/runtime/<locale>.json` (via `src/i18n/request.ts`) guarantees:
+    - **Zero network latency**: Instant server execution and rapid First Contentful Paint (FCP).
+    - **Zero build or render failures**: Builds do not depend on external CDN uptime or network requests.
+    - **Optimal bundling**: Next.js automatically tree-shakes and splits locale chunks.
 
 ---
 
@@ -38,7 +104,7 @@ Shallok pairs a deterministic AST CLI (`shallok`) with **the AI Agent as the sem
 1. **The CLI (`shallok`)** handles deterministic tasks:
    - Scaffolds Next.js configuration files: `src/i18n/routing.ts` and `src/i18n/request.ts`.
    - AST parses Next.js JSX/TSX components (`shallok extract`).
-   - Replaces user-facing strings with `{t('key')}` and imports `useTranslations` (`shallok extract --apply`).
+   - Replaces user-facing strings with `{t('key')}` using normalized dot-separated key paths and imports `useTranslations` (`shallok extract --apply`).
    - Maintains per-key versioning and SHA-256 content hashes in `localization_base/`.
    - Compiles runtime message bundles to `messages/runtime/<locale>.json` (`shallok build`).
    - Validates 0 missing keys, 0 stale versions, and 0 broken placeholders (`shallok validate`).
@@ -62,6 +128,9 @@ shallok analyze --json
 - Check router type:
   - **App Router**: `app/` directory exists (with or without `src/`).
   - **Pages Router**: `pages/` directory exists.
+- Check existing translation catalogs and folders (`locales/`, `src/locales/`, `messages/`).
+- Check if existing base locale is referencing legacy files (e.g., `src/locales/en.json`, `locales/en.json`).
+- Check if existing keys use object-based nesting or normalized dot-paths.
 - Identify candidate page and component files (`app/**/*.tsx`, `src/**/*.tsx`).
 - Note user-requested target locales (e.g. `kn`, `hi`, `fr`, `es`).
 - **Never edit files during Phase A.**
@@ -70,34 +139,35 @@ shallok analyze --json
 Present a concise plan to the developer:
 1. **Framework & Library**: Next.js with `next-intl`.
 2. **Locales**: Source locale (e.g. `en`) and target locales (e.g. `kn` Kannada, `hi` Hindi).
-3. **Dependencies to Install**:
+3. **Directory & Legacy File Migration**:
+   - If non-standard locale folders or legacy base locale files exist (e.g. `src/locales/en.json`): Propose migrating to `messages/` and updating all import references to `messages/runtime/<locale>.json`.
+   - If object-based keys exist: Explicitly propose flattening to normalized dot-path keys and request user confirmation.
+4. **Loading Strategy Recommendation**:
+   - Recommend keeping translations local in `messages/runtime/` for zero-latency Server Components and SSR.
+5. **Dependencies to Install**:
    ```bash
    pnpm add next-intl
    ```
-4. **Planned Scaffolding**:
+6. **Planned Scaffolding**:
    - `localization.json` (Project manifest)
    - `src/i18n/routing.ts` (Routing definition for `next-intl`)
-   - `src/i18n/request.ts` (Request configuration for server-side loading)
+   - `src/i18n/request.ts` (Request configuration for server-side loading from `messages/runtime/`)
    - `middleware.ts` (Locale detection and URL rewriting)
-   - `localization_base/<source_locale>.json` & `metadata.json` (Canonical catalog & versions)
+   - `localization_base/<source_locale>.json` & `metadata.json` (Canonical catalog & versions — **never delete**)
    - `messages/<locale>.json` (Authoritative target catalogs)
    - `messages/runtime/<locale>.json` (Compiled runtime message bundles)
-5. **App Structure Updates**:
+7. **App Structure Updates**:
    - Nest routes under `app/[locale]/...`
    - Setup `NextIntlClientProvider` in `app/[locale]/layout.tsx` for client components
-6. Confirm with user before proceeding (or proceed if `--yes` / requested).
+8. Confirm with user before proceeding (or proceed if `--yes` / requested).
 
 ### Phase C: Initialize
 Initialize configuration and Next.js scaffolding:
 ```bash
 shallok init --source en --locales kn,hi --framework nextjs --yes
 ```
-This automatically creates:
-- `localization.json`
-- `src/i18n/routing.ts`
-- `src/i18n/request.ts`
-- Initial catalogs in `localization_base/` and `messages/`
-- Installs `shallok` in `devDependencies` and copies skills
+1. If migrating from an existing non-standard folder (e.g., `locales/` or `src/locales/`), copy/migrate existing translations into `messages/` and reconcile into `localization_base/`.
+2. **Update all code references**: Replace legacy imports like `import en from './locales/en.json'` with `messages/runtime/<locale>.json` in `src/i18n/request.ts`.
 
 Ensure dependencies are installed:
 ```bash
@@ -115,33 +185,33 @@ pnpm add next-intl
    ```
    This extracts text and JSX expressions to `localization_base/en.json`.
 
-3. **Enrich Context (Performed by AI Agent)**:
+3. **Mandatory Context Check & Enrichment (Performed by AI Agent)**:
    - Review `localization_base/en.json`.
-   - Inspect the Next.js page or component where each key originates.
-   - Enrich `context` with semantic clarity (e.g. *"Dashboard banner heading welcoming returning user"*).
-   - Use the CLI:
+   - **Verify that every key has proper semantic context**.
+   - If any key lacks context or has only generic AST descriptions, inspect the Next.js page or component where each key originates and enrich `context` with semantic clarity BEFORE proceeding to translation:
      ```bash
      shallok context set <key> "<semantic_description>"
      ```
      Or edit `"context"` directly in `localization_base/en.json`.
    - **Preservation Guarantee**: Shallok's reconciliation algorithm strictly preserves your enriched context across all future extractions.
 
-### Phase E: Translate (AI-Driven)
+### Phase E: Translate (AI-Driven from `localization_base/{source_locale}` + Context)
 **No external API keys or LLM provider subscriptions are required.**
 You, as the AI coding agent, generate the translations directly:
 
-1. **Check translation status**:
+1. **Verify Context First**: Ensure all keys in `localization_base/<source_locale>.json` have proper semantic context.
+2. **Check translation status**:
    ```bash
    shallok status --json
    ```
    Identify keys in `messages/<locale>.json` marked with `status: "missing"` or `status: "stale"`.
 
-2. **Generate Natural Translations**:
-   - Translate each string into culturally authentic target language (e.g. Kannada, Hindi).
-   - Use the source `content` and your enriched `context` for accurate terminology.
+3. **Generate Translations from `localization_base/{source_locale}` and Its Context**:
+   - Translate each string into culturally authentic target language (e.g. Kannada, Hindi) reading source `content` AND its semantic `context` from `localization_base/<source_locale>.json`.
+   - Use the source `content` and your enriched `context` for accurate terminology and nuance.
    - **Strict Rule**: Preserve all interpolation tokens (`{count}`, `{name}`, `{date}`) exactly as formatted in the source text.
 
-3. **Update Target Locale Catalogs**:
+4. **Update Target Locale Catalogs**:
    Write directly into `messages/<locale>.json` or use:
    ```bash
    shallok entry set <locale> <key> "<translated_text>"
@@ -152,6 +222,8 @@ You, as the AI coding agent, generate the translations directly:
    ```bash
    shallok extract --apply
    ```
+   Replaces hardcoded strings with `{t('key')}` using normalized dot-separated key paths.
+
 2. **Verify Next.js Integration Files**:
    - `middleware.ts` configured with `next-intl/middleware`.
    - `next.config.mjs` with `createNextIntlPlugin`.