shallok 0.1.1 → 0.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/README.md +132 -24
  2. package/dist/index.js +13 -13
  3. package/dist/index.js.map +1 -1
  4. package/package.json +10 -11
  5. package/skills/astro-localization-and-translation-i18n/SKILL.md +105 -32
  6. package/skills/astro-localization-and-translation-i18n/references/astro-i18n.md +15 -0
  7. package/skills/astro-localization-and-translation-i18n/references/extraction.md +6 -1
  8. package/skills/astro-localization-and-translation-i18n/references/frameworks.md +59 -0
  9. package/skills/astro-localization-and-translation-i18n/references/translation.md +30 -4
  10. package/skills/astro-localization-and-translation-i18n/references/versioning.md +22 -6
  11. package/skills/expo-localization-and-translation-i18n/SKILL.md +116 -35
  12. package/skills/expo-localization-and-translation-i18n/references/expo-localization.md +56 -11
  13. package/skills/expo-localization-and-translation-i18n/references/extraction.md +6 -1
  14. package/skills/expo-localization-and-translation-i18n/references/frameworks.md +59 -0
  15. package/skills/expo-localization-and-translation-i18n/references/translation.md +30 -4
  16. package/skills/expo-localization-and-translation-i18n/references/versioning.md +22 -6
  17. package/skills/localization-and-translation-i18n/SKILL.md +116 -28
  18. package/skills/localization-and-translation-i18n/references/extraction.md +6 -1
  19. package/skills/localization-and-translation-i18n/references/frameworks.md +26 -7
  20. package/skills/localization-and-translation-i18n/references/translation.md +30 -4
  21. package/skills/localization-and-translation-i18n/references/versioning.md +22 -6
  22. package/skills/nextjs-localization-and-translation-i18n/SKILL.md +107 -35
  23. package/skills/nextjs-localization-and-translation-i18n/references/extraction.md +6 -1
  24. package/skills/nextjs-localization-and-translation-i18n/references/frameworks.md +59 -0
  25. package/skills/nextjs-localization-and-translation-i18n/references/next-intl.md +14 -2
  26. package/skills/nextjs-localization-and-translation-i18n/references/translation.md +30 -4
  27. package/skills/nextjs-localization-and-translation-i18n/references/versioning.md +22 -6
  28. package/skills/react-localization-and-translation-i18n/SKILL.md +116 -35
  29. package/skills/react-localization-and-translation-i18n/references/extraction.md +6 -1
  30. package/skills/react-localization-and-translation-i18n/references/frameworks.md +59 -0
  31. package/skills/react-localization-and-translation-i18n/references/react-i18next.md +54 -8
  32. package/skills/react-localization-and-translation-i18n/references/translation.md +30 -4
  33. package/skills/react-localization-and-translation-i18n/references/versioning.md +22 -6
  34. package/skills/localize-app/SKILL.md +0 -80
  35. package/skills/localize-app/references/extraction.md +0 -29
  36. package/skills/localize-app/references/frameworks.md +0 -25
  37. package/skills/localize-app/references/translation.md +0 -45
  38. package/skills/localize-app/references/versioning.md +0 -57
@@ -14,20 +14,91 @@ Use this skill when asked to add, configure, update, or audit localization in **
14
14
 
15
15
  ---
16
16
 
17
- ## Agent Step 0: Verify & Install Shallok CLI
17
+ ## Agent Step 0: Mandatory Pre-Flight Checklist (Run Before ANY Code or Config Changes)
18
+
19
+ **Before making ANY changes to a new repo or an existing repo, you (the AI Agent) MUST execute the following checks in order:**
20
+
21
+ ### 1. Check if Shallok is Installed
22
+ - Inspect `package.json` for `"shallok"` or `"@shallok/cli"` in `devDependencies` (or `dependencies`), or run:
23
+ ```bash
24
+ npx shallok --version
25
+ ```
26
+ - **If `shallok` is NOT installed, install it immediately into `devDependencies`:**
27
+ - If `pnpm-lock.yaml` exists: `pnpm add -D shallok`
28
+ - If `yarn.lock` exists: `yarn add -D shallok`
29
+ - If `bun.lockb` / `bun.lock` exists: `bun add -d shallok`
30
+ - Otherwise: `npm install -D shallok`
31
+
32
+ ### 2. Check if `localization.json` and `localization_base/` Exist
33
+ - Verify that `localization.json` exists in the project root.
34
+ - Verify that the `localization_base/` directory exists with `<source_locale>.json` and `metadata.json`.
35
+ - **If either does NOT exist, create them using Shallok:**
36
+ ```bash
37
+ shallok init --source en --locales kn,hi --framework expo --yes
38
+ ```
39
+
40
+ ### 3. NEVER Delete `localization.json` or `localization_base/`
41
+ > [!CAUTION]
42
+ > **CRITICAL PERSISTENCE RULE: NEVER DELETE `localization.json` OR `localization_base/`!**
43
+ > - `localization.json` is the single source-of-truth project manifest.
44
+ > - `localization_base/` contains the canonical catalog (`<source_locale>.json`) with per-key version numbers (`key_version`), cryptographic content hashes (`sha256:`), source code occurrences, and enriched semantic context, alongside `metadata.json`.
45
+ > - **Under NO circumstances should you EVER delete `localization.json` or `localization_base/`**. Deleting them destroys the versioning history, wipes out human- and AI-enriched context, resets version counters, breaks incremental translation, and corrupts catalog synchronization.
18
46
 
19
- **Before executing any localization commands, you (the AI Agent) MUST verify that `shallok` is available:**
47
+ ---
20
48
 
21
- 1. Check `package.json` for `"shallok"` in `devDependencies` or run:
22
- ```bash
23
- npx shallok --version
24
- ```
25
- 2. **If `shallok` is NOT installed, install it immediately** using the project's package manager:
26
- - If `pnpm-lock.yaml` exists: `pnpm add -D shallok`
27
- - If `yarn.lock` exists: `yarn add -D shallok`
28
- - If `bun.lockb` / `bun.lock` exists: `bun add -d shallok`
29
- - Otherwise: `npm install -D shallok`
30
- *(Alternatively, run commands directly via `npx shallok <command>`)*
49
+ ## Core Architectural & Formatting Rules
50
+
51
+ ### Rule 1: Always Use Normalized Paths for Keys (Never Object-Based Paths)
52
+ - **Always use normalized dot-separated key paths**: e.g., `"auth.login.title"`, `"checkout.pay_now"`, `"settings.profile.change_password"`.
53
+ - **Never use nested object-based paths**: Do not structure keys or catalogs as nested JSON objects (e.g., `{ "auth": { "login": { "title": "..." } } }`). All keys in `localization_base/<source_locale>.json`, `messages/<locale>.json`, and runtime bundles MUST be flat normalized string paths.
54
+ - **If an existing project is using object-based keys**:
55
+ - **STOP and ask the user** for confirmation to flatten them into normalized dot-separated key paths before proceeding with extraction or catalog edits.
56
+ - Explain that normalized dot-paths are required for deterministic AST extraction, per-key versioning, and collision-free reconciliation.
57
+ - Upon user approval, migrate and flatten all nested keys to normalized dot-paths.
58
+
59
+ ### Rule 2: Always Use the `messages/` Directory Structure (Ask User & Migrate If Different)
60
+ - **Standard translations folder**: Shallok strictly standardizes on `messages/` for authoritative locale catalogs (`messages/<locale>.json`) and compiled runtime bundles (`messages/runtime/<locale>.json`).
61
+ - **If the project uses any other folder structure** (e.g., `locales/`, `src/locales/`, `translations/`, `lang/`, `src/i18n/locales/`):
62
+ - **STOP and ask the user** for approval to switch to the standardized `messages/` directory structure.
63
+ - Upon user approval, **migrate** all existing translation files into `messages/`, update framework configuration and import paths, reconcile existing strings into `localization_base/`, and clean up the old directory.
64
+
65
+ ### Rule 3: Check Context for All Keys in `localization_base/{source_locale}` Before Translating
66
+ - **Before generating any translations for any target locale**:
67
+ - Inspect all active keys in `localization_base/<source_locale>.json` (e.g., `localization_base/en.json`).
68
+ - Verify that **every key has meaningful, accurate semantic `context`**.
69
+ - If any key lacks context, has an empty context (`""`), or contains only generic AST descriptions (e.g., `"JSX text in button"`), you MUST inspect the component/screen and **generate rich semantic context for all such keys BEFORE generating translations**.
70
+ - Enrich context via:
71
+ ```bash
72
+ shallok context set <key> "<semantic_description>"
73
+ ```
74
+ or directly edit the `"context"` field in `localization_base/<source_locale>.json`.
75
+
76
+ ### Rule 4: Always Generate Translations from `localization_base/{source_locale}` and Its Context
77
+ - **Never translate raw strings in isolation**:
78
+ - All translations MUST be generated directly from `localization_base/<source_locale>.json`, reading BOTH the source string (`content`) and its semantic `context`.
79
+ - The `context` provides critical disambiguation (grammatical gender, button vs label vs heading, formality, mobile character limits).
80
+ - Strictly preserve all interpolation tokens (`{userName}`, `{count}`, `{amount}`) exactly as formatted in the source `content`.
81
+
82
+ ### Rule 5: Migrate Legacy Base Locale References to Specified Structure
83
+ - **If the mobile app already has an existing localization setup**:
84
+ - Check how the base/source locale (`source_locale`, e.g., `en`) is imported and loaded.
85
+ - If any file (e.g. `i18n.ts`, `src/i18n.ts`, `app/_layout.tsx`, `App.tsx`) is importing a legacy file (such as `src/locales/en.json`, `locales/en.json`, or arbitrary JSON files):
86
+ 1. **Reconcile strings** into `localization_base/<source_locale>.json` and `messages/<source_locale>.json`.
87
+ 2. **Update all code imports** to point to the standardized compiled runtime bundle (`messages/runtime/<locale>.json`) or the CDN loader.
88
+ 3. Remove or deprecate the legacy base locale file so that Shallok does NOT extract it as source code (preventing false occurrences like `"relative_file_path": "src/locales/en.json"`).
89
+
90
+ ### Rule 6: Mobile Translation Architecture: CDN-Backed Dynamic Loading (OTA Updates)
91
+ - **Suggest loading translations from a CDN for Expo & React Native**:
92
+ - App Store and Google Play binary releases require review cycles that take days or weeks. Shipping translations solely inside the mobile binary prevents quick fixes for typos, copy tweaks, or adding languages.
93
+ - Recommend hosting compiled `messages/runtime/<locale>.json` on a CDN or cloud storage bucket (e.g. Cloudflare R2, AWS S3/CloudFront, Supabase Storage, Vercel Blob).
94
+ - **Environment Variables & Configuration**:
95
+ - Define `EXPO_PUBLIC_TRANSLATIONS_CDN_URL=https://cdn.yourdomain.com/locales` in `.env` and `.env.example`.
96
+ - **Prompt and ask the user to configure/update `EXPO_PUBLIC_TRANSLATIONS_CDN_URL`** with their actual CDN endpoint.
97
+ - **Resilient Mobile Runtime Loader with Local Offline Fallback**:
98
+ - Configure `i18n.ts` to fetch dynamic translations from `${EXPO_PUBLIC_TRANSLATIONS_CDN_URL}/${locale}.json` and cache them locally using `AsyncStorage`.
99
+ - Always bundle `messages/runtime/<locale>.json` locally as fallback resources, ensuring the app works 100% offline or on first launch before network connectivity is established.
100
+ - **Upload Script**:
101
+ - Suggest / scaffold an upload script (e.g., `scripts/upload-locales.mjs`) to automate publishing compiled `messages/runtime/` dictionaries to the CDN.
31
102
 
32
103
  ---
33
104
 
@@ -38,7 +109,7 @@ Shallok pairs a deterministic AST CLI (`shallok`) with **the AI Agent as the sem
38
109
  1. **The CLI (`shallok`)** handles deterministic tasks:
39
110
  - Scaffolds `i18n.ts` configuring `expo-localization` and `i18next`.
40
111
  - AST parses mobile React Native components (`shallok extract`).
41
- - Replaces user-facing strings in `<Text>`, `<TextInput>`, and props with `{t('key')}` (`shallok extract --apply`).
112
+ - Replaces user-facing strings in `<Text>`, `<TextInput>`, and props with `{t('key')}` using normalized dot-separated key paths (`shallok extract --apply`).
42
113
  - Maintains per-key versioning and SHA-256 content hashes in `localization_base/`.
43
114
  - Compiles static runtime message bundles to `messages/runtime/<locale>.json` for Metro bundling (`shallok build`).
44
115
  - Validates 0 missing keys, 0 stale versions, and 0 broken placeholders (`shallok validate`).
@@ -59,6 +130,9 @@ shallok analyze --json
59
130
  ```
60
131
  - Verify framework is detected as `expo` (or `react-native`).
61
132
  - Inspect `package.json` for Expo SDK version and navigation library (Expo Router `expo-router` or `@react-navigation`).
133
+ - Check existing translation catalogs and folders (`locales/`, `src/locales/`, `messages/`).
134
+ - Check if existing base locale is referencing legacy files (e.g., `src/locales/en.json`, `locales/en.json`).
135
+ - Check if existing keys use object-based nesting or normalized dot-paths.
62
136
  - Identify mobile entry point (`App.tsx`, `index.js`, or `app/_layout.tsx`).
63
137
  - Identify candidate screen and component files (`src/**/*.tsx`, `app/**/*.tsx`, `components/**/*.tsx`).
64
138
  - Note user-requested target locales (e.g. `kn`, `hi`, `fr`, `es`).
@@ -68,30 +142,37 @@ shallok analyze --json
68
142
  Present a concise plan to the developer:
69
143
  1. **Framework & Libraries**: Expo / React Native with `expo-localization`, `i18next`, and `react-i18next`.
70
144
  2. **Locales**: Source locale (e.g. `en`) and target locales (e.g. `kn` Kannada, `hi` Hindi).
71
- 3. **Dependencies to Install**:
145
+ 3. **Directory & Legacy File Migration**:
146
+ - If non-standard locale folders or legacy base locale files exist (e.g. `src/locales/en.json`): Propose migrating to `messages/` and updating all import references to `messages/runtime/<locale>.json`.
147
+ - If object-based keys exist: Explicitly propose flattening to normalized dot-path keys and request user confirmation.
148
+ 4. **Loading Strategy Recommendation**:
149
+ - Propose loading translations from a CDN via `EXPO_PUBLIC_TRANSLATIONS_CDN_URL` with local offline fallback for OTA updates without app store reviews.
150
+ - Ask the user to configure their CDN endpoint in `.env`.
151
+ 5. **Dependencies to Install**:
72
152
  ```bash
73
- npx expo install expo-localization
153
+ npx expo install expo-localization @react-native-async-storage/async-storage
74
154
  pnpm add i18next react-i18next
75
155
  ```
76
- 4. **Planned Scaffolding**:
156
+ 6. **Planned Scaffolding**:
77
157
  - `localization.json` (Project manifest)
78
- - `i18n.ts` (Mobile i18n initialization at root)
79
- - `localization_base/<source_locale>.json` & `metadata.json` (Canonical catalog & versions)
158
+ - `i18n.ts` (Mobile i18n initialization at root with CDN loader & local fallback)
159
+ - `.env` / `.env.example` (Configured with `EXPO_PUBLIC_TRANSLATIONS_CDN_URL`)
160
+ - `scripts/upload-locales.mjs` (Script to sync `messages/runtime/` to CDN)
161
+ - `localization_base/<source_locale>.json` & `metadata.json` (Canonical catalog & versions — **never delete**)
80
162
  - `messages/<locale>.json` (Authoritative target catalogs)
81
163
  - `messages/runtime/<locale>.json` (Static runtime bundles imported by Metro)
82
- 5. **Root Integration**: Adding `import './i18n';` in `App.tsx` or `app/_layout.tsx`.
83
- 6. Confirm with user before proceeding (or proceed if `--yes` / requested).
164
+ 7. **Root Integration**: Adding `import './i18n';` in `App.tsx` or `app/_layout.tsx`.
165
+ 8. Confirm with user before proceeding (or proceed if `--yes` / requested).
84
166
 
85
167
  ### Phase C: Initialize
86
168
  Initialize configuration and Expo scaffolding:
87
169
  ```bash
88
170
  shallok init --source en --locales kn,hi --framework expo --yes
89
171
  ```
90
- This automatically creates:
91
- - `localization.json`
92
- - `i18n.ts` (with `getLocales()` device language detection)
93
- - Initial catalogs in `localization_base/` and `messages/`
94
- - Installs `shallok` in `devDependencies` and copies skills
172
+ 1. If migrating from an existing non-standard folder (e.g., `locales/` or `src/locales/`), copy/migrate existing translations into `messages/` and reconcile into `localization_base/`.
173
+ 2. **Update all code references**: Replace legacy imports like `import en from './locales/en.json'` with `messages/runtime/` or CDN loader.
174
+ 3. Configure `.env` and `.env.example` with `EXPO_PUBLIC_TRANSLATIONS_CDN_URL` and ask the user to update it with their CDN URL.
175
+ 4. Scaffold `scripts/upload-locales.mjs` if CDN loading is chosen.
95
176
 
96
177
  Ensure mobile dependencies are installed:
97
178
  ```bash
@@ -110,34 +191,34 @@ pnpm add i18next react-i18next
110
191
  ```
111
192
  This extracts text inside `<Text>`, `<Button title="...">`, and `<TextInput placeholder="...">` into `localization_base/en.json`.
112
193
 
113
- 3. **Enrich Context (Performed by AI Agent)**:
194
+ 3. **Mandatory Context Check & Enrichment (Performed by AI Agent)**:
114
195
  - Review `localization_base/en.json`.
115
- - Inspect the mobile screen and component context.
116
- - Enrich `context` with semantic clarity (e.g. *"Header label on Bluetooth pairing bottom sheet modal"*).
117
- - Use the CLI:
196
+ - **Verify that every key has proper semantic context**.
197
+ - If any key lacks context or has only generic AST descriptions, inspect the mobile screen and component context and enrich `context` with semantic clarity BEFORE proceeding to translation:
118
198
  ```bash
119
199
  shallok context set <key> "<semantic_description>"
120
200
  ```
121
201
  Or edit `"context"` directly in `localization_base/en.json`.
122
202
  - **Preservation Guarantee**: Shallok's reconciliation algorithm strictly preserves your enriched context across all future extractions.
123
203
 
124
- ### Phase E: Translate (AI-Driven)
204
+ ### Phase E: Translate (AI-Driven from `localization_base/{source_locale}` + Context)
125
205
  **No external API keys or LLM provider subscriptions are required.**
126
206
  You, as the AI coding agent, generate the translations directly:
127
207
 
128
- 1. **Check translation status**:
208
+ 1. **Verify Context First**: Ensure all keys in `localization_base/<source_locale>.json` have proper semantic context.
209
+ 2. **Check translation status**:
129
210
  ```bash
130
211
  shallok status --json
131
212
  ```
132
213
  Identify keys in `messages/<locale>.json` marked with `status: "missing"` or `status: "stale"`.
133
214
 
134
- 2. **Generate Natural Translations**:
135
- - Translate each string into culturally authentic target language (e.g. Kannada, Hindi).
215
+ 3. **Generate Translations from `localization_base/{source_locale}` and Its Context**:
216
+ - Translate each string into culturally authentic target language (e.g. Kannada, Hindi) reading source `content` AND its semantic `context` from `localization_base/<source_locale>.json`.
136
217
  - Keep mobile screen space constraints in mind (e.g. concise button labels).
137
218
  - Use the source `content` and your enriched `context` for accurate terminology.
138
219
  - **Strict Rule**: Preserve all interpolation tokens (`{count}`, `{name}`, `{date}`) exactly as formatted in the source text.
139
220
 
140
- 3. **Update Target Locale Catalogs**:
221
+ 4. **Update Target Locale Catalogs**:
141
222
  Write directly into `messages/<locale>.json` or use:
142
223
  ```bash
143
224
  shallok entry set <locale> <key> "<translated_text>"
@@ -148,7 +229,7 @@ You, as the AI coding agent, generate the translations directly:
148
229
  ```bash
149
230
  shallok extract --apply
150
231
  ```
151
- This injects `const { t } = useTranslation();` and replaces hardcoded text with `{t('key')}`.
232
+ This injects `const { t } = useTranslation();` and replaces hardcoded text with `{t('key')}` using normalized dot-separated key paths.
152
233
 
153
234
  2. **Verify Root Entry Integration**:
154
235
  Ensure `App.tsx` or `app/_layout.tsx` imports `i18n.ts`:
@@ -1,39 +1,84 @@
1
1
  # Expo & React Native Technical Reference
2
2
 
3
- ## Configuration Pattern (`i18n.ts`)
4
- React Native applications require static message loading because dynamic Node.js filesystem operations (`fs.readFile`) are not available in native runtimes.
3
+ ## Deployment Architecture: CDN-Backed Loading (OTA) vs. Local Bundling
5
4
 
5
+ Mobile apps benefit greatly from CDN-backed translation loading because updating text via App Store / Google Play requires full binary reviews.
6
+
7
+ ### 1. CDN Dynamic Loading Pattern (Recommended for Production & OTA Updates)
8
+
9
+ Configure `.env`:
10
+ ```env
11
+ EXPO_PUBLIC_TRANSLATIONS_CDN_URL=https://cdn.yourdomain.com/locales
12
+ ```
13
+
14
+ `i18n.ts`:
6
15
  ```typescript
7
16
  import i18n from 'i18next';
8
17
  import { initReactI18next } from 'react-i18next';
9
18
  import { getLocales } from 'expo-localization';
19
+ import AsyncStorage from '@react-native-async-storage/async-storage';
20
+
21
+ // Local bundled fallbacks (from our specified structure)
22
+ import enFallback from './messages/runtime/en.json';
23
+ import knFallback from './messages/runtime/kn.json';
10
24
 
11
- import enMessages from './messages/runtime/en.json';
12
- import knMessages from './messages/runtime/kn.json';
13
- import hiMessages from './messages/runtime/hi.json';
25
+ const CDN_URL = process.env.EXPO_PUBLIC_TRANSLATIONS_CDN_URL;
14
26
 
15
27
  const resources = {
16
- en: { translation: enMessages },
17
- kn: { translation: knMessages },
18
- hi: { translation: hiMessages },
28
+ en: { translation: enFallback },
29
+ kn: { translation: knFallback },
19
30
  };
20
31
 
21
32
  const deviceLanguage = getLocales()[0]?.languageCode ?? 'en';
22
33
 
34
+ export async function fetchRemoteTranslations(locale: string) {
35
+ if (!CDN_URL) return;
36
+ try {
37
+ const res = await fetch(`${CDN_URL}/${locale}.json`);
38
+ if (res.ok) {
39
+ const messages = await res.json();
40
+ i18n.addResourceBundle(locale, 'translation', messages, true, true);
41
+ await AsyncStorage.setItem(`translations_${locale}`, JSON.stringify(messages));
42
+ }
43
+ } catch (err) {
44
+ // Falls back seamlessly to bundled local messages
45
+ }
46
+ }
47
+
23
48
  i18n
24
49
  .use(initReactI18next)
25
50
  .init({
26
51
  resources,
27
52
  lng: deviceLanguage,
28
53
  fallbackLng: 'en',
29
- interpolation: {
30
- escapeValue: false,
31
- },
54
+ interpolation: { escapeValue: false },
32
55
  });
33
56
 
34
57
  export default i18n;
35
58
  ```
36
59
 
60
+ ### 2. Upload Script (`scripts/upload-locales.mjs`)
61
+ Suggest creating an upload script to sync `messages/runtime/` to CDN/bucket:
62
+ ```javascript
63
+ import fs from 'node:fs';
64
+ import path from 'node:path';
65
+
66
+ // Upload all JSON files in messages/runtime/ to Cloudflare R2 / S3 / Supabase Storage
67
+ const runtimeDir = path.resolve('messages/runtime');
68
+ const files = fs.readdirSync(runtimeDir);
69
+ for (const file of files) {
70
+ console.log(`Uploading ${file} to CDN...`);
71
+ }
72
+ ```
73
+
74
+ ### 3. Legacy Base Locale Migration
75
+ If existing code imported `src/locales/en.json`:
76
+ - Reconcile keys into `localization_base/en.json`.
77
+ - Update imports in `i18n.ts` and `app/_layout.tsx` to point to `./messages/runtime/en.json` (or CDN loader).
78
+ - Deprecate or remove `src/locales/en.json`.
79
+
80
+ ---
81
+
37
82
  ## React Native Component Rules
38
83
  1. **Text Nodes**:
39
84
  Raw strings cannot exist outside `<Text>` in React Native.
@@ -3,6 +3,11 @@
3
3
  ## Extraction Strategy
4
4
  Shallok uses Babel AST parsing (`@babel/parser` and `@babel/traverse`) with full JSX and TypeScript support to safely inspect and transform source files without regular expressions.
5
5
 
6
+ ## Normalized Dot-Paths for Keys (Never Object-Based)
7
+ - **Always use normalized dot-separated key paths**: e.g., `"checkout.pay_now"`, `"auth.login.title"`, `"dashboard.metrics.active_users"`.
8
+ - **Never use object-based or nested object paths**: Catalogs must never store nested object trees.
9
+ - **If an existing project uses object-based keys**: Ask the user for confirmation to flatten them to normalized dot-path keys before running extraction or migrating catalogs.
10
+
6
11
  ## What is Extracted
7
12
  1. **JSX Text Nodes**:
8
13
  - `<h1>Welcome</h1>` → `{t('home.welcome')}`
@@ -26,4 +31,4 @@ Shallok uses Babel AST parsing (`@babel/parser` and `@babel/traverse`) with full
26
31
  - Already localized calls: `t(...)`, `i18n.t(...)`.
27
32
 
28
33
  ## Dry-Run Mode
29
- Always use `localize extract --dry-run` to preview keys and occurrences before committing changes to source files.
34
+ Always use `shallok extract --dry-run` to preview keys and occurrences before committing changes to source files.
@@ -0,0 +1,59 @@
1
+ # Framework Adapters Reference
2
+
3
+ ## Loading Strategies: Local vs. CDN
4
+
5
+ | Framework Type | Recommended Strategy | Primary Rationale | Configuration & ENV |
6
+ | :--- | :--- | :--- | :--- |
7
+ | **Astro** (`astro:i18n`) | **Local** (`messages/runtime/`) | Static builds (SSG) & SSR; zero network hops, zero build dependency on external services. | Bundled in `src/i18n/utils.ts` |
8
+ | **Next.js** (`next-intl`) | **Local** (`messages/runtime/`) | Server Components & SSR; instant First Contentful Paint, zero latency in Node/Edge. | Bundled in `src/i18n/request.ts` |
9
+ | **Expo / React Native** | **CDN-Backed** (+ Local Fallback) | Instant Over-The-Air (OTA) translation & typo fixes without waiting for App Store review. | `EXPO_PUBLIC_TRANSLATIONS_CDN_URL` + AsyncStorage caching + local fallback |
10
+ | **React SPA** (Vite / CRA) | **CDN-Backed** (+ Local Fallback) | Instant copy updates without rebuilding web bundle. | `VITE_TRANSLATIONS_CDN_URL` + localStorage caching + local fallback |
11
+
12
+ ---
13
+
14
+ ## Supported Frameworks & Adapters
15
+
16
+ ### 1. React (`react-i18next`)
17
+ - **Initialization**: Generates `src/i18n.ts` configuring `i18next` with `initReactI18next`.
18
+ - **Runtime Messages**: Generates clean key-value message files in `messages/runtime/<locale>.json`.
19
+ - **CDN Loading**: Suggest `VITE_TRANSLATIONS_CDN_URL` in `.env` with local `messages/runtime/` fallback and `scripts/upload-locales.mjs`.
20
+ - **Legacy Migration**: If `src/main.tsx` or `src/i18n.ts` imported `src/locales/en.json`, migrate to `messages/runtime/` or CDN loader.
21
+ - **AST Transformation**:
22
+ - Adds `import { useTranslation } from 'react-i18next';`
23
+ - Injects `const { t } = useTranslation();` in components
24
+ - Replaces text with `{t('key')}` using normalized dot-separated key paths.
25
+
26
+ ### 2. Next.js (`next-intl`)
27
+ - **App Router Support**:
28
+ - Generates `src/i18n/routing.ts` defining `locales` and `defaultLocale`.
29
+ - Generates `src/i18n/request.ts` using `getRequestConfig` loading local `messages/runtime/<locale>.json`.
30
+ - **Runtime Messages**: Generates runtime message files in `messages/runtime/<locale>.json`.
31
+ - **Loading Strategy**: Local loading recommended for Server Components and SSR.
32
+ - **Legacy Migration**: If `src/i18n/request.ts` or `middleware.ts` imported old locale JSONs, update imports to `messages/runtime/<locale>.json`.
33
+ - **AST Transformation**:
34
+ - Adds `import { useTranslations } from 'next-intl';`
35
+ - Injects `const t = useTranslations();` in components
36
+ - Replaces text with `{t('key')}` using normalized dot-separated key paths.
37
+
38
+ ### 3. Expo / React Native (`expo-localization` + `react-i18next`)
39
+ - **Initialization**: Generates `i18n.ts` using `getLocales` from `expo-localization` and `i18next`.
40
+ - **Runtime Messages**: Generates runtime message files in `messages/runtime/<locale>.json`.
41
+ - **CDN Loading**: Suggest `EXPO_PUBLIC_TRANSLATIONS_CDN_URL` in `.env` with `AsyncStorage` caching and local `messages/runtime/` fallback.
42
+ - **Legacy Migration**: If `app/_layout.tsx` or `src/i18n.ts` imported `src/locales/en.json`, migrate to `messages/runtime/` or CDN loader.
43
+ - **AST Transformation**:
44
+ - Adds `import { useTranslation } from 'react-i18next';`
45
+ - Injects `const { t } = useTranslation();` in mobile components
46
+ - Replaces text inside `<Text>` and attributes like `placeholder` with `{t('key')}` using normalized dot-separated key paths.
47
+
48
+ ### 4. Astro (`astro:i18n`)
49
+ - **Initialization**: Generates `src/i18n/utils.ts` defining `locales`, `defaultLocale`, and `useTranslations` loading local `messages/runtime/<locale>.json`.
50
+ - **Runtime Messages**: Generates runtime message files in `messages/runtime/<locale>.json`.
51
+ - **Loading Strategy**: Local loading recommended for fast static SSG and hybrid SSR.
52
+ - **Legacy Migration**: If `src/i18n/utils.ts` or `.astro` files imported old locale JSONs, update imports to `messages/runtime/<locale>.json`.
53
+ - **Component Usage**:
54
+ - In Astro components frontmatter: `const t = await useTranslations(locale);`
55
+ - Renders `{t('key')}` in templates and islands using normalized dot-separated key paths.
56
+
57
+ ### 5. Vue (`vue-i18n`)
58
+ - **Initialization**: Generates `src/i18n.ts` using `createI18n`.
59
+ - **Runtime Messages**: Generates runtime message files in `messages/runtime/<locale>.json`.
@@ -1,11 +1,37 @@
1
1
  # Translation Engine & Agent-Native Translation Reference
2
2
 
3
3
  ## Agent-Native Translation (Primary Workflow)
4
- When using Shallok via an AI coding agent (e.g., in Claude, Cursor, Copilot, Antigravity):
4
+ When using Shallok via an AI coding agent (e.g., in Antigravity, Claude, Cursor, Copilot):
5
5
  - **No external API keys or LLM provider subscriptions are needed.**
6
6
  - The AI agent itself generates accurate, natural translations for each target locale.
7
- - The AI agent inspects `messages/<locale>.json` or runs `localize status --json` to identify entries with `status: "missing"` or `status: "stale"`.
8
- - The AI agent writes the translated strings into `messages/<locale>.json` (or uses `localize entry set <locale> <key> "<translated_text>"`).
7
+ - **Always verify context first**: Before translating any key, check that all keys in `localization_base/<source_locale>.json` have proper semantic context. If any key lacks context or has generic AST context, generate rich context before translating.
8
+ - **Always translate from `localization_base/<source_locale>.json` and its context**: Never translate raw isolated string fragments. Combine the canonical source `content` and its semantic `context` to generate culturally authentic, grammatically correct translations.
9
+ - Inspect `messages/<locale>.json` or run `shallok status --json` to identify entries with `status: "missing"` or `status: "stale"`.
10
+ - Write the translated strings into `messages/<locale>.json` (or use `shallok entry set <locale> <key> "<translated_text>"`).
11
+
12
+ ## Core Rules for Translations
13
+
14
+ ### 1. Pre-Flight Verification & Persistence
15
+ - Before making changes to a new or existing repo, verify `shallok` is installed and check that `localization.json` and `localization_base/` exist. If not, initialize with `shallok init`.
16
+ - **NEVER delete `localization.json` or `localization_base/`**. They hold the single source of truth, content hashes, version counters, and context.
17
+
18
+ ### 2. Normalized Dot-Paths for Keys
19
+ - All keys must use normalized dot notation (e.g., `"checkout.pay_now"`, `"auth.login.title"`).
20
+ - Never use nested object structures. If an existing codebase uses nested objects for keys, ask the user to convert/flatten them to normalized dot-path keys.
21
+
22
+ ### 3. Standardized `messages/` Folder
23
+ - All translations must reside in `messages/` (`messages/<locale>.json` and `messages/runtime/<locale>.json`).
24
+ - If an existing project uses another directory (such as `locales/`, `src/locales/`, `translations/`), ask the user to switch to `messages/` and migrate existing files.
25
+
26
+ ### 4. Context Verification Before Translation
27
+ - Check all keys in `localization_base/<source_locale>.json` have meaningful semantic context.
28
+ - If any key lacks context, generate clear context describing the screen, element, and user action before initiating translation.
29
+
30
+ ### 5. Always Translate from Source Locale Catalog + Context
31
+ - Read both `content` and `context` from `localization_base/<source_locale>.json`.
32
+ - Preserve all interpolation placeholders (`{name}`, `{count}`, `{amount}`) exactly as formatted.
33
+
34
+ ---
9
35
 
10
36
  ### Translation Schema
11
37
  Each entry in `messages/<locale>.json` maintains:
@@ -35,7 +61,7 @@ The AI agent must strictly preserve all interpolation placeholders in target tra
35
61
  - Kannada: `"ಮರಳಿ ಸ್ವಾಗತ, {userName}! ನಿಮ್ಮ ಕಾರ್ಟ್‌ನಲ್ಲಿ {itemCount} ಐಟಂಗಳು ಇವೆ."`
36
62
  - Hindi: `"वापसी पर स्वागत है, {userName}! आपके कार्ट में {itemCount} आइटम हैं।"`
37
63
 
38
- If a translation drops a placeholder or introduces an invalid placeholder, `localize validate` will fail with an error.
64
+ If a translation drops a placeholder or introduces an invalid placeholder, `shallok validate` will fail with an error.
39
65
 
40
66
  ---
41
67
 
@@ -1,8 +1,22 @@
1
1
  # Per-Key Versioning & Context Reference
2
2
 
3
+ ## Strict Catalog Persistence: NEVER Delete `localization.json` or `localization_base/`
4
+ > [!CAUTION]
5
+ > `localization.json` and `localization_base/` are the permanent foundation of the localization system:
6
+ > - `localization.json` stores project settings, locales, paths, and key strategies.
7
+ > - `localization_base/` stores canonical source strings, per-key version counters (`key_version`), cryptographic SHA-256 hashes (`content_hash`), occurrence positions, and enriched semantic `context`.
8
+ > - **Under NO circumstances should you EVER delete `localization.json` or `localization_base/`**. Deleting them resets version counters, destroys human- and AI-enriched context, breaks incremental translation, and corrupts catalog synchronization.
9
+
10
+ ---
11
+
3
12
  ## Canonical Source Catalog (`localization_base/`)
4
13
  The canonical catalog is the single source of truth for all active and obsolete keys, including their cryptographic content hashes and version counters.
5
14
 
15
+ ### Key Format: Normalized Dot-Separated Paths Only
16
+ - All keys must use normalized dot-separated string paths (e.g. `"checkout.pay_now"`, `"auth.login.title"`).
17
+ - **Never use nested object-based keys**.
18
+ - If an existing project uses object-based keys, ask the user to convert and flatten them to normalized dot-paths before modifying catalogs.
19
+
6
20
  ### Structure of a Canonical Entry
7
21
  ```json
8
22
  {
@@ -26,12 +40,14 @@ The canonical catalog is the single source of truth for all active and obsolete
26
40
  }
27
41
  ```
28
42
 
29
- ## Context Enrichment by AI Agents
30
- 1. When strings are extracted via `localize extract`, the CLI provides baseline AST context (e.g. `JSX text in <button> (in App <button>)`).
31
- 2. The AI Agent analyzes the source code, component hierarchy, and user flow, and enriches the `context` with semantic clarity:
32
- - CLI method: `localize context set <key> "<semantic_description>"`
43
+ ## Context Enrichment & Verification Before Translation
44
+ 1. When strings are extracted via `shallok extract`, the CLI provides baseline AST context (e.g. `JSX text in <button> (in App <button>)`).
45
+ 2. **Mandatory Check Before Translation**: Before generating any translations, check that all active keys in `localization_base/<source_locale>.json` have proper, high-quality semantic context.
46
+ 3. If any key lacks context or has only generic AST descriptions, analyze the source code, component hierarchy, and user flow, and enrich the `context` with semantic clarity:
47
+ - CLI method: `shallok context set <key> "<semantic_description>"`
33
48
  - Direct edit: Update `"context"` in `localization_base/<source_locale>.json`.
34
- 3. **Preservation Guarantee**:
49
+ 4. **Always Translate from Source Locale + Context**: Translations must be derived directly from `localization_base/<source_locale>.json` using BOTH `content` and `context`.
50
+ 5. **Preservation Guarantee**:
35
51
  Shallok's reconciliation algorithm (`reconcileCanonicalCatalog`) checks if a key already has an enriched context. Unless the source code introduces an explicit inline comment (`{/* context: ... */}`), the AI agent's enriched context is **strictly preserved** across all future extractions and will not be overwritten by generic AST node descriptions.
36
52
 
37
53
  ## Reconciliation & Versioning Rules
@@ -54,4 +70,4 @@ In `messages/<locale>.json`, each entry records:
54
70
  }
55
71
  }
56
72
  ```
57
- A translation is **stale** when `entry.source_version < canonicalEntry.key_version`. Stale entries retain their existing translated value for reference until the AI agent updates them.
73
+ A translation is **stale** when `entry.source_version < canonicalEntry.key_version`. Stale entries retain their existing translated value for reference until the AI agent updates them from `localization_base/<source_locale>.json` and its enriched `context`.