@techninja/clearstack 0.4.1 → 0.4.4

package/templates/shared/docs/clearstack/I18N.md

@@ -4,144 +4,319 @@
 
 Hardcoding UI strings is a foot gun. Retrofitting i18n into an existing app
 requires touching every component — a painful, error-prone refactor. Starting
-with `t('key')` or `msg\`` from day one costs almost nothing and keeps the door
-open while also clearly separating UI copy from component logic.
+with `msg\`` from day one costs almost nothing and keeps the door open while
+also clearly separating UI copy from component logic.
 
-## Two Patterns — Use Both
+## The Pattern — Hybrids-Native (`localize` + `msg\``)
 
-Clearstack supports two complementary approaches. They are not mutually
-exclusive; most apps will use both.
+Hybrids has first-class i18n built in. Template text content is **translated
+automatically** — no source changes needed for simple strings. Use `msg\`` for
+dynamic values, plural forms, and attribute content.
+
+```js
+import { html, define, localize, msg } from 'hybrids';
 
-### Pattern A — `t()` cascade (non-template strings)
+// Simple static text — translated automatically, no code change needed
+html`<button>Save</button>`
 
-A 4-layer message cascade resolved at call time. Best for strings outside
-Hybrids templates: error messages, queue status, validators, utility functions.
+// Dynamic value in a template
+html`<div>${msg`${count} variants scored`}</div>`
 
+// As an attribute (title, placeholder, aria-label)
+html`<button title="${msg`Add item`}">+</button>`
+
+// Outside a template (utils, handlers, queues) — works anywhere
+const status = msg`Loading…`;
 ```
-1. App defaults     → DEFAULTS in utils/i18n.js (English, ships with app)
-2. Locale file      → /locales/<lang>.json (e.g. es.json)
-3. Overrides        → /locales/overrides.json (project customizes English)
-4. Locale overrides → /locales/overrides.<lang>.json (project customizes locale)
+
+---
+
+## Six Translation Scenarios
+
+Understanding which strings need what treatment is the key to avoiding bugs.
+
+| Scenario | Auto-translated? | What to do |
+|---|---|---|
+| Static text in `html\`` template | ✅ Yes | Nothing — Hybrids handles it |
+| Attributes (title, placeholder, aria-label) | ❌ No | Wrap value with `msg\`` |
+| Dynamic expressions in helper functions | ❌ No | Wrap with `msg\`` |
+| Module-level constants | ❌ No | Use lazy getters (see below) |
+| Plural forms | ❌ No | Use `msg\`` + dictionary entry |
+| Strings outside templates (utils, handlers) | ❌ No | Use `msg\`` (works anywhere) |
+
+### The Module-Level Constant Trap
+
+```js
+// BAD — msg evaluates at import time, before locale loads
+const TABS = [
+  { id: 'home', label: msg`Home` },
+];
+
+// GOOD — lazy getter defers evaluation to render time
+const TABS = [
+  { id: 'home', get label() { return msg`Home`; } },
+];
 ```
 
+This matters because ES module static imports resolve before top-level `await`
+executes. Any `msg` call in a module-level constant evaluates before the locale
+file finishes loading.
+
+---
+
+## App Init — Timing Is Everything
+
+`localize()` **must** be called before templates compile (first render).
+Use top-level `await` in the router module:
+
 ```js
-import { t, loadLocale } from '#utils/i18n.js';
+// router/index.js
+import '#utils/i18n-init.js';  // sync — registers English plurals
 
-// Call once at app init
-await loadLocale(navigator.language);
+// Load locale translations — MUST await before any component renders
+const _lang = navigator.language?.slice(0, 2);
+if (_lang && _lang !== 'en') {
+  await import(`#locales/${_lang}.js`).catch(() => {});
+}
 
-t('general.loading')                    // → 'Loading…'
-t('task.empty')                         // → 'No tasks yet.'
-t('greeting', { name: 'James' })        // → 'Hello, James!'
-t('missing.key')                        // → 'missing.key' (never throws)
+import AppView from '#pages/app/view.js';
+// ... define router component
 ```
 
-### Pattern B — `localize` + `msg\`` (Hybrids-native, template strings)
+**Why `await`?** Without it, the dynamic import is fire-and-forget. Components
+render before translations register, and static template text compiles with
+English keys permanently (Hybrids compiles templates once on first render).
 
-Hybrids has first-class i18n built in. Template text content is **translated
-automatically** — no source changes needed for simple strings. Use `msg\`` for
-dynamic values, plural forms, and attribute content.
+---
+
+## File Layout
+
+```
+src/
+├── utils/
+│   └── i18n-init.js      ← English plural forms + identity entries
+├── locales/
+│   └── es.js             ← Full Spanish dictionary
+└── router/
+    └── index.js          ← await import locale before render
+```
+
+### `i18n-init.js` — English Plurals + Identity Entries
 
 ```js
-import { html, define, localize, msg } from 'hybrids';
+import { localize } from 'hybrids';
 
-// Simple static text — translated automatically, no code change needed
-html`<button>Save</button>`
+localize('default', {
+  // Plural forms (English needs these for correct singular/plural)
+  '${0} items scored': {
+    message: { one: '${0} item scored', other: '${0} items scored' },
+  },
+
+  // Identity entries — required for every msg``-wrapped string
+  'Add item': { message: 'Add item' },
+  'Loading…': { message: 'Loading…' },
+});
+```
 
-// Dynamic value in a template
-html`<div>${msg`${count} variants scored`}</div>`
+**Why identity entries?** Once any `localize()` call populates the dictionary,
+Hybrids logs "Missing translation" warnings for any `msg` key not found. Since
+`i18n-init.js` always loads (for English plurals), the dictionary is never
+empty. Every `msg`-wrapped string must be registered.
 
-// As an attribute
-html`<my-button label="${msg`Submit`}"></my-button>`
+Static template text does NOT need identity entries — Hybrids auto-extracts
+those keys during template compilation.
 
-// Load translations once before first render (router/index.js or app init)
+### `locales/es.js` — Full Locale Dictionary
+
+```js
 import { localize } from 'hybrids';
-const msgs = await fetch('/locales/es.json').then(r => r.json());
-localize('es', msgs);
+
+localize('default', {
+  // Static template text (auto-extracted by `hybrids extract`)
+  'Save': { message: 'Guardar' },
+  'Cancel': { message: 'Cancelar' },
+
+  // msg``-wrapped strings
+  'Add item': { message: 'Agregar elemento' },
+  'Loading…': { message: 'Cargando…' },
+
+  // Plural forms
+  '${0} items scored': {
+    message: { one: '${0} elemento puntuado', other: '${0} elementos puntuados' },
+  },
+});
 ```
 
-#### Plural Forms
+**Why `'default'` not `'es'`?** The `'default'` key is appended to the language
+search list as a universal fallback. This means a single file works for `es`,
+`es-MX`, `es-AR` without registering each variant. Only load it for non-English
+users (via the conditional `await import`).
 
-The `msg\`` helper uses`Intl.PluralRules` automatically when the dictionary
-entry is an object with plural keys:
+---
+
+## Plural Forms
+
+### Single Noun
 
 ```js
-// In a template:
-html`<p>${msg`${count} variants matched`}</p>`
+html`<p>${msg`${count} items in cart`}</p>`
+```
 
-// In locales/es.json (or via localize()):
-{
-  "${0} variants matched": {
-    "message": {
-      "one": "${0} variante encontrada",
-      "other": "${0} variantes encontradas",
-      "zero": "Ninguna variante encontrada"
-    }
-  }
-}
+Hybrids uses the **first** `${N}` value for `Intl.PluralRules` selection:
+
+```js
+localize('default', {
+  '${0} items in cart': {
+    message: { one: '${0} item in cart', other: '${0} items in cart' },
+  },
+});
 ```
 
-For English-only apps with irregular plurals, use `msg\`` with a`localize('default', ...)` entry:
+### Multiple Nouns in One String
+
+Each noun needs its own `msg` call for independent plural selection:
 
 ```js
-// Avoids inline ternaries like: `${n} trait${n !== 1 ? 's' : ''}`
-html`<p>${msg`${n} traits scored`}</p>`
+// BAD — only first value drives plural selection
+msg`${files} files and ${folders} folders`
+
+// GOOD — nested msg calls
+msg`${msg`${files} file${files}`} and ${msg`${folders} folder${folders}`}`
+```
 
+The inner `msg` calls produce keys like `${0} file${1}` — the second
+placeholder is the same count value, used as a "hint" that this is a
+pluralizable noun:
+
+```js
 localize('default', {
-  '${0} traits scored': {
-    message: { one: '${0} trait scored', other: '${0} traits scored' }
-  }
+  '${0} file${1}': {
+    message: { one: '${0} file', other: '${0} files' },
+  },
+  '${0} folder${1}': {
+    message: { one: '${0} folder', other: '${0} folders' },
+  },
+  '${0} and ${1}': { message: '${0} and ${1}' },
 });
 ```
 
-## Which Pattern to Use
+### Zero Special Case
 
-| Situation | Pattern |
-|---|---|
-| String inside `html\`` template | B — auto-translated or `msg\`` |
-| Plural form | B — `msg\`` + plural object |
-| String outside a template (util, queue, validator) | A — `t()` |
-| Attribute value on a component | B — `msg\`` in expression |
-| Brand voice / English overrides | A — `overrides.json` |
-| Adding a full language | B — `localize(lang, msgs)` |
+```js
+'${0} items': {
+  message: {
+    zero: 'No items',           // count === 0
+    one: '${0} item',           // count === 1
+    other: '${0} items',        // everything else
+  },
+}
+```
 
-## App Init
+---
 
-Call `loadLocale()` (Pattern A) and/or `localize()` (Pattern B) once before
-first render. The router's `connect` is the right place in a Hybrids app:
+## Helper Functions — The Invisible Boundary
+
+Strings in standalone helper functions (outside `html\`` templates) are
+**invisible to Hybrids' auto-translation**. This is the most common source
+of missed translations.
 
 ```js
-// router/index.js
-import { loadLocale } from '#utils/i18n.js';
-import { localize } from 'hybrids';
+// BAD — stat() is a helper, label is a dynamic expression
+function stat(label, value) {
+  return html`<span class="stat-label">${label}</span>`;
+}
+stat('Coverage', '95%');  // "Coverage" never gets translated
+
+// GOOD — wrap at the call site
+stat(msg`Coverage`, '95%');
+```
 
-// Pattern A — cascade for non-template strings
-loadLocale(navigator.language);
+**Rule**: If a string passes through a function parameter before reaching a
+template, it needs `msg` at the call site.
 
-// Pattern B — load a locale file for template auto-translation
-const lang = navigator.language.split('-')[0];
-if (lang !== 'en') {
-  fetch(`/locales/${lang}.json`)
-    .then(r => r.ok ? r.json() : {})
-    .then(msgs => localize(lang, msgs));
+---
+
+## Attributes
+
+Hybrids auto-translates text content inside elements but **never** attributes:
+
+```js
+// "Save" auto-translates, but title does NOT
+html`<button title="Save changes">Save</button>`
+
+// Fix: wrap attribute value with msg
+html`<button title="${msg`Save changes`}">Save</button>`
+```
+
+Common attributes that need `msg`:
+
+- `title` (tooltips)
+- `placeholder` (inputs)
+- `aria-label` / `aria-description`
+- Custom element string attributes
+
+---
+
+## Content Data Translations (Large Datasets)
+
+For apps with large translatable content (product catalogs, editorial content,
+CMS data), use a separate data-layer approach:
+
+```js
+// utils/i18n-data.js — fetches translation packs from CDN
+export async function loadContentPack(lang) { ... }
+export function translateItem(item) { ... }
+```
+
+Key patterns:
+
+- **Lazy loading** — only fetch when needed (e.g., gene explorer opens)
+- **In-place mutation** — translate object fields directly
+- **Preserve originals** — store English in `_field_en` for bilingual search
+- **Identity for English** — `loadContentPack()` returns null immediately,
+  all translate functions no-op
+
+### Bilingual Search
+
+When translations mutate searchable fields in-place, search must match both:
+
+```js
+function translateItem(item) {
+  if (!_pack) return;
+  item._name_en = item.name;          // preserve original
+  item.name = _pack[item.id]?.name;   // overwrite with translation
+}
+
+function matchesSearch(item, query) {
+  return item.name.includes(query) || item._name_en?.includes(query);
 }
 ```
 
-## Key Naming Convention (Pattern A)
+### Category/Tag Lookups
 
-`domain.concept` — dot-separated, lowercase, no spaces. Keys should read as
-documentation.
+Finite enumerable sets (categories, tags, statuses) translate via flat maps:
 
+```json
+{
+  "categories": { "Electronics": "Electrónica", "Books": "Libros" },
+  "tags": { "bestseller": "más vendido", "new": "nuevo" }
+}
 ```
-general.loading     general.error      general.save       general.cancel
-nav.home            nav.back           nav.signIn
-error.notFound      error.offline
-task.empty          task.addTask       task.deleteConfirm
+
+Use the English key internally (filtering, routing, Set membership) and the
+translated value for display only:
+
+```js
+// Internal — always English key
+const filtered = items.filter(i => i.category === activeCategory);
+
+// Display — translated
+html`<span>${translateCategory(item.category)}</span>`
 ```
 
-Avoid `button1`, `label`, or anything that doesn't describe the content.
+---
 
-## Extracting Keys (Pattern B)
+## Extracting Keys
 
 Hybrids ships a CLI extractor that scans source files and generates a
 translation-ready JSON file:
@@ -151,47 +326,133 @@ npx hybrids extract ./src ./locales/en.json
 npx hybrids extract --include-path ./src ./locales/en.json
 ```
 
-Wire it as a script so translators always have a fresh key file:
+Wire it as a script:
 
 ```json
 { "scripts": { "i18n:extract": "hybrids extract ./src ./locales/en.json" } }
 ```
 
-The extractor is optional — use it when you're ready to hand off to a
-translator or add a second language.
+**What the extractor catches:**
+
+- Static text content in `html\`` templates
+- `msg\`` calls (both in templates and standalone)
+
+**What it misses:**
+
+- Strings passed as arguments to helper functions
+- Template literals without `msg` tag
+- Content from external data sources
+
+---
+
+## Validation Script
+
+Add `i18n:check` to catch drift between source and locale files:
+
+```bash
+pnpm run i18n:check
+```
+
+What it reports:
+
+```
+i18n check (es)
+  ✓ 247/247 template keys translated
+  ✗ 3 msg`` keys missing: 'Rescore', 'Data Source', 'Imputed'
+  ⚠ 2 stale keys (removed from source, still in locale)
+  ✓ 12/12 plural forms complete
+```
+
+Implementation:
+
+1. Run `hybrids extract` to get all template keys
+2. Scan source for `msg\`` calls via regex to get dynamic keys
+3. Compare against each locale file
+4. Report missing, stale, and incomplete plural entries
+
+Wire into CI to catch regressions:
+
+```yaml
+- run: pnpm run i18n:check --fail-on-missing
+```
+
+---
 
 ## Adding a Language
 
-1. Run `npm run i18n:extract` to get the current key list
-2. Translate into `locales/<lang>.json` — only include keys that differ
-3. Load it at init via `localize(lang, msgs)`
-4. For Pattern A overrides: add `locales/overrides.<lang>.json`
+1. Run `pnpm run i18n:extract` to get the current key list
+2. Create `locales/<lang>.js` with `localize('default', { ... })`
+3. Add the conditional import in `router/index.js`
+4. Run `pnpm run i18n:check` to verify coverage
+
+---
+
+## Common Pitfalls
 
-## Spec Check
+| Pitfall | Symptom | Fix |
+|---|---|---|
+| Locale loads after first render | Static text stuck in English forever | `await` the locale import |
+| `msg` in module-level constant | Returns English despite locale loaded | Use lazy getter |
+| String in helper function param | Never translated | Wrap with `msg` at call site |
+| Attribute value | Never translated | Use `msg` in expression |
+| Missing identity entry | Console warning, works in English, breaks locale lookup | Add to `i18n-init.js` |
+| Translated category used for filtering | Filters break (Set mismatch) | Use English key internally, translate for display only |
+| Plain template literal for plural | "1 items" | Use `msg` + plural dictionary entry |
+| Multiple plurals in one string | Only first noun pluralizes correctly | Nested `msg` calls |
 
-`npm run spec code i18n` reports readiness without blocking the build:
+---
 
+## Pattern A — `t()` Cascade (Optional)
+
+> **Note**: In practice, `msg\`` covers all use cases including strings outside
+> templates. Pattern A is available for projects that prefer structured keys
+> (`domain.concept`) over natural-language keys, or need a 4-layer override
+> cascade for white-labeling.
+
+A 4-layer message cascade resolved at call time:
+
+```
+1. App defaults     → DEFAULTS in utils/i18n.js (English, ships with app)
+2. Locale file      → /locales/<lang>.json
+3. Overrides        → /locales/overrides.json (project customizes English)
+4. Locale overrides → /locales/overrides.<lang>.json
 ```
-✅ i18n readiness (12ms)
-   Pattern:   t() ×9, msg` ×3
-   Init:      yes · Overrides: yes · Languages: es, fr
-   Unwrapped: ~4 prose strings across 2 template files
+
+```js
+import { t, loadLocale } from '#utils/i18n.js';
+await loadLocale(navigator.language);
+
+t('general.loading')                    // → 'Loading…'
+t('greeting', { name: 'James' })        // → 'Hello, James!'
+t('missing.key')                        // → 'missing.key' (never throws)
 ```
 
-- **Pattern** — which i18n calls are present and how many files use them
-- **Init** — whether `loadLocale` or `localize` is called at startup
-- **Overrides** — whether `locales/overrides.json` exists
-- **Languages** — locale files found beyond English
-- **Unwrapped** — estimated prose strings in templates not yet wrapped in
-  `t()`/`msg\`` (heuristic, not exhaustive)
+### When to Use Pattern A Over `msg\``
+
+- White-label apps where English copy itself varies per deployment
+- Structured key namespacing (`error.notFound` vs natural text)
+- Server-side rendering where Hybrids isn't available
+- Shared validation/error messages between client and server
+
+### Key Naming Convention
+
+`domain.concept` — dot-separated, lowercase, no spaces:
+
+```
+general.loading     general.error      general.save       general.cancel
+nav.home            nav.back           nav.signIn
+error.notFound      error.offline
+```
 
-The check always passes — it surfaces gaps, not failures.
+---
 
 ## Rules
 
-- **Always use `t()` or `msg\`** for any user-visible string in components
-- **Never hardcode UI strings** directly in render functions
+- **Use `msg\`` for any user-visible string** that isn't static template text
+- **Never hardcode UI strings** in helper function parameters
 - **Plural forms belong in the dictionary**, not inline ternaries
-- **`loadLocale()` is safe to call multiple times** — last call wins
-- **Locale files are optional** — English-only apps just use `overrides.json`
-- **Keys fall back to themselves** — `t('missing')` returns `'missing'`, never throws
+- **`await` the locale import** before any component renders
+- **Locale files are optional** — English-only apps just use `i18n-init.js`
+- **Keys fall back to themselves** — missing translations show English, never crash
+- **Use English keys internally** for filtering/routing, translate for display
+- **Test with a non-English locale** early — don't wait until "i18n sprint"