langsys-js-typescript 0.4.3 → 0.6.0

package/CHANGELOG.md

@@ -1,10 +1,46 @@
-## 0.4.3 - 2026-07-08
+## 0.6.0 - unreleased
+
+### Fixed
+
+- **`md5` now agrees with a standard UTF-8 MD5 for all input.** It packed UTF-16 code units straight into byte lanes, so it matched a real MD5 only for ASCII. Two consequences, both confirmed against the published 0.5.0 build:
+  - *Cross-SDK divergence (certain, universal).* Every non-ASCII content block received a different `custom_id` in the JS SDK than in `langsys-php`, which hashes UTF-8 bytes. Any project rendering the same block through both SDKs got two catalog entries for one block.
+  - *Self-collision (real, rare).* A character's high byte was OR-ed into the neighbouring byte lane, vanishing whenever that lane already carried those bits — so `"éa"` and `"ǩa"` hashed identically, as did `"Café"`/`"Cafǩ"` and `"Don’t"`/`"Don<0x19>t"`. Distinct content blocks could share one `custom_id`, letting one block's translations serve another's content. A 20,000-phrase scan of realistic copy found no collisions, so reaching one needs a specific pairing — but the failure mode is silent and wrong when it happens.
+
+  The message bit-length was also a code-unit count rather than a byte count, so astral characters (`😀`) mis-lengthed the message independently of the packing bug.
+
+  **Migration is automatic and lossless.** `Translate` now resolves a content block against the corrected id first and falls back to the legacy id when that misses, so blocks registered by an older SDK keep serving their existing translations instead of orphaning. The fallback is **lookup-only** — registration always uses the corrected id, so the legacy-keyed population can only shrink. **Pure-ASCII ids are byte-identical to before**, so only non-ASCII blocks are affected at all.
+
+  Found by the `langsys-skill` and `langsys-php` agents; the collision mechanism and the JSON-offset dependency below were established jointly with them.
+
+### Added
+
+- **`md5Legacy(input)`** and **`generateLegacyCustomId(category, tokens)`** — the pre-0.6.0 hash, exported for migration tooling and cross-SDK reconciliation. Lookup-only; never register new content under these. Deprecated on arrival, to be removed once catalogs have been rebased.
+
+### Notes
+
+- Collisions are a property of the **final hashed string**, not the phrase: `JSON.stringify` shifts every character's offset, so a pair that collides standalone may not collide once wrapped, and changing `category` moves every character into different lanes. Reason at the `generateCustomId` level, never at bare `md5()`.
+
+## 0.5.0 - 2026-08-14
+
+### Changed (breaking, types only)
+
+- **`PhraseOptions.params` is now `Record<string, ParamPrimitive>`**, matching `TranslateOptions.params` and `t()`. It was `Record<string, unknown>`, which allowed values — objects, arrays, functions — that interpolation can only render as `[object Object]`. No runtime behavior changes; this is a compile-time narrowing that surfaces those calls as errors instead of silently rendering garbage. `Phrase.setParams()` now also accepts `undefined`, matching `Translate.setParams()`.
+
+  **Impact — two distinct cases.** First, and most visible to application code: **passing an object or array as a param value no longer compiles** (`params={{ user: someObject }}`). That code was already broken at runtime — interpolation could only render it as `[object Object]` — so this converts a silent bug into a build error, but it does bite at build time. Second: code typing its params as `Record<string, unknown>` and forwarding it no longer compiles, which covered all three framework bindings; the fix there is one line per call site, declaring `Record<string, ParamPrimitive>` (exported from the package root). Usefully, that narrowed declaration compiles against 0.4.x *and* 0.5.0, so wrappers can land it independently of the version bump. Runtime is unaffected throughout — code that skips the type change still *works*, it just fails `tsc`.
+
+### Added
+
+- **`Phrase` is documented in the README for the first time.** It's been exported since 0.1.0 but had no section, so the only documented tool for markup-bearing content was `Translate` — which splits at tag boundaries and is the wrong choice for a sentence. The new section covers the three reasons it exists (grammatical agreement, reordering, phrase-key stability), and the `Translate` section now shows the literal per-text-node split so the trade-off is visible at the point of decision.
+- Warning against building phrases with template literals (`` t(`Hello, ${name}!`) ``), which registers every runtime value as its own catalog phrase and pollutes the catalog shared across every Langsys SDK.
+- Params doc comments now name the `%name%` markup spelling alongside the canonical `{name}`, including Vue's `{{ name }}` case.
+
+## 0.4.3 - 2026-08-08
 
 ### Fixed
 
 - **The params diagnostic no longer names the wrong framework.** 0.4.2's warning hardcoded "the framework compiler (Svelte/JSX)" and the `{key}` spelling, which misdiagnoses Vue users — Vue consumes `{{ key }}`, not single braces, so a Vue developer hitting the warning was pointed at a syntax their framework never touches. The message is now framework-neutral and names both brace spellings.
 
-## 0.4.2 - 2026-07-08
+## 0.4.2 - 2026-08-08
 
 ### Added
 
@@ -50,7 +86,23 @@ Full CLDR compliance, matching the backend's CLDR migration. Everything rides on
 
 - `canonicalizeLocale(locale)` — exported BCP 47 canonicalizer, useful when seeding your own `UserLocaleStore`.
 
-## 0.2.1 - 2026-06-11
+## 0.2.3 - 2026-07-01
+
+> Reconstructed retroactively (2026-08-15) from git history — this release shipped without a changelog entry.
+
+### Fixed
+
+- **`ready()` never settled on an SSR-seeded locale, hanging every content-block consumer.** The SSR handoff calls `markLoaded()` to prime the fetch cache so `change()` short-circuits, but `readyResolve()` lived only in `getTranslations()`. With no fetch ever firing, `Translate` and `Phrase` awaited `ready()` forever on the seeded locale and never rendered. Now resolved in `markLoaded()` — a seeded catalog counts as ready — and in the legacy `skipFetch` branch of `change()`, which had the same hole.
+
+## 0.2.2 - 2026-06-24
+
+> Reconstructed retroactively (2026-08-15) from git history — this release shipped without a changelog entry.
+
+### Fixed
+
+- **Switching back to the initial/SSR locale left the previous language on screen.** The init-time locale subscription recomputed `skipFetch` on every settle as `initialTranslationsLocale === locale && !!initialTranslations`. Since `initialTranslations` stays truthy for the whole session, every *return* to the initial locale skipped `change()`'s store updates, so the previous locale's catalog stayed active and the `t` signal never rebuilt. Replaced the persistent skip flag with `Translations.markLoaded()`, which primes the same `lastLoaded` cache `change()` consults: the first settle after the SSR handoff is still a cache hit with no redundant fetch, but switching back later flows through `change()` normally.
+
+## 0.2.1 - 2026-06-12
 
 ### Changed
 
@@ -60,6 +112,25 @@ Full CLDR compliance, matching the backend's CLDR migration. Everything rides on
 - Missing-phrase registration now chunks into batches of 200 to respect the `/translatable-items` per-request cap.
 - Added `LangsysAppAPI.createTranslatableItems()` as the single write entry point for both phrases and content blocks.
 
+## 0.2.0 - 2026-06-10
+
+> Reconstructed retroactively (2026-08-15). This release shipped without a changelog entry, which is how the reversed `t()` signature survived in `CLAUDE.md` for four months — there was no release note to prompt the update. Details recovered from git history (`v0.1.0..v0.2.0`).
+
+### Changed (breaking)
+
+- **`t()` argument order flipped to `t(phrase, category?, params?)`.** 0.1.0 shipped `t(category, phrase, params?)`; the phrase now comes first and the category is optional. Implemented via overloads that discriminate on the type at position 2 (string ⇒ category, object ⇒ params). This is the single most consequential API change in the package's history and the one most likely to be miswritten from memory: with a placeholder-free phrase, the old order still type-checks, silently registering the category as the phrase. Nothing catches it until someone reads the catalog.
+- Narrowed `LocaleSource`; extracted the framework-agnostic content-block helpers (`generateCustomId`, `isContentBlockKnown`, `registerContentBlock`, `tokenizeElement`) so non-DOM wrappers can reuse registration without the DOM mutator.
+- Missing phrases are sent with a null category on the wire rather than the `__uncategorized__` sentinel, which is server-internal and must not be sent by clients.
+
+### Added
+
+- **`<Phrase>` and `<DontTranslate>`** — markup-sentinel rich-text handling, so a markup-bearing sentence stays one translatable phrase.
+- **ICU MessageFormat** wired through `interpolate()`, with the SDK declaring ICU capability so the server sends raw ICU rather than pre-rendered text.
+
+### Fixed
+
+- Content-block dedup unified, and a `custom_id` hash collision fixed. (Note: the deeper `md5` encoding defect behind that class of collision was not found until 0.6.0.)
+
 ## 0.1.0 - 2026-05-19
 
 Initial public release. Framework-agnostic TypeScript SDK for the Langsys Translation Manager — the foundation that `langsys-js-svelte` (and future framework-specific bindings) build on.

package/README.md

@@ -72,6 +72,15 @@ Allowed value types: `string | number | Date | boolean`. Numbers and Dates forma
 
 > Translations containing ICU MessageFormat syntax — `{count, plural, one {# item} other {# items}}`, `{gender, select, …}`, `{n, number}`, `{d, date}` — are rendered with full CLDR plural rules for the active locale (Arabic's six categories, Russian's four, etc.). Plain `{name}` slots keep working unchanged.
 
+> **⚠️ Always pass values through `params` — never build the phrase yourself.**
+>
+> ```ts
+> t(`Hello, ${name}!`, 'Greetings');              // ❌ never do this
+> t('Hello, {name}!', 'Greetings', { name });     // ✓
+> ```
+>
+> The template literal resolves *before* `t()` runs, so with a write key the SDK registers `"Hello, Sarah!"` as a brand-new phrase — and every distinct runtime value mints another one ("Hello, Priya!", "Hello, Ahmed!"). Your Translation Manager fills with untranslatable one-offs, and since projects share one catalog across SDKs, it pollutes what every other binding reads too. The SDK cannot detect this: an interpolated string is indistinguishable from an authored one. The `params` form registers the phrase once, with the placeholder intact for translators.
+
 #### Categorization disambiguates context
 
 ```ts
@@ -158,6 +167,14 @@ Attributes honored on contained elements:
 - Several validation-message `data-*` attributes
 - `translate="no"` — elements marked this way (and their children) are skipped
 
+> **`Translate` tokenizes per text node — so inline markup splits a sentence.**
+>
+> ```html
+> <p>My content <strong>is the best</strong> when internationalized.</p>
+> ```
+>
+> registers **three** phrases: `"My content"`, `"is the best"`, `"when internationalized."` That's correct for a block of independent runs, and it's what `Translate` is for. But a translator sees three fragments with no context, and neither reordering nor grammatical agreement can cross a fragment boundary — fatal for a sentence whose word order or noun forms change with the language. When inline markup sits *inside a sentence*, reach for [`Phrase`](#the-phrase-class) instead.
+
 ### Interpolation params
 
 `Translate` accepts the same single-brace `{key}` placeholders as `t()`. Pass `params` and every translated text node and attribute is interpolated after lookup — unknown keys stay visible as-is, and `Number`/`Date` values are formatted per the active locale's CLDR rules:
@@ -206,6 +223,33 @@ instead.
 
 The check runs whenever the params key-set changes, so a ticking `count` won't spam the console, and it's silent in production.
 
+## The `Phrase` class
+
+`Phrase` wraps an element whose contents are **one** translatable sentence *even though it contains inline markup*. Where `Translate` splits at tag boundaries, `Phrase` keeps the run whole.
+
+```ts
+import { Phrase } from 'langsys-js-typescript';
+
+const el = document.querySelector<HTMLElement>('#reviews')!;
+new Phrase(el, { category: 'ProductCard', params: { n: reviewCount } });
+```
+
+```html
+<span id="reviews">Based on %n% <strong>reviews</strong></span>
+```
+
+That registers as a single phrase — `Based on {n} {m0o}reviews{m0c}` — rather than `"Based on {n}"` and `"reviews"`. Three things depend on it:
+
+1. **Grammatical agreement.** The count and the noun it inflects must live in one phrase, or no ICU plural rule can select the right noun form. Russian has four plural categories, Arabic six, Polish four — split them apart and the output is confidently wrong, not merely awkward.
+2. **Reordering.** What you write as `<span>White</span> House` renders in Spanish as `Casa <span>Blanca</span>` — the emphasis follows the word it belongs to, across a word-order change. That works because the markup becomes ordinary ICU placeholders (`{m0o}White{m0c} House` → `Casa {m0o}Blanca{m0c}`), so a translator or model *places* them around the translated word rather than being stuck with the original order. Impossible once the sentence is split at the tag boundary.
+3. **Phrase-key stability.** Your real elements never reach the translator — they're replaced by neutral tokens and reconstituted at render. Framework scoped-CSS classes (`svelte-a1b2c3`) change every build; if they rode along in the phrase, the key would drift and silently re-translate everything.
+
+Use it standalone, or nested inside a `Translate` block to protect specific runs — the block tokenizer skips any subtree carrying `data-ls-phrase` (exported as `PHRASE_MARKER_ATTR`), letting the `Phrase` own it. `setParams()` re-renders on a changed count, same as `Translate`.
+
+> **Who sets `data-ls-phrase`.** In the framework bindings it's internal — `<Phrase>` puts it on its own host element and you never write it yourself. In **vanilla usage it is yours to set**: the `Phrase` class does not add it, so a `Phrase` nested inside a `Translate` block needs the attribute present in your HTML *before* the wrapping `Translate` tokenizes, or the block will tokenize straight through the phrase and both will fight over the same nodes. Standalone `Phrase` doesn't need it at all. (Not to be confused with `langsys-php`'s `data-langsys-*` attributes, which are author-facing by design.)
+
+If a translation drops or unbalances a markup token, rendering degrades to plain text with the markers stripped — the meaning survives even when the markup doesn't.
+
 ## Server-Side Rendering
 
 Pre-fetch translations on the server and seed them through `initialTranslations` to skip the duplicate client fetch on hydration:

package/dist/index.d.mts

@@ -375,7 +375,13 @@ interface TranslateOptions {
     custom_id?: string;
     /** Optional human-readable label shown in the Translation Manager. */
     label?: string;
-    /** Interpolation params — `{name}`, `{count}`, etc. Same single-brace syntax as `t()`. */
+    /**
+     * Interpolation params — `{name}`, `{count}`, etc. Same single-brace syntax
+     * as `t()`, and what you write in plain DOM. In a framework component's
+     * markup write `%name%` / `%count%` instead: Svelte and JSX consume
+     * `{name}` at compile time (Vue consumes `{{ name }}`), and `%name%` is
+     * normalized back to `{name}` at capture.
+     */
     params?: Record<string, ParamPrimitive>;
 }
 /**
@@ -433,8 +439,15 @@ declare const PHRASE_MARKER_ATTR = "data-ls-phrase";
 interface PhraseOptions {
     /** Category the phrase registers under (disambiguation for translators). */
     category?: string;
-    /** Interpolation params — `{n}` for pluralization, `{name}`, etc. */
-    params?: Record<string, unknown>;
+    /**
+     * Interpolation params — `{n}` for pluralization, `{name}`, etc.
+     *
+     * Those are the canonical spellings, and what you write in plain DOM. In a
+     * framework component's markup write `%n%` / `%name%` instead: Svelte and
+     * JSX consume `{n}` at compile time (Vue consumes `{{ n }}`), and `%n%` is
+     * normalized back to `{n}` at capture.
+     */
+    params?: Record<string, ParamPrimitive>;
 }
 /**
  * DOM handler for a markup-bearing "rich" phrase (`<Phrase>`).
@@ -460,7 +473,7 @@ declare class Phrase {
     private checkedParamKeys;
     constructor(host: HTMLElement, options?: PhraseOptions);
     /** Update interpolation params (e.g. a changed count) and re-render. */
-    setParams(params: Record<string, unknown>): void;
+    setParams(params: Record<string, ParamPrimitive> | undefined): void;
     /** Stop reacting to locale/translation changes. Safe to call multiple times. */
     destroy(): void;
     private _init;
@@ -599,6 +612,24 @@ interface iContentBlock {
  *      and render via your framework's templating.
  */
 
+/**
+ * Attributes that mark a subtree as a self-managed "keep-together" phrase,
+ * which the block tokenizer must skip rather than split at tag boundaries.
+ *
+ * Two spellings, because two SDKs mark the same thing:
+ *  - `data-ls-phrase` — set by this SDK's framework `<Phrase>` components.
+ *  - `data-langsys-phrase` — `langsys-php`'s author-facing marker, which
+ *    survives into `translatePage()` output.
+ *
+ * The distinction is invisible in normal use, but on SSR handoff there is only
+ * ONE DOM: a page rendered by PHP and then hydrated by a JS SDK is walked by
+ * both implementations, so this tokenizer has to recognise PHP's marker or it
+ * re-tokenizes a subtree PHP deliberately kept whole. Recognising both is
+ * additive — `data-langsys-phrase` never appears in DOM our components emit.
+ */
+declare const PHRASE_MARKER_ATTRS: readonly ["data-ls-phrase", "data-langsys-phrase"];
+/** True when an element is marked as a self-managed phrase by either SDK. */
+declare function isPhraseMarked(element: Element): boolean;
 /**
  * Compute the deterministic content-block id from a `(category, tokens)` pair.
  *
@@ -611,6 +642,23 @@ interface iContentBlock {
  * whether to call `registerContentBlock`.
  */
 declare function generateCustomId(category: string, tokens: string[]): string;
+/**
+ * The id this block would have had before the 0.6.0 MD5 fix.
+ *
+ * **Lookup only — never register under this.** Blocks registered by an older
+ * SDK are keyed by it, so `Translate` falls back to it when the corrected id
+ * misses, which keeps existing translations resolving instead of orphaning
+ * them. Registering under it would keep minting ids from a hash that both
+ * diverges across SDKs and can collide.
+ *
+ * Note the collision is a property of the FINAL hashed string, not the phrase:
+ * `JSON.stringify` shifts every character's offset, so the same phrase pair can
+ * collide standalone and not collide here — and changing `category` moves every
+ * character into different lanes. Always reason at this level, not at `md5()`.
+ *
+ * @deprecated Migration aid; will be removed once catalogs have been rebased.
+ */
+declare function generateLegacyCustomId(category: string, tokens: string[]): string;
 /**
  * True iff the local translations cache already has the content block
  * for the given `(category, customId)` — meaning the backend knows about
@@ -671,7 +719,26 @@ declare const sTranslations: Signal<iCategories>;
 declare const currentlyLoadedLocale: Signal<string>;
 
 declare const isEmpty: (data: unknown) => boolean;
+/**
+ * MD5 of a string, matching a standard UTF-8 MD5 for all input.
+ *
+ * Content-block `custom_id`s are derived from this, so it's a wire identity —
+ * see `generateCustomId`. Before 0.6.0 this packed UTF-16 code units directly
+ * into byte lanes, which agreed with a real MD5 only for ASCII: non-ASCII
+ * input both diverged from every other Langsys SDK and could collide (a
+ * character's high byte was OR-ed into the neighbouring lane, vanishing
+ * whenever that lane already carried those bits — so `éa` and `ǩa` hashed
+ * alike). `md5Legacy` preserves the old behavior for migration lookups only.
+ */
 declare function md5(inputString: string): string;
+/**
+ * The pre-0.6.0 MD5, retained ONLY so existing content blocks stay resolvable:
+ * ids minted by an older SDK are keyed by this. Lookup-only — never register
+ * new content under it, or the colliding-id population keeps growing.
+ *
+ * @deprecated Migration aid; will be removed once catalogs have been rebased.
+ */
+declare function md5Legacy(inputString: string): string;
 
 /**
  * Detects ICU MessageFormat syntax in a translation string.
@@ -778,4 +845,4 @@ interface iProject {
 declare const t: TFunction;
 declare const tSignal: Signal<TFunction>;
 
-export { type EncodedRichText, type ExtractParamKeys, LangsysApp, LangsysAppAPI, type LocaleSource, Logger, PHRASE_MARKER_ATTR, type ParamPrimitive, type ParamsFor, Phrase, type PhraseOptions, type RichSlot, type Signal, type Subscriber, type TArgs, type TFunction, Translate, type TranslateOptions, type TranslationParams, Translations, type Unsubscriber, type Updater, canonicalizeLocale, createSignal, currentlyLoadedLocale, encodeRichText, findUnusedParamKeys, generateCustomId, getValue, type iCategories, type iContentBlock, type iCountry, type iCountryDialCode, type iCountryList, type iCurrency, type iCurrencyList, type iLangsysConfig, type iLangsysInitConfig, type ResponseObject as iLangsysResponse, type iLanguageName, type iLocaleData, type iLocaleDefault, type iLocaleFlat, type iProject, type iTranslations, interpolate, isContentBlockKnown, isEmpty, isICU, logger, markupTokenValues, md5, normalizeMarkupPlaceholders, persist, reconstitute, registerContentBlock, sTranslations, stripSentinels, t, tSignal, tokenizeElement };
+export { type EncodedRichText, type ExtractParamKeys, LangsysApp, LangsysAppAPI, type LocaleSource, Logger, PHRASE_MARKER_ATTR, PHRASE_MARKER_ATTRS, type ParamPrimitive, type ParamsFor, Phrase, type PhraseOptions, type RichSlot, type Signal, type Subscriber, type TArgs, type TFunction, Translate, type TranslateOptions, type TranslationParams, Translations, type Unsubscriber, type Updater, canonicalizeLocale, createSignal, currentlyLoadedLocale, encodeRichText, findUnusedParamKeys, generateCustomId, generateLegacyCustomId, getValue, type iCategories, type iContentBlock, type iCountry, type iCountryDialCode, type iCountryList, type iCurrency, type iCurrencyList, type iLangsysConfig, type iLangsysInitConfig, type ResponseObject as iLangsysResponse, type iLanguageName, type iLocaleData, type iLocaleDefault, type iLocaleFlat, type iProject, type iTranslations, interpolate, isContentBlockKnown, isEmpty, isICU, isPhraseMarked, logger, markupTokenValues, md5, md5Legacy, normalizeMarkupPlaceholders, persist, reconstitute, registerContentBlock, sTranslations, stripSentinels, t, tSignal, tokenizeElement };

package/dist/index.d.ts

@@ -375,7 +375,13 @@ interface TranslateOptions {
     custom_id?: string;
     /** Optional human-readable label shown in the Translation Manager. */
     label?: string;
-    /** Interpolation params — `{name}`, `{count}`, etc. Same single-brace syntax as `t()`. */
+    /**
+     * Interpolation params — `{name}`, `{count}`, etc. Same single-brace syntax
+     * as `t()`, and what you write in plain DOM. In a framework component's
+     * markup write `%name%` / `%count%` instead: Svelte and JSX consume
+     * `{name}` at compile time (Vue consumes `{{ name }}`), and `%name%` is
+     * normalized back to `{name}` at capture.
+     */
     params?: Record<string, ParamPrimitive>;
 }
 /**
@@ -433,8 +439,15 @@ declare const PHRASE_MARKER_ATTR = "data-ls-phrase";
 interface PhraseOptions {
     /** Category the phrase registers under (disambiguation for translators). */
     category?: string;
-    /** Interpolation params — `{n}` for pluralization, `{name}`, etc. */
-    params?: Record<string, unknown>;
+    /**
+     * Interpolation params — `{n}` for pluralization, `{name}`, etc.
+     *
+     * Those are the canonical spellings, and what you write in plain DOM. In a
+     * framework component's markup write `%n%` / `%name%` instead: Svelte and
+     * JSX consume `{n}` at compile time (Vue consumes `{{ n }}`), and `%n%` is
+     * normalized back to `{n}` at capture.
+     */
+    params?: Record<string, ParamPrimitive>;
 }
 /**
  * DOM handler for a markup-bearing "rich" phrase (`<Phrase>`).
@@ -460,7 +473,7 @@ declare class Phrase {
     private checkedParamKeys;
     constructor(host: HTMLElement, options?: PhraseOptions);
     /** Update interpolation params (e.g. a changed count) and re-render. */
-    setParams(params: Record<string, unknown>): void;
+    setParams(params: Record<string, ParamPrimitive> | undefined): void;
     /** Stop reacting to locale/translation changes. Safe to call multiple times. */
     destroy(): void;
     private _init;
@@ -599,6 +612,24 @@ interface iContentBlock {
  *      and render via your framework's templating.
  */
 
+/**
+ * Attributes that mark a subtree as a self-managed "keep-together" phrase,
+ * which the block tokenizer must skip rather than split at tag boundaries.
+ *
+ * Two spellings, because two SDKs mark the same thing:
+ *  - `data-ls-phrase` — set by this SDK's framework `<Phrase>` components.
+ *  - `data-langsys-phrase` — `langsys-php`'s author-facing marker, which
+ *    survives into `translatePage()` output.
+ *
+ * The distinction is invisible in normal use, but on SSR handoff there is only
+ * ONE DOM: a page rendered by PHP and then hydrated by a JS SDK is walked by
+ * both implementations, so this tokenizer has to recognise PHP's marker or it
+ * re-tokenizes a subtree PHP deliberately kept whole. Recognising both is
+ * additive — `data-langsys-phrase` never appears in DOM our components emit.
+ */
+declare const PHRASE_MARKER_ATTRS: readonly ["data-ls-phrase", "data-langsys-phrase"];
+/** True when an element is marked as a self-managed phrase by either SDK. */
+declare function isPhraseMarked(element: Element): boolean;
 /**
  * Compute the deterministic content-block id from a `(category, tokens)` pair.
  *
@@ -611,6 +642,23 @@ interface iContentBlock {
  * whether to call `registerContentBlock`.
  */
 declare function generateCustomId(category: string, tokens: string[]): string;
+/**
+ * The id this block would have had before the 0.6.0 MD5 fix.
+ *
+ * **Lookup only — never register under this.** Blocks registered by an older
+ * SDK are keyed by it, so `Translate` falls back to it when the corrected id
+ * misses, which keeps existing translations resolving instead of orphaning
+ * them. Registering under it would keep minting ids from a hash that both
+ * diverges across SDKs and can collide.
+ *
+ * Note the collision is a property of the FINAL hashed string, not the phrase:
+ * `JSON.stringify` shifts every character's offset, so the same phrase pair can
+ * collide standalone and not collide here — and changing `category` moves every
+ * character into different lanes. Always reason at this level, not at `md5()`.
+ *
+ * @deprecated Migration aid; will be removed once catalogs have been rebased.
+ */
+declare function generateLegacyCustomId(category: string, tokens: string[]): string;
 /**
  * True iff the local translations cache already has the content block
  * for the given `(category, customId)` — meaning the backend knows about
@@ -671,7 +719,26 @@ declare const sTranslations: Signal<iCategories>;
 declare const currentlyLoadedLocale: Signal<string>;
 
 declare const isEmpty: (data: unknown) => boolean;
+/**
+ * MD5 of a string, matching a standard UTF-8 MD5 for all input.
+ *
+ * Content-block `custom_id`s are derived from this, so it's a wire identity —
+ * see `generateCustomId`. Before 0.6.0 this packed UTF-16 code units directly
+ * into byte lanes, which agreed with a real MD5 only for ASCII: non-ASCII
+ * input both diverged from every other Langsys SDK and could collide (a
+ * character's high byte was OR-ed into the neighbouring lane, vanishing
+ * whenever that lane already carried those bits — so `éa` and `ǩa` hashed
+ * alike). `md5Legacy` preserves the old behavior for migration lookups only.
+ */
 declare function md5(inputString: string): string;
+/**
+ * The pre-0.6.0 MD5, retained ONLY so existing content blocks stay resolvable:
+ * ids minted by an older SDK are keyed by this. Lookup-only — never register
+ * new content under it, or the colliding-id population keeps growing.
+ *
+ * @deprecated Migration aid; will be removed once catalogs have been rebased.
+ */
+declare function md5Legacy(inputString: string): string;
 
 /**
  * Detects ICU MessageFormat syntax in a translation string.
@@ -778,4 +845,4 @@ interface iProject {
 declare const t: TFunction;
 declare const tSignal: Signal<TFunction>;
 
-export { type EncodedRichText, type ExtractParamKeys, LangsysApp, LangsysAppAPI, type LocaleSource, Logger, PHRASE_MARKER_ATTR, type ParamPrimitive, type ParamsFor, Phrase, type PhraseOptions, type RichSlot, type Signal, type Subscriber, type TArgs, type TFunction, Translate, type TranslateOptions, type TranslationParams, Translations, type Unsubscriber, type Updater, canonicalizeLocale, createSignal, currentlyLoadedLocale, encodeRichText, findUnusedParamKeys, generateCustomId, getValue, type iCategories, type iContentBlock, type iCountry, type iCountryDialCode, type iCountryList, type iCurrency, type iCurrencyList, type iLangsysConfig, type iLangsysInitConfig, type ResponseObject as iLangsysResponse, type iLanguageName, type iLocaleData, type iLocaleDefault, type iLocaleFlat, type iProject, type iTranslations, interpolate, isContentBlockKnown, isEmpty, isICU, logger, markupTokenValues, md5, normalizeMarkupPlaceholders, persist, reconstitute, registerContentBlock, sTranslations, stripSentinels, t, tSignal, tokenizeElement };
+export { type EncodedRichText, type ExtractParamKeys, LangsysApp, LangsysAppAPI, type LocaleSource, Logger, PHRASE_MARKER_ATTR, PHRASE_MARKER_ATTRS, type ParamPrimitive, type ParamsFor, Phrase, type PhraseOptions, type RichSlot, type Signal, type Subscriber, type TArgs, type TFunction, Translate, type TranslateOptions, type TranslationParams, Translations, type Unsubscriber, type Updater, canonicalizeLocale, createSignal, currentlyLoadedLocale, encodeRichText, findUnusedParamKeys, generateCustomId, generateLegacyCustomId, getValue, type iCategories, type iContentBlock, type iCountry, type iCountryDialCode, type iCountryList, type iCurrency, type iCurrencyList, type iLangsysConfig, type iLangsysInitConfig, type ResponseObject as iLangsysResponse, type iLanguageName, type iLocaleData, type iLocaleDefault, type iLocaleFlat, type iProject, type iTranslations, interpolate, isContentBlockKnown, isEmpty, isICU, isPhraseMarked, logger, markupTokenValues, md5, md5Legacy, normalizeMarkupPlaceholders, persist, reconstitute, registerContentBlock, sTranslations, stripSentinels, t, tSignal, tokenizeElement };

package/dist/index.js

@@ -937,8 +937,23 @@ var isEmpty = (data) => {
   if (typeof data === "object") return Object.keys(data).length === 0;
   return false;
 };
+function toUtf8ByteString(input) {
+  const bytes = new TextEncoder().encode(input);
+  let out = "";
+  for (let i = 0; i < bytes.length; i += 8192) {
+    out += String.fromCharCode(...bytes.subarray(i, i + 8192));
+  }
+  return out;
+}
 function md5(inputString) {
   if (typeof inputString !== "string") inputString = JSON.stringify(inputString);
+  return md5Core(toUtf8ByteString(inputString));
+}
+function md5Legacy(inputString) {
+  if (typeof inputString !== "string") inputString = JSON.stringify(inputString);
+  return md5Core(inputString);
+}
+function md5Core(inputString) {
   const hc = "0123456789abcdef";
   function rh(n) {
     let j, s = "";
@@ -1070,6 +1085,10 @@ var TRANSLATABLE_ATTRIBUTES = [
   "data-required-message",
   "data-pattern-message"
 ];
+var PHRASE_MARKER_ATTRS = ["data-ls-phrase", "data-langsys-phrase"];
+function isPhraseMarked(element) {
+  return PHRASE_MARKER_ATTRS.some((attr) => element.hasAttribute(attr));
+}
 var VALUE_TRANSLATABLE_ELEMENTS = ["button"];
 var VALUE_TRANSLATABLE_INPUT_TYPES = ["submit", "button"];
 var SEMANTIC_STYLE_PROPERTIES = [
@@ -1124,6 +1143,9 @@ var SEMANTIC_STYLE_PROPERTIES = [
 function generateCustomId(category, tokens) {
   return md5(JSON.stringify([category, tokens]));
 }
+function generateLegacyCustomId(category, tokens) {
+  return md5Legacy(JSON.stringify([category, tokens]));
+}
 function isContentBlockKnown(category, customId) {
   const cats = sTranslations.get();
   const lookupCat = category || "__uncategorized__";
@@ -1183,7 +1205,7 @@ function _walkForTokens(liveRoot, cloneNodes, tokens, indices) {
     if (node.nodeType === Node.ELEMENT_NODE) {
       const el = node;
       if (el.getAttribute("translate") === "no") return;
-      if (el.hasAttribute("data-ls-phrase")) return;
+      if (isPhraseMarked(el)) return;
     }
     if (node.hasChildNodes()) {
       _applyStylesToClone(liveRoot, node, [...indices, index]);
@@ -1333,7 +1355,8 @@ var Translate = class {
     return true;
   }
   async handleContentBlock(contentBlock) {
-    if (isEmpty(this.custom_id)) {
+    const derivedId = isEmpty(this.custom_id);
+    if (derivedId) {
       this.custom_id = generateCustomId(contentBlock.category, contentBlock.tokens);
     }
     contentBlock.custom_id = this.custom_id;
@@ -1345,6 +1368,17 @@ var Translate = class {
       }
       return;
     }
+    if (derivedId) {
+      const legacyId = generateLegacyCustomId(contentBlock.category, contentBlock.tokens);
+      if (legacyId !== this.custom_id && isContentBlockKnown(contentBlock.category, legacyId)) {
+        this.custom_id = legacyId;
+        if (this.tokens.length > 1 && this.element) {
+          this.translate(Array.from(this.element.childNodes));
+          this.lastTranslatedLocale = currentlyLoadedLocale.get();
+        }
+        return;
+      }
+    }
     void registerContentBlock(contentBlock);
     if (this.tokens.length > 1 && this.element) {
       this.translate(Array.from(this.element.childNodes));
@@ -1360,7 +1394,7 @@ var Translate = class {
       if (node?.nodeType === Node.ELEMENT_NODE) {
         const element = node;
         if (element.getAttribute("translate") === "no") return;
-        if (element.hasAttribute("data-ls-phrase")) return;
+        if (isPhraseMarked(element)) return;
         this.translateAttributes(node);
       }
       if (node?.nodeType !== Node.TEXT_NODE) {
@@ -1605,6 +1639,7 @@ exports.LangsysApp = LangsysApp;
 exports.LangsysAppAPI = LangsysAppAPI;
 exports.Logger = Logger;
 exports.PHRASE_MARKER_ATTR = PHRASE_MARKER_ATTR;
+exports.PHRASE_MARKER_ATTRS = PHRASE_MARKER_ATTRS;
 exports.Phrase = Phrase;
 exports.Translate = Translate;
 exports.Translations = Translations;
@@ -1614,14 +1649,17 @@ exports.currentlyLoadedLocale = currentlyLoadedLocale;
 exports.encodeRichText = encodeRichText;
 exports.findUnusedParamKeys = findUnusedParamKeys;
 exports.generateCustomId = generateCustomId;
+exports.generateLegacyCustomId = generateLegacyCustomId;
 exports.getValue = getValue;
 exports.interpolate = interpolate;
 exports.isContentBlockKnown = isContentBlockKnown;
 exports.isEmpty = isEmpty;
 exports.isICU = isICU;
+exports.isPhraseMarked = isPhraseMarked;
 exports.logger = logger;
 exports.markupTokenValues = markupTokenValues;
 exports.md5 = md5;
+exports.md5Legacy = md5Legacy;
 exports.normalizeMarkupPlaceholders = normalizeMarkupPlaceholders;
 exports.persist = persist;
 exports.reconstitute = reconstitute;