langsys-js-typescript 0.6.1 → 0.6.3

package/CHANGELOG.md

@@ -1,3 +1,44 @@
+## 0.6.3 - 2026-08-16
+
+### Fixed
+
+- **`<select>` option text was harvested twice, diverging `custom_id` from every other Langsys SDK.** A `<select>` special case pushed each `<option>`'s text, and the ordinary text-node walk then pushed it again — `<option>` cannot contain elements, so the walker already covered it. Any content block containing a `<select>` therefore produced `["S","S"]` where `langsys-php` produced `["S"]`, and had a different `custom_id` in the two SDKs since before either was audited. The fixture suite could not see it: those fixtures are synthetic token lists that never touch a DOM.
+
+  **Migration is automatic and lookup-only**, extending the mechanism 0.6.0 introduced. Three historical id shapes are now tried before registering:
+
+  | tokens | hash | registered by |
+  |---|---|---|
+  | corrected | corrected | 0.6.3+ (current) |
+  | legacy (duplicated) | corrected | 0.6.0 – 0.6.2 |
+  | legacy (duplicated) | legacy | before 0.6.0 |
+
+  Registration always uses the corrected id, so the legacy-keyed population can only shrink. Content without a `<select>` is unaffected — its token list is identical either way, so nothing needlessly rebases. Without this fallback the upgrade would have been silent in the worst direction: a dropdown quietly reverting to source text, with nothing erroring to explain it.
+
+- **`legacyTokenizeElement(element)`** — the pre-0.6.3 token list, exported for migration tooling. Lookup-only; never register under an id derived from it. Deprecated on arrival.
+
+### Added
+
+- **Four standard attributes are now harvested: `label`, `aria-description`, `aria-valuetext`, `aria-roledescription`.** `langsys-php` harvests a superset of our list; diffing the two showed nothing JS-only and 16 PHP-only, of which these four carry real user-visible text. `label` is what a user reads in a `<select>` picker (`<option>`, `<optgroup>`, `<track>`) and is invisible to sighted testing that only inspects option text; the three ARIA attributes are read aloud by screen readers, so leaving them untranslated degrades accessibility specifically for the users least able to work around it.
+
+  The remaining twelve PHP-only attributes are framework conventions (Bootstrap `data-bs-*`, Rails-style `data-confirm`, loading/notification text) and are deliberately **not** mirrored — they're written by server-rendered templates, and a JS app renders those strings through its own components.
+
+  **Note:** a content block containing one of these attributes now yields a different token list, and therefore a different `custom_id`. Such blocks re-register and their existing translations orphan. The 0.6.0 legacy-id fallback cannot help here — it re-hashes the *same* tokens, and this changes the tokens themselves. Affected population is only blocks that use these four attributes.
+
+## 0.6.2 - 2026-08-16
+
+### Added
+
+- **`data-notrans` is now honored as an alias for `translate="no"`.** `langsys-php` accepts it as an author-facing opt-out, for hosts whose templating strips unknown bare attributes or where `translate` collides with another tool — and it survives into `translatePage()` output. On SSR handoff there is one DOM, so content an author marked "do not translate" was being extracted and registered by whichever SDK walked it. Presence is intent; an explicit `"false"`/`"0"` opts out of the opt-out, compared after trimming. `translate="no"` is now matched case-insensitively too, also mirroring PHP.
+- **`isTranslationExcluded(element)`** — exported alongside `isPhraseMarked`, for wrappers that tokenize through their own templating rather than delegating to the DOM class.
+
+### Fixed
+
+- **Phrase-marker values are trimmed before comparison**, so `data-langsys-phrase=" false "` opts out like `="false"` does. This matched `langsys-php` at the time it shipped; auditing the difference showed the gap was drift — their exclusion check gained a trim their phrase check never did — and they normalized on their side, so both SDKs now trim in both places. `isPhraseMarked` is also the method name in both SDKs now, so the correspondence is checkable at a glance.
+
+### Notes
+
+- Asking `langsys-php` whether to mirror `data-notrans` rather than implementing against their code surfaced a bug on their side: their truthiness test was inverted at both ends — bare `data-notrans` did nothing, while `data-notrans="false"` excluded. Had this been implemented from their source as it stood, it would have faithfully reproduced that. Their fix landed first; this mirrors the corrected semantics.
+
 ## 0.6.1 - 2026-08-15
 
 ### Fixed

package/dist/index.d.mts

@@ -628,14 +628,37 @@ interface iContentBlock {
  * additive — `data-langsys-phrase` never appears in DOM our components emit.
  */
 declare const PHRASE_MARKER_ATTRS: readonly ["data-ls-phrase", "data-langsys-phrase"];
+/**
+ * True when an element opts out of translation entirely — it and its subtree
+ * are skipped by the tokenizer.
+ *
+ * Mirrors `langsys-php`'s `HtmlParser::isTranslationExcluded()` exactly:
+ *  - `translate="no"` — the HTML standard attribute, matched case-insensitively.
+ *  - `data-notrans` — PHP's author-facing alias, for hosts whose templating
+ *    strips unknown bare attributes or where `translate` collides with another
+ *    tool. Presence is intent; an explicit `"false"`/`"0"` opts out of the
+ *    opt-out, compared after trimming.
+ *
+ * Honoring PHP's alias matters for the same reason as the phrase marker: on SSR
+ * handoff there is ONE DOM, and content an author marked "do not translate"
+ * must not be extracted and registered by whichever SDK happens to walk it.
+ *
+ * Both this and `isPhraseMarked` trim before comparing. They briefly differed
+ * — PHP's phrase check predated the trim its exclusion check gained — and the
+ * gap was resolved by asking rather than tidying: mirror exactly, flag the
+ * wart, let the owner decide. Divergence-by-tidying is the same failure class
+ * as divergence-by-oversight, just better intentioned.
+ */
+declare function isTranslationExcluded(element: Element): boolean;
 /**
  * True when an element is marked as a self-managed phrase by either SDK.
  *
  * Presence alone means intent, like any boolean HTML attribute — but an
- * explicit `="false"` or `="0"` opts OUT, matching `langsys-php`'s
- * `hasPhraseAttribute()` exactly. Without the opt-out we would skip a subtree
- * the author had deliberately un-marked, and it would then be translated by
- * neither SDK.
+ * explicit `="false"` or `="0"` opts OUT, compared after trimming. Mirrors
+ * `langsys-php`'s `isPhraseMarked()` exactly (same method name in both SDKs,
+ * so the correspondence is checkable at a glance). Without the opt-out we
+ * would skip a subtree the author had deliberately un-marked, and it would
+ * then be translated by neither SDK.
  */
 declare function isPhraseMarked(element: Element): boolean;
 /**
@@ -715,6 +738,23 @@ declare function tokenizeElement(element: HTMLElement): {
     tokens: string[];
     content: string;
 };
+/**
+ * The token list this element would have produced before 0.6.3, when every
+ * `<option>`'s text was harvested twice — once by a `<select>` special case and
+ * again by the ordinary text-node walk.
+ *
+ * **Lookup only.** Content blocks registered by an older SDK are keyed by an id
+ * derived from this list, so `Translate` falls back to it when the corrected id
+ * misses; without that, every block containing a `<select>` would silently
+ * re-key and lose its translations on upgrade. Registration always uses the
+ * corrected list, so the legacy-keyed population can only shrink.
+ *
+ * Skips the computed-style capture that `tokenizeElement` performs — this is a
+ * cache-key computation, not a snapshot for translators.
+ *
+ * @deprecated Migration aid; will be removed once catalogs have been rebased.
+ */
+declare function legacyTokenizeElement(element: HTMLElement): string[];
 
 /**
  * Create a Signal whose value is persisted to localStorage (when available).
@@ -853,4 +893,4 @@ interface iProject {
 declare const t: TFunction;
 declare const tSignal: Signal<TFunction>;
 
-export { type EncodedRichText, type ExtractParamKeys, LangsysApp, LangsysAppAPI, type LocaleSource, Logger, PHRASE_MARKER_ATTR, PHRASE_MARKER_ATTRS, type ParamPrimitive, type ParamsFor, Phrase, type PhraseOptions, type RichSlot, type Signal, type Subscriber, type TArgs, type TFunction, Translate, type TranslateOptions, type TranslationParams, Translations, type Unsubscriber, type Updater, canonicalizeLocale, createSignal, currentlyLoadedLocale, encodeRichText, findUnusedParamKeys, generateCustomId, generateLegacyCustomId, getValue, type iCategories, type iContentBlock, type iCountry, type iCountryDialCode, type iCountryList, type iCurrency, type iCurrencyList, type iLangsysConfig, type iLangsysInitConfig, type ResponseObject as iLangsysResponse, type iLanguageName, type iLocaleData, type iLocaleDefault, type iLocaleFlat, type iProject, type iTranslations, interpolate, isContentBlockKnown, isEmpty, isICU, isPhraseMarked, logger, markupTokenValues, md5, md5Legacy, normalizeMarkupPlaceholders, persist, reconstitute, registerContentBlock, sTranslations, stripSentinels, t, tSignal, tokenizeElement };
+export { type EncodedRichText, type ExtractParamKeys, LangsysApp, LangsysAppAPI, type LocaleSource, Logger, PHRASE_MARKER_ATTR, PHRASE_MARKER_ATTRS, type ParamPrimitive, type ParamsFor, Phrase, type PhraseOptions, type RichSlot, type Signal, type Subscriber, type TArgs, type TFunction, Translate, type TranslateOptions, type TranslationParams, Translations, type Unsubscriber, type Updater, canonicalizeLocale, createSignal, currentlyLoadedLocale, encodeRichText, findUnusedParamKeys, generateCustomId, generateLegacyCustomId, getValue, type iCategories, type iContentBlock, type iCountry, type iCountryDialCode, type iCountryList, type iCurrency, type iCurrencyList, type iLangsysConfig, type iLangsysInitConfig, type ResponseObject as iLangsysResponse, type iLanguageName, type iLocaleData, type iLocaleDefault, type iLocaleFlat, type iProject, type iTranslations, interpolate, isContentBlockKnown, isEmpty, isICU, isPhraseMarked, isTranslationExcluded, legacyTokenizeElement, logger, markupTokenValues, md5, md5Legacy, normalizeMarkupPlaceholders, persist, reconstitute, registerContentBlock, sTranslations, stripSentinels, t, tSignal, tokenizeElement };

package/dist/index.d.ts

@@ -628,14 +628,37 @@ interface iContentBlock {
  * additive — `data-langsys-phrase` never appears in DOM our components emit.
  */
 declare const PHRASE_MARKER_ATTRS: readonly ["data-ls-phrase", "data-langsys-phrase"];
+/**
+ * True when an element opts out of translation entirely — it and its subtree
+ * are skipped by the tokenizer.
+ *
+ * Mirrors `langsys-php`'s `HtmlParser::isTranslationExcluded()` exactly:
+ *  - `translate="no"` — the HTML standard attribute, matched case-insensitively.
+ *  - `data-notrans` — PHP's author-facing alias, for hosts whose templating
+ *    strips unknown bare attributes or where `translate` collides with another
+ *    tool. Presence is intent; an explicit `"false"`/`"0"` opts out of the
+ *    opt-out, compared after trimming.
+ *
+ * Honoring PHP's alias matters for the same reason as the phrase marker: on SSR
+ * handoff there is ONE DOM, and content an author marked "do not translate"
+ * must not be extracted and registered by whichever SDK happens to walk it.
+ *
+ * Both this and `isPhraseMarked` trim before comparing. They briefly differed
+ * — PHP's phrase check predated the trim its exclusion check gained — and the
+ * gap was resolved by asking rather than tidying: mirror exactly, flag the
+ * wart, let the owner decide. Divergence-by-tidying is the same failure class
+ * as divergence-by-oversight, just better intentioned.
+ */
+declare function isTranslationExcluded(element: Element): boolean;
 /**
  * True when an element is marked as a self-managed phrase by either SDK.
  *
  * Presence alone means intent, like any boolean HTML attribute — but an
- * explicit `="false"` or `="0"` opts OUT, matching `langsys-php`'s
- * `hasPhraseAttribute()` exactly. Without the opt-out we would skip a subtree
- * the author had deliberately un-marked, and it would then be translated by
- * neither SDK.
+ * explicit `="false"` or `="0"` opts OUT, compared after trimming. Mirrors
+ * `langsys-php`'s `isPhraseMarked()` exactly (same method name in both SDKs,
+ * so the correspondence is checkable at a glance). Without the opt-out we
+ * would skip a subtree the author had deliberately un-marked, and it would
+ * then be translated by neither SDK.
  */
 declare function isPhraseMarked(element: Element): boolean;
 /**
@@ -715,6 +738,23 @@ declare function tokenizeElement(element: HTMLElement): {
     tokens: string[];
     content: string;
 };
+/**
+ * The token list this element would have produced before 0.6.3, when every
+ * `<option>`'s text was harvested twice — once by a `<select>` special case and
+ * again by the ordinary text-node walk.
+ *
+ * **Lookup only.** Content blocks registered by an older SDK are keyed by an id
+ * derived from this list, so `Translate` falls back to it when the corrected id
+ * misses; without that, every block containing a `<select>` would silently
+ * re-key and lose its translations on upgrade. Registration always uses the
+ * corrected list, so the legacy-keyed population can only shrink.
+ *
+ * Skips the computed-style capture that `tokenizeElement` performs — this is a
+ * cache-key computation, not a snapshot for translators.
+ *
+ * @deprecated Migration aid; will be removed once catalogs have been rebased.
+ */
+declare function legacyTokenizeElement(element: HTMLElement): string[];
 
 /**
  * Create a Signal whose value is persisted to localStorage (when available).
@@ -853,4 +893,4 @@ interface iProject {
 declare const t: TFunction;
 declare const tSignal: Signal<TFunction>;
 
-export { type EncodedRichText, type ExtractParamKeys, LangsysApp, LangsysAppAPI, type LocaleSource, Logger, PHRASE_MARKER_ATTR, PHRASE_MARKER_ATTRS, type ParamPrimitive, type ParamsFor, Phrase, type PhraseOptions, type RichSlot, type Signal, type Subscriber, type TArgs, type TFunction, Translate, type TranslateOptions, type TranslationParams, Translations, type Unsubscriber, type Updater, canonicalizeLocale, createSignal, currentlyLoadedLocale, encodeRichText, findUnusedParamKeys, generateCustomId, generateLegacyCustomId, getValue, type iCategories, type iContentBlock, type iCountry, type iCountryDialCode, type iCountryList, type iCurrency, type iCurrencyList, type iLangsysConfig, type iLangsysInitConfig, type ResponseObject as iLangsysResponse, type iLanguageName, type iLocaleData, type iLocaleDefault, type iLocaleFlat, type iProject, type iTranslations, interpolate, isContentBlockKnown, isEmpty, isICU, isPhraseMarked, logger, markupTokenValues, md5, md5Legacy, normalizeMarkupPlaceholders, persist, reconstitute, registerContentBlock, sTranslations, stripSentinels, t, tSignal, tokenizeElement };
+export { type EncodedRichText, type ExtractParamKeys, LangsysApp, LangsysAppAPI, type LocaleSource, Logger, PHRASE_MARKER_ATTR, PHRASE_MARKER_ATTRS, type ParamPrimitive, type ParamsFor, Phrase, type PhraseOptions, type RichSlot, type Signal, type Subscriber, type TArgs, type TFunction, Translate, type TranslateOptions, type TranslationParams, Translations, type Unsubscriber, type Updater, canonicalizeLocale, createSignal, currentlyLoadedLocale, encodeRichText, findUnusedParamKeys, generateCustomId, generateLegacyCustomId, getValue, type iCategories, type iContentBlock, type iCountry, type iCountryDialCode, type iCountryList, type iCurrency, type iCurrencyList, type iLangsysConfig, type iLangsysInitConfig, type ResponseObject as iLangsysResponse, type iLanguageName, type iLocaleData, type iLocaleDefault, type iLocaleFlat, type iProject, type iTranslations, interpolate, isContentBlockKnown, isEmpty, isICU, isPhraseMarked, isTranslationExcluded, legacyTokenizeElement, logger, markupTokenValues, md5, md5Legacy, normalizeMarkupPlaceholders, persist, reconstitute, registerContentBlock, sTranslations, stripSentinels, t, tSignal, tokenizeElement };

package/dist/index.js

@@ -1076,8 +1076,14 @@ var TRANSLATABLE_ATTRIBUTES = [
   "placeholder",
   "alt",
   "title",
+  "label",
+  // <option>, <optgroup>, <track> — the text a user reads in the picker
   "aria-label",
   "aria-placeholder",
+  "aria-description",
+  "aria-valuetext",
+  // the spoken value of a slider/meter
+  "aria-roledescription",
   "data-error",
   "data-error-message",
   "data-validation-message",
@@ -1086,10 +1092,16 @@ var TRANSLATABLE_ATTRIBUTES = [
   "data-pattern-message"
 ];
 var PHRASE_MARKER_ATTRS = ["data-ls-phrase", "data-langsys-phrase"];
+function isTranslationExcluded(element) {
+  if ((element.getAttribute("translate") ?? "").toLowerCase() === "no") return true;
+  if (!element.hasAttribute("data-notrans")) return false;
+  const value = (element.getAttribute("data-notrans") ?? "").trim().toLowerCase();
+  return value !== "false" && value !== "0";
+}
 function isPhraseMarked(element) {
   return PHRASE_MARKER_ATTRS.some((attr) => {
     if (!element.hasAttribute(attr)) return false;
-    const value = (element.getAttribute(attr) ?? "").toLowerCase();
+    const value = (element.getAttribute(attr) ?? "").trim().toLowerCase();
     return value !== "false" && value !== "0";
   });
 }
@@ -1188,9 +1200,15 @@ async function registerContentBlock(contentBlock) {
 function tokenizeElement(element) {
   const tokens = [];
   const clone = element.cloneNode(true);
-  _walkForTokens(element, Array.from(clone.childNodes), tokens, []);
+  _walkForTokens(element, Array.from(clone.childNodes), tokens, [], false, true);
   return { tokens, content: normalizeMarkupPlaceholders(clone.outerHTML) };
 }
+function legacyTokenizeElement(element) {
+  const tokens = [];
+  const clone = element.cloneNode(true);
+  _walkForTokens(element, Array.from(clone.childNodes), tokens, [], true, false);
+  return tokens;
+}
 function _writeKnownContentBlockToCache(category, customId) {
   const lookupCat = category || "__uncategorized__";
   sTranslations.update((current) => {
@@ -1204,18 +1222,18 @@ function _writeKnownContentBlockToCache(category, customId) {
     return { ...current };
   });
 }
-function _walkForTokens(liveRoot, cloneNodes, tokens, indices) {
+function _walkForTokens(liveRoot, cloneNodes, tokens, indices, duplicateSelectOptions, applyStyles) {
   cloneNodes.forEach((node, index) => {
     if (node.nodeType === Node.ELEMENT_NODE) {
       const el = node;
-      if (el.getAttribute("translate") === "no") return;
+      if (isTranslationExcluded(el)) return;
       if (isPhraseMarked(el)) return;
     }
-    if (node.hasChildNodes()) {
+    if (applyStyles && node.hasChildNodes()) {
       _applyStylesToClone(liveRoot, node, [...indices, index]);
     }
     if (node.nodeType === Node.ELEMENT_NODE) {
-      _tokenizeAttributes(node, tokens);
+      _tokenizeAttributes(node, tokens, duplicateSelectOptions);
     }
     const contentToken = node.nodeValue?.replace(/\s+/g, " ").trim();
     if (node.nodeType === Node.TEXT_NODE && contentToken) {
@@ -1223,10 +1241,10 @@ function _walkForTokens(liveRoot, cloneNodes, tokens, indices) {
       return;
     }
     if (!node.hasChildNodes()) return;
-    _walkForTokens(liveRoot, Array.from(node.childNodes), tokens, [...indices, index]);
+    _walkForTokens(liveRoot, Array.from(node.childNodes), tokens, [...indices, index], duplicateSelectOptions, applyStyles);
   });
 }
-function _tokenizeAttributes(element, tokens) {
+function _tokenizeAttributes(element, tokens, duplicateSelectOptions) {
   const tagName = element.tagName.toLowerCase();
   if (tagName === "img") {
     const img = element;
@@ -1247,7 +1265,7 @@ function _tokenizeAttributes(element, tokens) {
       if (value) tokens.push(normalizeMarkupPlaceholders(value));
     }
   }
-  if (tagName === "select") {
+  if (duplicateSelectOptions && tagName === "select") {
     const options = element.querySelectorAll("option");
     options.forEach((option) => {
       const optionText = option.textContent?.trim();
@@ -1373,9 +1391,15 @@ var Translate = class {
       return;
     }
     if (derivedId) {
-      const legacyId = generateLegacyCustomId(contentBlock.category, contentBlock.tokens);
-      if (legacyId !== this.custom_id && isContentBlockKnown(contentBlock.category, legacyId)) {
-        this.custom_id = legacyId;
+      const legacyTokens = legacyTokenizeElement(this.element);
+      const candidates = [
+        generateCustomId(contentBlock.category, legacyTokens),
+        generateLegacyCustomId(contentBlock.category, legacyTokens)
+      ];
+      for (const candidate of candidates) {
+        if (candidate === this.custom_id) continue;
+        if (!isContentBlockKnown(contentBlock.category, candidate)) continue;
+        this.custom_id = candidate;
         if (this.tokens.length > 1 && this.element) {
           this.translate(Array.from(this.element.childNodes));
           this.lastTranslatedLocale = currentlyLoadedLocale.get();
@@ -1397,7 +1421,7 @@ var Translate = class {
     nodes.forEach((node) => {
       if (node?.nodeType === Node.ELEMENT_NODE) {
         const element = node;
-        if (element.getAttribute("translate") === "no") return;
+        if (isTranslationExcluded(element)) return;
         if (isPhraseMarked(element)) return;
         this.translateAttributes(node);
       }
@@ -1660,6 +1684,8 @@ exports.isContentBlockKnown = isContentBlockKnown;
 exports.isEmpty = isEmpty;
 exports.isICU = isICU;
 exports.isPhraseMarked = isPhraseMarked;
+exports.isTranslationExcluded = isTranslationExcluded;
+exports.legacyTokenizeElement = legacyTokenizeElement;
 exports.logger = logger;
 exports.markupTokenValues = markupTokenValues;
 exports.md5 = md5;