@vestra-player/sdk 1.5.8 → 1.5.9

package/CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## 1.5.9
+
+**Translations** (player 0.4.3). Additive; `apiVersion` stays `vestra/1`, and plugins written for 1.5.8 need no change. One test-visible change: a core action's `declaration.path` in `TestHost` is now the player's English key (read `display` for what the menu shows). The player now shows its interface in English, Simplified Chinese, Japanese, Korean, German, French or Spanish, chosen at start-up; a plugin writes every text in English and ships one gettext file per language. Plugins write no translation code: the player translates. Guides: [Translate a plugin](docs/how-to/i18n.md), [Migrate a plugin to English source text](docs/how-to/i18n-migration.md); example: `examples/i18n`.
+
+- **`plugin.i18n.locale`** (new `I18n`, `Locale`): the interface language of this run, `"en"`, `"zh-Hans"`, `"ja"`, `"ko"`, `"de"`, `"fr"` or `"es"`; it never changes while the player runs. `"en"` from older players.
+- **Templates with arguments**: `osd.show(text, args?)`, `OsdPanel.set(text, args?)` and `ConfirmOptions.args` (new `TemplateArgs`: an array for `{}` / `{0}`, an object for `{name}`). The player translates the template with the plugin's catalog, then fills it in. Without `args` the text shows as it is, so existing calls are unchanged; older players show the template unfilled.
+- **Declarations are translated by the player** from the plugin's catalog: command titles, path segments and `reason`, settings page and panel titles, and the manifest's display fields. Menu paths are English source text and menus merge by them; the player's own menus have the English keys `Open`, `Playback`, `Subtitles`, `Video`, `Audio`, `Screen`, `Settings` (the `@video`-style ids still work, and the player still accepts the old Chinese titles of its own menus). New `PluginInfo.display` (`PluginDisplay`): a package's name, punchline, description and tags as shown.
+- **`appearance.language`** (`LanguagePreference`: `"system"` or a locale) and **`appearance.setLanguage(language)`** (needs `ui.theme`): the saved choice, used from the next start.
+- **Catalogs**: `locales/messages.pot` and `locales/<lang>.po` in the plugin's folder. New commands **`vestra-plugin i18n extract`** (finds `@tr` in `.slint` files, the texts of `commands.add`, `osd.show`, dialogs, settings and panel titles through the TypeScript type checker, and the manifest's display fields; merges into every `.po`, keeping translations; `--locale <lang>` starts one) and **`vestra-plugin i18n check`** (coverage, placeholder and plural problems, a stale template, sentences composed in TypeScript, non-literal values, non-English source text, Slint text without `@tr`; warnings only). `vestra-plugin check` prints the same warnings when the plugin has `locales/`. `vestra-plugin build` compiles each `.po` into `dist/locales/<lang>.json` and no longer packs `.po` files.
+- **`TestHost`**: `locale` (also `startPlugin({ locale })`) loads the plugin's catalog for that language; `FakeCommand.display` is the menu as shown, `host.osd` and `osdPanel` the shown text (the template and args stay in `host.sent(...)`), new `host.confirmTexts`, `host.translate(text, args)`, `host.loadCatalog(directory)`, `host.language` / `setLanguage(...)`, settings page and panel titles as shown. The core actions' declarations use the player's English keys (`["Playback", "Seek", "Forward 5 s"]`); their `display` and the binding table show the player's words. While `locale` is not set, the player's own messages (core actions, refusals) stay in the Chinese of 1.5.8, so existing tests keep passing; with a `locale` they are the player's English (or its Chinese for `"zh-Hans"`).
+- `LOCALES` in `@vestra-player/sdk/testing`.
+- The Slint library copy (`slint/vestra/widgets.slint`, `markdown.slint`) uses `@tr` for its own texts (`UrlInputRow`'s default buttons, the code block's Copy button, the image placeholder).
+
 ## 1.5.8
 
 **Trial loading is removed; `resolvers.fileTypes`; the CLI under `--permission`.** `apiVersion` stays `vestra/1`. The removal breaks code that called `PackageCandidate.trial()`: only Vestra Player AI did, and it moves to installing (an install can be rolled back).

package/README.md

@@ -72,6 +72,7 @@ Complete, tested plugins in [`examples/`](https://github.com/vestrasys/vestra-pl
 | [menus](https://github.com/vestrasys/vestra-player-sdk/tree/main/examples/menus)                       | Radio items, separator groups and a menu item whose title changes.       | [Step by step](https://vestrasys.github.io/vestra-player-sdk/documents/Example__Menus.html)            |
 | [theme-widgets](https://github.com/vestrasys/vestra-player-sdk/tree/main/examples/theme-widgets)       | A tool window drawn with the player's theme and widgets.                 | [Step by step](https://vestrasys.github.io/vestra-player-sdk/documents/Example__Theme_widgets.html)    |
 | [markdown](https://github.com/vestrasys/vestra-player-sdk/tree/main/examples/markdown)                 | A window that shows a Markdown document.                                 | [Step by step](https://vestrasys.github.io/vestra-player-sdk/documents/Example__Markdown.html)         |
+| [i18n](https://github.com/vestrasys/vestra-player-sdk/tree/main/examples/i18n)                         | A plugin in the player's language: English source text, a translation.  | [Step by step](https://vestrasys.github.io/vestra-player-sdk/documents/Counter__i18n_.html)            |
 
 ## Command line
 
@@ -81,6 +82,8 @@ Complete, tested plugins in [`examples/`](https://github.com/vestrasys/vestra-pl
 | `vestra-plugin test [dir...]`                               | Runs `node --test` on `tests/*.test.{ts,mjs}`.                             |
 | `vestra-plugin build [dir...] [--version x.y.z]`            | Writes `dist/`: types stripped, files copied, manifest versioned.          |
 | `vestra-plugin pack [dir...] [--version x.y.z] [--out dir]` | Builds, then packs `dist/` as `<id>-<version>.vxp`.                        |
+| `vestra-plugin i18n extract [dir...] [--locale lang]`       | Collects the texts to translate into `locales/`, keeping translations.     |
+| `vestra-plugin i18n check [dir...]`                         | Translation coverage and untranslatable text (warnings only).              |
 
 ## Compatibility
 
@@ -91,7 +94,7 @@ Complete, tested plugins in [`examples/`](https://github.com/vestrasys/vestra-pl
 | 1.2 | `vestra/1` | 0.2 or newer                                                                              | 26 (shipped with the player) |
 | 1.3 | `vestra/1` | 0.2 or newer; 0.3 for the 1.3 additions (menu ids, `Theme.danger`, empty video chains)    | 26 (shipped with the player) |
 | 1.4 | `vestra/1` | 0.2 or newer; 0.3 for the 1.3 additions. `ThemeId` is `"vestra"` only (see the CHANGELOG) | 26 (shipped with the player) |
-| 1.5 | `vestra/1` | 0.2 or newer; 0.3 for the 1.3 additions; 0.4 for the 1.5 additions (`vestra-light`, `@vestra/icons.slint`) | 26 (shipped with the player) |
+| 1.5 | `vestra/1` | 0.2 or newer; 0.3 for the 1.3 additions; 0.4 for the 1.5 additions (`vestra-light`, `@vestra/icons.slint`); 0.4.3 for translations (1.5.9) | 26 (shipped with the player) |
 
 Minor releases only add backwards-compatible API; a breaking change gets a new `apiVersion` and a new major version. `@vestra-player/sdk/runtime` is for the player only and not covered by semver.
 

package/bin/vestra-plugin.mjs

@@ -1,8 +1,11 @@
 #!/usr/bin/env node
 /**
- * vestra-plugin: check, build, test and pack a Vestra Player plugin.
+ * vestra-plugin: check, build, test and pack a Vestra Player plugin, and
+ * keep its translations up to date.
  *
  *   vestra-plugin <check|build|test|pack> [plugin-dir...] [--version x.y.z] [--out dir]
+ *   vestra-plugin i18n extract [plugin-dir...] [--locale <lang>]...
+ *   vestra-plugin i18n check [plugin-dir...]
  *
  * A plugin directory holds manifest.json; the default is the current
  * directory. Works in any project that depends on @vestra-player/sdk.
@@ -14,6 +17,24 @@
  *   pack    build, then dist/ → <out>/<id>-<version>.vxp (a zip; default
  *           out: the current directory) with its sha256 and size
  *
+ * Translations (docs/how-to/i18n.md). English is the source text; each
+ * language is a gettext file locales/<lang>.po:
+ *
+ *   i18n extract  find the texts to translate (@tr in .slint files, the
+ *                 texts handed to commands.add, osd.show, dialogs, settings
+ *                 and panel titles, the manifest's display fields), write
+ *                 locales/messages.pot and merge it into every
+ *                 locales/<lang>.po (translations kept, vanished ones marked
+ *                 obsolete); --locale <lang> starts a new language
+ *   i18n check    coverage per language, placeholder and plural problems, a
+ *                 stale messages.pot, and texts that cannot be translated
+ *                 (composed in TypeScript, not literals, not English, Slint
+ *                 text without @tr); warnings only, an unreadable .po fails
+ *
+ * `check` prints the same i18n warnings when the plugin has locales/;
+ * `build` compiles locales/<lang>.po to dist/locales/<lang>.json (the .po
+ * files themselves are not packed).
+ *
  * Every command first checks manifest.json against the player's rules
  * (display fields, paths inside the package, unknown fields); `check` also
  * prints hints, such as an id without the `<owner>.` prefix the official
@@ -62,6 +83,10 @@ const sdkRoot = resolve(import.meta.dirname, "..");
 const { manifestProblems } = await import(
   pathToFileURL(join(sdkRoot, "dist/manifest.js")).href
 );
+const i18n = await import(pathToFileURL(join(sdkRoot, "dist/i18n.js")).href);
+const i18nTools = await import(
+  pathToFileURL(join(sdkRoot, "dist/i18n-tools.js")).href
+);
 const SDK = "@vestra-player/sdk";
 
 /** Never part of a production package. */
@@ -79,7 +104,8 @@ const SKIP_FILES = new Set([
   "package.json",
   "package-lock.json",
 ]);
-const SKIP_EXTENSIONS = new Set([".md", ".log", ".zip", ".vxp"]);
+// Translation sources: build packs the compiled dist/locales/<lang>.json.
+const SKIP_EXTENSIONS = new Set([".md", ".log", ".zip", ".vxp", ".po", ".pot"]);
 const isTypeScript = (file) => file.endsWith(".ts") && !file.endsWith(".d.ts");
 const isDeclaration = (file) => /\.d\.(m|c)?ts$/.test(file);
 
@@ -299,6 +325,113 @@ async function check(target) {
     await rm(scratch, { recursive: true, force: true });
   }
   console.log(`${target.name}: type check passed (${sources.length} files)`);
+  // Translation warnings never fail the check.
+  if (present(join(target.directory, "locales"))) {
+    try {
+      const { warnings, errors } = await i18nReport(target);
+      for (const line of [...warnings, ...errors])
+        console.log(`${target.name}: i18n: ${line}`);
+    } catch (error) {
+      console.log(`${target.name}: i18n: ${error.message}`);
+    }
+  }
+}
+
+/** The plugin's own production .ts / .mts and .slint files. */
+async function sourceFiles(target) {
+  const all = await files(target.directory);
+  return {
+    typescript: all.filter(
+      (file) =>
+        (isTypeScript(file) || file.endsWith(".mts")) && !isDeclaration(file),
+    ),
+    slint: all.filter((file) => file.endsWith(".slint")),
+  };
+}
+
+/** Compiler options of the type check (its tsconfig), for extraction. */
+async function extraction(target) {
+  const ts = await typescript();
+  const { typescript: sources, slint } = await sourceFiles(target);
+  const config = checkConfig(
+    target,
+    sources.map((file) => join(target.directory, file)),
+  );
+  const parsed = ts.parseJsonConfigFileContent(
+    config,
+    ts.sys,
+    target.directory,
+  );
+  return i18nTools.extract({
+    directory: target.directory,
+    manifest: target.manifest,
+    slint,
+    typescript: sources,
+    compilerOptions: parsed.options,
+  });
+}
+
+async function i18nReport(target) {
+  const { messages, findings } = await extraction(target);
+  return i18nTools.checkCatalogs(target.directory, messages, findings);
+}
+
+async function i18nExtract(target, locales) {
+  for (const locale of locales) {
+    if (locale === "en")
+      fail("--locale en: English is the source language, it has no .po file");
+    if (!i18n.LOCALES.includes(locale))
+      fail(
+        `--locale ${locale}: the player's languages are ${i18n.LOCALES.filter((l) => l !== "en").join(", ")}`,
+      );
+  }
+  dependencies(target);
+  const { messages, findings } = await extraction(target);
+  const directory = join(target.directory, "locales");
+  await mkdir(directory, { recursive: true });
+  await writeFile(
+    join(directory, "messages.pot"),
+    i18nTools.template(target.manifest, messages),
+  );
+  console.log(
+    `${target.name}: locales/messages.pot: ${messages.length} messages`,
+  );
+  const languages = new Set(locales);
+  for (const name of await readdir(directory))
+    if (name.endsWith(".po")) languages.add(name.slice(0, -3));
+  for (const language of [...languages].sort()) {
+    if (language === "en" || !i18n.LOCALES.includes(language)) {
+      console.log(
+        `${target.name}: locales/${language}.po: skipped (not a player language)`,
+      );
+      continue;
+    }
+    const file = join(directory, `${language}.po`);
+    let existing = null;
+    if (present(file)) existing = await readFile(file, "utf8");
+    let merged;
+    try {
+      merged = i18nTools.merge(existing, language, messages);
+    } catch (error) {
+      fail(`${target.name}: locales/${language}.po: ${error.message}`);
+    }
+    await writeFile(file, merged.text);
+    console.log(
+      `${target.name}: locales/${language}.po: ${merged.translated}/${merged.total} translated, ${merged.added} new, ${merged.obsolete} obsolete${existing === null ? " (created)" : ""}`,
+    );
+  }
+  for (const finding of findings)
+    console.log(
+      `${target.name}: i18n: ${finding.file}:${finding.line}: ${finding.message}`,
+    );
+}
+
+async function i18nCheck(target) {
+  dependencies(target);
+  const { warnings, errors } = await i18nReport(target);
+  for (const line of warnings) console.log(`${target.name}: i18n: ${line}`);
+  if (errors.length) fail(`${target.name}: ${errors.join("; ")}`);
+  if (!warnings.length) console.log(`${target.name}: i18n: no problems`);
 }
 
 function version(target, flag) {
@@ -455,6 +588,14 @@ async function build(target, flag) {
       entry.built,
       await readFile(join(out, entry.built), "utf8"),
     );
+  let localeWarnings;
+  try {
+    localeWarnings = i18nTools.compileLocales(target.directory, out);
+  } catch (error) {
+    fail(`${target.name}: ${error.message}`);
+  }
+  for (const line of localeWarnings)
+    console.log(`${target.name}: warning: ${line}`);
   await writeFile(join(out, "package.json"), '{"type":"module"}\n');
   await writeFile(
     join(out, "manifest.json"),
@@ -558,8 +699,11 @@ async function pack(target, flag, outDirectory) {
   );
 }
 
-const USAGE =
-  "usage: vestra-plugin <check|build|test|pack> [plugin-dir...] [--version x.y.z] [--out dir]";
+const USAGE = [
+  "usage: vestra-plugin <check|build|test|pack> [plugin-dir...] [--version x.y.z] [--out dir]",
+  "       vestra-plugin i18n extract [plugin-dir...] [--locale <lang>]...",
+  "       vestra-plugin i18n check [plugin-dir...]",
+].join("\n");
 
 async function main() {
   const args = process.argv.slice(2);
@@ -575,10 +719,25 @@ async function main() {
     console.log(USAGE);
     return;
   }
-  if (!["check", "build", "test", "pack"].includes(command)) {
+  let subcommand = null;
+  if (command === "i18n") {
+    subcommand = args.shift();
+    if (!["extract", "check"].includes(subcommand)) {
+      console.error(USAGE);
+      process.exit(2);
+    }
+  } else if (!["check", "build", "test", "pack"].includes(command)) {
     console.error(USAGE);
     process.exit(2);
   }
+  const locales = [];
+  for (let index; (index = args.indexOf("--locale")) >= 0;) {
+    const [, value] = args.splice(index, 2);
+    if (value === undefined) fail("--locale needs a value");
+    locales.push(value);
+  }
+  if (locales.length && subcommand !== "extract")
+    fail("--locale applies to i18n extract");
   const flag = option("--version");
   const out = option("--out");
   const directories = args.length ? args : ["."];
@@ -588,7 +747,9 @@ async function main() {
   for (const directory of directories) {
     try {
       const target = await plugin(directory);
-      if (command === "check") await check(target);
+      if (subcommand === "extract") await i18nExtract(target, locales);
+      else if (subcommand === "check") await i18nCheck(target);
+      else if (command === "check") await check(target);
       else if (command === "build") await build(target, flag);
       else if (command === "pack") await pack(target, flag, out);
       else await test(target);
@@ -599,7 +760,9 @@ async function main() {
     }
   }
   if (failed.length) {
-    console.error(`${command} failed for: ${failed.join(", ")}`);
+    console.error(
+      `${subcommand ? `i18n ${subcommand}` : command} failed for: ${failed.join(", ")}`,
+    );
     process.exit(1);
   }
 }

package/dist/i18n-tools.d.ts

@@ -0,0 +1,67 @@
+import ts from "typescript";
+/** A translatable text found in the sources. */
+export interface Message {
+    context?: string;
+    id: string;
+    plural?: string;
+    /** `#.` notes (from `TRANSLATORS:` comments). */
+    notes: string[];
+    /** `path:line` (or `manifest.json`), relative to the plugin, sorted. */
+    references: string[];
+}
+/** Something extraction or the heuristics want the author to look at. */
+export interface Finding {
+    /** Relative path, forward slashes. */
+    file: string;
+    line: number;
+    kind: "dynamic" | "composed" | "not-english" | "slint-text";
+    message: string;
+}
+export interface ExtractOptions {
+    /** The plugin directory. */
+    directory: string;
+    /** The plugin's `manifest.json` (display fields are extracted). */
+    manifest: Record<string, unknown>;
+    /** The plugin's `.slint` files, relative to `directory`. */
+    slint: readonly string[];
+    /** The plugin's production `.ts` / `.mts` files, relative to `directory`. */
+    typescript: readonly string[];
+    /** Compiler options for a program over those files (as `check` uses). */
+    compilerOptions?: ts.CompilerOptions;
+}
+export declare const pluralForms: (language: string) => "nplurals=1; plural=0;" | "nplurals=2; plural=(n != 1);";
+/** Find every translatable text of a plugin, in reference order. */
+export declare function extract(options: ExtractOptions): {
+    messages: Message[];
+    findings: Finding[];
+};
+/** `locales/messages.pot` for these messages. */
+export declare function template(manifest: Record<string, unknown>, messages: readonly Message[]): string;
+/** The header of a new `<lang>.po`. */
+export declare const newHeader: (language: string) => string;
+/**
+ * Merge the template into a language's file: translations and fuzzy flags
+ * of present messages stay, new ones come untranslated, notes and
+ * references follow the template, vanished translated messages become
+ * obsolete (`#~`, at the end) and come back with their translation.
+ */
+export declare function merge(existing: string | null, language: string, messages: readonly Message[]): {
+    text: string;
+    total: number;
+    translated: number;
+    added: number;
+    obsolete: number;
+};
+/** Problems found by `check` in one plugin's catalogs, as text lines. */
+export declare function checkCatalogs(directory: string, messages: readonly Message[], findings: readonly Finding[]): {
+    warnings: string[];
+    errors: string[];
+};
+/**
+ * Compile every `locales/<lang>.po` of a player language to
+ * `<out>/locales/<lang>.json`. Returns the warnings (files the player never
+ * reads); throws on a `.po` that does not parse.
+ */
+export declare function compileLocales(directory: string, out: string): string[];
+/** The relative path of a file from the plugin directory, forward slashes. */
+export declare const relativePath: (directory: string, file: string) => string;