i18n-js 4.0.0 → 4.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0b2fdace9fca14d471ff34c3b24055d4093d82ce2a1f72f2df06364a98454a31
-  data.tar.gz: 9ea6bd71626357c8fce24c28e991b9edc96f8a3230e55c4d55cf1dec3813c94e
+  metadata.gz: 60f1ad0e9477d8e6570573f64ac2a55136653da8e4791c6fc32e69c9547e197b
+  data.tar.gz: d46a3d8b8a25020635d9c108b2cb2caf3a256e419112a21a0d986f94e1e19c37
 SHA512:
-  metadata.gz: 2c8da0348d02f04d805a013dadd24beb08fb7de555bfdded51e4e6c7c29ba392978a723b7e95be9d752dda2b35bbe6d6aefe4d4cf9166c1ddca09f003712533f
-  data.tar.gz: 882af1975b1f2e479ad4d8bdcbe812175fae165c80a2f915223e14fc71fb0b3236774254e103754169eab5a3d577e219e10803fd05e14fa3bac82471e4c770e1
+  metadata.gz: a1d63d094866a836a69a9b3fc8f2989608a1b794daf5d5c82062e17b3bcb07ab7fb0ec8f2c05aa27a2585236cff33d974c313ca81be96de129340ecce8f78490
+  data.tar.gz: 7ec126b41663f765a421950c1ddcb215d460681355088fc77731673c44f3baf590cc58c6e5841bc9dda6f58732249488cdad3b020287b57d7d1de70cf8e95716

data/.github/workflows/ruby-tests.yml

@@ -9,12 +9,15 @@ on:
 
 jobs:
   build:
-    name: Tests with Ruby ${{ matrix.ruby }} and ${{ matrix.gemfile }}
+    name:
+      Tests with Ruby ${{ matrix.ruby }}, Node ${{ matrix.node }} and ${{
+      matrix.gemfile }}
     runs-on: "ubuntu-latest"
     strategy:
       fail-fast: false
       matrix:
         ruby: ["2.7", "3.0", "3.1"]
+        node: ["16", "18"]
         gemfile:
           - Gemfile
     if: |
@@ -32,6 +35,23 @@ jobs:
             ${{ runner.os }}-${{ matrix.ruby }}-gems-${{
             hashFiles(matrix.gemfile) }}
 
+      - uses: actions/cache@v3
+        id: npm-cache
+        with:
+          path: vendor/bundle
+          key: >
+            ${{ runner.os }}-${{ matrix.node }}-npm-${{
+            hashFiles('package.json') }}
+
+      - name: Set up Node
+        uses: actions/setup-node@v2-beta
+        with:
+          node-version: ${{ matrix.node }}
+
+      - name: Install npm dependencies
+        run: |
+          yarn install
+
       - name: Set up Ruby
         uses: ruby/setup-ruby@v1
         with:
@@ -49,4 +69,5 @@ jobs:
         env:
           BUNDLE_GEMFILE: ${{ matrix.gemfile }}
         run: |
+          yarn compile
           bundle exec rake

data/.gitignore

@@ -9,3 +9,5 @@
 /vendor/
 *.log
 *.lock
+/lib/**/*.js
+/test/output

data/.rubocop.yml

@@ -14,3 +14,6 @@ Naming/FileName:
   Exclude:
     - lib/i18n-js.rb
     - lib/guard/i18n-js.rb
+
+Style/PerlBackrefs:
+  Enabled: false

data/CHANGELOG.md

@@ -11,6 +11,25 @@ Prefix your message with one of the following:
 - [Security] in case of vulnerabilities.
 -->
 
-## Jul 29, 2022
+## v4.1.0 - Dec 09, 2022
+
+- [Added] Parse configuration files as erb.
+- [Changed] `I18n.listen(run_on_start:)` was added to control if files should be
+  exported during `I18n.listen`'s boot. The default value is `true`.
+- [Added] Now it's possible to transform translations before exporting them
+  using a stable plugin api.
+- [Deprecated] The `i18n check` has been deprecated. Use
+  `i18n lint:translations` instead.
+- [Added] Use `i18n lint:scripts` to lint JavaScript/TypeScript.
+- [Fixed] Expand paths passed to `I18nJS.listen(locales_dir:)`.
+
+## v4.0.1 - Aug 25, 2022
+
+- [Fixed] Shell out export to avoid handling I18n reloading heuristics.
+- [Changed] `I18nJS.listen` now accepts a directories list to watch.
+- [Changed] `I18nJS.listen` now accepts
+  [listen](https://rubygems.org/gems/listen) options via `:options`.
+
+## v4.0.0 - Jul 29, 2022
 
 - Official release of i18n-js v4.0.0.

data/MIGRATING_FROM_V3_TO_V4.md

@@ -0,0 +1,191 @@
+# Migrating from v3 to v4
+
+I18n-js v4 is a breaking change release and diverges quite a lot from how the
+previous version worked. This guides summarizes the process of upgrading an app
+that uses i18n-js v3 to v4.
+
+## Development
+
+Previously, you could use a middleware to export translations (some people even
+used this in production 😬). In development, you can now use whatever your want,
+because i18n-js doesn't make any assumptions. All you need to do is running
+`i18n export`, either manually or by using something that listens to file
+changes.
+
+If you like watchman, you can use something like this:
+
+```bash
+#!/usr/bin/env bash
+
+root=`pwd`
+
+watchman watch-del "$root"
+watchman watch-project "$root"
+watchman trigger-del "$root" i18n
+
+watchman -j <<-JSON
+[
+  "trigger",
+  "$root",
+  {
+    "name": "i18n",
+    "expression": [
+      "anyof",
+      ["match", "config/locales/**/*.yml", "wholename"],
+      ["match", "config/locales/**/*.po", "wholename"],
+      ["match", "config/i18n.yml", "wholename"]
+    ],
+    "command": ["i18n", "export"]
+  }
+]
+JSON
+
+# If you're running this through Foreman, then uncomment the following lines:
+# while true; do
+#   sleep 1
+# done
+```
+
+You can also use guard. Make sure you have both
+[guard](https://rubygems.org/packages/guard) and
+[guard-compat](https://rubygems.org/packages/guard-compat) installed and use
+Guardfile file with the following contents:
+
+```ruby
+guard(:"i18n-js",
+      run_on_start: true,
+      config_file: "./config/i18n.yml",
+      require_file: "./config/environment.rb") do
+  watch(%r{^config/locales/.+\.(yml|po)$})
+  watch(%r{^config/i18n.yml$})
+  watch("Gemfile")
+end
+```
+
+To run guard, use `guard start -i`.
+
+Finally, you can use [listen](https://rubygems.org/gems/listen). Create the file
+`config/initializers/i18n.rb` with the following content:
+
+```ruby
+Rails.application.config.after_initialize do
+  require "i18n-js/listen"
+  # This will only run in development.
+  I18nJS.listen
+end
+```
+
+> **Warning**:
+>
+> No matter which approach you choose, the idea is that you _precompile_ your
+> translations when going to production. DO NOT RUN any of the above in
+> production.
+
+## Exporting translations
+
+The build process for i18n now relies on an external CLI called `i18n`. All you
+need to do is executing `i18n export` in your build step to generate the json
+files for your translations.
+
+## Using your translations
+
+The JavaScript package is now a separate thing and need to be installed using
+your favorite tooling (e.g. yarn, npm, pnpm, etc).
+
+```console
+$ yarn add i18n-js@latest
+$ npm i --save-dev i18n-js@latest
+```
+
+From now on, the way you load translations and set up I18n-js is totally up to
+you, but means you need to load the json files and attach to the I18n-js
+instance. This is how I do it in a project I'm doing right now (Rails 7 +
+esbuild + TypeScript). First, we need to load the I18n-js configuration from the
+main JavaScript file:
+
+```typescript
+// app/javascript/application.ts
+import { i18n } from "./config/i18n";
+```
+
+Then we need to load our translations and instantiate the I18n-js class.
+
+```typescript
+// app/javascript/config/i18n.ts
+import { I18n } from "i18n-js";
+import translations from "translations.json";
+
+// Fetch user locale from html#lang.
+// This value is being set on `app/views/layouts/application.html.erb` and
+// is inferred from `ACCEPT-LANGUAGE` header.
+const userLocale = document.documentElement.lang;
+
+export const i18n = new I18n();
+i18n.store(translations);
+i18n.defaultLocale = "en";
+i18n.enableFallback = true;
+i18n.locale = userLocale;
+```
+
+The best thing about the above is that it is a pretty straightforward pattern in
+the JavaScript community. It doesn't rely on specific parts from Sprockets (I'm
+not even using it on my projects) or eRb files.
+
+## Ruby on Rails
+
+### Upgrading the configuration file
+
+The configuration file loaded from `config/i18n.yml` has changed. Given the v3
+configuration below
+
+```yaml
+---
+translations:
+  - file: "app/assets/javascripts/date_formats.js"
+    only: "*.date.formats"
+  - file: "app/assets/javascripts/other.js"
+    only: ["*.activerecord", "*.admin.*.title"]
+  - file: "app/assets/javascripts/everything_else.js"
+    except:
+      - "*.activerecord"
+      - "*.admin.*.title"
+      - "*.date.formats"
+```
+
+the equivalent configuration file for v4 would be
+
+```yaml
+---
+translations:
+  - file: "app/assets/javascripts/date_formats.js"
+    patterns:
+      - "*.date.formats"
+  - file: "app/assets/javascripts/other.js"
+    patterns:
+      - "*.activerecord"
+      - "*.admin.*.title"
+  - file: "app/assets/javascripts/everything_else.js"
+    patterns:
+      # Notice the exclamation mark.
+      - "!*.activerecord"
+      - "!*.admin.*.title"
+      - "!*.date.formats"
+```
+
+Other configuration options:
+
+- `export_i18n_js`: removed without an equivalent
+- `fallbacks`: removed without an equivalent
+- `js_available_locales`: removed (on v4 you can use groups, like in
+  `{pt-BR,en}.*`)
+- `namespace`: removed without an equivalent
+- `sort_translation_keys`: removed (on v4 keys will always be sorted)
+- `translations[].prefix`: removed without an equivalent
+- `translations[].pretty_print`: removed (on v4 files will always be exported in
+  a readable format)
+
+### Placeholders
+
+Previously, v3 had the `%{locale}` placeholder, which can be used as part of the
+directory and/or file name. Now, the syntax is just `:locale`. Additionally, you
+can also use `:digest`, which uses a MD5 hex digest of the exported file.

data/README.md

@@ -1,5 +1,5 @@
 <p align="center">
-  <img width="250" height="58" src="https://github.com/fnando/i18n-js/raw/main/i18njs.png" alt="i18n.js">
+  <img width="250" height="58" src="https://github.com/fnando/i18n-js/raw/main/images/i18njs.png" alt="i18n.js">
 </p>
 
 <p align="center">
@@ -12,14 +12,14 @@
   <small>
     Oh, you don't use Ruby? No problem! You can still use i18n-js
     <br>
-    and the 
-    <a href="https://www.npmjs.com/package/i18n-js/v/next">companion JavaScript package</a>.
+    and the
+    <a href="https://www.npmjs.com/package/i18n-js/v/latest">companion JavaScript package</a>.
   </small>
 </p>
 
 <p align="center">
   <a href="https://github.com/fnando/i18n-js"><img src="https://github.com/fnando/i18n-js/workflows/ruby-tests/badge.svg" alt="Tests"></a>
-  <a href="https://rubygems.org/gems/i18n-js"><img src="https://img.shields.io/gem/v/i18n-js.svg?include_prereleases" alt="Gem"></a>
+  <a href="https://rubygems.org/gems/i18n-js"><img src="https://img.shields.io/gem/v/i18n-js.svg" alt="Gem"></a>
   <a href="https://rubygems.org/gems/i18n-js"><img src="https://img.shields.io/gem/dt/i18n-js.svg" alt="Gem"></a>
   <a href="https://tldrlegal.com/license/mit-license"><img src="https://img.shields.io/:License-MIT-blue.svg" alt="MIT License"></a>
 </p>
@@ -45,6 +45,14 @@ About patterns:
   - `*.messages.*`
 - Patterns starting with `!` are excluded.
   - `!*.activerecord.*` will exclude all ActiveRecord translations.
+- You can use groups:
+  - `{pt-BR,en}.js.*` will include only `pt-BR` and `en` translations, even if
+    more languages are available.
+
+> **Note**:
+>
+> Patterns use [glob](https://rubygems.org/gems/glob), so check it out for the
+> most up-to-date documentation about what's available.
 
 The config file:
 
@@ -68,6 +76,22 @@ The output path can use the following placeholders:
 - `:locale`: the language that's being exported.
 - `:digest`: the MD5 hex digest of the exported file.
 
+The config file is processed as erb, so you can have dynamic content on it if
+you want. The following example shows how to use groups from a variable.
+
+```yml
+---
+<% group = "{en,pt}" %>
+
+translations:
+  - file: app/frontend/translations.json
+    patterns:
+      - "<%= group %>.*"
+      - "!<%= group %>.activerecord"
+      - "!<%= group %>.errors"
+      - "!<%= group %>.number.nth"
+```
+
 The Ruby API:
 
 ```ruby
@@ -88,7 +112,9 @@ Commands:
 - init: Initialize a project
 - export: Export translations as JSON files
 - version: Show package version
-- check: Check for missing translations
+- plugins: List plugins that will be activated
+- lint:translations: Check for missing translations
+- lint:scripts: Lint files using TypeScript
 
 Run `i18n COMMAND --help` for more information on specific commands.
 ```
@@ -97,14 +123,126 @@ By default, `i18n` will use `config/i18n.yml` and `config/environment.rb` as the
 configuration files. If you don't have these files, then you'll need to specify
 both `--config` and `--require`.
 
+### Plugins
+
+#### Built-in plugins:
+
+##### `embed_fallback_translations`:
+
+Embed fallback translations inferred from the default locale. This can be useful
+in cases where you have multiple large translation files and don't want to load
+the default locale together with the target locale.
+
+To use it, add the following to your configuration file:
+
+```yaml
+embed_fallback_translations:
+  enabled: true
+```
+
+#### Plugin API
+
+You can transform the exported translations by adding plugins. A plugin must
+inherit from `I18nJS::Plugin` and can have 3 class methods. The following
+example shows how the built-in `embed_fallback_translations` plugin is
+implemented.
+
+```ruby
+# frozen_string_literal: true
+
+module I18nJS
+  require "i18n-js/plugin"
+
+  class EmbedFallbackTranslationsPlugin < I18nJS::Plugin
+    CONFIG_KEY = :embed_fallback_translations
+
+    # This method must set up the basic plugin configuration, like adding the
+    # config's root key in case your plugin accepts configuration (defined via
+    # the config file).
+    #
+    # If you don't add this key, the linter will prevent non-default keys from
+    # being added to the configuration file.
+    def self.setup
+      I18nJS::Schema.root_keys << CONFIG_KEY
+    end
+
+    # In case your plugin accepts configuration, this is where you must validate
+    # the configuration, making sure only valid keys and type is provided.
+    # If the configuration contains invalid data, then you must raise an
+    # exception using something like
+    # `raise I18nJS::Schema::InvalidError, error_message`.
+    def self.validate_schema(config:)
+      return unless config.key?(CONFIG_KEY)
+
+      plugin_config = config[CONFIG_KEY]
+      valid_keys = %i[enabled]
+      schema = I18nJS::Schema.new(config)
+
+      schema.expect_required_keys(valid_keys, plugin_config)
+      schema.reject_extraneous_keys(valid_keys, plugin_config)
+      schema.expect_enabled_config(CONFIG_KEY, plugin_config[:enabled])
+    end
+
+    # This method is responsible for transforming the translations. The
+    # translations you'll receive may be already be filtered by other plugins
+    # and by the default filtering itself. If you need to access the original
+    # translations, use `I18nJS.translations`.
+    #
+    # Make sure you always check whether your plugin is active before
+    # transforming translations; otherwise, opting out transformation won't be
+    # possible.
+    def self.transform(translations:, config:)
+      return translations unless config.dig(CONFIG_KEY, :enabled)
+
+      translations_glob = Glob.new(translations)
+      translations_glob << "*"
+
+      mapping = translations.keys.each_with_object({}) do |locale, buffer|
+        buffer[locale] = Glob.new(translations[locale]).tap do |glob|
+          glob << "*"
+        end
+      end
+
+      default_locale = I18n.default_locale
+      default_locale_glob = mapping.delete(default_locale)
+      default_locale_paths = default_locale_glob.paths
+
+      mapping.each do |locale, glob|
+        missing_keys = default_locale_paths - glob.paths
+
+        missing_keys.each do |key|
+          components = key.split(".").map(&:to_sym)
+          fallback_translation = translations.dig(default_locale, *components)
+
+          next unless fallback_translation
+
+          translations_glob.set([locale, key].join("."), fallback_translation)
+        end
+      end
+
+      translations_glob.to_h
+    end
+  end
+
+  I18nJS.register_plugin(EmbedFallbackTranslationsPlugin)
+end
+```
+
+To distribute this plugin, you need to create a gem package that matches the
+pattern `i18n-js/*_plugin.rb`. You can test whether your plugin will be found by
+installing your gem, opening a iRB session and running
+`Gem.find_files("i18n-js/*_plugin.rb")`. If your plugin is not listed, then you
+need to double check your gem load path and see why the file is not being
+loaded.
+
 ### Listing missing translations
 
-To list missing and extraneous translations, you can use `i18n check`. This
-command will load your translations similarly to how `i18n export` does, but
-will output the list of keys that don't have a matching translation against the
-default locale. Here's an example:
+To list missing and extraneous translations, you can use
+`i18n lint:translations`. This command will load your translations similarly to
+how `i18n export` does, but will output the list of keys that don't have a
+matching translation against the default locale. Here's an example:
 
-![`i18n check` command in action](https://github.com/fnando/i18n-js/raw/main/images/i18njs-check.gif)
+![`i18n lint:translations` command in action](https://github.com/fnando/i18n-js/raw/main/images/i18njs-check.gif)
 
 This command will exist with status 1 whenever there are missing translations.
 This way you can use it as a CI linting.
@@ -125,19 +263,97 @@ translations:
     patterns:
       - "*"
 
-check:
+lint_translations:
   ignore:
     - en.mailer.login.subject
     - en.mailer.login.body
 ```
 
-> **Note**: In order to avoid mistakenly ignoring keys, this configuration
-> option only accepts the full translation scope, rather than accepting a
-> pattern like `pt.ignored.scope.*`.
+> **Note**:
+>
+> In order to avoid mistakenly ignoring keys, this configuration option only
+> accepts the full translation scope, rather than accepting a pattern like
+> `pt.ignored.scope.*`.
+
+### Linting your JavaScript/TypeScript files
+
+To lint your script files and check for missing translations (which can signal
+that you're either using wrong scopes or forgot to add the translation), use
+`i18n lint:scripts`. This command will parse your JavaScript/TypeScript files
+and extract all scopes being used. This command requires a Node.js runtime. You
+can either specify one via `--node-path`, or let the plugin infer a binary from
+your `$PATH`.
+
+The comparison will be made against the export JSON files, which means it'll
+consider transformations performed by plugins (e.g. the output files may be
+affected by `embed_fallback_translations` plugin).
+
+The translations that will be extract must be called as one of the following
+ways:
+
+- `i18n.t(scope, options)`
+- `i18n.translate(scope, options)`
+- `t(scope, options)`
+
+Notice that only literal strings can be used, as in `i18n.t("message")`. If
+you're using dynamic scoping through variables (e.g.
+`const scope = "message"; i18n.t(scope)`), they will be skipped.
+
+```console
+$ i18n lint:scripts --config test/config/lint.yml --require test/config/require.rb
+=> Config file: "test/config/lint.yml"
+=> Require file: "test/config/require.rb"
+=> Node: "/Users/fnando/.asdf/shims/node"
+=> Available locales: [:en, :es, :pt]
+=> Patterns: ["!(node_modules)/**/*.js", "!(node_modules)/**/*.ts", "!(node_modules)/**/*.jsx", "!(node_modules)/**/*.tsx"]
+=> 9 translations, 11 missing, 4 ignored
+   - test/scripts/lint/file.js:1:1: en.js.missing
+   - test/scripts/lint/file.js:1:1: es.js.missing
+   - test/scripts/lint/file.js:1:1: pt.js.missing
+   - test/scripts/lint/file.js:2:8: en.base.js.missing
+   - test/scripts/lint/file.js:2:8: es.base.js.missing
+   - test/scripts/lint/file.js:2:8: pt.base.js.missing
+   - test/scripts/lint/file.js:4:8: en.js.missing
+   - test/scripts/lint/file.js:4:8: es.js.missing
+   - test/scripts/lint/file.js:4:8: pt.js.missing
+   - test/scripts/lint/file.js:6:1: en.another_ignore_scope
+   - test/scripts/lint/file.js:6:1: es.another_ignore_scope
+```
+
+This command will list all locales and their missing translations. Avoid listing
+a particular translation, you can set `lint.ignore` on your config file.
+
+```yaml
+---
+translations:
+  - file: app/frontend/translations.json
+    patterns:
+      - "*"
+
+lint_scripts:
+  ignore:
+    - ignore_scope # will ignore this scope on all languages
+    - pt.another_ignore_scope # will ignore this scope only on `pt`
+```
+
+You can also set the patterns that will be looked up. By default, it scans all
+JavaScript and TypeScript files that don't live on `node_modules`.
+
+```yaml
+---
+translations:
+  - file: app/frontend/translations.json
+    patterns:
+      - "*"
+
+lint:
+  patterns:
+    - "app/assets/**/*.ts"
+```
 
 ## Automatically export translations
 
-### Using watchman
+### Using [watchman](https://facebook.github.io/watchman/)
 
 Create a script at `bin/i18n-watch`.
 
@@ -182,11 +398,11 @@ line to your Procfile:
 i18n: ./bin/i18n-watch
 ```
 
-### Using guard
+### Using [guard](https://rubygems.org/gems/guard)
 
-Install [guard](https://rubygems.org/packages/guard) and
-[guard-compat](https://rubygems.org/packages/guard-compat). Then create a
-Guardfile with the following configuration:
+Install [guard](https://rubygems.org/gems/guard) and
+[guard-compat](https://rubygems.org/gems/guard-compat). Then create a Guardfile
+with the following configuration:
 
 ```ruby
 guard(:"i18n-js",
@@ -204,7 +420,7 @@ accordingly.
 
 Now you can run `guard start -i`.
 
-### Using listen
+### Using [listen](https://rubygems.org/gems/listen)
 
 Create a file under `config/initializers/i18n.rb` with the following content:
 
@@ -216,8 +432,25 @@ end
 ```
 
 The code above will watch for changes based on `config/i18n.yml` and
-`config/locales`. You can customize these options with
-`I18nJS.listen(config_file: "config/i18n.yml", locales_dir: "config/locales")`.
+`config/locales`. You can customize these options:
+
+- `config_file` - i18n-js configuration file
+- `locales_dir` - one or multiple directories to watch for locales changes
+- `options` - passed directly to
+  [listen](https://github.com/guard/listen/#options)
+- `run_on_start` - export files on start. Defaults to `true`. When disabled,
+  files will be exported only when there are file changes.
+
+Example:
+
+```ruby
+I18nJS.listen(
+  config_file: "config/i18n.yml",
+  locales_dir: ["config/locales", "app/views"],
+  options: {only: %r{.yml$}},
+  run_on_start: false
+)
+```
 
 ### Integrating with your frontend
 
@@ -227,6 +460,13 @@ that loads all the exported translation.
 
 ### FAQ
 
+#### I'm running v3. Is there a migration plan?
+
+[There's a document](https://github.com/fnando/i18n-js/tree/main/MIGRATING_FROM_V3_TO_V4.md)
+outlining some of the things you need to do to migrate from v3 to v4. It may not
+be as complete as we'd like it to be, so let's know if you face any issues
+during the migration is not outline is that document.
+
 #### How can I export translations without having a database around?
 
 Some people may have a build process using something like Docker that don't
@@ -246,9 +486,11 @@ require "action_view/railtie"
 I18n.load_path += Dir["./config/locales/**/*.yml"]
 ```
 
-Notice that you may not need to load ActiveSupport and ActionView lines, or even
-may need to add additional requires for other libs. With this approach you have
-full control on what's going to be loaded.
+> **Note**:
+>
+> You may not need to load ActiveSupport and ActionView lines, or even may need
+> to add additional requires for other libs. With this approach you have full
+> control on what's going to be loaded.
 
 ## Maintainer