lokalise_manager 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: f0640f0599d5b4967f79d2ced69f63d7cb7edde3e2a3a49336d08d17e5fb4c0f
+  data.tar.gz: 0766d8cb7f6b478e427f8ef7dd1409d86072a74a4606b026eaa718f6fe4761b5
+SHA512:
+  metadata.gz: 38a3adfcc0bf4a78f595547cdc3ba2962a4122d8bbeaa7e132d47dc4e76992711e653710a802e395fafcb0c316743607876a10f0914c2633472994891fd4850e
+  data.tar.gz: 98b2edd83de044615db56ee71d090afa03fcfc7793c55854e322ab466996e0eb46c74f3b22dbdbd44e570f31746d40394912ba39c29dd17337b17e08d0417761

data/.github/CODE_OF_CONDUCT.md

@@ -0,0 +1,46 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at hello@lokalise.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/.github/CONTRIBUTING.md

@@ -0,0 +1,14 @@
+# Contributing
+
+1. [Fork the repository.][fork]
+2. [Create a topic branch.][branch]
+3. Implement your feature or bug fix.
+4. Don't forget to add specs and make sure they pass by running `rspec .`.
+5. Make sure your code complies with the style guide by running `rubocop`. `rubocop -a` can automatically fix most issues for you.
+6. If necessary, add documentation for your feature or bug fix.
+7. Commit and push your changes.
+8. [Submit a pull request.][pr]
+
+[fork]: http://help.github.com/fork-a-repo/
+[branch]: https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-branches
+[pr]: https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests

data/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,11 @@
+### Summary
+
+Provide a general description of the code changes in your pull
+request. Were there any bugs you had fixed? If so, mention them. If
+these bugs have open GitHub issues, be sure to tag them here as well,
+to keep the conversation linked together.
+
+### Other Information
+
+If there's anything else that's important and relevant to your pull
+request, mention that information here. 

data/CHANGELOG.md

@@ -0,0 +1,5 @@
+# Changelog
+
+## 1.0.0 (14-Oct-21)
+
+* Initial release

data/Gemfile

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+source 'http://rubygems.org'
+
+gemspec

data/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2021 Lokalise team, Ilya Bodrov
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,266 @@
+# LokaliseManager
+
+![Gem](https://img.shields.io/gem/v/lokalise_manager)
+[![Build Status](https://travis-ci.com/bodrovis/lokalise_manager.svg?branch=master)](https://travis-ci.com/github/bodrovis/lokalise_manager)
+[![Test Coverage](https://codecov.io/gh/bodrovis/lokalise_manager/graph/badge.svg)](https://codecov.io/gh/bodrovis/lokalise_manager)
+![Downloads total](https://img.shields.io/gem/dt/lokalise_manager)
+
+This gem provides [Lokalise](http://lokalise.com) integration for Ruby and allows to exchange translation files between your project and TMS easily. It relies on [ruby-lokalise-api](https://lokalise.github.io/ruby-lokalise-api) to send APIv2 requests.
+
+If you are looking for a Rails integration, please check [lokalise_rails](https://github.com/bodrovis/lokalise_rails) which provides a set of Rake tasks for importing/exporting.
+
+## Getting started
+
+### Requirements
+
+This gem requires Ruby 2.5+. You will also need to [setup a Lokalise account](https://app.lokalise.com/signup) and create a [translation project](https://docs.lokalise.com/en/articles/1400460-projects). Finally, you will need to generate a [read/write API token](https://docs.lokalise.com/en/articles/1929556-api-tokens) at your Lokalise profile.
+
+### Installation
+
+Add the gem to your `Gemfile`:
+
+```ruby
+gem 'lokalise_manager'
+```
+
+and run:
+
+```
+bundle
+```
+
+### Creating a client
+
+To import or export translation files, you'll have to create the corresponding client:
+
+```ruby
+importer = LokaliseManager.importer api_token: '1234abc', project_id: '123.abc'
+
+# OR
+
+exporter = LokaliseManager.exporter api_token: '1234abc', project_id: '123.abc'
+```
+
+You *must* provide an API token and a project ID (your project ID can be found under Lokalise project settings). [Other options can be customized as well (see below)](https://github.com/bodrovis/lokalise_manager#configuration) but they have sensible defaults.
+
+### Importing files from Lokalise into your project
+
+To download translation files from Lokalise into your project (by default all files will be stored under the `locales/` directory), run the following code:
+
+```ruby
+result = importer.import!
+```
+
+The `result` will contain a boolean value which says whether the operation was successfull or not.
+
+Please note that upon importing translations any duplicating files inside the `locales` directory (or any other directory that you've specified in the options) will be overwritten! You can enable [safe mode](https://github.com/bodrovis/lokalise_manager#import-config) to check whether the folder is empty or not.
+
+### Exporting files from your project to Lokalise
+
+To upload your translation files from a local directory (defaults to `locales/`) to a Lokalise project, run the following code: 
+
+```ruby
+processes = exporter.export!
+```
+
+`processes` will contain an array of queued background processes as uploading is done in the background on Lokalise. You can perform periodic checks to read the status of the process. Here's a very simple example:
+
+```ruby
+def uploaded?(process)
+  5.times do # try to check the status 5 times
+    process = process.reload_data # load new data
+    return(true) if process.status == 'finished' # return true is the upload has finished
+    sleep 1 # wait for 1 second, adjust this number with regards to the upload size
+  end
+
+  false # if all 5 checks failed, return false (probably something is wrong)
+end
+
+processes = exporter.export!
+uploaded? processes[0]
+```
+
+Please don't forget that Lokalise API has rate limiting and you cannot send more than six requests per second.
+
+## Configuration
+
+### Common config
+
+* `api_token` (`string`, required) — Lokalise API token with read/write permissions.
+* `project_id` (`string`, required) — Lokalise project ID. You must have import/export permissions in the specified project.
+* `locales_path` (`string`) — path to the directory with your translation files. Defaults to `"#{Dir.getwd}/locales"`.
+* `branch` (`string`) — Lokalise project branch to use. Defaults to `""` (no branch is provided).
+* `timeouts` (`hash`) — set [request timeouts for the Lokalise API client](https://lokalise.github.io/ruby-lokalise-api/additional_info/customization#setting-timeouts). By default, requests have no timeouts: `{open_timeout: nil, timeout: nil}`. Both values are in seconds.
+
+### Import config
+
+* `import_opts` (`hash`) — options that will be passed to Lokalise API when downloading translations to your app. Here are the default options:
+
+```ruby
+{
+  format: 'yaml',
+  placeholder_format: :icu,
+  yaml_include_root: true,
+  original_filenames: true,
+  directory_prefix: '',
+  indentation: '2sp'
+}
+```
+
+Full list of available import options [can be found in the official API documentation](https://app.lokalise.com/api2docs/curl/#transition-download-files-post).
+
+You can provide additional options, and they will be merged with the default ones. For example:
+
+```ruby
+importer = LokaliseManager.importer api_token: '1234abc',
+                                    project_id: '123.abc',
+                                    import_opts: {original_filenames: true}
+```
+
+In this case the `import_opts` will have `original_filenames` set to `true` and will also contain all the defaults (`format`, `placeholder_format`, and others). Of course, you can override defaults as well:
+
+```ruby
+importer = LokaliseManager.importer api_token: '1234abc',
+                                    project_id: '123.abc',
+                                    import_opts: {indentation: '4sp}
+```
+
+* `import_safe_mode` (`boolean`) — default to `false`. When this option is enabled, the import task will check whether the directory set with `locales_path` is empty or not. If it is not empty, you will be prompted to continue.
+* `max_retries_import` (`integer`) — this option is introduced to properly handle Lokalise API rate limiting. If the HTTP status code 429 (too many requests) has been received, this gem will apply an exponential backoff mechanism with a very simple formula: `2 ** retries`. If the maximum number of retries has been reached, a `Lokalise::Error::TooManyRequests` exception will be raised and the operation will be halted.
+
+### Export config
+
+* `export_opts` (`hash`) — options that will be passed to Lokalise API when uploading translations. Full list of available export options [can be found in the official documentation](https://app.lokalise.com/api2docs/curl/#transition-upload-a-file-post). By default, the following options are provided:
+  + `data` (`string`, required) — base64-encoded contents of the translation file.
+  + `filename` (`string`, required) — translation file name. If the file is stored under a subdirectory (for example, `nested/en.yml` inside the `locales/` directory), the whole path acts as a name. Later when importing files with such names, they will be placed into the proper subdirectories.
+  + `lang_iso` (`string`, required) — language ISO code which is determined using the root key inside your YAML file. For example, in this case the `lang_iso` is `en_US`:
+
+```yaml
+en_US:
+  my_key: "my value"
+```
+
+You can provide additional options, and they will be merged with the default ones. For example:
+
+```ruby
+exporter = LokaliseManager.exporter api_token: '1234abc',
+                                    project_id: '123.abc',
+                                    export_opts: {detect_icu_plurals: true}
+```
+
+In this case the `export_opts` will have `detect_icu_plurals` set to `true` and will also contain all the defaults (`data`, `filename`, and `lang_iso`).
+
+**Please note** that if your Lokalise project does not have a language with the specified `lang_iso` code, the export will fail. It means that you first have to add all the locales to the project and then start the exporting process.
+
+* `skip_file_export` (`lambda` or `proc`) — specify additional exclusion criteria for the exported files. By default, the rake task will ignore all non-file entries and all files with improper extensions (the latter is controlled by the `file_ext_regexp`). Lambda passed to this option should accept a single argument which is full path to the file (instance of the [`Pathname` class](https://ruby-doc.org/stdlib-2.7.1/libdoc/pathname/rdoc/Pathname.html)). For example, to exclude all files that have `fr` part in their names, add the following config:
+
+```ruby
+c.skip_file_export = ->(file) { f.split[1].to_s.include?('fr') }
+```
+
+* `max_retries_export` (`integer`) — this option is introduced to properly handle Lokalise API rate limiting. If the HTTP status code 429 (too many requests) has been received, LokaliseRails will apply an exponential backoff mechanism with a very simple formula: `2 ** retries` (initially `retries` is `0`). If the maximum number of retries has been reached, a `Lokalise::Error::TooManyRequests` exception will be raised and the export operation will be halted. By default, LokaliseRails will make up to `5` retries which potentially means `1 + 2 + 4 + 8 + 16 + 32 = 63` seconds of waiting time. If the `max_retries_export` is less than `1`, LokaliseRails will not perform any retries and give up immediately after receiving error 429.
+
+### Config to work with formats other than YAML
+
+If your translation files are not in YAML format, you will need to adjust the following options:
+
+* `file_ext_regexp` (`regexp`) — regular expression applied to file extensions to determine which files should be imported and exported. Defaults to `/\.ya?ml\z/i` (YAML files).
+* `translations_loader` (`lambda` or `proc`) — loads translations data and makes sure they are valid before saving them to a translation file. Defaults to `->(raw_data) { YAML.safe_load raw_data }`. In the simplest case you may just return the data back, for example `-> (raw_data) { raw_data }`.
+* `translations_converter` (`lambda` or `proc`) — converts translations data to a proper format before saving them to a translation file. Defaults to `->(raw_data) { raw_data.to_yaml }`. In the simplest case you may just return the data back, for example `-> (raw_data) { raw_data }`.
+* `lang_iso_inferer` (`lambda` or `proc`) — infers language ISO code based on the translation file data before uploading it to Lokalise. Defaults to `->(data) { YAML.safe_load(data)&.keys&.first }`.
+
+## Providing config options
+
+### Per-client
+
+The simplest way to provide your config is on per-client basis, for example:
+
+```ruby
+importer = LokaliseManager.importer api_token: '1234abc',
+                                    project_id: '123.abc',
+                                    import_opts: {original_filenames: true},
+                                    export_opts: {detect_icu_plurals: true},
+                                    translations_converter: -> (raw_data) { raw_data }
+```
+
+These options will be merged with the default ones. Please note that per-client config has the highest priority.
+
+You can also adjust individual options later using the `#config` instance variable (it contains an OpenStruct object):
+
+```
+importer.config.project_id = '678xyz'
+importer.config.branch = 'develop'
+```
+
+### Globally
+
+You can also provide config globally. To achieve that, call `.config` on `LokaliseManager::Config` class:
+
+```ruby
+LokaliseManager::GlobalConfig.config do |c|
+  c.api_token = '12345'
+  c.project_id = '123.abc'
+
+  c.branch = 'develop'
+end
+```
+
+Global config takes precedence over the default options, however per-client config has higher precedence.
+
+### Overriding defaults
+
+You can even subclass the default `GlobalConfig` class and provide the new defaults:
+
+```ruby
+class CustomConfig < LokaliseManager::GlobalConfig
+  class << self
+    def branch
+      @branch || 'develop'
+    end
+
+    def locales_path
+      @locales_path || "#{Dir.getwd}/i18n/locales"
+    end
+  end
+end
+```
+
+Check [`global_config.rb` source](https://github.com/bodrovis/lokalise_manager/blob/master/lib/lokalise_manager/global_config.rb) to find other defaults.
+
+Now when you have created a custom config, you can also set some global options (this is not required though):
+
+```ruby
+CustomConfig.config do |c|
+  c.api_token = '123'
+  c.project_id = '456.abc'
+end
+```
+
+However, it's required to pass your new class when instantiating the clients:
+
+```ruby
+importer = LokaliseManager.importer({}, CustomConfig)
+```
+
+Please note that round brackets are required in this case.
+
+The first argument is your per-client options, whereas the second argument contains the class with all the defaults.
+
+Of course, you can still provide options on per-client basis:
+
+```ruby
+importer = LokaliseManager.importer({api_token: '890abcdef'}, CustomConfig)
+
+# Now we can run the import process as before:
+
+importer.import!
+```
+
+## Running tests
+
+1. Copypaste `.env.example` file as `.env`. Put your Lokalise API token and project ID inside. The `.env` file is excluded from version control so your data is safe. All in all, we use pre-recorded VCR cassettes, so the actual API requests won’t be sent. However, providing at least some values is required.
+2. Run `rspec .`. Observe test results and code coverage.
+
+## License
+
+Copyright (c) [Lokalise team](http://lokalise.com), [Ilya Bodrov](http://bodrovis.tech). License type is [MIT](https://github.com/bodrovis/lokalise_manager/blob/master/LICENSE).

data/Rakefile

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+require 'rake'
+
+begin
+  require 'bundler/setup'
+  Bundler::GemHelper.install_tasks
+rescue LoadError
+  puts 'although not required, bundler is recommened for running the tests'
+end
+
+task default: :spec
+
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec)
+
+require 'rubocop/rake_task'
+RuboCop::RakeTask.new do |task|
+  task.requires << 'rubocop-performance'
+  task.requires << 'rubocop-rspec'
+end

data/lib/lokalise_manager/error.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+module LokaliseManager
+  class Error < StandardError
+    # Initializes a new Error object
+    def initialize(message = '')
+      super(message)
+    end
+  end
+end

data/lib/lokalise_manager/global_config.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+module LokaliseManager
+  class GlobalConfig
+    class << self
+      attr_accessor :api_token, :project_id
+      attr_writer :import_opts, :import_safe_mode, :export_opts, :locales_path,
+                  :file_ext_regexp, :skip_file_export, :branch, :timeouts,
+                  :translations_loader, :translations_converter, :lang_iso_inferer,
+                  :max_retries_export, :max_retries_import
+
+      # Main interface to provide configuration options
+      def config
+        yield self
+      end
+
+      # Full path to directory with translation files
+      def locales_path
+        @locales_path || "#{Dir.getwd}/locales"
+      end
+
+      # Project branch to use
+      def branch
+        @branch || ''
+      end
+
+      # Set request timeouts for the Lokalise API client
+      def timeouts
+        @timeouts || {}
+      end
+
+      # Maximum number of retries for file exporting
+      def max_retries_export
+        @max_retries_export || 5
+      end
+
+      # Maximum number of retries for file importing
+      def max_retries_import
+        @max_retries_import || 5
+      end
+
+      # Regular expression used to select translation files with proper extensions
+      def file_ext_regexp
+        @file_ext_regexp || /\.ya?ml\z/i
+      end
+
+      # Options for import rake task
+      def import_opts
+        @import_opts || {
+          format: 'yaml',
+          placeholder_format: :icu,
+          yaml_include_root: true,
+          original_filenames: true,
+          directory_prefix: '',
+          indentation: '2sp'
+        }
+      end
+
+      # Options for export rake task
+      def export_opts
+        @export_opts || {}
+      end
+
+      # Enables safe mode for import. When enabled, will check whether the target folder is empty or not
+      def import_safe_mode
+        @import_safe_mode.nil? ? false : @import_safe_mode
+      end
+
+      # Additional file skip criteria to apply when performing export
+      def skip_file_export
+        @skip_file_export || ->(_) { false }
+      end
+
+      def translations_loader
+        @translations_loader || ->(raw_data) { YAML.safe_load raw_data }
+      end
+
+      # Converts translations data to the proper format
+      def translations_converter
+        @translations_converter || ->(raw_data) { raw_data.to_yaml }
+      end
+
+      # Infers lang ISO for the given translation file
+      def lang_iso_inferer
+        @lang_iso_inferer || ->(data) { YAML.safe_load(data)&.keys&.first }
+      end
+    end
+  end
+end

data/lib/lokalise_manager/task_definitions/base.rb

@@ -0,0 +1,98 @@
+# frozen_string_literal: true
+
+require 'ruby-lokalise-api'
+require 'pathname'
+require 'ostruct'
+
+module LokaliseManager
+  module TaskDefinitions
+    class Base
+      attr_accessor :config
+
+      # Creates a new importer or exporter. It accepts custom config and merges it
+      # with the global config (custom config take precendence)
+      #
+      # @param custom_opts [Hash]
+      # @param global_config [Object]
+      def initialize(custom_opts = {}, global_config = LokaliseManager::GlobalConfig)
+        primary_opts = global_config.
+                       singleton_methods.
+                       filter { |m| m.to_s.end_with?('=') }.
+                       each_with_object({}) do |method, opts|
+          reader = method.to_s.delete_suffix('=')
+          opts[reader.to_sym] = global_config.send(reader)
+        end
+
+        @config = OpenStruct.new primary_opts.merge(custom_opts)
+      end
+
+      # Creates a Lokalise API client
+      #
+      # @return [Lokalise::Client]
+      def api_client
+        @api_client ||= ::Lokalise.client config.api_token, {enable_compression: true}.merge(config.timeouts)
+      end
+
+      # Resets API client
+      def reset_api_client!
+        Lokalise.reset_client!
+        @api_client = nil
+      end
+
+      private
+
+      # Checks task options
+      #
+      # @return Array
+      def check_options_errors!
+        errors = []
+        errors << 'Project ID is not set!' if config.project_id.nil? || config.project_id.empty?
+        errors << 'Lokalise API token is not set!' if config.api_token.nil? || config.api_token.empty?
+
+        raise(LokaliseManager::Error, errors.join(' ')) if errors.any?
+      end
+
+      # Checks whether the provided file has a proper extension as dictated by the `file_ext_regexp` option
+      #
+      # @return Boolean
+      # @param raw_path [String, Pathname]
+      def proper_ext?(raw_path)
+        path = raw_path.is_a?(Pathname) ? raw_path : Pathname.new(raw_path)
+        config.file_ext_regexp.match? path.extname
+      end
+
+      # Returns directory and filename for the given entry
+      #
+      # @return Array
+      # @param entry [String]
+      def subdir_and_filename_for(entry)
+        Pathname.new(entry).split
+      end
+
+      # Returns Lokalise project ID and branch, semicolumn separated
+      #
+      # @return [String]
+      def project_id_with_branch
+        return config.project_id.to_s if config.branch.to_s.strip.empty?
+
+        "#{config.project_id}:#{config.branch}"
+      end
+
+      # Sends request with exponential backoff mechanism
+      def with_exp_backoff(max_retries)
+        return unless block_given?
+
+        retries = 0
+        begin
+          yield
+        rescue Lokalise::Error::TooManyRequests => e
+          raise(e.class, "Gave up after #{retries} retries") if retries >= max_retries
+
+          sleep 2**retries
+          retries += 1
+          retry
+        end
+      end
+    end
+  end
+end