lokalise_manager 1.1.0 → 2.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a6dbbda54dc255ed4ec13547b954213e41ab38c90d1520ffe3762d36dc2917f8
-  data.tar.gz: a341a91e4fea6ff76e0de87e5efa9660f5a0c5a7659656fbcfa14600fc4937d8
+  metadata.gz: f3a7da094cb7939fd6826c89afbbbc525dff59b553a23fba7b49ed38046e68b9
+  data.tar.gz: c0a323464c864d12042f126d765a483adc128a4397604d012cdeadee31003fcf
 SHA512:
-  metadata.gz: 2f5f277f431596f70ee52f45186a972dcd507d1be08f6fc71bd04117556ca803efbe3e8b0d6a123ec6f81b959a5a1c99d9aa63a3189a4700a195fb823ca766e2
-  data.tar.gz: c64848ee98ee90cd77507bae719048ac425c60b7f16581b52de6ed6ab2301976487f6999027eab729244c4d3e8b27af1d5d6875c74d8b3d7cbd35112e0ededf1
+  metadata.gz: 79c0b5a93767b5ffa8cd55dbf6591a90cafbb586043e8e75ca040761b8fa882882971783358e6d342afab96e298d34c67ca5d414d376c909fa564e8ad03d2466
+  data.tar.gz: ed2ad8e59fb6e15db8ba7c60de48e808c2eab0ff94b8f287c54bb2e618e1c930d5d3244a2483a6e61e69b7094943a05db59ea54ef32fc3596ef344f5956553ee

data/CHANGELOG.md

@@ -1,5 +1,67 @@
 # Changelog
 
+## 2.1.0 (27-Jan-22)
+
+* **Breaking change**: `export!` will now return an array of objects responding to the following methods:
+  + `success` — usually returns `true` (to learn more, check documentation for the `:raise_on_export_fail` option below)
+  + `process` — returns an object (an instance of the `Lokalise::Resources::QueuedProcess`) representing a [queued background process](https://lokalise.github.io/ruby-lokalise-api/api/queued-processes) as uploading is done in the background on Lokalise. You can use this object to check the process status (whether the uploading is completed or not).
+  + `path` — returns an instance of the `Pathname` class which represent the file being uploaded.
+* Here's an example:
+
+```ruby
+def uploaded?(process)
+  5.times do # try to check the status 5 times
+    process = process.reload_data # load new data
+    return(true) if process.status == 'finished' # return true is the upload has finished
+    sleep 1 # wait for 1 second, adjust this number with regards to the upload size
+  end
+
+  false # if all 5 checks failed, return false (probably something is wrong)
+end
+
+processes = exporter.export!
+puts "Checking status for the #{processes[0].path} file"
+uploaded? processes[0].process
+```
+
+* Introduced a new option `raise_on_export_fail` (`boolean`) which is `true` by default. When this option is enabled, LokaliseManager will re-raise any exceptions that happened during the file uploading. When this option is disabled, the exporting process will continue even if something goes wrong. In this case you'll probably need to check the result yourself and make the necessary actions. For example:
+
+```ruby
+processes = exporter.export!
+
+processes.each do |proc_data|
+  if proc_data.success
+    # Everything is good, the uploading is queued
+    puts "#{proc_data.path} is sent to Lokalise!"
+    process = proc_data.process
+    puts "Current process status is #{process.status}"
+  else
+    # Something bad has happened
+    puts "Could not send #{proc_data.path} to Lokalise"
+    puts "Error #{proc_data.error.class}: #{proc_data.error.message}"
+    # Or you could re-raise this exception:
+    # raise proc_data.error.class
+  end
+end
+```
+
+## 2.0.0 (27-Jan-22)
+
+* `export!` method is now taking advantage of multi-threading (as Lokalise API allows to send requests in parallel since January 2022)
+* Test with Ruby 3.1.0
+* Other minor fixes
+
+## 1.2.1 (26-Nov-21)
+
+* Use refinements instead of monkey patching to add hash methods
+* Don't use `OpenStruct` anymore to store config opts
+* Minor fixes
+
+## 1.2.0 (26-Oct-21)
+
+* Add a new option `:silent_mode` which is `false` by default. When silent mode is enabled, no debug info will be printed out to `$stdout`. The only exception are the "safe mode" messages — you'll still be prompted to continue if the target directory is not empty.
+* Use `#deep_merge` instead of a simple merge when processing options.
+
 ## 1.1.0 (25-Oct-21)
 
 * Add a new option `:use_oauth2_token` which is `false` by default. When enabled, you'll be able to provide a token obtained via [OAuth 2 flow](https://docs.lokalise.com/en/articles/5574713-oauth-2) rather than generated via Lokalise profile. The token should still be provided via the `:api_token` option:

data/README.md

@@ -1,7 +1,7 @@
 # LokaliseManager
 
 ![Gem](https://img.shields.io/gem/v/lokalise_manager)
-[![Build Status](https://travis-ci.com/bodrovis/lokalise_manager.svg?branch=master)](https://travis-ci.com/github/bodrovis/lokalise_manager)
+![CI](https://github.com/bodrovis/lokalise_manager/actions/workflows/ci.yml/badge.svg)
 [![Test Coverage](https://codecov.io/gh/bodrovis/lokalise_manager/graph/badge.svg)](https://codecov.io/gh/bodrovis/lokalise_manager)
 ![Downloads total](https://img.shields.io/gem/dt/lokalise_manager)
 
@@ -15,6 +15,8 @@ If you are looking for a Rails integration, please check [lokalise_rails](https:
 
 This gem requires Ruby 2.5+. You will also need to [setup a Lokalise account](https://app.lokalise.com/signup) and create a [translation project](https://docs.lokalise.com/en/articles/1400460-projects). Finally, you will need to generate a [read/write API token](https://docs.lokalise.com/en/articles/1929556-api-tokens) at your Lokalise profile.
 
+Alternatively, you can utilize a token obtained via OAuth 2 flow. When using such a token, you'll have to set `:use_oauth2_token` option to `true` (see below).
+
 ### Installation
 
 Add the gem to your `Gemfile`:
@@ -63,7 +65,13 @@ To upload your translation files from a local directory (defaults to `locales/`)
 processes = exporter.export!
 ```
 
-`processes` will contain an array of queued background processes as uploading is done in the background on Lokalise. You can perform periodic checks to read the status of the process. Here's a very simple example:
+`processes` will contain an array of objects responding to the following methods:
+
+* `success` — usually returns `true` (to learn more, check documentation for the `:raise_on_export_fail` option below)
+* `process` — returns an object (an instance of the `Lokalise::Resources::QueuedProcess`) representing a [queued background process](https://lokalise.github.io/ruby-lokalise-api/api/queued-processes) as uploading is done in the background on Lokalise.
+* `path` — returns an instance of the `Pathname` class which represent the file being uploaded.
+
+You can perform periodic checks to read the status of the process. Here's a very simple example:
 
 ```ruby
 def uploaded?(process)
@@ -77,7 +85,8 @@ def uploaded?(process)
 end
 
 processes = exporter.export!
-uploaded? processes[0]
+puts "Checking status for the #{processes[0].path} file"
+uploaded? processes[0].process
 ```
 
 Please don't forget that Lokalise API has rate limiting and you cannot send more than six requests per second.
@@ -92,6 +101,7 @@ Please don't forget that Lokalise API has rate limiting and you cannot send more
 * `locales_path` (`string`) — path to the directory with your translation files. Defaults to `"#{Dir.getwd}/locales"`.
 * `branch` (`string`) — Lokalise project branch to use. Defaults to `""` (no branch is provided).
 * `timeouts` (`hash`) — set [request timeouts for the Lokalise API client](https://lokalise.github.io/ruby-lokalise-api/additional_info/customization#setting-timeouts). By default, requests have no timeouts: `{open_timeout: nil, timeout: nil}`. Both values are in seconds.
+* `silent_mode` (`boolean`) — whether you would like to output debugging information to `$stdout`. By default, after a task is performed, a short notification message will be printed out to the terminal. When set to `false`, notifications won't be printed. Please note that currently `import_safe_mode` has higher priority. Even if you enable `silent_mode`, and the `import_safe_mode` is enabled as well, you will be prompted to confirm the import operation if the target directory is not empty.
 
 ### Import config
 
@@ -159,7 +169,29 @@ In this case the `export_opts` will have `detect_icu_plurals` set to `true` and
 c.skip_file_export = ->(file) { f.split[1].to_s.include?('fr') }
 ```
 
-* `max_retries_export` (`integer`) — this option is introduced to properly handle Lokalise API rate limiting. If the HTTP status code 429 (too many requests) has been received, LokaliseRails will apply an exponential backoff mechanism with a very simple formula: `2 ** retries` (initially `retries` is `0`). If the maximum number of retries has been reached, a `Lokalise::Error::TooManyRequests` exception will be raised and the export operation will be halted. By default, LokaliseRails will make up to `5` retries which potentially means `1 + 2 + 4 + 8 + 16 + 32 = 63` seconds of waiting time. If the `max_retries_export` is less than `1`, LokaliseRails will not perform any retries and give up immediately after receiving error 429.
+* `max_retries_export` (`integer`) — this option is introduced to properly handle Lokalise API rate limiting. If the HTTP status code 429 (too many requests) has been received, LokaliseRails will apply an exponential backoff mechanism with a very simple formula: `2 ** retries` (initially `retries` is `0`). If the maximum number of retries has been reached, a `Lokalise::Error::TooManyRequests` exception will be raised and the export operation will be halted. By default, LokaliseManager will make up to `5` retries which potentially means `1 + 2 + 4 + 8 + 16 + 32 = 63` seconds of waiting time. If the `max_retries_export` is less than `1`, LokaliseRails will not perform any retries and give up immediately after receiving error 429.
+* `raise_on_export_fail` (`boolean`) — default is `true`. When this option is enabled, LokaliseManager will re-raise any exceptions that happened during the file uploading. In other words, if any uploading thread raised an exception, your exporting process will exit with an exception. Suppose, you are uploading 12 translation files; these files will be split in 2 groups with 6 files each, and each group will be uploaded in parallel (using threads). However, suppose some exception happens when uploading the first group. By default this exception will be re-raised for the whole process and the script will never try to upload the second group. If you would like to continue uploading even if an exception happened, set the `raise_on_export_fail` to `false`. In this case the `export!` method will return an array with scheduled processes and with information about processes that were not successfully scheduled. This information is represented as an object with three methods: `path` (contains an instance of the `Pathname` class which says which file could not be uploaded), `error` (the actual exception), and `success` (returns `false`). So, you can use the following snippet to check your processes:
+
+```ruby
+processes = exporter.export!
+
+processes.each do |proc_data|
+  if proc_data.success
+    # Everything is good, the uploading is queued
+    puts "#{proc_data.path} is sent to Lokalise!"
+    process = proc_data.process
+    puts "Current process status is #{process.status}"
+  else
+    # Something bad has happened
+    puts "Could not send #{proc_data.path} to Lokalise"
+    puts "Error #{proc_data.error.class}: #{proc_data.error.message}"
+    # Or you could re-raise this exception:
+    # raise proc_data.error.class
+  end
+end
+```
+
+* For example, you could collect all the files that were uploaded successfully, re-create the exporter object with the `skip_file_export` option (skipping all files that were successfully imported), and re-run the whole exporting process once again.
 
 ### Config to work with formats other than YAML
 
@@ -268,4 +300,4 @@ importer.import!
 
 ## License
 
-Copyright (c) [Lokalise team](http://lokalise.com), [Ilya Bodrov](http://bodrovis.tech). License type is [MIT](https://github.com/bodrovis/lokalise_manager/blob/master/LICENSE).
+Copyright (c) [Lokalise team](http://lokalise.com), [Ilya Bodrov](http://bodrovis.tech). License type is [MIT](https://github.com/bodrovis/lokalise_manager/blob/master/LICENSE).

data/lib/lokalise_manager/global_config.rb

@@ -7,13 +7,24 @@ module LokaliseManager
       attr_writer :import_opts, :import_safe_mode, :export_opts, :locales_path,
                   :file_ext_regexp, :skip_file_export, :branch, :timeouts,
                   :translations_loader, :translations_converter, :lang_iso_inferer,
-                  :max_retries_export, :max_retries_import, :use_oauth2_token
+                  :max_retries_export, :max_retries_import, :use_oauth2_token, :silent_mode,
+                  :raise_on_export_fail
 
       # Main interface to provide configuration options
       def config
         yield self
       end
 
+      # When enabled, will re-raise any exception that happens during file exporting
+      def raise_on_export_fail
+        @raise_on_export_fail || true
+      end
+
+      # When enabled, won't print any debugging info to $stdout
+      def silent_mode
+        @silent_mode || false
+      end
+
       # When enabled, will use OAuth 2 Lokalise client and will require to provide a token obtained via OAuth 2 flow
       # rather than via Lokalise profile
       def use_oauth2_token

data/lib/lokalise_manager/task_definitions/base.rb

@@ -2,11 +2,12 @@
 
 require 'ruby-lokalise-api'
 require 'pathname'
-require 'ostruct'
 
 module LokaliseManager
   module TaskDefinitions
     class Base
+      using LokaliseManager::Utils::HashUtils
+
       attr_accessor :config
 
       # Creates a new importer or exporter. It accepts custom config and merges it
@@ -23,7 +24,11 @@ module LokaliseManager
           opts[reader.to_sym] = global_config.send(reader)
         end
 
-        @config = OpenStruct.new primary_opts.merge(custom_opts)
+        all_opts = primary_opts.deep_merge(custom_opts)
+
+        config_klass = Struct.new(*all_opts.keys, keyword_init: true)
+
+        @config = config_klass.new all_opts
       end
 
       # Creates a Lokalise API client
@@ -33,12 +38,13 @@ module LokaliseManager
         client_opts = [config.api_token, {enable_compression: true}.merge(config.timeouts)]
         client_method = config.use_oauth2_token ? :oauth_client : :client
 
-        @api_client ||= ::Lokalise.send(client_method, *client_opts)
+        @api_client = ::Lokalise.send(client_method, *client_opts)
       end
 
       # Resets API client
       def reset_api_client!
-        Lokalise.reset_client!
+        ::Lokalise.reset_client!
+        ::Lokalise.reset_oauth_client!
         @api_client = nil
       end
 

data/lib/lokalise_manager/task_definitions/exporter.rb

@@ -5,6 +5,10 @@ require 'base64'
 module LokaliseManager
   module TaskDefinitions
     class Exporter < Base
+      using LokaliseManager::Utils::ArrayUtils
+
+      MAX_THREADS = 6
+
       # Performs translation file export to Lokalise and returns an array of queued processes
       #
       # @return [Array]
@@ -12,31 +16,50 @@ module LokaliseManager
         check_options_errors!
 
         queued_processes = []
-        each_file do |full_path, relative_path|
-          queued_processes << do_upload(full_path, relative_path)
-        rescue StandardError => e
-          raise e.class, "Error while trying to upload #{full_path}: #{e.message}"
+
+        all_files.in_groups_of(MAX_THREADS) do |files_group|
+          parallel_upload(files_group).each do |thr|
+            raise_on_fail(thr) if config.raise_on_export_fail
+
+            queued_processes.push thr
+          end
         end
 
-        $stdout.print 'Task complete!'
+        $stdout.print('Task complete!') unless config.silent_mode
 
         queued_processes
       end
 
       private
 
+      def parallel_upload(files_group)
+        files_group.compact.map do |file_data|
+          do_upload(*file_data)
+        end.map(&:value)
+      end
+
+      def raise_on_fail(thread)
+        raise(thread.error.class, "Error while trying to upload #{thread.path}: #{thread.error.message}") unless thread.success
+      end
+
       # Performs the actual file uploading to Lokalise. If the API rate limit is exceeed,
       # applies exponential backoff
       def do_upload(f_path, r_path)
-        with_exp_backoff(config.max_retries_export) do
-          api_client.upload_file project_id_with_branch, opts(f_path, r_path)
+        proc_klass = Struct.new(:success, :process, :path, :error, keyword_init: true)
+
+        Thread.new do
+          process = with_exp_backoff(config.max_retries_export) do
+            api_client.upload_file project_id_with_branch, opts(f_path, r_path)
+          end
+          proc_klass.new success: true, process: process, path: f_path
+        rescue StandardError => e
+          proc_klass.new success: false, path: f_path, error: e
         end
       end
 
-      # Processes each translation file in the specified directory
-      def each_file
-        return unless block_given?
-
+      # Gets translation files from the specified directory
+      def all_files
+        files = []
         loc_path = config.locales_path
         Dir["#{loc_path}/**/*"].sort.each do |f|
           full_path = Pathname.new f
@@ -45,8 +68,9 @@ module LokaliseManager
 
           relative_path = full_path.relative_path_from Pathname.new(loc_path)
 
-          yield full_path, relative_path
+          files << [full_path, relative_path]
         end
+        files
       end
 
       # Generates export options

data/lib/lokalise_manager/task_definitions/importer.rb

@@ -14,13 +14,13 @@ module LokaliseManager
         check_options_errors!
 
         unless proceed_when_safe_mode?
-          $stdout.print 'Task cancelled!'
+          $stdout.print('Task cancelled!') unless config.silent_mode
           return false
         end
 
         open_and_process_zip download_files['bundle_url']
 
-        $stdout.print 'Task complete!'
+        $stdout.print('Task complete!') unless config.silent_mode
         true
       end
 

data/lib/lokalise_manager/utils/array_utils.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+# Taken from https://github.com/rails/rails/blob/6bfc637659248df5d6719a86d2981b52662d9b50/activesupport/lib/active_support/core_ext/array/grouping.rb
+
+module LokaliseManager
+  module Utils
+    module ArrayUtils
+      refine Array do
+        def in_groups_of(number, fill_with = nil, &block)
+          if number.to_i <= 0
+            raise ArgumentError,
+                  "Group size must be a positive integer, was #{number.inspect}"
+          end
+
+          if fill_with == false
+            collection = self
+          else
+            padding = (number - (size % number)) % number
+            collection = dup.concat(Array.new(padding, fill_with))
+          end
+
+          collection.each_slice(number, &block)
+        end
+      end
+    end
+  end
+end

data/lib/lokalise_manager/utils/hash_utils.rb

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+# Taken from https://github.com/rails/rails/blob/83217025a171593547d1268651b446d3533e2019/activesupport/lib/active_support/core_ext/hash/deep_merge.rb
+
+module LokaliseManager
+  module Utils
+    module HashUtils
+      refine Hash do
+        def deep_merge(other_hash, &block)
+          dup.deep_merge!(other_hash, &block)
+        end
+
+        # Same as +deep_merge+, but modifies +self+.
+        def deep_merge!(other_hash, &block)
+          merge!(other_hash) do |key, this_val, other_val|
+            if this_val.is_a?(Hash) && other_val.is_a?(Hash)
+              this_val.deep_merge(other_val, &block)
+            elsif block
+              yield(key, this_val, other_val)
+            else
+              other_val
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/lokalise_manager/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LokaliseManager
-  VERSION = '1.1.0'
+  VERSION = '2.1.0'
 end

data/lib/lokalise_manager.rb

@@ -2,6 +2,9 @@
 
 require 'yaml'
 
+require 'lokalise_manager/utils/hash_utils'
+require 'lokalise_manager/utils/array_utils'
+
 require 'lokalise_manager/version'
 require 'lokalise_manager/error'
 require 'lokalise_manager/global_config'

data/lokalise_manager.gemspec

@@ -32,7 +32,10 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency 'rspec', '~> 3.6'
   spec.add_development_dependency 'rubocop', '~> 1.0'
   spec.add_development_dependency 'rubocop-performance', '~> 1.5'
-  spec.add_development_dependency 'rubocop-rspec', '~> 2.5.0'
+  spec.add_development_dependency 'rubocop-rspec', '~> 2.6'
   spec.add_development_dependency 'simplecov', '~> 0.16'
   spec.add_development_dependency 'vcr', '~> 6.0'
+  spec.metadata = {
+    'rubygems_mfa_required' => 'true'
+  }
 end

data/spec/lib/lokalise_manager/global_config_spec.rb

@@ -14,6 +14,16 @@ describe LokaliseManager::GlobalConfig do
     fake_class.project_id = '123.abc'
   end
 
+  it 'is possible to set raise_on_export_fail' do
+    allow(fake_class).to receive(:raise_on_export_fail=).with(false)
+    fake_class.raise_on_export_fail = false
+  end
+
+  it 'is possible to set silent_mode' do
+    allow(fake_class).to receive(:silent_mode=).with(true)
+    fake_class.silent_mode = true
+  end
+
   it 'is possible to set use_oauth2_token' do
     allow(fake_class).to receive(:use_oauth2_token=).with(true)
     fake_class.use_oauth2_token = true

data/spec/lib/lokalise_manager/task_definitions/base_spec.rb

@@ -24,7 +24,7 @@ describe LokaliseManager::TaskDefinitions::Base do
   specify '.reset_client!' do
     expect(described_object.api_client).to be_an_instance_of(Lokalise::Client)
     described_object.reset_api_client!
-    current_client = described_object.instance_variable_get '@api_client'
+    current_client = described_object.instance_variable_get :@api_client
     expect(current_client).to be_nil
   end
 

data/spec/lib/lokalise_manager/task_definitions/exporter_spec.rb

@@ -24,26 +24,52 @@ describe LokaliseManager::TaskDefinitions::Exporter do
 
     describe '.export!' do
       it 'sends a proper API request and handles rate limiting' do
-        process = VCR.use_cassette('upload_files_multiple') do
-          described_object.export!
-        end.first
+        process = nil
+
+        VCR.use_cassette('upload_files_multiple') do
+          expect(-> { process = described_object.export!.first.process }).to output(/complete!/).to_stdout
+        end
 
         expect(process.project_id).to eq(project_id)
         expect(process.status).to eq('queued')
       end
 
       it 'handles too many requests' do
+        allow(described_object.config).to receive(:max_retries_export).and_return(1)
+        allow(described_object).to receive(:sleep).and_return(0)
+
+        fake_client = instance_double('Lokalise::Client')
+        allow(fake_client).to receive(:token).with(any_args).and_return('fake_token')
+        allow(fake_client).to receive(:upload_file).with(any_args).and_raise(Lokalise::Error::TooManyRequests)
+        allow(described_object).to receive(:api_client).and_return(fake_client)
+
+        expect(-> { described_object.export! }).to raise_error(Lokalise::Error::TooManyRequests, /Gave up after 1 retries/i)
+
+        expect(described_object).to have_received(:sleep).exactly(6).times
+        expect(described_object).to have_received(:api_client).at_least(12).times
+        expect(fake_client).to have_received(:upload_file).exactly(12).times
+      end
+
+      it 'handles too many requests but does not re-raise anything when raise_on_export_fail is false' do
+        allow(described_object.config).to receive(:max_retries_export).and_return(1)
+        allow(described_object.config).to receive(:raise_on_export_fail).and_return(false)
         allow(described_object).to receive(:sleep).and_return(0)
 
         fake_client = instance_double('Lokalise::Client')
+        allow(fake_client).to receive(:token).with(any_args).and_return('fake_token')
         allow(fake_client).to receive(:upload_file).with(any_args).and_raise(Lokalise::Error::TooManyRequests)
         allow(described_object).to receive(:api_client).and_return(fake_client)
+        processes = []
+        expect(-> { processes = described_object.export! }).not_to raise_error
 
-        expect(-> { described_object.export! }).to raise_error(Lokalise::Error::TooManyRequests, /Gave up after 2 retries/i)
+        expect(processes[0].path.to_s).to include('en_0')
+        expect(processes[0].success).to be false
+        expect(processes[1].error.class).to eq(Lokalise::Error::TooManyRequests)
+        expect(processes.count).to eq(7)
 
-        expect(described_object).to have_received(:sleep).exactly(2).times
-        expect(described_object).to have_received(:api_client).exactly(3).times
-        expect(fake_client).to have_received(:upload_file).exactly(3).times
+        expect(described_object).to have_received(:sleep).exactly(7).times
+        expect(described_object).to have_received(:api_client).at_least(14).times
+        expect(fake_client).to have_received(:upload_file).exactly(14).times
       end
     end
   end
@@ -58,10 +84,23 @@ describe LokaliseManager::TaskDefinitions::Exporter do
     end
 
     describe '.export!' do
+      it 'sends a proper API request but does not output anything when silent_mode is enabled' do
+        allow(described_object.config).to receive(:silent_mode).and_return(true)
+
+        process = nil
+
+        VCR.use_cassette('upload_files') do
+          expect(-> { process = described_object.export!.first.process }).not_to output(/complete!/).to_stdout
+        end
+
+        expect(process.status).to eq('queued')
+        expect(described_object.config).to have_received(:silent_mode).at_most(1).times
+      end
+
       it 'sends a proper API request' do
         process = VCR.use_cassette('upload_files') do
           described_object.export!
-        end.first
+        end.first.process
 
         expect(process.project_id).to eq(project_id)
         expect(process.status).to eq('queued')
@@ -70,11 +109,16 @@ describe LokaliseManager::TaskDefinitions::Exporter do
       it 'sends a proper API request when a different branch is provided' do
         allow(described_object.config).to receive(:branch).and_return('develop')
 
-        process = VCR.use_cassette('upload_files_branch') do
+        process_data = VCR.use_cassette('upload_files_branch') do
           described_object.export!
         end.first
 
         expect(described_object.config).to have_received(:branch).at_most(2).times
+        expect(process_data.success).to be true
+        expect(process_data.path.to_s).to include('en.yml')
+
+        process = process_data.process
+        expect(process).to be_an_instance_of(Lokalise::Resources::QueuedProcess)
         expect(process.project_id).to eq(project_id)
         expect(process.status).to eq('queued')
       end
@@ -93,9 +137,9 @@ describe LokaliseManager::TaskDefinitions::Exporter do
       end
     end
 
-    describe '.each_file' do
+    describe '#all_files' do
       it 'yield proper arguments' do
-        expect { |b| described_object.send(:each_file, &b) }.to yield_with_args(
+        expect(described_object.send(:all_files).first).to include(
           Pathname.new(path),
           Pathname.new(relative_name)
         )
@@ -165,30 +209,29 @@ describe LokaliseManager::TaskDefinitions::Exporter do
       end
     end
 
-    describe '.each_file' do
-      it 'yields every translation file' do
-        expect { |b| described_object.send(:each_file, &b) }.to yield_successive_args(
-          [
-            Pathname.new(path),
-            Pathname.new(relative_name)
-          ],
-          [
-            Pathname.new(path_ru),
-            Pathname.new(filename_ru)
-          ]
+    describe '#each_file' do
+      it 'returns all files' do
+        files = described_object.send(:all_files)
+        expect(files[0]).to include(
+          Pathname.new(path),
+          Pathname.new(relative_name)
+        )
+        expect(files[1]).to include(
+          Pathname.new(path_ru),
+          Pathname.new(filename_ru)
         )
       end
 
-      it 'does not yield files that have to be skipped' do
+      it 'does not return files that have to be skipped' do
         allow(described_object.config).to receive(:skip_file_export).twice.and_return(
           ->(f) { f.split[1].to_s.include?('ru') }
         )
-        expect { |b| described_object.send(:each_file, &b) }.to yield_successive_args(
-          [
-            Pathname.new(path),
-            Pathname.new(relative_name)
-          ]
+        files = described_object.send(:all_files)
+        expect(files[0]).to include(
+          Pathname.new(path),
+          Pathname.new(relative_name)
         )
+        expect(files.count).to eq(1)
 
         expect(described_object.config).to have_received(:skip_file_export).twice
       end

data/spec/lib/lokalise_manager/task_definitions/importer_spec.rb

@@ -9,12 +9,16 @@ describe LokaliseManager::TaskDefinitions::Importer do
   let(:loc_path) { described_object.config.locales_path }
   let(:project_id) { ENV['LOKALISE_PROJECT_ID'] }
   let(:local_trans) { "#{Dir.getwd}/spec/fixtures/trans.zip" }
-  let(:faulty_trans) { "#{Dir.getwd}/spec/fixtures/faulty_trans.zip" }
 
-  describe '.open_and_process_zip' do
+  describe '#open_and_process_zip' do
     it 're-raises errors during file processing' do
-      expect(-> { described_object.send(:open_and_process_zip, faulty_trans) }).
-        to raise_error(Psych::DisallowedClass, /Error when trying to process fail\.yml/)
+      entry = double
+      allow(entry).to receive(:name).and_return('fail.yml')
+      allow(described_object).to receive(:data_from).with(entry).and_raise(EncodingError)
+      expect(-> { described_object.send(:process!, entry) }).
+        to raise_error(EncodingError, /Error when trying to process fail\.yml/)
+
+      expect(described_object).to have_received(:data_from)
     end
 
     it 're-raises errors during file opening' do
@@ -23,7 +27,7 @@ describe LokaliseManager::TaskDefinitions::Importer do
     end
   end
 
-  describe '.download_files' do
+  describe '#download_files' do
     it 'returns a proper download URL' do
       response = VCR.use_cassette('download_files') do
         described_object.send :download_files
@@ -103,15 +107,31 @@ describe LokaliseManager::TaskDefinitions::Importer do
       end
 
       it 'runs import successfully' do
-        result = VCR.use_cassette('import') do
-          described_object.import!
+        result = nil
+
+        VCR.use_cassette('import') do
+          expect(-> { result = described_object.import! }).to output(/complete!/).to_stdout
         end
 
         expect(result).to be true
 
-        expect(count_translations).to eq(8)
-        expect_file_exist loc_path, 'en.yml'
-        expect_file_exist loc_path, 'ru.yml'
+        expect(count_translations).to eq(24)
+        expect_file_exist loc_path, 'en_1.yml'
+        expect_file_exist loc_path, 'ru_2.yml'
+      end
+
+      it 'runs import successfully but does not provide any output when silent_mode is enabled' do
+        allow(described_object.config).to receive(:silent_mode).and_return(true)
+        result = nil
+
+        VCR.use_cassette('import') do
+          expect(-> { result = described_object.import! }).not_to output(/complete!/).to_stdout
+        end
+
+        expect(result).to be true
+        expect_file_exist loc_path, 'en_1.yml'
+        expect_file_exist loc_path, 'ru_2.yml'
+        expect(described_object.config).to have_received(:silent_mode).at_most(1).times
       end
     end
 
@@ -157,9 +177,21 @@ describe LokaliseManager::TaskDefinitions::Importer do
       it 'import halts when a user chooses not to proceed' do
         allow(safe_mode_obj).to receive(:download_files).at_most(0).times
         allow($stdin).to receive(:gets).and_return('N')
-        expect(-> { safe_mode_obj.import! }).to output(/is not empty/).to_stdout
+        expect(-> { safe_mode_obj.import! }).to output(/cancelled/).to_stdout
+
+        expect(safe_mode_obj).not_to have_received(:download_files)
+        expect($stdin).to have_received(:gets)
+        expect(count_translations).to eq(1)
+      end
+
+      it 'import halts when a user chooses not to proceed and debug info is not printed out when silent_mode is enabled' do
+        allow(safe_mode_obj.config).to receive(:silent_mode).and_return(true)
+        allow(safe_mode_obj).to receive(:download_files).at_most(0).times
+        allow($stdin).to receive(:gets).and_return('N')
+        expect(-> { safe_mode_obj.import! }).not_to output(/cancelled/).to_stdout
 
         expect(safe_mode_obj).not_to have_received(:download_files)
+        expect(safe_mode_obj.config).to have_received(:silent_mode)
         expect($stdin).to have_received(:gets)
         expect(count_translations).to eq(1)
       end

data/spec/lib/utils/array_utils_spec.rb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+describe LokaliseManager::Utils::ArrayUtils do
+  using described_class
+  let(:arr) { (1..8).to_a }
+
+  describe '#in_groups_of' do
+    it 'raises an exception when the number is less than 1' do
+      expect(-> { arr.in_groups_of(-1) }).to raise_error(ArgumentError)
+    end
+
+    it 'uses collection itself if fill_with is false' do
+      enum = arr.in_groups_of(5, false)
+      enum.next
+      expect(enum.next.count).to eq(3)
+    end
+  end
+end

data/spec/lib/utils/hash_utils_spec.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+describe LokaliseManager::Utils::HashUtils do
+  using described_class
+  let(:h1) { {a: 100, b: 200, c: {c1: 100}} }
+  let(:h2) { {b: 250, c: {c1: 200}} }
+
+  specify '#deep_merge' do
+    result = h1.deep_merge(h2) { |_key, this_val, other_val| this_val + other_val }
+    expect(result[:b]).to eq(450)
+    expect(result[:c][:c1]).to eq(300)
+  end
+end

data/spec/support/vcr.rb

@@ -6,6 +6,8 @@ VCR.configure do |c|
   c.ignore_hosts 'codeclimate.com'
   c.hook_into :faraday
   c.cassette_library_dir = File.join(File.dirname(__FILE__), '..', 'fixtures', 'vcr_cassettes')
-  c.filter_sensitive_data('<LOKALISE_TOKEN>') { ENV.fetch('LOKALISE_API_TOKEN') }
+  c.filter_sensitive_data('<LOKALISE_TOKEN>') do |_i|
+    ENV.fetch('LOKALISE_API_TOKEN')
+  end
   c.configure_rspec_metadata!
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: lokalise_manager
 version: !ruby/object:Gem::Version
-  version: 1.1.0
+  version: 2.1.0
 platform: ruby
 authors:
 - Ilya Bodrov
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-10-25 00:00:00.000000000 Z
+date: 2022-01-27 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: ruby-lokalise-api
@@ -128,14 +128,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 2.5.0
+        version: '2.6'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 2.5.0
+        version: '2.6'
 - !ruby/object:Gem::Dependency
   name: simplecov
   requirement: !ruby/object:Gem::Requirement
@@ -187,6 +187,8 @@ files:
 - lib/lokalise_manager/task_definitions/base.rb
 - lib/lokalise_manager/task_definitions/exporter.rb
 - lib/lokalise_manager/task_definitions/importer.rb
+- lib/lokalise_manager/utils/array_utils.rb
+- lib/lokalise_manager/utils/hash_utils.rb
 - lib/lokalise_manager/version.rb
 - lokalise_manager.gemspec
 - spec/lib/lokalise_manager/global_config_spec.rb
@@ -194,6 +196,8 @@ files:
 - spec/lib/lokalise_manager/task_definitions/exporter_spec.rb
 - spec/lib/lokalise_manager/task_definitions/importer_spec.rb
 - spec/lib/lokalise_manager_spec.rb
+- spec/lib/utils/array_utils_spec.rb
+- spec/lib/utils/hash_utils_spec.rb
 - spec/spec_helper.rb
 - spec/support/file_manager.rb
 - spec/support/spec_addons.rb
@@ -201,7 +205,8 @@ files:
 homepage: https://github.com/bodrovis/lokalise_manager
 licenses:
 - MIT
-metadata: {}
+metadata:
+  rubygems_mfa_required: 'true'
 post_install_message:
 rdoc_options: []
 require_paths:
@@ -217,7 +222,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.26
+rubygems_version: 3.3.5
 signing_key:
 specification_version: 4
 summary: Lokalise integration for Ruby
@@ -227,6 +232,8 @@ test_files:
 - spec/lib/lokalise_manager/task_definitions/exporter_spec.rb
 - spec/lib/lokalise_manager/task_definitions/importer_spec.rb
 - spec/lib/lokalise_manager_spec.rb
+- spec/lib/utils/array_utils_spec.rb
+- spec/lib/utils/hash_utils_spec.rb
 - spec/spec_helper.rb
 - spec/support/file_manager.rb
 - spec/support/spec_addons.rb