poesie 1.3.0 → 1.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 8f8bff2658b4d9825300ecb79bd1836dec4aa4cb
-  data.tar.gz: c8a17bc54ad8bcc73bf98c5edaede5aafcba2a4c
+  metadata.gz: 3faa652a9f6ad87e650ba88a394d2f5d577e66b3
+  data.tar.gz: 40ede2cb29bea6fd4abc16d0b27bb13daf00a308
 SHA512:
-  metadata.gz: 31faa35c132decdc0bcc018eca372d767dbe951b5b3ad09c03180f6cb41a3c1fc0fc1d1d6960e39dc1adb716f746a2f51690f77663b174072ff1aeb8b83e0e1b
-  data.tar.gz: 4d54dbbbc8abbdd960fd4e3af1074d862469e4d0e57cc10788b82882190e8c06e1baff590c88e286f10877b1506b75485484e98bceffaeeb554164d0ec485d78
+  metadata.gz: 3d20a015a7232e8973d5a86ff956c5bb35fa1295889db5d3d70af44eee44077aa7d0b5dd9ec238a5a44b14457964baf4db7daeca4cef56a0531e2417b6389548
+  data.tar.gz: dc1cfc402b131d401c0f0c0384ec5e06246f7882ce9d87002bc41cf59c19bd705d9a2fd1c7bae5f3d18a3d270e8fe50710106fcc58f20f820a8fff0e30dd79db

data/README.md

@@ -29,11 +29,19 @@ Even though POEditor's web interface allows you to export the strings in those f
 
 ## Installation
 
+The easiest solution to install the latest version of `poesie` is to simply run:
+
+```sh
+gem install poesie
+```
+
+Alternatively, the following solutions will allow you to install the latest version from master, so that you can use the newest features even if they haven't been released to RubyGems yet:
+
 <details>
-<summary>Solution 1: Clone the repo</summary>
+<summary>Alternate Solution 1: Clone the repo and use it directly</summary>
 
 * `git clone` the project on you computer
-* Install `bundler` using `gem install bundler` if you don't have it already
+* If you don't have it already, install `bundler` using `gem install bundler`
 * Install `poesie`'s dependencies by running `bundle install` from the directory where you cloned the repository
 * Invoke the tool using its full path `<path/where/you/cloned/Poesie>/bin/poesie`.
 
@@ -42,15 +50,16 @@ You could also add the `<path/where/you/cloned/Poesie>/bin/` path to your `PATH`
 </details>
 
 <details>
-<summary>Solution 2: Build and install it as a gem</summary>
+<summary>Alternate Solution 2: Build and install the gem yourself</summary>
 
 * `git clone` the project on you computer
 * Run `gem build poesie.gemspec` to build the gem
-* Run `gem install poesie-*.gem` to install the gem you just built (`*` will be the version of the gem)
+* Run `gem install poesie-*.gem` to install the gem you just built (where `*` is the version of the gem)
 * Now that it's installed in your system, you can invoke the tool using `poesie` from anywhere
 
-</details>
+_This solution has the drawback of being potentially easily confused between versions of `poesie` that you installed yourself vs. official versions though._
 
+</details>
 
 ## Using POEditor properly
 
@@ -112,7 +121,7 @@ Usage: poesie [options]
     -a, --android PATH               Path of the Android strings.xml file path to generate
     -c, --context PATH               Path of the *.json file to generate for contexts
     -d, --date                       Generate the current date in file headers
-    -r, --replacements FILE          Path to a YAML file listing all replacements
+    -s, --subst FILE                 Path to a YAML file listing all text substitutions
     -h, --help                       Show this message
     -v, --version                    Show version
 ```
@@ -147,12 +156,12 @@ This can be useful:
 * Either to use that JSON file directly in your project to do whatever you want with the contexts (e.g. parsing the JSON file at runtime using `JSONSerialization`, and use it as you please)
 * Or use that JSON file with a template engine (like [Liquid](https://github.com/Shopify/liquid)) to generate code specific to your needs. See the example script in `examples/gen-context-with-liquid.rb`.
 
-## Providing text replacements
+## Providing text substitutions
 
-In case you need replacements to be applied to your translations, you can use the `--replacements` flag (`-r` for short) to provide a YAML file listing all replacements to be applied.
+In case you need substitutions to be applied to your translations, you can use the `--subst` flag (`-s` for short) to provide a YAML file listing all substitutions to be applied.
 
 ```
-$ poesie --token "..." --project "..." --lang fr --ios .../localizable.strings -r replacements.yaml
+$ poesie --token "..." --project "..." --lang fr --ios .../localizable.strings -s substitutions.yaml
 ```
 
 This can be useful:
@@ -163,16 +172,51 @@ This can be useful:
 * To ensure proper capitalization and orthography of your brand name every time it's spelled in the translations
 * etc.
 
-The YAML file provided must be of the form of a single Hash of String pairs, for example:
+The YAML file provided must be of the form of a single Hash of String pairs, or an Array listing Hashes of String pairs.
+
+* Substitutions will be performed in the given order if listed in an Array.
+* Order of substitutions isn't guaranteed if they are listed in a Hash. That's why it's sometimes preferable to use an Array of key/value pairs rather than a Hash (because an Array is ordered)
+* If a key is surrounded with slashes, it will be interpreted as a regular expression.
+
+**Example:**
 
 ```yaml
-" :": "\u00A0:"
-" ;": "\u00A0;"
-" !": "\u00A0!"
-" ?": "\u00A0?"
-"...": "…"
+- " :": "\u00A0:"
+  " ;": "\u00A0;"
+  " !": "\u00A0!"
+  " ?": "\u00A0?"
+- "...": "…"
+- /^\s+/: ""
+  /\s+$/: ""
 ```
 
-_Note: given that the JSON format is a subset of the YAML format, using a JSON file is also possible, as long as it still represents a dictionary with Strings as keys and as values._
+In this example:
+
+* The first substitutions to be applied will be the 4 first ones about the `:;!?` punctuation. The order of the substitutions between those 4 is undetermined (as Hashes are unordered).
+* Then the substitution of `...` to `…` will be applied
+* The last two substitutions are interpreted as Regular Expressions, trimming the beginning and end of each text. They will be applied last (but those two will be applied in any order w/r/t each other).
+
+<details>
+<summary>_Note: given that the JSON format is a subset of the YAML format, using a JSON file to represent the same substitutions is also possible._</summary>
+
+```json
+[
+  {
+    " :": " :",
+    " ;": " ;",
+    " !": " !",
+    " ?": " ?"
+  },
+  {
+    "...": "…"
+  },
+  {
+    "/^\\s+/": "",
+    "/\\s+$/": ""
+  }
+]
+```
+
+</details>
 
-You can find an example of a YAML file in `examples/replacements.yaml` in this repository.
+You can find an example of a YAML file in `examples/substitutions.yaml` in this repository.

data/bin/poesie

@@ -11,7 +11,8 @@ end
 begin
   require 'optparse'
   require 'yaml'
-  require 'poesie'
+  require File.expand_path('../lib/poesie', File.dirname(__FILE__))
+  require File.expand_path('../lib/version', File.dirname(__FILE__))
 rescue LoadError => e
   abort 'error: missing some dependencies for poesie. Be sure to install it using `gem install` or to run `bundle installl` to install dependencies - ' + e.message
 end
@@ -24,7 +25,7 @@ options = {
     :strings_xml => nil,
     :context_file => nil,
     :print_date => false,
-    :replacements => nil
+    :substitutions => nil
 }
 
 opts = OptionParser.new do |opts|
@@ -50,16 +51,17 @@ opts = OptionParser.new do |opts|
   opts.on('-d', '--date', %q(Generate the current date in file headers)) do
     options[:print_date] = true
   end
-  opts.on('-r FILE', '--replacements FILE', %q(Path to a YAML file listing all replacements)) do |path|
-    Poesie.exit_with_error("The replacements file #{path} was not found") unless File.exists?(path)  
+  opts.on('-s FILE', '--subst FILE', %q(Path to a YAML file listing all substitutions)) do |path|
+    Poesie.exit_with_error("The substitutions file #{path} was not found") unless File.exists?(path)  
     begin
-      dict = YAML.load_file(path)
+      subst = YAML.load_file(path)
     rescue Psych::SyntaxError => e
       Poesie.exit_with_error("Invalid YAML file #{e.message}")  
     end
-    valid = dict.keys.all? { |k| k.is_a?(String) } && dict.values.all? { |k| k.is_a?(String) }
-    Poesie.exit_with_error("The provided replacements file #{path} should only contain a Dictionary with String keys and values") unless valid
-    options[:replacements] = dict
+    subst = [subst] if subst.is_a?(Hash)
+    valid = subst.all? { |hash| hash.keys.all? { |k| k.is_a?(String) } && hash.values.all? { |k| k.is_a?(String) } }
+    Poesie.exit_with_error("The provided substitutions file #{path} should only contain a (Array of) Hashes with String keys and values") unless valid
+    options[:substitutions] = subst
   end
   opts.on_tail('-h', '--help', %q(Show this message)) { puts opts; exit 1 }
   opts.on_tail('-v', '--version', 'Show version') { puts Poesie::VERSION; exit }
@@ -81,14 +83,14 @@ Poesie.exit_with_error('You need to specify your POEditor project language using
 
 # iOS
 unless options[:localizable_strings].nil?
-  Log::title('== iOS ==')
+  Poesie::Log::title('== iOS ==')
   exporter.run(options[:lang]) do |terms|
     # Localizable.strings
-    Log::subtitle("== Language #{options[:lang]} ==")
+    Poesie::Log::subtitle("== Language #{options[:lang]} ==")
     Poesie::AppleFormatter::write_strings_file(
       terms,
       options[:localizable_strings],
-      replacements: options[:replacements],
+      substitutions: options[:substitutions],
       print_date: options[:print_date]
     )
 
@@ -97,13 +99,13 @@ unless options[:localizable_strings].nil?
     Poesie::AppleFormatter::write_stringsdict_file(
       terms,
       strings_dict_path,
-      replacements: options[:replacements],
+      substitutions: options[:substitutions],
       print_date: options[:print_date]
     )
 
     # Context.json (if specified)
     if options[:context_file]
-      Log::subtitle("== Context #{options[:context_file]} ==")
+      Poesie::Log::subtitle("== Context #{options[:context_file]} ==")
       Poesie::AppleFormatter::write_context_json(terms, options[:context_file])
     end
   end
@@ -111,14 +113,14 @@ end
 
 # Android
 unless options[:strings_xml].nil?
-  Log::title('== Android ==')
+  Poesie::Log::title('== Android ==')
   exporter.run(options[:lang]) do |terms|
     # Strings.xml
-    Log::subtitle("== Language #{options[:lang]} ==")
+    Poesie::Log::subtitle("== Language #{options[:lang]} ==")
     Poesie::AndroidFormatter::write_strings_xml(
       terms,
       options[:strings_xml],
-      replacements: options[:replacements],
+      substitutions: options[:substitutions],
       print_date: options[:print_date]
     )
   end

data/lib/android_formatter.rb

@@ -9,15 +9,16 @@ module Poesie
     #        The json parsed terms exported by POEditor and sorted alphabetically
     # @param [String] file
     #        The path to the file to write the content to
-    # @param [Hash<String,String>] replacements
-    #        The list of replacements to apply to the translations
+    # @param [Hash<String,String>] substitutions
+    #        The list of substitutions to apply to the translations
     # @param [Bool] print_date
     #        Should we print the date in the header of the generated file
     #
-    def self.write_strings_xml(terms, file, replacements: nil, print_date: false)
+    def self.write_strings_xml(terms, file, substitutions: nil, print_date: false)
+      stats = { :ios => 0, :nil => [], :count => 0 }
+
       Log::info(" - Save to file: #{file}")
-      fh = File.open(file, "w")
-      begin
+      File.open(file, "w") do |fh|
         xml_builder = Builder::XmlMarkup.new(:target => fh, :indent => 4)
         xml_builder.instruct!
         xml_builder.comment!("Exported from POEditor   ")
@@ -26,25 +27,32 @@ module Poesie
         xml_builder.resources do |resources_node|
           terms.each do |term|
             (term, definition, plurals, comment, context) = ['term', 'definition', 'term_plural', 'comment', 'context'].map { |k| term[k] }
-            # Skip ugly cases if POEditor is buggy for some entries
-            next if term.nil? || term.empty? || definition.nil?
-            next if term =~ /_ios$/
+            
+            # Filter terms and update stats
+            next if (term.nil? || term.empty? || definition.nil?) && stats[:nil] << term
+            next if (term =~ /_ios$/) && stats[:ios] += 1
+            stats[:count] += 1
+
             xml_builder.comment!(context) unless context.empty?
             if plurals.empty?
-              definition = Poesie::process(definition, replacements).gsub('"', '\\"')
+              definition = Poesie::process(definition, substitutions).gsub('"', '\\"')
               resources_node.string("\"#{definition}\"", :name => term)
             else
               resources_node.plurals(:name => plurals) do |plurals_node|
                 definition.each do |plural_quantity, plural_value|
-                  plural_value = Poesie::process(plural_value, replacements).gsub('"', '\\"')
+                  plural_value = Poesie::process(plural_value, substitutions).gsub('"', '\\"')
                   plurals_node.item("\"#{plural_value}\"", :quantity => plural_quantity)
                 end
               end
             end
           end
         end
-      ensure
-        fh.close
+      end
+
+      Log::info("   [Stats] #{stats[:count]} strings processed (Filtered out #{stats[:ios]} iOS strings)")
+      unless stats[:nil].empty?
+        Log::error("   Found #{stats[:nil].count} empty value(s) for the following term(s):")
+        stats[:nil].each { |key| Log::error("    - #{key.inspect}") }
       end
     end
   end

data/lib/apple_formatter.rb

@@ -9,23 +9,23 @@ module Poesie
     #        JSON returned by the POEditor API
     # @param [String] file
     #        The path of the file to write
-    # @param [Hash<String,String>] replacements
-    #        The list of replacements to apply to the translations
+    # @param [Hash<String,String>] substitutions
+    #        The list of substitutions to apply to the translations
     # @param [Bool] print_date
     #        Should we print the date in the header of the generated file
     #
-    def self.write_strings_file(terms, file, replacements: nil, print_date: false)
+    def self.write_strings_file(terms, file, substitutions: nil, print_date: false)
       out_lines = ['/'+'*'*79, ' * Exported from POEditor - https://poeditor.com']
       out_lines << " * #{Time.now}" if print_date
       out_lines += [' '+'*'*79+'/', '']
       last_prefix = ''
-      stats = { :android => 0, :nil => 0, :count => 0 }
+      stats = { :android => 0, :nil => [], :count => 0 }
 
       terms.each do |term|
         (term, definition, comment, context) = ['term', 'definition', 'comment', 'context'].map { |k| term[k] }
 
         # Filter terms and update stats
-        next if (term.nil? || term.empty? || definition.nil? || definition.empty?) && stats[:nil] += 1
+        next if (term.nil? || term.empty? || definition.nil? || definition.empty?) && stats[:nil] << term
         next if (term =~ /_android$/) && stats[:android] += 1 # Remove android-specific strings
         stats[:count] += 1
 
@@ -42,7 +42,7 @@ module Poesie
           definition = definition["one"]
         end
 
-        definition = Poesie::process(definition, replacements)
+        definition = Poesie::process(definition, substitutions)
                     .gsub("\u2028", '') # Sometimes inserted by the POEditor exporter
                     .gsub("\n", '\n') # Replace actual CRLF with '\n'
                     .gsub('"', '\\"') # Escape quotes
@@ -53,12 +53,15 @@ module Poesie
 
       content = out_lines.join("\n") + "\n"
 
-
-      Log::info("   [Stats] #{stats[:count]} strings processed (Filtered out #{stats[:android]} android strings, #{stats[:nil]} nil entries)")
       Log::info(" - Save to file: #{file}")
       File.open(file, "w") do |fh|
         fh.write(content)
       end
+      Log::info("   [Stats] #{stats[:count]} strings processed (Filtered out #{stats[:android]} android strings)")
+      unless stats[:nil].empty?
+        Log::error("   Found #{stats[:nil].count} empty value(s) for the following term(s):")
+        stats[:nil].each { |key| Log::error("    - #{key.inspect}") }
+      end
     end
 
     # Write the Localizable.stringsdict output file
@@ -67,17 +70,16 @@ module Poesie
     #        JSON returned by the POEditor API
     # @param [String] file
     #        The path of the file to write
-    # @param [Hash<String,String>] replacements
-    #        The list of replacements to apply to the translations
+    # @param [Hash<String,String>] substitutions
+    #        The list of substitutions to apply to the translations
     # @param [Bool] print_date
     #        Should we print the date in the header of the generated file
     #
-    def self.write_stringsdict_file(terms, file, replacements: nil, print_date: false)
-      stats = { :android => 0, :nil => 0, :count => 0 }
+    def self.write_stringsdict_file(terms, file, substitutions: nil, print_date: false)
+      stats = { :android => 0, :nil => [], :count => 0 }
 
       Log::info(" - Save to file: #{file}")
-      fh = File.open(file, "w")
-      begin
+      File.open(file, "w") do |fh|
         xml_builder = Builder::XmlMarkup.new(:target => fh, :indent => 4)
         xml_builder.instruct!
         xml_builder.comment!("Exported from POEditor   ")
@@ -89,14 +91,13 @@ module Poesie
               (term, term_plural, definition) = ['term', 'term_plural', 'definition'].map { |k| term[k] }
 
               # Filter terms and update stats
-              next if (term.nil? || term.empty? || definition.nil?) && stats[:nil] += 1
+              next if (term.nil? || term.empty? || definition.nil?) && stats[:nil] << term
               next if (term =~ /_android$/) && stats[:android] += 1 # Remove android-specific strings
               next unless definition.is_a? Hash
               stats[:count] += 1
               
               key = term_plural || term
 
-              
               root_node.key(key)
               root_node.dict do |dict_node|
                 dict_node.key('NSStringLocalizedFormatKey')
@@ -109,7 +110,8 @@ module Poesie
                   format_node.string('d')
 
                   definition.each do |(quantity, text)|
-                    text = Poesie::process(text, replacements)
+                    text = Poesie::process(text, substitutions)
+                    text = Poesie::process(text, substitutions)
                               .gsub("\u2028", '') # Sometimes inserted by the POEditor exporter
                               .gsub('\n', "\n") # Replace '\n' with actual CRLF
                               .gsub(/%(\d+\$)?s/, '%\1@') # replace %s with %@ for iOS
@@ -121,8 +123,11 @@ module Poesie
             end
           end
         end
-      ensure
-        fh.close
+      end
+      Log::info("   [Stats] #{stats[:count]} strings processed (Filtered out #{stats[:android]} android strings)")
+      unless stats[:nil].empty?
+        Log::error("   Found #{stats[:nil].count} empty value(s) for the following term(s):")
+        stats[:nil].each { |key| Log::error("    - #{key.inspect}") }
       end
     end
 
@@ -162,11 +167,11 @@ module Poesie
 
       context_json = JSON.pretty_generate(json_hash)
 
-      Log::info("   [Stats] #{stats[:count]} contexts processed (Filtered out #{stats[:android]} android entries, #{stats[:nil]} nil contexts)")
       Log::info(" - Save to file: #{file}")
       File.open(file, "w") do |fh|
         fh.write(context_json)
       end
+      Log::info("   [Stats] #{stats[:count]} contexts processed (Filtered out #{stats[:android]} android entries, #{stats[:nil]} nil contexts)")
     end
 
   end

data/lib/exporter.rb

@@ -41,16 +41,15 @@ module Poesie
     # @return [String] URL of the exported file ready to be downloaded
     #
     def generate_export_uri(lang)
-      uri = URI('https://poeditor.com/api/')
-      res = Net::HTTP.post_form(uri, 'api_token' => @api_token, 'action' => 'export', 'id' => @project_id, 'type' => 'json', 'language' => lang)
+      uri = URI('https://api.poeditor.com/v2/projects/export')
+      res = Net::HTTP.post_form(uri, 'api_token' => @api_token, 'id' => @project_id, 'type' => 'json', 'language' => lang)
       json = JSON.parse(res.body)
-      if json['response']['status'] != 'success'
+      unless json['response']['status'] == 'success'
         r = json['response']
         puts "Error #{r['code']} (#{r['status']})\n#{r['message']}"
         exit 1
-      else
-        json['item']
       end
+      json['result']['url']
     end
   end
 end

data/lib/log.rb

@@ -0,0 +1,24 @@
+module Poesie
+  module Log
+    @@quiet = false
+    def self.quiet=(value)
+      @@quiet = value
+    end
+
+    def self.title(str) # bg yellow
+      puts "\e[44;37m#{str}\e[0m" unless @@quiet
+    end
+
+    def self.error(str)
+      puts "\e[1;31m! #{str}\e[0m" unless @@quiet
+    end
+
+    def self.info(str)
+      puts "\e[1;32m√ #{str}\e[0m" unless @@quiet
+    end
+
+    def self.subtitle(str)
+      puts "\e[1;33m√ #{str}\e[0m" unless @@quiet
+    end
+  end
+end

data/lib/poesie.rb

@@ -1,5 +1,4 @@
-require File.expand_path('utils/log', File.dirname(__FILE__))
-
+require File.expand_path('log', File.dirname(__FILE__))
 require File.expand_path('exporter', File.dirname(__FILE__))
 require File.expand_path('android_formatter', File.dirname(__FILE__))
 require File.expand_path('apple_formatter', File.dirname(__FILE__))
@@ -12,18 +11,24 @@ module Poesie
     exit 1
   end
 
-  # Apply the list of text replacements to the given string
+  # Apply the list of text substitutions to the given string
   #
   # @param [String] text
   #        The text to process
-  # @param [Hash<String,String>] replacements
-  #        The replacements to apply
+  # @param [Hash<String,String>] substitutions
+  #        The substitutions to apply
   #
-  def self.process(text, replacements)
-    return text if replacements.nil?
+  def self.process(text, substitutions)
+    return text if substitutions.nil?
     replaced = text.dup
-    replacements.each do |k,v|
-      replaced.gsub!(k, v)
+    list = substitutions
+    list = [substitutions] if substitutions.is_a?(Hash)
+    list.each do |hash|
+      hash.each do |k,v|
+        # If the key is surrounding by slashes, interpret as a RegExp
+        k = Regexp.new($1) if k =~ %r(^/(.*)/$)
+        replaced.gsub!(k, v)
+      end
     end
     replaced
   end

data/lib/version.rb

@@ -1,3 +1,3 @@
 module Poesie
-  VERSION = '1.3.0'
+  VERSION = '1.4.0'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: poesie
 version: !ruby/object:Gem::Version
-  version: 1.3.0
+  version: 1.4.0
 platform: ruby
 authors:
 - Olivier Halligon
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-02-07 00:00:00.000000000 Z
+date: 2017-02-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: net
@@ -67,8 +67,8 @@ files:
 - lib/android_formatter.rb
 - lib/apple_formatter.rb
 - lib/exporter.rb
+- lib/log.rb
 - lib/poesie.rb
-- lib/utils/log.rb
 - lib/version.rb
 homepage: https://github.com/NijiDigital/poesie
 licenses:
@@ -90,9 +90,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.4.7
+rubygems_version: 2.6.7
 signing_key: 
 specification_version: 4
 summary: Automate and post-process the export of POEditor strings
 test_files: []
-has_rdoc: 

data/lib/utils/log.rb

@@ -1,23 +0,0 @@
-module Log
-  @@quiet = false
-  def self.quiet=(value)
-    @@quiet = value
-  end
-
-  def self.title(str) # bg yellow
-    puts "\e[44;37m#{str}\e[0m" unless @@quiet
-  end
-
-  def self.error(str)
-    puts "\e[1;31m! #{str}\e[0m" unless @@quiet
-  end
-
-  def self.info(str)
-    puts "\e[1;32m√ #{str}\e[0m" unless @@quiet
-  end
-
-  def self.subtitle(str)
-    puts "\e[1;33m√ #{str}\e[0m" unless @@quiet
-  end
-
-end