i18n-translators-tools 0.1

data/README.md

@@ -0,0 +1,229 @@
+I18n translation and locales management utility
+===============================================
+
+This package brings you useful utility which can help you to handle locale files
+and translations in your Ruby projects.
+
+
+Interesting features
+--------------------
+
+* no database required
+* merging and changes propagation (adding, removing and changed default text)
+  keeping default file untouched
+* creating new locale file based on another (usually default) file
+* converting from one format to another (yml <=> rb <=> po)
+* statistics
+* built-in simple console translator
+* support for locales split into sub-directories like:
+  * locale
+    * controller
+      * main
+        * default.yml
+        * en_US.yml
+        * en_GB.yml
+        * cs_CZ.yml
+      * auth
+        * default.yml
+        * en_US.yml
+        * en_GB.yml
+        * cs_CZ.yml
+    * model
+      * user
+        * default.yml
+        * en_US.yml
+        * en_GB.yml
+        * cs_CZ.yml
+    * rules
+      * en_US.rb
+      * en_GB.yml
+      * cs_CZ.rb
+* adds extra translation metadata right to the locale files (translators whose
+  translate with text editors don't have to tackle with diffs)
+* can strip all extra metadata and revert back to the
+  I18n::Backend::Simple format
+* in configuration files
+  (~/.config/ruby/i18n-translate; locale/.i18n-translate) you can put your
+  common or project related configurations (e.g: exclude => ['rules'])
+  i18n-translate utility reads firstly config in your home 
+  (~/.config/ruby/i18n-translate) then merge it with the one in your project
+  locale directory (e.g: locale/.i18n-translate) and then change it
+  using command line arguments
+
+
+WARNING
+-------
+
+* **i18n-translate can NOT handle lambdas and procedures.** The solution is to
+  put all your rules to separate file and use exclude (repeatable) argument
+  or create configuration file for your project in locales directory including
+  exclude array.
+* **po files are supported only partialy.** If you convert from yaml or ruby to
+  po and back you don't have to care even if you are using pluralization.
+  If you are converting from po origin files then you can lose header of the
+  file, pluralization, extracted comments, some flags (fuzzy will stay) and
+  previous-context. Strings over multiple lines are supported, however.
+* **po files are not compatible with I18n::Gettext.** The main purpose of
+  enabling conversions to po files is effort to allow usage of many po editors
+  for ruby projects. It is supposed that you will keep your files in yml (rb)
+  and conversions will be used only for translators and then you convert it
+  back. Then you lose nothing.
+
+
+How to add support into your application
+----------------------------------------
+
+i18n-translate brings additional metadata to locales. Therefore you have to
+include new backend into I18n. This new backend works as a transparent proxy
+and can work with both simple and enhanced format. If you are already using
+I18n then adding this backend will be probably the only change you have to do.
+I highly recommend to use this extension together with Fallback backend
+and all default values (usually in english) put into special file like
+'default.yml'.
+
+This will enable to have locales like en_US and en_GB where
+can native speakers put their polished english. I think it is good thing 
+to put all default strings into separate file(s). If you need to change
+some text you don't have to touch source code.
+
+Default files have to be in the I18n simple format. The enhanced format
+is only for locales.
+
+So in your application you should do something like this:
+
+    require 'i18n-translate'
+
+    I18n::Backend::Simple.send(:include, I18n::Backend::Translate)
+    I18n::Backend::Simple.send(:include, I18n::Backend::Fallbacks)
+    I18n.default_locale = 'default'
+    I18n.load_path << Dir[ File.expand_path("../locale/*.yml", __FILE__) ]
+
+and then you can use
+
+    I18n.t('some.key', :count => 3, :var => 'interpolated string')
+
+as usual.
+
+
+Examples
+--------
+
+Suppose we have our locales in 'locale/' directory without sub-directories and
+default values are inside 'locale/default.yml' file. And we have two files
+with locales 'locale/de_DE.yml', 'locale/cs_CZ.yml' and 'locale/extra/cs_CZ.yml'
+
+**Merging new changes (additions, removes, field changes) to all files:**
+
+    $> i18n-translate merge
+    locale/cs_CZ.yml...merged
+    locale/de_DE.yml...merged
+
+**Converting all cs_CZ files to rb format:**
+
+    $> i18n-translate convert -f yml -t rb -l cs_CZ -r
+    locale/cs_CZ.yml...converted
+    locale/extra/cs_CZ.yml...converted
+
+**Show some statistics:**
+
+    $> i18n-translate stat
+    locale/cs_CZ.yml...65% (650/1000)
+    locale/de_DE.yml...90% (900/1000)
+
+**Translate more entries (built-in translator invocation):**
+
+    $> i18n-translate translate -l cs_CZ
+
+For more help run i18n-translate without parameters.
+
+There also exists example web application. It is simple web standalone
+translator. You can download it at [i18n-web-translator][1]
+
+
+Supported formats
+-----------------
+
+* **po**; supported format looks like
+
+	#  there is some comment
+	#, fuzzy, changed
+	#| msgid Old default
+	msgctxt "there.is.some.key"
+	msgid "Default text"
+	msgstr "Prelozeny defaultni text"
+
+  Such po file is pretty usable with po editors.
+
+* **yml**; standard yaml files in I18n simple format
+* **rb**; typical ruby files in I18n simple format
+
+
+New locale files format
+------------------------
+
+Old format using in Simple backend is:
+
+    key: "String"
+
+or for pluralization (depends on rules) it can be similar to this:
+
+    key:
+      one: "String for one"
+      other: "plural string"
+
+New format looks like:
+
+    key:
+      old: "old default string"
+      default: "new default string"
+      comment: "translator's comments"
+      t: "translation itself"
+      flag: "one of (ok || incomplete || changed || untranslated)"
+      fuzzy: true # exists only where flag != ok (nice to have when you want
+                    edit files manually)
+
+Pluralized variant should look like:
+
+    key:
+      one:
+        old:
+        default:
+        t:
+        ...
+      other:
+        old:
+        default:
+        t:
+        ...
+
+As you can see the old format is string and the new format is hash.
+If you use lambdas and procs objects, you should save them in separate
+file(s) because i18n-translate utility can't handle them but Translate backend
+can.
+
+
+Configure file format
+-------------------
+
+Configuration files are normal ruby files which should return hash.
+It is not necessary to use all switches.
+
+*Example config file:*
+
+    {
+      :exclude => ['rules'],
+      :format => 'yml',
+      :default => 'en_US'
+      :default_format => 'rb',
+      :verbose => true,
+      :locale_dir => 'locales',
+      :separator => '.',
+      :target => 'yml',
+      :deep => true,
+      :force_encoding => true,
+      :encoding => 'utf-8',
+      :quiet => false
+    }
+
+[1]: http://github.com/pejuko/i18n-web-translator
+

data/Rakefile

@@ -0,0 +1,18 @@
+# -*- coding: utf-8 -*-
+# vi: fenc=utf-8:expandtab:ts=2:sw=2:sts=2
+# 
+# @author: Petr Kovar <pejuko@gmail.com>
+
+require 'rake/testtask'
+require 'rake/gempackagetask'
+require 'rake/clean'
+
+CLEAN << "coverage" << "pkg"
+
+task :default => [:test]
+Rake::TestTask.new(:test) do |t|
+  t.pattern = File.join(File.dirname(__FILE__), 'test/tc_*.rb')
+  t.verbose = true
+end
+
+Rake::GemPackageTask.new(eval(File.read("i18n-translators-tools.gemspec"))) {|pkg|}

data/bin/i18n-translate

@@ -0,0 +1,304 @@
+#!/usr/bin/env ruby
+# -*- coding: utf-8 -*-
+# vi: fenc=utf-8:expandtab:ts=2:sw=2:sts=2
+# 
+# @author: Petr Kovar <pejuko@gmail.com>
+
+$KCODE="UTF8"
+
+def print_help
+  print $0
+  puts " <command> [options] [locale directory]"
+  puts ""
+  puts "commands:
+  list        -- list available locales
+  stat        -- shows statistics
+  create      -- adds new locale into locale directory
+  merge       -- scans locale directory and merge locales with default file
+  convert     -- convert from -f format to -c format
+  strip       -- throw away translation metadata from locales and makes them
+                 compatible again with I18n::Backend::Simple
+  translate   -- simple console translator; --language must be set
+
+
+options:
+  --exclude[=pattern], -x [pattern]   -- exludes directory/file from scan
+                                         if pattern is missing, exclude
+                                         is reset to empty array. useful if
+                                         you have exclude configured in your
+                                         config file and want to reset it
+                                         e.g: merge -x -x rules
+                                         the first -x resets exclude
+  --locale_dir=<dir>, -p <dir>        -- where to search for locales
+                                         (default: ./locale)
+  --default=<name>, -d <name>         -- name of file with default strings
+                                         without suffix (default: default)
+  --format=<format>, -f <format>      -- format of locale files. one from
+                                         auto, yml, rb, (po - not implemented)
+                                         (default: auto)
+  --default_format=<format>           -- format of default locale
+                                         default: same as --format
+  --separator=<string>, -s <string>   -- separator in keys (default: .)
+                                         (key e.g.: main.message.error)
+  --target=<format>, -t <format>      -- convert from one format to second
+                                         e.g: -f rb -t yml
+  --locale=<locale>, -l <locale>      -- work with specific locale only
+                                         e.g: create -p locale -l cs_CZ
+  --deep, -r                          -- scan locale_dir recursively
+  --quiet, -q                         -- show less information
+  --verbose, -v                       -- shows more information during
+                                         locales processing
+
+
+CONFIG FILE
+
+         ~/.config/ruby/i18n-translate
+         locale_dir/.i18n-translate
+
+         i18n-translate reads its configuration firstly from file and then
+         overwrites them with command line arguments. It is normal ruby file
+         where is defined variable options as hash.
+
+         Example:
+
+         options = {
+           :exclude => ['rules'],
+           :verbose => true,
+           :format => 'yml',
+           :default => 'en_US'
+         }
+
+
+!!! WARNING !!!
+
+         It can NOT handle locales with proc or lambda.
+         Store your procedures into diferent file/directory and include
+         this file/directory in --exclude. Optionaly you can set
+         excludes in your config file.
+"
+end
+
+
+COMMANDS = %w(list stat create merge convert strip translate)
+command = ARGV.shift
+
+unless COMMANDS.include?(command)
+  print_help()
+  exit
+end
+
+
+
+begin
+  require 'rubygems'
+rescue
+end
+
+require 'i18n'
+require 'i18n-translate'
+
+
+def process_locale(tr, command, opts)
+  options = opts.dup
+  options.delete(:quiet) if command == 'list'
+
+  print "#{tr.lang_file}..." unless options[:quiet]
+  
+  msg = ""
+  save_locale = false
+  case command
+  when 'strip'
+    tr.strip!
+    msg="striped"
+    save_locale = true
+  when 'merge'
+    tr.assign(tr.merge)
+    msg="merged"
+    save_locale = true
+  when 'convert'
+    tr.options[:format] = options[:target]
+    msg="converted"
+    save_locale = true
+  when 'list'
+    msg="exists"
+  when 'stat'
+    stat = tr.stat
+    if stat[:total] > 0
+      msg = "%3d%% (#{stat[:ok]}/#{stat[:total]})" % stat[:progress].to_i
+    else
+      msg = "doesn't exist #{tr.default_file}: Try --default_format" unless File.exists?(tr.default_file)
+    end
+  end
+  
+  if save_locale
+    tr.export!
+  end
+
+  puts msg unless options[:quiet]
+end
+
+
+# setup command  line arguments
+require 'getoptlong'
+opts = GetoptLong.new(
+  ["--exclude", "-x", GetoptLong::OPTIONAL_ARGUMENT],
+  ["--locale_dir", "-p", GetoptLong::REQUIRED_ARGUMENT],
+  ["--default", "-d", GetoptLong::REQUIRED_ARGUMENT],
+  ["--format", "-f", GetoptLong::REQUIRED_ARGUMENT],
+  ["--default_format", "-g", GetoptLong::REQUIRED_ARGUMENT],
+  ["--separator", "-s", GetoptLong::REQUIRED_ARGUMENT],
+  ["--target", "-t", GetoptLong::REQUIRED_ARGUMENT],
+  ["--deep", "-r", GetoptLong::NO_ARGUMENT],
+  ["--verbose", "-v", GetoptLong::NO_ARGUMENT],
+  ["--quiet", "-q", GetoptLong::NO_ARGUMENT],
+  ["--locale", "-l", GetoptLong::REQUIRED_ARGUMENT]
+)
+
+
+# setting up default options
+user_config_file = File.join(ENV["HOME"], ".config/ruby/i18n-translate")
+options = I18n::Translate::Translate::DEFAULT_OPTIONS.dup
+options.merge!(eval( File.read(user_config_file) )) if File.exists?(user_config_file)
+options[:exclude] ||= []
+
+
+tmp = {}
+# process command line arguments
+opts.each do |opt, val|
+  optkey = opt[2..-1].to_sym
+  case optkey
+  when :exclude
+    unless val.to_s.strip.empty?
+      tmp[:exclude] << val
+    else
+      tmp[:exclude] = []
+    end
+  when :format, :default_format
+    if (not I18n::Translate::FORMATS.include?(val)) and (val != 'auto')
+      puts "!!! Unknown file format"
+      exit 5
+    end
+    tmp[optkey] = val
+  when :deep
+    tmp[optkey] = val
+  when :verbose
+    tmp[:verbose] = true
+    tmp[:quiet] = false
+  when :quiet
+    tmp[:quiet] = true
+    tmp[:verbose] = false
+  else
+    tmp[optkey] = val
+  end
+end
+
+locale_dir = tmp[:locale_dir] ? tmp[:locale_dir] : options[:locale_dir]
+# reading project locale config options
+locale_config_file = File.join(locale_dir, ".i18n-translate")
+options.merge!(eval( File.read(locale_config_file) )) if File.exists?(locale_config_file)
+
+# merge command line arguments
+options.merge!(tmp)
+
+
+# process command
+case command
+
+when 'create'
+
+  unless options[:locale]
+    puts "!!! With command create you must set --locale parametr"
+    exit 1
+  end
+
+  format = (options[:format] == "auto") ? I18n::Translate::FORMATS.first : options[:format]
+  fname = File.join(options[:locale_dir], "#{options[:locale]}.#{format}")
+
+  if File.exists?(fname)
+    puts "!!! File '#{fname}' already exists"
+    exit 2
+  end
+
+  tr = I18n::Translate::Translate.new(options[:locale], options.merge({:format => format}))
+  tr.assign(tr.merge)
+  tr.export!
+
+  puts "#{tr.options[:locale_dir]}/#{tr.lang}...created" unless options[:quiet]
+
+when 'translate'
+
+  unless options[:locale]
+    puts "!!! With command translate you must set --locale parametr"
+    exit 1
+  end
+
+  tr = I18n::Translate::Translate.new(options[:locale], options)
+  stat = tr.stat
+  tr.merge.select{|x| x["flag"] != "ok"}.each_with_index do |entry, i|
+    next_entry = false
+    while not next_entry
+      puts ""
+      puts ""
+      puts "(#{i+1}/#{stat[:fuzzy]}) #{entry["key"]} (#{entry["flag"]})"
+      puts "comment: #{entry["comment"]}" unless entry["comment"].empty?
+      puts "old default: #{entry["old_default"]}" unless entry["old_default"].empty?
+      puts "old translation: #{entry["old_t"]}" unless entry["old_t"].empty?
+      puts "default: #{entry["default"]}"
+      puts "translation: #{entry["t"]}"
+      puts ""
+      puts "Actions:"
+      puts "n (next) t (translate) f (change flag) c (comment) s (save) q (save & quit) x(exit no saving)"
+      action = STDIN.readline.strip
+      puts ""
+      case action
+      when 'n'
+        next_entry = true
+      when 't'
+        puts "Enter translation:"
+        entry["t"] = STDIN.readline.strip
+        entry["flag"] = "ok" unless entry["t"].empty?
+        tr.assign( [entry] )
+        puts "Flag sets to #{entry["flag"]}"
+      when 'f'
+        puts "Change flag to:"
+        puts "o (ok), i (incomplete), c (changed), u (untranslated)"
+        f = STDIN.readline.strip
+        I18n::Translate::FLAGS.each do |fname|
+          if fname[0,1] == f
+            entry["flag"] = fname
+            break
+          end
+        end
+        tr.assign( [entry] )
+        puts "Flag sets to #{entry["flag"]}"
+      when 'c'
+        puts "Enter comment:"
+        entry["comment"] = STDIN.readline.strip
+        tr.assign( [entry] )
+        puts "Comment has changed."
+      when 's'
+        tr.export!
+        puts "Translation saved"
+      when 'q'
+        tr.export!
+        puts "Translation saved"
+        exit
+      when 'x'
+        exit
+      end # case
+    end # while
+  end # each_with_index
+
+  tr.export!
+  tr.reload!
+else
+
+  I18n::Translate.scan(options) do |tr|
+    # skip if not desired locale
+    next if options[:locale] and (options[:locale] != tr.lang)
+
+    process_locale(tr, command, options)
+  end
+
+end
+