i18n_googledocs 0.1.4

data/CHANGELOG

@@ -0,0 +1,37 @@
+## 
+
+### 0.1.3
+
+2012-05-02 @kristianmandrup
+
+Various fixes. Renamed gem.
+
+### 0.1.1 
+
+2012-04-23 @kristianmandrup
+
+- Fixed compatibility with Ruby 1.8.7
+- Added import, export and missing_keys generator for i18n-docs generator namespace
+- Allow export generator to use any locale as base (I18n.default_locale is now  the default!)
+- Fixed export bug when locale is 'no' for norway ('no' is treated as false value)
+- Added 'locales' generator to generate base locale files
+- Use I18n.default_locale as the default master locale for all generators
+- Added 'normalize' generator to normalize tabs to spaces
+- Added 'copy_master' generator to use one locale translation as base for other locales
+- Added :normalize parameter on export to allow pre-normalization before export :)
+- Added :output-dir to export and allow ~ as HOME alias
+- Added :locales option to export to control which locales are exported
+- Added :auto-translate option to export and copy_master, using Google Translate API. Note: Requires key registration at Google (not free).
+
+### 0.0.4
+
+Update docs, license. Push to Rubygems.org.
+
+### 0.0.3
+
+* Open sourced: changed name and description.
+
+### 0.0.2
+
+* removed loading of `awesome_print` from the rake task. It was breaking
+  download

data/Gemfile

@@ -0,0 +1,27 @@
+source :rubygems
+
+gem 'deep_merge'
+
+group :test do
+	gem 'to_lang'
+
+	# other Translator APIs or adapters
+	# gem 'google-translate'
+	# gem 'bing_translator'
+end
+
+group :development do
+  gem "rspec", 		">= 2.8.0"
+  gem "rdoc", 		">= 3.12"
+  gem "bundler", 	">= 1.0.0"
+  gem "jeweler", 	">= 1.8.3"
+  gem "simplecov",">= 0.5"
+  gem 'rake'
+  gem 'mocha'  
+end
+
+
+group :test, :development do
+	gem 'awesome_print'
+end
+

data/LICENSE.md

@@ -0,0 +1,8 @@
+Copyright (c) 2012 Local.ch AG
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+

data/README.md

@@ -0,0 +1,173 @@
+
+# i18n-docs
+
+**i18n-docs** is a ruby gem that helps you to keep translations stored in Google Docs. This makes it easier for translators and project members to coordinate changes and updates. In addition, the standard features of Google Docs are super handy for this process: revision tracking, authorization, publishing, etc. 
+
+Although we at use it with Google Docs, it could be used with any CSV file.
+
+*this gem is currently in use and tested with Rails 3.1. It probably works with other 3.x versions, but probably not 2.x at the moment.*
+
+## Features: 
+
+* download translations from multiple Google spreadsheets and store to YAML files
+* import/export YAML files with a Google Spreadsheet
+* check YAML files for missing translations (`rake i18n:find_missing_keys`)
+
+## Usage
+
+### Configuration
+
+Add the GEM to your Rails project:
+
+    gem 'i18n-docs'
+
+Create a configuration file in `config/translations.yml`:
+
+    files:
+      pre_signup.yml: "https://docs.google.com/spreadsheet/pub?key=ab43...34f3&single=true&gid=0&output=csv"
+      navigation.yml: "https://docs.google.com/spreadsheet/pub?key=0Ap...XveWc&single=true&gid=0&output=csv"
+      ... etc ...
+
+Make the directories and files to hold the translations:
+
+```
+$ rails g i18n_docs:locales da se no
+      create  config/locales/da/your_combination.yml.yml
+      create  config/locales/da/pre_signup.yml.yml
+      create  config/locales/da/navigation.yml.yml
+      create  config/locales/da/extras.yml.yml
+      create  config/locales/da/your_team.yml.yml
+      create  config/locales/se/your_combination.yml.yml
+      create  config/locales/se/pre_signup.yml.yml
+      create  config/locales/se/navigation.yml.yml
+      create  config/locales/se/extras.yml.yml
+      create  config/locales/se/your_team.yml.yml
+      create  config/locales/no/your_combination.yml.yml
+      create  config/locales/no/pre_signup.yml.yml
+      create  config/locales/no/navigation.yml.yml
+      create  config/locales/no/extras.yml.yml
+      create  config/locales/no/your_team.yml.yml
+```
+      
+Finally, let Rails know what locales you will be using. Add this to `config/application.rb`:
+
+    # locales to support:
+    I18n.available_locales = [:en,:de,:it,:fr]
+
+This defines which languages and translation files to import from a Google Spreadsheet. The content of the Spreadsheet URL is stored to a file called e.g. `example1.yml` within folders `config/locales/en` and all other detected locales.
+
+## Generators
+
+## Translation APIs
+
+The built in (optional) auto-translation currently only supports Google Translation API.
+
+Please help us add Bing or some other translation API support :)
+
+### Normalizer
+
+Normalizes yml files, converting tabs to 2 spaces pr. default
+This is useful before export in order to better ensure that yaml files have valid syntax.
+
+`$ rails g i18n_docs:normalize da`
+
+By default, the normalized files are prefixed with '_'. In order to overwrite existing files, use the overwrite option.
+
+`$ rails g i18n_docs:normalize da --overwrite true`
+
+In addition you can use the space option to customize spaces pr tab.
+
+`$ rails g i18n_docs:normalize da se --spaces 4`
+
+If you want to clean up and remove the normalized files, use the delete option
+
+`$ rails g i18n_docs:normalize da se --delete`
+
+If you want to accept all normalized files prefixed with '_'
+
+`$ rails g i18n_docs:normalize da se --accept`
+
+## Copy master
+
+Use one locale as master and copy into other locales while changing the root key :) 
+
+`$ rails g i18n_docs:copy_master da --into se no`
+
+Auto translate from the master locale file to each locale generated. 
+
+`$ rails g i18n_docs:copy_master da --into sv --auto-translate`
+
+### Export
+
+Export locale files as CSV files to upload to Google Docs
+
+`$ rails g i18n_docs:export`
+
+Use specific (here danish) locale as the master locale
+
+`$ rails g i18n_docs:export da`
+
+Auto translate from the master locale file to each csv generated. 
+
+`$ rails g i18n_docs:export da --auto-translate`
+
+It also supports a locales option to control for which locales to export.
+The normalize option can be used to pre-normalize before export using the normalize generator.
+
+`$ rails g i18n_docs:export da --locales sv --auto-translate --normalize`
+
+Finally the outpur-dir option can be used to control where to export the cvs files. If you use the ~ (HOME alias) it will substitute with ENV['HOME'] and work as expected ;)
+
+`$ rails g i18n_docs:export da --locales sv --output-dir "~/Documents/csv"`
+
+### Find missing keys
+
+Find and list translation keys that do not exist in all locales
+
+`$ rails g i18n_docs:missing_keys`
+
+### Import
+
+Import csv files from Google Docs as locale files
+
+`$ rails g i18n_docs:import`
+
+### Rake Tasks
+
+Following Rake tasks are added by the GEM to your Rails project:
+
+* `rake i18n:export_translations`
+  Export all language files to CSV files (only files contained in 'en' folder are considered)
+* `rake i18n:export_translations[da]` (override default: only files in 'da' folder)
+
+* `rake i18n:import_translations`
+  Download translations from Google Spreadsheet and save them to YAML files.
+* `rake i18n:missing_keys`
+  Find and list translation keys that do not exist in all locales
+
+
+## Setup at Google Docs
+
+* At Google Docs, create a new spreadsheet for your translations. The first column is the translation key, the other columns are the languages:
+![screenshot](http://dl.dropbox.com/u/385855/Screenshots/bamr.png)
+* use option `File -> Publish To the Web` 
+![screenshot](http://dl.dropbox.com/u/385855/Screenshots/2-lp.png)
+* use **get a link** to generate a unique url that points to the CSV spreadsheet
+![screenshot](http://dl.dropbox.com/u/385855/Screenshots/oom_.png)
+* From now on you should only update translations in Google Docs and run `rake i18n:import_translations` in the application to get changes. You can also export your 
+
+## Todo
+
+*Pull requests welcome!*
+
+* Create mocked tests that do not have to download our sample CSV files
+* Support more Translation APIs besides Google, fx Bing etc.
+
+## CHANGELOG
+
+See CHANGELOG file
+
+### Credits/License
+
+This gem is sponsored by [local.ch](http://www.local.ch/). It is licensed under the [MIT license](http://en.wikipedia.org/wiki/MIT_License). If you're a ruby developer and want to work with us in Switzerland, please check out our [jobs page](http://local-ch.github.com/).
+

data/Rakefile

@@ -0,0 +1,25 @@
+# encoding: utf-8
+
+require 'rubygems'
+require 'bundler'
+begin
+  Bundler.setup(:default, :development)
+rescue Bundler::BundlerError => e
+  $stderr.puts e.message
+  $stderr.puts "Run `bundle install` to install missing gems"
+  exit e.status_code
+end
+require 'rake'
+
+require 'jeweler'
+Jeweler::Tasks.new do |gem|
+  # gem is a Gem::Specification... see http://docgem.rubygemgem.org/read/chapter/20 for more options
+  gem.name 				= "i18n_googledocs"
+  gem.homepage 		= "http://github.com/kristianmandrup/i18n-docs"
+  gem.summary     = "Maintain translations in Google Docs and export them to your Rails project."
+  gem.description = "GEM providing helper scripts to manage i18n translations in Google Docgem. Features: check YAML files for missing translations; export YAML files to CSV; download translations from multiple Google spreadsheets and store to YAML files"
+  gem.authors     = ["Georg Kunz", "Ivan Jovanovic", "Jeremy Seitz", "Kristian Mandrup"]
+  gem.email       = 'jeremy.seitz@local.ch'
+end
+Jeweler::RubygemsDotOrgTasks.new
+

data/VERSION

@@ -0,0 +1 @@
+0.1.4

data/i18n_googledocs.gemspec

@@ -0,0 +1,95 @@
+# Generated by jeweler
+# DO NOT EDIT THIS FILE DIRECTLY
+# Instead, edit Jeweler::Tasks in Rakefile, and run 'rake gemspec'
+# -*- encoding: utf-8 -*-
+
+Gem::Specification.new do |s|
+  s.name = "i18n_googledocs"
+  s.version = "0.1.4"
+
+  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+  s.authors = ["Georg Kunz", "Ivan Jovanovic", "Jeremy Seitz", "Kristian Mandrup"]
+  s.date = "2012-05-03"
+  s.description = "GEM providing helper scripts to manage i18n translations in Google Docgem. Features: check YAML files for missing translations; export YAML files to CSV; download translations from multiple Google spreadsheets and store to YAML files"
+  s.email = "jeremy.seitz@local.ch"
+  s.extra_rdoc_files = [
+    "LICENSE.md",
+    "README.md"
+  ]
+  s.files = [
+    "CHANGELOG",
+    "Gemfile",
+    "LICENSE.md",
+    "README.md",
+    "Rakefile",
+    "VERSION",
+    "i18n_googledocs.gemspec",
+    "lib/generators/i18n_docs/copy_master_generator.rb",
+    "lib/generators/i18n_docs/export_generator.rb",
+    "lib/generators/i18n_docs/import_generator.rb",
+    "lib/generators/i18n_docs/locales_generator.rb",
+    "lib/generators/i18n_docs/missing_keys_generator.rb",
+    "lib/generators/i18n_docs/normalize_generator.rb",
+    "lib/generators/i18n_docs/templates/locale.erb",
+    "lib/i18n_googledocs.rb",
+    "lib/localch_i18n/csv_to_yaml.rb",
+    "lib/localch_i18n/missing_keys_finder.rb",
+    "lib/localch_i18n/translation_file_export.rb",
+    "lib/localch_i18n/translations.rb",
+    "lib/localch_i18n/translator.rb",
+    "lib/localch_i18n/translator/service.rb",
+    "lib/localch_i18n/util.rb",
+    "lib/tasks/store_translations.rake",
+    "tasks/test.rake",
+    "test/fixtures/config.yml",
+    "test/fixtures/de/header.yml",
+    "test/fixtures/en/header.yml",
+    "test/fixtures/minimal.csv",
+    "test/fixtures/test.csv",
+    "test/test_helper.rb",
+    "test/unit/csv_to_yaml_test.rb",
+    "test/unit/translation_file_export_test.rb",
+    "test/unit/translations_test.rb"
+  ]
+  s.homepage = "http://github.com/kristianmandrup/i18n-docs"
+  s.require_paths = ["lib"]
+  s.rubygems_version = "1.8.24"
+  s.summary = "Maintain translations in Google Docs and export them to your Rails project."
+
+  if s.respond_to? :specification_version then
+    s.specification_version = 3
+
+    if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
+      s.add_runtime_dependency(%q<deep_merge>, [">= 0"])
+      s.add_development_dependency(%q<rspec>, [">= 2.8.0"])
+      s.add_development_dependency(%q<rdoc>, [">= 3.12"])
+      s.add_development_dependency(%q<bundler>, [">= 1.0.0"])
+      s.add_development_dependency(%q<jeweler>, [">= 1.8.3"])
+      s.add_development_dependency(%q<simplecov>, [">= 0.5"])
+      s.add_development_dependency(%q<rake>, [">= 0"])
+      s.add_development_dependency(%q<mocha>, [">= 0"])
+      s.add_development_dependency(%q<awesome_print>, [">= 0"])
+    else
+      s.add_dependency(%q<deep_merge>, [">= 0"])
+      s.add_dependency(%q<rspec>, [">= 2.8.0"])
+      s.add_dependency(%q<rdoc>, [">= 3.12"])
+      s.add_dependency(%q<bundler>, [">= 1.0.0"])
+      s.add_dependency(%q<jeweler>, [">= 1.8.3"])
+      s.add_dependency(%q<simplecov>, [">= 0.5"])
+      s.add_dependency(%q<rake>, [">= 0"])
+      s.add_dependency(%q<mocha>, [">= 0"])
+      s.add_dependency(%q<awesome_print>, [">= 0"])
+    end
+  else
+    s.add_dependency(%q<deep_merge>, [">= 0"])
+    s.add_dependency(%q<rspec>, [">= 2.8.0"])
+    s.add_dependency(%q<rdoc>, [">= 3.12"])
+    s.add_dependency(%q<bundler>, [">= 1.0.0"])
+    s.add_dependency(%q<jeweler>, [">= 1.8.3"])
+    s.add_dependency(%q<simplecov>, [">= 0.5"])
+    s.add_dependency(%q<rake>, [">= 0"])
+    s.add_dependency(%q<mocha>, [">= 0"])
+    s.add_dependency(%q<awesome_print>, [">= 0"])
+  end
+end
+

data/lib/generators/i18n_docs/copy_master_generator.rb

@@ -0,0 +1,118 @@
+require 'localch_i18n/util'
+
+module I18nDocs
+  module Generators
+    class CopyMasterGenerator < Rails::Generators::Base
+			include LocalchI18n::Util
+
+      desc "Copy master locale files to other locales to be used as base"
+
+      class_option  :into,  :type => :array, :default => [], 
+          :desc => "Which locales to copy master files into"
+
+      class_option  :auto_translate,  :type => :boolean, :default => false, 
+          :desc => "Use Google Translate API to auto-translate?"
+
+      argument      :master, :type => :string,  :default => nil, 
+          :desc => "Master locale, default = I18n.default_locale"
+
+      def main_flow        
+        check_translate_key if options[:auto_translate]
+        copy_master master_locale
+      end        
+
+      protected
+
+      # TODO: refactor this when/if we support other translation services 
+      def check_translate_key
+        unless I18n.google_translation_key
+          say %q{You must set I18n.google_translation_key to a valid API key.
+  See https://code.google.com/apis/console/b/0/?api=translate}, :red
+          exit(0)
+        end
+
+        say "Using google translation key: #{I18n.google_translation_key}"        
+      end
+
+      def master_locale
+        master || I18n.default_locale || :en
+      end
+
+      def locales
+        @locales ||= options[:into].empty? ? I18n.available_locales : options[:into]
+      end
+
+      def unnormalized_files path
+        Dir[File.join(path,'[^_]*.yml')]
+      end
+
+      def locales_root
+        Rails.root.join 'config', 'locales'
+      end
+
+      def locale_names
+      	locales.join(' ')
+      end
+
+      def master_locale
+        master || I18n.default_locale || :en
+      end
+
+      def make_local_copy file, locale
+      	local_content = content.sub /#{master_locale}:/, "#{locale}:"
+      	file = file.gsub /\\/, '/'
+      	local_file = file.gsub /\/#{master_locale}\//, "\/#{locale}\/"
+
+      	File.open(local_file, 'w') do |f|
+      		f.puts local_content
+      	end
+
+        # auto translate local file just written 
+      	auto_translate_for(local_file, locale) if auto_translate?
+      end
+
+     	attr_accessor :content
+      attr_reader :current_locale
+
+      # override method in Util module to use options
+      def auto_translate?
+        options[:auto_translate]
+      end
+
+      # flatten_translations_hash has already done the translation
+      # if auto_translate? returned true
+     	def auto_translate_for(local_file, locale)
+        @current_locale = locale
+     		flat_hash = flatten_translations_hash load_translations_for(local_file, locale)
+
+        translated_hash = {}
+        flat_hash.each do |key, text|
+          translation = text
+          trans_row_hash = row_to_hash key, translation
+          translated_hash.deep_merge!(trans_row_hash)
+        end
+        translated_hash
+
+        # write the translated file
+        File.open(local_file, 'w') do |file|
+          final_translation_hash = {locale => translated_hash}
+          file.puts YAML::dump(final_translation_hash)
+        end
+     	end
+
+      def locale_path locale
+        (locale != :all) ? File.join(locales_root, locale) : locales_root
+      end
+
+      def copy_master locale = :en
+        path = locale_path(locale)
+        say "Using #{locale} as master for locales: #{locale_names}"
+        unnormalized_files(path).each do |file|
+        	say "Making local copies for: #{file}"
+        	self.content = File.open(file).read
+          locales.each{|locale| make_local_copy file, locale }
+        end       
+      end
+		end
+	end
+end

data/lib/generators/i18n_docs/export_generator.rb

@@ -0,0 +1,92 @@
+module I18nDocs
+  module Generators
+    class ExportGenerator < Rails::Generators::Base
+      desc "Export locale files as CSV files"
+
+      argument     :master,  :type => :string,  :default => nil, :desc => "Master locale, default = I18n.default_locale"
+      
+      class_option  :normalize,  :type => :boolean, :default => false, 
+          :desc => "Normalize locale files (with overwrite) before export?"
+
+      class_option  :locales,  :type => :array, :default => [], 
+          :desc => "locales to export for, default = all"
+
+      class_option  :output_dir,  :type => :string, :default => nil, 
+          :desc => "Output directory"
+
+      def main_flow
+        generate "i18n_docs:normalize #{locale_names} --overwrite" if normalize?
+        show_files
+        export_files
+      end        
+
+      protected
+
+      def normalize?
+        options[:normalize]
+      end
+
+      def locale_names
+        locales.join(' ')
+      end
+
+      def show_files
+        say ""
+        say "  Valid locales to export: #{export_locales}", :green
+        say "  Detected files:"
+        input_files.each {|f| say "    * #{File.basename(f)}", :green }
+      end        
+
+      def export_files
+        say ""
+        say "  Start exporting files:", :green
+        say ""
+        say "    source dir: #{source_dir}"
+        say "    output dir: #{output_dir}"
+        
+        input_files.each do |file|
+          file = File.basename(file)
+          exporter = LocalchI18n::TranslationFileExport.new(source_dir, file, output_dir, locales_to_export)
+          exporter.export
+        end
+        
+        say ""
+        say "  CSV files can be removed safely after uploading them manually to Google Spreadsheet."
+        say ""
+      end
+
+
+      def source_dir
+        Rails.root.join('config', 'locales')
+      end
+
+      def output_dir
+        @output_dir ||= begin
+          odir = options[:output_dir] || Rails.root.join('tmp')
+          odir = odir.gsub(/~/, ENV['HOME']) if odir =~ /~/
+          odir
+        end
+      end
+    
+      def locales_to_export
+        export_locales.empty? ? I18n.available_locales : export_locales
+      end
+    
+      def export_locales
+        locales.map(&:to_sym) & I18n.available_locales.map(&:to_sym)
+      end
+
+      def locales
+        options[:locales]
+      end
+
+      def master_locale
+        master || I18n.default_locale || :en
+      end
+
+      def input_files
+        Dir[File.join(source_dir, master_locale.to_s, '*.yml')]
+      end
+    end
+  end
+end

data/lib/generators/i18n_docs/import_generator.rb

@@ -0,0 +1,30 @@
+module I18nDocs
+  module Generators
+    class ImportGenerator < Rails::Generators::Base
+      desc "Import locale files from CSV files"
+
+      def main_flow
+        find_translations_file
+        import
+      end        
+
+      protected
+
+			def find_translations_file
+	    	config_file = Rails.root.join('config', 'translations.yml')
+	    	raise "No config file 'config/translations.yml' found." if !File.exists?(config_file)
+	    end
+
+			def import		   		   
+		    translations = LocalchI18n::Translations.new(config_file, tmp_dir)
+		    translations.download_files
+		    translations.store_translations
+		    translations.clean_up
+		  end
+
+		  def tmp_dir
+		  	Rails.root.join('tmp')
+		  end
+		end
+	end
+end

data/lib/generators/i18n_docs/locales_generator.rb

@@ -0,0 +1,59 @@
+# for lang in de en it fr; do \
+#     mkdir -p config/locales/$lang; \
+#     touch config/locales/$lang/navigation.yml; \
+#     touch config/locales/$lang/forms.yml; \
+#   done
+module I18nDocs
+  module Generators
+    class LocalesGenerator < Rails::Generators::Base
+      desc "Creates Locale files for I18n-docs"
+
+      argument     :locales,           :type => :array,      :desc => "Locale codes"
+
+      source_root File.dirname(__FILE__) + '/templates'
+
+      def main_flow
+        locales.each do |locale|
+        	self.locale = locale
+          exec_template
+        end
+      end
+
+      def instructions
+      	say "----------------------------------"
+      	say "Add this to config/application.rb:"
+      	say "config.i18n.available_locales = #{available_locales}", :green
+      end
+
+      protected
+
+      def available_locales
+      	locales.map(&:to_sym).inspect
+      end
+
+      def files_src_path
+      	File.join Rails.root, 'config', 'translations.yml'
+      end
+
+      def yaml
+        say("Missing #{files_src_path}", :red) and exit(0) unless File.exist?(files_src_path)
+      	YAML::load File.open(files_src_path)
+      end
+
+      def files
+      	yaml['files'].keys
+      end
+
+      attr_accessor :locale, :file, :key, :text
+
+      def exec_template
+      	files.each do |file|      
+          self.file = file
+          self.key = file.gsub /\..+$/, ''	
+          self.text = options[:text] || "Hello from #{locale} locale"
+        	template "locale.erb", "config/locales/#{locale}/#{file}"
+        end
+      end
+    end
+  end
+end

data/lib/generators/i18n_docs/missing_keys_generator.rb

@@ -0,0 +1,12 @@
+module I18nDocs
+  module Generators
+    class MissingKeysGenerator < Rails::Generators::Base
+      desc "Show missing keys"
+
+      def main
+		    finder = LocalchI18n::MissingKeysFinder.new(I18n.backend)
+		    finder.find_missing_keys
+		  end
+		end
+	end
+end