slimkeyfy 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 51e9f1359e2f3cec0cca00636171ddcdb452fc13
+  data.tar.gz: f8fe3487b10951a60283ff911bb7acd9d080d8e0
+SHA512:
+  metadata.gz: 6ec0ec5446e2fa6853695ea7fb96d3ac4921666e001c449b39d9da379c7b6a253fd7a5b955e5e6f796ffc0314164a925f37da1ee6144ae7c4e072a61ed7ed633
+  data.tar.gz: 65b12829f6513726748d6a646da5524343af0b19de8f74cdefdf489fd7eeabc1fb1214e4812cbd326ab77a7567797c05fb77b4b2f3830fa1d78d8325e8b97333

data/.gitignore

@@ -0,0 +1,34 @@
+*.gem
+*.rbc
+/.config
+/coverage/
+/InstalledFiles
+/pkg/
+/spec/reports/
+/test/tmp/
+/test/version_tmp/
+/tmp/
+
+## Specific to RubyMotion:
+.dat*
+.repl_history
+build/
+
+## Documentation cache and generated files:
+/.yardoc/
+/_yardoc/
+/doc/
+/rdoc/
+
+## Environment normalisation:
+/.bundle/
+/lib/bundler/man/
+
+# for a library or gem, you might want to ignore these files since the code is
+# intended to run in multiple environments; otherwise, check them in:
+# Gemfile.lock
+# .ruby-version
+# .ruby-gemset
+
+# unless supporting rvm < 1.11.0 or doing something fancy, ignore this:
+.rvmrc

data/.rspec

@@ -0,0 +1,3 @@
+--color
+--warnings
+--require spec_helper

data/Gemfile

@@ -0,0 +1,6 @@
+source 'https://rubygems.org'
+
+ruby '2.1.1'
+
+gem 'rspec', :require => 'spec'
+

data/Gemfile.lock

@@ -0,0 +1,22 @@
+GEM
+  remote: https://rubygems.org/
+  specs:
+    diff-lcs (1.2.5)
+    rspec (3.0.0)
+      rspec-core (~> 3.0.0)
+      rspec-expectations (~> 3.0.0)
+      rspec-mocks (~> 3.0.0)
+    rspec-core (3.0.2)
+      rspec-support (~> 3.0.0)
+    rspec-expectations (3.0.2)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.0.0)
+    rspec-mocks (3.0.2)
+      rspec-support (~> 3.0.0)
+    rspec-support (3.0.2)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  rspec

data/LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2014 PhraseApp.com
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,172 @@
+Slimkeyfy
+=
+Extract plain Strings from .slim views and Rails controllers to replace them with I18n's t() method. Keys with it's translations will be streamed to a YAML file.
+```ruby
+slimkeyfy app/views/users/show.html.slim en
+# users/show.html.slim
+h1 Hello World!
+# converts to
+h1= t('.hello_world') 
+# config/locales/users.en.yaml
+# en: 
+#   user:
+#     show:
+#       hello_world: Hello World!
+```
+
+Install
+-
+
+With Ruby Gems
+```ruby
+> gem install slimkeyfy
+```
+
+With Bundler
+```ruby
+group :development do
+  # Github
+  gem 'slimkeyfy', github: 'https://github.com/phrase/slimkeyfy.git'
+  # or with concrete version
+  gem 'slimkeyfy', '~> 0.1'
+end
+```
+
+Approach
+-
+Most of the time tools like this go for a 80/20 approach stating that 80% can be done right and 20% have to be done manually. Our approach is similar but a bit different. Translating and tagging your Rails app can be error prone. In order to reduce this Slimkeyfy streams in every line that matches the regular expression engine and prompts you to take an action. This guarantees a higher quality but also gets us closer to the 90%. It might take a little more time than full automation. The collected data is then processed into a YAML file. If you don't provide a YAML one will be created. All your processed views and resulting translations will be merged with the existing YAML.
+
+Usage
+-
+```unix
+slimkeyfy INPUT_FILENAME_OR_DIRECTORY LOCALE (e.g. en, fr) [YAML_FILE] [Options]
+```
+- If you do not provide a YAML file - one will be created at configs/locales/view_folder_name.locale.yml
+- If you provide one make sure that the top level locale matches your provided locale
+- The YAML file will be loaded as a hash and deep_merged with the new found translations
+
+**Stream** - walks through the given file/files and if a regex hits you will be prompted to apply (y)es, discard (n)o, tag (x) (comments suggestions above the processed line) or (a)bort (only aborts the current file).
+
+without YAML file
+```unix
+slimkeyfy path/to/your/file.html.slim en
+```
+with given YAML file (has to be valid YAML! with the top level locale matching your provided locale)
+```unix
+slimkeyfy path/to/your/dir/ en path/to/your/en.yml
+```
+A Backup (.bak) of the old file will be created e.g. index.html.slim => index.html.slim.bak.
+Providing ---no-backup creates the backups temporarily and deletes them right after you apply your changes. (be careful!)
+
+Example Usage
+-
+```unix
+your_app_name/
+  |- app/
+    |- views/
+      |- user/
+        new.html.slim
+        show.html.slim
+      |- project/
+        index.html.slim
+        ...
+    ...
+  |- config/
+    |- locales/
+      en.yml
+      ...
+  ...
+
+> pwd
+../your_app_name/
+ 
+> slimkeyfy app/views/users/ en
+... choose your changes here (y/n/x/a)
+
+> ls app/views/users/
+    new.html.slim
+    show.html.slim
+    new.html.slim.bak
+    show.html.slim.bak
+    
+> ls config/locales/
+    user.en.yml
+    en.yml
+    
+> cat ../users/new.html.slim.bak
+  h1 Hello World!
+    
+> cat ../users/new.html.slim
+  h1= t('.hello_world')
+ 
+> cat config/locales/en.yml
+  --
+  en:
+    keys..
+          
+> cat config/locales/users.en.yml
+  ---
+  en:
+    users:
+      new:
+        hello_world: Hello World!
+      show:
+        ...
+```
+Suggested Workflow
+-
+As HTML is not 100% parsable there will be errors in the conversion. To minimize your error rate we suggest to approach each view or view_folder individually. The i18n-tasks gem helped a lot by finding errors. Always double check your views and make sure that everything went smoothly. Especially double check all your links. Here is an example workflow:
+```ruby
+# 1. create a branch for a view folder 
+> git checkout -b users_localization
+
+# 2. slimkeyfy the view folder you would like to tag
+> slimkeyfy app/views/users/ en
+
+# 3. go through all files and verify/add missing translations 
+# (check against the .bak files (use git diff))
+> git diff app/views/users/views.html.slim
+
+# 4. add your translations + keys to your locale file(s)
+# 4.1 optional: Use the I18n-tasks gem to find missing translations
+
+# 5. go through all your views and click through everything to actually "see" what changed
+
+# 6. If everything is fine - clean up (remove .baks)
+> rm app/views/users/*.bak
+# Optional: remove temporary YAML files
+> rm config/locales/users.en.yml
+```
+Testing
+-
+To test the application simply call rspec spec/.. from the root directory of slimkeyfy
+```unix
+bundle install
+bundle exec rspec spec/
+```
+PhraseApp Integration
+-
+Now that you processed your views and moved the generated keys to your localization files it is quite easy to push it to PhraseApp. If you have not set up your account yet [take a look here](https://phraseapp.com/docs/about/setup-your-translations-with-phraseapp?language=en). Make sure that the latest gem of PhraseApp is installed by simply typing:
+```unix
+> gem install phrase
+```
+If you are already familiar with the PhraseApp gem you can upload your translation/localization files now. For more details on your setup have a look at our [detailed guide](https://phraseapp.com/docs/about/access-your-locale-files-with-the-api-client?language=en).
+
+Todo
+-
+- YAML placeholders are currently not supported (We are working on it!).
+- a dry run option where you can see what will happen if you convert.
+- currently you are prompted for all hits - We would like to do 70-80% automatically and prompt for the 20-30% that cannot be decided upon.
+- adding / generalizing Regular Expressions for better hit rate
+
+Issues
+-
+- If you choose to take a lot of files at one time make sure to go through with it. It is not an issue to completely rerun everything (already translated strings are ignored) but should be avoided.
+- some hits will not be correct, some things that should be found are not and sometimes the regex engine won't work for mysterious reasons. Slimkeyfy is a helper, it does not provide full automation. It might make your work a little easier.
+
+References
+-
+* Other tools, not slim specific for this task is the [i15r gem](https://github.com/balinterdi/i15r). It can process .haml and .erb.
+* I strongly recommend checking your translated app with the [i18n-tasks gem](https://github.com/glebm/i18n-tasks). It is a great tool in     finding missing and unused translations.
+* Always consult the [Official Rails Internationalization guide](http://guides.rubyonrails.org/i18n.html) if in doubt.
+* Visit [PhraseApp on Github](https://github.com/phrase/phrase)

data/bin/slimkeyfy

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+
+require_relative '../lib/slimkeyfy'
+
+SlimKeyfy.translate(ARGV)

data/lib/slimkeyfy/console/command_line.rb

@@ -0,0 +1,90 @@
+require 'optparse'
+
+class SlimKeyfy::Console::Commandline
+  def initialize(args)
+    @options = {}
+    @args = args
+    OptionParser.new(&method(:opt_scan)).parse!(@args)
+  end
+
+  def wrong_usage
+    puts "Please provide a path to .slims and a locale name. See -h for more information."
+    exit
+  end
+
+  def opt_scan(opts)
+    opts.banner = "
+#{"Usage".white}: slimkeyfy #{"INPUT_FILENAME_OR_DIRECTORY".green} #{"LOCALE".red} #{"[YAML_FILE]".yellow} [Options]
+  e.g. slimkeyfy #{"app/views/users/".green} #{"en".red} #{"phrase/locales/en.yml".yellow} [Options]
+
+#{"Description".white}: Extract plain Strings from .slim views and Rails controllers to 
+  replace them with I18n's t() method. Keys with it's translations will be streamed 
+  to a YAML file.
+
+  the following options are available:
+
+"
+    opts.on_tail('-h', '--help', 'Show this message') do
+      puts opts
+      exit
+    end
+    opts.on_tail('-v', '--version', 'Show current version') do
+      puts "0.1"
+      exit
+    end
+    opts.on_tail('-R', '--recursive', 'If a directory is given all subdirectories will be walked either. 
+                                      Without -r and a directory just the files within the first level are walked.
+                                      ') do
+      @options[:recursive] = true
+    end
+    opts.on_tail('-b', '--no-backup', 'No Backups - for safety reasons - are created. For minimum safety we 
+                                      recommend a version control like git. Backups will still be created for 
+                                      comparison but deleted right after you agree to the changes.
+                                ') do
+      @options[:no_backup] = true
+    end
+  end
+
+  def run
+    @options[:input] = input = @args.shift
+    @options[:locale] = locale = @args.shift
+    @options[:yaml_output] = @args.shift
+
+    wrong_usage if (input.nil? or locale.nil?)
+
+    if File.directory?(input) then
+      directory_translate(input)
+    elsif File.file?(input) then
+      translate
+    else
+      raise "Please provide a file or directory!"
+      exit
+    end
+  end
+
+  def directory_translate(input)
+    rec = @options.fetch(:recursive, false)
+    SlimKeyfy::Slimutils::MFileUtils.walk(input, rec).each do |rec_input|
+      if File.file?(rec_input) and is_valid_ext?(rec_input) then
+        @options[:input] = rec_input
+        translate
+      end
+    end
+  end
+
+  def translate
+    unless SlimKeyfy::Slimutils::MFileUtils.is_valid_extension?(@options[:input]) then
+      puts "invalid extension!"
+      return
+    end
+    @options[:ext] = SlimKeyfy::Slimutils::MFileUtils.file_extension(@options[:input])
+
+    puts "file=#{@options[:input]}"
+    translate_slim = SlimKeyfy::Console::Translate.new(@options)
+    begin
+      translate_slim.stream_mode
+    rescue Interrupt
+      SlimKeyfy::Slimutils::MFileUtils.restore(translate_slim.bak_path, translate_slim.original_file_path)
+    end
+  end
+end

data/lib/slimkeyfy/console/io_action.rb

@@ -0,0 +1,30 @@
+class SlimKeyfy::Console::IOAction
+  def self.yes_or_no?(msg)
+    puts "#{msg} (y|n)"
+    arg = STDIN.gets.chomp
+    if arg =~ /[yY](es)?/ then
+      true
+    elsif arg =~ /[nN]o?/ then
+      false
+    else
+      puts "Provide either (y)es or (n)o!"
+      self.yes_or_no?(msg)
+    end
+  end
+  def self.choose(msg)
+    puts "#{msg} (y|n) (x for tagging | (a)bort)"
+    arg = STDIN.gets.chomp
+    if arg =~ /[yY](es)?/ then
+      "y"
+    elsif arg =~ /[nN]o?/ then
+      "n"
+    elsif arg =~ /x/ then
+      "x"
+    elsif arg =~ /a/ then
+      "a"
+    else
+      puts "Either (y)es, (n)o, (x) for tagging or (a) to abort"
+      self.choose(msg)
+    end
+  end
+end

data/lib/slimkeyfy/console/printer.rb

@@ -0,0 +1,50 @@
+class SlimKeyfy::Console::Printer
+  def self.difference(old_line, new_line, translations)
+    puts "#{'-'.red} #{normalize(old_line).red}"
+    puts "#{'+'.green} #{normalize(new_line).green} => #{translations.to_s.yellow}"
+    puts "-"*40
+  end
+  def self.unix_diff(bak_path, file_path)
+    result = "Please install colordiff or diff (brew install colordiff)"
+    colordiff, diff = `which colordiff`, `which diff`
+    if not colordiff.empty? then
+      result = `colordiff #{bak_path} #{file_path}`
+    elsif not diff.empty? then
+      result =`diff #{bak_path} #{file_path}`
+    end
+    if result.nil? or result.strip.empty? then
+      puts "No changes for comparison found!"
+    else
+      puts "#{result}"
+    end
+  end
+  def self.normalize(line)
+    line.sub(/^\s*/, " ")
+  end
+  def self.tag(old_line, translations, comment_tag)
+    prettified_translations = translations.map{|k, v| "#{k}: #{v}, t('.#{k.split(".").last}')" }.join(" | ")
+    "#{indentation(old_line)}#{comment_tag} #{prettified_translations}\n#{old_line}"
+  end
+  def self.indentation(line)
+    return "" if line.nil? or line.empty?
+    " " * (line.size - line.gsub(/^\s+/, "").size)
+  end
+end
+
+class String
+  def colorize(color_code)
+    "\e[#{color_code}m#{self}\e[0m"
+  end
+  def red
+    colorize(31)
+  end
+  def green
+    colorize(32)
+  end
+  def yellow
+    colorize(34)
+  end
+  def white
+    colorize(37)
+  end
+end

data/lib/slimkeyfy/console/translate.rb

@@ -0,0 +1,101 @@
+class SlimKeyfy::Console::Translate
+
+  attr_reader :original_file_path, :bak_path
+
+  def initialize(options={})
+    @extension = options[:ext]
+    @transformer = transformer
+    @original_file_path = options[:input]
+    @no_backup = options.fetch(:no_backup, false)
+    @bak_path = SlimKeyfy::Slimutils::MFileUtils.backup(@original_file_path)
+    @content = SlimKeyfy::Slimutils::FileReader.read(@bak_path).split("\n")
+    @file_path = SlimKeyfy::Slimutils::MFileUtils.create_new_file(@original_file_path)
+    @key_base = generate_key_base
+    @yaml_processor = create_yaml_processor(options)
+    @new_content = []
+    @changes = false
+  end
+
+  def transformer
+    if @extension == "slim" then
+      SlimKeyfy::Transformer::SlimTransformer
+    elsif @extension == "rb" then
+      SlimKeyfy::Transformer::ControllerTransformer
+    else
+      puts "Unknown extension type!"
+      exit
+    end
+  end
+
+  def create_yaml_processor(options)
+    SlimKeyfy::Slimutils::YamlProcessor.new(options[:locale], @key_base, options.fetch(:yaml_output, nil))
+  end
+
+  def generate_key_base
+    SlimKeyfy::Slimutils::BaseKeyGenerator.generate_key_base_from_path(@original_file_path, @extension)
+  end
+
+  def stream_mode
+    @content.each_with_index do |old_line, idx|
+      word = SlimKeyfy::Transformer::Word.new(old_line, @key_base, @extension)
+      new_line, translations = @transformer.new(word, @yaml_processor).transform
+      if translations_are_invalid?(translations)
+        @yaml_processor.delete_translations(translations)
+        update_with(idx, old_line)
+      else
+        process_new_line(idx, old_line, new_line, translations)
+        @changes = true
+      end
+    end
+    SlimKeyfy::Slimutils::FileWriter.write(@file_path, @new_content.join("\n"))
+    finalize!
+  end
+
+  def process_new_line(idx, old_line, new_line, translations)
+    SlimKeyfy::Console::Printer.difference(old_line, new_line, translations)
+    case SlimKeyfy::Console::IOAction.choose("Changes wanted?")
+      when "y" then update_with(idx, new_line)
+      when "n" then 
+        update_with(idx, old_line)
+        @yaml_processor.delete_translations(translations)
+      when "x" then 
+        update_with(idx, SlimKeyfy::Console::Printer.tag(old_line, translations, comment_tag))
+        @yaml_processor.delete_translations(translations)
+      when "a" then
+        SlimKeyfy::Slimutils::MFileUtils.restore(@bak_path, @original_file_path)
+        puts "Aborted!"
+        exit
+    end
+  end
+
+  def finalize!
+    if @changes then
+      if SlimKeyfy::Console::IOAction.yes_or_no?("Do you like to review your changes?") then
+        SlimKeyfy::Console::Printer.unix_diff(@bak_path, @original_file_path)
+      end
+      if SlimKeyfy::Console::IOAction.yes_or_no?("Do you like to save your changes?") then
+        @yaml_processor.store!
+        puts "Saved! at #{@original_file_path}"
+        SlimKeyfy::Slimutils::MFileUtils.rm(@bak_path) if @no_backup
+      else
+        SlimKeyfy::Slimutils::MFileUtils.restore(@bak_path, @original_file_path)
+        puts "Restored!"
+      end
+    else 
+      SlimKeyfy::Slimutils::MFileUtils.restore(@bak_path, @original_file_path)
+      puts "Nothing was changed!" 
+    end
+  end
+
+  def update_with(idx, line)
+    @new_content[idx] = line
+  end
+
+  def translations_are_invalid?(translations)
+    translations.nil? or translations.empty?
+  end
+
+  def comment_tag
+    @transformer.slim? ? "//" : "#"
+  end
+end

data/lib/slimkeyfy/console.rb

@@ -0,0 +1,8 @@
+module SlimKeyfy::Console
+
+end
+
+require 'slimkeyfy/console/printer'
+require 'slimkeyfy/console/io_action'
+require 'slimkeyfy/console/command_line'
+require 'slimkeyfy/console/translate'

data/lib/slimkeyfy/slimutils/file_utils.rb

@@ -0,0 +1,61 @@
+require 'find'
+require 'fileutils'
+
+class SlimKeyfy::Slimutils::MFileUtils
+  def self.restore(backup_path, original_file_path)
+    FileUtils.cp(backup_path, original_file_path)
+    FileUtils.rm(backup_path)
+  end
+  def self.rm(file_path)
+    FileUtils.rm(file_path)
+  end
+  def self.backup(input)
+    original_file_path = abs_path(input)
+    backup_path = "#{original_file_path}.bak"
+    FileUtils.cp(original_file_path, backup_path)
+    backup_path
+  end
+  def self.create_new_file(input)
+    new_file_path = self.abs_path(input)
+    SlimKeyfy::Slimutils::FileWriter.write(new_file_path, "")
+    new_file_path
+  end
+  def self.abs_path(file)
+    File.absolute_path(file)
+  end
+  def self.walk(arg, recursive=false)
+    directory = abs_path(arg)
+    files = []
+    if recursive then
+      Find.find(directory) do |f|
+        files << f if File.file?(f)
+      end
+    else
+      files = Dir["#{directory}/*"]
+    end
+    files
+  end
+  def self.file_extension(file_path)
+    file_path.split(".").last
+  end
+  def self.is_valid_extension?(file_path)
+    ext = self.file_extension(file_path)
+    return false if (ext.nil? or ext.empty?)
+    (ext.end_with?("slim") or ext.end_with?("rb"))
+  end
+end
+
+class SlimKeyfy::Slimutils::FileReader
+  def self.read(file)
+    File.read(File.expand_path(file))
+  end
+end
+
+class SlimKeyfy::Slimutils::FileWriter
+  def self.write(full_path, content)
+    File.open(full_path, "w+") { |f| f.write(content) }
+  end
+  def self.append(full_path, content)
+    open(full_path, 'a') { |f| f.puts content}
+  end
+end