traduco 0.9.0

data/Gemfile

@@ -0,0 +1,2 @@
+source :rubygems
+gemspec

data/LICENSE

@@ -0,0 +1,33 @@
+Software License Agreement (BSD License)
+
+Copyright (c) 2013, Joseph Earl
+All rights reserved.
+
+Copyright (c) 2012, Mobiata, LLC
+All rights reserved.
+
+Redistribution and use of this software in source and binary forms, with or
+without modification, are permitted provided that the following conditions are
+met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+* Neither the name of the organization nor the names of its contributors may be
+  used to endorse or promote products derived from this software without
+  specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.md

@@ -0,0 +1,182 @@
+# Traduco
+
+Traduco is a command line tool for managing your strings and their translations. These strings are all stored in a master text file and then Traduco uses this file to import and export strings in a variety of file types, including iOS and Mac OS X `.strings` files, Android `.xml` files, gettext `.po` files, and [jquery-localize][jquerylocalize] `.json` files. This allows individuals and companies to easily share strings across multiple projects, as well as export strings in any format the user wants.
+
+## Install
+
+### As a Gem
+
+Traduco is most easily installed as a Gem.
+
+	$ gem install traduco
+
+### From Source
+
+You can also run Traduco directly from source. However, it requires [rubyzip][rubyzip] in order to create and read standard zip files.
+
+	$ gem install rubyzip
+	$ git clone git://github.com/mobiata/traduco.git
+	$ cd traduco
+	$ ./traduco --help
+
+Make sure you run the `traduco` executable at the root of the project as it properly sets up your Ruby library path. The `bin/traduco` executable does not.
+
+## String File Format
+
+Traduco stores all of its strings in a single file. The format of this file is a slight variant of the [Git][git] config file format, which itself is based on the old [Windows INI file][INI] format. The entire file is broken up into sections, which are created by placing the section name between two pairs of square brackets. Sections are optional, but they are a recommended way of breaking your strings into smaller, more manageable chunks.
+
+Each grouping section contains N string definitions. These string definitions start with the string key placed within a single pair of square brackets. This string definition then contains a number of key-value pairs, including a comment, a comma-separated list of tags (which are used by Traduco to select a subset of strings), and all of the translations.
+
+### Tags
+
+Tags are used by Traduco as a way to only work with a subset of your strings at any given point in time. Each string can be assigned zero or more tags which are separated by commas. Tags are optional, though highly recommended. You can get a list of all strings currently missing tags by executing the `generate-report` command.
+
+### Whitespace
+
+Whitepace in this file is mostly ignored. If you absolutely need to put spaces at the beginning or end of your translated string, you can wrap the entire string in a pair of `` ` `` characters. If your actual string needs to start *and* end with a grave accent, you can wrap it in another pair of `` ` `` characters. See the example, below.
+
+### Example
+
+	[[General]]
+		[yes]
+			en = Yes
+			es = Sí
+			fr = Oui
+			ja = はい
+		[no]
+			en = No
+			fr = Non
+			ja = いいえ
+	
+	[[Errors]]
+		[path_not_found_error]
+			en = The file '%@' could not be found.
+			tags = app1,app6
+			comment = An error describing when a path on the filesystem could not be found.
+		[network_unavailable_error]
+			en = The network is currently unavailable.
+			tags = app1
+			comment = An error describing when the device can not connect to the internet.
+	
+	[[Escaping Example]]
+		[list_item_separator]
+			en = `, `
+			tags = mytag
+			comment = A string that should be placed between multiple items in a list. For example: Red, Green, Blue
+		[grave_accent_quoted_string]
+			en = ``%@``
+			tags = myothertag
+			comment = This string will evaluate to `%@`.
+
+## Supported Output Formats
+
+Traduco currently supports the following formats for outputting strings:
+
+* [iOS and OS X String Resources][applestrings] (format: apple)
+* [Android String Resources][androidstrings] (format: android)
+* [Gettext PO Files][gettextpo] (format: gettext)
+* [jquery-localize Language Files][jquerylocalize] (format: jquery)
+
+If you would like to enable traduco to create language files in another format, create an appropriate formatter in `lib/traduco/formatters`.
+
+## Usage
+
+	Usage: traduco COMMAND STRINGS_FILE [INPUT_OR_OUTPUT_PATH] [--lang LANG1,LANG2...] [--tags TAG1,TAG2,TAG3...] [--format FORMAT]
+	
+### Commands
+
+#### `generate-string-file`
+
+This command creates an Apple or Android strings file from the master strings data file.
+
+	$ traduco generate-string-file /path/to/strings.txt values-ja.xml --tags common,app1
+	$ traduco generate-string-file /path/to/strings.txt Localizable.strings --lang ja --tags mytag
+	$ traduco generate-string-file /path/to/strings.txt all-english.strings --lang en
+
+#### `generate-all-string-files`
+
+This command is a convenient way to call `generate-string-file` multiple times. It uses standard Mac OS X, iOS, and Android conventions to figure out exactly which files to create given a parent directory. For example, if you point it to a parent directory containing `en.lproj`, `fr.lproj`, and `ja.lproj` subdirectories, Traduco will create a `Localizable.strings` file of the appropriate language in each of them. This is often the command you will want to execute during the build phase of your project.
+
+	$ traduco generate-all-string-files /path/to/strings.txt /path/to/project/locales/directory --tags common,app1
+
+#### `consume-string-file`
+
+This command slurps all of the strings from a `.strings` or `.xml` file and incorporates the translated text into the master strings data file. This is a simple way to incorporate any changes made to a single file by one of your translators. It will only identify strings that already exist in the master data file.
+
+	$ traduco consume-string-file /path/to/strings.txt fr.strings
+	$ traduco consume-string-file /path/to/strings.txt Localizable.strings --lang ja
+	$ traduco consume-string-file /path/to/strings.txt es.xml
+
+#### `consume-all-string-files`
+
+This command reads in a folder containing many `.strings` or `.xml` files. These files should be in a standard folder hierarchy so that traduco knows the language of each file. When combined with the `--developer-language`, `--consume-comments`, and `--consume-all` flags, this command is a great way to create your initial strings data file from an existing iOS or Android project. Just make sure that you create a blank strings.txt file, first!
+
+	$ traduco consume-all-string-files strings.txt Resources/Locales --developer-language en --consume-all --consume-comments
+
+#### `generate-loc-drop`
+
+This command is a convenient way to generate a zip file containing files created by the `generate-string-file` command. It is often used for creating a single zip containing a large number of strings in all languages which you can then hand off to your translation team.
+
+	$ traduco generate-loc-drop /path/to/strings.txt LocDrop1.zip
+	$ traduco generate-loc-drop /path/to/strings.txt LocDrop2.zip --lang en,fr,ja,ko --tags common,app1
+
+#### `consume-loc-drop`
+
+This command is a convenient way of taking a zip file and executing the `consume-string-file` command on each file within the archive. It is most often used to incorporate all of the changes made by the translation team after they have completed work on a localization drop.
+
+	$ traduco consume-loc-drop /path/to/strings.txt LocDrop2.zip
+
+#### `generate-report`
+
+This command gives you useful information about your strings. It will tell you how many strings you have, how many have been translated into each language, and whether your master strings data file has any duplicate string keys.
+
+	$ traduco generate-report /path/to/strings.txt
+
+## Creating Your First strings.txt File
+
+The easiest way to create your first strings.txt file is to run the `consume-all-string-files` command. The one caveat is to first create a blank strings.txt file to use as your starting point. Then, just point the `consume-all-string-files` command at a directory in your project containing all of your iOS, OS X, or Android strings files.
+
+	$ touch strings.txt
+	$ traduco consume-all-string-files strings.txt Resources/Locales --developer-language en --consume-all --consume-comments
+
+## Traduco and Your Build Process
+
+It is easy to incorporate Traduco right into your iOS and OS X app build processes.
+
+1. In your project folder, create all of the `.lproj` directories that you need. It does not really matter where they are. We tend to put them in `Resources/Locales/`.
+2. Run the `generate-all-string-files` command to create all of the string files you need in these directories. For example,
+
+		$ traduco generate-all-string-files strings.txt Resources/Locales/ --tags tag1,tag2
+
+	Make sure you point Traduco at your strings data file, the directory that contains all of your `.lproj` directories, and the tags that describe the strings you want to use for this project.
+3. Drag the `Resources/Locales/` directory to the Xcode project navigator so that Xcode knows to include all of these strings files in your build.
+4. In Xcode, navigate to the "Build Phases" tab of your target.
+5. Click on the "Add Build Phase" button and select "Add Run Script".
+6. Drag the new "Run Script" build phase up so that it runs earlier in the build process. It doesn't really matter where, as long as it happens before the resources are copied to your bundle.
+7. Edit your script to run the exact same command you ran in step (2) above.
+
+Now, whenever you build your application, Xcode will automatically invoke Traduco to make sure that your `.strings` files are up-to-date.
+
+## User Interface
+
+* [Traduco TextMate 2 Bundle](https://github.com/mobiata/traduco.tmbundle) — This [TextMate 2](https://github.com/textmate/textmate) bundle will make it easier for you to work with Traduco strings files. In particular, it lets you use code folding to easily collapse and expand both strings and sections.
+* [traduco_ui](https://github.com/Daij-Djan/traduco_ui) — A user interface for Traduco written by [Dominik Pich](https://github.com/Daij-Djan/). Consider using this if you would prefer to use Traduco without dropping to a command line.
+
+## Contributors
+
+Many thanks to all of the contributors to the Traduco project, including:
+
+* [Ishitoya Kentaro](https://github.com/kent013)
+* [Kevin Everets](https://github.com/keverets)
+* [Kevin Wood](https://github.com/kwood)
+* [Mohammad Hejazi](https://github.com/MohammadHejazi)
+* [Robert Guo](http://www.robertguo.me/)
+
+
+[rubyzip]: http://rubygems.org/gems/rubyzip
+[git]: http://git-scm.org/
+[INI]: http://en.wikipedia.org/wiki/INI_file
+[applestrings]: http://developer.apple.com/documentation/Cocoa/Conceptual/LoadingResources/Strings/Strings.html
+[androidstrings]: http://developer.android.com/guide/topics/resources/string-resource.html
+[gettextpo]: http://www.gnu.org/savannah-checkouts/gnu/gettext/manual/html_node/PO-Files.html
+[jquerylocalize]: https://github.com/coderifous/jquery-localize

data/bin/traduco

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+require 'twine'
+begin
+  Twine::Runner.run(ARGV)
+rescue Twine::Error => e
+  STDERR.puts e.message
+  exit
+end

data/library/traduco.rb

@@ -0,0 +1,10 @@
+module Traduco
+  class Error < StandardError
+  end
+
+  require 'traduco/cli'
+  require 'traduco/encoding'
+  require 'traduco/formatters'
+  require 'traduco/runner'
+  require 'traduco/l10nfile'
+end

data/library/traduco/cli.rb

@@ -0,0 +1,186 @@
+require 'optparse'
+
+module Traduco
+  class CLI
+    def initialize(args, options)
+      @options = options
+      @args = args
+    end
+
+    def self.parse_args(args, options)
+      new(args, options).parse_args
+    end
+
+    def parse_args
+      parser = OptionParser.new do |opts|
+        opts.banner = 'Usage: twine COMMAND STRINGS_FILE [INPUT_OR_OUTPUT_PATH] [--lang LANG1,LANG2...] [--tags TAG1,TAG2,TAG3...] [--format FORMAT]'
+        opts.separator ''
+        opts.separator 'The purpose of this script is to convert back and forth between multiple data formats, allowing us to treat our strings (and translations) as data stored in a text file. We can then use the data file to create drops for the localization team, consume similar drops returned by the localization team, generate reports on the strings, as well as create formatted string files to ship with your products. Traduco currently supports iOS, OS X, Android, gettext, and jquery-localize string files.'
+        opts.separator ''
+        opts.separator 'Commands:'
+        opts.separator ''
+        opts.separator 'generate-string-file -- Generates a string file in a certain LANGUAGE given a particular FORMAT. This script will attempt to guess both the language and the format given the filename and extension. For example, "ko.xml" will generate a Korean language file for Android.'
+        opts.separator ''
+        opts.separator 'generate-all-string-files -- Generates all the string files necessary for a given project. The parent directory to all of the locale-specific directories in your project should be specified as the INPUT_OR_OUTPUT_PATH. This command will most often be executed by your build script so that each build always contains the most recent strings.'
+        opts.separator ''
+        opts.separator 'consume-string-file -- Slurps all of the strings from a translated strings file into the specified STRINGS_FILE. If you have some files returned to you by your translators you can use this command to incorporate all of their changes. This script will attempt to guess both the language and the format given the filename and extension. For example, "ja.strings" will assume that the file is a Japanese iOS strings file.'
+        opts.separator ''
+        opts.separator 'consume-all-string-files -- Slurps all of the strings from a directory into the specified STRINGS_FILE. If you have some files returned to you by your translators you can use this command to incorporate all of their changes. This script will attempt to guess both the language and the format given the filename and extension. For example, "ja.strings" will assume that the file is a Japanese iOS strings file.'
+        opts.separator ''
+        opts.separator 'generate-loc-drop -- Generates a zip archive of strings files in any format. The purpose of this command is to create a very simple archive that can be handed off to a translation team. The translation team can unzip the archive, translate all of the strings in the archived files, zip everything back up, and then hand that final archive back to be consumed by the consume-loc-drop command. This command assumes that --include-untranslated has been specified on the command line.'
+        opts.separator ''
+        opts.separator 'consume-loc-drop -- Consumes an archive of translated files. This archive should be in the same format as the one created by the generate-loc-drop command.'
+        opts.separator ''
+        opts.separator 'generate-report -- Generates a report containing data about your strings. For example, it will tell you if you have any duplicate strings or if any of your strings are missing tags. In addition, it will tell you how many strings you have and how many of those strings have been translated into each language.'
+        opts.separator ''
+        opts.separator 'General Options:'
+        opts.separator ''
+        opts.on('-l', '--lang LANGUAGES', Array, 'The language code(s) to use for the specified action.') do |langs|
+          @options[:languages] = langs
+        end
+        opts.on('-t', '--tags TAGS', Array, 'The tag(s) to use for the specified action. Only strings with that tag will be processed. Do not specify any tags to match all strings in the strings data file.') do |tags|
+          @options[:tags] = tags
+        end
+        opts.on('-u', '--untagged', 'If you have specified tags using the --tags flag, then only those tags will be selected. If you also want to select all strings that are untagged, then you can specify this option to do so.') do |u|
+          @options[:untagged] = true
+        end
+        formats = []
+        Formatters::FORMATTERS.each do |formatter|
+          formats << formatter::FORMAT_NAME
+        end
+        opts.on('-f', '--format FORMAT', "The file format to read or write (#{formats.join(', ')}). Additional formatters can be placed in the formats/ directory.") do |format|
+          lformat = format.downcase
+          if !formats.include?(lformat)
+            STDERR.puts "Invalid format: #{format}"
+          end
+          @options[:format] = lformat
+        end
+        opts.on('-a', '--consume-all', 'Normally, when consuming a string file, Traduco will ignore any string keys that do not exist in your master file.') do |a|
+          @options[:consume_all] = true
+        end
+        opts.on('-s', '--include-untranslated', 'This flag will cause any Android string files that are generated to include strings that have not yet been translated for the current language.') do |s|
+          @options[:include_untranslated] = true
+        end
+        opts.on('-o', '--output-file OUTPUT_FILE', 'Write the new strings database to this file instead of replacing the original file. This flag is only useful when running the consume-string-file or consume-loc-drop commands.') do |o|
+          @options[:output_path] = o
+        end
+        opts.on('-n', '--file-name FILE_NAME', 'When running the generate-all-string-files command, this flag may be used to overwrite the default file name of the format.') do |n|
+          @options[:file_name] = n
+        end
+        opts.on('-d', '--developer-language LANG', 'When writing the strings data file, set the specified language as the "developer language". In practice, this just means that this language will appear first in the strings data file.') do |d|
+          @options[:developer_language] = d
+        end
+        opts.on('-c', '--consume-comments', 'Normally, when consuming a string file, Traduco will ignore all comments in the file. With this flag set, any comments encountered will be read and parsed into the strings data file. This is especially useful when creating your first strings data file from an existing project.') do |c|
+          @options[:consume_comments] = true
+        end
+        opts.on('-e', '--encoding ENCODING', 'Traduco defaults to encoding all output files in UTF-8. This flag will tell Traduco to use an alternate encoding for these files. For example, you could use this to write Apple .strings files in UTF-16. This flag currently only works with Apple .strings files and is currently only supported in Ruby 1.9.3 or greater.') do |e|
+          if !"".respond_to?(:encode)
+            raise Traduco::Error.new "The --encoding flag is only supported on Ruby 1.9.3 or greater."
+          end
+          @options[:output_encoding] = e
+        end
+        opts.on('-h', '--help', 'Show this message.') do |h|
+          puts opts.help
+          exit
+        end
+        opts.on('--version', 'Print the version number and exit.') do |x|
+          puts "Traduco version #{Traduco::VERSION}"
+          exit
+        end
+        opts.separator ''
+        opts.separator 'Examples:'
+        opts.separator ''
+        opts.separator '> twine generate-string-file strings.txt ko.xml --tags FT'
+        opts.separator '> twine generate-all-string-files strings.txt Resources/Locales/ --tags FT,FB'
+        opts.separator '> twine consume-string-file strings.txt ja.strings'
+        opts.separator '> twine consume-all-string-files strings.txt Resources/Locales/ --developer-language en'
+        opts.separator '> twine generate-loc-drop strings.txt LocDrop5.zip --tags FT,FB --format android --lang de,en,en-GB,ja,ko'
+        opts.separator '> twine consume-loc-drop strings.txt LocDrop5.zip'
+        opts.separator '> twine generate-report strings.txt'
+      end
+      parser.parse! @args
+
+      if @args.length == 0
+        puts parser.help
+        exit
+      end
+
+      @options[:command] = @args[0]
+
+      if !VALID_COMMANDS.include? @options[:command]
+        raise Traduco::Error.new "Invalid command: #{@options[:command]}"
+      end
+
+      if @args.length == 1
+        raise Traduco::Error.new 'You must specify your strings file.'
+      end
+
+      @options[:strings_file] = @args[1]
+
+      case @options[:command]
+      when 'generate-string-file'
+        if @args.length == 3
+          @options[:output_path] = @args[2]
+        elsif @args.length > 3
+          raise Traduco::Error.new "Unknown argument: #{@args[3]}"
+        else
+          raise Traduco::Error.new 'Not enough arguments.'
+        end
+        if @options[:languages] and @options[:languages].length > 1
+          raise Traduco::Error.new 'Please only specify a single language for the generate-string-file command.'
+        end
+      when 'generate-all-string-files'
+        if ARGV.length == 3
+          @options[:output_path] = @args[2]
+        elsif @args.length > 3
+          raise Traduco::Error.new "Unknown argument: #{@args[3]}"
+        else
+          raise Traduco::Error.new 'Not enough arguments.'
+        end
+      when 'consume-string-file'
+        if @args.length == 3
+          @options[:input_path] = @args[2]
+        elsif @args.length > 3
+          raise Traduco::Error.new "Unknown argument: #{@args[3]}"
+        else
+          raise Traduco::Error.new 'Not enough arguments.'
+        end
+        if @options[:languages] and @options[:languages].length > 1
+          raise Traduco::Error.new 'Please only specify a single language for the consume-string-file command.'
+        end
+      when 'consume-all-string-files'
+        if @args.length == 3
+          @options[:input_path] = @args[2]
+        elsif @args.length > 3
+          raise Traduco::Error.new "Unknown argument: #{@args[3]}"
+        else
+          raise Traduco::Error.new 'Not enough arguments.'
+        end
+      when 'generate-loc-drop'
+        @options[:include_untranslated] = true
+        if @args.length == 3
+          @options[:output_path] = @args[2]
+        elsif @args.length > 3
+          raise Traduco::Error.new "Unknown argument: #{@args[3]}"
+        else
+          raise Traduco::Error.new 'Not enough arguments.'
+        end
+        if !@options[:format]
+          raise Traduco::Error.new 'You must specify a format.'
+        end
+      when 'consume-loc-drop'
+        if @args.length == 3
+          @options[:input_path] = @args[2]
+        elsif @args.length > 3
+          raise Traduco::Error.new "Unknown argument: #{@args[3]}"
+        else
+          raise Traduco::Error.new 'Not enough arguments.'
+        end
+      when 'generate-report'
+        if @args.length > 2
+          raise Traduco::Error.new "Unknown argument: #{@args[2]}"
+        end
+      end
+    end
+  end
+end

data/library/traduco/encoding.rb

@@ -0,0 +1,20 @@
+module Traduco
+  module Encoding
+    def self.encoding_for_path path
+      File.open(path, 'rb') do |f|
+        begin
+          a = f.readbyte
+          b = f.readbyte
+          if (a == 0xfe && b == 0xff)
+            return 'UTF-16BE'
+          elsif (a == 0xff && b == 0xfe)
+            return 'UTF-16LE'
+          end
+        rescue EOFError
+        end
+      end
+
+      'UTF-8'
+    end
+  end
+end