twine 1.0.2 → 1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 0e7971931654800b4d398dd6c3d9e86b96e4f5ba
-  data.tar.gz: 64a1eccc6c604439e10c82bded869527b123f8db
+SHA256:
+  metadata.gz: d8667f50ad7147b8b7edac0d82ce542c2ecc973604c8334e878513b6acaa9a14
+  data.tar.gz: 1ceb2b2b18dea8a3585bd91a1b427b30ebd9c888817dcf7140b3d211cd9e157c
 SHA512:
-  metadata.gz: 808d4045076dfef7fcba6d26797eb05d7638fd3c2adc964c24059a20f378beb8c5d978f1598d651e1117a13a1fbdbdf102d21c767aee3a68c7eb5699fae7215a
-  data.tar.gz: a63755c14cbadb14c0bba2960ff178a5785280bbb72e300dba5dc449ea510c823c6e248a1b848306f7c5de532c658ef8d4f034e80347f58ec2a4525673cf1ab6
+  metadata.gz: a961953ec7b74510c10e17114b84c227bac43117a869ba4254d7410a16ea9413e4133a2c9f099c3d67922fd6e1632d841f366e79d28687e61f73861721dea43a
+  data.tar.gz: 954b69b0d6e0fadec38cf145e0e1772eb1fde0259337b45ffac172d41b6f6c031b33373c768cfbb48ad3e9cfb1565dc9e91db8e825023421fdc30098439a4d86

data/README.md

@@ -1,6 +1,6 @@
 # Twine
 
-[![Continuous Integration by CircleCI](https://circleci.com/gh/teespring/twine.svg?style=shield)](https://circleci.com/gh/teespring/twine)
+[![Continuous Integration by CircleCI](https://circleci.com/gh/scelis/twine.svg?style=shield)](https://circleci.com/gh/scelis/twine)
 
 Twine is a command line tool for managing your strings and their translations. These are all stored in a master text file and then Twine uses this file to import and export localization files in a variety of types, including iOS and Mac OS X `.strings` files, Android `.xml` files, gettext `.po` files, and [jquery-localize][jquerylocalize] `.json` files. This allows individuals and companies to easily share translations across multiple projects, as well as export localization files in any format the user wants.
 
@@ -80,7 +80,7 @@ Twine currently supports the following output formats:
 * [Android String Resources][androidstrings] (format: android)
     * HTML tags will be escaped by replacing `<` with `&lt`
     * Tags inside `<![CDATA[` won't be escaped.
-    * Supports [basic styling][androidstyling] with `<b>`, `<i>`, `<u>` and `<a>` links.
+    * Supports [basic styling][androidstyling] according to [Android documentation](https://developer.android.com/guide/topics/resources/string-resource.html#StylingWithHTML).  All of the documented tags are supported, in addition to `<a>` links.
 	    * These tags will *not* be escaped if the string doesn't contain placeholders. You can reference them directly in your layouts or by using [`getText()`](https://developer.android.com/reference/android/content/res/Resources.html#getText(int)) to read them programatically.
 	    * These tags *will* be escaped if the string contains placeholders. You can use [`getString()`](https://developer.android.com/reference/android/content/res/Resources.html#getString(int,%20java.lang.Object...)) combined with [`fromHtml`](https://developer.android.com/reference/android/text/Html.html#fromHtml(java.lang.String)) as shown in the [documentation][androidstyling] to display them.
 	* See [\#212](https://github.com/scelis/twine/issues/212) for details.
@@ -150,7 +150,7 @@ This command validates that the Twine data file can be parsed, contains no dupli
 The easiest way to create your first Twine data file is to run the [`consume-all-localization-files`](#consume-all-localization-files) command. The one caveat is to first create a blank file to use as your starting point. Then, just point the `consume-all-localization-files` command at a directory in your project containing all of your localization files.
 
 	$ touch twine.txt
-	$ twine consume-all-localization-files twine.txt Resources/Locales --developer-language en --consume-all --consume-comments
+	$ twine consume-all-localization-files twine.txt Resources/Locales --developer-language en --consume-all --consume-comments --format apple/android/gettext/jquery/django/tizen/flash
 
 ## Twine and Your Build Process
 
@@ -174,7 +174,10 @@ Now, whenever you build your application, Xcode will automatically invoke Twine
 
 ### Android Studio/Gradle
 
-Add the following task at the top level in app/build.gradle:
+#### Standard
+
+Add the following code to `app/build.gradle`:
+
 ```
 task generateLocalizations {
 	String script = 'if hash twine 2>/dev/null; then twine generate-localization-file twine.txt ./src/main/res/values/generated_strings.xml; fi'
@@ -183,10 +186,46 @@ task generateLocalizations {
 		args '-c', script
 	}
 }
+
+preBuild {
+	dependsOn generateLocalizations
+}
 ```
 
-Now every time you build your app the localization files are generated from the Twine file.
+#### Using [jruby](http://jruby.org)
 
+With this approach, developers do not need to manually install ruby, gem, or twine.
+
+Add the following code to `app/build.gradle`:
+
+```
+buildscript {
+	repositories { jcenter() }
+
+	dependencies {
+		/* NOTE: Set your prefered version of jruby here. */
+		classpath "com.github.jruby-gradle:jruby-gradle-plugin:1.5.0"
+	}
+}
+
+apply plugin: 'com.github.jruby-gradle.base'
+
+dependencies {
+	/* NOTE: Set your prefered version of twine here. */
+	jrubyExec 'rubygems:twine:1.0.3'
+}
+
+task generateLocalizations (type: JRubyExec) {
+	dependsOn jrubyPrepare
+	jrubyArgs '-S'
+	script "twine"
+	scriptArgs 'generate-localization-file', 'twine.txt', './src/main/res/values/generated_strings.xml'
+}
+
+preBuild {
+	dependsOn generateLocalizations
+}
+```
 
 ## User Interface
 
@@ -224,4 +263,4 @@ Many thanks to all of the contributors to the Twine project, including:
 [djangopo]: https://docs.djangoproject.com/en/dev/topics/i18n/translation/
 [tizen]: https://developer.tizen.org/documentation/articles/localization
 [flash]: http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/mx/resources/IResourceManager.html#getString()
-[printf]: https://en.wikipedia.org/wiki/Printf_format_string
+[printf]: https://en.wikipedia.org/wiki/Printf_format_string

data/lib/twine/cli.rb

@@ -45,6 +45,14 @@ module Twine
           files are UTF-16 without BOM, you need to specify if it's UTF-16LE or UTF16-BE.
         DESC
       },
+      escape_all_tags: {
+        switch: ['--[no-]escape-all-tags'],
+        description: <<-DESC,
+          Always escape all HTML tags. By default the Android formatter will ONLY escape styling tags, if a 
+          string also contains placeholders. This flag enforces that styling tags are escaped regardless of
+          placeholders.
+        DESC
+      },
       file_name: {
         switch: ['-n', '--file-name FILE_NAME'],
         description: 'This flag may be used to overwrite the default file name of the format.'
@@ -78,6 +86,10 @@ module Twine
         switch: ['-p', '--[no-]pedantic'],
         description: 'When validating a Twine file, perform additional checks that go beyond pure validity (like presence of tags).'
       },
+      quiet: {
+        switch: ['-q', '--[no-]quiet'],
+        description: 'Suppress all console output except error messages.'
+      },
       tags: {
         switch: ['-t', '--tags TAG1,TAG2,TAG3', Array],
         description: <<-DESC,
@@ -107,9 +119,11 @@ module Twine
         optional_options: [
           :developer_language,
           :encoding,
+          :escape_all_tags,
           :format,
           :include,
           :languages,
+          :quiet,
           :tags,
           :untagged,
           :validate
@@ -128,9 +142,11 @@ module Twine
           :create_folders,
           :developer_language,
           :encoding,
+          :escape_all_tags,
           :file_name,
           :format,
           :include,
+          :quiet,
           :tags,
           :untagged,
           :validate
@@ -146,7 +162,9 @@ module Twine
         optional_options: [
           :developer_language,
           :encoding,
+          :escape_all_tags,
           :include,
+          :quiet,
           :tags,
           :untagged,
           :validate
@@ -164,6 +182,7 @@ module Twine
           :format,
           :languages,
           :output_path,
+          :quiet,
           :tags
         ],
         option_validation: Proc.new { |options|
@@ -183,6 +202,7 @@ module Twine
           :encoding,
           :format,
           :output_path,
+          :quiet,
           :tags
         ],
         example: 'twine consume-all-localization-files twine.txt Resources/Locales/ --developer-language en --tags DefaultTag1,DefaultTag2'
@@ -197,6 +217,7 @@ module Twine
           :encoding,
           :format,
           :output_path,
+          :quiet,
           :tags
         ],
         example: 'twine consume-localization-archive twine.txt LocDrop5.zip'
@@ -206,7 +227,8 @@ module Twine
         arguments: [:twine_file],
         optional_options: [
           :developer_language,
-          :pedantic
+          :pedantic,
+          :quiet
         ],
         example: 'twine validate-twine-file twine.txt'
       }
@@ -227,7 +249,7 @@ module Twine
 
       mapped_command = DEPRECATED_COMMAND_MAPPINGS[command]
       if mapped_command
-        Twine::stderr.puts "WARNING: Twine commands names have changed. `#{command}` is now `#{mapped_command}`. The old command is deprecated and will soon stop working. For more information please check the documentation at https://github.com/mobiata/twine"
+        Twine::stdout.puts "WARNING: Twine commands names have changed. `#{command}` is now `#{mapped_command}`. The old command is deprecated and will soon stop working. For more information please check the documentation at https://github.com/mobiata/twine"
         command = mapped_command
       end
 

data/lib/twine/formatters/abstract.rb

@@ -3,6 +3,8 @@ require 'fileutils'
 module Twine
   module Formatters
     class Abstract
+      LANGUAGE_CODE_WITH_OPTIONAL_REGION_CODE = "[a-z]{2}(?:-[A-Za-z]{2})?"
+
       attr_accessor :twine_file
       attr_accessor :options
 
@@ -38,7 +40,7 @@ module Twine
             definition.translations[lang] = value
           end
         elsif @options[:consume_all]
-          Twine::stderr.puts "Adding new definition '#{key}' to twine file."
+          Twine::stdout.puts "Adding new definition '#{key}' to twine file."
           current_section = @twine_file.sections.find { |s| s.name == 'Uncategorized' }
           unless current_section
             current_section = TwineSection.new('Uncategorized')
@@ -54,7 +56,7 @@ module Twine
           @twine_file.definitions_by_key[key] = current_definition
           @twine_file.definitions_by_key[key].translations[lang] = value
         else
-          Twine::stderr.puts "Warning: '#{key}' not found in twine file."
+          Twine::stdout.puts "WARNING: '#{key}' not found in twine file."
         end
         if !@twine_file.language_codes.include?(lang)
           @twine_file.add_language_code(lang)
@@ -76,7 +78,12 @@ module Twine
       end
 
       def determine_language_given_path(path)
-        raise NotImplementedError.new("You must implement determine_language_given_path in your formatter class.")
+        only_language_and_region = /^#{LANGUAGE_CODE_WITH_OPTIONAL_REGION_CODE}$/i
+        basename = File.basename(path, File.extname(path))
+        return basename if basename =~ only_language_and_region
+        return basename if @twine_file.language_codes.include? basename
+        
+        path.split(File::SEPARATOR).reverse.find { |segment| segment =~ only_language_and_region }
       end
 
       def output_path_for_language(lang)

data/lib/twine/formatters/android.rb

@@ -37,11 +37,15 @@ module Twine
           end
         end
 
-        return
+        return super
       end
 
       def output_path_for_language(lang)
-        "values-#{lang}".gsub(/-(\p{Lu})/, '-r\1')
+        if lang == @twine_file.language_codes[0]
+          "values"
+        else
+          "values-#{lang}".gsub(/-(\p{Lu})/, '-r\1')
+        end
       end
 
       def set_translation_for_key(key, lang, value)
@@ -56,7 +60,7 @@ module Twine
 
       def read(io, lang)
         document = REXML::Document.new io, :compress_whitespace => %w{ string }
-
+        document.context[:attribute_quote] = :quote
         comment = nil
         document.root.children.each do |child|
           if child.is_a? REXML::Comment
@@ -67,7 +71,8 @@ module Twine
 
             key = child.attributes['name']
 
-            set_translation_for_key(key, lang, child.text)
+            content = child.children.map(&:to_s).join
+            set_translation_for_key(key, lang, content)
             set_comment_for_key(key, comment) if comment
 
             comment = nil
@@ -108,22 +113,26 @@ module Twine
 
       # http://developer.android.com/guide/topics/resources/string-resource.html#FormattingAndStyling
       def escape_value(value)
-        inside_cdata = /<\!\[CDATA\[((?!\]\]>).)*$/       # opening CDATA tag ('<![CDATA[') not followed by a closing tag (']]>')
-        inside_opening_anchor_tag = /<a\s?((?!>).)*$/     # anchor tag start ('<a ') not followed by a '>'
+        inside_cdata = /<\!\[CDATA\[((?!\]\]>).)*$/              # opening CDATA tag ('<![CDATA[') not followed by a closing tag (']]>')
+        inside_opening_tag = /<(a|font|span|p)\s?((?!>).)*$/     # tag start ('<a ', '<font ', '<span ' or '<p ') not followed by a '>'
 
         # escape double and single quotes and & signs
-        value = gsub_unless(value, '"', '\\"') { |substring| substring =~ inside_cdata || substring =~ inside_opening_anchor_tag }
+        value = gsub_unless(value, '"', '\\"') { |substring| substring =~ inside_cdata || substring =~ inside_opening_tag }
         value = gsub_unless(value, "'", "\\'") { |substring| substring =~ inside_cdata }
-        value = gsub_unless(value, /&/, '&amp;') { |substring| substring =~ inside_cdata || substring =~ inside_opening_anchor_tag }
+        value = gsub_unless(value, /&/, '&amp;') { |substring| substring =~ inside_cdata || substring =~ inside_opening_tag }
 
         # if `value` contains a placeholder, escape all angle brackets
         # if not, escape opening angle brackes unless it's a supported styling tag
         # https://github.com/scelis/twine/issues/212
         # https://stackoverflow.com/questions/3235131/#18199543
-        if number_of_twine_placeholders(value) > 0
-          angle_bracket = /<(?!(\/?(\!\[CDATA)))/           # matches all `<` but <![CDATA
-        else  
-          angle_bracket = /<(?!(\/?(b|u|i|a|\!\[CDATA)))/   # matches all `<` but <b>, <u>, <i>, <a> and <![CDATA
+        if number_of_twine_placeholders(value) > 0 or @options[:escape_all_tags]
+          # matches all `<` but <![CDATA
+          angle_bracket = /<(?!(\/?(\!\[CDATA)))/
+        else
+          # matches all '<' but <b>, <em>, <i>, <cite>, <dfn>, <big>, <small>, <font>, <tt>, <s>,
+          # <strike>, <del>, <u>, <super>, <sub>, <ul>, <li>, <br>, <div>, <span>, <p>, <a>
+          # and <![CDATA
+          angle_bracket = /<(?!(\/?(b|em|i|cite|dfn|big|small|font|tt|s|strike|del|u|super|sub|ul|li|br|div|span|p|a|\!\[CDATA)))/
         end
         value = gsub_unless(value, angle_bracket, '&lt;') { |substring| substring =~ inside_cdata }
 

data/lib/twine/formatters/apple.rb

@@ -30,7 +30,7 @@ module Twine
           end
         end
 
-        return
+        return super
       end
 
       def output_path_for_language(lang)

data/lib/twine/formatters/django.rb

@@ -1,5 +1,6 @@
 module Twine
   module Formatters
+    # For a description of the .po file format, see https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html
     class Django < Abstract
       def format_name
         'django'
@@ -13,22 +14,11 @@ module Twine
         'strings.po'
       end
 
-      def determine_language_given_path(path)
-        path_arr = path.split(File::SEPARATOR)
-        path_arr.each do |segment|
-          match = /([a-z]{2}(-[A-Za-z]{2})?)\.po$/.match(segment)
-          return match[1] if match
-        end
-        
-        return
-      end
-
       def read(io, lang)
-        comment_regex = /#\. *"?(.*)"?$/
-        key_regex = /msgid *"(.*)"$/
-        value_regex = /msgstr *"(.*)"$/m
+        comment_regex = /^\s*#\. *"?(.*)"?$/
+        key_regex = /^msgid *"(.*)"$/
+        value_regex = /^msgstr *"(.*)"$/m
 
-        last_comment = nil
         while line = io.gets          
           comment_match = comment_regex.match(line)
           if comment_match
@@ -64,11 +54,12 @@ module Twine
       end
 
       def format_header(lang)
-        "##\n # Django Strings File\n # Generated by Twine #{Twine::VERSION}\n # Language: #{lang}\nmsgid \"\"\nmsgstr \"\"\n\"Content-Type: text/plain; charset=UTF-8\\n\""
+        # see https://www.gnu.org/software/trans-coord/manual/gnun/html_node/PO-Header.html for details
+        "# Django Strings File\n# Generated by Twine #{Twine::VERSION}\n# Language: #{lang}\nmsgid \"\"\nmsgstr \"\"\n\"Content-Type: text/plain; charset=UTF-8\\n\""
       end
 
       def format_section_header(section)
-        "#--------- #{section.name} ---------#\n"
+        "# --------- #{section.name} --------- #\n"
       end
 
       def format_definition(definition, lang)

data/lib/twine/formatters/flash.rb

@@ -15,11 +15,6 @@ module Twine
         'resources.properties'
       end
 
-      def determine_language_given_path(path)
-        # match two-letter language code, optionally followed by a two letter region code
-        path.split(File::SEPARATOR).reverse.find { |segment| segment =~ /^([a-z]{2}(-[a-z]{2})?)$/i }
-      end
-
       def set_translation_for_key(key, lang, value)
         value = convert_placeholders_from_flash_to_twine(value)
         super(key, lang, value)

data/lib/twine/formatters/gettext.rb

@@ -15,16 +15,6 @@ module Twine
         'strings.po'
       end
 
-      def determine_language_given_path(path)
-        path_arr = path.split(File::SEPARATOR)
-        path_arr.each do |segment|
-          match = /([a-z]{2}(-[A-Za-z]{2})?)\.po$/.match(segment)
-          return match[1] if match
-        end
-
-        return
-      end
-
       def read(io, lang)
         comment_regex = /#.? *"(.*)"$/
         key_regex = /msgctxt *"(.*)"$/
@@ -65,7 +55,7 @@ module Twine
       end
 
       def format_header(lang)
-        "msgid \"\"\nmsgstr \"\"\n\"Language: #{lang}\\n\"\n\"X-Generator: Twine #{Twine::VERSION}\\n\"\n"
+        "msgid \"\"\nmsgstr \"\"\n\"Language: #{lang}\"\n\"X-Generator: Twine #{Twine::VERSION}\"\n"
       end
 
       def format_section_header(section)
@@ -86,15 +76,15 @@ module Twine
       end
 
       def format_key(key)
-        "msgctxt \"#{key}\"\n"
+        "msgctxt \"#{escape_quotes(key)}\"\n"
       end
 
       def format_base_translation(definition)
-        "msgid \"#{definition.translations[@default_lang]}\"\n"
+        "msgid \"#{escape_quotes(definition.translations[@default_lang])}\"\n"
       end
 
       def format_value(value)
-        "msgstr \"#{value}\"\n"
+        "msgstr \"#{escape_quotes(value)}\"\n"
       end
     end
   end

data/lib/twine/formatters/jquery.rb

@@ -14,22 +14,17 @@ module Twine
       end
 
       def determine_language_given_path(path)
-        path_arr = path.split(File::SEPARATOR)
-        path_arr.each do |segment|
-          match = /^((.+)-)?([^-]+)\.json$/.match(segment)
-          if match
-            return match[3]
-          end
-        end
+        match = /^.+-([^-]{2})\.json$/.match File.basename(path)
+        return match[1] if match
 
-        return
+        return super
       end
 
       def read(io, lang)
         begin
           require "json"
         rescue LoadError
-          raise Twine::Error.new "You must run 'gem install json' in order to read or write jquery-localize files."
+          raise Twine::Error.new "You must run `gem install json` in order to read or write jquery-localize files."
         end
 
         json = JSON.load(io)
@@ -46,7 +41,7 @@ module Twine
 
       def format_sections(twine_file, lang)
         sections = twine_file.sections.map { |section| format_section(section, lang) }
-        sections.delete_if &:empty?
+        sections.delete_if(&:empty?)
         sections.join(",\n\n")
       end