twitter_cldr 1.7.0 → 1.8.0

data/History.txt

@@ -1,3 +1,9 @@
+=== 1.8.0
+
+* Added support for language code conversion.
+* `#localize` methods (eg. for Hash, String, etc) now dynamically generated, part of the `TwitterCldr::Localized` namespace.
+* New convenience method `TwitterCldr::Normalization#normalize`.
+
 === 1.7.0
 
 * Wrote rake tasks to update CLDR and ICU resources.

data/README.md

@@ -1,4 +1,4 @@
-## twitter-cldr-rb [![Build Status](https://secure.travis-ci.org/twitter/twitter-cldr-rb.png?branch=master)](http://travis-ci.org/twitter/twitter-cldr-rb)
+## twitter-cldr-rb [![Build Status](https://secure.travis-ci.org/twitter/twitter-cldr-rb.png?branch=master)](http://travis-ci.org/twitter/twitter-cldr-rb) [![Code Climate](https://codeclimate.com/badge.png)](https://codeclimate.com/github/twitter/twitter-cldr-rb)
 
 TwitterCldr uses Unicode's Common Locale Data Repository (CLDR) to format certain types of text into their
 localized equivalents.  Currently supported types of text include dates, times, currencies, decimals, percentages, and symbols.
@@ -57,7 +57,7 @@ TwitterCldr patches core Ruby objects like `Fixnum` and `Date` to make localizat
 Behind the scenes, these convenience methods are creating instances of `LocalizedNumber`.  You can do the same thing if you're feeling adventurous:
 
 ```ruby
-num = TwitterCldr::LocalizedNumber.new(1337, :es)
+num = TwitterCldr::Localized::LocalizedNumber.new(1337, :es)
 num.to_currency.to_s  # ...etc
 ```
 
@@ -105,7 +105,7 @@ The CLDR data set only includes 4 specific date formats, full, long, medium, and
 Behind the scenes, these convenience methods are creating instances of `LocalizedDate`, `LocalizedTime`, and `LocalizedDateTime`.  You can do the same thing if you're feeling adventurous:
 
 ```ruby
-dt = TwitterCldr::LocalizedDateTime.new(DateTime.now, :es)
+dt = TwitterCldr::Localized::LocalizedDateTime.new(DateTime.now, :es)
 dt.to_short_s  # ...etc
 ```
 
@@ -145,11 +145,11 @@ Specify a different reference point for the time span calculation:
 Behind the scenes, these convenience methods are creating instances of `LocalizedTimespan`, whose constructor accepts a number of seconds as the first argument.  You can do the same thing if you're feeling adventurous:
 
 ```ruby
-ts = TwitterCldr::LocalizedTimespan.new(86400, :locale => :de)
+ts = TwitterCldr::Localized::LocalizedTimespan.new(86400, :locale => :de)
 ts.to_s                         # In 1 Tag
 ts.to_s(:unit => :hour)         # In 24 Stunden
 
-ts = TwitterCldr::LocalizedTimespan.new(-86400, :locale => :de)
+ts = TwitterCldr::Localized::LocalizedTimespan.new(-86400, :locale => :de)
 ts.to_s                         # Vor 1 Tag
 ts.to_s(:unit => :hour)         # Vor 24 Stunden
 ```
@@ -326,6 +326,62 @@ Get a list of supported territories by using the `#territories` method:
 TwitterCldr::Shared::PhoneCodes.territories  # [:zw, :an, :tr, :by, :mh, ...]
 ```
 
+### Language Codes
+
+Over the years, different standards for language codes have accumulated.  Probably the two most popular are ISO-639 and BCP-47 and their children.  TwitterCLDR provides a way to convert between these codes systematically.
+
+```ruby
+TwitterCldr::Shared::LanguageCodes.convert(:es, :from => :bcp_47, :to => :iso_639_2)  # :spa
+```
+
+Use the `standards_for` method to get the standards that are available for conversion from a given code.  In the example below, note that the first argument, `:es`, is the correct BCP-47 language code for Spanish, which is the second argument.  The return value comprises all the available conversions:
+
+```ruby
+# [:bcp_47, :iso_639_1, :iso_639_2, :iso_639_3]
+TwitterCldr::Shared::LanguageCodes.standards_for(:es, :bcp_47)
+```
+
+Get a list of supported standards for a full English language name:
+
+```ruby
+# [:bcp_47, :iso_639_1, :iso_639_2, :iso_639_3]
+TwitterCldr::Shared::LanguageCodes.standards_for_language(:Spanish)
+```
+
+Get a list of supported languages:
+
+```ruby
+TwitterCldr::Shared::LanguageCodes.languages  # [:Spanish, :German, :Norwegian, :Arabic ... ]
+```
+
+Determine valid standards:
+
+```ruby
+TwitterCldr::Shared::LanguageCodes.valid_standard?(:iso_639_1)  # true
+TwitterCldr::Shared::LanguageCodes.valid_standard?(:blarg)      # false
+```
+
+Determine valid codes:
+
+```ruby
+TwitterCldr::Shared::LanguageCodes.valid_code?(:es, :bcp_47)     # true
+TwitterCldr::Shared::LanguageCodes.valid_code?(:es, :iso_639_2)  # false
+```
+
+Convert the full English name of a language into a language code:
+
+```ruby
+TwitterCldr::Shared::LanguageCodes.from_language(:Spanish, :iso_639_2)  # :spa
+```
+
+Convert a language code into it's full English name:
+
+```ruby
+TwitterCldr::Shared::LanguageCodes.to_language(:spa, :iso_639_2)  # "Spanish"
+```
+
+**NOTE**: All of the functions in `TwitterCldr::Shared::LanguageCodes` accept both symbol and string parameters.
+
 ### Unicode Data
 
 TwitterCLDR provides ways to retrieve individual code points as well as normalize and decompose Unicode text.

data/Rakefile

@@ -40,79 +40,83 @@ if RUBY_VERSION < '1.9.0'
   end
 end
 
-namespace :resources do
-  namespace :update do
-
-    desc 'Import locales resources'
-    task :locales_resources, :cldr_path do |_, args|
-      TwitterCldr::Resources::LocalesResourcesImporter.new(
-          args[:cldr_path] || '../cldr',
-          './resources/locales'
-      ).import
-    end
+namespace :update do
+  desc 'Import locales resources'
+  task :locales_resources, :cldr_path do |_, args|
+    TwitterCldr::Resources::LocalesResourcesImporter.new(
+        args[:cldr_path] || './vendor/cldr',
+        './resources/locales'
+    ).import
+  end
 
-    desc 'Import custom locales resources'
-    task :custom_locales_resources do
-      TwitterCldr::Resources::CustomLocalesResourcesImporter.new('./resources/custom/locales').import
-    end
+  desc 'Import custom locales resources'
+  task :custom_locales_resources do
+    TwitterCldr::Resources::CustomLocalesResourcesImporter.new('./resources/custom/locales').import
+  end
 
-    desc 'Import tailoring resources from CLDR data (should be executed using JRuby 1.7 in 1.9 mode)'
-    task :tailoring, :cldr_path, :icu4j_jar_path do |_, args|
-      TwitterCldr::Resources::TailoringImporter.new(
-          args[:cldr_path] || '../cldr',
-          './resources/collation/tailoring',
-          args[:icu4j_jar_path] ||'../icu4j-49_1.jar'
-      ).import(TwitterCldr.supported_locales)
-    end
+  desc 'Import tailoring resources from CLDR data (should be executed using JRuby 1.7 in 1.9 mode)'
+  task :tailoring_data, :cldr_path, :icu4j_jar_path do |_, args|
+    TwitterCldr::Resources::TailoringImporter.new(
+        args[:cldr_path] || './vendor/cldr',
+        './resources/collation/tailoring',
+        args[:icu4j_jar_path] ||'./vendor/icu4j-49_1.jar'
+    ).import(TwitterCldr.supported_locales)
+  end
 
-    desc 'Import Unicode data resources'
-    task :unicode_data, :unicode_data_path do |_, args|
-      TwitterCldr::Resources::UnicodeDataImporter.new(
-          args[:unicode_data_path] || '../unicode-data',
-          './resources/unicode_data'
-      ).import
-    end
+  desc 'Import Unicode data resources'
+  task :unicode_data, :unicode_data_path do |_, args|
+    TwitterCldr::Resources::UnicodeDataImporter.new(
+        args[:unicode_data_path] || './vendor/unicode-data',
+        './resources/unicode_data'
+    ).import
+  end
 
-    desc 'Import composition exclusions resource'
-    task :composition_exclusions, :derived_normalization_props_path do |_, args|
-      TwitterCldr::Resources::CompositionExclusionsImporter.new(
-          args[:derived_normalization_props_path] || '../unicode-data/DerivedNormalizationProps.txt',
-          './resources/unicode_data'
-      ).import
-    end
+  desc 'Import composition exclusions resource'
+  task :composition_exclusions, :derived_normalization_props_path do |_, args|
+    TwitterCldr::Resources::CompositionExclusionsImporter.new(
+        args[:derived_normalization_props_path] || './vendor/unicode-data/DerivedNormalizationProps.txt',
+        './resources/unicode_data'
+    ).import
+  end
 
-    desc 'Import postal codes resource'
-    task :postal_codes, :cldr_path do |_, args|
-      TwitterCldr::Resources::PostalCodesImporter.new(
-          args[:cldr_path] || '../cldr',
-          './resources/shared'
-      ).import
-    end
+  desc 'Import postal codes resource'
+  task :postal_codes, :cldr_path do |_, args|
+    TwitterCldr::Resources::PostalCodesImporter.new(
+        args[:cldr_path] || './vendor/cldr',
+        './resources/shared'
+    ).import
+  end
 
-    desc 'Import phone codes resource'
-    task :phone_codes, :cldr_path do |_, args|
-      TwitterCldr::Resources::PhoneCodesImporter.new(
-          args[:cldr_path] || '../cldr',
-          './resources/shared'
-      ).import
-    end
+  desc 'Import phone codes resource'
+  task :phone_codes, :cldr_path do |_, args|
+    TwitterCldr::Resources::PhoneCodesImporter.new(
+        args[:cldr_path] || './vendor/cldr',
+        './resources/shared'
+    ).import
+  end
 
-    desc 'Update default and tailoring tries dumps'
-    task :tries do
-      TwitterCldr::Resources::TriesDumper.update_dumps
-    end
+  desc 'Import language codes'
+  task :language_codes, :language_codes_data do |_, args|
+    TwitterCldr::Resources::LanguageCodesImporter.new(
+        args[:language_codes_data] || './vendor/language-codes',
+        './resources/shared'
+    ).import
+  end
 
-    desc 'Update canonical compositions resource'
-    task :canonical_compositions do
-      TwitterCldr::Resources::CanonicalCompositionsUpdater.new('./resources/unicode_data').update
-    end
+  desc 'Update default and tailoring tries dumps'
+  task :collation_tries do
+    TwitterCldr::Resources::CollationTriesDumper.update_dumps
+  end
 
+  desc 'Update canonical compositions resource'
+  task :canonical_compositions do
+    TwitterCldr::Resources::CanonicalCompositionsUpdater.new('./resources/unicode_data').update
   end
 end
 
 namespace :js do
+  desc 'Build JS files'
   task :build do
-    require File.expand_path(File.join(File.dirname(__FILE__), %w[lib twitter_cldr]))
     TwitterCldr.require_js
     FileUtils.mkdir_p(TwitterCldr::Js.build_dir)
     TwitterCldr::Js.output_dir = File.expand_path(ENV["OUTPUT_DIR"])
@@ -120,8 +124,8 @@ namespace :js do
     TwitterCldr::Js.install
   end
 
+  desc 'Run JS specs'
   task :test do
-    require File.expand_path(File.join(File.dirname(__FILE__), %w[lib twitter_cldr]))
     TwitterCldr.require_js
     FileUtils.mkdir_p(TwitterCldr::Js.build_dir)
     TwitterCldr::Js.make(:locales => [:en])

data/js/lib/twitter_cldr_js.rb

@@ -3,8 +3,6 @@
 # Copyright 2012 Twitter, Inc
 # http://www.apache.org/licenses/LICENSE-2.0
 
-$:.push(File.dirname(__FILE__))
-
 require 'mustache'
 require 'uglifier'
 require 'jasmine-headless-webkit'

data/lib/twitter_cldr/core_ext.rb

@@ -1,12 +1,12 @@
-require 'twitter_cldr/core_ext/localized_object'
-require 'twitter_cldr/core_ext/calendars/datetime'
-require 'twitter_cldr/core_ext/calendars/date'
-require 'twitter_cldr/core_ext/calendars/time'
-require 'twitter_cldr/core_ext/calendars/timespan'
-require 'twitter_cldr/core_ext/numbers/localized_number'
-require 'twitter_cldr/core_ext/numbers/bignum'
-require 'twitter_cldr/core_ext/numbers/fixnum'
-require 'twitter_cldr/core_ext/numbers/float'
-require 'twitter_cldr/core_ext/symbol'
-require 'twitter_cldr/core_ext/string'
-require 'twitter_cldr/core_ext/array'
+# encoding: UTF-8
+
+# Copyright 2012 Twitter, Inc
+# http://www.apache.org/licenses/LICENSE-2.0
+
+[Bignum, Fixnum, Float].each do |klass|
+  TwitterCldr::Localized::LocalizedNumber.localize(klass)
+end
+
+[Array, Date, DateTime, String, Symbol, Time].each do |klass|
+  TwitterCldr::Localized::const_get("Localized#{klass}").localize(klass)
+end

data/lib/twitter_cldr/formatters/calendars/timespan_formatter.rb

@@ -6,16 +6,17 @@
 module TwitterCldr
   module Formatters
     class TimespanFormatter < Base
+
       DEFAULT_TYPE = :default
 
-      TIME_IN_SECONDS = { 
-        :second => 1,
-        :minute => 60,
-        :hour => 3600,
-        :day => 86400,
-        :week => 604800,
-        :month => 2629743.83,
-        :year => 31556926 
+      TIME_IN_SECONDS = {
+          :second => 1,
+          :minute => 60,
+          :hour   => 3600,
+          :day    => 86400,
+          :week   => 604800,
+          :month  => 2629743.83,
+          :year   => 31556926
       }
 
       def initialize(options = {})
@@ -24,13 +25,14 @@ module TwitterCldr
       end
 
       def format(seconds, options = {})
+        options[:type]      ||= DEFAULT_TYPE
         options[:direction] ||= @direction || (seconds < 0 ? :ago : :until)
-        options[:unit] ||= self.calculate_unit(seconds.abs)
+        options[:unit]      ||= calculate_unit(seconds.abs)
+
         options[:number] = calculate_time(seconds.abs, options[:unit])
-        options[:type] ||= DEFAULT_TYPE
 
         tokens = @tokenizer.tokens(options)
-        strings = tokens.map { |token| token[:value]}
+        strings = tokens.map { |token| token[:value] }
         strings.join.gsub(/\{[0-9]\}/, options[:number].to_s)
       end
 

data/lib/twitter_cldr/localized/localized_array.rb

@@ -0,0 +1,33 @@
+# encoding: UTF-8
+
+# Copyright 2012 Twitter, Inc
+# http://www.apache.org/licenses/LICENSE-2.0
+
+module TwitterCldr
+  module Localized
+
+    class LocalizedArray < LocalizedObject
+      def code_points_to_string
+        TwitterCldr::Utils::CodePoints.to_string(base_obj)
+      end
+
+      def sort
+        TwitterCldr::Collation::Collator.new(locale).sort(base_obj).localize
+      end
+
+      def sort!
+        TwitterCldr::Collation::Collator.new(locale).sort!(base_obj)
+        self
+      end
+
+      def formatter_const
+        nil
+      end
+
+      def to_a
+        @base_obj.dup
+      end
+    end
+
+  end
+end

data/lib/twitter_cldr/localized/localized_date.rb

@@ -0,0 +1,23 @@
+# encoding: UTF-8
+
+# Copyright 2012 Twitter, Inc
+# http://www.apache.org/licenses/LICENSE-2.0
+
+module TwitterCldr
+  module Localized
+
+    class LocalizedDate < LocalizedDateTime
+      def to_datetime(time)
+        time_obj = time.is_a?(LocalizedTime) ? time.base_obj : time
+        LocalizedDateTime.new(DateTime.parse("#{@base_obj.strftime("%Y-%m-%d")}T#{time_obj.strftime("%H:%M:%S%z")}"), @locale, :calendar_type => @calendar_type)
+      end
+
+      protected
+
+      def formatter_const
+        TwitterCldr::Formatters::DateFormatter
+      end
+    end
+
+  end
+end

data/lib/twitter_cldr/localized/localized_datetime.rb

@@ -0,0 +1,63 @@
+# encoding: UTF-8
+
+# Copyright 2012 Twitter, Inc
+# http://www.apache.org/licenses/LICENSE-2.0
+
+module TwitterCldr
+  module Localized
+
+    class LocalizedDateTime < LocalizedObject
+      attr_reader :calendar_type
+
+      def initialize(obj, locale, options = {})
+        super
+        @calendar_type = options[:calendar_type] || TwitterCldr::DEFAULT_CALENDAR_TYPE
+      end
+
+      TwitterCldr::Tokenizers::DateTimeTokenizer::VALID_TYPES.each do |format_type|
+        define_method "to_#{format_type}_s" do
+          @formatter.format(@base_obj, :type => format_type.to_sym)
+        end
+      end
+
+      def to_timespan(options = {})
+        base_time = options[:base_time] || Time.now
+        seconds = (self.to_time.base_obj.to_i - base_time.to_i).abs
+        TwitterCldr::Localized::LocalizedTimespan.new(seconds, options.merge(:locale => @locale, :direction => :none))
+      end
+
+      def ago(options = {})
+        base_time = options[:base_time] || Time.now
+        seconds = self.to_time.base_obj.to_i - base_time.to_i
+        raise ArgumentError.new('Start date is after end date. Consider using "until" function.') if seconds > 0
+        TwitterCldr::Localized::LocalizedTimespan.new(seconds, options.merge(:locale => @locale))
+      end
+
+      def until(options = {})
+        base_time = options[:base_time] || Time.now
+        seconds = self.to_time.base_obj.to_i - base_time.to_i
+        raise ArgumentError.new('End date is before start date. Consider using "ago" function.') if seconds < 0
+        TwitterCldr::Localized::LocalizedTimespan.new(seconds, options.merge(:locale => @locale))
+      end
+
+      def to_s
+        to_default_s
+      end
+
+      def to_date
+        LocalizedDate.new(Date.parse(@base_obj.strftime("%Y-%m-%dT%H:%M:%S%z")), @locale, :calendar_type => @calendar_type)
+      end
+
+      def to_time
+        LocalizedTime.new(Time.parse(@base_obj.strftime("%Y-%m-%dT%H:%M:%S%z")), @locale, :calendar_type => @calendar_type)
+      end
+
+      protected
+
+      def formatter_const
+        TwitterCldr::Formatters::DateTimeFormatter
+      end
+    end
+
+  end
+end