twitter_cldr 1.9.1 → 2.0.0

data/Gemfile

@@ -4,6 +4,8 @@ gemspec
 
 group :development, :test do
   gem 'rake'
+  gem 'pry'
+  gem 'pry-nav'
 end
 
 group :development do
@@ -19,5 +21,10 @@ group :test do
   platform :mri_18 do
     gem 'rcov'
   end
+
+  platform :mri_19 do
+    gem 'simplecov'
+    gem 'launchy'
+  end
 end
 

data/History.txt

@@ -1,3 +1,14 @@
+== 2.0.0
+
+* Added locales ga, ta, gl, cy, sr, bg, ku, ro, lv, be, sq, sk, and bn.
+* Added additional date formats.
+* Upgraded to CLDR 22.1.
+* Imported currency symbols and formatting rules from CLDR.
+* Added support for short/long numbers (eg. 1M for 1,000,000).
+* Improved RCov/Simplecov support.
+* Added custom Hungarian plurals rule.
+* Added support for approximate timespans (relative times).
+
 === 1.9.1
 
 * Locale resources now exported without Unicode escape sequences.

data/README.md

@@ -79,28 +79,40 @@ TwitterCldr::Shared::Currencies.for_country("Canada")      # { :currency => "Dol
 TwitterCldr::Shared::Currencies.for_code("CAD")            # { :currency => "Dollar", :symbol => "$", :country => "Canada"}
 ```
 
+#### Short / Long Decimals
+
+In addition to formatting regular decimals, TwitterCLDR supports short and long decimals.  Short decimals abbreviate the notation for the appropriate power of ten, for example "1M" for 1,000,000 or "2K" for 2,000.  Long decimals include the full notation, for example "1 million" or "2 thousand".  Long and short decimals can be generated using the appropriate `to_` method:
+
+```ruby
+2337.localize.to_short_decimal.to_s     # 2K
+1337123.localize.to_short_decimal.to_s  # 1M
+
+2337.localize.to_long_decimal.to_s      # 2 thousand
+1337123.localize.to_long_decimal.to_s   # 1 million
+```
+
 ### Dates and Times
 
 `Date`, `Time`, and `DateTime` objects are supported:
 
 ```ruby
-DateTime.now.localize(:es).to_full_s                    # "lunes, 12 de diciembre de 2011 21:44:57 UTC -0800"
-DateTime.now.localize(:es).to_long_s                    # "12 de diciembre de 2011 21:44:57 -08:00"
-DateTime.now.localize(:es).to_medium_s                  # "12/12/2011 21:44:57"
-DateTime.now.localize(:es).to_short_s                   # "12/12/11 21:44"
+DateTime.now.localize(:es).to_full_s               # "lunes, 12 de diciembre de 2011 21:44:57 UTC -0800"
+DateTime.now.localize(:es).to_long_s               # "12 de diciembre de 2011 21:44:57 -08:00"
+DateTime.now.localize(:es).to_medium_s             # "12/12/2011 21:44:57"
+DateTime.now.localize(:es).to_short_s              # "12/12/11 21:44"
 
-Date.today.localize(:es).to_full_s                      # "lunes 12 de diciembre de 2011"
-Date.today.localize(:es).to_long_s                      # "12 de diciembre de 2011"
-Date.today.localize(:es).to_medium_s                    # "12/12/2011"
-Date.today.localize(:es).to_short_s                     # "12/12/11"
+Date.today.localize(:es).to_full_s                 # "lunes 12 de diciembre de 2011"
+Date.today.localize(:es).to_long_s                 # "12 de diciembre de 2011"
+Date.today.localize(:es).to_medium_s               # "12/12/2011"
+Date.today.localize(:es).to_short_s                # "12/12/11"
 
-Time.now.localize(:es).to_full_s                        # "21:44:57 UTC -0800"
-Time.now.localize(:es).to_long_s                        # "21:44:57 UTC"
-Time.now.localize(:es).to_medium_s                      # "21:44:57"
-Time.now.localize(:es).to_short_s                       # "21:44"
+Time.now.localize(:es).to_full_s                   # "21:44:57 UTC -0800"
+Time.now.localize(:es).to_long_s                   # "21:44:57 UTC"
+Time.now.localize(:es).to_medium_s                 # "21:44:57"
+Time.now.localize(:es).to_short_s                  # "21:44"
 ```
 
-The CLDR data set only includes 4 specific date formats, full, long, medium, and short, so you'll have to choose amongst them for the one that best fits your needs.  Yes, it's limiting, but the 4 formats get the job done most of the time :)
+The default CLDR data set only includes 4 date formats, full, long, medium, and short.  See below for a list of additional formats.
 
 Behind the scenes, these convenience methods are creating instances of `LocalizedDate`, `LocalizedTime`, and `LocalizedDateTime`.  You can do the same thing if you're feeling adventurous:
 
@@ -109,6 +121,62 @@ dt = TwitterCldr::Localized::LocalizedDateTime.new(DateTime.now, :es)
 dt.to_short_s  # ...etc
 ```
 
+#### Additional Date Formats
+
+Besides the default date formats, CLDR supports a number of additional ones.  The list of available formats varys for each locale.  To get a full list, use the `additional_formats_for` method:
+
+```ruby
+# ["EEEEd", "Ed", "GGGGyMd", "H", "Hm", "Hms", "M", "MEd", "MMM", "MMMEEEEd", "MMMEd", ... ] 
+TwitterCldr::Formatters::DateTimeFormatter.additional_formats_for(:ja)
+```
+
+You can use any of the returned formats as the `:format` option when creating new instances of `LocalizedDateTime` or `DateTimeFormatter`:
+
+```ruby
+# 2011/12/12 21:44:57
+DateTime.now.localize(:ja).to_s
+
+# 12日月曜日
+DateTime.now.localize(:ja).to_s(:format => "EEEEd")
+```
+
+It's important to know that, even though a format may not be available across locales, TwitterCLDR will do it's best to approximate if no exact match can be found.
+
+##### List of additional date format examples for English:
+
+| Format | Output           |
+|:-------|------------------|
+| EHm    | Wed 17:05        |
+| EHms   | Wed 17:05:33     |
+| Ed     | 28 Wed           |
+| Ehm    | Wed 5:05 p.m.    |
+| Ehms   | Wed 5:05:33 p.m. |
+| Gy     | 2012 AD          |
+| H      | 17               |
+| Hm     | 17:05            |
+| Hms    | 17:05:33         |
+| M      | 11               |
+| MEd    | Wed 11/28        |
+| MMM    | Nov              |
+| MMMEd  | Wed Nov 28       |
+| MMMd   | Nov 28           |
+| Md     | 11/28            |
+| d      | 28               |
+| h      | 5 p.m.           |
+| hm     | 5:05 p.m.        |
+| hms    | 5:05:33 p.m.     |
+| ms     | 05:33            |
+| y      | 2012             |
+| yM     | 11/2012          |
+| yMEd   | Wed 11/28/2012   |
+| yMMM   | Nov 2012         |
+| yMMMEd | Wed Nov 28 2012  |
+| yMMMd  | Nov 28 2012      |
+| yMd    | 11/28/2012       |
+| yQQQ   | Q4 2012          |
+| yQQQQ  | 4th quarter 2012 |
+
+
 #### Relative Dates and Times
 
 In addition to formatting full dates and times, TwitterCLDR supports relative time spans via several convenience methods and the `LocalizedTimespan` class.  TwitterCLDR tries to guess the best time unit (eg. days, hours, minutes, etc) based on the length of the time span.  Unless otherwise specified, TwitterCLDR will use the current date and time as the reference point for the calculation.
@@ -154,6 +222,14 @@ ts.to_s                         # Vor 1 Tag
 ts.to_s(:unit => :hour)         # Vor 24 Stunden
 ```
 
+By default, timespans are exact representations of a given unit of elapsed time.  TwitterCLDR also supports approximate timespans which round up to the nearest larger unit.  For example, "44 seconds" remains "44 seconds" while "45 seconds" becomes "1 minute".  To approximate, pass the `:approximate => true` option into `to_s`:
+
+```ruby
+TwitterCldr::Localized::LocalizedTimespan.new(44).to_s(:approximate => true)  # In 44 seconds
+TwitterCldr::Localized::LocalizedTimespan.new(45).to_s(:approximate => true)  # In 1 minute
+TwitterCldr::Localized::LocalizedTimespan.new(52).to_s(:approximate => true)  # In 1 minute
+```
+
 ### Lists
 
 TwitterCLDR supports formatting lists of strings as you might do in English by using commas, eg: "Apples, cherries, and oranges".  Use the `localize` method on an array followed by a call to `to_sentence`:
@@ -565,6 +641,10 @@ No external requirements.
 
 Tests are written in RSpec using RR as the mocking framework.
 
+## Test Coverage
+
+You can run the development test coverage suite with `bundle exec rake spec:cov`, or the full suite with `bundle exec rake spec:cov:full`.  TwitterCLDR uses RCov under Ruby 1.8 and Simplecov under Ruby 1.9.
+
 ## JavaScript Support
 
 TwitterCLDR currently supports localization of certain textual objects in JavaScript via the twitter-cldr-js gem.  See [http://github.com/twitter/twitter-cldr-js](http://github.com/twitter/twitter-cldr-js) for details.

data/Rakefile

@@ -32,11 +32,32 @@ namespace :spec do
 end
 
 if RUBY_VERSION < '1.9.0'
-  desc 'Run all examples with RCov'
-  RSpec::Core::RakeTask.new('spec:rcov') do |t|
-    t.rcov      = true
-    t.pattern   = './spec/**/*_spec.rb'
-    t.rcov_opts = %w(-T --sort coverage --exclude gems/,spec/)
+  desc 'Run specs with RCov'
+  RSpec::Core::RakeTask.new('spec:cov') do |t|
+    t.rcov       = true
+    t.pattern    = './spec/**/*_spec.rb'
+    t.rcov_opts  = '-T --sort coverage --exclude gems/,spec/'
+  end
+
+  desc 'Run full specs suit with RCov'
+  task 'spec:cov:full' => %w[spec:full_spec_env spec:cov]
+else
+  namespace :spec do
+    desc 'Run specs with SimpleCov'
+    task :cov => ['spec:simplecov_env', :spec] do
+      require 'launchy'
+      Launchy.open 'coverage/index.html'
+    end
+
+    desc 'Run full specs suit with SimpleCov'
+    task 'cov:full' => %w[spec:full_spec_env spec:cov]
+
+    task :simplecov_env do
+      puts 'Cleaning up coverage reports'
+      rm_rf 'coverage'
+
+      ENV['SCOV'] = 'true'
+    end
   end
 end
 
@@ -45,7 +66,7 @@ namespace :update do
   task :locales_resources, :cldr_path do |_, args|
     TwitterCldr::Resources::LocalesResourcesImporter.new(
         args[:cldr_path] || './vendor/cldr',
-        './resources/locales'
+        './resources'
     ).import
   end
 

data/lib/twitter_cldr/formatters.rb

@@ -7,20 +7,23 @@ module TwitterCldr
   module Formatters
     autoload :Base, 'twitter_cldr/formatters/base'
 
-    autoload :DateTimeFormatter, 'twitter_cldr/formatters/calendars/datetime_formatter'
-    autoload :DateFormatter,     'twitter_cldr/formatters/calendars/date_formatter'
-    autoload :TimeFormatter,     'twitter_cldr/formatters/calendars/time_formatter'
-    autoload :TimespanFormatter, 'twitter_cldr/formatters/calendars/timespan_formatter'
+    autoload :DateTimeFormatter,          'twitter_cldr/formatters/calendars/datetime_formatter'
+    autoload :DateFormatter,              'twitter_cldr/formatters/calendars/date_formatter'
+    autoload :TimeFormatter,              'twitter_cldr/formatters/calendars/time_formatter'
+    autoload :TimespanFormatter,          'twitter_cldr/formatters/calendars/timespan_formatter'
 
-    autoload :Numbers,           'twitter_cldr/formatters/numbers'
-    autoload :NumberFormatter,   'twitter_cldr/formatters/numbers/number_formatter'
-    autoload :DecimalFormatter,  'twitter_cldr/formatters/numbers/decimal_formatter'
-    autoload :CurrencyFormatter, 'twitter_cldr/formatters/numbers/currency_formatter'
-    autoload :PercentFormatter,  'twitter_cldr/formatters/numbers/percent_formatter'
+    autoload :Numbers,                    'twitter_cldr/formatters/numbers'
+    autoload :NumberFormatter,            'twitter_cldr/formatters/numbers/number_formatter'
+    autoload :AbbreviatedNumberFormatter, 'twitter_cldr/formatters/numbers/abbreviated/abbreviated_number_formatter'
+    autoload :DecimalFormatter,           'twitter_cldr/formatters/numbers/decimal_formatter'
+    autoload :ShortDecimalFormatter,      'twitter_cldr/formatters/numbers/abbreviated/short_decimal_formatter'
+    autoload :LongDecimalFormatter,       'twitter_cldr/formatters/numbers/abbreviated/long_decimal_formatter'
+    autoload :CurrencyFormatter,          'twitter_cldr/formatters/numbers/currency_formatter'
+    autoload :PercentFormatter,           'twitter_cldr/formatters/numbers/percent_formatter'
 
-    autoload :Plurals,           'twitter_cldr/formatters/plurals'
-    autoload :PluralFormatter,   'twitter_cldr/formatters/plurals/plural_formatter'
+    autoload :Plurals,                    'twitter_cldr/formatters/plurals'
+    autoload :PluralFormatter,            'twitter_cldr/formatters/plurals/plural_formatter'
 
-    autoload :ListFormatter,     'twitter_cldr/formatters/list_formatter'
+    autoload :ListFormatter,              'twitter_cldr/formatters/list_formatter'
   end
 end

data/lib/twitter_cldr/formatters/base.rb

@@ -15,7 +15,7 @@ module TwitterCldr
       protected
 
       def process_tokens(tokens, obj)
-        result = ''
+        result = ""
 
         tokens.each_with_index do |token, index|
           case token.type

data/lib/twitter_cldr/formatters/calendars/datetime_formatter.rb

@@ -48,16 +48,33 @@ module TwitterCldr
         self.send(METHODS[token.value[0].chr], date, token.value, token.value.size)
       end
 
+      def additional_format_selector
+        @tokenizer.additional_format_selector
+      end
+
+      def self.additional_formats_for(locale)
+        new(:locale => locale).additional_format_selector.patterns.map(&:to_s)
+      end
+
       protected
 
+      # there is incomplete era data in CLDR for certain locales like Hindi
+      # fall back if that happens
       def era(date, pattern, length)
         choices = case length
+          when 0
+            ["", ""]
           when 1..3
             @tokenizer.calendar[:eras][:abbr]
           else
             @tokenizer.calendar[:eras][:name]
         end
-        choices[date.year < 0 ? 0 : 1]
+
+        if result = choices[date.year < 0 ? 0 : 1]
+          result
+        else
+          era(date, pattern[0..-2], length - 1)
+        end
       end
 
       def year(date, pattern, length)
@@ -132,10 +149,8 @@ module TwitterCldr
           when 2
             date.month.to_s.rjust(length, '0')
           when 3
-            raise NotImplementedError, 'requires cldr\'s "multiple inheritance"'
             @tokenizer.calendar[:months][:'stand-alone'][:abbreviated][date.month]
           when 4
-            raise NotImplementedError, 'requires cldr\'s "multiple inheritance"'
             @tokenizer.calendar[:months][:'stand-alone'][:wide][date.month]
           when 5
             @tokenizer.calendar[:months][:'stand-alone'][:narrow][date.month]

data/lib/twitter_cldr/formatters/calendars/timespan_formatter.rb

@@ -8,15 +8,16 @@ module TwitterCldr
     class TimespanFormatter < Base
 
       DEFAULT_TYPE = :default
+      APPROXIMATE_MULTIPLIER = 0.75
 
       TIME_IN_SECONDS = {
-          :second => 1,
-          :minute => 60,
-          :hour   => 3600,
-          :day    => 86400,
-          :week   => 604800,
-          :month  => 2629743.83,
-          :year   => 31556926
+        :second => 1,
+        :minute => 60,
+        :hour   => 3600,
+        :day    => 86400,
+        :week   => 604800,
+        :month  => 2629743.83,
+        :year   => 31556926
       }
 
       def initialize(options = {})
@@ -24,10 +25,11 @@ module TwitterCldr
         @tokenizer = TwitterCldr::Tokenizers::TimespanTokenizer.new(:locale => extract_locale(options))
       end
 
-      def format(seconds, options = {})
+      def format(seconds, fmt_options = {})
+        options = fmt_options.dup
         options[:type]      ||= DEFAULT_TYPE
         options[:direction] ||= @direction || (seconds < 0 ? :ago : :until)
-        options[:unit]      ||= calculate_unit(seconds.abs)
+        options[:unit]      ||= calculate_unit(seconds.abs, options)
 
         options[:number] = calculate_time(seconds.abs, options[:unit])
 
@@ -36,22 +38,18 @@ module TwitterCldr
         strings.join.gsub(/\{[0-9]\}/, options[:number].to_s)
       end
 
-      def calculate_unit(seconds)
-        if seconds < 30
-          :second
-        elsif seconds < 2670
-          :minute
-        elsif seconds < 86369
-          :hour
-        elsif seconds < 604800
-          :day
-        elsif seconds < 2591969
-          :week
-        elsif seconds < 31556926
-          :month
-        else
-          :year
-        end
+      def calculate_unit(seconds, unit_options = {})
+        options = unit_options.dup
+        options[:approximate] = false if options[:approximate].nil?
+        multiplier = options[:approximate] ? APPROXIMATE_MULTIPLIER : 1
+
+        if seconds < (TIME_IN_SECONDS[:minute] * multiplier) then :second
+        elsif seconds < (TIME_IN_SECONDS[:hour] * multiplier) then :minute
+        elsif seconds < (TIME_IN_SECONDS[:day] * multiplier) then :hour
+        elsif seconds < (TIME_IN_SECONDS[:week] * multiplier) then :day
+        elsif seconds < (TIME_IN_SECONDS[:month] * multiplier) then :week
+        elsif seconds < (TIME_IN_SECONDS[:year] * multiplier) then :month
+        else :year end
       end
 
       # 0 <-> 29 secs                                                   # => seconds
@@ -61,7 +59,7 @@ module TwitterCldr
       # 29 days, 23 hrs, 59 mins, 29 secs <-> 1 yr minus 1 sec          # => months
       # 1 yr <-> max time or date                                       # => years
       def calculate_time(seconds, unit)
-        (seconds / TIME_IN_SECONDS[unit]).round.to_i
+        (seconds.to_f / TIME_IN_SECONDS[unit].to_f).round.to_i
       end
 
     end

data/lib/twitter_cldr/formatters/numbers/abbreviated/abbreviated_number_formatter.rb

@@ -0,0 +1,47 @@
+# encoding: UTF-8
+
+# Copyright 2012 Twitter, Inc
+# http://www.apache.org/licenses/LICENSE-2.0
+
+module TwitterCldr
+  module Formatters
+    class AbbreviatedNumberFormatter < NumberFormatter
+      NUMBER_MAX = 10 ** (TwitterCldr::Tokenizers::NumberTokenizer::ABBREVIATED_MAX_POWER + 1)
+      NUMBER_MIN = 10 ** TwitterCldr::Tokenizers::NumberTokenizer::ABBREVIATED_MIN_POWER
+
+      def default_format_options_for(number)
+        { :precision => 0 }
+      end
+
+      def get_tokens(obj, options = {})
+        type = (obj < NUMBER_MAX) && (obj >= NUMBER_MIN) ? get_type : :decimal
+        format = type == get_type ? get_key(obj) : nil
+        opts = options.merge(
+          :sign => obj.abs == obj ? :positive : :negative,
+          :type => type,
+          :format => format
+        )
+        @tokenizer.tokens(opts)
+      end
+
+      protected
+
+      def get_type
+        :decimal
+      end
+
+      def get_key(number)
+        "1#{"0" * (number.to_i.to_s.size - 1)}".to_i
+      end
+
+      def transform_number(number)
+        if (number < NUMBER_MAX) && (number >= NUMBER_MIN)
+          sig_figs = ((number.to_i.to_s.size - 1) % 3)
+          number.to_s[0..sig_figs].to_i
+        else
+          number
+        end
+      end
+    end
+  end
+end