ruby-cldr 0.0.1

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2009 Sven Fuchs
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.textile

@@ -0,0 +1,104 @@
+h1. Ruby library for exporting and using data from CLDR
+
+CLDR ("Common Locale Data Repository":http://cldr.unicode.org) contains tons of high-quality locale data such as formatting rules for dates, times, numbers, currencies as well as language, country, calendar-specific names etc.
+
+For localizing applications in Ruby we'll obviously be able to use this incredibly comprehensive and well-maintained resource.
+
+This library is a first stab at that goal. You can: 
+
+* export CLDR data to YAML and Ruby formatted files which are supposed to be usable in an  "I18n":http://github.com/svenfuchs/i18n context but might be usable elsewhere, too. 
+* use CLDR compliant formatters for the following types: number, percentage, currency, date, time, datetime.
+
+h2. Requirements
+
+  * Ruby 1.9 (if you want well-ordered Hashes to be exported)
+  * Thor
+
+h2. Installation
+
+<pre>
+gem install cldr (not there, yet. install locally)
+thor install http://github.com/svenfuchs/cldr/raw/master/cldr.thor --as=cldr
+</pre>
+
+h2. Export
+
+The following command will export all known components from all locales to the target directory ./data/[locale]/[component].{yml,rb}:
+
+<pre>
+$ thor cldr:export
+</pre>
+
+You can also optionally specify locales and/or components to export as well as the target directory:
+
+<pre>
+$ thor cldr:export --locales de fr en --components numbers plurals --target=./tmp/export
+</pre>
+
+This will export the components :numbers and :plurals from the locales :de, :fr and :en to the same target directory.
+
+Also note that CLDR natively builds on a locale fallback concept where all locales eventually fall back to a :root locale. E.g. the :de-AT locale only contains a single format for numbers, which means that an application is supposed to use other formats from the :de locale (fallback). Particular bits of information are only present in the :root locale where all locales fall back to eventually.
+
+By default this library just exports data that is present in CLDR for a given locale. If you do not want to use locale fallbacks in your application you'll need to "flatten" locale fallbacks and merge the data during export time. To do that you can use the --merge option:
+
+<pre>
+$ thor cldr:export --merge
+</pre>
+
+h2. Formatters
+
+The library includes a bunch of formatter classes that can be used to format Ruby objects like Numerics, Date, Time, DateTime etc. using the format information provided by CLDR.
+
+E.g.:
+
+<pre>
+	options = { :decimal => ',', :group => ' ' }
+	format  = Cldr::Format::Numeric.new('#,##0.##', options)
+	format.apply(1234.567)
+	# => "1 234,57"
+
+	calendar = Cldr::Data::Calendars.new(:de)[:calendars][:gregorian]
+	format   = Cldr::Format::Date.new('EEEE, d. MMMM y', calendar)
+	format.apply(Date.new(2010, 1, 11))
+	# => "Montag, 11. Januar 2010"
+
+	calendar = Cldr::Data::Calendars.new(:de)[:calendars][:gregorian]
+	format   = Cldr::Format::Time.new('HH:mm:ss z', calendar)
+	format.apply(Time.utc(2010, 1, 1, 13, 12, 11))
+	# => "13:12:11 UTC"
+</pre>
+
+In order to make these things easier to use the library provides a bunch of helpers defined in the module Cldr::Format. (This module is supposed to work as an extension to the Simple backend in the I18n gem but is included here to suggest a common API to formatters and make development easier for usecases outside of the I18n gem. If you want to include this module somewhere else you have to implement a few abstract methods to provide translations.)
+
+E.g.:
+
+<pre>
+	format(:de, 123456.78)
+	# => "123.456,78"
+
+	format(:de, 123456.78, :as => :percent)
+	# => "123.457 %"
+
+	format(:de, 123456.78, :currency => 'EUR') 
+	# => "123.456,78 EUR"
+
+	format(:de, Date.new(2010, 1, 1), :format => :full)
+	# => "Freitag, 1. Januar 2010"
+
+	format(:de, Time.utc(2010, 1, 1, 13, 15, 17), :format => :long)
+	# => "13:15:17 UTC"
+
+	format(:de, DateTime.new(2010, 11, 12, 13, 14, 15), :format => :long)
+	# => "12. November 2010 13:14:15 +00:00"
+
+	format(:de, DateTime.new(2010, 11, 12, 13, 14, 15), :date_format => :long, :time_format => :short)
+	# => "12. November 2010 13:14"
+</pre>
+
+h2. Resources
+
+For additional information on CLDR plural rules see:
+
+  * "http://unicode.org/draft/reports/tr35/tr35.html#Language_Plural_Rules":http://unicode.org/draft/reports/tr35/tr35.html#Language_Plural_Rules
+  * "http://www.unicode.org/cldr/data/charts/supplemental/language_plural_rules.html":http://www.unicode.org/cldr/data/charts/supplemental/language_plural_rules.html
+

data/Rakefile

@@ -0,0 +1,18 @@
+require 'rubygems'
+require 'rake'
+
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gem|
+    gem.name = "ruby-cldr"
+    gem.summary = %Q{Ruby library for exporting and using data from CLDR }
+    gem.description = %Q{Ruby library for exporting and using data from CLDR, see http://cldr.unicode.org}
+    gem.email = "svenfuchs@artweb-design.de"
+    gem.homepage = "http://github.com/svenfuchs/ruby-cldr"
+    gem.authors = ["Sven Fuchs"]
+    gem.files =  FileList["*.thor", "[A-Z]*", "{lib,test}/**/*"]
+  end
+  Jeweler::GemcutterTasks.new
+rescue LoadError
+  puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
+end

data/TODO

@@ -0,0 +1,68 @@
+- during export resolve links if :merge => true
+- make sense of timezone data
+- hook into Rails' select_month and DateTime helpers
+
+extract:
+
+- territoryContainment (groups territories)
+- languageData (maps languages/scripts to territories)
+- territoryInfo (?)
+- calendarData
+- weekData
+
+what do the following tags in supplementalMetadata mean?
+- variable
+- defaultContent
+
+
+Instead of exporting YAML, maybe by default export Ruby classes:
+
+	module Cldr
+		module Data
+			class De::Numbers
+				def currency_format
+					"#,##0.00 ¤"
+				end
+				
+				def symbols
+					{ ... }
+				end
+			end
+			
+			class De::Calendars::Gregorian
+				def months(context, type)
+					case context
+					when :format
+						case type
+						when :wide
+						  { :sat => "Samstag", :sun => "Sonntag", :mon => "Montag", ...}
+						end
+					when :'stand-alone'
+						case type
+						when :narrow
+							{ ... }
+						else
+						  months(:format, :wide)
+						end
+					end
+				end
+				
+				def date_format(type)
+					case type
+					when :full
+						"EEEE, d. MMMM y"
+					when :medium
+						"dd.MM.yyyy"
+					else
+						date_format(:medium)
+					end
+				end
+			end
+
+			class DeAt::Numbers < De::Numbers
+			end
+
+			class DeAt::Calendars::Gregorian < De::Calendars::Gregorian
+			end
+		end
+	end

data/VERSION

@@ -0,0 +1 @@
+0.0.1

data/cldr.thor

@@ -0,0 +1,5 @@
+$:.unshift(File.dirname(__FILE__) + '/lib')
+
+require 'rubygems'
+require 'thor'
+require 'cldr/thor'

data/lib/cldr/data/base.rb

@@ -0,0 +1,66 @@
+require 'core_ext/string/underscore'
+require 'core_ext/hash/deep_merge'
+require 'nokogiri'
+
+class Cldr
+  module Data
+    class Base < Hash
+      attr_reader :locale
+
+      def initialize(*args)
+        hash    = args.last.is_a?(Hash) ? args.pop : {}
+        @locale = args.pop.to_sym unless args.empty?
+      end
+
+      def []=(keys, value)
+        return if value.nil? || value.respond_to?(:empty?) && value.empty?
+
+        keys = keys.to_s.split('.')
+        last_key = keys.pop.to_sym
+
+        target = keys.inject(self) { |target, key| target[key.to_sym] || target.store(key.to_sym, {}) }
+        target.store(last_key, value)
+      end
+      
+      def update(hash)
+        hash.each { |key, value| self[key] = value }
+      end
+
+      protected
+
+        def plural?(node)
+          !!node.attribute('count')
+        end
+
+        def draft?(node)
+          draft = node.attribute('draft')
+          draft && draft.value == 'unconfirmed'
+        end
+
+        def name(node)
+          node.name.underscore
+        end
+
+        def count(node)
+          node.attribute('count').value
+        end
+
+        def select(*sources)
+          doc.xpath(xpath(sources))
+        end
+        
+        def xpath(sources)
+          path = sources.map { |source| source.respond_to?(:path) ? source.path : source }.join('/')
+          path =~ /^\/?\/ldml/ ? path : "//ldml/#{path}"
+        end
+
+        def doc
+          @doc ||= Nokogiri::XML(File.read(path))
+        end
+
+        def path
+          @path ||= "#{Cldr::Data.dir}/main/#{locale.to_s.gsub('-', '_')}.xml"
+        end
+    end
+  end
+end

data/lib/cldr/data/calendars/gregorian.rb

@@ -0,0 +1,124 @@
+class Cldr
+  module Data
+    class Calendars
+      class Gregorian < Base
+        def initialize(locale)
+          super
+          update(
+            :months              => contexts('month'),
+            :days                => contexts('day'),
+            :eras                => contexts('era'),
+            :quarters            => contexts('quarter'),
+            :periods             => periods,
+            :fields              => fields,
+            :'formats.date'      => formats('date'),
+            :'formats.time'      => formats('time'),
+            :'formats.datetime'  => formats('dateTime')
+          )
+        end
+
+        def calendar
+          @calendar ||= select('dates/calendars/calendar[@type="gregorian"]').first
+        end
+
+        def contexts(kind)
+          select(calendar, "#{kind}s/#{kind}Context").inject({}) do |result, node|
+            context = node.attribute('type').value.to_sym
+            result[context] = widths(node, kind, context)
+            result
+          end
+        end
+
+        def widths(node, kind, context)
+          select(node, "#{kind}Width").inject({}) do |result, node|
+            width  = node.attribute('type').value.to_sym
+            result[width] = elements(node, kind, context, width)
+            result
+          end
+        end
+
+        def elements(node, kind, context, width)
+          aliased = select(node, 'alias').first
+          if aliased
+            xpath_to_key(aliased.attribute('path').value, kind, context, width)
+          else
+            select(node, kind).inject({}) do |result, node|
+              key = node.attribute('type').value
+              key = key =~ /^\d*$/ ? key.to_i : key.to_sym
+              result[key] = node.content
+              result
+            end
+          end
+        end
+        
+        def xpath_to_key(xpath, kind, context, width)
+          kind    = (xpath =~ %r(/([^\/]*)Width) && $1) || kind
+          context = (xpath =~ %r(Context\[@type='([^\/]*)'\]) && $1) || context
+          width   = (xpath =~ %r(Width\[@type='([^\/]*)'\]) && $1) || width
+          :"calendars.gregorian.#{kind}s.#{context}.#{width}"
+        end
+        
+        def xpath_width
+        end
+
+        def periods
+          am = select(calendar, "am").first
+          pm = select(calendar, "pm").first
+
+          result = {}
+          result[:am] = am.content if am
+          result[:pm] = pm.content if pm
+          result
+        end
+
+        def eras
+          base_path = calendar.path.gsub('/ldml/', '') + '/eras'
+          keys = select("#{base_path}/*").map { |node| node.name }
+
+          eras = keys.inject([]) do |result, name|
+            path = "#{base_path}/#{name}/*"
+            key  = name.gsub('era', '').gsub(/s$/, '').downcase.to_sym
+            result << extract(path, :key   => lambda { |node| node.attribute('type').value.to_i rescue 0 },
+                                    :value => lambda { |node| { key => node.content } })
+          end
+          eras.inject({}) { |result, data| result.deep_merge(data) }
+        end
+
+        def formats(type)
+          formats = select(calendar, "#{type}Formats/#{type}FormatLength").inject({}) do |result, node|
+            key = node.attribute('type').value.to_sym rescue :format
+            result[key] = pattern(node, type)
+            result
+          end
+          if default = default_format(type)
+            formats = default.merge(formats)
+          end
+          formats
+        end
+        
+        def default_format(type)
+          if node = select(calendar, "#{type}Formats/default").first
+            key = node.attribute('choice').value.to_sym
+            { :default => :"calendars.gregorian.formats.#{type.downcase}.#{key}" }
+          end
+        end
+
+        def pattern(node, type)
+          select(node, "#{type}Format/pattern").inject({}) do |result, node|
+            result[:pattern] = node.content
+            result
+          end
+        end
+
+        def fields
+          select(calendar, "fields/field").inject({}) do |result, node|
+            key  = node.attribute('type').value.to_sym
+            name = node.xpath('displayName').first
+            result[key] = name.content if name
+            result
+          end
+        end
+      end
+    end
+  end
+end

data/lib/cldr/data/calendars.rb

@@ -0,0 +1,12 @@
+class Cldr
+  module Data
+    class Calendars < Base
+      autoload :Gregorian, 'cldr/data/calendars/gregorian'
+
+      def initialize(locale)
+        super
+        self['calendars.gregorian'] = Gregorian.new(locale)
+      end
+    end
+  end
+end

data/lib/cldr/data/currencies.rb

@@ -0,0 +1,26 @@
+class Cldr
+  module Data
+    class Currencies < Base
+      def initialize(locale)
+        super
+        self[:currencies] = currencies
+      end
+
+      def currencies
+        select('numbers/currencies/*').inject({}) do |result, node|
+          currency = self.currency(node)
+          result[node.attribute('type').value.to_sym] = currency unless currency.empty?
+          result
+        end
+      end
+
+      def currency(node)
+        select(node, 'displayName').inject({}) do |result, node|
+          count = node.attribute('count') ? node.attribute('count').value.to_sym : :one
+          result[count] = node.content unless draft?(node)
+          result
+        end
+      end
+    end
+  end
+end

data/lib/cldr/data/delimiters.rb

@@ -0,0 +1,21 @@
+class Cldr
+  module Data
+    class Delimiters < Base
+      def initialize(locale)
+        super
+        self[:'delimiters.quotes.default']   = quotes('quotation')
+        self[:'delimiters.quotes.alternate'] = quotes('alternateQuotation')
+      end
+
+      def quotes(type)
+        start = select("delimiters/#{type}Start").first
+        end_  = select("delimiters/#{type}End").first
+
+        result = {}
+        result[:start] = start.content if start
+        result[:end]   = end_.content  if end_
+        result
+      end
+    end
+  end
+end

data/lib/cldr/data/languages.rb

@@ -0,0 +1,17 @@
+class Cldr
+  module Data
+    class Languages < Base
+      def initialize(locale)
+        super
+        self[:languages] = languages
+      end
+
+      def languages
+        @languages ||= select('localeDisplayNames/languages/language').inject({}) do |result, node|
+          result[node.attribute('type').value.gsub('_', '-').to_sym] = node.content unless draft?(node)
+          result
+        end
+      end
+    end
+  end
+end

data/lib/cldr/data/numbers.rb

@@ -0,0 +1,60 @@
+class Cldr
+  module Data
+    class Numbers < Base
+      def initialize(locale)
+        super
+        self[:'numbers.symbols'] = symbols
+        self[:'numbers.formats'] = {
+          :decimal => {
+            :patterns => {
+              :default => format('decimal')
+            }
+          },
+          :scientific => {
+            :patterns => {
+              :default => format('scientific')
+            }
+          },
+          :percent => {
+            :patterns => {
+              :default => format('percent')
+            }
+          },
+          :currency => {
+            :patterns => {
+              :default => format('currency')
+            }
+          },
+        }
+        self[:'numbers.formats.currency.unit'] = unit
+      end
+      
+      def currency
+        currency = format('currency')
+        currency.update(:unit => unit) unless unit.empty?
+        currency
+      end
+
+      def symbols
+        select('numbers/symbols/*').inject({}) do |result, node|
+          result[name(node).to_sym] = node.content unless draft?(node)
+          result
+        end
+      end
+
+      def format(type)
+        select("numbers/#{type}Formats/#{type}FormatLength/#{type}Format/pattern").inject({}) do |result, node|
+          node.content unless draft?(node)
+        end
+      end
+
+      def unit
+        @unit ||= select("numbers/currencyFormats/unitPattern").inject({}) do |result, node|
+          count = node.attribute('count').value rescue 'one'
+          result[count] = node.content
+          result
+        end
+      end
+    end
+  end
+end

data/lib/cldr/data/plurals/cldr_grammar.treetop

@@ -0,0 +1,50 @@
+# http://unicode.org/draft/reports/tr35/tr35.html#Language_Plural_Rules
+#
+# condition       = and_condition ('or' and_condition)*
+# and_condition   = relation ('and' relation)*
+# relation        = is_relation | in_relation | within_relation | 'n' <EOL>
+# is_relation     = expr 'is' ('not')? value
+# in_relation     = expr ('not')? 'in' range
+# within_relation = expr ('not')? 'within' range
+# expr            = 'n' ('mod' value)?
+# value           = digit+
+# digit           = 0|1|2|3|4|5|6|7|8|9
+# range           = value'..'value
+
+grammar CldrPluralGrammar
+	rule or_condition
+		and_condition (" or " and_condition)?
+	end
+	
+	rule and_condition
+		relation (" and " relation)?
+	end
+	
+	rule relation
+		is_relation / in_relation / within_relation / "n"
+	end
+	
+	rule is_relation
+		expr " is " "not "? value
+	end	
+	
+	rule in_relation
+		expr " not"? " in " range
+	end
+	
+	rule within_relation
+		expr " not"? " within " range
+	end
+	
+	rule expr
+		"n" (" mod " value)?
+	end	
+	
+	rule range
+		value ".." value
+	end
+
+  rule value
+    [0-9]*
+  end
+end