fast_gettext 0.4.16

data/.gitignore

@@ -0,0 +1,2 @@
+pkg
+benchmark/locle

data/CHANGELOG

@@ -0,0 +1,6 @@
+0.4.14 -- "" is translated as "", not as gettext meta information
+0.4.0 -- pluralisation_rules is no longer stored in each repository, only retrived. Added Chain and Logger repository.
+0.3.6 -- FastGettext.default_locale=
+0.3.5 -- FastGettext.default_text_domain=
+0.3.4 -- Exceptions are thrown, not returned when translating without text domain
+0.3 -- pluralisation methods accept/return n plural forms, contrary to singular/plural before

data/README.markdown

@@ -0,0 +1,181 @@
+FastGettext
+===========
+GetText but 3.5 x faster, 560 x less memory, simple, clean namespace (7 vs 34) and threadsave!  
+
+It supports multiple backends (atm: .mo files, .po files, Database(ActiveRecor + any other), Chain, Loggers) and can easily be extended.
+
+[Example Rails application](https://github.com/grosser/gettext_i18n_rails_example)
+
+
+Setup
+=====
+### 1. Install
+    sudo gem install fast_gettext
+
+### 2. Add a translation repository
+
+From mo files (traditional/default)
+    FastGettext.add_text_domain('my_app',:path=>'locale')
+
+Or po files (less maintenacnce than mo)
+    FastGettext.add_text_domain('my_app',:path=>'locale', :type=>:po)
+
+Or database (scaleable, great for many locales/translators)
+    require "fast_gettext/translation_repository/db"
+    include FastGettext::TranslationRepository::Db.require_models #load and include default models
+    FastGettext.add_text_domain('my_app', :type=>:db, :model=>TranslationKey)
+
+### 3. Choose text domain and locale for translation
+Do this once in every Thread. (e.g. Rails -> ApplicationController)
+    FastGettext.text_domain = 'my_app'
+    FastGettext.available_locales = ['de','en','fr','en_US','en_UK'] # only allow these locales to be set (optional)
+    FastGettext.locale = 'de'
+
+### 4. Start translating
+    include FastGettext::Translation
+    _('Car') == 'Auto'
+    _('not-found') == 'not-found'
+    s_('Namespace|no-found') == 'not-found'
+    n_('Axis','Axis',3) == 'Achsen' #German plural of Axis
+
+
+Managing translations
+============
+### mo/po-files
+Generate .po or .mo files using GetText parser (example tasks at [gettext_i18n_rails](http://github.com/grosser/gettext_i18n_rails))
+
+Tell Gettext where your .mo or .po files lie, e.g. for locale/de/my_app.po and locale/de/LC_MESSAGES/my_app.mo
+    FastGettext.add_text_domain('my_app',:path=>'locale')
+
+Use the [original GetText](http://github.com/mutoh/gettext) to create and manage po/mo-files.
+(Work on a po/mo parser & reader that is easier to use has started, contributions welcome @ [pomo](http://github.com/grosser/pomo) )
+
+###Database
+!!!new/only short time in production, please report back any ideas/suggestions you have!!!  
+[Example migration for ActiveRecord](http://github.com/grosser/fast_gettext/blob/master/examples/db/migration.rb)  
+The default plural seperator is `||||` but you may overwrite it (or suggest a better one..).
+
+This is usable with any model DataMapper/Sequel or any other(non-database) backend, the only thing you need to do is respond to the self.translation(key, locale) call.
+If you want to use your own models, have a look at the [default models](http://github.com/grosser/fast_gettext/tree/master/lib/fast_gettext/translation_repository/db_models) to see what you want/need to implement.
+
+To manage translations via a Web GUI, use a [Rails application and the translation_db_engine](http://github.com/grosser/translation_db_engine)
+
+
+Performance
+===========
+50_000 translations speed / memory  
+small translation file <-> large translation file  
+(ruby enterprise 1.8.6, your results may vary, try `rake benchmark`)
+    Baseline: (doing nothing in a loop)
+    0.250000s / 0K <->
+
+    Ideal: (primitive Hash lookup)
+    0.820000s / 4K <-> 0.760000s / 4K
+
+    FastGettext:
+    1.360000s / 8K <-> 1.320000s / 8K
+
+    GetText 2.0.1:
+    4.880000s / 4480K <-> 4.810000s / 4480K
+
+    ActiveSupport I18n::Backend::Simple :
+    21.770000s / 10100K <->
+
+
+Rails
+=======================
+Try the [gettext_i18n_rails plugin](http://github.com/grosser/gettext_i18n_rails), it simplifies the setup.  
+Try the [translation_db_engine](http://github.com/grosser/translation_db_engine), to manage your translations in a db.
+
+Setting `available_locales`,`text_domain` or `locale` will not work inside the `evironment.rb`,
+since it runs in a different thread then e.g. controllers, so set them inside your application_controller.
+
+    #environment.rb after initializers
+    Object.send(:include,FastGettext::Translation)
+    FastGettext.add_text_domain('accounting',:path=>'locale')
+    FastGettext.add_text_domain('frontend',:path=>'locale')
+    ...
+
+    #application_controller.rb
+    class ApplicationController ...
+      include FastGettext::Translation
+      before_filter :set_locale
+      def set_locale
+        FastGettext.available_locales = ['de','en',...]
+        FastGettext.text_domain = 'frontend'
+        session[:locale] = I18n.locale = FastGettext.set_locale(params[:locale] || session[:locale] || request.env['HTTP_ACCEPT_LANGUAGE'] || 'en')
+      end
+
+
+Advanced features
+=================
+###Abnormal pluralisation
+Pluralisation rules can be set directly via a lambda (see specs/), or by using the Gettext
+plural definition (see spec/locale/en/test_plural.po or [Plural expressions for all languages](http://translate.sourceforge.net/wiki/l10n/pluralforms).
+
+
+###default_text_domain
+If you only use one text domain, setting `FastGettext.default_text_domain = 'app'`
+is sufficient and no more `text_domain=` is needed
+
+###default_locale
+If the simple rule of "first `availble_locale` or 'en'" is not suficcient for you, set `FastGettext.default_locale = 'de'`.
+
+###default_available_locales
+Fallback when no available_locales are set
+
+###Chains
+You can use any number of repositories to find a translation. Simply add them to a chain and when
+the first cannot translate a given key, the next is asked and so forth.
+    repos = [
+      FastGettext::TranslationRepository.build('new', :path=>'....'),
+      FastGettext::TranslationRepository.build('old', :path=>'....')
+    ]
+    FastGettext.add_text_domain 'combined', :type=>:chain, :chain=>repos
+
+###Logger
+When you want to know which keys could not be translated or were used, add a Logger to a Chain:
+    repos = [
+      FastGettext::TranslationRepository.build('app', :path=>'....')
+      FastGettext::TranslationRepository.build('logger', :type=>:logger, :callback=>lamda{|key_or_array_of_ids| ... }),
+    }
+    FastGettext.add_text_domain 'combined', :type=>:chain, :chain=>repos
+If the Logger is in position #1 it will see all translations, if it is in position #2 it will only see the unfound.
+Unfound may not always mean missing, if you chose not to translate a word because the key is a good translation, it will appear nevertheless.
+A lambda or anything that responds to `call` will do as callback. A good starting point may be `examples/missing_translations_logger.rb`.
+
+###Plugins
+Want a yml, xml version ?
+Write your own TranslationRepository!
+    #fast_gettext/translation_repository/xxx.rb
+    module FastGettext
+      module TranslationRepository
+        class Wtf
+          define initialize(name,options), [key], plural(*keys) and
+          either inherit from TranslationRepository::Base or define available_locales and pluralisation_rule
+        end
+      end
+    end
+
+
+FAQ
+===
+ - [Problems with ActiveRecord messages?](http://wiki.github.com/grosser/fast_gettext/activerecord)
+
+
+TODO
+====
+ - YML backend that reads ActiveSupport::I18n files
+ - any ideas ? :D
+
+Author
+======
+Mo/Po-file parsing from Masao Mutoh, see vendor/README
+
+###Contributors
+ - [geekq](http://www.innoq.com/blog/vd)
+ - Rudolf Gavlas
+
+[Michael Grosser](http://pragmatig.wordpress.com)  
+grosser.michael@gmail.com  
+Hereby placed under public domain, do what you want, just do not hold me accountable...  

data/Rakefile

@@ -0,0 +1,42 @@
+desc "Run all specs in spec directory"
+task :default do |t|
+  options = "--colour --format progress --loadby --reverse"
+  files = FileList['spec/**/*_spec.rb']
+  system("spec #{options} #{files}")
+end
+
+task :benchmark do
+  puts "Running on #{RUBY}"
+  %w[baseline ideal fast_gettext original i18n_simple].each do |bench|
+    puts `ruby benchmark/#{bench}.rb`
+    puts ""
+  end
+end
+
+task :namespaces do
+  puts `ruby benchmark/namespace/original.rb`
+  puts `ruby benchmark/namespace/fast_gettext.rb`
+end
+
+begin
+  require 'jeweler'
+  project_name = 'fast_gettext'
+  Jeweler::Tasks.new do |gem|
+    gem.name = project_name
+    gem.summary = "A simple, fast and threadsafe implementation of GetText"
+    gem.email = "grosser.michael@gmail.com"
+    gem.homepage = "http://github.com/grosser/#{project_name}"
+    gem.authors = ["Michael Grosser"]
+    gem.rubyforge_project = project_name.sub('_','-')
+  end
+
+  # fake task so that rubyforge:release works
+  task :rdoc do
+    `mkdir rdoc`
+    `echo documentation is at http://github.com/grosser/#{project_name} > rdoc/README.rdoc`
+  end
+
+  Jeweler::RubyforgeTasks.new
+rescue LoadError
+  puts "Jeweler, or one of its dependencies, is not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
+end

data/VERSION

@@ -0,0 +1 @@
+0.4.16

data/benchmark/base.rb

@@ -0,0 +1,42 @@
+require 'rubygems'
+require 'benchmark'
+
+RUNS = 50_0000
+DEFAULTS = {:memory=>0}
+
+def locale_folder(domain)
+  path = case domain
+  when 'test' then File.join(File.expand_path(File.dirname(__FILE__)),'..','spec','locale')
+  when 'large' then File.join(File.expand_path(File.dirname(__FILE__)),'locale')
+  end
+
+  mo = File.join(path,'de','LC_MESSAGES',"#{domain}.mo")
+  raise unless File.exist?(mo)
+  path
+end
+
+def results_test(&block)
+  print "#{(result(&block)).to_s.strip.split(' ').first}s / #{memory}K <-> "
+end
+
+def results_large
+  print "#{(result {_('login') == 'anmelden'}).to_s.strip.split(' ').first}s / #{memory}K"
+  puts ""
+end
+
+def result
+  result =Benchmark.measure do
+    RUNS.times do
+      raise "not translated" unless yield
+    end
+  end
+  result
+end
+
+def memory
+  pid = Process.pid
+  map = `pmap -d #{pid}`
+  map.split("\n").last.strip.squeeze(' ').split(' ')[3].to_i - DEFAULTS[:memory]
+end
+
+DEFAULTS[:memory] = memory + 4 #4 => 0 for base calls

data/benchmark/baseline.rb

@@ -0,0 +1,5 @@
+require 'benchmark/base'
+
+puts "Baseline: (doing nothing in a loop)"
+results_test{true}
+puts ""

data/benchmark/fast_gettext.rb

@@ -0,0 +1,18 @@
+require 'benchmark/base'
+
+$LOAD_PATH.unshift 'lib'
+require 'fast_gettext'
+include FastGettext::Translation
+
+FastGettext.available_locales  = ['de','en']
+FastGettext.locale = 'de'
+
+puts "FastGettext:"
+FastGettext.add_text_domain('test',:path=>locale_folder('test'))
+FastGettext.text_domain = 'test'
+results_test{_('car') == 'Auto'}
+
+#i cannot add the large file, since its an internal applications mo file
+FastGettext.add_text_domain('large',:path=>locale_folder('large'))
+FastGettext.text_domain = 'large'
+results_large

data/benchmark/i18n_simple.rb

@@ -0,0 +1,7 @@
+require 'benchmark/base'
+require 'activesupport'
+I18n.backend = I18n::Backend::Simple.new
+I18n.load_path = ['benchmark/locale/de.yml']
+I18n.locale = :de
+puts "ActiveSupport I18n::Backend::Simple :"
+results_test{I18n.translate('activerecord.models.car')=='Auto'}

data/benchmark/ideal.rb

@@ -0,0 +1,22 @@
+require 'benchmark/base'
+
+module FastestGettext
+  def set_domain(folder,domain,locale)
+    @data = {}
+    require File.join(File.dirname(__FILE__),'..','vendor','mofile')
+    FastGettext::GetText::MOFile.open(File.join(folder,locale,'LC_MESSAGES',"#{domain}.mo"), "UTF-8").each{|k,v|@data[k]=v}
+  end
+  def _(word)
+    @data[word]
+  end
+end
+
+
+include FastestGettext
+set_domain(locale_folder('test'),'test','de')
+puts "Ideal: (primitive Hash lookup)"
+results_test{_('car') == 'Auto'}
+
+#i cannot add the large file, since its an internal applications mo file
+set_domain(locale_folder('large'),'large','de')
+results_large

data/benchmark/locale/de.yml

@@ -0,0 +1,127 @@
+# German translations for Ruby on Rails 
+# by Clemens Kofler (clemens@railway.at)
+
+de:
+  date:
+    formats:
+      default: "%d.%m.%Y"
+      short: "%e. %b"
+      long: "%e. %B %Y"
+      only_day: "%e"
+
+    day_names: [Sonntag, Montag, Dienstag, Mittwoch, Donnerstag, Freitag, Samstag]
+    abbr_day_names: [So, Mo, Di, Mi, Do, Fr, Sa]
+    month_names: [~, Januar, Februar, März, April, Mai, Juni, Juli, August, September, Oktober, November, Dezember]
+    abbr_month_names: [~, Jan, Feb, Mär, Apr, Mai, Jun, Jul, Aug, Sep, Okt, Nov, Dez]
+    order: [ :day, :month, :year ]
+  
+  time:
+    formats:
+      default: "%A, %e. %B %Y, %H:%M Uhr"
+      short: "%e. %B, %H:%M Uhr"
+      long: "%A, %e. %B %Y, %H:%M Uhr"
+      time: "%H:%M"
+
+    am: "vormittags"
+    pm: "nachmittags"
+      
+  datetime:
+    distance_in_words:
+      half_a_minute: 'eine halbe Minute'
+      less_than_x_seconds:
+        zero: 'weniger als 1 Sekunde'
+        one: 'weniger als 1 Sekunde'
+        other: 'weniger als {{count}} Sekunden'
+      x_seconds:
+        one: '1 Sekunde'
+        other: '{{count}} Sekunden'
+      less_than_x_minutes:
+        zero: 'weniger als 1 Minute'
+        one: 'weniger als eine Minute'
+        other: 'weniger als {{count}} Minuten'
+      x_minutes:
+        one: '1 Minute'
+        other: '{{count}} Minuten'
+      about_x_hours:
+        one: 'etwa 1 Stunde'
+        other: 'etwa {{count}} Stunden'
+      x_days:
+        one: '1 Tag'
+        other: '{{count}} Tage'
+      about_x_months:
+        one: 'etwa 1 Monat'
+        other: 'etwa {{count}} Monate'
+      x_months:
+        one: '1 Monat'
+        other: '{{count}} Monate'
+      about_x_years:
+        one: 'etwa 1 Jahr'
+        other: 'etwa {{count}} Jahre'
+      over_x_years:
+        one: 'mehr als 1 Jahr'
+        other: 'mehr als {{count}} Jahre'
+      
+  number:
+    format:
+      precision: 2
+      separator: ','
+      delimiter: '.'
+    currency:
+      format:
+        unit: '€'
+        format: '%n%u'
+        separator: 
+        delimiter: 
+        precision: 
+    percentage:
+      format:
+        delimiter: ""
+    precision:
+      format:
+        delimiter: ""
+    human:
+      format:
+        delimiter: ""
+        precision: 1
+
+  support:
+    array:
+      sentence_connector: "und"
+      skip_last_comma: true
+        
+  activerecord:
+    errors:
+      template:
+        header:
+          one:    "Konnte dieses {{model}} Objekt nicht speichern: 1 Fehler."
+          other:  "Konnte dieses {{model}} Objekt nicht speichern: {{count}} Fehler."
+        body: "Bitte überprüfen Sie die folgenden Felder:"
+      format:
+        seperator: ' '
+      messages:
+        inclusion: "ist kein gültiger Wert"
+        exclusion: "ist nicht verfügbar"
+        invalid: "ist nicht gültig"
+        confirmation: "stimmt nicht mit der Bestätigung überein"
+        accepted: "muss akzeptiert werden"
+        empty: "muss ausgefüllt werden"
+        blank: "muss ausgefüllt werden"
+        too_long: "ist zu lang (nicht mehr als {{count}} Zeichen)"
+        too_short: "ist zu kurz (nicht weniger als {{count}} Zeichen)"
+        wrong_length: "hat die falsche Länge (muss genau {{count}} Zeichen haben)"
+        taken: "ist bereits vergeben"
+        not_a_number: "ist keine Zahl"
+        greater_than: "muss größer als {{count}} sein"
+        greater_than_or_equal_to: "muss größer oder gleich {{count}} sein"
+        equal_to: "muss genau {{count}} sein"
+        less_than: "muss kleiner als {{count}} sein"
+        less_than_or_equal_to: "muss kleiner oder gleich {{count}} sein"
+        odd: "muss ungerade sein"
+        even: "muss gerade sein"
+      models:
+        car: 'BAUTO'
+        cars: 'CAUTO'
+        Car: 'DAUTO'
+
+    models:
+      car: 'Auto'