vacuum_cleaner 1.0.0 → 1.0.1

data/.gitignore

@@ -1,3 +1,4 @@
 doc
 pkg
 .yardoc
+*.watcher

data/README.md

@@ -3,12 +3,15 @@ Vacuum Cleaner
 
 A Ruby (and Rails) attribute normalization gem, which is supposedly [semver](http://semver.org/)-compliant.
 
+Known to work with Ruby-1.9, JRuby and both Rails 2.3.x and 3. If there are any issues, please file a
+[bug report](http://github.com/lwe/vacuum_cleaner/issues), or fix it and send me a pull request.
+
 Installation
 ------------
 
 1. Install as gem from [rubygems](http://rubygems.org/gems/vacuum_cleaner): `sudo gem install vacuum_cleaner`.
    Then load it in your app: `require 'rubygems'; require 'vacuum_cleaner'`      
-2. Install as gem using [bundler](http://github.com/carlhuda/bundler), add `gem "vacuum_cleaner"` to your
+2. Install as gem using [bundler](http://gembundler.com), add `gem "vacuum_cleaner"` to your
    `Gemfile` and run `bundle install`
 3. Or as a Rails plugin, for Rails 2.x run `./script/plugin install git://github.com/lwe/vacuum_cleaner.git`, when using Rails 3.x
    goodeness run `rails plugin install git://github.com/lwe/vacuum_cleaner.git`
@@ -16,26 +19,29 @@ Installation
 Usage
 -----
 
-It creates a new setter method for an attribute and thus allows the gem to reprozess the input value.
+It creates a new setter method for an attribute and thus allows the normalizers defined to reprocess the input value.
 
     class Doctor
-      include VacuumCleaner::Normalizations  # enables #normalizes, NOTE: not required for ActiveRecord models                                             
+      include VacuumCleaner::Normalizations  # enables #normalizes, NOTE: not required for ActiveRecord models
       attr_accessor :name                    # create some reader/writter
       
       normalizes :name                       # enables strip/clean-up magic on attribute :name
     end
     
+Using `normalizes` just adds a default normalization implemenation, which removes leading/trailing
+whitespace and converts spaces only to `nil`. Everything happens upon "set".
+
     @doc = Doctor.new
-    
-    # set name with leading/trailing spaces
     @doc.name = "  Elliot Reid\n\t"
-    @doc.name   # => "Elliot Reid"
+    @doc.name   # => "Elliot Reid" => trailing space was stripped
     
-    # empty strings => nil
     @doc.name = "\t\n"
-    @doc.name   # => nil
+    @doc.name   # => nil => converted to nil
+
+Okay, this is how it basically works, the `normalizes` call just generates a new setter method,
+which `normalizes` the input value and then calls the original setter method.
 
-Okay, this is it. Now, let the fun part begin...
+What else can be done then?
 
     # can be used with multiple attributes (if they all share the same normalizer)
     normalizes :name, :company
@@ -54,9 +60,12 @@ Okay, this is it. Now, let the fun part begin...
     normalizes :homepage, :url => true
     # "google.com" => "http://google.com"
     # "http://example.com" => "http://example.com" PS: left as is
-    
-Assuming this already allows to fix 99.9% of all attribute normalization cases, if there's
-that special need, then `normalizes` accepts a block:
+
+Take a look at `VacuumCleaner::Normalizer`, about how the process works and how custom
+reusable normalizers can be written. For the-quick-fix-that-shouldnt-have-been-used-but-was
+case or if there's no reuse, `normalizes` takes a block as argument which is called
+after any other normalizer in the chain. Note that normalizers are not halted, nor stopped
+if they return `nil` or `false` or something similar, so ensure that case is handled properly.
 
     # strips all whitespace within a string
     normalizes(:phone) { |value| value.to_s.gsub(/\s+/, '') unless value.nil? }    
@@ -68,9 +77,40 @@ that special need, then `normalizes` accepts a block:
     # "\t\n" => ""
     # nil => ""
     
-Need access to the object within the block? As easy as:
+Need access to the full object within the block? As easy as:
 
     # naming J.D. after some girly girl?
     normalizes(:first_name) do |obj, attribute, value|
       obj.name == "Dorian" ? %w{Agnes Shirley Denise}[rand(3)] : value
     end
+
+Background
+----------
+
+As mentoined earlier `normalizes` creates a new setter method, so let's shortly take a look
+at how.
+
+    normalizes(:name)
+    
+    # 1. creates a :normalize_name method, which contains the normalization chain, block etc.
+    # 2. if :name= exists, it's aliased to :name_without_normalization=
+    # 3. creates a new :name= method, which calls :normalize_name, then tries to
+    #    set the normalized value by one of:
+    #    a) calling :name_without_normalization=, if defined
+    #    b) self[:name] = v, if it responds to :[]= (for ActiveRecord support)
+    #    c) or, as a fallback, sets @name to the result of :normalize_name
+
+Lessons learned: when the need arises to set the value without any normalization and there's
+a setter just use `@object.name_without_normalization = "har har har\n\t"`. Feel free to
+completly override `normalize_<attribute>`, but a much smarter way to add very custom normalizers
+is by a) providing a block to `normalizes` or b) create a custom `VacuumCleaner::Normalizer`
+implementation.
+
+Some info about the different files, might be a good place to look at when trying to figure
+out how to write custom `VacuumCleaner::Normalizer` implemenations, or for a look at how
+it works.
+
+    lib/vacuum_cleaner/normalizer.rb        # Base Normalizer implementation
+    lib/vacuum_cleaner/normalizations/*.rb  # Some default Normalizer implementations, like url, downcase etc.
+    lib/vacuum_cleaner/normalizations.rb    # Provides the `normalizes` method and all the logic etc.
+    

data/Rakefile

@@ -1,7 +1,7 @@
 require 'rake'
 require 'rake/testtask'
-
-require File.join(File.dirname(__FILE__), 'lib', 'vacuum_cleaner')
+$: << File.join(File.dirname(__FILE__), 'lib')
+require 'vacuum_cleaner'
 
 begin
   require 'yard'
@@ -35,7 +35,7 @@ begin
     gemspec.add_development_dependency('shoulda', '>= 2.10.2')
     gemspec.add_development_dependency('activesupport', '>= 2.3.5')
     
-    gemspec.files.reject! { |file| file =~ /\.gemspec$/ } # kinda redundant
+    gemspec.files.reject! { |file| file =~ /\.gemspec$/ || file =~ /\.png$/ || file =~ /\.watcher_example+$/} # do not include gemspec + png files
   end
   Jeweler::GemcutterTasks.new
 rescue LoadError
@@ -64,7 +64,19 @@ namespace :test do
     t.verbose = true
   end
   
+  desc 'Run both the integration and unit tests'
   task :all => [:'test:unit', :'test:integration']
+  
+  desc 'Run test with supplied rakes like RAKES=jrake,rake19,rake'
+  task :portability do |t|
+    raise "No RAKES specified e.g. add RAKES=rake,rake19" unless ENV['RAKES']
+    ENV['RAKES'].split(/[,;\s]\s*/).each do |rake_cmd|
+      puts
+      puts " * * * runing `#{rake_cmd} test:all`"
+      puts      
+      system("#{rake_cmd} test:all")
+    end
+  end
 end  
 
 namespace :metrics do
@@ -85,7 +97,7 @@ namespace :metrics do
   task :coverage do |t|
     rm_f "doc/coverage"
     mkdir_p "doc/coverage"
-    rcov = %(rcov -Ilib:test --exclude '\/gems\/' -o doc/coverage -T test/unit/vacuum_cleaner/*_test.rb -T test/unit/vacuum_cleaner/*/*_test.rb)
+    rcov = %(rcov -Ilib:test --exclude '\/gems\/' -o doc/coverage -T test/integration/*_test.rb -T test/unit/vacuum_cleaner/*_test.rb -T test/unit/vacuum_cleaner/*/*_test.rb)
     system rcov
   end
   

data/init.rb

@@ -1 +1 @@
-require File.join(File.dirname(__FILE__), 'rails', 'init')
+require 'vacuum_cleaner'

data/lib/vacuum_cleaner/normalizations/active_support.rb

@@ -0,0 +1,33 @@
+# Okay, because about 99.9% of the usage is with Rails/AS, I guess it should
+# have better/deeper support for it, like support for Multibyte stuff and some
+# (useful) inflections.
+#
+# Please note that the +TransliterateNormalizer+ requires rails 3.0.0.beta3 and
+# when run with ruby 1.9.
+#
+if defined?(::ActiveSupport)
+  require 'active_support/inflector'
+    
+  require 'vacuum_cleaner/normalizations/method'  
+  
+  module VacuumCleaner
+    module Normalizations      
+      # Calls `ActiveSupport::Inflector.transliterate` if supplied
+      # +value+ responds to +to_str+, so it basically only works on strings.
+      class TransliterateNormalizer < Normalizer
+        def normalize_value(value)
+          ::ActiveSupport::Inflector.transliterate(value.to_str).to_s if value.respond_to?(:to_str)
+        end
+      end
+          
+      # Calls the `titleize` method from AS on the supplied value, if possible.
+      TitleizeNormalizer = MethodNormalizer.build(:titleize)
+    
+      # Calls the `humanize` method from AS on the supplied value.
+      HumanizeNormalizer = MethodNormalizer.build(:humanize)      
+    end
+  end
+
+  # Set the multibyte proxy class to AS::Multibyte::Chars, which in turn works perfectly with UTF8 chars et al.
+  VacuumCleaner::Normalizations::MethodNormalizer.multibyte_wrapper = ::ActiveSupport::Multibyte::Chars
+end

data/lib/vacuum_cleaner/normalizations/method.rb

@@ -13,13 +13,26 @@ module VacuumCleaner
     # Subclasses of the +MethodNormalizer+ can take advantage of it's
     # +normalize_if_respond_to+ method, to easily create custom
     # normalizers based on methods availble on the result value.
-    class MethodNormalizer < Normalizer      
-      # Helper method to "bake" a method normalizer from a method, enabling us to do stuff like.
-      #
-      #   TitelizeNormalizer = MethodNormalizer.build(:titleize)
-      #
-      def self.build(sym)
-        module_eval "Class.new(MethodNormalizer) do; def initialize(*args); super({ :method => #{sym.inspect}}) end; end", __FILE__, __LINE__
+    class MethodNormalizer < Normalizer
+      
+      class << self  
+        # Helper method to "bake" a method normalizer from a method, enabling us to do stuff like.
+        #
+        #   TitelizeNormalizer = MethodNormalizer.build(:titleize)
+        #
+        def build(sym)
+          module_eval "Class.new(MethodNormalizer) do; def initialize(*args); super({ :method => #{sym.inspect}}) end; end", __FILE__, __LINE__
+        end
+        
+        # Due to the lack of multibyte support in ruby core, a proxy class like
+        # {ActiveSupport::Multibyte::Chars} can be registered here and the proxy
+        # is then used to wrap string values within, so that methods like `upcase`
+        # or `downcase` work with characters outside the ASCII range as well.
+        #
+        # The wrapper is only used if the value supplied is a string, i.e. responds to
+        # +to_str+.
+        def multibyte_wrapper=(clazz); @multibyte_wrapper = clazz end        
+        def multibyte_wrapper; @multibyte_wrapper end
       end
       
       # Accept either a hash or symbol name.
@@ -27,18 +40,19 @@ module VacuumCleaner
         args = { :method => args } unless args.is_a?(Hash)
         super(args)
       end
-      
+            
       # Normalize value by trying to call the method at hand, if
       # +value+ does not respond to the defined method, returns +nil+.
       def normalize_value(value)
         sym = options[:method]
+        value = MethodNormalizer.multibyte_wrapper.new(value) if MethodNormalizer.multibyte_wrapper and value.respond_to?(:to_str)
         value.respond_to?(sym) ? value.send(sym) : nil
       end      
     end
     
     # Downcase value unless nil or empty.
     DowncaseNormalizer = MethodNormalizer.build(:downcase)
-    
+  
     # Upcases value unless nil or empty.
     UpcaseNormalizer = MethodNormalizer.build(:upcase)
   end

data/lib/vacuum_cleaner/normalizations/numeric.rb

@@ -0,0 +1,39 @@
+module VacuumCleaner
+  module Normalizations
+    
+    # The numeric normalizer tries to normalize a numeric string input
+    # like <tt>CHF 12'000.--</tt> into a string parseable by {{Kernel.Float}}
+    # or similar (result would be <tt>12000.</tt>).
+    #
+    # It basically strips any non valid character for a number from the
+    # input string (scientific notation is currently not support), so all
+    # whitespace, currency symbols and units are stripped. Furthermore also
+    # the decimal points are normalized, because in Germany numbers can look
+    # like: <tt>EUR 12.000,50</tt> and in Switzerland they can look like
+    # <tt>1,5 Mio.</tt>. So the normalizer tries to ensure that both are
+    # numbers parseable by {{Kernel.Float}}, by intelligently trying to
+    # figure out the seperator used and converting it to <tt>.</tt>.
+    #
+    # All values which do not respond to <tt>to_str</tt> are left as is.
+    #
+    # Note: no conversion or anything similar is done! The value wont be
+    # converted to a +Fixnum+ or whatever and will be left as string.
+    # When used with Rails validations, this might also certainly render
+    # the validations to check if it's a valid number obsolete, yet all
+    # the stuff about min/max, fixed only etc. work of course like a charm.
+    #
+    class NumericNormalizer < Normalizer
+      def normalize_value(value)
+        if value.respond_to?(:to_str)
+          num = value.to_str.gsub(/\s*/, '') # I. remove all spaces
+          num.gsub!(/([^\d\-])\./, '\1')     # II. remove misleading points in units like "Mio." or "SFr."
+          num.gsub!(/[^\d,\.\-]/i, '')       # III. remove all chars we are not interested in, like anything not related to numeric :)
+          num.gsub!("--", '')                # IV. remove double dashes, like often used in CH
+          num = num.gsub(",", '.').gsub(".") { $'.include?(".") ? "" : "." }.gsub(/\.\z/, '')  # V. intelligently convert comma to points...
+        else
+          value
+        end
+      end      
+    end
+  end
+end

data/lib/vacuum_cleaner/railtie.rb

@@ -0,0 +1,7 @@
+module VacuumCleaner
+  class Railtie < Rails::Railtie
+    config.after_initialize do
+      ActiveRecord::Base.send(:include, VacuumCleaner::Normalizations)
+    end
+  end
+end

data/lib/vacuum_cleaner.rb

@@ -1,12 +1,19 @@
 # Fancy value normalization utility for ruby (and rails),
 # see {VacuumCleaner::Normalizations} for more information about usage.
 #
+
+require 'vacuum_cleaner/normalizer'
+require 'vacuum_cleaner/normalizations'
+
 # @see VacuumCleaner::Normalizations
 # @see VacuumCleaner::Normalizer
 module VacuumCleaner
   # +VacuumCleaner+ version
-  VERSION = "1.0.0".freeze
-  
-  autoload :Normalizer, 'vacuum_cleaner/normalizer'
-  autoload :Normalizations, 'vacuum_cleaner/normalizations'
-end
+  VERSION = "1.0.1".freeze
+end
+
+if defined?(ActiveSupport) && ActiveSupport.respond_to?(:on_load)
+  ActiveSupport.on_load(:active_record) { include VacuumCleaner::Normalizations }
+else
+  ActiveRecord::Base.send(:include, VacuumCleaner::Normalizations) if defined?(ActiveRecord)  
+end

data/test/integration/active_support_integration_test.rb

@@ -0,0 +1,55 @@
+# encoding: utf-8
+$KCODE = 'U' if RUBY_VERSION < '1.9'
+
+require 'test_helper'
+
+# run init.rb, which should load VacuumCleaner
+require 'active_support'
+require 'active_support/version'
+
+require File.join(File.dirname(__FILE__), '..', '..', 'init')
+
+puts "Running integration tests against: active_support-#{ActiveSupport::VERSION::STRING}"
+
+class ActiveSupportIntegrationTest < ::Test::Unit::TestCase
+  include VacuumCleaner::Normalizations
+  
+  context "VacuumCleaner::Normalizations::ActiveSupport" do
+    context "TransliterateNormalizer" do
+      should "call AS::Inflector#translitrate if value responds to to_str" do
+        assert_equal "Geneve", TransliterateNormalizer.new.normalize_value("Genève")
+        assert_equal "Zurich", TransliterateNormalizer.new.normalize_value("Zürich")
+        assert_equal "Bern", TransliterateNormalizer.new.normalize_value("Bern")
+        assert_equal "", TransliterateNormalizer.new.normalize_value("")
+        assert_equal "\n ", TransliterateNormalizer.new.normalize_value("\n ")
+        
+        assert_nil TransliterateNormalizer.new.normalize_value(nil)
+        assert_nil TransliterateNormalizer.new.normalize_value(12.5)
+      end
+    end
+    
+    context "TitleizeNormalizer" do
+      should "call #titleize on string, else return nil" do
+        assert_equal "My First Day", TitleizeNormalizer.new.normalize_value("my first day")
+        assert_equal "My Mentor", TitleizeNormalizer.new.normalize_value("MY MENTOR")
+        assert_nil TitleizeNormalizer.new.normalize_value(nil)
+        assert_nil TitleizeNormalizer.new.normalize_value(42)
+      end
+    end
+    
+    context "HumanizeNormalizer" do
+      should "call #humanize on string, else return nil" do
+        assert_equal "My old lady", HumanizeNormalizer.new.normalize_value("My Old Lady")
+        assert_equal "My two dads", HumanizeNormalizer.new.normalize_value("my two dads")
+        assert_nil HumanizeNormalizer.new.normalize_value(nil)
+        assert_nil HumanizeNormalizer.new.normalize_value(123.5)
+      end
+    end
+
+    should "translitarte and upcase input value" do
+      obj = Class.new { include VacuumCleaner::Normalizations; attr_reader :prefix; normalizes(:prefix, :upcase => true, :transliterate => true) }.new
+      obj.prefix = "Genève\n\t"
+      assert_equal "GENEVE", obj.prefix
+    end
+  end
+end

data/test/test_helper.rb

@@ -1,3 +1,6 @@
 require 'rubygems'
 require 'test/unit'
 require 'shoulda'
+
+# load redgreen, if available
+begin; require 'redgreen'; rescue LoadError; end

data/test/unit/vacuum_cleaner/normalizations/numeric_test.rb

@@ -0,0 +1,35 @@
+require 'test_helper'
+require 'vacuum_cleaner/normalizer'
+require 'vacuum_cleaner/normalizations/numeric'
+
+class VacuumCleaner::Normalizations::NumericTtest < Test::Unit::TestCase
+  include VacuumCleaner::Normalizations
+  
+  context "NumericNormalizer#normalize_value" do
+    should "remove any non numeric character, except decimal character" do
+      assert_equal "10.5", NumericNormalizer.new.normalize_value("10.5")
+      assert_equal "-12.3434", NumericNormalizer.new.normalize_value("-12.3434")      
+      assert_equal "121250", NumericNormalizer.new.normalize_value("121'250 Mio. USD")
+    end
+    
+    should "accept a negative prefix" do
+      assert_equal "-450.00", NumericNormalizer.new.normalize_value("CHF -450.00")
+      assert_equal "-.50", NumericNormalizer.new.normalize_value("- .50 SFr.")
+    end
+    
+    should "strip single trailing points" do
+      assert_equal "1250", NumericNormalizer.new.normalize_value("CHF 1250.--")      
+    end
+    
+    should "work with German/Swiss notation of comma as decimal separator" do
+      assert_equal "40.50", NumericNormalizer.new.normalize_value("CHF 40,50")
+      assert_equal "1040.50", NumericNormalizer.new.normalize_value("EUR 1.040,50")
+      assert_equal "-1100040.50", NumericNormalizer.new.normalize_value("EUR -1.100.040,50")
+    end
+    
+    should "leave numeric and nil values as is" do
+      assert_nil NumericNormalizer.new.normalize_value(nil)
+      assert_equal 12.5, NumericNormalizer.new.normalize_value(12.5)
+    end
+  end  
+end

data/test/unit/vacuum_cleaner/normalizations_test.rb

@@ -36,13 +36,16 @@ class VacuumCleaner::NormalizationsTest < Test::Unit::TestCase
     context "ClassMethods#normalizes" do
       should "throw ArgumentError if no attributes are passed in" do
         assert_raise ArgumentError do
-          klass = Class.new do
-            include VacuumCleaner::Normalizations
-            normalizes
-          end
-        end
+          klass = Class.new { include VacuumCleaner::Normalizations; normalizes }
+        end        
       end
       
+      should "throw ArgumentError if invalid/unknown normalizer is used" do
+        assert_raise ArgumentError do
+          klass = Class.new { include VacuumCleaner::Normalizations; normalizes(:name, :foobar_unkown_normalizer => true) }
+        end
+      end        
+       
       should "take a symbol as argument" do
         assert_respond_to Class.new { include VacuumCleaner::Normalizations; normalizes(:name) }, :normalizes
       end

metadata

@@ -1,12 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: vacuum_cleaner
 version: !ruby/object:Gem::Version 
+  hash: 21
   prerelease: false
   segments: 
   - 1
   - 0
-  - 0
-  version: 1.0.0
+  - 1
+  version: 1.0.1
 platform: ruby
 authors: 
 - Lukas Westermann
@@ -14,16 +15,18 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2010-04-21 00:00:00 +02:00
+date: 2010-09-09 00:00:00 +02:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
   name: shoulda
   prerelease: false
   requirement: &id001 !ruby/object:Gem::Requirement 
+    none: false
     requirements: 
     - - ">="
       - !ruby/object:Gem::Version 
+        hash: 35
         segments: 
         - 2
         - 10
@@ -35,9 +38,11 @@ dependencies:
   name: activesupport
   prerelease: false
   requirement: &id002 !ruby/object:Gem::Requirement 
+    none: false
     requirements: 
     - - ">="
       - !ruby/object:Gem::Version 
+        hash: 9
         segments: 
         - 2
         - 3
@@ -61,13 +66,17 @@ files:
 - init.rb
 - lib/vacuum_cleaner.rb
 - lib/vacuum_cleaner/normalizations.rb
+- lib/vacuum_cleaner/normalizations/active_support.rb
 - lib/vacuum_cleaner/normalizations/method.rb
+- lib/vacuum_cleaner/normalizations/numeric.rb
 - lib/vacuum_cleaner/normalizations/url.rb
 - lib/vacuum_cleaner/normalizer.rb
-- rails/init.rb
+- lib/vacuum_cleaner/railtie.rb
 - test/integration/active_record_integration_test.rb
+- test/integration/active_support_integration_test.rb
 - test/test_helper.rb
 - test/unit/vacuum_cleaner/normalizations/method_test.rb
+- test/unit/vacuum_cleaner/normalizations/numeric_test.rb
 - test/unit/vacuum_cleaner/normalizations/url_test.rb
 - test/unit/vacuum_cleaner/normalizations_test.rb
 - test/unit/vacuum_cleaner/normalizer_test.rb
@@ -81,30 +90,36 @@ rdoc_options:
 require_paths: 
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement 
+  none: false
   requirements: 
   - - ">="
     - !ruby/object:Gem::Version 
+      hash: 3
       segments: 
       - 0
       version: "0"
 required_rubygems_version: !ruby/object:Gem::Requirement 
+  none: false
   requirements: 
   - - ">="
     - !ruby/object:Gem::Version 
+      hash: 3
       segments: 
       - 0
       version: "0"
 requirements: []
 
 rubyforge_project: 
-rubygems_version: 1.3.6
+rubygems_version: 1.3.7
 signing_key: 
 specification_version: 3
 summary: Ruby (and Rails) attribute cleaning support, provides some nice default normalization strategies.
 test_files: 
 - test/integration/active_record_integration_test.rb
+- test/integration/active_support_integration_test.rb
 - test/test_helper.rb
 - test/unit/vacuum_cleaner/normalizations/method_test.rb
+- test/unit/vacuum_cleaner/normalizations/numeric_test.rb
 - test/unit/vacuum_cleaner/normalizations/url_test.rb
 - test/unit/vacuum_cleaner/normalizations_test.rb
 - test/unit/vacuum_cleaner/normalizer_test.rb

data/rails/init.rb

@@ -1,3 +0,0 @@
-require 'vacuum_cleaner'
-
-ActiveRecord::Base.class_eval { include VacuumCleaner::Normalizations } if defined?(ActiveRecord::Base) # all versions of rails