data-anonymization 0.2.0 → 0.3.0

data/README.md

@@ -2,7 +2,7 @@
 Tool to create anonymized production data dump to use for PERF and other TEST environments.
 
 ## Getting started
-Install gem using (use `pre` option to tryout edge version):
+Install gem using:
 
     $ gem install data-anonymization
 
@@ -41,23 +41,59 @@ Run using:
 
 ## Examples
 
-1. [Whitelist](https://github.com/sunitparekh/data-anonymization/blob/master/whitelist_dsl.rb)
-2. [Blacklist](https://github.com/sunitparekh/data-anonymization/blob/master/blacklist_dsl.rb)
+1. [Whitelist using Chinoook sample database](https://github.com/sunitparekh/data-anonymization/blob/master/whitelist_dsl.rb)
+2. [Blacklist using Chinoook sample database](https://github.com/sunitparekh/data-anonymization/blob/master/blacklist_dsl.rb)
+3. [Whitelist with composite primary key using DellStore sample database](https://github.com/sunitparekh/test-anonymization/blob/master/dell_whitelist.rb)
+4. [Blacklist with composite primary key using DellStore sample database](https://github.com/sunitparekh/test-anonymization/blob/master/dell_blacklist.rb)
+
+## Changelog
+
+#### 0.3.0 (Sep 4, 2012)
+
+Major changes:
+
+1. Added support for Parallel table execution
+2. Change in default String strategy from LoremIpsum to RandomString based on end user feedback.
+3. Fixed issue with table column name 'type' as this is default name for STI in activerecord.
+
+Please see the [Github 0.3.0 milestone page](https://github.com/sunitparekh/data-anonymization/issues?milestone=1&page=1&state=open) for more details on changes/fixes in release 0.3.0
+
+#### 0.2.0 (August 16, 2012)
+
+1. Added the progress bar using 'powerbar' gem. Which also shows the ETA for each table.
+2. Added More strategies
+3. Fixed default anonymization strategies for boolean and integer values
+4. Added support for composite primary key
+
+#### 0.1.2 (August 14, 2012)
+
+1. First initial release
+
+## Roadmap
+
+#### 0.4.0
+
+1. MongoDB anonymization support (NoSQL document based database support)
+
+#### 0.5.0
+
+1. Generate DSL from database and build schema from source as part of Whitelist approach.
 
 #### Share feedback
 Please use Github [issues](https://github.com/sunitparekh/data-anonymization/issues) to share feedback, feature suggestions and report issues.
 
 ## What is data anonymization?
 
-For almost all projects there is a need to have production data dump in order to run performance tests, rehearsal production releases and debugging production issues.
-However, getting production data and using it is not feasible due to multiple reasons, one of them being that personal user data would be exposed. And thus arises the need for data anonymization.
+For almost all projects there is a need for production data dump in order to run performance tests, rehearse production releases and debug production issues.
+However, getting production data and using it is not feasible due to multiple reasons, primary being privacy concerns for user data. And thus the need for data anonymization.
 This tool helps you to get anonymized production data dump using either Blacklist or Whitelist strategies.
 
 ## Anonymization Strategies
 
 ### Blacklist
 This approach essentially leaves all fields unchanged with the exception of those specified by the user, which are scrambled/anonymized (hence the name blacklist).
-Blacklist create a copy of prod database and chooses the fields to be anonymized like e.g. username, password, email, name, geo location etc. based on user specification Most of the fields had different rules e.g. password as always set to same value for all users, email needs to be valid.
+For `Blacklist` create a copy of prod database and chooses the fields to be anonymized like e.g. username, password, email, name, geo location etc. based on user specification. Most of the fields have different rules e.g. password should be set to same value for all users, email needs to be valid.
+
 The problem with this approach is that when new fields are added they will not be anonymized by default. Human error in omitting users personal data could be damaging.
 
 ```ruby
@@ -70,9 +106,9 @@ end
 
 ### Whitelist
 This approach, by default scrambles/anonymizes all fields except a list of fields which are allowed to copied as is. Hence the name whitelist.
-By default all data needs to be anonymized. So from production database sanitizing the data record by record and insert anonymized data into destination database. Source database need only be readonly.
-All fields would be anonymized using default anonymization strategies based on the datatype, unless an anonymization strategy is specified. For instance special strategies could be used for emails, passwords, usernames etc.
-A list of whitelisted fields which implies that it's okay to copy the data as is and anonymization isn't required.
+By default all data needs to be anonymized. So from production database data is sanitized record by record and inserted as anonymized data into destination database. Source database needs to be readonly.
+All fields would be anonymized using default anonymization strategy which is based on the datatype, unless a special anonymization strategy is specified. For instance special strategies could be used for emails, passwords, usernames etc.
+A whitelisted field implies that it's okay to copy the data as is and anonymization isn't required.
 This way any new field will be anonymized by default and if we need them as is, add it to the whitelist explicitly. This prevents any human error and protects sensitive information.
 
 ```ruby
@@ -84,6 +120,25 @@ database 'DatabaseName' do
 end
 ```
 
+## Tips
+
+1. In Whitelist approach make source database connection READONLY.
+2. Change [default field strategies](#default-field-strategies) to avoid using same strategy again and again in your DSL.
+3. To run anonymization in parallel at Table level, provided no FK constraint on tables use DataAnon::Parallel::Table strategy
+
+## Running in Parallel
+Currently provides capability of running anonymization in parallel at table level provided no FK constraints on tables.
+It uses [Parallel gem](https://github.com/grosser/parallel) provided by Michael Grosser.
+By default it starts multiple parallel ruby processes processing table one by one.
+```ruby
+database 'DellStore' do
+  strategy DataAnon::Strategy::Whitelist
+  execution_strategy DataAnon::Parallel::Table  # by default sequential table processing
+  ...
+end
+```
+
+
 ## DataAnon::Core::Field
 The object that gets passed along with the field strategies.
 
@@ -386,7 +441,6 @@ write your own anonymous field strategies within DSL,
   end
 ```
 
-
 ## Default field strategies
 
 ```ruby
@@ -428,38 +482,6 @@ ENV['show_progress'] = 'false'
 DataAnon::Utils::Logging.logger.level = Logger::INFO
 ```
 
-## Changelog
-
-#### 0.2.0
-
-1. Added the progress bar using 'powerbar' gem. Which also shows the ETA for each table.
-2. Added More strategies
-3. Fixed default anonymization strategies for boolean and integer values
-4. Added support for composite primary key
-
-#### 0.1.2 (August 14, 2012)
-
-1. First initial release
-
-## Roadmap
-
-#### 0.2.0
-
-1. Complete list of all the field strategies planned supporting all data types
-
-#### 0.3.0
-
-1. Run anonymization in parallel threads (performance enchantments)
-
-#### 0.4.0
-
-1. MongoDB anonymization support (NoSQL document based database support)
-
-#### 0.5.0
-
-1. Generate DSL from database and build schema from source as part of Whitelist approach.
-
-
 ## Want to contribute?
 
 1. Fork it
@@ -476,6 +498,8 @@ DataAnon::Utils::Logging.logger.level = Logger::INFO
 
 - [ThoughtWorks Inc](http://www.thoughtworks.com), for allowing us to build this tool and make it open source.
 - [Birinder](https://twitter.com/birinder_) and [Panda](https://twitter.com/sarbashrestha) for reviewing the documentation.
-
+- [Dan Abel](http://www.linkedin.com/pub/dan-abel/0/61b/9b0) for introducing me to Blacklist and Whitelist approach for data anonymization.
+- [Chirga Doshi](https://twitter.com/chiragsdoshi) for encouraging me to get this done.
+- [Aditya Karle](https://twitter.com/adityakarle) for the Logo. (Coming Soon...)
 
 

data/data-anonymization.gemspec

@@ -23,4 +23,5 @@ Gem::Specification.new do |gem|
   gem.add_dependency('rgeo', '~> 0.3.15')
   gem.add_dependency('rgeo-geojson', '~> 0.2.3')
   gem.add_dependency('powerbar', '~> 1.0.8')
+  gem.add_dependency('parallel', '~> 0.5.18')
 end

data/lib/core/database.rb

@@ -7,18 +7,26 @@ module DataAnon
         @name = name
         @strategy = DataAnon::Strategy::Whitelist
         @user_defaults = {}
+        @tables = []
+        @execution_strategy = DataAnon::Core::Sequential
+        ENV['parallel_execution'] = 'false'
       end
 
       def strategy strategy
         @strategy = strategy
       end
 
+      def execution_strategy execution_strategy
+        @execution_strategy = execution_strategy
+        ENV['parallel_execution'] = 'true' if execution_strategy == DataAnon::Parallel::Table
+      end
+
       def source_db connection_spec
-        DataAnon::Utils::SourceDatabase.establish_connection connection_spec
+        @source_database = connection_spec
       end
 
       def destination_db connection_spec
-        DataAnon::Utils::DestinationDatabase.establish_connection connection_spec
+        @destination_database = connection_spec
       end
 
       def default_field_strategies default_strategies
@@ -26,10 +34,20 @@ module DataAnon
       end
 
       def table (name, &block)
-        @strategy.new(name, @user_defaults).process_fields(&block).process
+        table = @strategy.new(@source_database, @destination_database, name, @user_defaults).process_fields(&block)
+        @tables<< table
+      end
+
+      def anonymize
+        @execution_strategy.new.anonymize @tables
       end
 
+    end
 
+    class Sequential
+      def anonymize tables
+        tables.each { |table| table.process }
+      end
     end
 
   end

data/lib/core/dsl.rb

@@ -5,7 +5,9 @@ module DataAnon
 
       def database(name, &block)
         logger.debug "Processing Database: #{name}"
-        DataAnon::Core::Database.new(name).instance_eval &block
+        database = DataAnon::Core::Database.new(name)
+        database.instance_eval &block
+        database.anonymize
       end
 
     end

data/lib/data-anonymization.rb

@@ -4,9 +4,11 @@ require "utils/logging"
 require "utils/random_int"
 require "utils/random_float"
 require "utils/random_string"
+require "utils/random_string_chars_only"
 require "utils/geojson_parser"
 require "utils/progress_bar"
 require "utils/resource"
+require "parallel/table"
 require "core/database"
 require "core/field"
 require "strategy/strategies"

data/lib/parallel/table.rb

@@ -0,0 +1,13 @@
+require 'parallel'
+
+module DataAnon
+  module Parallel
+    class Table
+
+      def anonymize tables
+        ::Parallel.each(tables) { |table| table.process }
+      end
+
+    end
+  end
+end

data/lib/strategy/base.rb

@@ -1,14 +1,14 @@
-require 'powerbar'
-
 module DataAnon
   module Strategy
     class Base
       include Utils::Logging
 
-      def initialize name, user_strategies
+      def initialize source_database, destination_database, name, user_strategies
         @name = name
         @user_strategies = user_strategies
         @fields = {}
+        @source_database = source_database
+        @destination_database = destination_database
       end
 
       def process_fields &block
@@ -50,11 +50,15 @@ module DataAnon
       end
 
       def dest_table
-        @dest_table ||= Utils::DestinationTable.create @name, @primary_keys
+        return @dest_table unless @dest_table.nil?
+        DataAnon::Utils::DestinationDatabase.establish_connection @destination_database if @destination_database
+        @dest_table = Utils::DestinationTable.create @name, @primary_keys
       end
 
       def source_table
-        @source_table ||= Utils::SourceTable.create @name, @primary_keys
+        return @source_table unless @source_table.nil?
+        DataAnon::Utils::SourceDatabase.establish_connection @source_database
+        @source_table = Utils::SourceTable.create @name, @primary_keys
       end
 
       def process

data/lib/strategy/field/default_anon.rb

@@ -4,10 +4,11 @@ module DataAnon
 
       class DefaultAnon
 
-        DEFAULT_STRATEGIES = {:string => FieldStrategy::LoremIpsum.new,
+        DEFAULT_STRATEGIES = {:string => FieldStrategy::RandomString.new,
                               :fixnum => FieldStrategy::RandomIntegerDelta.new(5),
                               :bignum => FieldStrategy::RandomIntegerDelta.new(5000),
                               :float => FieldStrategy::RandomFloatDelta.new(5.0),
+                              :bigdecimal => FieldStrategy::RandomBigDecimalDelta.new(500.0),
                               :datetime => FieldStrategy::DateTimeDelta.new,
                               :time => FieldStrategy::TimeDelta.new,
                               :date => FieldStrategy::DateDelta.new,
@@ -21,7 +22,7 @@ module DataAnon
 
         def anonymize field
           strategy = @user_defaults[field.value.class.to_s.downcase.to_sym]
-          raise "No strategy defined for datatype #{field.value.class}" unless strategy
+          raise "No strategy defined for datatype #{field.value.class}. Use 'default_field_strategies' option in your script. Refer to  http://sunitparekh.github.com/data-anonymization/#default-field-strategies for more details. " unless strategy
           strategy.anonymize field
         end
 

data/lib/strategy/field/fields.rb

@@ -7,6 +7,7 @@ require 'strategy/field/anonymous'
 require 'strategy/field/string/lorem_ipsum'
 require 'strategy/field/string/string_template'
 require 'strategy/field/string/random_string'
+require 'strategy/field/string/random_url'
 require 'strategy/field/string/formatted_string_numbers'
 
 require 'strategy/field/string/select_from_file'
@@ -18,6 +19,7 @@ require 'strategy/field/number/random_integer'
 require 'strategy/field/number/random_float'
 require 'strategy/field/number/random_integer_delta'
 require 'strategy/field/number/random_float_delta'
+require 'strategy/field/number/random_big_decimal_delta'
 
 # contact
 require 'strategy/field/contact/geojson_base'

data/lib/strategy/field/number/random_big_decimal_delta.rb

@@ -0,0 +1,19 @@
+require 'bigdecimal'
+
+module DataAnon
+  module Strategy
+    module Field
+      class RandomBigDecimalDelta
+
+        def initialize delta = 100.0
+          @delta = delta
+        end
+
+        def anonymize field
+          return BigDecimal.new("#{field.value + DataAnon::Utils::RandomFloat.generate(-@delta, +@delta)}")
+        end
+
+      end
+    end
+  end
+end

data/lib/strategy/field/number/random_float_delta.rb

@@ -8,14 +8,9 @@ module DataAnon
         end
 
         def anonymize field
-           return range(field.value-@delta,field.value+@delta)
+          return field.value + DataAnon::Utils::RandomFloat.generate(-@delta, +@delta)
         end
 
-        def range (min, max)
-          Random.new.rand * (max-min) + min
-        end
-
-
       end
     end
   end

data/lib/strategy/field/string/random_url.rb

@@ -0,0 +1,30 @@
+module DataAnon
+  module Strategy
+    module Field
+      class RandomUrl
+
+        def anonymize field
+
+          url = field.value
+          randomized_url = ""
+          protocols = url.scan(/http:\/\/|www\./)
+          protocols.each do |token|
+            url = url.gsub(token,"")
+            randomized_url += token
+          end
+
+          marker_position = 0
+
+          while marker_position < url.length
+            special_char_index = url.index(/\W/, marker_position) || url.length
+            text = url[marker_position...special_char_index]
+            randomized_url += "#{DataAnon::Utils::RandomStringCharsOnly.generate(text.length)}#{url[special_char_index]}"
+            marker_position = special_char_index + 1
+          end
+
+          randomized_url
+        end
+      end
+    end
+  end
+end

data/lib/strategy/field/string/select_from_database.rb

@@ -5,7 +5,8 @@ module DataAnon
       class SelectFromDatabase
         include Utils::Logging
 
-        def initialize table_name, field_name
+        def initialize table_name, field_name, connection_spec
+          DataAnon::Utils::SourceDatabase.establish_connection connection_spec
           source = Utils::SourceTable.create table_name, []
           @values = source.select(field_name).uniq.collect { |record| record[field_name]}
           logger.debug "For field strategy #{table_name}:#{field_name} using values #{@values} "

data/lib/utils/database.rb

@@ -29,6 +29,7 @@ module DataAnon
           self.table_name = table_name
           self.primary_keys = primary_keys if primary_keys.length > 1
           self.primary_key = primary_keys[0] if primary_keys.length == 1
+          self.inheritance_column = :_type_disabled
           self.mass_assignment_sanitizer = MassAssignmentIgnoreSanitizer.new(self)
         end
       end

data/lib/utils/progress_bar.rb

@@ -1,24 +1,51 @@
+require 'powerbar'
+
 module DataAnon
   module Utils
 
     class ProgressBar
+      include Utils::Logging
 
       def initialize table_name, total
         @total = total
         @table_name = table_name
-        @progress_bar = PowerBar.new if total > 0 && show_progress
+        @progress_bar = PowerBar.new if total > 0 && show_progress && !parallel?
       end
 
       def show_progress
         ENV['show_progress'] != 'false'
       end
 
+      def parallel?
+        ENV['parallel_execution'] == 'true'
+      end
+
       def show index
-        if @progress_bar && ((index % 1000 == 0) || (index == @total) || (index == 1))
-          @progress_bar.show(:msg => "Table: #{@table_name} (#{index}/#{@total})", :done => index, :total => @total)
+        if started(index) || regular_interval(index) || complete(index)
+          if @progress_bar
+            msg = "Table: %-15s [ %6d/%-6d ]" % [ @table_name,index,@total]
+            @progress_bar.show(:msg => msg, :done => index, :total => @total)
+          elsif parallel?
+            suffix = ""
+            suffix = "STARTED" if started(index)
+            suffix = "COMPLETE" if complete(index)
+            logger.info("Table: %-15s [ %6d/%-6d ] %s" % [ @table_name,index,@total, suffix])
+          end
         end
       end
 
+      def complete index
+        index == @total
+      end
+
+      def regular_interval index
+        index % 1000 == 0
+      end
+
+      def started index
+        index == 1
+      end
+
       def close
         @progress_bar.close if @progress_bar
       end

data/lib/utils/random_string_chars_only.rb

@@ -0,0 +1,14 @@
+module DataAnon
+  module Utils
+    class RandomStringCharsOnly
+
+      def self.generate length = nil
+        length ||= Random.new.rand 5...15
+        chars = 'abcdefghjkmnpqrstuvwxyz'
+        random_string = ''
+        length.times { random_string << chars[rand(chars.size)] }
+        random_string
+      end
+    end
+  end
+end

data/lib/version.rb

@@ -1,3 +1,3 @@
 module DataAnonymization
-  VERSION = "0.2.0"
+  VERSION = "0.3.0"
 end

data/spec/acceptance/rdbms_blacklist_spec.rb

@@ -18,6 +18,7 @@ describe "End 2 End RDBMS Blacklist Acceptance Test using SQLite database" do
       table 'customers' do
         primary_key 'cust_id'
         anonymize('email').using FieldStrategy::StringTemplate.new('test+#{row_number}@gmail.com')
+        anonymize 'terms_n_condition', 'age'
       end
     end
 

data/spec/acceptance/rdbms_whitelist_spec.rb

@@ -22,12 +22,14 @@ describe "End 2 End RDBMS Whitelist Acceptance Test using SQLite database" do
 
       table 'customers' do
         primary_key 'cust_id'
-        whitelist 'cust_id', 'address', 'zipcode'
+        whitelist 'cust_id', 'address', 'zipcode', 'blog_url'
         anonymize('first_name').using FieldStrategy::RandomFirstName.new
         anonymize('last_name').using FieldStrategy::RandomLastName.new
         anonymize('state').using FieldStrategy::SelectFromList.new(['Gujrat','Karnataka'])
         anonymize('phone').using FieldStrategy::RandomPhoneNumber.new
         anonymize('email').using FieldStrategy::StringTemplate.new('test+#{row_number}@gmail.com')
+        anonymize 'terms_n_condition', 'age', 'longitude'
+        anonymize('latitude').using FieldStrategy::RandomFloatDelta.new(2.0)
       end
     end
 
@@ -42,7 +44,10 @@ describe "End 2 End RDBMS Whitelist Acceptance Test using SQLite database" do
     new_rec.zipcode.should == '411048'
     new_rec.phone.should_not be "9923700662"
     new_rec.email.should == 'test+1@gmail.com'
-
+    [true,false].should include(new_rec.terms_n_condition)
+    new_rec.age.should be_between(0,100)
+    new_rec.latitude.should be_between( 38.689060, 42.689060)
+    new_rec.longitude.should be_between( -84.044636, -64.044636)
 
   end
 end

data/spec/spec_helper.rb

@@ -7,7 +7,7 @@ ENV['show_progress'] = 'false'
 
 Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f}
 
-DataAnon::Utils::Logging.logger.level = Logger::INFO
+DataAnon::Utils::Logging.logger.level = Logger::WARN
 
 RSpec.configure do |config|
   config.expect_with :rspec

data/spec/strategy/field/datetime/date_delta_spec.rb

@@ -11,7 +11,6 @@ describe FieldStrategy::DateDelta do
     let(:date_difference) {anonymized_value - field.value}
 
     it { anonymized_value.should be_kind_of Date}
-    it { anonymized_value.should_not == Date.new(2011,4,7) }
     it { date_difference.should be_between(-5.days, 5.days) }
   end
 

data/spec/strategy/field/default_anon_spec.rb

@@ -32,6 +32,13 @@ describe FieldStrategy::DefaultAnon do
     it { anonymized_value.should be_kind_of Fixnum  }
   end
 
+  describe 'anonymized bignum value' do
+    let(:field) {DataAnon::Core::Field.new('int_field',2348723489723847382947,1,nil)}
+    let(:anonymized_value) {DefaultAnon.new.anonymize(field)}
+
+    it { anonymized_value.should be_kind_of Bignum  }
+  end
+
   describe 'anonymized string value' do
     let(:field) {DataAnon::Core::Field.new('string_field','String',1,nil)}
     let(:anonymized_value) {DefaultAnon.new.anonymize(field)}

data/spec/strategy/field/number/random_big_decimal_delta_spec.rb

@@ -0,0 +1,20 @@
+require "spec_helper"
+require 'bigdecimal'
+
+describe FieldStrategy::RandomBigDecimalDelta do
+
+  RandomBigDecimalDelta = FieldStrategy::RandomBigDecimalDelta
+  let(:field) {DataAnon::Core::Field.new('decimal_field',BigDecimal.new("53422342378687687342893.23324"),1,nil)}
+
+  describe 'anonymized big decimal should not be the same as original value' do
+    let(:anonymized_value) {RandomBigDecimalDelta.new.anonymize(field)}
+
+    it {anonymized_value.should_not equal field.value}
+  end
+
+  describe 'anonymized value returned should be big decimal' do
+    let(:anonymized_value) {RandomBigDecimalDelta.new.anonymize(field)}
+
+    it { anonymized_value.should be_kind_of BigDecimal }
+  end
+end

data/spec/strategy/field/number/random_float_delta_spec.rb

@@ -6,16 +6,14 @@ describe FieldStrategy::RandomFloatDelta do
   let(:field) {DataAnon::Core::Field.new('float_field',5.5,1,nil)}
 
   describe 'anonymized float should not be the same as original value' do
-    let(:anonymized_float) {RandomFloatDelta.new(5).anonymize(field)}
+    let(:anonymized_value) {RandomFloatDelta.new(5).anonymize(field)}
 
-    it {anonymized_float.should_not equal field.value}
+    it {anonymized_value.should_not equal field.value}
   end
 
   describe 'anonymized value returned should be a float' do
-    let(:anonymized_float) {RandomFloatDelta.new(5).anonymize(field)}
+    let(:anonymized_value) {RandomFloatDelta.new(5).anonymize(field)}
 
-    it { is_float = anonymized_float.is_a? Float
-         is_float.should be true
-    }
+    it { anonymized_value.should be_kind_of Float }
   end
 end

data/spec/strategy/field/string/random_url_spec.rb

@@ -0,0 +1,15 @@
+require "spec_helper"
+
+describe FieldStrategy::RandomUrl do
+
+  RandomUrl = FieldStrategy::RandomUrl
+
+  describe 'anonymized url must not be the same as original url' do
+    let(:field) {DataAnon::Core::Field.new('string_field','http://fakeurl.com',1,nil)}
+    let(:anonymized_url) {RandomUrl.new.anonymize(field)}
+
+    it {anonymized_url.should_not equal field.value}
+    it {anonymized_url.should match /https?:\/\/[\S]+/}
+  end
+
+end

data/spec/strategy/field/string/select_from_database_spec.rb

@@ -2,17 +2,13 @@ require "spec_helper"
 
 describe FieldStrategy::SelectFromDatabase do
 
-  before(:each) do
-    source = {:adapter => 'sqlite3', :database => 'sample-data/chinook.sqlite'}
-    DataAnon::Utils::SourceDatabase.establish_connection source
-  end
-
   SelectFromDatabase = FieldStrategy::SelectFromDatabase
   let(:field) { DataAnon::Core::Field.new('name', 'Abcd', 1, nil) }
+  let(:source)  { {:adapter => 'sqlite3', :database => 'sample-data/chinook.sqlite'} }
 
   describe 'more than one values in predefined list' do
 
-    let(:anonymized_value) { SelectFromDatabase.new('MediaType','Name').anonymize(field) }
+    let(:anonymized_value) { SelectFromDatabase.new('MediaType','Name', source).anonymize(field) }
 
     it { anonymized_value.should_not be('Abcd') }
     it { anonymized_value.should_not be_empty }

data/spec/support/customer_sample.rb

@@ -12,6 +12,11 @@ class CustomerSample
         t.string :zipcode
         t.string :phone
         t.string :email
+        t.string :blog_url
+        t.boolean :terms_n_condition
+        t.integer :age
+        t.float :latitude
+        t.float :longitude
       end
     end
   end
@@ -30,7 +35,8 @@ class CustomerSample
   SAMPLE_DATA = {:cust_id => 100, :first_name => "Sunit", :last_name => "Parekh",
                  :birth_date => Date.new(1977,7,8), :address => "F 501 Shanti Nagar",
                  :state => "Maharastra", :zipcode => "411048", :phone => "9923700662",
-                 :email => "parekh.sunit@gmail.com"}
+                 :email => "parekh.sunit@gmail.com", :terms_n_condition => true,
+                 :age => 34, :longitude => -74.044636, :latitude => +40.689060}
 
   def self.insert_record connection_spec, data_hash = SAMPLE_DATA
     DataAnon::Utils::TempDatabase.establish_connection connection_spec

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: data-anonymization
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
   prerelease: 
 platform: ruby
 authors:
@@ -11,7 +11,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-08-17 00:00:00.000000000 Z
+date: 2012-09-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -109,6 +109,22 @@ dependencies:
     - - ~>
       - !ruby/object:Gem::Version
         version: 1.0.8
+- !ruby/object:Gem::Dependency
+  name: parallel
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 0.5.18
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 0.5.18
 description: Data anonymization tool for RDBMS databases
 email:
 - parekh.sunit@gmail.com
@@ -134,6 +150,7 @@ files:
 - lib/core/dsl.rb
 - lib/core/field.rb
 - lib/data-anonymization.rb
+- lib/parallel/table.rb
 - lib/strategy/base.rb
 - lib/strategy/blacklist.rb
 - lib/strategy/field/anonymous.rb
@@ -158,6 +175,7 @@ files:
 - lib/strategy/field/name/random_full_name.rb
 - lib/strategy/field/name/random_last_name.rb
 - lib/strategy/field/name/random_user_name.rb
+- lib/strategy/field/number/random_big_decimal_delta.rb
 - lib/strategy/field/number/random_float.rb
 - lib/strategy/field/number/random_float_delta.rb
 - lib/strategy/field/number/random_integer.rb
@@ -166,6 +184,7 @@ files:
 - lib/strategy/field/string/formatted_string_numbers.rb
 - lib/strategy/field/string/lorem_ipsum.rb
 - lib/strategy/field/string/random_string.rb
+- lib/strategy/field/string/random_url.rb
 - lib/strategy/field/string/select_from_database.rb
 - lib/strategy/field/string/select_from_file.rb
 - lib/strategy/field/string/select_from_list.rb
@@ -181,6 +200,7 @@ files:
 - lib/utils/random_float.rb
 - lib/utils/random_int.rb
 - lib/utils/random_string.rb
+- lib/utils/random_string_chars_only.rb
 - lib/utils/resource.rb
 - lib/version.rb
 - resources/UK_addresses.geojson
@@ -210,6 +230,7 @@ files:
 - spec/strategy/field/name/random_full_name_spec.rb
 - spec/strategy/field/name/random_last_name_spec.rb
 - spec/strategy/field/name/random_user_name_spec.rb
+- spec/strategy/field/number/random_big_decimal_delta_spec.rb
 - spec/strategy/field/number/random_float_delta_spec.rb
 - spec/strategy/field/number/random_float_spec.rb
 - spec/strategy/field/number/random_integer_delta_spec.rb
@@ -218,6 +239,7 @@ files:
 - spec/strategy/field/string/formatted_string_numbers_spec.rb
 - spec/strategy/field/string/lorem_ipsum_spec.rb
 - spec/strategy/field/string/random_string_spec.rb
+- spec/strategy/field/string/random_url_spec.rb
 - spec/strategy/field/string/select_from_database_spec.rb
 - spec/strategy/field/string/select_from_list_spec.rb
 - spec/strategy/field/string/string_template_spec.rb
@@ -277,6 +299,7 @@ test_files:
 - spec/strategy/field/name/random_full_name_spec.rb
 - spec/strategy/field/name/random_last_name_spec.rb
 - spec/strategy/field/name/random_user_name_spec.rb
+- spec/strategy/field/number/random_big_decimal_delta_spec.rb
 - spec/strategy/field/number/random_float_delta_spec.rb
 - spec/strategy/field/number/random_float_spec.rb
 - spec/strategy/field/number/random_integer_delta_spec.rb
@@ -285,6 +308,7 @@ test_files:
 - spec/strategy/field/string/formatted_string_numbers_spec.rb
 - spec/strategy/field/string/lorem_ipsum_spec.rb
 - spec/strategy/field/string/random_string_spec.rb
+- spec/strategy/field/string/random_url_spec.rb
 - spec/strategy/field/string/select_from_database_spec.rb
 - spec/strategy/field/string/select_from_list_spec.rb
 - spec/strategy/field/string/string_template_spec.rb