mongo_populator 0.1.0

data/CHANGELOG.md

@@ -0,0 +1,3 @@
+# 0.1.0 (September 14th, 2011)
+
+* initial release based on fork of http://github.com/ryanb/populator

data/Gemfile

@@ -0,0 +1,5 @@
+source "http://rubygems.org"
+
+gem 'mongo'
+
+gemspec

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2011 Benjamin Cullen-Kerney, Ryan Bates
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,76 @@
+# MongoPopulator
+
+`mongo_populator` populates a MongoDB database with placeholder data. It is built upon [`populator`](https://github.com/ryanb/populator) by Ryan Bates, but it works with the `mongo` gem in standalone scripts, and is not tied to any particular framework.
+
+## Installation
+
+    gem install mongo_populator
+
+## Usage
+
+This gem adds a "populate" method to a `Mongo::Collection`. Pass the number of documents you want to create along with a block. In the block you can set the field values for each document.
+
+    require 'rubygems'
+    require 'mongo_populator'
+
+    db = Mongo::Connection.new("localhost", 27017).db("test-db")    
+    
+    article_collection = db.collection('article')
+    article_collection.populate(100) do |article|
+      article.title       = MongoPopulator.words(4..6).capitalize
+      article.slug        = article.title.downcase.tr(' ','-')
+      article.published   = true
+      article.created_at  = (Time.now - 604800)..Time.now
+      article.body        = MongoPopulator.paragraphs(5..7)
+    end
+
+Unlike the original Populator, we are letting MongoDB set each ObjectId. This makes setting up relationships only slightly more laborious.
+
+    article_collection = db.collection('article')
+    article_collection.populate(100) do |article|
+      ...
+    end
+
+    # create array of article _ids to select from
+    article_ids = db.collection('article').distinct('_id')
+
+    comment_collection = db.collection('comments')
+    comment_collection.populate(1000) do |comment|
+      ...
+      comment.article = article_ids
+    end
+
+That will create an average of 10 comments for each article. Embedded documents are not yet supported.
+
+Passing a range or array of values will randomly select one.
+
+    person.gender = ['male', 'female']
+
+This will create 1000 to 5000 men or women with the annual income between 10,000 and 200,000.
+
+If you need to generate fake data, there are a few methods to do this.
+
+    MongoPopulator.words(3) # generates 3 random words separated by spaces
+    MongoPopulator.words(10..20) # generates between 10 and 20 random words
+    MongoPopulator.sentences(5) # generates 5 sentences
+    MongoPopulator.paragraphs(3) # generates 3 paragraphs
+
+For fancier data generation, try the [Faker gem](http://faker.rubyforge.org).
+
+To persist arrays in your documents, use either #items to save a certain number of items randomly selected from a set, or #array to save a specific array.
+
+    MongoPopulator.items(1..5, %w(ape bear cat dog elephant firefox)) # populates array with provided terms
+    MongoPopulator.items(10..20) # populates array with random words
+    MongoPopulator.array('red', 'green', 'blue') # saves `['red', 'green', 'blue']` exactly
+
+## Development
+
+Problems or questions? Add an [issue on GitHub](https://github.com/bak/mongo_populator/issues) or fork the project and send a pull request.
+
+## Special Thanks
+
+MongoPopulator is highly derivative of, and heavily reuses, the work of Ryan Bates [via Populator](https://github.com/ryanb/populator/). Thanks, Ryan.
+
+## Special Thanks for the original Populator
+
+Special thanks to Zach Dennis for his ar-extensions gem which some of this code is based on. Also many thanks to the [contributors](https://github.com/ryanb/populator/contributors). See the [CHANGELOG](https://github.com/ryanb/populator/blob/master/CHANGELOG.rdoc) for the full list.

data/Rakefile

@@ -0,0 +1,13 @@
+require 'rubygems'
+require 'rake'
+require 'rspec/core/rake_task'
+require 'yaml'
+
+task :default => :spec
+
+task :spec do
+    desc "Run specs"
+    RSpec::Core::RakeTask.new do |t|
+      t.rspec_opts = ["-c"]
+    end
+end

data/lib/mongo_populator/collection_additions.rb

@@ -0,0 +1,18 @@
+module MongoPopulator
+  module CollectionAdditions
+
+    # Populate a Mongo::Collection with data
+    def populate(amount, &block)
+      Factory.for_collection(self).populate(amount, &block)
+    end
+  end
+  
+  # extend host class with class methods when we're included
+  def self.included(host_class)
+    host_class.extend(CollectionAdditions)
+  end
+end
+
+Mongo::Collection.class_eval do
+  include MongoPopulator::CollectionAdditions
+end

data/lib/mongo_populator/factory.rb

@@ -0,0 +1,73 @@
+module MongoPopulator
+  # Builds multiple Populator::Record instances and saves them to the database
+  class Factory
+
+    @factories = {}
+    @depth = 0
+
+    # Fetches the factory dedicated to a given collection. You should always use this
+    # method instead of instatiating a factory directly so that a single factory is
+    # shared on multiple calls.
+    def self.for_collection(collection)
+      @factories[collection] ||= new(collection)
+    end    
+
+    # Find all remaining factories and call save_records on them.
+    def self.save_remaining_records
+      @factories.values.each do |factory|
+        factory.save_records
+      end
+      @factories = {}
+    end    
+
+    # Keep track of nested factory calls so we can save the remaining records once we
+    # are done with the base factory. This makes Populator more efficient when nesting
+    # factories.
+    def self.remember_depth
+      @depth += 1
+      yield
+      @depth -= 1
+      save_remaining_records if @depth.zero?
+    end    
+
+    # Use for_collection instead of instatiating a record directly.
+    def initialize(collection)
+      @collection = collection
+      @records = []
+    end    
+
+    # Entry method for building records. Delegates to build_records after remember_depth.
+    def populate(amount, &block)
+      self.class.remember_depth do
+        build_records(MongoPopulator.interpret_value(amount), &block)
+      end
+    end
+
+    # Builds multiple MongoPopulator::Record instances and calls save_records them when
+    # :per_query limit option is reached.
+    def build_records(amount, &block)
+      amount.times do
+        # index = last_id_in_database + @records.size + 1
+        record = Record.new(@collection)
+        block.call(record) if block
+        @records << record.attributes
+        save_records 
+      end
+    end
+
+    # Saves the records to the database 
+    def save_records
+      @records.each do |record|
+        @collection.insert(record)
+      end
+      # @last_id_in_database = @records.last[:_id]
+      @records.clear
+    end
+
+    private
+
+    # def last_id_in_database
+    #   @last_id_in_database ||= @collection.distinct('_id').last
+    # end
+  end
+end

data/lib/mongo_populator/mongo_array.rb

@@ -0,0 +1,5 @@
+module MongoPopulator
+
+  # MongoArrays can be passed directly to attributes and will be saved as arrays in mongo.
+  class MongoArray < Array; end
+end

data/lib/mongo_populator/random.rb

@@ -0,0 +1,86 @@
+module MongoPopulator
+  # This module adds several methods for generating random data which can be
+  # called directly on Populator.
+  
+  module Random
+    WORDS = %w(alias consequatur aut perferendis sit voluptatem accusantium doloremque aperiam eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo aspernatur aut odit aut fugit sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt neque dolorem ipsum quia dolor sit amet consectetur adipisci velit sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem ut enim ad minima veniam quis nostrum exercitationem ullam corporis nemo enim ipsam voluptatem quia voluptas sit suscipit laboriosam nisi ut aliquid ex ea commodi consequatur quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae et iusto odio dignissimos ducimus qui blanditiis praesentium laudantium totam rem voluptatum deleniti atque corrupti quos dolores et quas molestias excepturi sint occaecati cupiditate non provident sed ut perspiciatis unde omnis iste natus error similique sunt in culpa qui officia deserunt mollitia animi id est laborum et dolorum fuga et harum quidem rerum facilis est et expedita distinctio nam libero tempore soluta nobis est eligendi optio cumque nihil impedit quo porro quisquam est qui minus id quod maxime placeat facere possimus omnis voluptas assumenda est omnis dolor repellendus temporibus autem quibusdam et aut consequatur vel illum qui dolorem eum fugiat quo voluptas nulla pariatur at vero eos et accusamus officiis debitis aut rerum necessitatibus saepe eveniet ut et voluptates repudiandae sint et molestiae non recusandae itaque earum rerum hic tenetur a sapiente delectus ut aut reiciendis voluptatibus maiores doloribus asperiores repellat)
+
+    # Pick a random value out of a given range.
+    def value_in_range(range)
+      case range.first
+      when Integer then number_in_range(range)
+      when Time then time_in_range(range)
+      when Date then date_in_range(range)
+      else range.to_a[rand(range.to_a.size)]
+      end
+    end
+
+    # Generate a given number of words. If a range is passed, it will generate
+    # a random number of words within that range.
+    def words(total)
+      (1..interpret_value(total)).map { WORDS[rand(WORDS.size)] }.join(' ')
+    end
+
+    # Generate a given number of sentences. If a range is passed, it will generate
+    # a random number of sentences within that range.
+    def sentences(total)
+      (1..interpret_value(total)).map do
+        words(5..20).capitalize
+      end.join('. ')
+    end
+
+    # Generate a given number of paragraphs. If a range is passed, it will generate
+    # a random number of paragraphs within that range.
+    def paragraphs(total)
+      (1..interpret_value(total)).map do
+        sentences(3..8).capitalize
+      end.join("\n\n")
+    end
+        
+    # Generate a given number of items, or for a range, generate a random number of
+    # items within that range, using values in array, or random words. Returns MongoArray.
+    def items(total, arr=nil)
+      out = MongoArray.new
+      (1..interpret_value(total)).map do
+        out << (arr ? arr[rand(arr.size)] : words(1))
+      end
+      return out
+    end
+
+    # Simply pass the values back out as a MongoArray
+    def array(*values)
+      MongoArray.new(values)
+    end
+
+    # If an array or range is passed, a random value will be selected to match.
+    # All other values are simply returned.
+    def interpret_value(value)
+      case value
+      when MongoArray then value
+      when Array then value[rand(value.size)]
+      when Range then value_in_range(value)
+      else value
+      end
+    end
+
+    private
+
+    def time_in_range(range)
+      Time.at number_in_range(Range.new(range.first.to_i, range.last.to_i, range.exclude_end?))
+    end
+
+    def date_in_range(range)
+      Date.jd number_in_range(Range.new(range.first.jd, range.last.jd, range.exclude_end?))
+    end
+
+    def number_in_range(range)
+      if range.exclude_end?
+        rand(range.last - range.first) + range.first
+      else
+        rand((range.last+1) - range.first) + range.first
+      end
+    end
+  end
+
+  extend Random # load it into the populator module directly so we can call the methods
+end

data/lib/mongo_populator/record.rb

@@ -0,0 +1,30 @@
+module MongoPopulator
+  # This is what is passed to the block when calling populate.
+  class Record
+    attr_accessor :attributes
+
+    # Creates a new instance of Record. 
+    def initialize(collection)
+      @attributes = {}
+    end
+
+    def attributes=(values_hash)
+      values_hash.each_pair do |key, value|
+        value = value.call if value.is_a?(Proc)
+        self.send((key.to_s + "=").to_sym, value)
+      end
+    end
+
+    private
+
+    def method_missing(sym, *args, &block)
+      name = sym.to_s
+      if name.include? '='
+        @attributes[name.sub('=', '').to_sym] = MongoPopulator.interpret_value(args.first)
+      else
+        @attributes[sym]
+      end
+    end
+  end
+end
+

data/lib/mongo_populator.rb

@@ -0,0 +1,12 @@
+$:.unshift(File.dirname(__FILE__))
+
+require 'mongo'
+require 'mongo_populator/collection_additions'
+require 'mongo_populator/factory'
+require 'mongo_populator/record'
+require 'mongo_populator/mongo_array'
+require 'mongo_populator/random'
+
+# MongoPopulator is made up of several parts. To start, see MongoPopulator::ModelAdditions.
+module MongoPopulator
+end

data/spec/README.rdoc

@@ -0,0 +1,5 @@
+= Running Populator Specs
+
+To run the specs, make sure you have mongo running on localhost port 27017, and
+
+  rake spec

data/spec/mongo_populator/collection_additions_spec.rb

@@ -0,0 +1,27 @@
+require "spec_helper"
+
+describe MongoPopulator::CollectionAdditions do
+  before(:each) do
+    @collection = $db.collection('test-collection')
+  end
+
+  it "should add populate method to collection class" do
+    @collection.should respond_to(:populate)
+  end
+
+  it "should add 10 records to database" do
+    @collection.populate(10)
+    @collection.count.should == 10
+  end
+
+  it "should set attribute columns" do
+    @collection.populate(1) do |record|
+      record.name = "foo"
+    end
+    @collection.distinct('name').last.should == "foo"
+  end
+
+  after(:each) do
+    @collection.drop
+  end
+end

data/spec/mongo_populator/factory_spec.rb

@@ -0,0 +1,18 @@
+require "spec_helper"
+
+describe MongoPopulator::Factory do
+  before(:each) do
+    @collection = $db.collection('test-collection')
+    @factory = MongoPopulator::Factory.for_collection(@collection)
+  end
+
+  it "should generate within range" do
+    @factory.populate(2..4)
+    @collection.count.should >= 2
+    @collection.count.should <= 4
+  end
+
+  after(:each) do
+    @collection.drop
+  end
+end

data/spec/mongo_populator/random_spec.rb

@@ -0,0 +1,49 @@
+require "spec_helper"
+
+describe MongoPopulator::Random do
+  it "should pick a random number in range excluding last value" do
+    MongoPopulator.stubs(:rand).with(5).returns(3)
+    MongoPopulator.value_in_range(10...15).should == 13
+  end
+
+  it "should pick a random number in range including last value" do
+    MongoPopulator.stubs(:rand).with(5).returns(3)
+    MongoPopulator.value_in_range(10..14).should == 13
+  end
+
+  it "should pick a random time in range" do
+    start_time = Time.now - 172800
+    end_time = Time.now
+    MongoPopulator.stubs(:rand).with(end_time.to_i-start_time.to_i).returns(1)
+    MongoPopulator.value_in_range(start_time...end_time).should == Time.at(start_time.to_i + 1)
+  end
+
+  it "should pick a random date in range" do
+    start_date = Date.today - 63072000 
+    end_date = Date.today
+    MongoPopulator.stubs(:rand).with(end_date.jd-start_date.jd).returns(1)
+    MongoPopulator.value_in_range(start_date...end_date).should == Date.jd(start_date.jd + 1)
+  end
+
+  it "should pick a random string by converting to array" do
+    MongoPopulator.stubs(:rand).with(5).returns(2)
+    MongoPopulator.value_in_range('a'..'e').should == 'c'
+  end
+
+  it "should pick 3 random words" do
+    MongoPopulator.words(3).split.should have(3).records
+  end
+
+  it "should pick a random number of random words" do
+    MongoPopulator.stubs(:rand).returns(3)
+    MongoPopulator.words(10...15).split.should have(13).records
+  end
+
+  it "should generate 3 random sentences" do
+    MongoPopulator.sentences(3).split(/\. [A-Z]/).should have(3).records
+  end
+
+  it "should generate 3 random paragraphs" do
+    MongoPopulator.paragraphs(3).split(/\n\n/).should have(3).records
+  end
+end

data/spec/mongo_populator/record_spec.rb

@@ -0,0 +1,37 @@
+require "spec_helper"
+
+describe MongoPopulator::Record do
+  before(:each) do
+    @collection = $db.collection('test-collection')
+  end
+
+  it "should pick random number from range" do
+    record = MongoPopulator::Record.new(@collection)
+    record.stock = 2..5
+    record.stock.should >= 2
+    record.stock.should <= 5
+  end
+
+  it "should pick random value from array" do
+    record = MongoPopulator::Record.new(@collection)
+    record.name = %w[foo bar]
+    %w[foo bar].should include(record.name)
+  end
+
+  it "should allow set via attributes hash" do
+    record = MongoPopulator::Record.new(@collection)
+    record.attributes = {:stock => 2..5}
+    record.stock.should >= 2
+    record.stock.should <= 5
+  end
+
+  it "should take a proc object via attributes hash" do
+    record = MongoPopulator::Record.new(@collection)
+    record.attributes = {:stock => lambda {15}}
+    record.stock.should == 15
+  end
+
+  after(:each) do
+    @collection.drop
+  end
+end

data/spec/spec_helper.rb

@@ -0,0 +1,17 @@
+require 'rubygems'
+require 'bundler/setup'
+
+Bundler.require(:default)
+
+require 'mongo'
+require 'mongo_populator'
+
+RSpec.configure do |config|
+  config.mock_with :mocha
+
+  begin
+    $db = Mongo::Connection.new("localhost", 27017).db("test-db")    
+  rescue Mongo::ConnectionFailure
+    p "Could not connect to Mongo. Ensure that db server is running at localhost port 27017."
+  end
+end

metadata

@@ -0,0 +1,95 @@
+--- !ruby/object:Gem::Specification
+name: mongo_populator
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+  prerelease: 
+platform: ruby
+authors:
+- Benjamin Cullen-Kerney
+- Ryan Bates
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2011-09-14 00:00:00.000000000Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: mongo
+  requirement: &70101862524240 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.3.1
+  type: :runtime
+  prerelease: false
+  version_requirements: *70101862524240
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: &70101862523760 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 2.6.0
+  type: :development
+  prerelease: false
+  version_requirements: *70101862523760
+- !ruby/object:Gem::Dependency
+  name: mocha
+  requirement: &70101862550920 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 0.10.0
+  type: :development
+  prerelease: false
+  version_requirements: *70101862550920
+description: Mass populate MongoDB.
+email: ben.kerney -> gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- lib/mongo_populator/collection_additions.rb
+- lib/mongo_populator/factory.rb
+- lib/mongo_populator/mongo_array.rb
+- lib/mongo_populator/random.rb
+- lib/mongo_populator/record.rb
+- lib/mongo_populator.rb
+- spec/mongo_populator/collection_additions_spec.rb
+- spec/mongo_populator/factory_spec.rb
+- spec/mongo_populator/random_spec.rb
+- spec/mongo_populator/record_spec.rb
+- spec/README.rdoc
+- spec/spec_helper.rb
+- CHANGELOG.md
+- Gemfile
+- LICENSE
+- Rakefile
+- README.md
+homepage: http://github.com/bak/mongo_populator
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: 1.8.0
+requirements: []
+rubyforge_project: mongo_populator
+rubygems_version: 1.8.6
+signing_key: 
+specification_version: 3
+summary: Mass populate MongoDB.
+test_files: []