rubberband_flamethrower 0.4.0 → 0.4.1

data/README.md

@@ -1,6 +1,6 @@
 # Rubberband Flamethrower
 
-Rubberband Flamethrower is a gem for inserting faked data into an Elastic Search server and providing basic benchmarks. It creates and inserts fake data objects with three fields (message, username, and post_date) and times the process. It inserts in batches of 5,000 objects by default but can be configured to insert any number of objects. Here is an sample generated data object:
+Rubberband Flamethrower is a gem for inserting faked data into an Elastic Search server and providing basic benchmarks. It creates and inserts fake data objects with three fields (message, username, and post_date) and times the process. It inserts in batches of 500 objects by default but can be configured to insert any number of objects. Here is an sample generated data object:
 
 	{
 		"message":"utilizing plowed popularizing demeanor anesthesia specializes chaperon pedaling.",
@@ -29,7 +29,7 @@ Start an Elastic Search node:
 
 ### Ruby
 
-You do not need a rails project to use this gem though it is easier to use if you do.  It has been designed with ruby 1.9.1 and above in mind. The sample method of the Array class is used in the code and was not a part of the 1.8.7 release.
+It has been designed with ruby 1.9.1 and above in mind. The sample method of the Array class is used in the code and was not a part of the 1.8.7 release.
 
 ## Installation
 
@@ -43,9 +43,9 @@ Or if you will be using this as a part of a Rails project, you can add the gem t
 
 ### Command Line Executable
 
-Once the gem is installed and you have an Elastic Search server running you are ready to begin inserting fake data. You can run the gem from the command line using the "flamethrower" command.  The command without any arguments will display the help screen:
+Once the gem is installed and you have an Elastic Search server running you are ready to begin inserting fake data. You can run the gem from the command line using the "flamethrower" command.  
 
-	flamethrower
+#### Fire
 
 To start a batch insert into the local Elastic Search server you add the argument "fire" to the command:
 
@@ -53,9 +53,9 @@ To start a batch insert into the local Elastic Search server you add the argumen
 
 By default it will insert 5000 documents starting with document ID 1 into an Elastic Search index named "twitter" of type "tweet" into a server node located at http://localhost:9200.
 
-You can configure what is inserted by passing additional parameters. There are 5 parameters accepted by the `flamethrower fire` command, all of which have a default value if left blank. Here are the parameters in order with their default values: (how_many=5000, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet")
+You can configure what is inserted by passing additional parameters. The parameters accepted by the `flamethrower fire` command all have a default value if left blank. Here are the parameters in order with their default values: (how_many=500, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet")
 
-To Insert 10,000 instead of 5,000:
+To Insert 10,000 instead of 500:
 
 	flamethrower fire 10000
 
@@ -69,8 +69,31 @@ To Insert 2,000 starting with the ID 1 to a server located at http://es.com:9200
 
 To put your documents into an index named "facebook" instead of "twitter" with a type of "message" instead of "tweet"
 
-	flamethrower fire 5000 1 "http://localhost:9200" "facebook" "message"
+	flamethrower fire 500 1 "http://localhost:9200" "facebook" "message"
+
+#### Auto
+
+The "auto" argument can be used to repeatedly run the "flamethrower fire" command, timing each run.  By default it will run the command 3 times.
+
+	flamethrower auto
+
+This can be configured much like the above example, though there is one additional parameter, which is supplied first and represents the number of times to run the "flamethrower fire" command. Here are the parameters in order with their default values: (how_many_batches=3, per_batch=5000, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet", id_overwrite="n")
+
+To run the "flamethrower fire" command 5 times in a row instead of the default 3:
+
+	flamethrower auto 5
+
+To run the "flamethrower  fire" command 5 times, inserting 5,000 objects each time:
+
+	flamethrower auto 5 5000
+	
+The id_overwrite parameter determines the ID strategy used for subsequent batches in the auto command. When set to \"n\" (which it is by default) each batch will be writing new data with fresh IDs to the Elastic Search server, simulating a system where data is constantly being inserted and not updated. 5 batches of 500 with an \"n\" would use the IDs 1-2,500. When it is set to \"y\" each batch will simulate overwriting existing data in the Elastic Search server, simulating a system where data is constantly being updated (after the initial batch). 5 batches of 500 with a setting of \"y\" would use the IDs 1-500 on each batch.
+	
+#### Help
+
+The command with the argument "help" or without any arguments will display the help screen:
 
+	flamethrower help
 
 ### IRB/Ruby Scripts
 

data/VERSION

@@ -1 +1 @@
-0.4.0
+0.4.1

data/bin/flamethrower

@@ -7,15 +7,15 @@ require 'rubberband_flamethrower'
 
 args = ARGV.dup
 ARGV.clear
-command = args.shift.strip rescue 'help'
+command = args.shift.strip rescue 'auto'
 
 case command
 when "fire"
   RubberbandFlamethrower.fire(*args)
-when "auto"
-  RubberbandFlamethrower.auto(*args)
-else
+when "help"
   RubberbandFlamethrower.help
+else
+  RubberbandFlamethrower.auto(*args)
 end
 
 #binary adapted from old heroku source

data/lib/rubberband_flamethrower/flamethrower.rb

@@ -1,9 +1,30 @@
 # This class is designed to insert objects created with the DataGenerator class into Elastic Search
+
+require "active_support/core_ext"
+require 'httparty'
+require "benchmark"
+require File.dirname(__FILE__)+"/data_generator.rb"
+
 module RubberbandFlamethrower
   class Flamethrower
+
+    # Batch inserts randomly generated JSON data objects into an Elastic Search server.
+    # Example of the object inserted:
+    #  {
+    #   "message":"utilizing plowed popularizing demeanor anesthesia specializes chaperon pedaling.",
+    #   "username":"pummeling",
+    #   "post_date":"20130408T15:41:28"
+    #  }
+    # 
+    # @param [Integer] how_many - how many randomly generated data objects to insert
+    # @param [Integer] starting_id - starting id for randomly generated data objects, will increment from this number
+    # @param [String] server_url - url of the Elastic Search server
+    # @param [String] index - name of the Elastic Search index to insert data into
+    # @param [String] type - name of the Elastic Search type to insert data into
+    # @param [Integer] printing_level - 0 = no printing | 1 = progress dots | 2 = debug level
     def fire(how_many, starting_id, server_url, index, type, printing_level=0)
 
-      
+      # this will constantly flush the printing to the display so the progress dots appear for each insert
       ($stdout.sync = true) if printing_level == 1
 
       # a unique ID must be provided for each document stored in Elastic Search
@@ -12,8 +33,8 @@ module RubberbandFlamethrower
       # initialize the random data generator object
       data = DataGenerator.new
 
+      # loop through inserting random data
       how_many.to_i.times do |i|
-
         # generate a piece of random data to insert that approxiamates a tweet
         insert_data = data.generate_random_insert_data
         (puts insert_data) if printing_level == 2
@@ -27,6 +48,5 @@ module RubberbandFlamethrower
         id = id + 1
       end
     end
-    
   end
 end

data/lib/rubberband_flamethrower.rb

@@ -1,9 +1,12 @@
 module RubberbandFlamethrower
-  def self.fire(how_many=5000, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet")
-    require "active_support/core_ext"
-    require 'httparty'
-    require "benchmark"
-    require File.dirname(__FILE__)+"/rubberband_flamethrower/data_generator.rb"
+  
+  # Benchmarks a call to the fire method (which inserts a batch of random data into Elastic Search)
+  # @param [Integer] how_many - how many randomly generated data objects to insert
+  # @param [Integer] starting_id - starting id for randomly generated data objects, will increment from this number
+  # @param [String] server_url - url of the Elastic Search server
+  # @param [String] index - name of the Elastic Search index to insert data into
+  # @param [String] type - name of the Elastic Search type to insert data into
+  def self.fire(how_many=500, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet")
     require File.dirname(__FILE__)+"/rubberband_flamethrower/flamethrower.rb"
     flamethrower = Flamethrower.new
     time = Benchmark.measure do
@@ -14,23 +17,64 @@ module RubberbandFlamethrower
     puts time
   end
 
-  def self.auto(how_many_batches=3, per_batch=5000, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet")
-    require "active_support/core_ext"
-    require 'httparty'
-    require "benchmark"
-    require File.dirname(__FILE__)+"/rubberband_flamethrower/data_generator.rb"
+  # Benchmarks a series of calls to the fire method (which inserts a batch of random data into Elastic Search)
+  # @param [Integer] how_many_batches - how many batches to run
+  # @param [Integer] per_batch - how many randomly generated data objects to insert
+  # @param [Integer] starting_id - starting id for randomly generated data objects, will increment from this number
+  # @param [String] server_url - url of the Elastic Search server
+  # @param [String] index - name of the Elastic Search index to insert data into
+  # @param [String] type - name of the Elastic Search type to insert data into
+  def self.auto(how_many_batches=3, per_batch=500, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet", id_overwrite="n")
     require File.dirname(__FILE__)+"/rubberband_flamethrower/flamethrower.rb"
     flamethrower = Flamethrower.new
+    puts "#{per_batch} documents inserted into Elastic Search per set"
     Benchmark.bm(8) do |x|
       how_many_batches.to_i.times do |i|
-        x.report("set #{i+1}:")   { flamethrower.fire(per_batch, starting_id, server_url, index, type) }
-        starting_id = starting_id.to_i + per_batch.to_i
+        x.report("set #{i+1} of #{how_many_batches}:")   { flamethrower.fire(per_batch, starting_id, server_url, index, type) }
+        # increment the starting id  by the batch size on each loop unless "id_overwrite" is set to "y"
+        # when it is set to "n" (which it is by default)
+        # each batch will be writing new data with fresh IDs to the Elastic Search server
+        # simulating a system where data is constantly being inserted and not updated
+        # when it is set to "y" each batch will simulate overwriting existing data in the Elastic Search server
+        # simulating a system where data is constantly being updated (after the initial batch)
+        (starting_id = starting_id.to_i + per_batch.to_i) unless id_overwrite == "y"
       end
     end
   end
 
+  # Displays help menu of the available help menu commands
   def self.help
-    puts "Flamethrower Commands Available:\n\nflamethrower #display this help message\nflamethrower fire #sends a batch to the elastic search server"
+    puts "Rubberband Flamethrower is a gem for inserting faked data into an Elastic Search server"
+    puts "and providing basic benchmarks. It creates and inserts fake data objects with three"
+    puts "fields (message, username, and post_date) and reports timing information."
+    
+    puts "\n\nFlamethrower Commands Available:\n\n"
+    puts "flamethrower fire #benchmark a batch insert of data to an elastic search server"
+    puts "flamethrower auto #benchmark a series of batch inserts to an elastic search server"
+    puts "flamethrower help #display this help message"
+    
+    puts "\n\nThe fire and auto commands can be configured by passing arguments."
+    puts "The parameters accepted by fire and auto all have a default value if left blank."
+    
+    puts "\n\n\"fire\" parameters in order with their default values:"
+    puts "(how_many=500, starting_id=1, server_url=http://localhost:9200, index=twitter, type=tweet)"
+
+    puts "\n\n\"auto\" parameters in order with their default values:"
+    puts "(how_many_batches=3, per_batch=500, starting_id=1, server_url=http://localhost:9200, index=twitter, type=tweet, id_overwrite=n)"
+    
+    puts "\n\nUsage Examples With Parameters:"
+    puts "flamethrower fire 10000 #To Insert 10,000 instead of 500"
+    puts "flamethrower fire 5000 5001 #To Insert 5,000 starting with the ID 5001"
+    puts "flamethrower fire 2000 1 \"http://es.com:9200\" #Elastic Search Node located at http://es.com:9200"
+    puts "flamethrower fire 500 1 \"http://localhost:9200\" \"facebook\" \"message\""
+    puts "\t#Insert into an index named \"facebook\" instead of \"twitter\""
+    puts "\t#with a type of \"message\" instead of \"tweet\""
+    
+    puts "\n\nThe id_overwrite parameter determines the ID strategy used for subsequent batches in the auto command."
+    puts "When set to \"n\" (which it is by default)each batch will be writing new data with fresh IDs to the Elastic"
+    puts "Search server, simulating a system where data is constantly being inserted and not updated. When it is set"
+    puts "to \"y\" each batch will simulate overwriting existing data in the Elastic Search server, simulating a"
+    puts "system where data is constantly being updated (after the initial batch)."
   end
 
 end

data/rubberband_flamethrower.gemspec

@@ -5,7 +5,7 @@
 
 Gem::Specification.new do |s|
   s.name = "rubberband_flamethrower"
-  s.version = "0.4.0"
+  s.version = "0.4.1"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Michael Orr"]

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: rubberband_flamethrower
 version: !ruby/object:Gem::Version
-  version: 0.4.0
+  version: 0.4.1
   prerelease: 
 platform: ruby
 authors:
@@ -13,7 +13,7 @@ date: 2013-04-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: httparty
-  requirement: &17477820 !ruby/object:Gem::Requirement
+  requirement: &15221100 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -21,10 +21,10 @@ dependencies:
         version: 0.10.2
   type: :runtime
   prerelease: false
-  version_requirements: *17477820
+  version_requirements: *15221100
 - !ruby/object:Gem::Dependency
   name: activesupport
-  requirement: &17477280 !ruby/object:Gem::Requirement
+  requirement: &15219640 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -32,10 +32,10 @@ dependencies:
         version: 3.2.13
   type: :runtime
   prerelease: false
-  version_requirements: *17477280
+  version_requirements: *15219640
 - !ruby/object:Gem::Dependency
   name: rdoc
-  requirement: &17476800 !ruby/object:Gem::Requirement
+  requirement: &15217040 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -43,10 +43,10 @@ dependencies:
         version: '3.12'
   type: :development
   prerelease: false
-  version_requirements: *17476800
+  version_requirements: *15217040
 - !ruby/object:Gem::Dependency
   name: bundler
-  requirement: &17476280 !ruby/object:Gem::Requirement
+  requirement: &15225080 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -54,10 +54,10 @@ dependencies:
         version: 1.3.0
   type: :development
   prerelease: false
-  version_requirements: *17476280
+  version_requirements: *15225080
 - !ruby/object:Gem::Dependency
   name: jeweler
-  requirement: &17475800 !ruby/object:Gem::Requirement
+  requirement: &15239460 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -65,7 +65,7 @@ dependencies:
         version: 1.8.4
   type: :development
   prerelease: false
-  version_requirements: *17475800
+  version_requirements: *15239460
 description: Use to quickly fill up some indicies in Elastic Search and to retrieve
   statistics about insertion rates
 email: michael@cloudspace.com
@@ -117,7 +117,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: 3868063792966642724
+      hash: 1869112594906546010
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements: