rubberband_flamethrower 0.2.2 → 0.3.0
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +117 -0
- data/Rakefile +1 -1
- data/VERSION +1 -1
- data/bin/flamethrower +19 -0
- data/lib/rubberband_flamethrower/flamethrower.rb +3 -3
- data/lib/rubberband_flamethrower.rb +7 -2
- data/rubberband_flamethrower.gemspec +6 -4
- metadata +18 -16
- data/README.rdoc +0 -64
data/README.md
ADDED
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
# Rubberband Flamethrower
|
|
2
|
+
|
|
3
|
+
Rubberband Flamethrower is a gem for inserting faked data into an Elastic Search server. It creates and inserts fake data objects with three fields: message, username, and post_date. Here is an sample generated data object:
|
|
4
|
+
|
|
5
|
+
{"message":"utilizing plowed popularizing demeanor anesthesia specializes chaperon pedaling.","username":"pummeling","post_date":"20130408T15:41:28"}
|
|
6
|
+
|
|
7
|
+
|
|
8
|
+
## Pre-Requisites
|
|
9
|
+
|
|
10
|
+
### Elastic Search
|
|
11
|
+
|
|
12
|
+
You should install and have an Elastic Search node running before trying to use this gem.
|
|
13
|
+
|
|
14
|
+
Download Elastic Search:
|
|
15
|
+
|
|
16
|
+
curl -k -L -o elasticsearch-0.20.6.tar.gz http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.6.tar.gz
|
|
17
|
+
|
|
18
|
+
Unarchive Elastic Search:
|
|
19
|
+
|
|
20
|
+
tar -zxvf elasticsearch-0.20.6.tar.gz
|
|
21
|
+
|
|
22
|
+
Start an Elastic Search node:
|
|
23
|
+
|
|
24
|
+
./elasticsearch-0.20.6/bin/elasticsearch -f
|
|
25
|
+
|
|
26
|
+
### Ruby
|
|
27
|
+
|
|
28
|
+
You do not need a rails project to use this gem though it is easier to use if you do. It has been designed with ruby 1.9.1 and above in mind. The sample method of the Array class is used in the code and was not a part of the 1.8.7 release.
|
|
29
|
+
|
|
30
|
+
## Installation
|
|
31
|
+
|
|
32
|
+
install the gem manually
|
|
33
|
+
|
|
34
|
+
gem install rubberband_flamethrower
|
|
35
|
+
|
|
36
|
+
Or if you will be using this as a part of a Rails project, you can add the gem to your Gemfile and then `bundle install`.
|
|
37
|
+
|
|
38
|
+
## Flamethrower Use
|
|
39
|
+
|
|
40
|
+
### Command Line Executable
|
|
41
|
+
|
|
42
|
+
Once the gem is installed and you have an Elastic Search server running you are ready to begin inserting fake data. You can run the gem from the command line using the "flamethrower" command. The command without any arguments will display the help screen:
|
|
43
|
+
|
|
44
|
+
flamethrower
|
|
45
|
+
|
|
46
|
+
To start a batch insert into the local Elastic Search server you add the argument "fire" to the command:
|
|
47
|
+
|
|
48
|
+
flamethrower fire
|
|
49
|
+
|
|
50
|
+
By default it will insert 5000 documents starting with document ID 1 into an Elastic Search index named "twitter" of type "tweet" into a server node located at http://localhost:9200.
|
|
51
|
+
|
|
52
|
+
You can configure what is inserted by passing additional parameters. There are 5 parameters accepted by the `flamethrower fire` command, all of which have a default value if left blank. Here are the parameters in order with their default values: (how_many=5000, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet")
|
|
53
|
+
|
|
54
|
+
To Insert 10,000 instead of 5,000:
|
|
55
|
+
|
|
56
|
+
flamethrower fire 10000
|
|
57
|
+
|
|
58
|
+
To Insert 5,000 starting with the ID 5001
|
|
59
|
+
|
|
60
|
+
flamethrower fire 5000 5001
|
|
61
|
+
|
|
62
|
+
To Insert 2,000 starting with the ID 1 to a server located at http://es.com:9200
|
|
63
|
+
|
|
64
|
+
flamethrower fire 2000 1 "http://es.com:9200"
|
|
65
|
+
|
|
66
|
+
To put your documents into an index named "facebook" instead of "twitter" with a type of "message" instead of "tweet"
|
|
67
|
+
|
|
68
|
+
flamethrower fire 5000 1 "http://localhost:9200" "facebook" "message"
|
|
69
|
+
|
|
70
|
+
|
|
71
|
+
### IRB/Ruby Scripts
|
|
72
|
+
|
|
73
|
+
Instead of using the gem from the command line you may want to use this gem from inside of a ruby script or through IRB.
|
|
74
|
+
|
|
75
|
+
First require the gem:
|
|
76
|
+
|
|
77
|
+
require 'rubberband_flamethrower'
|
|
78
|
+
|
|
79
|
+
Then you are ready to use it!
|
|
80
|
+
|
|
81
|
+
RubberbandFlamethrower.fire
|
|
82
|
+
|
|
83
|
+
By default it will insert 5000 documents starting with document ID 1 into an Elastic Search index named "twitter" of type "tweet" into a server node located at http://localhost:9200.
|
|
84
|
+
|
|
85
|
+
The fire method can be configured by passing parameters to it. There are 5 parameters accepted by the fire method, all of which have a default value if left blank. Here are the parameters in order with their default values: (how_many=5000, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet")
|
|
86
|
+
|
|
87
|
+
To Insert 10,000 instead of 5,000:
|
|
88
|
+
|
|
89
|
+
RubberbandFlamethrower.fire(10000)
|
|
90
|
+
|
|
91
|
+
To Insert 5,000 starting with the ID 5001
|
|
92
|
+
|
|
93
|
+
RubberbandFlamethrower.fire(5000, 5001)
|
|
94
|
+
|
|
95
|
+
To Insert 2,000 starting with the ID 1 to a server located at http://es.com:9200
|
|
96
|
+
|
|
97
|
+
RubberbandFlamethrower.fire(2000, 1,"http://es.com:9200")
|
|
98
|
+
|
|
99
|
+
To put your documents into an index named "facebook" instead of "twitter" with a type of "message" instead of "tweet"
|
|
100
|
+
|
|
101
|
+
RubberbandFlamethrower.fire(5000, 1, "http://localhost:9200", "facebook", "message")
|
|
102
|
+
|
|
103
|
+
## Contributing to rubberband_flamethrower
|
|
104
|
+
|
|
105
|
+
* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
|
|
106
|
+
* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
|
|
107
|
+
* Fork the project.
|
|
108
|
+
* Start a feature/bugfix branch.
|
|
109
|
+
* Commit and push until you are happy with your contribution.
|
|
110
|
+
* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
|
|
111
|
+
* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
|
|
112
|
+
|
|
113
|
+
## Copyright
|
|
114
|
+
|
|
115
|
+
Copyright (c) 2013 Michael Orr. See LICENSE.txt for
|
|
116
|
+
further details.
|
|
117
|
+
|
data/Rakefile
CHANGED
|
@@ -57,7 +57,7 @@ namespace :rubberband_flamethrower do
|
|
|
57
57
|
desc "Insert fake data into Elastic Search, use arguments to change defaults"
|
|
58
58
|
task :fire, :how_many, :starting_id, :server_url, :index, :type do |t, args|
|
|
59
59
|
args.with_defaults(:how_many => 5000, :starting_id => 1, :server_url => "http://localhost:9200", :index => "twitter", :type => "tweet")
|
|
60
|
-
RubberbandFlamethrower.
|
|
60
|
+
RubberbandFlamethrower.fire(args[:how_many].to_i, args[:starting_id].to_i, args[:server_url], args[:index], args[:type])
|
|
61
61
|
end
|
|
62
62
|
end
|
|
63
63
|
|
data/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.
|
|
1
|
+
0.3.0
|
data/bin/flamethrower
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
|
|
3
|
+
lib = File.expand_path(File.dirname(__FILE__) + '/../lib')
|
|
4
|
+
$LOAD_PATH.unshift(lib) if File.directory?(lib) && !$LOAD_PATH.include?(lib)
|
|
5
|
+
|
|
6
|
+
require 'rubberband_flamethrower'
|
|
7
|
+
|
|
8
|
+
args = ARGV.dup
|
|
9
|
+
ARGV.clear
|
|
10
|
+
command = args.shift.strip rescue 'help'
|
|
11
|
+
|
|
12
|
+
case command
|
|
13
|
+
when "fire"
|
|
14
|
+
RubberbandFlamethrower.fire(*args)
|
|
15
|
+
else
|
|
16
|
+
RubberbandFlamethrower.help
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
#binary adapted from old heroku source
|
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
# This class is designed to insert objects created with the DataGenerator class into Elastic Search
|
|
2
2
|
module RubberbandFlamethrower
|
|
3
3
|
class Flamethrower
|
|
4
|
-
def
|
|
4
|
+
def fire(how_many, starting_id, server_url, index, type)
|
|
5
5
|
# a unique ID must be provided for each document stored in Elastic Search
|
|
6
|
-
id = starting_id
|
|
6
|
+
id = starting_id.to_i
|
|
7
7
|
|
|
8
8
|
# initialize the random data generator object
|
|
9
9
|
data = DataGenerator.new
|
|
10
10
|
|
|
11
|
-
how_many.times do |i|
|
|
11
|
+
how_many.to_i.times do |i|
|
|
12
12
|
|
|
13
13
|
# generate a piece of random data to insert that approxiamates a tweet
|
|
14
14
|
insert_data = data.generate_random_insert_data
|
|
@@ -1,10 +1,15 @@
|
|
|
1
1
|
module RubberbandFlamethrower
|
|
2
|
-
def self.
|
|
2
|
+
def self.fire(how_many=5000, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet")
|
|
3
3
|
require "active_support/core_ext"
|
|
4
4
|
require 'httparty'
|
|
5
5
|
require File.dirname(__FILE__)+"/rubberband_flamethrower/data_generator.rb"
|
|
6
6
|
require File.dirname(__FILE__)+"/rubberband_flamethrower/flamethrower.rb"
|
|
7
7
|
flamethrower = Flamethrower.new
|
|
8
|
-
flamethrower.
|
|
8
|
+
flamethrower.fire(how_many, starting_id, server_url, index, type)
|
|
9
9
|
end
|
|
10
|
+
|
|
11
|
+
def self.help
|
|
12
|
+
puts "Flamethrower Commands Available:\n\nflamethrower #display this help message\nflamethrower fire #sends a batch to the elastic search server"
|
|
13
|
+
end
|
|
14
|
+
|
|
10
15
|
end
|
|
@@ -5,25 +5,27 @@
|
|
|
5
5
|
|
|
6
6
|
Gem::Specification.new do |s|
|
|
7
7
|
s.name = "rubberband_flamethrower"
|
|
8
|
-
s.version = "0.
|
|
8
|
+
s.version = "0.3.0"
|
|
9
9
|
|
|
10
10
|
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
|
|
11
11
|
s.authors = ["Michael Orr"]
|
|
12
|
-
s.date = "2013-04-
|
|
12
|
+
s.date = "2013-04-08"
|
|
13
13
|
s.description = "Use to quickly fill up some indicies in Elastic Search and to retrieve statistics about insertion rates"
|
|
14
14
|
s.email = "michael@cloudspace.com"
|
|
15
|
+
s.executables = ["flamethrower"]
|
|
15
16
|
s.extra_rdoc_files = [
|
|
16
17
|
"LICENSE.txt",
|
|
17
|
-
"README.
|
|
18
|
+
"README.md"
|
|
18
19
|
]
|
|
19
20
|
s.files = [
|
|
20
21
|
".document",
|
|
21
22
|
"Gemfile",
|
|
22
23
|
"Gemfile.lock",
|
|
23
24
|
"LICENSE.txt",
|
|
24
|
-
"README.
|
|
25
|
+
"README.md",
|
|
25
26
|
"Rakefile",
|
|
26
27
|
"VERSION",
|
|
28
|
+
"bin/flamethrower",
|
|
27
29
|
"lib/rubberband_flamethrower.rb",
|
|
28
30
|
"lib/rubberband_flamethrower/.DS_Store",
|
|
29
31
|
"lib/rubberband_flamethrower/data_generator.rb",
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: rubberband_flamethrower
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
prerelease:
|
|
6
6
|
platform: ruby
|
|
7
7
|
authors:
|
|
@@ -9,11 +9,11 @@ authors:
|
|
|
9
9
|
autorequire:
|
|
10
10
|
bindir: bin
|
|
11
11
|
cert_chain: []
|
|
12
|
-
date: 2013-04-
|
|
12
|
+
date: 2013-04-08 00:00:00.000000000 Z
|
|
13
13
|
dependencies:
|
|
14
14
|
- !ruby/object:Gem::Dependency
|
|
15
15
|
name: httparty
|
|
16
|
-
requirement: &
|
|
16
|
+
requirement: &12287740 !ruby/object:Gem::Requirement
|
|
17
17
|
none: false
|
|
18
18
|
requirements:
|
|
19
19
|
- - ~>
|
|
@@ -21,10 +21,10 @@ dependencies:
|
|
|
21
21
|
version: 0.10.2
|
|
22
22
|
type: :runtime
|
|
23
23
|
prerelease: false
|
|
24
|
-
version_requirements: *
|
|
24
|
+
version_requirements: *12287740
|
|
25
25
|
- !ruby/object:Gem::Dependency
|
|
26
26
|
name: activesupport
|
|
27
|
-
requirement: &
|
|
27
|
+
requirement: &12287240 !ruby/object:Gem::Requirement
|
|
28
28
|
none: false
|
|
29
29
|
requirements:
|
|
30
30
|
- - ~>
|
|
@@ -32,10 +32,10 @@ dependencies:
|
|
|
32
32
|
version: 3.2.13
|
|
33
33
|
type: :runtime
|
|
34
34
|
prerelease: false
|
|
35
|
-
version_requirements: *
|
|
35
|
+
version_requirements: *12287240
|
|
36
36
|
- !ruby/object:Gem::Dependency
|
|
37
37
|
name: rdoc
|
|
38
|
-
requirement: &
|
|
38
|
+
requirement: &12286760 !ruby/object:Gem::Requirement
|
|
39
39
|
none: false
|
|
40
40
|
requirements:
|
|
41
41
|
- - ~>
|
|
@@ -43,10 +43,10 @@ dependencies:
|
|
|
43
43
|
version: '3.12'
|
|
44
44
|
type: :development
|
|
45
45
|
prerelease: false
|
|
46
|
-
version_requirements: *
|
|
46
|
+
version_requirements: *12286760
|
|
47
47
|
- !ruby/object:Gem::Dependency
|
|
48
48
|
name: bundler
|
|
49
|
-
requirement: &
|
|
49
|
+
requirement: &12302580 !ruby/object:Gem::Requirement
|
|
50
50
|
none: false
|
|
51
51
|
requirements:
|
|
52
52
|
- - ~>
|
|
@@ -54,10 +54,10 @@ dependencies:
|
|
|
54
54
|
version: 1.3.0
|
|
55
55
|
type: :development
|
|
56
56
|
prerelease: false
|
|
57
|
-
version_requirements: *
|
|
57
|
+
version_requirements: *12302580
|
|
58
58
|
- !ruby/object:Gem::Dependency
|
|
59
59
|
name: jeweler
|
|
60
|
-
requirement: &
|
|
60
|
+
requirement: &12301800 !ruby/object:Gem::Requirement
|
|
61
61
|
none: false
|
|
62
62
|
requirements:
|
|
63
63
|
- - ~>
|
|
@@ -65,23 +65,25 @@ dependencies:
|
|
|
65
65
|
version: 1.8.4
|
|
66
66
|
type: :development
|
|
67
67
|
prerelease: false
|
|
68
|
-
version_requirements: *
|
|
68
|
+
version_requirements: *12301800
|
|
69
69
|
description: Use to quickly fill up some indicies in Elastic Search and to retrieve
|
|
70
70
|
statistics about insertion rates
|
|
71
71
|
email: michael@cloudspace.com
|
|
72
|
-
executables:
|
|
72
|
+
executables:
|
|
73
|
+
- flamethrower
|
|
73
74
|
extensions: []
|
|
74
75
|
extra_rdoc_files:
|
|
75
76
|
- LICENSE.txt
|
|
76
|
-
- README.
|
|
77
|
+
- README.md
|
|
77
78
|
files:
|
|
78
79
|
- .document
|
|
79
80
|
- Gemfile
|
|
80
81
|
- Gemfile.lock
|
|
81
82
|
- LICENSE.txt
|
|
82
|
-
- README.
|
|
83
|
+
- README.md
|
|
83
84
|
- Rakefile
|
|
84
85
|
- VERSION
|
|
86
|
+
- bin/flamethrower
|
|
85
87
|
- lib/rubberband_flamethrower.rb
|
|
86
88
|
- lib/rubberband_flamethrower/.DS_Store
|
|
87
89
|
- lib/rubberband_flamethrower/data_generator.rb
|
|
@@ -115,7 +117,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
|
115
117
|
version: '0'
|
|
116
118
|
segments:
|
|
117
119
|
- 0
|
|
118
|
-
hash:
|
|
120
|
+
hash: 2982825154253671074
|
|
119
121
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
120
122
|
none: false
|
|
121
123
|
requirements:
|
data/README.rdoc
DELETED
|
@@ -1,64 +0,0 @@
|
|
|
1
|
-
= Rubberband Flamethrower
|
|
2
|
-
|
|
3
|
-
Rubberband Flamethrower is a gem for inserting faked data into an Elastic Search server.
|
|
4
|
-
|
|
5
|
-
== Pre-Requisites
|
|
6
|
-
|
|
7
|
-
=== Elastic Search
|
|
8
|
-
|
|
9
|
-
You should install and have an Elastic Search node running before trying to use this gem
|
|
10
|
-
|
|
11
|
-
To download Elastic Search, unarchive, and start a node:
|
|
12
|
-
|
|
13
|
-
$ curl -k -L -o elasticsearch-0.20.6.tar.gz http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.6.tar.gz
|
|
14
|
-
$ tar -zxvf elasticsearch-0.20.6.tar.gz
|
|
15
|
-
$ ./elasticsearch-0.20.6/bin/elasticsearch -f
|
|
16
|
-
|
|
17
|
-
=== Ruby
|
|
18
|
-
|
|
19
|
-
You do not need a rails project to use this gem though it is easier to use if you do. It has been designed with ruby 1.9.1 and above in mind. The sample method of the Array class is used in the code and was not a part of the 1.8.7 release.
|
|
20
|
-
|
|
21
|
-
== Installation
|
|
22
|
-
|
|
23
|
-
install the gem manually
|
|
24
|
-
|
|
25
|
-
gem install rubberband_flamethrower
|
|
26
|
-
|
|
27
|
-
== Use
|
|
28
|
-
|
|
29
|
-
Insert into Elastic Search by using the gem through irb
|
|
30
|
-
|
|
31
|
-
require 'rubberband_flamethrower'
|
|
32
|
-
RubberbandFlamethrower.send_batch
|
|
33
|
-
|
|
34
|
-
The send_batch method can be configured by passing parameters. By default it will insert 5000 documents starting with document ID 1 into an Elastic Search index named "twitter" of type "tweet" into a server node located at http://localhost:9200.
|
|
35
|
-
|
|
36
|
-
There are 5 parameters accepted by the send_batch method, all of which have a default value if left blank. Here are the parameters in order with their default values: (how_many=5000, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet")
|
|
37
|
-
|
|
38
|
-
To Insert 10,000 instead of 5,000:
|
|
39
|
-
RubberbandFlamethrower.send_batch(10000)
|
|
40
|
-
|
|
41
|
-
To Insert 5,000 starting with the ID 5001
|
|
42
|
-
RubberbandFlamethrower.send_batch(5000,5001)
|
|
43
|
-
|
|
44
|
-
To Insert 2,000 starting with the ID 1 to a server located at http://es.com:9200
|
|
45
|
-
RubberbandFlamethrower.send_batch(2000,1,"http://es.com:9200")
|
|
46
|
-
|
|
47
|
-
To put your documents into an index named "facebook" instead of "twitter" with a type of "message" instead of "tweet"
|
|
48
|
-
RubberbandFlamethrower.send_batch(5000,1,"http://localhost:9200","facebook","message")
|
|
49
|
-
|
|
50
|
-
== Contributing to rubberband_flamethrower
|
|
51
|
-
|
|
52
|
-
* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
|
|
53
|
-
* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
|
|
54
|
-
* Fork the project.
|
|
55
|
-
* Start a feature/bugfix branch.
|
|
56
|
-
* Commit and push until you are happy with your contribution.
|
|
57
|
-
* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
|
|
58
|
-
* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
|
|
59
|
-
|
|
60
|
-
== Copyright
|
|
61
|
-
|
|
62
|
-
Copyright (c) 2013 Michael Orr. See LICENSE.txt for
|
|
63
|
-
further details.
|
|
64
|
-
|