murder 0.1.0 → 0.1.1
Sign up to get free protection for your applications and to get access to all the features.
- data/{README → README.md} +30 -25
- data/VERSION +1 -1
- data/lib/murder/admin.rb +1 -1
- data/murder.gemspec +4 -4
- metadata +5 -5
data/{README → README.md}
RENAMED
@@ -1,4 +1,5 @@
|
|
1
|
-
Murder by Larry Gadea <lg@twitter.com> and Matt Freels <freels@twitter.com>
|
1
|
+
Murder by Larry Gadea <lg@twitter.com> and Matt Freels <freels@twitter.com>
|
2
|
+
|
2
3
|
Copyright 2010 Twitter Inc.
|
3
4
|
|
4
5
|
|
@@ -118,19 +119,23 @@ To distribute a directory of files, first make sure that murder is set
|
|
118
119
|
up on all hosts, then manually run the murder cap tasks:
|
119
120
|
|
120
121
|
1. Start the tracker:
|
121
|
-
|
122
|
+
|
123
|
+
cap murder:start_tracker
|
122
124
|
|
123
125
|
2. Create a torrent from a directory of files on the seeder, and start seeding:
|
124
|
-
|
125
|
-
|
126
|
-
|
126
|
+
|
127
|
+
scp -r ./files host1:~/files
|
128
|
+
cap murder:create_torrent tag="Deploy1" files_path="~/files"
|
129
|
+
cap murder:start_seeding tag="Deploy1"
|
127
130
|
|
128
131
|
3. Distribute the torrent to all peers:
|
129
|
-
|
132
|
+
|
133
|
+
cap murder:peer tag="Deploy1" destination_path="/tmp"
|
130
134
|
|
131
135
|
4. Stop the seeder and tracker:
|
132
|
-
|
133
|
-
|
136
|
+
|
137
|
+
cap murder:stop_seeding
|
138
|
+
cap murder:stop_tracker
|
134
139
|
|
135
140
|
When this finishes, all peers will have the files in /tmp/Deploy1
|
136
141
|
|
@@ -138,51 +143,51 @@ When this finishes, all peers will have the files in /tmp/Deploy1
|
|
138
143
|
TASK REFERENCE
|
139
144
|
--------------
|
140
145
|
|
141
|
-
distribute_files
|
146
|
+
`distribute_files`:
|
142
147
|
SCPs a compressed version of all files from ./dist (the python Bittorrent
|
143
148
|
library and custom scripts) to all server. The entire directory is sent,
|
144
149
|
regardless of the role of each individual server. The path on the server is
|
145
150
|
specified by remote_murder_path and will be cleared prior to transferring
|
146
151
|
files over.
|
147
152
|
|
148
|
-
start_tracker
|
153
|
+
`start_tracker`:
|
149
154
|
Starts the Bittorrent tracker (essentially a mini-web-server) listening on
|
150
155
|
port 8998.
|
151
156
|
|
152
|
-
stop_tracker
|
157
|
+
`stop_tracker`:
|
153
158
|
If the Bittorrent tracker is running, this will kill the process. Note that
|
154
159
|
if it is not running you will receive an error.
|
155
160
|
|
156
|
-
create_torrent
|
161
|
+
`create_torrent`:
|
157
162
|
Compresses the directory specified by the passed-in argument 'files_path'
|
158
163
|
and creates a .torrent file identified by the 'tag' argument. Be sure to use
|
159
164
|
the same 'tag' value with any following commands. Any .git directories will be
|
160
165
|
skipped. Once completed, the .torrent will be downloaded to your local
|
161
166
|
/tmp/TAG.tgz.torrent.
|
162
167
|
|
163
|
-
download_torrent
|
168
|
+
`download_torrent`:
|
164
169
|
Although not necessary to run, if the file from create_torrent was lost, you
|
165
170
|
can redownload it from the seeder using this task. You must specify a valid
|
166
171
|
'tag' argument.
|
167
172
|
|
168
|
-
start_seeding
|
173
|
+
`start_seeding`:
|
169
174
|
Will cause the seeder machine to connect to the tracker and start seeding.
|
170
175
|
The ip address returned by the 'host' bash command will be announced to the
|
171
176
|
tracker. The server will not stop seeding until the stop_seeding task is
|
172
177
|
called. You must specify a valid 'tag' argument (which identifies the .torrent
|
173
178
|
in /tmp to use)
|
174
179
|
|
175
|
-
stop_seeding
|
180
|
+
`stop_seeding`:
|
176
181
|
If the seeder is currently seeding, this will kill the process. Note that if
|
177
182
|
it is not running, you will receive an error. If a peer was downloading from
|
178
183
|
this seed, the peer will find another host to receive any remaining data. You
|
179
184
|
must specify a valid 'tag' argument.
|
180
185
|
|
181
|
-
stop_all_seeding
|
186
|
+
`stop_all_seeding`:
|
182
187
|
Identical to stop_seeding, except this will kill all seeding processes. No
|
183
188
|
'tag' argument is needed.
|
184
189
|
|
185
|
-
peer
|
190
|
+
`peer`:
|
186
191
|
Instructs all the peer servers to connect to the tracker and start download
|
187
192
|
and spreading pieces and files amongst themselves. You must specify a valid
|
188
193
|
'tag' argument. Once the download is complete on a server, that server will
|
@@ -195,7 +200,7 @@ is empty, this command will fail. To clean it, pass in the
|
|
195
200
|
removed. When this task completes, all files have been transferred and moved
|
196
201
|
into the requested directory.
|
197
202
|
|
198
|
-
stop_all_peering
|
203
|
+
`stop_all_peering`:
|
199
204
|
Sometimes peers can go on forever (usually because of an error). This
|
200
205
|
command will forcibly kill all "murder_client.py peer" commands that are
|
201
206
|
running.
|
@@ -206,20 +211,20 @@ CONFIG REFERENCE
|
|
206
211
|
Variables
|
207
212
|
---------
|
208
213
|
|
209
|
-
default_tag
|
214
|
+
`default_tag`:
|
210
215
|
A tag name to use by default such that a tag parameter doesn't need to be
|
211
216
|
manually entered on every task. Not recommended to be used since files will be
|
212
217
|
overwritten.
|
213
218
|
|
214
|
-
default_seeder_files_path
|
219
|
+
`default_seeder_files_path`:
|
215
220
|
A path on the seeder's file system where the files to be distributed are
|
216
221
|
stored.
|
217
222
|
|
218
|
-
default_destination_path
|
223
|
+
`default_destination_path`:
|
219
224
|
A path on the peers' file system where the files that were distributed
|
220
225
|
should be decompressed into.
|
221
226
|
|
222
|
-
remote_murder_path
|
227
|
+
`remote_murder_path`:
|
223
228
|
A path where murder will look for its support files on each host. `cap
|
224
229
|
murder:distribute_files` will upload murder support files here.
|
225
230
|
|
@@ -227,11 +232,11 @@ murder:distribute_files` will upload murder support files here.
|
|
227
232
|
Roles
|
228
233
|
-----
|
229
234
|
|
230
|
-
tracker
|
235
|
+
`tracker`:
|
231
236
|
Host on which to run the BitTorrent tracker
|
232
237
|
|
233
|
-
seeder
|
238
|
+
`seeder`:
|
234
239
|
Host which will be the source of the files to be distributed via BitTorrent
|
235
240
|
|
236
|
-
peers
|
241
|
+
`peers`:
|
237
242
|
All hosts to which files should be distributed
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
0.1.
|
1
|
+
0.1.1
|
data/lib/murder/admin.rb
CHANGED
@@ -19,7 +19,7 @@ namespace :murder do
|
|
19
19
|
SCPs a compressed version of all files from ./dist (the python Bittorrent library and custom scripts) to all server. The entire directory is sent, regardless of the role of each individual server. The path on the server is specified by remote_murder_path and will be cleared prior to transferring files over.
|
20
20
|
DESC
|
21
21
|
task :distribute_files, :roles => [:tracker, :seeder, :peer] do
|
22
|
-
dist_path = File.expand_path('
|
22
|
+
dist_path = File.expand_path('../../../dist', __FILE__)
|
23
23
|
|
24
24
|
run "mkdir -p #{remote_murder_path}/"
|
25
25
|
run "[ $(find '#{remote_murder_path}/'* | wc -l ) -lt 1000 ] && rm -rf '#{remote_murder_path}/'* || ( echo 'Cowardly refusing to remove files! Check the remote_murder_path.' ; exit 1 )"
|
data/murder.gemspec
CHANGED
@@ -5,21 +5,21 @@
|
|
5
5
|
|
6
6
|
Gem::Specification.new do |s|
|
7
7
|
s.name = %q{murder}
|
8
|
-
s.version = "0.1.
|
8
|
+
s.version = "0.1.1"
|
9
9
|
|
10
10
|
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
|
11
11
|
s.authors = ["Larry Gadea", "Matt Freels"]
|
12
|
-
s.date = %q{2010-
|
12
|
+
s.date = %q{2010-04-13}
|
13
13
|
s.description = %q{Large scale server deploys using BitTorrent and the BitTornado library}
|
14
14
|
s.email = %q{lg@twitter.com}
|
15
15
|
s.extra_rdoc_files = [
|
16
16
|
"LICENSE",
|
17
|
-
"README"
|
17
|
+
"README.md"
|
18
18
|
]
|
19
19
|
s.files = [
|
20
20
|
".gitignore",
|
21
21
|
"LICENSE",
|
22
|
-
"README",
|
22
|
+
"README.md",
|
23
23
|
"Rakefile",
|
24
24
|
"VERSION",
|
25
25
|
"dist/BitTornado/BT1/Choker.py",
|
metadata
CHANGED
@@ -5,8 +5,8 @@ version: !ruby/object:Gem::Version
|
|
5
5
|
segments:
|
6
6
|
- 0
|
7
7
|
- 1
|
8
|
-
-
|
9
|
-
version: 0.1.
|
8
|
+
- 1
|
9
|
+
version: 0.1.1
|
10
10
|
platform: ruby
|
11
11
|
authors:
|
12
12
|
- Larry Gadea
|
@@ -15,7 +15,7 @@ autorequire:
|
|
15
15
|
bindir: bin
|
16
16
|
cert_chain: []
|
17
17
|
|
18
|
-
date: 2010-
|
18
|
+
date: 2010-04-13 00:00:00 -07:00
|
19
19
|
default_executable:
|
20
20
|
dependencies: []
|
21
21
|
|
@@ -27,11 +27,11 @@ extensions: []
|
|
27
27
|
|
28
28
|
extra_rdoc_files:
|
29
29
|
- LICENSE
|
30
|
-
- README
|
30
|
+
- README.md
|
31
31
|
files:
|
32
32
|
- .gitignore
|
33
33
|
- LICENSE
|
34
|
-
- README
|
34
|
+
- README.md
|
35
35
|
- Rakefile
|
36
36
|
- VERSION
|
37
37
|
- dist/BitTornado/BT1/Choker.py
|