mobilize-ssh 1.35 → 1.36

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -1,291 +1,4 @@
1
- Mobilize-Ssh
2
- ============
1
+ Mobilize
2
+ ========
3
3
 
4
- Mobilize-Ssh adds the power of ssh to [mobilize-base][mobilize-base].
5
- * move files, execute scripts, and output logs and datasets, all through
6
- Google Spreadsheets.
7
-
8
- Table Of Contents
9
- -----------------
10
- * [Overview](#section_Overview)
11
- * [Install](#section_Install)
12
- * [Mobilize-Ssh](#section_Install_Mobilize-Ssh)
13
- * [Install Dirs and Files](#section_Install_Dirs_and_Files)
14
- * [Configure](#section_Configure)
15
- * [Ssh](#section_Configure_Ssh)
16
- * [Git](#section_Configure_Git)
17
- * [Start](#section_Start)
18
- * [Create Job](#section_Start_Create_Job)
19
- * [Run Test](#section_Start_Run_Test)
20
- * [Meta](#section_Meta)
21
- * [Author](#section_Author)
22
-
23
- <a name='section_Overview'></a>
24
- Overview
25
- -----------
26
-
27
- * Mobilize-ssh adds script deployment to mobilize-base.
28
-
29
- <a name='section_Install'></a>
30
- Install
31
- ------------
32
-
33
- Make sure you go through all the steps in the [mobilize-base][mobilize-base]
34
- install section first.
35
-
36
- <a name='section_Install_Mobilize-Ssh'></a>
37
- ### Mobilize-Ssh
38
-
39
- add this to your Gemfile:
40
-
41
- ``` ruby
42
- gem "mobilize-ssh"
43
- ```
44
-
45
- or do
46
-
47
- $ gem install mobilize-ssh
48
-
49
- for a ruby-wide install.
50
-
51
- <a name='section_Install_Dirs_and_Files'></a>
52
- ### Dirs and Files
53
-
54
- ### Rakefile
55
-
56
- Inside the Rakefile in your project's root dir, make sure you have:
57
-
58
- ``` ruby
59
- require 'mobilize-base/tasks'
60
- require 'mobilize-ssh/tasks'
61
- ```
62
-
63
- This defines rake tasks essential to run the environment.
64
-
65
- ### Config Dir
66
-
67
- run
68
-
69
- $ rake mobilize_ssh:setup
70
-
71
- This will copy over a sample ssh.yml to your config dir.
72
-
73
- It will also add mobilize-ssh to the extensions in jobtracker.yml.
74
-
75
- <a name='section_Configure'></a>
76
- Configure
77
- ------------
78
-
79
- <a name='section_Configure_Ssh'></a>
80
- ### Configure Ssh
81
-
82
- The Ssh configuration consists of:
83
- * nodes, identified by aliases, such as `test_node`. This alias is what you should
84
- pass into the "node" param over in the ssh.run task.
85
- * if no node is specified, commands will default to the first node listed.
86
-
87
- Each node has:
88
- * a host;
89
- * a gateway (optional); If you don't need a gateway, remove that row from the configuration file.
90
- * sudoers; these are user names that are allowed to pass user params
91
- to the run call. This requires passwordless sudo for the host user.
92
- * su_all_users true/false option, which ensures that commands are executed by the
93
- user on the Runner. It prefixes all commands with sudo su <user> before executing the
94
- command. This is strongly recommended if possible as it ensures users do
95
- not overstep their permissions. This requires passwordless sudo for the
96
- host user and accounts on the host machine for each user.
97
-
98
- Each host and gateway has a series of ssh params:
99
- * name - the ip address or name of the host
100
- * key - the relative path of the ssh key file. Default is
101
- "config/mobilize/ssh_private.key"
102
- * port - the port to connect on
103
- * user - the user you are connecting as
104
-
105
- Sample ssh.yml:
106
-
107
- ``` yml
108
- ---
109
- development:
110
- nodes:
111
- dev_node:
112
- sudoers:
113
- - sudo_user
114
- su_all_users: true
115
- host:
116
- name: dev-host.com
117
- key: config/mobilize/ssh_private.key
118
- port: 22
119
- user: host_user
120
- gateway:
121
- name: dev-gateway.com
122
- key: config/mobilize/ssh_private.key
123
- port: 22
124
- user: gateway_user
125
- test:
126
- nodes:
127
- test_node:
128
- sudoers:
129
- - sudo_user
130
- su_all_users: true
131
- host:
132
- name: test-host.com
133
- key: config/mobilize/ssh_private.key
134
- port: 22
135
- user: host_user
136
- gateway:
137
- name: test-gateway.com
138
- key: config/mobilize/ssh_private.key
139
- port: 22
140
- user: gateway_user
141
- production:
142
- nodes:
143
- prod_node:
144
- sudoers:
145
- - sudo_user
146
- su_all_users: true
147
- host:
148
- name: prod-host.com
149
- key: config/mobilize/ssh_private.key
150
- port: 22
151
- user: host_user
152
- gateway:
153
- name: prod-gateway.com
154
- key: config/mobilize/ssh_private.key
155
- port: 22
156
- user: gateway_user
157
- ```
158
-
159
- <a name='section_Configure_Git'></a>
160
- ### Configure Git
161
-
162
- Git configuration is not required but recommended, as it allows you to
163
- pull files directly from public or private Git repositories.
164
-
165
- The Git configuration consists of:
166
- * domains, identified by aliases, such as `private` and `public`.
167
- * domains are passed into the source parameters in the git.run task.
168
- * if no domain is specified, commands will default to the first domain listed.
169
-
170
- Each domain has:
171
- * a host;
172
- * a user, which is the user used for the git clone command.
173
- * a set of repo keys (optional) which correspond to "deploy keys" on
174
- github. Each repo must have its own ssh key, and the public key must be
175
- stored in the repo.
176
- * if your repo doesn't need an ssh key to work, this is not necessary
177
- to add.
178
-
179
-
180
- Sample git.yml:
181
-
182
- ``` yml
183
- ---
184
- development:
185
- domains:
186
- private:
187
- host: github.private.com
188
- user: git
189
- repo_keys:
190
- "repo_path_1": 'local/path/to/key1'
191
- "repo_path_2": 'local/path/to/key1'
192
- public:
193
- host: github.com
194
- user: git
195
- test:
196
- domains:
197
- public:
198
- host: github.com
199
- user: git
200
- private:
201
- host: github.private.com
202
- user: git
203
- repo_keys:
204
- "repo_path_1": 'local/path/to/key1'
205
- "repo_path_2": 'local/path/to/key1'
206
- production:
207
- domains:
208
- private:
209
- host: github.private.com
210
- user: git
211
- repo_keys:
212
- "repo_path_1": 'local/path/to/key1'
213
- "repo_path_2": 'local/path/to/key1'
214
- public:
215
- host: github.com
216
- user: git
217
- ```
218
-
219
-
220
- <a name='section_Start'></a>
221
- Start
222
- -----
223
-
224
- <a name='section_Start_Create_Job'></a>
225
- ### Create Job
226
-
227
- * For mobilize-ssh, the following task is available:
228
- * ssh.run `node: <node_alias>, cmd: <command>, user: user, sources:[*<source_paths>], params:[{<key,value pairs>}]`, which reads sources, copies them to a temporary folder on the selected node, and runs the command inside that folder.
229
- * user, sources, node, and params are optional; cmd is required.
230
- * specifying user will cause the command to be prefixed with sudo su <user> -c.
231
- * non-google sources will also be read as the specified user.
232
- * git sources can be specified with syntax `git://<domain>/<repo_owner>/<repo_name>/<file_path>`.
233
- * Accessing private repos requires that you add the Mobilize public key to the repository as a deploy key.
234
- * there is no user-level access control for git repositories at this time.
235
- * domain defaults to the first one listed, if not included.
236
- * params are also optional for all of the below. They replace tokens in sources and the command.
237
- * params are passed as a YML or JSON, as in:
238
- * `ssh.run source:<source_path>, params:{'date':'2013-03-01', 'unit':'widgets'}`
239
- * this example replaces all the keys, preceded by '@' in all source hqls with the value.
240
- * The preceding '@' is used to keep from replacing instances
241
- of "date" and "unit" in the command/source file; you should have `@date` and `@unit` in your actual HQL
242
- if you'd like to replace those tokens.
243
- * not specifying node will cause the command to be run on the default node.
244
- * ssh sources can be specified with syntax
245
- `ssh://<node><file_full_path>`. If node is omitted, default node will be used.
246
- * `<node><file_full_path>` and `<file_full_path>` can be used in the context of ssh.run, but if the path has only 1 slash, or none, it will try to find a google sheet or file instead.
247
- * The test uses `ssh.run node:"test_node", cmd:"ruby code.rb", user: "root", sources:["code.rb","code.sh"]`
248
-
249
- <a name='section_Start_Run_Test'></a>
250
- ### Run Test
251
-
252
- To run tests, you will need to
253
-
254
- 1) go through the [mobilize-base][mobilize-base] test first
255
-
256
- 2) clone the mobilize-ssh repository
257
-
258
- From the project folder, run
259
-
260
- 3) $ rake mobilize_ssh:setup
261
-
262
- Copy over the config files from the mobilize-base project into the
263
- config dir, and populate the values in the ssh.yml file, esp. the
264
- test_node item.
265
-
266
- You should also copy the ssh private key you wish to use into your
267
- desired path (by default: config/mobilize/ssh_private.key), and make sure it is referenced in ssh.yml
268
-
269
- 3) $ rake test
270
-
271
- This will populate your test Runner from mobilize-base with a sample ssh job.
272
-
273
- The purpose of the test will be to deploy two code files, have the first
274
- execute the second, which is a "tail /var/log/syslog" command, and write the resulting output to a gsheet.
275
-
276
- <a name='section_Meta'></a>
277
- Meta
278
- ----
279
-
280
- * Code: `git clone git://github.com/dena/mobilize-ssh.git`
281
- * Home: <https://github.com/dena/mobilize-ssh>
282
- * Bugs: <https://github.com/dena/mobilize-ssh/issues>
283
- * Gems: <http://rubygems.org/gems/mobilize-ssh>
284
-
285
- <a name='section_Author'></a>
286
- Author
287
- ------
288
-
289
- Cassio Paes-Leme :: cpaesleme@dena.com :: @cpaesleme
290
-
291
- [mobilize-base]: https://github.com/dena/mobilize-base
4
+ Please refer to the mobilize-server wiki: https://github.com/DeNA/mobilize-server/wiki
@@ -1,7 +1,7 @@
1
- namespace :mobilize_ssh do
1
+ require 'yaml'
2
+ namespace :mobilize do
2
3
  desc "Set up config and log folders and files"
3
- task :setup do
4
- require 'yaml'
4
+ task :setup_ssh do
5
5
  sample_dir = File.dirname(__FILE__) + '/../samples/'
6
6
  sample_files = Dir.entries(sample_dir)
7
7
  config_dir = (ENV['MOBILIZE_CONFIG_DIR'] ||= "config/mobilize/")
@@ -1,5 +1,5 @@
1
1
  module Mobilize
2
2
  module Ssh
3
- VERSION = "1.35"
3
+ VERSION = "1.36"
4
4
  end
5
5
  end
data/lib/mobilize-ssh.rb CHANGED
@@ -9,6 +9,9 @@ require "mobilize-ssh/extensions/socket"
9
9
 
10
10
  module Mobilize
11
11
  module Ssh
12
+ def Ssh.home_dir
13
+ File.expand_path('..',File.dirname(__FILE__))
14
+ end
12
15
  end
13
16
  end
14
17
  require "mobilize-ssh/handlers/ssh"
data/mobilize-ssh.gemspec CHANGED
@@ -16,7 +16,7 @@ Gem::Specification.new do |gem|
16
16
  gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
17
17
  gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
18
18
  gem.require_paths = ["lib"]
19
- gem.add_runtime_dependency "mobilize-base","1.35"
19
+ gem.add_runtime_dependency "mobilize-base","1.36"
20
20
  gem.add_runtime_dependency "net-ssh"
21
21
  gem.add_runtime_dependency "net-scp"
22
22
  gem.add_runtime_dependency "net-ssh-gateway"
File without changes
File without changes
File without changes
@@ -0,0 +1,37 @@
1
+ ---
2
+ - path: "Runner_mobilize(test)/jobs"
3
+ state: working
4
+ count: 1
5
+ confirmed_ats: []
6
+ - path: "Runner_mobilize(test)/jobs/ssh1/stage1"
7
+ state: working
8
+ count: 1
9
+ confirmed_ats: []
10
+ - path: "Runner_mobilize(test)/jobs/ssh1/stage2"
11
+ state: working
12
+ count: 1
13
+ confirmed_ats: []
14
+ - path: "Runner_mobilize(test)/jobs/ssh2/stage1"
15
+ state: working
16
+ count: 1
17
+ confirmed_ats: []
18
+ - path: "Runner_mobilize(test)/jobs/ssh2/stage2"
19
+ state: working
20
+ count: 1
21
+ confirmed_ats: []
22
+ - path: "Runner_mobilize(test)/jobs/ssh3/stage1"
23
+ state: working
24
+ count: 1
25
+ confirmed_ats: []
26
+ - path: "Runner_mobilize(test)/jobs/ssh3/stage2"
27
+ state: working
28
+ count: 1
29
+ confirmed_ats: []
30
+ - path: "Runner_mobilize(test)/jobs/ssh4/stage1"
31
+ state: working
32
+ count: 1
33
+ confirmed_ats: []
34
+ - path: "Runner_mobilize(test)/jobs/ssh4/stage2"
35
+ state: working
36
+ count: 1
37
+ confirmed_ats: []
@@ -1,25 +1,25 @@
1
1
  ---
2
- - name: test_ssh_1
2
+ - name: ssh1
3
3
  active: true
4
4
  trigger: once
5
5
  status: ""
6
6
  stage1: ssh.run node:"test_node", cmd:"ruby code.rb", user:"root", sources:["code.rb", "code.sh"]
7
- stage2: gsheet.write source:"stage1", target:"test_ssh_1.out"
8
- - name: test_ssh_2
7
+ stage2: gsheet.write source:"stage1", target:"ssh1.out"
8
+ - name: ssh2
9
9
  active: true
10
- trigger: "after test_ssh_1"
10
+ trigger: "after ssh1"
11
11
  status: ""
12
12
  stage1: ssh.run cmd:"sh code2.sh", user:"root", sources:["code2.sh","test_node/var/log/syslog"], params:{file:"syslog"}
13
- stage2: gsheet.write source:"stage1", target:"test_ssh_2.out"
14
- - name: test_ssh_3
13
+ stage2: gsheet.write source:"stage1", target:"ssh2.out"
14
+ - name: ssh3
15
15
  active: true
16
- trigger: "after test_ssh_2"
16
+ trigger: "after ssh2"
17
17
  status: ""
18
18
  stage1: ssh.run cmd:"echo '@test_param'", params:{test_param:"test param successful"}
19
- stage2: gsheet.write source:"stage1", target:"test_ssh_3.out"
20
- - name: test_ssh_4
19
+ stage2: gsheet.write source:"stage1", target:"ssh3.out"
20
+ - name: ssh4
21
21
  active: true
22
- trigger: "after test_ssh_3"
22
+ trigger: "after ssh3"
23
23
  status: ""
24
24
  stage1: ssh.run node:"test_node", user:root, sources:["git://DeNA/mobilize-ssh/test/code.rb","git://DeNA/mobilize-ssh/test/code.sh"], cmd:"ruby code.rb"
25
- stage2: gsheet.write source:stage1, target:"test_ssh_4.out"
25
+ stage2: gsheet.write source:stage1, target:"ssh4.out"
@@ -0,0 +1,46 @@
1
+ require 'test_helper'
2
+ describe "Mobilize" do
3
+ # enqueues 4 workers on Resque
4
+ it "runs integration test" do
5
+
6
+ puts "restart workers"
7
+ Mobilize::Jobtracker.restart_workers!
8
+
9
+ u = TestHelper.owner_user
10
+ r = u.runner
11
+ user_name = u.name
12
+ gdrive_slot = u.email
13
+
14
+ puts "add test code"
15
+ ["code.rb","code.sh","code2.sh"].each do |fixture_name|
16
+ target_url = "gsheet://#{r.title}/#{fixture_name}"
17
+ TestHelper.write_fixture(fixture_name, target_url, 'replace')
18
+ end
19
+
20
+ puts "add/update jobs"
21
+ u.jobs.each{|j| j.delete}
22
+ jobs_fixture_name = "integration_jobs"
23
+ jobs_target_url = "gsheet://#{r.title}/jobs"
24
+ TestHelper.write_fixture(jobs_fixture_name, jobs_target_url, 'update')
25
+
26
+ puts "job rows added, force enqueue runner, wait for stages"
27
+ #wait for stages to complete
28
+ expected_fixture_name = "integration_expected"
29
+ Mobilize::Jobtracker.stop!
30
+ r.enqueue!
31
+ TestHelper.confirm_expected_jobs(expected_fixture_name)
32
+
33
+ puts "update job status and activity"
34
+ r.update_gsheet(gdrive_slot)
35
+
36
+ puts "jobtracker posted data to test sheets"
37
+ ['ssh1.out','ssh2.out','ssh4.out'].each do |out_name|
38
+ url = "gsheet://#{r.title}/#{out_name}"
39
+ assert TestHelper.check_output(url, 'min_length' => 100) == true
40
+ end
41
+
42
+ #shorter
43
+ url = "gsheet://#{r.title}/ssh3.out"
44
+ assert TestHelper.check_output(url, 'min_length' => 3) == true
45
+ end
46
+ end
data/test/test_helper.rb CHANGED
@@ -8,3 +8,4 @@ $dir = File.dirname(File.expand_path(__FILE__))
8
8
  ENV['MOBILIZE_ENV'] = 'test'
9
9
  require 'mobilize-ssh'
10
10
  $TESTING = true
11
+ require "#{Mobilize::Base.home_dir}/test/test_helper"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: mobilize-ssh
3
3
  version: !ruby/object:Gem::Version
4
- version: '1.35'
4
+ version: '1.36'
5
5
  prerelease:
6
6
  platform: ruby
7
7
  authors:
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2013-04-23 00:00:00.000000000 Z
12
+ date: 2013-05-21 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: mobilize-base
@@ -18,7 +18,7 @@ dependencies:
18
18
  requirements:
19
19
  - - '='
20
20
  - !ruby/object:Gem::Version
21
- version: '1.35'
21
+ version: '1.36'
22
22
  type: :runtime
23
23
  prerelease: false
24
24
  version_requirements: !ruby/object:Gem::Requirement
@@ -26,7 +26,7 @@ dependencies:
26
26
  requirements:
27
27
  - - '='
28
28
  - !ruby/object:Gem::Version
29
- version: '1.35'
29
+ version: '1.36'
30
30
  - !ruby/object:Gem::Dependency
31
31
  name: net-ssh
32
32
  requirement: !ruby/object:Gem::Requirement
@@ -99,12 +99,13 @@ files:
99
99
  - lib/samples/git.yml
100
100
  - lib/samples/ssh.yml
101
101
  - mobilize-ssh.gemspec
102
- - test/code.rb
103
- - test/code.sh
104
- - test/code2.sh
105
- - test/mobilize-ssh_test.rb
102
+ - test/fixtures/code.rb
103
+ - test/fixtures/code.sh
104
+ - test/fixtures/code2.sh
105
+ - test/fixtures/integration_expected.yml
106
+ - test/fixtures/integration_jobs.yml
107
+ - test/integration/mobilize-ssh_test.rb
106
108
  - test/redis-test.conf
107
- - test/ssh_job_rows.yml
108
109
  - test/test_helper.rb
109
110
  homepage: http://github.com/dena/mobilize-ssh
110
111
  licenses: []
@@ -120,7 +121,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
120
121
  version: '0'
121
122
  segments:
122
123
  - 0
123
- hash: -1934183260655929417
124
+ hash: 3534560230256585865
124
125
  required_rubygems_version: !ruby/object:Gem::Requirement
125
126
  none: false
126
127
  requirements:
@@ -129,7 +130,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
129
130
  version: '0'
130
131
  segments:
131
132
  - 0
132
- hash: -1934183260655929417
133
+ hash: 3534560230256585865
133
134
  requirements: []
134
135
  rubyforge_project:
135
136
  rubygems_version: 1.8.25
@@ -137,10 +138,11 @@ signing_key:
137
138
  specification_version: 3
138
139
  summary: extend mobilize-base with the ability to run files across hosts
139
140
  test_files:
140
- - test/code.rb
141
- - test/code.sh
142
- - test/code2.sh
143
- - test/mobilize-ssh_test.rb
141
+ - test/fixtures/code.rb
142
+ - test/fixtures/code.sh
143
+ - test/fixtures/code2.sh
144
+ - test/fixtures/integration_expected.yml
145
+ - test/fixtures/integration_jobs.yml
146
+ - test/integration/mobilize-ssh_test.rb
144
147
  - test/redis-test.conf
145
- - test/ssh_job_rows.yml
146
148
  - test/test_helper.rb
@@ -1,95 +0,0 @@
1
- require 'test_helper'
2
-
3
- describe "Mobilize" do
4
-
5
- def before
6
- puts 'nothing before'
7
- end
8
-
9
- # enqueues 4 workers on Resque
10
- it "runs integration test" do
11
-
12
- puts "restart workers"
13
- Mobilize::Jobtracker.restart_workers!
14
-
15
- gdrive_slot = Mobilize::Gdrive.owner_email
16
- puts "create user 'mobilize'"
17
- user_name = gdrive_slot.split("@").first
18
- u = Mobilize::User.where(:name=>user_name).first
19
- r = u.runner
20
-
21
- rb_code_sheet = Mobilize::Gsheet.find_by_path("#{r.path.split("/")[0..-2].join("/")}/code.rb",gdrive_slot)
22
- sh_code_sheet = Mobilize::Gsheet.find_by_path("#{r.path.split("/")[0..-2].join("/")}/code.sh",gdrive_slot)
23
- sh_code_sheet2 = Mobilize::Gsheet.find_by_path("#{r.path.split("/")[0..-2].join("/")}/code2.sh",gdrive_slot)
24
- [rb_code_sheet,sh_code_sheet].each {|s| s.delete if s}
25
-
26
- puts "add test code"
27
- rb_code_sheet = Mobilize::Gsheet.find_or_create_by_path("#{r.path.split("/")[0..-2].join("/")}/code.rb",gdrive_slot)
28
- rb_code_tsv = File.open("#{Mobilize::Base.root}/test/code.rb").read
29
- rb_code_sheet.write(rb_code_tsv,Mobilize::Gdrive.owner_name)
30
-
31
- sh_code_sheet = Mobilize::Gsheet.find_or_create_by_path("#{r.path.split("/")[0..-2].join("/")}/code.sh",gdrive_slot)
32
- sh_code_tsv = File.open("#{Mobilize::Base.root}/test/code.sh").read
33
- sh_code_sheet.write(sh_code_tsv,Mobilize::Gdrive.owner_name)
34
-
35
- sh_code_sheet2 = Mobilize::Gsheet.find_or_create_by_path("#{r.path.split("/")[0..-2].join("/")}/code2.sh",gdrive_slot)
36
- sh_code_tsv2 = File.open("#{Mobilize::Base.root}/test/code2.sh").read
37
- sh_code_sheet2.write(sh_code_tsv2,Mobilize::Gdrive.owner_name)
38
-
39
- jobs_sheet = r.gsheet(gdrive_slot)
40
-
41
- #delete target sheets if they exist
42
- ssh_target_sheet_1 = Mobilize::Gsheet.find_by_path("#{r.path.split("/")[0..-2].join("/")}/test_ssh_1.out",gdrive_slot)
43
- ssh_target_sheet_2 = Mobilize::Gsheet.find_by_path("#{r.path.split("/")[0..-2].join("/")}/test_ssh_2.out",gdrive_slot)
44
- ssh_target_sheet_3 = Mobilize::Gsheet.find_by_path("#{r.path.split("/")[0..-2].join("/")}/test_ssh_3.out",gdrive_slot)
45
- ssh_target_sheet_4 = Mobilize::Gsheet.find_by_path("#{r.path.split("/")[0..-2].join("/")}/test_ssh_4.out",gdrive_slot)
46
- [ssh_target_sheet_1,ssh_target_sheet_2,ssh_target_sheet_3].each {|s| s.delete if s}
47
-
48
- ssh_job_rows = ::YAML.load_file("#{Mobilize::Base.root}/test/ssh_job_rows.yml")
49
- ssh_job_rows.map{|j| r.jobs(j['name'])}.each{|j| j.delete if j}
50
- jobs_sheet.add_or_update_rows(ssh_job_rows)
51
-
52
- puts "job row added, force enqueue runner, wait for stages"
53
- r.enqueue!
54
- wait_for_stages(900)
55
-
56
- puts "update job status and activity"
57
- r.update_gsheet(gdrive_slot)
58
-
59
- puts "jobtracker posted data to test sheet"
60
- ssh_target_sheet_1 = Mobilize::Gsheet.find_by_path("#{r.path.split("/")[0..-2].join("/")}/test_ssh_1.out",gdrive_slot)
61
- ssh_target_sheet_2 = Mobilize::Gsheet.find_by_path("#{r.path.split("/")[0..-2].join("/")}/test_ssh_2.out",gdrive_slot)
62
- ssh_target_sheet_3 = Mobilize::Gsheet.find_by_path("#{r.path.split("/")[0..-2].join("/")}/test_ssh_3.out",gdrive_slot)
63
- ssh_target_sheet_4 = Mobilize::Gsheet.find_by_path("#{r.path.split("/")[0..-2].join("/")}/test_ssh_4.out",gdrive_slot)
64
-
65
- assert ssh_target_sheet_1.to_tsv.length > 100
66
- assert ssh_target_sheet_2.to_tsv.length > 100
67
- assert ssh_target_sheet_3.to_tsv.length > 3
68
- assert ssh_target_sheet_4.to_tsv.length > 100
69
-
70
- end
71
-
72
- def wait_for_stages(time_limit=600,stage_limit=120,wait_length=10)
73
- time = 0
74
- time_since_stage = 0
75
- #check for 10 min
76
- while time < time_limit and time_since_stage < stage_limit
77
- sleep wait_length
78
- job_classes = Mobilize::Resque.jobs.map{|j| j['class']}
79
- if job_classes.include?("Mobilize::Stage")
80
- time_since_stage = 0
81
- puts "saw stage at #{time.to_s} seconds"
82
- else
83
- time_since_stage += wait_length
84
- puts "#{time_since_stage.to_s} seconds since stage seen"
85
- end
86
- time += wait_length
87
- puts "total wait time #{time.to_s} seconds"
88
- end
89
-
90
- if time >= time_limit
91
- raise "Timed out before stage completion"
92
- end
93
- end
94
-
95
- end