beaker 3.22.0 → 3.23.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,15 +1,15 @@
1
1
  ---
2
2
  !binary "U0hBMQ==":
3
3
  metadata.gz: !binary |-
4
- MDNlYWQ5NWZjNDVjNzA3OGU2NjZiYjJjYzZmMGE2ZTQ1M2E0NTAyOQ==
4
+ NmM3NDg3NTZhMTAzZjM5Mzc5NGMzMDJjMjU1MDQ4NDY5NmRkZGViOA==
5
5
  data.tar.gz: !binary |-
6
- OGE5ZGYxYzg4MzNjMWEzZWExMjcwY2E3OWVhYjEyODM4ZmQ2ZWI2ZA==
6
+ ODg4NTRkZjBmZDEyZmE3NDhiNjI3OTNlYWYxOTk5ZmM4ZDIwMmE5Ng==
7
7
  SHA512:
8
8
  metadata.gz: !binary |-
9
- MjRiNTAyNmNlNmU2OTA3OTJhMWYxZDFkZWJkMjVkODRmMWNmMTYzNzljMGEx
10
- NTRkNDdiMmI5YWJkNmEyNmJhNTBkMzliNTgwMDc3N2VmZDczY2UyYzZmZWI5
11
- OTUxMmQ3ZGNjZjE3NTlkZTk5NTJiNjIwNDY4NmVlMmE4MTE5OWQ=
9
+ ZjgwNzAyZTIxZGNjYzA1MDgwYzVkMTEyZjBjZTU0NTU3ZDQ5ODNiNTA4OTk4
10
+ NGI4OTljMmJiMmM3N2M4YWNkNTliNzQ1YTExMmU3MzJhYmIzZmFmOTNhYjYx
11
+ MWFhNGEzMjgzOTRlM2YzNzRlN2Q0YzhlOTdjZmExNmNkYThmOWM=
12
12
  data.tar.gz: !binary |-
13
- ODQwYThmMTc3NmQwYTQ1ODhkMGNlMDlmYjkxMWFhN2YxYzZhYTlmNTE5OWMw
14
- YzAyMTg0NGU2NDE0NDNiZjA3ODJhMWQ3MzE5NzhjNTM2N2Y4ZWY3ZjE0NzAy
15
- MmJlODdhMWMwYTIxODY3Mjg5ZTk3MGVlYjExM2U5ZmU0MDc1ZWY=
13
+ MTNkMmUzMWQ1MjBhZDY3OWM5ZjhiMzYzYzMxYTRiYTZkMDIxMzc2MjM5ZTI0
14
+ MzA3MGExYWE2OWQzYmVmZDg1MWJkOWMxZDg1MjMzNjNkNWIwMTkwNTcyMTUw
15
+ MmI3NDQwY2RiYThiYjA3ZDE3YmE5OWE3ZWFjYmFkMThlMmU1Yjg=
data/.gitignore CHANGED
@@ -24,3 +24,6 @@ doc
24
24
  # Vagrant folder
25
25
  .vagrant/
26
26
  .vagrant_files/
27
+ # Byebug
28
+ .byebugrc
29
+ .byebug_history
@@ -24,7 +24,6 @@ Gem::Specification.new do |s|
24
24
  s.add_development_dependency 'rspec-its'
25
25
  s.add_development_dependency 'fakefs', '~> 0.6'
26
26
  s.add_development_dependency 'simplecov'
27
- s.add_development_dependency 'pry', '~> 0.10'
28
27
  s.add_development_dependency 'rake', '~> 10.0'
29
28
 
30
29
  # Documentation dependencies
@@ -33,6 +32,9 @@ Gem::Specification.new do |s|
33
32
  # Run time dependencies
34
33
  s.add_runtime_dependency 'minitest', '~> 5.4'
35
34
  s.add_runtime_dependency 'minitar', '~> 0.6'
35
+ s.add_runtime_dependency 'pry-byebug', '~> 3.4.2'
36
+ # pry-byebug can have issues with native readline libs so add rb-readline
37
+ s.add_runtime_dependency 'rb-readline', '~> 0.5.3'
36
38
 
37
39
  s.add_runtime_dependency 'hocon', '~> 1.0'
38
40
  s.add_runtime_dependency 'net-ssh', '~> 4.0'
@@ -50,19 +52,14 @@ Gem::Specification.new do |s|
50
52
  s.add_runtime_dependency 'beaker-hostgenerator'
51
53
  s.add_runtime_dependency 'beaker-puppet', '~> 0.0'
52
54
 
53
- # A minor setback for a major comeback (BKR-841)
54
- #
55
- # Beaker uses nokogiri but it was not declared as direct dependency
56
- # before as one of the hypervisor gems included it
57
- s.add_runtime_dependency 'nokogiri', '~> 1.8.0'
58
-
59
55
  # Optional provisioner specific support
60
56
  s.add_runtime_dependency 'beaker-docker', '~> 0.1'
61
57
  s.add_runtime_dependency 'beaker-aws', '~> 0.1'
62
- s.add_runtime_dependency 'beaker-vmpooler', '~> 0.1'
58
+ s.add_runtime_dependency 'beaker-vmpooler', '~> 1.0'
63
59
  s.add_runtime_dependency 'beaker-google', '~> 0.1'
64
60
  s.add_runtime_dependency 'beaker-vagrant', '~> 0.1'
65
61
  s.add_runtime_dependency 'beaker-vmware', '~> 0.1'
66
62
  s.add_runtime_dependency 'beaker-openstack', '~> 0.1'
63
+ s.add_runtime_dependency 'beaker-vcloud', '~> 0.1'
67
64
 
68
65
  end
@@ -0,0 +1,404 @@
1
+ # Debug beaker tests
2
+
3
+ beaker includes [pry-byebug](https://github.com/deivid-rodriguez/pry-byebug), a gem that combines two powerful related tools: pry and byebug
4
+
5
+ ### What is Pry?
6
+
7
+ [Pry](http://pryrepl.org/) is a powerful Ruby editing and debugging tool. Beaker uses Pry runtime invocation to create a developer console.
8
+
9
+ ### What is byebug?
10
+ [Byebug](https://github.com/deivid-rodriguez/byebug) is a powerful debugger for ruby. It allows for flexible control of breakpoints, stepping through lines or the call stack. It lacks some features of pry such as source code editing and replaying code execution from within the debugging session, but can be used in combination with pry.
11
+
12
+ There are several ways to have your tests break and enter debugging:
13
+
14
+ 1. [Add a breakpoint to your test file](#add-a-breakpoint-to-your-test-file)
15
+ 2. [Use the --debug-errors option](#use-the---debug-errors-option)
16
+ 3. [Defining external breakpoints with byebug](#defining-external-breakpoints-with-byebug)
17
+
18
+ ## Add a breakpoint to your test file
19
+
20
+ Add Pry to individual tests by adding `require 'pry'` to the Ruby test file.
21
+ Then add the statement `binding.pry` at the point in code where you want access to the full, current Beaker environment.
22
+
23
+ ### Example
24
+ #### Example test trypry.rb
25
+ Here's a test file that exercises different ways of running commands on Beaker hosts. At the end of the main `hosts.each` loop I've included `binding.pry` to invoke the console.
26
+
27
+ ```
28
+ hosts.each do |h|
29
+ on h, "echo hello"
30
+ if h['platform'] =~ /windows/
31
+ scp_to h, "beaker.gemspec", "/cygdrive/c/Documents\ and\ Settings/All\ Users/Application\ Data/"
32
+ end
33
+ on(h, "echo test block") do |result|
34
+ puts "block result.stdout: #{result.stdout}"
35
+ puts "block result.raw_stdout: #{result.raw_stdout}"
36
+ end
37
+ on(h, "echo test block, built in functions") do
38
+ puts "built in function stdout: #{stdout}"
39
+ puts "built in function stderr: #{stderr}"
40
+ end
41
+
42
+ result = on(h, "echo no block")
43
+ puts "return var result.stdout: #{result.stdout}"
44
+ puts "return var result.raw_stdout: #{result.raw_stdout}"
45
+
46
+ binding.pry
47
+
48
+ end
49
+ ```
50
+
51
+ #### Sample output to the first `binding.pry` call:
52
+ ```
53
+ $ bundle exec beaker --tests tests/trypry.rb --hosts configs/fusion/winfusion.cfg --no-provision
54
+ {
55
+ "project": "Beaker",
56
+ ...
57
+ "helper": [],
58
+ "load_path": [],
59
+ "pre_suite": [],
60
+ "post_suite": [],
61
+ "install": [],
62
+ "modules": [],
63
+ "logger": "#<Beaker::Logger:0x007f925a6b4218>"
64
+ }
65
+ Hypervisor for pe-centos6 is none
66
+ Hypervisor for w2k8r2 is none
67
+ Hypervisor for w2k3r2 is none
68
+ Beaker::Hypervisor, found some none boxes to create
69
+
70
+ pe-centos6 10:55:27$ which curl
71
+ /usr/bin/curl
72
+
73
+ pe-centos6 executed in 0.14 seconds
74
+
75
+ pe-centos6 10:55:27$ which ntpdate
76
+ /usr/sbin/ntpdate
77
+
78
+ pe-centos6 executed in 0.01 seconds
79
+
80
+ w2k8r2 10:55:27$ which curl
81
+ /bin/curl
82
+
83
+ w2k8r2 executed in 0.42 seconds
84
+
85
+ w2k3r2 10:55:27$ which curl
86
+ /bin/curl
87
+
88
+ w2k3r2 executed in 0.29 seconds
89
+ No tests to run for suite 'pre_suite'
90
+ Begin tests/trypry.rb
91
+
92
+ pe-centos6 10:55:28$ echo hello
93
+ hello
94
+
95
+ pe-centos6 executed in 0.01 seconds
96
+
97
+ pe-centos6 10:55:28$ echo test block
98
+ test block
99
+
100
+ pe-centos6 executed in 0.01 seconds
101
+ block result.stdout: test block
102
+ block result.raw_stdout: test block
103
+
104
+ pe-centos6 10:55:28$ echo test block, built in functions
105
+ test block, built in functions
106
+
107
+ pe-centos6 executed in 0.00 seconds
108
+ built in function stdout: test block, built in functions
109
+ built in function stderr:
110
+
111
+ pe-centos6 10:55:28$ echo no block
112
+ no block
113
+
114
+ pe-centos6 executed in 0.00 seconds
115
+ return var result.stdout: no block
116
+ return var result.raw_stdout: no block
117
+
118
+ From: /Users/anode/beaker/tests/trypry.rb @ line 19 self.run_test:
119
+
120
+ 14:
121
+ 15: result = on(h, "echo no block")
122
+ 16: puts "return var result.stdout: #{result.stdout}"
123
+ 17: puts "return var result.raw_stdout: #{result.raw_stdout}"
124
+ 18:
125
+ => 19: binding.pry
126
+ 20:
127
+ 21: end
128
+
129
+ [1] pry(#<Beaker::TestCase>)>
130
+ ```
131
+ #### Using the console
132
+ At this point I have access to the console. I have full access to Beaker hosts, the Beaker DSL and Ruby.
133
+
134
+ Here's some sample console calls:
135
+ ```
136
+ [1] pry(#<Beaker::TestCase>)> hosts
137
+ => [pe-centos6, w2k8r2, w2k3r2]
138
+ [2] pry(#<Beaker::TestCase>)> on hosts[1], 'echo hello'
139
+
140
+ w2k8r2 10:54:11$ echo hello
141
+ hello
142
+
143
+ w2k8r2 executed in 0.07 seconds
144
+ => #<Beaker::Result:0x007f9f6b7a3408
145
+ @cmd=" echo hello ",
146
+ @exit_code=0,
147
+ @host="w2k8r2",
148
+ @output="hello\n",
149
+ @raw_output="hello\n",
150
+ @raw_stderr="",
151
+ @raw_stdout="hello\n",
152
+ @stderr="",
153
+ @stdout="hello\n">
154
+ [3] pry(#<Beaker::TestCase>)> on hosts[1], 'ls /cygdrive/c/Documents\ and\ Settings/All\ Users/Application\ Data/'
155
+
156
+ w2k8r2 10:56:15$ ls /cygdrive/c/Documents\ and\ Settings/All\ Users/Application\ Data/
157
+ Application Data
158
+ Desktop
159
+ Documents
160
+ Favorites
161
+ Microsoft
162
+ Package Cache
163
+ Start Menu
164
+ Templates
165
+ VMware
166
+ beaker.gemspec
167
+ ntuser.pol
168
+
169
+ w2k8r2 executed in 0.09 seconds
170
+ => #<Beaker::Result:0x007f925b227898
171
+ @cmd=
172
+ " ls /cygdrive/c/Documents\\ and\\ Settings/All\\ Users/Application\\ Data/ ",
173
+ @exit_code=0,
174
+ @host="w2k8r2",
175
+ @output=
176
+ "Application Data\nDesktop\nDocuments\nFavorites\nMicrosoft\nPackage Cache\nStart Menu\nTemplates\nVMware\nbeaker.gemspec\nntuser.pol\n",
177
+ @raw_output=
178
+ "Application Data\nDesktop\nDocuments\nFavorites\nMicrosoft\nPackage Cache\nStart Menu\nTemplates\nVMware\nbeaker.gemspec\nntuser.pol\n",
179
+ @raw_stderr="",
180
+ @raw_stdout=
181
+ "Application Data\nDesktop\nDocuments\nFavorites\nMicrosoft\nPackage Cache\nStart Menu\nTemplates\nVMware\nbeaker.gemspec\nntuser.pol\n",
182
+ @stderr="",
183
+ @stdout=
184
+ "Application Data\nDesktop\nDocuments\nFavorites\nMicrosoft\nPackage Cache\nStart Menu\nTemplates\nVMware\nbeaker.gemspec\nntuser.pol\n">
185
+ [4] pry(#<Beaker::TestCase>)> result = on hosts[1], 'ls /cygdrive/c/Documents\ and\ Settings/All\ Users/Application\ Data/'
186
+
187
+ w2k8r2 10:56:34$ ls /cygdrive/c/Documents\ and\ Settings/All\ Users/Application\ Data/
188
+ Application Data
189
+ Desktop
190
+ Documents
191
+ Favorites
192
+ Microsoft
193
+ Package Cache
194
+ Start Menu
195
+ Templates
196
+ VMware
197
+ beaker.gemspec
198
+ ntuser.pol
199
+
200
+ w2k8r2 executed in 0.08 seconds
201
+ => #<Beaker::Result:0x007f925a387018
202
+ @cmd=
203
+ " ls /cygdrive/c/Documents\\ and\\ Settings/All\\ Users/Application\\ Data/ ",
204
+ @exit_code=0,
205
+ @host="w2k8r2",
206
+ @output=
207
+ "Application Data\nDesktop\nDocuments\nFavorites\nMicrosoft\nPackage Cache\nStart Menu\nTemplates\nVMware\nbeaker.gemspec\nntuser.pol\n",
208
+ @raw_output=
209
+ "Application Data\nDesktop\nDocuments\nFavorites\nMicrosoft\nPackage Cache\nStart Menu\nTemplates\nVMware\nbeaker.gemspec\nntuser.pol\n",
210
+ @raw_stderr="",
211
+ @raw_stdout=
212
+ "Application Data\nDesktop\nDocuments\nFavorites\nMicrosoft\nPackage Cache\nStart Menu\nTemplates\nVMware\nbeaker.gemspec\nntuser.pol\n",
213
+ @stderr="",
214
+ @stdout=
215
+ "Application Data\nDesktop\nDocuments\nFavorites\nMicrosoft\nPackage Cache\nStart Menu\nTemplates\nVMware\nbeaker.gemspec\nntuser.pol\n">
216
+ [5] pry(#<Beaker::TestCase>)> result.stdout.chomp
217
+ => "Application Data\nDesktop\nDocuments\nFavorites\nMicrosoft\nPackage Cache\nStart Menu\nTemplates\nVMware\nbeaker.gemspec\nntuser.pol"
218
+ ```
219
+ #### Continue regular test execution
220
+ Simply `exit` the console.
221
+ ```
222
+ [6] pry(#<Beaker::TestCase>)> exit
223
+ ```
224
+
225
+ ## Use the --debug-errors option
226
+
227
+ You can run beaker and pass the option `--debug-errors` to have beaker enter the pry console if or when a test error occurs. This is useful for transient/intermittent errors where you may need to run the test a large number of times before seeing the failure. It's also useful to keep your hosts at the state where error occurred, as the test's teardown will not (yet) have executed.
228
+
229
+ ### Example
230
+
231
+ Consider this example test case _test.rb_, which is going to fail:
232
+
233
+ test_name 'An important aspect of my product'
234
+
235
+ important_expected_value = 3
236
+
237
+ step 'Assert expected matches actual' do
238
+ actual_value = 2
239
+ assert_equal(important_expected_value, actual_value, 'This product is faulty')
240
+ end
241
+
242
+ You can run this test using `beaker -t test.rb` or (to use the `run` subcommand and just run the test and bypass host provisioning etc) `beaker run -t test.rb`
243
+ It fails.
244
+
245
+ Now try `beaker run -t test.rb --debug-errors`
246
+ This will enter a pry console when the failure occurs.
247
+
248
+ * Assert expected matches actual
249
+ Exception raised during step execution and debug-errors option is set, entering pry. Exception was: #<Minitest::Assertion: This product is faulty.
250
+ Expected: 3
251
+ Actual: 2>
252
+ HINT: Use the pry 'backtrace' and 'up' commands to navigate to the test code
253
+
254
+ From: /home/puppet/code/beaker/lib/beaker/dsl/structure.rb @ line 51 Beaker::DSL::Structure#step:
255
+
256
+ 38: def step step_name, &block
257
+ 39: logger.notify "\n* #{step_name}\n"
258
+ 40: set_current_step_name(step_name)
259
+ 41: if block_given?
260
+ 42: logger.step_in()
261
+ 43: begin
262
+ 44: yield
263
+ 45: rescue Exception => e
264
+ 46: if(@options.has_key?(:debug_errors) && @options[:debug_errors] == true)
265
+ 47: logger.info("Exception raised during step execution and debug-errors option is set, entering pry. Exception was: #{e.inspect}")
266
+ 48: logger.info("HINT: Use the pry 'backtrace' and 'up' commands to navigate to the test code")
267
+ 49: binding.pry
268
+ 50: end
269
+ => 51: raise e
270
+ 52: end
271
+ 53: logger.step_out()
272
+ 54: end
273
+ 55: end
274
+
275
+ [1] pry(#<Beaker::TestCase>)>
276
+
277
+ Because beaker is catching the exception, pry is showing you beaker code. But you can navigate to your own test code using the `backtrace` and `up` commands.
278
+
279
+ `backtrace` shows output like:
280
+
281
+ [1] pry(#<Beaker::TestCase>)> backtrace
282
+ --> #0 rescue in Beaker::DSL::Structure.rescue in step(step_name#String, &block#Proc) at /home/puppet/code/beaker/lib/beaker/dsl/structure.rb:51
283
+ #1 Beaker::DSL::Structure.step(step_name#String, &block#Proc) at /home/puppet/code/beaker/lib/beaker/dsl/structure.rb:43
284
+ #2 block in #<Class:#<Beaker::TestCase:0x00000001761848>>.block in run_test at /home/puppet/code/beaker/test.rb:5
285
+ I+--- #3 Kernel.eval(*args) at /home/puppet/code/beaker/lib/beaker/test_case.rb:133
286
+ #4 block in #<Class:#<Beaker::TestCase:0x00000001761848>>.block in run_test at /home/puppet/code/beaker/lib/beaker/test_case.rb:133
287
+
288
+ The arrow indicates pry is showing code at the top of the stack trace. At position #2 in the stack you can see line 5 of test.rb - this is the actual test code that failed. You can navigate there by entering the `up` command twice.
289
+
290
+ From: /home/puppet/code/beaker/test.rb @ line 5 self.run_test:
291
+
292
+ 1: test_name 'An important aspect of my product'
293
+ 2:
294
+ 3: important_expected_value = 3
295
+ 4:
296
+ => 5: step 'Assert expected matches actual' do
297
+ 6: actual_value = 2
298
+ 7: assert_equal(important_expected_value, actual_value, 'This product is faulty')
299
+ 8: end
300
+
301
+ [2] pry(#<Beaker::TestCase>)>
302
+
303
+ > **NOTE:** Bear in mind that though you are navigating through your code here; pry is not rewinding code execution. When you exit pry, beaker will continue its execution from the top of the backtrace, failing the test due to the error that occurred.
304
+
305
+ From here, you can examine variables in your test. If you have hosts provisioned, you can send them commands using the [on](http://www.rubydoc.info/github/puppetlabs/beaker/Beaker/DSL/Helpers/HostHelpers#on-instance_method) method, or in a separate terminal ssh into them to investigate their state at the point of failure.
306
+
307
+ ## Defining external breakpoints with byebug
308
+ This method is perhaps the least intuitive to use; but combines the advantages of letting you pick the line(s) of code to set breakpoints on; and being able to debug without needing to edit your test code directly. It is sometimes desirable to debug a test without changing the test's code at all; for example, if you want to compare a test at two or more different git commits, your test source needs to remain unmodified from its committed form.
309
+
310
+ byebug allows you to externally define breakpoints to keep them separate from your source code.
311
+
312
+ ### Example
313
+ Consider the following test file test.rb:
314
+
315
+ ```ruby
316
+ test_name 'An important aspect of my product'
317
+
318
+ important_expected_value = 3
319
+
320
+ step 'Assert expected matches actual' do
321
+ actual_value = 2
322
+ assert_equal(important_expected_value, actual_value, 'This product is faulty')
323
+ end
324
+ ```
325
+
326
+ If you wanted to debug at line 6 to investigate the state of your System Under Test before the failing assertion executes, then create a file in your working directory named .byebugrc with contents:
327
+
328
+ ```
329
+ break test.rb:7
330
+ ```
331
+
332
+ You can then run beaker within byebug like this:
333
+
334
+ ```
335
+ [centos@test-development project]$ bundle exec byebug beaker -t test.rb
336
+
337
+ [4, 13] in /home/puppet/code/beaker/vendor/bundle/ruby/2.3.0/bin/beaker
338
+ 4: #
339
+ 5: # The application 'beaker' is installed as part of a gem, and
340
+ 6: # this file is here to facilitate running it.
341
+ 7: #
342
+ 8:
343
+ => 9: require 'rubygems'
344
+ 10:
345
+ 11: version = ">= 0.a"
346
+ 12:
347
+ 13: if ARGV.first
348
+ (byebug)
349
+ ```
350
+
351
+ Note that byebug has immediately stopped at the first line of beaker and awaits your command. Type 'continue' to command byebug to continue execution until it next hits a breakpoint.
352
+
353
+ ```
354
+ 8:
355
+ => 9: require 'rubygems'
356
+ 10:
357
+ 11: version = ">= 0.a"
358
+ 12:
359
+ 13: if ARGV.first
360
+ (byebug) continue
361
+ Beaker!
362
+ wWWWw
363
+ |o o|
364
+ | O | 3.18.0!
365
+ |(")|
366
+ / \X/ \
367
+ | V |
368
+ | | |
369
+ {
370
+ "project": "Beaker",
371
+ "department": "unknown",
372
+ "created_by": "centos",
373
+ "host_tags": {},
374
+ "openstack_api_key": null,
375
+ "openstack_username": null,
376
+ ...
377
+ "timestamp": "2017-06-23 13:21:11 +0000",
378
+ "beaker_version": "3.18.0",
379
+ "log_prefix": "beaker_logs",
380
+ "xml_dated_dir": "junit/beaker_logs/2017-06-23_13_21_11",
381
+ "log_dated_dir": "log/beaker_logs/2017-06-23_13_21_11",
382
+ "logger_sut": "#<Beaker::Logger:0x000000034a2998>"
383
+ }
384
+ No tests to run for suite 'pre_suite'
385
+ Begin test.rb
386
+
387
+ An important aspect of my product
388
+
389
+ * Assert expected matches actual
390
+ Stopped by breakpoint 1 at test.rb:7
391
+
392
+ [1, 8] in test.rb
393
+ 1: test_name 'An important aspect of my product'
394
+ 2:
395
+ 3: important_expected_value = 3
396
+ 4:
397
+ 5: step 'Assert expected matches actual' do
398
+ 6: actual_value = 2
399
+ => 7: assert_equal(important_expected_value, actual_value, 'This product is faulty')
400
+ 8: end
401
+ (byebug)
402
+ ```
403
+
404
+ You can now debug at this point in the file using [byebug commands](https://github.com/deivid-rodriguez/byebug#byebugs-commands)