beaker 3.35.0 → 3.36.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +32 -9
- data/CHANGELOG.md +15 -1
- data/CONTRIBUTING.md +37 -31
- data/DOCUMENTING.md +68 -44
- data/README.md +14 -31
- data/acceptance/tests/base/dsl/helpers/host_helpers/curl_on_test.rb +3 -3
- data/beaker.gemspec +1 -1
- data/docs/README.md +5 -12
- data/docs/concepts/beaker_libraries.md +2 -2
- data/docs/concepts/glossary.md +11 -0
- data/docs/concepts/style_guide.md +24 -91
- data/docs/concepts/test_tagging.md +3 -10
- data/docs/concepts/testing_beaker_itself.md +3 -2
- data/docs/concepts/ticket_process.md +17 -45
- data/docs/how_to/archive_sut_files.md +7 -33
- data/docs/how_to/change_terminal_output_coloring.md +7 -4
- data/docs/how_to/confine.md +5 -16
- data/docs/how_to/debug_beaker_tests.md +14 -6
- data/docs/how_to/hosts/README.md +2 -5
- data/docs/how_to/hosts/archlinux.md +4 -2
- data/docs/how_to/hosts/cisco.md +9 -30
- data/docs/how_to/hosts/eos.md +5 -14
- data/docs/how_to/hypervisors/README.md +9 -31
- data/docs/how_to/install_puppet.md +16 -51
- data/docs/how_to/platform_specific_tag_confines.md +11 -31
- data/docs/how_to/preserve_hosts.md +13 -54
- data/docs/how_to/rake_tasks.md +1 -4
- data/docs/how_to/recipes.md +9 -2
- data/docs/how_to/run_in_parallel.md +10 -10
- data/docs/how_to/ssh_agent_forwarding.md +8 -13
- data/docs/how_to/ssh_connection_preference.md +10 -10
- data/docs/how_to/test_arbitrary_beaker_versions.md +5 -12
- data/docs/how_to/the_beaker_dsl.md +44 -21
- data/docs/how_to/upgrade_from_2_to_3.md +13 -53
- data/docs/how_to/use_hocon_helpers.md +10 -28
- data/docs/how_to/use_user_password_authentication.md +13 -10
- data/docs/how_to/write_a_beaker_test_for_a_module.md +6 -2
- data/docs/tutorials/README.md +14 -55
- data/docs/tutorials/creating_a_test_environment.md +47 -52
- data/docs/tutorials/installation.md +57 -21
- data/docs/tutorials/lets_write_a_test.md +8 -1
- data/docs/tutorials/quick_start_rake_tasks.md +63 -62
- data/docs/tutorials/subcommands.md +19 -32
- data/docs/tutorials/test_run.md +14 -16
- data/docs/tutorials/test_suites.md +14 -43
- data/docs/tutorials/the_command_line.md +14 -2
- data/lib/beaker/host/unix/pkg.rb +9 -2
- data/lib/beaker/host_prebuilt_steps.rb +1 -1
- data/lib/beaker/shared/host_manager.rb +6 -3
- data/lib/beaker/version.rb +1 -1
- data/spec/beaker/host/unix/pkg_spec.rb +66 -17
- data/spec/beaker/host/unix_spec.rb +4 -4
- data/spec/beaker/host_prebuilt_steps_spec.rb +10 -6
- data/spec/beaker/hypervisor/hypervisor_spec.rb +2 -2
- data/spec/beaker/shared/host_manager_spec.rb +8 -0
- metadata +9 -2
@@ -1,6 +1,7 @@
|
|
1
1
|
## The Task
|
2
2
|
|
3
3
|
We will write a test to check if a package (specifically HTTPD) is installed and running. To do this we will write two files:
|
4
|
+
|
4
5
|
1. `install.rb` - This file will install the package and start the service
|
5
6
|
2. `mytest.rb` - This file will have our core tests that checks if the package is installed and running
|
6
7
|
|
@@ -18,7 +19,10 @@ What needs to happen in this test:
|
|
18
19
|
* Test HTTPD service is running
|
19
20
|
|
20
21
|
## Create a host configuration file
|
22
|
+
|
23
|
+
```console
|
21
24
|
$ beaker-hostgenerator redhat7-64 > redhat7-64.yaml
|
25
|
+
```
|
22
26
|
|
23
27
|
This command will generate a host file for our system under test (SUT). It will use vmpooler as hypervisor for the host. Please check out [this](https://github.com/puppetlabs/beaker/tree/master/docs/how_to/hypervisors) doc to learn more about hypervisors for beaker.
|
24
28
|
|
@@ -81,8 +85,11 @@ end
|
|
81
85
|
```
|
82
86
|
|
83
87
|
## Run it!
|
88
|
+
|
84
89
|
You can now run this with
|
85
90
|
|
86
|
-
|
91
|
+
```console
|
92
|
+
$ beaker --host redhat7-64ma.yaml --pre-suite install.rb --tests mytest.rb
|
93
|
+
```
|
87
94
|
|
88
95
|
Next up you may want to look at the [Beaker test for a module](../how_to/write_a_beaker_test_for_a_module.md) page.
|
@@ -2,49 +2,52 @@
|
|
2
2
|
|
3
3
|
We have developed some rake tasks to help new Beaker users get up and running quickly with writing and running tests.
|
4
4
|
|
5
|
-
|
6
5
|
## Pre-requisites
|
7
6
|
|
8
7
|
* You will need to have already completed the Beaker installation tutorial - [Beaker Installation](installation.md)
|
9
|
-
|
10
|
-
* Hypervisors are services that provision SUTs for Beaker. We have made two available in this quick start guide to allow you to get up
|
11
|
-
and running. See the docs on how to setup [Vmpooler](https://github.com/puppetlabs/beaker-vmpooler/blob/master/vmpooler.md) and [Vagrant](https://github.com/puppetlabs/beaker-vagrant/blob/master/docs/vagrant.md).
|
12
|
-
|
8
|
+
* Hypervisors are services that provision SUTs for Beaker. We have made two available in this quick start guide to allow you to get up and running. See the docs on how to setup [Vmpooler](https://github.com/puppetlabs/beaker-vmpooler/blob/master/vmpooler.md) and [Vagrant](https://github.com/puppetlabs/beaker-vagrant/blob/master/docs/vagrant.md).
|
13
9
|
|
14
10
|
## How to use them
|
15
11
|
|
16
12
|
To use the tasks, you need to put the following line at the top of your project's rake file:
|
17
13
|
|
14
|
+
```rake
|
18
15
|
require 'beaker/tasks/quick_start'
|
16
|
+
```
|
19
17
|
|
20
18
|
To check that you have access to the quickstart tasks from your project, run:
|
21
19
|
|
20
|
+
```console
|
22
21
|
rake --tasks
|
23
|
-
|
22
|
+
```
|
23
|
+
|
24
24
|
You should see them listed along with any rake tasks you have defined in your local project rakefile:
|
25
25
|
|
26
|
+
```rake
|
26
27
|
rake beaker_quickstart:gen_hosts[hypervisor] # Generate Default Beaker Host Config File, valid options are: vmpooler or vagrant
|
27
28
|
rake beaker_quickstart:gen_pre_suite # Generate Default Pre-Suite
|
28
29
|
rake beaker_quickstart:gen_smoke_test # Generate Default Smoke Test
|
29
|
-
rake beaker_quickstart:run_test[hypervisor] # Run Default Smoke Test, after generating default host config and test files, valid
|
30
|
+
rake beaker_quickstart:run_test[hypervisor] # Run Default Smoke Test, after generating default host config and test files, valid
|
30
31
|
options are: vmpooler or vagrant
|
31
|
-
|
32
|
+
```
|
32
33
|
|
33
34
|
## Tasks
|
34
35
|
|
35
36
|
### Hypervisor Argument
|
36
37
|
|
37
|
-
Some of the tasks below take a 'hypervisor' argument that you can pass either 'vmpooler' or 'vagrant' to. If you leave it empty, the
|
38
|
-
task will default to using 'vagrant'.
|
38
|
+
Some of the tasks below take a 'hypervisor' argument that you can pass either 'vmpooler' or 'vagrant' to. If you leave it empty, the task will default to using 'vagrant'.
|
39
39
|
|
40
40
|
Example:
|
41
41
|
|
42
|
-
|
42
|
+
```console
|
43
|
+
$ rake beaker_quickstart:gen_hosts[vmpooler]
|
44
|
+
```
|
43
45
|
|
44
46
|
If you have the zsh shell then you will need to escape the square brackets:
|
45
47
|
|
48
|
+
```console
|
46
49
|
rake beaker_quickstart:gen_hosts\[vmpooler\]
|
47
|
-
|
50
|
+
```
|
48
51
|
|
49
52
|
### Generate tasks
|
50
53
|
|
@@ -58,23 +61,24 @@ These tasks are standalone and can be run independently from each other to gener
|
|
58
61
|
|
59
62
|
To run:
|
60
63
|
|
61
|
-
|
64
|
+
```console
|
65
|
+
$ rake beaker_quickstart:gen_hosts[hypervisor]
|
66
|
+
```
|
62
67
|
|
63
68
|
The gen_hosts task will create a file 'default_hypervisor_hosts.yaml' in acceptance/config.
|
64
69
|
|
65
|
-
If the file already exists, it will not be overwritten. This will allow you to play around with the config yourself, either by manually
|
66
|
-
editing the file or by using beaker-hostgenerator to generate a new hosts config.
|
67
|
-
|
70
|
+
If the file already exists, it will not be overwritten. This will allow you to play around with the config yourself, either by manually editing the file or by using beaker-hostgenerator to generate a new hosts config.
|
68
71
|
|
69
72
|
Vmpooler file (redhat 7 master and agent):
|
70
73
|
|
74
|
+
```yaml
|
71
75
|
---
|
72
76
|
HOSTS:
|
73
77
|
redhat7-64-1:
|
74
|
-
pe_dir:
|
75
|
-
pe_ver:
|
76
|
-
pe_upgrade_dir:
|
77
|
-
pe_upgrade_ver:
|
78
|
+
pe_dir:
|
79
|
+
pe_ver:
|
80
|
+
pe_upgrade_dir:
|
81
|
+
pe_upgrade_ver:
|
78
82
|
hypervisor: vmpooler
|
79
83
|
platform: el-7-x86_64
|
80
84
|
template: redhat-7-x86_64
|
@@ -86,10 +90,10 @@ Vmpooler file (redhat 7 master and agent):
|
|
86
90
|
- classifier
|
87
91
|
- default
|
88
92
|
redhat7-64-2:
|
89
|
-
pe_dir:
|
90
|
-
pe_ver:
|
91
|
-
pe_upgrade_dir:
|
92
|
-
pe_upgrade_ver:
|
93
|
+
pe_dir:
|
94
|
+
pe_ver:
|
95
|
+
pe_upgrade_dir:
|
96
|
+
pe_upgrade_ver:
|
93
97
|
hypervisor: vmpooler
|
94
98
|
platform: el-7-x86_64
|
95
99
|
template: redhat-7-x86_64
|
@@ -100,18 +104,18 @@ Vmpooler file (redhat 7 master and agent):
|
|
100
104
|
nfs_server: none
|
101
105
|
consoleport: 443
|
102
106
|
pooling_api: http://vmpooler.delivery.puppetlabs.net/
|
103
|
-
|
104
|
-
|
107
|
+
```
|
105
108
|
|
106
109
|
Vagrant file (ubuntu 14 master and agent):
|
107
110
|
|
111
|
+
```yaml
|
108
112
|
---
|
109
113
|
HOSTS:
|
110
114
|
ubuntu1404-64-1:
|
111
|
-
pe_dir:
|
112
|
-
pe_ver:
|
113
|
-
pe_upgrade_dir:
|
114
|
-
pe_upgrade_ver:
|
115
|
+
pe_dir:
|
116
|
+
pe_ver:
|
117
|
+
pe_upgrade_dir:
|
118
|
+
pe_upgrade_ver:
|
115
119
|
platform: ubuntu-14.04-amd64
|
116
120
|
hypervisor: vagrant
|
117
121
|
roles:
|
@@ -122,10 +126,10 @@ Vagrant file (ubuntu 14 master and agent):
|
|
122
126
|
- classifier
|
123
127
|
- default
|
124
128
|
ubuntu1404-64-2:
|
125
|
-
pe_dir:
|
126
|
-
pe_ver:
|
127
|
-
pe_upgrade_dir:
|
128
|
-
pe_upgrade_ver:
|
129
|
+
pe_dir:
|
130
|
+
pe_ver:
|
131
|
+
pe_upgrade_dir:
|
132
|
+
pe_upgrade_ver:
|
129
133
|
platform: ubuntu-14.04-amd64
|
130
134
|
hypervisor: vagrant
|
131
135
|
roles:
|
@@ -136,44 +140,41 @@ Vagrant file (ubuntu 14 master and agent):
|
|
136
140
|
consoleport: 443
|
137
141
|
box_url: https://vagrantcloud.com/puppetlabs/boxes/ubuntu-14.04-64-nocm
|
138
142
|
box: puppetlabs/ubuntu-14.04-64-nocm
|
139
|
-
|
143
|
+
```
|
140
144
|
|
141
145
|
For more info on host generation and what these configs represent see - [Creating A Test Environment](creating_a_test_environment.md)
|
142
146
|
|
143
|
-
|
144
|
-
### gen_pre_suite
|
147
|
+
### `gen_pre_suite`
|
145
148
|
|
146
149
|
To run:
|
147
150
|
|
148
|
-
|
149
|
-
|
150
|
-
|
151
|
+
```console
|
152
|
+
$ rake beaker_quickstart:gen_pre_suite
|
153
|
+
```
|
151
154
|
|
152
|
-
The gen_pre-suite task will create a file 'default_pre_suite.rb' in acceptance/setup.
|
153
|
-
|
154
|
-
If the file already exists, it will not be overwritten. This will allow you to edit the pre_suite file yourself if required.
|
155
|
+
This task will generate a default Beaker pre-suite file. The gen_pre-suite task will create a file 'default_pre_suite.rb' in acceptance/setup. If the file already exists, it will not be overwritten. This will allow you to edit the pre_suite file yourself if required.
|
155
156
|
|
156
157
|
pre_suite file:
|
157
158
|
|
159
|
+
```ruby
|
158
160
|
install_puppet
|
159
|
-
|
160
|
-
See the [Test Suites doc](test_suites.md) in this directory for more information on pre-suites.
|
161
|
+
```
|
161
162
|
|
163
|
+
See the [Test Suites doc](test_suites.md) in this directory for more information on pre-suites.
|
162
164
|
|
163
|
-
### gen_smoke_test
|
165
|
+
### `gen_smoke_test`
|
164
166
|
|
165
167
|
To run:
|
166
168
|
|
169
|
+
```console
|
167
170
|
rake beaker_quickstart:gen_smoke_test
|
168
|
-
|
169
|
-
This task will generate a default Beaker smoke test file.
|
170
|
-
|
171
|
-
The gen_smoke_test task will create a file 'default_smoke_test.rb' in acceptance/tests.
|
171
|
+
```
|
172
172
|
|
173
|
-
If the file already exists, it will not be overwritten. This will allow you to edit the test file yourself if required.
|
173
|
+
This task will generate a default Beaker smoke test file. The gen_smoke_test task will create a file 'default_smoke_test.rb' in acceptance/tests. If the file already exists, it will not be overwritten. This will allow you to edit the test file yourself if required.
|
174
174
|
|
175
175
|
smoke test file:
|
176
176
|
|
177
|
+
```ruby
|
177
178
|
test_name 'puppet install smoketest' do
|
178
179
|
step 'puppet install smoketest: verify \'puppet help\' can be successfully called on
|
179
180
|
all hosts' do
|
@@ -182,29 +183,29 @@ smoke test file:
|
|
182
183
|
end
|
183
184
|
end
|
184
185
|
end
|
185
|
-
|
186
|
+
```
|
187
|
+
|
186
188
|
This smoke test will check that Puppet has been successfully installed on the hosts.
|
187
189
|
|
188
190
|
For more information on the Beaker dsl methods available to you in your tests see - [Beaker dsl](../how_to/the_beaker_dsl.md)
|
189
191
|
|
190
|
-
|
191
192
|
### Run task
|
192
193
|
|
193
|
-
The beaker_quickstart:run_test task will run all the above tasks in sequential order, to generate a hosts file, pre-suite file, smoke
|
194
|
-
test and then use these files to perform a Beaker test run. If the files already exist (see below for further info on file names and
|
195
|
-
location) then they will not be overwritten.
|
196
|
-
|
194
|
+
The beaker_quickstart:run_test task will run all the above tasks in sequential order, to generate a hosts file, pre-suite file, smoke test and then use these files to perform a Beaker test run. If the files already exist (see below for further info on file names and location) then they will not be overwritten.
|
197
195
|
|
198
196
|
#### run_test
|
199
197
|
|
200
198
|
To run:
|
201
|
-
|
202
|
-
|
203
|
-
|
199
|
+
|
200
|
+
```console
|
201
|
+
$ rake beaker_quickstart:run_test[hypervisor]
|
202
|
+
```
|
203
|
+
|
204
204
|
This task will run the above 3 tasks in sequential order and then execute a Beaker test run using all 3 files.
|
205
205
|
|
206
|
-
|
207
|
-
acceptance/tests/default_smoke_test.rb
|
206
|
+
```console
|
207
|
+
$ beaker --hosts acceptance/config/default_vmpooler_hosts.yaml --pre-suite acceptance/setup/default_pre_suite.rb --tests acceptance/tests/default_smoke_test.rb
|
208
|
+
```
|
208
209
|
|
209
210
|
You will end up with provisioned hosts with puppet installed and a test check executed to verify that puppet was installed.
|
210
211
|
|
@@ -1,55 +1,42 @@
|
|
1
1
|
# Using Subcommands
|
2
2
|
|
3
|
-
The document gives an overview of the subcommands that Beaker supports and
|
4
|
-
describes how to use them.
|
3
|
+
The document gives an overview of the subcommands that Beaker supports and describes how to use them.
|
5
4
|
|
6
5
|
## Why Subcommands?
|
7
6
|
|
8
|
-
Subcommands are designed to make test development and iteration simpler by
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
Most subcommands pass through flags to the Beaker options. For instance, you can
|
16
|
-
pass through `--hosts` to the `init` subcommand and it will parse the `--hosts`
|
17
|
-
argument as if you were executing a Beaker run*. Please review the
|
18
|
-
subcommand specific help for further information. You can see the help for a
|
19
|
-
specific subcommand by running `beaker help SUBCOMMAND`.
|
20
|
-
|
21
|
-
*Please note that in this document, a Beaker `run` is standard beaker invocation
|
22
|
-
without any subcommands.
|
7
|
+
Subcommands are designed to make test development and iteration simpler by separating out all of the phases of a Beaker [test run](test_run.md)*. Instead of requiring the entirety of your Beaker execution in one command, subcommands allow you to execute each phase independently. This allows for faster feedback for potential failures and better control for iterating over actual test development.
|
8
|
+
|
9
|
+
Most subcommands pass through flags to the Beaker options. For instance, you can pass through `--hosts` to the `init` subcommand and it will parse the `--hosts` argument as if you were executing a Beaker run*. Please review the subcommand specific help for further information. You can see the help for a specific subcommand by running `beaker help SUBCOMMAND`.
|
10
|
+
|
11
|
+
Please note that in this document, a Beaker `run` is standard beaker invocation without any subcommands.
|
12
|
+
|
23
13
|
## Available Subcommands
|
24
14
|
|
25
15
|
### beaker init
|
26
|
-
|
27
|
-
`subcommand_options.yaml` file that is user-facing; altering this file will
|
28
|
-
alter the options for subcommand execution.
|
16
|
+
|
17
|
+
Initializes the required `.beaker/` configuration folder. This folder contains a `subcommand_options.yaml` file that is user-facing; altering this file will alter the options for subcommand execution.
|
29
18
|
|
30
19
|
### beaker provision
|
31
|
-
|
32
|
-
`--hosts` flag here to override any hosts provided there.
|
20
|
+
|
21
|
+
Provisions hosts defined in your `subcommand_options file`. You can pass the `--hosts` flag here to override any hosts provided there.
|
33
22
|
|
34
23
|
### beaker exec
|
35
|
-
|
36
|
-
that resource will be run in the context of the `tests` suite; if supplied a
|
37
|
-
Beaker suite, then just that suite will run. If no resource is supplied, then
|
38
|
-
this command executes the suites as they are defined in the configuration in the
|
39
|
-
`subcommand_options.yaml`.
|
24
|
+
|
25
|
+
Run a single file, directory, or Beaker suite. If supplied a file or directory, that resource will be run in the context of the `tests` suite; if supplied a Beaker suite, then just that suite will run. If no resource is supplied, then this command executes the suites as they are defined in the configuration in the `subcommand_options.yaml`.
|
40
26
|
|
41
27
|
### beaker destroy
|
28
|
+
|
42
29
|
Execute this command to deprovision your systems under test(SUTs).
|
43
30
|
|
44
31
|
## Basic workflow
|
45
32
|
|
46
|
-
```
|
47
|
-
beaker init -h hosts_file -o options_file --keyfile ssh_key --pre-suite ./setup/pre-suit/my_presuite.rb
|
48
|
-
beaker provision
|
33
|
+
```console
|
34
|
+
$ beaker init -h hosts_file -o options_file --keyfile ssh_key --pre-suite ./setup/pre-suit/my_presuite.rb
|
35
|
+
$ beaker provision
|
49
36
|
# Note: do not pass in hosts file, or use the '-t' flag! Just the file
|
50
37
|
# or directory. Do not pass GO. Do not collect $200.
|
51
|
-
beaker exec ./tests/my_test.rb
|
38
|
+
$ beaker exec ./tests/my_test.rb
|
52
39
|
# Repeating the above command as needed
|
53
40
|
# When you're done testing using the VM that Beaker provisioned
|
54
|
-
beaker destroy
|
41
|
+
$ beaker destroy
|
55
42
|
```
|
data/docs/tutorials/test_run.md
CHANGED
@@ -1,23 +1,21 @@
|
|
1
1
|
# Beaker Test Runs
|
2
2
|
|
3
|
-
A Beaker test run consists of two primary phases: an SUT provision phase and a
|
4
|
-
suite execution phase. After suite execution, Beaker defaults to cleaning up and
|
5
|
-
destroying the SUTs. Leave SUTs running with the `--preserve-hosts` flag.
|
3
|
+
A Beaker test run consists of two primary phases: an SUT provision phase and a suite execution phase. After suite execution, Beaker defaults to cleaning up and destroying the SUTs. Leave SUTs running with the `--preserve-hosts` flag.
|
6
4
|
|
5
|
+
## Provisioning
|
7
6
|
|
8
|
-
*
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
*
|
13
|
-
*
|
14
|
-
|
15
|
-
|
16
|
-
*
|
17
|
-
|
18
|
-
* skip with `--no-configure`
|
7
|
+
* Using supported hypervisors, provision SUTs for testing on
|
8
|
+
* Do any initial configuration to ensure that the SUTs can communicate with beaker and each other
|
9
|
+
* skip with `--no-provision`
|
10
|
+
* Provisioning also runs some basic setup on the SUTs
|
11
|
+
* Validation
|
12
|
+
* Check the SUTs for necessary packages (curl, ntpdate)
|
13
|
+
* skip with `--no-validate`
|
14
|
+
* Configuration
|
15
|
+
* Do any post-provisioning configuration to the SUTs
|
16
|
+
* skip with `--no-configure`
|
19
17
|
|
18
|
+
## Execution
|
20
19
|
|
21
|
-
*
|
22
|
-
* Execute the files specified in each of the suites; for further documentation,
|
20
|
+
* Execute the files specified in each of the suites; for further documentation,
|
23
21
|
please refer to [Test Suites & Failure Modes](test_suites.md)
|
@@ -1,83 +1,54 @@
|
|
1
1
|
# Test Suites & Failure Modes
|
2
2
|
|
3
|
-
Beaker test suites correspond to test suites in most testing frameworks,
|
4
|
-
being containers for tests or pre- or post-testing files to execute.
|
3
|
+
Beaker test suites correspond to test suites in most testing frameworks, being containers for tests or pre- or post-testing files to execute.
|
5
4
|
|
6
|
-
There are two main ways that you specify which test suites a particular
|
7
|
-
file belongs in. The first way is to use the Beaker command line interface
|
8
|
-
(CLI) arguments. These are specified in runtime order below:
|
5
|
+
There are two main ways that you specify which test suites a particular file belongs in. The first way is to use the Beaker command line interface (CLI) arguments. These are specified in runtime order below:
|
9
6
|
|
10
7
|
--pre-suite
|
11
8
|
—-tests
|
12
9
|
—-post-suite
|
13
10
|
—-pre-cleanup
|
14
11
|
|
15
|
-
If you’d like to find out more information about these arguments, or how
|
16
|
-
exactly you pass the specified files to each suite, execute `beaker —-help`
|
17
|
-
at the CLI.
|
12
|
+
If you’d like to find out more information about these arguments, or how exactly you pass the specified files to each suite, execute `beaker —-help` at the CLI.
|
18
13
|
|
19
|
-
The second way is to provide suite arguments through config files. There
|
20
|
-
are two places that you can provide this info:
|
14
|
+
The second way is to provide suite arguments through config files. There are two places that you can provide this info:
|
21
15
|
|
22
16
|
1. The `CONFIG` section of a hosts file.
|
23
17
|
2. A local options file passed through the `--options-file` CLI argument.
|
24
18
|
|
25
|
-
Either way, the keys for the arguments needed are listed below, respective
|
26
|
-
to the arguments listed above:
|
19
|
+
Either way, the keys for the arguments needed are listed below, respective to the arguments listed above:
|
27
20
|
|
28
21
|
:pre_suite
|
29
22
|
:tests
|
30
23
|
:post_suite
|
31
24
|
:pre_cleanup
|
32
25
|
|
33
|
-
*Note* that one difference between the two methods is that if you provide the
|
34
|
-
options via the CLI, we support various input methods such as specifying
|
35
|
-
individual files vs paths to folders. The second option (providing keys to
|
36
|
-
config files) only works if each file is fully specified. Beaker will not
|
37
|
-
find all files in a directory in the second case, that expansion is only done
|
38
|
-
on the CLI inputs.
|
26
|
+
*Note* that one difference between the two methods is that if you provide the options via the CLI, we support various input methods such as specifying individual files vs paths to folders. The second option (providing keys to config files) only works if each file is fully specified. Beaker will not find all files in a directory in the second case, that expansion is only done on the CLI inputs.
|
39
27
|
|
40
28
|
# Suite Details, & Failure Mode Behavior
|
41
29
|
|
42
|
-
This section is to explain the particulars of any suite, and any warnings or
|
43
|
-
notes about using them, including how they behave in Beaker’s different
|
44
|
-
failure modes.
|
30
|
+
This section is to explain the particulars of any suite, and any warnings or notes about using them, including how they behave in Beaker’s different failure modes.
|
45
31
|
|
46
32
|
## Pre-Suite
|
47
33
|
|
48
|
-
The pre-suite is for setting up the Systems Under Test (SUTs) for the testing
|
49
|
-
suite. No surprises here, usually these files are filled with the setup and
|
50
|
-
installation code needed to verify that the operating assumptions of the
|
51
|
-
software being tested are true.
|
34
|
+
The pre-suite is for setting up the Systems Under Test (SUTs) for the testing suite. No surprises here, usually these files are filled with the setup and installation code needed to verify that the operating assumptions of the software being tested are true.
|
52
35
|
|
53
|
-
Pre-suites, since they’re supposed to contain just setup code, will fail-fast
|
54
|
-
and the entire Beaker run will be abandoned if setup fails, since testing
|
55
|
-
assumes that setup has succeeded.
|
36
|
+
Pre-suites, since they’re supposed to contain just setup code, will fail-fast and the entire Beaker run will be abandoned if setup fails, since testing assumes that setup has succeeded.
|
56
37
|
|
57
38
|
## Tests
|
58
39
|
|
59
|
-
Time to actually test! This suite contains the test files that you would
|
60
|
-
like to verify new code with.
|
40
|
+
Time to actually test! This suite contains the test files that you would like to verify new code with.
|
61
41
|
|
62
42
|
The test suite behaves according to the global fail-mode setting.
|
63
43
|
|
64
44
|
## Post-Suite
|
65
45
|
|
66
|
-
Usually the post-suite is used to clean up any fixtures that your tests might
|
67
|
-
have poisoned, collect any log files that you’d like to later review, and
|
68
|
-
to get any resources from the SUTs before they are cleaned up / deleted.
|
46
|
+
Usually the post-suite is used to clean up any fixtures that your tests might have poisoned, collect any log files that you’d like to later review, and to get any resources from the SUTs before they are cleaned up / deleted.
|
69
47
|
|
70
|
-
The post-suite only runs if the fail-mode is set to slow, which it is by
|
71
|
-
default. Fast fail-mode will skip this suite, so that you can get feedback
|
72
|
-
quicker, and you can potentially check out the system in the state exactly
|
73
|
-
that it failed in.
|
48
|
+
The post-suite only runs if the fail-mode is set to slow, which it is by default. Fast fail-mode will skip this suite, so that you can get feedback quicker, and you can potentially check out the system in the state exactly that it failed in.
|
74
49
|
|
75
50
|
## Pre-cleanup
|
76
51
|
|
77
|
-
The pre-cleanup suite is for tasks that you’d like to run at the end of your
|
78
|
-
tests, regardless of Beaker’s fail-mode. You can think of this behaving as a
|
79
|
-
`finally` block at the end of a `try-rescue` one.
|
52
|
+
The pre-cleanup suite is for tasks that you’d like to run at the end of your tests, regardless of Beaker’s fail-mode. You can think of this behaving as a `finally` block at the end of a `try-rescue` one.
|
80
53
|
|
81
|
-
The pre-cleanup suite falls back to the global fail-mode setting, which
|
82
|
-
defaults to slow, meaning it will run all tests regardless of any early
|
83
|
-
failures.
|
54
|
+
The pre-cleanup suite falls back to the global fail-mode setting, which defaults to slow, meaning it will run all tests regardless of any early failures.
|