rcomp 0.1.1 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.gitignore +0 -6
- data/CHANGELOG.rdoc +8 -0
- data/Gemfile.lock +2 -1
- data/README.md +24 -18
- data/Rakefile +5 -0
- data/features/generate.feature +94 -2
- data/features/init.feature +36 -12
- data/features/step_definitions/test_steps.rb +5 -0
- data/features/step_definitions/version_steps.rb +7 -0
- data/features/support/env.rb +27 -20
- data/features/test.feature +104 -2
- data/features/version.feature +1 -1
- data/lib/rcomp.rb +2 -0
- data/lib/rcomp/actions.rb +6 -0
- data/lib/rcomp/cli.rb +20 -57
- data/lib/rcomp/conf.rb +30 -74
- data/lib/rcomp/initializer.rb +108 -0
- data/lib/rcomp/process.rb +44 -0
- data/lib/rcomp/reporter.rb +12 -15
- data/lib/rcomp/runner.rb +20 -22
- data/lib/rcomp/suite.rb +19 -2
- data/lib/rcomp/version.rb +1 -1
- data/rcomp.gemspec +5 -5
- metadata +24 -6
- data/features/set_command.feature +0 -24
- data/features/set_directory.feature +0 -22
data/.gitignore
CHANGED
data/CHANGELOG.rdoc
CHANGED
@@ -1,3 +1,11 @@
|
|
1
|
+
== 0.2.0, released 2012-11-25
|
2
|
+
|
3
|
+
* Add "ignore" configuration option
|
4
|
+
* Add "timeout" configuration option
|
5
|
+
* Remove "set-command"
|
6
|
+
* Remove "set-directory"
|
7
|
+
* Upgrade "init" to handle command setting when neccessary
|
8
|
+
|
1
9
|
== 0.1.1, released 2012-11-20
|
2
10
|
|
3
11
|
* Ignore dotfiles in tests directory
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -14,13 +14,11 @@ $ gem install rcomp
|
|
14
14
|
```
|
15
15
|
$ rcomp
|
16
16
|
Tasks:
|
17
|
-
rcomp generate
|
18
|
-
rcomp help [TASK]
|
19
|
-
rcomp init
|
20
|
-
rcomp
|
21
|
-
rcomp
|
22
|
-
rcomp test # Run all tests
|
23
|
-
rcomp version # Prints RComp's version information
|
17
|
+
rcomp generate # Generate expected output for all tests
|
18
|
+
rcomp help [TASK] # Describe available tasks or one specific task
|
19
|
+
rcomp init # Setup rcomp test directory
|
20
|
+
rcomp test # Run all tests
|
21
|
+
rcomp version # Prints RComp's version information
|
24
22
|
```
|
25
23
|
|
26
24
|
## Setup
|
@@ -28,7 +26,6 @@ Tasks:
|
|
28
26
|
In your project root directory, run:
|
29
27
|
|
30
28
|
```
|
31
|
-
$ rcomp set-command ./some-executable
|
32
29
|
$ rcomp init
|
33
30
|
```
|
34
31
|
|
@@ -57,7 +54,7 @@ Managed by RComp. Stores the results of your most recent test suite run.
|
|
57
54
|
|
58
55
|
---
|
59
56
|
|
60
|
-
A simple RComp suite might look something like
|
57
|
+
A simple RComp suite might look something like:
|
61
58
|
|
62
59
|
```
|
63
60
|
.
|
@@ -75,7 +72,7 @@ A simple RComp suite might look something like this:
|
|
75
72
|
|
76
73
|
## Configuration
|
77
74
|
|
78
|
-
All custom configuration is stored in
|
75
|
+
All custom configuration is stored in `.rcomp` as YAML
|
79
76
|
|
80
77
|
<table>
|
81
78
|
<th>Setting</th><th>Config</th><th>Default</th><th>Description</th>
|
@@ -91,6 +88,23 @@ All custom configuration is stored in a `.rcomp` file as YAML
|
|
91
88
|
<td><code>rcomp</code></td>
|
92
89
|
<td>Directory RComp will store tests, results and expected in</td>
|
93
90
|
</tr>
|
91
|
+
<tr>
|
92
|
+
<td>ignore</td>
|
93
|
+
<td>
|
94
|
+
<code>
|
95
|
+
ignore:
|
96
|
+
- [PATTERN]
|
97
|
+
</code>
|
98
|
+
</td>
|
99
|
+
<td></td>
|
100
|
+
<td>List of patterns RComp will ignore when finding tests</td>
|
101
|
+
</tr>
|
102
|
+
<tr>
|
103
|
+
<td>timeout</td>
|
104
|
+
<td><code>timeout: [TIMEOUT]</code></td>
|
105
|
+
<td><code>5</code></td>
|
106
|
+
<td>Test execution time limit (seconds)</td>
|
107
|
+
</tr>
|
94
108
|
</table>
|
95
109
|
|
96
110
|
## Aliases
|
@@ -105,14 +119,6 @@ All custom configuration is stored in a `.rcomp` file as YAML
|
|
105
119
|
<td><code>generate</code></td>
|
106
120
|
<td><code>g</code></td>
|
107
121
|
</tr>
|
108
|
-
<tr>
|
109
|
-
<td><code>set-command</code></td>
|
110
|
-
<td><code>c</code></td>
|
111
|
-
</tr>
|
112
|
-
<tr>
|
113
|
-
<td><code>set-directory</code></td>
|
114
|
-
<td><code>d</code></td>
|
115
|
-
</tr>
|
116
122
|
<tr>
|
117
123
|
<td><code>version</code></td>
|
118
124
|
<td><code>-v</code><code>--version</code></td>
|
data/Rakefile
CHANGED
data/features/generate.feature
CHANGED
@@ -8,13 +8,21 @@ Feature: Generate
|
|
8
8
|
And the exit status should be 1
|
9
9
|
|
10
10
|
Scenario: Generate without init
|
11
|
-
Given
|
11
|
+
Given a file named ".rcomp" with:
|
12
|
+
"""
|
13
|
+
command: ./test_exec
|
14
|
+
|
15
|
+
"""
|
12
16
|
When I run `rcomp generate`
|
13
17
|
Then the output should contain "No RComp directory"
|
14
18
|
And the exit status should be 1
|
15
19
|
|
16
20
|
Scenario: Generate with partial init
|
17
|
-
Given
|
21
|
+
Given a file named ".rcomp" with:
|
22
|
+
"""
|
23
|
+
command: ./test_exec
|
24
|
+
|
25
|
+
"""
|
18
26
|
And a directory named "rcomp"
|
19
27
|
And a directory named "rcomp/tests"
|
20
28
|
When I run `rcomp generate`
|
@@ -211,6 +219,60 @@ Feature: Generate
|
|
211
219
|
Then the output should contain "2 files (2 generated)"
|
212
220
|
And the exit status should be 0
|
213
221
|
|
222
|
+
# ignore from config
|
223
|
+
@basic-conf
|
224
|
+
@load-assorted-tests
|
225
|
+
Scenario: Test with config ignore filter set: catches all
|
226
|
+
Given I append to ".rcomp" with:
|
227
|
+
"""
|
228
|
+
ignore:
|
229
|
+
- xyz
|
230
|
+
|
231
|
+
"""
|
232
|
+
When I run `rcomp g`
|
233
|
+
Then the output should contain "3 files (2 skipped, 1 generated)"
|
234
|
+
And the exit status should be 0
|
235
|
+
|
236
|
+
@basic-conf
|
237
|
+
@load-assorted-tests
|
238
|
+
Scenario: Test with config ignore filter set: catches some
|
239
|
+
Given I append to ".rcomp" with:
|
240
|
+
"""
|
241
|
+
ignore:
|
242
|
+
- foo
|
243
|
+
|
244
|
+
"""
|
245
|
+
When I run `rcomp g`
|
246
|
+
Then the output should contain "2 files (2 skipped)"
|
247
|
+
And the exit status should be 0
|
248
|
+
|
249
|
+
@basic-conf
|
250
|
+
@load-assorted-tests
|
251
|
+
Scenario: Test with config ignore filter set: catches none
|
252
|
+
Given I append to ".rcomp" with:
|
253
|
+
"""
|
254
|
+
ignore:
|
255
|
+
- test
|
256
|
+
|
257
|
+
"""
|
258
|
+
When I run `rcomp g`
|
259
|
+
Then the output should contain "0 files ()"
|
260
|
+
And the exit status should be 0
|
261
|
+
|
262
|
+
@basic-conf
|
263
|
+
@load-assorted-tests
|
264
|
+
Scenario: Test with multiple config ignore filters set
|
265
|
+
Given I append to ".rcomp" with:
|
266
|
+
"""
|
267
|
+
ignore:
|
268
|
+
- test_a
|
269
|
+
- test_b
|
270
|
+
|
271
|
+
"""
|
272
|
+
When I run `rcomp g`
|
273
|
+
Then the output should contain "1 file (1 generated)"
|
274
|
+
And the exit status should be 0
|
275
|
+
|
214
276
|
# custom generate directory
|
215
277
|
@custom-conf
|
216
278
|
Scenario: Custom conf generate
|
@@ -251,3 +313,33 @@ Feature: Generate
|
|
251
313
|
When I run `rcomp generate`
|
252
314
|
Then the output should contain "3 files (2 skipped, 1 generated)"
|
253
315
|
And the exit status should be 0
|
316
|
+
|
317
|
+
# timeout
|
318
|
+
@loop-conf
|
319
|
+
Scenario: Test hanging test with default timeout
|
320
|
+
Given a file named "rcomp/tests/test1.test" with:
|
321
|
+
"""
|
322
|
+
ABC
|
323
|
+
|
324
|
+
"""
|
325
|
+
When I run `rcomp generate` for up to 6 seconds
|
326
|
+
Then the output should contain "1 file (1 failed)"
|
327
|
+
And the output should contain "timeout : /test1"
|
328
|
+
And the exit status should be 1
|
329
|
+
|
330
|
+
@loop-conf
|
331
|
+
Scenario: Test hanging test with custom timeout
|
332
|
+
Given I append to ".rcomp" with:
|
333
|
+
"""
|
334
|
+
timeout: 1
|
335
|
+
|
336
|
+
"""
|
337
|
+
And a file named "rcomp/tests/test1.test" with:
|
338
|
+
"""
|
339
|
+
ABC
|
340
|
+
|
341
|
+
"""
|
342
|
+
When I run `rcomp generate` for up to 2 seconds
|
343
|
+
Then the output should contain "1 file (1 failed)"
|
344
|
+
And the output should contain "timeout : /test1"
|
345
|
+
And the exit status should be 1
|
data/features/init.feature
CHANGED
@@ -2,43 +2,67 @@ Feature: Init
|
|
2
2
|
A user should be able to initialize the RComp's test directories
|
3
3
|
|
4
4
|
Scenario: Blind init
|
5
|
-
When I run `rcomp init`
|
5
|
+
When I run `rcomp init` interactively
|
6
|
+
And I type "./something"
|
6
7
|
Then the following directories should exist:
|
7
8
|
| rcomp |
|
8
9
|
| rcomp/tests |
|
9
10
|
| rcomp/results |
|
10
11
|
| rcomp/expected |
|
12
|
+
And the file ".rcomp" should contain "command: ./something"
|
11
13
|
And the exit status should be 0
|
12
14
|
|
13
15
|
Scenario: Init with directory set in project root
|
14
|
-
Given
|
15
|
-
|
16
|
+
Given a file named ".rcomp" with:
|
17
|
+
"""
|
18
|
+
directory: dir
|
19
|
+
|
20
|
+
"""
|
21
|
+
When I run `rcomp init` interactively
|
22
|
+
And I type "./something"
|
16
23
|
Then the following directories should exist:
|
17
24
|
| dir |
|
18
25
|
| dir/tests |
|
19
26
|
| dir/results |
|
20
27
|
| dir/expected |
|
28
|
+
And the file ".rcomp" should contain "command: ./something"
|
21
29
|
And the exit status should be 0
|
22
30
|
|
23
|
-
Scenario: Init with directory set in subdirectory
|
31
|
+
Scenario: Init with directory set in subdirectory that exists
|
24
32
|
Given a directory named "spec/dir"
|
25
|
-
And
|
26
|
-
|
33
|
+
And a file named ".rcomp" with:
|
34
|
+
"""
|
35
|
+
directory: spec/dir/rcomp
|
36
|
+
|
37
|
+
"""
|
38
|
+
When I run `rcomp init` interactively
|
39
|
+
And I type "./something"
|
27
40
|
Then the following directories should exist:
|
28
41
|
| spec/dir/rcomp |
|
29
42
|
| spec/dir/rcomp/tests |
|
30
43
|
| spec/dir/rcomp/results |
|
31
44
|
| spec/dir/rcomp/expected |
|
45
|
+
And the file ".rcomp" should contain "command: ./something"
|
32
46
|
And the exit status should be 0
|
47
|
+
|
48
|
+
Scenario: Init with directory set in subdirectory that doesn't exist
|
49
|
+
Given a file named ".rcomp" with:
|
50
|
+
"""
|
51
|
+
directory: spec/dir/rcomp
|
33
52
|
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
Then the
|
38
|
-
|
53
|
+
"""
|
54
|
+
When I run `rcomp init` interactively
|
55
|
+
And I type "./something"
|
56
|
+
Then the following directories should exist:
|
57
|
+
| spec/dir/rcomp |
|
58
|
+
| spec/dir/rcomp/tests |
|
59
|
+
| spec/dir/rcomp/results |
|
60
|
+
| spec/dir/rcomp/expected |
|
61
|
+
And the file ".rcomp" should contain "command: ./something"
|
62
|
+
And the exit status should be 0
|
39
63
|
|
64
|
+
@basic-conf
|
40
65
|
Scenario: Already initialized
|
41
|
-
Given I run `rcomp init`
|
42
66
|
When I run `rcomp init`
|
43
67
|
Then the output should contain "already initialized"
|
44
68
|
And the exit status should be 1
|
data/features/support/env.rb
CHANGED
@@ -4,20 +4,23 @@ require 'aruba/cucumber'
|
|
4
4
|
require 'test/unit/assertions'
|
5
5
|
|
6
6
|
# Helpers
|
7
|
-
def create_executable
|
8
|
-
content = "#!/usr/bin/env ruby\nputs IO.binread(ARGV[0])"
|
7
|
+
def create_executable(content)
|
9
8
|
write_file("test_exec", content)
|
10
9
|
in_current_dir do
|
11
10
|
FileUtils.chmod(0755, "test_exec")
|
12
11
|
end
|
13
12
|
end
|
14
13
|
|
15
|
-
def
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
14
|
+
def basic_executable
|
15
|
+
"#!/usr/bin/env ruby\nputs IO.binread(ARGV[0])"
|
16
|
+
end
|
17
|
+
|
18
|
+
def err_executable
|
19
|
+
"#!/usr/bin/env ruby\n$stderr.puts IO.binread(ARGV[0])"
|
20
|
+
end
|
21
|
+
|
22
|
+
def loop_executable
|
23
|
+
"!#/usr/bin/env ruby\nsleep 30"
|
21
24
|
end
|
22
25
|
|
23
26
|
def create_assorted_tests
|
@@ -28,27 +31,31 @@ end
|
|
28
31
|
|
29
32
|
# Custom Tags
|
30
33
|
Before('@basic-conf') do
|
31
|
-
create_executable
|
34
|
+
create_executable(basic_executable)
|
32
35
|
# Spin up basic RComp configuration
|
33
|
-
|
34
|
-
run_simple('rcomp init'
|
36
|
+
write_file(".rcomp", "command: ./test_exec\n")
|
37
|
+
run_simple('rcomp init')
|
35
38
|
end
|
36
39
|
|
37
40
|
Before('@err-conf') do
|
38
|
-
|
41
|
+
create_executable(err_executable)
|
39
42
|
# Spin up RComp configuraton with an erroring executable
|
40
|
-
|
41
|
-
run_simple('rcomp init'
|
43
|
+
write_file(".rcomp", "command: ./test_exec\n")
|
44
|
+
run_simple('rcomp init')
|
42
45
|
end
|
43
46
|
|
44
47
|
Before('@custom-conf') do
|
45
|
-
create_executable
|
48
|
+
create_executable(basic_executable)
|
46
49
|
# Spin up custom path RComp configuration
|
47
|
-
|
48
|
-
|
49
|
-
run_simple('rcomp
|
50
|
-
|
51
|
-
|
50
|
+
write_file(".rcomp",
|
51
|
+
"command: ./test_exec\ndirectory: test/integration/rcomp\n")
|
52
|
+
run_simple('rcomp init')
|
53
|
+
end
|
54
|
+
|
55
|
+
Before('@loop-conf') do
|
56
|
+
create_executable(loop_executable)
|
57
|
+
write_file(".rcomp", "command: ./test_exec\n")
|
58
|
+
run_simple('rcomp init')
|
52
59
|
end
|
53
60
|
|
54
61
|
Before('@load-assorted-tests') do
|
data/features/test.feature
CHANGED
@@ -8,13 +8,21 @@ Feature: Test
|
|
8
8
|
And the exit status should be 1
|
9
9
|
|
10
10
|
Scenario: Test without init
|
11
|
-
Given
|
11
|
+
Given a file named ".rcomp" with:
|
12
|
+
"""
|
13
|
+
command: ./test_exec
|
14
|
+
|
15
|
+
"""
|
12
16
|
When I run `rcomp test`
|
13
17
|
Then the output should contain "No RComp directory"
|
14
18
|
And the exit status should be 1
|
15
19
|
|
16
20
|
Scenario: Test with partial init
|
17
|
-
Given
|
21
|
+
Given a file named ".rcomp" with:
|
22
|
+
"""
|
23
|
+
command: ./test_exec
|
24
|
+
|
25
|
+
"""
|
18
26
|
And a directory named "rcomp"
|
19
27
|
And a directory named "rcomp/tests"
|
20
28
|
When I run `rcomp test`
|
@@ -151,6 +159,60 @@ Feature: Test
|
|
151
159
|
Then the output should contain "0 tests ()"
|
152
160
|
And the exit status should be 0
|
153
161
|
|
162
|
+
# ignore from config
|
163
|
+
@basic-conf
|
164
|
+
@load-assorted-tests
|
165
|
+
Scenario: Test with config ignore filter set: catches all
|
166
|
+
Given I append to ".rcomp" with:
|
167
|
+
"""
|
168
|
+
ignore:
|
169
|
+
- xyz
|
170
|
+
|
171
|
+
"""
|
172
|
+
When I run `rcomp t`
|
173
|
+
Then the output should contain "3 tests (1 failed, 1 skipped, 1 passed)"
|
174
|
+
And the exit status should be 1
|
175
|
+
|
176
|
+
@basic-conf
|
177
|
+
@load-assorted-tests
|
178
|
+
Scenario: Test with config ignore filter set: catches some
|
179
|
+
Given I append to ".rcomp" with:
|
180
|
+
"""
|
181
|
+
ignore:
|
182
|
+
- foo
|
183
|
+
|
184
|
+
"""
|
185
|
+
When I run `rcomp t`
|
186
|
+
Then the output should contain "2 tests (1 failed, 1 passed)"
|
187
|
+
And the exit status should be 1
|
188
|
+
|
189
|
+
@basic-conf
|
190
|
+
@load-assorted-tests
|
191
|
+
Scenario: Test with config ignore filter set: catches none
|
192
|
+
Given I append to ".rcomp" with:
|
193
|
+
"""
|
194
|
+
ignore:
|
195
|
+
- test
|
196
|
+
|
197
|
+
"""
|
198
|
+
When I run `rcomp t`
|
199
|
+
Then the output should contain "0 tests ()"
|
200
|
+
And the exit status should be 0
|
201
|
+
|
202
|
+
@basic-conf
|
203
|
+
@load-assorted-tests
|
204
|
+
Scenario: Test with multiple config ignore filters set
|
205
|
+
Given I append to ".rcomp" with:
|
206
|
+
"""
|
207
|
+
ignore:
|
208
|
+
- test_a
|
209
|
+
- test_b
|
210
|
+
|
211
|
+
"""
|
212
|
+
When I run `rcomp t`
|
213
|
+
Then the output should contain "1 test (1 skipped)"
|
214
|
+
And the exit status should be 0
|
215
|
+
|
154
216
|
# custom tests directory
|
155
217
|
@custom-conf
|
156
218
|
Scenario: Custom conf test
|
@@ -196,3 +258,43 @@ Feature: Test
|
|
196
258
|
When I run `rcomp test`
|
197
259
|
Then the output should contain "3 tests (1 failed, 1 skipped, 1 passed)"
|
198
260
|
And the exit status should be 1
|
261
|
+
|
262
|
+
# timeout
|
263
|
+
@loop-conf
|
264
|
+
Scenario: Test hanging test with default timeout
|
265
|
+
Given a file named "rcomp/tests/test1.test" with:
|
266
|
+
"""
|
267
|
+
ABC
|
268
|
+
|
269
|
+
"""
|
270
|
+
And a file named "rcomp/expected/test1.out" with:
|
271
|
+
"""
|
272
|
+
ABC
|
273
|
+
|
274
|
+
"""
|
275
|
+
When I run `rcomp test` for up to 6 seconds
|
276
|
+
Then the output should contain "1 test (1 failed)"
|
277
|
+
And the output should contain "timeout : /test1"
|
278
|
+
And the exit status should be 1
|
279
|
+
|
280
|
+
@loop-conf
|
281
|
+
Scenario: Test hanging test with custom timeout
|
282
|
+
Given I append to ".rcomp" with:
|
283
|
+
"""
|
284
|
+
timeout: 1
|
285
|
+
|
286
|
+
"""
|
287
|
+
And a file named "rcomp/tests/test1.test" with:
|
288
|
+
"""
|
289
|
+
ABC
|
290
|
+
|
291
|
+
"""
|
292
|
+
And a file named "rcomp/expected/test1.out" with:
|
293
|
+
"""
|
294
|
+
ABC
|
295
|
+
|
296
|
+
"""
|
297
|
+
When I run `rcomp test` for up to 2 seconds
|
298
|
+
Then the output should contain "1 test (1 failed)"
|
299
|
+
And the output should contain "timeout : /test1"
|
300
|
+
And the exit status should be 1
|