rcomp 0.1.1 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.gitignore CHANGED
@@ -16,9 +16,3 @@ tmp
16
16
  .yardoc
17
17
  _yardoc
18
18
  doc/
19
-
20
- # RComp files
21
- .rcomp
22
-
23
- # simplecov
24
- coverage
data/CHANGELOG.rdoc CHANGED
@@ -1,3 +1,11 @@
1
+ == 0.2.0, released 2012-11-25
2
+
3
+ * Add "ignore" configuration option
4
+ * Add "timeout" configuration option
5
+ * Remove "set-command"
6
+ * Remove "set-directory"
7
+ * Upgrade "init" to handle command setting when neccessary
8
+
1
9
  == 0.1.1, released 2012-11-20
2
10
 
3
11
  * Ignore dotfiles in tests directory
data/Gemfile.lock CHANGED
@@ -1,7 +1,8 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- rcomp (0.1.0)
4
+ rcomp (0.2.0)
5
+ childprocess (~> 0.3.6)
5
6
  thor (~> 0.16.0)
6
7
 
7
8
  GEM
data/README.md CHANGED
@@ -14,13 +14,11 @@ $ gem install rcomp
14
14
  ```
15
15
  $ rcomp
16
16
  Tasks:
17
- rcomp generate # Generate expected output for all tests
18
- rcomp help [TASK] # Describe available tasks or one specific task
19
- rcomp init # Setup rcomp test directory
20
- rcomp set_command COMMAND # Sets the command RComp will run tests with
21
- rcomp set_directory PATH # Set the directory RComp will store files
22
- rcomp test # Run all tests
23
- rcomp version # Prints RComp's version information
17
+ rcomp generate # Generate expected output for all tests
18
+ rcomp help [TASK] # Describe available tasks or one specific task
19
+ rcomp init # Setup rcomp test directory
20
+ rcomp test # Run all tests
21
+ rcomp version # Prints RComp's version information
24
22
  ```
25
23
 
26
24
  ## Setup
@@ -28,7 +26,6 @@ Tasks:
28
26
  In your project root directory, run:
29
27
 
30
28
  ```
31
- $ rcomp set-command ./some-executable
32
29
  $ rcomp init
33
30
  ```
34
31
 
@@ -57,7 +54,7 @@ Managed by RComp. Stores the results of your most recent test suite run.
57
54
 
58
55
  ---
59
56
 
60
- A simple RComp suite might look something like this:
57
+ A simple RComp suite might look something like:
61
58
 
62
59
  ```
63
60
  .
@@ -75,7 +72,7 @@ A simple RComp suite might look something like this:
75
72
 
76
73
  ## Configuration
77
74
 
78
- All custom configuration is stored in a `.rcomp` file as YAML
75
+ All custom configuration is stored in `.rcomp` as YAML
79
76
 
80
77
  <table>
81
78
  <th>Setting</th><th>Config</th><th>Default</th><th>Description</th>
@@ -91,6 +88,23 @@ All custom configuration is stored in a `.rcomp` file as YAML
91
88
  <td><code>rcomp</code></td>
92
89
  <td>Directory RComp will store tests, results and expected in</td>
93
90
  </tr>
91
+ <tr>
92
+ <td>ignore</td>
93
+ <td>
94
+ <code>
95
+ ignore:
96
+ - [PATTERN]
97
+ </code>
98
+ </td>
99
+ <td></td>
100
+ <td>List of patterns RComp will ignore when finding tests</td>
101
+ </tr>
102
+ <tr>
103
+ <td>timeout</td>
104
+ <td><code>timeout: [TIMEOUT]</code></td>
105
+ <td><code>5</code></td>
106
+ <td>Test execution time limit (seconds)</td>
107
+ </tr>
94
108
  </table>
95
109
 
96
110
  ## Aliases
@@ -105,14 +119,6 @@ All custom configuration is stored in a `.rcomp` file as YAML
105
119
  <td><code>generate</code></td>
106
120
  <td><code>g</code></td>
107
121
  </tr>
108
- <tr>
109
- <td><code>set-command</code></td>
110
- <td><code>c</code></td>
111
- </tr>
112
- <tr>
113
- <td><code>set-directory</code></td>
114
- <td><code>d</code></td>
115
- </tr>
116
122
  <tr>
117
123
  <td><code>version</code></td>
118
124
  <td><code>-v</code><code>--version</code></td>
data/Rakefile CHANGED
@@ -4,4 +4,9 @@ require 'bundler/gem_tasks'
4
4
  desc "Run features"
5
5
  Cucumber::Rake::Task.new(:features)
6
6
 
7
+ desc "Clean"
8
+ task :clean do
9
+ rm_rf "pkg"
10
+ end
11
+
7
12
  task :default => :features
@@ -8,13 +8,21 @@ Feature: Generate
8
8
  And the exit status should be 1
9
9
 
10
10
  Scenario: Generate without init
11
- Given I run `rcomp c ./exec`
11
+ Given a file named ".rcomp" with:
12
+ """
13
+ command: ./test_exec
14
+
15
+ """
12
16
  When I run `rcomp generate`
13
17
  Then the output should contain "No RComp directory"
14
18
  And the exit status should be 1
15
19
 
16
20
  Scenario: Generate with partial init
17
- Given I run `rcomp c ./exec`
21
+ Given a file named ".rcomp" with:
22
+ """
23
+ command: ./test_exec
24
+
25
+ """
18
26
  And a directory named "rcomp"
19
27
  And a directory named "rcomp/tests"
20
28
  When I run `rcomp generate`
@@ -211,6 +219,60 @@ Feature: Generate
211
219
  Then the output should contain "2 files (2 generated)"
212
220
  And the exit status should be 0
213
221
 
222
+ # ignore from config
223
+ @basic-conf
224
+ @load-assorted-tests
225
+ Scenario: Test with config ignore filter set: catches all
226
+ Given I append to ".rcomp" with:
227
+ """
228
+ ignore:
229
+ - xyz
230
+
231
+ """
232
+ When I run `rcomp g`
233
+ Then the output should contain "3 files (2 skipped, 1 generated)"
234
+ And the exit status should be 0
235
+
236
+ @basic-conf
237
+ @load-assorted-tests
238
+ Scenario: Test with config ignore filter set: catches some
239
+ Given I append to ".rcomp" with:
240
+ """
241
+ ignore:
242
+ - foo
243
+
244
+ """
245
+ When I run `rcomp g`
246
+ Then the output should contain "2 files (2 skipped)"
247
+ And the exit status should be 0
248
+
249
+ @basic-conf
250
+ @load-assorted-tests
251
+ Scenario: Test with config ignore filter set: catches none
252
+ Given I append to ".rcomp" with:
253
+ """
254
+ ignore:
255
+ - test
256
+
257
+ """
258
+ When I run `rcomp g`
259
+ Then the output should contain "0 files ()"
260
+ And the exit status should be 0
261
+
262
+ @basic-conf
263
+ @load-assorted-tests
264
+ Scenario: Test with multiple config ignore filters set
265
+ Given I append to ".rcomp" with:
266
+ """
267
+ ignore:
268
+ - test_a
269
+ - test_b
270
+
271
+ """
272
+ When I run `rcomp g`
273
+ Then the output should contain "1 file (1 generated)"
274
+ And the exit status should be 0
275
+
214
276
  # custom generate directory
215
277
  @custom-conf
216
278
  Scenario: Custom conf generate
@@ -251,3 +313,33 @@ Feature: Generate
251
313
  When I run `rcomp generate`
252
314
  Then the output should contain "3 files (2 skipped, 1 generated)"
253
315
  And the exit status should be 0
316
+
317
+ # timeout
318
+ @loop-conf
319
+ Scenario: Test hanging test with default timeout
320
+ Given a file named "rcomp/tests/test1.test" with:
321
+ """
322
+ ABC
323
+
324
+ """
325
+ When I run `rcomp generate` for up to 6 seconds
326
+ Then the output should contain "1 file (1 failed)"
327
+ And the output should contain "timeout : /test1"
328
+ And the exit status should be 1
329
+
330
+ @loop-conf
331
+ Scenario: Test hanging test with custom timeout
332
+ Given I append to ".rcomp" with:
333
+ """
334
+ timeout: 1
335
+
336
+ """
337
+ And a file named "rcomp/tests/test1.test" with:
338
+ """
339
+ ABC
340
+
341
+ """
342
+ When I run `rcomp generate` for up to 2 seconds
343
+ Then the output should contain "1 file (1 failed)"
344
+ And the output should contain "timeout : /test1"
345
+ And the exit status should be 1
@@ -2,43 +2,67 @@ Feature: Init
2
2
  A user should be able to initialize the RComp's test directories
3
3
 
4
4
  Scenario: Blind init
5
- When I run `rcomp init`
5
+ When I run `rcomp init` interactively
6
+ And I type "./something"
6
7
  Then the following directories should exist:
7
8
  | rcomp |
8
9
  | rcomp/tests |
9
10
  | rcomp/results |
10
11
  | rcomp/expected |
12
+ And the file ".rcomp" should contain "command: ./something"
11
13
  And the exit status should be 0
12
14
 
13
15
  Scenario: Init with directory set in project root
14
- Given I run `rcomp d dir`
15
- When I run `rcomp init`
16
+ Given a file named ".rcomp" with:
17
+ """
18
+ directory: dir
19
+
20
+ """
21
+ When I run `rcomp init` interactively
22
+ And I type "./something"
16
23
  Then the following directories should exist:
17
24
  | dir |
18
25
  | dir/tests |
19
26
  | dir/results |
20
27
  | dir/expected |
28
+ And the file ".rcomp" should contain "command: ./something"
21
29
  And the exit status should be 0
22
30
 
23
- Scenario: Init with directory set in subdirectory
31
+ Scenario: Init with directory set in subdirectory that exists
24
32
  Given a directory named "spec/dir"
25
- And I run `rcomp d spec/dir/rcomp`
26
- When I run `rcomp init`
33
+ And a file named ".rcomp" with:
34
+ """
35
+ directory: spec/dir/rcomp
36
+
37
+ """
38
+ When I run `rcomp init` interactively
39
+ And I type "./something"
27
40
  Then the following directories should exist:
28
41
  | spec/dir/rcomp |
29
42
  | spec/dir/rcomp/tests |
30
43
  | spec/dir/rcomp/results |
31
44
  | spec/dir/rcomp/expected |
45
+ And the file ".rcomp" should contain "command: ./something"
32
46
  And the exit status should be 0
47
+
48
+ Scenario: Init with directory set in subdirectory that doesn't exist
49
+ Given a file named ".rcomp" with:
50
+ """
51
+ directory: spec/dir/rcomp
33
52
 
34
- Scenario: Init with directory set in nonexistant subdirectory
35
- Given I run `rcomp d nonexistant/rcomp`
36
- When I run `rcomp init`
37
- Then the output should contain "No directory nonexistant"
38
- And the exit status should be 1
53
+ """
54
+ When I run `rcomp init` interactively
55
+ And I type "./something"
56
+ Then the following directories should exist:
57
+ | spec/dir/rcomp |
58
+ | spec/dir/rcomp/tests |
59
+ | spec/dir/rcomp/results |
60
+ | spec/dir/rcomp/expected |
61
+ And the file ".rcomp" should contain "command: ./something"
62
+ And the exit status should be 0
39
63
 
64
+ @basic-conf
40
65
  Scenario: Already initialized
41
- Given I run `rcomp init`
42
66
  When I run `rcomp init`
43
67
  Then the output should contain "already initialized"
44
68
  And the exit status should be 1
@@ -0,0 +1,5 @@
1
+ require 'aruba/api'
2
+
3
+ When /^I run `(.*?)` for up to (\d+) seconds$/ do |cmd, secs|
4
+ run_simple(unescape(cmd), false, secs && secs.to_i)
5
+ end
@@ -0,0 +1,7 @@
1
+ require 'aruba/api'
2
+ require 'rcomp/version'
3
+
4
+ Then /^the output should contain the version number$/ do
5
+ expected = "RComp version #{RComp::VERSION}\n"
6
+ assert_exact_output(expected, all_output)
7
+ end
@@ -4,20 +4,23 @@ require 'aruba/cucumber'
4
4
  require 'test/unit/assertions'
5
5
 
6
6
  # Helpers
7
- def create_executable
8
- content = "#!/usr/bin/env ruby\nputs IO.binread(ARGV[0])"
7
+ def create_executable(content)
9
8
  write_file("test_exec", content)
10
9
  in_current_dir do
11
10
  FileUtils.chmod(0755, "test_exec")
12
11
  end
13
12
  end
14
13
 
15
- def create_err_executable
16
- content = "#!/usr/bin/env ruby\n$stderr.puts IO.binread(ARGV[0])"
17
- write_file("test_exec", content)
18
- in_current_dir do
19
- FileUtils.chmod(0755, "test_exec")
20
- end
14
+ def basic_executable
15
+ "#!/usr/bin/env ruby\nputs IO.binread(ARGV[0])"
16
+ end
17
+
18
+ def err_executable
19
+ "#!/usr/bin/env ruby\n$stderr.puts IO.binread(ARGV[0])"
20
+ end
21
+
22
+ def loop_executable
23
+ "!#/usr/bin/env ruby\nsleep 30"
21
24
  end
22
25
 
23
26
  def create_assorted_tests
@@ -28,27 +31,31 @@ end
28
31
 
29
32
  # Custom Tags
30
33
  Before('@basic-conf') do
31
- create_executable
34
+ create_executable(basic_executable)
32
35
  # Spin up basic RComp configuration
33
- run_simple('rcomp c ./test_exec', false)
34
- run_simple('rcomp init', false)
36
+ write_file(".rcomp", "command: ./test_exec\n")
37
+ run_simple('rcomp init')
35
38
  end
36
39
 
37
40
  Before('@err-conf') do
38
- create_err_executable
41
+ create_executable(err_executable)
39
42
  # Spin up RComp configuraton with an erroring executable
40
- run_simple('rcomp c ./test_exec', false)
41
- run_simple('rcomp init', false)
43
+ write_file(".rcomp", "command: ./test_exec\n")
44
+ run_simple('rcomp init')
42
45
  end
43
46
 
44
47
  Before('@custom-conf') do
45
- create_executable
48
+ create_executable(basic_executable)
46
49
  # Spin up custom path RComp configuration
47
- run_simple('mkdir test', false)
48
- run_simple('mkdir test/integration', false)
49
- run_simple('rcomp d test/integration/rcomp')
50
- run_simple('rcomp c ./test_exec', false)
51
- run_simple('rcomp init', false)
50
+ write_file(".rcomp",
51
+ "command: ./test_exec\ndirectory: test/integration/rcomp\n")
52
+ run_simple('rcomp init')
53
+ end
54
+
55
+ Before('@loop-conf') do
56
+ create_executable(loop_executable)
57
+ write_file(".rcomp", "command: ./test_exec\n")
58
+ run_simple('rcomp init')
52
59
  end
53
60
 
54
61
  Before('@load-assorted-tests') do
@@ -8,13 +8,21 @@ Feature: Test
8
8
  And the exit status should be 1
9
9
 
10
10
  Scenario: Test without init
11
- Given I run `rcomp c ./exec`
11
+ Given a file named ".rcomp" with:
12
+ """
13
+ command: ./test_exec
14
+
15
+ """
12
16
  When I run `rcomp test`
13
17
  Then the output should contain "No RComp directory"
14
18
  And the exit status should be 1
15
19
 
16
20
  Scenario: Test with partial init
17
- Given I run `rcomp c ./exec`
21
+ Given a file named ".rcomp" with:
22
+ """
23
+ command: ./test_exec
24
+
25
+ """
18
26
  And a directory named "rcomp"
19
27
  And a directory named "rcomp/tests"
20
28
  When I run `rcomp test`
@@ -151,6 +159,60 @@ Feature: Test
151
159
  Then the output should contain "0 tests ()"
152
160
  And the exit status should be 0
153
161
 
162
+ # ignore from config
163
+ @basic-conf
164
+ @load-assorted-tests
165
+ Scenario: Test with config ignore filter set: catches all
166
+ Given I append to ".rcomp" with:
167
+ """
168
+ ignore:
169
+ - xyz
170
+
171
+ """
172
+ When I run `rcomp t`
173
+ Then the output should contain "3 tests (1 failed, 1 skipped, 1 passed)"
174
+ And the exit status should be 1
175
+
176
+ @basic-conf
177
+ @load-assorted-tests
178
+ Scenario: Test with config ignore filter set: catches some
179
+ Given I append to ".rcomp" with:
180
+ """
181
+ ignore:
182
+ - foo
183
+
184
+ """
185
+ When I run `rcomp t`
186
+ Then the output should contain "2 tests (1 failed, 1 passed)"
187
+ And the exit status should be 1
188
+
189
+ @basic-conf
190
+ @load-assorted-tests
191
+ Scenario: Test with config ignore filter set: catches none
192
+ Given I append to ".rcomp" with:
193
+ """
194
+ ignore:
195
+ - test
196
+
197
+ """
198
+ When I run `rcomp t`
199
+ Then the output should contain "0 tests ()"
200
+ And the exit status should be 0
201
+
202
+ @basic-conf
203
+ @load-assorted-tests
204
+ Scenario: Test with multiple config ignore filters set
205
+ Given I append to ".rcomp" with:
206
+ """
207
+ ignore:
208
+ - test_a
209
+ - test_b
210
+
211
+ """
212
+ When I run `rcomp t`
213
+ Then the output should contain "1 test (1 skipped)"
214
+ And the exit status should be 0
215
+
154
216
  # custom tests directory
155
217
  @custom-conf
156
218
  Scenario: Custom conf test
@@ -196,3 +258,43 @@ Feature: Test
196
258
  When I run `rcomp test`
197
259
  Then the output should contain "3 tests (1 failed, 1 skipped, 1 passed)"
198
260
  And the exit status should be 1
261
+
262
+ # timeout
263
+ @loop-conf
264
+ Scenario: Test hanging test with default timeout
265
+ Given a file named "rcomp/tests/test1.test" with:
266
+ """
267
+ ABC
268
+
269
+ """
270
+ And a file named "rcomp/expected/test1.out" with:
271
+ """
272
+ ABC
273
+
274
+ """
275
+ When I run `rcomp test` for up to 6 seconds
276
+ Then the output should contain "1 test (1 failed)"
277
+ And the output should contain "timeout : /test1"
278
+ And the exit status should be 1
279
+
280
+ @loop-conf
281
+ Scenario: Test hanging test with custom timeout
282
+ Given I append to ".rcomp" with:
283
+ """
284
+ timeout: 1
285
+
286
+ """
287
+ And a file named "rcomp/tests/test1.test" with:
288
+ """
289
+ ABC
290
+
291
+ """
292
+ And a file named "rcomp/expected/test1.out" with:
293
+ """
294
+ ABC
295
+
296
+ """
297
+ When I run `rcomp test` for up to 2 seconds
298
+ Then the output should contain "1 test (1 failed)"
299
+ And the output should contain "timeout : /test1"
300
+ And the exit status should be 1