specbandit 1.3.1 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +26 -1
- data/lib/specbandit/cli.rb +9 -1
- data/lib/specbandit/configuration.rb +16 -1
- data/lib/specbandit/version.rb +1 -1
- data/lib/specbandit/worker.rb +25 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d2aab71e7d9a6d7f5e3be3ae3aa4c0bb1c44933ec7b6d5fd0b3c4d00530ed6f3
|
|
4
|
+
data.tar.gz: 12e1f2b14710316d042acf34e1a1fabb888c9817ec82270823cf6229f76702de
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 76484e1fc1740b9b416fc69348156b80c536b218ec3a762e1d3366e57f2efe4946ad708e00258017fc0342b1e1155985d25aed486fb2e6fb7c024276c85c4f92
|
|
7
|
+
data.tar.gz: 27dcfd5025a9e32d4bbb79e73cccb25e862ca455758746e02fa9347677974e7abdc0fea1baf237e63d36e477bed0ff571dacdd3d95d4abf5c1c86ef6f904aa78
|
data/README.md
CHANGED
|
@@ -135,7 +135,7 @@ Each worker loops:
|
|
|
135
135
|
3. Repeat until the queue is empty
|
|
136
136
|
4. Exit 0 if all batches passed, 1 if any failed
|
|
137
137
|
|
|
138
|
-
|
|
138
|
+
By default, a failing batch does **not** stop the worker. It continues stealing remaining work so other runners aren't blocked waiting on files that will never be consumed. To stop the whole run at the first failure, use [`--fail-fast`](#fail-fast---fail-fast).
|
|
139
139
|
|
|
140
140
|
### CLI reference
|
|
141
141
|
|
|
@@ -163,6 +163,7 @@ specbandit work [options] [-- extra-opts...]
|
|
|
163
163
|
--key-failed KEY Redis key to record failed test files
|
|
164
164
|
--key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
|
|
165
165
|
--verbose Show per-batch file list and full command output
|
|
166
|
+
--fail-fast Stop at the first failed batch and empty the shared queue (see below)
|
|
166
167
|
|
|
167
168
|
Arguments after -- are forwarded to the adapter. They are merged with
|
|
168
169
|
--command-opts (cli adapter) or --rspec-opts (rspec adapter).
|
|
@@ -185,6 +186,7 @@ All CLI options can be set via environment variables:
|
|
|
185
186
|
| `SPECBANDIT_KEY_RERUN` | Per-runner rerun key | _(none)_ |
|
|
186
187
|
| `SPECBANDIT_KEY_FAILED` | Redis key for failed test files | _(none)_ |
|
|
187
188
|
| `SPECBANDIT_VERBOSE` | Enable verbose output (`1`/`true`/`yes`) | _(false)_ |
|
|
189
|
+
| `SPECBANDIT_FAIL_FAST` | Stop at the first failed batch (`1`/`true`/`yes`) | _(false)_ |
|
|
188
190
|
|
|
189
191
|
CLI flags take precedence over environment variables.
|
|
190
192
|
|
|
@@ -464,6 +466,28 @@ Both leave the per-runner rerun keys and the failed keys alone, so a single-runn
|
|
|
464
466
|
|
|
465
467
|
A reset with nothing to reset is not an error. Neither command clears the key when there is nothing to push in its place: dropping the marker on its own would make every worker on that key crash as "never published".
|
|
466
468
|
|
|
469
|
+
## Fail-fast: `--fail-fast`
|
|
470
|
+
|
|
471
|
+
Use `--fail-fast` when a single failure makes the rest of the run useless, and you want CI to report it as soon as possible.
|
|
472
|
+
|
|
473
|
+
```bash
|
|
474
|
+
specbandit work --key pr-123-run-456 --adapter rspec --fail-fast
|
|
475
|
+
```
|
|
476
|
+
|
|
477
|
+
When a batch fails, the worker:
|
|
478
|
+
|
|
479
|
+
1. Records the batch in the summary and in the `--report` file, as usual.
|
|
480
|
+
2. Deletes the shared queue list (`DEL <key>`), so no worker can steal the pending files.
|
|
481
|
+
3. Stops and exits 1.
|
|
482
|
+
|
|
483
|
+
The `<key>:published` marker stays. A worker that checks the queue after the prune sees "published but drained" and exits 0, instead of crashing with "never published". A worker that has a batch in flight finishes that batch, then finds the queue empty and stops. Specbandit does not interrupt a batch in flight.
|
|
484
|
+
|
|
485
|
+
Specbandit sees results per batch, so the worker stops after the failing batch, not after the failing example. To also stop inside a batch, forward the option of your test runner, for example `--rspec-opts "--fail-fast"`.
|
|
486
|
+
|
|
487
|
+
In replay mode, the worker stops after the first failed batch and skips the remaining replay files. Replay never touches the shared queue, so there is nothing to prune.
|
|
488
|
+
|
|
489
|
+
`--fail-fast` cannot be used with `--key-failed`. The failed key feeds a later retry pass, and fail-fast stops before most files run, so that list would be incomplete. Specbandit rejects the combination at startup and exits 1. An empty `--key-failed ""` counts as not set.
|
|
490
|
+
|
|
467
491
|
## How it works
|
|
468
492
|
|
|
469
493
|
- **Push** uses `RPUSH` to append all file paths to a Redis list in a single command, sets `EXPIRE` on the key, and writes a durable `<key>:published` marker (with the same TTL). Empty Redis lists are auto-deleted, so this marker is what lets `work` tell "never pushed" (crash) apart from "drained, arriving late" (exit 0).
|
|
@@ -476,6 +500,7 @@ A reset with nothing to reset is not an error. Neither command clears the key wh
|
|
|
476
500
|
- **Run** delegates to the configured adapter:
|
|
477
501
|
- **CLI adapter**: spawns a shell command per batch via `Open3`, appending file paths as arguments. Works with any test runner.
|
|
478
502
|
- **RSpec adapter**: uses `RSpec::Core::Runner.run` in-process with `RSpec.clear_examples` between batches to reset example state while preserving configuration. No subprocess forking overhead.
|
|
503
|
+
- **Fail-fast** (`--fail-fast`): after the first failed batch, the shared queue list is removed with `DEL` (the `<key>:published` marker stays), and the worker stops.
|
|
479
504
|
- **Exit code** is 0 if every batch passed (or the queue was already empty), 1 if any batch had failures.
|
|
480
505
|
|
|
481
506
|
## Development
|
data/lib/specbandit/cli.rb
CHANGED
|
@@ -185,6 +185,10 @@ module Specbandit
|
|
|
185
185
|
Specbandit.configuration.verbose = true
|
|
186
186
|
end
|
|
187
187
|
|
|
188
|
+
opts.on('--fail-fast', 'Stop at the first failed batch and empty the shared queue') do
|
|
189
|
+
Specbandit.configuration.fail_fast = true
|
|
190
|
+
end
|
|
191
|
+
|
|
188
192
|
opts.on('-h', '--help', 'Show this help') do
|
|
189
193
|
puts opts
|
|
190
194
|
return 0
|
|
@@ -200,7 +204,7 @@ module Specbandit
|
|
|
200
204
|
config = Specbandit.configuration
|
|
201
205
|
adapter = build_adapter(config, extra_opts)
|
|
202
206
|
|
|
203
|
-
config.
|
|
207
|
+
config.validate_work!
|
|
204
208
|
|
|
205
209
|
worker = Worker.new(adapter: adapter)
|
|
206
210
|
worker.run
|
|
@@ -276,6 +280,9 @@ module Specbandit
|
|
|
276
280
|
--key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
|
|
277
281
|
--report FILE Write JSON report to FILE after run
|
|
278
282
|
--verbose Show per-batch file list and full command output
|
|
283
|
+
--fail-fast Stop at the first failed batch and remove the pending
|
|
284
|
+
files from the shared queue, so other workers stop too.
|
|
285
|
+
Cannot be used with --key-failed
|
|
279
286
|
|
|
280
287
|
Arguments after -- are forwarded to the adapter (rspec opts, command opts, etc.).
|
|
281
288
|
They are merged with --command-opts or --rspec-opts if both are provided.
|
|
@@ -293,6 +300,7 @@ module Specbandit
|
|
|
293
300
|
SPECBANDIT_KEY_FAILED Redis key for failed test files
|
|
294
301
|
SPECBANDIT_VERBOSE Enable verbose output (1/true/yes)
|
|
295
302
|
SPECBANDIT_REPORT Path to write JSON report file
|
|
303
|
+
SPECBANDIT_FAIL_FAST Stop at the first failed batch (1/true/yes)
|
|
296
304
|
|
|
297
305
|
File input priority for push:
|
|
298
306
|
1. stdin (piped) echo "spec/a_spec.rb" | specbandit push --key KEY
|
|
@@ -5,7 +5,7 @@ module Specbandit
|
|
|
5
5
|
attr_accessor :redis_url, :batch_size, :key, :rspec_opts, :key_ttl,
|
|
6
6
|
:key_rerun, :verbose,
|
|
7
7
|
:adapter, :command, :command_opts,
|
|
8
|
-
:key_failed, :report,
|
|
8
|
+
:key_failed, :report, :fail_fast,
|
|
9
9
|
:redis_max_attempts, :redis_connect_timeout, :redis_timeout,
|
|
10
10
|
:redis_reconnect_attempts
|
|
11
11
|
|
|
@@ -42,6 +42,7 @@ module Specbandit
|
|
|
42
42
|
@command_opts = parse_space_separated(ENV.fetch('SPECBANDIT_COMMAND_OPTS', nil))
|
|
43
43
|
@key_failed = ENV.fetch('SPECBANDIT_KEY_FAILED', nil)
|
|
44
44
|
@report = ENV.fetch('SPECBANDIT_REPORT', nil)
|
|
45
|
+
@fail_fast = env_truthy?('SPECBANDIT_FAIL_FAST')
|
|
45
46
|
|
|
46
47
|
@redis_max_attempts = Integer(ENV.fetch('SPECBANDIT_REDIS_MAX_ATTEMPTS', DEFAULT_REDIS_MAX_ATTEMPTS))
|
|
47
48
|
@redis_connect_timeout = Float(ENV.fetch('SPECBANDIT_REDIS_CONNECT_TIMEOUT', DEFAULT_REDIS_CONNECT_TIMEOUT))
|
|
@@ -56,6 +57,20 @@ module Specbandit
|
|
|
56
57
|
raise Error, 'redis_max_attempts must be a positive integer' unless redis_max_attempts.positive?
|
|
57
58
|
end
|
|
58
59
|
|
|
60
|
+
# Checks that only `work` needs. Kept apart from validate! so that a
|
|
61
|
+
# CI-wide SPECBANDIT_FAIL_FAST cannot break `push` or `reset`.
|
|
62
|
+
def validate_work!
|
|
63
|
+
validate!
|
|
64
|
+
|
|
65
|
+
# The failed key feeds a later retry pass. Fail-fast prunes the queue at
|
|
66
|
+
# the first failure, so most files never run and the retry list would be
|
|
67
|
+
# incomplete. An empty key name counts as not set, like everywhere else.
|
|
68
|
+
return unless fail_fast && !key_failed.nil? && !key_failed.empty?
|
|
69
|
+
|
|
70
|
+
raise Error, '--fail-fast cannot be used with --key-failed: fail-fast stops at the first failure, ' \
|
|
71
|
+
'so the failed key would not hold a complete list to retry'
|
|
72
|
+
end
|
|
73
|
+
|
|
59
74
|
private
|
|
60
75
|
|
|
61
76
|
def parse_rspec_opts(opts)
|
data/lib/specbandit/version.rb
CHANGED
data/lib/specbandit/worker.rb
CHANGED
|
@@ -5,7 +5,7 @@ require 'json'
|
|
|
5
5
|
module Specbandit
|
|
6
6
|
class Worker
|
|
7
7
|
attr_reader :queue, :key, :batch_size, :adapter, :key_rerun, :key_ttl, :key_failed,
|
|
8
|
-
:output, :verbose, :report
|
|
8
|
+
:output, :verbose, :report, :fail_fast
|
|
9
9
|
|
|
10
10
|
def initialize(
|
|
11
11
|
key: Specbandit.configuration.key,
|
|
@@ -16,6 +16,7 @@ module Specbandit
|
|
|
16
16
|
key_failed: Specbandit.configuration.key_failed,
|
|
17
17
|
verbose: Specbandit.configuration.verbose,
|
|
18
18
|
report: Specbandit.configuration.report,
|
|
19
|
+
fail_fast: Specbandit.configuration.fail_fast,
|
|
19
20
|
queue: nil,
|
|
20
21
|
output: $stdout,
|
|
21
22
|
# Legacy parameter for backward compatibility.
|
|
@@ -29,6 +30,7 @@ module Specbandit
|
|
|
29
30
|
@key_failed = key_failed
|
|
30
31
|
@verbose = verbose
|
|
31
32
|
@report = report
|
|
33
|
+
@fail_fast = fail_fast
|
|
32
34
|
@queue = queue || RedisQueue.new
|
|
33
35
|
@output = output
|
|
34
36
|
@batch_results = []
|
|
@@ -153,6 +155,11 @@ module Specbandit
|
|
|
153
155
|
if result.exit_code != 0
|
|
154
156
|
output.puts "[specbandit] Batch ##{batch_num} FAILED (exit code: #{result.exit_code})" if verbose
|
|
155
157
|
failed = true
|
|
158
|
+
if fail_fast
|
|
159
|
+
# Replay never touches the shared queue, so there is nothing to prune.
|
|
160
|
+
output.puts "[specbandit] Fail-fast: batch ##{batch_num} failed. Skipping the remaining replay files."
|
|
161
|
+
break
|
|
162
|
+
end
|
|
156
163
|
elsif verbose
|
|
157
164
|
output.puts "[specbandit] Batch ##{batch_num} passed."
|
|
158
165
|
end
|
|
@@ -197,6 +204,10 @@ module Specbandit
|
|
|
197
204
|
if result.exit_code != 0
|
|
198
205
|
output.puts "[specbandit] Batch ##{batch_num} FAILED (exit code: #{result.exit_code})" if verbose
|
|
199
206
|
failed = true
|
|
207
|
+
if fail_fast
|
|
208
|
+
prune_queue(batch_num)
|
|
209
|
+
break
|
|
210
|
+
end
|
|
200
211
|
elsif verbose
|
|
201
212
|
output.puts "[specbandit] Batch ##{batch_num} passed."
|
|
202
213
|
end
|
|
@@ -211,6 +222,19 @@ module Specbandit
|
|
|
211
222
|
failed ? 1 : 0
|
|
212
223
|
end
|
|
213
224
|
|
|
225
|
+
# Fail-fast: empty the shared queue so no worker steals more files.
|
|
226
|
+
#
|
|
227
|
+
# Only the list is deleted. The published marker stays, so a worker that
|
|
228
|
+
# checks the queue after this sees "published but drained" and exits 0,
|
|
229
|
+
# instead of crashing with "never published". Workers with a batch in
|
|
230
|
+
# flight finish it, then find the queue empty and stop.
|
|
231
|
+
def prune_queue(batch_num)
|
|
232
|
+
pending = queue.length(key)
|
|
233
|
+
queue.delete(key)
|
|
234
|
+
output.puts "[specbandit] Fail-fast: batch ##{batch_num} failed. " \
|
|
235
|
+
"Pruned #{pending} pending files from '#{key}'. Stopping."
|
|
236
|
+
end
|
|
237
|
+
|
|
214
238
|
# Record failed files to the failed key in Redis for later review.
|
|
215
239
|
# Called after each batch; only pushes when key_failed is configured
|
|
216
240
|
# and the batch had a non-zero exit code.
|