specbandit 1.3.1 → 1.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: df38ee5f158f484e5e0952106172f864957d6a97af3ec73b7ce6c4214d264638
4
- data.tar.gz: 39e6ad3ffc12353dca6f72f1286fdea1c4646349d5e230a3b926ce313ebf8bd4
3
+ metadata.gz: d2aab71e7d9a6d7f5e3be3ae3aa4c0bb1c44933ec7b6d5fd0b3c4d00530ed6f3
4
+ data.tar.gz: 12e1f2b14710316d042acf34e1a1fabb888c9817ec82270823cf6229f76702de
5
5
  SHA512:
6
- metadata.gz: 1c6969d9d69d6bb39ef3554c542681b9bffe4d77e78829180ecf85fa41560f2c2e07789fd0bff31dd628f15045aa8ce64d33f091798afdcc9cf8aa15abcfb04d
7
- data.tar.gz: 5c9ab6cc2373c70daf21f040c5049013d5fd958c86ab06ef9a59f82326684eddc398f30233ff661359974aa6b7f614a0ee7d05551e503285c058215294d44f07
6
+ metadata.gz: 76484e1fc1740b9b416fc69348156b80c536b218ec3a762e1d3366e57f2efe4946ad708e00258017fc0342b1e1155985d25aed486fb2e6fb7c024276c85c4f92
7
+ data.tar.gz: 27dcfd5025a9e32d4bbb79e73cccb25e862ca455758746e02fa9347677974e7abdc0fea1baf237e63d36e477bed0ff571dacdd3d95d4abf5c1c86ef6f904aa78
data/README.md CHANGED
@@ -135,7 +135,7 @@ Each worker loops:
135
135
  3. Repeat until the queue is empty
136
136
  4. Exit 0 if all batches passed, 1 if any failed
137
137
 
138
- A failing batch does **not** stop the worker. It continues stealing remaining work so other runners aren't blocked waiting on files that will never be consumed.
138
+ By default, a failing batch does **not** stop the worker. It continues stealing remaining work so other runners aren't blocked waiting on files that will never be consumed. To stop the whole run at the first failure, use [`--fail-fast`](#fail-fast---fail-fast).
139
139
 
140
140
  ### CLI reference
141
141
 
@@ -163,6 +163,7 @@ specbandit work [options] [-- extra-opts...]
163
163
  --key-failed KEY Redis key to record failed test files
164
164
  --key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
165
165
  --verbose Show per-batch file list and full command output
166
+ --fail-fast Stop at the first failed batch and empty the shared queue (see below)
166
167
 
167
168
  Arguments after -- are forwarded to the adapter. They are merged with
168
169
  --command-opts (cli adapter) or --rspec-opts (rspec adapter).
@@ -185,6 +186,7 @@ All CLI options can be set via environment variables:
185
186
  | `SPECBANDIT_KEY_RERUN` | Per-runner rerun key | _(none)_ |
186
187
  | `SPECBANDIT_KEY_FAILED` | Redis key for failed test files | _(none)_ |
187
188
  | `SPECBANDIT_VERBOSE` | Enable verbose output (`1`/`true`/`yes`) | _(false)_ |
189
+ | `SPECBANDIT_FAIL_FAST` | Stop at the first failed batch (`1`/`true`/`yes`) | _(false)_ |
188
190
 
189
191
  CLI flags take precedence over environment variables.
190
192
 
@@ -464,6 +466,28 @@ Both leave the per-runner rerun keys and the failed keys alone, so a single-runn
464
466
 
465
467
  A reset with nothing to reset is not an error. Neither command clears the key when there is nothing to push in its place: dropping the marker on its own would make every worker on that key crash as "never published".
466
468
 
469
+ ## Fail-fast: `--fail-fast`
470
+
471
+ Use `--fail-fast` when a single failure makes the rest of the run useless, and you want CI to report it as soon as possible.
472
+
473
+ ```bash
474
+ specbandit work --key pr-123-run-456 --adapter rspec --fail-fast
475
+ ```
476
+
477
+ When a batch fails, the worker:
478
+
479
+ 1. Records the batch in the summary and in the `--report` file, as usual.
480
+ 2. Deletes the shared queue list (`DEL <key>`), so no worker can steal the pending files.
481
+ 3. Stops and exits 1.
482
+
483
+ The `<key>:published` marker stays. A worker that checks the queue after the prune sees "published but drained" and exits 0, instead of crashing with "never published". A worker that has a batch in flight finishes that batch, then finds the queue empty and stops. Specbandit does not interrupt a batch in flight.
484
+
485
+ Specbandit sees results per batch, so the worker stops after the failing batch, not after the failing example. To also stop inside a batch, forward the option of your test runner, for example `--rspec-opts "--fail-fast"`.
486
+
487
+ In replay mode, the worker stops after the first failed batch and skips the remaining replay files. Replay never touches the shared queue, so there is nothing to prune.
488
+
489
+ `--fail-fast` cannot be used with `--key-failed`. The failed key feeds a later retry pass, and fail-fast stops before most files run, so that list would be incomplete. Specbandit rejects the combination at startup and exits 1. An empty `--key-failed ""` counts as not set.
490
+
467
491
  ## How it works
468
492
 
469
493
  - **Push** uses `RPUSH` to append all file paths to a Redis list in a single command, sets `EXPIRE` on the key, and writes a durable `<key>:published` marker (with the same TTL). Empty Redis lists are auto-deleted, so this marker is what lets `work` tell "never pushed" (crash) apart from "drained, arriving late" (exit 0).
@@ -476,6 +500,7 @@ A reset with nothing to reset is not an error. Neither command clears the key wh
476
500
  - **Run** delegates to the configured adapter:
477
501
  - **CLI adapter**: spawns a shell command per batch via `Open3`, appending file paths as arguments. Works with any test runner.
478
502
  - **RSpec adapter**: uses `RSpec::Core::Runner.run` in-process with `RSpec.clear_examples` between batches to reset example state while preserving configuration. No subprocess forking overhead.
503
+ - **Fail-fast** (`--fail-fast`): after the first failed batch, the shared queue list is removed with `DEL` (the `<key>:published` marker stays), and the worker stops.
479
504
  - **Exit code** is 0 if every batch passed (or the queue was already empty), 1 if any batch had failures.
480
505
 
481
506
  ## Development
@@ -185,6 +185,10 @@ module Specbandit
185
185
  Specbandit.configuration.verbose = true
186
186
  end
187
187
 
188
+ opts.on('--fail-fast', 'Stop at the first failed batch and empty the shared queue') do
189
+ Specbandit.configuration.fail_fast = true
190
+ end
191
+
188
192
  opts.on('-h', '--help', 'Show this help') do
189
193
  puts opts
190
194
  return 0
@@ -200,7 +204,7 @@ module Specbandit
200
204
  config = Specbandit.configuration
201
205
  adapter = build_adapter(config, extra_opts)
202
206
 
203
- config.validate!
207
+ config.validate_work!
204
208
 
205
209
  worker = Worker.new(adapter: adapter)
206
210
  worker.run
@@ -276,6 +280,9 @@ module Specbandit
276
280
  --key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
277
281
  --report FILE Write JSON report to FILE after run
278
282
  --verbose Show per-batch file list and full command output
283
+ --fail-fast Stop at the first failed batch and remove the pending
284
+ files from the shared queue, so other workers stop too.
285
+ Cannot be used with --key-failed
279
286
 
280
287
  Arguments after -- are forwarded to the adapter (rspec opts, command opts, etc.).
281
288
  They are merged with --command-opts or --rspec-opts if both are provided.
@@ -293,6 +300,7 @@ module Specbandit
293
300
  SPECBANDIT_KEY_FAILED Redis key for failed test files
294
301
  SPECBANDIT_VERBOSE Enable verbose output (1/true/yes)
295
302
  SPECBANDIT_REPORT Path to write JSON report file
303
+ SPECBANDIT_FAIL_FAST Stop at the first failed batch (1/true/yes)
296
304
 
297
305
  File input priority for push:
298
306
  1. stdin (piped) echo "spec/a_spec.rb" | specbandit push --key KEY
@@ -5,7 +5,7 @@ module Specbandit
5
5
  attr_accessor :redis_url, :batch_size, :key, :rspec_opts, :key_ttl,
6
6
  :key_rerun, :verbose,
7
7
  :adapter, :command, :command_opts,
8
- :key_failed, :report,
8
+ :key_failed, :report, :fail_fast,
9
9
  :redis_max_attempts, :redis_connect_timeout, :redis_timeout,
10
10
  :redis_reconnect_attempts
11
11
 
@@ -42,6 +42,7 @@ module Specbandit
42
42
  @command_opts = parse_space_separated(ENV.fetch('SPECBANDIT_COMMAND_OPTS', nil))
43
43
  @key_failed = ENV.fetch('SPECBANDIT_KEY_FAILED', nil)
44
44
  @report = ENV.fetch('SPECBANDIT_REPORT', nil)
45
+ @fail_fast = env_truthy?('SPECBANDIT_FAIL_FAST')
45
46
 
46
47
  @redis_max_attempts = Integer(ENV.fetch('SPECBANDIT_REDIS_MAX_ATTEMPTS', DEFAULT_REDIS_MAX_ATTEMPTS))
47
48
  @redis_connect_timeout = Float(ENV.fetch('SPECBANDIT_REDIS_CONNECT_TIMEOUT', DEFAULT_REDIS_CONNECT_TIMEOUT))
@@ -56,6 +57,20 @@ module Specbandit
56
57
  raise Error, 'redis_max_attempts must be a positive integer' unless redis_max_attempts.positive?
57
58
  end
58
59
 
60
+ # Checks that only `work` needs. Kept apart from validate! so that a
61
+ # CI-wide SPECBANDIT_FAIL_FAST cannot break `push` or `reset`.
62
+ def validate_work!
63
+ validate!
64
+
65
+ # The failed key feeds a later retry pass. Fail-fast prunes the queue at
66
+ # the first failure, so most files never run and the retry list would be
67
+ # incomplete. An empty key name counts as not set, like everywhere else.
68
+ return unless fail_fast && !key_failed.nil? && !key_failed.empty?
69
+
70
+ raise Error, '--fail-fast cannot be used with --key-failed: fail-fast stops at the first failure, ' \
71
+ 'so the failed key would not hold a complete list to retry'
72
+ end
73
+
59
74
  private
60
75
 
61
76
  def parse_rspec_opts(opts)
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Specbandit
4
- VERSION = '1.3.1'
4
+ VERSION = '1.4.0'
5
5
  end
@@ -5,7 +5,7 @@ require 'json'
5
5
  module Specbandit
6
6
  class Worker
7
7
  attr_reader :queue, :key, :batch_size, :adapter, :key_rerun, :key_ttl, :key_failed,
8
- :output, :verbose, :report
8
+ :output, :verbose, :report, :fail_fast
9
9
 
10
10
  def initialize(
11
11
  key: Specbandit.configuration.key,
@@ -16,6 +16,7 @@ module Specbandit
16
16
  key_failed: Specbandit.configuration.key_failed,
17
17
  verbose: Specbandit.configuration.verbose,
18
18
  report: Specbandit.configuration.report,
19
+ fail_fast: Specbandit.configuration.fail_fast,
19
20
  queue: nil,
20
21
  output: $stdout,
21
22
  # Legacy parameter for backward compatibility.
@@ -29,6 +30,7 @@ module Specbandit
29
30
  @key_failed = key_failed
30
31
  @verbose = verbose
31
32
  @report = report
33
+ @fail_fast = fail_fast
32
34
  @queue = queue || RedisQueue.new
33
35
  @output = output
34
36
  @batch_results = []
@@ -153,6 +155,11 @@ module Specbandit
153
155
  if result.exit_code != 0
154
156
  output.puts "[specbandit] Batch ##{batch_num} FAILED (exit code: #{result.exit_code})" if verbose
155
157
  failed = true
158
+ if fail_fast
159
+ # Replay never touches the shared queue, so there is nothing to prune.
160
+ output.puts "[specbandit] Fail-fast: batch ##{batch_num} failed. Skipping the remaining replay files."
161
+ break
162
+ end
156
163
  elsif verbose
157
164
  output.puts "[specbandit] Batch ##{batch_num} passed."
158
165
  end
@@ -197,6 +204,10 @@ module Specbandit
197
204
  if result.exit_code != 0
198
205
  output.puts "[specbandit] Batch ##{batch_num} FAILED (exit code: #{result.exit_code})" if verbose
199
206
  failed = true
207
+ if fail_fast
208
+ prune_queue(batch_num)
209
+ break
210
+ end
200
211
  elsif verbose
201
212
  output.puts "[specbandit] Batch ##{batch_num} passed."
202
213
  end
@@ -211,6 +222,19 @@ module Specbandit
211
222
  failed ? 1 : 0
212
223
  end
213
224
 
225
+ # Fail-fast: empty the shared queue so no worker steals more files.
226
+ #
227
+ # Only the list is deleted. The published marker stays, so a worker that
228
+ # checks the queue after this sees "published but drained" and exits 0,
229
+ # instead of crashing with "never published". Workers with a batch in
230
+ # flight finish it, then find the queue empty and stop.
231
+ def prune_queue(batch_num)
232
+ pending = queue.length(key)
233
+ queue.delete(key)
234
+ output.puts "[specbandit] Fail-fast: batch ##{batch_num} failed. " \
235
+ "Pruned #{pending} pending files from '#{key}'. Stopping."
236
+ end
237
+
214
238
  # Record failed files to the failed key in Redis for later review.
215
239
  # Called after each batch; only pushes when key_failed is configured
216
240
  # and the batch had a non-zero exit code.
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: specbandit
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.3.1
4
+ version: 1.4.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ferran Basora