specbandit 0.13.1 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 14a1d9a07eb3ab1b19c59c9e054aec4fb32e5da1ac3f33841feb1fb068085648
4
- data.tar.gz: 756a15f2a9c7f9cce3db7fa06e499235fd18ec332a51d6db1900ee1a9a2d1fb8
3
+ metadata.gz: 89e101cef42f9c643d1c651261efff4a5d28c640761dc3f025ce133c78b1446d
4
+ data.tar.gz: 5e93530207fb1fb476fca6c2b373581cc22ba070efd3cd7f30b6a637fc1804c2
5
5
  SHA512:
6
- metadata.gz: 6bc49db9d7bd80540b17d19b15fb759a2ab0c1c867026f3234610074323237cf4cd5ee8d7e590c472b324dbc152b0db83d35fa05bbf443175efad1f5fc658c1c
7
- data.tar.gz: '09cdd5433154bc6a35d13006beccdcba288d8538fc79585e79b8939edf1a0c2e1149d98d21cbdc8ee4eb7f756640d00575b4ebb3791217f5359c511a8ddea6a0'
6
+ metadata.gz: 164896ab49dd19b6309e4edc567b87fe31aa9bfe81a6837e5e945a46c2e38694aac1ae68ee99469160d02cea79d6e64d3ce5a4ea4edb88ef48b61977cd0142a9
7
+ data.tar.gz: 01047ed0fc228b16c70c93f49644379a8bddff86be657a1599cbc83196e54fd7e0722fccf3bb64529911297834464ee107a7c5f875e645c0c42d49597c1f09d6
data/README.md CHANGED
@@ -136,7 +136,7 @@ specbandit push [options] [files...]
136
136
  --key KEY Redis queue key (required)
137
137
  --pattern PATTERN Glob pattern for file discovery
138
138
  --redis-url URL Redis URL (default: redis://localhost:6379)
139
- --key-ttl SECONDS TTL for the Redis key (default: 21600 / 6 hours)
139
+ --key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
140
140
 
141
141
  specbandit work [options] [-- extra-opts...]
142
142
  --key KEY Redis queue key (required)
@@ -147,8 +147,8 @@ specbandit work [options] [-- extra-opts...]
147
147
  --batch-size N Files per batch (default: 5)
148
148
  --redis-url URL Redis URL (default: redis://localhost:6379)
149
149
  --key-rerun KEY Per-runner rerun key for re-run support (see below)
150
- --key-rerun-ttl SECS TTL for rerun key (default: 604800 / 1 week)
151
- --rerun Signal this is a re-run (fail if rerun key is empty)
150
+ --key-failed KEY Redis key to record failed test files
151
+ --key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
152
152
  --verbose Show per-batch file list and full command output
153
153
 
154
154
  Arguments after -- are forwarded to the adapter. They are merged with
@@ -167,11 +167,10 @@ All CLI options can be set via environment variables:
167
167
  | `SPECBANDIT_COMMAND` | Command to run (cli adapter) | _(none)_ |
168
168
  | `SPECBANDIT_COMMAND_OPTS` | Space-separated command options | _(none)_ |
169
169
  | `SPECBANDIT_BATCH_SIZE` | Files per steal | `5` |
170
- | `SPECBANDIT_KEY_TTL` | Key expiry in seconds | `21600` (6 hours) |
170
+ | `SPECBANDIT_KEY_TTL` | Expiry (seconds) for **all** Redis keys | `604800` (1 week) |
171
171
  | `SPECBANDIT_RSPEC_OPTS` | Space-separated RSpec options (rspec adapter) | _(none)_ |
172
172
  | `SPECBANDIT_KEY_RERUN` | Per-runner rerun key | _(none)_ |
173
- | `SPECBANDIT_KEY_RERUN_TTL` | Rerun key expiry in seconds | `604800` (1 week) |
174
- | `SPECBANDIT_RERUN` | Signal re-run mode (`1`/`true`/`yes`) | _(false)_ |
173
+ | `SPECBANDIT_KEY_FAILED` | Redis key for failed test files | _(none)_ |
175
174
  | `SPECBANDIT_VERBOSE` | Enable verbose output (`1`/`true`/`yes`) | _(false)_ |
176
175
 
177
176
  CLI flags take precedence over environment variables.
@@ -185,9 +184,8 @@ Specbandit.configure do |c|
185
184
  c.redis_url = "redis://my-redis:6379"
186
185
  c.key = "pr-123-run-456"
187
186
  c.batch_size = 10
188
- c.key_ttl = 7200 # 2 hours (default: 21600 / 6 hours)
187
+ c.key_ttl = 604_800 # 1 week (default) -- applies to every key specbandit writes
189
188
  c.key_rerun = "pr-123-run-456-runner-3"
190
- c.key_rerun_ttl = 604_800 # 1 week (default)
191
189
  end
192
190
 
193
191
  # Push
@@ -289,7 +287,7 @@ When you use specbandit to distribute tests across multiple CI runners (e.g. a G
289
287
  This creates a subtle but serious problem with CI re-runs:
290
288
 
291
289
  1. **First run**: Runner #3 steals and executes files X, Y, Z. File Y fails. The shared queue is now empty (all files were consumed across all runners).
292
- 2. **Re-run of runner #3**: GitHub Actions re-runs only the failed runner. It starts `specbandit work` again with the same `--key`, but the shared queue is already empty. Runner #3 sees nothing to do and **exits 0 -- the failing test silently passes**.
290
+ 2. **Re-run of runner #3**: GitHub Actions re-runs only the failed runner. It starts `specbandit work` again with the same `--key`, but the shared queue is already empty. Without a per-runner memory the runner would see nothing to do and exit 0 -- silently passing the failing test.
293
291
 
294
292
  This happens because GitHub Actions re-runs **reuse the same `run_id`**, so the key resolves to the same (now empty) Redis list.
295
293
 
@@ -306,33 +304,26 @@ specbandit work \
306
304
  --batch-size 10
307
305
  ```
308
306
 
309
- ### How it works: three operating modes
307
+ ### How it works: the mode is derived from Redis, never from a flag
310
308
 
311
- Specbandit detects the mode automatically based on the state of `--key-rerun`:
309
+ Specbandit decides what to do purely from the **state of Redis** -- there is no `--rerun` flag or environment variable to get wrong. Two signals drive the decision:
312
310
 
313
- | `--key-rerun` provided? | Rerun key in Redis | `--rerun` flag? | Mode | Behavior |
314
- |---|---|---|---|---|
315
- | No | -- | No | **Steal** | Original behavior. Steal from shared queue, run, done. |
316
- | Yes | Empty | No | **Record** | Steal from shared queue + record each batch to the rerun key. |
317
- | Yes | Has data | No/Yes | **Replay** | Ignore shared queue entirely. Re-run exactly the recorded files. |
318
- | Yes | Empty | Yes | **Fail** | Exit 1 with error. Prevents silent false pass on stale re-runs. |
319
- | No | -- | Yes | **Error** | Validation error: `--rerun` requires `--key-rerun`. |
311
+ - **Published marker.** `specbandit push` writes a durable companion key (`<key>:published`) alongside the queue. Because Redis auto-deletes empty lists, a drained queue is indistinguishable from one that was never created -- the marker is the only reliable proof that work was ever enqueued.
312
+ - **Rerun key.** The per-runner `--key-rerun` that records what this runner executed.
320
313
 
321
- > **Empty key names count as "not provided".** A `--key-rerun` whose value is an empty string -- e.g. `--key-rerun "$VAR"` where `$VAR` is unset in CI -- is treated exactly like `--key-rerun` being absent (the **No** rows above), not as a configured-but-empty key. Combined with `--rerun`, that means an empty/unset name fails hard (exit 1) rather than silently steal-and-pass. The same applies to `--key-failed`: an empty/unset name is treated as "not configured" and no failed files are recorded.
314
+ The full decision table:
322
315
 
323
- #### The `--rerun` safety flag
316
+ | Published | Shared queue (`--key`) | Rerun key (`--key-rerun`) | Behavior |
317
+ |---|---|---|---|
318
+ | **No** | -- | -- | **Crash** (exit 1). Nothing was ever pushed for this key (or it expired). Refuses to run rather than silently pass. |
319
+ | Yes | Drained / empty | Empty | **OK, exit 0.** Worker arriving late -- everything was already taken by peers and this runner has no re-run memory. |
320
+ | Yes | Has data | Empty | **Steal.** Classic run: pop batches from the shared queue (and record them to the rerun key if one is configured). |
321
+ | Yes | Drained / empty | Has data | **Replay.** Classic re-run: ignore the shared queue and re-run exactly the recorded files. |
322
+ | Yes | Has data | Has data | **Crash** (exit 1). Inconsistent state -- refuses to run to avoid double-executing. |
324
323
 
325
- Without `--rerun`, specbandit cannot distinguish a first run from a re-run when the rerun key is empty (e.g., TTL expired or Redis was flushed). In that case it silently falls back to Record mode, which may find an empty shared queue and exit 0 with zero tests -- a **silent false pass**.
324
+ > **Empty key names count as "not provided".** A `--key-rerun` whose value is an empty string -- e.g. `--key-rerun "$VAR"` where `$VAR` is unset in CI -- is treated exactly like `--key-rerun` being absent. The same applies to `--key-failed`: an empty/unset name is treated as "not configured" and no failed files are recorded.
326
325
 
327
- The `--rerun` flag tells specbandit "this is definitely a re-run". If the rerun key is empty, it fails hard with exit code 1 and a clear error message instead of silently passing.
328
-
329
- Set it on re-run attempts using your CI's run attempt counter:
330
-
331
- ```yaml
332
- # GitHub Actions: github.run_attempt is "1" on first run, "2"+ on re-runs
333
- env:
334
- SPECBANDIT_RERUN: ${{ github.run_attempt != '1' && '1' || '' }}
335
- ```
326
+ > **The published marker replaces the old `--rerun` flag.** Earlier versions used a `--rerun` / `SPECBANDIT_RERUN` flag (driven by `github.run_attempt`) to decide whether to fail on an empty rerun key. That flag is gone: the marker lets specbandit tell "never pushed" (crash) apart from "drained, arriving late" (exit 0) without any CI-provided hint. Drop `--rerun` / `SPECBANDIT_RERUN` from your config.
336
327
 
337
328
  **On first run**, the rerun key doesn't exist yet (empty), so specbandit enters **record mode**:
338
329
 
@@ -394,11 +385,10 @@ jobs:
394
385
  --key-rerun "pr-${{ github.event.number }}-${{ github.run_id }}-runner-${{ matrix.runner }}" \
395
386
  --redis-url "${{ secrets.REDIS_URL }}" \
396
387
  --adapter rspec \
397
- --batch-size 10 \
398
- ${{ github.run_attempt != '1' && '--rerun' || '' }}
388
+ --batch-size 10
399
389
  ```
400
390
 
401
- The only difference from the basic example is the addition of `--key-rerun` and `--rerun`. The key structure:
391
+ The only difference from the basic example is the addition of `--key-rerun` (no re-run flag needed -- the mode is derived from Redis). The key structure:
402
392
 
403
393
  - `--key` = `pr-42-run-100` -- **shared** across all 4 runners, same on re-run (because `run_id` is reused)
404
394
  - `--key-rerun` = `pr-42-run-100-runner-3` -- **unique per runner**, same on re-run
@@ -422,27 +412,24 @@ The only difference from the basic example is the addition of `--key-rerun` and
422
412
 
423
413
  Runners 1, 2, 4 are not started at all.
424
414
 
425
- ### Rerun key TTL
415
+ ### Key TTL
426
416
 
427
- The rerun key defaults to a **1 week TTL** (`604800` seconds). This is intentionally longer than the shared queue TTL (6 hours) because re-runs can happen hours or even days after the original CI run.
417
+ A **single** TTL governs every key specbandit writes -- the shared queue, its published marker, the per-runner rerun key, and the failed key. It defaults to **1 week** (`604800` seconds), long enough that re-runs happening hours or even days after the original CI run still find their rerun key and published marker alive.
428
418
 
429
- Override via `--key-rerun-ttl` or `SPECBANDIT_KEY_RERUN_TTL`:
419
+ Override via `--key-ttl` or `SPECBANDIT_KEY_TTL` (set it on `push`, which is where the queue and its marker are created):
430
420
 
431
421
  ```bash
432
- # Set rerun key to expire after 3 days
433
- specbandit work \
434
- --key "pr-42-run-100" \
435
- --key-rerun "pr-42-run-100-runner-3" \
436
- --key-rerun-ttl 259200
422
+ # Everything expires after 3 days
423
+ specbandit push --key "pr-42-run-100" --key-ttl 259200 --pattern 'spec/**/*_spec.rb'
437
424
  ```
438
425
 
439
426
  ## How it works
440
427
 
441
- - **Push** uses `RPUSH` to append all file paths to a Redis list in a single command, then sets `EXPIRE` on the key (default: 6 hours) to ensure stale queues are automatically cleaned up.
428
+ - **Push** uses `RPUSH` to append all file paths to a Redis list in a single command, sets `EXPIRE` on the key, and writes a durable `<key>:published` marker (with the same TTL). Empty Redis lists are auto-deleted, so this marker is what lets `work` tell "never pushed" (crash) apart from "drained, arriving late" (exit 0).
442
429
  - **Steal** uses `LPOP key count` (Redis 6.2+), which atomically pops up to N elements. No Lua scripts, no locks, no race conditions.
443
- - **Record** (when `--key-rerun` is set): after each steal, the batch is also `RPUSH`ed to the per-runner rerun key with its own TTL (default: 1 week).
444
- - **Replay** (when `--key-rerun` has data): reads all files from the rerun key via `LRANGE` (non-destructive), splits into batches, and runs them locally. The shared queue is never touched.
445
- - **Rerun safety** (when `--rerun` is set): if the rerun key is empty, specbandit exits 1 immediately instead of falling through to record mode. This prevents silent false passes when the rerun key TTL has expired or Redis was flushed.
430
+ - **Record** (when `--key-rerun` is set): after each steal, the batch is also `RPUSH`ed to the per-runner rerun key.
431
+ - **Replay** (when the rerun key has data and the shared queue is drained): reads all files from the rerun key via `LRANGE` (non-destructive), splits into batches, and runs them locally. The shared queue is never touched.
432
+ - **Mode selection** is derived entirely from Redis (published marker + queue/rerun state) -- see the decision table above. There is no re-run flag or environment variable.
446
433
  - **Run** delegates to the configured adapter:
447
434
  - **CLI adapter**: spawns a shell command per batch via `Open3`, appending file paths as arguments. Works with any test runner.
448
435
  - **RSpec adapter**: uses `RSpec::Core::Runner.run` in-process with `RSpec.clear_examples` between batches to reset example state while preserving configuration. No subprocess forking overhead.
@@ -63,7 +63,7 @@ module Specbandit
63
63
  Specbandit.configuration.redis_url = v
64
64
  end
65
65
 
66
- opts.on('--key-ttl SECONDS', Integer, 'TTL for the Redis key in seconds (default: 21600 / 6 hours)') do |v|
66
+ opts.on('--key-ttl SECONDS', Integer, 'TTL for all Redis keys in seconds (default: 604800 / 1 week)') do |v|
67
67
  Specbandit.configuration.key_ttl = v
68
68
  end
69
69
 
@@ -119,20 +119,12 @@ module Specbandit
119
119
  Specbandit.configuration.key_rerun = v
120
120
  end
121
121
 
122
- opts.on('--key-rerun-ttl SECONDS', Integer, 'TTL for rerun key in seconds (default: 604800 / 1 week)') do |v|
123
- Specbandit.configuration.key_rerun_ttl = v
124
- end
125
-
126
122
  opts.on('--key-failed KEY', 'Redis key to record failed test files for later review') do |v|
127
123
  Specbandit.configuration.key_failed = v
128
124
  end
129
125
 
130
- opts.on('--key-failed-ttl SECONDS', Integer, 'TTL for failed key in seconds (default: 604800 / 1 week)') do |v|
131
- Specbandit.configuration.key_failed_ttl = v
132
- end
133
-
134
- opts.on('--rerun', 'Signal this is a re-run (fail hard if rerun key is empty)') do
135
- Specbandit.configuration.rerun = true
126
+ opts.on('--key-ttl SECONDS', Integer, 'TTL for all Redis keys in seconds (default: 604800 / 1 week)') do |v|
127
+ Specbandit.configuration.key_ttl = v
136
128
  end
137
129
 
138
130
  opts.on('--report FILE', 'Write JSON report with run statistics to FILE') do |v|
@@ -209,7 +201,7 @@ module Specbandit
209
201
  --key KEY Redis queue key (required, or set SPECBANDIT_KEY)
210
202
  --pattern PATTERN Glob pattern for file discovery (e.g. 'spec/**/*_spec.rb')
211
203
  --redis-url URL Redis URL (default: redis://localhost:6379)
212
- --key-ttl SECONDS TTL for the Redis key (default: 21600 / 6 hours)
204
+ --key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
213
205
 
214
206
  Work options:
215
207
  --key KEY Redis queue key (required, or set SPECBANDIT_KEY)
@@ -219,11 +211,9 @@ module Specbandit
219
211
  --rspec-opts OPTS Extra options forwarded to RSpec (for rspec adapter)
220
212
  --batch-size N Files per batch (default: 5, or set SPECBANDIT_BATCH_SIZE)
221
213
  --redis-url URL Redis URL (default: redis://localhost:6379)
222
- --key-rerun KEY Per-runner rerun key for re-run support
223
- --key-rerun-ttl N TTL for rerun key (default: 604800 / 1 week)
224
- --key-failed KEY Redis key to record failed test files
225
- --key-failed-ttl N TTL for failed key (default: 604800 / 1 week)
226
- --rerun Signal this is a re-run (fail hard if rerun key is empty)
214
+ --key-rerun KEY Per-runner rerun key for re-run support
215
+ --key-failed KEY Redis key to record failed test files
216
+ --key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
227
217
  --report FILE Write JSON report to FILE after run
228
218
  --verbose Show per-batch file list and full command output
229
219
 
@@ -237,13 +227,10 @@ module Specbandit
237
227
  SPECBANDIT_COMMAND Command to run (cli adapter)
238
228
  SPECBANDIT_COMMAND_OPTS Command options (space-separated)
239
229
  SPECBANDIT_BATCH_SIZE Batch size
240
- SPECBANDIT_KEY_TTL Key TTL in seconds (default: 21600)
230
+ SPECBANDIT_KEY_TTL TTL for all Redis keys in seconds (default: 604800)
241
231
  SPECBANDIT_RSPEC_OPTS RSpec options (rspec adapter)
242
232
  SPECBANDIT_KEY_RERUN Per-runner rerun key
243
- SPECBANDIT_KEY_RERUN_TTL Rerun key TTL in seconds (default: 604800)
244
233
  SPECBANDIT_KEY_FAILED Redis key for failed test files
245
- SPECBANDIT_KEY_FAILED_TTL Failed key TTL in seconds (default: 604800)
246
- SPECBANDIT_RERUN Signal re-run mode (1/true/yes)
247
234
  SPECBANDIT_VERBOSE Enable verbose output (1/true/yes)
248
235
  SPECBANDIT_REPORT Path to write JSON report file
249
236
 
@@ -3,15 +3,17 @@
3
3
  module Specbandit
4
4
  class Configuration
5
5
  attr_accessor :redis_url, :batch_size, :key, :rspec_opts, :key_ttl,
6
- :key_rerun, :key_rerun_ttl, :rerun, :verbose,
6
+ :key_rerun, :verbose,
7
7
  :adapter, :command, :command_opts,
8
- :key_failed, :key_failed_ttl, :report
8
+ :key_failed, :report
9
9
 
10
10
  DEFAULT_REDIS_URL = 'redis://localhost:6379'
11
11
  DEFAULT_BATCH_SIZE = 5
12
- DEFAULT_KEY_TTL = 21_600 # 6 hours in seconds
13
- DEFAULT_KEY_RERUN_TTL = 604_800 # 1 week in seconds
14
- DEFAULT_KEY_FAILED_TTL = 604_800 # 1 week in seconds
12
+ # A single TTL governs every key specbandit writes: the shared queue, its
13
+ # published marker, the per-runner rerun key and the failed key. It defaults
14
+ # to 1 week because re-runs can happen hours or days after the original run,
15
+ # and the rerun key + published marker must still be alive when they do.
16
+ DEFAULT_KEY_TTL = 604_800 # 1 week in seconds
15
17
  DEFAULT_ADAPTER = 'cli'
16
18
 
17
19
  def initialize
@@ -21,14 +23,11 @@ module Specbandit
21
23
  @rspec_opts = parse_rspec_opts(ENV.fetch('SPECBANDIT_RSPEC_OPTS', nil))
22
24
  @key_ttl = Integer(ENV.fetch('SPECBANDIT_KEY_TTL', DEFAULT_KEY_TTL))
23
25
  @key_rerun = ENV.fetch('SPECBANDIT_KEY_RERUN', nil)
24
- @key_rerun_ttl = Integer(ENV.fetch('SPECBANDIT_KEY_RERUN_TTL', DEFAULT_KEY_RERUN_TTL))
25
- @rerun = env_truthy?('SPECBANDIT_RERUN')
26
26
  @verbose = env_truthy?('SPECBANDIT_VERBOSE')
27
27
  @adapter = ENV.fetch('SPECBANDIT_ADAPTER', DEFAULT_ADAPTER)
28
28
  @command = ENV.fetch('SPECBANDIT_COMMAND', nil)
29
29
  @command_opts = parse_space_separated(ENV.fetch('SPECBANDIT_COMMAND_OPTS', nil))
30
30
  @key_failed = ENV.fetch('SPECBANDIT_KEY_FAILED', nil)
31
- @key_failed_ttl = Integer(ENV.fetch('SPECBANDIT_KEY_FAILED_TTL', DEFAULT_KEY_FAILED_TTL))
32
31
  @report = ENV.fetch('SPECBANDIT_REPORT', nil)
33
32
  end
34
33
 
@@ -36,9 +35,6 @@ module Specbandit
36
35
  raise Error, 'key is required (set via --key or SPECBANDIT_KEY)' if key.nil? || key.empty?
37
36
  raise Error, 'batch_size must be a positive integer' unless batch_size.positive?
38
37
  raise Error, 'key_ttl must be a positive integer' unless key_ttl.positive?
39
- raise Error, 'key_rerun_ttl must be a positive integer' unless key_rerun_ttl.positive?
40
- raise Error, 'key_failed_ttl must be a positive integer' unless key_failed_ttl.positive?
41
- raise Error, '--rerun requires --key-rerun to be set' if rerun && (key_rerun.nil? || key_rerun.empty?)
42
38
  end
43
39
 
44
40
  private
@@ -26,13 +26,28 @@ module Specbandit
26
26
  return 0
27
27
  end
28
28
 
29
- queue.push(key, resolved, ttl: key_ttl)
29
+ push_ms = measure { queue.push(key, resolved, ttl: key_ttl) }
30
+ # Record a durable "published" marker so workers can tell a drained
31
+ # queue ("worker arriving late", OK) apart from one that was never
32
+ # pushed ("you didn't push work", crash). Redis auto-deletes empty
33
+ # lists, so the list itself cannot carry this signal.
34
+ mark_ms = measure { queue.mark_published(key, ttl: key_ttl) }
35
+
30
36
  output.puts "[specbandit] Enqueued #{resolved.size} files onto key '#{key}' (TTL: #{key_ttl}s)."
37
+ output.puts format('[specbandit] Redis latency: push %.1fms, mark published %.1fms.', push_ms, mark_ms)
31
38
  resolved.size
32
39
  end
33
40
 
34
41
  private
35
42
 
43
+ # Run the block and return the wall-clock time it took, in milliseconds.
44
+ # Used to surface how long the Redis round-trips took in the push log.
45
+ def measure
46
+ start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
47
+ yield
48
+ (Process.clock_gettime(Process::CLOCK_MONOTONIC) - start) * 1000
49
+ end
50
+
36
51
  def resolve_files(files:, pattern:)
37
52
  # Priority 1: stdin (only when data is actually piped in)
38
53
  if !$stdin.tty? && $stdin.ready?
@@ -45,6 +45,27 @@ module Specbandit
45
45
  with_retries { redis.llen(key) }
46
46
  end
47
47
 
48
+ # Mark a queue key as "published" by setting a companion marker key.
49
+ #
50
+ # The marker is a separate string key (`<key>:published`) that survives
51
+ # the queue list being fully drained. It is the source of truth for
52
+ # "was work ever pushed for this key?" -- Redis auto-deletes empty lists,
53
+ # so the list alone cannot distinguish "never pushed" from "drained".
54
+ def mark_published(key, ttl: nil)
55
+ with_retries do
56
+ if ttl
57
+ redis.set(published_marker(key), '1', ex: ttl)
58
+ else
59
+ redis.set(published_marker(key), '1')
60
+ end
61
+ end
62
+ end
63
+
64
+ # Whether a queue key has been published (its marker exists).
65
+ def published?(key)
66
+ with_retries { redis.exists?(published_marker(key)) }
67
+ end
68
+
48
69
  # Read all file paths from the list non-destructively.
49
70
  # Returns an array of file paths (empty array when key doesn't exist).
50
71
  def read_all(key)
@@ -57,6 +78,11 @@ module Specbandit
57
78
 
58
79
  private
59
80
 
81
+ # Companion marker key name for a given queue key.
82
+ def published_marker(key)
83
+ "#{key}:published"
84
+ end
85
+
60
86
  def with_retries(attempts: MAX_ATTEMPTS)
61
87
  retries = 0
62
88
  begin
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Specbandit
4
- VERSION = '0.13.1'
4
+ VERSION = '1.0.0'
5
5
  end
@@ -4,7 +4,7 @@ require 'json'
4
4
 
5
5
  module Specbandit
6
6
  class Worker
7
- attr_reader :queue, :key, :batch_size, :adapter, :key_rerun, :key_rerun_ttl, :key_failed, :key_failed_ttl, :rerun,
7
+ attr_reader :queue, :key, :batch_size, :adapter, :key_rerun, :key_ttl, :key_failed,
8
8
  :output, :verbose, :report
9
9
 
10
10
  def initialize(
@@ -12,10 +12,8 @@ module Specbandit
12
12
  batch_size: Specbandit.configuration.batch_size,
13
13
  adapter: nil,
14
14
  key_rerun: Specbandit.configuration.key_rerun,
15
- key_rerun_ttl: Specbandit.configuration.key_rerun_ttl,
15
+ key_ttl: Specbandit.configuration.key_ttl,
16
16
  key_failed: Specbandit.configuration.key_failed,
17
- key_failed_ttl: Specbandit.configuration.key_failed_ttl,
18
- rerun: Specbandit.configuration.rerun,
19
17
  verbose: Specbandit.configuration.verbose,
20
18
  report: Specbandit.configuration.report,
21
19
  queue: nil,
@@ -27,10 +25,8 @@ module Specbandit
27
25
  @key = key
28
26
  @batch_size = batch_size
29
27
  @key_rerun = key_rerun
30
- @key_rerun_ttl = key_rerun_ttl
28
+ @key_ttl = key_ttl
31
29
  @key_failed = key_failed
32
- @key_failed_ttl = key_failed_ttl
33
- @rerun = rerun
34
30
  @verbose = verbose
35
31
  @report = report
36
32
  @queue = queue || RedisQueue.new
@@ -69,22 +65,42 @@ module Specbandit
69
65
 
70
66
  private
71
67
 
72
- # Decide the operating mode and execute it, returning the exit code.
73
- # - no usable rerun key → steal mode (or crash if --rerun was requested)
74
- # - rerun key has data → replay mode
75
- # - rerun key but empty → record mode (or crash if --rerun was requested)
68
+ # Decide the operating mode from the state of Redis and execute it.
69
+ #
70
+ # The mode is derived entirely from Redis -- never from an environment
71
+ # variable or flag -- following this table:
72
+ #
73
+ # Published | Key (queue) | Rerun key | Behavior
74
+ # ----------|----------------|-------------|-------------------------------
75
+ # No | -- | -- | Crash: nothing was ever pushed
76
+ # Yes | empty/drained | empty | OK: worker arriving late (0)
77
+ # Yes | has data | empty | Steal (record if rerun key set)
78
+ # Yes | empty/drained | has data | Replay recorded files
79
+ # Yes | has data | has data | Crash: inconsistent (weird case)
80
+ #
81
+ # "Published" is a durable marker written by `specbandit push`; it is the
82
+ # only reliable signal that work was ever enqueued, because Redis
83
+ # auto-deletes empty lists (so a drained queue is indistinguishable from a
84
+ # never-created one by the list alone).
76
85
  def determine_exit_code
77
- unless key_present?(key_rerun)
78
- return fail_stale_rerun if rerun
86
+ return fail_not_published unless queue.published?(key)
79
87
 
80
- return run_steal(record: false)
81
- end
82
-
83
- rerun_files = queue.read_all(key_rerun)
84
- return run_replay(rerun_files) if rerun_files.any?
85
- return fail_stale_rerun if rerun
88
+ key_has_data = queue.length(key).positive?
89
+ rerun_files = key_present?(key_rerun) ? queue.read_all(key_rerun) : []
86
90
 
87
- run_steal(record: true)
91
+ if key_has_data && rerun_files.any?
92
+ fail_inconsistent_state
93
+ elsif rerun_files.any?
94
+ run_replay(rerun_files)
95
+ elsif key_has_data
96
+ run_steal(record: key_present?(key_rerun))
97
+ else
98
+ # Published, but the queue is drained and this runner has no rerun
99
+ # memory: it simply arrived after everything was already taken.
100
+ output.puts "[specbandit] Queue '#{key}' already drained and no rerun files to replay. " \
101
+ 'Nothing to do (worker arriving late).' if verbose
102
+ 0
103
+ end
88
104
  end
89
105
 
90
106
  # A Redis key name is usable only when it is a non-nil, non-empty string.
@@ -94,15 +110,22 @@ module Specbandit
94
110
  !value.nil? && !value.empty?
95
111
  end
96
112
 
97
- # Emit the stale/missing-rerun error and return exit code 1.
98
- def fail_stale_rerun
99
- if key_present?(key_rerun)
100
- output.puts "[specbandit] ERROR: --rerun flag is set but rerun key '#{key_rerun}' is empty."
101
- else
102
- output.puts '[specbandit] ERROR: --rerun flag is set but no rerun key is configured.'
103
- end
104
- output.puts '[specbandit] The rerun key may have expired (TTL) or Redis was flushed.'
105
- output.puts '[specbandit] Cannot replay — failing to prevent silent false pass.'
113
+ # No published marker for this key: `specbandit push` was never run for it,
114
+ # or the key expired. Crash rather than silently pass with zero tests.
115
+ def fail_not_published
116
+ output.puts "[specbandit] ERROR: queue '#{key}' was never published."
117
+ output.puts '[specbandit] Run `specbandit push` before `specbandit work`, or the key/TTL may have expired.'
118
+ output.puts '[specbandit] Refusing to run to prevent a silent false pass.'
119
+ 1
120
+ end
121
+
122
+ # Both the shared queue and this runner's rerun key hold files at once.
123
+ # That should never happen: a fresh run has no rerun memory yet, and a
124
+ # re-run reads from a drained queue. Crash instead of double-executing.
125
+ def fail_inconsistent_state
126
+ output.puts "[specbandit] ERROR: inconsistent state — shared queue '#{key}' still has files " \
127
+ "while rerun key '#{key_rerun}' also has recorded files."
128
+ output.puts '[specbandit] Refusing to run to avoid double-execution / undefined behavior.'
106
129
  1
107
130
  end
108
131
 
@@ -159,7 +182,7 @@ module Specbandit
159
182
  end
160
183
 
161
184
  # Record the stolen batch so this runner can replay on re-run
162
- queue.push(key_rerun, files, ttl: key_rerun_ttl) if record
185
+ queue.push(key_rerun, files, ttl: key_ttl) if record
163
186
 
164
187
  batch_num += 1
165
188
  output.puts "[specbandit] Batch ##{batch_num}: running #{files.size} files" if verbose
@@ -200,7 +223,7 @@ module Specbandit
200
223
  failed_files = extract_failed_files(result) || files
201
224
  return if failed_files.empty?
202
225
 
203
- queue.push(key_failed, failed_files, ttl: key_failed_ttl)
226
+ queue.push(key_failed, failed_files, ttl: key_ttl)
204
227
  end
205
228
 
206
229
  # Extract individual failed file paths from an RspecBatchResult's JSON output.
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: specbandit
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.13.1
4
+ version: 1.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ferran Basora