specbandit 0.13.1 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +32 -45
- data/lib/specbandit/cli.rb +8 -21
- data/lib/specbandit/configuration.rb +7 -11
- data/lib/specbandit/publisher.rb +16 -1
- data/lib/specbandit/redis_queue.rb +26 -0
- data/lib/specbandit/version.rb +1 -1
- data/lib/specbandit/worker.rb +54 -31
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 89e101cef42f9c643d1c651261efff4a5d28c640761dc3f025ce133c78b1446d
|
|
4
|
+
data.tar.gz: 5e93530207fb1fb476fca6c2b373581cc22ba070efd3cd7f30b6a637fc1804c2
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 164896ab49dd19b6309e4edc567b87fe31aa9bfe81a6837e5e945a46c2e38694aac1ae68ee99469160d02cea79d6e64d3ce5a4ea4edb88ef48b61977cd0142a9
|
|
7
|
+
data.tar.gz: 01047ed0fc228b16c70c93f49644379a8bddff86be657a1599cbc83196e54fd7e0722fccf3bb64529911297834464ee107a7c5f875e645c0c42d49597c1f09d6
|
data/README.md
CHANGED
|
@@ -136,7 +136,7 @@ specbandit push [options] [files...]
|
|
|
136
136
|
--key KEY Redis queue key (required)
|
|
137
137
|
--pattern PATTERN Glob pattern for file discovery
|
|
138
138
|
--redis-url URL Redis URL (default: redis://localhost:6379)
|
|
139
|
-
--key-ttl SECONDS TTL for
|
|
139
|
+
--key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
|
|
140
140
|
|
|
141
141
|
specbandit work [options] [-- extra-opts...]
|
|
142
142
|
--key KEY Redis queue key (required)
|
|
@@ -147,8 +147,8 @@ specbandit work [options] [-- extra-opts...]
|
|
|
147
147
|
--batch-size N Files per batch (default: 5)
|
|
148
148
|
--redis-url URL Redis URL (default: redis://localhost:6379)
|
|
149
149
|
--key-rerun KEY Per-runner rerun key for re-run support (see below)
|
|
150
|
-
--key-
|
|
151
|
-
--
|
|
150
|
+
--key-failed KEY Redis key to record failed test files
|
|
151
|
+
--key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
|
|
152
152
|
--verbose Show per-batch file list and full command output
|
|
153
153
|
|
|
154
154
|
Arguments after -- are forwarded to the adapter. They are merged with
|
|
@@ -167,11 +167,10 @@ All CLI options can be set via environment variables:
|
|
|
167
167
|
| `SPECBANDIT_COMMAND` | Command to run (cli adapter) | _(none)_ |
|
|
168
168
|
| `SPECBANDIT_COMMAND_OPTS` | Space-separated command options | _(none)_ |
|
|
169
169
|
| `SPECBANDIT_BATCH_SIZE` | Files per steal | `5` |
|
|
170
|
-
| `SPECBANDIT_KEY_TTL` |
|
|
170
|
+
| `SPECBANDIT_KEY_TTL` | Expiry (seconds) for **all** Redis keys | `604800` (1 week) |
|
|
171
171
|
| `SPECBANDIT_RSPEC_OPTS` | Space-separated RSpec options (rspec adapter) | _(none)_ |
|
|
172
172
|
| `SPECBANDIT_KEY_RERUN` | Per-runner rerun key | _(none)_ |
|
|
173
|
-
| `
|
|
174
|
-
| `SPECBANDIT_RERUN` | Signal re-run mode (`1`/`true`/`yes`) | _(false)_ |
|
|
173
|
+
| `SPECBANDIT_KEY_FAILED` | Redis key for failed test files | _(none)_ |
|
|
175
174
|
| `SPECBANDIT_VERBOSE` | Enable verbose output (`1`/`true`/`yes`) | _(false)_ |
|
|
176
175
|
|
|
177
176
|
CLI flags take precedence over environment variables.
|
|
@@ -185,9 +184,8 @@ Specbandit.configure do |c|
|
|
|
185
184
|
c.redis_url = "redis://my-redis:6379"
|
|
186
185
|
c.key = "pr-123-run-456"
|
|
187
186
|
c.batch_size = 10
|
|
188
|
-
c.key_ttl =
|
|
187
|
+
c.key_ttl = 604_800 # 1 week (default) -- applies to every key specbandit writes
|
|
189
188
|
c.key_rerun = "pr-123-run-456-runner-3"
|
|
190
|
-
c.key_rerun_ttl = 604_800 # 1 week (default)
|
|
191
189
|
end
|
|
192
190
|
|
|
193
191
|
# Push
|
|
@@ -289,7 +287,7 @@ When you use specbandit to distribute tests across multiple CI runners (e.g. a G
|
|
|
289
287
|
This creates a subtle but serious problem with CI re-runs:
|
|
290
288
|
|
|
291
289
|
1. **First run**: Runner #3 steals and executes files X, Y, Z. File Y fails. The shared queue is now empty (all files were consumed across all runners).
|
|
292
|
-
2. **Re-run of runner #3**: GitHub Actions re-runs only the failed runner. It starts `specbandit work` again with the same `--key`, but the shared queue is already empty.
|
|
290
|
+
2. **Re-run of runner #3**: GitHub Actions re-runs only the failed runner. It starts `specbandit work` again with the same `--key`, but the shared queue is already empty. Without a per-runner memory the runner would see nothing to do and exit 0 -- silently passing the failing test.
|
|
293
291
|
|
|
294
292
|
This happens because GitHub Actions re-runs **reuse the same `run_id`**, so the key resolves to the same (now empty) Redis list.
|
|
295
293
|
|
|
@@ -306,33 +304,26 @@ specbandit work \
|
|
|
306
304
|
--batch-size 10
|
|
307
305
|
```
|
|
308
306
|
|
|
309
|
-
### How it works:
|
|
307
|
+
### How it works: the mode is derived from Redis, never from a flag
|
|
310
308
|
|
|
311
|
-
Specbandit
|
|
309
|
+
Specbandit decides what to do purely from the **state of Redis** -- there is no `--rerun` flag or environment variable to get wrong. Two signals drive the decision:
|
|
312
310
|
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
| No | -- | No | **Steal** | Original behavior. Steal from shared queue, run, done. |
|
|
316
|
-
| Yes | Empty | No | **Record** | Steal from shared queue + record each batch to the rerun key. |
|
|
317
|
-
| Yes | Has data | No/Yes | **Replay** | Ignore shared queue entirely. Re-run exactly the recorded files. |
|
|
318
|
-
| Yes | Empty | Yes | **Fail** | Exit 1 with error. Prevents silent false pass on stale re-runs. |
|
|
319
|
-
| No | -- | Yes | **Error** | Validation error: `--rerun` requires `--key-rerun`. |
|
|
311
|
+
- **Published marker.** `specbandit push` writes a durable companion key (`<key>:published`) alongside the queue. Because Redis auto-deletes empty lists, a drained queue is indistinguishable from one that was never created -- the marker is the only reliable proof that work was ever enqueued.
|
|
312
|
+
- **Rerun key.** The per-runner `--key-rerun` that records what this runner executed.
|
|
320
313
|
|
|
321
|
-
|
|
314
|
+
The full decision table:
|
|
322
315
|
|
|
323
|
-
|
|
316
|
+
| Published | Shared queue (`--key`) | Rerun key (`--key-rerun`) | Behavior |
|
|
317
|
+
|---|---|---|---|
|
|
318
|
+
| **No** | -- | -- | **Crash** (exit 1). Nothing was ever pushed for this key (or it expired). Refuses to run rather than silently pass. |
|
|
319
|
+
| Yes | Drained / empty | Empty | **OK, exit 0.** Worker arriving late -- everything was already taken by peers and this runner has no re-run memory. |
|
|
320
|
+
| Yes | Has data | Empty | **Steal.** Classic run: pop batches from the shared queue (and record them to the rerun key if one is configured). |
|
|
321
|
+
| Yes | Drained / empty | Has data | **Replay.** Classic re-run: ignore the shared queue and re-run exactly the recorded files. |
|
|
322
|
+
| Yes | Has data | Has data | **Crash** (exit 1). Inconsistent state -- refuses to run to avoid double-executing. |
|
|
324
323
|
|
|
325
|
-
|
|
324
|
+
> **Empty key names count as "not provided".** A `--key-rerun` whose value is an empty string -- e.g. `--key-rerun "$VAR"` where `$VAR` is unset in CI -- is treated exactly like `--key-rerun` being absent. The same applies to `--key-failed`: an empty/unset name is treated as "not configured" and no failed files are recorded.
|
|
326
325
|
|
|
327
|
-
The `--rerun` flag
|
|
328
|
-
|
|
329
|
-
Set it on re-run attempts using your CI's run attempt counter:
|
|
330
|
-
|
|
331
|
-
```yaml
|
|
332
|
-
# GitHub Actions: github.run_attempt is "1" on first run, "2"+ on re-runs
|
|
333
|
-
env:
|
|
334
|
-
SPECBANDIT_RERUN: ${{ github.run_attempt != '1' && '1' || '' }}
|
|
335
|
-
```
|
|
326
|
+
> **The published marker replaces the old `--rerun` flag.** Earlier versions used a `--rerun` / `SPECBANDIT_RERUN` flag (driven by `github.run_attempt`) to decide whether to fail on an empty rerun key. That flag is gone: the marker lets specbandit tell "never pushed" (crash) apart from "drained, arriving late" (exit 0) without any CI-provided hint. Drop `--rerun` / `SPECBANDIT_RERUN` from your config.
|
|
336
327
|
|
|
337
328
|
**On first run**, the rerun key doesn't exist yet (empty), so specbandit enters **record mode**:
|
|
338
329
|
|
|
@@ -394,11 +385,10 @@ jobs:
|
|
|
394
385
|
--key-rerun "pr-${{ github.event.number }}-${{ github.run_id }}-runner-${{ matrix.runner }}" \
|
|
395
386
|
--redis-url "${{ secrets.REDIS_URL }}" \
|
|
396
387
|
--adapter rspec \
|
|
397
|
-
--batch-size 10
|
|
398
|
-
${{ github.run_attempt != '1' && '--rerun' || '' }}
|
|
388
|
+
--batch-size 10
|
|
399
389
|
```
|
|
400
390
|
|
|
401
|
-
The only difference from the basic example is the addition of `--key-rerun`
|
|
391
|
+
The only difference from the basic example is the addition of `--key-rerun` (no re-run flag needed -- the mode is derived from Redis). The key structure:
|
|
402
392
|
|
|
403
393
|
- `--key` = `pr-42-run-100` -- **shared** across all 4 runners, same on re-run (because `run_id` is reused)
|
|
404
394
|
- `--key-rerun` = `pr-42-run-100-runner-3` -- **unique per runner**, same on re-run
|
|
@@ -422,27 +412,24 @@ The only difference from the basic example is the addition of `--key-rerun` and
|
|
|
422
412
|
|
|
423
413
|
Runners 1, 2, 4 are not started at all.
|
|
424
414
|
|
|
425
|
-
###
|
|
415
|
+
### Key TTL
|
|
426
416
|
|
|
427
|
-
|
|
417
|
+
A **single** TTL governs every key specbandit writes -- the shared queue, its published marker, the per-runner rerun key, and the failed key. It defaults to **1 week** (`604800` seconds), long enough that re-runs happening hours or even days after the original CI run still find their rerun key and published marker alive.
|
|
428
418
|
|
|
429
|
-
Override via `--key-
|
|
419
|
+
Override via `--key-ttl` or `SPECBANDIT_KEY_TTL` (set it on `push`, which is where the queue and its marker are created):
|
|
430
420
|
|
|
431
421
|
```bash
|
|
432
|
-
#
|
|
433
|
-
specbandit
|
|
434
|
-
--key "pr-42-run-100" \
|
|
435
|
-
--key-rerun "pr-42-run-100-runner-3" \
|
|
436
|
-
--key-rerun-ttl 259200
|
|
422
|
+
# Everything expires after 3 days
|
|
423
|
+
specbandit push --key "pr-42-run-100" --key-ttl 259200 --pattern 'spec/**/*_spec.rb'
|
|
437
424
|
```
|
|
438
425
|
|
|
439
426
|
## How it works
|
|
440
427
|
|
|
441
|
-
- **Push** uses `RPUSH` to append all file paths to a Redis list in a single command,
|
|
428
|
+
- **Push** uses `RPUSH` to append all file paths to a Redis list in a single command, sets `EXPIRE` on the key, and writes a durable `<key>:published` marker (with the same TTL). Empty Redis lists are auto-deleted, so this marker is what lets `work` tell "never pushed" (crash) apart from "drained, arriving late" (exit 0).
|
|
442
429
|
- **Steal** uses `LPOP key count` (Redis 6.2+), which atomically pops up to N elements. No Lua scripts, no locks, no race conditions.
|
|
443
|
-
- **Record** (when `--key-rerun` is set): after each steal, the batch is also `RPUSH`ed to the per-runner rerun key
|
|
444
|
-
- **Replay** (when
|
|
445
|
-
- **
|
|
430
|
+
- **Record** (when `--key-rerun` is set): after each steal, the batch is also `RPUSH`ed to the per-runner rerun key.
|
|
431
|
+
- **Replay** (when the rerun key has data and the shared queue is drained): reads all files from the rerun key via `LRANGE` (non-destructive), splits into batches, and runs them locally. The shared queue is never touched.
|
|
432
|
+
- **Mode selection** is derived entirely from Redis (published marker + queue/rerun state) -- see the decision table above. There is no re-run flag or environment variable.
|
|
446
433
|
- **Run** delegates to the configured adapter:
|
|
447
434
|
- **CLI adapter**: spawns a shell command per batch via `Open3`, appending file paths as arguments. Works with any test runner.
|
|
448
435
|
- **RSpec adapter**: uses `RSpec::Core::Runner.run` in-process with `RSpec.clear_examples` between batches to reset example state while preserving configuration. No subprocess forking overhead.
|
data/lib/specbandit/cli.rb
CHANGED
|
@@ -63,7 +63,7 @@ module Specbandit
|
|
|
63
63
|
Specbandit.configuration.redis_url = v
|
|
64
64
|
end
|
|
65
65
|
|
|
66
|
-
opts.on('--key-ttl SECONDS', Integer, 'TTL for
|
|
66
|
+
opts.on('--key-ttl SECONDS', Integer, 'TTL for all Redis keys in seconds (default: 604800 / 1 week)') do |v|
|
|
67
67
|
Specbandit.configuration.key_ttl = v
|
|
68
68
|
end
|
|
69
69
|
|
|
@@ -119,20 +119,12 @@ module Specbandit
|
|
|
119
119
|
Specbandit.configuration.key_rerun = v
|
|
120
120
|
end
|
|
121
121
|
|
|
122
|
-
opts.on('--key-rerun-ttl SECONDS', Integer, 'TTL for rerun key in seconds (default: 604800 / 1 week)') do |v|
|
|
123
|
-
Specbandit.configuration.key_rerun_ttl = v
|
|
124
|
-
end
|
|
125
|
-
|
|
126
122
|
opts.on('--key-failed KEY', 'Redis key to record failed test files for later review') do |v|
|
|
127
123
|
Specbandit.configuration.key_failed = v
|
|
128
124
|
end
|
|
129
125
|
|
|
130
|
-
opts.on('--key-
|
|
131
|
-
Specbandit.configuration.
|
|
132
|
-
end
|
|
133
|
-
|
|
134
|
-
opts.on('--rerun', 'Signal this is a re-run (fail hard if rerun key is empty)') do
|
|
135
|
-
Specbandit.configuration.rerun = true
|
|
126
|
+
opts.on('--key-ttl SECONDS', Integer, 'TTL for all Redis keys in seconds (default: 604800 / 1 week)') do |v|
|
|
127
|
+
Specbandit.configuration.key_ttl = v
|
|
136
128
|
end
|
|
137
129
|
|
|
138
130
|
opts.on('--report FILE', 'Write JSON report with run statistics to FILE') do |v|
|
|
@@ -209,7 +201,7 @@ module Specbandit
|
|
|
209
201
|
--key KEY Redis queue key (required, or set SPECBANDIT_KEY)
|
|
210
202
|
--pattern PATTERN Glob pattern for file discovery (e.g. 'spec/**/*_spec.rb')
|
|
211
203
|
--redis-url URL Redis URL (default: redis://localhost:6379)
|
|
212
|
-
--key-ttl SECONDS TTL for
|
|
204
|
+
--key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
|
|
213
205
|
|
|
214
206
|
Work options:
|
|
215
207
|
--key KEY Redis queue key (required, or set SPECBANDIT_KEY)
|
|
@@ -219,11 +211,9 @@ module Specbandit
|
|
|
219
211
|
--rspec-opts OPTS Extra options forwarded to RSpec (for rspec adapter)
|
|
220
212
|
--batch-size N Files per batch (default: 5, or set SPECBANDIT_BATCH_SIZE)
|
|
221
213
|
--redis-url URL Redis URL (default: redis://localhost:6379)
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
--key-failed-ttl N TTL for failed key (default: 604800 / 1 week)
|
|
226
|
-
--rerun Signal this is a re-run (fail hard if rerun key is empty)
|
|
214
|
+
--key-rerun KEY Per-runner rerun key for re-run support
|
|
215
|
+
--key-failed KEY Redis key to record failed test files
|
|
216
|
+
--key-ttl SECONDS TTL for all Redis keys (default: 604800 / 1 week)
|
|
227
217
|
--report FILE Write JSON report to FILE after run
|
|
228
218
|
--verbose Show per-batch file list and full command output
|
|
229
219
|
|
|
@@ -237,13 +227,10 @@ module Specbandit
|
|
|
237
227
|
SPECBANDIT_COMMAND Command to run (cli adapter)
|
|
238
228
|
SPECBANDIT_COMMAND_OPTS Command options (space-separated)
|
|
239
229
|
SPECBANDIT_BATCH_SIZE Batch size
|
|
240
|
-
SPECBANDIT_KEY_TTL
|
|
230
|
+
SPECBANDIT_KEY_TTL TTL for all Redis keys in seconds (default: 604800)
|
|
241
231
|
SPECBANDIT_RSPEC_OPTS RSpec options (rspec adapter)
|
|
242
232
|
SPECBANDIT_KEY_RERUN Per-runner rerun key
|
|
243
|
-
SPECBANDIT_KEY_RERUN_TTL Rerun key TTL in seconds (default: 604800)
|
|
244
233
|
SPECBANDIT_KEY_FAILED Redis key for failed test files
|
|
245
|
-
SPECBANDIT_KEY_FAILED_TTL Failed key TTL in seconds (default: 604800)
|
|
246
|
-
SPECBANDIT_RERUN Signal re-run mode (1/true/yes)
|
|
247
234
|
SPECBANDIT_VERBOSE Enable verbose output (1/true/yes)
|
|
248
235
|
SPECBANDIT_REPORT Path to write JSON report file
|
|
249
236
|
|
|
@@ -3,15 +3,17 @@
|
|
|
3
3
|
module Specbandit
|
|
4
4
|
class Configuration
|
|
5
5
|
attr_accessor :redis_url, :batch_size, :key, :rspec_opts, :key_ttl,
|
|
6
|
-
:key_rerun, :
|
|
6
|
+
:key_rerun, :verbose,
|
|
7
7
|
:adapter, :command, :command_opts,
|
|
8
|
-
:key_failed, :
|
|
8
|
+
:key_failed, :report
|
|
9
9
|
|
|
10
10
|
DEFAULT_REDIS_URL = 'redis://localhost:6379'
|
|
11
11
|
DEFAULT_BATCH_SIZE = 5
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
12
|
+
# A single TTL governs every key specbandit writes: the shared queue, its
|
|
13
|
+
# published marker, the per-runner rerun key and the failed key. It defaults
|
|
14
|
+
# to 1 week because re-runs can happen hours or days after the original run,
|
|
15
|
+
# and the rerun key + published marker must still be alive when they do.
|
|
16
|
+
DEFAULT_KEY_TTL = 604_800 # 1 week in seconds
|
|
15
17
|
DEFAULT_ADAPTER = 'cli'
|
|
16
18
|
|
|
17
19
|
def initialize
|
|
@@ -21,14 +23,11 @@ module Specbandit
|
|
|
21
23
|
@rspec_opts = parse_rspec_opts(ENV.fetch('SPECBANDIT_RSPEC_OPTS', nil))
|
|
22
24
|
@key_ttl = Integer(ENV.fetch('SPECBANDIT_KEY_TTL', DEFAULT_KEY_TTL))
|
|
23
25
|
@key_rerun = ENV.fetch('SPECBANDIT_KEY_RERUN', nil)
|
|
24
|
-
@key_rerun_ttl = Integer(ENV.fetch('SPECBANDIT_KEY_RERUN_TTL', DEFAULT_KEY_RERUN_TTL))
|
|
25
|
-
@rerun = env_truthy?('SPECBANDIT_RERUN')
|
|
26
26
|
@verbose = env_truthy?('SPECBANDIT_VERBOSE')
|
|
27
27
|
@adapter = ENV.fetch('SPECBANDIT_ADAPTER', DEFAULT_ADAPTER)
|
|
28
28
|
@command = ENV.fetch('SPECBANDIT_COMMAND', nil)
|
|
29
29
|
@command_opts = parse_space_separated(ENV.fetch('SPECBANDIT_COMMAND_OPTS', nil))
|
|
30
30
|
@key_failed = ENV.fetch('SPECBANDIT_KEY_FAILED', nil)
|
|
31
|
-
@key_failed_ttl = Integer(ENV.fetch('SPECBANDIT_KEY_FAILED_TTL', DEFAULT_KEY_FAILED_TTL))
|
|
32
31
|
@report = ENV.fetch('SPECBANDIT_REPORT', nil)
|
|
33
32
|
end
|
|
34
33
|
|
|
@@ -36,9 +35,6 @@ module Specbandit
|
|
|
36
35
|
raise Error, 'key is required (set via --key or SPECBANDIT_KEY)' if key.nil? || key.empty?
|
|
37
36
|
raise Error, 'batch_size must be a positive integer' unless batch_size.positive?
|
|
38
37
|
raise Error, 'key_ttl must be a positive integer' unless key_ttl.positive?
|
|
39
|
-
raise Error, 'key_rerun_ttl must be a positive integer' unless key_rerun_ttl.positive?
|
|
40
|
-
raise Error, 'key_failed_ttl must be a positive integer' unless key_failed_ttl.positive?
|
|
41
|
-
raise Error, '--rerun requires --key-rerun to be set' if rerun && (key_rerun.nil? || key_rerun.empty?)
|
|
42
38
|
end
|
|
43
39
|
|
|
44
40
|
private
|
data/lib/specbandit/publisher.rb
CHANGED
|
@@ -26,13 +26,28 @@ module Specbandit
|
|
|
26
26
|
return 0
|
|
27
27
|
end
|
|
28
28
|
|
|
29
|
-
queue.push(key, resolved, ttl: key_ttl)
|
|
29
|
+
push_ms = measure { queue.push(key, resolved, ttl: key_ttl) }
|
|
30
|
+
# Record a durable "published" marker so workers can tell a drained
|
|
31
|
+
# queue ("worker arriving late", OK) apart from one that was never
|
|
32
|
+
# pushed ("you didn't push work", crash). Redis auto-deletes empty
|
|
33
|
+
# lists, so the list itself cannot carry this signal.
|
|
34
|
+
mark_ms = measure { queue.mark_published(key, ttl: key_ttl) }
|
|
35
|
+
|
|
30
36
|
output.puts "[specbandit] Enqueued #{resolved.size} files onto key '#{key}' (TTL: #{key_ttl}s)."
|
|
37
|
+
output.puts format('[specbandit] Redis latency: push %.1fms, mark published %.1fms.', push_ms, mark_ms)
|
|
31
38
|
resolved.size
|
|
32
39
|
end
|
|
33
40
|
|
|
34
41
|
private
|
|
35
42
|
|
|
43
|
+
# Run the block and return the wall-clock time it took, in milliseconds.
|
|
44
|
+
# Used to surface how long the Redis round-trips took in the push log.
|
|
45
|
+
def measure
|
|
46
|
+
start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
47
|
+
yield
|
|
48
|
+
(Process.clock_gettime(Process::CLOCK_MONOTONIC) - start) * 1000
|
|
49
|
+
end
|
|
50
|
+
|
|
36
51
|
def resolve_files(files:, pattern:)
|
|
37
52
|
# Priority 1: stdin (only when data is actually piped in)
|
|
38
53
|
if !$stdin.tty? && $stdin.ready?
|
|
@@ -45,6 +45,27 @@ module Specbandit
|
|
|
45
45
|
with_retries { redis.llen(key) }
|
|
46
46
|
end
|
|
47
47
|
|
|
48
|
+
# Mark a queue key as "published" by setting a companion marker key.
|
|
49
|
+
#
|
|
50
|
+
# The marker is a separate string key (`<key>:published`) that survives
|
|
51
|
+
# the queue list being fully drained. It is the source of truth for
|
|
52
|
+
# "was work ever pushed for this key?" -- Redis auto-deletes empty lists,
|
|
53
|
+
# so the list alone cannot distinguish "never pushed" from "drained".
|
|
54
|
+
def mark_published(key, ttl: nil)
|
|
55
|
+
with_retries do
|
|
56
|
+
if ttl
|
|
57
|
+
redis.set(published_marker(key), '1', ex: ttl)
|
|
58
|
+
else
|
|
59
|
+
redis.set(published_marker(key), '1')
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
# Whether a queue key has been published (its marker exists).
|
|
65
|
+
def published?(key)
|
|
66
|
+
with_retries { redis.exists?(published_marker(key)) }
|
|
67
|
+
end
|
|
68
|
+
|
|
48
69
|
# Read all file paths from the list non-destructively.
|
|
49
70
|
# Returns an array of file paths (empty array when key doesn't exist).
|
|
50
71
|
def read_all(key)
|
|
@@ -57,6 +78,11 @@ module Specbandit
|
|
|
57
78
|
|
|
58
79
|
private
|
|
59
80
|
|
|
81
|
+
# Companion marker key name for a given queue key.
|
|
82
|
+
def published_marker(key)
|
|
83
|
+
"#{key}:published"
|
|
84
|
+
end
|
|
85
|
+
|
|
60
86
|
def with_retries(attempts: MAX_ATTEMPTS)
|
|
61
87
|
retries = 0
|
|
62
88
|
begin
|
data/lib/specbandit/version.rb
CHANGED
data/lib/specbandit/worker.rb
CHANGED
|
@@ -4,7 +4,7 @@ require 'json'
|
|
|
4
4
|
|
|
5
5
|
module Specbandit
|
|
6
6
|
class Worker
|
|
7
|
-
attr_reader :queue, :key, :batch_size, :adapter, :key_rerun, :
|
|
7
|
+
attr_reader :queue, :key, :batch_size, :adapter, :key_rerun, :key_ttl, :key_failed,
|
|
8
8
|
:output, :verbose, :report
|
|
9
9
|
|
|
10
10
|
def initialize(
|
|
@@ -12,10 +12,8 @@ module Specbandit
|
|
|
12
12
|
batch_size: Specbandit.configuration.batch_size,
|
|
13
13
|
adapter: nil,
|
|
14
14
|
key_rerun: Specbandit.configuration.key_rerun,
|
|
15
|
-
|
|
15
|
+
key_ttl: Specbandit.configuration.key_ttl,
|
|
16
16
|
key_failed: Specbandit.configuration.key_failed,
|
|
17
|
-
key_failed_ttl: Specbandit.configuration.key_failed_ttl,
|
|
18
|
-
rerun: Specbandit.configuration.rerun,
|
|
19
17
|
verbose: Specbandit.configuration.verbose,
|
|
20
18
|
report: Specbandit.configuration.report,
|
|
21
19
|
queue: nil,
|
|
@@ -27,10 +25,8 @@ module Specbandit
|
|
|
27
25
|
@key = key
|
|
28
26
|
@batch_size = batch_size
|
|
29
27
|
@key_rerun = key_rerun
|
|
30
|
-
@
|
|
28
|
+
@key_ttl = key_ttl
|
|
31
29
|
@key_failed = key_failed
|
|
32
|
-
@key_failed_ttl = key_failed_ttl
|
|
33
|
-
@rerun = rerun
|
|
34
30
|
@verbose = verbose
|
|
35
31
|
@report = report
|
|
36
32
|
@queue = queue || RedisQueue.new
|
|
@@ -69,22 +65,42 @@ module Specbandit
|
|
|
69
65
|
|
|
70
66
|
private
|
|
71
67
|
|
|
72
|
-
# Decide the operating mode
|
|
73
|
-
#
|
|
74
|
-
#
|
|
75
|
-
#
|
|
68
|
+
# Decide the operating mode from the state of Redis and execute it.
|
|
69
|
+
#
|
|
70
|
+
# The mode is derived entirely from Redis -- never from an environment
|
|
71
|
+
# variable or flag -- following this table:
|
|
72
|
+
#
|
|
73
|
+
# Published | Key (queue) | Rerun key | Behavior
|
|
74
|
+
# ----------|----------------|-------------|-------------------------------
|
|
75
|
+
# No | -- | -- | Crash: nothing was ever pushed
|
|
76
|
+
# Yes | empty/drained | empty | OK: worker arriving late (0)
|
|
77
|
+
# Yes | has data | empty | Steal (record if rerun key set)
|
|
78
|
+
# Yes | empty/drained | has data | Replay recorded files
|
|
79
|
+
# Yes | has data | has data | Crash: inconsistent (weird case)
|
|
80
|
+
#
|
|
81
|
+
# "Published" is a durable marker written by `specbandit push`; it is the
|
|
82
|
+
# only reliable signal that work was ever enqueued, because Redis
|
|
83
|
+
# auto-deletes empty lists (so a drained queue is indistinguishable from a
|
|
84
|
+
# never-created one by the list alone).
|
|
76
85
|
def determine_exit_code
|
|
77
|
-
unless
|
|
78
|
-
return fail_stale_rerun if rerun
|
|
86
|
+
return fail_not_published unless queue.published?(key)
|
|
79
87
|
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
rerun_files = queue.read_all(key_rerun)
|
|
84
|
-
return run_replay(rerun_files) if rerun_files.any?
|
|
85
|
-
return fail_stale_rerun if rerun
|
|
88
|
+
key_has_data = queue.length(key).positive?
|
|
89
|
+
rerun_files = key_present?(key_rerun) ? queue.read_all(key_rerun) : []
|
|
86
90
|
|
|
87
|
-
|
|
91
|
+
if key_has_data && rerun_files.any?
|
|
92
|
+
fail_inconsistent_state
|
|
93
|
+
elsif rerun_files.any?
|
|
94
|
+
run_replay(rerun_files)
|
|
95
|
+
elsif key_has_data
|
|
96
|
+
run_steal(record: key_present?(key_rerun))
|
|
97
|
+
else
|
|
98
|
+
# Published, but the queue is drained and this runner has no rerun
|
|
99
|
+
# memory: it simply arrived after everything was already taken.
|
|
100
|
+
output.puts "[specbandit] Queue '#{key}' already drained and no rerun files to replay. " \
|
|
101
|
+
'Nothing to do (worker arriving late).' if verbose
|
|
102
|
+
0
|
|
103
|
+
end
|
|
88
104
|
end
|
|
89
105
|
|
|
90
106
|
# A Redis key name is usable only when it is a non-nil, non-empty string.
|
|
@@ -94,15 +110,22 @@ module Specbandit
|
|
|
94
110
|
!value.nil? && !value.empty?
|
|
95
111
|
end
|
|
96
112
|
|
|
97
|
-
#
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
113
|
+
# No published marker for this key: `specbandit push` was never run for it,
|
|
114
|
+
# or the key expired. Crash rather than silently pass with zero tests.
|
|
115
|
+
def fail_not_published
|
|
116
|
+
output.puts "[specbandit] ERROR: queue '#{key}' was never published."
|
|
117
|
+
output.puts '[specbandit] Run `specbandit push` before `specbandit work`, or the key/TTL may have expired.'
|
|
118
|
+
output.puts '[specbandit] Refusing to run to prevent a silent false pass.'
|
|
119
|
+
1
|
|
120
|
+
end
|
|
121
|
+
|
|
122
|
+
# Both the shared queue and this runner's rerun key hold files at once.
|
|
123
|
+
# That should never happen: a fresh run has no rerun memory yet, and a
|
|
124
|
+
# re-run reads from a drained queue. Crash instead of double-executing.
|
|
125
|
+
def fail_inconsistent_state
|
|
126
|
+
output.puts "[specbandit] ERROR: inconsistent state — shared queue '#{key}' still has files " \
|
|
127
|
+
"while rerun key '#{key_rerun}' also has recorded files."
|
|
128
|
+
output.puts '[specbandit] Refusing to run to avoid double-execution / undefined behavior.'
|
|
106
129
|
1
|
|
107
130
|
end
|
|
108
131
|
|
|
@@ -159,7 +182,7 @@ module Specbandit
|
|
|
159
182
|
end
|
|
160
183
|
|
|
161
184
|
# Record the stolen batch so this runner can replay on re-run
|
|
162
|
-
queue.push(key_rerun, files, ttl:
|
|
185
|
+
queue.push(key_rerun, files, ttl: key_ttl) if record
|
|
163
186
|
|
|
164
187
|
batch_num += 1
|
|
165
188
|
output.puts "[specbandit] Batch ##{batch_num}: running #{files.size} files" if verbose
|
|
@@ -200,7 +223,7 @@ module Specbandit
|
|
|
200
223
|
failed_files = extract_failed_files(result) || files
|
|
201
224
|
return if failed_files.empty?
|
|
202
225
|
|
|
203
|
-
queue.push(key_failed, failed_files, ttl:
|
|
226
|
+
queue.push(key_failed, failed_files, ttl: key_ttl)
|
|
204
227
|
end
|
|
205
228
|
|
|
206
229
|
# Extract individual failed file paths from an RspecBatchResult's JSON output.
|