parallel_matrix_formatter 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +35 -0
- data/README.md +99 -14
- data/config/parallel_matrix_formatter.yml +5 -0
- data/lib/parallel_matrix_formatter/error.rb +5 -0
- data/lib/parallel_matrix_formatter/formatter.rb +48 -12
- data/lib/parallel_matrix_formatter/ipc/client.rb +1 -1
- data/lib/parallel_matrix_formatter/ipc/server.rb +28 -5
- data/lib/parallel_matrix_formatter/ipc.rb +3 -4
- data/lib/parallel_matrix_formatter/orchestrator.rb +46 -9
- data/lib/parallel_matrix_formatter/output/silencer.rb +20 -5
- data/lib/parallel_matrix_formatter/rendering/progress_update_policy.rb +1 -1
- data/lib/parallel_matrix_formatter/rendering/summary.rb +14 -1
- data/lib/parallel_matrix_formatter/runner.rb +45 -0
- data/lib/parallel_matrix_formatter/silence.rb +1 -0
- data/lib/parallel_matrix_formatter/version.rb +1 -1
- data/lib/parallel_matrix_formatter.rb +7 -24
- metadata +11 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 87461fe9450a052070b6692e41304d365cf182ff6c3ed24a54c763dac8921e60
|
|
4
|
+
data.tar.gz: 2249990d6263c6021be4b68916868371ad883eacbd56cfc8ceb79df4a86524fe
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c2b8bc1f533a22055fae037c4ddae821852e7e3b27b5afc522cb98a1258a101a52c018eec59f6c43515fa1bb3694198662b75281f60d3ddfa87804b37e7b3eab
|
|
7
|
+
data.tar.gz: 05b0b6dae008cba2545a9b8dd0fd4eef58d65b538338ecb3c5596bf328746433b178f96d32478e853978ad9d7ee8bb4befcfa8183673eca2808dc1b378df8a0b
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,41 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [0.2.0] - 2026-10-03
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
- Support for [`parallel_tests`](https://github.com/grosser/parallel_tests) (`parallel_rspec`) next to
|
|
14
|
+
`parallel_split_test`. The formatter detects the runner and takes the number of processes and the identity of
|
|
15
|
+
the run from it; the README lists the `parallel_rspec` options that are not supported.
|
|
16
|
+
|
|
17
|
+
### Fixed
|
|
18
|
+
- The stderr of every process is written to a log file in the temporary directory instead of `/dev/null`; the
|
|
19
|
+
summary lists the logs when a process is missing, an error happened outside of the examples or a log is not
|
|
20
|
+
empty.
|
|
21
|
+
- The closing `Summary:` of `parallel_split_test` is no longer empty: every process writes its totals line to the
|
|
22
|
+
stream RSpec gave its formatter, which `parallel_split_test` records.
|
|
23
|
+
- Process 1 no longer waits forever for a process that died before its first example: every process announces
|
|
24
|
+
itself as soon as it connects. Without a pid file (`parallel_split_test`) process 1 stops waiting for a process
|
|
25
|
+
that has not connected within the new `connect_timeout_seconds` setting (default 120, the timeout the processes
|
|
26
|
+
already used to connect) and reports it as missing.
|
|
27
|
+
- Under `parallel_tests` the summary could leave out a fast process: it had already left the pid file while its
|
|
28
|
+
messages were still unread. The orchestrator now also waits until every connection has been read to the end.
|
|
29
|
+
- Without a runner (plain `rspec`) the socket is named after the process's own pid instead of its parent's, so
|
|
30
|
+
two `rspec` runs started from the same shell no longer remove each other's socket.
|
|
31
|
+
- `--format ParallelMatrixFormatter::Formatter` works without requiring the gem first; the formatter file now
|
|
32
|
+
loads everything it needs.
|
|
33
|
+
|
|
34
|
+
### Changed
|
|
35
|
+
- README: documents other formatters writing to stdout, the `--out` handling of `parallel_split_test` and the
|
|
36
|
+
default colors.
|
|
37
|
+
- The orchestrator waits for the processes that actually connected instead of trusting the announced number of
|
|
38
|
+
processes. Under `parallel_tests` it also waits for every process listed in the run's pid file, so it neither
|
|
39
|
+
hangs when `parallel_tests` starts fewer processes than announced nor finishes before a slow process reports.
|
|
40
|
+
- The socket is named after the run (the pid file under `parallel_tests`, the parent pid under `parallel_split_test`);
|
|
41
|
+
`Ipc.socket_path`, `Ipc::Server.new` and `Ipc::Client.connect` take the path explicitly.
|
|
42
|
+
|
|
8
43
|
## [0.1.0] - 2026-10-03
|
|
9
44
|
|
|
10
45
|
### Added
|
data/README.md
CHANGED
|
@@ -1,9 +1,29 @@
|
|
|
1
1
|
# ParallelMatrixFormatter
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
3
|
+
[](https://rubygems.org/gems/parallel_matrix_formatter)
|
|
4
|
+
[](https://github.com/vovka/parallel_matrix_formatter/actions/workflows/ci.yml)
|
|
5
|
+
[](LICENSE.txt)
|
|
6
|
+
|
|
7
|
+
Parallel RSpec output turns into unreadable interleaved noise. This formatter merges every process into one live
|
|
8
|
+
progress display and one consolidated failure report, rendered as Matrix digital rain.
|
|
9
|
+
|
|
10
|
+

|
|
11
|
+
|
|
12
|
+
An RSpec formatter for suites run with [`parallel_split_test`](https://github.com/grosser/parallel_split_test)
|
|
13
|
+
or [`parallel_tests`](https://github.com/grosser/parallel_tests). It prints a progress line with the percentage of
|
|
14
|
+
each process surrounded by falling katakana "rain", a colored symbol for every finished example, and, at the end, a
|
|
15
|
+
single RSpec-style summary with all failures from all processes.
|
|
16
|
+
|
|
17
|
+
## Quick start
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
bundle add parallel_matrix_formatter --group test
|
|
21
|
+
bundle exec parallel_split_test --format ParallelMatrixFormatter::Formatter spec
|
|
22
|
+
# or
|
|
23
|
+
bundle exec parallel_rspec -o "--format ParallelMatrixFormatter::Formatter" spec
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
See [Installation](#installation) and [Usage](#usage) for details.
|
|
7
27
|
|
|
8
28
|
## What you are looking at
|
|
9
29
|
|
|
@@ -34,7 +54,7 @@ When the last process finishes, the rain stops and one consolidated report follo
|
|
|
34
54
|
the failures of all processes with their messages in red and backtraces in cyan, the wall-clock time next to the
|
|
35
55
|
time summed across processes, the totals (red when something failed, yellow when examples are only pending, green
|
|
36
56
|
otherwise) and the commands to rerun the failures. The first line and the closing `Summary:` block are printed by
|
|
37
|
-
`parallel_split_test` itself.
|
|
57
|
+
`parallel_split_test` itself (`parallel_tests` prints its own first line and totals in the same places).
|
|
38
58
|
|
|
39
59
|

|
|
40
60
|
|
|
@@ -55,15 +75,30 @@ gem 'parallel_matrix_formatter', group: :test
|
|
|
55
75
|
|
|
56
76
|
and run `bundle install`.
|
|
57
77
|
|
|
58
|
-
Requirements: Ruby 3.2 or newer, `rspec-core` 3.x
|
|
59
|
-
supported; Windows is
|
|
78
|
+
Requirements: Ruby 3.2 or newer, `rspec-core` 3.x, and either `parallel_split_test` or `parallel_tests` (or
|
|
79
|
+
neither, for a single process). The processes talk over UNIX sockets, so Linux and macOS are supported; Windows is
|
|
80
|
+
not.
|
|
60
81
|
|
|
61
82
|
## Usage
|
|
62
83
|
|
|
84
|
+
With `parallel_split_test`:
|
|
85
|
+
|
|
63
86
|
```sh
|
|
64
87
|
bundle exec parallel_split_test --format ParallelMatrixFormatter::Formatter spec
|
|
65
88
|
```
|
|
66
89
|
|
|
90
|
+
With `parallel_tests`:
|
|
91
|
+
|
|
92
|
+
```sh
|
|
93
|
+
bundle exec parallel_rspec -o "--format ParallelMatrixFormatter::Formatter" spec
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
or, to make it the default, put the option in a `.rspec_parallel` file, which `parallel_rspec` reads:
|
|
97
|
+
|
|
98
|
+
```
|
|
99
|
+
--format ParallelMatrixFormatter::Formatter
|
|
100
|
+
```
|
|
101
|
+
|
|
67
102
|
The formatter also works with a single process:
|
|
68
103
|
|
|
69
104
|
```sh
|
|
@@ -103,6 +138,10 @@ The full schema, with the default values:
|
|
|
103
138
|
# Redirect STDOUT and STDERR of every test process to /dev/null.
|
|
104
139
|
suppress_output: true
|
|
105
140
|
|
|
141
|
+
# How long the processes wait to connect to process 1, and process 1 waits for them. A process that has not
|
|
142
|
+
# connected by then is reported as missing.
|
|
143
|
+
connect_timeout_seconds: 120
|
|
144
|
+
|
|
106
145
|
# Ten characters that replace the digits 0-9 in the time and the percentages.
|
|
107
146
|
# Leave empty to keep plain digits.
|
|
108
147
|
digits: ""
|
|
@@ -137,7 +176,8 @@ example_status:
|
|
|
137
176
|
```
|
|
138
177
|
|
|
139
178
|
Colors can be any name known to RSpec's console codes: `black`, `red`, `green`, `yellow`, `blue`, `magenta`,
|
|
140
|
-
`cyan`, `white` and their `bold_*` variants. Colors are always emitted,
|
|
179
|
+
`cyan`, `white` and their `bold_*` variants. Colors are always emitted, even when stdout is not a terminal, because
|
|
180
|
+
CI logs render ANSI codes; set the
|
|
141
181
|
`NO_COLOR` environment variable to turn them off.
|
|
142
182
|
|
|
143
183
|
Example override: emoji for the examples, and a progress line whenever a process advances by 10 percent
|
|
@@ -166,10 +206,21 @@ deprecation warnings, output of C extensions and child processes therefore canno
|
|
|
166
206
|
|
|
167
207
|
```sh
|
|
168
208
|
RUBYOPT="-rparallel_matrix_formatter/silence" bundle exec parallel_split_test --format ParallelMatrixFormatter::Formatter spec
|
|
209
|
+
RUBYOPT="-rparallel_matrix_formatter/silence" bundle exec parallel_rspec -o "--format ParallelMatrixFormatter::Formatter" spec
|
|
169
210
|
```
|
|
170
211
|
|
|
171
|
-
-
|
|
172
|
-
|
|
212
|
+
- STDOUT goes to `/dev/null`, but STDERR of every process goes to a log file in the temporary directory
|
|
213
|
+
(`parallel_matrix_formatter-<run>-<process>.stderr.log`), so crashes, load errors and warnings are not lost.
|
|
214
|
+
When a process is missing, an error happened outside of the examples or a log is not empty, the summary lists
|
|
215
|
+
the logs. A process that dies without reporting is also named in a yellow warning.
|
|
216
|
+
- The closing `Summary:` of `parallel_split_test` is built from what each process writes to the stream RSpec
|
|
217
|
+
hands to its formatter, so every process writes its own totals line (`3 examples, 0 failures`) to that stream,
|
|
218
|
+
which is recorded but not displayed. `parallel_tests` totals the `N examples, M failures` lines it reads from
|
|
219
|
+
stdout; the consolidated totals the display prints in process 1 are the only such line, so its total is right.
|
|
220
|
+
- Another formatter writing to stdout (for example `--format progress`) is silenced too. Send it to a file:
|
|
221
|
+
`--format progress --out progress.txt`. `parallel_split_test` only rewrites the first `-o`/`--out` per process
|
|
222
|
+
(to `name.<process>.ext`, merged afterwards), so a second formatter with `--out` may be written by every
|
|
223
|
+
process to the same file.
|
|
173
224
|
- If you pass `--out FILE` to RSpec, the display is written to that file instead of the terminal.
|
|
174
225
|
|
|
175
226
|
## How it works
|
|
@@ -178,7 +229,37 @@ Every test process loads the formatter. Process 1 (`TEST_ENV_NUMBER` empty or `1
|
|
|
178
229
|
orchestrator, which listens on a UNIX socket in the temporary directory (one per run). The other processes
|
|
179
230
|
connect to it and send the result of every example and, at the end, a summary of their run. The orchestrator
|
|
180
231
|
renders the progress lines and status symbols as messages arrive. When every process has sent its summary or has
|
|
181
|
-
disconnected,
|
|
232
|
+
disconnected (or, without a pid file to consult, has not connected within `connect_timeout_seconds`), it prints
|
|
233
|
+
the consolidated summary.
|
|
234
|
+
|
|
235
|
+
The formatter detects the runner it is started by:
|
|
236
|
+
|
|
237
|
+
| Runner | Number of processes | Identifies the run (socket name) |
|
|
238
|
+
| --- | --- | --- |
|
|
239
|
+
| `parallel_split_test` | `ParallelSplitTest.processes` | pid of the parent process |
|
|
240
|
+
| `parallel_tests` | `PARALLEL_TEST_GROUPS` | name of the `PARALLEL_PID_FILE` |
|
|
241
|
+
| none | 1 | pid of the process |
|
|
242
|
+
|
|
243
|
+
`parallel_tests` can start fewer processes than it announces: it drops empty groups (more processes than spec
|
|
244
|
+
files) without correcting `PARALLEL_TEST_GROUPS`. So under `parallel_tests` the orchestrator does not wait for
|
|
245
|
+
the announced number. It waits for the processes that connected, and for every process still listed in the
|
|
246
|
+
pid file that `parallel_tests` keeps for the run, so it neither hangs for a process that never existed nor
|
|
247
|
+
finishes before a slow one reports.
|
|
248
|
+
|
|
249
|
+
## Using it with parallel_tests
|
|
250
|
+
|
|
251
|
+
These `parallel_rspec` options are not supported, because they change how the output of the processes reaches
|
|
252
|
+
the terminal or how many times a process runs:
|
|
253
|
+
|
|
254
|
+
- `--serialize-stdout` holds back the output of process 1 until it has finished, so the display is not live.
|
|
255
|
+
- `--prefix-output-with-test-env-number` prefixes every chunk of output, which garbles the display.
|
|
256
|
+
- `--test-file-limit` runs several RSpec processes one after another under the same process number.
|
|
257
|
+
- `--only-group-continuous-test-env` numbers the processes after their group, so there may be no process 1 to
|
|
258
|
+
host the display.
|
|
259
|
+
|
|
260
|
+
A process that dies before RSpec has loaded the formatter (for example because of a syntax error in a required
|
|
261
|
+
file) never connects, so the yellow warning about missing processes cannot name it. `parallel_rspec` still
|
|
262
|
+
exits with a failure status.
|
|
182
263
|
|
|
183
264
|
## Development
|
|
184
265
|
|
|
@@ -189,10 +270,14 @@ bundle exec rspec # runs the specs in one process with RSpec's own forma
|
|
|
189
270
|
ruby demo/matrix_demo.rb # previews the display without a test suite
|
|
190
271
|
```
|
|
191
272
|
|
|
192
|
-
The specs eat their own dog food: `rake` (and CI) runs them with `parallel_split_test` and this formatter
|
|
193
|
-
|
|
273
|
+
The specs eat their own dog food: `rake` (and CI) runs them with `parallel_split_test` and this formatter, as
|
|
274
|
+
released on RubyGems. The release is installed into `tmp/` on the first run and renders the report, so a bug in
|
|
275
|
+
the code under test cannot garble the report of its own specs; its version is pinned in
|
|
276
|
+
[`spec/support/released_formatter.rb`](spec/support/released_formatter.rb). CI also sets
|
|
277
|
+
[`.github/parallel_matrix_formatter.yml`](.github/parallel_matrix_formatter.yml), which prints a progress line
|
|
194
278
|
whenever a process advances by 10 percent; to see the same locally:
|
|
195
|
-
`PARALLEL_MATRIX_FORMATTER_CONFIG=.github/parallel_matrix_formatter.yml bundle exec rake`.
|
|
279
|
+
`PARALLEL_MATRIX_FORMATTER_CONFIG=.github/parallel_matrix_formatter.yml bundle exec rake`. The integration specs
|
|
280
|
+
run the formatter for real under both `parallel_split_test` (faked by a stub) and `parallel_rspec`.
|
|
196
281
|
|
|
197
282
|
## Contributing
|
|
198
283
|
|
|
@@ -9,6 +9,11 @@
|
|
|
9
9
|
# when something goes wrong and you need to see why.
|
|
10
10
|
suppress_output: true
|
|
11
11
|
|
|
12
|
+
# How long a process waits for process 1 to start listening, and process 1
|
|
13
|
+
# waits for the others to connect. A process that never connects (for example
|
|
14
|
+
# because spec_helper failed to load) is reported as missing once it passes.
|
|
15
|
+
connect_timeout_seconds: 120
|
|
16
|
+
|
|
12
17
|
# Ten characters that replace the digits 0-9 in the time and the percentages,
|
|
13
18
|
# e.g. "ロイクヨムラレヌメワ". Leave empty to keep plain digits.
|
|
14
19
|
digits: ""
|
|
@@ -2,6 +2,26 @@
|
|
|
2
2
|
|
|
3
3
|
require 'rspec/core/formatters/base_formatter'
|
|
4
4
|
|
|
5
|
+
# RSpec requires only this file for --format ParallelMatrixFormatter::Formatter,
|
|
6
|
+
# so it loads everything the formatter needs.
|
|
7
|
+
require_relative 'error'
|
|
8
|
+
require_relative 'config'
|
|
9
|
+
require_relative 'output/silencer'
|
|
10
|
+
require_relative 'runner'
|
|
11
|
+
require_relative 'ipc'
|
|
12
|
+
require_relative 'ipc/client'
|
|
13
|
+
require_relative 'ipc/server'
|
|
14
|
+
require_relative 'rendering/colors'
|
|
15
|
+
require_relative 'rendering/digits'
|
|
16
|
+
require_relative 'rendering/progress_update_policy'
|
|
17
|
+
require_relative 'rendering/progress_column'
|
|
18
|
+
require_relative 'rendering/progress_line'
|
|
19
|
+
require_relative 'rendering/example_status'
|
|
20
|
+
require_relative 'rendering/summary'
|
|
21
|
+
require_relative 'rendering/display'
|
|
22
|
+
require_relative 'null_orchestrator'
|
|
23
|
+
require_relative 'orchestrator'
|
|
24
|
+
|
|
5
25
|
module ParallelMatrixFormatter
|
|
6
26
|
# The RSpec formatter loaded into every test process. It silences the
|
|
7
27
|
# process, turns RSpec notifications into IPC messages for the orchestrator
|
|
@@ -13,15 +33,21 @@ module ParallelMatrixFormatter
|
|
|
13
33
|
def initialize(output)
|
|
14
34
|
config = Config.load
|
|
15
35
|
super(display_output(config, output))
|
|
36
|
+
@connect_timeout = config['connect_timeout_seconds']
|
|
37
|
+
@stream = output
|
|
16
38
|
@process_number = [ENV['TEST_ENV_NUMBER'].to_i, 1].max
|
|
17
|
-
@
|
|
39
|
+
@runner = Runner.detect
|
|
40
|
+
@orchestrator = Orchestrator.for(@process_number, @runner, self.output, config)
|
|
18
41
|
@examples_run = 0
|
|
19
42
|
@failures = []
|
|
20
43
|
end
|
|
21
44
|
|
|
22
45
|
def start(notification)
|
|
23
46
|
@total_examples = notification.count
|
|
24
|
-
@client = Ipc::Client.connect
|
|
47
|
+
@client = Ipc::Client.connect(Ipc.socket_path(@runner.run_id), timeout: @connect_timeout)
|
|
48
|
+
# Announces the process at once, so the orchestrator notices it dying
|
|
49
|
+
# even before its first example.
|
|
50
|
+
@client.notify(**sender, type: 'hello')
|
|
25
51
|
end
|
|
26
52
|
|
|
27
53
|
def example_started(_notification)
|
|
@@ -42,9 +68,11 @@ module ParallelMatrixFormatter
|
|
|
42
68
|
end
|
|
43
69
|
|
|
44
70
|
def dump_summary(summary)
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
71
|
+
record_totals(summary)
|
|
72
|
+
@client.notify(**sender, type: 'summary', duration: summary.duration,
|
|
73
|
+
examples: summary.example_count, failures: summary.failure_count,
|
|
74
|
+
pending: summary.pending_count, errors: summary.errors_outside_of_examples_count,
|
|
75
|
+
failed_examples: @failures)
|
|
48
76
|
end
|
|
49
77
|
|
|
50
78
|
def close(_notification)
|
|
@@ -54,12 +82,6 @@ module ParallelMatrixFormatter
|
|
|
54
82
|
|
|
55
83
|
private
|
|
56
84
|
|
|
57
|
-
# Under parallel_split_test every process is forked by the same runner,
|
|
58
|
-
# which records the process count before forking.
|
|
59
|
-
def total_processes
|
|
60
|
-
(ParallelSplitTest.processes if defined?(ParallelSplitTest)) || 1
|
|
61
|
-
end
|
|
62
|
-
|
|
63
85
|
# The silenced terminal is where the display goes, unless RSpec was asked
|
|
64
86
|
# to write to a file with --out.
|
|
65
87
|
def display_output(config, output)
|
|
@@ -69,9 +91,23 @@ module ParallelMatrixFormatter
|
|
|
69
91
|
output.is_a?(File) ? output : terminal
|
|
70
92
|
end
|
|
71
93
|
|
|
94
|
+
# parallel_split_test builds its closing "Summary:" from what every process
|
|
95
|
+
# wrote to the stream RSpec handed to the formatter. The display goes to the
|
|
96
|
+
# silenced terminal instead, so the totals line goes to the stream, which
|
|
97
|
+
# parallel_split_test records while its own stdout is silenced.
|
|
98
|
+
def record_totals(summary)
|
|
99
|
+
@stream.puts summary.totals_line unless @stream.equal?(output)
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
# Identifies this process to the orchestrator: its number and the pids it
|
|
103
|
+
# can be told apart by in the runner's pid file.
|
|
104
|
+
def sender
|
|
105
|
+
{ process: @process_number, pid: Process.pid, ppid: Process.ppid }
|
|
106
|
+
end
|
|
107
|
+
|
|
72
108
|
def report(status)
|
|
73
109
|
progress = @examples_run.fdiv(@total_examples)
|
|
74
|
-
@client.notify(type: 'example',
|
|
110
|
+
@client.notify(**sender, type: 'example', status: status, progress: progress)
|
|
75
111
|
end
|
|
76
112
|
|
|
77
113
|
def failure_details(notification)
|
|
@@ -10,7 +10,7 @@ module ParallelMatrixFormatter
|
|
|
10
10
|
# Process 1 may still be loading spec files when the others start.
|
|
11
11
|
CONNECT_TIMEOUT = 120
|
|
12
12
|
|
|
13
|
-
def self.connect(path
|
|
13
|
+
def self.connect(path, timeout: CONNECT_TIMEOUT)
|
|
14
14
|
deadline = Time.now + timeout
|
|
15
15
|
begin
|
|
16
16
|
new(UNIXSocket.new(path))
|
|
@@ -10,21 +10,36 @@ module ParallelMatrixFormatter
|
|
|
10
10
|
# When a client disconnects, a synthetic `disconnected` message carrying its
|
|
11
11
|
# process number is queued, so a crashed process is noticed too.
|
|
12
12
|
class Server
|
|
13
|
-
def initialize(path
|
|
13
|
+
def initialize(path)
|
|
14
14
|
FileUtils.rm_f(path)
|
|
15
15
|
@path = path
|
|
16
16
|
@socket = UNIXServer.new(path)
|
|
17
17
|
@messages = Queue.new
|
|
18
|
+
@lock = Mutex.new
|
|
19
|
+
@open_clients = 0
|
|
18
20
|
@acceptor = Thread.new { accept_clients }
|
|
19
21
|
end
|
|
20
22
|
|
|
21
|
-
# Yields messages in arrival order until the server is closed.
|
|
22
|
-
|
|
23
|
-
|
|
23
|
+
# Yields messages in arrival order until the server is closed. With a
|
|
24
|
+
# poll interval it also yields nil whenever that long passes without a
|
|
25
|
+
# message, so the caller can check on the processes it is waiting for.
|
|
26
|
+
def each_message(poll_interval: nil)
|
|
27
|
+
loop do
|
|
28
|
+
message = @messages.pop(timeout: poll_interval)
|
|
29
|
+
break if message.nil? && @messages.closed?
|
|
30
|
+
|
|
24
31
|
yield message
|
|
25
32
|
end
|
|
26
33
|
end
|
|
27
34
|
|
|
35
|
+
# Whether every client that connected, or is waiting to be accepted, has
|
|
36
|
+
# disconnected and every message has been yielded. A process can exit
|
|
37
|
+
# before its messages are read, so its absence from a runner's pid file
|
|
38
|
+
# alone does not mean it has been heard from.
|
|
39
|
+
def idle?
|
|
40
|
+
@lock.synchronize { @open_clients.zero? && !@socket.wait_readable(0) } && @messages.empty?
|
|
41
|
+
end
|
|
42
|
+
|
|
28
43
|
def close
|
|
29
44
|
@socket.close
|
|
30
45
|
@messages.close
|
|
@@ -34,7 +49,14 @@ module ParallelMatrixFormatter
|
|
|
34
49
|
private
|
|
35
50
|
|
|
36
51
|
def accept_clients
|
|
37
|
-
loop
|
|
52
|
+
loop do
|
|
53
|
+
@socket.wait_readable
|
|
54
|
+
client = @lock.synchronize do
|
|
55
|
+
@open_clients += 1
|
|
56
|
+
@socket.accept
|
|
57
|
+
end
|
|
58
|
+
Thread.new(client) { |connection| read(connection) }
|
|
59
|
+
end
|
|
38
60
|
rescue IOError, Errno::EBADF
|
|
39
61
|
nil # the socket was closed
|
|
40
62
|
end
|
|
@@ -49,6 +71,7 @@ module ParallelMatrixFormatter
|
|
|
49
71
|
enqueue('type' => 'disconnected', 'process' => process) if process
|
|
50
72
|
ensure
|
|
51
73
|
client.close
|
|
74
|
+
@lock.synchronize { @open_clients -= 1 }
|
|
52
75
|
end
|
|
53
76
|
|
|
54
77
|
def enqueue(message)
|
|
@@ -6,10 +6,9 @@ module ParallelMatrixFormatter
|
|
|
6
6
|
# Communication between the test processes and the orchestrator: newline
|
|
7
7
|
# separated JSON messages over a UNIX socket.
|
|
8
8
|
module Ipc
|
|
9
|
-
#
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
File.join(Dir.tmpdir, "parallel_matrix_formatter-#{Process.ppid}.sock")
|
|
9
|
+
# @param run_id [String, Integer] identifies the run, see Runner#run_id
|
|
10
|
+
def self.socket_path(run_id)
|
|
11
|
+
File.join(Dir.tmpdir, "parallel_matrix_formatter-#{run_id}.sock")
|
|
13
12
|
end
|
|
14
13
|
end
|
|
15
14
|
end
|
|
@@ -4,20 +4,36 @@ module ParallelMatrixFormatter
|
|
|
4
4
|
# Runs in process 1 only. Receives the messages of every test process over
|
|
5
5
|
# IPC, renders them as they arrive and prints the consolidated summary once
|
|
6
6
|
# every process has reported its summary or disconnected.
|
|
7
|
+
#
|
|
8
|
+
# The runner's process count is only a lower bound: parallel_tests drops
|
|
9
|
+
# empty groups and numbers processes by group under --only-group. So the
|
|
10
|
+
# processes to wait for are the ones that actually connected, plus, without
|
|
11
|
+
# a pid file, the ones the count promises. With a pid file the orchestrator
|
|
12
|
+
# also waits for every live test process it has not heard from yet. A process
|
|
13
|
+
# counts as heard from when its pid, or its parent's (a wrapper such as spring
|
|
14
|
+
# that parallel_tests started instead of rspec), arrived in a message.
|
|
15
|
+
# Without a pid file, a process that has not connected within the connect
|
|
16
|
+
# timeout counts as gone: it died before loading the formatter, or gave up.
|
|
17
|
+
# Either way it only finishes once the server has read every connection.
|
|
7
18
|
class Orchestrator
|
|
8
|
-
|
|
19
|
+
POLL_INTERVAL = 1
|
|
20
|
+
|
|
21
|
+
def self.for(process_number, runner, output, config)
|
|
9
22
|
return NullOrchestrator.new unless process_number == 1
|
|
10
23
|
|
|
11
|
-
new(
|
|
24
|
+
new(runner, output, Rendering::Display.new(config, runner.process_count), config['connect_timeout_seconds'])
|
|
12
25
|
end
|
|
13
26
|
|
|
14
|
-
def initialize(
|
|
15
|
-
@
|
|
27
|
+
def initialize(runner, output, display, connect_timeout = Ipc::Client::CONNECT_TIMEOUT)
|
|
28
|
+
@runner = runner
|
|
16
29
|
@output = output
|
|
17
30
|
@display = display
|
|
18
31
|
@summaries = {}
|
|
19
32
|
@disconnected = []
|
|
20
|
-
@
|
|
33
|
+
@processes = []
|
|
34
|
+
@known_pids = []
|
|
35
|
+
@connect_deadline = Time.now + connect_timeout
|
|
36
|
+
@server = Ipc::Server.new(Ipc.socket_path(runner.run_id))
|
|
21
37
|
@collector = Thread.new { collect_messages }
|
|
22
38
|
end
|
|
23
39
|
|
|
@@ -32,13 +48,14 @@ module ParallelMatrixFormatter
|
|
|
32
48
|
private
|
|
33
49
|
|
|
34
50
|
def collect_messages
|
|
35
|
-
@server.each_message do |message|
|
|
36
|
-
handle(message)
|
|
51
|
+
@server.each_message(poll_interval: POLL_INTERVAL) do |message|
|
|
52
|
+
handle(message) if message
|
|
37
53
|
break if all_finished?
|
|
38
54
|
end
|
|
39
55
|
end
|
|
40
56
|
|
|
41
57
|
def handle(message)
|
|
58
|
+
remember(message)
|
|
42
59
|
case message['type']
|
|
43
60
|
when 'example' then print_example(message)
|
|
44
61
|
when 'summary' then @summaries[message['process']] = message
|
|
@@ -46,17 +63,37 @@ module ParallelMatrixFormatter
|
|
|
46
63
|
end
|
|
47
64
|
end
|
|
48
65
|
|
|
66
|
+
def remember(message)
|
|
67
|
+
@processes |= [message['process']]
|
|
68
|
+
@known_pids |= message.values_at('pid', 'ppid').compact
|
|
69
|
+
end
|
|
70
|
+
|
|
49
71
|
def print_example(message)
|
|
50
72
|
@output.print @display.example(message['process'], message['status'], message['progress'])
|
|
51
73
|
@output.flush
|
|
52
74
|
end
|
|
53
75
|
|
|
54
76
|
def all_finished?
|
|
55
|
-
missing_processes.all? { |process|
|
|
77
|
+
missing_processes.all? { |process| gone?(process) } && no_unknown_live_process? && @server.idle?
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
def gone?(process)
|
|
81
|
+
@disconnected.include?(process) || (!@processes.include?(process) && Time.now > @connect_deadline)
|
|
56
82
|
end
|
|
57
83
|
|
|
58
84
|
def missing_processes
|
|
59
|
-
|
|
85
|
+
awaited_processes.reject { |process| @summaries.key?(process) }
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
def awaited_processes
|
|
89
|
+
@runner.pid_file ? @processes : @processes | (1..@runner.process_count).to_a
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
def no_unknown_live_process?
|
|
93
|
+
return true unless @runner.pid_file
|
|
94
|
+
|
|
95
|
+
live = @runner.live_pids
|
|
96
|
+
!live.nil? && (live - @known_pids).empty?
|
|
60
97
|
end
|
|
61
98
|
end
|
|
62
99
|
end
|
|
@@ -1,11 +1,15 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require 'tmpdir'
|
|
4
|
+
|
|
3
5
|
module ParallelMatrixFormatter
|
|
4
6
|
module Output
|
|
5
|
-
# Redirects the process's stdout
|
|
6
|
-
# copy of the original stdout for the display. The
|
|
7
|
-
# the file descriptor level, so output written
|
|
8
|
-
# STDOUT, C extensions or child processes is
|
|
7
|
+
# Redirects the process's stdout to /dev/null and its stderr to a log file,
|
|
8
|
+
# and keeps a private copy of the original stdout for the display. The
|
|
9
|
+
# redirection happens at the file descriptor level, so output written
|
|
10
|
+
# through loggers holding STDOUT, C extensions or child processes is
|
|
11
|
+
# silenced as well. There is one log per process, named after the run (the
|
|
12
|
+
# parent pid) and the process number.
|
|
9
13
|
module Silencer
|
|
10
14
|
class << self
|
|
11
15
|
# @return [IO] the original stdout
|
|
@@ -18,15 +22,26 @@ module ParallelMatrixFormatter
|
|
|
18
22
|
!@terminal.nil?
|
|
19
23
|
end
|
|
20
24
|
|
|
25
|
+
# @return [Array<String>] the stderr logs of every process of the run
|
|
26
|
+
def stderr_logs
|
|
27
|
+
@log_prefix ? Dir["#{@log_prefix}-*.stderr.log"] : []
|
|
28
|
+
end
|
|
29
|
+
|
|
21
30
|
private
|
|
22
31
|
|
|
23
32
|
def redirect_to_null
|
|
24
33
|
terminal = STDOUT.dup
|
|
25
34
|
terminal.sync = true
|
|
26
35
|
STDOUT.reopen(File::NULL, 'w')
|
|
27
|
-
STDERR.reopen(
|
|
36
|
+
STDERR.reopen(stderr_log_path, 'w')
|
|
37
|
+
STDERR.sync = true
|
|
28
38
|
terminal
|
|
29
39
|
end
|
|
40
|
+
|
|
41
|
+
def stderr_log_path
|
|
42
|
+
@log_prefix = File.join(Dir.tmpdir, "parallel_matrix_formatter-#{Process.ppid}")
|
|
43
|
+
"#{@log_prefix}-#{[ENV['TEST_ENV_NUMBER'].to_i, 1].max}.stderr.log"
|
|
44
|
+
end
|
|
30
45
|
end
|
|
31
46
|
end
|
|
32
47
|
end
|
|
@@ -45,7 +45,7 @@ module ParallelMatrixFormatter
|
|
|
45
45
|
end
|
|
46
46
|
|
|
47
47
|
def all_complete?(progress)
|
|
48
|
-
progress.size
|
|
48
|
+
progress.size >= @total_processes && progress.values.all? { |value| value >= 1.0 }
|
|
49
49
|
end
|
|
50
50
|
end
|
|
51
51
|
end
|
|
@@ -17,7 +17,8 @@ module ParallelMatrixFormatter
|
|
|
17
17
|
# @param missing_processes [Array<Integer>] processes that never sent a summary
|
|
18
18
|
def render(summaries, missing_processes)
|
|
19
19
|
failures = summaries.flat_map { |summary| summary['failed_examples'] }
|
|
20
|
-
sections = [failures_section(failures), warnings(missing_processes),
|
|
20
|
+
sections = [failures_section(failures), warnings(missing_processes),
|
|
21
|
+
logs_section(summaries, missing_processes), totals(summaries), rerun_section(failures)]
|
|
21
22
|
"\n#{sections.compact.join("\n")}"
|
|
22
23
|
end
|
|
23
24
|
|
|
@@ -44,6 +45,18 @@ module ParallelMatrixFormatter
|
|
|
44
45
|
'Set suppress_output: false to see its output.', :yellow)}"
|
|
45
46
|
end
|
|
46
47
|
|
|
48
|
+
# The stderr of the processes is silenced into logs, so point at them
|
|
49
|
+
# when something went wrong: a missing process, an error outside of the
|
|
50
|
+
# examples (a load error) or anything a process wrote to stderr.
|
|
51
|
+
def logs_section(summaries, missing_processes)
|
|
52
|
+
logs = Output::Silencer.stderr_logs
|
|
53
|
+
errors = summaries.sum { |summary| summary['errors'].to_i }
|
|
54
|
+
return unless errors.positive? || missing_processes.any? || logs.any? { |log| File.size?(log) }
|
|
55
|
+
|
|
56
|
+
title = Colors.wrap("stderr of the processes (#{errors} errors outside of examples):", :yellow)
|
|
57
|
+
"\n#{title}\n#{logs.join("\n")}"
|
|
58
|
+
end
|
|
59
|
+
|
|
47
60
|
def totals(summaries)
|
|
48
61
|
examples, failures, pending = %w[examples failures pending].map { |key| summaries.sum { |s| s[key] } }
|
|
49
62
|
process_time = Helpers.format_duration(summaries.sum { |summary| summary['duration'] })
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'json'
|
|
4
|
+
|
|
5
|
+
module ParallelMatrixFormatter
|
|
6
|
+
# Describes the parallel runner that started this process: how many processes
|
|
7
|
+
# it runs, which run they belong to and, where the runner keeps one, the file
|
|
8
|
+
# listing the processes that are still alive.
|
|
9
|
+
#
|
|
10
|
+
# parallel_split_test forks every process from one runner, so the constant
|
|
11
|
+
# ParallelSplitTest knows the count and the parent pid identifies the run.
|
|
12
|
+
# parallel_tests spawns every process from one parallel_rspec, which exports
|
|
13
|
+
# the requested number of groups and a pid file that is unique per run.
|
|
14
|
+
# Without a runner the only process is process 1 itself, so its own pid
|
|
15
|
+
# identifies the run; the parent pid would be shared by every rspec started
|
|
16
|
+
# from the same shell.
|
|
17
|
+
Runner = Struct.new(:process_count, :run_id, :pid_file, keyword_init: true) do
|
|
18
|
+
def self.detect
|
|
19
|
+
if defined?(::ParallelSplitTest)
|
|
20
|
+
new(process_count: ::ParallelSplitTest.processes, run_id: Process.ppid)
|
|
21
|
+
elsif ENV['PARALLEL_TEST_GROUPS']
|
|
22
|
+
parallel_tests
|
|
23
|
+
else
|
|
24
|
+
new(process_count: 1, run_id: Process.pid)
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def self.parallel_tests
|
|
29
|
+
pid_file = ENV.fetch('PARALLEL_PID_FILE', nil)
|
|
30
|
+
new(process_count: [ENV['PARALLEL_TEST_GROUPS'].to_i, 1].max,
|
|
31
|
+
run_id: pid_file ? File.basename(pid_file) : Process.ppid, pid_file: pid_file)
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
# @return [Array<Integer>, nil] pids of the test processes still running, or
|
|
35
|
+
# nil when unknown: no pid file, or one that is empty or half written
|
|
36
|
+
def live_pids
|
|
37
|
+
return unless pid_file
|
|
38
|
+
|
|
39
|
+
pids = JSON.parse(File.read(pid_file))
|
|
40
|
+
pids if pids.is_a?(Array)
|
|
41
|
+
rescue JSON::ParserError, SystemCallError
|
|
42
|
+
nil
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
end
|
|
@@ -6,6 +6,7 @@
|
|
|
6
6
|
# before anything else to silence it:
|
|
7
7
|
#
|
|
8
8
|
# RUBYOPT="-rparallel_matrix_formatter/silence" bundle exec parallel_split_test ...
|
|
9
|
+
# RUBYOPT="-rparallel_matrix_formatter/silence" bundle exec parallel_rspec ...
|
|
9
10
|
require_relative 'config'
|
|
10
11
|
require_relative 'output/silencer'
|
|
11
12
|
|
|
@@ -1,32 +1,15 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require 'rspec/core'
|
|
4
|
-
|
|
5
|
-
require_relative 'parallel_matrix_formatter/version'
|
|
6
|
-
require_relative 'parallel_matrix_formatter/config'
|
|
7
|
-
require_relative 'parallel_matrix_formatter/output/silencer'
|
|
8
|
-
require_relative 'parallel_matrix_formatter/ipc'
|
|
9
|
-
require_relative 'parallel_matrix_formatter/ipc/client'
|
|
10
|
-
require_relative 'parallel_matrix_formatter/ipc/server'
|
|
11
|
-
require_relative 'parallel_matrix_formatter/rendering/colors'
|
|
12
|
-
require_relative 'parallel_matrix_formatter/rendering/digits'
|
|
13
|
-
require_relative 'parallel_matrix_formatter/rendering/progress_update_policy'
|
|
14
|
-
require_relative 'parallel_matrix_formatter/rendering/progress_column'
|
|
15
|
-
require_relative 'parallel_matrix_formatter/rendering/progress_line'
|
|
16
|
-
require_relative 'parallel_matrix_formatter/rendering/example_status'
|
|
17
|
-
require_relative 'parallel_matrix_formatter/rendering/summary'
|
|
18
|
-
require_relative 'parallel_matrix_formatter/rendering/display'
|
|
19
|
-
require_relative 'parallel_matrix_formatter/null_orchestrator'
|
|
20
|
-
require_relative 'parallel_matrix_formatter/orchestrator'
|
|
21
|
-
require_relative 'parallel_matrix_formatter/formatter'
|
|
22
|
-
|
|
23
3
|
# Matrix digital rain RSpec formatter for test suites split across processes
|
|
24
|
-
# with parallel_split_test.
|
|
4
|
+
# with parallel_split_test or parallel_tests.
|
|
25
5
|
#
|
|
26
6
|
# Every test process loads the Formatter, which silences the process and sends
|
|
27
7
|
# each example's result to process 1 over a UNIX socket. Process 1 also hosts
|
|
28
8
|
# the Orchestrator, which renders the shared display and, once every process has
|
|
29
9
|
# reported, the consolidated summary.
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
10
|
+
|
|
11
|
+
require 'rspec/core'
|
|
12
|
+
|
|
13
|
+
require_relative 'parallel_matrix_formatter/version'
|
|
14
|
+
require_relative 'parallel_matrix_formatter/error'
|
|
15
|
+
require_relative 'parallel_matrix_formatter/formatter'
|
metadata
CHANGED
|
@@ -1,13 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: parallel_matrix_formatter
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Volodymyr Shcherbyna
|
|
8
|
+
autorequire:
|
|
8
9
|
bindir: bin
|
|
9
10
|
cert_chain: []
|
|
10
|
-
date:
|
|
11
|
+
date: 2026-10-03 00:00:00.000000000 Z
|
|
11
12
|
dependencies:
|
|
12
13
|
- !ruby/object:Gem::Dependency
|
|
13
14
|
name: rspec-core
|
|
@@ -24,7 +25,8 @@ dependencies:
|
|
|
24
25
|
- !ruby/object:Gem::Version
|
|
25
26
|
version: '3.0'
|
|
26
27
|
description: An RSpec formatter that renders the progress of every parallel_split_test
|
|
27
|
-
process as one Matrix-style digital rain and prints a single consolidated
|
|
28
|
+
or parallel_tests process as one Matrix-style digital rain and prints a single consolidated
|
|
29
|
+
summary.
|
|
28
30
|
email:
|
|
29
31
|
- scherbina.v@gmail.com
|
|
30
32
|
executables: []
|
|
@@ -37,6 +39,7 @@ files:
|
|
|
37
39
|
- config/parallel_matrix_formatter.yml
|
|
38
40
|
- lib/parallel_matrix_formatter.rb
|
|
39
41
|
- lib/parallel_matrix_formatter/config.rb
|
|
42
|
+
- lib/parallel_matrix_formatter/error.rb
|
|
40
43
|
- lib/parallel_matrix_formatter/formatter.rb
|
|
41
44
|
- lib/parallel_matrix_formatter/ipc.rb
|
|
42
45
|
- lib/parallel_matrix_formatter/ipc/client.rb
|
|
@@ -52,6 +55,7 @@ files:
|
|
|
52
55
|
- lib/parallel_matrix_formatter/rendering/progress_line.rb
|
|
53
56
|
- lib/parallel_matrix_formatter/rendering/progress_update_policy.rb
|
|
54
57
|
- lib/parallel_matrix_formatter/rendering/summary.rb
|
|
58
|
+
- lib/parallel_matrix_formatter/runner.rb
|
|
55
59
|
- lib/parallel_matrix_formatter/silence.rb
|
|
56
60
|
- lib/parallel_matrix_formatter/version.rb
|
|
57
61
|
homepage: https://github.com/vovka/parallel_matrix_formatter
|
|
@@ -62,6 +66,7 @@ metadata:
|
|
|
62
66
|
source_code_uri: https://github.com/vovka/parallel_matrix_formatter
|
|
63
67
|
changelog_uri: https://github.com/vovka/parallel_matrix_formatter/blob/main/CHANGELOG.md
|
|
64
68
|
rubygems_mfa_required: 'true'
|
|
69
|
+
post_install_message:
|
|
65
70
|
rdoc_options: []
|
|
66
71
|
require_paths:
|
|
67
72
|
- lib
|
|
@@ -76,7 +81,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
76
81
|
- !ruby/object:Gem::Version
|
|
77
82
|
version: '0'
|
|
78
83
|
requirements: []
|
|
79
|
-
rubygems_version: 3.
|
|
84
|
+
rubygems_version: 3.5.22
|
|
85
|
+
signing_key:
|
|
80
86
|
specification_version: 4
|
|
81
|
-
summary: Matrix digital rain RSpec formatter for parallel_split_test
|
|
87
|
+
summary: Matrix digital rain RSpec formatter for parallel_split_test and parallel_tests
|
|
82
88
|
test_files: []
|