tldr 0.2.1 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +11 -0
- data/README.md +153 -70
- data/lib/tldr/parallel_controls.rb +47 -0
- data/lib/tldr/parallelizer.rb +42 -0
- data/lib/tldr/planner.rb +1 -2
- data/lib/tldr/reporters/default.rb +6 -1
- data/lib/tldr/runner.rb +12 -17
- data/lib/tldr/strategizer.rb +54 -0
- data/lib/tldr/value/config.rb +0 -2
- data/lib/tldr/value/test.rb +8 -3
- data/lib/tldr/value/test_group.rb +22 -0
- data/lib/tldr/value.rb +1 -0
- data/lib/tldr/version.rb +1 -1
- data/lib/tldr.rb +10 -5
- metadata +7 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 91190a0094836b21df8aaeccb57684848ada68707a4c52241ebd74437fb514c4
|
4
|
+
data.tar.gz: a6cc8f1573bdb317f6fd5f337a9670bbeef95f735bdc1003774f4bca0a7b294b
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 93b9fdf0d61dcaba8c3c6dff464700ca260bbdef433eaf799e33130baf5ef63d0e7dec1c6dc3558e5d0f1bc580b53fbacc31c466da1f4a9803533470bb1b5ec5
|
7
|
+
data.tar.gz: f2a8de04ed3e9b84c277db21f111177849138c1b1b692a43d96492b0707c0e9a8c255c26a9991d899ed27312a04ebe13887a6db7d4417a4878b4e2cfc542acc4
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,16 @@
|
|
1
1
|
## [Unreleased]
|
2
2
|
|
3
|
+
## [0.4.0]
|
4
|
+
|
5
|
+
* Add `TLDR.dont_run_these_in_parallel!` method to allow tests to indicate that they
|
6
|
+
must be run in isolation and not concurrently with any other tests
|
7
|
+
* Add support for `around` hooks (similar to [minitest-around](https://github.com/splattael/minitest-around))
|
8
|
+
|
9
|
+
## [0.3.0]
|
10
|
+
|
11
|
+
* Add `TLDR.run_these_together!` method to allow tests that can't safely be run
|
12
|
+
concurrently to be grouped and run together
|
13
|
+
|
3
14
|
## [0.2.1]
|
4
15
|
|
5
16
|
* Define a default empty setup/teardown in the base class, to guard against
|
data/README.md
CHANGED
@@ -1,6 +1,8 @@
|
|
1
1
|
# TLDR - for people who don't have time for slow tests
|
2
2
|
|
3
|
-
|
3
|
+
Okay, you might need to sit down for this:
|
4
|
+
|
5
|
+
**tl;dr, TLDR is a Ruby test framework that stops running your tests after 1.8 seconds.**
|
4
6
|
|
5
7
|
We initially meant this as a joke [while
|
6
8
|
pairin'](https://www.youtube.com/live/bmi-SWeH4MA?si=p5g1j1FQZrbYEOCg&t=63), but
|
@@ -9,16 +11,28 @@ in addition to being funny, it was also a pretty good idea. So we fleshed out
|
|
9
11
|
[Minitest-compatible](#minitest-compatibility), and downright pleasant test
|
10
12
|
framework for Ruby.
|
11
13
|
|
14
|
+
The "big idea" here is TLDR is designed for users to run the `tldr` command
|
15
|
+
repeatedly as they work—as opposed to only running the tests for whatever is
|
16
|
+
being worked on. Even if the suite run over the 1.8 second time limit. Because
|
17
|
+
TLDR shuffles and runs in parallel and is guaranteed to take less than two
|
18
|
+
seconds,
|
19
|
+
**you'll actually wind up running _all_ of your tests quite often as you work**,
|
20
|
+
catching any problems much earlier than if you had waited until the end of the
|
21
|
+
day to push your work and let a continuous integration server run the full
|
22
|
+
suite.
|
23
|
+
|
12
24
|
Some stuff you might like:
|
13
25
|
|
14
26
|
* A CLI that can run tests by line number(s) (e.g. `foo.rb:5 bar.rb:3:10`) and
|
15
27
|
by names or patterns (e.g. `--name test_fail,test_error --name "/_\d/"`)
|
16
|
-
* Everything is **parallel by default**, and seems pretty fast
|
17
|
-
|
28
|
+
* Everything is **parallel by default**, and seems pretty darn fast; TLDR
|
29
|
+
also provides [several escape hatches to sequester tests that aren't thread-safe](#parallel-by-default-is-nice-in-theory-but-half-my-tests-are-failing-wat)
|
30
|
+
* Surprisingly delightful color diff output when two things fail to equal one
|
31
|
+
another, care of [@mcmire's super_diff gem](https://github.com/mcmire/super_diff)
|
18
32
|
* By default, the CLI will prepend your most-recently-edited test file to the
|
19
|
-
front of your suite so its tests will run first. The
|
20
|
-
the most likely
|
21
|
-
`--prepend` option)
|
33
|
+
front of your suite so its tests will run first. The test you worked on most recently
|
34
|
+
is the one you most likely want to ensure runs, so TLDR runs it first (see the
|
35
|
+
`--prepend` option for how to control this behavior)
|
22
36
|
* And, of course, our signature feature: your test suite will never grow into
|
23
37
|
a glacially slow, soul-sucking albatross around your neck, because **after 1.8
|
24
38
|
seconds, it stops running your tests**, with a report on what it _was_ able to
|
@@ -50,7 +64,7 @@ end
|
|
50
64
|
```
|
51
65
|
|
52
66
|
A TLDR subclass defines its tests with instance methods that begin with
|
53
|
-
`
|
67
|
+
`test_`. They can define `setup` and/or `teardown` methods which will run before
|
54
68
|
and after each test, respectively.
|
55
69
|
|
56
70
|
If you place your tests in `test/**/*_test.rb` (and/or `test/**/test_*.rb`)
|
@@ -82,68 +96,9 @@ flags:
|
|
82
96
|
$ tldr --name FooTest#test_foo -n test_bar,test_baz -n /_qux/
|
83
97
|
```
|
84
98
|
|
85
|
-
(The above will translate to this array of name
|
99
|
+
(The above will translate to this array of name filters internally:
|
86
100
|
`["FooTest#test_foo", "test_bar", "test_baz", "/_qux/"]`.)
|
87
101
|
|
88
|
-
### Running tests without the CLI
|
89
|
-
|
90
|
-
If you'd rather use TLDR by running Ruby files instead of the `tldr` CLI
|
91
|
-
(similar to `require "minitest/autorun"`), here's how to do it!
|
92
|
-
|
93
|
-
Given a file `test/some_test.rb`:
|
94
|
-
|
95
|
-
```ruby
|
96
|
-
require "tldr"
|
97
|
-
TLDR::Run.at_exit! TLDR::Config.new(no_emoji: true)
|
98
|
-
|
99
|
-
class SomeTest < TLDR
|
100
|
-
def test_truth
|
101
|
-
assert true
|
102
|
-
end
|
103
|
-
end
|
104
|
-
```
|
105
|
-
|
106
|
-
You could run the test with:
|
107
|
-
|
108
|
-
```
|
109
|
-
$ ruby test/some_test.rb
|
110
|
-
```
|
111
|
-
|
112
|
-
To maximize control and to avoid running code accidentally (and _unlike_ the
|
113
|
-
`tldr` CLI), running `at_exit!` will not set default values to the `paths`,
|
114
|
-
`helper`, `load_paths`, and `prepend_tests` config properties. You'll have to
|
115
|
-
pass any values you want to set on a [Config object](/lib/tldr/value/config.rb)
|
116
|
-
and pass it to `at_exit!`.
|
117
|
-
|
118
|
-
To avoid running multiple suites accidentally, if `TLDR::Run.at_exit!` is
|
119
|
-
encountered multiple times, only the first hook will be registered. If the
|
120
|
-
`tldr` CLI is running and encounters a call to `at_exit!`, it will be ignored.
|
121
|
-
|
122
|
-
#### Setting up the load path
|
123
|
-
|
124
|
-
When running TLDR from a Ruby script, one thing the framework can't help you with
|
125
|
-
is setting up load paths for you.
|
126
|
-
|
127
|
-
If you want to require code in `test/` or `lib/` without using
|
128
|
-
`require_relative`, you'll need to add those directories to the load path. You
|
129
|
-
can do this programmatically by prepending the path to `$LOAD_PATH`, like
|
130
|
-
this:
|
131
|
-
|
132
|
-
```ruby
|
133
|
-
$LOAD_PATH.unshift "test"
|
134
|
-
|
135
|
-
require "tldr"
|
136
|
-
TLDR::Run.at_exit! TLDR::Config.new(no_emoji: true)
|
137
|
-
|
138
|
-
require "helper"
|
139
|
-
```
|
140
|
-
|
141
|
-
Or by using Ruby's `-I` flag to include it:
|
142
|
-
|
143
|
-
```
|
144
|
-
$ ruby -Itest test/some_test.rb
|
145
|
-
```
|
146
|
-
|
147
102
|
### Options
|
148
103
|
|
149
104
|
Here is the full list of CLI options:
|
@@ -192,7 +147,8 @@ well as some [internal tests](/tests/dotfile_test.rb) demonstrating its behavior
|
|
192
147
|
Tests you write with tldr are designed to be mostly-compatible with
|
193
148
|
[Minitest](https://github.com/minitest/minitest) tests. Some notes:
|
194
149
|
|
195
|
-
* `setup` and `teardown` hook methods should work as you expect
|
150
|
+
* `setup` and `teardown` hook methods should work as you expect. (We even threw
|
151
|
+
in [an `around` hook](https://github.com/splattael/minitest-around) as a bonus!)
|
196
152
|
* All of Minitest's assertions (e.g. `assert`, `assert_equals`) are provided,
|
197
153
|
with these caveats:
|
198
154
|
* To retain the `expected, actual` argument ordering, `tldr` defines
|
@@ -201,7 +157,7 @@ with these caveats:
|
|
201
157
|
* If you want to maximize compatibility and mix in `assert_includes` and the
|
202
158
|
deprecated `assert_send`, just `include
|
203
159
|
TLDR::Assertions::MinitestCompatibility` into the `TLDR` base class or
|
204
|
-
individual test
|
160
|
+
individual test classesJust set it
|
205
161
|
|
206
162
|
### Running TLDR with Rake
|
207
163
|
|
@@ -225,9 +181,136 @@ You could then run the task with:
|
|
225
181
|
$ TLDR_OPTS="--no-parallel" bundle exec rake tldr
|
226
182
|
```
|
227
183
|
|
184
|
+
### Running tests without the CLI
|
185
|
+
|
186
|
+
If you'd rather use TLDR by running Ruby files instead of the `tldr` CLI
|
187
|
+
(similar to `require "minitest/autorun"`), here's how to do it!
|
188
|
+
|
189
|
+
Given a file `test/some_test.rb`:
|
190
|
+
|
191
|
+
```ruby
|
192
|
+
require "tldr"
|
193
|
+
TLDR::Run.at_exit! TLDR::Config.new(no_emoji: true)
|
194
|
+
|
195
|
+
class SomeTest < TLDR
|
196
|
+
def test_truth
|
197
|
+
assert true
|
198
|
+
end
|
199
|
+
end
|
200
|
+
```
|
201
|
+
|
202
|
+
You could run the test with:
|
203
|
+
|
204
|
+
```
|
205
|
+
$ ruby test/some_test.rb
|
206
|
+
```
|
207
|
+
|
208
|
+
To maximize control and to avoid running code accidentally (and _unlike_ the
|
209
|
+
`tldr` CLI), running `at_exit!` will not set default values to the `paths`,
|
210
|
+
`helper`, `load_paths`, and `prepend_tests` config properties. You'll have to
|
211
|
+
pass any values you want to set on a [Config object](/lib/tldr/value/config.rb)
|
212
|
+
and pass it to `at_exit!`.
|
213
|
+
|
214
|
+
To avoid running multiple suites accidentally, if `TLDR::Run.at_exit!` is
|
215
|
+
encountered multiple times, only the first hook will be registered. If the
|
216
|
+
`tldr` CLI is running and encounters a call to `at_exit!`, it will be ignored.
|
217
|
+
|
218
|
+
#### Setting up the load path
|
219
|
+
|
220
|
+
When running TLDR from a Ruby script, one thing the framework can't help you with
|
221
|
+
is setting up load paths for you.
|
222
|
+
|
223
|
+
If you want to require code in `test/` or `lib/` without using
|
224
|
+
`require_relative`, you'll need to add those directories to the load path. You
|
225
|
+
can do this programmatically by prepending the path to `$LOAD_PATH`, like
|
226
|
+
this:
|
227
|
+
|
228
|
+
```ruby
|
229
|
+
$LOAD_PATH.unshift "test"
|
230
|
+
|
231
|
+
require "tldr"
|
232
|
+
TLDR::Run.at_exit! TLDR::Config.new(no_emoji: true)
|
233
|
+
|
234
|
+
require "helper"
|
235
|
+
```
|
236
|
+
|
237
|
+
Or by using Ruby's `-I` flag to include it:
|
238
|
+
|
239
|
+
```
|
240
|
+
$ ruby -Itest test/some_test.rb
|
241
|
+
```
|
242
|
+
|
243
|
+
## Questions you might be asking
|
244
|
+
|
245
|
+
TLDR is very similar to Minitest in API, but different in enough ways that you
|
246
|
+
probably have some questions.
|
247
|
+
|
248
|
+
### Parallel-by-default is nice in theory but half my tests are failing. Wat?
|
249
|
+
|
250
|
+
**Read this before you add `--no-parallel` because some tests are failing when
|
251
|
+
you run `tldr`.**
|
252
|
+
|
253
|
+
The vast majority of test suites in the wild are not parallelized and the vast
|
254
|
+
majority of _those_ will only parallelize by forking processes as opposed to
|
255
|
+
using a thread pool. We wanted to encourage more people to save time (after all,
|
256
|
+
you only get 1.8 seconds here) by making your test suite run as fast as it can,
|
257
|
+
so your tests run in parallel threads by default.
|
258
|
+
|
259
|
+
If you're writing new code and tests with TLDR and dutifully running `tldr`
|
260
|
+
constantly for fast feedback, odds are that this will help you catch thread
|
261
|
+
safety issues early—this is a good thing, because it gives you a chance to
|
262
|
+
address them before they're too hard to fix! But maybe you're porting an
|
263
|
+
existing test suite to TLDR and running in parallel for the first time, or maybe
|
264
|
+
you need to test something that simply _can't_ be exercised in a thread-safe
|
265
|
+
way. For those cases, TLDR's goal is to give you some tools to prevent you from
|
266
|
+
giving up and adding `--no-parallel` to your entire test suite and **slowing
|
267
|
+
everything down for the sake of a few tests**.
|
268
|
+
|
269
|
+
So, when you see a test that is failing when run in parallel with the rest of your
|
270
|
+
suite, here is what we recommend doing, in priority order:
|
271
|
+
|
272
|
+
1. Figure out a way to redesign the test (or the code under test) to be
|
273
|
+
thread-safe. Modern versions of Ruby provide a number of tools to make this
|
274
|
+
easier than it used to be, and it may be as simple as making an instance
|
275
|
+
variable thread-local
|
276
|
+
2. If the problem is that a subset of your tests depend on the same resource,
|
277
|
+
try using [TLDR.run_these_together!](lib/tldr/parallel_controls.rb) class to
|
278
|
+
group the tests together. This will ensure that those tests run in the same
|
279
|
+
thread in sequence (here's a [simple
|
280
|
+
example](/tests/fixture/run_these_together.rb))
|
281
|
+
3. For tests that affect process-wide resources like setting the system clock or
|
282
|
+
changing the process's working directory (i.e. `Dir.chdir`), you can sequester
|
283
|
+
them to run sequentially _after_ all parallel tests in your suite have run with
|
284
|
+
[TLDR.dont_run_these_in_parallel!](lib/tldr/parallel_controls.rb), which takes
|
285
|
+
the same arguments as `run_these_together!`
|
286
|
+
([example](/tests/fixture/dont_run_these_in_parallel.rb))
|
287
|
+
4. Give up and make the whole suite `--no-parallel`. If you find that you need
|
288
|
+
to resort to this, you might save some keystrokes by adding `parallel: false` in
|
289
|
+
a [.tldr.yml](#setting-defaults-in-tldryml) file
|
290
|
+
|
291
|
+
We have a couple other ideas of ways to incorporate non-thread-safe tests into
|
292
|
+
your suite without slowing down the rest of your tests, so stay tuned!
|
293
|
+
|
228
294
|
### How will I run all my tests in CI without the time bomb going off?
|
229
295
|
|
230
|
-
TLDR will run all your tests in CI without the time bomb going off.
|
296
|
+
TLDR will run all your tests in CI without the time bomb going off. If
|
297
|
+
`tldr` is run in a non-interactive shell and a `CI` environment variable is set
|
298
|
+
(as it is on virtually every CI service), then the bomb will be defused.
|
299
|
+
|
300
|
+
### Is there a plugin system?
|
301
|
+
|
302
|
+
There is not.
|
303
|
+
|
304
|
+
Currently, the only pluggable aspect of TLDR are reporters, which can be set
|
305
|
+
with the `--reporter` command line option. It can be set to any fully-qualified
|
306
|
+
class name that extends from
|
307
|
+
[TLDR::Reporters::Base](/lib/tldr/reporters/base.rb).
|
308
|
+
|
309
|
+
### What about mocking?
|
310
|
+
|
311
|
+
TLDR is laser-focused on running tests, so it doesn't provide a built-in mocking
|
312
|
+
facility. Might we interest you in a refreshing
|
313
|
+
[mocktail](https://github.com/testdouble/mocktail), instead?
|
231
314
|
|
232
315
|
## Acknowledgements
|
233
316
|
|
@@ -0,0 +1,47 @@
|
|
1
|
+
class TLDR
|
2
|
+
# If it's not safe to run a set of tests in parallel, you can force them to
|
3
|
+
# run in a group together (in a single worker) with `run_these_together!` in
|
4
|
+
# your test.
|
5
|
+
#
|
6
|
+
# This method takes an array of tuples, where the first element is the class
|
7
|
+
# (or its name as a string, if the class is not yet defined in the current
|
8
|
+
# file) and the second element is the method name. If the second element is
|
9
|
+
# nil, then all the tests on the class will be run together.
|
10
|
+
#
|
11
|
+
# Examples:
|
12
|
+
# - `run_these_together!` will run all the tests defined on the current
|
13
|
+
# class to be run in a group
|
14
|
+
# - `run_these_together!([[ClassA, nil], ["ClassB", :test_1], [ClassB, :test_2]])`
|
15
|
+
# will run all the tests defined on ClassA, and test_1 and test_2 from ClassB
|
16
|
+
#
|
17
|
+
GROUPED_TESTS = Concurrent::Array.new
|
18
|
+
def self.run_these_together! klass_method_tuples = [[self, nil]]
|
19
|
+
GROUPED_TESTS << TestGroup.new(klass_method_tuples)
|
20
|
+
end
|
21
|
+
|
22
|
+
# This is a similar API to run_these_together! but its effect is more drastic
|
23
|
+
# Rather than running the provided (class, method) tuples in a group within a
|
24
|
+
# thread as part of a parallel run, it will reserve all tests specified by
|
25
|
+
# all calls to `dont_run_these_in_parallel!` to be run after all parallel tests have
|
26
|
+
# finished.
|
27
|
+
#
|
28
|
+
# This has an important implication! If your test suite is over TLDR's time
|
29
|
+
# limit, it means that these tests will never be run outside of CI unless you
|
30
|
+
# run them manually.
|
31
|
+
#
|
32
|
+
# Like `run_these_together!`, `dont_run_these_in_parallel!` takes an array of
|
33
|
+
# tuples, where the first element is the class (or its fully-qualified name as
|
34
|
+
# a string) and the second element is `nil` (matching all the class's test
|
35
|
+
# methods) or else one of the methods on the class.
|
36
|
+
#
|
37
|
+
# Examples:
|
38
|
+
# - `dont_run_these_in_parallel!` will run all the tests defined on the current
|
39
|
+
# class after all parallel tests have finished
|
40
|
+
# - `dont_run_these_in_parallel!([[ClassA, nil], ["ClassB", :test_1], [ClassB, :test_2]])`
|
41
|
+
# will run all the tests defined on ClassA, and test_1 and test_2 from ClassB
|
42
|
+
#
|
43
|
+
THREAD_UNSAFE_TESTS = Concurrent::Array.new
|
44
|
+
def self.dont_run_these_in_parallel! klass_method_tuples = [[self, nil]]
|
45
|
+
THREAD_UNSAFE_TESTS << TestGroup.new(klass_method_tuples)
|
46
|
+
end
|
47
|
+
end
|
@@ -0,0 +1,42 @@
|
|
1
|
+
class TLDR
|
2
|
+
class Parallelizer
|
3
|
+
def initialize
|
4
|
+
@strategizer = Strategizer.new
|
5
|
+
@thread_pool = Concurrent::ThreadPoolExecutor.new(
|
6
|
+
name: "tldr",
|
7
|
+
auto_terminate: true
|
8
|
+
)
|
9
|
+
end
|
10
|
+
|
11
|
+
def parallelize all_tests, parallel, &blk
|
12
|
+
return run_in_sequence(all_tests, &blk) if all_tests.size < 2 || !parallel
|
13
|
+
|
14
|
+
strategy = @strategizer.strategize all_tests, GROUPED_TESTS, THREAD_UNSAFE_TESTS
|
15
|
+
|
16
|
+
run_in_parallel(strategy.parallel_tests_and_groups, &blk) +
|
17
|
+
run_in_sequence(strategy.thread_unsafe_tests, &blk)
|
18
|
+
end
|
19
|
+
|
20
|
+
private
|
21
|
+
|
22
|
+
def run_in_sequence tests, &blk
|
23
|
+
tests.map(&blk)
|
24
|
+
end
|
25
|
+
|
26
|
+
def run_in_parallel tests_and_groups, &blk
|
27
|
+
tests_and_groups.map { |test_or_group|
|
28
|
+
tests_to_run = if test_or_group.group?
|
29
|
+
test_or_group.tests
|
30
|
+
else
|
31
|
+
[test_or_group]
|
32
|
+
end
|
33
|
+
|
34
|
+
unless tests_to_run.empty?
|
35
|
+
Concurrent::Promises.future_on(@thread_pool) {
|
36
|
+
tests_to_run.map(&blk)
|
37
|
+
}
|
38
|
+
end
|
39
|
+
}.compact.flat_map(&:value)
|
40
|
+
end
|
41
|
+
end
|
42
|
+
end
|
data/lib/tldr/planner.rb
CHANGED
@@ -50,8 +50,7 @@ class TLDR
|
|
50
50
|
def gather_tests
|
51
51
|
gather_descendants(TLDR).flat_map { |subklass|
|
52
52
|
subklass.instance_methods.grep(/^test_/).sort.map { |method|
|
53
|
-
|
54
|
-
Test.new subklass, method, file, line
|
53
|
+
Test.new subklass, method
|
55
54
|
}
|
56
55
|
}
|
57
56
|
end
|
@@ -17,12 +17,17 @@ class TLDR
|
|
17
17
|
end
|
18
18
|
|
19
19
|
def after_test result
|
20
|
-
|
20
|
+
output = case result.type
|
21
21
|
when :success then @icons.success
|
22
22
|
when :skip then @icons.skip
|
23
23
|
when :failure then @icons.failure
|
24
24
|
when :error then @icons.error
|
25
25
|
end
|
26
|
+
if @config.verbose
|
27
|
+
@out.puts "#{output} #{result.type.capitalize} - #{describe(result.test, result.relevant_location)}"
|
28
|
+
else
|
29
|
+
@out.print output
|
30
|
+
end
|
26
31
|
end
|
27
32
|
|
28
33
|
def time_diff start, stop = Process.clock_gettime(Process::CLOCK_MONOTONIC, :microsecond)
|
data/lib/tldr/runner.rb
CHANGED
@@ -1,9 +1,9 @@
|
|
1
1
|
require "irb"
|
2
|
-
require "concurrent"
|
3
2
|
|
4
3
|
class TLDR
|
5
4
|
class Runner
|
6
5
|
def initialize
|
6
|
+
@parallelizer = Parallelizer.new
|
7
7
|
@wip = Concurrent::Array.new
|
8
8
|
@results = Concurrent::Array.new
|
9
9
|
@run_aborted = Concurrent::AtomicBoolean.new false
|
@@ -34,7 +34,7 @@ class TLDR
|
|
34
34
|
end
|
35
35
|
}
|
36
36
|
|
37
|
-
results = parallelize(plan.tests, config.parallel) { |test|
|
37
|
+
results = @parallelizer.parallelize(plan.tests, config.parallel) { |test|
|
38
38
|
e = nil
|
39
39
|
start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC, :microsecond)
|
40
40
|
wip_test = WIPTest.new test, start_time
|
@@ -42,7 +42,16 @@ class TLDR
|
|
42
42
|
runtime = time_it(start_time) do
|
43
43
|
instance = test.klass.new
|
44
44
|
instance.setup if instance.respond_to? :setup
|
45
|
-
instance.
|
45
|
+
if instance.respond_to? :around
|
46
|
+
did_run = false
|
47
|
+
instance.around {
|
48
|
+
did_run = true
|
49
|
+
instance.send(test.method)
|
50
|
+
}
|
51
|
+
raise Error, "#{test.klass}#around failed to yield or call the passed test block" unless did_run
|
52
|
+
else
|
53
|
+
instance.send(test.method)
|
54
|
+
end
|
46
55
|
instance.teardown if instance.respond_to? :teardown
|
47
56
|
rescue Skip, Failure, StandardError => e
|
48
57
|
end
|
@@ -65,20 +74,6 @@ class TLDR
|
|
65
74
|
|
66
75
|
private
|
67
76
|
|
68
|
-
def parallelize tests, parallel, &blk
|
69
|
-
return tests.map(&blk) if tests.size < 2 || !parallel
|
70
|
-
tldr_pool = Concurrent::ThreadPoolExecutor.new(
|
71
|
-
name: "tldr",
|
72
|
-
auto_terminate: true
|
73
|
-
)
|
74
|
-
|
75
|
-
tests.map { |test|
|
76
|
-
Concurrent::Promises.future_on(tldr_pool) {
|
77
|
-
blk.call test
|
78
|
-
}
|
79
|
-
}.flat_map(&:value)
|
80
|
-
end
|
81
|
-
|
82
77
|
def fail_fast reporter, plan, fast_failed_result
|
83
78
|
unless @run_aborted.true?
|
84
79
|
@run_aborted.make_true
|
@@ -0,0 +1,54 @@
|
|
1
|
+
class TLDR
|
2
|
+
class Strategizer
|
3
|
+
Strategy = Struct.new :parallel_tests_and_groups, :thread_unsafe_tests
|
4
|
+
|
5
|
+
# Combine all discovered test methods with any methods grouped by run_these_together!
|
6
|
+
#
|
7
|
+
# Priorities:
|
8
|
+
# - Map over tests to build out groups in order to retain shuffle order
|
9
|
+
# (group will run in position of first test in the group)
|
10
|
+
# - If a test is in multiple groups, only run it once
|
11
|
+
def strategize all_tests, run_these_together_groups, thread_unsafe_test_groups
|
12
|
+
thread_unsafe_tests, thread_safe_tests = partition_unsafe(all_tests, thread_unsafe_test_groups)
|
13
|
+
|
14
|
+
grouped_tests = prepare_run_together_groups run_these_together_groups, thread_safe_tests, thread_unsafe_tests
|
15
|
+
already_included_groups = []
|
16
|
+
|
17
|
+
Strategy.new thread_safe_tests.map { |test|
|
18
|
+
if (group = grouped_tests.find { |group| group.tests.include? test })
|
19
|
+
if already_included_groups.include? group
|
20
|
+
next
|
21
|
+
elsif (other = already_included_groups.find { |other| (group.tests & other.tests).any? })
|
22
|
+
other.tests |= group.tests
|
23
|
+
next
|
24
|
+
else
|
25
|
+
already_included_groups << group
|
26
|
+
group
|
27
|
+
end
|
28
|
+
else
|
29
|
+
test
|
30
|
+
end
|
31
|
+
}.compact, thread_unsafe_tests
|
32
|
+
end
|
33
|
+
|
34
|
+
private
|
35
|
+
|
36
|
+
def partition_unsafe tests, thread_unsafe_test_groups
|
37
|
+
tests.partition { |test|
|
38
|
+
thread_unsafe_test_groups.any? { |group| group.tests.include? test }
|
39
|
+
}
|
40
|
+
end
|
41
|
+
|
42
|
+
def prepare_run_together_groups run_these_together_groups, thread_safe_tests, thread_unsafe_tests
|
43
|
+
grouped_tests = run_these_together_groups.map(&:dup)
|
44
|
+
|
45
|
+
grouped_tests.each do |group|
|
46
|
+
group.tests = group.tests.select { |test|
|
47
|
+
thread_safe_tests.include?(test) && !thread_unsafe_tests.include?(test)
|
48
|
+
}
|
49
|
+
end
|
50
|
+
|
51
|
+
grouped_tests.reject { |group| group.tests.size < 2 }
|
52
|
+
end
|
53
|
+
end
|
54
|
+
end
|
data/lib/tldr/value/config.rb
CHANGED
data/lib/tldr/value/test.rb
CHANGED
@@ -1,10 +1,11 @@
|
|
1
1
|
class TLDR
|
2
|
-
Test = Struct.new :klass, :method
|
3
|
-
attr_reader :location
|
2
|
+
Test = Struct.new :klass, :method do
|
3
|
+
attr_reader :file, :line, :location
|
4
4
|
|
5
5
|
def initialize(*args)
|
6
6
|
super
|
7
|
-
@
|
7
|
+
@file, @line = SorbetCompatibility.unwrap_method(klass.instance_method(method)).source_location
|
8
|
+
@location = Location.new file, line
|
8
9
|
end
|
9
10
|
|
10
11
|
# Memoizing at call time, because re-parsing isn't free and isn't usually necessary
|
@@ -19,5 +20,9 @@ class TLDR
|
|
19
20
|
def covers_line? l
|
20
21
|
line == l || (l >= line && l <= end_line)
|
21
22
|
end
|
23
|
+
|
24
|
+
def group?
|
25
|
+
false
|
26
|
+
end
|
22
27
|
end
|
23
28
|
end
|
@@ -0,0 +1,22 @@
|
|
1
|
+
class TLDR
|
2
|
+
TestGroup = Struct.new :configuration do
|
3
|
+
attr_writer :tests
|
4
|
+
|
5
|
+
def tests
|
6
|
+
@tests ||= configuration.flat_map { |(klass, method)|
|
7
|
+
klass = Kernel.const_get(klass) if klass.is_a? String
|
8
|
+
if method.nil?
|
9
|
+
klass.instance_methods.grep(/^test_/).sort.map { |method|
|
10
|
+
Test.new klass, method
|
11
|
+
}
|
12
|
+
else
|
13
|
+
Test.new klass, method
|
14
|
+
end
|
15
|
+
}
|
16
|
+
end
|
17
|
+
|
18
|
+
def group?
|
19
|
+
true
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
data/lib/tldr/value.rb
CHANGED
data/lib/tldr/version.rb
CHANGED
data/lib/tldr.rb
CHANGED
@@ -1,14 +1,19 @@
|
|
1
|
-
|
1
|
+
require "concurrent-ruby"
|
2
|
+
|
3
|
+
require_relative "tldr/argv_parser"
|
4
|
+
require_relative "tldr/assertions"
|
2
5
|
require_relative "tldr/backtrace_filter"
|
3
6
|
require_relative "tldr/error"
|
4
|
-
require_relative "tldr/
|
5
|
-
require_relative "tldr/
|
6
|
-
require_relative "tldr/argv_parser"
|
7
|
+
require_relative "tldr/parallel_controls"
|
8
|
+
require_relative "tldr/parallelizer"
|
7
9
|
require_relative "tldr/planner"
|
10
|
+
require_relative "tldr/reporters"
|
8
11
|
require_relative "tldr/runner"
|
9
|
-
require_relative "tldr/assertions"
|
10
12
|
require_relative "tldr/skippable"
|
11
13
|
require_relative "tldr/sorbet_compatibility"
|
14
|
+
require_relative "tldr/strategizer"
|
15
|
+
require_relative "tldr/value"
|
16
|
+
require_relative "tldr/version"
|
12
17
|
|
13
18
|
class TLDR
|
14
19
|
include Assertions
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: tldr
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Justin Searls
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: exe
|
11
11
|
cert_chain: []
|
12
|
-
date: 2023-09-
|
12
|
+
date: 2023-09-27 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: super_diff
|
@@ -60,6 +60,8 @@ files:
|
|
60
60
|
- lib/tldr/assertions/minitest_compatibility.rb
|
61
61
|
- lib/tldr/backtrace_filter.rb
|
62
62
|
- lib/tldr/error.rb
|
63
|
+
- lib/tldr/parallel_controls.rb
|
64
|
+
- lib/tldr/parallelizer.rb
|
63
65
|
- lib/tldr/planner.rb
|
64
66
|
- lib/tldr/rake.rb
|
65
67
|
- lib/tldr/reporters.rb
|
@@ -69,11 +71,13 @@ files:
|
|
69
71
|
- lib/tldr/runner.rb
|
70
72
|
- lib/tldr/skippable.rb
|
71
73
|
- lib/tldr/sorbet_compatibility.rb
|
74
|
+
- lib/tldr/strategizer.rb
|
72
75
|
- lib/tldr/value.rb
|
73
76
|
- lib/tldr/value/config.rb
|
74
77
|
- lib/tldr/value/location.rb
|
75
78
|
- lib/tldr/value/plan.rb
|
76
79
|
- lib/tldr/value/test.rb
|
80
|
+
- lib/tldr/value/test_group.rb
|
77
81
|
- lib/tldr/value/test_result.rb
|
78
82
|
- lib/tldr/value/wip_test.rb
|
79
83
|
- lib/tldr/version.rb
|
@@ -101,7 +105,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
101
105
|
- !ruby/object:Gem::Version
|
102
106
|
version: '0'
|
103
107
|
requirements: []
|
104
|
-
rubygems_version: 3.4.
|
108
|
+
rubygems_version: 3.4.17
|
105
109
|
signing_key:
|
106
110
|
specification_version: 4
|
107
111
|
summary: TLDR will run your tests, but only for 1.8 seconds.
|