smartest 0.1.0.alpha1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +16 -0
- data/DEVELOPMENT.md +774 -0
- data/Gemfile +5 -0
- data/LICENSE +21 -0
- data/README.md +518 -0
- data/Rakefile +12 -0
- data/SMARTEST_DESIGN.md +1137 -0
- data/exe/smartest +63 -0
- data/lib/smartest/autorun.rb +8 -0
- data/lib/smartest/dsl.rb +22 -0
- data/lib/smartest/errors.rb +52 -0
- data/lib/smartest/execution_context.rb +8 -0
- data/lib/smartest/expectation_target.rb +21 -0
- data/lib/smartest/expectations.rb +9 -0
- data/lib/smartest/fixture.rb +78 -0
- data/lib/smartest/fixture_class_registry.rb +27 -0
- data/lib/smartest/fixture_definition.rb +31 -0
- data/lib/smartest/fixture_set.rb +119 -0
- data/lib/smartest/init_generator.rb +70 -0
- data/lib/smartest/matchers.rb +109 -0
- data/lib/smartest/parameter_extractor.rb +51 -0
- data/lib/smartest/reporter.rb +91 -0
- data/lib/smartest/runner.rb +80 -0
- data/lib/smartest/suite.rb +12 -0
- data/lib/smartest/test_case.rb +18 -0
- data/lib/smartest/test_registry.rb +25 -0
- data/lib/smartest/test_result.rb +43 -0
- data/lib/smartest/version.rb +5 -0
- data/lib/smartest.rb +59 -0
- data/smartest/smartest_test.rb +634 -0
- data/smartest.gemspec +48 -0
- metadata +95 -0
data/Gemfile
ADDED
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Yusuke Iwaki
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,518 @@
|
|
|
1
|
+
# Smartest
|
|
2
|
+
|
|
3
|
+
Smartest is a small Ruby test runner with a keyword-fixture-first design.
|
|
4
|
+
|
|
5
|
+
It lets you write tests like this:
|
|
6
|
+
|
|
7
|
+
```ruby
|
|
8
|
+
test("factorial") do
|
|
9
|
+
expect(1 * 2 * 3).to eq(6)
|
|
10
|
+
end
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
and fixture-driven tests like this:
|
|
14
|
+
|
|
15
|
+
```ruby
|
|
16
|
+
test("GET /me") do |logged_in_client:|
|
|
17
|
+
response = logged_in_client.get("/me")
|
|
18
|
+
|
|
19
|
+
expect(response.status).to eq(200)
|
|
20
|
+
end
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
Smartest is designed around three ideas:
|
|
24
|
+
|
|
25
|
+
1. Tests should be readable at the top level.
|
|
26
|
+
2. Fixture dependencies should be explicit.
|
|
27
|
+
3. Teardown should be written only when it is needed.
|
|
28
|
+
|
|
29
|
+
## Installation
|
|
30
|
+
|
|
31
|
+
Add this line to your application's Gemfile:
|
|
32
|
+
|
|
33
|
+
```ruby
|
|
34
|
+
gem "smartest"
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Then run:
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
bundle install
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Or install it directly:
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
gem install smartest
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
## Quick start
|
|
50
|
+
|
|
51
|
+
Initialize a test scaffold:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
bundle exec smartest --init
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
This creates:
|
|
58
|
+
|
|
59
|
+
```text
|
|
60
|
+
smartest/test_helper.rb
|
|
61
|
+
smartest/fixtures/
|
|
62
|
+
smartest/example_test.rb
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
The generated example looks like this:
|
|
66
|
+
|
|
67
|
+
```ruby
|
|
68
|
+
# smartest/example_test.rb
|
|
69
|
+
require "test_helper"
|
|
70
|
+
|
|
71
|
+
test("example") do
|
|
72
|
+
expect(1 + 1).to eq(2)
|
|
73
|
+
end
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Run the suite:
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
bundle exec smartest
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
By default, Smartest loads `smartest/**/*_test.rb`, so a separate `test/`
|
|
83
|
+
directory can remain available for Minitest.
|
|
84
|
+
|
|
85
|
+
You can also pass explicit paths:
|
|
86
|
+
|
|
87
|
+
```bash
|
|
88
|
+
bundle exec smartest smartest/**/*_test.rb
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
CLI help and version output are available with:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
bundle exec smartest --help
|
|
95
|
+
bundle exec smartest --version
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Expected output:
|
|
99
|
+
|
|
100
|
+
```text
|
|
101
|
+
Running 1 test
|
|
102
|
+
|
|
103
|
+
✓ example
|
|
104
|
+
|
|
105
|
+
1 test, 1 passed, 0 failed
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
## Defining tests
|
|
109
|
+
|
|
110
|
+
Use `test` at the top level:
|
|
111
|
+
|
|
112
|
+
```ruby
|
|
113
|
+
test("adds numbers") do
|
|
114
|
+
expect(1 + 2).to eq(3)
|
|
115
|
+
end
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
A test can request fixtures using required keyword arguments:
|
|
119
|
+
|
|
120
|
+
```ruby
|
|
121
|
+
test("uses a user") do |user:|
|
|
122
|
+
expect(user.name).to eq("Alice")
|
|
123
|
+
end
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Smartest intentionally favors keyword arguments for fixture injection:
|
|
127
|
+
|
|
128
|
+
```ruby
|
|
129
|
+
test("GET /me") do |logged_in_client:|
|
|
130
|
+
# ...
|
|
131
|
+
end
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
This makes fixture usage explicit and avoids relying on positional argument order.
|
|
135
|
+
|
|
136
|
+
## Expectations
|
|
137
|
+
|
|
138
|
+
Smartest uses an expectation style:
|
|
139
|
+
|
|
140
|
+
```ruby
|
|
141
|
+
expect(actual).to eq(expected)
|
|
142
|
+
expect(actual).not_to eq(expected)
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
Examples:
|
|
146
|
+
|
|
147
|
+
```ruby
|
|
148
|
+
test("string") do
|
|
149
|
+
expect("hello").to eq("hello")
|
|
150
|
+
end
|
|
151
|
+
|
|
152
|
+
test("array") do
|
|
153
|
+
expect([1, 2, 3]).to include(2)
|
|
154
|
+
end
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
Supported matchers include:
|
|
158
|
+
|
|
159
|
+
```ruby
|
|
160
|
+
eq(expected)
|
|
161
|
+
include(expected)
|
|
162
|
+
be_nil
|
|
163
|
+
raise_error(ErrorClass)
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
## Fixtures
|
|
167
|
+
|
|
168
|
+
Fixtures are defined in classes.
|
|
169
|
+
|
|
170
|
+
```ruby
|
|
171
|
+
class AppFixture < Smartest::Fixture
|
|
172
|
+
fixture :user do
|
|
173
|
+
User.create!(
|
|
174
|
+
name: "Alice",
|
|
175
|
+
email: "alice@example.com"
|
|
176
|
+
)
|
|
177
|
+
end
|
|
178
|
+
end
|
|
179
|
+
|
|
180
|
+
use_fixture AppFixture
|
|
181
|
+
|
|
182
|
+
test("user") do |user:|
|
|
183
|
+
expect(user.name).to eq("Alice")
|
|
184
|
+
end
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
A fixture is requested by name from a test block keyword argument.
|
|
188
|
+
|
|
189
|
+
```ruby
|
|
190
|
+
test("user") do |user:|
|
|
191
|
+
# Smartest resolves the `user` fixture
|
|
192
|
+
end
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
## Fixture dependencies
|
|
196
|
+
|
|
197
|
+
Fixtures can depend on other fixtures using required keyword arguments.
|
|
198
|
+
|
|
199
|
+
```ruby
|
|
200
|
+
class AppFixture < Smartest::Fixture
|
|
201
|
+
suite_fixture :server do
|
|
202
|
+
TestServer.start
|
|
203
|
+
end
|
|
204
|
+
|
|
205
|
+
fixture :client do |server:|
|
|
206
|
+
Client.new(base_url: server.url)
|
|
207
|
+
end
|
|
208
|
+
end
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
The dependency is explicit:
|
|
212
|
+
|
|
213
|
+
```ruby
|
|
214
|
+
fixture :client do |server:|
|
|
215
|
+
Client.new(base_url: server.url)
|
|
216
|
+
end
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
When a test requests `client`, Smartest resolves `server` first.
|
|
220
|
+
|
|
221
|
+
```ruby
|
|
222
|
+
test("GET /health") do |client:|
|
|
223
|
+
response = client.get("/health")
|
|
224
|
+
|
|
225
|
+
expect(response.status).to eq(200)
|
|
226
|
+
end
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
## Fixture scopes
|
|
230
|
+
|
|
231
|
+
Regular `fixture` definitions are test-scoped. Smartest creates a fresh value
|
|
232
|
+
for each test that requests the fixture.
|
|
233
|
+
|
|
234
|
+
Use `suite_fixture` for expensive resources that should be created once and
|
|
235
|
+
released after the full suite finishes:
|
|
236
|
+
|
|
237
|
+
```ruby
|
|
238
|
+
class BrowserFixture < Smartest::Fixture
|
|
239
|
+
suite_fixture :browser do
|
|
240
|
+
browser = Browser.launch
|
|
241
|
+
cleanup { browser.close }
|
|
242
|
+
browser
|
|
243
|
+
end
|
|
244
|
+
|
|
245
|
+
fixture :page do |browser:|
|
|
246
|
+
browser.new_page
|
|
247
|
+
end
|
|
248
|
+
end
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
Suite fixtures are lazy: setup runs the first time a test requests the fixture,
|
|
252
|
+
and cleanup runs once after all tests finish. Test-scoped fixtures can depend on
|
|
253
|
+
suite fixtures, but suite fixtures cannot depend on test-scoped fixtures.
|
|
254
|
+
|
|
255
|
+
## Fixtures with teardown
|
|
256
|
+
|
|
257
|
+
Not every fixture needs teardown. For fixtures that do, use `cleanup`.
|
|
258
|
+
|
|
259
|
+
```ruby
|
|
260
|
+
class WebFixture < Smartest::Fixture
|
|
261
|
+
fixture :server do
|
|
262
|
+
server = TestServer.start
|
|
263
|
+
cleanup { server.stop }
|
|
264
|
+
|
|
265
|
+
server.wait_until_ready!
|
|
266
|
+
server
|
|
267
|
+
end
|
|
268
|
+
|
|
269
|
+
fixture :client do |server:|
|
|
270
|
+
Client.new(base_url: server.url)
|
|
271
|
+
end
|
|
272
|
+
end
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
`cleanup` blocks run after the fixture's scope finishes. For regular fixtures
|
|
276
|
+
that means after the test. For `suite_fixture`, cleanup runs after the full
|
|
277
|
+
suite.
|
|
278
|
+
|
|
279
|
+
They are executed in reverse order of registration.
|
|
280
|
+
|
|
281
|
+
```ruby
|
|
282
|
+
fixture :temp_dir do
|
|
283
|
+
dir = Dir.mktmpdir
|
|
284
|
+
cleanup { FileUtils.rm_rf(dir) }
|
|
285
|
+
|
|
286
|
+
dir
|
|
287
|
+
end
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
Recommended pattern:
|
|
291
|
+
|
|
292
|
+
```ruby
|
|
293
|
+
fixture :server do
|
|
294
|
+
server = TestServer.start
|
|
295
|
+
cleanup { server.stop }
|
|
296
|
+
|
|
297
|
+
server.wait_until_ready!
|
|
298
|
+
server
|
|
299
|
+
end
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
Register cleanup immediately after acquiring the resource, before later setup steps that may fail.
|
|
303
|
+
|
|
304
|
+
## Logged-in client example
|
|
305
|
+
|
|
306
|
+
```ruby
|
|
307
|
+
class WebFixture < Smartest::Fixture
|
|
308
|
+
fixture :server do
|
|
309
|
+
server = TestServer.start
|
|
310
|
+
cleanup { server.stop }
|
|
311
|
+
|
|
312
|
+
server.wait_until_ready!
|
|
313
|
+
server
|
|
314
|
+
end
|
|
315
|
+
|
|
316
|
+
fixture :client do |server:|
|
|
317
|
+
Client.new(base_url: server.url)
|
|
318
|
+
end
|
|
319
|
+
|
|
320
|
+
fixture :user do
|
|
321
|
+
User.create!(
|
|
322
|
+
name: "Alice",
|
|
323
|
+
email: "alice@example.com"
|
|
324
|
+
)
|
|
325
|
+
end
|
|
326
|
+
|
|
327
|
+
fixture :logged_in_client do |client:, user:|
|
|
328
|
+
client.login(user)
|
|
329
|
+
client
|
|
330
|
+
end
|
|
331
|
+
end
|
|
332
|
+
|
|
333
|
+
use_fixture WebFixture
|
|
334
|
+
|
|
335
|
+
test("GET /me") do |logged_in_client:|
|
|
336
|
+
response = logged_in_client.get("/me")
|
|
337
|
+
|
|
338
|
+
expect(response.status).to eq(200)
|
|
339
|
+
end
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
Dependency graph:
|
|
343
|
+
|
|
344
|
+
```text
|
|
345
|
+
logged_in_client
|
|
346
|
+
├── client
|
|
347
|
+
│ └── server
|
|
348
|
+
└── user
|
|
349
|
+
```
|
|
350
|
+
|
|
351
|
+
Execution flow:
|
|
352
|
+
|
|
353
|
+
```text
|
|
354
|
+
server setup
|
|
355
|
+
client setup
|
|
356
|
+
user setup
|
|
357
|
+
logged_in_client setup
|
|
358
|
+
test body
|
|
359
|
+
server cleanup
|
|
360
|
+
```
|
|
361
|
+
|
|
362
|
+
## Registering fixture classes
|
|
363
|
+
|
|
364
|
+
Use `use_fixture`:
|
|
365
|
+
|
|
366
|
+
```ruby
|
|
367
|
+
use_fixture AppFixture
|
|
368
|
+
```
|
|
369
|
+
|
|
370
|
+
Multiple fixture classes can be registered:
|
|
371
|
+
|
|
372
|
+
```ruby
|
|
373
|
+
use_fixture UserFixture
|
|
374
|
+
use_fixture WebFixture
|
|
375
|
+
use_fixture ApiFixture
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
Fixture names must be unique across registered fixture classes.
|
|
379
|
+
|
|
380
|
+
If two fixture classes define the same fixture name, Smartest raises an error.
|
|
381
|
+
|
|
382
|
+
## Hooks
|
|
383
|
+
|
|
384
|
+
Smartest may support simple hooks:
|
|
385
|
+
|
|
386
|
+
```ruby
|
|
387
|
+
before do
|
|
388
|
+
DatabaseCleaner.start
|
|
389
|
+
end
|
|
390
|
+
|
|
391
|
+
after do
|
|
392
|
+
DatabaseCleaner.clean
|
|
393
|
+
end
|
|
394
|
+
```
|
|
395
|
+
|
|
396
|
+
Hooks are separate from fixture cleanup.
|
|
397
|
+
|
|
398
|
+
Use fixture cleanup for resource-specific teardown:
|
|
399
|
+
|
|
400
|
+
```ruby
|
|
401
|
+
fixture :server do
|
|
402
|
+
server = TestServer.start
|
|
403
|
+
cleanup { server.stop }
|
|
404
|
+
server
|
|
405
|
+
end
|
|
406
|
+
```
|
|
407
|
+
|
|
408
|
+
Use hooks for broad test-level behavior:
|
|
409
|
+
|
|
410
|
+
```ruby
|
|
411
|
+
before do
|
|
412
|
+
reset_global_state
|
|
413
|
+
end
|
|
414
|
+
```
|
|
415
|
+
|
|
416
|
+
## Recommended file structure
|
|
417
|
+
|
|
418
|
+
```text
|
|
419
|
+
smartest/
|
|
420
|
+
test_helper.rb
|
|
421
|
+
fixtures/
|
|
422
|
+
app_fixture.rb
|
|
423
|
+
web_fixture.rb
|
|
424
|
+
example_test.rb
|
|
425
|
+
```
|
|
426
|
+
|
|
427
|
+
```ruby
|
|
428
|
+
# smartest/test_helper.rb
|
|
429
|
+
require "smartest/autorun"
|
|
430
|
+
|
|
431
|
+
Dir[File.join(__dir__, "fixtures", "**", "*.rb")].sort.each do |fixture_file|
|
|
432
|
+
require fixture_file
|
|
433
|
+
end
|
|
434
|
+
```
|
|
435
|
+
|
|
436
|
+
The generated helper loads Ruby files under `smartest/fixtures/` in sorted order.
|
|
437
|
+
Test files still register the fixture classes they need with `use_fixture`.
|
|
438
|
+
|
|
439
|
+
Example:
|
|
440
|
+
|
|
441
|
+
```ruby
|
|
442
|
+
# smartest/fixtures/web_fixture.rb
|
|
443
|
+
class WebFixture < Smartest::Fixture
|
|
444
|
+
fixture :server do
|
|
445
|
+
server = TestServer.start
|
|
446
|
+
cleanup { server.stop }
|
|
447
|
+
server
|
|
448
|
+
end
|
|
449
|
+
|
|
450
|
+
fixture :client do |server:|
|
|
451
|
+
Client.new(base_url: server.url)
|
|
452
|
+
end
|
|
453
|
+
end
|
|
454
|
+
```
|
|
455
|
+
|
|
456
|
+
```ruby
|
|
457
|
+
# smartest/example_test.rb
|
|
458
|
+
require "test_helper"
|
|
459
|
+
|
|
460
|
+
use_fixture WebFixture
|
|
461
|
+
|
|
462
|
+
test("GET /health") do |client:|
|
|
463
|
+
expect(client.get("/health").status).to eq(200)
|
|
464
|
+
end
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
## Design choices
|
|
468
|
+
|
|
469
|
+
Smartest intentionally does not use this style as the primary API:
|
|
470
|
+
|
|
471
|
+
```ruby
|
|
472
|
+
test("GET /me") do |logged_in_client|
|
|
473
|
+
end
|
|
474
|
+
```
|
|
475
|
+
|
|
476
|
+
Instead, Smartest prefers:
|
|
477
|
+
|
|
478
|
+
```ruby
|
|
479
|
+
test("GET /me") do |logged_in_client:|
|
|
480
|
+
end
|
|
481
|
+
```
|
|
482
|
+
|
|
483
|
+
Keyword arguments make fixture injection explicit.
|
|
484
|
+
|
|
485
|
+
Smartest also avoids this fixture dependency style:
|
|
486
|
+
|
|
487
|
+
```ruby
|
|
488
|
+
fixture :client, with: [:server] do |server|
|
|
489
|
+
Client.new(base_url: server.url)
|
|
490
|
+
end
|
|
491
|
+
```
|
|
492
|
+
|
|
493
|
+
Instead, it prefers:
|
|
494
|
+
|
|
495
|
+
```ruby
|
|
496
|
+
fixture :client do |server:|
|
|
497
|
+
Client.new(base_url: server.url)
|
|
498
|
+
end
|
|
499
|
+
```
|
|
500
|
+
|
|
501
|
+
The dependency declaration and usage are in one place.
|
|
502
|
+
|
|
503
|
+
## Status
|
|
504
|
+
|
|
505
|
+
Smartest is currently a design-stage test runner.
|
|
506
|
+
|
|
507
|
+
The intended MVP includes:
|
|
508
|
+
|
|
509
|
+
- top-level `test`
|
|
510
|
+
- class-based fixtures
|
|
511
|
+
- keyword-argument fixture injection
|
|
512
|
+
- fixture dependencies through keyword arguments
|
|
513
|
+
- fixture cleanup
|
|
514
|
+
- `expect(...).to eq(...)`
|
|
515
|
+
- console reporter
|
|
516
|
+
- CLI runner
|
|
517
|
+
- circular fixture dependency detection
|
|
518
|
+
- duplicate fixture detection
|