test_data 0.2.0 → 0.2.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.standard.yml +2 -0
- data/CHANGELOG.md +7 -0
- data/Gemfile.lock +1 -1
- data/README.md +588 -693
- data/example/.gitignore +1 -4
- data/example/Gemfile.lock +1 -1
- data/example/config/application.rb +3 -0
- data/example/spec/requests/rails_fixtures_override_spec.rb +26 -4
- data/example/test/integration/test_data_hooks_test.rb +89 -0
- data/lib/generators/test_data/initializer_generator.rb +18 -6
- data/lib/test_data/config.rb +29 -1
- data/lib/test_data/custom_loaders/rails_fixtures.rb +2 -0
- data/lib/test_data/manager.rb +2 -0
- data/lib/test_data/rake.rb +2 -3
- data/lib/test_data/version.rb +1 -1
- data/script/reset_example_app +1 -0
- data/script/test +18 -6
- metadata +4 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 84a9fc3fe7b3dda0cf1ebdc775c9191757d21de11e033189c6450b91f63332e7
|
4
|
+
data.tar.gz: 0a47179f0607d9c57f63706e35de176b14d93989c6e5406094524a0fb3b09704
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 4c10de3f9e234ae53e47553d099612dc7474cb3e98022b6a953d60beb8da3f502b00f506f29729dad0a9eb37c8e7f05a662263963300d7c531f48fef660c9173
|
7
|
+
data.tar.gz: d718e34070ca15bd9d0e27c75789e170e42da183fac8efc973404d9cea909ce984453f9393277b37ad36ff42fa4866c66de14d2a82a149ae536f0555c2f5f52d
|
data/.standard.yml
ADDED
data/CHANGELOG.md
CHANGED
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -1,9 +1,5 @@
|
|
1
1
|
# The `test_data` gem
|
2
2
|
|
3
|
-
**HEADS UP: 0.2.0 made a whole bunch of breaking changes to the public API and
|
4
|
-
we haven't finished rewriting the README yet. Please bear with us while we work
|
5
|
-
through it. 🙇**
|
6
|
-
|
7
3
|
`test_data` does what it says on the tin: it provides a fast & reliable system
|
8
4
|
for managing your Rails application's test data.
|
9
5
|
|
@@ -22,23 +18,24 @@ What it does:
|
|
22
18
|
files, no precarious approximations of realism: **real data created by your
|
23
19
|
app**
|
24
20
|
|
25
|
-
* Exposes a simple API for ensuring that
|
26
|
-
|
27
|
-
|
21
|
+
* Exposes a simple API for ensuring that your data will be pristine for each of
|
22
|
+
your tests, whether the test depends on test_data, an empty database, or Rails
|
23
|
+
fixtures
|
28
24
|
|
29
25
|
* Safeguards your tests from flaky failures and supercharges your build by
|
30
26
|
providing a sophisticated transaction manager that isolates each test while
|
31
27
|
ensuring your data is only loaded once
|
32
28
|
|
33
29
|
If you've despaired over the seeming inevitability that all Rails test suites
|
34
|
-
will eventually grow to become slow,
|
35
|
-
|
30
|
+
will eventually grow to become slow, flaky, and incomprehensible, then this gem
|
31
|
+
is for you! And even if you're [a factory_bot
|
36
32
|
fan](https://twitter.com/searls/status/1379491813099253762?s=20), we hope you'll
|
37
33
|
be open to the idea that [there might be a better way](
|
38
34
|
#but-we-use-and-like-factory_bot-and-so-i-am-inclined-to-dislike-everything-about-this-gem).
|
39
35
|
|
40
36
|
_[Full disclosure: because the gem is still brand new, it makes a number of
|
41
|
-
[assumptions](#assumptions)
|
37
|
+
[assumptions](#assumptions)—chief among them being that **Postgres & Rails 6+
|
38
|
+
are required**—so it may not work for every project just yet.]_
|
42
39
|
|
43
40
|
## Documentation
|
44
41
|
|
@@ -46,10 +43,9 @@ This gem requires a lot of documentation—not because `test_data` does a lot of
|
|
46
43
|
things, but because managing one's test data is an inherently complex task. If
|
47
44
|
one reason Rails apps chronically suffer from slow tests is that other
|
48
45
|
approaches oversimplify test data management, it stands to reason that any
|
49
|
-
discomfort caused by `test_data`'s scope may not
|
50
|
-
|
51
|
-
|
52
|
-
application.
|
46
|
+
discomfort caused by `test_data`'s scope may not be _unnecessary complexity_ but
|
47
|
+
instead be an indication of how little of the problem's _essential complexity_
|
48
|
+
we have reckoned with to this point.
|
53
49
|
|
54
50
|
1. [Getting Started Guide](#getting-started-guide)
|
55
51
|
1. [Install and initialize `test_data`](#step-1-install-and-initialize-test_data)
|
@@ -70,14 +66,26 @@ application.
|
|
70
66
|
* [test_data:create_database](#test_datacreate_database)
|
71
67
|
* [test_data:drop_database](#test_datadrop_database)
|
72
68
|
4. [API Reference](#api-reference)
|
73
|
-
* [TestData.config](#testdataconfig)
|
74
69
|
* [TestData.uses_test_data](#testdatauses_test_data)
|
75
70
|
* [TestData.uses_clean_slate](#testdatauses_clean_slate)
|
76
|
-
* [TestData.prevent_rails_fixtures_from_loading_automatically!](#testdataprevent_rails_fixtures_from_loading_automatically)
|
77
71
|
* [TestData.uses_rails_fixtures(self)](#testdatauses_rails_fixtures)
|
72
|
+
* [TestData.prevent_rails_fixtures_from_loading_automatically!](#testdataprevent_rails_fixtures_from_loading_automatically)
|
73
|
+
* [TestData.config](#testdataconfig)
|
78
74
|
* [TestData.insert_test_data_dump](#testdatainsert_test_data_dump)
|
79
75
|
5. [Assumptions](#assumptions)
|
80
76
|
6. [Fears, Uncertainties, and Doubts](#fears-uncertainties-and-doubts) (Q & A)
|
77
|
+
* [But we're already happy with
|
78
|
+
factory_bot!](#but-we-use-and-like-factory_bot-and-so-i-am-inclined-to-dislike-everything-about-this-gem)
|
79
|
+
* [How will we handle merge conflicts in the schema
|
80
|
+
dumps?](#how-will-i-handle-merge-conflicts-in-these-sql-files-if-i-have-lots-of-people-working-on-lots-of-feature-branches-all-adding-to-the-test_data-database-dumps)
|
81
|
+
* [Why can't I manage different SQL dumps for different
|
82
|
+
scenarios?](#why-cant-i-save-multiple-database-dumps-to-cover-different-scenarios)
|
83
|
+
* [These SQL dumps are way too large to commit to
|
84
|
+
git!](#are-you-sure-i-should-commit-these-sql-dumps-theyre-way-too-big)
|
85
|
+
* [Tests shouldn't rely on shared test data if they don't need
|
86
|
+
to](#tests-shouldnt-use-shared-test-data-they-should-instantiate-the-objects-they-need)
|
87
|
+
* [My tests aren't as fast as they should
|
88
|
+
be](#im-worried-my-tests-arent-as-fast-as-they-should-be)
|
81
89
|
7. [Code of Conduct](#code-of-conduct)
|
82
90
|
8. [Changelog](/CHANGELOG.md)
|
83
91
|
9. [MIT License](/LICENSE.txt)
|
@@ -146,20 +154,11 @@ your server (or any command) to create some test data like so:
|
|
146
154
|
````
|
147
155
|
|
148
156
|
The purpose of the `test_data` database is to provide a sandbox in which you
|
149
|
-
will manually generate test data by playing
|
150
|
-
|
151
|
-
|
152
|
-
|
153
|
-
|
154
|
-
models and their associations increase in complexity), your test data will
|
155
|
-
always be realistic because your real application will have created it!
|
156
|
-
|
157
|
-
The database dumps are meant to be committed in git and versioned alongside your
|
158
|
-
tests over the life of the application. Its schema & data are should be
|
159
|
-
incrementally migrated over time, just like your production database. (As a
|
160
|
-
happy side effect of running your migrations against your test data, this means
|
161
|
-
your `test_data` database may help you identify hard-to-catch migration bugs
|
162
|
-
early, before being deployed to production!)
|
157
|
+
will manually generate test data by playing around with your app. Rather than
|
158
|
+
try to imitate realistic data using factories and fixtures (a task which only
|
159
|
+
grows more difficult as your models and their associations increase in
|
160
|
+
complexity), your test data will always be realistic because your real
|
161
|
+
application will have created it!
|
163
162
|
|
164
163
|
### Step 2: Create some test data
|
165
164
|
|
@@ -186,31 +185,47 @@ number of Rails commands or Rake tasks against its database by setting
|
|
186
185
|
|
187
186
|
_[Aside: If you experience any hiccups in getting your server to work, please
|
188
187
|
[open an issue](https://github.com/testdouble/test_data/issues/new) and let us
|
189
|
-
know—it may present an opportunity to improve the `test_data:configure`
|
188
|
+
know—it may present an opportunity for us to improve the `test_data:configure`
|
189
|
+
task!]_
|
190
190
|
|
191
191
|
#### Create test data by using your app
|
192
192
|
|
193
193
|
Once the app is running, it's time to generate some test data. You'll know how
|
194
194
|
to accomplish this step better than anyone—it's your app, after all!
|
195
195
|
|
196
|
-
|
197
|
-
|
198
|
-
|
199
|
-
|
200
|
-
|
201
|
-
|
202
|
-
|
203
|
-
|
204
|
-
|
205
|
-
|
206
|
-
|
207
|
-
|
208
|
-
|
196
|
+
A few bits of advice click & type some test data into existence:
|
197
|
+
|
198
|
+
* Spend a little time thoughtfully navigating each feature of your app in order
|
199
|
+
to generate enough data to be representative of what would be needed to test
|
200
|
+
them (e.g. one `User` per role, one of each kind of `Order`, etc.)
|
201
|
+
* Less is more: the less test data you create, the more meaningful & memorable
|
202
|
+
it will be to yourself and your teammates when writing tests. Don't keep
|
203
|
+
adding test data unless it will allow you to exercise additional application
|
204
|
+
code (e.g. enough `Project` models to require pagination, but not hundreds of
|
205
|
+
them for the sake of looking "production-like")
|
206
|
+
* Memorable names can become memes for the team to quickly recall and reference
|
207
|
+
later (if the admin user is named "Angela" and the manager is "Maria", that'll
|
208
|
+
probably serve you better than generic names like "TestUser #1")
|
209
|
+
|
210
|
+
If you make a mistake when creating your initial set of test data, it's
|
211
|
+
perfectly okay to reset the database and start over! Your future tests will be
|
212
|
+
coupled to this data as your application grows and evolves, so it's worth taking
|
213
|
+
the time to ensure the foundation is solid. (But that's not to say everything
|
214
|
+
needs to be perfect; you can always change things or add more data later—you'll
|
215
|
+
just have to update your tests accordingly.)
|
209
216
|
|
210
217
|
### Step 3: Dump your `test_data` database
|
211
218
|
|
219
|
+
Once you've created a good sampling of test data by interacting with your app,
|
220
|
+
the next step is to flush it from the `test_data` database to SQL files. These
|
221
|
+
database dumps are meant to be committed to source control and versioned
|
222
|
+
alongside your tests over the life of the application. Additionally, they are
|
223
|
+
designed to be incrementally
|
224
|
+
[migrated](#step-5-keeping-your-test-data-up-to-date) over time, just like you
|
225
|
+
migrate production database with every release.
|
226
|
+
|
212
227
|
Once you have your test data how you want it, dump the schema and data to SQL
|
213
|
-
files:
|
228
|
+
files with the `test_data:dump` Rake task:
|
214
229
|
|
215
230
|
```
|
216
231
|
$ bin/rake test_data:dump
|
@@ -218,12 +233,15 @@ $ bin/rake test_data:dump
|
|
218
233
|
|
219
234
|
This will dump three files into `test/support/test_data`:
|
220
235
|
|
221
|
-
* Schema DDL
|
236
|
+
* `schema.sql` - Schema DDL used to (re-)initialize the `test_data` environment
|
237
|
+
database for anyone looking to update your test data
|
222
238
|
|
223
|
-
*
|
239
|
+
* `data.sql` - The test data itself, exported as a bunch of SQL `INSERT`
|
240
|
+
statements, which will be executed by your tests to load your test data
|
224
241
|
|
225
|
-
*
|
226
|
-
`
|
242
|
+
* `non_test_data.sql` - Data needed to run the `test_data` environment, but
|
243
|
+
which shouldn't be inserted by your tests (the `ar_internal_metadata` and
|
244
|
+
`schema_migrations` tables, by default; see `config.non_test_data_tables`)
|
227
245
|
|
228
246
|
You probably won't need to, but these paths can be overridden with
|
229
247
|
[TestData.config](#testdataconfig) method. Additional details can also be found
|
@@ -241,25 +259,34 @@ environment exists only for creating your test data. Your tests will, in turn,
|
|
241
259
|
load the SQL dump of your data into the `test` database, and things will proceed
|
242
260
|
just as if you'd been loading [Rails' built-in
|
243
261
|
fixtures](https://guides.rubyonrails.org/testing.html#the-low-down-on-fixtures)
|
244
|
-
from a set of YAML files
|
245
|
-
are generated through realistic use, whereas fixtures are defined manually in
|
246
|
-
(sometimes painstaking) YAML.
|
262
|
+
from a set of YAML files.
|
247
263
|
|
248
264
|
### Step 4: Load your data in your tests
|
249
265
|
|
250
266
|
Now that you've dumped the contents of your `test_data` database, you can start
|
251
267
|
writing tests that rely on this test data.
|
252
268
|
|
253
|
-
To accomplish this, you'll likely want to add hooks to run before
|
254
|
-
|
255
|
-
|
256
|
-
|
257
|
-
|
269
|
+
To accomplish this, you'll likely want to add hooks to run before each test to
|
270
|
+
put the database into whatever state the test needs.
|
271
|
+
|
272
|
+
For the simplest case—ensuring your test data is loaded into the `test` database
|
273
|
+
and available to your test, you'll want to call the
|
274
|
+
[TestData.uses_test_data](#testdatauses_test_data) method at the beginning of
|
275
|
+
the test. The first time `uses_test_data` is called, `test_data` will start a
|
276
|
+
transaction and insert your test data. On subsequent calls to `uses_test_data`
|
277
|
+
by later tests, the transaction will be rolled back to a save point taken just
|
278
|
+
after the data was initially loaded, so that each test gets a clean starting
|
279
|
+
point without repeatedly executing the expensive SQL operation.
|
280
|
+
|
281
|
+
#### If you want every single test to have access to your test data
|
258
282
|
|
259
|
-
If you
|
260
|
-
|
261
|
-
|
262
|
-
|
283
|
+
If, for the sake of consistency & simplicity you want every single Rails-aware
|
284
|
+
test to have access to your test data, you
|
285
|
+
can accomplish this with a single global before-each hook.
|
286
|
+
|
287
|
+
If you're using Rails' default
|
288
|
+
[Minitest](https://github.com/seattlerb/minitest), you can load it in a `setup`
|
289
|
+
hook in `ActiveSupport::TestCase`:
|
263
290
|
|
264
291
|
```ruby
|
265
292
|
class ActiveSupport::TestCase
|
@@ -269,8 +296,8 @@ class ActiveSupport::TestCase
|
|
269
296
|
end
|
270
297
|
```
|
271
298
|
|
272
|
-
|
273
|
-
global `before(:each)` hook in your `rails_helper.rb` file:
|
299
|
+
Likewise, if you use [RSpec](https://rspec.info), you can accomplish the same
|
300
|
+
thing with global `before(:each)` hook in your `rails_helper.rb` file:
|
274
301
|
|
275
302
|
```ruby
|
276
303
|
RSpec.configure do |config|
|
@@ -280,82 +307,139 @@ RSpec.configure do |config|
|
|
280
307
|
end
|
281
308
|
```
|
282
309
|
|
283
|
-
|
284
|
-
|
285
|
-
|
286
|
-
|
287
|
-
|
288
|
-
|
289
|
-
guide](#factory--fixture-interoperability-guide) for help.
|
310
|
+
#### If some tests rely on test data and others need a clean slate
|
311
|
+
|
312
|
+
Of course, for simple units of code, it may be more prudent to manually create
|
313
|
+
the test data they need inline as opposed to relying on a shared source of test
|
314
|
+
data. For these tests, you can call
|
315
|
+
[TestData.uses_clean_slate](#testdatauses_clean_slate) in a `setup` hook.
|
290
316
|
|
291
|
-
|
292
|
-
|
293
|
-
[TestData.uses_clean_slate](#testdatauses_clean_slate) to clear data generated
|
294
|
-
by this gem out before they run. One way to do that would be to define two test
|
295
|
-
types:
|
317
|
+
For the best performance, you might consider a mode-switching method that's
|
318
|
+
invoked at the top of each test listing like this:
|
296
319
|
|
297
320
|
```ruby
|
298
|
-
|
299
|
-
|
300
|
-
|
301
|
-
|
321
|
+
class ActiveSupport::TestCase
|
322
|
+
def self.uses(mode)
|
323
|
+
case mode
|
324
|
+
when :clean_slate
|
325
|
+
setup { TestData.uses_clean_slate }
|
326
|
+
when :test_data
|
327
|
+
setup { TestData.uses_test_data }
|
328
|
+
else
|
329
|
+
raise "Invalid test data mode: #{mode}"
|
330
|
+
end
|
302
331
|
end
|
303
332
|
end
|
304
333
|
|
305
|
-
#
|
306
|
-
class
|
307
|
-
|
308
|
-
|
334
|
+
# A simple model that will `create` its own data
|
335
|
+
class WidgetTest < ActiveSupport::TestCase
|
336
|
+
uses :clean_slate
|
337
|
+
# …
|
338
|
+
end
|
339
|
+
|
340
|
+
# An integrated test that depends on a lot of data
|
341
|
+
class KitchenSinkTest < ActionDispatch::IntegrationTest
|
342
|
+
uses :test_data
|
343
|
+
# …
|
344
|
+
end
|
345
|
+
```
|
346
|
+
|
347
|
+
Or, with RSpec:
|
348
|
+
|
349
|
+
```ruby
|
350
|
+
module TestDataModes
|
351
|
+
def uses(mode)
|
352
|
+
case mode
|
353
|
+
when :clean_slate
|
354
|
+
before(:each) { TestData.uses_clean_slate }
|
355
|
+
when :test_data
|
356
|
+
before(:each) { TestData.uses_test_data }
|
357
|
+
else
|
358
|
+
raise "Invalid test data mode: #{mode}"
|
359
|
+
end
|
309
360
|
end
|
310
361
|
end
|
362
|
+
|
363
|
+
RSpec.configure do |config|
|
364
|
+
config.extend(TestDataModes)
|
365
|
+
end
|
366
|
+
|
367
|
+
RSpec.describe Widget, type: :model do
|
368
|
+
uses :clean_slate
|
369
|
+
# …
|
370
|
+
end
|
371
|
+
|
372
|
+
RSpec.describe "Kitchen sink", type: :request do
|
373
|
+
uses :test_data
|
374
|
+
# …
|
375
|
+
end
|
311
376
|
```
|
312
377
|
|
378
|
+
#### If your situation is more complicated
|
379
|
+
|
380
|
+
If you're adding `test_data` to an existing application, it's likely that you
|
381
|
+
won't be able to easily adopt a one-size-fits-all approach to test setup across
|
382
|
+
your entire suite. Some points of reference, if that's the situation you're in:
|
383
|
+
|
384
|
+
* If your test suite is already using fixtures or factories and the above hooks
|
385
|
+
just broke everything, check out our [interoperability
|
386
|
+
guide](#factory--fixture-interoperability-guide) for help.
|
387
|
+
* If you don't want `test_data` managing transactions and cleanup for you and
|
388
|
+
just want to load the SQL dump, you can call
|
389
|
+
[TestData.insert_test_data_dump](#testdatainsert_test_data_dump)
|
390
|
+
* For more information on how all this works, see the [API
|
391
|
+
reference](#api-reference).
|
392
|
+
|
313
393
|
### Step 5: Keeping your test data up-to-date
|
314
394
|
|
315
395
|
Your app relies on its tests and your tests rely on their test data. This
|
316
396
|
creates a bit of a paradox: creating & maintaining test data is _literally_ a
|
317
397
|
tertiary concern but simultaneously an inescapable responsibility that will live
|
318
398
|
with you for the life of your application. That's true whether you use this gem,
|
319
|
-
`factory_bot`, Rails fixtures, or
|
320
|
-
|
399
|
+
`factory_bot`, Rails fixtures, or something else as a source of shared test
|
400
|
+
data.
|
321
401
|
|
322
|
-
|
323
|
-
|
324
|
-
your schema evolves is a going concern of this gem.
|
325
|
-
|
326
|
-
Fortunately, because your `test_data` database needs to be maintained for the
|
327
|
-
entire life of your application and because production databases need the same
|
328
|
-
thing, we already have a fantastic tool for the job: [Rails
|
402
|
+
Fortunately, we already have a fantastic tool available for keeping our
|
403
|
+
`test_data` database up-to-date over the life of our application: [Rails
|
329
404
|
migrations](https://guides.rubyonrails.org/active_record_migrations.html). If
|
330
|
-
your migrations are resilient enough for your production
|
331
|
-
be able to keep your `test_data` database up-to-date.
|
405
|
+
your migrations are resilient enough for your production database, they should
|
406
|
+
also be able to keep your `test_data` database up-to-date. (As a happy side
|
407
|
+
effect of running your migrations against your test data, this means your
|
408
|
+
`test_data` database may help you identify hard-to-catch migration bugs early,
|
409
|
+
before being deployed to production!)
|
410
|
+
|
411
|
+
Whenever you create a new migration or add a major feature, you'll probably need
|
412
|
+
to update your test data. Here's how to do it:
|
332
413
|
|
333
|
-
|
334
|
-
|
335
|
-
data. Here's a rough outline to updating your `test_data` database:
|
414
|
+
* If the current SQL dumps in `test/support/test_data` are newer than your local
|
415
|
+
`test_data` database:
|
336
416
|
|
337
|
-
1.
|
338
|
-
|
339
|
-
database is out of date), drop it with `rake test_data:drop_database`
|
417
|
+
1. Be sure there's nothing in your local `test_data` database that you added
|
418
|
+
intentionally and forgot to dump, because it's about to be erased
|
340
419
|
|
341
|
-
2.
|
342
|
-
test_data:load`
|
420
|
+
2. Run `rake test_data:drop_database`
|
343
421
|
|
344
|
-
3. Run
|
422
|
+
3. Run `rake test_data:load` to recreate the `test_data` database and load
|
423
|
+
the latest SQL dumps into it
|
345
424
|
|
346
|
-
4.
|
347
|
-
(`RAILS_ENV=test_data bin/rails s`), just like in [Step
|
348
|
-
2](#step-2-create-some-test-data)
|
425
|
+
4. Run any pending migrations with `RAILS_ENV=test_data bin/rake db:migrate`
|
349
426
|
|
350
|
-
5.
|
427
|
+
5. If you need to create any additional data, start up the server
|
428
|
+
(`RAILS_ENV=test_data bin/rails s`), just like in [Step
|
429
|
+
2](#step-2-create-some-test-data)
|
351
430
|
|
352
|
-
6.
|
431
|
+
6. Export your newly-updated `test_data` database with `rake test_data:dump`
|
432
|
+
|
433
|
+
7. Ensure your tests are passing and then commit the resulting SQL files
|
434
|
+
|
435
|
+
* If the local `test_data` database is already up-to-date with the current SQL
|
436
|
+
dumps, follow steps **4 through 7** above
|
353
437
|
|
354
438
|
It's important to keep in mind that your test data SQL dumps are a shared,
|
355
439
|
generated resource among your team (just like a `structure.sql` or `schema.rb`
|
356
440
|
file). As a result, if your team doesn't integrate code frequently or if the
|
357
|
-
test data
|
358
|
-
|
441
|
+
test data changes frequently, you'd be right to be concerned that [the resulting
|
442
|
+
merge conflicts could become
|
359
443
|
significant](#how-will-i-handle-merge-conflicts-in-these-sql-files-if-i-have-lots-of-people-working-on-lots-of-feature-branches-all-adding-to-the-test_data-database-dumps),
|
360
444
|
so sweeping changes should be made deliberately and in collaboration with other
|
361
445
|
contributors.
|
@@ -364,7 +448,10 @@ _[Aside: some Rails teams are averse to using migrations to migrate data as well
|
|
364
448
|
as schemas, instead preferring one-off scripts and tasks. You'll have an easier
|
365
449
|
time of things if you use migrations for both schema and data changes. Here are
|
366
450
|
some notes on [how to write data migrations
|
367
|
-
safely](https://blog.testdouble.com/posts/2014-11-04-healthy-migration-habits/#habit-4-dont-reference-models).
|
451
|
+
safely](https://blog.testdouble.com/posts/2014-11-04-healthy-migration-habits/#habit-4-dont-reference-models).
|
452
|
+
Otherwise, you'll need to remember to run any ad hoc deployment scripts against
|
453
|
+
your `test_data` Rails environment along with each of your other deployed
|
454
|
+
environments.]_
|
368
455
|
|
369
456
|
## Factory & Fixture Interoperability Guide
|
370
457
|
|
@@ -383,8 +470,8 @@ all three, or none-of-the-above.
|
|
383
470
|
|
384
471
|
This section will hopefully make it a little easier to incorporate new
|
385
472
|
`test_data` tests into a codebase that's already using `factory_bot` and/or
|
386
|
-
Rails fixtures, whether you choose to incrementally
|
387
|
-
|
473
|
+
Rails fixtures, whether you choose to incrementally migrate to using `test_data`
|
474
|
+
over time.
|
388
475
|
|
389
476
|
### Using `test_data` with `factory_bot`
|
390
477
|
|
@@ -395,185 +482,43 @@ This section will document some thoughts and strategies for introducing
|
|
395
482
|
|
396
483
|
Depending on the assumptions your tests make about the state of the database
|
397
484
|
before you've loaded any factories, it's possible that everything will "just
|
398
|
-
work" after adding
|
399
|
-
|
400
|
-
|
401
|
-
just passes.
|
485
|
+
work" after adding [TestData.uses_test_data](#testdatauses_test_data) in a
|
486
|
+
before-each hook (as shown in the [setup
|
487
|
+
guide](#step-4-load-your-data-in-your-tests)). So by all means, try running your
|
488
|
+
suite after following the initial setup guide and see if the suite just passes.
|
402
489
|
|
403
490
|
If you find that your test suite is failing after adding
|
404
|
-
`TestData.uses_test_data` to your setup, don't panic!
|
405
|
-
|
406
|
-
|
407
|
-
|
408
|
-
|
409
|
-
factories or editing your `test_data` database to
|
410
|
-
another. Care should be taken to preserve the
|
411
|
-
test, however, as naively squashing errors can
|
412
|
-
factories and your `test_data`
|
413
|
-
|
414
|
-
|
415
|
-
|
416
|
-
|
417
|
-
|
418
|
-
|
419
|
-
|
420
|
-
[TestData.truncate](#testdatatruncate) in a before-each hook and
|
421
|
-
[TestData.rollback(:after_data_truncate)](#rolling-back-to-after-test-data-was-truncated)
|
422
|
-
in an after-each hook, like this:
|
491
|
+
`TestData.uses_test_data` to your setup, don't panic! Test failures are most
|
492
|
+
likely caused by the combination of your `test_data` database with the data
|
493
|
+
persisted by your factories.
|
494
|
+
|
495
|
+
One approach would be to attempt to resolve each such failure one-by-one—usually
|
496
|
+
by updating the offending factories or editing your `test_data` database to
|
497
|
+
ensure they steer clear of one another. Care should be taken to preserve the
|
498
|
+
conceptual encapsulation of each test, however, as naively squashing errors can
|
499
|
+
introduce inadvertent coupling between your factories and your `test_data`
|
500
|
+
database such that neither can be used independently of the other.
|
501
|
+
|
502
|
+
Another approach that the `test_data` gem provides is an additional mode with
|
503
|
+
`TestData.uses_clean_slate`, which—when called at the top of a factory-dependent
|
504
|
+
test—will ensure that the tables that `test_data` had written to will be
|
505
|
+
truncated, allowing the test to create whatever factories it needs without fear
|
506
|
+
of conflicts.
|
423
507
|
|
424
508
|
```ruby
|
425
509
|
class AnExistingFactoryUsingTest < ActiveSupport::Testcase
|
426
|
-
|
427
|
-
TestData.
|
510
|
+
setup do
|
511
|
+
TestData.uses_clean_slate
|
428
512
|
# pre-existing setup
|
429
513
|
end
|
430
|
-
|
431
|
-
def test_stuff
|
432
|
-
#… etc
|
433
|
-
end
|
434
|
-
|
435
|
-
def teardown
|
436
|
-
TestData.rollback(:after_truncate)
|
437
|
-
end
|
438
|
-
end
|
439
|
-
```
|
440
|
-
|
441
|
-
What this will do is complicated and counter-intuitive, but also fast and
|
442
|
-
reliable: [TestData.truncate](#testdatatruncate) will first ensure that your
|
443
|
-
`test_data` database is loaded inside a transaction, then will truncate that
|
444
|
-
data (set the `truncate_these_test_data_tables` [config option](#testdataconfig)
|
445
|
-
if necessary), and will finally create _yet another_ transaction save point
|
446
|
-
named `:after_data_truncate`. From that point onward, your test is free to
|
447
|
-
create all the factories it needs without fear of colliding with whatever you've
|
448
|
-
got stored in your `test_data` tables.
|
449
|
-
|
450
|
-
_[Why does this approach potentially load all the `test_data` data only to
|
451
|
-
immediately truncate it? Because it's actually much faster to truncate a large
|
452
|
-
data load in a live transaction, rollback the truncation, and then re-truncate
|
453
|
-
the data for a subsequent test than it would be to rollback the large data load
|
454
|
-
itself and re-load it for a subsequent test. It's silly but it works.]_
|
455
|
-
|
456
|
-
Hopefully one of these approaches, or some combination of them will get your
|
457
|
-
test suite passing after you've introduced `test_data`.
|
458
|
-
|
459
|
-
#### Separating your `test_data` and factory tests
|
460
|
-
|
461
|
-
Just because your tests _can_ access both your `factory_bot` factories and
|
462
|
-
`test_data` database doesn't mean they _should_.
|
463
|
-
|
464
|
-
Integration tests inevitably become coupled to the data that's available to
|
465
|
-
them, and if a test has access to both records created by a factory and a
|
466
|
-
`test_data` SQL dump, it is likely to unintentionally become inextricable from
|
467
|
-
both. This could result in the test having more ways to fail than necessary and
|
468
|
-
make it harder to simplify your test data strategy later. Instead, consider
|
469
|
-
explicitly opting into a single type of test data by separating your tests based
|
470
|
-
on which source of test data they use.
|
471
|
-
|
472
|
-
Every situation will be different, but one strategy that suits a lot of
|
473
|
-
circumstances would be to write a class method that runs at test-load time to
|
474
|
-
declare and configure the test data strategy for the current test.
|
475
|
-
|
476
|
-
Taking from [this
|
477
|
-
example](/example/test/integration/better_mode_switching_demo_test.rb) test, you
|
478
|
-
could implement a class method like this:
|
479
|
-
|
480
|
-
```ruby
|
481
|
-
class ActiveSupport::TestCase
|
482
|
-
def self.test_data_mode(mode)
|
483
|
-
case mode
|
484
|
-
when :factory_bot
|
485
|
-
require "factory_bot_rails"
|
486
|
-
include FactoryBot::Syntax::Methods
|
487
|
-
|
488
|
-
setup do
|
489
|
-
TestData.truncate
|
490
|
-
end
|
491
|
-
|
492
|
-
teardown do
|
493
|
-
TestData.rollback(:after_data_truncate)
|
494
|
-
end
|
495
|
-
when :test_data
|
496
|
-
setup do
|
497
|
-
TestData.load
|
498
|
-
end
|
499
|
-
|
500
|
-
teardown do
|
501
|
-
TestData.rollback
|
502
|
-
end
|
503
|
-
end
|
504
|
-
end
|
505
|
-
end
|
506
|
-
```
|
507
|
-
|
508
|
-
And then (without any class inheritance complications), simply declare which
|
509
|
-
kind of test you're specifying:
|
510
|
-
|
511
|
-
```ruby
|
512
|
-
class SomeFactoryUsingTest < ActiveSupport::TestCase
|
513
|
-
test_data_mode :factory_bot
|
514
|
-
|
515
|
-
# … tests go here
|
516
|
-
end
|
517
|
-
|
518
|
-
class SomeTestDataUsingTest < ActionDispatch::IntegrationTest
|
519
|
-
test_data_mode :test_data
|
520
|
-
|
521
|
-
# etc.
|
514
|
+
# …
|
522
515
|
end
|
523
516
|
```
|
524
517
|
|
525
|
-
|
526
|
-
|
527
|
-
|
528
|
-
|
529
|
-
tangling your tests' dependency graph further.
|
530
|
-
|
531
|
-
#### Speeding up your test suite when using factories
|
532
|
-
|
533
|
-
##### Addressing redundant data cleanup
|
534
|
-
|
535
|
-
After adding `test_data` to your test suite, consider is how database cleanup
|
536
|
-
was being handled previously to make sure it isn't unnecessarily truncating
|
537
|
-
everything or resetting the transaction between tests. It's possible that your
|
538
|
-
suite is relying on Rails' built-in `use_transactional_tests` feature to wrap
|
539
|
-
your tests in always-rolled-back transactions, even if you're not using
|
540
|
-
fixtures. Or perhaps your suite uses
|
541
|
-
[database_cleaner](https://github.com/DatabaseCleaner/database_cleaner) to
|
542
|
-
truncate the database before or after each test. In either case, it's important
|
543
|
-
to know that by default [TestData.load](#testdataload) and
|
544
|
-
[TestData.rollback](#testdatarollback) will start and rollback a nested
|
545
|
-
transaction, respectively. That means—so long as they're called at the top of a
|
546
|
-
before-each hook and the end of an after-each hook—you might be able to disable
|
547
|
-
`use_transactional_tests` or remove your dependency on `database_cleaner` or any
|
548
|
-
other custom truncation logic you might have. Even if you get your suite running
|
549
|
-
immediately after adding `test_data`, it's still worth taking the time to
|
550
|
-
understand what's going on during test setup & teardown, because there may be an
|
551
|
-
opportunity to make your tests faster and more comprehensible by eliminating
|
552
|
-
redundant clean-up steps.
|
553
|
-
|
554
|
-
##### Avoiding truncate rollback churn
|
555
|
-
|
556
|
-
It's important to know that if your test suite has a mix of tests that call
|
557
|
-
[TestData.load](#testdataload) and tests that call
|
558
|
-
[TestData.truncate](#testdatatruncate), each time the test runner switches
|
559
|
-
between the two types, each call to `TestData.load` will cause the transaction
|
560
|
-
state to be rolled back from
|
561
|
-
[:after_data_truncate](#rolling-back-to-after-test-data-was-truncated) to
|
562
|
-
[:after_data_load](#rolling-back-to-after-the-data-was-loaded), only for the
|
563
|
-
next test to call `TestData.truncate` truncates all the tables again. In
|
564
|
-
practice, this shouldn't be too costly an operation, but if your test order is
|
565
|
-
randomized you might find that your build will run faster if you separate each
|
566
|
-
set of tests at runtime.
|
567
|
-
|
568
|
-
Separating your `test_data` and `factory_bot` tests is pretty trivial if you're
|
569
|
-
using RSpec, as the
|
570
|
-
[tag](https://relishapp.com/rspec/rspec-core/v/3-10/docs/command-line/tag-option)
|
571
|
-
feature was built with this sort of need in mind. Otherwise, you might consider
|
572
|
-
organizing the tests in different directories and running multiple commands to
|
573
|
-
execute them (e.g. `bin/rails test test/test_data_tests` and `bin/rails
|
574
|
-
test/factory_tests`). Every CI configuration is different, however, and you may
|
575
|
-
find yourself needing to get creative in configuring things to achieve the
|
576
|
-
fastest build time.
|
518
|
+
If you have a lot of tests, you can find a more sophisticated approaches for
|
519
|
+
logically switching between types of test data declaratively above in the
|
520
|
+
[getting started
|
521
|
+
section](#if-some-tests-rely-on-test-data-and-others-need-a-clean-slate)
|
577
522
|
|
578
523
|
### Using `test_data` with Rails fixtures
|
579
524
|
|
@@ -584,146 +529,53 @@ permanently committed to the test database actually makes them a little trickier
|
|
584
529
|
to work with. This section will cover a couple approaches for integrating
|
585
530
|
`test_data` into suites that use fixtures.
|
586
531
|
|
587
|
-
#### Getting your fixtures-dependent tests passing with `test_data`
|
588
|
-
|
589
532
|
It's more likely than not that all your tests will explode in dramatic fashion
|
590
|
-
as soon as you add `TestData.
|
591
|
-
fixtures will be loaded
|
592
|
-
|
593
|
-
|
594
|
-
|
595
|
-
|
596
|
-
|
597
|
-
|
598
|
-
|
599
|
-
|
600
|
-
|
601
|
-
Rails' built-in fixtures behavior with a monkey patch**. If that bold text
|
602
|
-
warning wasn't enough to scare you from reading on, here's how to do it.
|
603
|
-
|
604
|
-
_[Note that the following requires `use_transactional_data_loader` to be enabled
|
605
|
-
in your [config](#testdataconfig), because it depends on transaction
|
606
|
-
rollbacks.]_
|
607
|
-
|
608
|
-
Here's what you can do if you can't get your fixtures to play nicely with your
|
609
|
-
`test_data` dump:
|
610
|
-
|
611
|
-
1. Near the top of your test helper, call:
|
533
|
+
as soon as you add `TestData.uses_test_data` to a `setup` or `before(:each)`
|
534
|
+
hook. Typically, your fixtures will be loaded and committed immediately with
|
535
|
+
your `test_data` dump inserted afterward, which makes it exceedingly likely that
|
536
|
+
your tests will fail with primary key and unique constraint conflicts. If that's
|
537
|
+
the case you find yourself in, `test_data` provides an API that **overrides
|
538
|
+
Rails' built-in fixtures behavior with a monkey patch**.
|
539
|
+
|
540
|
+
And if that bold text wasn't enough to scare you off, here's how to do
|
541
|
+
it:
|
542
|
+
|
543
|
+
1. Before your tests have loaded (e.g. near the top of your test helper), call:
|
612
544
|
[TestData.prevent_rails_fixtures_from_loading_automatically!](#testdataprevent_rails_fixtures_from_loading_automatically)
|
613
|
-
This will
|
545
|
+
This will patch Rails'
|
614
546
|
[setup_fixtures](https://github.com/rails/rails/blob/main/activerecord/lib/active_record/test_fixtures.rb#L105)
|
615
|
-
into a no-op, which means that your test fixtures
|
616
|
-
loaded into your test database
|
547
|
+
and effectively render it into a no-op, which means that your test fixtures
|
548
|
+
will not be automatically loaded into your test database
|
617
549
|
|
618
|
-
2. In tests that rely on your `test_data` dump, call
|
619
|
-
TestData.
|
620
|
-
would. Because your fixtures won't be loaded automatically, they
|
621
|
-
available to these tests
|
550
|
+
2. In tests that rely on your `test_data` dump, call
|
551
|
+
[TestData.uses_test_data](#step-4-load-your-data-in-your-tests) as you
|
552
|
+
normally would. Because your fixtures won't be loaded automatically, they
|
553
|
+
won't be available to these tests
|
622
554
|
|
623
555
|
3. In tests that need fixtures, call
|
624
|
-
[TestData.
|
625
|
-
|
626
|
-
|
627
|
-
|
628
|
-
transaction-nesting) ensure your `test_data` dump is loaded in an initial
|
629
|
-
transaction, then ensure that it is truncated in a second transaction, before
|
630
|
-
loading your rails fixtures in a third transaction. These tests will have
|
631
|
-
access to all your fixture data without being tainted by any of your
|
632
|
-
`test_data` data
|
556
|
+
[TestData.uses_rails_fixtures(self)](#testdatauses_rails_fixtures) in a
|
557
|
+
before-each hook. This will first ensure that any tables written to by
|
558
|
+
`test_data` are truncated (as with `TestData.uses_clean_slate`) before
|
559
|
+
loading your Rails fixtures
|
633
560
|
|
634
561
|
For example, you might add the following to an existing fixtures-dependent
|
635
562
|
test to get it passing:
|
636
563
|
|
637
564
|
```ruby
|
638
565
|
class AnExistingFixtureUsingTest < ActiveSupport::Testcase
|
639
|
-
|
640
|
-
TestData.
|
566
|
+
setup do
|
567
|
+
TestData.uses_rails_fixtures(self)
|
641
568
|
# pre-existing setup
|
642
569
|
end
|
643
570
|
|
644
|
-
|
645
|
-
#… etc
|
646
|
-
end
|
647
|
-
|
648
|
-
def teardown
|
649
|
-
TestData.rollback(:after_load_rails_fixtures)
|
650
|
-
end
|
571
|
+
# …
|
651
572
|
end
|
652
573
|
```
|
653
574
|
|
654
|
-
|
655
|
-
|
656
|
-
|
657
|
-
|
658
|
-
#### Separating your `test_data` and fixture tests
|
659
|
-
|
660
|
-
*This only applies if you had to use
|
661
|
-
[TestData.load_rails_fixtures(self)](#testdataload_rails_fixtures) as shown
|
662
|
-
above.*
|
663
|
-
|
664
|
-
Just [like with factories](#separating-your-test_data-and-factory-tests), you
|
665
|
-
might benefit from a test helper to clearly declare whether a test uses fixtures
|
666
|
-
or `test_data` right at the top. Following the same pattern, you might do this:
|
667
|
-
|
668
|
-
```ruby
|
669
|
-
class ActiveSupport::TestCase
|
670
|
-
def self.test_data_mode(mode)
|
671
|
-
case mode
|
672
|
-
when :fixtures
|
673
|
-
fixtures :all
|
674
|
-
|
675
|
-
setup do
|
676
|
-
TestData.load_rails_fixtures(self)
|
677
|
-
end
|
678
|
-
|
679
|
-
teardown do
|
680
|
-
TestData.rollback(:after_load_rails_fixtures)
|
681
|
-
end
|
682
|
-
when :test_data
|
683
|
-
setup do
|
684
|
-
TestData.load
|
685
|
-
end
|
686
|
-
|
687
|
-
teardown do
|
688
|
-
TestData.rollback
|
689
|
-
end
|
690
|
-
end
|
691
|
-
end
|
692
|
-
end
|
693
|
-
```
|
694
|
-
|
695
|
-
Which would allow you to simplify the above fixtures-using test to:
|
696
|
-
|
697
|
-
```ruby
|
698
|
-
class AnExistingFixtureUsingTest < ActiveSupport::Testcase
|
699
|
-
test_data_mode :fixtures
|
700
|
-
|
701
|
-
def test_stuff
|
702
|
-
#… etc
|
703
|
-
end
|
704
|
-
end
|
705
|
-
```
|
706
|
-
|
707
|
-
#### Improving test suite speed with fixtures
|
708
|
-
|
709
|
-
Again, as is [the case with
|
710
|
-
factories](#improving-test-suite-speed-with-factories), every time your test
|
711
|
-
runner randomly picks a `test_data` test after running a fixtures-dependent
|
712
|
-
test, it will roll back your fixtures and the truncation of your `test_data`,
|
713
|
-
only to re-truncate your `test_data` data and reload your fixtures for the next
|
714
|
-
test that happens to use fixtures. But unlike truncation alone, loading your
|
715
|
-
fixtures is a non-trivial operation that can chew up a some serious time as your
|
716
|
-
suite runs.
|
717
|
-
|
718
|
-
As a result, we strongly encourage breaking up your test suite to avoid this
|
719
|
-
churn, even if it means splitting your test run over multiple CLI commands. If
|
720
|
-
you're using the Rails test runner and Minitest, that likely means sequestering
|
721
|
-
one set of tests to one directory and the other to a different directory, as
|
722
|
-
there is no granular control over to how the runner randomizes suites. And for
|
723
|
-
RSpec,
|
724
|
-
[tagging](https://relishapp.com/rspec/rspec-core/v/3-10/docs/command-line/tag-option)
|
725
|
-
each spec and running separate commands for each tag could yield significant
|
726
|
-
performance improvements.
|
575
|
+
If you've adopted a mode-switching helper method [like the one described
|
576
|
+
above](#if-some-tests-rely-on-test-data-and-others-need-a-clean-slate), you
|
577
|
+
could of course add a third mode to cover any tests that depend on Rails
|
578
|
+
fixtures.
|
727
579
|
|
728
580
|
## Rake Task Reference
|
729
581
|
|
@@ -768,11 +620,9 @@ This task runs several generators:
|
|
768
620
|
have numerous secrets in this file's `development:` stanza, you may want to
|
769
621
|
alias and inherit it into `test_data:` like the `webpacker.yml` generator does
|
770
622
|
|
771
|
-
* `config/cable.yml` -
|
772
|
-
[ActionCable](https://guides.rubyonrails.org/action_cable_overview.html)
|
773
|
-
|
774
|
-
generator explicitly specifies `async` instead, since that's the default for
|
775
|
-
`development:`
|
623
|
+
* `config/cable.yml` - Simply defines a `test_data:` entry that tells
|
624
|
+
[ActionCable](https://guides.rubyonrails.org/action_cable_overview.html) to
|
625
|
+
use the `async` adapter, since that's also the default for `development`
|
776
626
|
|
777
627
|
### test_data:verify_config
|
778
628
|
|
@@ -789,8 +639,8 @@ your seed file. Specifically:
|
|
789
639
|
1. Creates the `test_data` environment's database, if it doesn't already exist
|
790
640
|
|
791
641
|
2. Ensures the database is non-empty to preserve data integrity (run
|
792
|
-
[test_data:drop_database](#test_datadrop_database) first if
|
793
|
-
|
642
|
+
[test_data:drop_database](#test_datadrop_database) first if you intend to
|
643
|
+
reinitialize it)
|
794
644
|
|
795
645
|
3. Checks to see if a dump of the database already exists (by default, stored in
|
796
646
|
`test/support/test_data/`)
|
@@ -804,8 +654,9 @@ your seed file. Specifically:
|
|
804
654
|
### test_data:dump
|
805
655
|
|
806
656
|
This task is designed to be run after you've created or updated your test data
|
807
|
-
and you're ready to run your tests against it. The
|
808
|
-
SQL dumps from your `test_data` environment's
|
657
|
+
in the `test_data` database and you're ready to run your tests against it. The
|
658
|
+
task creates several plain SQL dumps from your `test_data` environment's
|
659
|
+
database:
|
809
660
|
|
810
661
|
* A schema-only dump, by default in `test/support/test_data/schema.sql`
|
811
662
|
|
@@ -813,13 +664,13 @@ SQL dumps from your `test_data` environment's database:
|
|
813
664
|
`test/support/test_data/data.sql`
|
814
665
|
|
815
666
|
* A data-only dump of records that you *don't* want loaded in your tests in
|
816
|
-
`test/support/test_data/non_test_data.sql
|
667
|
+
`test/support/test_data/non_test_data.sql`. By default, this includes Rails'
|
817
668
|
internal tables: `ar_internal_metadata` and `schema_migrations`, configurable
|
818
|
-
with [TestData.config](#testdataconfig)'s `non_test_data_tables`
|
669
|
+
with [TestData.config](#testdataconfig)'s `non_test_data_tables`
|
819
670
|
|
820
671
|
Each of these files are designed to be committed and versioned with the rest of
|
821
672
|
your application. [TestData.config](#testdataconfig) includes several
|
822
|
-
options to control
|
673
|
+
options to control this task.
|
823
674
|
|
824
675
|
### test_data:load
|
825
676
|
|
@@ -836,7 +687,7 @@ This task will load your SQL dumps into your `test_data` database by:
|
|
836
687
|
4. Warning if there are pending migrations that haven't been run yet
|
837
688
|
|
838
689
|
If there are pending migrations, you'll probably want to run them and then
|
839
|
-
dump & commit your test data so that they're
|
690
|
+
dump & commit your test data so that they're up-to-date:
|
840
691
|
|
841
692
|
```
|
842
693
|
$ RAILS_ENV=test_data bin/rake db:migrate
|
@@ -849,278 +700,285 @@ This task will create the `test_data` environment's database if it does not
|
|
849
700
|
already exist. It also
|
850
701
|
[enhances](https://dev.to/molly/rake-task-enhance-method-explained-3bo0) Rails'
|
851
702
|
`db:create` task so that `test_data` is created along with `development` and
|
852
|
-
`test
|
703
|
+
`test` whenever `rake db:create` is run.
|
853
704
|
|
854
705
|
### test_data:drop_database
|
855
706
|
|
856
707
|
This task will drop the `test_data` environment's database if it exists. It also
|
857
708
|
enhances Rails' `db:drop` task so that `test_data` is dropped along with
|
858
|
-
`development` and `test
|
709
|
+
`development` and `test` whenever `rake db:drop` is run.
|
859
710
|
|
860
711
|
## API Reference
|
861
712
|
|
713
|
+
### TestData.uses_test_data
|
714
|
+
|
715
|
+
This is the method designed to be used by your tests to load your test data
|
716
|
+
into your `test` database so that your tests can rely on it. Typically, you'll
|
717
|
+
want to call it at the beginning of each test that relies on the test data
|
718
|
+
managed by this gem—most often, in a before-each hook.
|
719
|
+
|
720
|
+
For the sake of speed and integrity, `TestData.uses_test_data` is designed to
|
721
|
+
take advantage of nested transactions ([Postgres
|
722
|
+
savepoints](https://www.postgresql.org/docs/current/sql-savepoint.html)). By
|
723
|
+
default, data is loaded in a transaction and intended to be rolled back to the
|
724
|
+
point _immediately after_ the data was imported between tests. This way, your
|
725
|
+
test suite only pays the cost of importing the SQL file once, but each of your
|
726
|
+
tests can enjoy a clean slate that's free of data pollution from other tests.
|
727
|
+
(This is similar to, but separate from, Rails fixtures'
|
728
|
+
[use_transactional_tests](https://edgeguides.rubyonrails.org/testing.html#testing-parallel-transactions)
|
729
|
+
option.)
|
730
|
+
|
731
|
+
_See configuration option:
|
732
|
+
[config.after_test_data_load](#configafter_test_data_load)_
|
733
|
+
|
734
|
+
### TestData.uses_clean_slate
|
735
|
+
|
736
|
+
If a test does not rely on your `test_data` data, you can instead ensure that it
|
737
|
+
runs against empty tables by calling `TestData.uses_clean_slate`. Like
|
738
|
+
`TestData.uses_test_data`, this would normally be called at the beginning of
|
739
|
+
each such test in a before-each hook.
|
740
|
+
|
741
|
+
This method works by first ensuring that your test data is loaded (and the
|
742
|
+
correspondent savepoint created), then will truncate all affected tables and
|
743
|
+
create another savepoint. It's a little counter-intuitive that you'd first
|
744
|
+
litter your database with data only to wipe it clean again, but it's much faster
|
745
|
+
to repeatedly truncate tables than to repeatedly import large SQL files.
|
746
|
+
|
747
|
+
_See configuration options:
|
748
|
+
[config.after_test_data_truncate](#configafter_test_data_truncate),
|
749
|
+
[config.truncate_these_test_data_tables](#configtruncate_these_test_data_tables)_
|
750
|
+
|
751
|
+
### TestData.uses_rails_fixtures
|
752
|
+
|
753
|
+
As described in this README's [fixture interop
|
754
|
+
guide](#using-test_data-with-rails-fixtures), `TestData.uses_rails_fixtures`
|
755
|
+
will load your app's [Rails
|
756
|
+
fixtures](https://guides.rubyonrails.org/testing.html#the-low-down-on-fixtures)
|
757
|
+
by intercepting Rails' built-in fixture-loading code. As with the other "uses"
|
758
|
+
methods, you'll likely want to call it in a before-each hook before any test
|
759
|
+
that needs access to your Rails fixtures.
|
760
|
+
|
761
|
+
There are two additional things to keep in mind if using this method:
|
762
|
+
|
763
|
+
1. Using this feature requires that you've first invoked
|
764
|
+
[TestData.prevent_rails_fixtures_from_loading_automatically!](#testdataprevent_rails_fixtures_from_loading_automatically)
|
765
|
+
to override Rails' default behavior before any of your tests have loaded or
|
766
|
+
started running
|
767
|
+
|
768
|
+
2. Because the method depends on Rails' fixture caching mechanism, it must be
|
769
|
+
passed an instance of the running test class (e.g.
|
770
|
+
`TestData.uses_rails_fixtures(self)`)
|
771
|
+
|
772
|
+
Under the hood, this method effectively ensures a clean slate the same way
|
773
|
+
`TestData.uses_clean_slate` does, except that after creating the truncation
|
774
|
+
savepoint, it will then load your fixtures and finally create—wait for it—yet
|
775
|
+
another savepoint that subsequent calls to `uses_rails_fixtures` can rollback
|
776
|
+
to.
|
777
|
+
|
778
|
+
_See configuration option:
|
779
|
+
[config.after_rails_fixture_load](#configafter_rails_fixture_load)_
|
780
|
+
|
781
|
+
#### TestData.prevent_rails_fixtures_from_loading_automatically!
|
782
|
+
|
783
|
+
Call this method before any tests have been loaded or executed by your test
|
784
|
+
runner if you're planning to use
|
785
|
+
[TestData.uses_rails_fixtures](#testdatauses_rails_fixtures) to load Rails
|
786
|
+
fixtures into any of your tests. This method will disable the default behavior
|
787
|
+
of loading your Rails fixtures into the test database as soon as the first test
|
788
|
+
case with fixtures enabled is executed. (Inspect the [source for the
|
789
|
+
patch](/lib/test_data/active_record_ext.rb) to make sure you're comfortable with
|
790
|
+
what it's doing.)
|
791
|
+
|
862
792
|
### TestData.config
|
863
793
|
|
864
794
|
The generated `config/initializers/test_data.rb` initializer will include a call
|
865
795
|
to `TestData.config`, which takes a block that yields a mutable configuration
|
866
|
-
object (similar to `Rails.application.config`)
|
796
|
+
object (similar to `Rails.application.config`). If anything is unclear after
|
797
|
+
reading the documentation, feel free to review the
|
798
|
+
[initializer](lib/generators/test_data/initializer_generator.rb) and the [Config
|
799
|
+
class](/lib/test_data/config.rb) themselves.
|
800
|
+
|
801
|
+
#### Lifecycle hooks
|
802
|
+
|
803
|
+
Want to shift forward several timestamp fields after your `test_data` SQL dumps
|
804
|
+
are loaded into your test database? Need to refresh a materialized view after
|
805
|
+
your Rails fixtures are loaded? You _could_ do these things after calling
|
806
|
+
`TestData.uses_test_data` and `TestData.uses_rails_fixtures`, respectively, but
|
807
|
+
you'd take the corresponding performance hit in each and every test.
|
808
|
+
|
809
|
+
Instead, you can pass a callable or a block and `test_data` will execute it just
|
810
|
+
_after_ performing the associated data operation but just _before_ creating a
|
811
|
+
transaction savepoint. That way, whenever the gem rolls back between tests, your
|
812
|
+
hook won't need to be run again.
|
813
|
+
|
814
|
+
##### config.after_test_data_load
|
815
|
+
|
816
|
+
This is hook is run immediately after `TestData.uses_test_data` has loaded your
|
817
|
+
SQL dumps into the `test` database, but before creating a savepoint. Takes a
|
818
|
+
block or anything that responds to `call`.
|
819
|
+
|
867
820
|
|
868
821
|
```ruby
|
869
822
|
TestData.config do |config|
|
870
|
-
#
|
871
|
-
|
872
|
-
|
873
|
-
|
874
|
-
|
823
|
+
# Example: roll time forward
|
824
|
+
config.after_test_data_load do
|
825
|
+
Boop.connection.exec_update(<<~SQL, nil, [[nil, Time.zone.now - System.epoch]])
|
826
|
+
update boops set booped_at = booped_at + $1
|
827
|
+
SQL
|
828
|
+
end
|
829
|
+
end
|
830
|
+
```
|
875
831
|
|
876
|
-
|
877
|
-
# config.non_test_data_dump_path = "test/support/test_data/non_test_data.sql"
|
832
|
+
##### config.after_test_data_truncate
|
878
833
|
|
879
|
-
|
880
|
-
|
881
|
-
|
834
|
+
This is hook is run immediately after `TestData.uses_clean_slate` has truncated
|
835
|
+
your test data, but before creating a savepoint. Takes a block or anything that
|
836
|
+
responds to `call`.
|
882
837
|
|
883
|
-
|
884
|
-
|
838
|
+
```ruby
|
839
|
+
TestData.config do |config|
|
840
|
+
# Example: pass a callable instead of a block
|
841
|
+
config.after_test_data_truncate(SomethingThatRespondsToCall.new)
|
842
|
+
end
|
843
|
+
```
|
885
844
|
|
886
|
-
|
887
|
-
# If left as `nil`, all tables inserted into by the SQL file at
|
888
|
-
# `data_dump_path` will be truncated
|
889
|
-
# config.truncate_these_test_data_tables = nil
|
845
|
+
##### config.after_rails_fixture_load
|
890
846
|
|
891
|
-
|
892
|
-
|
893
|
-
|
894
|
-
# being used
|
895
|
-
# config.use_transactional_data_loader = true
|
847
|
+
This is hook is run immediately after `TestData.uses_rails_fixtures` has loaded
|
848
|
+
your Rails fixtures into the `test` database, but before creating a savepoint.
|
849
|
+
Takes a block or anything that responds to `call`.
|
896
850
|
|
897
|
-
|
898
|
-
|
899
|
-
#
|
851
|
+
```ruby
|
852
|
+
TestData.config do |config|
|
853
|
+
# Example: refresh Postgres assets like materialized views
|
854
|
+
config.after_rails_fixture_load do
|
855
|
+
RefreshesMaterializedViews.new.call
|
856
|
+
end
|
900
857
|
end
|
901
858
|
```
|
902
859
|
|
903
|
-
|
860
|
+
#### test_data:dump options
|
904
861
|
|
905
|
-
|
906
|
-
|
862
|
+
The gem provides several options governing the behavior of the
|
863
|
+
[test_data:dump](#test_datadump) Rake task. You probably won't need to set these
|
864
|
+
unless you run into a problem with the defaults.
|
907
865
|
|
908
|
-
|
866
|
+
##### config.non_test_data_tables
|
909
867
|
|
910
|
-
|
911
|
-
|
912
|
-
|
913
|
-
|
914
|
-
|
915
|
-
|
916
|
-
tests can enjoy a clean slate that's free of data pollution from other tests.
|
917
|
-
(This is similar to, but separate from, Rails fixtures'
|
918
|
-
[use_transactional_tests](https://edgeguides.rubyonrails.org/testing.html#testing-parallel-transactions)
|
919
|
-
option.)
|
868
|
+
Your application may have some tables that are necessary for the operation of
|
869
|
+
the application, but irrelevant or incompatible with you your tests. This data
|
870
|
+
is still dumped for the sake of being able to restore the database with [rake
|
871
|
+
test_data:load](#test_dataload), but will not be loaded when your tests are
|
872
|
+
running. Defaults to `[]`, (but will always include `ar_internal_metadata` and
|
873
|
+
`schema_migrations`).
|
920
874
|
|
921
|
-
|
922
|
-
|
923
|
-
|
924
|
-
|
925
|
-
|
926
|
-
|
927
|
-
|
928
|
-
|
929
|
-
|
930
|
-
|
931
|
-
|
932
|
-
|
933
|
-
of the test data.
|
934
|
-
|
935
|
-
As an additional safeguard, in case a rollback is triggered unexpectedly (i.e.
|
936
|
-
calling `rollback_transaction` on `ActiveRecord::Base.connection` instead of via
|
937
|
-
`TestData.rollback`), `test_data` writes a memo indicating that the data is
|
938
|
-
loaded in `ar_internal_metadata`. `TestData.load` uses this memo to detect this
|
939
|
-
issue and will recreate the `:after_data_load` savepoint rather than attempt to
|
940
|
-
erroneously reload your SQL data dump. (Similar error-handling is built-into
|
941
|
-
[TestData.truncate](#testdatatruncate) and
|
942
|
-
[TestData.load_rails_fixtures](#testdataload_rails_fixtures), as well.)
|
943
|
-
|
944
|
-
#### Loading without transactions
|
945
|
-
|
946
|
-
For most cases, we strongly recommend using the default transactional testing
|
947
|
-
strategy, both because it's faster and because it reduces the risk of test
|
948
|
-
pollution. However, you may need to commit your test data if the data needs to
|
949
|
-
be loaded by multiple processes or over multiple connections.
|
950
|
-
|
951
|
-
If you need to load the test data and commit it to the database, simply set
|
952
|
-
`TestData.config.use_transactional_data_loader = false`.
|
953
|
-
|
954
|
-
If transactions are disabled, you'll need to decide whether and how to clear the
|
955
|
-
data out after each test. Many folks use
|
956
|
-
[database_cleaner](https://github.com/DatabaseCleaner/database_cleaner) for
|
957
|
-
this, while `test_data` offers a rudimentary
|
958
|
-
[TestData.truncate](https://github.com/testdouble/test_data#testdatatruncate)
|
959
|
-
method that may be sufficient for your needs.
|
960
|
-
|
961
|
-
You might imagine something like this if you were loading the data just once for
|
962
|
-
the full run of a test suite:
|
875
|
+
```ruby
|
876
|
+
TestData.config do |config|
|
877
|
+
config.non_test_data_tables = []
|
878
|
+
end
|
879
|
+
```
|
880
|
+
|
881
|
+
##### config.dont_dump_these_tables
|
882
|
+
|
883
|
+
Some tables populated by your application may not be necessary to either its
|
884
|
+
proper functioning or useful to your tests (e.g. audit logs), so you can save
|
885
|
+
time and storage by preventing those tables from being dumped entirely. Defaults
|
886
|
+
to `[]`.
|
963
887
|
|
964
888
|
```ruby
|
965
|
-
|
966
|
-
config.
|
967
|
-
|
968
|
-
|
889
|
+
TestData.config do |config|
|
890
|
+
config.dont_dump_these_tables = []
|
891
|
+
end
|
892
|
+
```
|
969
893
|
|
970
|
-
|
971
|
-
|
972
|
-
|
894
|
+
##### config.schema_dump_path
|
895
|
+
|
896
|
+
The path to which the schema DDL of your `test_data` database will be written.
|
897
|
+
This is only used by [rake test_data:load](#test_dataload) when initializing the
|
898
|
+
`test_data` database. Defaults to `"test/support/test_data/schema.sql"`.
|
899
|
+
|
900
|
+
```ruby
|
901
|
+
TestData.config do |config|
|
902
|
+
config.schema_dump_path = "test/support/test_data/schema.sql"
|
973
903
|
end
|
974
904
|
```
|
975
905
|
|
976
|
-
|
977
|
-
|
978
|
-
|
979
|
-
|
980
|
-
|
981
|
-
### TestData.rollback
|
982
|
-
|
983
|
-
Because the gem loads your data in a transaction, it makes it easy to rollback
|
984
|
-
to any of its defined savepoints. In most cases you'll want to roll back to
|
985
|
-
`:after_data_load` after each test, and that's what `TestData.rollback` will do
|
986
|
-
when called without an argument. If the specified savepoint isn't active,
|
987
|
-
calling `rollback` is a no-op.
|
988
|
-
|
989
|
-
The gem may create up to four nested savepoints in a single transaction, and
|
990
|
-
this method allows you to rollback to any of them. They form the following
|
991
|
-
stack:
|
992
|
-
|
993
|
-
* `:before_data_load` - Taken before loading your `test_data` dump
|
994
|
-
* `:after_data_load` - Taken after loading your `test_data` dump
|
995
|
-
* `:after_truncate` - Taken after your `test_data` is truncated
|
996
|
-
* `:after_load_rails_fixtures` - Taken after Rails fixtures are loaded via
|
997
|
-
[TestData.load_rails_fixtures](#testdataload_rails_fixtures)
|
998
|
-
|
999
|
-
More details on rolling back to each of the gem's savepoints follows below.
|
1000
|
-
|
1001
|
-
#### Rolling back to before test data was loaded
|
1002
|
-
|
1003
|
-
If some tests rely on data loaded by `TestData.load` and you have other tests
|
1004
|
-
that depend on that data _not being there_, you probably want to call
|
1005
|
-
[TestData.truncate](#testdatatruncate). But if that won't work for your needs,
|
1006
|
-
you can rewind to the moment just before your test data was loaded by calling
|
1007
|
-
`TestData.rollback(:before_data_load)`.
|
1008
|
-
|
1009
|
-
**⚠️ Warning⚠️** Repeatedly loading and rolling back to `:before_data_load` is
|
1010
|
-
expensive! If your test suite calls `TestData.rollback(:before_data_load)`
|
1011
|
-
multiple times, it's likely you're re-loading your (possibly large) SQL file of
|
1012
|
-
test data many more times than is necessary. Consider using
|
1013
|
-
[TestData.truncate](#testdatatruncate) to achieve the same goal with faster
|
1014
|
-
performance. Failing that, it might be preferable to partition your test suite
|
1015
|
-
so that similar tests are run in separate groups (as opposed to in a fully
|
1016
|
-
random or arbitrary order) to avoid repeatedly thrashing between rollbacks and
|
1017
|
-
reloads. This partitioning could be accomplished by either configuring your test
|
1018
|
-
runner or by running separate test commands for each group of tests.
|
1019
|
-
|
1020
|
-
#### Rolling back to after the data was loaded
|
1021
|
-
|
1022
|
-
This is the way you're likely to call this method most often.
|
1023
|
-
|
1024
|
-
When `TestData.rollback` is passed no arguments or called more explicitly as
|
1025
|
-
`TestData.rollback(:after_data_load)`, the method will rollback to the
|
1026
|
-
`:after_data_load` transaction savepoint taken immediately after the SQL dump
|
1027
|
-
was loaded. As a result, it is intended to be run after each test (e.g. in an
|
1028
|
-
`after(:each)` or `teardown`), to undo any changes made by the test.
|
1029
|
-
|
1030
|
-
#### Rolling back to after test data was truncated
|
1031
|
-
|
1032
|
-
If some of your tests call [TestData.truncate](#testdatatruncate) to clear out
|
1033
|
-
your test data after it's been loaded (as
|
1034
|
-
[described](#getting-your-factory-tests-passing-after-adding-test_data) when
|
1035
|
-
using `test_data` in conjunction with `factory_bot`), then you will likely want
|
1036
|
-
to run `TestData.rollback(:after_data_truncate)` after each of them. This will
|
1037
|
-
rewind your test database's state to when those tables were first
|
1038
|
-
truncated—effectively re-cleaning the slate for the next test.
|
1039
|
-
|
1040
|
-
#### Rolling back to after Rails fixtures were loaded
|
1041
|
-
|
1042
|
-
If you're using
|
1043
|
-
[TestData.load_rails_fixtures(self)](#testdataload_rails_fixtures) in your
|
1044
|
-
test's before-each hook, you'll probably want to teardown that test by rolling
|
1045
|
-
back with `TestData.rollback(:after_load_rails_fixtures)` in an after-each hook,
|
1046
|
-
which will rewind to the point just after your Rails fixtures were loaded.
|
1047
|
-
|
1048
|
-
### TestData.truncate
|
1049
|
-
|
1050
|
-
Do you have some tests that _shouldn't_ access your test data? Or did some
|
1051
|
-
existing tests started failing after `test_data` was added? If you want to clear
|
1052
|
-
the state of your `test` database to support these tests, you can accomplish
|
1053
|
-
this with `TestData.truncate`. It truncates all the tables that `TestData.load`
|
1054
|
-
inserted into and then creates a savepoint named `:after_data_truncate`.
|
1055
|
-
|
1056
|
-
Most often, you'll want to call `TestData.truncate` before each test that
|
1057
|
-
should _not_ have access to your test data created with this gem. After each
|
1058
|
-
such test, it can clean up by calling `TestData.rollback(:after_data_truncate)`:
|
906
|
+
##### config.data_dump_path
|
907
|
+
|
908
|
+
The path that the SQL dump of your test data will be written. This is the dump
|
909
|
+
that will be executed by `TestData.uses_test_data` in your tests. Defaults to
|
910
|
+
`"test/support/test_data/data.sql"`.
|
1059
911
|
|
1060
912
|
```ruby
|
1061
|
-
|
1062
|
-
|
1063
|
-
|
1064
|
-
|
913
|
+
TestData.config do |config|
|
914
|
+
config.data_dump_path = "test/support/test_data/data.sql"
|
915
|
+
end
|
916
|
+
```
|
1065
917
|
|
1066
|
-
|
1067
|
-
|
1068
|
-
|
918
|
+
##### config.non_test_data_dump_path
|
919
|
+
|
920
|
+
The path to which the [non_test_data_tables](#confignon_test_data_tables) in
|
921
|
+
your `test_data` database will be written. This is only used by [rake
|
922
|
+
test_data:load](#test_dataload) when initializing the `test_data` database.
|
923
|
+
Defaults to `"test/support/test_data/non_test_data.sql"`.
|
924
|
+
|
925
|
+
```ruby
|
926
|
+
TestData.config do |config|
|
927
|
+
config.non_test_data_dump_path = "test/support/test_data/non_test_data.sql"
|
1069
928
|
end
|
1070
929
|
```
|
1071
930
|
|
1072
|
-
|
1073
|
-
test data SQL dump will be truncated (and cascading to any tables with foreign
|
1074
|
-
keys pointing to those tables), but you can also explicitly specify which tables
|
1075
|
-
should be truncated yourself by setting the `truncate_these_test_data_tables`
|
1076
|
-
property on [TestData.config](#testdataconfig) to an array of table names.
|
931
|
+
#### Other configuration options
|
1077
932
|
|
1078
|
-
|
933
|
+
##### config.truncate_these_test_data_tables
|
1079
934
|
|
1080
|
-
|
1081
|
-
|
1082
|
-
|
935
|
+
By default, when [TestData.uses_clean_slate](#testdatauses_clean_slate) is
|
936
|
+
called, it will truncate any tables for which an `INSERT` operation was
|
937
|
+
detected in your test data SQL dump. This may not be suitable for every case,
|
938
|
+
however, so this option allows you to specify which tables are truncated.
|
939
|
+
Defaults to `nil`.
|
1083
940
|
|
1084
|
-
|
941
|
+
```ruby
|
942
|
+
TestData.config do |config|
|
943
|
+
config.truncate_these_test_data_tables = []
|
944
|
+
end
|
945
|
+
```
|
1085
946
|
|
1086
|
-
|
1087
|
-
runner if you're planning to use
|
1088
|
-
[TestData.load_rails_fixtures](#testdataload_rails_fixtures) to load Rails
|
1089
|
-
fixtures into any of your tests. This method will disable the default behavior
|
1090
|
-
of loading your Rails fixtures into the test database as soon as the first test
|
1091
|
-
case with fixtures enabled is executed. (Inspect the [source for the
|
1092
|
-
patch](/lib/test_data/active_record_ext.rb) to make sure you're comfortable with
|
1093
|
-
what it's doing.)
|
947
|
+
##### config.log_level
|
1094
948
|
|
1095
|
-
|
949
|
+
The gem outputs its messages to standard output and error by assigning a log
|
950
|
+
level to each message. Valid values are `:debug`, `:info`, `:warn`, `:error`,
|
951
|
+
`:quiet`. Defaults to `:info`.
|
1096
952
|
|
1097
|
-
|
1098
|
-
|
1099
|
-
|
1100
|
-
|
1101
|
-
|
1102
|
-
feature uses Rails built-in fixtures-loading code as well as its caching
|
1103
|
-
mechanism, the method must be passed an instance of the running test class (in
|
1104
|
-
a Minitest `setup` hook, that means `TestData.load_rails_fixtures(self)`)
|
953
|
+
```ruby
|
954
|
+
TestData.config do |config|
|
955
|
+
config.log_level = :info
|
956
|
+
end
|
957
|
+
```
|
1105
958
|
|
1106
|
-
|
959
|
+
### TestData.insert_test_data_dump
|
1107
960
|
|
1108
|
-
|
1109
|
-
|
1110
|
-
|
1111
|
-
|
961
|
+
If you just want to insert the test data in your application's SQL dumps without
|
962
|
+
any of the transaction management or test runner assumptions inherent in
|
963
|
+
[TestData.uses_test_data](#testdatauses_test_data), then you can call
|
964
|
+
`TestData.insert_test_data_dump` to load and execute the dump.
|
1112
965
|
|
1113
|
-
|
966
|
+
This might be necessary in a few different situations:
|
1114
967
|
|
1115
|
-
|
1116
|
-
|
1117
|
-
|
1118
|
-
|
1119
|
-
|
968
|
+
* Running tests in environments that can't be isolated to a single database
|
969
|
+
transaction (e.g. orchestrating tests across multiple databases, processes,
|
970
|
+
etc.)
|
971
|
+
* You might ant to use your test data to seed pre-production environments with
|
972
|
+
enough data to exploratory test (as you might do in a `postdeploy` script with
|
973
|
+
your [Heroku Review
|
974
|
+
Apps](https://devcenter.heroku.com/articles/github-integration-review-apps))
|
975
|
+
* Your tests require complex heterogeneous sources of data that aren't a good
|
976
|
+
fit for the assumptions and constraints of this library's default methods for
|
977
|
+
preparing test data
|
1120
978
|
|
1121
|
-
|
1122
|
-
transaction
|
1123
|
-
|
979
|
+
In any case, since `TestData.insert_test_data_dump` is not wrapped in a
|
980
|
+
transaction, when used for automated tests, data cleanup becomes your
|
981
|
+
responsibility.
|
1124
982
|
|
1125
983
|
## Assumptions
|
1126
984
|
|
@@ -1148,12 +1006,11 @@ yet. Here are some existing assumptions and limitations:
|
|
1148
1006
|
|
1149
1007
|
If you use `factory_bot` and all of these are true:
|
1150
1008
|
|
1151
|
-
* Your integration tests are super fast and not getting significantly slower
|
1009
|
+
* Your integration tests are super fast and are not getting significantly slower
|
1152
1010
|
over time
|
1153
1011
|
|
1154
|
-
*
|
1155
|
-
|
1156
|
-
each of those tests be analyzed & updated to get them passing again
|
1012
|
+
* Minor changes to existing factories rarely result in test failures that
|
1013
|
+
require unrelated tests to be read & updated to get them passing again
|
1157
1014
|
|
1158
1015
|
* The number of associated records generated between your most-used factories
|
1159
1016
|
are representative of production data, as opposed to generating a sprawling
|
@@ -1169,16 +1026,16 @@ If you use `factory_bot` and all of these are true:
|
|
1169
1026
|
confidence-eroding nested factories with names like `:user`, `:basic_user`,
|
1170
1027
|
`:lite_user`, and `:plain_user_no_associations_allowed`
|
1171
1028
|
|
1172
|
-
If none of these things are true, then congratulations! You are using
|
1173
|
-
`factory_bot`
|
1029
|
+
If none of these things are true, then congratulations! You are probably using
|
1030
|
+
`factory_bot` to great effect! Unfortunately, in our experience, this outcome
|
1174
1031
|
is exceedingly rare, especially for large and long-lived applications.
|
1175
1032
|
|
1176
|
-
However, if any of the above
|
1177
|
-
|
1178
|
-
|
1179
|
-
|
1180
|
-
|
1181
|
-
|
1033
|
+
However, if you'd answer "no" to any of the above questions, just know that
|
1034
|
+
these are the sorts of failure modes the `test_data` gem was designed to
|
1035
|
+
avoid—and we hope you'll consider trying it with an open mind. At the same time,
|
1036
|
+
we acknowledge that large test suites can't be rewritten and migrated to a
|
1037
|
+
different source of test data overnight—nor should they be! See our notes on
|
1038
|
+
[migrating to `test_data`
|
1182
1039
|
incrementally](#factory--fixture-interoperability-guide)
|
1183
1040
|
|
1184
1041
|
### How will I handle merge conflicts in these SQL files if I have lots of people working on lots of feature branches all adding to the `test_data` database dumps?
|
@@ -1198,13 +1055,13 @@ this risk. The reason that the dumps are stored as plain SQL (aside from the
|
|
1198
1055
|
fact that git's text compression is very good) is to make merge conflicts with
|
1199
1056
|
other branches feasible, if not entirely painless.
|
1200
1057
|
|
1201
|
-
However, if your app is in the very initial stages of development
|
1202
|
-
making breaking changes to your schema very frequently, our
|
1203
|
-
hold off a bit on writing _any_ integration tests that depend
|
1204
|
-
of test data, as they'll be more likely
|
1205
|
-
iterate than detect bugs. Once you you have
|
1206
|
-
end-to-end, that's a good moment to start
|
1207
|
-
pulling in a
|
1058
|
+
However, if your app is in the very initial stages of development or you're
|
1059
|
+
otherwise making breaking changes to your schema and data very frequently, our
|
1060
|
+
best advice is to hold off a bit on writing _any_ integration tests that depend
|
1061
|
+
on shared sources of test data (regardless of tool), as they'll be more likely
|
1062
|
+
to frustrate your ability to rapidly iterate than detect bugs. Once you you have
|
1063
|
+
a reasonably stable feature working end-to-end, that's a good moment to start
|
1064
|
+
adding integration tests—and perhaps pulling in a gem like this one to help you.
|
1208
1065
|
|
1209
1066
|
### Why can't I save multiple database dumps to cover different scenarios?
|
1210
1067
|
|
@@ -1220,8 +1077,7 @@ By having a single `test_data` database that grows up with your application just
|
|
1220
1077
|
like `production` does—with both having their schemas and data migrated
|
1221
1078
|
incrementally over time—your integration tests that depend on `test_data` will
|
1222
1079
|
have an early opportunity to catch bugs that otherwise wouldn't be found until
|
1223
|
-
they were deployed into a long-lived
|
1224
|
-
production itself.
|
1080
|
+
they were deployed into a long-lived staging or (gasp!) production environment.
|
1225
1081
|
|
1226
1082
|
### Are you sure I should commit these SQL dumps? They're way too big!
|
1227
1083
|
|
@@ -1232,16 +1088,17 @@ cause:
|
|
1232
1088
|
resetting (or rolling back) your changes and making another attempt at
|
1233
1089
|
generating a more minimal set of test data
|
1234
1090
|
|
1235
|
-
2. If
|
1236
|
-
tests
|
1091
|
+
2. If some records persisted by your application aren't very relevant to your
|
1092
|
+
tests, you might consider either of these options:
|
1237
1093
|
|
1238
|
-
*
|
1239
|
-
|
1094
|
+
* If certain tables are necessary for running the app but aren't needed by
|
1095
|
+
your tests, you can add them to the `config.non_test_data_tables`
|
1096
|
+
configuration array. They'll still be committed to git, but won't loaded
|
1097
|
+
by your tests
|
1240
1098
|
|
1241
|
-
*
|
1242
|
-
`config.dont_dump_these_tables` array
|
1243
|
-
won't be
|
1244
|
-
so if the data is needed for the app to operate, you'll need to dump them)
|
1099
|
+
* If the certain tables are not needed by your application or by your tests
|
1100
|
+
(e.g. audit logs), add them to the `config.dont_dump_these_tables` array,
|
1101
|
+
and they won't be persisted by `rake test_data:dump`
|
1245
1102
|
|
1246
1103
|
3. If the dumps are _necessarily_ really big (some apps are complex!), consider
|
1247
1104
|
looking into [git-lfs](https://git-lfs.github.com) for tracking them without
|
@@ -1264,7 +1121,7 @@ test data loaded from this gem or any other:
|
|
1264
1121
|
def test_exclude_cancelled_orders
|
1265
1122
|
good_order = Order.new
|
1266
1123
|
bad_order = Order.new(cancelled: true)
|
1267
|
-
user = User.create!(orders: good_order, bad_order)
|
1124
|
+
user = User.create!(orders: [good_order, bad_order])
|
1268
1125
|
|
1269
1126
|
result = user.active_orders
|
1270
1127
|
|
@@ -1273,17 +1130,18 @@ def test_exclude_cancelled_orders
|
|
1273
1130
|
end
|
1274
1131
|
```
|
1275
1132
|
|
1276
|
-
This test is simple, self-contained, clearly
|
1277
|
-
[arrange-act-assert](https://github.com/testdouble/contributing-tests/wiki/Arrange-Act-Assert)
|
1278
|
-
and (most importantly) will only fail if the functionality stops
|
1279
|
-
Maximizing the number of tests that can be written expressively and
|
1280
|
-
without the aid of shared test data is a laudable goal that more
|
1281
|
-
embrace.
|
1133
|
+
This test is simple, self-contained, clearly demarcates the
|
1134
|
+
[arrange-act-assert](https://github.com/testdouble/contributing-tests/wiki/Arrange-Act-Assert)
|
1135
|
+
phases, and (most importantly) will only fail if the functionality stops
|
1136
|
+
working. Maximizing the number of tests that can be written expressively and
|
1137
|
+
succinctly without the aid of shared test data is a laudable goal that more
|
1138
|
+
teams should embrace.
|
1282
1139
|
|
1283
1140
|
However, what if the code you're writing doesn't need 3 records in the database,
|
1284
|
-
but 30? Writing that much test setup would be painstaking
|
1285
|
-
fully-encapsulated
|
1286
|
-
|
1141
|
+
but 30? Writing that much test setup would be painstaking, despite being
|
1142
|
+
fully-encapsulated. Long test setup is harder for others to read and understand.
|
1143
|
+
And because that setup depends on more of your system's code, it will have more
|
1144
|
+
reasons to break as your codebase changes. At that point, you have two options:
|
1287
1145
|
|
1288
1146
|
1. Critically validate your design: why is it so hard to set up? Does it
|
1289
1147
|
_really_ require so much persisted data to exercise this behavior? Would a
|
@@ -1294,8 +1152,8 @@ point, you have two options:
|
|
1294
1152
|
[subject](https://github.com/testdouble/contributing-tests/wiki/Subject)
|
1295
1153
|
instead of loading everything from the database? When automated testing is
|
1296
1154
|
saved for the very end of a feature's development, it can feel too costly to
|
1297
|
-
reexamine design decisions like this, but it
|
1298
|
-
same. *Easy to test code is easy to use code*
|
1155
|
+
reexamine design decisions like this, but it can be valuable to consider all
|
1156
|
+
the same. *Easy to test code is easy to use code*
|
1299
1157
|
|
1300
1158
|
2. If the complex setup is a necessary reality of the situation that your app
|
1301
1159
|
needs to handle (and it often will be!), then having _some_ kind of shared
|
@@ -1304,16 +1162,17 @@ point, you have two options:
|
|
1304
1162
|
|
1305
1163
|
As a result, there is no one-size-fits-all approach. Straightforward behavior
|
1306
1164
|
that can be invoked with a clear, concise test has no reason to be coupled to a
|
1307
|
-
shared source of test data.
|
1308
|
-
carefully-arranged data
|
1309
|
-
|
1165
|
+
shared source of test data. Meanwhile, tests of more complex behaviors that
|
1166
|
+
require lots of carefully-arranged data might be unmaintainable without a shared
|
1167
|
+
source of test data to lean on. So both kinds of test clearly have their place.
|
1310
1168
|
|
1311
1169
|
But this is a pretty nuanced discussion that can be hard to keep in mind when
|
1312
1170
|
under deadline pressure or on a large team where building consensus around norms
|
1313
1171
|
is challenging. As a result, leaving the decision of which type of test to write
|
1314
1172
|
to spur-of-the-moment judgment is likely to result in inconsistent test design.
|
1315
1173
|
Instead, you might consider separating these two categories into separate test
|
1316
|
-
types or suites
|
1174
|
+
types or suites, with simple heuristics to determine which types of code demand
|
1175
|
+
which type of test.
|
1317
1176
|
|
1318
1177
|
For example, it would be completely reasonable to load this gem's test data for
|
1319
1178
|
integration tests, but not for basic tests of models, like so:
|
@@ -1321,21 +1180,13 @@ integration tests, but not for basic tests of models, like so:
|
|
1321
1180
|
```ruby
|
1322
1181
|
class ActionDispatch::IntegrationTest
|
1323
1182
|
setup do
|
1324
|
-
TestData.
|
1325
|
-
end
|
1326
|
-
|
1327
|
-
teardown do
|
1328
|
-
TestData.rollback
|
1183
|
+
TestData.uses_test_data
|
1329
1184
|
end
|
1330
1185
|
end
|
1331
1186
|
|
1332
1187
|
class ActiveSupport::TestCase
|
1333
1188
|
setup do
|
1334
|
-
TestData.
|
1335
|
-
end
|
1336
|
-
|
1337
|
-
teardown do
|
1338
|
-
TestData.rollback(:after_data_truncate)
|
1189
|
+
TestData.uses_clean_slate
|
1339
1190
|
end
|
1340
1191
|
end
|
1341
1192
|
```
|
@@ -1344,6 +1195,50 @@ In short, this skepticism is generally healthy, and encapsulated tests that
|
|
1344
1195
|
forego reliance on shared sources of test data should be maximized. For
|
1345
1196
|
everything else, there's `test_data`.
|
1346
1197
|
|
1198
|
+
### I'm worried my tests aren't as fast as they should be
|
1199
|
+
|
1200
|
+
The `test_data` gem was written to enable tests that are not only more
|
1201
|
+
comprehensible and maintainable over the long-term, but also _much faster_ to
|
1202
|
+
run. That said—and especially if you're adding `test_data` to an existing test
|
1203
|
+
suite—care should be taken to audit everything the suite does between tests in
|
1204
|
+
order to optimize its overall runtime.
|
1205
|
+
|
1206
|
+
The first and most likely source of unnecessary slowness is redundant test
|
1207
|
+
cleanup—the speed gained from sandwiching every expensive operation between
|
1208
|
+
transaction savepoints can be profound… but can also easily be erased by a
|
1209
|
+
single before-each hook calling
|
1210
|
+
[database_cleaner](https://github.com/DatabaseCleaner/database_cleaner) to
|
1211
|
+
commit a truncation of the database. As a result, it's worth taking a little
|
1212
|
+
time to take stock of everything that's called between tests during setup &
|
1213
|
+
teardown to ensure multiple tools aren't attempting to clean up the state of the
|
1214
|
+
database and potentially interfering with one another.
|
1215
|
+
|
1216
|
+
A second opportunity for optimization is to group tests that use the same type
|
1217
|
+
of test data together, either into separate suites or by preventing them from
|
1218
|
+
being run in random order across said types. For example, suppose you have 10
|
1219
|
+
tests that call `TestData.uses_test_data` and 10 that call
|
1220
|
+
`TestData.uses_rails_fixtures`. If a test that calls `TestData.uses_test_data`
|
1221
|
+
is followed by another that calls `uses_test_data`, the only operation needed by
|
1222
|
+
the second call will be a rollback to the savepoint taken after the test data
|
1223
|
+
was loaded. If, however, a `uses_test_data` test is followed by a
|
1224
|
+
`uses_rails_fixtures` test, then the test data will be truncated and the
|
1225
|
+
fixtures loaded and new savepoints created (which would then be undone again if
|
1226
|
+
the _next_ test happened to call `uses_test_data`).
|
1227
|
+
|
1228
|
+
As a result of the above, the marginal runtime cost for each `TestData.uses_*`
|
1229
|
+
method depends on which kinds of test precedes and follows it. That means your
|
1230
|
+
tests will run faster overall if the tests that call `TestData.uses_test_data`
|
1231
|
+
are run as a group separately from your tests that rely on
|
1232
|
+
`TestData.uses_clean_slate` or `TestData.uses_rails_fixtures`. Separating your
|
1233
|
+
tests into logical groups pretty trivial if you're using RSpec, as the
|
1234
|
+
[tag](https://relishapp.com/rspec/rspec-core/v/3-10/docs/command-line/tag-option)
|
1235
|
+
feature was built with this sort of need in mind. If you're using Minitest, you
|
1236
|
+
might consider organizing the tests in different directories and running
|
1237
|
+
multiple commands to execute them (e.g. `bin/rails test test/test_data_tests`
|
1238
|
+
and `bin/rails test/factory_tests`). Every CI configuration is different,
|
1239
|
+
however, and you may find yourself needing to get creative in configuring things
|
1240
|
+
to achieve the fastest build time.
|
1241
|
+
|
1347
1242
|
## Code of Conduct
|
1348
1243
|
|
1349
1244
|
This project follows Test Double's [code of
|