test_data 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/ruby.yml +1 -5
- data/.standard.yml +2 -0
- data/CHANGELOG.md +38 -1
- data/Gemfile.lock +15 -15
- data/LICENSE.txt +1 -1
- data/README.md +701 -712
- data/example/.gitignore +1 -4
- data/example/Gemfile.lock +1 -1
- data/example/config/application.rb +3 -0
- data/example/config/credentials.yml.enc +1 -2
- data/example/spec/rails_helper.rb +1 -1
- data/example/spec/requests/boops_spec.rb +1 -5
- data/example/spec/requests/rails_fixtures_override_spec.rb +106 -0
- data/example/test/integration/better_mode_switching_demo_test.rb +2 -10
- data/example/test/integration/fixture_load_count_test.rb +82 -0
- data/example/test/integration/load_rollback_truncate_test.rb +40 -45
- data/example/test/integration/mode_switching_demo_test.rb +4 -14
- data/example/test/integration/parallel_boops_with_fixtures_test.rb +1 -5
- data/example/test/integration/parallel_boops_without_fixtures_test.rb +1 -5
- data/example/test/integration/rails_fixtures_double_load_test.rb +2 -2
- data/example/test/integration/rails_fixtures_override_test.rb +18 -35
- data/example/test/integration/test_data_hooks_test.rb +89 -0
- data/example/test/integration/transaction_committing_boops_test.rb +1 -10
- data/example/test/test_helper.rb +1 -5
- data/lib/generators/test_data/environment_file_generator.rb +4 -0
- data/lib/generators/test_data/initializer_generator.rb +19 -13
- data/lib/test_data/config.rb +30 -12
- data/lib/test_data/custom_loaders/abstract_base.rb +25 -0
- data/lib/test_data/custom_loaders/rails_fixtures.rb +45 -0
- data/lib/test_data/detects_database_existence.rb +19 -0
- data/lib/test_data/determines_databases_associated_dump_time.rb +13 -0
- data/lib/test_data/determines_when_sql_dump_was_made.rb +24 -0
- data/lib/test_data/dumps_database.rb +3 -0
- data/lib/test_data/inserts_test_data.rb +25 -0
- data/lib/test_data/manager.rb +187 -0
- data/lib/test_data/railtie.rb +4 -0
- data/lib/test_data/rake.rb +41 -12
- data/lib/test_data/records_dump_metadata.rb +9 -0
- data/lib/test_data/truncates_test_data.rb +31 -0
- data/lib/test_data/version.rb +1 -1
- data/lib/test_data/warns_if_database_is_newer_than_dump.rb +32 -0
- data/lib/test_data/warns_if_dump_is_newer_than_database.rb +36 -0
- data/lib/test_data.rb +43 -1
- data/script/reset_example_app +1 -0
- data/script/test +54 -6
- metadata +17 -3
- data/lib/test_data/transactional_data_loader.rb +0 -300
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 8c9bcdeaebcbad6163fb495b327408b64f7d85f90efd7ab171275ba4fd61e484
|
4
|
+
data.tar.gz: c08433086b9cfcb57e88e275e63f8600083945afdd738962a952e94573b95f77
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: e69305fb469eab8714569cea4b57fb6a3b9050ea97065230ef86afa27e8c089d094b75673515348e31f8ec989a61924fb7924d70c21fdbd31decdef8e490459e
|
7
|
+
data.tar.gz: 1592fccf3753f33c74f449ad2debe029806778383fd07f089c9bb67e49861e5a3885d04497720dd04525511182ca8b87ff6774c90837845204f210d85f3db270
|
data/.github/workflows/ruby.yml
CHANGED
data/.standard.yml
ADDED
data/CHANGELOG.md
CHANGED
@@ -1,4 +1,41 @@
|
|
1
|
-
#
|
1
|
+
# 0.3.0
|
2
|
+
|
3
|
+
- Add a `test_data:reinitialize` task that will delete the `test_data` database
|
4
|
+
if necessary before invoking `test_data:initialize`
|
5
|
+
- Warn if re-initializing and the local database appears to have been dumped
|
6
|
+
or loaded from a dump that is newer than the dumps on disk
|
7
|
+
- Add a warning on app load if the dumps on disk appear
|
8
|
+
newer than the local `test_data` database
|
9
|
+
|
10
|
+
# 0.2.2
|
11
|
+
|
12
|
+
- Improve performance of Rails fixtures being repeatedly loaded by changing the
|
13
|
+
caching strategy
|
14
|
+
|
15
|
+
# 0.2.1
|
16
|
+
|
17
|
+
- Adds several lifecycle hooks:
|
18
|
+
- config.after_test_data_load
|
19
|
+
- config.after_test_data_truncate
|
20
|
+
- config.after_rails_fixture_load
|
21
|
+
|
22
|
+
# 0.2.0
|
23
|
+
|
24
|
+
- BREAKING CHANGES: Remove or rename a bunch of APIs that aren't quite necessary
|
25
|
+
and leak too much implementation, requiring too much cognitive load for users.
|
26
|
+
- Remove config.use_transactional_data_loader
|
27
|
+
- Remove TestData.rollback
|
28
|
+
- Change TestData.load to TestData.uses_test_data and make it transaction-only
|
29
|
+
- Change TestData.truncate to TestData.uses_clean_slate and make it
|
30
|
+
transaction-only
|
31
|
+
- Change TestData.load_rails_fixtures to TestData.uses_rails_fixtures and make
|
32
|
+
it transaction-only
|
33
|
+
- Add TestData.insert_test_data_dump, which will blindly insert the test SQL
|
34
|
+
dump of test data without any transaction management
|
35
|
+
- [#2](https://github.com/testdouble/test_data/issues/2) - Work around
|
36
|
+
hard-coded environment names when initializing test_data environment secrets
|
37
|
+
|
38
|
+
# 0.1.0
|
2
39
|
|
3
40
|
- New feature: `TestData.load_rails_fixtures` to override default fixtures
|
4
41
|
behavior by loading it in a nested transaction after `TestData.truncate`
|
data/Gemfile.lock
CHANGED
@@ -1,26 +1,26 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
test_data (0.
|
4
|
+
test_data (0.3.0)
|
5
5
|
railties (~> 6.0)
|
6
6
|
|
7
7
|
GEM
|
8
8
|
remote: https://rubygems.org/
|
9
9
|
specs:
|
10
|
-
actionpack (6.1.4)
|
11
|
-
actionview (= 6.1.4)
|
12
|
-
activesupport (= 6.1.4)
|
10
|
+
actionpack (6.1.4.1)
|
11
|
+
actionview (= 6.1.4.1)
|
12
|
+
activesupport (= 6.1.4.1)
|
13
13
|
rack (~> 2.0, >= 2.0.9)
|
14
14
|
rack-test (>= 0.6.3)
|
15
15
|
rails-dom-testing (~> 2.0)
|
16
16
|
rails-html-sanitizer (~> 1.0, >= 1.2.0)
|
17
|
-
actionview (6.1.4)
|
18
|
-
activesupport (= 6.1.4)
|
17
|
+
actionview (6.1.4.1)
|
18
|
+
activesupport (= 6.1.4.1)
|
19
19
|
builder (~> 3.1)
|
20
20
|
erubi (~> 1.4)
|
21
21
|
rails-dom-testing (~> 2.0)
|
22
22
|
rails-html-sanitizer (~> 1.1, >= 1.2.0)
|
23
|
-
activesupport (6.1.4)
|
23
|
+
activesupport (6.1.4.1)
|
24
24
|
concurrent-ruby (~> 1.0, >= 1.0.2)
|
25
25
|
i18n (>= 1.6, < 2)
|
26
26
|
minitest (>= 5.1)
|
@@ -34,14 +34,14 @@ GEM
|
|
34
34
|
erubi (1.10.0)
|
35
35
|
i18n (1.8.10)
|
36
36
|
concurrent-ruby (~> 1.0)
|
37
|
-
loofah (2.
|
37
|
+
loofah (2.12.0)
|
38
38
|
crass (~> 1.0.2)
|
39
39
|
nokogiri (>= 1.5.9)
|
40
40
|
method_source (1.0.0)
|
41
|
-
mini_portile2 (2.
|
41
|
+
mini_portile2 (2.6.1)
|
42
42
|
minitest (5.14.4)
|
43
|
-
nokogiri (1.
|
44
|
-
mini_portile2 (~> 2.
|
43
|
+
nokogiri (1.12.4)
|
44
|
+
mini_portile2 (~> 2.6.1)
|
45
45
|
racc (~> 1.4)
|
46
46
|
parallel (1.20.1)
|
47
47
|
parser (3.0.1.0)
|
@@ -56,11 +56,11 @@ GEM
|
|
56
56
|
rails-dom-testing (2.0.3)
|
57
57
|
activesupport (>= 4.2.0)
|
58
58
|
nokogiri (>= 1.6)
|
59
|
-
rails-html-sanitizer (1.
|
59
|
+
rails-html-sanitizer (1.4.2)
|
60
60
|
loofah (~> 2.3)
|
61
|
-
railties (6.1.4)
|
62
|
-
actionpack (= 6.1.4)
|
63
|
-
activesupport (= 6.1.4)
|
61
|
+
railties (6.1.4.1)
|
62
|
+
actionpack (= 6.1.4.1)
|
63
|
+
activesupport (= 6.1.4.1)
|
64
64
|
method_source
|
65
65
|
rake (>= 0.13)
|
66
66
|
thor (~> 1.0)
|
data/LICENSE.txt
CHANGED
data/README.md
CHANGED
@@ -18,23 +18,24 @@ What it does:
|
|
18
18
|
files, no precarious approximations of realism: **real data created by your
|
19
19
|
app**
|
20
20
|
|
21
|
-
* Exposes a simple API for
|
22
|
-
tests
|
23
|
-
|
21
|
+
* Exposes a simple API for ensuring that your data will be pristine for each of
|
22
|
+
your tests, whether the test depends on test_data, an empty database, or Rails
|
23
|
+
fixtures
|
24
24
|
|
25
25
|
* Safeguards your tests from flaky failures and supercharges your build by
|
26
26
|
providing a sophisticated transaction manager that isolates each test while
|
27
27
|
ensuring your data is only loaded once
|
28
28
|
|
29
29
|
If you've despaired over the seeming inevitability that all Rails test suites
|
30
|
-
will eventually grow to become slow,
|
31
|
-
|
30
|
+
will eventually grow to become slow, flaky, and incomprehensible, then this gem
|
31
|
+
is for you! And even if you're [a factory_bot
|
32
32
|
fan](https://twitter.com/searls/status/1379491813099253762?s=20), we hope you'll
|
33
33
|
be open to the idea that [there might be a better way](
|
34
34
|
#but-we-use-and-like-factory_bot-and-so-i-am-inclined-to-dislike-everything-about-this-gem).
|
35
35
|
|
36
36
|
_[Full disclosure: because the gem is still brand new, it makes a number of
|
37
|
-
[assumptions](#assumptions)
|
37
|
+
[assumptions](#assumptions)—chief among them being that **Postgres & Rails 6+
|
38
|
+
are required**—so it may not work for every project just yet.]_
|
38
39
|
|
39
40
|
## Documentation
|
40
41
|
|
@@ -42,21 +43,17 @@ This gem requires a lot of documentation—not because `test_data` does a lot of
|
|
42
43
|
things, but because managing one's test data is an inherently complex task. If
|
43
44
|
one reason Rails apps chronically suffer from slow tests is that other
|
44
45
|
approaches oversimplify test data management, it stands to reason that any
|
45
|
-
discomfort caused by `test_data`'s scope may not
|
46
|
-
|
47
|
-
|
48
|
-
application.
|
46
|
+
discomfort caused by `test_data`'s scope may not be _unnecessary complexity_ but
|
47
|
+
instead be an indication of how little of the problem's _essential complexity_
|
48
|
+
we have reckoned with to this point.
|
49
49
|
|
50
50
|
1. [Getting Started Guide](#getting-started-guide)
|
51
|
-
1. [Install and initialize
|
52
|
-
`test_data`](#step-1-install-and-initialize-test_data)
|
51
|
+
1. [Install and initialize `test_data`](#step-1-install-and-initialize-test_data)
|
53
52
|
2. [Create some test data](#step-2-create-some-test-data)
|
54
53
|
3. [Dump your `test_data` database](#step-3-dump-your-test_data-database)
|
55
54
|
4. [Load your data in your tests](#step-4-load-your-data-in-your-tests)
|
56
|
-
5. [Keeping your test data
|
57
|
-
|
58
|
-
2. [Factory & Fixture Interoperability
|
59
|
-
Guide](#factory--fixture-interoperability-guide)
|
55
|
+
5. [Keeping your test data up-to-date](#step-5-keeping-your-test-data-up-to-date)
|
56
|
+
2. [Factory & Fixture Interoperability Guide](#factory--fixture-interoperability-guide)
|
60
57
|
* [Using `test_data` with `factory_bot`](#using-test_data-with-factory_bot)
|
61
58
|
* [Using `test_data` with Rails fixtures](#using-test_data-with-rails-fixtures)
|
62
59
|
3. [Rake Task Reference](#rake-task-reference)
|
@@ -64,23 +61,32 @@ application.
|
|
64
61
|
* [test_data:configure](#test_dataconfigure)
|
65
62
|
* [test_data:verify_config](#test_dataverify_config)
|
66
63
|
* [test_data:initialize](#test_datainitialize)
|
64
|
+
* [test_data:reinitialize](#test_datareinitialize)
|
67
65
|
* [test_data:dump](#test_datadump)
|
68
66
|
* [test_data:load](#test_dataload)
|
69
67
|
* [test_data:create_database](#test_datacreate_database)
|
70
68
|
* [test_data:drop_database](#test_datadrop_database)
|
71
69
|
4. [API Reference](#api-reference)
|
70
|
+
* [TestData.uses_test_data](#testdatauses_test_data)
|
71
|
+
* [TestData.uses_clean_slate](#testdatauses_clean_slate)
|
72
|
+
* [TestData.uses_rails_fixtures(self)](#testdatauses_rails_fixtures)
|
73
|
+
* [TestData.prevent_rails_fixtures_from_loading_automatically!](#testdataprevent_rails_fixtures_from_loading_automatically)
|
72
74
|
* [TestData.config](#testdataconfig)
|
73
|
-
* [TestData.
|
74
|
-
* [TestData.rollback](#testdatarollback)
|
75
|
-
* [TestData.rollback(:before_data_load)](#rolling-back-to-before-test-data-was-loaded)
|
76
|
-
* [TestData.rollback(:after_data_load)](#rolling-back-to-after-the-data-was-loaded)
|
77
|
-
* [TestData.rollback(:after_data_truncate)](#rolling-back-to-after-test-data-was-truncated)
|
78
|
-
* [TestData.rollback(:after_load_rails_fixtures)](#rolling-back-to-after-rails-fixtures-were-loaded)
|
79
|
-
* [TestData.truncate](#testdatatruncate)
|
80
|
-
* [TestData.prevent_rails_fixtures_from_loading_automatically!](#testdataprevent_rails_fixtures_from_loading_automatically)
|
81
|
-
* [TestData.load_rails_fixtures](#testdataload_rails_fixtures)
|
75
|
+
* [TestData.insert_test_data_dump](#testdatainsert_test_data_dump)
|
82
76
|
5. [Assumptions](#assumptions)
|
83
77
|
6. [Fears, Uncertainties, and Doubts](#fears-uncertainties-and-doubts) (Q & A)
|
78
|
+
* [But we're already happy with
|
79
|
+
factory_bot!](#but-we-use-and-like-factory_bot-and-so-i-am-inclined-to-dislike-everything-about-this-gem)
|
80
|
+
* [How will we handle merge conflicts in the schema
|
81
|
+
dumps?](#how-will-i-handle-merge-conflicts-in-these-sql-files-if-i-have-lots-of-people-working-on-lots-of-feature-branches-all-adding-to-the-test_data-database-dumps)
|
82
|
+
* [Why can't I manage different SQL dumps for different
|
83
|
+
scenarios?](#why-cant-i-save-multiple-database-dumps-to-cover-different-scenarios)
|
84
|
+
* [These SQL dumps are way too large to commit to
|
85
|
+
git!](#are-you-sure-i-should-commit-these-sql-dumps-theyre-way-too-big)
|
86
|
+
* [Tests shouldn't rely on shared test data if they don't need
|
87
|
+
to](#tests-shouldnt-use-shared-test-data-they-should-instantiate-the-objects-they-need)
|
88
|
+
* [My tests aren't as fast as they should
|
89
|
+
be](#im-worried-my-tests-arent-as-fast-as-they-should-be)
|
84
90
|
7. [Code of Conduct](#code-of-conduct)
|
85
91
|
8. [Changelog](/CHANGELOG.md)
|
86
92
|
9. [MIT License](/LICENSE.txt)
|
@@ -140,21 +146,20 @@ Created database 'yourappname_test_data'
|
|
140
146
|
------------
|
141
147
|
|
142
148
|
(1 row)
|
143
|
-
````
|
144
149
|
|
145
|
-
|
146
|
-
|
147
|
-
|
148
|
-
|
149
|
-
|
150
|
-
|
151
|
-
created it!
|
150
|
+
Your test_data environment and database are ready for use! You can now run
|
151
|
+
your server (or any command) to create some test data like so:
|
152
|
+
|
153
|
+
$ RAILS_ENV=test_data bin/rails server
|
154
|
+
|
155
|
+
````
|
152
156
|
|
153
|
-
The
|
154
|
-
|
155
|
-
|
156
|
-
|
157
|
-
|
157
|
+
The purpose of the `test_data` database is to provide a sandbox in which you
|
158
|
+
will manually generate test data by playing around with your app. Rather than
|
159
|
+
try to imitate realistic data using factories and fixtures (a task which only
|
160
|
+
grows more difficult as your models and their associations increase in
|
161
|
+
complexity), your test data will always be realistic because your real
|
162
|
+
application will have created it!
|
158
163
|
|
159
164
|
### Step 2: Create some test data
|
160
165
|
|
@@ -181,31 +186,47 @@ number of Rails commands or Rake tasks against its database by setting
|
|
181
186
|
|
182
187
|
_[Aside: If you experience any hiccups in getting your server to work, please
|
183
188
|
[open an issue](https://github.com/testdouble/test_data/issues/new) and let us
|
184
|
-
know—it may present an opportunity to improve the `test_data:configure`
|
189
|
+
know—it may present an opportunity for us to improve the `test_data:configure`
|
190
|
+
task!]_
|
185
191
|
|
186
192
|
#### Create test data by using your app
|
187
193
|
|
188
194
|
Once the app is running, it's time to generate some test data. You'll know how
|
189
195
|
to accomplish this step better than anyone—it's your app, after all!
|
190
196
|
|
191
|
-
|
192
|
-
|
193
|
-
|
194
|
-
|
195
|
-
|
196
|
-
|
197
|
-
|
198
|
-
|
199
|
-
|
200
|
-
|
201
|
-
|
202
|
-
|
203
|
-
|
197
|
+
A few bits of advice click & type some test data into existence:
|
198
|
+
|
199
|
+
* Spend a little time thoughtfully navigating each feature of your app in order
|
200
|
+
to generate enough data to be representative of what would be needed to test
|
201
|
+
them (e.g. one `User` per role, one of each kind of `Order`, etc.)
|
202
|
+
* Less is more: the less test data you create, the more meaningful & memorable
|
203
|
+
it will be to yourself and your teammates when writing tests. Don't keep
|
204
|
+
adding test data unless it will allow you to exercise additional application
|
205
|
+
code (e.g. enough `Project` models to require pagination, but not hundreds of
|
206
|
+
them for the sake of looking "production-like")
|
207
|
+
* Memorable names can become memes for the team to quickly recall and reference
|
208
|
+
later (if the admin user is named "Angela" and the manager is "Maria", that'll
|
209
|
+
probably serve you better than generic names like "TestUser #1")
|
210
|
+
|
211
|
+
If you make a mistake when creating your initial set of test data, it's
|
212
|
+
perfectly okay to reset the database and start over! Your future tests will be
|
213
|
+
coupled to this data as your application grows and evolves, so it's worth taking
|
214
|
+
the time to ensure the foundation is solid. (But that's not to say everything
|
215
|
+
needs to be perfect; you can always change things or add more data later—you'll
|
216
|
+
just have to update your tests accordingly.)
|
204
217
|
|
205
218
|
### Step 3: Dump your `test_data` database
|
206
219
|
|
220
|
+
Once you've created a good sampling of test data by interacting with your app,
|
221
|
+
the next step is to flush it from the `test_data` database to SQL files. These
|
222
|
+
database dumps are meant to be committed to source control and versioned
|
223
|
+
alongside your tests over the life of the application. Additionally, they are
|
224
|
+
designed to be incrementally
|
225
|
+
[migrated](#step-5-keeping-your-test-data-up-to-date) over time, just like you
|
226
|
+
migrate production database with every release.
|
227
|
+
|
207
228
|
Once you have your test data how you want it, dump the schema and data to SQL
|
208
|
-
files:
|
229
|
+
files with the `test_data:dump` Rake task:
|
209
230
|
|
210
231
|
```
|
211
232
|
$ bin/rake test_data:dump
|
@@ -213,16 +234,19 @@ $ bin/rake test_data:dump
|
|
213
234
|
|
214
235
|
This will dump three files into `test/support/test_data`:
|
215
236
|
|
216
|
-
* Schema DDL
|
237
|
+
* `schema.sql` - Schema DDL used to (re-)initialize the `test_data` environment
|
238
|
+
database for anyone looking to update your test data
|
217
239
|
|
218
|
-
*
|
240
|
+
* `data.sql` - The test data itself, exported as a bunch of SQL `INSERT`
|
241
|
+
statements, which will be executed by your tests to load your test data
|
219
242
|
|
220
|
-
*
|
221
|
-
`
|
243
|
+
* `non_test_data.sql` - Data needed to run the `test_data` environment, but
|
244
|
+
which shouldn't be inserted by your tests (the `ar_internal_metadata` and
|
245
|
+
`schema_migrations` tables, by default; see `config.non_test_data_tables`)
|
222
246
|
|
223
|
-
|
224
|
-
Additional details can be found
|
225
|
-
Rake task reference.
|
247
|
+
You probably won't need to, but these paths can be overridden with
|
248
|
+
[TestData.config](#testdataconfig) method. Additional details can also be found
|
249
|
+
in the [test_data:dump](#test_datadump) Rake task reference.
|
226
250
|
|
227
251
|
Once you've made your initial set of dumps, briefly inspect them and—if
|
228
252
|
everything looks good—commit them. (And if the files are gigantic or full of
|
@@ -233,136 +257,201 @@ Does it feel weird to dump and commit SQL files? That's okay! It's [healthy to
|
|
233
257
|
be skeptical](https://twitter.com/searls/status/860553435116187649?s=20)
|
234
258
|
whenever you're asked to commit a generated file! Remember that the `test_data`
|
235
259
|
environment exists only for creating your test data. Your tests will, in turn,
|
236
|
-
load the SQL dump of your data into the
|
237
|
-
|
260
|
+
load the SQL dump of your data into the `test` database, and things will proceed
|
261
|
+
just as if you'd been loading [Rails' built-in
|
238
262
|
fixtures](https://guides.rubyonrails.org/testing.html#the-low-down-on-fixtures)
|
239
|
-
from a set of YAML files
|
240
|
-
are generated through realistic use, whereas fixtures are defined manually in
|
241
|
-
(sometimes painstaking) YAML.
|
263
|
+
from a set of YAML files.
|
242
264
|
|
243
265
|
### Step 4: Load your data in your tests
|
244
266
|
|
245
267
|
Now that you've dumped the contents of your `test_data` database, you can start
|
246
268
|
writing tests that rely on this test data.
|
247
269
|
|
248
|
-
To accomplish this, you'll likely want to add hooks to run before
|
249
|
-
|
250
|
-
|
251
|
-
|
252
|
-
|
270
|
+
To accomplish this, you'll likely want to add hooks to run before each test to
|
271
|
+
put the database into whatever state the test needs.
|
272
|
+
|
273
|
+
For the simplest case—ensuring your test data is loaded into the `test` database
|
274
|
+
and available to your test, you'll want to call the
|
275
|
+
[TestData.uses_test_data](#testdatauses_test_data) method at the beginning of
|
276
|
+
the test. The first time `uses_test_data` is called, `test_data` will start a
|
277
|
+
transaction and insert your test data. On subsequent calls to `uses_test_data`
|
278
|
+
by later tests, the transaction will be rolled back to a save point taken just
|
279
|
+
after the data was initially loaded, so that each test gets a clean starting
|
280
|
+
point without repeatedly executing the expensive SQL operation.
|
281
|
+
|
282
|
+
#### If you want every single test to have access to your test data
|
253
283
|
|
254
|
-
If you
|
255
|
-
|
256
|
-
|
284
|
+
If, for the sake of consistency & simplicity you want every single Rails-aware
|
285
|
+
test to have access to your test data, you
|
286
|
+
can accomplish this with a single global before-each hook.
|
287
|
+
|
288
|
+
If you're using Rails' default
|
289
|
+
[Minitest](https://github.com/seattlerb/minitest), you can load it in a `setup`
|
290
|
+
hook in `ActiveSupport::TestCase`:
|
257
291
|
|
258
292
|
```ruby
|
259
293
|
class ActiveSupport::TestCase
|
260
294
|
setup do
|
261
|
-
TestData.
|
262
|
-
end
|
263
|
-
|
264
|
-
teardown do
|
265
|
-
TestData.rollback
|
295
|
+
TestData.uses_test_data
|
266
296
|
end
|
267
297
|
end
|
268
298
|
```
|
269
299
|
|
270
|
-
|
271
|
-
global `before(:each)`
|
300
|
+
Likewise, if you use [RSpec](https://rspec.info), you can accomplish the same
|
301
|
+
thing with global `before(:each)` hook in your `rails_helper.rb` file:
|
272
302
|
|
273
303
|
```ruby
|
274
304
|
RSpec.configure do |config|
|
275
305
|
config.before(:each) do
|
276
|
-
TestData.
|
277
|
-
end
|
278
|
-
|
279
|
-
config.after(:each) do
|
280
|
-
TestData.rollback
|
306
|
+
TestData.uses_test_data
|
281
307
|
end
|
282
308
|
end
|
283
309
|
```
|
284
310
|
|
285
|
-
|
286
|
-
tests! Your tests will also benefit from the speed and data integrity that comes
|
287
|
-
with wrapping each test in an always-rolled-back transaction. For more
|
288
|
-
information on how all this works, see the [API reference](#api-reference). If
|
289
|
-
your test suite is already using fixtures or factories and the above hooks just
|
290
|
-
broke everything, check out our [interoperability
|
291
|
-
guide](#factory--fixture-interoperability-guide) for help.
|
311
|
+
#### If some tests rely on test data and others need a clean slate
|
292
312
|
|
293
|
-
|
294
|
-
|
295
|
-
|
296
|
-
|
313
|
+
Of course, for simple units of code, it may be more prudent to manually create
|
314
|
+
the test data they need inline as opposed to relying on a shared source of test
|
315
|
+
data. For these tests, you can call
|
316
|
+
[TestData.uses_clean_slate](#testdatauses_clean_slate) in a `setup` hook.
|
297
317
|
|
298
|
-
|
299
|
-
|
300
|
-
class TestDataTestCase < ActiveSupport::TestCase
|
301
|
-
setup do
|
302
|
-
TestData.load
|
303
|
-
end
|
318
|
+
For the best performance, you might consider a mode-switching method that's
|
319
|
+
invoked at the top of each test listing like this:
|
304
320
|
|
305
|
-
|
306
|
-
|
321
|
+
```ruby
|
322
|
+
class ActiveSupport::TestCase
|
323
|
+
def self.uses(mode)
|
324
|
+
case mode
|
325
|
+
when :clean_slate
|
326
|
+
setup { TestData.uses_clean_slate }
|
327
|
+
when :test_data
|
328
|
+
setup { TestData.uses_test_data }
|
329
|
+
else
|
330
|
+
raise "Invalid test data mode: #{mode}"
|
331
|
+
end
|
307
332
|
end
|
308
333
|
end
|
309
334
|
|
310
|
-
#
|
311
|
-
class
|
312
|
-
|
313
|
-
|
314
|
-
|
335
|
+
# A simple model that will `create` its own data
|
336
|
+
class WidgetTest < ActiveSupport::TestCase
|
337
|
+
uses :clean_slate
|
338
|
+
# …
|
339
|
+
end
|
315
340
|
|
316
|
-
|
317
|
-
|
341
|
+
# An integrated test that depends on a lot of data
|
342
|
+
class KitchenSinkTest < ActionDispatch::IntegrationTest
|
343
|
+
uses :test_data
|
344
|
+
# …
|
345
|
+
end
|
346
|
+
```
|
318
347
|
|
319
|
-
|
348
|
+
Or, with RSpec:
|
320
349
|
|
321
|
-
|
350
|
+
```ruby
|
351
|
+
module TestDataModes
|
352
|
+
def uses(mode)
|
353
|
+
case mode
|
354
|
+
when :clean_slate
|
355
|
+
before(:each) { TestData.uses_clean_slate }
|
356
|
+
when :test_data
|
357
|
+
before(:each) { TestData.uses_test_data }
|
358
|
+
else
|
359
|
+
raise "Invalid test data mode: #{mode}"
|
360
|
+
end
|
322
361
|
end
|
362
|
+
end
|
323
363
|
|
324
|
-
|
325
|
-
|
326
|
-
|
364
|
+
RSpec.configure do |config|
|
365
|
+
config.extend(TestDataModes)
|
366
|
+
end
|
367
|
+
|
368
|
+
RSpec.describe Widget, type: :model do
|
369
|
+
uses :clean_slate
|
370
|
+
# …
|
327
371
|
end
|
328
372
|
|
373
|
+
RSpec.describe "Kitchen sink", type: :request do
|
374
|
+
uses :test_data
|
375
|
+
# …
|
376
|
+
end
|
329
377
|
```
|
330
378
|
|
379
|
+
But wait, there's more! If your test suite switches between multiple modes from
|
380
|
+
test-to-test, it's important to be aware of the marginal cost _between_ each of
|
381
|
+
those tests. For example, two tests in a row that call `TestData.uses_test_data`
|
382
|
+
only need a simple rollback as test setup, but a `TestData.uses_test_data`
|
383
|
+
followed by a `TestData.uses_clean_slate` requires a rollback, a truncation, and
|
384
|
+
another savepoint. These small costs add up, so consider [speeding up your
|
385
|
+
build](#im-worried-my-tests-arent-as-fast-as-they-should-be) by grouping your
|
386
|
+
tests into sub-suites based on their source of test data.
|
387
|
+
|
388
|
+
#### If your situation is more complicated
|
389
|
+
|
390
|
+
If you're adding `test_data` to an existing application, it's likely that you
|
391
|
+
won't be able to easily adopt a one-size-fits-all approach to test setup across
|
392
|
+
your entire suite. Some points of reference, if that's the situation you're in:
|
393
|
+
|
394
|
+
* If your test suite is **already using fixtures or factories** and the above
|
395
|
+
hooks just broke everything, check out our [interoperability
|
396
|
+
guide](#factory--fixture-interoperability-guide) for help.
|
397
|
+
* If you need to make any changes to the data after it's loaded, truncated, or
|
398
|
+
after Rails fixtures are loaded, you can configure [lifecycle
|
399
|
+
hooks](#lifecycle-hooks) that will help you achieve a **very fast test suite**
|
400
|
+
by including those changes inside the transaction savepoints
|
401
|
+
* If you **don't want `test_data` managing transactions** and cleanup for you
|
402
|
+
and just want to load the SQL dump, you can call
|
403
|
+
[TestData.insert_test_data_dump](#testdatainsert_test_data_dump)
|
404
|
+
* For more information on how all this works, see the [API
|
405
|
+
reference](#api-reference).
|
406
|
+
|
331
407
|
### Step 5: Keeping your test data up-to-date
|
332
408
|
|
333
|
-
|
334
|
-
|
335
|
-
|
336
|
-
|
409
|
+
Your app relies on its tests and your tests rely on their test data. This
|
410
|
+
creates a bit of a paradox: creating & maintaining test data is _literally_ a
|
411
|
+
tertiary concern but simultaneously an inescapable responsibility that will live
|
412
|
+
with you for the life of your application. That's true whether you use this gem,
|
413
|
+
`factory_bot`, Rails fixtures, or something else as a source of shared test
|
414
|
+
data.
|
415
|
+
|
416
|
+
Fortunately, we already have a fantastic tool available for keeping our
|
417
|
+
`test_data` database up-to-date over the life of our application: [Rails
|
337
418
|
migrations](https://guides.rubyonrails.org/active_record_migrations.html). If
|
338
|
-
your migrations are resilient enough for your production
|
339
|
-
be able to keep your `test_data` database up-to-date.
|
419
|
+
your migrations are resilient enough for your production database, they should
|
420
|
+
also be able to keep your `test_data` database up-to-date. (As a happy side
|
421
|
+
effect of running your migrations against your test data, this means your
|
422
|
+
`test_data` database may help you identify hard-to-catch migration bugs early,
|
423
|
+
before being deployed to production!)
|
340
424
|
|
341
|
-
Whenever you
|
342
|
-
|
343
|
-
data. Here's a rough outline to updating your `test_data` database:
|
425
|
+
Whenever you create a new migration or add a major feature, you'll probably need
|
426
|
+
to update your test data. Here's how to do it:
|
344
427
|
|
345
|
-
|
346
|
-
|
428
|
+
* If the current SQL dumps in `test/support/test_data` are newer than your local
|
429
|
+
`test_data` database:
|
347
430
|
|
348
|
-
|
349
|
-
|
431
|
+
1. Be sure there's nothing in your local `test_data` database that you added
|
432
|
+
intentionally and forgot to dump, because it's about to be erased
|
350
433
|
|
351
|
-
|
434
|
+
2. Run `rake test_data:reinitialize` drop and recreate the `test_data`
|
435
|
+
database and load the latest SQL dumps into it
|
352
436
|
|
353
|
-
|
354
|
-
(`RAILS_ENV=test_data bin/rails s`), just like in [Step
|
355
|
-
2](#step-2-create-some-test-data)
|
437
|
+
3. Run any pending migrations with `RAILS_ENV=test_data bin/rake db:migrate`
|
356
438
|
|
357
|
-
|
439
|
+
4. If you need to create any additional data, start up the server
|
440
|
+
(`RAILS_ENV=test_data bin/rails s`), just like in [Step
|
441
|
+
2](#step-2-create-some-test-data)
|
358
442
|
|
359
|
-
|
443
|
+
5. Export your newly-updated `test_data` database with `rake test_data:dump`
|
444
|
+
|
445
|
+
6. Ensure your tests are passing and then commit the resulting SQL files
|
446
|
+
|
447
|
+
* If the local `test_data` database is already up-to-date with the current SQL
|
448
|
+
dumps, follow steps **3 through 6** above
|
360
449
|
|
361
450
|
It's important to keep in mind that your test data SQL dumps are a shared,
|
362
451
|
generated resource among your team (just like a `structure.sql` or `schema.rb`
|
363
452
|
file). As a result, if your team doesn't integrate code frequently or if the
|
364
|
-
test data
|
365
|
-
|
453
|
+
test data changes frequently, you'd be right to be concerned that [the resulting
|
454
|
+
merge conflicts could become
|
366
455
|
significant](#how-will-i-handle-merge-conflicts-in-these-sql-files-if-i-have-lots-of-people-working-on-lots-of-feature-branches-all-adding-to-the-test_data-database-dumps),
|
367
456
|
so sweeping changes should be made deliberately and in collaboration with other
|
368
457
|
contributors.
|
@@ -371,7 +460,10 @@ _[Aside: some Rails teams are averse to using migrations to migrate data as well
|
|
371
460
|
as schemas, instead preferring one-off scripts and tasks. You'll have an easier
|
372
461
|
time of things if you use migrations for both schema and data changes. Here are
|
373
462
|
some notes on [how to write data migrations
|
374
|
-
safely](https://blog.testdouble.com/posts/2014-11-04-healthy-migration-habits/#habit-4-dont-reference-models).
|
463
|
+
safely](https://blog.testdouble.com/posts/2014-11-04-healthy-migration-habits/#habit-4-dont-reference-models).
|
464
|
+
Otherwise, you'll need to remember to run any ad hoc deployment scripts against
|
465
|
+
your `test_data` Rails environment along with each of your other deployed
|
466
|
+
environments.]_
|
375
467
|
|
376
468
|
## Factory & Fixture Interoperability Guide
|
377
469
|
|
@@ -390,8 +482,8 @@ all three, or none-of-the-above.
|
|
390
482
|
|
391
483
|
This section will hopefully make it a little easier to incorporate new
|
392
484
|
`test_data` tests into a codebase that's already using `factory_bot` and/or
|
393
|
-
Rails fixtures, whether you choose to incrementally
|
394
|
-
|
485
|
+
Rails fixtures, whether you choose to incrementally migrate to using `test_data`
|
486
|
+
over time.
|
395
487
|
|
396
488
|
### Using `test_data` with `factory_bot`
|
397
489
|
|
@@ -402,182 +494,43 @@ This section will document some thoughts and strategies for introducing
|
|
402
494
|
|
403
495
|
Depending on the assumptions your tests make about the state of the database
|
404
496
|
before you've loaded any factories, it's possible that everything will "just
|
405
|
-
work" after adding
|
406
|
-
|
497
|
+
work" after adding [TestData.uses_test_data](#testdatauses_test_data) in a
|
498
|
+
before-each hook (as shown in the [setup
|
407
499
|
guide](#step-4-load-your-data-in-your-tests)). So by all means, try running your
|
408
500
|
suite after following the initial setup guide and see if the suite just passes.
|
409
501
|
|
410
|
-
If you find that your test suite is failing after adding
|
411
|
-
setup, don't panic!
|
412
|
-
|
413
|
-
|
414
|
-
well you understand the intentions of older tests) you might try to resolve
|
415
|
-
these errors one-by-one, usually by updating the offending factories or
|
416
|
-
editing your `test_data` database to ensure they steer clear of one another.
|
417
|
-
This can be tedious, however, and can undermine the completeness of either data
|
418
|
-
source in the absence of the other.
|
419
|
-
|
420
|
-
If your tests are failing after introducing `test_data` and it's not desirable
|
421
|
-
or feasible to work through the individual failures, you can accomplish a clean
|
422
|
-
segregation between your factory-dependent tests and your tests that rely on
|
423
|
-
`test_data` by wrapping each test that depends on `factory_bot` with
|
424
|
-
[TestData.truncate](#testdatatruncate) in a before-each hook and
|
425
|
-
[TestData.rollback(:after_data_truncate)](#rolling-back-to-after-test-data-was-truncated)
|
426
|
-
in an after-each hook, like this:
|
502
|
+
If you find that your test suite is failing after adding
|
503
|
+
`TestData.uses_test_data` to your setup, don't panic! Test failures are most
|
504
|
+
likely caused by the combination of your `test_data` SQL dump with the records
|
505
|
+
inserted by your factories.
|
427
506
|
|
428
|
-
|
429
|
-
|
430
|
-
|
431
|
-
|
432
|
-
|
433
|
-
|
507
|
+
One approach would be to attempt to resolve each such failure one-by-one—usually
|
508
|
+
by updating the offending factories or editing your `test_data` database to
|
509
|
+
ensure they steer clear of one another. Care should be taken to preserve the
|
510
|
+
conceptual encapsulation of each test, however, as naively squashing errors
|
511
|
+
risks introducing inadvertent coupling between your factories and your
|
512
|
+
`test_data` data such that neither can be used independently of the other.
|
434
513
|
|
435
|
-
|
436
|
-
|
437
|
-
|
438
|
-
|
439
|
-
|
440
|
-
TestData.rollback(:after_truncate)
|
441
|
-
end
|
442
|
-
end
|
443
|
-
```
|
444
|
-
|
445
|
-
What this will do is complicated and counter-intuitive, but also fast and
|
446
|
-
reliable: [TestData.truncate](#testdatatruncate) will first ensure that your
|
447
|
-
`test_data` database is loaded inside a transaction, then will truncate that
|
448
|
-
data (set the `truncate_these_test_data_tables` [config option](#testdataconfig)
|
449
|
-
if necessary), and will finally create _yet another_ transaction save point
|
450
|
-
named `:after_data_truncate`. From that point onward, your test is free to
|
451
|
-
create all the factories it needs without fear of colliding with whatever you've
|
452
|
-
got stored in your `test_data` tables.
|
453
|
-
|
454
|
-
_[Why does this approach potentially load all the `test_data` data only to
|
455
|
-
immediately truncate it? Because it's actually much faster to truncate a large
|
456
|
-
data load in a live transaction, rollback the truncation, and then re-truncate
|
457
|
-
the data for a subsequent test than it would be to rollback the large data load
|
458
|
-
itself and re-load it for a subsequent test. It's silly but it works.]_
|
459
|
-
|
460
|
-
Hopefully one of these approaches, or some combination of them will get your
|
461
|
-
test suite passing after you've introduced `test_data`.
|
462
|
-
|
463
|
-
#### Separating your `test_data` and factory tests
|
464
|
-
|
465
|
-
Just because your tests _can_ access both your `factory_bot` factories and
|
466
|
-
`test_data` database doesn't mean they _should_.
|
467
|
-
|
468
|
-
Integration tests inevitably become coupled to the data that's available to
|
469
|
-
them, and if a test has access to both records created by a factory and a
|
470
|
-
`test_data` SQL dump, it is likely to unintentionally become inextricable from
|
471
|
-
both. This could result in the test having more ways to fail than necessary and
|
472
|
-
make it harder to simplify your test data strategy later. Instead, consider
|
473
|
-
explicitly opting into a single type of test data by separating your tests based
|
474
|
-
on which source of test data they use.
|
475
|
-
|
476
|
-
Every situation will be different, but one strategy that suits a lot of
|
477
|
-
circumstances would be to write a class method that runs at test-load time to
|
478
|
-
declare and configure the test data strategy for the current test.
|
479
|
-
|
480
|
-
Taking from [this
|
481
|
-
example](/example/test/integration/better_mode_switching_demo_test.rb) test, you
|
482
|
-
could implement a class method like this:
|
514
|
+
Another approach that the `test_data` gem provides is an additional mode with
|
515
|
+
`TestData.uses_clean_slate`, which—when called at the top of a factory-dependent
|
516
|
+
test—will ensure that the tables that `test_data` had written to will be
|
517
|
+
truncated, allowing the test to create whatever factories it needs without fear
|
518
|
+
of conflicts.
|
483
519
|
|
484
520
|
```ruby
|
485
|
-
class ActiveSupport::
|
486
|
-
|
487
|
-
|
488
|
-
|
489
|
-
require "factory_bot_rails"
|
490
|
-
include FactoryBot::Syntax::Methods
|
491
|
-
|
492
|
-
setup do
|
493
|
-
TestData.truncate
|
494
|
-
end
|
495
|
-
|
496
|
-
teardown do
|
497
|
-
TestData.rollback(:after_data_truncate)
|
498
|
-
end
|
499
|
-
when :test_data
|
500
|
-
setup do
|
501
|
-
TestData.load
|
502
|
-
end
|
503
|
-
|
504
|
-
teardown do
|
505
|
-
TestData.rollback
|
506
|
-
end
|
507
|
-
end
|
521
|
+
class AnExistingFactoryUsingTest < ActiveSupport::Testcase
|
522
|
+
setup do
|
523
|
+
TestData.uses_clean_slate
|
524
|
+
# pre-existing setup
|
508
525
|
end
|
526
|
+
# …
|
509
527
|
end
|
510
528
|
```
|
511
529
|
|
512
|
-
|
513
|
-
|
514
|
-
|
515
|
-
|
516
|
-
class SomeFactoryUsingTest < ActiveSupport::TestCase
|
517
|
-
test_data_mode :factory_bot
|
518
|
-
|
519
|
-
# … tests go here
|
520
|
-
end
|
521
|
-
|
522
|
-
class SomeTestDataUsingTest < ActionDispatch::IntegrationTest
|
523
|
-
test_data_mode :test_data
|
524
|
-
|
525
|
-
# etc.
|
526
|
-
end
|
527
|
-
```
|
528
|
-
|
529
|
-
By following an approach like this one, your `test_data` tests won't even see
|
530
|
-
your `create_*` factory methods and your `factory_bot` tests won't have access
|
531
|
-
to any of your `test_data`, either. From there, you can migrate tests onto
|
532
|
-
`test_data` incrementally, secure in the knowledge that you're not inadvertently
|
533
|
-
tangling your tests' dependency graph further.
|
534
|
-
|
535
|
-
#### Speeding up your test suite when using factories
|
536
|
-
|
537
|
-
##### Addressing redundant data cleanup
|
538
|
-
|
539
|
-
After adding `test_data` to your test suite, consider is how database cleanup
|
540
|
-
was being handled previously to make sure it isn't unnecessarily truncating
|
541
|
-
everything or resetting the transaction between tests. It's possible that your
|
542
|
-
suite is relying on Rails' built-in `use_transactional_tests` feature to wrap
|
543
|
-
your tests in always-rolled-back transactions, even if you're not using
|
544
|
-
fixtures. Or perhaps your suite uses
|
545
|
-
[database_cleaner](https://github.com/DatabaseCleaner/database_cleaner) to
|
546
|
-
truncate the database before or after each test. In either case, it's important
|
547
|
-
to know that by default [TestData.load](#testdataload) and
|
548
|
-
[TestData.rollback](#testdatarollback) will start and rollback a nested
|
549
|
-
transaction, respectively. That means—so long as they're called at the top of a
|
550
|
-
before-each hook and the end of an after-each hook—you might be able to disable
|
551
|
-
`use_transactional_tests` or remove your dependency on `database_cleaner` or any
|
552
|
-
other custom truncation logic you might have. Even if you get your suite running
|
553
|
-
immediately after adding `test_data`, it's still worth taking the time to
|
554
|
-
understand what's going on during test setup & teardown, because there may be an
|
555
|
-
opportunity to make your tests faster and more comprehensible by eliminating
|
556
|
-
redundant clean-up steps.
|
557
|
-
|
558
|
-
##### Avoiding truncate rollback churn
|
559
|
-
|
560
|
-
It's important to know that if your test suite has a mix of tests that call
|
561
|
-
[TestData.load](#testdataload) and tests that call
|
562
|
-
[TestData.truncate](#testdatatruncate), each time the test runner switches
|
563
|
-
between the two types, each call to `TestData.load` will cause the transaction
|
564
|
-
state to be rolled back from
|
565
|
-
[:after_data_truncate](#rolling-back-to-after-test-data-was-truncated) to
|
566
|
-
[:after_data_load](#rolling-back-to-after-the-data-was-loaded), only for the
|
567
|
-
next test to call `TestData.truncate` truncates all the tables again. In
|
568
|
-
practice, this shouldn't be too costly an operation, but if your test order is
|
569
|
-
randomized you might find that your build will run faster if you separate each
|
570
|
-
set of tests at runtime.
|
571
|
-
|
572
|
-
Separating your `test_data` and `factory_bot` tests is pretty trivial if you're
|
573
|
-
using RSpec, as the
|
574
|
-
[tag](https://relishapp.com/rspec/rspec-core/v/3-10/docs/command-line/tag-option)
|
575
|
-
feature was built with this sort of need in mind. Otherwise, you might consider
|
576
|
-
organizing the tests in different directories and running multiple commands to
|
577
|
-
execute them (e.g. `bin/rails test test/test_data_tests` and `bin/rails
|
578
|
-
test/factory_tests`). Every CI configuration is different, however, and you may
|
579
|
-
find yourself needing to get creative in configuring things to achieve the
|
580
|
-
fastest build time.
|
530
|
+
If you have a lot of tests, you can find a more sophisticated approaches for
|
531
|
+
logically switching between types of test data declaratively above in the
|
532
|
+
[getting started
|
533
|
+
section](#if-some-tests-rely-on-test-data-and-others-need-a-clean-slate)
|
581
534
|
|
582
535
|
### Using `test_data` with Rails fixtures
|
583
536
|
|
@@ -588,145 +541,53 @@ permanently committed to the test database actually makes them a little trickier
|
|
588
541
|
to work with. This section will cover a couple approaches for integrating
|
589
542
|
`test_data` into suites that use fixtures.
|
590
543
|
|
591
|
-
#### Getting your fixtures-dependent tests passing with `test_data`
|
592
|
-
|
593
544
|
It's more likely than not that all your tests will explode in dramatic fashion
|
594
|
-
as soon as you add `TestData.
|
595
|
-
fixtures will be loaded
|
596
|
-
|
597
|
-
|
598
|
-
|
599
|
-
|
600
|
-
|
601
|
-
|
602
|
-
|
603
|
-
|
604
|
-
|
605
|
-
Rails' built-in fixtures behavior with a monkey patch**. If that bold text
|
606
|
-
warning wasn't enough to scare you from reading on, here's how to do it.
|
607
|
-
|
608
|
-
_[Note that the following requires `use_transactional_data_loader` to be enabled
|
609
|
-
in your [config](#testdataconfig), because it depends on transaction
|
610
|
-
rollbacks.]_
|
611
|
-
|
612
|
-
Here's what you can do if you can't get your fixtures to play nicely with your
|
613
|
-
`test_data` dump:
|
614
|
-
|
615
|
-
1. Near the top of your test helper, call:
|
545
|
+
as soon as you add `TestData.uses_test_data` to a `setup` or `before(:each)`
|
546
|
+
hook. Typically, your fixtures will be loaded and committed immediately with
|
547
|
+
your `test_data` dump inserted afterward, which makes it exceedingly likely that
|
548
|
+
your tests will fail with primary key and unique constraint conflicts. If that's
|
549
|
+
the case you find yourself in, `test_data` provides an API that **overrides
|
550
|
+
Rails' built-in fixtures behavior with a monkey patch**.
|
551
|
+
|
552
|
+
And if that bold text wasn't enough to scare you off, here's how to do
|
553
|
+
it:
|
554
|
+
|
555
|
+
1. Before your tests have loaded (e.g. near the top of your test helper), call:
|
616
556
|
[TestData.prevent_rails_fixtures_from_loading_automatically!](#testdataprevent_rails_fixtures_from_loading_automatically)
|
617
|
-
This will
|
557
|
+
This will patch Rails'
|
618
558
|
[setup_fixtures](https://github.com/rails/rails/blob/main/activerecord/lib/active_record/test_fixtures.rb#L105)
|
619
|
-
into a no-op, which means that your test fixtures
|
620
|
-
loaded into your test database
|
559
|
+
and effectively render it into a no-op, which means that your test fixtures
|
560
|
+
will not be automatically loaded into your test database
|
621
561
|
|
622
|
-
2. In tests that rely on your `test_data` dump, call
|
623
|
-
TestData.
|
624
|
-
would. Because your fixtures won't be loaded automatically, they
|
625
|
-
available to these tests
|
562
|
+
2. In tests that rely on your `test_data` dump, call
|
563
|
+
[TestData.uses_test_data](#step-4-load-your-data-in-your-tests) as you
|
564
|
+
normally would. Because your fixtures won't be loaded automatically, they
|
565
|
+
won't be available to these tests
|
626
566
|
|
627
567
|
3. In tests that need fixtures, call
|
628
|
-
[TestData.
|
629
|
-
|
630
|
-
|
631
|
-
|
632
|
-
transaction-nesting) ensure your `test_data` dump is loaded in an initial
|
633
|
-
transaction, then ensure that it is truncated in a second transaction, before
|
634
|
-
loading your rails fixtures in a third transaction. These tests will have
|
635
|
-
access to all your fixture data without being tainted by any of your
|
636
|
-
`test_data` data
|
568
|
+
[TestData.uses_rails_fixtures(self)](#testdatauses_rails_fixtures) in a
|
569
|
+
before-each hook. This will first ensure that any tables written to by
|
570
|
+
`test_data` are truncated (as with `TestData.uses_clean_slate`) before
|
571
|
+
loading your Rails fixtures
|
637
572
|
|
638
573
|
For example, you might add the following to an existing fixtures-dependent
|
639
574
|
test to get it passing:
|
640
575
|
|
641
576
|
```ruby
|
642
577
|
class AnExistingFixtureUsingTest < ActiveSupport::Testcase
|
643
|
-
|
644
|
-
TestData.
|
578
|
+
setup do
|
579
|
+
TestData.uses_rails_fixtures(self)
|
645
580
|
# pre-existing setup
|
646
581
|
end
|
647
582
|
|
648
|
-
|
649
|
-
#… etc
|
650
|
-
end
|
651
|
-
|
652
|
-
def teardown
|
653
|
-
TestData.rollback(:after_load_rails_fixtures)
|
654
|
-
end
|
655
|
-
end
|
656
|
-
```
|
657
|
-
|
658
|
-
_[You don't need to worry about whether `TestData.load` has been called
|
659
|
-
previously, the loader will infer your intent and ensure that the transaction
|
660
|
-
state is correct before loading your fixtures.]_
|
661
|
-
|
662
|
-
#### Separating your `test_data` and fixture tests
|
663
|
-
|
664
|
-
*This only applies if you had to use
|
665
|
-
[TestData.load_rails_fixtures](#testdataload_rails_fixtures) as shown above.*
|
666
|
-
|
667
|
-
Just [like with factories](#separating-your-test_data-and-factory-tests), you
|
668
|
-
might benefit from a test helper to clearly declare whether a test uses fixtures
|
669
|
-
or `test_data` right at the top. Following the same pattern, you might do this:
|
670
|
-
|
671
|
-
```ruby
|
672
|
-
class ActiveSupport::TestCase
|
673
|
-
def self.test_data_mode(mode)
|
674
|
-
case mode
|
675
|
-
when :fixtures
|
676
|
-
fixtures :all
|
677
|
-
|
678
|
-
setup do
|
679
|
-
TestData.load_rails_fixtures
|
680
|
-
end
|
681
|
-
|
682
|
-
teardown do
|
683
|
-
TestData.rollback(:after_load_rails_fixtures)
|
684
|
-
end
|
685
|
-
when :test_data
|
686
|
-
setup do
|
687
|
-
TestData.load
|
688
|
-
end
|
689
|
-
|
690
|
-
teardown do
|
691
|
-
TestData.rollback
|
692
|
-
end
|
693
|
-
end
|
694
|
-
end
|
695
|
-
end
|
696
|
-
```
|
697
|
-
|
698
|
-
Which would allow you to simplify the above fixtures-using test to:
|
699
|
-
|
700
|
-
```ruby
|
701
|
-
class AnExistingFixtureUsingTest < ActiveSupport::Testcase
|
702
|
-
test_data_mode :fixtures
|
703
|
-
|
704
|
-
def test_stuff
|
705
|
-
#… etc
|
706
|
-
end
|
583
|
+
# …
|
707
584
|
end
|
708
585
|
```
|
709
586
|
|
710
|
-
|
711
|
-
|
712
|
-
|
713
|
-
|
714
|
-
runner randomly picks a `test_data` test after running a fixtures-dependent
|
715
|
-
test, it will roll back your fixtures and the truncation of your `test_data`,
|
716
|
-
only to re-truncate your `test_data` data and reload your fixtures for the next
|
717
|
-
test that happens to use fixtures. But unlike truncation alone, loading your
|
718
|
-
fixtures is a non-trivial operation that can chew up a some serious time as your
|
719
|
-
suite runs.
|
720
|
-
|
721
|
-
As a result, we strongly encourage breaking up your test suite to avoid this
|
722
|
-
churn, even if it means splitting your test run over multiple CLI commands. If
|
723
|
-
you're using the Rails test runner and Minitest, that likely means sequestering
|
724
|
-
one set of tests to one directory and the other to a different directory, as
|
725
|
-
there is no granular control over to how the runner randomizes suites. And for
|
726
|
-
RSpec,
|
727
|
-
[tagging](https://relishapp.com/rspec/rspec-core/v/3-10/docs/command-line/tag-option)
|
728
|
-
each spec and running separate commands for each tag could yield significant
|
729
|
-
performance improvements.
|
587
|
+
If you've adopted a mode-switching helper method [like the one described
|
588
|
+
above](#if-some-tests-rely-on-test-data-and-others-need-a-clean-slate), you
|
589
|
+
could of course add a third mode to cover any tests that depend on Rails
|
590
|
+
fixtures.
|
730
591
|
|
731
592
|
## Rake Task Reference
|
732
593
|
|
@@ -771,11 +632,9 @@ This task runs several generators:
|
|
771
632
|
have numerous secrets in this file's `development:` stanza, you may want to
|
772
633
|
alias and inherit it into `test_data:` like the `webpacker.yml` generator does
|
773
634
|
|
774
|
-
* `config/cable.yml` -
|
775
|
-
[ActionCable](https://guides.rubyonrails.org/action_cable_overview.html)
|
776
|
-
|
777
|
-
generator explicitly specifies `async` instead, since that's the default for
|
778
|
-
`development:`
|
635
|
+
* `config/cable.yml` - Simply defines a `test_data:` entry that tells
|
636
|
+
[ActionCable](https://guides.rubyonrails.org/action_cable_overview.html) to
|
637
|
+
use the `async` adapter, since that's also the default for `development`
|
779
638
|
|
780
639
|
### test_data:verify_config
|
781
640
|
|
@@ -792,8 +651,8 @@ your seed file. Specifically:
|
|
792
651
|
1. Creates the `test_data` environment's database, if it doesn't already exist
|
793
652
|
|
794
653
|
2. Ensures the database is non-empty to preserve data integrity (run
|
795
|
-
[test_data:drop_database](#test_datadrop_database) first if
|
796
|
-
|
654
|
+
[test_data:drop_database](#test_datadrop_database) first if you intend to
|
655
|
+
reinitialize it)
|
797
656
|
|
798
657
|
3. Checks to see if a dump of the database already exists (by default, stored in
|
799
658
|
`test/support/test_data/`)
|
@@ -804,11 +663,24 @@ your seed file. Specifically:
|
|
804
663
|
* Otherwise, it invokes the task `db:schema:load` and `db:seed` (similar to
|
805
664
|
Rails' built-in `db:setup` task)
|
806
665
|
|
666
|
+
### test_data:reinitialize
|
667
|
+
|
668
|
+
This task is designed for the situation where you may already have a `test_data`
|
669
|
+
database created and simply want to drop it and replace it with whatever dumps
|
670
|
+
are in the `test/support/test_data` directory.
|
671
|
+
|
672
|
+
Dropping the database requires confirmation, either interactively or by setting
|
673
|
+
the environment variable `TEST_DATA_CONFIRM`. It will additionally warn you in
|
674
|
+
the event that the local database appears to be newer than the dumps on disk
|
675
|
+
that would replace it. From there, this task behaves the same way as `rake
|
676
|
+
test_data:initialize`.
|
677
|
+
|
807
678
|
### test_data:dump
|
808
679
|
|
809
680
|
This task is designed to be run after you've created or updated your test data
|
810
|
-
and you're ready to run your tests against it. The
|
811
|
-
SQL dumps from your `test_data` environment's
|
681
|
+
in the `test_data` database and you're ready to run your tests against it. The
|
682
|
+
task creates several plain SQL dumps from your `test_data` environment's
|
683
|
+
database:
|
812
684
|
|
813
685
|
* A schema-only dump, by default in `test/support/test_data/schema.sql`
|
814
686
|
|
@@ -816,13 +688,13 @@ SQL dumps from your `test_data` environment's database:
|
|
816
688
|
`test/support/test_data/data.sql`
|
817
689
|
|
818
690
|
* A data-only dump of records that you *don't* want loaded in your tests in
|
819
|
-
`test/support/test_data/non_test_data.sql
|
691
|
+
`test/support/test_data/non_test_data.sql`. By default, this includes Rails'
|
820
692
|
internal tables: `ar_internal_metadata` and `schema_migrations`, configurable
|
821
|
-
with [TestData.config](#testdataconfig)'s `non_test_data_tables`
|
693
|
+
with [TestData.config](#testdataconfig)'s `non_test_data_tables`
|
822
694
|
|
823
695
|
Each of these files are designed to be committed and versioned with the rest of
|
824
696
|
your application. [TestData.config](#testdataconfig) includes several
|
825
|
-
options to control
|
697
|
+
options to control this task.
|
826
698
|
|
827
699
|
### test_data:load
|
828
700
|
|
@@ -839,7 +711,7 @@ This task will load your SQL dumps into your `test_data` database by:
|
|
839
711
|
4. Warning if there are pending migrations that haven't been run yet
|
840
712
|
|
841
713
|
If there are pending migrations, you'll probably want to run them and then
|
842
|
-
dump & commit your test data so that they're
|
714
|
+
dump & commit your test data so that they're up-to-date:
|
843
715
|
|
844
716
|
```
|
845
717
|
$ RAILS_ENV=test_data bin/rake db:migrate
|
@@ -852,275 +724,285 @@ This task will create the `test_data` environment's database if it does not
|
|
852
724
|
already exist. It also
|
853
725
|
[enhances](https://dev.to/molly/rake-task-enhance-method-explained-3bo0) Rails'
|
854
726
|
`db:create` task so that `test_data` is created along with `development` and
|
855
|
-
`test
|
727
|
+
`test` whenever `rake db:create` is run.
|
856
728
|
|
857
729
|
### test_data:drop_database
|
858
730
|
|
859
731
|
This task will drop the `test_data` environment's database if it exists. It also
|
860
732
|
enhances Rails' `db:drop` task so that `test_data` is dropped along with
|
861
|
-
`development` and `test
|
733
|
+
`development` and `test` whenever `rake db:drop` is run.
|
862
734
|
|
863
735
|
## API Reference
|
864
736
|
|
737
|
+
### TestData.uses_test_data
|
738
|
+
|
739
|
+
This is the method designed to be used by your tests to load your test data
|
740
|
+
into your `test` database so that your tests can rely on it. Typically, you'll
|
741
|
+
want to call it at the beginning of each test that relies on the test data
|
742
|
+
managed by this gem—most often, in a before-each hook.
|
743
|
+
|
744
|
+
For the sake of speed and integrity, `TestData.uses_test_data` is designed to
|
745
|
+
take advantage of nested transactions ([Postgres
|
746
|
+
savepoints](https://www.postgresql.org/docs/current/sql-savepoint.html)). By
|
747
|
+
default, data is loaded in a transaction and intended to be rolled back to the
|
748
|
+
point _immediately after_ the data was imported between tests. This way, your
|
749
|
+
test suite only pays the cost of importing the SQL file once, but each of your
|
750
|
+
tests can enjoy a clean slate that's free of data pollution from other tests.
|
751
|
+
(This is similar to, but separate from, Rails fixtures'
|
752
|
+
[use_transactional_tests](https://edgeguides.rubyonrails.org/testing.html#testing-parallel-transactions)
|
753
|
+
option.)
|
754
|
+
|
755
|
+
_See configuration option:
|
756
|
+
[config.after_test_data_load](#configafter_test_data_load)_
|
757
|
+
|
758
|
+
### TestData.uses_clean_slate
|
759
|
+
|
760
|
+
If a test does not rely on your `test_data` data, you can instead ensure that it
|
761
|
+
runs against empty tables by calling `TestData.uses_clean_slate`. Like
|
762
|
+
`TestData.uses_test_data`, this would normally be called at the beginning of
|
763
|
+
each such test in a before-each hook.
|
764
|
+
|
765
|
+
This method works by first ensuring that your test data is loaded (and the
|
766
|
+
correspondent savepoint created), then will truncate all affected tables and
|
767
|
+
create another savepoint. It's a little counter-intuitive that you'd first
|
768
|
+
litter your database with data only to wipe it clean again, but it's much faster
|
769
|
+
to repeatedly truncate tables than to repeatedly import large SQL files.
|
770
|
+
|
771
|
+
_See configuration options:
|
772
|
+
[config.after_test_data_truncate](#configafter_test_data_truncate),
|
773
|
+
[config.truncate_these_test_data_tables](#configtruncate_these_test_data_tables)_
|
774
|
+
|
775
|
+
### TestData.uses_rails_fixtures
|
776
|
+
|
777
|
+
As described in this README's [fixture interop
|
778
|
+
guide](#using-test_data-with-rails-fixtures), `TestData.uses_rails_fixtures`
|
779
|
+
will load your app's [Rails
|
780
|
+
fixtures](https://guides.rubyonrails.org/testing.html#the-low-down-on-fixtures)
|
781
|
+
by intercepting Rails' built-in fixture-loading code. As with the other "uses"
|
782
|
+
methods, you'll likely want to call it in a before-each hook before any test
|
783
|
+
that needs access to your Rails fixtures.
|
784
|
+
|
785
|
+
There are two additional things to keep in mind if using this method:
|
786
|
+
|
787
|
+
1. Using this feature requires that you've first invoked
|
788
|
+
[TestData.prevent_rails_fixtures_from_loading_automatically!](#testdataprevent_rails_fixtures_from_loading_automatically)
|
789
|
+
before your tests have started running to override Rails' default behavior
|
790
|
+
before any of your tests have loaded or started running
|
791
|
+
|
792
|
+
2. Because the method depends on Rails' fixture caching mechanism, it must be
|
793
|
+
passed an instance of the running test class (e.g.
|
794
|
+
`TestData.uses_rails_fixtures(self)`)
|
795
|
+
|
796
|
+
Under the hood, this method effectively ensures a clean slate the same way
|
797
|
+
`TestData.uses_clean_slate` does, except that after creating the truncation
|
798
|
+
savepoint, it will then load your fixtures and finally create—wait for it—yet
|
799
|
+
another savepoint that subsequent calls to `uses_rails_fixtures` can rollback
|
800
|
+
to.
|
801
|
+
|
802
|
+
_See configuration option:
|
803
|
+
[config.after_rails_fixture_load](#configafter_rails_fixture_load)_
|
804
|
+
|
805
|
+
#### TestData.prevent_rails_fixtures_from_loading_automatically!
|
806
|
+
|
807
|
+
Call this method before any tests have been loaded or executed by your test
|
808
|
+
runner if you're planning to use
|
809
|
+
[TestData.uses_rails_fixtures](#testdatauses_rails_fixtures) to load Rails
|
810
|
+
fixtures into any of your tests. This method will disable the default behavior
|
811
|
+
of loading your Rails fixtures into the test database as soon as the first test
|
812
|
+
case with fixtures enabled is executed. (Inspect the [source for the
|
813
|
+
patch](/lib/test_data/active_record_ext.rb) to make sure you're comfortable with
|
814
|
+
what it's doing.)
|
815
|
+
|
865
816
|
### TestData.config
|
866
817
|
|
867
818
|
The generated `config/initializers/test_data.rb` initializer will include a call
|
868
819
|
to `TestData.config`, which takes a block that yields a mutable configuration
|
869
|
-
object (similar to `Rails.application.config`)
|
820
|
+
object (similar to `Rails.application.config`). If anything is unclear after
|
821
|
+
reading the documentation, feel free to review the
|
822
|
+
[initializer](lib/generators/test_data/initializer_generator.rb) and the [Config
|
823
|
+
class](/lib/test_data/config.rb) themselves.
|
824
|
+
|
825
|
+
#### Lifecycle hooks
|
826
|
+
|
827
|
+
Want to shift forward several timestamp fields after your `test_data` SQL dumps
|
828
|
+
are loaded into your test database? Need to refresh a materialized view after
|
829
|
+
your Rails fixtures are loaded? You _could_ do these things after calling
|
830
|
+
`TestData.uses_test_data` and `TestData.uses_rails_fixtures`, respectively, but
|
831
|
+
you'd take the corresponding performance hit in each and every test.
|
832
|
+
|
833
|
+
Instead, you can pass a callable or a block and `test_data` will execute it just
|
834
|
+
_after_ performing the associated data operation but just _before_ creating a
|
835
|
+
transaction savepoint. That way, whenever the gem rolls back between tests, your
|
836
|
+
hook won't need to be run again.
|
837
|
+
|
838
|
+
##### config.after_test_data_load
|
839
|
+
|
840
|
+
This is hook is run immediately after `TestData.uses_test_data` has loaded your
|
841
|
+
SQL dumps into the `test` database, but before creating a savepoint. Takes a
|
842
|
+
block or anything that responds to `call`.
|
843
|
+
|
870
844
|
|
871
845
|
```ruby
|
872
846
|
TestData.config do |config|
|
873
|
-
#
|
874
|
-
|
875
|
-
|
876
|
-
|
877
|
-
|
847
|
+
# Example: roll time forward
|
848
|
+
config.after_test_data_load do
|
849
|
+
Boop.connection.exec_update(<<~SQL, nil, [[nil, Time.zone.now - System.epoch]])
|
850
|
+
update boops set booped_at = booped_at + $1
|
851
|
+
SQL
|
852
|
+
end
|
853
|
+
end
|
854
|
+
```
|
878
855
|
|
879
|
-
|
880
|
-
# config.non_test_data_dump_path = "test/support/test_data/non_test_data.sql"
|
856
|
+
##### config.after_test_data_truncate
|
881
857
|
|
882
|
-
|
883
|
-
|
884
|
-
|
858
|
+
This is hook is run immediately after `TestData.uses_clean_slate` has truncated
|
859
|
+
your test data, but before creating a savepoint. Takes a block or anything that
|
860
|
+
responds to `call`.
|
885
861
|
|
886
|
-
|
887
|
-
|
862
|
+
```ruby
|
863
|
+
TestData.config do |config|
|
864
|
+
# Example: pass a callable instead of a block
|
865
|
+
config.after_test_data_truncate(SomethingThatRespondsToCall.new)
|
866
|
+
end
|
867
|
+
```
|
888
868
|
|
889
|
-
|
890
|
-
# If left as `nil`, all tables inserted into by the SQL file at
|
891
|
-
# `data_dump_path` will be truncated
|
892
|
-
# config.truncate_these_test_data_tables = nil
|
869
|
+
##### config.after_rails_fixture_load
|
893
870
|
|
894
|
-
|
895
|
-
|
896
|
-
|
897
|
-
# being used
|
898
|
-
# config.use_transactional_data_loader = true
|
871
|
+
This is hook is run immediately after `TestData.uses_rails_fixtures` has loaded
|
872
|
+
your Rails fixtures into the `test` database, but before creating a savepoint.
|
873
|
+
Takes a block or anything that responds to `call`.
|
899
874
|
|
900
|
-
|
901
|
-
|
902
|
-
#
|
875
|
+
```ruby
|
876
|
+
TestData.config do |config|
|
877
|
+
# Example: refresh Postgres assets like materialized views
|
878
|
+
config.after_rails_fixture_load do
|
879
|
+
RefreshesMaterializedViews.new.call
|
880
|
+
end
|
903
881
|
end
|
904
882
|
```
|
905
883
|
|
906
|
-
|
884
|
+
#### test_data:dump options
|
907
885
|
|
908
|
-
|
909
|
-
|
886
|
+
The gem provides several options governing the behavior of the
|
887
|
+
[test_data:dump](#test_datadump) Rake task. You probably won't need to set these
|
888
|
+
unless you run into a problem with the defaults.
|
910
889
|
|
911
|
-
|
890
|
+
##### config.non_test_data_tables
|
912
891
|
|
913
|
-
|
914
|
-
|
915
|
-
|
916
|
-
|
917
|
-
|
918
|
-
|
919
|
-
|
920
|
-
|
921
|
-
|
922
|
-
|
892
|
+
Your application may have some tables that are necessary for the operation of
|
893
|
+
the application, but irrelevant or incompatible with you your tests. This data
|
894
|
+
is still dumped for the sake of being able to restore the database with [rake
|
895
|
+
test_data:load](#test_dataload), but will not be loaded when your tests are
|
896
|
+
running. Defaults to `[]`, (but will always include `ar_internal_metadata` and
|
897
|
+
`schema_migrations`).
|
898
|
+
|
899
|
+
```ruby
|
900
|
+
TestData.config do |config|
|
901
|
+
config.non_test_data_tables = []
|
902
|
+
end
|
903
|
+
```
|
923
904
|
|
924
|
-
|
925
|
-
|
926
|
-
|
927
|
-
|
928
|
-
|
929
|
-
|
930
|
-
`test/support/test_data/data.sql`) to insert your test data
|
931
|
-
3. Creates the `:after_data_load` savepoint
|
932
|
-
|
933
|
-
If the method is called and the `:after_data_load` savepoint is already active
|
934
|
-
(indicating that the data is loaded), the method rolls back to
|
935
|
-
`:after_data_load`, inferring that the user's intention is to have a clean load
|
936
|
-
of the test data.
|
937
|
-
|
938
|
-
As an additional safeguard, in case a rollback is triggered unexpectedly (i.e.
|
939
|
-
calling `rollback_transaction` on `ActiveRecord::Base.connection` instead of via
|
940
|
-
`TestData.rollback`), `test_data` writes a memo indicating that the data is
|
941
|
-
loaded in `ar_internal_metadata`. `TestData.load` uses this memo to detect this
|
942
|
-
issue and will recreate the `:after_data_load` savepoint rather than attempt to
|
943
|
-
erroneously reload your SQL data dump. (Similar error-handling is built-into
|
944
|
-
[TestData.truncate](#testdatatruncate) and
|
945
|
-
[TestData.load_rails_fixtures](#testdataload_rails_fixtures), as well.)
|
946
|
-
|
947
|
-
#### Loading without transactions
|
948
|
-
|
949
|
-
For most cases, we strongly recommend using the default transactional testing
|
950
|
-
strategy, both because it's faster and because it reduces the risk of test
|
951
|
-
pollution. However, you may need to commit your test data if the data needs to
|
952
|
-
be loaded by multiple processes or over multiple connections.
|
953
|
-
|
954
|
-
If you need to load the test data and commit it to the database, simply set
|
955
|
-
`TestData.config.use_transactional_data_loader = false`.
|
956
|
-
|
957
|
-
If transactions are disabled, you'll need to decide whether and how to clear the
|
958
|
-
data out after each test. Many folks use
|
959
|
-
[database_cleaner](https://github.com/DatabaseCleaner/database_cleaner) for
|
960
|
-
this, while `test_data` offers a rudimentary
|
961
|
-
[TestData.truncate](https://github.com/testdouble/test_data#testdatatruncate)
|
962
|
-
method that may be sufficient for your needs.
|
963
|
-
|
964
|
-
You might imagine something like this if you were loading the data just once for
|
965
|
-
the full run of a test suite:
|
905
|
+
##### config.dont_dump_these_tables
|
906
|
+
|
907
|
+
Some tables populated by your application may not be necessary to either its
|
908
|
+
proper functioning or useful to your tests (e.g. audit logs), so you can save
|
909
|
+
time and storage by preventing those tables from being dumped entirely. Defaults
|
910
|
+
to `[]`.
|
966
911
|
|
967
912
|
```ruby
|
968
|
-
|
969
|
-
config.
|
970
|
-
|
971
|
-
|
913
|
+
TestData.config do |config|
|
914
|
+
config.dont_dump_these_tables = []
|
915
|
+
end
|
916
|
+
```
|
972
917
|
|
973
|
-
|
974
|
-
|
975
|
-
|
918
|
+
##### config.schema_dump_path
|
919
|
+
|
920
|
+
The path to which the schema DDL of your `test_data` database will be written.
|
921
|
+
This is only used by [rake test_data:load](#test_dataload) when initializing the
|
922
|
+
`test_data` database. Defaults to `"test/support/test_data/schema.sql"`.
|
923
|
+
|
924
|
+
```ruby
|
925
|
+
TestData.config do |config|
|
926
|
+
config.schema_dump_path = "test/support/test_data/schema.sql"
|
976
927
|
end
|
977
928
|
```
|
978
929
|
|
979
|
-
|
980
|
-
|
981
|
-
|
982
|
-
|
983
|
-
|
984
|
-
### TestData.rollback
|
985
|
-
|
986
|
-
Because the gem loads your data in a transaction, it makes it easy to rollback
|
987
|
-
to any of its defined savepoints. In most cases you'll want to roll back to
|
988
|
-
`:after_data_load` after each test, and that's what `TestData.rollback` will do
|
989
|
-
when called without an argument. If the specified savepoint isn't active,
|
990
|
-
calling `rollback` is a no-op.
|
991
|
-
|
992
|
-
The gem may create up to four nested savepoints in a single transaction, and
|
993
|
-
this method allows you to rollback to any of them. They form the following
|
994
|
-
stack:
|
995
|
-
|
996
|
-
* `:before_data_load` - Taken before loading your `test_data` dump
|
997
|
-
* `:after_data_load` - Taken after loading your `test_data` dump
|
998
|
-
* `:after_truncate` - Taken after your `test_data` is truncated
|
999
|
-
* `:after_load_rails_fixtures` - Taken after Rails fixtures are loaded via
|
1000
|
-
[TestData.load_rails_fixtures](#testdataload_rails_fixtures)
|
1001
|
-
|
1002
|
-
More details on rolling back to each of the gem's savepoints follows below.
|
1003
|
-
|
1004
|
-
#### Rolling back to before test data was loaded
|
1005
|
-
|
1006
|
-
If some tests rely on data loaded by `TestData.load` and you have other tests
|
1007
|
-
that depend on that data _not being there_, you probably want to call
|
1008
|
-
[TestData.truncate](#testdatatruncate). But if that won't work for your needs,
|
1009
|
-
you can rewind to the moment just before your test data was loaded by calling
|
1010
|
-
`TestData.rollback(:before_data_load)`.
|
1011
|
-
|
1012
|
-
**⚠️ Warning⚠️** Repeatedly loading and rolling back to `:before_data_load` is
|
1013
|
-
expensive! If your test suite calls `TestData.rollback(:before_data_load)`
|
1014
|
-
multiple times, it's likely you're re-loading your (possibly large) SQL file of
|
1015
|
-
test data many more times than is necessary. Consider using
|
1016
|
-
[TestData.truncate](#testdatatruncate) to achieve the same goal with faster
|
1017
|
-
performance. Failing that, it might be preferable to partition your test suite
|
1018
|
-
so that similar tests are run in separate groups (as opposed to in a fully
|
1019
|
-
random or arbitrary order) to avoid repeatedly thrashing between rollbacks and
|
1020
|
-
reloads. This partitioning could be accomplished by either configuring your test
|
1021
|
-
runner or by running separate test commands for each group of tests.
|
1022
|
-
|
1023
|
-
#### Rolling back to after the data was loaded
|
1024
|
-
|
1025
|
-
This is the way you're likely to call this method most often.
|
1026
|
-
|
1027
|
-
When `TestData.rollback` is passed no arguments or called more explicitly as
|
1028
|
-
`TestData.rollback(:after_data_load)`, the method will rollback to the
|
1029
|
-
`:after_data_load` transaction savepoint taken immediately after the SQL dump
|
1030
|
-
was loaded. As a result, it is intended to be run after each test (e.g. in an
|
1031
|
-
`after(:each)` or `teardown`), to undo any changes made by the test.
|
1032
|
-
|
1033
|
-
#### Rolling back to after test data was truncated
|
1034
|
-
|
1035
|
-
If some of your tests call [TestData.truncate](#testdatatruncate) to clear out
|
1036
|
-
your test data after it's been loaded (as
|
1037
|
-
[described](#getting-your-factory-tests-passing-after-adding-test_data) when
|
1038
|
-
using `test_data` in conjunction with `factory_bot`), then you will likely want
|
1039
|
-
to run `TestData.rollback(:after_data_truncate)` after each of them. This will
|
1040
|
-
rewind your test database's state to when those tables were first
|
1041
|
-
truncated—effectively re-cleaning the slate for the next test.
|
1042
|
-
|
1043
|
-
#### Rolling back to after Rails fixtures were loaded
|
1044
|
-
|
1045
|
-
If you're using [TestData.load_rails_fixtures](#testdataload_rails_fixtures) in
|
1046
|
-
your test's before-each hook, you'll probably want to teardown that test by
|
1047
|
-
rolling back with `TestData.rollback(:after_load_rails_fixtures)` in an
|
1048
|
-
after-each hook, which will rewind to the point just after your Rails fixtures
|
1049
|
-
were loaded.
|
1050
|
-
|
1051
|
-
### TestData.truncate
|
1052
|
-
|
1053
|
-
Do you have some tests that _shouldn't_ access your test data? Or did some
|
1054
|
-
existing tests started failing after `test_data` was added? If you want to clear
|
1055
|
-
the state of your `test` database to support these tests, you can accomplish
|
1056
|
-
this with `TestData.truncate`. It truncates all the tables that `TestData.load`
|
1057
|
-
inserted into and then creates a savepoint named `:after_data_truncate`.
|
1058
|
-
|
1059
|
-
Most often, you'll want to call `TestData.truncate` before each test that
|
1060
|
-
should _not_ have access to your test data created with this gem. After each
|
1061
|
-
such test, it can clean up by calling `TestData.rollback(:after_data_truncate)`:
|
930
|
+
##### config.data_dump_path
|
931
|
+
|
932
|
+
The path that the SQL dump of your test data will be written. This is the dump
|
933
|
+
that will be executed by `TestData.uses_test_data` in your tests. Defaults to
|
934
|
+
`"test/support/test_data/data.sql"`.
|
1062
935
|
|
1063
936
|
```ruby
|
1064
|
-
|
1065
|
-
|
1066
|
-
|
1067
|
-
|
937
|
+
TestData.config do |config|
|
938
|
+
config.data_dump_path = "test/support/test_data/data.sql"
|
939
|
+
end
|
940
|
+
```
|
1068
941
|
|
1069
|
-
|
1070
|
-
|
1071
|
-
|
942
|
+
##### config.non_test_data_dump_path
|
943
|
+
|
944
|
+
The path to which the [non_test_data_tables](#confignon_test_data_tables) in
|
945
|
+
your `test_data` database will be written. This is only used by [rake
|
946
|
+
test_data:load](#test_dataload) when initializing the `test_data` database.
|
947
|
+
Defaults to `"test/support/test_data/non_test_data.sql"`.
|
948
|
+
|
949
|
+
```ruby
|
950
|
+
TestData.config do |config|
|
951
|
+
config.non_test_data_dump_path = "test/support/test_data/non_test_data.sql"
|
1072
952
|
end
|
1073
953
|
```
|
1074
954
|
|
1075
|
-
|
1076
|
-
test data SQL dump will be truncated (and cascading to any tables with foreign
|
1077
|
-
keys pointing to those tables), but you can also explicitly specify which tables
|
1078
|
-
should be truncated yourself by setting the `truncate_these_test_data_tables`
|
1079
|
-
property on [TestData.config](#testdataconfig) to an array of table names.
|
955
|
+
#### Other configuration options
|
1080
956
|
|
1081
|
-
|
957
|
+
##### config.truncate_these_test_data_tables
|
1082
958
|
|
1083
|
-
|
1084
|
-
|
1085
|
-
|
959
|
+
By default, when [TestData.uses_clean_slate](#testdatauses_clean_slate) is
|
960
|
+
called, it will truncate any tables for which an `INSERT` operation was
|
961
|
+
detected in your test data SQL dump. This may not be suitable for every case,
|
962
|
+
however, so this option allows you to specify which tables are truncated.
|
963
|
+
Defaults to `nil`.
|
1086
964
|
|
1087
|
-
|
965
|
+
```ruby
|
966
|
+
TestData.config do |config|
|
967
|
+
config.truncate_these_test_data_tables = []
|
968
|
+
end
|
969
|
+
```
|
1088
970
|
|
1089
|
-
|
1090
|
-
runner if you're planning to use
|
1091
|
-
[TestData.load_rails_fixtures](#testdataload_rails_fixtures) to load Rails
|
1092
|
-
fixtures into any of your tests. This method will disable the default behavior
|
1093
|
-
of loading your Rails fixtures into the test database as soon as the first test
|
1094
|
-
case with fixtures enabled is executed. (Inspect the [source for the
|
1095
|
-
patch](/lib/test_data/active_record_ext.rb) to make sure you're comfortable with
|
1096
|
-
what it's doing.)
|
971
|
+
##### config.log_level
|
1097
972
|
|
1098
|
-
|
973
|
+
The gem outputs its messages to standard output and error by assigning a log
|
974
|
+
level to each message. Valid values are `:debug`, `:info`, `:warn`, `:error`,
|
975
|
+
`:quiet`. Defaults to `:info`.
|
1099
976
|
|
1100
|
-
|
1101
|
-
|
1102
|
-
|
1103
|
-
|
1104
|
-
|
977
|
+
```ruby
|
978
|
+
TestData.config do |config|
|
979
|
+
config.log_level = :info
|
980
|
+
end
|
981
|
+
```
|
1105
982
|
|
1106
|
-
|
983
|
+
### TestData.insert_test_data_dump
|
1107
984
|
|
1108
|
-
|
1109
|
-
|
1110
|
-
|
1111
|
-
|
985
|
+
If you just want to insert the test data in your application's SQL dumps without
|
986
|
+
any of the transaction management or test runner assumptions inherent in
|
987
|
+
[TestData.uses_test_data](#testdatauses_test_data), then you can call
|
988
|
+
`TestData.insert_test_data_dump` to load and execute the dump.
|
1112
989
|
|
1113
|
-
|
990
|
+
This might be necessary in a few different situations:
|
1114
991
|
|
1115
|
-
|
1116
|
-
|
1117
|
-
|
1118
|
-
|
1119
|
-
|
992
|
+
* Running tests in environments that can't be isolated to a single database
|
993
|
+
transaction (e.g. orchestrating tests across multiple databases, processes,
|
994
|
+
etc.)
|
995
|
+
* You might ant to use your test data to seed pre-production environments with
|
996
|
+
enough data to exploratory test (as you might do in a `postdeploy` script with
|
997
|
+
your [Heroku Review
|
998
|
+
Apps](https://devcenter.heroku.com/articles/github-integration-review-apps))
|
999
|
+
* Your tests require complex heterogeneous sources of data that aren't a good
|
1000
|
+
fit for the assumptions and constraints of this library's default methods for
|
1001
|
+
preparing test data
|
1120
1002
|
|
1121
|
-
|
1122
|
-
transaction
|
1123
|
-
|
1003
|
+
In any case, since `TestData.insert_test_data_dump` is not wrapped in a
|
1004
|
+
transaction, when used for automated tests, data cleanup becomes your
|
1005
|
+
responsibility.
|
1124
1006
|
|
1125
1007
|
## Assumptions
|
1126
1008
|
|
@@ -1148,12 +1030,11 @@ yet. Here are some existing assumptions and limitations:
|
|
1148
1030
|
|
1149
1031
|
If you use `factory_bot` and all of these are true:
|
1150
1032
|
|
1151
|
-
* Your integration tests are super fast and not getting significantly slower
|
1033
|
+
* Your integration tests are super fast and are not getting significantly slower
|
1152
1034
|
over time
|
1153
1035
|
|
1154
|
-
*
|
1155
|
-
|
1156
|
-
each of those tests be analyzed & updated to get them passing again
|
1036
|
+
* Minor changes to existing factories rarely result in test failures that
|
1037
|
+
require unrelated tests to be read & updated to get them passing again
|
1157
1038
|
|
1158
1039
|
* The number of associated records generated between your most-used factories
|
1159
1040
|
are representative of production data, as opposed to generating a sprawling
|
@@ -1169,16 +1050,16 @@ If you use `factory_bot` and all of these are true:
|
|
1169
1050
|
confidence-eroding nested factories with names like `:user`, `:basic_user`,
|
1170
1051
|
`:lite_user`, and `:plain_user_no_associations_allowed`
|
1171
1052
|
|
1172
|
-
If none of these things are true, then congratulations! You are using
|
1173
|
-
`factory_bot`
|
1053
|
+
If none of these things are true, then congratulations! You are probably using
|
1054
|
+
`factory_bot` to great effect! Unfortunately, in our experience, this outcome
|
1174
1055
|
is exceedingly rare, especially for large and long-lived applications.
|
1175
1056
|
|
1176
|
-
However, if any of the above
|
1177
|
-
|
1178
|
-
|
1179
|
-
|
1180
|
-
|
1181
|
-
|
1057
|
+
However, if you'd answer "no" to any of the above questions, just know that
|
1058
|
+
these are the sorts of failure modes the `test_data` gem was designed to
|
1059
|
+
avoid—and we hope you'll consider trying it with an open mind. At the same time,
|
1060
|
+
we acknowledge that large test suites can't be rewritten and migrated to a
|
1061
|
+
different source of test data overnight—nor should they be! See our notes on
|
1062
|
+
[migrating to `test_data`
|
1182
1063
|
incrementally](#factory--fixture-interoperability-guide)
|
1183
1064
|
|
1184
1065
|
### How will I handle merge conflicts in these SQL files if I have lots of people working on lots of feature branches all adding to the `test_data` database dumps?
|
@@ -1198,13 +1079,13 @@ this risk. The reason that the dumps are stored as plain SQL (aside from the
|
|
1198
1079
|
fact that git's text compression is very good) is to make merge conflicts with
|
1199
1080
|
other branches feasible, if not entirely painless.
|
1200
1081
|
|
1201
|
-
However, if your app is in the very initial stages of development
|
1202
|
-
making breaking changes to your schema very frequently, our
|
1203
|
-
hold off a bit on writing _any_ integration tests that depend
|
1204
|
-
of test data, as they'll be more likely
|
1205
|
-
iterate than detect bugs. Once you you have
|
1206
|
-
end-to-end, that's a good moment to start
|
1207
|
-
pulling in a
|
1082
|
+
However, if your app is in the very initial stages of development or you're
|
1083
|
+
otherwise making breaking changes to your schema and data very frequently, our
|
1084
|
+
best advice is to hold off a bit on writing _any_ integration tests that depend
|
1085
|
+
on shared sources of test data (regardless of tool), as they'll be more likely
|
1086
|
+
to frustrate your ability to rapidly iterate than detect bugs. Once you you have
|
1087
|
+
a reasonably stable feature working end-to-end, that's a good moment to start
|
1088
|
+
adding integration tests—and perhaps pulling in a gem like this one to help you.
|
1208
1089
|
|
1209
1090
|
### Why can't I save multiple database dumps to cover different scenarios?
|
1210
1091
|
|
@@ -1220,8 +1101,7 @@ By having a single `test_data` database that grows up with your application just
|
|
1220
1101
|
like `production` does—with both having their schemas and data migrated
|
1221
1102
|
incrementally over time—your integration tests that depend on `test_data` will
|
1222
1103
|
have an early opportunity to catch bugs that otherwise wouldn't be found until
|
1223
|
-
they were deployed into a long-lived
|
1224
|
-
production itself.
|
1104
|
+
they were deployed into a long-lived staging or (gasp!) production environment.
|
1225
1105
|
|
1226
1106
|
### Are you sure I should commit these SQL dumps? They're way too big!
|
1227
1107
|
|
@@ -1232,16 +1112,17 @@ cause:
|
|
1232
1112
|
resetting (or rolling back) your changes and making another attempt at
|
1233
1113
|
generating a more minimal set of test data
|
1234
1114
|
|
1235
|
-
2. If
|
1236
|
-
tests
|
1115
|
+
2. If some records persisted by your application aren't very relevant to your
|
1116
|
+
tests, you might consider either of these options:
|
1237
1117
|
|
1238
|
-
*
|
1239
|
-
|
1118
|
+
* If certain tables are necessary for running the app but aren't needed by
|
1119
|
+
your tests, you can add them to the `config.non_test_data_tables`
|
1120
|
+
configuration array. They'll still be committed to git, but won't loaded
|
1121
|
+
by your tests
|
1240
1122
|
|
1241
|
-
*
|
1242
|
-
`config.dont_dump_these_tables` array
|
1243
|
-
won't be
|
1244
|
-
so if the data is needed for the app to operate, you'll need to dump them)
|
1123
|
+
* If the certain tables are not needed by your application or by your tests
|
1124
|
+
(e.g. audit logs), add them to the `config.dont_dump_these_tables` array,
|
1125
|
+
and they won't be persisted by `rake test_data:dump`
|
1245
1126
|
|
1246
1127
|
3. If the dumps are _necessarily_ really big (some apps are complex!), consider
|
1247
1128
|
looking into [git-lfs](https://git-lfs.github.com) for tracking them without
|
@@ -1264,7 +1145,7 @@ test data loaded from this gem or any other:
|
|
1264
1145
|
def test_exclude_cancelled_orders
|
1265
1146
|
good_order = Order.new
|
1266
1147
|
bad_order = Order.new(cancelled: true)
|
1267
|
-
user = User.create!(orders: good_order, bad_order)
|
1148
|
+
user = User.create!(orders: [good_order, bad_order])
|
1268
1149
|
|
1269
1150
|
result = user.active_orders
|
1270
1151
|
|
@@ -1273,17 +1154,18 @@ def test_exclude_cancelled_orders
|
|
1273
1154
|
end
|
1274
1155
|
```
|
1275
1156
|
|
1276
|
-
This test is simple, self-contained, clearly
|
1277
|
-
[arrange-act-assert](https://github.com/testdouble/contributing-tests/wiki/Arrange-Act-Assert)
|
1278
|
-
and (most importantly) will only fail if the functionality stops
|
1279
|
-
Maximizing the number of tests that can be written expressively and
|
1280
|
-
without the aid of shared test data is a laudable goal that more
|
1281
|
-
embrace.
|
1157
|
+
This test is simple, self-contained, clearly demarcates the
|
1158
|
+
[arrange-act-assert](https://github.com/testdouble/contributing-tests/wiki/Arrange-Act-Assert)
|
1159
|
+
phases, and (most importantly) will only fail if the functionality stops
|
1160
|
+
working. Maximizing the number of tests that can be written expressively and
|
1161
|
+
succinctly without the aid of shared test data is a laudable goal that more
|
1162
|
+
teams should embrace.
|
1282
1163
|
|
1283
1164
|
However, what if the code you're writing doesn't need 3 records in the database,
|
1284
|
-
but 30? Writing that much test setup would be painstaking
|
1285
|
-
fully-encapsulated
|
1286
|
-
|
1165
|
+
but 30? Writing that much test setup would be painstaking, despite being
|
1166
|
+
fully-encapsulated. Long test setup is harder for others to read and understand.
|
1167
|
+
And because that setup depends on more of your system's code, it will have more
|
1168
|
+
reasons to break as your codebase changes. At that point, you have two options:
|
1287
1169
|
|
1288
1170
|
1. Critically validate your design: why is it so hard to set up? Does it
|
1289
1171
|
_really_ require so much persisted data to exercise this behavior? Would a
|
@@ -1294,8 +1176,8 @@ point, you have two options:
|
|
1294
1176
|
[subject](https://github.com/testdouble/contributing-tests/wiki/Subject)
|
1295
1177
|
instead of loading everything from the database? When automated testing is
|
1296
1178
|
saved for the very end of a feature's development, it can feel too costly to
|
1297
|
-
reexamine design decisions like this, but it
|
1298
|
-
same. *Easy to test code is easy to use code*
|
1179
|
+
reexamine design decisions like this, but it can be valuable to consider all
|
1180
|
+
the same. *Easy to test code is easy to use code*
|
1299
1181
|
|
1300
1182
|
2. If the complex setup is a necessary reality of the situation that your app
|
1301
1183
|
needs to handle (and it often will be!), then having _some_ kind of shared
|
@@ -1304,16 +1186,17 @@ point, you have two options:
|
|
1304
1186
|
|
1305
1187
|
As a result, there is no one-size-fits-all approach. Straightforward behavior
|
1306
1188
|
that can be invoked with a clear, concise test has no reason to be coupled to a
|
1307
|
-
shared source of test data.
|
1308
|
-
carefully-arranged data
|
1309
|
-
|
1189
|
+
shared source of test data. Meanwhile, tests of more complex behaviors that
|
1190
|
+
require lots of carefully-arranged data might be unmaintainable without a shared
|
1191
|
+
source of test data to lean on. So both kinds of test clearly have their place.
|
1310
1192
|
|
1311
1193
|
But this is a pretty nuanced discussion that can be hard to keep in mind when
|
1312
1194
|
under deadline pressure or on a large team where building consensus around norms
|
1313
1195
|
is challenging. As a result, leaving the decision of which type of test to write
|
1314
1196
|
to spur-of-the-moment judgment is likely to result in inconsistent test design.
|
1315
1197
|
Instead, you might consider separating these two categories into separate test
|
1316
|
-
types or suites
|
1198
|
+
types or suites, with simple heuristics to determine which types of code demand
|
1199
|
+
which type of test.
|
1317
1200
|
|
1318
1201
|
For example, it would be completely reasonable to load this gem's test data for
|
1319
1202
|
integration tests, but not for basic tests of models, like so:
|
@@ -1321,21 +1204,13 @@ integration tests, but not for basic tests of models, like so:
|
|
1321
1204
|
```ruby
|
1322
1205
|
class ActionDispatch::IntegrationTest
|
1323
1206
|
setup do
|
1324
|
-
TestData.
|
1325
|
-
end
|
1326
|
-
|
1327
|
-
teardown do
|
1328
|
-
TestData.rollback
|
1207
|
+
TestData.uses_test_data
|
1329
1208
|
end
|
1330
1209
|
end
|
1331
1210
|
|
1332
1211
|
class ActiveSupport::TestCase
|
1333
1212
|
setup do
|
1334
|
-
TestData.
|
1335
|
-
end
|
1336
|
-
|
1337
|
-
teardown do
|
1338
|
-
TestData.rollback(:after_data_truncate)
|
1213
|
+
TestData.uses_clean_slate
|
1339
1214
|
end
|
1340
1215
|
end
|
1341
1216
|
```
|
@@ -1344,6 +1219,120 @@ In short, this skepticism is generally healthy, and encapsulated tests that
|
|
1344
1219
|
forego reliance on shared sources of test data should be maximized. For
|
1345
1220
|
everything else, there's `test_data`.
|
1346
1221
|
|
1222
|
+
### I'm worried my tests aren't as fast as they should be
|
1223
|
+
|
1224
|
+
The `test_data` gem was written to enable tests that are not only more
|
1225
|
+
comprehensible and maintainable over the long-term, but also _much faster_ to
|
1226
|
+
run. That said—and especially if you're adding `test_data` to an existing test
|
1227
|
+
suite—care should be taken to audit everything the suite does between tests in
|
1228
|
+
order to optimize its overall runtime.
|
1229
|
+
|
1230
|
+
#### Randomized test order leading to data churn
|
1231
|
+
|
1232
|
+
Generally speaking, randomizing the order in which tests run is an unmitigated
|
1233
|
+
win: randomizing helps you catch any unintended dependency between two tests
|
1234
|
+
early, when it's still cheap & easy to fix. However, if your tests use different
|
1235
|
+
sources of test data (e.g. some call `TestData.uses_test_data` and some call
|
1236
|
+
`TestData.uses_clean_slate`), it's very likely that randomizing your tests will
|
1237
|
+
result in a significantly slower overall test suite. Instead, if you group tests
|
1238
|
+
that use the same type of test data together (e.g. by separating them into
|
1239
|
+
separate suites), you might find profound speed gains.
|
1240
|
+
|
1241
|
+
To illustrate this, suppose you had 5 tests that relied on your `test_data` data
|
1242
|
+
and 5 that relied on Rails fixtures. If all of these tests ran in random order
|
1243
|
+
(the default), you might see the following behavior at run-time:
|
1244
|
+
|
1245
|
+
```
|
1246
|
+
$ bin/rails test test/example_test.rb
|
1247
|
+
Run options: --seed 63999
|
1248
|
+
|
1249
|
+
# Running:
|
1250
|
+
|
1251
|
+
test_data -- loading test_data SQL dump
|
1252
|
+
. fixtures -- truncating tables, loading Rails fixtures
|
1253
|
+
. fixtures -- rolling back to Rails fixtures
|
1254
|
+
. test_data -- rolling back to clean test_data
|
1255
|
+
. fixtures -- truncating tables, loading Rails fixtures
|
1256
|
+
. test_data -- rolling back to clean test_data
|
1257
|
+
. fixtures -- truncating tables, loading Rails fixtures
|
1258
|
+
. test_data -- rolling back to clean test_data
|
1259
|
+
. fixtures -- truncating tables, loading Rails fixtures
|
1260
|
+
. test_data -- rolling back to clean test_data
|
1261
|
+
.
|
1262
|
+
|
1263
|
+
Finished in 2.449957s, 4.0817 runs/s, 4.0817 assertions/s.
|
1264
|
+
10 runs, 10 assertions, 0 failures, 0 errors, 0 skips
|
1265
|
+
```
|
1266
|
+
|
1267
|
+
So, what can you do to speed this up? The most effective strategy to avoiding
|
1268
|
+
this churn is to group the execution of each tests that use each source of test
|
1269
|
+
data into sub-suites that are run serially, on e after the other.
|
1270
|
+
|
1271
|
+
* If you're using Rails' defualt Minitest, we wrote a gem called
|
1272
|
+
[minitest-suite](https://github.com/testdouble/minitest-suite) to accomplish
|
1273
|
+
exactly this. Just declare something like `suite :test_data` or `suite
|
1274
|
+
:fixtures` at the top of each test class
|
1275
|
+
* If you're using RSpec, the [suite option combined with a custom
|
1276
|
+
ordering](https://gist.github.com/myronmarston/8fea012b9eb21b637335bb29069bce6b)
|
1277
|
+
can accomplish this. You might also consider using
|
1278
|
+
[tags](https://relishapp.com/rspec/rspec-core/v/3-10/docs/command-line/tag-option)
|
1279
|
+
to organize your tests by type, but you'll likely have to
|
1280
|
+
run a separate CLI invocation for each to avoid the tests from being
|
1281
|
+
interleaved
|
1282
|
+
|
1283
|
+
Here's what the same example would do at run-time after adding
|
1284
|
+
[minitest-suite](https://github.com/testdouble/minitest-suite):
|
1285
|
+
|
1286
|
+
```
|
1287
|
+
$ bin/rails test test/example_test.rb
|
1288
|
+
Run options: --seed 50105
|
1289
|
+
|
1290
|
+
# Running:
|
1291
|
+
|
1292
|
+
test_data -- loading test_data SQL dump
|
1293
|
+
. test_data -- rolling back to clean test_data
|
1294
|
+
. test_data -- rolling back to clean test_data
|
1295
|
+
. test_data -- rolling back to clean test_data
|
1296
|
+
. test_data -- rolling back to clean test_data
|
1297
|
+
. fixtures -- truncating tables, loading Rails fixtures
|
1298
|
+
. fixtures -- rolling back to clean fixtures
|
1299
|
+
. fixtures -- rolling back to clean fixtures
|
1300
|
+
. fixtures -- rolling back to clean fixtures
|
1301
|
+
. fixtures -- rolling back to clean fixtures
|
1302
|
+
.
|
1303
|
+
|
1304
|
+
Finished in 2.377050s, 4.2069 runs/s, 4.2069 assertions/s.
|
1305
|
+
10 runs, 10 assertions, 0 failures, 0 errors, 0 skips
|
1306
|
+
```
|
1307
|
+
|
1308
|
+
By grouping the execution in this way, the most expensive operations will
|
1309
|
+
usually only be run once: at the beginning of the first test in each suite.
|
1310
|
+
|
1311
|
+
#### Expensive data manipulation
|
1312
|
+
|
1313
|
+
If you're doing anything repeatedly that's data-intensive in your test setup
|
1314
|
+
after calling one of the `TestData.uses_*` methods, that operation is being
|
1315
|
+
repeated once per test, which could be very slow. Instead, you might consider
|
1316
|
+
moving that behavior into a [lifecycle hook](#lifecycle-hooks).
|
1317
|
+
|
1318
|
+
Any code passed to a lifecycle hook will only be executed when data is
|
1319
|
+
_actually_ loaded or truncated and its effect will be included in the
|
1320
|
+
transaction savepoint that the `test_data` gem rolls back between tests.
|
1321
|
+
Seriously, appropriately moving data adjustments into these hooks can cut your
|
1322
|
+
test suite's runtime by an order of magnitude.
|
1323
|
+
|
1324
|
+
#### Redundant test setup tasks
|
1325
|
+
|
1326
|
+
One of the most likely sources of unnecessary slowness is redundant test
|
1327
|
+
cleanup. The speed gained from sandwiching every expensive operation between
|
1328
|
+
transaction savepoints can be profound… but can also easily be erased by a
|
1329
|
+
single before-each hook calling
|
1330
|
+
[database_cleaner](https://github.com/DatabaseCleaner/database_cleaner) to
|
1331
|
+
commit a truncation of the database. As a result, it's worth taking a little
|
1332
|
+
time to take stock of everything that's called between tests during setup &
|
1333
|
+
teardown to ensure multiple tools aren't attempting to clean up the state of the
|
1334
|
+
database and potentially interfering with one another.
|
1335
|
+
|
1347
1336
|
## Code of Conduct
|
1348
1337
|
|
1349
1338
|
This project follows Test Double's [code of
|