test_data 0.0.2 → 0.2.2

Sign up to get free protection for your applications and to get access to all the features.
Files changed (56) hide show
  1. checksums.yaml +4 -4
  2. data/.github/workflows/ruby.yml +1 -5
  3. data/.standard.yml +2 -0
  4. data/CHANGELOG.md +41 -0
  5. data/Gemfile.lock +16 -16
  6. data/LICENSE.txt +1 -6
  7. data/README.md +864 -501
  8. data/example/.gitignore +1 -4
  9. data/example/Gemfile.lock +74 -74
  10. data/example/config/application.rb +3 -0
  11. data/example/config/credentials.yml.enc +1 -2
  12. data/example/spec/rails_helper.rb +1 -1
  13. data/example/spec/requests/boops_spec.rb +1 -5
  14. data/example/spec/requests/rails_fixtures_override_spec.rb +106 -0
  15. data/example/test/integration/better_mode_switching_demo_test.rb +6 -10
  16. data/example/test/integration/fixture_load_count_test.rb +82 -0
  17. data/example/test/integration/load_rollback_truncate_test.rb +40 -45
  18. data/example/test/integration/mode_switching_demo_test.rb +4 -14
  19. data/example/test/integration/parallel_boops_with_fixtures_test.rb +2 -6
  20. data/example/test/integration/parallel_boops_without_fixtures_test.rb +2 -6
  21. data/example/test/integration/rails_fixtures_double_load_test.rb +10 -0
  22. data/example/test/integration/rails_fixtures_override_test.rb +110 -0
  23. data/example/test/integration/test_data_hooks_test.rb +89 -0
  24. data/example/test/integration/transaction_committing_boops_test.rb +5 -3
  25. data/example/test/test_helper.rb +2 -6
  26. data/lib/generators/test_data/cable_yaml_generator.rb +18 -0
  27. data/lib/generators/test_data/database_yaml_generator.rb +2 -3
  28. data/lib/generators/test_data/environment_file_generator.rb +7 -0
  29. data/lib/generators/test_data/initializer_generator.rb +20 -7
  30. data/lib/generators/test_data/secrets_yaml_generator.rb +19 -0
  31. data/lib/generators/test_data/webpacker_yaml_generator.rb +3 -2
  32. data/lib/test_data.rb +37 -1
  33. data/lib/test_data/active_record_ext.rb +11 -0
  34. data/lib/test_data/config.rb +33 -3
  35. data/lib/test_data/configurators.rb +2 -0
  36. data/lib/test_data/configurators/cable_yaml.rb +25 -0
  37. data/lib/test_data/configurators/environment_file.rb +3 -2
  38. data/lib/test_data/configurators/initializer.rb +3 -2
  39. data/lib/test_data/configurators/secrets_yaml.rb +25 -0
  40. data/lib/test_data/configurators/webpacker_yaml.rb +4 -3
  41. data/lib/test_data/custom_loaders/abstract_base.rb +25 -0
  42. data/lib/test_data/custom_loaders/rails_fixtures.rb +45 -0
  43. data/lib/test_data/dumps_database.rb +24 -1
  44. data/lib/test_data/generator_support.rb +3 -0
  45. data/lib/test_data/inserts_test_data.rb +25 -0
  46. data/lib/test_data/loads_database_dumps.rb +1 -1
  47. data/lib/test_data/log.rb +19 -1
  48. data/lib/test_data/{transactional_data_loader.rb → manager.rb} +78 -81
  49. data/lib/test_data/rake.rb +16 -7
  50. data/lib/test_data/statistics.rb +6 -1
  51. data/lib/test_data/truncates_test_data.rb +31 -0
  52. data/lib/test_data/version.rb +1 -1
  53. data/script/reset_example_app +1 -0
  54. data/script/test +31 -6
  55. data/test_data.gemspec +1 -1
  56. metadata +20 -4
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: eb3ddac1f4153d6b3995975a7b3016089c0f64ef783b45af3b21d5a2ea00f671
4
- data.tar.gz: 0d45ac53789a0015c88f6d3f8cc545409c4c42fbd112f63edb1489cc5d603c7c
3
+ metadata.gz: 18d3469e9d0173f81ef3b2d8d082861094d79354b8fb544e983bdcfdd8eb788b
4
+ data.tar.gz: 983a1645132b22024799670ec463f11fe5f8e0d1b9427b96940dea8c684fbafb
5
5
  SHA512:
6
- metadata.gz: 7368531e5dcc41910350809eb2d5a36763f395a749f69dbe85579792cca513fbe5663a0c02c2595009617a63d274caba073572a854cb3f334a07b4b57ce98ff8
7
- data.tar.gz: 358f87044733160c2d4ee49b6e8ec1b422cc08e6aa4f0a12a7b47f190899b67ea707159e5cc2e66c307934ad4c1315c45fcbde4666eeb5497b98868afbae31ee
6
+ metadata.gz: c7fb1b86f23f074bb185cb1cf11c437d4ba57445d65907509384838993628c5759803ee90f3f290dbb219754806bb1c1133b7e84bf430918c1a23b3fca86b752
7
+ data.tar.gz: d00e512b415c74ab1650a3b3d58e36759d085056375e07326d8790d0033bc18066a2d82c791a578fb9d11cb7c08f9056d4d5e380b9cc52f97dd467c245469771
@@ -7,11 +7,7 @@
7
7
 
8
8
  name: Ruby
9
9
 
10
- on:
11
- push:
12
- branches: [ master ]
13
- pull_request:
14
- branches: [ master ]
10
+ on: [push, pull_request, workflow_dispatch]
15
11
 
16
12
  jobs:
17
13
  test:
data/.standard.yml ADDED
@@ -0,0 +1,2 @@
1
+ ruby_version: 2.4.0
2
+
data/CHANGELOG.md CHANGED
@@ -1,3 +1,44 @@
1
+ # 0.2.2
2
+
3
+ - Improve performance of Rails fixtures being repeatedly loaded by changing the
4
+ caching strategy
5
+
6
+ # 0.2.1
7
+
8
+ - Adds several lifecycle hooks:
9
+ - config.after_test_data_load
10
+ - config.after_test_data_truncate
11
+ - config.after_rails_fixture_load
12
+
13
+ # 0.2.0
14
+
15
+ - BREAKING CHANGES: Remove or rename a bunch of APIs that aren't quite necessary
16
+ and leak too much implementation, requiring too much cognitive load for users.
17
+ - Remove config.use_transactional_data_loader
18
+ - Remove TestData.rollback
19
+ - Change TestData.load to TestData.uses_test_data and make it transaction-only
20
+ - Change TestData.truncate to TestData.uses_clean_slate and make it
21
+ transaction-only
22
+ - Change TestData.load_rails_fixtures to TestData.uses_rails_fixtures and make
23
+ it transaction-only
24
+ - Add TestData.insert_test_data_dump, which will blindly insert the test SQL
25
+ dump of test data without any transaction management
26
+ - [#2](https://github.com/testdouble/test_data/issues/2) - Work around
27
+ hard-coded environment names when initializing test_data environment secrets
28
+
29
+ # 0.1.0
30
+
31
+ - New feature: `TestData.load_rails_fixtures` to override default fixtures
32
+ behavior by loading it in a nested transaction after `TestData.truncate`
33
+ - Breaking change: move transactions configuration out of `TestData.load` and
34
+ instead a global setting for `TestData.config` named
35
+ `use_transactional_data_loader`
36
+ - Cascades truncation of test_data tables unless they're explicitly specified by
37
+ the truncate_these_test_data_tables` option
38
+ - Add secrets.yml and cable.yml generators to `test_data:configure` task
39
+ - Print the size of each dump and warn when dump size reaches certain thresholds
40
+ or increases significantly in the `test_data:dump` task
41
+
1
42
  # 0.0.2
2
43
 
3
44
  - Make the rest of the gem better
data/Gemfile.lock CHANGED
@@ -1,26 +1,26 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- test_data (0.0.2)
4
+ test_data (0.2.2)
5
5
  railties (~> 6.0)
6
6
 
7
7
  GEM
8
8
  remote: https://rubygems.org/
9
9
  specs:
10
- actionpack (6.1.3.1)
11
- actionview (= 6.1.3.1)
12
- activesupport (= 6.1.3.1)
10
+ actionpack (6.1.4)
11
+ actionview (= 6.1.4)
12
+ activesupport (= 6.1.4)
13
13
  rack (~> 2.0, >= 2.0.9)
14
14
  rack-test (>= 0.6.3)
15
15
  rails-dom-testing (~> 2.0)
16
16
  rails-html-sanitizer (~> 1.0, >= 1.2.0)
17
- actionview (6.1.3.1)
18
- activesupport (= 6.1.3.1)
17
+ actionview (6.1.4)
18
+ activesupport (= 6.1.4)
19
19
  builder (~> 3.1)
20
20
  erubi (~> 1.4)
21
21
  rails-dom-testing (~> 2.0)
22
22
  rails-html-sanitizer (~> 1.1, >= 1.2.0)
23
- activesupport (6.1.3.1)
23
+ activesupport (6.1.4)
24
24
  concurrent-ruby (~> 1.0, >= 1.0.2)
25
25
  i18n (>= 1.6, < 2)
26
26
  minitest (>= 5.1)
@@ -29,19 +29,19 @@ GEM
29
29
  ast (2.4.2)
30
30
  builder (3.2.4)
31
31
  coderay (1.1.3)
32
- concurrent-ruby (1.1.8)
32
+ concurrent-ruby (1.1.9)
33
33
  crass (1.0.6)
34
34
  erubi (1.10.0)
35
35
  i18n (1.8.10)
36
36
  concurrent-ruby (~> 1.0)
37
- loofah (2.9.1)
37
+ loofah (2.11.0)
38
38
  crass (~> 1.0.2)
39
39
  nokogiri (>= 1.5.9)
40
40
  method_source (1.0.0)
41
- mini_portile2 (2.5.1)
41
+ mini_portile2 (2.6.1)
42
42
  minitest (5.14.4)
43
- nokogiri (1.11.3)
44
- mini_portile2 (~> 2.5.0)
43
+ nokogiri (1.12.0)
44
+ mini_portile2 (~> 2.6.1)
45
45
  racc (~> 1.4)
46
46
  parallel (1.20.1)
47
47
  parser (3.0.1.0)
@@ -58,11 +58,11 @@ GEM
58
58
  nokogiri (>= 1.6)
59
59
  rails-html-sanitizer (1.3.0)
60
60
  loofah (~> 2.3)
61
- railties (6.1.3.1)
62
- actionpack (= 6.1.3.1)
63
- activesupport (= 6.1.3.1)
61
+ railties (6.1.4)
62
+ actionpack (= 6.1.4)
63
+ activesupport (= 6.1.4)
64
64
  method_source
65
- rake (>= 0.8.7)
65
+ rake (>= 0.13)
66
66
  thor (~> 1.0)
67
67
  rainbow (3.0.0)
68
68
  rake (13.0.3)
data/LICENSE.txt CHANGED
@@ -1,9 +1,4 @@
1
- Copyright (c) 2019 Test Double, LLC
2
-
3
- Portions of these files Copyright (c) 2012-18 Bozhidar Batsov:
4
- - config/base.yml
5
- - lib/standard/cop/block_single_line_braces.rb
6
- - test/cop_invoker.rb
1
+ Copyright (c) 2021 Test Double, LLC
7
2
 
8
3
  Permission is hereby granted, free of charge, to any person obtaining
9
4
  a copy of this software and associated documentation files (the
data/README.md CHANGED
@@ -1,48 +1,104 @@
1
1
  # The `test_data` gem
2
2
 
3
- `test_data` is what it says on the tin: a fast & reliable system for managing
4
- your Rails application's test data.
3
+ `test_data` does what it says on the tin: it provides a fast & reliable system
4
+ for managing your Rails application's test data.
5
5
 
6
6
  The gem serves as both an alternative to
7
7
  [fixtures](https://guides.rubyonrails.org/testing.html#the-low-down-on-fixtures)
8
8
  & [factory_bot](https://github.com/thoughtbot/factory_bot), as well a broader
9
- workflow for building high-performance test suites that will scale with your
10
- application.
9
+ workflow for building test suites that will scale gracefully as your application
10
+ grows in size and complexity.
11
11
 
12
12
  What it does:
13
13
 
14
- * Establishes a fourth Rails environment named `test_data` designed to help you
15
- create a universe of data your tests can rely on by simply using your
16
- application. No Ruby DSL, no YAML files, no precarious approximations of
17
- realism: **real data created by your app**
14
+ * Establishes a fourth Rails environment (you can [define custom Rails
15
+ environments](https://guides.rubyonrails.org/configuring.html#creating-rails-environments)!)
16
+ named `test_data`, which you'll use to create a universe of data for your
17
+ tests by simply running and using your application. No Ruby DSL, no YAML
18
+ files, no precarious approximations of realism: **real data created by your
19
+ app**
18
20
 
19
- * Exposes a simple API for loading your test data and cleaning up between
20
- tests. Common edge cases are handled gracefully: tests that need access to
21
- your data will see it and tests that don't, won't
21
+ * Exposes a simple API for ensuring that your data will be pristine for each of
22
+ your tests, whether the test depends on test_data, an empty database, or Rails
23
+ fixtures
22
24
 
23
- * Safeguards your tests from flaky failures and supercharges their speed by
24
- providing a sophisticated transaction manager that isolates each test's
25
- changes to your data while ensuring your data is only loaded once
25
+ * Safeguards your tests from flaky failures and supercharges your build by
26
+ providing a sophisticated transaction manager that isolates each test while
27
+ ensuring your data is only loaded once
26
28
 
27
29
  If you've despaired over the seeming inevitability that all Rails test suites
28
- will eventually grow to become slow, unreliable, and brittle, then this gem is
29
- for you! And even if you're [a factory_bot
30
+ will eventually grow to become slow, flaky, and incomprehensible, then this gem
31
+ is for you! And even if you're [a factory_bot
30
32
  fan](https://twitter.com/searls/status/1379491813099253762?s=20), we hope you'll
31
33
  be open to the idea that [there might be a better way](
32
34
  #but-we-use-and-like-factory_bot-and-so-i-am-inclined-to-dislike-everything-about-this-gem).
33
35
 
34
- _[Because the gem is still brand new, it makes a number of
35
- [assumptions](#assumptions) and may not work for every project just yet.]_
36
+ _[Full disclosure: because the gem is still brand new, it makes a number of
37
+ [assumptions](#assumptions)—chief among them being that **Postgres & Rails 6+
38
+ are required**—so it may not work for every project just yet.]_
39
+
40
+ ## Documentation
41
+
42
+ This gem requires a lot of documentation—not because `test_data` does a lot of
43
+ things, but because managing one's test data is an inherently complex task. If
44
+ one reason Rails apps chronically suffer from slow tests is that other
45
+ approaches oversimplify test data management, it stands to reason that any
46
+ discomfort caused by `test_data`'s scope may not be _unnecessary complexity_ but
47
+ instead be an indication of how little of the problem's _essential complexity_
48
+ we have reckoned with to this point.
49
+
50
+ 1. [Getting Started Guide](#getting-started-guide)
51
+ 1. [Install and initialize `test_data`](#step-1-install-and-initialize-test_data)
52
+ 2. [Create some test data](#step-2-create-some-test-data)
53
+ 3. [Dump your `test_data` database](#step-3-dump-your-test_data-database)
54
+ 4. [Load your data in your tests](#step-4-load-your-data-in-your-tests)
55
+ 5. [Keeping your test data up-to-date](#step-5-keeping-your-test-data-up-to-date)
56
+ 2. [Factory & Fixture Interoperability Guide](#factory--fixture-interoperability-guide)
57
+ * [Using `test_data` with `factory_bot`](#using-test_data-with-factory_bot)
58
+ * [Using `test_data` with Rails fixtures](#using-test_data-with-rails-fixtures)
59
+ 3. [Rake Task Reference](#rake-task-reference)
60
+ * [test_data:install](#test_datainstall)
61
+ * [test_data:configure](#test_dataconfigure)
62
+ * [test_data:verify_config](#test_dataverify_config)
63
+ * [test_data:initialize](#test_datainitialize)
64
+ * [test_data:dump](#test_datadump)
65
+ * [test_data:load](#test_dataload)
66
+ * [test_data:create_database](#test_datacreate_database)
67
+ * [test_data:drop_database](#test_datadrop_database)
68
+ 4. [API Reference](#api-reference)
69
+ * [TestData.uses_test_data](#testdatauses_test_data)
70
+ * [TestData.uses_clean_slate](#testdatauses_clean_slate)
71
+ * [TestData.uses_rails_fixtures(self)](#testdatauses_rails_fixtures)
72
+ * [TestData.prevent_rails_fixtures_from_loading_automatically!](#testdataprevent_rails_fixtures_from_loading_automatically)
73
+ * [TestData.config](#testdataconfig)
74
+ * [TestData.insert_test_data_dump](#testdatainsert_test_data_dump)
75
+ 5. [Assumptions](#assumptions)
76
+ 6. [Fears, Uncertainties, and Doubts](#fears-uncertainties-and-doubts) (Q & A)
77
+ * [But we're already happy with
78
+ factory_bot!](#but-we-use-and-like-factory_bot-and-so-i-am-inclined-to-dislike-everything-about-this-gem)
79
+ * [How will we handle merge conflicts in the schema
80
+ dumps?](#how-will-i-handle-merge-conflicts-in-these-sql-files-if-i-have-lots-of-people-working-on-lots-of-feature-branches-all-adding-to-the-test_data-database-dumps)
81
+ * [Why can't I manage different SQL dumps for different
82
+ scenarios?](#why-cant-i-save-multiple-database-dumps-to-cover-different-scenarios)
83
+ * [These SQL dumps are way too large to commit to
84
+ git!](#are-you-sure-i-should-commit-these-sql-dumps-theyre-way-too-big)
85
+ * [Tests shouldn't rely on shared test data if they don't need
86
+ to](#tests-shouldnt-use-shared-test-data-they-should-instantiate-the-objects-they-need)
87
+ * [My tests aren't as fast as they should
88
+ be](#im-worried-my-tests-arent-as-fast-as-they-should-be)
89
+ 7. [Code of Conduct](#code-of-conduct)
90
+ 8. [Changelog](/CHANGELOG.md)
91
+ 9. [MIT License](/LICENSE.txt)
36
92
 
37
93
  ## Getting started guide
38
94
 
39
95
  This guide will walk you through setting up `test_data` in your application. You
40
96
  might notice that it's more complicated than installing a gem and declaring some
41
- default `Widget` attributes! The truth is that designing robust and reliable
42
- test data that is inherently complex and takes some thoughtful planning. There
43
- are plenty of shortcuts available, but experience has shown they tend to
44
- collapse under their own weight as your app scales and your team grows—exactly
45
- when fast & reliable tests are most valuable.
97
+ default `Widget` attributes! The hard truth is that designing robust and
98
+ reliable test data is an inherently complex problem and takes some thoughtful
99
+ planning. There are plenty of shortcuts available, but experience has shown they
100
+ tend to collapse under their own weight as your app scales and your team
101
+ grows—exactly when having a suite of fast & reliable tests is most valuable.
46
102
 
47
103
  And if you get stuck or need help as you're getting started, please feel free to
48
104
  [ask us for help](https://github.com/testdouble/test_data/discussions/new)!
@@ -62,14 +118,15 @@ end
62
118
  ```
63
119
 
64
120
  Since the `test_data` environment is designed to be used similarly to
65
- `development` (i.e. with a running server and interacting via a browser) the
66
- `:test_data` gem group should probably include everything that's available to
67
- the `:development` group.
121
+ `development` (i.e. with a running server and interacting via a browser), any
122
+ gems in your `:development` gem group should likely be included in a
123
+ `:test_data` gem group as well.
68
124
 
69
125
  #### Configuring the gem and initializing the database
70
126
 
71
- The gem ships with a number of Rake tasks, including `test_data:install`, which
72
- will generate the necessary configuration and initialize a `test_data` database:
127
+ The gem ships with a number of Rake tasks, including
128
+ [test_data:install](#test_datainstall), which will generate the necessary
129
+ configuration and initialize a `test_data` database:
73
130
 
74
131
  ```
75
132
  $ bin/rake test_data:install
@@ -88,21 +145,20 @@ Created database 'yourappname_test_data'
88
145
  ------------
89
146
 
90
147
  (1 row)
91
- ````
92
148
 
93
- The purpose of the `test_data` database is to provide a sandbox in which to
94
- generate test data by interacting with your app and then dumping the resulting
95
- state of the database so that your tests can load it later. Rather than try to
96
- imitate realistic data using factories and fixtures (a task which only grows
97
- more difficult as your models and their associations increase in complexity),
98
- your test data will always be realistic because your real application will have
99
- created it!
149
+ Your test_data environment and database are ready for use! You can now run
150
+ your server (or any command) to create some test data like so:
151
+
152
+ $ RAILS_ENV=test_data bin/rails server
153
+
154
+ ````
100
155
 
101
- The database dumps are meant to be committed in git and versioned alongside your
102
- tests over the life of the application. Its schema & data are should be
103
- incrementally migrated over time, just like your production database. (As a
104
- happy side effect, this means your `test_data` database may help you identify
105
- hard-to-catch migration bugs early, before being deployed to production!)
156
+ The purpose of the `test_data` database is to provide a sandbox in which you
157
+ will manually generate test data by playing around with your app. Rather than
158
+ try to imitate realistic data using factories and fixtures (a task which only
159
+ grows more difficult as your models and their associations increase in
160
+ complexity), your test data will always be realistic because your real
161
+ application will have created it!
106
162
 
107
163
  ### Step 2: Create some test data
108
164
 
@@ -118,6 +174,10 @@ environment variable:
118
174
  $ RAILS_ENV=test_data bin/rails server
119
175
  ```
120
176
 
177
+ _[If you're using [webpacker](https://github.com/rails/webpacker), you may also
178
+ need to start its development server as well with `RAILS_ENV=test_data
179
+ bin/webpack-dev-server`]_
180
+
121
181
  Because `test_data` creates a full-fledged Rails environment, you can run any
122
182
  number of Rails commands or Rake tasks against its database by setting
123
183
  `RAILS_ENV=test_data`, either in your shell environment or with each command
@@ -125,31 +185,47 @@ number of Rails commands or Rake tasks against its database by setting
125
185
 
126
186
  _[Aside: If you experience any hiccups in getting your server to work, please
127
187
  [open an issue](https://github.com/testdouble/test_data/issues/new) and let us
128
- know—it may present an opportunity to improve the `test_data:configure` task!]_
188
+ know—it may present an opportunity for us to improve the `test_data:configure`
189
+ task!]_
129
190
 
130
191
  #### Create test data by using your app
131
192
 
132
193
  Once the app is running, it's time to generate some test data. You'll know how
133
194
  to accomplish this step better than anyone—it's your app, after all!
134
195
 
135
- Our advice? Spend a little time thoughtfully navigating each feature of your app
136
- in order to generate enough data to be _representative_ of what would be needed
137
- to test your system's main behaviors (e.g. one `User` for each role, one of each
138
- kind of `Order`, etc.), while still being _minimal_ enough that the universe of
139
- data will be comprehensible & memorable to yourself and your teammates. It can
140
- also help to give new records memorable names, perhaps in keeping with a common
141
- theme (easier to refer to "Rafael" than "TestUser #1").
142
-
143
- If you make a mistake, it's perfectly okay to reset the database and start over!
144
- Your future tests will be coupled to this data as your application grows and
145
- evolves, so it's worth taking the time to ensure the foundation is solid. (But
146
- that's not to say everything needs to be perfect; you can always change things
147
- or add more data later—you'll just have to update your tests accordingly.)
196
+ A few bits of advice click & type some test data into existence:
197
+
198
+ * Spend a little time thoughtfully navigating each feature of your app in order
199
+ to generate enough data to be representative of what would be needed to test
200
+ them (e.g. one `User` per role, one of each kind of `Order`, etc.)
201
+ * Less is more: the less test data you create, the more meaningful & memorable
202
+ it will be to yourself and your teammates when writing tests. Don't keep
203
+ adding test data unless it will allow you to exercise additional application
204
+ code (e.g. enough `Project` models to require pagination, but not hundreds of
205
+ them for the sake of looking "production-like")
206
+ * Memorable names can become memes for the team to quickly recall and reference
207
+ later (if the admin user is named "Angela" and the manager is "Maria", that'll
208
+ probably serve you better than generic names like "TestUser #1")
209
+
210
+ If you make a mistake when creating your initial set of test data, it's
211
+ perfectly okay to reset the database and start over! Your future tests will be
212
+ coupled to this data as your application grows and evolves, so it's worth taking
213
+ the time to ensure the foundation is solid. (But that's not to say everything
214
+ needs to be perfect; you can always change things or add more data later—you'll
215
+ just have to update your tests accordingly.)
148
216
 
149
217
  ### Step 3: Dump your `test_data` database
150
218
 
219
+ Once you've created a good sampling of test data by interacting with your app,
220
+ the next step is to flush it from the `test_data` database to SQL files. These
221
+ database dumps are meant to be committed to source control and versioned
222
+ alongside your tests over the life of the application. Additionally, they are
223
+ designed to be incrementally
224
+ [migrated](#step-5-keeping-your-test-data-up-to-date) over time, just like you
225
+ migrate production database with every release.
226
+
151
227
  Once you have your test data how you want it, dump the schema and data to SQL
152
- files:
228
+ files with the `test_data:dump` Rake task:
153
229
 
154
230
  ```
155
231
  $ bin/rake test_data:dump
@@ -157,148 +233,226 @@ $ bin/rake test_data:dump
157
233
 
158
234
  This will dump three files into `test/support/test_data`:
159
235
 
160
- * Schema DDL in `schema.sql`
236
+ * `schema.sql` - Schema DDL used to (re-)initialize the `test_data` environment
237
+ database for anyone looking to update your test data
161
238
 
162
- * Test data in `data.sql`
239
+ * `data.sql` - The test data itself, exported as a bunch of SQL `INSERT`
240
+ statements, which will be executed by your tests to load your test data
163
241
 
164
- * Non-test data (`ar_internal_metadata` and `schema_migrations` by default) in
165
- `non_test_data.sql`
242
+ * `non_test_data.sql` - Data needed to run the `test_data` environment, but
243
+ which shouldn't be inserted by your tests (the `ar_internal_metadata` and
244
+ `schema_migrations` tables, by default; see `config.non_test_data_tables`)
166
245
 
167
- These paths can be overridden with [TestData.config](#testdataconfig) method.
168
- Additional details can be found in the [test_data:dump](#test_datadump)
169
- Rake task reference.
246
+ You probably won't need to, but these paths can be overridden with
247
+ [TestData.config](#testdataconfig) method. Additional details can also be found
248
+ in the [test_data:dump](#test_datadump) Rake task reference.
170
249
 
171
250
  Once you've made your initial set of dumps, briefly inspect them and—if
172
251
  everything looks good—commit them. (And if the files are gigantic or full of
173
252
  noise, you might find [these ideas
174
253
  helpful](#are-you-sure-i-should-commit-these-sql-dumps-theyre-way-too-big)).
175
254
 
176
- _[Feel weird to dump and commit SQL files? That's okay! It's [healthy to be
177
- skeptical](https://twitter.com/searls/status/860553435116187649?s=20) whenever
178
- you're asked to commit a generated file! Remember that the `test_data`
255
+ Does it feel weird to dump and commit SQL files? That's okay! It's [healthy to
256
+ be skeptical](https://twitter.com/searls/status/860553435116187649?s=20)
257
+ whenever you're asked to commit a generated file! Remember that the `test_data`
179
258
  environment exists only for creating your test data. Your tests will, in turn,
180
- load the SQL dump of your data into the familiar `test` database, and things
181
- will proceed just as if you'd been loading [Rails' built-in
259
+ load the SQL dump of your data into the `test` database, and things will proceed
260
+ just as if you'd been loading [Rails' built-in
182
261
  fixtures](https://guides.rubyonrails.org/testing.html#the-low-down-on-fixtures)
183
- from a set of YAML files—the major difference being how the data is authored.]_
262
+ from a set of YAML files.
184
263
 
185
264
  ### Step 4: Load your data in your tests
186
265
 
187
266
  Now that you've dumped the contents of your `test_data` database, you can start
188
267
  writing tests that rely on this test data.
189
268
 
190
- To accomplish this, you'll likely want to add hooks to run before & after each
191
- test—first to load your test data and then to rollback any changes made by the
192
- test. The `test_data` gem accomplishes this with its
193
- [TestData.load](#testdataload) and [TestData.rollback](#testdatarollback)
194
- methods.
269
+ To accomplish this, you'll likely want to add hooks to run before each test to
270
+ put the database into whatever state the test needs.
195
271
 
196
- If you're using (Rails' default)
197
- [Minitest](https://github.com/seattlerb/minitest) and want to include your test
198
- data with every test, you can add these hooks to `ActiveSupport::TestCase`:
272
+ For the simplest case—ensuring your test data is loaded into the `test` database
273
+ and available to your test, you'll want to call the
274
+ [TestData.uses_test_data](#testdatauses_test_data) method at the beginning of
275
+ the test. The first time `uses_test_data` is called, `test_data` will start a
276
+ transaction and insert your test data. On subsequent calls to `uses_test_data`
277
+ by later tests, the transaction will be rolled back to a save point taken just
278
+ after the data was initially loaded, so that each test gets a clean starting
279
+ point without repeatedly executing the expensive SQL operation.
280
+
281
+ #### If you want every single test to have access to your test data
282
+
283
+ If, for the sake of consistency & simplicity you want every single Rails-aware
284
+ test to have access to your test data, you
285
+ can accomplish this with a single global before-each hook.
286
+
287
+ If you're using Rails' default
288
+ [Minitest](https://github.com/seattlerb/minitest), you can load it in a `setup`
289
+ hook in `ActiveSupport::TestCase`:
199
290
 
200
291
  ```ruby
201
292
  class ActiveSupport::TestCase
202
- def setup
203
- TestData.load
204
- end
205
-
206
- def teardown
207
- TestData.rollback
293
+ setup do
294
+ TestData.uses_test_data
208
295
  end
209
296
  end
210
297
  ```
211
298
 
212
- If you use [RSpec](https://rspec.info), you can accomplish the same thing with
213
- global `before(:each)` and `after(:each)` hooks in your `rails_helper.rb` file:
299
+ Likewise, if you use [RSpec](https://rspec.info), you can accomplish the same
300
+ thing with global `before(:each)` hook in your `rails_helper.rb` file:
214
301
 
215
302
  ```ruby
216
303
  RSpec.configure do |config|
217
304
  config.before(:each) do
218
- TestData.load
219
- end
220
-
221
- config.after(:each) do
222
- TestData.rollback
305
+ TestData.uses_test_data
223
306
  end
224
307
  end
225
308
  ```
226
309
 
227
- That should be all you need to have access to your test data in all of your
228
- tests, along with the speed and data integrity of wrapping each test in an
229
- always-rolled-back transaction. For more information on how all this works, see
230
- the [API reference](#api-reference).
310
+ #### If some tests rely on test data and others need a clean slate
311
+
312
+ Of course, for simple units of code, it may be more prudent to manually create
313
+ the test data they need inline as opposed to relying on a shared source of test
314
+ data. For these tests, you can call
315
+ [TestData.uses_clean_slate](#testdatauses_clean_slate) in a `setup` hook.
231
316
 
232
- If you _don't_ want all of your Rails-aware tests to see this test data (suppose
233
- you have existing tests that use factories or fixtures instead), you probably
234
- want to use [TestData.truncate](#testdatatruncate) to clear data generated by
235
- this gem out before they run. You might do that by defining two test types:
317
+ For the best performance, you might consider a mode-switching method that's
318
+ invoked at the top of each test listing like this:
236
319
 
237
320
  ```ruby
238
- # Tests using data created by `test_data`
239
- class TestDataTestCase < ActiveSupport::TestCase
240
- def setup
241
- TestData.load
321
+ class ActiveSupport::TestCase
322
+ def self.uses(mode)
323
+ case mode
324
+ when :clean_slate
325
+ setup { TestData.uses_clean_slate }
326
+ when :test_data
327
+ setup { TestData.uses_test_data }
328
+ else
329
+ raise "Invalid test data mode: #{mode}"
330
+ end
242
331
  end
332
+ end
243
333
 
244
- def teardown
245
- TestData.rollback
246
- end
334
+ # A simple model that will `create` its own data
335
+ class WidgetTest < ActiveSupport::TestCase
336
+ uses :clean_slate
337
+ # …
247
338
  end
248
339
 
249
- # Tests using data created by `factory_bot`
250
- class FactoryBotTestCase < ActiveSupport::TestCase
251
- include FactoryBot::Syntax::Methods
340
+ # An integrated test that depends on a lot of data
341
+ class KitchenSinkTest < ActionDispatch::IntegrationTest
342
+ uses :test_data
343
+ # …
344
+ end
345
+ ```
252
346
 
253
- def setup
254
- TestData.truncate
255
- end
347
+ Or, with RSpec:
256
348
 
257
- def teardown
258
- TestData.rollback(:after_data_truncate)
349
+ ```ruby
350
+ module TestDataModes
351
+ def uses(mode)
352
+ case mode
353
+ when :clean_slate
354
+ before(:each) { TestData.uses_clean_slate }
355
+ when :test_data
356
+ before(:each) { TestData.uses_test_data }
357
+ else
358
+ raise "Invalid test data mode: #{mode}"
359
+ end
259
360
  end
260
361
  end
261
362
 
363
+ RSpec.configure do |config|
364
+ config.extend(TestDataModes)
365
+ end
366
+
367
+ RSpec.describe Widget, type: :model do
368
+ uses :clean_slate
369
+ # …
370
+ end
371
+
372
+ RSpec.describe "Kitchen sink", type: :request do
373
+ uses :test_data
374
+ # …
375
+ end
262
376
  ```
263
377
 
264
- For more thoughts on migrating to `test_data` when you have existing tests,
265
- [some ideas are discussed
266
- here](#we-already-have-thousands-of-tests-that-depend-on-rails-fixtures-or-factory_bot-can-we-start-using-test_data-without-throwing-them-away-and-starting-over).
378
+ But wait, there's more! If your test suite switches between multiple modes from
379
+ test-to-test, it's important to be aware of the marginal cost _between_ each of
380
+ those tests. For example, two tests in a row that call `TestData.uses_test_data`
381
+ only need a simple rollback as test setup, but a `TestData.uses_test_data`
382
+ followed by a `TestData.uses_clean_slate` requires a rollback, a truncation, and
383
+ another savepoint. These small costs add up, so consider [speeding up your
384
+ build](#im-worried-my-tests-arent-as-fast-as-they-should-be) by grouping your
385
+ tests into sub-suites based on their source of test data.
386
+
387
+ #### If your situation is more complicated
388
+
389
+ If you're adding `test_data` to an existing application, it's likely that you
390
+ won't be able to easily adopt a one-size-fits-all approach to test setup across
391
+ your entire suite. Some points of reference, if that's the situation you're in:
392
+
393
+ * If your test suite is **already using fixtures or factories** and the above
394
+ hooks just broke everything, check out our [interoperability
395
+ guide](#factory--fixture-interoperability-guide) for help.
396
+ * If you need to make any changes to the data after it's loaded, truncated, or
397
+ after Rails fixtures are loaded, you can configure [lifecycle
398
+ hooks](#lifecycle-hooks) that will help you achieve a **very fast test suite**
399
+ by including those changes inside the transaction savepoints
400
+ * If you **don't want `test_data` managing transactions** and cleanup for you
401
+ and just want to load the SQL dump, you can call
402
+ [TestData.insert_test_data_dump](#testdatainsert_test_data_dump)
403
+ * For more information on how all this works, see the [API
404
+ reference](#api-reference).
267
405
 
268
406
  ### Step 5: Keeping your test data up-to-date
269
407
 
270
- Because your app relies on its tests and your tests rely on their test data,
271
- your test data needs to be maintained for the entire life of your application.
272
- Fortunately, and because production databases need the same thing, we already
273
- have a fantastic tool for the job: [Rails
408
+ Your app relies on its tests and your tests rely on their test data. This
409
+ creates a bit of a paradox: creating & maintaining test data is _literally_ a
410
+ tertiary concern but simultaneously an inescapable responsibility that will live
411
+ with you for the life of your application. That's true whether you use this gem,
412
+ `factory_bot`, Rails fixtures, or something else as a source of shared test
413
+ data.
414
+
415
+ Fortunately, we already have a fantastic tool available for keeping our
416
+ `test_data` database up-to-date over the life of our application: [Rails
274
417
  migrations](https://guides.rubyonrails.org/active_record_migrations.html). If
275
- your migrations are resilient enough for your production data, they should also
276
- be able to keep your `test_data` database up-to-date.
418
+ your migrations are resilient enough for your production database, they should
419
+ also be able to keep your `test_data` database up-to-date. (As a happy side
420
+ effect of running your migrations against your test data, this means your
421
+ `test_data` database may help you identify hard-to-catch migration bugs early,
422
+ before being deployed to production!)
423
+
424
+ Whenever you create a new migration or add a major feature, you'll probably need
425
+ to update your test data. Here's how to do it:
426
+
427
+ * If the current SQL dumps in `test/support/test_data` are newer than your local
428
+ `test_data` database:
429
+
430
+ 1. Be sure there's nothing in your local `test_data` database that you added
431
+ intentionally and forgot to dump, because it's about to be erased
277
432
 
278
- Whenever you update your schema, migrate your data, or add a feature that
279
- necessitates the creation of more test data, you'll need to update your test
280
- data. Here's a rough outline to updating your `test_data` database:
433
+ 2. Run `rake test_data:drop_database`
281
434
 
282
- 1. If your local `test_data` database is out-of-date with your latest SQL dump
283
- files, drop it with `rake test_data:drop_database`
435
+ 3. Run `rake test_data:load` to recreate the `test_data` database and load
436
+ the latest SQL dumps into it
284
437
 
285
- 2. Load your schema & data into the database with `rake test_data:load`
438
+ 4. Run any pending migrations with `RAILS_ENV=test_data bin/rake db:migrate`
286
439
 
287
- 3. Run any pending migrations with `RAILS_ENV=test_data bin/rake db:migrate`
440
+ 5. If you need to create any additional data, start up the server
441
+ (`RAILS_ENV=test_data bin/rails s`), just like in [Step
442
+ 2](#step-2-create-some-test-data)
288
443
 
289
- 4. If you need to create any additional data, start up the server
290
- (`RAILS_ENV=test_data bin/rails s`), just like [Step
291
- 2](#step-2-create-some-test-data)
444
+ 6. Export your newly-updated `test_data` database with `rake test_data:dump`
292
445
 
293
- 5. Export your newly-updated `test_data` database with `rake test_data:dump`
446
+ 7. Ensure your tests are passing and then commit the resulting SQL files
294
447
 
295
- 6. Ensure your tests are passing and commit the resulting SQL files
448
+ * If the local `test_data` database is already up-to-date with the current SQL
449
+ dumps, follow steps **4 through 7** above
296
450
 
297
451
  It's important to keep in mind that your test data SQL dumps are a shared,
298
452
  generated resource among your team (just like a `structure.sql` or `schema.rb`
299
453
  file). As a result, if your team doesn't integrate code frequently or if the
300
- test data experiences a lot of churn in coincident feature branches, you'd be
301
- right to be concerned that [the resulting merge conflicts could become
454
+ test data changes frequently, you'd be right to be concerned that [the resulting
455
+ merge conflicts could become
302
456
  significant](#how-will-i-handle-merge-conflicts-in-these-sql-files-if-i-have-lots-of-people-working-on-lots-of-feature-branches-all-adding-to-the-test_data-database-dumps),
303
457
  so sweeping changes should be made deliberately and in collaboration with other
304
458
  contributors.
@@ -307,7 +461,134 @@ _[Aside: some Rails teams are averse to using migrations to migrate data as well
307
461
  as schemas, instead preferring one-off scripts and tasks. You'll have an easier
308
462
  time of things if you use migrations for both schema and data changes. Here are
309
463
  some notes on [how to write data migrations
310
- safely](https://blog.testdouble.com/posts/2014-11-04-healthy-migration-habits/#habit-4-dont-reference-models).]_
464
+ safely](https://blog.testdouble.com/posts/2014-11-04-healthy-migration-habits/#habit-4-dont-reference-models).
465
+ Otherwise, you'll need to remember to run any ad hoc deployment scripts against
466
+ your `test_data` Rails environment along with each of your other deployed
467
+ environments.]_
468
+
469
+ ## Factory & Fixture Interoperability Guide
470
+
471
+ Let's be real, most Rails apps already have some tests, and most of those test
472
+ suites will already be relying on
473
+ [factory_bot](https://github.com/thoughtbot/factory_bot) or Rails' built-in
474
+ [test
475
+ fixtures](https://guides.rubyonrails.org/testing.html#the-low-down-on-fixtures).
476
+ While `test_data` is designed to be an alternative to both of these approaches
477
+ to managing your test data, it wouldn't be practical to ask a team to rewrite
478
+ all their existing tests in order to migrate to a different tool. That's why the
479
+ `test_data` gem goes to great lengths to play nicely with your existing tests,
480
+ while ensuring each test is wrapped in an isolated and fast always-rolled-back
481
+ transaction—regardless if the test depends on `test_data`, factories, fixtures,
482
+ all three, or none-of-the-above.
483
+
484
+ This section will hopefully make it a little easier to incorporate new
485
+ `test_data` tests into a codebase that's already using `factory_bot` and/or
486
+ Rails fixtures, whether you choose to incrementally migrate to using `test_data`
487
+ over time.
488
+
489
+ ### Using `test_data` with `factory_bot`
490
+
491
+ This section will document some thoughts and strategies for introducing
492
+ `test_data` to a test suite that's already using `factory_bot`.
493
+
494
+ #### Getting your factory tests passing after adding `test_data`
495
+
496
+ Depending on the assumptions your tests make about the state of the database
497
+ before you've loaded any factories, it's possible that everything will "just
498
+ work" after adding [TestData.uses_test_data](#testdatauses_test_data) in a
499
+ before-each hook (as shown in the [setup
500
+ guide](#step-4-load-your-data-in-your-tests)). So by all means, try running your
501
+ suite after following the initial setup guide and see if the suite just passes.
502
+
503
+ If you find that your test suite is failing after adding
504
+ `TestData.uses_test_data` to your setup, don't panic! Test failures are most
505
+ likely caused by the combination of your `test_data` SQL dump with the records
506
+ inserted by your factories.
507
+
508
+ One approach would be to attempt to resolve each such failure one-by-one—usually
509
+ by updating the offending factories or editing your `test_data` database to
510
+ ensure they steer clear of one another. Care should be taken to preserve the
511
+ conceptual encapsulation of each test, however, as naively squashing errors
512
+ risks introducing inadvertent coupling between your factories and your
513
+ `test_data` data such that neither can be used independently of the other.
514
+
515
+ Another approach that the `test_data` gem provides is an additional mode with
516
+ `TestData.uses_clean_slate`, which—when called at the top of a factory-dependent
517
+ test—will ensure that the tables that `test_data` had written to will be
518
+ truncated, allowing the test to create whatever factories it needs without fear
519
+ of conflicts.
520
+
521
+ ```ruby
522
+ class AnExistingFactoryUsingTest < ActiveSupport::Testcase
523
+ setup do
524
+ TestData.uses_clean_slate
525
+ # pre-existing setup
526
+ end
527
+ # …
528
+ end
529
+ ```
530
+
531
+ If you have a lot of tests, you can find a more sophisticated approaches for
532
+ logically switching between types of test data declaratively above in the
533
+ [getting started
534
+ section](#if-some-tests-rely-on-test-data-and-others-need-a-clean-slate)
535
+
536
+ ### Using `test_data` with Rails fixtures
537
+
538
+ While [Rails
539
+ fixtures](https://guides.rubyonrails.org/testing.html#the-low-down-on-fixtures)
540
+ are similar to factories, the fact that they're run globally by Rails and
541
+ permanently committed to the test database actually makes them a little trickier
542
+ to work with. This section will cover a couple approaches for integrating
543
+ `test_data` into suites that use fixtures.
544
+
545
+ It's more likely than not that all your tests will explode in dramatic fashion
546
+ as soon as you add `TestData.uses_test_data` to a `setup` or `before(:each)`
547
+ hook. Typically, your fixtures will be loaded and committed immediately with
548
+ your `test_data` dump inserted afterward, which makes it exceedingly likely that
549
+ your tests will fail with primary key and unique constraint conflicts. If that's
550
+ the case you find yourself in, `test_data` provides an API that **overrides
551
+ Rails' built-in fixtures behavior with a monkey patch**.
552
+
553
+ And if that bold text wasn't enough to scare you off, here's how to do
554
+ it:
555
+
556
+ 1. Before your tests have loaded (e.g. near the top of your test helper), call:
557
+ [TestData.prevent_rails_fixtures_from_loading_automatically!](#testdataprevent_rails_fixtures_from_loading_automatically)
558
+ This will patch Rails'
559
+ [setup_fixtures](https://github.com/rails/rails/blob/main/activerecord/lib/active_record/test_fixtures.rb#L105)
560
+ and effectively render it into a no-op, which means that your test fixtures
561
+ will not be automatically loaded into your test database
562
+
563
+ 2. In tests that rely on your `test_data` dump, call
564
+ [TestData.uses_test_data](#step-4-load-your-data-in-your-tests) as you
565
+ normally would. Because your fixtures won't be loaded automatically, they
566
+ won't be available to these tests
567
+
568
+ 3. In tests that need fixtures, call
569
+ [TestData.uses_rails_fixtures(self)](#testdatauses_rails_fixtures) in a
570
+ before-each hook. This will first ensure that any tables written to by
571
+ `test_data` are truncated (as with `TestData.uses_clean_slate`) before
572
+ loading your Rails fixtures
573
+
574
+ For example, you might add the following to an existing fixtures-dependent
575
+ test to get it passing:
576
+
577
+ ```ruby
578
+ class AnExistingFixtureUsingTest < ActiveSupport::Testcase
579
+ setup do
580
+ TestData.uses_rails_fixtures(self)
581
+ # pre-existing setup
582
+ end
583
+
584
+ # …
585
+ end
586
+ ```
587
+
588
+ If you've adopted a mode-switching helper method [like the one described
589
+ above](#if-some-tests-rely-on-test-data-and-others-need-a-clean-slate), you
590
+ could of course add a third mode to cover any tests that depend on Rails
591
+ fixtures.
311
592
 
312
593
  ## Rake Task Reference
313
594
 
@@ -326,23 +607,41 @@ This task runs several generators:
326
607
  too! This gem adds a new `test_data` environment and database that's intended
327
608
  to be used to create and dump your test data. This new environment file loads
328
609
  your `development` environment's configuration and disables migration schema
329
- dumps so that you can run migrations of your `test_data` database without
610
+ dumps so that you can run migrations against your `test_data` database without
330
611
  affecting your app's `schema.rb` or `structure.sql`.
331
612
 
332
- * `config/initializers/test_data.rb` - Calls [TestData.config](#testdataconfig)
333
- with an empty block and comments documenting the currently-available options
334
- and their default values
613
+ * `config/initializers/test_data.rb` - Creates an initializer for the gem that
614
+ calls [TestData.config](#testdataconfig) with an empty block and comments
615
+ documenting the currently-available options and their default values
335
616
 
336
617
  * `config/database.yml` - This generator adds a new `test_data` section to your
337
618
  database configuration, named with the same scheme as your other databases
338
619
  (e.g. `your_app_test_data`). If your configuration resembles Rails' generated
339
- database.yml and has a working `&default` alias, then this should "just work"
620
+ `database.yml` and has a working `&default` alias, then this should "just
621
+ work"
622
+
623
+ * `config/webpacker.yml` - The gem has nothing to do with web assets, but
624
+ [webpacker](https://github.com/rails/webpacker) will display some prominent
625
+ warnings or errors if it is loaded without a configuration entry for the
626
+ currently-running environment, so this generator defines an alias based on
627
+ your `development` config and then defines `test_data` as extending it
628
+
629
+ * `config/secrets.yml` - If your app still uses (the now-deprecated)
630
+ [secrets.yml](https://guides.rubyonrails.org/4_1_release_notes.html#config-secrets-yml)
631
+ file introduced in Rails 4.1, this generator will ensure that the `test_data`
632
+ environment is accounted for with a generated `secret_key_base` value. If you
633
+ have numerous secrets in this file's `development:` stanza, you may want to
634
+ alias and inherit it into `test_data:` like the `webpacker.yml` generator does
635
+
636
+ * `config/cable.yml` - Simply defines a `test_data:` entry that tells
637
+ [ActionCable](https://guides.rubyonrails.org/action_cable_overview.html) to
638
+ use the `async` adapter, since that's also the default for `development`
340
639
 
341
- * `config/webpacker.yml` - The gem has nothing to do with web assets or
342
- webpacker, but webpacker will display some prominent warnings or errors if it
343
- is loaded without a configuration entry for the currently-running environment,
344
- so this generator defines an alias based on your `development` config and then
345
- defines `test_data` as extending it
640
+ ### test_data:verify_config
641
+
642
+ This task will verify that your configuration appears to be valid by checking
643
+ with each of the gem's generators to inspect your configuration files, and will
644
+ error whenever a configuration problem is detected.
346
645
 
347
646
  ### test_data:initialize
348
647
 
@@ -353,22 +652,24 @@ your seed file. Specifically:
353
652
  1. Creates the `test_data` environment's database, if it doesn't already exist
354
653
 
355
654
  2. Ensures the database is non-empty to preserve data integrity (run
356
- `test_data:drop_database` first if it contains outdated test data)
655
+ [test_data:drop_database](#test_datadrop_database) first if you intend to
656
+ reinitialize it)
357
657
 
358
658
  3. Checks to see if a dump of the database already exists (by default, stored in
359
659
  `test/support/test_data/`)
360
660
 
361
- * If dumps do exist, it invokes `test_data:load` to load them into the
362
- database
661
+ * If dumps do exist, it invokes [test_data:load](#test_dataload) to load
662
+ them into the database
363
663
 
364
664
  * Otherwise, it invokes the task `db:schema:load` and `db:seed` (similar to
365
- the `db:setup` task)
665
+ Rails' built-in `db:setup` task)
366
666
 
367
667
  ### test_data:dump
368
668
 
369
669
  This task is designed to be run after you've created or updated your test data
370
- and you want to run tests against it. The task creates several plain SQL dumps
371
- from your `test_data` environment's database:
670
+ in the `test_data` database and you're ready to run your tests against it. The
671
+ task creates several plain SQL dumps from your `test_data` environment's
672
+ database:
372
673
 
373
674
  * A schema-only dump, by default in `test/support/test_data/schema.sql`
374
675
 
@@ -376,20 +677,20 @@ from your `test_data` environment's database:
376
677
  `test/support/test_data/data.sql`
377
678
 
378
679
  * A data-only dump of records that you *don't* want loaded in your tests in
379
- `test/support/test_data/non_test_data.sql` (by default, this includes Rails'
680
+ `test/support/test_data/non_test_data.sql`. By default, this includes Rails'
380
681
  internal tables: `ar_internal_metadata` and `schema_migrations`, configurable
381
- with [TestData.config](#testdataconfig)'s `non_test_data_tables`)
682
+ with [TestData.config](#testdataconfig)'s `non_test_data_tables`
382
683
 
383
684
  Each of these files are designed to be committed and versioned with the rest of
384
685
  your application. [TestData.config](#testdataconfig) includes several
385
- options to control which tables are exported into which group.
686
+ options to control this task.
386
687
 
387
688
  ### test_data:load
388
689
 
389
690
  This task will load your SQL dumps into your `test_data` database by:
390
691
 
391
692
  1. Verifying the `test_data` environment's database is empty (creating it if it
392
- doesn't exist)
693
+ doesn't exist and failing if it's not empty)
393
694
 
394
695
  2. Verifying that your schema, test data, and non-test data SQL dumps can be
395
696
  found at the configured paths
@@ -399,7 +700,7 @@ This task will load your SQL dumps into your `test_data` database by:
399
700
  4. Warning if there are pending migrations that haven't been run yet
400
701
 
401
702
  If there are pending migrations, you'll probably want to run them and then
402
- dump & commit your test data so that they're all up-to-date:
703
+ dump & commit your test data so that they're up-to-date:
403
704
 
404
705
  ```
405
706
  $ RAILS_ENV=test_data bin/rake db:migrate
@@ -409,230 +710,288 @@ $ bin/rake test_data:dump
409
710
  ### test_data:create_database
410
711
 
411
712
  This task will create the `test_data` environment's database if it does not
412
- already exist. It also enhances Rails' `db:create` task so that `test_data` is
413
- created along with `development` and `test`.
713
+ already exist. It also
714
+ [enhances](https://dev.to/molly/rake-task-enhance-method-explained-3bo0) Rails'
715
+ `db:create` task so that `test_data` is created along with `development` and
716
+ `test` whenever `rake db:create` is run.
414
717
 
415
718
  ### test_data:drop_database
416
719
 
417
720
  This task will drop the `test_data` environment's database if it exists. It also
418
721
  enhances Rails' `db:drop` task so that `test_data` is dropped along with
419
- `development` and `test`.
722
+ `development` and `test` whenever `rake db:drop` is run.
420
723
 
421
724
  ## API Reference
422
725
 
423
- ### TestData.config
726
+ ### TestData.uses_test_data
424
727
 
425
- The generated `config/initializers/test_data.rb` initializer will include a call
426
- to `TestData.config`, which takes a block that yields a mutable configuration
427
- object (similar to `Rails.application.config`):
728
+ This is the method designed to be used by your tests to load your test data
729
+ into your `test` database so that your tests can rely on it. Typically, you'll
730
+ want to call it at the beginning of each test that relies on the test data
731
+ managed by this gem—most often, in a before-each hook.
428
732
 
429
- ```ruby
430
- TestData.config do |config|
431
- # Where to store SQL dumps of the test_data database schema
432
- # config.schema_dump_path = "test/support/test_data/schema.sql"
733
+ For the sake of speed and integrity, `TestData.uses_test_data` is designed to
734
+ take advantage of nested transactions ([Postgres
735
+ savepoints](https://www.postgresql.org/docs/current/sql-savepoint.html)). By
736
+ default, data is loaded in a transaction and intended to be rolled back to the
737
+ point _immediately after_ the data was imported between tests. This way, your
738
+ test suite only pays the cost of importing the SQL file once, but each of your
739
+ tests can enjoy a clean slate that's free of data pollution from other tests.
740
+ (This is similar to, but separate from, Rails fixtures'
741
+ [use_transactional_tests](https://edgeguides.rubyonrails.org/testing.html#testing-parallel-transactions)
742
+ option.)
433
743
 
434
- # Where to store SQL dumps of the test_data database test data
435
- # config.data_dump_path = "test/support/test_data/data.sql"
744
+ _See configuration option:
745
+ [config.after_test_data_load](#configafter_test_data_load)_
436
746
 
437
- # Where to store SQL dumps of the test_data database non-test data
438
- # config.non_test_data_dump_path = "test/support/test_data/non_test_data.sql"
747
+ ### TestData.uses_clean_slate
439
748
 
440
- # Tables whose data shouldn't be loaded into tests.
441
- # ("ar_internal_metadata" and "schema_migrations" are always excluded)
442
- # config.non_test_data_tables = []
749
+ If a test does not rely on your `test_data` data, you can instead ensure that it
750
+ runs against empty tables by calling `TestData.uses_clean_slate`. Like
751
+ `TestData.uses_test_data`, this would normally be called at the beginning of
752
+ each such test in a before-each hook.
443
753
 
444
- # Tables whose data should be excluded from all dumps (does not affect schema DDL)
445
- # config.dont_dump_these_tables = []
754
+ This method works by first ensuring that your test data is loaded (and the
755
+ correspondent savepoint created), then will truncate all affected tables and
756
+ create another savepoint. It's a little counter-intuitive that you'd first
757
+ litter your database with data only to wipe it clean again, but it's much faster
758
+ to repeatedly truncate tables than to repeatedly import large SQL files.
446
759
 
447
- # Tables whose data should be truncated by TestData.truncate
448
- # If left as `nil`, all tables inserted into by the SQL file at
449
- # `data_dump_path` will be truncated
450
- # config.truncate_these_test_data_tables = nil
760
+ _See configuration options:
761
+ [config.after_test_data_truncate](#configafter_test_data_truncate),
762
+ [config.truncate_these_test_data_tables](#configtruncate_these_test_data_tables)_
451
763
 
452
- # Log level (valid values: [:debug, :info, :warn, :error, :quiet])
453
- # Can also be set with env var TEST_DATA_LOG_LEVEL
454
- # config.log_level = :info
455
- end
456
- ```
764
+ ### TestData.uses_rails_fixtures
457
765
 
458
- ### TestData.load
766
+ As described in this README's [fixture interop
767
+ guide](#using-test_data-with-rails-fixtures), `TestData.uses_rails_fixtures`
768
+ will load your app's [Rails
769
+ fixtures](https://guides.rubyonrails.org/testing.html#the-low-down-on-fixtures)
770
+ by intercepting Rails' built-in fixture-loading code. As with the other "uses"
771
+ methods, you'll likely want to call it in a before-each hook before any test
772
+ that needs access to your Rails fixtures.
459
773
 
460
- This is the method designed to be used by your tests to load your test data
461
- into your `test` database so that your tests can rely on it.
774
+ There are two additional things to keep in mind if using this method:
462
775
 
463
- #### Loading with the speed & safety of transaction savepoints
776
+ 1. Using this feature requires that you've first invoked
777
+ [TestData.prevent_rails_fixtures_from_loading_automatically!](#testdataprevent_rails_fixtures_from_loading_automatically)
778
+ to override Rails' default behavior before any of your tests have loaded or
779
+ started running
464
780
 
465
- For the sake of speed and integrity, `TestData.load` is designed to take
466
- advantage of nested transactions ([Postgres
467
- savepoints](https://www.postgresql.org/docs/current/sql-savepoint.html)). By
468
- default, data is loaded in a transaction and intended to be rolled back to the
469
- point _immediately after_ the data was imported after each test. This way, your
470
- test suite only pays the cost of importing the SQL file once, but each of your
471
- tests can enjoy a clean slate free of potential data pollution from prior tests.
472
- (This is similar to, but separate from, Rails fixtures'
473
- [use_transactional_tests](https://edgeguides.rubyonrails.org/testing.html#testing-parallel-transactions)
474
- option.)
781
+ 2. Because the method depends on Rails' fixture caching mechanism, it must be
782
+ passed an instance of the running test class (e.g.
783
+ `TestData.uses_rails_fixtures(self)`)
475
784
 
476
- To help think through the method's behavior, the method nicknames its
477
- transactions `:before_data_load` and `:after_data_load`. The first time you call
478
- `TestData.load`:
785
+ Under the hood, this method effectively ensures a clean slate the same way
786
+ `TestData.uses_clean_slate` does, except that after creating the truncation
787
+ savepoint, it will then load your fixtures and finally create—wait for it—yet
788
+ another savepoint that subsequent calls to `uses_rails_fixtures` can rollback
789
+ to.
479
790
 
480
- 1. Starts the `:before_data_load` transaction
481
- 2. Executes the SQL found in the data dump (e.g.
482
- `test/support/test_data/data.sql`) to insert your test data
483
- 3. Starts the `:after_data_load` transaction
791
+ _See configuration option:
792
+ [config.after_rails_fixture_load](#configafter_rails_fixture_load)_
484
793
 
485
- If the method is called and the `:after_data_load` savepoint is already active
486
- (indicating that the data is loaded), the method rolls back to
487
- `:after_data_load`, inferring that the user's intention is to have a clean load
488
- of the test data.
794
+ #### TestData.prevent_rails_fixtures_from_loading_automatically!
489
795
 
490
- _[As an additional safeguard, in case a rollback is triggered unexpectedly (i.e.
491
- calling `rollback_transaction` on `ActiveRecord::Base.connection` instead of via
492
- `TestData.rollback`), `test_data` writes a memo that the data is loaded in
493
- `ar_internal_metadata`. `TestData.load` uses this memo to detect this issue and
494
- will recreate the `:after_data_load` savepoint rather than attempt to
495
- erroneously reload your SQL data dump.]_
796
+ Call this method before any tests have been loaded or executed by your test
797
+ runner if you're planning to use
798
+ [TestData.uses_rails_fixtures](#testdatauses_rails_fixtures) to load Rails
799
+ fixtures into any of your tests. This method will disable the default behavior
800
+ of loading your Rails fixtures into the test database as soon as the first test
801
+ case with fixtures enabled is executed. (Inspect the [source for the
802
+ patch](/lib/test_data/active_record_ext.rb) to make sure you're comfortable with
803
+ what it's doing.)
496
804
 
497
- #### Loading without transactions
805
+ ### TestData.config
498
806
 
499
- For most cases, we strongly recommend using the default transactional testing
500
- strategy, both because it's faster and because it reduces the risk of test
501
- pollution. However, you may need to commit your test data if the data needs to
502
- be loaded by multiple processes or over multiple connections.
807
+ The generated `config/initializers/test_data.rb` initializer will include a call
808
+ to `TestData.config`, which takes a block that yields a mutable configuration
809
+ object (similar to `Rails.application.config`). If anything is unclear after
810
+ reading the documentation, feel free to review the
811
+ [initializer](lib/generators/test_data/initializer_generator.rb) and the [Config
812
+ class](/lib/test_data/config.rb) themselves.
813
+
814
+ #### Lifecycle hooks
815
+
816
+ Want to shift forward several timestamp fields after your `test_data` SQL dumps
817
+ are loaded into your test database? Need to refresh a materialized view after
818
+ your Rails fixtures are loaded? You _could_ do these things after calling
819
+ `TestData.uses_test_data` and `TestData.uses_rails_fixtures`, respectively, but
820
+ you'd take the corresponding performance hit in each and every test.
503
821
 
504
- If you need to load the test data and commit it to the database, simply call
505
- `TestData.load(transactions: false)`.
822
+ Instead, you can pass a callable or a block and `test_data` will execute it just
823
+ _after_ performing the associated data operation but just _before_ creating a
824
+ transaction savepoint. That way, whenever the gem rolls back between tests, your
825
+ hook won't need to be run again.
506
826
 
507
- Once committed, figuring out when and how to clear it out after each test run
508
- then becomes an additional responsibility. Many folks use
509
- [database_cleaner](https://github.com/DatabaseCleaner/database_cleaner) for
510
- this, while this gem offers a rudimentary
511
- [TestData.truncate](https://github.com/testdouble/test_data#testdatatruncate)
512
- method that can be passed `transactions: false` and which may be sufficient.
827
+ ##### config.after_test_data_load
828
+
829
+ This is hook is run immediately after `TestData.uses_test_data` has loaded your
830
+ SQL dumps into the `test` database, but before creating a savepoint. Takes a
831
+ block or anything that responds to `call`.
513
832
 
514
- You might imagine something like this if you were loading the data just once for
515
- the full run of a test suite:
516
833
 
517
834
  ```ruby
518
- RSpec.configure do |config|
519
- config.before :all do
520
- TestData.load(transactions: false)
835
+ TestData.config do |config|
836
+ # Example: roll time forward
837
+ config.after_test_data_load do
838
+ Boop.connection.exec_update(<<~SQL, nil, [[nil, Time.zone.now - System.epoch]])
839
+ update boops set booped_at = booped_at + $1
840
+ SQL
521
841
  end
842
+ end
843
+ ```
844
+
845
+ ##### config.after_test_data_truncate
846
+
847
+ This is hook is run immediately after `TestData.uses_clean_slate` has truncated
848
+ your test data, but before creating a savepoint. Takes a block or anything that
849
+ responds to `call`.
850
+
851
+ ```ruby
852
+ TestData.config do |config|
853
+ # Example: pass a callable instead of a block
854
+ config.after_test_data_truncate(SomethingThatRespondsToCall.new)
855
+ end
856
+ ```
522
857
 
523
- config.after :all do
524
- TestData.truncate(transactions: false)
858
+ ##### config.after_rails_fixture_load
859
+
860
+ This is hook is run immediately after `TestData.uses_rails_fixtures` has loaded
861
+ your Rails fixtures into the `test` database, but before creating a savepoint.
862
+ Takes a block or anything that responds to `call`.
863
+
864
+ ```ruby
865
+ TestData.config do |config|
866
+ # Example: refresh Postgres assets like materialized views
867
+ config.after_rails_fixture_load do
868
+ RefreshesMaterializedViews.new.call
525
869
  end
526
870
  end
527
871
  ```
528
872
 
529
- Note that subsequent `TestData.load(transactions: false)` calls won't be able to
530
- detect whether the data is already loaded and will try to re-insert the data,
531
- which will almost certainly result in primary key conflicts.
873
+ #### test_data:dump options
532
874
 
533
- ### TestData.rollback
875
+ The gem provides several options governing the behavior of the
876
+ [test_data:dump](#test_datadump) Rake task. You probably won't need to set these
877
+ unless you run into a problem with the defaults.
534
878
 
535
- Because the gem loads your data in a transaction, it makes it easy to rollback
536
- to any of its defined savepoints (`:before_data_load`, `:after_data_load`, and
537
- `:after_data_truncate`). In most cases you'll want to roll back to
538
- `:after_data_load` after each test, and that's what `TestData.rollback` will do
539
- when called without an argument. More details on rolling back to each of the
540
- gem's savepoints follows below.
879
+ ##### config.non_test_data_tables
541
880
 
542
- #### Rolling back to after the data was loaded
881
+ Your application may have some tables that are necessary for the operation of
882
+ the application, but irrelevant or incompatible with you your tests. This data
883
+ is still dumped for the sake of being able to restore the database with [rake
884
+ test_data:load](#test_dataload), but will not be loaded when your tests are
885
+ running. Defaults to `[]`, (but will always include `ar_internal_metadata` and
886
+ `schema_migrations`).
543
887
 
544
- When `TestData.rollback` is passed no arguments or called more explicitly as
545
- `TestData.rollback(:after_data_load)`, the method will rollback to the
546
- `:after_data_load` transaction savepoint taken immediately after the SQL dump
547
- was loaded. As a result, it is intended to be run after each test (e.g. in an
548
- `after(:each)` or `teardown`), to undo any changes made by the test.
888
+ ```ruby
889
+ TestData.config do |config|
890
+ config.non_test_data_tables = []
891
+ end
892
+ ```
549
893
 
550
- (Calling `TestData.rollback` when no `:after_data_load` save point is active is
551
- a no-op.)
894
+ ##### config.dont_dump_these_tables
552
895
 
553
- #### Rolling back to before test data was loaded
896
+ Some tables populated by your application may not be necessary to either its
897
+ proper functioning or useful to your tests (e.g. audit logs), so you can save
898
+ time and storage by preventing those tables from being dumped entirely. Defaults
899
+ to `[]`.
554
900
 
555
- If some tests rely on data loaded by `TestData.load` and you have other tests
556
- that depend on that data _not being there_, you probably want to call
557
- [TestData.truncate](#testdatatruncate). But if that won't work for your needs,
558
- you can rewind to the moment just before your test data was loaded by calling
559
- `TestData.rollback(:before_data_load)`.
901
+ ```ruby
902
+ TestData.config do |config|
903
+ config.dont_dump_these_tables = []
904
+ end
905
+ ```
560
906
 
561
- (Calling `TestData.rollback` when no `:before_data_load` save point is active is
562
- a no-op.)
907
+ ##### config.schema_dump_path
563
908
 
564
- **⚠️ Warning:** Repeatedly loading and rolling back to `:before_data_load` is
565
- expensive! If your test suite calls `TestData.rollback(:before_data_load)`
566
- multiple times, it's likely you're re-loading your (possibly large) SQL file of
567
- test data more times than is necessary. Consider using
568
- [TestData.truncate](#testdatatruncate) to achieve the same goal with faster
569
- performance. Failing that, it might be preferable to partition your test suite
570
- so that similar tests are run in separate groups (as opposed to in a fully
571
- random or arbitrary order) to avoid repeatedly thrashing between rollbacks and
572
- reloads. This partitioning could be accomplished by either configuring your test
573
- runner or by running separate test commands for each group of tests.
909
+ The path to which the schema DDL of your `test_data` database will be written.
910
+ This is only used by [rake test_data:load](#test_dataload) when initializing the
911
+ `test_data` database. Defaults to `"test/support/test_data/schema.sql"`.
574
912
 
575
- #### Rolling back to after test data was truncated
913
+ ```ruby
914
+ TestData.config do |config|
915
+ config.schema_dump_path = "test/support/test_data/schema.sql"
916
+ end
917
+ ```
576
918
 
577
- If some of your tests call [TestData.truncate](#testdatatruncate) to clear out
578
- your test data after it's been loaded, then you will likely want to run
579
- `TestData.rollback(:after_data_truncate)` after each of them. This will rewind
580
- your test database's state to when those tables were first truncated—effectively
581
- re-cleaning the slate for the next test.
919
+ ##### config.data_dump_path
582
920
 
583
- (Calling `TestData.rollback` when no `:after_data_truncate` save point is active
584
- is a no-op.)
921
+ The path that the SQL dump of your test data will be written. This is the dump
922
+ that will be executed by `TestData.uses_test_data` in your tests. Defaults to
923
+ `"test/support/test_data/data.sql"`.
585
924
 
586
- ### TestData.truncate
925
+ ```ruby
926
+ TestData.config do |config|
927
+ config.data_dump_path = "test/support/test_data/data.sql"
928
+ end
929
+ ```
587
930
 
588
- Do you have some tests that _shouldn't_ access your test data? Or did some
589
- existing tests started failing after `test_data` was added? If you want to reset
590
- the state of your `test` database to support these tests, you have two options:
591
- (1) `TestData.rollback(:before_data_load)` and (2) `TestData.truncate`. As
592
- discussed above, the former will do the job, but may necessitate repetitive,
593
- slow reloads of your test data SQL file. `TestData.truncate`, meanwhile,
594
- truncates all the tables that `TestData.load` inserted into and then creates
595
- a savepoint nicknamed `:after_data_truncate`.
931
+ ##### config.non_test_data_dump_path
596
932
 
597
- Most often, you'll want to call `TestData.truncate` before each test that
598
- should _not_ have access to your test data created with this gem. After each
599
- such test, it can clean up by calling `TestData.rollback(:after_data_truncate)`:
933
+ The path to which the [non_test_data_tables](#confignon_test_data_tables) in
934
+ your `test_data` database will be written. This is only used by [rake
935
+ test_data:load](#test_dataload) when initializing the `test_data` database.
936
+ Defaults to `"test/support/test_data/non_test_data.sql"`.
600
937
 
601
938
  ```ruby
602
- class CleanSlateTest < ActiveDispatch::IntegrationTest
603
- def setup do
604
- TestData.truncate
605
- end
939
+ TestData.config do |config|
940
+ config.non_test_data_dump_path = "test/support/test_data/non_test_data.sql"
941
+ end
942
+ ```
606
943
 
607
- def teardown do
608
- TestData.rollback(:after_data_truncate)
609
- end
944
+ #### Other configuration options
945
+
946
+ ##### config.truncate_these_test_data_tables
947
+
948
+ By default, when [TestData.uses_clean_slate](#testdatauses_clean_slate) is
949
+ called, it will truncate any tables for which an `INSERT` operation was
950
+ detected in your test data SQL dump. This may not be suitable for every case,
951
+ however, so this option allows you to specify which tables are truncated.
952
+ Defaults to `nil`.
953
+
954
+ ```ruby
955
+ TestData.config do |config|
956
+ config.truncate_these_test_data_tables = []
610
957
  end
611
958
  ```
612
959
 
613
- By default, all tables for which there is an `INSERT INTO` statement in your
614
- test data SQL dump will be truncated, but you can specify which tables should be
615
- truncated yourself by setting the `truncate_these_test_data_tables` property on
616
- [TestData.config](#testdataconfig) to an array of table names.
960
+ ##### config.log_level
617
961
 
618
- Because tests are usually run in a random order, some fault tolerance is built
619
- in:
962
+ The gem outputs its messages to standard output and error by assigning a log
963
+ level to each message. Valid values are `:debug`, `:info`, `:warn`, `:error`,
964
+ `:quiet`. Defaults to `:info`.
965
+
966
+ ```ruby
967
+ TestData.config do |config|
968
+ config.log_level = :info
969
+ end
970
+ ```
620
971
 
621
- * If one test calls `TestData.rollback(:after_data_truncate)` in `teardown`, and
622
- the next test calls `TestData.load` in `setup`, the gem will do the
623
- probably-intended thing and rollback to `:after_data_load` so that the data is
624
- available
972
+ ### TestData.insert_test_data_dump
625
973
 
626
- * If a test calls `TestData.truncate` and the `:after_data_truncate` savepoint
627
- is rolled back outside the `TestData.rollback` method (e.g.
628
- `ActiveRecord::Base.connection.rollback_transaction`), the method will first
629
- check a memo written to `ar_internal_metadata` and recreate the
630
- `:after_data_truncate` savepoint
974
+ If you just want to insert the test data in your application's SQL dumps without
975
+ any of the transaction management or test runner assumptions inherent in
976
+ [TestData.uses_test_data](#testdatauses_test_data), then you can call
977
+ `TestData.insert_test_data_dump` to load and execute the dump.
631
978
 
632
- #### If you're not using transactions
979
+ This might be necessary in a few different situations:
633
980
 
634
- Just [like TestData.load](#loading-without-transactions), you can call
635
- `TestData.truncate(transactions: false)` and it'll commit the truncation.
981
+ * Running tests in environments that can't be isolated to a single database
982
+ transaction (e.g. orchestrating tests across multiple databases, processes,
983
+ etc.)
984
+ * You might ant to use your test data to seed pre-production environments with
985
+ enough data to exploratory test (as you might do in a `postdeploy` script with
986
+ your [Heroku Review
987
+ Apps](https://devcenter.heroku.com/articles/github-integration-review-apps))
988
+ * Your tests require complex heterogeneous sources of data that aren't a good
989
+ fit for the assumptions and constraints of this library's default methods for
990
+ preparing test data
991
+
992
+ In any case, since `TestData.insert_test_data_dump` is not wrapped in a
993
+ transaction, when used for automated tests, data cleanup becomes your
994
+ responsibility.
636
995
 
637
996
  ## Assumptions
638
997
 
@@ -643,15 +1002,16 @@ yet. Here are some existing assumptions and limitations:
643
1002
 
644
1003
  * You're using Rails 6 or higher
645
1004
 
646
- * Your app does not rely on Rails' [multi-database
1005
+ * Your app does not require Rails' [multi-database
647
1006
  support](https://guides.rubyonrails.org/active_record_multiple_databases.html)
1007
+ in order to be tested
648
1008
 
649
1009
  * Your app has the binstubs `bin/rake` and `bin/rails` that Rails generates and
650
- they work (you can regenerate them with `rails app:update:bin`)
1010
+ they work (protip: you can regenerate them with `rails app:update:bin`)
651
1011
 
652
1012
  * Your `database.yml` defines a `&default` alias from which to extend the
653
- `test_data` database configuration (if it lacks one, you'll need specify the
654
- database's configuration on your own)
1013
+ `test_data` database configuration (if your YAML file lacks one, you can
1014
+ always specify the `test_data` database configuration manually)
655
1015
 
656
1016
  ## Fears, Uncertainties, and Doubts
657
1017
 
@@ -659,12 +1019,11 @@ yet. Here are some existing assumptions and limitations:
659
1019
 
660
1020
  If you use `factory_bot` and all of these are true:
661
1021
 
662
- * Your integration tests are super fast and not getting significantly slower
1022
+ * Your integration tests are super fast and are not getting significantly slower
663
1023
  over time
664
1024
 
665
- * Innocuous changes to factories rarely result in unrelated test failures
666
- that—rather than indicating a bug in the production code—instead require that
667
- each of those tests be analyzed & updated to get them passing again
1025
+ * Minor changes to existing factories rarely result in test failures that
1026
+ require unrelated tests to be read & updated to get them passing again
668
1027
 
669
1028
  * The number of associated records generated between your most-used factories
670
1029
  are representative of production data, as opposed to generating a sprawling
@@ -672,7 +1031,7 @@ If you use `factory_bot` and all of these are true:
672
1031
  menu
673
1032
 
674
1033
  * Your default factories generate models that resemble real records created by
675
- your production application, as opposed to resembling the
1034
+ your production application, as opposed to representing the
676
1035
  sum-of-all-edge-cases with every boolean flag enabled and optional attribute
677
1036
  set
678
1037
 
@@ -680,17 +1039,17 @@ If you use `factory_bot` and all of these are true:
680
1039
  confidence-eroding nested factories with names like `:user`, `:basic_user`,
681
1040
  `:lite_user`, and `:plain_user_no_associations_allowed`
682
1041
 
683
- If none of these things are true, then congratulations! You are using
684
- `factory_bot` with great success! Unfortunately, in our experience, this outcome
1042
+ If none of these things are true, then congratulations! You are probably using
1043
+ `factory_bot` to great effect! Unfortunately, in our experience, this outcome
685
1044
  is exceedingly rare, especially for large and long-lived applications.
686
1045
 
687
- However, if any of the above issues resonate with your experience using
688
- `factory_bot`: these are the sorts of failure modes the `test_data` gem was
689
- designed to address. We hope you'll consider trying it with an open mind. At the
690
- same time, we acknowledge that large test suites can't be rewritten and migrated
691
- to a different source of test data overnight—nor should they be! See our notes
692
- on [migrating to `test_data`
693
- incrementally](#we-already-have-thousands-of-tests-that-depend-on-rails-fixtures-or-factory_bot-can-we-start-using-test_data-without-throwing-them-away-and-starting-over)
1046
+ However, if you'd answer "no" to any of the above questions, just know that
1047
+ these are the sorts of failure modes the `test_data` gem was designed to
1048
+ avoid—and we hope you'll consider trying it with an open mind. At the same time,
1049
+ we acknowledge that large test suites can't be rewritten and migrated to a
1050
+ different source of test data overnight—nor should they be! See our notes on
1051
+ [migrating to `test_data`
1052
+ incrementally](#factory--fixture-interoperability-guide)
694
1053
 
695
1054
  ### How will I handle merge conflicts in these SQL files if I have lots of people working on lots of feature branches all adding to the `test_data` database dumps?
696
1055
 
@@ -709,131 +1068,13 @@ this risk. The reason that the dumps are stored as plain SQL (aside from the
709
1068
  fact that git's text compression is very good) is to make merge conflicts with
710
1069
  other branches feasible, if not entirely painless.
711
1070
 
712
- However, if your app is in the very initial stages of development and you're
713
- making breaking changes to your schema very frequently, our best advice is to
714
- hold off a bit on writing _any_ integration tests that depend on shared sources
715
- of test data, as they'll be more likely to frustrate your ability to rapidly
716
- iterate than detect bugs. Once you you have a reasonably stable feature working
717
- end-to-end, that's a good moment to start adding integration tests (and thus
718
- pulling in a test data gem like this one to help you).
719
-
720
- ### We already have thousands of tests that depend on Rails fixtures or factory_bot; can we start using this gem without throwing them away and starting over?
721
-
722
- Yes, `test_data` was written with this sort of incremental transition in mind!
723
-
724
- Our strongest recommendation is to write each test against only a single source
725
- of test data. Integration tests inevitably become coupled to the data that's
726
- available to them, and if a test has access to fixtures, records created by a
727
- factory, and a `test_data` SQL dump, it is likely to unintentionally become
728
- dependent on all three. This could result in the test having more ways to fail
729
- than necessary and make it harder to simplify your test data strategy later.
730
- Instead, consider explicitly opting into a single type of test data for each
731
- test.
732
-
733
- The rest comes down to how you want to organize your tests, as discussed below.
734
-
735
- #### Approach 1: test-data-specific test subclasses
736
-
737
- If you're currently using `factory_bot`, you might already have something like
738
- this in your `test_helper.rb` file:
739
-
740
- ```ruby
741
- class ActiveSupport::TestCase
742
- include FactoryBot::Syntax::Methods
743
- end
744
- ```
745
-
746
- To avoid including these methods with every Rails-aware test, you might consider
747
- pushing down new classes for each source of test data:
748
-
749
- ```ruby
750
- # Tests using data created by `test_data`
751
- class TestDataTestCase < ActiveSupport::TestCase
752
- def setup
753
- TestData.load
754
- end
755
-
756
- def teardown
757
- TestData.rollback
758
- end
759
- end
760
-
761
- # Tests using data created by `factory_bot`
762
- class FactoryBotTestCase < ActiveSupport::TestCase
763
- include FactoryBot::Syntax::Methods
764
-
765
- def setup
766
- TestData.truncate
767
- end
768
-
769
- def teardown
770
- TestData.rollback(:after_data_truncate)
771
- end
772
- end
773
- ```
774
-
775
- From there, the class that each test extends will indicate which test data
776
- strategy it uses.
777
-
778
- This approach may work in simple cases, but won't be well-suited to if you're
779
- already extending any of the half-dozen subclasses of `ActiveSupport::TestCase`
780
- provided by Rails (or their analogues from
781
- [rspec-rails](https://github.com/rspec/rspec-rails)).
782
-
783
- #### Approach 2: write a helper method to set things up for you
784
-
785
- Every situation will be different, but one strategy that suits a lot of
786
- circumstances would be to write a method to declare and configure the test data
787
- strategy for the current test.
788
-
789
- Taking from [this
790
- example](/example/test/integration/better_mode_switching_demo_test.rb) test, you
791
- could implement a class method like this:
792
-
793
- ```ruby
794
- class ActiveSupport::TestCase
795
- def self.test_data_mode(mode)
796
- case mode
797
- when :factory_bot
798
- require "factory_bot_rails"
799
- include FactoryBot::Syntax::Methods
800
-
801
- setup do
802
- TestData.truncate
803
- end
804
-
805
- teardown do
806
- TestData.rollback(:after_data_truncate)
807
- end
808
- when :test_data
809
- setup do
810
- TestData.load
811
- end
812
-
813
- teardown do
814
- TestData.rollback
815
- end
816
- end
817
- end
818
- end
819
- ```
820
-
821
- And then (without any class inheritance complications), simply declare which
822
- kind of test you're specifying:
823
-
824
- ```ruby
825
- class SomeFactoryUsingTest < ActiveSupport::TestCase
826
- test_data_mode :factory_bot
827
-
828
- # … tests go here
829
- end
830
-
831
- class SomeTestDataUsingTest < ActionDispatch::IntegrationTest
832
- test_data_mode :test_data
833
-
834
- # etc.
835
- end
836
- ```
1071
+ However, if your app is in the very initial stages of development or you're
1072
+ otherwise making breaking changes to your schema and data very frequently, our
1073
+ best advice is to hold off a bit on writing _any_ integration tests that depend
1074
+ on shared sources of test data (regardless of tool), as they'll be more likely
1075
+ to frustrate your ability to rapidly iterate than detect bugs. Once you you have
1076
+ a reasonably stable feature working end-to-end, that's a good moment to start
1077
+ adding integration tests—and perhaps pulling in a gem like this one to help you.
837
1078
 
838
1079
  ### Why can't I save multiple database dumps to cover different scenarios?
839
1080
 
@@ -849,25 +1090,28 @@ By having a single `test_data` database that grows up with your application just
849
1090
  like `production` does—with both having their schemas and data migrated
850
1091
  incrementally over time—your integration tests that depend on `test_data` will
851
1092
  have an early opportunity to catch bugs that otherwise wouldn't be found until
852
- they were deployed into a long-lived environment like staging or (gasp!)
853
- production itself.
1093
+ they were deployed into a long-lived staging or (gasp!) production environment.
854
1094
 
855
1095
  ### Are you sure I should commit these SQL dumps? They're way too big!
856
1096
 
857
- If the dump files generated by `test_data:dump` are absolutely massive, consider the cause:
1097
+ If the dump files generated by `test_data:dump` seem massive, consider the
1098
+ cause:
858
1099
 
859
1100
  1. If you inadvertently created more data than necessary, you might consider
860
1101
  resetting (or rolling back) your changes and making another attempt at
861
1102
  generating a more minimal set of test data
862
1103
 
863
- 2. If certain tables have a lot of records but aren't very relevant to your
864
- tests (e.g. audit logs), you might consider either of these options:
1104
+ 2. If some records persisted by your application aren't very relevant to your
1105
+ tests, you might consider either of these options:
865
1106
 
866
- * Add those tables to the `config.non_test_data_tables` configuration array,
867
- where they'd still be committed to git, but won't loaded by your tests
1107
+ * If certain tables are necessary for running the app but aren't needed by
1108
+ your tests, you can add them to the `config.non_test_data_tables`
1109
+ configuration array. They'll still be committed to git, but won't loaded
1110
+ by your tests
868
1111
 
869
- * Exclude data from those tables entirely by adding them to the
870
- `config.dont_dump_these_tables` array
1112
+ * If the certain tables are not needed by your application or by your tests
1113
+ (e.g. audit logs), add them to the `config.dont_dump_these_tables` array,
1114
+ and they won't be persisted by `rake test_data:dump`
871
1115
 
872
1116
  3. If the dumps are _necessarily_ really big (some apps are complex!), consider
873
1117
  looking into [git-lfs](https://git-lfs.github.com) for tracking them without
@@ -890,7 +1134,7 @@ test data loaded from this gem or any other:
890
1134
  def test_exclude_cancelled_orders
891
1135
  good_order = Order.new
892
1136
  bad_order = Order.new(cancelled: true)
893
- user = User.create!(orders: good_order, bad_order)
1137
+ user = User.create!(orders: [good_order, bad_order])
894
1138
 
895
1139
  result = user.active_orders
896
1140
 
@@ -899,27 +1143,30 @@ def test_exclude_cancelled_orders
899
1143
  end
900
1144
  ```
901
1145
 
902
- This test is simple, self-contained, clearly denotes
903
- [arrange-act-assert](https://github.com/testdouble/contributing-tests/wiki/Arrange-Act-Assert),
904
- and (most importantly) will only fail if the functionality stops working.
905
- Maximizing the number of tests that can be written expressively and succinctly
906
- without the aid of an external helper is a laudable goal that more teams should
907
- embrace.
1146
+ This test is simple, self-contained, clearly demarcates the
1147
+ [arrange-act-assert](https://github.com/testdouble/contributing-tests/wiki/Arrange-Act-Assert)
1148
+ phases, and (most importantly) will only fail if the functionality stops
1149
+ working. Maximizing the number of tests that can be written expressively and
1150
+ succinctly without the aid of shared test data is a laudable goal that more
1151
+ teams should embrace.
908
1152
 
909
1153
  However, what if the code you're writing doesn't need 3 records in the database,
910
- but 30? Writing that much test setup would be painstaking and—despite being
911
- fully-encapsulated—hard for readers to understand what's going on. At that
912
- point, you have two options:
1154
+ but 30? Writing that much test setup would be painstaking, despite being
1155
+ fully-encapsulated. Long test setup is harder for others to read and understand.
1156
+ And because that setup depends on more of your system's code, it will have more
1157
+ reasons to break as your codebase changes. At that point, you have two options:
913
1158
 
914
1159
  1. Critically validate your design: why is it so hard to set up? Does it
915
1160
  _really_ require so much persisted data to exercise this behavior? Would a
916
- plain old Ruby object that defined a pure function have been feasible? Could
917
- a model instance or even a `Struct` be passed to the
1161
+ [plain old Ruby
1162
+ object](https://steveklabnik.com/writing/the-secret-to-rails-oo-design) that
1163
+ defined a pure function have been feasible? Could a model instance or even a
1164
+ `Struct` be passed to the
918
1165
  [subject](https://github.com/testdouble/contributing-tests/wiki/Subject)
919
1166
  instead of loading everything from the database? When automated testing is
920
1167
  saved for the very end of a feature's development, it can feel too costly to
921
- reexamine design decisions like this, but it's valuable feedback all the
922
- same. *Easy to test code is easy to use code*
1168
+ reexamine design decisions like this, but it can be valuable to consider all
1169
+ the same. *Easy to test code is easy to use code*
923
1170
 
924
1171
  2. If the complex setup is a necessary reality of the situation that your app
925
1172
  needs to handle (and it often will be!), then having _some_ kind of shared
@@ -928,38 +1175,31 @@ point, you have two options:
928
1175
 
929
1176
  As a result, there is no one-size-fits-all approach. Straightforward behavior
930
1177
  that can be invoked with a clear, concise test has no reason to be coupled to a
931
- shared source of test data. Subtle behavior that requires lots of data to be
932
- exercised would see its tests grow unwieldy without something to help populate
933
- that data. So both kinds of test clearly have their place.
1178
+ shared source of test data. Meanwhile, tests of more complex behaviors that
1179
+ require lots of carefully-arranged data might be unmaintainable without a shared
1180
+ source of test data to lean on. So both kinds of test clearly have their place.
934
1181
 
935
1182
  But this is a pretty nuanced discussion that can be hard to keep in mind when
936
1183
  under deadline pressure or on a large team where building consensus around norms
937
1184
  is challenging. As a result, leaving the decision of which type of test to write
938
1185
  to spur-of-the-moment judgment is likely to result in inconsistent test design.
939
1186
  Instead, you might consider separating these two categories into separate test
940
- types or suites.
1187
+ types or suites, with simple heuristics to determine which types of code demand
1188
+ which type of test.
941
1189
 
942
1190
  For example, it would be completely reasonable to load this gem's test data for
943
1191
  integration tests, but not for basic tests of models, like so:
944
1192
 
945
1193
  ```ruby
946
1194
  class ActionDispatch::IntegrationTest
947
- def setup
948
- TestData.load
949
- end
950
-
951
- def teardown
952
- TestData.rollback
1195
+ setup do
1196
+ TestData.uses_test_data
953
1197
  end
954
1198
  end
955
1199
 
956
1200
  class ActiveSupport::TestCase
957
- def setup
958
- TestData.truncate
959
- end
960
-
961
- def teardown
962
- TestData.rollback(:after_data_truncate)
1201
+ setup do
1202
+ TestData.uses_clean_slate
963
1203
  end
964
1204
  end
965
1205
  ```
@@ -968,6 +1208,129 @@ In short, this skepticism is generally healthy, and encapsulated tests that
968
1208
  forego reliance on shared sources of test data should be maximized. For
969
1209
  everything else, there's `test_data`.
970
1210
 
1211
+ ### I'm worried my tests aren't as fast as they should be
1212
+
1213
+ The `test_data` gem was written to enable tests that are not only more
1214
+ comprehensible and maintainable over the long-term, but also _much faster_ to
1215
+ run. That said—and especially if you're adding `test_data` to an existing test
1216
+ suite—care should be taken to audit everything the suite does between tests in
1217
+ order to optimize its overall runtime.
1218
+
1219
+ #### Randomized test order leading to data churn
1220
+
1221
+ Generally speaking, randomizing the order in which tests run is an unmitigated
1222
+ win: randomizing helps you catch any unintended dependency between two tests
1223
+ early, when it's still cheap & easy to fix. However, if your tests use different
1224
+ sources of test data (e.g. some call `TestData.uses_test_data` and some call
1225
+ `TestData.uses_clean_slate`), it's very likely that randomizing your tests will
1226
+ result in a significantly slower overall test suite. Instead, if you group tests
1227
+ that use the same type of test data together (e.g. by separating them into
1228
+ separate suites), you might find profound speed gains.
1229
+
1230
+ To illustrate why, suppose you have 5 tests that call `TestData.uses_test_data`
1231
+ and 5 that call `TestData.uses_rails_fixtures`. If a test that calls
1232
+ `TestData.uses_test_data` is followed by another that calls `uses_test_data`,
1233
+ the only operation needed by the second call will be a rollback to the savepoint
1234
+ taken after the test data was loaded. If, however, a `uses_test_data` test is
1235
+ followed by a `uses_rails_fixtures` test, then a lot more work is required:
1236
+ first a rollback, then the truncation of the test data, then a load of the
1237
+ fixtures followed by creation of a new savepoint—which would in tunr be undone
1238
+ again if the _next_ test happened to call `uses_test_data`. Switching between
1239
+ tests that use different sources of test data can cause significant unnecessary
1240
+ thrashing.
1241
+
1242
+ To illustrate the above, if all of these tests ran in random order (the
1243
+ default), you might see:
1244
+
1245
+ ```
1246
+ $ bin/rails test test/example_test.rb
1247
+ Run options: --seed 63999
1248
+
1249
+ # Running:
1250
+
1251
+ test_data -- loading test_data SQL dump
1252
+ . fixtures -- truncating tables, loading Rails fixtures
1253
+ . fixtures -- rolling back to Rails fixtures
1254
+ . test_data -- rolling back to clean test_data
1255
+ . fixtures -- truncating tables, loading Rails fixtures
1256
+ . test_data -- rolling back to clean test_data
1257
+ . fixtures -- truncating tables, loading Rails fixtures
1258
+ . test_data -- rolling back to clean test_data
1259
+ . fixtures -- truncating tables, loading Rails fixtures
1260
+ . test_data -- rolling back to clean test_data
1261
+ .
1262
+
1263
+ Finished in 2.449957s, 4.0817 runs/s, 4.0817 assertions/s.
1264
+ 10 runs, 10 assertions, 0 failures, 0 errors, 0 skips
1265
+ ```
1266
+
1267
+ So, what can you do to speed this up? The most effective strategy to avoiding
1268
+ this churn is to group the execution of each tests that use each source of test
1269
+ data into sub-suites that are run serially, on e after the other.
1270
+
1271
+ * If you're using Rails' defualt Minitest, we wrote a gem called
1272
+ [minitest-suite](https://github.com/testdouble/minitest-suite) to accomplish
1273
+ exactly this. Just declare something like `suite :test_data` or `suite
1274
+ :fixtures` at the top of each test class
1275
+ * If you're using RSpec, the
1276
+ [tag](https://relishapp.com/rspec/rspec-core/v/3-10/docs/command-line/tag-option)
1277
+ feature can help you organize your tests by type, but you'll likely have to
1278
+ run a separate CLI invocation for each to avoid the tests from being
1279
+ interleaved
1280
+
1281
+ Here's what the same example would do at run-time after adding
1282
+ [minitest-suite](https://github.com/testdouble/minitest-suite):
1283
+
1284
+ ```
1285
+ $ bin/rails test test/example_test.rb
1286
+ Run options: --seed 50105
1287
+
1288
+ # Running:
1289
+
1290
+ test_data -- loading test_data SQL dump
1291
+ . test_data -- rolling back to clean test_data
1292
+ . test_data -- rolling back to clean test_data
1293
+ . test_data -- rolling back to clean test_data
1294
+ . test_data -- rolling back to clean test_data
1295
+ . fixtures -- truncating tables, loading Rails fixtures
1296
+ . fixtures -- rolling back to clean fixtures
1297
+ . fixtures -- rolling back to clean fixtures
1298
+ . fixtures -- rolling back to clean fixtures
1299
+ . fixtures -- rolling back to clean fixtures
1300
+ .
1301
+
1302
+ Finished in 2.377050s, 4.2069 runs/s, 4.2069 assertions/s.
1303
+ 10 runs, 10 assertions, 0 failures, 0 errors, 0 skips
1304
+ ```
1305
+
1306
+ By grouping the execution in this way, the most expensive operations will
1307
+ usually only be run once: at the beginning of the first test in each suite.
1308
+
1309
+ #### Expensive data manipulation
1310
+
1311
+ If you're doing anything repeatedly that's data-intensive in your test setup
1312
+ after calling one of the `TestData.uses_*` methods, that operation is being
1313
+ repeated once per test, which could be very slow. Instead, you might consider
1314
+ moving that behavior into a [lifecycle hook](#lifecycle-hooks).
1315
+
1316
+ Any code passed to a lifecycle hook will only be executed when data is
1317
+ _actually_ loaded or truncated and its effect will be included in the
1318
+ transaction savepoint that the `test_data` gem rolls back between tests.
1319
+ Seriously, appropriately moving data adjustments into these hooks can cut your
1320
+ test suite's runtime by an order of magnitude.
1321
+
1322
+ #### Redundant test setup tasks
1323
+
1324
+ One of the most likely sources of unnecessary slowness is redundant test
1325
+ cleanup. The speed gained from sandwiching every expensive operation between
1326
+ transaction savepoints can be profound… but can also easily be erased by a
1327
+ single before-each hook calling
1328
+ [database_cleaner](https://github.com/DatabaseCleaner/database_cleaner) to
1329
+ commit a truncation of the database. As a result, it's worth taking a little
1330
+ time to take stock of everything that's called between tests during setup &
1331
+ teardown to ensure multiple tools aren't attempting to clean up the state of the
1332
+ database and potentially interfering with one another.
1333
+
971
1334
  ## Code of Conduct
972
1335
 
973
1336
  This project follows Test Double's [code of