nexus_seed 0.2.14 → 0.2.16
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +12 -0
- data/SEED_BUILDER.md +254 -0
- data/lib/nexus_seed/builder/base.rb +3 -2
- data/lib/nexus_seed/version.rb +2 -2
- metadata +2 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c56d4582669b8742a96cb06df56efda328b62f4916a49f4a9adb32521facff49
|
4
|
+
data.tar.gz: dc6146a31126777419cec6b29f5acf677ba0119b93e4905c6575465243f67c22
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: aedd5115dc4ca6a71c1b6ae574e41da3045ea8644ccd654bb0fed9f16baba4b6626cfb280d6380cfe292027de537a2f9566546b2df6715bbb86583e51c4575c6
|
7
|
+
data.tar.gz: 2c5f568773a14fbd9de9c8c8fc33c683387b0e610c129524ee16ec0418ab52851193a6d9476d2b732899420efbcfbd1f9d5b9cee5d776d763e930f468dcd3498
|
data/README.md
CHANGED
@@ -2,6 +2,10 @@
|
|
2
2
|
|
3
3
|
Common rails application seeding logic.
|
4
4
|
|
5
|
+
# Seed development
|
6
|
+
|
7
|
+
See [SEED_BUILDER](SEED_BUILDER.md) documentation.
|
8
|
+
|
5
9
|
# Running
|
6
10
|
|
7
11
|
## Running tasks on deploy
|
@@ -41,6 +45,14 @@ E.g. to retry 100 times every 3 seconds:
|
|
41
45
|
|
42
46
|
`bundle exec rake seed_common:run[100,3]`
|
43
47
|
|
48
|
+
## Removing seeded data
|
49
|
+
|
50
|
+
Use `NEXUS_SEED_DESTROY` env var to locate seeded data and remove it.
|
51
|
+
|
52
|
+
Especially useful when developing seeds, run with destroy and then run again without.
|
53
|
+
|
54
|
+
`NEXUS_SEED_DESTROY=true bundle exec rake seed_common:run`
|
55
|
+
|
44
56
|
# Local gem development
|
45
57
|
|
46
58
|
Steps to run this gem from local sources in one the nexus 'staged build' rails components:
|
data/SEED_BUILDER.md
ADDED
@@ -0,0 +1,254 @@
|
|
1
|
+
# Seed builder
|
2
|
+
|
3
|
+
The new seeding method proposes the following syntax:
|
4
|
+
|
5
|
+
```
|
6
|
+
NexusSeed::Builder.build(:collection_schema)
|
7
|
+
NexusSeed::Builder.build(:category)
|
8
|
+
|
9
|
+
game = NexusSeed::Builder.build(:game,
|
10
|
+
game_genre: NexusSeed::Builder.build(:game_genre, name: 'Some Game Genre'),
|
11
|
+
name: 'Some Random Game'
|
12
|
+
)
|
13
|
+
|
14
|
+
NexusSeed::Builder.build(:forum, { id: ENV['FORUM_COLLECTIONS_ID'] }, { find_by_params: :id })
|
15
|
+
```
|
16
|
+
|
17
|
+
`NexusSeed::Base.transaction` wraps the existing AR transaction and provides some QOL methods such as destroying
|
18
|
+
previously generated seeds and generating seed reports.
|
19
|
+
|
20
|
+
`NexusSeed::Builder.build(:collection_schema)` can be viewed as something
|
21
|
+
mimicking `FactoryBot.create(:collection_schema)`. There are a few key differences, however:
|
22
|
+
|
23
|
+
- `NexusSeed::Builder` ensures that the record generation is always idempotent
|
24
|
+
- It does not currently support traits, but it does provide `before_save` and `after_save` callbacks which have to be
|
25
|
+
defined in the corresponding builder class (think of builder classes as factory definitions in `FactoryBot`)
|
26
|
+
- it takes special parameters that can be passed as options
|
27
|
+
|
28
|
+
Each model that we are intending to build needs to have a corresponding builder class in `db/nexus_seed/builders`
|
29
|
+
directory. For example, if we want to create a builder class for a `Category` model, we will call it `CategoryBuilder`
|
30
|
+
and place it in that directory. Here's a simple example of a builder class:
|
31
|
+
|
32
|
+
```
|
33
|
+
# frozen_string_literal: true
|
34
|
+
module NexusSeed
|
35
|
+
module Builders
|
36
|
+
class CategoryBuilder < NexusSeed::Builder::Base
|
37
|
+
def defaults(params = {})
|
38
|
+
{
|
39
|
+
name: 'Default Category',
|
40
|
+
description: 'Category description',
|
41
|
+
approved: true,
|
42
|
+
suggested_by: 1,
|
43
|
+
parent_id: 0,
|
44
|
+
}
|
45
|
+
end
|
46
|
+
|
47
|
+
def find_by_params(instance, params)
|
48
|
+
:name
|
49
|
+
end
|
50
|
+
end
|
51
|
+
end
|
52
|
+
end
|
53
|
+
```
|
54
|
+
|
55
|
+
There are 2 things that are defined here:
|
56
|
+
|
57
|
+
- `defaults` is optional and can be used to provide default values for a model so that we don't need to specify them
|
58
|
+
each time. It takes `params` as an argument which is there simply to provide access to whatever is being passed into
|
59
|
+
an actual instance, e.g.:
|
60
|
+
|
61
|
+
```
|
62
|
+
NexusSeed::Builder.build(:category, name: 'some category', approved: false)
|
63
|
+
```
|
64
|
+
|
65
|
+
The `defaults` method will be able to see the `name` and `approved` params provided and if required, use them to resolve
|
66
|
+
the rest of the parameters, i.e. we could have a conditional statement such as:
|
67
|
+
|
68
|
+
```
|
69
|
+
def defaults(params)
|
70
|
+
defaults = {}
|
71
|
+
|
72
|
+
if params[:approved]
|
73
|
+
defaults[:approved_by] = 1337
|
74
|
+
end
|
75
|
+
|
76
|
+
...
|
77
|
+
defaults
|
78
|
+
end
|
79
|
+
```
|
80
|
+
|
81
|
+
To sum up, `defaults` is a method with access to current parameters used for the record generation and can accept any
|
82
|
+
arbitrary code, as long as it returns a hash that matches the available columns and relationships of the corresponding
|
83
|
+
model.
|
84
|
+
|
85
|
+
That said, you can use relationships when building your seed data, just as you would if you were using factory bot, or
|
86
|
+
the models directly e.g.:
|
87
|
+
|
88
|
+
```
|
89
|
+
genre = NexusSeed::Builder.build(:game_genre, name: 'Some Game Genre')
|
90
|
+
|
91
|
+
game = NexusSeed::Builder.build(:game, game_genre: genre)
|
92
|
+
```
|
93
|
+
|
94
|
+
This is equivalent to:
|
95
|
+
|
96
|
+
```
|
97
|
+
game = Game.create(game_genre: genre, ...rest of params)
|
98
|
+
# or
|
99
|
+
game = FactoryBot.create(:game, game_genre: genre)
|
100
|
+
```
|
101
|
+
|
102
|
+
**Idempotence:**
|
103
|
+
|
104
|
+
As you can see in the `CategoryBuilder` example, each builder class must have a `find_by_params` defined, which is used
|
105
|
+
to ensure that the records generated by the builder are idempotent. In the case of the category builder, that field is
|
106
|
+
set to `:name`, which means that the builder will never build two records with identical names.
|
107
|
+
|
108
|
+
`find_by_params` can also accept an array or a hash with a specific query, or it can be a method. Some examples:
|
109
|
+
|
110
|
+
```
|
111
|
+
# make this builder ensure the records are unique by name and parent_id
|
112
|
+
def find_by_params(instance, params)
|
113
|
+
[:name, :parent_id]
|
114
|
+
end
|
115
|
+
|
116
|
+
# make sure only one record with this parent_id can be created:
|
117
|
+
def find_by_params(instance, params)
|
118
|
+
{ parent_id: 1337 }
|
119
|
+
end
|
120
|
+
|
121
|
+
# do something with the params before returning the final `find_by_params`
|
122
|
+
def find_by_params(instance, params)
|
123
|
+
final_params = {}
|
124
|
+
|
125
|
+
if instance.foo? and params.bar?
|
126
|
+
final_params = ... run some code
|
127
|
+
end
|
128
|
+
|
129
|
+
final_params
|
130
|
+
end
|
131
|
+
```
|
132
|
+
|
133
|
+
You can do similar things in the `defaults` method, the difference is the `defaults` can only access your parameters,
|
134
|
+
whereas `find_by_params` can access the instance of a new model before it's saved. Here's the actual code from the base
|
135
|
+
builder class:
|
136
|
+
|
137
|
+
```
|
138
|
+
# /nexus_seed/builders/base.rb
|
139
|
+
module NexusSeed
|
140
|
+
module Builder
|
141
|
+
class Base
|
142
|
+
|
143
|
+
...
|
144
|
+
|
145
|
+
instance = klass.new(merged_params)
|
146
|
+
|
147
|
+
# set the custom find_by_params if provided
|
148
|
+
@find_by_params = if @options.key?(:find_by_params)
|
149
|
+
@options[:find_by_params]
|
150
|
+
else
|
151
|
+
find_by_params(instance, params)
|
152
|
+
end
|
153
|
+
|
154
|
+
raise StandardError, "Error: find_by_params must not be nil" if @find_by_params.nil?
|
155
|
+
|
156
|
+
before_save(instance, params)
|
157
|
+
|
158
|
+
find_by_query = if @find_by_params.is_a?(Hash)
|
159
|
+
@find_by_params
|
160
|
+
else
|
161
|
+
@find_by_params = @find_by_params.is_a?(Array) ? @find_by_params : [@find_by_params]
|
162
|
+
|
163
|
+
query_hash = {}
|
164
|
+
|
165
|
+
@find_by_params.each do |e|
|
166
|
+
query_hash[e] = instance[e]
|
167
|
+
end
|
168
|
+
|
169
|
+
query_hash
|
170
|
+
end
|
171
|
+
```
|
172
|
+
|
173
|
+
You can also override the default definition of `find_by_params` in your builder, by passing the `find_by_params` in
|
174
|
+
your options when generating new data:
|
175
|
+
|
176
|
+
```
|
177
|
+
NexusSeed::Builder.build(:category, {approved: true}, {find_by_params: [:name, :parent_id]})
|
178
|
+
```
|
179
|
+
|
180
|
+
Ensure that the `find_by_params` override goes into the **second** hash in the build method, which is intended to accept
|
181
|
+
optional stuff such as the above.
|
182
|
+
|
183
|
+
Finally, each builder can have `before_save` and `after_save` methods, which work in the same way and can access both
|
184
|
+
the instance and the params.
|
185
|
+
|
186
|
+
If a corresponding model is nested in a module, e.g. `Collections::BugReports::CollectionBugReport`, the builder will
|
187
|
+
not know how to resolve a call such as
|
188
|
+
|
189
|
+
```
|
190
|
+
NexusSeed::Builder.build(:collection_bug_report)
|
191
|
+
```
|
192
|
+
|
193
|
+
We need to therefore define the full modulized class inside the `CollectionBugReportBuilder.rb` like so:
|
194
|
+
|
195
|
+
```
|
196
|
+
def find_by_params ...
|
197
|
+
def defaults ...
|
198
|
+
def before_save...
|
199
|
+
|
200
|
+
# define modulized class here:
|
201
|
+
def model_class
|
202
|
+
Collections::BugReports::CollectionBugReport
|
203
|
+
end
|
204
|
+
```
|
205
|
+
|
206
|
+
Now you can simply run `NexusSeed::Builder.build(:collection_bug_report)` and the builder will know which active record
|
207
|
+
model is being used internally.
|
208
|
+
|
209
|
+
# Development assistance
|
210
|
+
|
211
|
+
It's often difficult to clear the seeds, especially if a project uses more than one database, so the seed builder
|
212
|
+
provides a handy interface to do just that. Simply pass `NEXUS_SEED_DESTROY=true` as your env variable when running seeds
|
213
|
+
manually, and it will destroy all of the previously generated seeds:
|
214
|
+
|
215
|
+
```
|
216
|
+
$ rake db:seeds NEXUS_SEED_DESTROY=true
|
217
|
+
1337 seeds destroyed.
|
218
|
+
Finished.
|
219
|
+
|
220
|
+
$ rake db:seeds
|
221
|
+
1337 created.
|
222
|
+
Finished
|
223
|
+
```
|
224
|
+
|
225
|
+
You can also request a detailed report of what's been generated:
|
226
|
+
|
227
|
+
```
|
228
|
+
$ rake db:seeds NEXUS_SEED_REPORT=true
|
229
|
+
Created 1000 Mods.
|
230
|
+
Created 2000 ModFiles.
|
231
|
+
... etc
|
232
|
+
35000 new seeds were created in total.
|
233
|
+
|
234
|
+
```
|
235
|
+
|
236
|
+
Reporting and idempotency play together nicely. Say you've added a couple more things to your seeds, a collection image
|
237
|
+
perhaps. When run again, you should see something along these lines:
|
238
|
+
|
239
|
+
```
|
240
|
+
$ rake db:seeds NEXUS_SEED_REPORT=true
|
241
|
+
Created 1 CollectionImage
|
242
|
+
1 new seed was created in total.
|
243
|
+
```
|
244
|
+
|
245
|
+
Running the seeds again after not changing anything should always result in:
|
246
|
+
|
247
|
+
```
|
248
|
+
$ rake db:seeds NEXUS_SEED_REPORT=true
|
249
|
+
0 new seed was created in total.
|
250
|
+
```
|
251
|
+
|
252
|
+
If that is NOT the case, then your `find_by_params` might be incorrectly defined. There is no hard and fast rule of how
|
253
|
+
you should declare your find_by_params to ensure idempotency, nor there is a one size fits all default go-to method to
|
254
|
+
use.
|
@@ -54,12 +54,13 @@ module NexusSeed
|
|
54
54
|
|
55
55
|
result = if existing.nil?
|
56
56
|
instance.save!
|
57
|
-
instance.
|
57
|
+
instance = klass.find_by(find_by_query)
|
58
|
+
instance
|
58
59
|
else
|
59
60
|
existing
|
60
61
|
end
|
61
62
|
|
62
|
-
NexusSeed::Builder.add_seed(result) if ENV['
|
63
|
+
NexusSeed::Builder.add_seed(result) if ENV['NEXUS_SEED_DESTROY'] == 'true'
|
63
64
|
|
64
65
|
after_save(result, params)
|
65
66
|
end
|
data/lib/nexus_seed/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: nexus_seed
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.2.
|
4
|
+
version: 0.2.16
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Johnathon Harris
|
@@ -35,6 +35,7 @@ files:
|
|
35
35
|
- ".rubocop.yml"
|
36
36
|
- Gemfile
|
37
37
|
- README.md
|
38
|
+
- SEED_BUILDER.md
|
38
39
|
- lib/nexus_seed.rb
|
39
40
|
- lib/nexus_seed/base.rb
|
40
41
|
- lib/nexus_seed/builder.rb
|