mongoid_lazy_migration 0.2 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.md +52 -42
- data/lib/mongoid/lazy_migration/document.rb +3 -1
- data/lib/mongoid/lazy_migration/version.rb +1 -1
- metadata +5 -5
data/README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
Mongoid Lazy Migration
|
2
2
|
======================
|
3
3
|
|
4
|
-

|
4
|
+
[](http://travis-ci.org/nviennot/mongoid_lazy_migration)
|
5
5
|
|
6
6
|
LazyMigration allows you to migrate your document on the fly.
|
7
7
|
The migration is run when an instance of your model is initialized.
|
@@ -52,9 +52,8 @@ A document migration goes through the following states:
|
|
52
52
|
|
53
53
|
### Atomic Migration Example
|
54
54
|
|
55
|
-
Suppose we have a model
|
56
|
-
|
57
|
-
We wish to add a field `num_people`. Here is how it goes:
|
55
|
+
Suppose we have a model GroupOfPeople with two fields: num_males and
|
56
|
+
num_females. We wish to add a field num_people. Here is how it goes:
|
58
57
|
|
59
58
|
```ruby
|
60
59
|
class GroupOfPeople
|
@@ -75,7 +74,7 @@ class GroupOfPeople
|
|
75
74
|
end
|
76
75
|
```
|
77
76
|
|
78
|
-
Note that calling
|
77
|
+
Note that calling inc_num_people is perfectly fine in presence of contention
|
79
78
|
because only one client will be able to commit the migration to the database.
|
80
79
|
|
81
80
|
Locked Mode
|
@@ -98,15 +97,16 @@ Because of the locking, a locked migration runs slower than an atomic one.
|
|
98
97
|
### Migration States
|
99
98
|
|
100
99
|
A document migration goes through the following states:
|
100
|
+
|
101
101
|
1. pending: the migration needs to be performed.
|
102
102
|
2. processing: the document is being migrated, blocking other clients.
|
103
103
|
3. done: the migration is complete.
|
104
104
|
|
105
105
|
### Locked Migration Example
|
106
106
|
|
107
|
-
Suppose we have a model
|
108
|
-
|
109
|
-
|
107
|
+
Suppose we have a model GroupOfPeople with an array people_names, which is an
|
108
|
+
array of strings. Our migration consists of introducing a new model Person, and
|
109
|
+
removing the array people_names from GroupOfPeople. Here it goes:
|
110
110
|
|
111
111
|
```ruby
|
112
112
|
class Person
|
@@ -124,7 +124,7 @@ class GroupOfPeople
|
|
124
124
|
has_many :people
|
125
125
|
|
126
126
|
migration(:lock => true) do
|
127
|
-
|
127
|
+
people_names.each do |name|
|
128
128
|
self.people.create(:name => name)
|
129
129
|
end
|
130
130
|
end
|
@@ -134,7 +134,7 @@ end
|
|
134
134
|
Since only one client can execute the migration block, we are guaranteed that
|
135
135
|
we only create the associations once.
|
136
136
|
|
137
|
-
Just to be safe, we don't unset the
|
137
|
+
Just to be safe, we don't unset the people_names in the migration. It would
|
138
138
|
be done manually once the migration is complete.
|
139
139
|
|
140
140
|
Background Migration
|
@@ -145,17 +145,24 @@ migrate the rest of the collection in the background.
|
|
145
145
|
|
146
146
|
It can be done programmatically:
|
147
147
|
|
148
|
-
|
149
|
-
|
150
|
-
|
151
|
-
* `Mongoid::LazyMigration.migrate(Model.where(:active => true))` will only
|
152
|
-
migrate the active models.
|
148
|
+
```ruby
|
149
|
+
# migrate all the models that have a migration block.
|
150
|
+
Mongoid::LazyMigration.migrate
|
153
151
|
|
154
|
-
|
152
|
+
# migrate only Model.
|
153
|
+
Mongoid::LazyMigration.migrate(Model)
|
155
154
|
|
156
|
-
|
157
|
-
|
158
|
-
|
155
|
+
# migrate only active documents of Model.
|
156
|
+
Mongoid::LazyMigration.migrate(Model.where(:active => true))
|
157
|
+
```
|
158
|
+
|
159
|
+
A rake task is provided wrapping the migrate method:
|
160
|
+
|
161
|
+
```ruby
|
162
|
+
rake db:mongoid:migrate
|
163
|
+
rake db:mongoid:migrate[Model]
|
164
|
+
rake db:mongoid:migrate[Model.where(:active => true)]
|
165
|
+
```
|
159
166
|
|
160
167
|
Both methods display a progress bar.
|
161
168
|
|
@@ -178,17 +185,24 @@ rake db:mongoid:migrate[User.where(:shard => 2, :active => true)]
|
|
178
185
|
Cleaning up
|
179
186
|
-----------
|
180
187
|
|
181
|
-
Once all the document have their migration state equal to done, you must
|
182
|
-
cleanup your collection (removing the migration_state field) to clear some
|
183
|
-
database space, and to ensure that you can run a future migration.
|
188
|
+
Once all the document have their migration state equal to done, you must do two things:
|
184
189
|
|
185
|
-
|
190
|
+
1. Remove the migration block from your model.
|
191
|
+
2. Cleanup the database (removing the migration_state field) to ensure that you
|
192
|
+
can run a future migration.
|
186
193
|
|
187
|
-
|
188
|
-
* `rake db:mongoid:cleanup_migration[Model]`
|
194
|
+
The two ways to cleanup a collection are:
|
189
195
|
|
190
|
-
|
191
|
-
|
196
|
+
```ruby
|
197
|
+
# programmatically
|
198
|
+
Mongoid::LazyMigration.cleanup(Model)
|
199
|
+
|
200
|
+
# rake task
|
201
|
+
rake db:mongoid:cleanup_migration[Model]
|
202
|
+
```
|
203
|
+
|
204
|
+
The cleanup process will be aborted if any of the document migrations are still
|
205
|
+
in the processing state, or if you haven't removed the migration block.
|
192
206
|
|
193
207
|
Important Considerations
|
194
208
|
------------------------
|
@@ -201,7 +215,7 @@ A couple of points that you need to be aware of:
|
|
201
215
|
interacting directly with the mongo driver and bypassing mongoid, you are on
|
202
216
|
your own.
|
203
217
|
* If you use only() in some of your queries, make sure that you include the
|
204
|
-
|
218
|
+
migration_state field.
|
205
219
|
* Do not use identity maps and threads at the same time. It is currently
|
206
220
|
unsupported, but support may be added in the future.
|
207
221
|
* Make sure you can migrate a document quickly, because migrations will be
|
@@ -212,26 +226,22 @@ A couple of points that you need to be aware of:
|
|
212
226
|
* The save and update callbacks will not be called when persisting the
|
213
227
|
migration.
|
214
228
|
* No validation will be performed during the migration.
|
215
|
-
* The migration will not update the
|
216
|
-
|
217
|
-
You can pass a criteria to `Mongoid::LazyMigration.migrate` (used at step 4
|
218
|
-
of the recipe). It allows you to:
|
219
|
-
|
220
|
-
* Update your most important document first (like your active members),
|
221
|
-
and do the rest after.
|
222
|
-
* Use many workers on different shards of your database. Note that it is
|
223
|
-
safe to try migrating the same document by different workers, but not
|
224
|
-
recommended for performance reasons.
|
229
|
+
* The migration will not update the updated_at field.
|
225
230
|
|
226
231
|
Workflow
|
227
232
|
--------
|
228
233
|
|
229
|
-
|
234
|
+
First, add the following line in your Gemfile:
|
235
|
+
|
236
|
+
```ruby
|
237
|
+
gem 'mongoid_lazy_migration'
|
238
|
+
```
|
239
|
+
|
240
|
+
Follow the following steps to perform a migration:
|
230
241
|
|
231
242
|
1. Run `rake db:mongoid:cleanup[Model]` just to be safe.
|
232
|
-
2. Include Mongoid::LazyMigration in your document and write your migration
|
233
|
-
|
234
|
-
migration.
|
243
|
+
2. Include `Mongoid::LazyMigration` in your document and write your migration.
|
244
|
+
Modify your application code to reflect the changes from the migration.
|
235
245
|
3. Deploy.
|
236
246
|
4. Run `rake db:mongoid:migrate`.
|
237
247
|
5. Remove the migration block from your model.
|
@@ -9,7 +9,9 @@ module Mongoid::LazyMigration::Document
|
|
9
9
|
end
|
10
10
|
|
11
11
|
def atomic_selector
|
12
|
-
|
12
|
+
return super unless @running_migrate_block
|
13
|
+
|
14
|
+
unless self.class.lock_migration
|
13
15
|
raise ["You cannot save during an atomic migration,",
|
14
16
|
"You are only allowed to set the document fields",
|
15
17
|
"The document will be commited once the migration is complete.",
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: mongoid_lazy_migration
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 0.2.1
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2012-07-
|
12
|
+
date: 2012-07-10 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: mongoid
|
@@ -66,13 +66,13 @@ executables: []
|
|
66
66
|
extensions: []
|
67
67
|
extra_rdoc_files: []
|
68
68
|
files:
|
69
|
+
- lib/mongoid_lazy_migration.rb
|
69
70
|
- lib/mongoid/lazy_migration/version.rb
|
70
|
-
- lib/mongoid/lazy_migration/railtie.rb
|
71
|
-
- lib/mongoid/lazy_migration/railtie/migrate.rake
|
72
71
|
- lib/mongoid/lazy_migration/tasks.rb
|
72
|
+
- lib/mongoid/lazy_migration/railtie/migrate.rake
|
73
|
+
- lib/mongoid/lazy_migration/railtie.rb
|
73
74
|
- lib/mongoid/lazy_migration/document.rb
|
74
75
|
- lib/mongoid/lazy_migration.rb
|
75
|
-
- lib/mongoid_lazy_migration.rb
|
76
76
|
- README.md
|
77
77
|
homepage: http://github.com/nviennot/mongoid_lazy_migration
|
78
78
|
licenses: []
|