online_migrations 0.16.0 → 0.17.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 48ddbce66257c9010d8ab7361d8860cd898854207b5a1f4c6aa2b6b797aec2e1
4
- data.tar.gz: c21fbb5f535f788e13068e339eebe77f05a378c68c00244c4522a10d50a28a0d
3
+ metadata.gz: 48361b40b0c34e603f7f18124d458993171da657cf74d0fd58b57529d11fdd6b
4
+ data.tar.gz: 2036e05bacfbbc000d1a24f1fd1e24406ffdbf89f39f4d125b45795011cb6dec
5
5
  SHA512:
6
- metadata.gz: a5bbff2a8cca45e5fce79d653db3246bc41a4ab9d314f44131bbf74f2659d610fdaf3e87ac5e22baf7928d0cdb481e19eacaa1ed8945e4ee32f1afac3223273f
7
- data.tar.gz: 6e3861f420327e0bccec7ab8e7c0a508e7cf224312c366297834091467d7cdb5e4fac8a305dbb59f7c0650d199d9a17062ab0408fd180b6fbaa05946bfd27545
6
+ metadata.gz: 774c4870cc1cab99053dea9bddfe4d3e5c37668ba2bb379409beaeb80ca525572a7140877d34005af412d7e4269e2ebc941d1769fb2a6fca1fa7b565207d3afa
7
+ data.tar.gz: dc57b344bfeb9b2d85ac5e0a56a0cc5dd23ec3081d13d81d6b9293c65ef8f9d5f3ac3b0f3f42b1f84306bc4793ecd95702bb44155fffdc98faaace69ecb3cb60
data/CHANGELOG.md CHANGED
@@ -1,5 +1,18 @@
1
1
  ## master (unreleased)
2
2
 
3
+ ## 0.17.0 (2024-04-23)
4
+
5
+ - Fix background migrations `#progress` possibility to fail with zero division error
6
+ - Add `ensure_background_data_migration_succeeded` and `ensure_background_schema_migration_succeeded` migration helpers
7
+ - Raise in development when background index creation/removal was not enqueued
8
+ - Suggest two migrations for adding foreign keys
9
+ - Reraise errors when running background schema migrations inline
10
+
11
+ ## 0.16.1 (2024-03-29)
12
+
13
+ - Improve error message when background schema migration name is already taken
14
+ - Fix copying column background migration to work with primary keys added via `initialize_column_type_change`
15
+
3
16
  ## 0.16.0 (2024-03-28)
4
17
 
5
18
  - Add support for asynchronous creation/removal of indexes
data/README.md CHANGED
@@ -355,6 +355,8 @@ A safer approach can be accomplished in several steps:
355
355
  disable_ddl_transaction!
356
356
 
357
357
  def up
358
+ # You can use `backfill_column_for_type_change_in_background` if want to
359
+ # backfill using background migrations.
358
360
  backfill_column_for_type_change :files, :size
359
361
  end
360
362
 
@@ -860,20 +862,25 @@ end
860
862
 
861
863
  :white_check_mark: **Good**
862
864
 
863
- Add the foreign key without validating existing rows, and then validate them in a separate transaction.
865
+ Add the foreign key without validating existing rows:
864
866
 
865
867
  ```ruby
866
868
  class AddForeignKeyToProjectsUser < ActiveRecord::Migration[7.1]
867
- disable_ddl_transaction!
868
-
869
869
  def change
870
870
  add_foreign_key :projects, :users, validate: false
871
- validate_foreign_key :projects, :users
872
871
  end
873
872
  end
874
873
  ```
875
874
 
876
- **Note**: If you forget `disable_ddl_transaction!`, the migration will fail.
875
+ Then validate them in a separate migration:
876
+
877
+ ```ruby
878
+ class ValidateForeignKeyOnProjectsUser < ActiveRecord::Migration[7.1]
879
+ def change
880
+ validate_foreign_key :projects, :users
881
+ end
882
+ end
883
+ ```
877
884
 
878
885
  ### Adding an exclusion constraint
879
886
 
@@ -80,16 +80,16 @@ You can enqueue your background migration to be run by the scheduler via:
80
80
  # db/migrate/xxxxxxxxxxxxxx_enqueue_backfill_project_issues_count.rb
81
81
  class EnqueueBackfillProjectIssuesCount < ActiveRecord::Migration[7.1]
82
82
  def up
83
- enqueue_background_migration("BackfillProjectIssuesCount")
83
+ enqueue_background_data_migration("BackfillProjectIssuesCount")
84
84
  end
85
85
 
86
86
  def down
87
- remove_background_migration("BackfillProjectIssuesCount")
87
+ remove_background_data_migration("BackfillProjectIssuesCount")
88
88
  end
89
89
  end
90
90
  ```
91
91
 
92
- `enqueue_background_migration` accepts additional configuration options which controls how the background migration is run. Check the [source code](https://github.com/fatkodima/online_migrations/blob/master/lib/online_migrations/background_migrations/migration_helpers.rb) for the list of all available configuration options.
92
+ `enqueue_background_data_migration` accepts additional configuration options which controls how the background migration is run. Check the [source code](https://github.com/fatkodima/online_migrations/blob/master/lib/online_migrations/background_migrations/migration_helpers.rb) for the list of all available configuration options.
93
93
 
94
94
  ## Custom Background Migration Arguments
95
95
 
@@ -112,7 +112,7 @@ And pass them when enqueuing:
112
112
 
113
113
  ```ruby
114
114
  def up
115
- enqueue_background_migration("MyMigrationWithArgs", arg1, arg2, ...)
115
+ enqueue_background_data_migration("MyMigrationWithArgs", arg1, arg2, ...)
116
116
  end
117
117
  ```
118
118
 
@@ -120,7 +120,7 @@ Make sure to also pass the arguments inside the `down` method of the migration:
120
120
 
121
121
  ```ruby
122
122
  def down
123
- remove_background_migration("MyMigrationWithArgs", arg1, arg2, ...)
123
+ remove_background_data_migration("MyMigrationWithArgs", arg1, arg2, ...)
124
124
  end
125
125
  ```
126
126
 
@@ -140,6 +140,10 @@ end
140
140
 
141
141
  **Note**: These migration helpers should be run inside the migration against the database where background migrations tables are defined.
142
142
 
143
+ ## Depending on migrated data
144
+
145
+ You shouldn't depend on the data until the background data migration is finished. If having 100% of the data migrated is a requirement, then the `ensure_background_data_migration_succeeded` helper can be used to guarantee that the migration succeeded and the data fully migrated.
146
+
143
147
  ## Testing
144
148
 
145
149
  At a minimum, it's recommended that the `#process_batch` method in your background migration is tested. You may also want to test the `#relation` and `#count` methods if they are sufficiently complex.
@@ -62,6 +62,10 @@ end
62
62
 
63
63
  `add_index_in_background`/`remove_index_in_background` accept additional configuration options which controls how the background schema migration is run. Check the [source code](https://github.com/fatkodima/online_migrations/blob/master/lib/online_migrations/background_schema_migrations/migration_helpers.rb) for the list of all available configuration options.
64
64
 
65
+ ## Depending on schema changes
66
+
67
+ You shouldn't depend on the schema until the background schema migration is finished. If having the schema migrated is a requirement, then the `ensure_background_schema_migration_succeeded` helper can be used to guarantee that the migration succeeded and the schema change applied.
68
+
65
69
  ## Instrumentation
66
70
 
67
71
  Background schema migrations use the [ActiveSupport::Notifications](http://api.rubyonrails.org/classes/ActiveSupport/Notifications.html) API.
@@ -11,7 +11,7 @@ module OnlineMigrations
11
11
  source_root File.expand_path("templates", __dir__)
12
12
  desc "This generator creates a background migration related files."
13
13
 
14
- def create_background_migration_file
14
+ def create_background_data_migration_file
15
15
  migrations_module_file_path = migrations_module.underscore
16
16
 
17
17
  template_file = File.join(
@@ -20,7 +20,7 @@ module OnlineMigrations
20
20
  class_path,
21
21
  "#{file_name}.rb"
22
22
  )
23
- template("background_migration.rb", template_file)
23
+ template("background_data_migration.rb", template_file)
24
24
  end
25
25
 
26
26
  def create_migration_file
@@ -1,10 +1,10 @@
1
1
  class Enqueue<%= class_name %> < <%= migration_parent %>
2
2
  def up
3
- enqueue_background_migration("<%= class_name %>", ...args)
3
+ enqueue_background_data_migration("<%= class_name %>", ...args)
4
4
  end
5
5
 
6
6
  def down
7
7
  # Make sure to pass the same arguments as in the "up" method, if any.
8
- remove_background_migration("<%= class_name %>", ...args)
8
+ remove_background_data_migration("<%= class_name %>", ...args)
9
9
  end
10
10
  end
@@ -25,10 +25,7 @@ module OnlineMigrations
25
25
  end
26
26
 
27
27
  def relation
28
- model
29
- .unscoped
30
- .where(copy_to.index_with(nil))
31
- .where.not(copy_from.index_with(nil))
28
+ model.unscoped
32
29
  end
33
30
 
34
31
  def process_batch(relation)
@@ -5,7 +5,7 @@ module OnlineMigrations
5
5
  # Class representing background data migration.
6
6
  #
7
7
  # @note The records of this class should not be created manually, but via
8
- # `enqueue_background_migration` helper inside migrations.
8
+ # `enqueue_background_data_migration` helper inside migrations.
9
9
  #
10
10
  class Migration < ApplicationRecord
11
11
  STATUSES = [
@@ -20,6 +20,7 @@ module OnlineMigrations
20
20
  self.table_name = :background_migrations
21
21
 
22
22
  scope :queue_order, -> { order(created_at: :asc) }
23
+ scope :parents, -> { where(parent_id: nil) }
23
24
  scope :runnable, -> { where(composite: false) }
24
25
  scope :active, -> { where(status: [statuses[:enqueued], statuses[:running]]) }
25
26
  scope :except_succeeded, -> { where.not(status: :succeeded) }
@@ -125,7 +126,7 @@ module OnlineMigrations
125
126
 
126
127
  progresses.sum.round(2)
127
128
  end
128
- elsif rows_count
129
+ elsif rows_count && rows_count > 0
129
130
  jobs_rows_count = migration_jobs.succeeded.sum(:batch_size)
130
131
  # The last migration job may need to process the amount of rows
131
132
  # less than the batch size, so we can get a value > 1.0.
@@ -179,6 +180,20 @@ module OnlineMigrations
179
180
  end
180
181
  end
181
182
 
183
+ # Returns the time this migration started running.
184
+ def started_at
185
+ # To be precise, we should get the minimum of `started_at` amongst the children jobs
186
+ # (for simple migrations) and amongst the children migrations (for composite migrations).
187
+ # But we do not have an appropriate index on the jobs table and using this will lead to
188
+ # N+1 queries if used inside some dashboard, for example.
189
+ created_at
190
+ end
191
+
192
+ # Returns the time this migration finished running.
193
+ def finished_at
194
+ updated_at if completed?
195
+ end
196
+
182
197
  # @private
183
198
  def on_shard(&block)
184
199
  abstract_class = Utils.find_connection_class(migration_model)
@@ -11,7 +11,7 @@ module OnlineMigrations
11
11
  # @param model_name [String] If Active Record multiple databases feature is used,
12
12
  # the class name of the model to get connection from.
13
13
  # @param options [Hash] used to control the behavior of background migration.
14
- # See `#enqueue_background_migration`
14
+ # See `#enqueue_background_data_migration`
15
15
  #
16
16
  # @return [OnlineMigrations::BackgroundMigrations::Migration]
17
17
  #
@@ -48,7 +48,7 @@ module OnlineMigrations
48
48
 
49
49
  model_name = model_name.name if model_name.is_a?(Class)
50
50
 
51
- enqueue_background_migration(
51
+ enqueue_background_data_migration(
52
52
  "BackfillColumn",
53
53
  table_name,
54
54
  updates,
@@ -67,7 +67,7 @@ module OnlineMigrations
67
67
  # For example when changing from `text` to `jsonb`. In this case, use the `type_cast_function` option.
68
68
  # You need to make sure there is no bad data and the cast will always succeed
69
69
  # @param options [Hash] used to control the behavior of background migration.
70
- # See `#enqueue_background_migration`
70
+ # See `#enqueue_background_data_migration`
71
71
  #
72
72
  # @return [OnlineMigrations::BackgroundMigrations::Migration]
73
73
  #
@@ -110,7 +110,7 @@ module OnlineMigrations
110
110
  tmp_columns = column_names.map { |column_name| "#{column_name}_for_type_change" }
111
111
  model_name = model_name.name if model_name.is_a?(Class)
112
112
 
113
- enqueue_background_migration(
113
+ enqueue_background_data_migration(
114
114
  "CopyColumn",
115
115
  table_name,
116
116
  column_names,
@@ -132,7 +132,7 @@ module OnlineMigrations
132
132
  # For example when changing from `text` to `jsonb`. In this case, use the `type_cast_function` option.
133
133
  # You need to make sure there is no bad data and the cast will always succeed
134
134
  # @param options [Hash] used to control the behavior of background migration.
135
- # See `#enqueue_background_migration`
135
+ # See `#enqueue_background_data_migration`
136
136
  #
137
137
  # @return [OnlineMigrations::BackgroundMigrations::Migration]
138
138
  #
@@ -167,7 +167,7 @@ module OnlineMigrations
167
167
 
168
168
  model_name = model_name.name if model_name.is_a?(Class)
169
169
 
170
- enqueue_background_migration(
170
+ enqueue_background_data_migration(
171
171
  "CopyColumn",
172
172
  table_name,
173
173
  copy_from,
@@ -187,7 +187,7 @@ module OnlineMigrations
187
187
  # - when `true` - will touch `updated_at` and/or `updated_on`
188
188
  # - when `Symbol` or `Array` - will touch specific column(s)
189
189
  # @param options [Hash] used to control the behavior of background migration.
190
- # See `#enqueue_background_migration`
190
+ # See `#enqueue_background_data_migration`
191
191
  #
192
192
  # @return [OnlineMigrations::BackgroundMigrations::Migration]
193
193
  #
@@ -208,7 +208,7 @@ module OnlineMigrations
208
208
  def reset_counters_in_background(model_name, *counters, touch: nil, **options)
209
209
  model_name = model_name.name if model_name.is_a?(Class)
210
210
 
211
- enqueue_background_migration(
211
+ enqueue_background_data_migration(
212
212
  "ResetCounters",
213
213
  model_name,
214
214
  counters,
@@ -224,7 +224,7 @@ module OnlineMigrations
224
224
  # @param model_name [String]
225
225
  # @param associations [Array]
226
226
  # @param options [Hash] used to control the behavior of background migration.
227
- # See `#enqueue_background_migration`
227
+ # See `#enqueue_background_data_migration`
228
228
  #
229
229
  # @return [OnlineMigrations::BackgroundMigrations::Migration]
230
230
  #
@@ -237,7 +237,7 @@ module OnlineMigrations
237
237
  def delete_orphaned_records_in_background(model_name, *associations, **options)
238
238
  model_name = model_name.name if model_name.is_a?(Class)
239
239
 
240
- enqueue_background_migration(
240
+ enqueue_background_data_migration(
241
241
  "DeleteOrphanedRecords",
242
242
  model_name,
243
243
  associations,
@@ -253,7 +253,7 @@ module OnlineMigrations
253
253
  # @param record_id [Integer, String] parent record primary key's value
254
254
  # @param association [String, Symbol] association name for which records will be removed
255
255
  # @param options [Hash] used to control the behavior of background migration.
256
- # See `#enqueue_background_migration`
256
+ # See `#enqueue_background_data_migration`
257
257
  #
258
258
  # @return [OnlineMigrations::BackgroundMigrations::Migration]
259
259
  #
@@ -266,7 +266,7 @@ module OnlineMigrations
266
266
  def delete_associated_records_in_background(model_name, record_id, association, **options)
267
267
  model_name = model_name.name if model_name.is_a?(Class)
268
268
 
269
- enqueue_background_migration(
269
+ enqueue_background_data_migration(
270
270
  "DeleteAssociatedRecords",
271
271
  model_name,
272
272
  record_id,
@@ -284,7 +284,7 @@ module OnlineMigrations
284
284
  # Relation-wide available actions: `:delete_all`, `:destroy_all`, and `:update_all`.
285
285
  # @param updates [Hash] updates to perform when `action` is set to `:update_all`
286
286
  # @param options [Hash] used to control the behavior of background migration.
287
- # See `#enqueue_background_migration`
287
+ # See `#enqueue_background_data_migration`
288
288
  #
289
289
  # @return [OnlineMigrations::BackgroundMigrations::Migration]
290
290
  #
@@ -312,7 +312,7 @@ module OnlineMigrations
312
312
  def perform_action_on_relation_in_background(model_name, conditions, action, updates: nil, **options)
313
313
  model_name = model_name.name if model_name.is_a?(Class)
314
314
 
315
- enqueue_background_migration(
315
+ enqueue_background_data_migration(
316
316
  "PerformActionOnRelation",
317
317
  model_name,
318
318
  conditions,
@@ -344,7 +344,7 @@ module OnlineMigrations
344
344
  # @return [OnlineMigrations::BackgroundMigrations::Migration]
345
345
  #
346
346
  # @example
347
- # enqueue_background_migration("BackfillProjectIssuesCount",
347
+ # enqueue_background_data_migration("BackfillProjectIssuesCount",
348
348
  # batch_size: 10_000, batch_max_attempts: 10)
349
349
  #
350
350
  # # Given the background migration exists:
@@ -369,8 +369,8 @@ module OnlineMigrations
369
369
  # @note For convenience, the enqueued background migration is run inline
370
370
  # in development and test environments
371
371
  #
372
- def enqueue_background_migration(migration_name, *arguments, **options)
373
- migration = create_background_migration(migration_name, *arguments, **options)
372
+ def enqueue_background_data_migration(migration_name, *arguments, **options)
373
+ migration = create_background_data_migration(migration_name, *arguments, **options)
374
374
 
375
375
  if Utils.run_background_migrations_inline?
376
376
  runner = MigrationRunner.new(migration)
@@ -379,6 +379,7 @@ module OnlineMigrations
379
379
 
380
380
  migration
381
381
  end
382
+ alias enqueue_background_migration enqueue_background_data_migration
382
383
 
383
384
  # Removes the background migration for the given class name and arguments, if exists.
384
385
  #
@@ -386,15 +387,54 @@ module OnlineMigrations
386
387
  # @param arguments [Array] Extra arguments the migration was originally created with
387
388
  #
388
389
  # @example
389
- # remove_background_migration("BackfillProjectIssuesCount")
390
+ # remove_background_data_migration("BackfillProjectIssuesCount")
390
391
  #
391
- def remove_background_migration(migration_name, *arguments)
392
+ def remove_background_data_migration(migration_name, *arguments)
392
393
  migration_name = migration_name.name if migration_name.is_a?(Class)
393
394
  Migration.for_configuration(migration_name, arguments).delete_all
394
395
  end
396
+ alias remove_background_migration remove_background_data_migration
397
+
398
+ # Ensures that the background data migration with the provided configuration succeeded.
399
+ #
400
+ # If the enqueued migration was not found in development (probably when resetting a dev environment
401
+ # followed by `db:migrate`), then a log warning is printed.
402
+ # If enqueued migration was not found in production, then the error is raised.
403
+ # If enqueued migration was found but is not succeeded, then the error is raised.
404
+ #
405
+ # @param migration_name [String, Class] Background migration job class name
406
+ # @param arguments [Array, nil] Arguments with which background migration was enqueued
407
+ #
408
+ # @example Without arguments
409
+ # ensure_background_data_migration_succeeded("BackfillProjectIssuesCount")
410
+ #
411
+ # @example With arguments
412
+ # ensure_background_data_migration_succeeded("CopyColumn", arguments: ["users", "id", "id_for_type_change"])
413
+ #
414
+ def ensure_background_data_migration_succeeded(migration_name, arguments: nil)
415
+ migration_name = migration_name.name if migration_name.is_a?(Class)
416
+
417
+ configuration = { migration_name: migration_name }
418
+
419
+ if arguments
420
+ arguments = Array(arguments)
421
+ migration = Migration.parents.for_configuration(migration_name, arguments).first
422
+ configuration[:arguments] = arguments.to_json
423
+ else
424
+ migration = Migration.parents.for_migration_name(migration_name).first
425
+ end
426
+
427
+ if migration.nil?
428
+ Utils.raise_in_prod_or_say_in_dev("Could not find background data migration for the given configuration: #{configuration}")
429
+ elsif !migration.succeeded?
430
+ raise "Expected background data migration for the given configuration to be marked as 'succeeded', " \
431
+ "but it is '#{migration.status}': #{configuration}"
432
+ end
433
+ end
434
+ alias ensure_background_migration_succeeded ensure_background_data_migration_succeeded
395
435
 
396
436
  # @private
397
- def create_background_migration(migration_name, *arguments, **options)
437
+ def create_background_data_migration(migration_name, *arguments, **options)
398
438
  options.assert_valid_keys(:batch_column_name, :min_value, :max_value, :batch_size, :sub_batch_size,
399
439
  :batch_pause, :sub_batch_pause_ms, :batch_max_attempts)
400
440
 
@@ -20,6 +20,7 @@ module OnlineMigrations
20
20
  self.table_name = :background_schema_migrations
21
21
 
22
22
  scope :queue_order, -> { order(created_at: :asc) }
23
+ scope :parents, -> { where(parent_id: nil) }
23
24
  scope :runnable, -> { where(composite: false) }
24
25
  scope :active, -> { where(status: [statuses[:enqueued], statuses[:running]]) }
25
26
  scope :except_succeeded, -> { where.not(status: :succeeded) }
@@ -57,9 +58,18 @@ module OnlineMigrations
57
58
  belongs_to :parent, class_name: name, optional: true
58
59
  has_many :children, class_name: name, foreign_key: :parent_id
59
60
 
60
- validates :migration_name, presence: true, uniqueness: { scope: :shard }
61
61
  validates :table_name, presence: true, length: { maximum: MAX_IDENTIFIER_LENGTH }
62
62
  validates :definition, presence: true
63
+ validates :migration_name, presence: true, uniqueness: {
64
+ scope: :shard,
65
+ message: ->(object, data) do
66
+ message = "(#{data[:value]}) has already been taken."
67
+ if object.index_addition?
68
+ message += " Consider enqueuing index creation with a different index name via a `:name` option."
69
+ end
70
+ message
71
+ end,
72
+ }
63
73
 
64
74
  validate :validate_children_statuses, if: -> { composite? && status_changed? }
65
75
  validate :validate_connection_class, if: :connection_class_name?
@@ -81,7 +91,14 @@ module OnlineMigrations
81
91
  100.0
82
92
  elsif composite?
83
93
  progresses = children.map(&:progress)
84
- (progresses.sum.to_f / progresses.size).round(2)
94
+ # There should not be composite migrations without children,
95
+ # but children may be deleted for some reason, so we need to
96
+ # make a check to avoid 0 division error.
97
+ if progresses.any?
98
+ (progresses.sum.to_f / progresses.size).round(2)
99
+ else
100
+ 0.0
101
+ end
85
102
  else
86
103
  0.0
87
104
  end
@@ -107,6 +124,10 @@ module OnlineMigrations
107
124
  end
108
125
  end
109
126
 
127
+ def index_addition?
128
+ definition.match?(/create (unique )?index/i)
129
+ end
130
+
110
131
  # @private
111
132
  def connection_class
112
133
  if connection_class_name && (klass = connection_class_name.safe_constantize)
@@ -130,8 +151,7 @@ module OnlineMigrations
130
151
  statement_timeout = self.statement_timeout || OnlineMigrations.config.statement_timeout
131
152
 
132
153
  with_statement_timeout(connection, statement_timeout) do
133
- case definition
134
- when /create (unique )?index/i
154
+ if index_addition?
135
155
  index = connection.indexes(table_name).find { |i| i.name == name }
136
156
  if index
137
157
  # Use index validity from https://github.com/rails/rails/pull/45160
@@ -7,7 +7,7 @@ module OnlineMigrations
7
7
  migration_options = options.extract!(:max_attempts, :statement_timeout, :connection_class_name)
8
8
 
9
9
  if index_exists?(table_name, column_name, **options)
10
- Utils.say("Index creation was not enqueued because the index already exists.")
10
+ Utils.raise_or_say("Index creation was not enqueued because the index already exists.")
11
11
  return
12
12
  end
13
13
 
@@ -27,7 +27,7 @@ module OnlineMigrations
27
27
  migration_options = options.extract!(:max_attempts, :statement_timeout, :connection_class_name)
28
28
 
29
29
  if !index_exists?(table_name, column_name, **options, name: name)
30
- Utils.say("Index deletion was not enqueued because the index does not exist.")
30
+ Utils.raise_or_say("Index deletion was not enqueued because the index does not exist.")
31
31
  return
32
32
  end
33
33
 
@@ -35,6 +35,29 @@ module OnlineMigrations
35
35
  enqueue_background_schema_migration(name, table_name, definition: definition, **migration_options)
36
36
  end
37
37
 
38
+ # Ensures that the background schema migration with the provided migration name succeeded.
39
+ #
40
+ # If the enqueued migration was not found in development (probably when resetting a dev environment
41
+ # followed by `db:migrate`), then a log warning is printed.
42
+ # If enqueued migration was not found in production, then the error is raised.
43
+ # If enqueued migration was found but is not succeeded, then the error is raised.
44
+ #
45
+ # @param migration_name [String, Symbol] Background schema migration name
46
+ #
47
+ # @example
48
+ # ensure_background_schema_migration_succeeded("index_users_on_email")
49
+ #
50
+ def ensure_background_schema_migration_succeeded(migration_name)
51
+ migration = Migration.parents.find_by(migration_name: migration_name)
52
+
53
+ if migration.nil?
54
+ Utils.raise_in_prod_or_say_in_dev("Could not find background schema migration: '#{migration_name}'")
55
+ elsif !migration.succeeded?
56
+ raise "Expected background schema migration '#{migration_name}' to be marked as 'succeeded', " \
57
+ "but it is '#{migration.status}'."
58
+ end
59
+ end
60
+
38
61
  def enqueue_background_schema_migration(name, table_name, **options)
39
62
  if options[:connection_class_name].nil? && Utils.multiple_databases?
40
63
  raise ArgumentError, "You must pass a :connection_class_name when using multiple databases."
@@ -76,6 +76,7 @@ module OnlineMigrations
76
76
  )
77
77
 
78
78
  ::OnlineMigrations.config.background_schema_migrations.error_handler.call(e, migration)
79
+ raise if Utils.run_background_migrations_inline?
79
80
  end
80
81
 
81
82
  def should_throttle?
@@ -25,8 +25,8 @@ module OnlineMigrations
25
25
  :remove_text_limit_constraint,
26
26
  :add_reference_concurrently,
27
27
  :change_column_type_in_background,
28
- :enqueue_background_migration,
29
- :remove_background_migration,
28
+ :enqueue_background_data_migration,
29
+ :remove_background_data_migration,
30
30
 
31
31
  # column type change helpers
32
32
  :initialize_column_type_change,
@@ -203,9 +203,9 @@ which will be passed to `add_column` when creating a new column, so you can over
203
203
  disable_ddl_transaction!
204
204
 
205
205
  def up
206
- <%= backfill_code %>
207
206
  # You can use `backfill_column_for_type_change_in_background` if want to
208
207
  # backfill using background migrations.
208
+ <%= backfill_code %>
209
209
  end
210
210
 
211
211
  def down
@@ -393,14 +393,17 @@ end",
393
393
  A safer approach is to create the new index and then delete the old one.",
394
394
 
395
395
  add_foreign_key:
396
- "Adding a foreign key blocks writes on both tables. Add the foreign key without validating existing rows,
397
- and then validate them in a separate transaction.
396
+ "Adding a foreign key blocks writes on both tables. Instead, add the foreign key without validating existing rows,
397
+ then validate them in a separate migration.
398
398
 
399
399
  class <%= migration_name %> < <%= migration_parent %>
400
- disable_ddl_transaction!
401
-
402
400
  def change
403
401
  <%= add_code %>
402
+ end
403
+ end
404
+
405
+ class Validate<%= migration_name %> < <%= migration_parent %>
406
+ def change
404
407
  <%= validate_code %>
405
408
  end
406
409
  end",
@@ -12,6 +12,7 @@ module OnlineMigrations
12
12
  super
13
13
  ensure
14
14
  VerboseSqlLogs.disable if verbose_sql_logs?
15
+ OnlineMigrations.current_migration = nil
15
16
  end
16
17
 
17
18
  # @private
@@ -32,6 +32,22 @@ module OnlineMigrations
32
32
  end
33
33
  end
34
34
 
35
+ def raise_or_say(message)
36
+ if developer_env?
37
+ raise message
38
+ else
39
+ say(message)
40
+ end
41
+ end
42
+
43
+ def raise_in_prod_or_say_in_dev(message)
44
+ if developer_env?
45
+ say(message)
46
+ else
47
+ raise message
48
+ end
49
+ end
50
+
35
51
  def warn(message)
36
52
  Kernel.warn("[online_migrations] #{message}")
37
53
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module OnlineMigrations
4
- VERSION = "0.16.0"
4
+ VERSION = "0.17.0"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: online_migrations
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.16.0
4
+ version: 0.17.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - fatkodima
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2024-03-28 00:00:00.000000000 Z
11
+ date: 2024-04-23 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activerecord
@@ -40,7 +40,7 @@ files:
40
40
  - lib/generators/online_migrations/background_migration_generator.rb
41
41
  - lib/generators/online_migrations/install_generator.rb
42
42
  - lib/generators/online_migrations/templates/add_sharding_to_online_migrations.rb.tt
43
- - lib/generators/online_migrations/templates/background_migration.rb.tt
43
+ - lib/generators/online_migrations/templates/background_data_migration.rb.tt
44
44
  - lib/generators/online_migrations/templates/create_background_schema_migrations.rb.tt
45
45
  - lib/generators/online_migrations/templates/initializer.rb.tt
46
46
  - lib/generators/online_migrations/templates/install_migration.rb.tt