data_shifter 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8143cec17a5f8cb7374ad327338e694cea0a9422bf0392e005a3eaeeee9ab83d
4
- data.tar.gz: b9a246478df8ef89377482951e74ad36b63655510f4bf3bbc6a1b4480edec85e
3
+ metadata.gz: 7c2c6cb1c13bba3100efe294ceeafd838f4c329650b323457b0a75296bbff28f
4
+ data.tar.gz: b2dfe9b104bcc97fe7f8d1524d9739cb6b5918885a64f0ccf58725a8477d4298
5
5
  SHA512:
6
- metadata.gz: e8f150f146151d8a82ccc79fd2e31f2ed813ee1b631d0eb9fc5490fa201ed31890a088f24bb7bb086d4158407f5bfb3f969c639dad75d85809c72d44e609172e
7
- data.tar.gz: b776d810b0819d216436e169414c9c09c0a0c1f7cc3123f58912fa638675f55c998c3c7be71bae40868c5fddbb031bd42d3b2b8a915491d9e6d792852712405f
6
+ metadata.gz: c501bef9515dae1a53a20dd4b40e4fb454c31b17a781b53df02b861f6ae0415f012a801f2b767f5975cd8cb5aa69555467de5a89dff4448b766ab410e23e7a5a
7
+ data.tar.gz: cd30ac7dcef93f800101c26a5472063859e8671270c58169083c7116fdd3cac2f6559efe6f9da084a2b7b939aa2ef0e909a96b816964043f8739d544a80fef34
data/CHANGELOG.md CHANGED
@@ -4,6 +4,20 @@
4
4
 
5
5
  * N/A
6
6
 
7
+ ## [0.3.0]
8
+
9
+ ### Added
10
+
11
+ - **Task-based shifts**: New `task` DSL for targeted, one-off changes without the `collection`/`process_record` pattern. Define one or more `task "label" do ... end` blocks that run in sequence with shared transaction and dry-run semantics. Labels appear in output and error messages.
12
+ - **Generator `--task` option**: `rails g data_shift fix_order_1234 --task` generates a shift with a `task` block instead of `collection`/`process_record`.
13
+ - **Colorized CLI output**: Headers, summaries, and status output now use ANSI colors for better readability. Colors are automatically disabled when output is not a TTY or when `NO_COLOR` environment variable is set.
14
+ - **Cleaner summaries**: `Failed` and `Skipped` lines are now omitted from summaries when their values are zero.
15
+
16
+ ### Changed
17
+
18
+ - **Improved error messages**: `NotImplementedError` messages for `collection` and `process_record` now suggest using `task` blocks as an alternative.
19
+ - **Task labels logged on execution**: When running task-based shifts, each labeled task logs its name (`>> label`) when it starts.
20
+
7
21
  ## [0.2.0]
8
22
 
9
23
  ### Added
data/README.md CHANGED
@@ -35,7 +35,9 @@ COMMIT=1 rake data:shift:backfill_foo
35
35
 
36
36
  ## Defining a shift
37
37
 
38
- Typical shifts implement:
38
+ ### Collection-based shifts (typical)
39
+
40
+ For systemic migrations across many records, implement:
39
41
 
40
42
  - **`collection`**: an `ActiveRecord::Relation` (uses `find_each`) or an `Array`/Enumerable
41
43
  - **`process_record(record)`**: applies the change for one record
@@ -56,9 +58,48 @@ module DataShifts
56
58
  end
57
59
  ```
58
60
 
61
+ ### Task-based shifts (targeted, one-off changes)
62
+
63
+ For targeted changes to specific records (e.g. fixing a bug for particular IDs), use `task` blocks instead:
64
+
65
+ ```ruby
66
+ module DataShifts
67
+ class FixOrderDiscrepancies < DataShifter::Shift
68
+ description "Fix order #1234 shipping and billing issues"
69
+
70
+ task "Correct shipping address" do
71
+ order.update!(shipping_address: "123 Main St")
72
+ end
73
+
74
+ task "Apply missing discount" do
75
+ order.update!(discount_cents: 500)
76
+ end
77
+
78
+ private
79
+
80
+ def order
81
+ @order ||= Order.find(1234)
82
+ end
83
+ end
84
+ end
85
+ ```
86
+
87
+ Task blocks run in the context of the shift instance, so they have access to private helper methods, `dry_run?`, `log`, `skip!`, `find_exactly!`, and any other instance methods you define. Use private methods to DRY up shared lookups across tasks.
88
+
89
+ Task blocks:
90
+
91
+ - Run in sequence within the same lifecycle (transaction, dry run protection, summary)
92
+ - Default to single transaction (all tasks commit or roll back together); use `transaction :per_record` for per-task transactions
93
+
94
+ Generate a task-based shift with:
95
+
96
+ ```bash
97
+ bin/rails generate data_shift fix_order_1234 --task
98
+ ```
99
+
59
100
  ## Dry run vs commit
60
101
 
61
- Shifts run in **dry run** mode by default. In the automatic transaction modes (`transaction :single` / `true`, and `transaction :per_record`), DB changes are rolled back automatically.
102
+ Shifts run in **dry run** mode by default. DB changes are always rolled back in dry run mode, regardless of transaction setting.
62
103
 
63
104
  - **Dry run (default)**: `rake data:shift:backfill_foo`
64
105
  - **Commit**: `COMMIT=1 rake data:shift:backfill_foo`
@@ -277,6 +318,7 @@ end
277
318
  | `bin/rails generate data_shift backfill_foo` | `lib/data_shifts/<timestamp>_backfill_foo.rb` with a `DataShifts::BackfillFoo` class |
278
319
  | `bin/rails generate data_shift backfill_users --model User` | Same, with `User.all` in `collection` and `process_record(user)` |
279
320
  | `bin/rails generate data_shift backfill_users --spec` | Also generates `spec/lib/data_shifts/backfill_users_spec.rb` when RSpec is enabled |
321
+ | `bin/rails generate data_shift fix_order_1234 --task` | Generates a shift with a `task` block instead of `collection`/`process_record` |
280
322
 
281
323
  The generator refuses to create a second shift if it would produce a duplicate rake task name.
282
324
 
@@ -0,0 +1,71 @@
1
+ # frozen_string_literal: true
2
+
3
+ module DataShifter
4
+ module Internal
5
+ # ANSI color utilities for CLI output.
6
+ # Automatically detects TTY and respects NO_COLOR environment variable.
7
+ module Colors
8
+ CODES = {
9
+ reset: "\e[0m",
10
+ bold: "\e[1m",
11
+ dim: "\e[2m",
12
+ green: "\e[32m",
13
+ yellow: "\e[33m",
14
+ red: "\e[31m",
15
+ cyan: "\e[36m",
16
+ }.freeze
17
+
18
+ module_function
19
+
20
+ def enabled?(io = $stdout)
21
+ return false if ENV["NO_COLOR"]
22
+ return false unless io.respond_to?(:tty?)
23
+
24
+ io.tty?
25
+ end
26
+
27
+ def wrap(text, *styles, io: $stdout)
28
+ return text unless enabled?(io)
29
+
30
+ codes = styles.map { |s| CODES[s] }.compact.join
31
+ "#{codes}#{text}#{CODES[:reset]}"
32
+ end
33
+
34
+ def bold(text, io: $stdout)
35
+ wrap(text, :bold, io:)
36
+ end
37
+
38
+ def dim(text, io: $stdout)
39
+ wrap(text, :dim, io:)
40
+ end
41
+
42
+ def green(text, io: $stdout)
43
+ wrap(text, :green, io:)
44
+ end
45
+
46
+ def yellow(text, io: $stdout)
47
+ wrap(text, :yellow, io:)
48
+ end
49
+
50
+ def red(text, io: $stdout)
51
+ wrap(text, :red, io:)
52
+ end
53
+
54
+ def cyan(text, io: $stdout)
55
+ wrap(text, :cyan, io:)
56
+ end
57
+
58
+ def success(text, io: $stdout)
59
+ wrap(text, :bold, :green, io:)
60
+ end
61
+
62
+ def warning(text, io: $stdout)
63
+ wrap(text, :bold, :yellow, io:)
64
+ end
65
+
66
+ def error(text, io: $stdout)
67
+ wrap(text, :bold, :red, io:)
68
+ end
69
+ end
70
+ end
71
+ end
@@ -1,5 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require_relative "colors"
4
+
3
5
  module DataShifter
4
6
  module Internal
5
7
  # Output formatting utilities for data shift runs.
@@ -12,131 +14,164 @@ module DataShifter
12
14
  }.freeze
13
15
 
14
16
  SKIP_REASONS_DISPLAY_LIMIT = 10
17
+ DIVIDER = "=" * 60
18
+ SEPARATOR = "-" * 60
15
19
 
16
20
  module_function
17
21
 
22
+ # --- Public header methods ---
23
+
18
24
  def print_header(io:, shift_class:, total:, label:, dry_run:, transaction_mode:, status_interval:)
19
- io.puts ""
20
- io.puts "=" * 60
21
- io.puts shift_class.name || "DataShifter::Shift (anonymous)"
22
- io.puts "\"#{shift_class.description}\"" if shift_class.description.present?
23
- io.puts "-" * 60
24
- io.puts "Mode: #{dry_run ? "DRY RUN (no changes will be persisted)" : "LIVE"}"
25
+ print_header_top(io:, shift_class:, dry_run:)
25
26
  io.puts "Records: #{total} #{label}"
26
27
  io.puts "Transaction: #{TRANSACTION_MODE_LABELS[transaction_mode]}"
28
+ print_header_bottom(io:, status_interval:)
29
+ end
27
30
 
28
- status_line = build_status_line(status_interval)
29
- io.puts "Status: #{status_line} for live progress (no abort)" if status_line
30
-
31
- io.puts "=" * 60
32
- io.puts ""
31
+ def print_task_header(io:, shift_class:, block_count:, dry_run:, transaction_mode:, status_interval:)
32
+ print_header_top(io:, shift_class:, dry_run:)
33
+ io.puts "Tasks: #{block_count}" if block_count >= 2
34
+ io.puts "Transaction: #{task_transaction_label(transaction_mode)}"
35
+ print_header_bottom(io:, status_interval:)
33
36
  end
34
37
 
38
+ # --- Public summary/progress methods ---
39
+
35
40
  def print_summary(io:, stats:, errors:, start_time:, dry_run:, transaction_mode:, interrupted:, task_name:, last_successful_id:, skip_reasons: {})
36
41
  return unless start_time
37
42
 
38
- elapsed = (Time.current - start_time).round(1)
43
+ has_failures = stats[:failed].positive? || interrupted
44
+
39
45
  io.puts ""
40
- io.puts "=" * 60
41
- io.puts summary_title(dry_run:, interrupted:)
42
- io.puts "-" * 60
43
- io.puts "Duration: #{elapsed}s"
44
- io.puts "Processed: #{stats[:processed]}"
45
- io.puts "Succeeded: #{stats[:succeeded]}"
46
- io.puts "Failed: #{stats[:failed]}"
47
- io.puts "Skipped: #{stats[:skipped]}"
48
- print_skip_reasons(io:, skip_reasons:) if skip_reasons.any?
46
+ io.puts summary_divider(has_failures:, io:)
47
+ io.puts summary_title(dry_run:, interrupted:, has_failures:, io:)
48
+ io.puts Colors.dim(SEPARATOR, io:)
49
+ print_stats(io:, stats:, start_time:, skip_reasons:)
49
50
 
50
51
  print_errors(io:, errors:) if errors.any?
51
52
  print_interrupt_warning(io:, transaction_mode:, dry_run:) if interrupted
52
53
  print_dry_run_instructions(io:, task_name:) if dry_run && !interrupted
53
54
  print_continue_from_hint(io:, task_name:, last_successful_id:, dry_run:, transaction_mode:, errors:)
54
55
 
55
- io.puts "=" * 60
56
+ io.puts summary_divider(has_failures:, io:)
56
57
  end
57
58
 
58
59
  def print_progress(io:, stats:, errors:, start_time:, status_interval:, skip_reasons: {})
59
60
  return unless start_time
60
61
 
61
- elapsed = (Time.current - start_time).round(1)
62
62
  io.puts ""
63
- io.puts "=" * 60
64
-
65
- trigger = if status_interval
66
- "every #{status_interval}s (STATUS_INTERVAL)"
67
- elsif Signal.list.key?("INFO")
68
- "Ctrl+T"
69
- else
70
- "SIGUSR1"
71
- end
72
-
73
- io.puts "STATUS (still running) — triggered by #{trigger}"
74
- io.puts "-" * 60
75
- io.puts "Duration: #{elapsed}s"
76
- io.puts "Processed: #{stats[:processed]}"
77
- io.puts "Succeeded: #{stats[:succeeded]}"
78
- io.puts "Failed: #{stats[:failed]}"
79
- io.puts "Skipped: #{stats[:skipped]}"
80
- print_skip_reasons(io:, skip_reasons:) if skip_reasons.any?
63
+ io.puts Colors.cyan(DIVIDER, io:)
64
+ io.puts "#{Colors.cyan("STATUS (still running)", io:)} — triggered by #{status_trigger(status_interval)}"
65
+ io.puts Colors.dim(SEPARATOR, io:)
66
+ print_stats(io:, stats:, start_time:, skip_reasons:)
81
67
 
82
68
  print_errors(io:, errors:) if errors.any?
83
69
 
84
- io.puts "=" * 60
70
+ io.puts Colors.cyan(DIVIDER, io:)
85
71
  io.puts ""
86
72
  end
87
73
 
88
74
  def print_errors(io:, errors:)
89
75
  io.puts ""
90
- io.puts "ERRORS:"
91
- errors.each do |err|
92
- lines = err[:error].to_s.split("\n")
93
- io.puts " #{err[:record]}: #{lines.first}"
94
- lines.drop(1).each { |line| io.puts " #{line}" }
95
- err[:backtrace]&.each { |line| io.puts " #{line}" }
76
+ io.puts Colors.error("ERRORS:", io:)
77
+ errors.each { |err| print_single_error(io:, err:) }
78
+ end
79
+
80
+ # --- Private helpers ---
81
+
82
+ def print_header_top(io:, shift_class:, dry_run:)
83
+ io.puts ""
84
+ io.puts Colors.dim(DIVIDER, io:)
85
+ io.puts Colors.bold(shift_class.name || "DataShifter::Shift (anonymous)", io:)
86
+ io.puts Colors.dim("\"#{shift_class.description}\"", io:) if shift_class.description.present?
87
+ io.puts Colors.dim(SEPARATOR, io:)
88
+ io.puts "Mode: #{mode_label(dry_run:, io:)}"
89
+ end
90
+
91
+ def print_header_bottom(io:, status_interval:)
92
+ status_line = build_status_line(status_interval)
93
+ io.puts Colors.dim("Status: #{status_line} for live progress (no abort)", io:) if status_line
94
+ io.puts Colors.dim(DIVIDER, io:)
95
+ io.puts ""
96
+ end
97
+
98
+ def print_stats(io:, stats:, start_time:, skip_reasons:)
99
+ elapsed = (Time.current - start_time).round(1)
100
+ io.puts "Duration: #{elapsed}s"
101
+ io.puts "Processed: #{stats[:processed]}"
102
+ io.puts "Succeeded: #{Colors.green(stats[:succeeded].to_s, io:)}"
103
+ io.puts "Failed: #{Colors.red(stats[:failed].to_s, io:)}" if stats[:failed].positive?
104
+ io.puts "Skipped: #{Colors.yellow(stats[:skipped].to_s, io:)}" if stats[:skipped].positive?
105
+ print_skip_reasons(io:, skip_reasons:) if skip_reasons.any?
106
+ end
107
+
108
+ def print_single_error(io:, err:)
109
+ lines = err[:error].to_s.split("\n")
110
+ io.puts " #{Colors.red(err[:record].to_s, io:)}: #{lines.first}"
111
+ lines.drop(1).each { |line| io.puts " #{line}" }
112
+ err[:backtrace]&.each { |line| io.puts Colors.dim(" #{line}", io:) }
113
+ end
114
+
115
+ def mode_label(dry_run:, io:)
116
+ if dry_run
117
+ "#{Colors.cyan("DRY RUN", io:)} (no changes will be persisted)"
118
+ else
119
+ Colors.warning("LIVE", io:)
96
120
  end
97
121
  end
98
122
 
99
- def summary_title(dry_run:, interrupted:)
123
+ def task_transaction_label(mode)
124
+ mode == :per_record ? "per-task" : TRANSACTION_MODE_LABELS[mode]
125
+ end
126
+
127
+ def summary_divider(has_failures:, io:)
128
+ has_failures ? Colors.red(DIVIDER, io:) : Colors.green(DIVIDER, io:)
129
+ end
130
+
131
+ def summary_title(dry_run:, interrupted:, has_failures: false, io: $stdout)
100
132
  base = dry_run ? "SUMMARY (DRY RUN)" : "SUMMARY"
101
- interrupted ? "#{base} - INTERRUPTED" : base
133
+ title = interrupted ? "#{base} - INTERRUPTED" : base
134
+ has_failures ? Colors.error(title, io:) : Colors.success(title, io:)
102
135
  end
103
136
 
104
- def print_interrupt_warning(io:, transaction_mode:, dry_run:)
105
- io.puts ""
106
- if transaction_mode == :none
107
- io.puts "[!] INTERRUPTED: `transaction false` mode was active."
108
- io.puts " Some DB changes may have been applied before interruption."
109
- io.puts " Non-DB side effects (API calls, emails, etc.) are not rolled back."
110
- io.puts " Review the database state before re-running."
111
- elsif dry_run
112
- io.puts "[!] INTERRUPTED: All DB changes have been rolled back (dry run)."
113
- io.puts " Non-DB side effects (API calls, emails, etc.) are not rolled back."
137
+ def status_trigger(status_interval)
138
+ if status_interval
139
+ "every #{status_interval}s (STATUS_INTERVAL)"
140
+ elsif Signal.list.key?("INFO")
141
+ "Ctrl+T"
114
142
  else
115
- io.puts "[!] INTERRUPTED: DB transaction has been rolled back."
116
- io.puts " No DB changes were persisted."
117
- io.puts " Non-DB side effects (API calls, emails, etc.) are not rolled back."
143
+ "SIGUSR1"
118
144
  end
119
145
  end
120
146
 
147
+ def print_interrupt_warning(io:, transaction_mode:, dry_run:)
148
+ msg = if transaction_mode == :none
149
+ "`transaction false` mode was active. Some DB changes may have been applied."
150
+ elsif dry_run
151
+ "All DB changes have been rolled back (dry run)."
152
+ else
153
+ "DB transaction has been rolled back. No DB changes were persisted."
154
+ end
155
+ io.puts ""
156
+ io.puts "#{Colors.warning("[!] INTERRUPTED:", io:)} #{msg}"
157
+ io.puts " Non-DB side effects (API calls, emails, etc.) are not rolled back."
158
+ end
159
+
121
160
  def print_dry_run_instructions(io:, task_name:)
122
161
  io.puts ""
123
- io.puts "[!] No changes were saved."
162
+ io.puts Colors.cyan("[!] No changes were saved.", io:)
124
163
  return unless task_name.present?
125
164
 
126
165
  io.puts "To apply these changes, run:"
127
- io.puts " COMMIT=1 rake data:shift:#{task_name}"
166
+ io.puts " #{Colors.bold("COMMIT=1 rake data:shift:#{task_name}", io:)}"
128
167
  end
129
168
 
130
169
  def print_continue_from_hint(io:, task_name:, last_successful_id:, dry_run:, transaction_mode:, errors:)
131
- return if dry_run
132
- return unless transaction_mode == :none
133
- return if errors.empty?
134
- return unless last_successful_id
135
- return unless task_name.present?
170
+ return if dry_run || transaction_mode != :none || errors.empty? || !last_successful_id || !task_name.present?
136
171
 
137
172
  io.puts ""
138
173
  io.puts "To resume from the last successful record:"
139
- io.puts " CONTINUE_FROM=#{last_successful_id} COMMIT=1 rake data:shift:#{task_name}"
174
+ io.puts " #{Colors.bold("CONTINUE_FROM=#{last_successful_id} COMMIT=1 rake data:shift:#{task_name}", io:)}"
140
175
  end
141
176
 
142
177
  def build_status_line(status_interval)
@@ -9,6 +9,7 @@ require_relative "internal/record_utils"
9
9
  require_relative "internal/progress_bar"
10
10
  require_relative "internal/side_effect_guards"
11
11
  require_relative "internal/log_deduplicator"
12
+ require_relative "internal/colors"
12
13
 
13
14
  # Base class for data shifts. Dry-run by default, progress bars, transaction modes, consistent summaries.
14
15
  #
@@ -67,6 +68,7 @@ module DataShifter
67
68
  class_attribute :_throttle_interval, default: nil
68
69
  class_attribute :_allow_external_requests, default: [], instance_accessor: false
69
70
  class_attribute :_suppress_repeated_logs, default: nil, instance_accessor: false
71
+ class_attribute :_task_blocks, default: [], instance_accessor: false
70
72
 
71
73
  # Internal exception used by skip! to abort the current process_record.
72
74
  # Rescued in _process_one; not propagated.
@@ -128,6 +130,21 @@ module DataShifter
128
130
  self._suppress_repeated_logs = !!enabled
129
131
  end
130
132
 
133
+ # Define a task block to run instead of collection/process_record.
134
+ # Multiple blocks run in sequence; labels appear in errors and summary.
135
+ # Example:
136
+ # task "Fix user A" do
137
+ # User.find(123).update!(...)
138
+ # end
139
+ # task "Fix user B" do
140
+ # User.find(456).update!(...)
141
+ # end
142
+ def task(label = nil, &block)
143
+ raise ArgumentError, "task requires a block" unless block_given?
144
+
145
+ self._task_blocks = (_task_blocks || []).dup + [{ label: label.presence, block: }]
146
+ end
147
+
131
148
  def run!
132
149
  dry_run = Internal::Env.dry_run?
133
150
  result = call(dry_run:)
@@ -139,7 +156,11 @@ module DataShifter
139
156
  # --- Public API (intentionally exposed to subclasses) ---
140
157
 
141
158
  def call
142
- _for_each_record_in(collection) { |record| process_record(record) }
159
+ if self.class._task_blocks.any?
160
+ _run_task_blocks
161
+ else
162
+ _for_each_record_in(collection) { |record| process_record(record) }
163
+ end
143
164
  end
144
165
 
145
166
  def find_exactly!(model, ids)
@@ -163,7 +184,7 @@ module DataShifter
163
184
  end
164
185
 
165
186
  def log(message)
166
- puts message
187
+ puts Internal::Colors.dim(message)
167
188
  end
168
189
 
169
190
  private
@@ -259,11 +280,13 @@ module DataShifter
259
280
  # --- Override points ---
260
281
 
261
282
  def collection
262
- raise NotImplementedError, "#{self.class.name}: override `collection`"
283
+ raise NotImplementedError,
284
+ "#{self.class.name}: override `collection` (or use one or more `task 'label' do ... end` blocks for targeted shifts)."
263
285
  end
264
286
 
265
287
  def process_record(_record)
266
- raise NotImplementedError, "#{self.class.name}: override `process_record`"
288
+ raise NotImplementedError,
289
+ "#{self.class.name}: override `process_record` (or use one or more `task 'label' do ... end` blocks for targeted shifts)."
267
290
  end
268
291
 
269
292
  # --- Record iteration ---
@@ -452,5 +475,88 @@ module DataShifter
452
475
  # Re-raise to trigger transaction rollback in the wrapping transaction block
453
476
  raise Interrupt
454
477
  end
478
+
479
+ # --- Task block execution ---
480
+
481
+ def _run_task_blocks
482
+ _validate_no_collection_or_process_record_override!
483
+
484
+ blocks = self.class._task_blocks
485
+ _reset_tracking
486
+ _print_task_header(blocks.size)
487
+
488
+ ActiveSupport::IsolatedExecutionState[:_data_shifter_current_run] = self
489
+ status_proc = proc { ActiveSupport::IsolatedExecutionState[:_data_shifter_current_run]&.send(:_print_progress) }
490
+ prev_handlers = Internal::SignalHandler.install_status_traps(status_proc)
491
+
492
+ begin
493
+ blocks.each do |entry|
494
+ _execute_task_block(entry)
495
+ end
496
+ fail! "#{@stats[:failed]} task(s) failed" if @errors.any?
497
+ rescue Interrupt
498
+ _handle_interrupt
499
+ ensure
500
+ ActiveSupport::IsolatedExecutionState.delete(:_data_shifter_current_run)
501
+ Internal::SignalHandler.restore_status_traps(prev_handlers)
502
+ end
503
+ end
504
+
505
+ def _validate_no_collection_or_process_record_override!
506
+ collection_overridden = instance_method_owner(:collection) != DataShifter::Shift
507
+ process_record_overridden = instance_method_owner(:process_record) != DataShifter::Shift
508
+
509
+ return unless collection_overridden || process_record_overridden
510
+
511
+ raise ArgumentError,
512
+ "Cannot use task blocks and override collection or process_record; use one mode or the other."
513
+ end
514
+
515
+ def instance_method_owner(method_name)
516
+ self.class.instance_method(method_name).owner
517
+ end
518
+
519
+ def _execute_task_block(entry)
520
+ label = entry[:label]
521
+ block = entry[:block]
522
+
523
+ if _transaction_mode == :per_record && !dry_run?
524
+ ::ActiveRecord::Base.transaction do
525
+ _run_single_task_block(label, block)
526
+ end
527
+ else
528
+ _run_single_task_block(label, block)
529
+ end
530
+ end
531
+
532
+ def _run_single_task_block(label, block)
533
+ puts Internal::Colors.cyan(">> #{label} <<") if label.present?
534
+ instance_exec(&block)
535
+ @stats[:processed] += 1
536
+ @stats[:succeeded] += 1
537
+ rescue SkipRecord
538
+ # skip! already updated @stats[:skipped] and @skip_reasons; continue to next block
539
+ nil
540
+ rescue StandardError => e
541
+ @stats[:failed] += 1
542
+ identifier = label || "task"
543
+ error_text = _format_error(e)
544
+ @errors << { record: identifier, error: error_text, backtrace: e.backtrace&.first(3) }
545
+ _log_error(identifier, error_text)
546
+
547
+ new_message = label.present? ? "#{label}: #{e.message}" : e.message
548
+ raise e.class, new_message, e.backtrace
549
+ end
550
+
551
+ def _print_task_header(block_count)
552
+ Internal::Output.print_task_header(
553
+ io: $stdout,
554
+ shift_class: self.class,
555
+ block_count:,
556
+ dry_run: dry_run?,
557
+ transaction_mode: _transaction_mode,
558
+ status_interval: Internal::Env.status_interval_seconds,
559
+ )
560
+ end
455
561
  end
456
562
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module DataShifter
4
- VERSION = "0.2.0"
4
+ VERSION = "0.3.0"
5
5
  end
@@ -5,6 +5,7 @@
5
5
  # Usage:
6
6
  # rails g data_shift backfill_users
7
7
  # rails g data_shift backfill_users --model=User
8
+ # rails g data_shift fix_order_1234 --task
8
9
  #
9
10
  class DataShiftGenerator < Rails::Generators::NamedBase
10
11
  class_option :model,
@@ -17,6 +18,11 @@ class DataShiftGenerator < Rails::Generators::NamedBase
17
18
  default: false,
18
19
  desc: "Generate RSpec file"
19
20
 
21
+ class_option :task,
22
+ type: :boolean,
23
+ default: false,
24
+ desc: "Generate a task-based shift (uses task blocks instead of collection/process_record)"
25
+
20
26
  def check_for_naming_conflict
21
27
  underscored_name = name.underscore
22
28
 
@@ -44,6 +50,29 @@ class DataShiftGenerator < Rails::Generators::NamedBase
44
50
  model_name_raw = options[:model].to_s.strip
45
51
  @model_name = model_name_raw.present? ? model_name_raw.underscore.singularize.camelize : nil
46
52
 
53
+ if options[:task]
54
+ _create_task_shift_file(underscored_name)
55
+ else
56
+ _create_standard_shift_file(underscored_name)
57
+ end
58
+ end
59
+
60
+ def create_spec_file
61
+ return unless options[:spec]
62
+ return unless rspec_enabled?
63
+
64
+ underscored_name = name.underscore
65
+
66
+ if options[:task]
67
+ _create_task_spec_file(underscored_name)
68
+ else
69
+ _create_standard_spec_file(underscored_name)
70
+ end
71
+ end
72
+
73
+ private
74
+
75
+ def _create_standard_shift_file(underscored_name)
47
76
  collection_body = if @model_name.present?
48
77
  "#{@model_name}.all"
49
78
  else
@@ -76,14 +105,34 @@ class DataShiftGenerator < Rails::Generators::NamedBase
76
105
  RUBY
77
106
  end
78
107
 
79
- def create_spec_file
80
- return unless options[:spec]
81
- return unless rspec_enabled?
108
+ def _create_task_shift_file(underscored_name)
109
+ model_comment = @model_name.present? ? "# #{@model_name}.find(...).update!(...)" : "# Model.find(...).update!(...)"
110
+ task_label = @class_name.underscore.humanize
82
111
 
83
- underscored_name = name.underscore
84
- record_arg = @model_name.present? ? @model_name.underscore : "record"
112
+ create_file "lib/data_shifts/#{@timestamp}_#{underscored_name}.rb", <<~RUBY
113
+ # frozen_string_literal: true
114
+
115
+ # rake data:shift:#{underscored_name} # Dry run (default)
116
+ # COMMIT=1 rake data:shift:#{underscored_name} # Apply changes
117
+
118
+ module DataShifts
119
+ class #{@class_name} < DataShifter::Shift
120
+ description "TODO: Describe this shift"
121
+
122
+ transaction true # or :per_record for per-task transactions
85
123
 
124
+ task "#{task_label}" do
125
+ #{model_comment}
126
+ end
127
+ end
128
+ end
129
+ RUBY
130
+ end
131
+
132
+ def _create_standard_spec_file(underscored_name)
133
+ record_arg = @model_name.present? ? @model_name.underscore : "record"
86
134
  model_for_change = @model_name.present? ? @model_name : "Model"
135
+
87
136
  create_file "spec/lib/data_shifts/#{underscored_name}_spec.rb", <<~RUBY
88
137
  # frozen_string_literal: true
89
138
 
@@ -98,28 +147,53 @@ class DataShiftGenerator < Rails::Generators::NamedBase
98
147
  # Set up test records as needed
99
148
  # let(:#{record_arg}) { create(:#{record_arg}) }
100
149
 
101
- describe "dry run" do
102
- it "does not persist changes" do
103
- expect do
104
- result = run_data_shift(described_class, dry_run: true)
105
- expect(result).to be_ok
106
- end.not_to change(#{model_for_change}, :count)
150
+ context "when dry run" do
151
+ subject(:result) { run_data_shift(described_class, dry_run: true) }
152
+
153
+ it "succeeds without persisting changes" do
154
+ expect { result }.not_to change(#{model_for_change}, :count)
155
+ expect(result).to be_ok
107
156
  end
108
157
  end
109
158
 
110
- describe "commit" do
159
+ context "when commit" do
160
+ subject(:result) { run_data_shift(described_class, commit: true) }
161
+
111
162
  it "applies changes" do
112
- expect do
113
- result = run_data_shift(described_class, commit: true)
114
- expect(result).to be_ok
115
- end.to change(#{model_for_change}, :count)
163
+ expect { result }.to change(#{model_for_change}, :count)
164
+ expect(result).to be_ok
116
165
  end
117
166
  end
118
167
  end
119
168
  RUBY
120
169
  end
121
170
 
122
- private
171
+ def _create_task_spec_file(underscored_name)
172
+ create_file "spec/lib/data_shifts/#{underscored_name}_spec.rb", <<~RUBY
173
+ # frozen_string_literal: true
174
+
175
+ require "rails_helper"
176
+ require "data_shifter/spec_helper"
177
+
178
+ RSpec.describe DataShifts::#{@class_name} do
179
+ include DataShifter::SpecHelper
180
+
181
+ before { allow($stdout).to receive(:puts) }
182
+
183
+ context "when dry run" do
184
+ subject(:result) { run_data_shift(described_class, dry_run: true) }
185
+
186
+ it { is_expected.to be_ok }
187
+ end
188
+
189
+ context "when commit" do
190
+ subject(:result) { run_data_shift(described_class, commit: true) }
191
+
192
+ it { is_expected.to be_ok }
193
+ end
194
+ end
195
+ RUBY
196
+ end
123
197
 
124
198
  def rspec_enabled?
125
199
  # Check if rspec-rails is available and configured as the test framework
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: data_shifter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.0
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Kali Donovan
@@ -116,6 +116,7 @@ files:
116
116
  - lib/data_shifter.rb
117
117
  - lib/data_shifter/configuration.rb
118
118
  - lib/data_shifter/errors.rb
119
+ - lib/data_shifter/internal/colors.rb
119
120
  - lib/data_shifter/internal/env.rb
120
121
  - lib/data_shifter/internal/log_deduplicator.rb
121
122
  - lib/data_shifter/internal/output.rb