data_shifter 0.3.3 → 0.3.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8f9ab924aaeb0638f2061907484bc7ba4f5e795a1ea58e96f1ae8b0a7542b150
4
- data.tar.gz: 24a81ebe9a5e1f0d27d4cec1494fd26c4c8d1209ddca2aba6cd606dec14e4dda
3
+ metadata.gz: 2b117c23db49ab1654f56e2901792147277a957da4b65e308a893197846d24e4
4
+ data.tar.gz: 26cae4d79c699e8f84e52570bde8de7150e31bc812562197c1ffd9f1e4fdaf58
5
5
  SHA512:
6
- metadata.gz: f5834ec7a2545de4342cc3f9205c8ce0f3a0b53fdfbebacd1452b7196c1b7cb606c4fa4d32e8e7d92532ff7e958f5537d2cba51e0176f9d760cdad9fa8b31498
7
- data.tar.gz: cefd020aedd8a63f65f0bb507bcf8843a65fac12debfb78e47cd0e74b485e98746a8d3b805e0c2be6071254488308a1c10b47fb9e5280f13425d6eade6f4113d
6
+ metadata.gz: 6cc508407a213fae51bb740e352b27d8c470496cc7d6608bdf5a209360f55a801e7a60e9028aa62336b2d32475da7c189d9c79215970e950c30dc5577e056b4d
7
+ data.tar.gz: d3d625d0876a3574994ba877cc89e4c698f4ce083ebab597f1b92051122a78047cec201c2794aac7096e1b958d141c15dddc24d24582c6092a1adbb198502370
data/CHANGELOG.md CHANGED
@@ -2,7 +2,15 @@
2
2
 
3
3
  ## [Unreleased]
4
4
 
5
- * N/A
5
+ ## [0.3.4]
6
+
7
+ * [Changed] `COMMIT` / `DRY_RUN` are now parsed as real booleans (`1/true/t/yes/y/on` vs `0/false/f/no/n/off`, case- and whitespace-insensitive). **`DRY_RUN=1` now means a dry run** — previously the `DRY_RUN == "true"` compare read `DRY_RUN=1` as "not dry" and committed the shift, which is data loss for a value that is truthy in every other tool. Unrecognized values (e.g. `COMMIT=garbage`) now raise `ArgumentError` instead of silently dry-running. `DRY_RUN=false` / `COMMIT=1` still commit.
8
+ * [Changed] Bumped the minimum Rails (ActiveRecord, ActiveSupport, Railties) to `>= 7.2`, matching upstream `axn`. Its `on_success` now fires via `ActiveRecord.after_all_transactions_commit`, which requires ActiveRecord 7.2+.
9
+ * [Feature] `throttle` now accepts an optional `per:` keyword. When set, the sleep is inserted only after processing that many records instead of after every record (e.g. `throttle 1.second, per: 100` sleeps once per 100 records).
10
+ * [Feature] `task` now accepts an Axn class as sugar for a one-axn task: `task "label", SomeAxn, foo: 1` forwards to `SomeAxn.call!(foo: 1)`, so a helper axn's failure is never silently swallowed. Keyword args are static (evaluated at class-load); use the block form when you need runtime values. Passing both a class and a block raises.
11
+ * [Feature] `inline_csv` reads CSV colocated with the shift after a `__END__` marker, so small data sets can live alongside the code. Returns the data rows (`CSV::Row` objects by default, so `row["id"]` works); options forward to `CSV.parse`. Typically used as `def collection = inline_csv`. `csv` is required lazily (not a hard dependency) — add `gem "csv"` to your Gemfile if you're on Ruby 3.4+ and hit a load error.
12
+ * [Changed] Configuration now rides on the upstream `Axn::Configurable` DSL instead of a hand-rolled config object. The public API is unchanged — `DataShifter.configure { |c| ... }`, `DataShifter.config.x`, and the per-shift `progress`/`suppress_repeated_logs` overrides all keep working. This raises the minimum `axn` to `>= 0.1.0.alpha.5` (the first release shipping `Axn::Configurable`), now depended on from rubygems rather than a git pin.
13
+ * [Changed] Replaced the removed `log_calls false` DSL with `auto_log false` (axn renamed the method on main). The previous `respond_to?(:log_calls)` guard silently no-op'd, re-enabling axn's per-call logging that shifts intend to suppress.
6
14
 
7
15
  ## [0.3.3]
8
16
 
data/README.md CHANGED
@@ -58,6 +58,33 @@ module DataShifts
58
58
  end
59
59
  ```
60
60
 
61
+ ### Inline CSV data (small data sets)
62
+
63
+ When the data driving a shift is small, you can colocate it with the code after a `__END__` marker instead of keeping a separate file. `inline_csv` parses that section and returns the rows (`CSV::Row` objects by default, so `row["col"]` works):
64
+
65
+ ```ruby
66
+ module DataShifts
67
+ class BackfillTimeZones < DataShifter::Shift
68
+ description "Set time zones from a fixed list"
69
+
70
+ def collection = inline_csv
71
+
72
+ def process_record(row)
73
+ User.find(row["id"]).update!(time_zone: row["time_zone"])
74
+ end
75
+ end
76
+ end
77
+
78
+ __END__
79
+ id,time_zone
80
+ 1,Pacific Time (US & Canada)
81
+ 2,Eastern Time (US & Canada)
82
+ ```
83
+
84
+ Options forward straight to `CSV.parse` (e.g. `inline_csv(col_sep: ";")`). Large data sets (the multi-thousand-row variety) are better kept in a separate `.csv` you load yourself, so they can be opened in a spreadsheet editor.
85
+
86
+ `inline_csv` requires the `csv` library lazily, so it isn't a dependency unless you use it. `csv` ships with Ruby through 3.3 and is a bundled gem on 3.4+; on Ruby 3.4+ you may need to add `gem "csv"` to your Gemfile.
87
+
61
88
  ### Task-based shifts (targeted, one-off changes)
62
89
 
63
90
  For targeted changes to specific records (e.g. fixing a bug for particular IDs), use `task` blocks instead:
@@ -86,6 +113,15 @@ end
86
113
 
87
114
  Task blocks run in the context of the shift instance, so they have access to private helper methods, `dry_run?`, `log`, `skip!`, `find_exactly!`, and any other instance methods you define. Use private methods to DRY up shared lookups across tasks.
88
115
 
116
+ When a task is just a single helper [axn](https://github.com/teamshares/axn), pass the class instead of a block — its keyword args are forwarded to `.call!`, so a failure is reported instead of silently swallowed:
117
+
118
+ ```ruby
119
+ task "Recalculate company totals", RecalculateTotals, company_id: 123
120
+ # equivalent to: task("Recalculate company totals") { RecalculateTotals.call!(company_id: 123) }
121
+ ```
122
+
123
+ The kwargs are evaluated at class-load time, so this form is for static values; use the block form when you need runtime or instance state.
124
+
89
125
  Task blocks:
90
126
 
91
127
  - Run in sequence within the same lifecycle (transaction, dry run protection, summary)
@@ -105,6 +141,8 @@ Shifts run in **dry run** mode by default. DB changes are always rolled back in
105
141
  - **Commit**: `COMMIT=1 rake data:shift:backfill_foo`
106
142
  - (`COMMIT=true` or `DRY_RUN=false` also commit)
107
143
 
144
+ `COMMIT` and `DRY_RUN` are parsed as booleans — `1`, `true`, `t`, `yes`, `y`, `on` are truthy; `0`, `false`, `f`, `no`, `n`, `off` are falsey (case- and whitespace-insensitive). `COMMIT=<truthy>` commits; otherwise `DRY_RUN` decides, defaulting to a dry run — so `DRY_RUN=1` is a **dry run**, not a commit. An unrecognized value (e.g. `COMMIT=please`) raises rather than guessing a side.
145
+
108
146
  ### Automatic side-effect guards (dry run)
109
147
 
110
148
  In **dry run** mode, DataShifter automatically blocks or fakes these side effects so unguarded code is less likely to hit the network or send mail/jobs:
@@ -310,11 +348,14 @@ Skip reasons are grouped: the summary shows the top 10 reasons by count (e.g. `"
310
348
 
311
349
  ```ruby
312
350
  class SomeShift < DataShifter::Shift
313
- throttle 0.1 # sleep seconds between records
314
- progress false # disable progress bar rendering
351
+ throttle 0.1 # sleep 0.1s after every record
352
+ throttle 1.second, per: 100 # sleep 1s after every 100 records
353
+ progress false # disable progress bar rendering
315
354
  end
316
355
  ```
317
356
 
357
+ `per:` is optional. Without it, the sleep runs after every record. With `per: N`, the sleep runs after every Nth record — useful for large collections where per-record sleeping would add too much wall-clock time.
358
+
318
359
 
319
360
  ## Generator
320
361
 
@@ -367,7 +408,8 @@ end
367
408
  ## Requirements
368
409
 
369
410
  - Ruby ≥ 3.2.1
370
- - Rails (ActiveRecord, ActiveSupport, Railties) ≥ 7.0
411
+ - Rails (ActiveRecord, ActiveSupport, Railties) ≥ 7.2
371
412
  - `axn` (Shift classes include `Axn`)
372
413
  - `ruby-progressbar` (for progress bars)
373
414
  - `webmock` (for dry-run HTTP blocking; optional allowlist via `allow_external_requests [...]` / `DataShifter.config.allow_external_requests`)
415
+ - `csv` — only if you use `inline_csv`; required lazily, not a hard dependency (ships with Ruby through 3.3, a bundled gem on 3.4+)
@@ -7,14 +7,27 @@ module DataShifter
7
7
  module Env
8
8
  module_function
9
9
 
10
- # Determine dry_run mode from environment variables.
11
- # COMMIT=1 or COMMIT=true means dry_run=false
12
- # DRY_RUN=false means dry_run=false; default is true
10
+ TRUTHY = %w[1 true t yes y on].freeze
11
+ FALSEY = %w[0 false f no n off].freeze
12
+
13
+ # COMMIT=<truthy> means commit. Otherwise DRY_RUN decides, defaulting to a dry run.
13
14
  def dry_run?
14
- if ENV["COMMIT"].present?
15
- !%w[1 true].include?(ENV["COMMIT"].to_s.downcase)
15
+ return !boolean!("COMMIT") if ENV["COMMIT"].present?
16
+ return true if ENV["DRY_RUN"].blank?
17
+
18
+ boolean!("DRY_RUN")
19
+ end
20
+
21
+ # Raises on anything unrecognized rather than picking a side. The old `DRY_RUN == "true"`
22
+ # compare silently read `DRY_RUN=1` as "not dry" and committed the shift — a value that is
23
+ # truthy in every other tool has to mean dry run here or it means data loss.
24
+ def boolean!(var)
25
+ raw = ENV.fetch(var, nil)
26
+ case raw.to_s.strip.downcase
27
+ when *TRUTHY then true
28
+ when *FALSEY then false
16
29
  else
17
- ENV.fetch("DRY_RUN", "true") == "true"
30
+ raise ArgumentError, "#{var}=#{raw.inspect} is not a boolean — use one of: #{(TRUTHY + FALSEY).join(", ")}"
18
31
  end
19
32
  end
20
33
 
@@ -0,0 +1,55 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "axn"
4
+
5
+ # Global configuration for DataShifter, declared via the Axn::Configurable DSL.
6
+ #
7
+ # Configure via:
8
+ # DataShifter.configure do |config|
9
+ # config.allow_external_requests = ["api.readonly.example.com"]
10
+ # config.suppress_repeated_logs = false
11
+ # end
12
+ #
13
+ # Or access directly:
14
+ # DataShifter.config.progress_enabled = false
15
+ #
16
+ # This supplies DataShifter.config / .configure / .reset_config! plus per-setting
17
+ # accessors and predicates. `progress_enabled` and `suppress_repeated_logs` are
18
+ # `overridable: true`, so individual shifts can override them per-class (see Shift).
19
+ #
20
+ # Required from both data_shifter.rb and shift.rb (idempotent, like any require)
21
+ # so `include DataShifter.overrides` in Shift works whichever is required first.
22
+ module DataShifter
23
+ extend Axn::Configurable
24
+
25
+ # Namespaces DataShifter's overridable settings (`suppress_repeated_logs`, `progress_enabled`) so
26
+ # they can't collide with axn core's own `:core`-namespaced overridable settings — both compose
27
+ # onto every Shift subclass, since Shift `include Axn` as well as `DataShifter.overrides`. Declared
28
+ # explicitly (matching Axn::Configuration's own `config_namespace :core`) even though the default
29
+ # (this module object) is already collision-safe; must come before any `setting`.
30
+ config_namespace :data_shifter
31
+
32
+ # Hosts or regexes allowed for HTTP during dry run only (combined with per-shift allow_external_requests).
33
+ # Has no effect in commit mode — HTTP is unrestricted when dry_run is false.
34
+ setting :allow_external_requests, default: []
35
+
36
+ # Whether to allow loopback HTTP (127.0.0.1, ::1, localhost) during dry runs. Default: true.
37
+ # Loopback is rarely "external" and is needed for tracing/metrics sidecars (Datadog agent on
38
+ # 8126, statsd on 8125, OTLP collector, etc.). Set to false if you want strict net blocking.
39
+ setting :allow_loopback_requests, default: true
40
+
41
+ # Whether to suppress repeated log messages during a shift run. Default: true.
42
+ # Can be overridden per shift with `suppress_repeated_logs true/false`.
43
+ setting :suppress_repeated_logs, default: true, overridable: true
44
+
45
+ # Maximum unique log messages to track for deduplication. Default: 1000.
46
+ # When exceeded, entries with count == 1 are cleared first; repeated entries are kept.
47
+ setting :repeated_log_cap, default: 1000
48
+
49
+ # Global default for progress bar visibility. Default: true.
50
+ # Per-shift `progress true/false` overrides this.
51
+ setting :progress_enabled, default: true, overridable: true
52
+
53
+ # Default status print interval in seconds when ENV STATUS_INTERVAL is not set. Default: nil.
54
+ setting :status_interval_seconds, default: nil
55
+ end
@@ -2,6 +2,7 @@
2
2
 
3
3
  require "axn"
4
4
  require "active_support/isolated_execution_state"
5
+ require_relative "settings"
5
6
  require_relative "internal/env"
6
7
  require_relative "internal/output"
7
8
  require_relative "internal/signal_handler"
@@ -49,10 +50,17 @@ require_relative "internal/colors"
49
50
  module DataShifter
50
51
  class Shift
51
52
  include Axn
53
+ # Per-class overrides for `progress_enabled` and `suppress_repeated_logs`
54
+ # (declared `overridable: true` on the DataShifter module). Adds class-level
55
+ # `progress_enabled` (bare reader resolves the override, else falls back to
56
+ # config) / `progress_enabled_override` (and likewise for suppress_repeated_logs)
57
+ # accessors; the thin `progress` / `suppress_repeated_logs` aliases below
58
+ # preserve the historical public DSL.
59
+ include DataShifter.overrides
52
60
 
53
61
  expects :dry_run, type: :boolean, default: true
54
62
 
55
- log_calls false if respond_to?(:log_calls)
63
+ auto_log false
56
64
 
57
65
  around :_with_log_deduplication
58
66
  around :_with_side_effect_guards
@@ -62,12 +70,11 @@ module DataShifter
62
70
  on_error :_print_summary_for_axn_error
63
71
 
64
72
  class_attribute :_transaction_mode, default: :single
65
- class_attribute :_progress_enabled, default: nil
66
73
  class_attribute :_description, default: nil
67
74
  class_attribute :_task_name, default: nil
68
75
  class_attribute :_throttle_interval, default: nil
76
+ class_attribute :_throttle_per, default: 1
69
77
  class_attribute :_allow_external_requests, default: [], instance_accessor: false
70
- class_attribute :_suppress_repeated_logs, default: nil, instance_accessor: false
71
78
  class_attribute :_task_blocks, default: [], instance_accessor: false
72
79
 
73
80
  # Internal exception used by skip! to abort the current process_record.
@@ -105,16 +112,22 @@ module DataShifter
105
112
  end
106
113
  end
107
114
 
115
+ # Per-shift override for progress bar visibility. Overrides DataShifter.config.progress_enabled.
116
+ # Thin alias over the `progress_enabled` override accessor generated by DataShifter.overrides.
117
+ # progress false # set override (coerced to a boolean)
118
+ # progress # read the raw override (nil when unset; does NOT fall back to config)
108
119
  def progress(enabled = nil)
109
120
  if enabled.nil?
110
- _progress_enabled
121
+ raw = progress_enabled_override
122
+ Axn::Configurable::UNSET.equal?(raw) ? nil : raw
111
123
  else
112
- self._progress_enabled = !!enabled
124
+ progress_enabled(!!enabled)
113
125
  end
114
126
  end
115
127
 
116
- def throttle(interval)
128
+ def throttle(interval, per: 1)
117
129
  self._throttle_interval = interval
130
+ self._throttle_per = per
118
131
  end
119
132
 
120
133
  # Allow these hosts (or regexes) for HTTP during dry run only. Combines with DataShifter.config.allow_external_requests.
@@ -125,22 +138,36 @@ module DataShifter
125
138
  end
126
139
 
127
140
  # Enable/disable log deduplication for this shift. Overrides DataShifter.config.suppress_repeated_logs.
141
+ # Thin alias over the `suppress_repeated_logs` override accessor generated by DataShifter.overrides
142
+ # (which would otherwise read the resolved value when called with no args); we always treat a call
143
+ # with an argument as a set, coercing to a boolean.
128
144
  # Example: suppress_repeated_logs false
129
145
  def suppress_repeated_logs(enabled)
130
- self._suppress_repeated_logs = !!enabled
146
+ super(!!enabled)
131
147
  end
132
148
 
133
- # Define a task block to run instead of collection/process_record.
134
- # Multiple blocks run in sequence; labels appear in errors and summary.
135
- # Example:
149
+ # Define a task to run instead of collection/process_record.
150
+ # Multiple tasks run in sequence; labels appear in errors and summary.
151
+ #
152
+ # Block form (arbitrary code, runtime/instance state available):
136
153
  # task "Fix user A" do
137
154
  # User.find(123).update!(...)
138
155
  # end
139
- # task "Fix user B" do
140
- # User.find(456).update!(...)
141
- # end
142
- def task(label = nil, &block)
143
- raise ArgumentError, "task requires a block" unless block_given?
156
+ #
157
+ # Axn class form (sugar for one helper axn; kwargs are static, evaluated
158
+ # at class-load time, and forwarded to .call! so a failure is never
159
+ # silently swallowed). Use the block form when you need runtime values:
160
+ # task "Recalculate totals", RecalculateTotals, company_id: 123
161
+ # # => RecalculateTotals.call!(company_id: 123)
162
+ def task(label = nil, axn = nil, **axn_kwargs, &block)
163
+ if axn
164
+ raise ArgumentError, "task accepts either an Axn class or a block, not both" if block
165
+ raise ArgumentError, "task expected an Axn class but got #{axn.inspect}" unless axn.is_a?(Class) && axn.include?(Axn)
166
+
167
+ block = -> { axn.call!(**axn_kwargs) }
168
+ elsif block.nil?
169
+ raise ArgumentError, "task requires a block or an Axn class"
170
+ end
144
171
 
145
172
  self._task_blocks = (_task_blocks || []).dup + [{ label: label.presence, block: }]
146
173
  end
@@ -174,6 +201,24 @@ module DataShifter
174
201
  ids.map { |id| records_by_id[id] }
175
202
  end
176
203
 
204
+ # Parse CSV colocated with the shift, after a `__END__` marker, so small
205
+ # data sets can live alongside the code. Returns the data rows as an array
206
+ # (CSV::Row objects when headers: true, the default — so `row["id"]` works);
207
+ # extra options forward straight to CSV.parse. Typically used as the
208
+ # collection:
209
+ # def collection = inline_csv
210
+ # def process_record(row) = User.find(row["id"]).update!(...)
211
+ # __END__
212
+ # id,...
213
+ #
214
+ # Lazily requires `csv` (a bundled gem on Ruby 3.4+); raises with a hint to
215
+ # add it to the Gemfile if unavailable.
216
+ def inline_csv(**csv_opts)
217
+ _require_csv!
218
+ parsed = CSV.parse(_inline_data_body, headers: true, **csv_opts)
219
+ parsed.is_a?(CSV::Table) ? parsed.each.to_a : parsed
220
+ end
221
+
177
222
  def dry_run? = dry_run
178
223
 
179
224
  def skip!(reason = nil)
@@ -192,7 +237,10 @@ module DataShifter
192
237
  # --- Axn lifecycle hooks ---
193
238
 
194
239
  def _with_log_deduplication(chain)
195
- effective = self.class._suppress_repeated_logs.nil? ? DataShifter.config.suppress_repeated_logs : self.class._suppress_repeated_logs
240
+ # DataShifter.resolve_override_for, not the generated `suppress_repeated_logs` reader:
241
+ # that name is shadowed by Shift's own class method (a required-arg setter, see above),
242
+ # which would raise ArgumentError rather than resolve the override + config fallback.
243
+ effective = DataShifter.resolve_override_for(self.class, :suppress_repeated_logs)
196
244
  unless effective && defined?(::Rails) && ::Rails.respond_to?(:logger) && ::Rails.logger
197
245
  chain.call
198
246
  return
@@ -399,19 +447,29 @@ module DataShifter
399
447
  end
400
448
 
401
449
  def _iterate(enum, total)
402
- progress_on = _progress_enabled.nil? ? DataShifter.config.progress_enabled : _progress_enabled
450
+ # DataShifter.resolve_override_for: the shadow-proof framework path (see
451
+ # _with_log_deduplication). progress_enabled isn't currently shadowed on Shift,
452
+ # but resolving this way is consistent and doesn't depend on that staying true.
453
+ progress_on = DataShifter.resolve_override_for(self.class, :progress_enabled)
403
454
  bar = Internal::ProgressBar.create(total:, dry_run: dry_run?, enabled: progress_on)
455
+ throttle_count = 0
404
456
  if enum.respond_to?(:find_each)
405
457
  enum.find_each do |record|
406
458
  _process_one(record) { yield record }
407
459
  bar&.increment
408
- sleep(_throttle_interval) if _throttle_interval
460
+ if _throttle_interval
461
+ throttle_count += 1
462
+ sleep(_throttle_interval) if (throttle_count % _throttle_per).zero?
463
+ end
409
464
  end
410
465
  else
411
466
  enum.each do |record|
412
467
  _process_one(record) { yield record }
413
468
  bar&.increment
414
- sleep(_throttle_interval) if _throttle_interval
469
+ if _throttle_interval
470
+ throttle_count += 1
471
+ sleep(_throttle_interval) if (throttle_count % _throttle_per).zero?
472
+ end
415
473
  end
416
474
  end
417
475
  end
@@ -445,6 +503,44 @@ module DataShifter
445
503
  _print_progress
446
504
  end
447
505
 
506
+ # `csv` is lazily required (not a hard dependency) so it isn't dragged into
507
+ # apps that never call inline_csv. csv ships with Ruby through 3.3 and is a
508
+ # bundled gem on 3.4+, so this normally just works; on 3.5+ without it in the
509
+ # Gemfile we surface an actionable message instead of a bare LoadError.
510
+ def _require_csv!
511
+ require "csv"
512
+ rescue LoadError
513
+ raise LoadError, 'inline_csv needs the csv library. Add `gem "csv"` to your Gemfile (csv is no longer a default gem on Ruby 3.4+).'
514
+ end
515
+
516
+ # The raw text after this shift file's `__END__` marker. Resolves the
517
+ # source file via the class constant (works however the shift is invoked,
518
+ # unlike Ruby's `DATA`, which is only defined for the main script). Uses
519
+ # Ripper (not a text-based split) to find the real `__END__` token, so a
520
+ # `__END__`-looking line inside a heredoc or comment isn't mistaken for it.
521
+ def _inline_data_body
522
+ @_inline_data_body ||= begin
523
+ class_name = self.class.name
524
+ source = class_name && Object.const_source_location(class_name)&.first
525
+ raise ArgumentError, "inline_csv requires the shift to be a named class defined in a file" unless source && File.exist?(source)
526
+
527
+ require "ripper"
528
+ content = File.read(source)
529
+ offset = 0
530
+ found = false
531
+ Ripper.lex(content).each do |(_pos, type, token, _state)|
532
+ offset += token.length
533
+ if type == :on___end__
534
+ found = true
535
+ break
536
+ end
537
+ end
538
+ raise ArgumentError, "inline_csv: no __END__ data section found in #{source}" unless found
539
+
540
+ content[offset..]
541
+ end
542
+ end
543
+
448
544
  def _format_error(e)
449
545
  msg = e.message.to_s
450
546
  msg += "\n Caused by: #{e.cause.class}: #{e.cause.message}" if e.respond_to?(:cause) && e.cause
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module DataShifter
4
- VERSION = "0.3.3"
4
+ VERSION = "0.3.4"
5
5
  end
data/lib/data_shifter.rb CHANGED
@@ -1,27 +1,8 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require_relative "data_shifter/version"
4
- require_relative "data_shifter/configuration"
4
+ require_relative "data_shifter/settings"
5
5
  require_relative "data_shifter/errors"
6
6
  require_relative "data_shifter/internal/rake_exception_reporting"
7
7
  require_relative "data_shifter/shift"
8
8
  require_relative "data_shifter/railtie"
9
-
10
- module DataShifter
11
- class << self
12
- # Returns the global configuration instance.
13
- def config
14
- @config ||= Configuration.new
15
- end
16
-
17
- # Yields the configuration for block-style setup.
18
- #
19
- # DataShifter.configure do |c|
20
- # c.allow_external_requests = ["api.readonly.example.com"]
21
- # c.suppress_repeated_logs = false
22
- # end
23
- def configure
24
- yield config
25
- end
26
- end
27
- end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: data_shifter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.3
4
+ version: 0.3.4
5
5
  platform: ruby
6
6
  authors:
7
7
  - Kali Donovan
@@ -15,35 +15,35 @@ dependencies:
15
15
  requirements:
16
16
  - - ">="
17
17
  - !ruby/object:Gem::Version
18
- version: '7.0'
18
+ version: '7.2'
19
19
  type: :runtime
20
20
  prerelease: false
21
21
  version_requirements: !ruby/object:Gem::Requirement
22
22
  requirements:
23
23
  - - ">="
24
24
  - !ruby/object:Gem::Version
25
- version: '7.0'
25
+ version: '7.2'
26
26
  - !ruby/object:Gem::Dependency
27
27
  name: activesupport
28
28
  requirement: !ruby/object:Gem::Requirement
29
29
  requirements:
30
30
  - - ">="
31
31
  - !ruby/object:Gem::Version
32
- version: '7.0'
32
+ version: '7.2'
33
33
  type: :runtime
34
34
  prerelease: false
35
35
  version_requirements: !ruby/object:Gem::Requirement
36
36
  requirements:
37
37
  - - ">="
38
38
  - !ruby/object:Gem::Version
39
- version: '7.0'
39
+ version: '7.2'
40
40
  - !ruby/object:Gem::Dependency
41
41
  name: axn
42
42
  requirement: !ruby/object:Gem::Requirement
43
43
  requirements:
44
44
  - - ">="
45
45
  - !ruby/object:Gem::Version
46
- version: 0.1.0.pre.alpha.4
46
+ version: 0.1.0.pre.alpha.5
47
47
  - - "<"
48
48
  - !ruby/object:Gem::Version
49
49
  version: 0.2.0
@@ -53,7 +53,7 @@ dependencies:
53
53
  requirements:
54
54
  - - ">="
55
55
  - !ruby/object:Gem::Version
56
- version: 0.1.0.pre.alpha.4
56
+ version: 0.1.0.pre.alpha.5
57
57
  - - "<"
58
58
  - !ruby/object:Gem::Version
59
59
  version: 0.2.0
@@ -63,14 +63,14 @@ dependencies:
63
63
  requirements:
64
64
  - - ">="
65
65
  - !ruby/object:Gem::Version
66
- version: '7.0'
66
+ version: '7.2'
67
67
  type: :runtime
68
68
  prerelease: false
69
69
  version_requirements: !ruby/object:Gem::Requirement
70
70
  requirements:
71
71
  - - ">="
72
72
  - !ruby/object:Gem::Version
73
- version: '7.0'
73
+ version: '7.2'
74
74
  - !ruby/object:Gem::Dependency
75
75
  name: ruby-progressbar
76
76
  requirement: !ruby/object:Gem::Requirement
@@ -107,14 +107,10 @@ executables: []
107
107
  extensions: []
108
108
  extra_rdoc_files: []
109
109
  files:
110
- - ".husky/pre-commit"
111
- - ".lintstagedrc"
112
110
  - CHANGELOG.md
113
111
  - LICENSE.txt
114
112
  - README.md
115
- - Rakefile
116
113
  - lib/data_shifter.rb
117
- - lib/data_shifter/configuration.rb
118
114
  - lib/data_shifter/errors.rb
119
115
  - lib/data_shifter/internal/colors.rb
120
116
  - lib/data_shifter/internal/env.rb
@@ -127,6 +123,7 @@ files:
127
123
  - lib/data_shifter/internal/side_effect_guards.rb
128
124
  - lib/data_shifter/internal/signal_handler.rb
129
125
  - lib/data_shifter/railtie.rb
126
+ - lib/data_shifter/settings.rb
130
127
  - lib/data_shifter/shift.rb
131
128
  - lib/data_shifter/spec_helper.rb
132
129
  - lib/data_shifter/version.rb
data/.husky/pre-commit DELETED
@@ -1 +0,0 @@
1
- npx lint-staged
data/.lintstagedrc DELETED
@@ -1,3 +0,0 @@
1
- {
2
- "*.rb": ["bundle exec rubocop -A -c .rubocop.yml --force-exclusion"]
3
- }
data/Rakefile DELETED
@@ -1,16 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require "bundler/gem_tasks"
4
- require "rubocop/rake_task"
5
-
6
- task :spec do
7
- sh "bundle exec rspec"
8
- end
9
-
10
- RuboCop::RakeTask.new
11
-
12
- task default: %i[spec rubocop]
13
-
14
- # Ensure specs and rubocop pass before release (must run first; enhance appends)
15
- release_task = Rake::Task["release"]
16
- release_task.prerequisites.unshift(:default)
@@ -1,48 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module DataShifter
4
- # Global configuration for DataShifter.
5
- #
6
- # Configure via:
7
- # DataShifter.configure do |config|
8
- # config.allow_external_requests = ["api.readonly.example.com"]
9
- # config.suppress_repeated_logs = true
10
- # end
11
- #
12
- # Or access directly:
13
- # DataShifter.config.progress_enabled = false
14
- class Configuration
15
- # Hosts or regexes allowed for HTTP during dry run only (combined with per-shift allow_external_requests).
16
- # Has no effect in commit mode — HTTP is unrestricted when dry_run is false.
17
- attr_accessor :allow_external_requests
18
-
19
- # Whether to allow loopback HTTP (127.0.0.1, ::1, localhost) during dry runs. Default: true.
20
- # Loopback is rarely "external" and is needed for tracing/metrics sidecars (Datadog agent on
21
- # 8126, statsd on 8125, OTLP collector, etc.). Set to false if you want strict net blocking.
22
- attr_accessor :allow_loopback_requests
23
-
24
- # Whether to suppress repeated log messages during a shift run. Default: true.
25
- # Can be overridden per shift with `suppress_repeated_logs true/false`.
26
- attr_accessor :suppress_repeated_logs
27
-
28
- # Maximum unique log messages to track for deduplication. Default: 1000.
29
- # When exceeded, entries with count == 1 are cleared first; repeated entries are kept.
30
- attr_accessor :repeated_log_cap
31
-
32
- # Global default for progress bar visibility. Default: true.
33
- # Per-shift `progress true/false` overrides this.
34
- attr_accessor :progress_enabled
35
-
36
- # Default status print interval in seconds when ENV STATUS_INTERVAL is not set. Default: nil.
37
- attr_accessor :status_interval_seconds
38
-
39
- def initialize
40
- @allow_external_requests = []
41
- @allow_loopback_requests = true
42
- @suppress_repeated_logs = true
43
- @repeated_log_cap = 1000
44
- @progress_enabled = true
45
- @status_interval_seconds = nil
46
- end
47
- end
48
- end