dekiru 0.8.0 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: a1c032693e2658b61ee9744b7e872a4379eb194d9f4e3a4ac1147c4b5025682b
4
- data.tar.gz: 14321e4bf0b2574358bfc29147bf801480801181af1c8e40879d8f4357cc7fca
3
+ metadata.gz: 54931a9e4393d521550361c655a87e561e548691c3be644cb2b881e4a9c55aa9
4
+ data.tar.gz: 663476875859ca0dcd5a5fa755f1b48c26eadbeca4df848f717ed8cde5aea9be
5
5
  SHA512:
6
- metadata.gz: a5f36a181ddd30cf6500c5d454ceffb5cb058a5aeb5de29b1f44b96eb9eb6fe4796522bd8e0b20f0e3ae09cb5ca83f86688ac7ff1df4421522c0bbacc83ad9ac
7
- data.tar.gz: e39f08ff48c0f412c7fbe7ff755b2c2dd77378aca03172bff6622b09d26c6bc7f94e08cefbe5ebdff575504550520da146e7b4f312b9c3b14fef6b6f96242523
6
+ metadata.gz: 6c6ad53a0f07879ddcaed43927406cc58dbc176d346655f400539fc27685f1470239358c2cea4a29ec4c9c2f375fce40fed98cc2cb521e2a0e8b382026357e7a
7
+ data.tar.gz: 3d9074d7047a77b7089cc7b9d4e4bdacd8e6c9273dd03649227e073d0c84e7c5f8286bc03cba48072c121f8c244d7e87d6e358260aed5ada59c73ccdc70d208c
data/.gitignore CHANGED
@@ -16,3 +16,4 @@ spec/reports
16
16
  test/tmp
17
17
  test/version_tmp
18
18
  tmp
19
+ vendor/bundle
data/README.md CHANGED
@@ -124,122 +124,7 @@ end
124
124
 
125
125
  ## Data Migration Operator
126
126
 
127
- 実行しながら進捗を表示したり、処理の最後に実行の確認をしたりといった、データ移行作業をするときに必要な処理を以下のような script を作成することで、実現できるようになります。
128
-
129
- ```ruby
130
- # scripts/demo.rb
131
- Dekiru::DataMigrationOperator.execute('Demo migration') do
132
- targets = User.where("email LIKE '%sonicgarden%'")
133
-
134
- log "all targets count: #{targets.count}"
135
- find_each_with_progress(targets) do |user|
136
- user.update(admin: true)
137
- end
138
-
139
- log "updated user count: #{User.where("email LIKE '%sonicgarden%'").where(admin: true).count}"
140
- end
141
- ```
142
-
143
- ```
144
- $ bin/rails r scripts/demo.rb
145
- Start: Demo migration at 2019-05-24 18:29:57 +0900
146
-
147
- all targets count: 30
148
- Time: 00:00:00 |=================>>| 100% Progress
149
- updated user count: 30
150
-
151
- Are you sure to commit? (yes/no) > yes
152
-
153
- Finished successfully: Demo migration
154
- Total time: 6.35 sec
155
- ```
156
-
157
- また`warning_side_effects: true`オプションを付けて実行することで、データ移行作業で発生した副作用が表示されるようになります。
158
-
159
- ```ruby
160
- Dekiru::DataMigrationOperator.execute('Demo migration', warning_side_effects: true) do
161
- # ...
162
- end
163
- ```
164
-
165
- ```
166
- $ bin/rails r scripts/demo.rb
167
- Start: Demo migration at 2019-05-24 18:29:57 +0900
168
-
169
- all targets count: 30
170
- Time: 00:00:00 |=================>>| 100% Progress
171
- updated user count: 30
172
-
173
- Write Queries!!
174
- 30 call: Update "users" SET ...
175
-
176
- Enqueued Jobs!!
177
- 10 call: NotifyJob
178
-
179
- Deliverd Mailers!!
180
- 10 call: UserMailer
181
-
182
- Are you sure to commit? (yes/no) > yes
183
- ```
184
-
185
- ジェネレータを使って `Dekiru::DataMigrationOperator` を使ったメンテナンススクリプトを生成することができます。ファイル名にはスクリプトファイルを生成した日付が prefix として付与されます。
186
-
187
- ```
188
- $ bin/rails g maintenance_script demo_migration
189
- $ cat scripts/20230118_demo_migration.rb
190
- Dekiru::DataMigrationOperator.execute('demo_migration') do
191
- # write here
192
- end
193
- ```
194
-
195
- ファイルの出力先ディレクトリは、デフォルトではアプリケーションルート直下の `scripts` ディレクトリです。設定で出力先ディレクトリを変更することもできます。
196
-
197
- ```ruby
198
- # config/initializer/dekiru.rb
199
- Dekiru.configure do |config|
200
- config.maintenance_script_directory = 'scripts/maintenance'
201
- end
202
- ```
203
-
204
- ### Dekiru::TransactionProvider
205
-
206
- `Dekiru::DataMigrationOperator` を使うスクリプトにおいて、複数データベースへの書き込みが必要な場合など、`ActiveRecord::Base.transaction` によるトランザクション開始だけでは不十分な場合があります。`Dekiru::TransactionProvider` を実装することで `Dekiru::DataMigrationOperator` のトランザクション開始の挙動をカスタマイズすることができます。
207
-
208
- 以下のようなアプリケーションコードがあるとします。
209
-
210
- ```ruby
211
- class LegacyRecord < ApplicationRecord
212
- connects_to database: { writing: :legacy, reading: :legacy }
213
- end
214
-
215
- class ApplicationRecord < ActiveRecord::Base
216
- connects_to database: { writing: :primary, reading: :primary }
217
-
218
- def self.with_legacy_transaction
219
- ActiveRecord::Base.transaction do
220
- LegacyRecord.transaction do
221
- yield
222
- end
223
- end
224
- end
225
- end
226
- ```
227
-
228
- `Dekiru::DataMigrationOperator` においても `ApplicationRecord.with_legacy_transaction` を使ってトランザクション開始するために、以下のような設定を用意します。
229
-
230
- ```ruby
231
- # config/initializer/dekiru.rb
232
- class MyTransactionProvider < Dekiru::TransactionProvider
233
- def within_transaction(&)
234
- ApplicationRecord.with_legacy_transaction(&)
235
- end
236
- end
237
-
238
- Dekiru.configure do |config|
239
- config.transaction_provider = MyTransactionProvider.new
240
- end
241
- ```
242
-
127
+ For data migration operations, please use the separate [`dekiru-data_migration` gem](https://github.com/SonicGarden/dekiru-data_migration) which provides the `Dekiru::DataMigrationOperator` functionality.
243
128
 
244
129
  ## Refinements
245
130
 
data/dekiru.gemspec CHANGED
@@ -22,7 +22,7 @@ Gem::Specification.new do |gem|
22
22
  gem.required_ruby_version = '>= 3.1.0'
23
23
 
24
24
  gem.add_dependency 'rails', '>= 7.0'
25
- gem.add_dependency 'ruby-progressbar'
25
+ gem.add_dependency 'dekiru-data_migration'
26
26
  gem.add_development_dependency 'rake'
27
27
  gem.add_development_dependency 'rspec'
28
28
  gem.add_development_dependency 'rubocop'
@@ -1,3 +1,3 @@
1
1
  module Dekiru
2
- VERSION = '0.8.0'
2
+ VERSION = '1.0.0'
3
3
  end
data/lib/dekiru.rb CHANGED
@@ -1,10 +1,8 @@
1
1
  require 'dekiru/version'
2
2
  require 'dekiru/railtie' if defined?(::Rails)
3
3
  require 'dekiru/helper'
4
- require 'dekiru/data_migration_operator'
5
4
  require 'dekiru/mail_security_interceptor'
6
5
  require 'dekiru/camelize_hash'
7
- require 'dekiru/transaction_provider'
8
6
 
9
7
  require 'active_support'
10
8
  require 'active_support/all'
@@ -21,12 +19,10 @@ module Dekiru
21
19
  end
22
20
 
23
21
  class Configuration
24
- attr_accessor :mail_security_hook, :maintenance_script_directory, :transaction_provider
22
+ attr_accessor :mail_security_hook
25
23
 
26
24
  def initialize
27
25
  @mail_security_hook = false # default
28
- @maintenance_script_directory = 'scripts'
29
- @transaction_provider = TransactionProvider.new
30
26
  end
31
27
  end
32
28
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: dekiru
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.8.0
4
+ version: 1.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Akihiro Matsumura
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2025-04-14 00:00:00.000000000 Z
11
+ date: 2025-06-23 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rails
@@ -25,7 +25,7 @@ dependencies:
25
25
  - !ruby/object:Gem::Version
26
26
  version: '7.0'
27
27
  - !ruby/object:Gem::Dependency
28
- name: ruby-progressbar
28
+ name: dekiru-data_migration
29
29
  requirement: !ruby/object:Gem::Requirement
30
30
  requirements:
31
31
  - - ">="
@@ -133,19 +133,13 @@ files:
133
133
  - lib/dekiru/capybara/helpers/wait_for_position_stable.rb
134
134
  - lib/dekiru/capybara/legacy_helpers.rb
135
135
  - lib/dekiru/capybara/matchers.rb
136
- - lib/dekiru/data_migration_operator.rb
137
136
  - lib/dekiru/helper.rb
138
137
  - lib/dekiru/mail_security_interceptor.rb
139
138
  - lib/dekiru/railtie.rb
140
139
  - lib/dekiru/task_with_logger.rb
141
140
  - lib/dekiru/tasks/db.rake
142
- - lib/dekiru/transaction_provider.rb
143
141
  - lib/dekiru/version.rb
144
- - lib/generators/maintenance_script/USAGE
145
- - lib/generators/maintenance_script/maintenance_script_generator.rb
146
- - lib/generators/maintenance_script/templates/maintenance_script.rb.erb
147
142
  - spec/dekiru/camelize_hash_spec.rb
148
- - spec/dekiru/data_migration_operator_spec.rb
149
143
  - spec/dekiru/mail_security_interceptor_spec.rb
150
144
  - spec/spec_helper.rb
151
145
  - spec/supports/action_mailer.rb
@@ -177,7 +171,6 @@ specification_version: 4
177
171
  summary: Usefull helper methods for Ruby on Rails
178
172
  test_files:
179
173
  - spec/dekiru/camelize_hash_spec.rb
180
- - spec/dekiru/data_migration_operator_spec.rb
181
174
  - spec/dekiru/mail_security_interceptor_spec.rb
182
175
  - spec/spec_helper.rb
183
176
  - spec/supports/action_mailer.rb
@@ -1,154 +0,0 @@
1
- require 'ruby-progressbar'
2
-
3
- module Dekiru
4
- class DataMigrationOperator
5
- class NestedTransactionError < StandardError ; end
6
-
7
- attr_reader :title, :stream, :logger, :result, :canceled, :started_at, :ended_at, :error
8
-
9
- def self.execute(title, options = {}, &block)
10
- self.new(title, options).execute(&block)
11
- end
12
-
13
- def initialize(title, options = {})
14
- @title = title
15
- @options = options
16
- @logger = @options.fetch(:logger) { Logger.new(Rails.root.join("log/data_migration_#{Time.current.strftime("%Y%m%d%H%M")}.log")) }
17
- @stream = @options.fetch(:output, $stdout)
18
- @without_transaction = @options.fetch(:without_transaction, false)
19
- @side_effects = Hash.new do |hash, key|
20
- hash[key] = Hash.new(0)
21
- end
22
- end
23
-
24
- def execute(&block)
25
- @started_at = Time.current
26
- log "Start: #{title} at #{started_at}\n\n"
27
- if @without_transaction
28
- run(&block)
29
- @result = true
30
- else
31
- raise NestedTransactionError if current_transaction_open?
32
-
33
- @result = transaction_provider.within_transaction do
34
- run(&block)
35
- log "Finished execution: #{title}"
36
- confirm?("\nAre you sure to commit?")
37
- end
38
- end
39
- log "Finished successfully: #{title}" if @result == true
40
- rescue => e
41
- @error = e
42
- @result = false
43
- ensure
44
- @ended_at = Time.current
45
- log "Total time: #{self.duration.round(2)} sec"
46
-
47
- raise error if error
48
-
49
- return @result
50
- end
51
-
52
- def duration
53
- ((self.ended_at || Time.current) - self.started_at)
54
- end
55
-
56
- def each_with_progress(enum, options = {})
57
- options = options.dup
58
- options[:total] ||= ((enum.size == Float::INFINITY ? nil : enum.size) rescue nil)
59
- options[:format] ||= options[:total] ? '%a |%b>>%i| %p%% %t' : '%a |%b>>%i| ??%% %t'
60
- options[:output] = stream
61
-
62
- @pb = ::ProgressBar.create(options)
63
- enum.each do |item|
64
- yield item
65
- @pb.increment
66
- end
67
- @pb.finish
68
- end
69
-
70
- def find_each_with_progress(target_scope, options = {}, &block)
71
- # `LocalJumpError: no block given (yield)` が出る場合、 find_each メソッドが enumerator を返していない可能性があります
72
- # 直接 each_with_progress を使うか、 find_each が enumerator を返すように修正してください
73
- each_with_progress(target_scope.find_each, options, &block)
74
- end
75
-
76
- private
77
-
78
- def log(message)
79
- if @pb && !@pb.finished?
80
- @pb.log(message)
81
- else
82
- stream.puts(message)
83
- end
84
-
85
- logger&.info(message.squish)
86
- end
87
-
88
- def confirm?(message = 'Are you sure?')
89
- loop do
90
- stream.print "#{message} (yes/no) > "
91
- case STDIN.gets.strip
92
- when 'yes'
93
- newline
94
- return true
95
- when 'no'
96
- newline
97
- cancel!
98
- end
99
- end
100
- end
101
-
102
- def newline
103
- stream.puts('')
104
- end
105
-
106
- def cancel!
107
- log "Canceled: #{title}"
108
- raise ActiveRecord::Rollback
109
- end
110
-
111
- def handle_notification(*args)
112
- event = ActiveSupport::Notifications::Event.new(*args)
113
-
114
- increment_side_effects(:enqueued_jobs, event.payload[:job].class.name) if event.payload[:job]
115
- increment_side_effects(:deliverd_mailers, event.payload[:mailer]) if event.payload[:mailer]
116
-
117
- if event.payload[:sql] && /\A\s*(insert|update|delete)/i.match?(event.payload[:sql])
118
- increment_side_effects(:write_queries, event.payload[:sql])
119
- end
120
- end
121
-
122
- def increment_side_effects(type, value)
123
- @side_effects[type][value] += 1
124
- end
125
-
126
- def warning_side_effects(&block)
127
- ActiveSupport::Notifications.subscribed(method(:handle_notification), /^(sql|enqueue|deliver)/) do
128
- instance_eval(&block)
129
- end
130
-
131
- @side_effects.each do |name, items|
132
- newline
133
- log "#{name.to_s.titlecase}!!"
134
- items.sort_by { |v, c| c }.reverse.slice(0, 20).each do |value, count|
135
- log "#{count} call: #{value}"
136
- end
137
- end
138
- end
139
-
140
- def run(&block)
141
- if @options.fetch(:warning_side_effects, true)
142
- warning_side_effects(&block)
143
- else
144
- instance_eval(&block)
145
- end
146
- end
147
-
148
- def transaction_provider
149
- Dekiru.configuration.transaction_provider
150
- end
151
-
152
- delegate :current_transaction_open?, to: :transaction_provider
153
- end
154
- end
@@ -1,11 +0,0 @@
1
- module Dekiru
2
- class TransactionProvider
3
- def within_transaction(&)
4
- ActiveRecord::Base.transaction(&)
5
- end
6
-
7
- def current_transaction_open?
8
- ActiveRecord::Base.connection.current_transaction.open?
9
- end
10
- end
11
- end
@@ -1,8 +0,0 @@
1
- Description:
2
- Create database maintenance script using Dekiru::DataMigrationOperator
3
-
4
- Example:
5
- rails generate maintenance_script UpdateUserName
6
-
7
- This will create:
8
- scripts/XXX_update_user_name.rb
@@ -1,16 +0,0 @@
1
- require 'rails/generators'
2
-
3
- class MaintenanceScriptGenerator < Rails::Generators::NamedBase
4
- source_root File.expand_path('templates', __dir__)
5
-
6
- def copy_maintenance_script_file
7
- template 'maintenance_script.rb.erb',
8
- "#{Dekiru.configuration.maintenance_script_directory}/#{filename_date}_#{file_name}.rb"
9
- end
10
-
11
- private
12
-
13
- def filename_date
14
- Time.current.strftime('%Y%m%d')
15
- end
16
- end
@@ -1,3 +0,0 @@
1
- Dekiru::DataMigrationOperator.execute('<%= @name %>') do
2
- # write here
3
- end
@@ -1,226 +0,0 @@
1
- require "spec_helper"
2
-
3
- describe Dekiru::DataMigrationOperator do
4
- let(:dummy_stream) do
5
- class Dekiru::DummyStream
6
- attr_reader :out
7
-
8
- def initialize
9
- @out = ''
10
- end
11
-
12
- def puts(text)
13
- @out = out << "#{text}\n"
14
- end
15
-
16
- def print(text)
17
- @out = out << text
18
- end
19
-
20
- def tty?
21
- false
22
- end
23
-
24
- def flush
25
- end
26
- end
27
- Dekiru::DummyStream.new
28
- end
29
- let(:without_transaction) { false }
30
- let(:operator) do
31
- op = Dekiru::DataMigrationOperator.new('dummy', output: dummy_stream, logger: nil, without_transaction: without_transaction)
32
- allow(op).to receive(:current_transaction_open?) { false }
33
- op
34
- end
35
-
36
- describe '#execute' do
37
- it 'confirm で yes' do
38
- allow(STDIN).to receive(:gets) do
39
- "yes\n"
40
- end
41
-
42
- expect do
43
- operator.execute { log 'processing'; sleep 1.0 }
44
- end.not_to raise_error
45
-
46
- expect(operator.result).to eq(true)
47
- expect(operator.duration).to be_within(0.1).of(1.0)
48
- expect(operator.error).to eq(nil)
49
- expect(operator.stream.out).to include('Are you sure to commit?')
50
- expect(operator.stream.out).to include('Finished successfully:')
51
- expect(operator.stream.out).to include('Total time:')
52
- end
53
-
54
- it 'confirm で no' do
55
- allow(STDIN).to receive(:gets) do
56
- "no\n"
57
- end
58
-
59
- expect do
60
- operator.execute { log 'processing'; sleep 1.0 }
61
- end.to raise_error(ActiveRecord::Rollback)
62
-
63
- expect(operator.result).to eq(false)
64
- expect(operator.duration).to be_within(0.1).of(1.0)
65
- expect(operator.error.class).to eq(ActiveRecord::Rollback)
66
- expect(operator.stream.out).to include('Are you sure to commit?')
67
- expect(operator.stream.out).to include('Canceled:')
68
- expect(operator.stream.out).to include('Total time:')
69
- end
70
-
71
- it '処理中に例外' do
72
- expect do
73
- operator.execute { raise ArgumentError }
74
- end.to raise_error(ArgumentError)
75
-
76
- expect(operator.result).to eq(false)
77
- expect(operator.error.class).to eq(ArgumentError)
78
- expect(operator.stream.out).not_to include('Are you sure to commit?')
79
- expect(operator.stream.out).not_to include('Canceled:')
80
- expect(operator.stream.out).to include('Total time:')
81
- end
82
-
83
- context 'トランザクション内で呼び出された場合' do
84
- before { allow(operator).to receive(:current_transaction_open?) { true } }
85
-
86
- it '例外が発生すること' do
87
- expect { operator.execute { log 'processing'; sleep 1.0 } }.to raise_error(Dekiru::DataMigrationOperator::NestedTransactionError)
88
- end
89
- end
90
-
91
- context 'without_transaction: true のとき' do
92
- let(:without_transaction) { true }
93
-
94
- it 'トランザクションがかからないこと' do
95
- expect do
96
- operator.execute { log 'processing'; sleep 1.0 }
97
- end.not_to raise_error
98
-
99
- expect(operator.result).to eq(true)
100
- expect(operator.duration).to be_within(0.1).of(1.0)
101
- expect(operator.error).to eq(nil)
102
- expect(operator.stream.out).not_to include('Are you sure to commit?')
103
- expect(operator.stream.out).to include('Finished successfully:')
104
- expect(operator.stream.out).to include('Total time:')
105
- end
106
- end
107
- end
108
-
109
- describe '#each_with_progress' do
110
- it '進捗が表示される' do
111
- record = (0...10)
112
-
113
- allow(STDIN).to receive(:gets) do
114
- "yes\n"
115
- end
116
-
117
- sum = 0
118
- operator.execute do
119
- each_with_progress(record, title: 'count up number') do |num|
120
- sum += num
121
- end
122
- end
123
-
124
- expect(sum).to eq(45)
125
- expect(operator.result).to eq(true)
126
- expect(operator.error).to eq(nil)
127
- expect(operator.stream.out).to include('Are you sure to commit?')
128
- expect(operator.stream.out).to include('count up number:')
129
- expect(operator.stream.out).to include('Finished successfully:')
130
- expect(operator.stream.out).to include('Total time:')
131
- end
132
-
133
- it 'total をオプションで渡すことができる' do
134
- class Dekiru::DummyRecord
135
- def self.count
136
- raise "won't call"
137
- end
138
-
139
- def self.each
140
- yield 99
141
- end
142
- end
143
-
144
- allow(STDIN).to receive(:gets) do
145
- "yes\n"
146
- end
147
-
148
- sum = 0
149
- operator.execute do
150
- each_with_progress(Dekiru::DummyRecord, title: 'pass total as option', total: 1) do |num|
151
- sum += num
152
- end
153
- end
154
-
155
- expect(sum).to eq(99)
156
- expect(operator.result).to eq(true)
157
- expect(operator.error).to eq(nil)
158
- expect(operator.stream.out).to include('Are you sure to commit?')
159
- expect(operator.stream.out).to include('pass total as option:')
160
- expect(operator.stream.out).to include('Finished successfully:')
161
- expect(operator.stream.out).to include('Total time:')
162
- end
163
- end
164
-
165
- describe '#find_each_with_progress' do
166
- it '進捗が表示される' do
167
- record = (0...10).to_a.tap do |r|
168
- r.singleton_class.alias_method(:find_each, :each)
169
- end
170
-
171
- allow(STDIN).to receive(:gets) do
172
- "yes\n"
173
- end
174
-
175
- sum = 0
176
- operator.execute do
177
- find_each_with_progress(record, title: 'count up number') do |num|
178
- sum += num
179
- end
180
- end
181
-
182
- expect(sum).to eq(45)
183
- expect(operator.result).to eq(true)
184
- expect(operator.error).to eq(nil)
185
- expect(operator.stream.out).to include('Are you sure to commit?')
186
- expect(operator.stream.out).to include('count up number:')
187
- expect(operator.stream.out).to include('Finished successfully:')
188
- expect(operator.stream.out).to include('Total time:')
189
- end
190
-
191
- it 'total をオプションで渡すことができる' do
192
- class Dekiru::DummyRecord
193
- def self.count
194
- raise "won't call"
195
- end
196
-
197
- def self.find_each
198
- if block_given?
199
- yield 99
200
- else
201
- Enumerator.new { |y| y << 99 }
202
- end
203
- end
204
- end
205
-
206
- allow(STDIN).to receive(:gets) do
207
- "yes\n"
208
- end
209
-
210
- sum = 0
211
- operator.execute do
212
- find_each_with_progress(Dekiru::DummyRecord, title: 'pass total as option', total: 1) do |num|
213
- sum += num
214
- end
215
- end
216
-
217
- expect(sum).to eq(99)
218
- expect(operator.result).to eq(true)
219
- expect(operator.error).to eq(nil)
220
- expect(operator.stream.out).to include('Are you sure to commit?')
221
- expect(operator.stream.out).to include('pass total as option:')
222
- expect(operator.stream.out).to include('Finished successfully:')
223
- expect(operator.stream.out).to include('Total time:')
224
- end
225
- end
226
- end