exwiw 0.1.9 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +7 -0
- data/README.md +59 -0
- data/docs/plans/2026-05-22-after-insert-hook.md +189 -0
- data/lib/exwiw/after_insert_hook.rb +63 -0
- data/lib/exwiw/cli.rb +41 -0
- data/lib/exwiw/runner.rb +20 -2
- data/lib/exwiw/version.rb +1 -1
- data/lib/exwiw.rb +1 -0
- metadata +3 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 72265c12853dbc7a70b44d81aca646870f23a2964e8433e0799edd6d6228c494
|
|
4
|
+
data.tar.gz: ea442e301f04910083c5ac39d35fdeba81cc8078a7cabae44f03defa40cdaf5b
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 23d468a3a8ef2900856d7bc2730e3708d2443980bd086e69912ce4e77a3b9cd500cb67032405d2db119a39059b55b78545a34dbd9f16204d64676a73687d2045
|
|
7
|
+
data.tar.gz: dc77ea5d357863e55556615537e095216c2c5cf5bbd3e3bd98de6b359e33e8bbe7ab92b27c3745c52a9723f068303e29859bebfe46a4dacd4bd673f8ad9b82ca
|
data/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
+
## [0.2.0] - 2026-05-22
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- `--insert-only` CLI flag to skip generating `delete-*.sql` files. ([#23](https://github.com/heyinc/exwiw/pull/23))
|
|
10
|
+
- `--after-insert-hook` CLI flag to run a hook after per-table insert/delete files are generated. `.rb` hooks evaluate `insert_sql` DSL via ERB and write the result to `insert-{N+1}-after_insert.{ext}`; other executables run as a child process with `EXWIW_*` environment variables for pure side-effect hooks. ([#24](https://github.com/heyinc/exwiw/pull/24))
|
|
11
|
+
|
|
5
12
|
## [0.1.9] - 2026-05-21
|
|
6
13
|
|
|
7
14
|
Added
|
data/README.md
CHANGED
|
@@ -160,6 +160,65 @@ psql -d app_dev -f dump/insert-001-shops.sql
|
|
|
160
160
|
|
|
161
161
|
`--output-format=copy` is only supported with the `postgresql` adapter.
|
|
162
162
|
|
|
163
|
+
### Skip DELETE SQL output
|
|
164
|
+
|
|
165
|
+
By default, exwiw generates `delete-*.sql` files alongside the `insert-*.sql` files so that an existing dataset can be cleared before re-inserting. Pass `--insert-only` when you only need the insert files:
|
|
166
|
+
|
|
167
|
+
```bash
|
|
168
|
+
exwiw \
|
|
169
|
+
--adapter=mysql2 \
|
|
170
|
+
--host=localhost \
|
|
171
|
+
--port=3306 \
|
|
172
|
+
--user=reader \
|
|
173
|
+
--database=app_production \
|
|
174
|
+
--config-dir=exwiw \
|
|
175
|
+
--target-table=shops \
|
|
176
|
+
--ids=1 \
|
|
177
|
+
--output-dir=dump \
|
|
178
|
+
--insert-only
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
### After-insert hook
|
|
182
|
+
|
|
183
|
+
`--after-insert-hook=PATH` runs a post-processing hook **after** all per-table insert/delete files have been written. The hook can be either a Ruby file (`.rb`) or any executable script (e.g. `.sh`).
|
|
184
|
+
|
|
185
|
+
**Ruby hook (`.rb`)**: provides a tiny DSL with two builtins:
|
|
186
|
+
|
|
187
|
+
- `cli_options` — Hash of all parsed CLI options (e.g. `cli_options.fetch(:ids)` returns the `--ids` array).
|
|
188
|
+
- `insert_sql(template)` — appends an ERB-rendered string to a buffer. After the hook finishes, the buffer is concatenated and written to `insert-{N+1}-after_insert.{ext}` where `{N+1}` is one past the last per-table insert file. For the MongoDB adapter the equivalent alias `insert_jsonl(template)` is available; output goes to `insert-{N+1}-after_insert.jsonl`. Multiple `insert_sql` calls in a single hook are joined with `"\n"` into the same file. If no `insert_sql` call is made, no file is created.
|
|
189
|
+
|
|
190
|
+
Example `hooks/seed_default_users.rb`:
|
|
191
|
+
|
|
192
|
+
```ruby
|
|
193
|
+
insert_sql <<~SQL
|
|
194
|
+
-- seed default users for tenants <%= cli_options.fetch(:ids).join(',') %>
|
|
195
|
+
<%- cli_options.fetch(:ids).each do |tenant_id| -%>
|
|
196
|
+
INSERT INTO users (tenant_id, email) VALUES (<%= tenant_id %>, 'default@example.com');
|
|
197
|
+
<%- end -%>
|
|
198
|
+
SQL
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
Run with:
|
|
202
|
+
|
|
203
|
+
```bash
|
|
204
|
+
exwiw \
|
|
205
|
+
--adapter=mysql2 --host=localhost --port=3306 --user=reader \
|
|
206
|
+
--database=app_production --config-dir=exwiw \
|
|
207
|
+
--target-table=shops --ids=1,2 \
|
|
208
|
+
--output-dir=dump \
|
|
209
|
+
--after-insert-hook=hooks/seed_default_users.rb
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
**Shell hook**: anything other than `.rb` is exec'd as a child process. It is a pure side-effect hook — exwiw does not capture its stdout. The hook receives these env vars and inherits `DATABASE_PASSWORD` from the parent:
|
|
213
|
+
|
|
214
|
+
- `EXWIW_OUTPUT_DIR`, `EXWIW_CONFIG_DIR`
|
|
215
|
+
- `EXWIW_DATABASE_ADAPTER`, `EXWIW_DATABASE_HOST`, `EXWIW_DATABASE_PORT`, `EXWIW_DATABASE_USER`, `EXWIW_DATABASE_NAME`
|
|
216
|
+
- `EXWIW_TARGET_TABLE`, `EXWIW_IDS` (comma-separated), `EXWIW_OUTPUT_FORMAT`
|
|
217
|
+
|
|
218
|
+
A non-zero exit code from the shell hook aborts exwiw.
|
|
219
|
+
|
|
220
|
+
Note: Ruby hooks are evaluated via `instance_eval` inside the exwiw process — only pass paths you trust.
|
|
221
|
+
|
|
163
222
|
### Bulk insert chunk size
|
|
164
223
|
|
|
165
224
|
`bulk_insert_chunk_size` splits the generated `INSERT` statement into multiple statements, each containing at most the specified number of rows. This is useful when the number of records per table is large enough to hit limits like MySQL's `max_allowed_packet`.
|
|
@@ -0,0 +1,189 @@
|
|
|
1
|
+
# Plan: `--after-insert-hook` フック (Ruby DSL / shell script)
|
|
2
|
+
|
|
3
|
+
## Context
|
|
4
|
+
|
|
5
|
+
現状 `exwiw` は `insert-000-schema.{sql,js}` → `insert-NNN-{table}.{sql,jsonl}` → `delete-NNN-...` を生成して終わる。実運用では「import 後に特定テナントへデフォルトユーザを挿入する」「監査ログを 1 行打つ」など、抽出結果を踏まえた**後処理 SQL / 副作用**を続けて流したいケースがある。
|
|
6
|
+
|
|
7
|
+
これを毎回別ファイルとして手で書き足すのは面倒なので、抽出ジョブの一部としてフックを記述できるようにする。フックは `--ids` などの CLI オプションを参照できるべき (例: 「抽出対象のテナント ID 配列に対してデフォルトユーザを seed」)。
|
|
8
|
+
|
|
9
|
+
ゴール:
|
|
10
|
+
|
|
11
|
+
- `--after-insert=PATH` オプションを追加。`PATH` には `.rb` または `.sh` を指定可。
|
|
12
|
+
- 拡張子 `.rb`: 軽量 DSL (`cli_options`, `insert_sql` / `insert_jsonl`) を提供。文字列引数は ERB として評価され、結果が連結されて最後尾の insert ファイルとして書き出される。
|
|
13
|
+
- 拡張子 `.sh` (および `.rb` 以外): 環境変数で CLI オプションを渡したうえで子プロセスとして実行。出力ファイルは生成しない (純粋な副作用フック)。
|
|
14
|
+
- フックは per-table の insert/delete ループ完了後に 1 度だけ実行される。
|
|
15
|
+
|
|
16
|
+
## Design
|
|
17
|
+
|
|
18
|
+
### CLI レイヤー
|
|
19
|
+
**File**: `lib/exwiw/cli.rb`
|
|
20
|
+
|
|
21
|
+
- `@after_insert_path = nil` を初期化 (`initialize`)。
|
|
22
|
+
- `parser` 内に `opts.on("--after-insert=[PATH]", "Path to a .rb or .sh post-processing hook") { |v| @after_insert_path = File.expand_path(v) }` を追加。
|
|
23
|
+
- `validate_options!` で以下を検証:
|
|
24
|
+
- パスが存在しないとき: `$stderr.puts "--after-insert file not found: #{@after_insert_path}"; exit 1`
|
|
25
|
+
- 拡張子が `.rb` でも `.sh` でもなく、かつ実行可能ビットも立っていないとき: `--after-insert must be a .rb file or an executable script` で exit 1。
|
|
26
|
+
- `cli_options` 用に **CLI 全オプションを Hash 化するメソッド** を追加 (`build_cli_options_hash`):
|
|
27
|
+
```ruby
|
|
28
|
+
{
|
|
29
|
+
database_host: @database_host, database_port: @database_port,
|
|
30
|
+
database_user: @database_user, database_password: @database_password,
|
|
31
|
+
output_dir: @output_dir, config_dir: @config_dir,
|
|
32
|
+
database_adapter: @database_adapter, database_name: @database_name,
|
|
33
|
+
target_table: @target_table_name, ids: @ids.dup.freeze,
|
|
34
|
+
output_format: @output_format, insert_only: @insert_only,
|
|
35
|
+
log_level: @log_level, after_insert: @after_insert_path,
|
|
36
|
+
}.freeze
|
|
37
|
+
```
|
|
38
|
+
- `Runner.new(...)` 呼び出しに `after_insert_path: @after_insert_path, cli_options: build_cli_options_hash` を追加。
|
|
39
|
+
|
|
40
|
+
### Runner 統合
|
|
41
|
+
**File**: `lib/exwiw/runner.rb`
|
|
42
|
+
|
|
43
|
+
- `initialize` のキーワード引数に `after_insert_path: nil, cli_options: {}` を追加し instance var に格納。
|
|
44
|
+
- `run` の per-table ループ (`ordered_table_names.each_with_index`) が終わった**直後** (現状の line 98 の直後) で:
|
|
45
|
+
```ruby
|
|
46
|
+
if @after_insert_path
|
|
47
|
+
@logger.info("Running after-insert hook: #{@after_insert_path}")
|
|
48
|
+
AfterInsertHook.run(
|
|
49
|
+
path: @after_insert_path,
|
|
50
|
+
cli_options: @cli_options,
|
|
51
|
+
output_dir: @output_dir,
|
|
52
|
+
next_idx: total_size + 1,
|
|
53
|
+
output_extension: adapter.output_extension,
|
|
54
|
+
logger: @logger,
|
|
55
|
+
)
|
|
56
|
+
end
|
|
57
|
+
```
|
|
58
|
+
- `total_size` は既存変数 (`ordered_table_names.size`)。schema が `000`、per-table が `001..total_size` を使うので、フック出力は `total_size + 1` 番。`delete-*` は逆順番号なので衝突しない。
|
|
59
|
+
|
|
60
|
+
### 新規ファイル: `lib/exwiw/after_insert_hook.rb`
|
|
61
|
+
|
|
62
|
+
```ruby
|
|
63
|
+
require 'erb'
|
|
64
|
+
require 'shellwords'
|
|
65
|
+
|
|
66
|
+
module Exwiw
|
|
67
|
+
class AfterInsertHook
|
|
68
|
+
def self.run(path:, cli_options:, output_dir:, next_idx:, output_extension:, logger:)
|
|
69
|
+
ext = File.extname(path)
|
|
70
|
+
idx_str = next_idx.to_s.rjust(3, '0')
|
|
71
|
+
output_path = File.join(output_dir, "insert-#{idx_str}-after_insert.#{output_extension}")
|
|
72
|
+
|
|
73
|
+
case ext
|
|
74
|
+
when '.rb'
|
|
75
|
+
run_ruby(path: path, cli_options: cli_options, output_path: output_path, logger: logger)
|
|
76
|
+
else
|
|
77
|
+
run_shell(path: path, cli_options: cli_options, output_dir: output_dir, logger: logger)
|
|
78
|
+
end
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def self.run_ruby(path:, cli_options:, output_path:, logger:)
|
|
82
|
+
ctx = Context.new(cli_options)
|
|
83
|
+
ctx.instance_eval(File.read(path), path)
|
|
84
|
+
sql = ctx.collected.join("\n")
|
|
85
|
+
if sql.empty?
|
|
86
|
+
logger.info("After-insert hook produced no output; skipping file write.")
|
|
87
|
+
return
|
|
88
|
+
end
|
|
89
|
+
File.write(output_path, sql)
|
|
90
|
+
logger.info("Wrote after-insert hook output to #{output_path}")
|
|
91
|
+
end
|
|
92
|
+
|
|
93
|
+
def self.run_shell(path:, cli_options:, output_dir:, logger:)
|
|
94
|
+
env = {
|
|
95
|
+
'EXWIW_OUTPUT_DIR' => output_dir,
|
|
96
|
+
'EXWIW_CONFIG_DIR' => cli_options[:config_dir].to_s,
|
|
97
|
+
'EXWIW_DATABASE_ADAPTER' => cli_options[:database_adapter].to_s,
|
|
98
|
+
'EXWIW_DATABASE_HOST' => cli_options[:database_host].to_s,
|
|
99
|
+
'EXWIW_DATABASE_PORT' => cli_options[:database_port].to_s,
|
|
100
|
+
'EXWIW_DATABASE_USER' => cli_options[:database_user].to_s,
|
|
101
|
+
'EXWIW_DATABASE_NAME' => cli_options[:database_name].to_s,
|
|
102
|
+
'EXWIW_TARGET_TABLE' => cli_options[:target_table].to_s,
|
|
103
|
+
'EXWIW_IDS' => Array(cli_options[:ids]).join(','),
|
|
104
|
+
'EXWIW_OUTPUT_FORMAT' => cli_options[:output_format].to_s,
|
|
105
|
+
}
|
|
106
|
+
# DATABASE_PASSWORD は既存 ENV をそのまま受け継がせる (env hash で上書きしない)。
|
|
107
|
+
ok = system(env, path)
|
|
108
|
+
raise "after-insert shell hook failed: #{path}" unless ok
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
class Context
|
|
112
|
+
attr_reader :cli_options, :collected
|
|
113
|
+
|
|
114
|
+
def initialize(cli_options)
|
|
115
|
+
@cli_options = cli_options
|
|
116
|
+
@collected = []
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
# ERB 評価。MongoDB 向けに `insert_jsonl` の別名も提供する (出力ファイル名・拡張子は
|
|
120
|
+
# Runner 側で adapter.output_extension を元に決まるので、どちらを呼んでも同じバッファに溜まる)。
|
|
121
|
+
def insert_sql(template)
|
|
122
|
+
@collected << ERB.new(template, trim_mode: '-').result(binding)
|
|
123
|
+
end
|
|
124
|
+
alias_method :insert_jsonl, :insert_sql
|
|
125
|
+
end
|
|
126
|
+
end
|
|
127
|
+
end
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
ポイント:
|
|
131
|
+
- `instance_eval(File.read(path), path)` で、フックファイル内では `cli_options` と `insert_sql` が単純なメソッド呼び出しとして使える (DSL 風)。
|
|
132
|
+
- ERB 評価は `Context#insert_sql` の `binding` を使うため、ERB テンプレート内でも `cli_options.fetch(:ids)` が呼べる。
|
|
133
|
+
- `insert_sql` / `insert_jsonl` を複数回呼ぶと `@collected` に積まれ、最後に `"\n"` で連結して 1 ファイルに書く。
|
|
134
|
+
- 空出力なら書き出さない (idempotency に近い挙動)。
|
|
135
|
+
- shell 実行は `system(env, path)`。`path` は単独引数として渡すのでシェル展開されない (Shellwords 必要なし)。失敗時は exception で停止 → exit。
|
|
136
|
+
- ENV 名は `EXWIW_*` 接頭辞で名前空間を切る。`DATABASE_PASSWORD` は親プロセスの ENV を継承させる (フック側で読みたければ読める)。
|
|
137
|
+
|
|
138
|
+
### lib/exwiw.rb への require 追加
|
|
139
|
+
`require_relative "exwiw/after_insert_hook"` を `runner.rb` の require の前後あたりに追加。
|
|
140
|
+
|
|
141
|
+
### 使用例 (README に追記)
|
|
142
|
+
|
|
143
|
+
`hooks/seed_default_users.rb`:
|
|
144
|
+
```ruby
|
|
145
|
+
# cli_options[:ids] には --ids で渡された配列が入る
|
|
146
|
+
insert_sql <<~SQL
|
|
147
|
+
-- seed default users for tenants <%= cli_options.fetch(:ids).join(',') %>
|
|
148
|
+
<%- cli_options.fetch(:ids).each do |tenant_id| -%>
|
|
149
|
+
INSERT INTO users (tenant_id, email) VALUES (<%= tenant_id %>, 'default@example.com');
|
|
150
|
+
<%- end -%>
|
|
151
|
+
SQL
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
実行:
|
|
155
|
+
```
|
|
156
|
+
exwiw --adapter=mysql2 ... --target-table=shops --ids=1,2 \
|
|
157
|
+
--after-insert=hooks/seed_default_users.rb
|
|
158
|
+
```
|
|
159
|
+
結果: `dump/insert-{total+1}-after_insert.sql` が、テナント 1,2 用の INSERT を含めて出力される。
|
|
160
|
+
|
|
161
|
+
## Files to modify / add
|
|
162
|
+
|
|
163
|
+
| パス | 変更 |
|
|
164
|
+
|---|---|
|
|
165
|
+
| `lib/exwiw/cli.rb` | `--after-insert` parse / validate / `build_cli_options_hash` 追加、Runner 呼び出しへ伝搬 |
|
|
166
|
+
| `lib/exwiw/runner.rb` | `initialize` に `after_insert_path:`, `cli_options:` 追加、per-table ループ完了後にフック呼び出し |
|
|
167
|
+
| `lib/exwiw/after_insert_hook.rb` (新規) | `AfterInsertHook.run` + `Context` (DSL + ERB) |
|
|
168
|
+
| `lib/exwiw.rb` | `require_relative "exwiw/after_insert_hook"` 追加 |
|
|
169
|
+
| `README.md` | `--after-insert` の節を追加 (Ruby DSL 例 / shell hook 例 / 環境変数一覧) |
|
|
170
|
+
| `spec/runner_spec.rb` | Ruby フックで `insert-{N+1}-after_insert.sql` が書き出され、ERB で `cli_options.fetch(:ids)` が展開されることを assert |
|
|
171
|
+
| `spec/after_insert_hook_spec.rb` (新規) | `Context#insert_sql` の ERB 評価、複数回呼び出しが `"\n"` 連結されることを assert |
|
|
172
|
+
|
|
173
|
+
## Verification
|
|
174
|
+
|
|
175
|
+
1. **ユニットテスト**: `bundle exec rspec spec/after_insert_hook_spec.rb` — `Context#insert_sql` を直接叩いて、ERB が `cli_options.fetch(:ids)` を解決できること、複数回呼び出しが `\n` 連結されることを確認。
|
|
176
|
+
2. **統合テスト**: `bundle exec rspec spec/runner_spec.rb` — sqlite3 経由で実際に Runner を流し、tmp に書いたフック `.rb` を `--after-insert` 相当で渡し、`tmp/.../insert-{N+1}-after_insert.sql` が生成されることと、ファイル内に ERB 展開後の `--ids` の値が含まれることを assert。
|
|
177
|
+
3. **CLI E2E**: scenario の `test_with_sqlite3.sh` を一時的に編集して `--after-insert=` を付けた呼び出しを試し、出力ディレクトリに想定どおりのファイルが置かれることを目視確認。MongoDB は `--after-insert=hook.rb` で `insert_jsonl` を使ったときに `.jsonl` が出ることのみ smoke-test。
|
|
178
|
+
4. **エッジケース確認**:
|
|
179
|
+
- `--after-insert=missing.rb` でわかりやすいエラー終了。
|
|
180
|
+
- フック内で `insert_sql` を 1 度も呼ばなかったとき、ファイルは作られず info ログのみ。
|
|
181
|
+
- shell hook の non-zero exit code で Runner が落ちる。
|
|
182
|
+
- `--ids` 省略時に `cli_options.fetch(:ids)` が空配列を返す (`@ids = []` 初期値が保たれる)。
|
|
183
|
+
|
|
184
|
+
## 留意点 / 既知のリスク
|
|
185
|
+
|
|
186
|
+
- **任意コード実行**: `.rb` フックは `instance_eval` で実行される (= exwiw プロセスと同じ権限で動く)。ユーザ自身が用意した hook を渡す前提なので問題ないが、README に「信頼するソースのみ」と注意書きを入れる。
|
|
187
|
+
- **ファイル番号の衝突**: `delete-*` は逆順番号 (`total_size - idx`) を使うので、`insert-{total+1}-after_insert` と直接衝突はしない (`delete-001`...`delete-{total}` の範囲)。
|
|
188
|
+
- **MongoDB 対応の限界**: `insert_jsonl` は ERB 出力結果を `.jsonl` としてそのまま書き出す。1 行 1 ドキュメントの形に揃えるのはユーザ責任。`mongoimport` で流せる前提。
|
|
189
|
+
- **password の取り扱い**: shell hook の env に `DATABASE_PASSWORD` を明示的に詰めない (親プロセス ENV を継承させる)。プロセス一覧経由の漏えいを防ぐため、ENV を介すのは hash で `EXWIW_*` のみ。
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'erb'
|
|
4
|
+
|
|
5
|
+
module Exwiw
|
|
6
|
+
class AfterInsertHook
|
|
7
|
+
def self.run(path:, cli_options:, output_dir:, next_idx:, output_extension:, logger:)
|
|
8
|
+
ext = File.extname(path)
|
|
9
|
+
idx_str = next_idx.to_s.rjust(3, '0')
|
|
10
|
+
output_path = File.join(output_dir, "insert-#{idx_str}-after_insert.#{output_extension}")
|
|
11
|
+
|
|
12
|
+
if ext == '.rb'
|
|
13
|
+
run_ruby(path: path, cli_options: cli_options, output_path: output_path, logger: logger)
|
|
14
|
+
else
|
|
15
|
+
run_shell(path: path, cli_options: cli_options, output_dir: output_dir, logger: logger)
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def self.run_ruby(path:, cli_options:, output_path:, logger:)
|
|
20
|
+
ctx = Context.new(cli_options)
|
|
21
|
+
ctx.instance_eval(File.read(path), path)
|
|
22
|
+
content = ctx.collected.join("\n")
|
|
23
|
+
if content.empty?
|
|
24
|
+
logger.info("After-insert hook produced no output; skipping file write.")
|
|
25
|
+
return
|
|
26
|
+
end
|
|
27
|
+
File.write(output_path, content)
|
|
28
|
+
logger.info("Wrote after-insert hook output to #{output_path}")
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def self.run_shell(path:, cli_options:, output_dir:, logger:)
|
|
32
|
+
env = {
|
|
33
|
+
'EXWIW_OUTPUT_DIR' => output_dir,
|
|
34
|
+
'EXWIW_CONFIG_DIR' => cli_options[:config_dir].to_s,
|
|
35
|
+
'EXWIW_DATABASE_ADAPTER' => cli_options[:database_adapter].to_s,
|
|
36
|
+
'EXWIW_DATABASE_HOST' => cli_options[:database_host].to_s,
|
|
37
|
+
'EXWIW_DATABASE_PORT' => cli_options[:database_port].to_s,
|
|
38
|
+
'EXWIW_DATABASE_USER' => cli_options[:database_user].to_s,
|
|
39
|
+
'EXWIW_DATABASE_NAME' => cli_options[:database_name].to_s,
|
|
40
|
+
'EXWIW_TARGET_TABLE' => cli_options[:target_table].to_s,
|
|
41
|
+
'EXWIW_IDS' => Array(cli_options[:ids]).join(','),
|
|
42
|
+
'EXWIW_OUTPUT_FORMAT' => cli_options[:output_format].to_s,
|
|
43
|
+
}
|
|
44
|
+
logger.info("Running after-insert shell hook: #{path}")
|
|
45
|
+
ok = system(env, path)
|
|
46
|
+
raise "after-insert shell hook failed: #{path}" unless ok
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
class Context
|
|
50
|
+
attr_reader :cli_options, :collected
|
|
51
|
+
|
|
52
|
+
def initialize(cli_options)
|
|
53
|
+
@cli_options = cli_options
|
|
54
|
+
@collected = []
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def insert_sql(template)
|
|
58
|
+
@collected << ERB.new(template, trim_mode: '-').result(binding)
|
|
59
|
+
end
|
|
60
|
+
alias_method :insert_jsonl, :insert_sql
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
end
|
data/lib/exwiw/cli.rb
CHANGED
|
@@ -29,6 +29,8 @@ module Exwiw
|
|
|
29
29
|
@target_table_name = nil
|
|
30
30
|
@ids = []
|
|
31
31
|
@output_format = 'insert'
|
|
32
|
+
@insert_only = false
|
|
33
|
+
@after_insert_hook_path = nil
|
|
32
34
|
@log_level = :info
|
|
33
35
|
|
|
34
36
|
parser.parse!(@argv)
|
|
@@ -62,6 +64,9 @@ module Exwiw
|
|
|
62
64
|
config_dir: @config_dir,
|
|
63
65
|
dump_target: dump_target,
|
|
64
66
|
output_format: @output_format,
|
|
67
|
+
insert_only: @insert_only,
|
|
68
|
+
after_insert_hook_path: @after_insert_hook_path,
|
|
69
|
+
cli_options: build_cli_options_hash,
|
|
65
70
|
logger: logger,
|
|
66
71
|
).run
|
|
67
72
|
end
|
|
@@ -129,6 +134,38 @@ module Exwiw
|
|
|
129
134
|
$stderr.puts "--target-table is required when --ids is specified"
|
|
130
135
|
exit 1
|
|
131
136
|
end
|
|
137
|
+
|
|
138
|
+
if @after_insert_hook_path
|
|
139
|
+
unless File.file?(@after_insert_hook_path)
|
|
140
|
+
$stderr.puts "--after-insert-hook file not found: #{@after_insert_hook_path}"
|
|
141
|
+
exit 1
|
|
142
|
+
end
|
|
143
|
+
|
|
144
|
+
ext = File.extname(@after_insert_hook_path)
|
|
145
|
+
if ext != '.rb' && !File.executable?(@after_insert_hook_path)
|
|
146
|
+
$stderr.puts "--after-insert-hook must be a .rb file or an executable script: #{@after_insert_hook_path}"
|
|
147
|
+
exit 1
|
|
148
|
+
end
|
|
149
|
+
end
|
|
150
|
+
end
|
|
151
|
+
|
|
152
|
+
private def build_cli_options_hash
|
|
153
|
+
{
|
|
154
|
+
database_host: @database_host,
|
|
155
|
+
database_port: @database_port,
|
|
156
|
+
database_user: @database_user,
|
|
157
|
+
database_password: @database_password,
|
|
158
|
+
output_dir: @output_dir,
|
|
159
|
+
config_dir: @config_dir,
|
|
160
|
+
database_adapter: @database_adapter,
|
|
161
|
+
database_name: @database_name,
|
|
162
|
+
target_table: @target_table_name,
|
|
163
|
+
ids: @ids.dup.freeze,
|
|
164
|
+
output_format: @output_format,
|
|
165
|
+
insert_only: @insert_only,
|
|
166
|
+
log_level: @log_level,
|
|
167
|
+
after_insert_hook: @after_insert_hook_path,
|
|
168
|
+
}.freeze
|
|
132
169
|
end
|
|
133
170
|
|
|
134
171
|
private def build_logger
|
|
@@ -167,6 +204,10 @@ module Exwiw
|
|
|
167
204
|
opts.on("--target-table=[TABLE]", "Target table for extraction. If omitted, dump all tables.") { |v| @target_table_name = v }
|
|
168
205
|
opts.on("--ids=[IDS]", "Comma-separated list of identifiers. Required when --target-table is given.") { |v| @ids = v.split(',') }
|
|
169
206
|
opts.on("--output-format=[FORMAT]", "Output format: insert (default) or copy (PostgreSQL only)") { |v| @output_format = v }
|
|
207
|
+
opts.on("--insert-only", "Do not generate DELETE SQL files") { @insert_only = true }
|
|
208
|
+
opts.on("--after-insert-hook=PATH", "Path to a .rb or .sh post-processing hook executed after all insert/delete files are written") do |v|
|
|
209
|
+
@after_insert_hook_path = File.expand_path(v)
|
|
210
|
+
end
|
|
170
211
|
opts.on("--log-level=LEVEL", "Log level (debug, info). default is info") { |v| @log_level = v.to_sym }
|
|
171
212
|
|
|
172
213
|
opts.on("--help", "Print this help") do
|
data/lib/exwiw/runner.rb
CHANGED
|
@@ -10,13 +10,19 @@ module Exwiw
|
|
|
10
10
|
config_dir:,
|
|
11
11
|
dump_target:,
|
|
12
12
|
logger:,
|
|
13
|
-
output_format: 'insert'
|
|
13
|
+
output_format: 'insert',
|
|
14
|
+
insert_only: false,
|
|
15
|
+
after_insert_hook_path: nil,
|
|
16
|
+
cli_options: {}
|
|
14
17
|
)
|
|
15
18
|
@connection_config = connection_config
|
|
16
19
|
@output_dir = output_dir
|
|
17
20
|
@config_dir = config_dir
|
|
18
21
|
@dump_target = dump_target
|
|
19
22
|
@output_format = output_format
|
|
23
|
+
@insert_only = insert_only
|
|
24
|
+
@after_insert_hook_path = after_insert_hook_path
|
|
25
|
+
@cli_options = cli_options
|
|
20
26
|
@logger = logger
|
|
21
27
|
end
|
|
22
28
|
|
|
@@ -80,7 +86,7 @@ module Exwiw
|
|
|
80
86
|
end
|
|
81
87
|
end
|
|
82
88
|
|
|
83
|
-
if adapter.supports_bulk_delete?
|
|
89
|
+
if adapter.supports_bulk_delete? && !@insert_only
|
|
84
90
|
@logger.debug(" Generate DELETE statement...")
|
|
85
91
|
delete_sql = adapter.to_bulk_delete(query_ast, table)
|
|
86
92
|
if @logger.debug?
|
|
@@ -94,6 +100,18 @@ module Exwiw
|
|
|
94
100
|
end
|
|
95
101
|
end
|
|
96
102
|
end
|
|
103
|
+
|
|
104
|
+
if @after_insert_hook_path
|
|
105
|
+
@logger.info("Running after-insert hook: #{@after_insert_hook_path}")
|
|
106
|
+
AfterInsertHook.run(
|
|
107
|
+
path: @after_insert_hook_path,
|
|
108
|
+
cli_options: @cli_options,
|
|
109
|
+
output_dir: @output_dir,
|
|
110
|
+
next_idx: total_size + 1,
|
|
111
|
+
output_extension: adapter.output_extension,
|
|
112
|
+
logger: @logger,
|
|
113
|
+
)
|
|
114
|
+
end
|
|
97
115
|
end
|
|
98
116
|
|
|
99
117
|
private def load_table_config(klass)
|
data/lib/exwiw/version.rb
CHANGED
data/lib/exwiw.rb
CHANGED
|
@@ -21,6 +21,7 @@ require_relative "exwiw/determine_table_processing_order"
|
|
|
21
21
|
require_relative "exwiw/mongo_query"
|
|
22
22
|
require_relative "exwiw/query_ast"
|
|
23
23
|
require_relative "exwiw/query_ast_builder"
|
|
24
|
+
require_relative "exwiw/after_insert_hook"
|
|
24
25
|
require_relative "exwiw/runner"
|
|
25
26
|
require_relative "exwiw/schema_generator"
|
|
26
27
|
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: exwiw
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Shia
|
|
@@ -37,6 +37,7 @@ files:
|
|
|
37
37
|
- README.md
|
|
38
38
|
- docs/plans/2026-05-15-insert-000-schema-file.md
|
|
39
39
|
- docs/plans/2026-05-16-mongodb-from-clean-scenario.md
|
|
40
|
+
- docs/plans/2026-05-22-after-insert-hook.md
|
|
40
41
|
- exe/exwiw
|
|
41
42
|
- lib/exwiw.rb
|
|
42
43
|
- lib/exwiw/adapter.rb
|
|
@@ -44,6 +45,7 @@ files:
|
|
|
44
45
|
- lib/exwiw/adapter/mysql2_adapter.rb
|
|
45
46
|
- lib/exwiw/adapter/postgresql_adapter.rb
|
|
46
47
|
- lib/exwiw/adapter/sqlite3_adapter.rb
|
|
48
|
+
- lib/exwiw/after_insert_hook.rb
|
|
47
49
|
- lib/exwiw/belongs_to.rb
|
|
48
50
|
- lib/exwiw/cli.rb
|
|
49
51
|
- lib/exwiw/ddl_postprocessor.rb
|