exwiw 0.2.9 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +13 -1
- data/README.md +6 -4
- data/docs/plans/2026-05-31-ids-column-for-sql-adapters.md +93 -0
- data/lib/exwiw/adapter/mysql2_adapter.rb +17 -6
- data/lib/exwiw/adapter/postgresql_adapter.rb +17 -6
- data/lib/exwiw/adapter/sqlite3_adapter.rb +17 -6
- data/lib/exwiw/belongs_to.rb +4 -3
- data/lib/exwiw/cli.rb +50 -27
- data/lib/exwiw/mongoid_schema_generator.rb +6 -4
- data/lib/exwiw/query_ast.rb +25 -6
- data/lib/exwiw/query_ast_builder.rb +51 -28
- data/lib/exwiw/schema_generator.rb +24 -21
- data/lib/exwiw/table_config.rb +6 -5
- data/lib/exwiw/version.rb +1 -1
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 55e0eecbd5d7117c263f00fb43c36e2bcc31c75eb4b7ef0255402bec2ac108dc
|
|
4
|
+
data.tar.gz: '0913f0804ad33023661b947f88cc86adfeb98ef6b047a93f0ba1f09cea52cec9'
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: bb152c10da5489d005660f458ee8cc526a199a15618e41a625694df6d8cca5df623916c14996f03df1ee969b1c51e43b12fc1b7a8676f2754ae01c7211deb251
|
|
7
|
+
data.tar.gz: 55ae136ec956f3a3e15d522e3d71388765ab206911e5662e653522d35e3126fb1d67711051bfd19d8cc2a7457194f84065ecc49dde38d7faaca67fddfb70da1f
|
data/CHANGELOG.md
CHANGED
|
@@ -2,11 +2,23 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
+
## [0.3.1] - 2026-05-31
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
|
|
9
|
+
- **Breaking:** the `dump` subcommand is renamed to `export` to match the gem name (Export What I Want). Invoke `exwiw export ...` (or omit the subcommand, which now defaults to `export`) instead of `exwiw dump ...`. There is no `dump` alias.
|
|
10
|
+
|
|
11
|
+
## [0.3.0] - 2026-05-31
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
- New `--ids-column=COLUMN` CLI option matches `--ids` against an arbitrary column on the target table instead of its primary key (e.g. `--target-table=users --ids=alice@example.com --ids-column=email`). This is the SQL-adapter (mysql2/postgresql/sqlite3) counterpart of the mongodb `--ids-field`; the two are mutually exclusive and each is rejected by the other adapter family (`--ids-field` is mongodb-only, `--ids-column` is sql-only), mirroring the existing `--target-table` / `--target-collection` split. Related tables are still extracted correctly: rather than propagating `--ids` directly onto foreign keys (which would be wrong when filtering on a non-primary-key column), each foreign key is resolved through the target via a subquery (`WHERE fk IN (SELECT pk FROM target WHERE COLUMN IN (...))`), so only the target table's filter column changes and direct / indirect / polymorphic relations all extract correctly. Note: if `COLUMN` is itself masked, re-running `delete-*` against an already-imported (masked) dump won't match, so prefer a stable natural key. ([#47](https://github.com/heyinc/exwiw/pull/47))
|
|
16
|
+
|
|
5
17
|
## [0.2.9] - 2026-05-31
|
|
6
18
|
|
|
7
19
|
### Added
|
|
8
20
|
|
|
9
|
-
- New `--ids-field=FIELD` CLI option matches `--ids` against an arbitrary field on the target collection instead of its primary key (e.g. `--target-collection=users --ids=a@example.com --ids-field=email`). Only the target collection's filter changes — downstream foreign-key propagation still keys off the primary key. Unlike the primary-key path, the supplied ids are **not** type-coerced (a custom field's stored type is unknown, so values are passed through as-is).
|
|
21
|
+
- New `--ids-field=FIELD` CLI option matches `--ids` against an arbitrary field on the target collection instead of its primary key (e.g. `--target-collection=users --ids=a@example.com --ids-field=email`). Only the target collection's filter changes — downstream foreign-key propagation still keys off the primary key. Unlike the primary-key path, the supplied ids are **not** type-coerced (a custom field's stored type is unknown, so values are passed through as-is). This flag is **mongodb-only**.
|
|
10
22
|
- New `--target-collection=COLLECTION` CLI option, a mongodb-only alias of `--target-table`. Specifying both, or using `--target-collection` with a non-mongodb adapter, is rejected at validation time.
|
|
11
23
|
- New rake task `exwiw:schema:generate_mongoid` (backed by `Exwiw::MongoidSchemaGenerator`) generates `MongodbCollectionConfig` files by introspecting Mongoid document models — a separate task/class from the ActiveRecord `schema:generate` because the ORMs expose different metadata. It derives the collection name, the `_id` primary key, `fields` (including referenced `belongs_to` foreign keys), `belongs_tos` from referenced `belongs_to` associations, and `embedded_in` from `embedded_in` / `embeds_many` / `embeds_one` associations (each embedded config names its immediate parent collection and `store_as` document key; nested embedding is emitted as a chain — `comments` embedded_in `posts`, `posts` embedded_in `users` — so the adapter can recurse through both array and Hash subdocuments). Regeneration preserves hand-edited `replace_with` / `filter` / `skip` / `bulk_insert_chunk_size`. Polymorphic `belongs_to` is not yet expanded. Models in an inheritance hierarchy whose subclasses share the base's collection (Mongoid STI, `_type` discriminator) collapse into a single config: subclasses are discovered via `descendants` (Mongoid registers only the base in `Mongoid.models`) and every class's `fields` / `belongs_tos` are unioned, so subclass-only fields and associations are preserved. A referenced `belongs_to` declared on an *embedded* document (e.g. `Comment embedded_in :post, belongs_to :author`) is dropped from the embedded config's `belongs_tos` (cross-collection refs from inside embedded subdocuments are unsupported and rejected on load), while its foreign-key column is still kept as an ordinary field. A `has_and_belongs_to_many` association is likewise dropped from `belongs_tos` (its foreign keys are stored as an array field such as `tag_ids`, which exwiw cannot follow as a single-valued foreign key), while that `*_ids` array column is kept as an ordinary field. A *polymorphic* `embedded_in` (`embedded_in :addressable, polymorphic: true`) has no single embedding parent collection and cannot be expressed as an `embedded_in` config, so the generator raises a clear, actionable error rather than crashing on the unresolvable parent class. A *self-referential / cyclic* embedding (Mongoid's `recursively_embeds_many` / `recursively_embeds_one`) makes a collection both top-level and embedded inside documents of its own type; since exwiw represents a collection as either top-level or embedded (not both), the generator likewise raises a clear error rather than emit an `embedded_in` config that would silently make the collection undumpable. The `created_at` / `updated_at` columns added by `include Mongoid::Timestamps` are tracked as ordinary fields, and their BSON `ObjectId` / `Date` values (the shape a live `find` returns) serialize as MongoDB Extended JSON (`$oid` / `$date`) through the dump path — now covered end-to-end against the generated configs. An aliased field (`field :ctry, as: :country`) is emitted by its **stored** document key (`ctry`), never the Ruby accessor (`country`), so masking and projection target the key that actually appears in the document; the accessor is additionally surfaced as `mongoid_field_name` on that field so the otherwise cryptic short key stays understandable (association aliases such as `shop => shop_id` and the built-in `id => _id` are not field renames and are not annotated).
|
|
12
24
|
|
data/README.md
CHANGED
|
@@ -46,10 +46,10 @@ gem install exwiw
|
|
|
46
46
|
|
|
47
47
|
exwiw has two subcommands:
|
|
48
48
|
|
|
49
|
-
- `
|
|
50
|
-
- `explain` — print the compiled SQL and its `EXPLAIN` output for each query that `
|
|
49
|
+
- `export` (default) — generate INSERT/COPY SQL files. If the subcommand is omitted, `export` is assumed.
|
|
50
|
+
- `explain` — print the compiled SQL and its `EXPLAIN` output for each query that `export` would run, without executing the SELECTs.
|
|
51
51
|
|
|
52
|
-
### `exwiw
|
|
52
|
+
### `exwiw export`
|
|
53
53
|
|
|
54
54
|
```bash
|
|
55
55
|
# dump & masking all records from database to dump.sql based on schema.json
|
|
@@ -67,6 +67,8 @@ exwiw \
|
|
|
67
67
|
--log-level=info
|
|
68
68
|
```
|
|
69
69
|
|
|
70
|
+
By default `--ids` are matched against the target table's primary key. `--ids-column=COLUMN` matches them against a different column instead (e.g. `--target-table=users --ids=alice@example.com --ids-column=email`). Related tables are still extracted correctly: their foreign keys are resolved through the target via a subquery (`WHERE fk IN (SELECT pk FROM target WHERE COLUMN IN (...))`), so only the target table's filter column changes. This is the SQL-adapter counterpart of the mongodb `--ids-field`; the two are mutually exclusive and each is rejected by the other adapter family. Note: if `COLUMN` is itself masked, re-running `delete-*` against an already-imported (masked) dump won't match, so prefer a stable natural key.
|
|
71
|
+
|
|
70
72
|
When `--target-table` and `--ids` are omitted, exwiw dumps all tables defined in `--config-dir`:
|
|
71
73
|
|
|
72
74
|
```bash
|
|
@@ -412,7 +414,7 @@ The MongoDB adapter is experimental. To use it:
|
|
|
412
414
|
- The MongoDB adapter consumes a separate config type, `MongodbCollectionConfig`, with MongoDB-native naming. Use `fields` (instead of the SQL adapters' `columns`), and set `"primary_key": "_id"`. Foreign keys (`shop_id`, `user_id`, ...) stay as ordinary fields.
|
|
413
415
|
- `--ids` values are coerced to the type actually stored in `_id` before filtering: integer-looking ids become `Integer`, 24-char hex ids become `BSON::ObjectId` (Mongoid's default `_id` type — a plain String would never match an ObjectId), and any other string is left as-is.
|
|
414
416
|
- `--target-collection=COLLECTION` is a mongodb-only alias of `--target-table` (use whichever reads better for MongoDB). Specifying both, or using `--target-collection` with a non-mongodb adapter, is an error.
|
|
415
|
-
- `--ids-field=FIELD` matches `--ids` against `FIELD` on the target collection instead of its primary key (e.g. `--target-collection=users --ids=a@example.com --ids-field=email`). Downstream foreign-key propagation still keys off the primary key, so only the target collection's filter changes. Unlike the primary-key path, the supplied ids are **not** type-coerced (the stored type of a custom field is unknown), so pass values matching the field's actual type. This flag is
|
|
417
|
+
- `--ids-field=FIELD` matches `--ids` against `FIELD` on the target collection instead of its primary key (e.g. `--target-collection=users --ids=a@example.com --ids-field=email`). Downstream foreign-key propagation still keys off the primary key, so only the target collection's filter changes. Unlike the primary-key path, the supplied ids are **not** type-coerced (the stored type of a custom field is unknown), so pass values matching the field's actual type. This flag is **mongodb-only**; the SQL adapters use `--ids-column` instead (see below).
|
|
416
418
|
- Output is JSON Lines (`insert-{idx}-{collection}.jsonl`) using MongoDB Extended JSON (relaxed mode). Import with `mongoimport`:
|
|
417
419
|
```bash
|
|
418
420
|
mongoimport --db app_dev --collection users --file dump/insert-002-users.jsonl
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
# `--ids-column` を SQL アダプタに実装
|
|
2
|
+
|
|
3
|
+
## Context
|
|
4
|
+
|
|
5
|
+
MongoDB アダプタには `--ids-field` フラグがあり、`--ids` を対象テーブルの主キー以外の
|
|
6
|
+
フィールドにマッチさせられる(`lib/exwiw/adapter/mongodb_adapter.rb:48-65`)。一方で
|
|
7
|
+
SQL アダプタ(mysql2 / postgresql / sqlite3)では未実装で、CLI が明示的に拒否している
|
|
8
|
+
(`lib/exwiw/cli.rb:183-189` の TODO、`lib/exwiw/query_ast_builder.rb:109-118` の TODO)。
|
|
9
|
+
|
|
10
|
+
このプランでは同等の機能を SQL アダプタにも提供する。命名・ゲーティングは既存の
|
|
11
|
+
`--target-table` / `--target-collection` の分け方を踏襲し、**アダプタ別に厳密分離**する:
|
|
12
|
+
|
|
13
|
+
- `--ids-field` … mongodb 専用(既存)
|
|
14
|
+
- `--ids-column` … SQL アダプタ専用(新規)
|
|
15
|
+
- 両方同時指定は拒否、不適合アダプタとの組み合わせも拒否
|
|
16
|
+
|
|
17
|
+
内部的には双方とも `DumpTarget#ids_field` に集約される(`--target-collection` が
|
|
18
|
+
`@target_table_name` に畳まれるのと同じパターン)。
|
|
19
|
+
|
|
20
|
+
## 変更内容
|
|
21
|
+
|
|
22
|
+
### 1. `lib/exwiw/cli.rb` — フラグ定義・畳み込み・バリデーション
|
|
23
|
+
|
|
24
|
+
- **インスタンス変数追加**(`initialize`, 42行目付近): `@ids_column = nil` を追加。
|
|
25
|
+
- **フラグ定義**(`parser`, 309行目付近): `--ids-field` の直後に追加。
|
|
26
|
+
```ruby
|
|
27
|
+
opts.on("--ids-column=[COLUMN]", "Column on the target table that --ids is matched against. Defaults to the primary key. (sql adapters only)") { |v| @ids_column = v }
|
|
28
|
+
```
|
|
29
|
+
- **エイリアス畳み込み**: `resolve_target_collection_alias!`(210行目)に倣い
|
|
30
|
+
`resolve_ids_column_alias!` を新設し `validate_options!` の冒頭で呼ぶ。挙動:
|
|
31
|
+
- `--ids-field` と `--ids-column` の同時指定を拒否
|
|
32
|
+
("Specify only one of --ids-field and --ids-column")。
|
|
33
|
+
- `--ids-column` を mongodb で使った場合は拒否
|
|
34
|
+
("--ids-column is only supported by the sql adapters (use --ids-field)")。
|
|
35
|
+
- 問題なければ `@ids_field = @ids_column` に畳み込む。
|
|
36
|
+
- **`--ids-field` の検証更新**(175-190行目):
|
|
37
|
+
- `--target-table` 必須チェックは「`@ids_field` が立っていれば」で共通化されるため、
|
|
38
|
+
畳み込み後はそのまま両方をカバーする(メッセージは ids_field/ids_column を
|
|
39
|
+
使った側に合わせて出し分けるか、汎用文言にする — 実装時に調整)。
|
|
40
|
+
- mongodb 限定チェック(186-189行目)は `--ids-field` 用に維持
|
|
41
|
+
("--ids-field is currently only supported by the mongodb adapter" のまま)。
|
|
42
|
+
|
|
43
|
+
実装方針: 畳み込み前にどちらのフラグが使われたかが分かる状態で
|
|
44
|
+
「target-table 必須」「アダプタ整合」を検証してから `@ids_field` に集約する。
|
|
45
|
+
|
|
46
|
+
### 2. `lib/exwiw/query_ast_builder.rb` — WHERE 句に反映
|
|
47
|
+
|
|
48
|
+
**当初の想定(1行変更)は関連テーブルで不正確になることが判明**したため、設計を変更した。
|
|
49
|
+
SQL アダプタは単一クエリで `dump_target.ids` を外部キーに直接伝播する
|
|
50
|
+
(`orders.user_id IN ids`)。これは `--ids` が主キーである前提のため、`--ids-column`
|
|
51
|
+
で別カラムを指定すると関連テーブルが壊れる(mongodb は @state に主キーを溜めて伝播する
|
|
52
|
+
ので正しい)。
|
|
53
|
+
|
|
54
|
+
採用したアプローチ: **ターゲットを介すサブクエリ**。`ids_field` 指定時、外部キー制約を
|
|
55
|
+
`fk IN (SELECT pk FROM target WHERE ids_field IN (ids))` に置き換える。direct /
|
|
56
|
+
indirect / polymorphic を一律に正しく扱える。
|
|
57
|
+
|
|
58
|
+
- `lib/exwiw/query_ast.rb`: `Subquery` 構造体を追加。`WhereClause` に
|
|
59
|
+
operator `:in_subquery`(value が `Subquery`)を導入し `to_h` を対応。
|
|
60
|
+
- `lib/exwiw/query_ast_builder.rb`: `dump_target_fk_clause(foreign_key)` ヘルパーを新設。
|
|
61
|
+
`ids_field` 無しなら従来通り `eq`、有りなら `:in_subquery` を返す。
|
|
62
|
+
- `build_where_clauses`(direct belongs_to)と `build_join_clauses`
|
|
63
|
+
(indirect の `relation_to_dump_target` hop)の両方で利用。
|
|
64
|
+
- ターゲットテーブル自身のフィルタは `ids_field || primary_key` の `eq`(従来どおり)。
|
|
65
|
+
- 各 SQL アダプタ(postgresql / mysql2 / sqlite3)の `compile_where_condition` に
|
|
66
|
+
`:in_subquery` 分岐と `compile_subquery` を追加。`is_a?(WhereClause)` のままなので
|
|
67
|
+
bulk_delete のサブクエリ生成・JoinClause.to_h もそのまま動く。
|
|
68
|
+
|
|
69
|
+
補足: `--ids-column` がマスク対象カラムの場合、`delete-*` の冪等性が崩れる
|
|
70
|
+
(README に注記済み)。
|
|
71
|
+
|
|
72
|
+
## 検証
|
|
73
|
+
|
|
74
|
+
- `bundle exec rspec spec/cli_spec.rb` — 既存の `--ids-field` validation を維持しつつ、
|
|
75
|
+
新規ケースを追加:
|
|
76
|
+
- `--ids-column` が `@ids_field`(畳み込み後)にパースされる
|
|
77
|
+
- `--ids-column` を mongodb で指定すると拒否される
|
|
78
|
+
- `--ids-field` と `--ids-column` の同時指定が拒否される
|
|
79
|
+
- `--ids-column` を target-table 無しで指定すると拒否される
|
|
80
|
+
- `bundle exec rspec spec/query_ast_builder_spec.rb`(存在すれば)に
|
|
81
|
+
`ids_field` 指定時に対象テーブルの WHERE が主キーではなく当該カラムになることを確認する
|
|
82
|
+
ケースを追加。
|
|
83
|
+
- `explain` サブコマンド(SQL のみ対応)で end-to-end 確認:
|
|
84
|
+
既存 scenario(例 `scenario/sqlite3-schema`)に対し
|
|
85
|
+
`--target-table=... --ids=... --ids-column=<col>` を渡し、出力 SQL の WHERE が
|
|
86
|
+
`<table>.<col> IN (...)` になることを目視確認。
|
|
87
|
+
- `bundle exec rspec`(全体)でリグレッションが無いこと。
|
|
88
|
+
|
|
89
|
+
## ドキュメント
|
|
90
|
+
|
|
91
|
+
`README.md:415` の `--ids-field` 説明を更新し、SQL では `--ids-column` を使う旨と例
|
|
92
|
+
(例: `--target-table=users --ids=a@example.com --ids-column=email`)を追記。
|
|
93
|
+
「SQL adapters reject it / TODO」の記述を解消する。
|
|
@@ -138,10 +138,11 @@ module Exwiw
|
|
|
138
138
|
subquery_sql = compile_ast(subquery_ast)
|
|
139
139
|
sql += "\nWHERE #{select_query_ast.from_table_name}.#{foreign_key} IN (#{subquery_sql})"
|
|
140
140
|
|
|
141
|
-
# first_join.base_where_clauses
|
|
142
|
-
# (from_table_name)
|
|
143
|
-
#
|
|
144
|
-
#
|
|
141
|
+
# first_join.base_where_clauses holds conditions on the outer
|
|
142
|
+
# delete-target table (from_table_name), such as a polymorphic type
|
|
143
|
+
# column. They are not part of the subquery, so add them to the outer
|
|
144
|
+
# WHERE. This prevents deleting rows that belong to a different
|
|
145
|
+
# polymorphic type.
|
|
145
146
|
first_join.base_where_clauses.each do |where|
|
|
146
147
|
next unless where.is_a?(Exwiw::QueryAst::WhereClause)
|
|
147
148
|
|
|
@@ -171,8 +172,9 @@ module Exwiw
|
|
|
171
172
|
sql += " AND #{compiled_where_condition}"
|
|
172
173
|
end
|
|
173
174
|
|
|
174
|
-
# base_where_clauses
|
|
175
|
-
#
|
|
175
|
+
# base_where_clauses is compiled against the joined-from table
|
|
176
|
+
# (base_table_name), e.g. the type-column filter on a polymorphic
|
|
177
|
+
# source table.
|
|
176
178
|
join.base_where_clauses.each do |where|
|
|
177
179
|
compiled_where_condition = compile_where_condition(where, join.base_table_name)
|
|
178
180
|
sql += " AND #{compiled_where_condition}"
|
|
@@ -201,11 +203,20 @@ module Exwiw
|
|
|
201
203
|
else
|
|
202
204
|
"#{key} IN (#{values.join(', ')})"
|
|
203
205
|
end
|
|
206
|
+
elsif where_clause.operator == :in_subquery
|
|
207
|
+
"#{key} IN (#{compile_subquery(where_clause.value)})"
|
|
204
208
|
else
|
|
205
209
|
raise "Unsupported operator: #{where_clause.operator}"
|
|
206
210
|
end
|
|
207
211
|
end
|
|
208
212
|
|
|
213
|
+
private def compile_subquery(subquery)
|
|
214
|
+
inner_values = subquery.where_values.map { |v| escape_value(v) }
|
|
215
|
+
"SELECT #{subquery.table_name}.#{subquery.select_column} " \
|
|
216
|
+
"FROM #{subquery.table_name} " \
|
|
217
|
+
"WHERE #{subquery.table_name}.#{subquery.where_column} IN (#{inner_values.join(', ')})"
|
|
218
|
+
end
|
|
219
|
+
|
|
209
220
|
private def escape_value(value)
|
|
210
221
|
case value
|
|
211
222
|
when nil
|
|
@@ -180,10 +180,11 @@ module Exwiw
|
|
|
180
180
|
subquery_sql = compile_ast(subquery_ast)
|
|
181
181
|
sql += "\nWHERE #{select_query_ast.from_table_name}.#{foreign_key} IN (#{subquery_sql})"
|
|
182
182
|
|
|
183
|
-
# first_join.base_where_clauses
|
|
184
|
-
# (from_table_name)
|
|
185
|
-
#
|
|
186
|
-
#
|
|
183
|
+
# first_join.base_where_clauses holds conditions on the outer
|
|
184
|
+
# delete-target table (from_table_name), such as a polymorphic type
|
|
185
|
+
# column. They are not part of the subquery, so add them to the outer
|
|
186
|
+
# WHERE. This prevents deleting rows that belong to a different
|
|
187
|
+
# polymorphic type.
|
|
187
188
|
first_join.base_where_clauses.each do |where|
|
|
188
189
|
next unless where.is_a?(Exwiw::QueryAst::WhereClause)
|
|
189
190
|
|
|
@@ -213,8 +214,9 @@ module Exwiw
|
|
|
213
214
|
sql += " AND #{compiled_where_condition}"
|
|
214
215
|
end
|
|
215
216
|
|
|
216
|
-
# base_where_clauses
|
|
217
|
-
#
|
|
217
|
+
# base_where_clauses is compiled against the joined-from table
|
|
218
|
+
# (base_table_name), e.g. the type-column filter on a polymorphic
|
|
219
|
+
# source table.
|
|
218
220
|
join.base_where_clauses.each do |where|
|
|
219
221
|
compiled_where_condition = compile_where_condition(where, join.base_table_name)
|
|
220
222
|
sql += " AND #{compiled_where_condition}"
|
|
@@ -243,11 +245,20 @@ module Exwiw
|
|
|
243
245
|
else
|
|
244
246
|
"#{key} IN (#{values.join(', ')})"
|
|
245
247
|
end
|
|
248
|
+
elsif where_clause.operator == :in_subquery
|
|
249
|
+
"#{key} IN (#{compile_subquery(where_clause.value)})"
|
|
246
250
|
else
|
|
247
251
|
raise "Unsupported operator: #{where_clause.operator}"
|
|
248
252
|
end
|
|
249
253
|
end
|
|
250
254
|
|
|
255
|
+
private def compile_subquery(subquery)
|
|
256
|
+
inner_values = subquery.where_values.map { |v| escape_value(v) }
|
|
257
|
+
"SELECT #{subquery.table_name}.#{subquery.select_column} " \
|
|
258
|
+
"FROM #{subquery.table_name} " \
|
|
259
|
+
"WHERE #{subquery.table_name}.#{subquery.where_column} IN (#{inner_values.join(', ')})"
|
|
260
|
+
end
|
|
261
|
+
|
|
251
262
|
private def escape_value(value)
|
|
252
263
|
case value
|
|
253
264
|
when nil
|
|
@@ -125,10 +125,11 @@ module Exwiw
|
|
|
125
125
|
subquery_sql = compile_ast(subquery_ast)
|
|
126
126
|
sql += "\nWHERE #{select_query_ast.from_table_name}.#{foreign_key} IN (#{subquery_sql})"
|
|
127
127
|
|
|
128
|
-
# first_join.base_where_clauses
|
|
129
|
-
# (from_table_name)
|
|
130
|
-
#
|
|
131
|
-
#
|
|
128
|
+
# first_join.base_where_clauses holds conditions on the outer
|
|
129
|
+
# delete-target table (from_table_name), such as a polymorphic type
|
|
130
|
+
# column. They are not part of the subquery, so add them to the outer
|
|
131
|
+
# WHERE. This prevents deleting rows that belong to a different
|
|
132
|
+
# polymorphic type.
|
|
132
133
|
first_join.base_where_clauses.each do |where|
|
|
133
134
|
next unless where.is_a?(Exwiw::QueryAst::WhereClause)
|
|
134
135
|
|
|
@@ -158,8 +159,9 @@ module Exwiw
|
|
|
158
159
|
sql += " AND #{compiled_where_condition}"
|
|
159
160
|
end
|
|
160
161
|
|
|
161
|
-
# base_where_clauses
|
|
162
|
-
#
|
|
162
|
+
# base_where_clauses is compiled against the joined-from table
|
|
163
|
+
# (base_table_name), e.g. the type-column filter on a polymorphic
|
|
164
|
+
# source table.
|
|
163
165
|
join.base_where_clauses.each do |where|
|
|
164
166
|
compiled_where_condition = compile_where_condition(where, join.base_table_name)
|
|
165
167
|
sql += " AND #{compiled_where_condition}"
|
|
@@ -188,11 +190,20 @@ module Exwiw
|
|
|
188
190
|
else
|
|
189
191
|
"#{key} IN (#{values.join(', ')})"
|
|
190
192
|
end
|
|
193
|
+
elsif where_clause.operator == :in_subquery
|
|
194
|
+
"#{key} IN (#{compile_subquery(where_clause.value)})"
|
|
191
195
|
else
|
|
192
196
|
raise "Unsupported operator: #{where_clause.operator}"
|
|
193
197
|
end
|
|
194
198
|
end
|
|
195
199
|
|
|
200
|
+
private def compile_subquery(subquery)
|
|
201
|
+
inner_values = subquery.where_values.map { |v| escape_value(v) }
|
|
202
|
+
"SELECT #{subquery.table_name}.#{subquery.select_column} " \
|
|
203
|
+
"FROM #{subquery.table_name} " \
|
|
204
|
+
"WHERE #{subquery.table_name}.#{subquery.where_column} IN (#{inner_values.join(', ')})"
|
|
205
|
+
end
|
|
206
|
+
|
|
196
207
|
private def escape_value(value)
|
|
197
208
|
case value
|
|
198
209
|
when nil
|
data/lib/exwiw/belongs_to.rb
CHANGED
|
@@ -6,9 +6,10 @@ module Exwiw
|
|
|
6
6
|
|
|
7
7
|
attribute :foreign_key, String
|
|
8
8
|
attribute :table_name, String
|
|
9
|
-
# polymorphic
|
|
10
|
-
# (
|
|
11
|
-
#
|
|
9
|
+
# Set only for a polymorphic association. `foreign_type` is the name of the
|
|
10
|
+
# column storing the type (e.g. `reviewable_type`), and `type_value` is the
|
|
11
|
+
# value held in that column (e.g. `"Product"`). Both are nil for a
|
|
12
|
+
# non-polymorphic belongs_to.
|
|
12
13
|
attribute :foreign_type, optional(String), skip_serializing_if_nil: true
|
|
13
14
|
attribute :type_value, optional(String), skip_serializing_if_nil: true
|
|
14
15
|
|
data/lib/exwiw/cli.rb
CHANGED
|
@@ -10,7 +10,7 @@ require 'exwiw'
|
|
|
10
10
|
|
|
11
11
|
module Exwiw
|
|
12
12
|
class CLI
|
|
13
|
-
KNOWN_SUBCOMMANDS = %w[
|
|
13
|
+
KNOWN_SUBCOMMANDS = %w[export explain].freeze
|
|
14
14
|
|
|
15
15
|
def self.start(argv)
|
|
16
16
|
new(argv).run
|
|
@@ -23,7 +23,7 @@ module Exwiw
|
|
|
23
23
|
if !@argv.empty? && !@argv.first.start_with?("-") && KNOWN_SUBCOMMANDS.include?(@argv.first)
|
|
24
24
|
@argv.shift
|
|
25
25
|
else
|
|
26
|
-
"
|
|
26
|
+
"export"
|
|
27
27
|
end
|
|
28
28
|
|
|
29
29
|
@help = @argv.empty?
|
|
@@ -40,6 +40,7 @@ module Exwiw
|
|
|
40
40
|
@target_collection_name = nil
|
|
41
41
|
@ids = []
|
|
42
42
|
@ids_field = nil
|
|
43
|
+
@ids_column = nil
|
|
43
44
|
@output_format = nil
|
|
44
45
|
@insert_only = nil
|
|
45
46
|
@after_insert_hook_path = nil
|
|
@@ -74,7 +75,7 @@ module Exwiw
|
|
|
74
75
|
logger = build_logger
|
|
75
76
|
|
|
76
77
|
case @subcommand
|
|
77
|
-
when "
|
|
78
|
+
when "export"
|
|
78
79
|
Runner.new(
|
|
79
80
|
connection_config: connection_config,
|
|
80
81
|
output_dir: @output_dir,
|
|
@@ -99,6 +100,7 @@ module Exwiw
|
|
|
99
100
|
|
|
100
101
|
private def validate_options!
|
|
101
102
|
resolve_target_collection_alias!
|
|
103
|
+
resolve_ids_column_alias!
|
|
102
104
|
|
|
103
105
|
if @subcommand == "explain"
|
|
104
106
|
validate_explain_only!
|
|
@@ -130,7 +132,7 @@ module Exwiw
|
|
|
130
132
|
exit 1
|
|
131
133
|
end
|
|
132
134
|
|
|
133
|
-
if @subcommand == "
|
|
135
|
+
if @subcommand == "export"
|
|
134
136
|
@output_dir ||= "dump"
|
|
135
137
|
@output_format ||= "insert"
|
|
136
138
|
@insert_only = @insert_only ? true : false
|
|
@@ -172,23 +174,6 @@ module Exwiw
|
|
|
172
174
|
exit 1
|
|
173
175
|
end
|
|
174
176
|
|
|
175
|
-
if @ids_field
|
|
176
|
-
# --ids-field overrides the field --ids filters against on the target
|
|
177
|
-
# table; it is meaningless without a target table to constrain.
|
|
178
|
-
if !@target_table_name
|
|
179
|
-
$stderr.puts "--target-table is required when --ids-field is specified"
|
|
180
|
-
exit 1
|
|
181
|
-
end
|
|
182
|
-
|
|
183
|
-
# TODO: support --ids-field for the sql adapters (mysql2/postgresql/
|
|
184
|
-
# sqlite3) by threading dump_target.ids_field through QueryAstBuilder's
|
|
185
|
-
# WHERE clause on the target table. For now it is mongodb-only.
|
|
186
|
-
if @database_adapter != "mongodb"
|
|
187
|
-
$stderr.puts "--ids-field is currently only supported by the mongodb adapter"
|
|
188
|
-
exit 1
|
|
189
|
-
end
|
|
190
|
-
end
|
|
191
|
-
|
|
192
177
|
if @after_insert_hook_path
|
|
193
178
|
unless File.file?(@after_insert_hook_path)
|
|
194
179
|
$stderr.puts "--after-insert-hook file not found: #{@after_insert_hook_path}"
|
|
@@ -223,6 +208,43 @@ module Exwiw
|
|
|
223
208
|
@target_table_name = @target_collection_name
|
|
224
209
|
end
|
|
225
210
|
|
|
211
|
+
# `--ids-column` is the sql-adapter spelling of `--ids-field` (the mongodb
|
|
212
|
+
# spelling). Both override which column/field `--ids` is matched against on
|
|
213
|
+
# the target table; internally they fold into the single @ids_field carried
|
|
214
|
+
# by DumpTarget. Mirror the --target-table/--target-collection split: each
|
|
215
|
+
# flag is restricted to its adapter family and the two are mutually
|
|
216
|
+
# exclusive. Runs after resolve_target_collection_alias! so
|
|
217
|
+
# @target_table_name already reflects the collection alias.
|
|
218
|
+
private def resolve_ids_column_alias!
|
|
219
|
+
if @ids_field && @ids_column
|
|
220
|
+
$stderr.puts "Specify only one of --ids-field and --ids-column"
|
|
221
|
+
exit 1
|
|
222
|
+
end
|
|
223
|
+
|
|
224
|
+
if @ids_field && @database_adapter != "mongodb"
|
|
225
|
+
$stderr.puts "--ids-field is only supported by the mongodb adapter (use --ids-column)"
|
|
226
|
+
exit 1
|
|
227
|
+
end
|
|
228
|
+
|
|
229
|
+
if @ids_column
|
|
230
|
+
sql_adapters = ["mysql2", "postgresql", "sqlite3"]
|
|
231
|
+
unless sql_adapters.include?(@database_adapter)
|
|
232
|
+
$stderr.puts "--ids-column is only supported by the sql adapters (use --ids-field)"
|
|
233
|
+
exit 1
|
|
234
|
+
end
|
|
235
|
+
|
|
236
|
+
@ids_field = @ids_column
|
|
237
|
+
end
|
|
238
|
+
|
|
239
|
+
# --ids-field/--ids-column override the column --ids filters against on
|
|
240
|
+
# the target table; meaningless without a target table to constrain.
|
|
241
|
+
if @ids_field && !@target_table_name
|
|
242
|
+
flag = @ids_column ? "--ids-column" : "--ids-field"
|
|
243
|
+
$stderr.puts "--target-table is required when #{flag} is specified"
|
|
244
|
+
exit 1
|
|
245
|
+
end
|
|
246
|
+
end
|
|
247
|
+
|
|
226
248
|
private def validate_explain_only!
|
|
227
249
|
if @database_adapter == "mongodb"
|
|
228
250
|
$stderr.puts "mongodb adapter is not yet supported by 'explain' subcommand"
|
|
@@ -284,7 +306,7 @@ module Exwiw
|
|
|
284
306
|
Usage: exwiw [SUBCOMMAND] [options]
|
|
285
307
|
|
|
286
308
|
Subcommands:
|
|
287
|
-
|
|
309
|
+
export Generate INSERT/COPY SQL files (default when omitted).
|
|
288
310
|
explain Print EXPLAIN output for each extraction query to stdout.
|
|
289
311
|
(not yet supported for the mongodb adapter)
|
|
290
312
|
BANNER
|
|
@@ -293,7 +315,7 @@ module Exwiw
|
|
|
293
315
|
opts.on("-h", "--host=HOST", "Target database host") { |v| @database_host = v }
|
|
294
316
|
opts.on("-p", "--port=PORT", "Target database port") { |v| @database_port = v }
|
|
295
317
|
opts.on("-u", "--user=USERNAME", "Target database user") { |v| @database_user = v }
|
|
296
|
-
opts.on("-o", "--output-dir=[DUMP_DIR_PATH]", "Output file path. default is dump/ (
|
|
318
|
+
opts.on("-o", "--output-dir=[DUMP_DIR_PATH]", "Output file path. default is dump/ (export subcommand only)") do |v|
|
|
297
319
|
v = v.end_with?("/") ? v[0..-2] : v
|
|
298
320
|
@output_dir = File.expand_path(v)
|
|
299
321
|
end
|
|
@@ -306,10 +328,11 @@ module Exwiw
|
|
|
306
328
|
opts.on("--target-table=[TABLE]", "Target table for extraction. If omitted, dump all tables.") { |v| @target_table_name = v }
|
|
307
329
|
opts.on("--target-collection=[COLLECTION]", "Alias of --target-table for the mongodb adapter.") { |v| @target_collection_name = v }
|
|
308
330
|
opts.on("--ids=[IDS]", "Comma-separated list of identifiers. Required when --target-table is given.") { |v| @ids = v.split(',') }
|
|
309
|
-
opts.on("--ids-field=[FIELD]", "Field on the target
|
|
310
|
-
opts.on("--
|
|
311
|
-
opts.on("--
|
|
312
|
-
opts.on("--
|
|
331
|
+
opts.on("--ids-field=[FIELD]", "Field on the target collection that --ids is matched against. Defaults to the primary key. (mongodb adapter only)") { |v| @ids_field = v }
|
|
332
|
+
opts.on("--ids-column=[COLUMN]", "Column on the target table that --ids is matched against. Defaults to the primary key. (sql adapters only)") { |v| @ids_column = v }
|
|
333
|
+
opts.on("--output-format=[FORMAT]", "Output format: insert (default) or copy (PostgreSQL only, export subcommand only)") { |v| @output_format = v }
|
|
334
|
+
opts.on("--insert-only", "Do not generate DELETE SQL files (export subcommand only)") { @insert_only = true }
|
|
335
|
+
opts.on("--after-insert-hook=PATH", "Path to a .rb or .sh post-processing hook executed after all insert/delete files are written (export subcommand only)") do |v|
|
|
313
336
|
@after_insert_hook_path = File.expand_path(v)
|
|
314
337
|
end
|
|
315
338
|
opts.on("--log-level=LEVEL", "Log level (debug, info). default is info") { |v| @log_level = v.to_sym }
|
|
@@ -159,11 +159,13 @@ module Exwiw
|
|
|
159
159
|
end
|
|
160
160
|
end
|
|
161
161
|
|
|
162
|
-
# polymorphic belongs_to (`belongs_to :reviewable, polymorphic: true`)
|
|
163
|
-
#
|
|
164
|
-
#
|
|
162
|
+
# A polymorphic belongs_to (`belongs_to :reviewable, polymorphic: true`)
|
|
163
|
+
# has no single target collection, so it is not supported yet. Exclude it
|
|
164
|
+
# here to avoid emitting an incorrect FK (leaving room to expand it later,
|
|
165
|
+
# like the ActiveRecord version does).
|
|
165
166
|
#
|
|
166
|
-
#
|
|
167
|
+
# In an inheritance hierarchy the base class and its subclasses carry the
|
|
168
|
+
# same belongs_to twice, so uniq them.
|
|
167
169
|
belongs_to_assocs
|
|
168
170
|
.reject(&:polymorphic?)
|
|
169
171
|
.map { |assoc| { table_name: assoc.klass.collection_name.to_s, foreign_key: assoc.foreign_key } }
|
data/lib/exwiw/query_ast.rb
CHANGED
|
@@ -3,11 +3,11 @@
|
|
|
3
3
|
module Exwiw
|
|
4
4
|
module QueryAst
|
|
5
5
|
class JoinClause
|
|
6
|
-
# `where_clauses`
|
|
7
|
-
#
|
|
8
|
-
#
|
|
9
|
-
#
|
|
10
|
-
#
|
|
6
|
+
# `where_clauses` is compiled against this join's join_table_name (the
|
|
7
|
+
# joined-to table). `base_where_clauses`, on the other hand, is compiled
|
|
8
|
+
# against base_table_name (the joined-from table). The latter is used for
|
|
9
|
+
# the case where the source table polymorphically belongs_to the joined-to
|
|
10
|
+
# table and the type column (foreign_type) lives on the source table.
|
|
11
11
|
attr_reader :base_table_name, :foreign_key, :join_table_name, :primary_key, :where_clauses, :base_where_clauses
|
|
12
12
|
|
|
13
13
|
def initialize(base_table_name:, foreign_key:, join_table_name:, primary_key:, where_clauses: [], base_where_clauses: [])
|
|
@@ -41,7 +41,26 @@ module Exwiw
|
|
|
41
41
|
{
|
|
42
42
|
column_name: column_name,
|
|
43
43
|
operator: operator,
|
|
44
|
-
value: value,
|
|
44
|
+
value: value.is_a?(Subquery) ? value.to_h : value,
|
|
45
|
+
}
|
|
46
|
+
end
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
# Resolves a set of values on `where_column` to the rows' `select_column`
|
|
50
|
+
# via a nested SELECT. Used as the `value` of a WhereClause whose operator
|
|
51
|
+
# is `:in_subquery`, so `--ids-column`/`--ids-field` can filter related
|
|
52
|
+
# tables through the target table's primary key:
|
|
53
|
+
#
|
|
54
|
+
# <table>.<fk> IN (SELECT <table_name>.<select_column>
|
|
55
|
+
# FROM <table_name>
|
|
56
|
+
# WHERE <table_name>.<where_column> IN (<where_values>))
|
|
57
|
+
Subquery = Struct.new(:table_name, :select_column, :where_column, :where_values, keyword_init: true) do
|
|
58
|
+
def to_h
|
|
59
|
+
{
|
|
60
|
+
table_name: table_name,
|
|
61
|
+
select_column: select_column,
|
|
62
|
+
where_column: where_column,
|
|
63
|
+
where_values: where_values,
|
|
45
64
|
}
|
|
46
65
|
end
|
|
47
66
|
end
|
|
@@ -58,12 +58,12 @@ module Exwiw
|
|
|
58
58
|
base_where_clauses: []
|
|
59
59
|
)
|
|
60
60
|
|
|
61
|
-
#
|
|
62
|
-
#
|
|
63
|
-
# (foreign_type)
|
|
64
|
-
#
|
|
65
|
-
#
|
|
66
|
-
#
|
|
61
|
+
# When this hop itself is a polymorphic belongs_to (e.g. comments
|
|
62
|
+
# polymorphically belongs_to posts as commentable), the type column
|
|
63
|
+
# (foreign_type) lives on the source table (from_table = base_table_name).
|
|
64
|
+
# The foreign key alone is not enough — a value like reviewable_id=1 can
|
|
65
|
+
# collide with rows of another model — so add the type condition to
|
|
66
|
+
# base_where_clauses to narrow down the source table.
|
|
67
67
|
if relation.polymorphic?
|
|
68
68
|
join_clause.base_where_clauses.push QueryAst::WhereClause.new(
|
|
69
69
|
column_name: relation.foreign_type,
|
|
@@ -73,16 +73,13 @@ module Exwiw
|
|
|
73
73
|
end
|
|
74
74
|
relation_to_dump_target = to_table.belongs_to(dump_target.table_name)
|
|
75
75
|
if relation_to_dump_target
|
|
76
|
-
join_clause.where_clauses.push
|
|
77
|
-
column_name: relation_to_dump_target.foreign_key,
|
|
78
|
-
operator: :eq,
|
|
79
|
-
value: dump_target.ids
|
|
80
|
-
)
|
|
76
|
+
join_clause.where_clauses.push dump_target_fk_clause(relation_to_dump_target.foreign_key)
|
|
81
77
|
|
|
82
|
-
#
|
|
83
|
-
#
|
|
84
|
-
# (= join_table_name)
|
|
85
|
-
#
|
|
78
|
+
# When the intermediate table polymorphically belongs_to the dump
|
|
79
|
+
# target, also add the type column (foreign_type) to the join
|
|
80
|
+
# condition. The type column lives on to_table (= join_table_name), so
|
|
81
|
+
# it rides on the existing mechanism where a JoinClause's where_clauses
|
|
82
|
+
# are compiled against join_table_name.
|
|
86
83
|
if relation_to_dump_target.polymorphic?
|
|
87
84
|
join_clause.where_clauses.push QueryAst::WhereClause.new(
|
|
88
85
|
column_name: relation_to_dump_target.foreign_type,
|
|
@@ -107,12 +104,12 @@ module Exwiw
|
|
|
107
104
|
clauses = []
|
|
108
105
|
|
|
109
106
|
if table.name == dump_target.table_name
|
|
110
|
-
#
|
|
111
|
-
# primary-key column on the target table
|
|
112
|
-
#
|
|
113
|
-
#
|
|
107
|
+
# `--ids-column` (folded into dump_target.ids_field by the CLI) lets
|
|
108
|
+
# `--ids` match a non primary-key column on the target table; otherwise
|
|
109
|
+
# fall back to the primary key. Only the target table's filter changes —
|
|
110
|
+
# downstream foreign-key propagation still keys off the primary key.
|
|
114
111
|
clauses.push Exwiw::QueryAst::WhereClause.new(
|
|
115
|
-
column_name: table.primary_key,
|
|
112
|
+
column_name: dump_target.ids_field || table.primary_key,
|
|
116
113
|
operator: :eq,
|
|
117
114
|
value: dump_target.ids
|
|
118
115
|
)
|
|
@@ -123,15 +120,11 @@ module Exwiw
|
|
|
123
120
|
belongs_to = table.belongs_to(dump_target.table_name)
|
|
124
121
|
return clauses if belongs_to.nil?
|
|
125
122
|
|
|
126
|
-
clauses.push
|
|
127
|
-
column_name: belongs_to.foreign_key,
|
|
128
|
-
operator: :eq,
|
|
129
|
-
value: dump_target.ids
|
|
130
|
-
)
|
|
123
|
+
clauses.push dump_target_fk_clause(belongs_to.foreign_key)
|
|
131
124
|
|
|
132
|
-
# polymorphic belongs_to
|
|
133
|
-
# (
|
|
134
|
-
#
|
|
125
|
+
# For a polymorphic belongs_to the foreign key alone cannot distinguish the
|
|
126
|
+
# type (e.g. reviewable_id=1 could be a Product or another model), so add a
|
|
127
|
+
# condition filtering the type column (foreign_type) by type_value.
|
|
135
128
|
if belongs_to.polymorphic?
|
|
136
129
|
clauses.push Exwiw::QueryAst::WhereClause.new(
|
|
137
130
|
column_name: belongs_to.foreign_type,
|
|
@@ -147,6 +140,36 @@ module Exwiw
|
|
|
147
140
|
clauses
|
|
148
141
|
end
|
|
149
142
|
|
|
143
|
+
# Builds the WHERE clause that constrains a `foreign_key` pointing at the
|
|
144
|
+
# dump target. Normally `--ids` are the target's primary keys, so a plain
|
|
145
|
+
# `foreign_key IN (ids)` suffices. When `--ids-column`/`--ids-field` is set
|
|
146
|
+
# (dump_target.ids_field), `--ids` match a non primary-key column instead,
|
|
147
|
+
# so the foreign key must be resolved through the target table:
|
|
148
|
+
# `foreign_key IN (SELECT pk FROM target WHERE ids_field IN (ids))`.
|
|
149
|
+
# This keeps related-table extraction correct regardless of whether the
|
|
150
|
+
# relation is direct, indirect, or polymorphic.
|
|
151
|
+
private def dump_target_fk_clause(foreign_key)
|
|
152
|
+
unless dump_target.ids_field
|
|
153
|
+
return Exwiw::QueryAst::WhereClause.new(
|
|
154
|
+
column_name: foreign_key,
|
|
155
|
+
operator: :eq,
|
|
156
|
+
value: dump_target.ids
|
|
157
|
+
)
|
|
158
|
+
end
|
|
159
|
+
|
|
160
|
+
target = table_by_name.fetch(dump_target.table_name)
|
|
161
|
+
Exwiw::QueryAst::WhereClause.new(
|
|
162
|
+
column_name: foreign_key,
|
|
163
|
+
operator: :in_subquery,
|
|
164
|
+
value: Exwiw::QueryAst::Subquery.new(
|
|
165
|
+
table_name: target.name,
|
|
166
|
+
select_column: target.primary_key,
|
|
167
|
+
where_column: dump_target.ids_field,
|
|
168
|
+
where_values: dump_target.ids
|
|
169
|
+
)
|
|
170
|
+
)
|
|
171
|
+
end
|
|
172
|
+
|
|
150
173
|
private def find_path_to_dump_target(table, table_by_name, dump_target)
|
|
151
174
|
return [] if table.name == dump_target.table_name
|
|
152
175
|
|
|
@@ -78,11 +78,12 @@ module Exwiw
|
|
|
78
78
|
representative = model_group.first
|
|
79
79
|
primary_key = representative.primary_key
|
|
80
80
|
|
|
81
|
-
#
|
|
82
|
-
#
|
|
83
|
-
#
|
|
84
|
-
#
|
|
85
|
-
#
|
|
81
|
+
# Tables with a composite primary key (`representative.primary_key` is an
|
|
82
|
+
# Array) are not supported yet. Emit them with `primary_key` omitted,
|
|
83
|
+
# `skip: true`, and a `type` that marks them as unsupported — the `type`
|
|
84
|
+
# acts as a signpost for adding support later. The config file itself is
|
|
85
|
+
# still generated so a user can manually remove `skip` and wire it up when
|
|
86
|
+
# needed.
|
|
86
87
|
if primary_key.is_a?(Array)
|
|
87
88
|
TableConfig.from_symbol_keys(
|
|
88
89
|
name: table_name,
|
|
@@ -110,12 +111,13 @@ module Exwiw
|
|
|
110
111
|
@models.reject(&:abstract_class?).select(&:table_exists?)
|
|
111
112
|
end
|
|
112
113
|
|
|
113
|
-
# rails-managed
|
|
114
|
-
#
|
|
115
|
-
# multi-DB
|
|
116
|
-
#
|
|
117
|
-
#
|
|
118
|
-
# (
|
|
114
|
+
# The rails-managed tables (`schema_migrations` / `ar_internal_metadata`)
|
|
115
|
+
# have no model class, so they cannot be picked up from
|
|
116
|
+
# `ActiveRecord::Base.descendants`. In a multi-DB setup each connection has
|
|
117
|
+
# its own migration history table, so we take the target connection and only
|
|
118
|
+
# emit an entry when the table actually exists on that connection. The table
|
|
119
|
+
# name itself (including any prefix/suffix) comes from the global settings
|
|
120
|
+
# (`ActiveRecord::Base.schema_migrations_table_name`, etc.).
|
|
119
121
|
private def build_rails_managed_tables(conn)
|
|
120
122
|
result = []
|
|
121
123
|
|
|
@@ -151,11 +153,11 @@ module Exwiw
|
|
|
151
153
|
.reject(&:polymorphic?)
|
|
152
154
|
.map { |assoc| { table_name: assoc.table_name, foreign_key: assoc.foreign_key } }
|
|
153
155
|
|
|
154
|
-
# polymorphic
|
|
155
|
-
#
|
|
156
|
-
# `has_many/has_one ..., as: <association_name
|
|
157
|
-
#
|
|
158
|
-
# (`
|
|
156
|
+
# A polymorphic belongs_to (`belongs_to :reviewable, polymorphic: true`)
|
|
157
|
+
# has no single target table. The candidate tables are found by looking up
|
|
158
|
+
# the other models that declare `has_many/has_one ..., as: <association_name>`.
|
|
159
|
+
# For each candidate table, expand one belongs_to entry carrying the type
|
|
160
|
+
# column (`foreign_type`) and the stored type value (`type_value`).
|
|
159
161
|
polymorphic = belongs_to_assocs
|
|
160
162
|
.select(&:polymorphic?)
|
|
161
163
|
.flat_map do |assoc|
|
|
@@ -172,11 +174,12 @@ module Exwiw
|
|
|
172
174
|
(non_polymorphic + polymorphic).uniq
|
|
173
175
|
end
|
|
174
176
|
|
|
175
|
-
#
|
|
176
|
-
# `
|
|
177
|
-
# `
|
|
178
|
-
# Ruby
|
|
179
|
-
#
|
|
177
|
+
# Enumerate the concrete models that can be targets of the polymorphic
|
|
178
|
+
# association `association_name`, by looking them up from every model's
|
|
179
|
+
# `has_many` / `has_one` `as:` option. The order of `concrete_models` depends
|
|
180
|
+
# on `ActiveRecord::Base.descendants`, which can vary by Ruby version, so sort
|
|
181
|
+
# by `table_name` to return a deterministic order and keep the generated
|
|
182
|
+
# belongs_to ordering stable.
|
|
180
183
|
private def polymorphic_target_models(association_name)
|
|
181
184
|
concrete_models.select do |model|
|
|
182
185
|
(model.reflect_on_all_associations(:has_many) +
|
data/lib/exwiw/table_config.rb
CHANGED
|
@@ -11,9 +11,10 @@ module Exwiw
|
|
|
11
11
|
RAILS_MANAGED_INTERNAL_METADATA,
|
|
12
12
|
].freeze
|
|
13
13
|
|
|
14
|
-
# exwiw
|
|
15
|
-
# schema:generate
|
|
16
|
-
#
|
|
14
|
+
# type marking a table with a composite primary key, which exwiw does not
|
|
15
|
+
# support yet. schema:generate attaches it together with skip:true. Unlike
|
|
16
|
+
# rails-managed tables, columns/belongs_tos are retained so it can serve as a
|
|
17
|
+
# signpost for adding support later.
|
|
17
18
|
UNSUPPORTED_COMPOSITE_PRIMARY_KEY = "unsupported_composite_primary_key"
|
|
18
19
|
|
|
19
20
|
attribute :name, String
|
|
@@ -143,8 +144,8 @@ module Exwiw
|
|
|
143
144
|
"Table '#{name}' has type=#{type}; columns must not be defined."
|
|
144
145
|
end
|
|
145
146
|
else
|
|
146
|
-
# skip:true
|
|
147
|
-
# (
|
|
147
|
+
# A skip:true table is not extracted, so primary_key is not required
|
|
148
|
+
# (e.g. a composite-primary-key table that exwiw does not support).
|
|
148
149
|
if primary_key.nil? && !skip
|
|
149
150
|
raise ArgumentError, "Table '#{name}' requires primary_key."
|
|
150
151
|
end
|
data/lib/exwiw/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: exwiw
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Shia
|
|
@@ -40,6 +40,7 @@ files:
|
|
|
40
40
|
- docs/plans/2026-05-22-after-insert-hook.md
|
|
41
41
|
- docs/plans/2026-05-22-postgres-copy-mode-scenario-test.md
|
|
42
42
|
- docs/plans/2026-05-29-rails-managed-tables.md
|
|
43
|
+
- docs/plans/2026-05-31-ids-column-for-sql-adapters.md
|
|
43
44
|
- exe/exwiw
|
|
44
45
|
- lib/exwiw.rb
|
|
45
46
|
- lib/exwiw/adapter.rb
|