exwiw 0.2.9 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7e5d922e740407599ecb3fa3992e0de402434bacfb38ca817a189a53acf16ab3
-  data.tar.gz: 2a8919778bb9395434587ebb69f49f4d8445cb1890e0999cb065b5577d802532
+  metadata.gz: e62483afc43054ae026c232502f40f6c1a92381a9c2c833ef1f7d603f6b845ae
+  data.tar.gz: 660a006272b9156bbc98af2a0c288f7943bdd40523adffc45de713b36a5d0426
 SHA512:
-  metadata.gz: e38a240087564c3e3909106268fba7ad3a8dce881924f8b210b1733d65f0bc12d3bb514af80be21e6f0ad39707f99e96edbaed3cc7c54e2d26677c3a0c7d203f
-  data.tar.gz: cca54067266034f8df074fce301623fd6c0c860e9fd1a13eac43d930b89f46a7b63d3e6230d155abcce3f6ab021c2299d518f8131d26ee5bcbe7c0a535093ee9
+  metadata.gz: 988813923c5e8a2aa1fd499c3af3a2d5467fc9a77b60664b1cd8d245ed1837a1cf43dc88a872bbbcfb7951902624e6b61fb50d290af6716a06c1fe8bb376ae6a
+  data.tar.gz: d8f751b794c20658c643ab2aeca769b01f7b50e9acc46c9605a4d7121d0d5fa8041bfbdad8021ede6bbfefc8cb36a937509120f25b7bdd059aaf6989f4c94e00

data/CHANGELOG.md

@@ -2,6 +2,8 @@
 
 ## [Unreleased]
 
+## [0.3.0] - 2026-05-31
+
 ## [0.2.9] - 2026-05-31
 
 ### Added

data/README.md

@@ -67,6 +67,8 @@ exwiw \
   --log-level=info
 ```
 
+By default `--ids` are matched against the target table's primary key. `--ids-column=COLUMN` matches them against a different column instead (e.g. `--target-table=users --ids=alice@example.com --ids-column=email`). Related tables are still extracted correctly: their foreign keys are resolved through the target via a subquery (`WHERE fk IN (SELECT pk FROM target WHERE COLUMN IN (...))`), so only the target table's filter column changes. This is the SQL-adapter counterpart of the mongodb `--ids-field`; the two are mutually exclusive and each is rejected by the other adapter family. Note: if `COLUMN` is itself masked, re-running `delete-*` against an already-imported (masked) dump won't match, so prefer a stable natural key.
+
 When `--target-table` and `--ids` are omitted, exwiw dumps all tables defined in `--config-dir`:
 
 ```bash
@@ -412,7 +414,7 @@ The MongoDB adapter is experimental. To use it:
 - The MongoDB adapter consumes a separate config type, `MongodbCollectionConfig`, with MongoDB-native naming. Use `fields` (instead of the SQL adapters' `columns`), and set `"primary_key": "_id"`. Foreign keys (`shop_id`, `user_id`, ...) stay as ordinary fields.
 - `--ids` values are coerced to the type actually stored in `_id` before filtering: integer-looking ids become `Integer`, 24-char hex ids become `BSON::ObjectId` (Mongoid's default `_id` type — a plain String would never match an ObjectId), and any other string is left as-is.
 - `--target-collection=COLLECTION` is a mongodb-only alias of `--target-table` (use whichever reads better for MongoDB). Specifying both, or using `--target-collection` with a non-mongodb adapter, is an error.
-- `--ids-field=FIELD` matches `--ids` against `FIELD` on the target collection instead of its primary key (e.g. `--target-collection=users --ids=a@example.com --ids-field=email`). Downstream foreign-key propagation still keys off the primary key, so only the target collection's filter changes. Unlike the primary-key path, the supplied ids are **not** type-coerced (the stored type of a custom field is unknown), so pass values matching the field's actual type. This flag is currently **mongodb-only** (the SQL adapters reject it; supporting them is a TODO).
+- `--ids-field=FIELD` matches `--ids` against `FIELD` on the target collection instead of its primary key (e.g. `--target-collection=users --ids=a@example.com --ids-field=email`). Downstream foreign-key propagation still keys off the primary key, so only the target collection's filter changes. Unlike the primary-key path, the supplied ids are **not** type-coerced (the stored type of a custom field is unknown), so pass values matching the field's actual type. This flag is **mongodb-only**; the SQL adapters use `--ids-column` instead (see below).
 - Output is JSON Lines (`insert-{idx}-{collection}.jsonl`) using MongoDB Extended JSON (relaxed mode). Import with `mongoimport`:
   ```bash
   mongoimport --db app_dev --collection users --file dump/insert-002-users.jsonl

data/docs/plans/2026-05-31-ids-column-for-sql-adapters.md

@@ -0,0 +1,93 @@
+# `--ids-column` を SQL アダプタに実装
+
+## Context
+
+MongoDB アダプタには `--ids-field` フラグがあり、`--ids` を対象テーブルの主キー以外の
+フィールドにマッチさせられる（`lib/exwiw/adapter/mongodb_adapter.rb:48-65`）。一方で
+SQL アダプタ（mysql2 / postgresql / sqlite3）では未実装で、CLI が明示的に拒否している
+（`lib/exwiw/cli.rb:183-189` の TODO、`lib/exwiw/query_ast_builder.rb:109-118` の TODO）。
+
+このプランでは同等の機能を SQL アダプタにも提供する。命名・ゲーティングは既存の
+`--target-table` / `--target-collection` の分け方を踏襲し、**アダプタ別に厳密分離**する:
+
+- `--ids-field` … mongodb 専用（既存）
+- `--ids-column` … SQL アダプタ専用（新規）
+- 両方同時指定は拒否、不適合アダプタとの組み合わせも拒否
+
+内部的には双方とも `DumpTarget#ids_field` に集約される（`--target-collection` が
+`@target_table_name` に畳まれるのと同じパターン）。
+
+## 変更内容
+
+### 1. `lib/exwiw/cli.rb` — フラグ定義・畳み込み・バリデーション
+
+- **インスタンス変数追加**（`initialize`, 42行目付近）: `@ids_column = nil` を追加。
+- **フラグ定義**（`parser`, 309行目付近）: `--ids-field` の直後に追加。
+  ```ruby
+  opts.on("--ids-column=[COLUMN]", "Column on the target table that --ids is matched against. Defaults to the primary key. (sql adapters only)") { |v| @ids_column = v }
+  ```
+- **エイリアス畳み込み**: `resolve_target_collection_alias!`（210行目）に倣い
+  `resolve_ids_column_alias!` を新設し `validate_options!` の冒頭で呼ぶ。挙動:
+  - `--ids-field` と `--ids-column` の同時指定を拒否
+    （"Specify only one of --ids-field and --ids-column"）。
+  - `--ids-column` を mongodb で使った場合は拒否
+    （"--ids-column is only supported by the sql adapters (use --ids-field)"）。
+  - 問題なければ `@ids_field = @ids_column` に畳み込む。
+- **`--ids-field` の検証更新**（175-190行目）:
+  - `--target-table` 必須チェックは「`@ids_field` が立っていれば」で共通化されるため、
+    畳み込み後はそのまま両方をカバーする（メッセージは ids_field/ids_column を
+    使った側に合わせて出し分けるか、汎用文言にする — 実装時に調整）。
+  - mongodb 限定チェック（186-189行目）は `--ids-field` 用に維持
+    （"--ids-field is currently only supported by the mongodb adapter" のまま）。
+
+  実装方針: 畳み込み前にどちらのフラグが使われたかが分かる状態で
+  「target-table 必須」「アダプタ整合」を検証してから `@ids_field` に集約する。
+
+### 2. `lib/exwiw/query_ast_builder.rb` — WHERE 句に反映
+
+**当初の想定（1行変更）は関連テーブルで不正確になることが判明**したため、設計を変更した。
+SQL アダプタは単一クエリで `dump_target.ids` を外部キーに直接伝播する
+（`orders.user_id IN ids`）。これは `--ids` が主キーである前提のため、`--ids-column`
+で別カラムを指定すると関連テーブルが壊れる（mongodb は @state に主キーを溜めて伝播する
+ので正しい）。
+
+採用したアプローチ: **ターゲットを介すサブクエリ**。`ids_field` 指定時、外部キー制約を
+`fk IN (SELECT pk FROM target WHERE ids_field IN (ids))` に置き換える。direct /
+indirect / polymorphic を一律に正しく扱える。
+
+- `lib/exwiw/query_ast.rb`: `Subquery` 構造体を追加。`WhereClause` に
+  operator `:in_subquery`（value が `Subquery`）を導入し `to_h` を対応。
+- `lib/exwiw/query_ast_builder.rb`: `dump_target_fk_clause(foreign_key)` ヘルパーを新設。
+  `ids_field` 無しなら従来通り `eq`、有りなら `:in_subquery` を返す。
+  - `build_where_clauses`（direct belongs_to）と `build_join_clauses`
+    （indirect の `relation_to_dump_target` hop）の両方で利用。
+  - ターゲットテーブル自身のフィルタは `ids_field || primary_key` の `eq`（従来どおり）。
+- 各 SQL アダプタ（postgresql / mysql2 / sqlite3）の `compile_where_condition` に
+  `:in_subquery` 分岐と `compile_subquery` を追加。`is_a?(WhereClause)` のままなので
+  bulk_delete のサブクエリ生成・JoinClause.to_h もそのまま動く。
+
+補足: `--ids-column` がマスク対象カラムの場合、`delete-*` の冪等性が崩れる
+（README に注記済み）。
+
+## 検証
+
+- `bundle exec rspec spec/cli_spec.rb` — 既存の `--ids-field` validation を維持しつつ、
+  新規ケースを追加:
+  - `--ids-column` が `@ids_field`（畳み込み後）にパースされる
+  - `--ids-column` を mongodb で指定すると拒否される
+  - `--ids-field` と `--ids-column` の同時指定が拒否される
+  - `--ids-column` を target-table 無しで指定すると拒否される
+- `bundle exec rspec spec/query_ast_builder_spec.rb`（存在すれば）に
+  `ids_field` 指定時に対象テーブルの WHERE が主キーではなく当該カラムになることを確認する
+  ケースを追加。
+- `explain` サブコマンド（SQL のみ対応）で end-to-end 確認:
+  既存 scenario（例 `scenario/sqlite3-schema`）に対し
+  `--target-table=... --ids=... --ids-column=<col>` を渡し、出力 SQL の WHERE が
+  `<table>.<col> IN (...)` になることを目視確認。
+- `bundle exec rspec`（全体）でリグレッションが無いこと。
+
+## ドキュメント
+
+`README.md:415` の `--ids-field` 説明を更新し、SQL では `--ids-column` を使う旨と例
+（例: `--target-table=users --ids=a@example.com --ids-column=email`）を追記。
+「SQL adapters reject it / TODO」の記述を解消する。

data/lib/exwiw/adapter/mysql2_adapter.rb

@@ -201,11 +201,20 @@ module Exwiw
           else
             "#{key} IN (#{values.join(', ')})"
           end
+        elsif where_clause.operator == :in_subquery
+          "#{key} IN (#{compile_subquery(where_clause.value)})"
         else
           raise "Unsupported operator: #{where_clause.operator}"
         end
       end
 
+      private def compile_subquery(subquery)
+        inner_values = subquery.where_values.map { |v| escape_value(v) }
+        "SELECT #{subquery.table_name}.#{subquery.select_column} " \
+          "FROM #{subquery.table_name} " \
+          "WHERE #{subquery.table_name}.#{subquery.where_column} IN (#{inner_values.join(', ')})"
+      end
+
       private def escape_value(value)
         case value
         when nil

data/lib/exwiw/adapter/postgresql_adapter.rb

@@ -243,11 +243,20 @@ module Exwiw
           else
             "#{key} IN (#{values.join(', ')})"
           end
+        elsif where_clause.operator == :in_subquery
+          "#{key} IN (#{compile_subquery(where_clause.value)})"
         else
           raise "Unsupported operator: #{where_clause.operator}"
         end
       end
 
+      private def compile_subquery(subquery)
+        inner_values = subquery.where_values.map { |v| escape_value(v) }
+        "SELECT #{subquery.table_name}.#{subquery.select_column} " \
+          "FROM #{subquery.table_name} " \
+          "WHERE #{subquery.table_name}.#{subquery.where_column} IN (#{inner_values.join(', ')})"
+      end
+
       private def escape_value(value)
         case value
         when nil

data/lib/exwiw/adapter/sqlite3_adapter.rb

@@ -188,11 +188,20 @@ module Exwiw
           else
             "#{key} IN (#{values.join(', ')})"
           end
+        elsif where_clause.operator == :in_subquery
+          "#{key} IN (#{compile_subquery(where_clause.value)})"
         else
           raise "Unsupported operator: #{where_clause.operator}"
         end
       end
 
+      private def compile_subquery(subquery)
+        inner_values = subquery.where_values.map { |v| escape_value(v) }
+        "SELECT #{subquery.table_name}.#{subquery.select_column} " \
+          "FROM #{subquery.table_name} " \
+          "WHERE #{subquery.table_name}.#{subquery.where_column} IN (#{inner_values.join(', ')})"
+      end
+
       private def escape_value(value)
         case value
         when nil

data/lib/exwiw/cli.rb

@@ -40,6 +40,7 @@ module Exwiw
       @target_collection_name = nil
       @ids = []
       @ids_field = nil
+      @ids_column = nil
       @output_format = nil
       @insert_only = nil
       @after_insert_hook_path = nil
@@ -99,6 +100,7 @@ module Exwiw
 
     private def validate_options!
       resolve_target_collection_alias!
+      resolve_ids_column_alias!
 
       if @subcommand == "explain"
         validate_explain_only!
@@ -172,23 +174,6 @@ module Exwiw
         exit 1
       end
 
-      if @ids_field
-        # --ids-field overrides the field --ids filters against on the target
-        # table; it is meaningless without a target table to constrain.
-        if !@target_table_name
-          $stderr.puts "--target-table is required when --ids-field is specified"
-          exit 1
-        end
-
-        # TODO: support --ids-field for the sql adapters (mysql2/postgresql/
-        # sqlite3) by threading dump_target.ids_field through QueryAstBuilder's
-        # WHERE clause on the target table. For now it is mongodb-only.
-        if @database_adapter != "mongodb"
-          $stderr.puts "--ids-field is currently only supported by the mongodb adapter"
-          exit 1
-        end
-      end
-
       if @after_insert_hook_path
         unless File.file?(@after_insert_hook_path)
           $stderr.puts "--after-insert-hook file not found: #{@after_insert_hook_path}"
@@ -223,6 +208,43 @@ module Exwiw
       @target_table_name = @target_collection_name
     end
 
+    # `--ids-column` is the sql-adapter spelling of `--ids-field` (the mongodb
+    # spelling). Both override which column/field `--ids` is matched against on
+    # the target table; internally they fold into the single @ids_field carried
+    # by DumpTarget. Mirror the --target-table/--target-collection split: each
+    # flag is restricted to its adapter family and the two are mutually
+    # exclusive. Runs after resolve_target_collection_alias! so
+    # @target_table_name already reflects the collection alias.
+    private def resolve_ids_column_alias!
+      if @ids_field && @ids_column
+        $stderr.puts "Specify only one of --ids-field and --ids-column"
+        exit 1
+      end
+
+      if @ids_field && @database_adapter != "mongodb"
+        $stderr.puts "--ids-field is only supported by the mongodb adapter (use --ids-column)"
+        exit 1
+      end
+
+      if @ids_column
+        sql_adapters = ["mysql2", "postgresql", "sqlite3"]
+        unless sql_adapters.include?(@database_adapter)
+          $stderr.puts "--ids-column is only supported by the sql adapters (use --ids-field)"
+          exit 1
+        end
+
+        @ids_field = @ids_column
+      end
+
+      # --ids-field/--ids-column override the column --ids filters against on
+      # the target table; meaningless without a target table to constrain.
+      if @ids_field && !@target_table_name
+        flag = @ids_column ? "--ids-column" : "--ids-field"
+        $stderr.puts "--target-table is required when #{flag} is specified"
+        exit 1
+      end
+    end
+
     private def validate_explain_only!
       if @database_adapter == "mongodb"
         $stderr.puts "mongodb adapter is not yet supported by 'explain' subcommand"
@@ -306,7 +328,8 @@ module Exwiw
         opts.on("--target-table=[TABLE]", "Target table for extraction. If omitted, dump all tables.") { |v| @target_table_name = v }
         opts.on("--target-collection=[COLLECTION]", "Alias of --target-table for the mongodb adapter.") { |v| @target_collection_name = v }
         opts.on("--ids=[IDS]", "Comma-separated list of identifiers. Required when --target-table is given.") { |v| @ids = v.split(',') }
-        opts.on("--ids-field=[FIELD]", "Field on the target table that --ids is matched against. Defaults to the primary key. (mongodb adapter only)") { |v| @ids_field = v }
+        opts.on("--ids-field=[FIELD]", "Field on the target collection that --ids is matched against. Defaults to the primary key. (mongodb adapter only)") { |v| @ids_field = v }
+        opts.on("--ids-column=[COLUMN]", "Column on the target table that --ids is matched against. Defaults to the primary key. (sql adapters only)") { |v| @ids_column = v }
         opts.on("--output-format=[FORMAT]", "Output format: insert (default) or copy (PostgreSQL only, dump subcommand only)") { |v| @output_format = v }
         opts.on("--insert-only", "Do not generate DELETE SQL files (dump subcommand only)") { @insert_only = true }
         opts.on("--after-insert-hook=PATH", "Path to a .rb or .sh post-processing hook executed after all insert/delete files are written (dump subcommand only)") do |v|

data/lib/exwiw/query_ast.rb

@@ -41,7 +41,26 @@ module Exwiw
         {
           column_name: column_name,
           operator: operator,
-          value: value,
+          value: value.is_a?(Subquery) ? value.to_h : value,
+        }
+      end
+    end
+
+    # Resolves a set of values on `where_column` to the rows' `select_column`
+    # via a nested SELECT. Used as the `value` of a WhereClause whose operator
+    # is `:in_subquery`, so `--ids-column`/`--ids-field` can filter related
+    # tables through the target table's primary key:
+    #
+    #   <table>.<fk> IN (SELECT <table_name>.<select_column>
+    #                    FROM <table_name>
+    #                    WHERE <table_name>.<where_column> IN (<where_values>))
+    Subquery = Struct.new(:table_name, :select_column, :where_column, :where_values, keyword_init: true) do
+      def to_h
+        {
+          table_name: table_name,
+          select_column: select_column,
+          where_column: where_column,
+          where_values: where_values,
         }
       end
     end

data/lib/exwiw/query_ast_builder.rb

@@ -73,11 +73,7 @@ module Exwiw
         end
         relation_to_dump_target = to_table.belongs_to(dump_target.table_name)
         if relation_to_dump_target
-          join_clause.where_clauses.push QueryAst::WhereClause.new(
-            column_name: relation_to_dump_target.foreign_key,
-            operator: :eq,
-            value: dump_target.ids
-          )
+          join_clause.where_clauses.push dump_target_fk_clause(relation_to_dump_target.foreign_key)
 
           # 中間テーブルが dump target へ polymorphic belongs_to している場合は、
           # 型カラム (foreign_type) も join 条件に追加する。型カラムは to_table
@@ -107,12 +103,12 @@ module Exwiw
       clauses = []
 
       if table.name == dump_target.table_name
-        # TODO: honor dump_target.ids_field here so `--ids` can match a non
-        # primary-key column on the target table (currently mongodb-only; the
-        # CLI rejects --ids-field for the sql adapters). When implemented, use
-        # `dump_target.ids_field || table.primary_key` as the column_name.
+        # `--ids-column` (folded into dump_target.ids_field by the CLI) lets
+        # `--ids` match a non primary-key column on the target table; otherwise
+        # fall back to the primary key. Only the target table's filter changes —
+        # downstream foreign-key propagation still keys off the primary key.
         clauses.push Exwiw::QueryAst::WhereClause.new(
-          column_name: table.primary_key,
+          column_name: dump_target.ids_field || table.primary_key,
           operator: :eq,
           value: dump_target.ids
         )
@@ -123,11 +119,7 @@ module Exwiw
       belongs_to = table.belongs_to(dump_target.table_name)
       return clauses if belongs_to.nil?
 
-      clauses.push Exwiw::QueryAst::WhereClause.new(
-        column_name: belongs_to.foreign_key,
-        operator: :eq,
-        value: dump_target.ids
-      )
+      clauses.push dump_target_fk_clause(belongs_to.foreign_key)
 
       # polymorphic belongs_to の場合は外部キーだけでは型を区別できないため
       # (例: reviewable_id=1 が Product なのか別モデルなのか判別できない)、
@@ -147,6 +139,36 @@ module Exwiw
       clauses
     end
 
+    # Builds the WHERE clause that constrains a `foreign_key` pointing at the
+    # dump target. Normally `--ids` are the target's primary keys, so a plain
+    # `foreign_key IN (ids)` suffices. When `--ids-column`/`--ids-field` is set
+    # (dump_target.ids_field), `--ids` match a non primary-key column instead,
+    # so the foreign key must be resolved through the target table:
+    # `foreign_key IN (SELECT pk FROM target WHERE ids_field IN (ids))`.
+    # This keeps related-table extraction correct regardless of whether the
+    # relation is direct, indirect, or polymorphic.
+    private def dump_target_fk_clause(foreign_key)
+      unless dump_target.ids_field
+        return Exwiw::QueryAst::WhereClause.new(
+          column_name: foreign_key,
+          operator: :eq,
+          value: dump_target.ids
+        )
+      end
+
+      target = table_by_name.fetch(dump_target.table_name)
+      Exwiw::QueryAst::WhereClause.new(
+        column_name: foreign_key,
+        operator: :in_subquery,
+        value: Exwiw::QueryAst::Subquery.new(
+          table_name: target.name,
+          select_column: target.primary_key,
+          where_column: dump_target.ids_field,
+          where_values: dump_target.ids
+        )
+      )
+    end
+
     private def find_path_to_dump_target(table, table_by_name, dump_target)
       return [] if table.name == dump_target.table_name
 

data/lib/exwiw/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Exwiw
-  VERSION = "0.2.9"
+  VERSION = "0.3.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: exwiw
 version: !ruby/object:Gem::Version
-  version: 0.2.9
+  version: 0.3.0
 platform: ruby
 authors:
 - Shia
@@ -40,6 +40,7 @@ files:
 - docs/plans/2026-05-22-after-insert-hook.md
 - docs/plans/2026-05-22-postgres-copy-mode-scenario-test.md
 - docs/plans/2026-05-29-rails-managed-tables.md
+- docs/plans/2026-05-31-ids-column-for-sql-adapters.md
 - exe/exwiw
 - lib/exwiw.rb
 - lib/exwiw/adapter.rb