exwiw 1.1.5 → 1.1.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2df915162dc88ac29ccc262c436d5d7ab1797a7e026398ced3e66d1e5b73a7b7
-  data.tar.gz: 3144680abf12865a3db9d4173e4338101fa318da9f5e61e3367c5924fb24f9c6
+  metadata.gz: 345a459c251b837b718e00ed92b011c8975016af2a92149868fcd24744356041
+  data.tar.gz: c79d6c2525ae322bab05956102c424eecc6fa40864e0bc7d581912a68d20928e
 SHA512:
-  metadata.gz: 2726b1b47c6584d6a85ab6f1342fdfb8ac760d237c09854f81a34e36d403a3720efbe25ee47185d3f381b4384e6a3f75f47ff6002fcbaa09d4613f937b39cc1f
-  data.tar.gz: 75bb6c28ecd32adaa2aeb3488d319784811ad4ab08104560dea54982e94de51c7a7ff606b81815f6d73a41608f5f6dae49c57774a35ad837a8b6b5dab994c45d
+  metadata.gz: 881bc0c87eb1121c9670075e059eafd77a3c551f444442575b4545a5c9b78387dd90e2726c0745618b7486e4b1f907d5cb49ce414aefd772c7fd10cd8c80060d
+  data.tar.gz: c6a2ee871b7f8ff89e1ddd048c4494c71b7b8776e9d26a1d677b24b11614d1ff4a82c418e5169935514d17404ca18d30626dcfdf105469f6a01e5b0d5664782e

data/CHANGELOG.md

@@ -2,6 +2,18 @@
 
 ## [Unreleased]
 
+## [1.1.6] - 2026-09-29
+
+### Added
+
+- **`--target-table` can be omitted in scope-column mode.** Scope-column mode starts from every table that declares a `scope_column`, so the target only had to name one of them and was otherwise unused. `--ids` alone now runs in scope-column mode when any table in the schema declares a `scope_column` (tables marked `ignore: true` do not count), for both `export` and `explain`. When none does, the run stops before any output with an error saying to either name the table the ids belong to with `--target-table` (single-target mode) or declare a `scope_column` (scope-column mode). This check moved from the CLI to the schema pre-flight (`QueryAstBuilder.validate_scope!`), since it needs the schema. Runs that pass `--target-table`, and the deprecated `--scope-column` flag, behave as before. The mongodb adapter has no scope-column mode, so `--ids` there still requires `--target-collection` and is rejected by the CLI without it.
+- **Scope-column mode can filter unrelated groups of tables by different kinds of id in one run.** A table can declare `id_space` next to its `scope_column` to say which ID space the column's values belong to, and `--ids` takes one value list per ID space: `--ids=1,2 --ids=org=<uuid>,<uuid>`. Tables without `id_space`, and `--ids` without a prefix, are the `default` ID space, so existing schemas, commands and configs keep their meaning. A table without a scope column is filtered by the ID space of the table it is scoped through. The config file's `ids:` also accepts a mapping (`ids: { default: [1, 2], org: [...] }`), and a shell after-insert hook receives each space given values as `EXWIW_IDS_<NAME>` (`EXWIW_IDS` stays the `default` space and is always set; Ruby hooks get `ids_for("<name>")`). The run aborts before extracting anything when a table's ID space was given no values, when a given ID space is used by no table, or when a table reaches scoped tables of more than one ID space, which the `belongs_to` walk would otherwise settle silently by picking the nearest. Single `--target-table` mode and the MongoDB adapter use only the `default` space and abort when a named one is given. See [Per-table `scope_column` and ID spaces](README.md#per-table-scope_column-and-id-spaces).
+
+### Changed
+
+- **`--ids` is read as `<id_space>=<ids>` when the part before its first `=` looks like an ID space name** (a lowercase letter followed by lowercase letters, digits or underscores). An id list whose first id contains such a prefix, like `--ids=key=1`, now names the ID space `key`; pass `--ids=default=key=1` to keep it as an id. Giving `--ids` more than once for the same ID space is now an error instead of the last one silently winning.
+- **`Exwiw::DumpTarget#ids` is a Hash from ID space name to values.** An Array passed to `DumpTarget.new(ids:)` is still accepted as the `default` space's values; read them with `#default_ids`.
+
 ## [1.1.5] - 2026-09-25
 
 ### Fixed

data/README.md

@@ -80,7 +80,13 @@ exwiw \
   --log-level=info
 ```
 
-By default `--ids` are matched against the target table's primary key. If the target table declares a per-table `scope_column`, exwiw runs in [scope-column mode](#scope-column-mode) instead — `--ids` are then values of that shared column, and the table is scoped like any other rather than anchored by primary key.
+By default `--ids` are matched against the target table's primary key. If the target table declares a per-table `scope_column`, exwiw runs in [scope-column mode](#scope-column-mode) instead — `--ids` are then values of that shared column, and the table is scoped like any other rather than anchored by primary key. In scope-column mode `--target-table` can be omitted: passing only `--ids` runs in scope-column mode as long as some table in the schema declares a `scope_column`.
+
+| `--target-table` | `--ids` | Mode |
+|---|---|---|
+| given | given | Scope-column mode if the table declares a `scope_column`, single-target mode otherwise |
+| omitted | given | Scope-column mode if any table declares a `scope_column`, an error otherwise (SQL adapters only) |
+| omitted | omitted | Every table is dumped in full |
 
 When `--target-table` and `--ids` are omitted, exwiw dumps all tables defined in `--schema-dir`:
 
@@ -173,7 +179,8 @@ all. Choosing one table as `--target-table` would leave the others unrelated to
 and an unrelated table is dumped in full — a problem if it holds personal data.
 
 Scope-column mode handles this shape: instead of anchoring on one table's primary
-key, **every table is filtered by a shared column** whose values are `--ids`.
+key, **every table is filtered by a shared column** whose values are `--ids` (tables
+keyed by an unrelated kind of id can use their own [ID space](#per-table-scope_column-and-id-spaces)).
 Declare that column per table in the schema config with `scope_column:`:
 
 ```json
@@ -185,7 +192,7 @@ Declare that column per table in the schema config with `scope_column:`:
 }
 ```
 
-Then name any scoped table as `--target-table` and pass the scope values as `--ids`:
+Then pass the scope values as `--ids`, without `--target-table`:
 
 ```bash
 exwiw \
@@ -193,15 +200,22 @@ exwiw \
   --host=localhost --port=5432 --user=reader \
   --database=app_production \
   --schema-dir=exwiw/schema \
-  --target-table=shops --ids=42,43 \
+  --ids=42,43 \
   --output-dir=dump
 ```
 
-Because `shops` declares a `scope_column`, exwiw switches to scope-column mode: the
+Because the schema declares a `scope_column`, exwiw runs in scope-column mode: the
 `--ids` (`42,43`) are **`business_entity_id` values, not shop primary keys**, and
-`shops` itself is scoped by `business_entity_id IN (42,43)` like every other scoped
-table — it is *not* used as a primary-key anchor. (A table that declares a
-`scope_column` therefore can no longer be single-extracted by primary key.)
+`shops` is scoped by `business_entity_id IN (42,43)` like every other scoped table.
+Where extraction starts is decided by the schema, so `--target-table` is not
+needed. If no table declares a `scope_column`, `--ids` alone is an error: name the
+table the ids belong to with `--target-table` (single-target mode), or declare a
+`scope_column`. Tables marked `ignore: true` do not count.
+
+Naming a scoped table as `--target-table` (`--target-table=shops --ids=42,43`)
+selects the same mode and extracts the same rows; the target is *not* used as a
+primary-key anchor. (A table that declares a `scope_column` therefore can no longer
+be single-extracted by primary key.)
 
 Each table is resolved as follows:
 
@@ -236,7 +250,8 @@ Each table is resolved as follows:
 > the broader hub cascade, set `ignore: true` on the child's `belongs_to` edge that
 > points at this table.
 
-Scope-column mode is SQL-only (mysql / postgresql / sqlite). It works with `exwiw
+Scope-column mode is SQL-only (mysql / postgresql / sqlite); with the mongodb
+adapter, `--ids` still requires `--target-collection`. It works with `exwiw
 explain` too, which is the recommended way to preview the queries before exporting.
 
 #### Cross-database foreign keys
@@ -266,11 +281,13 @@ opt out of the strict check and be exported in full:
 Rails-managed tables (`schema_migrations`, `ar_internal_metadata`) are treated as
 exempt automatically.
 
-#### Per-table `scope_column` and the value space
+#### Per-table `scope_column` and ID spaces
 
-Scope-column mode assumes a single shared **value** space — the same `--ids` apply
-to every scoped table. Each table names its own column, so a table that stores that
-same value under a differently named column simply declares that name:
+The values of every `scope_column` belong to an **ID space**, and `--ids` gives the
+values of each space. A table that does not declare `id_space` is in the `default`
+space, which a plain `--ids=1,2` fills, so without any `id_space` every scoped table
+is filtered by the same `--ids`. Each table names its own column, so a table that
+stores that same value under a differently named column simply declares that name:
 
 ```json
 {
@@ -281,8 +298,42 @@ same value under a differently named column simply declares that name:
 }
 ```
 
-Both `scope_exempt` and `scope_column` are user-maintained and preserved across
-`schema:generate` regeneration (the generators never emit them).
+When one database holds two groups of tables that no foreign key connects, each
+keyed by a different kind of id, put one group in a named ID space. Here `tenants`
+are identified by integer ids and `organizations` by UUIDs (each object is its own
+schema file):
+
+```json
+{ "name": "tenants", "primary_key": "id", "scope_column": "id", "columns": [{ "name": "id" }] }
+{ "name": "organizations", "primary_key": "id", "scope_column": "id", "id_space": "org", "columns": [{ "name": "id" }] }
+```
+
+```bash
+exwiw ... --target-table=tenants \
+  --ids=1,2 \
+  --ids=org=0b6f4c1e-0000-4000-8000-000000000001
+```
+
+`--ids=1,2` is the same as `--ids=default=1,2`. The part before the first `=` is
+read as an ID space name only when it is shaped like one (a lowercase letter
+followed by lowercase letters, digits or underscores), so an id that contains `=`
+itself is passed with an explicit `default=`. Each ID space may be given once. A
+table without a scope column is filtered by the ID space of the table it is scoped
+through (a `belongs_to` join, `reverse_scope`, referenced-by or the parent cascade).
+
+The run aborts before extracting anything when a table is filtered by an ID space
+that was given no values, when values are given for an ID space no table uses, or
+when a table reaches scoped tables of more than one ID space (otherwise the
+`belongs_to` walk would silently settle on the nearest one).
+
+Single `--target-table` mode and the MongoDB adapter use only the `default` space
+and abort when a named one is given. In the config file, `ids:` takes either a list
+(the `default` space) or a mapping from ID space name to values, such as
+`ids: { default: [1, 2], org: [0b6f4c1e-0000-4000-8000-000000000001] }`; as with
+the other keys, it is used only when `--ids` is not passed at all.
+
+`scope_exempt`, `scope_column` and `id_space` are user-maintained and preserved
+across `schema:generate` regeneration (the generators never emit them).
 
 #### Deprecated: the `--scope-column` flag
 
@@ -290,7 +341,7 @@ Before per-table declarations, scope-column mode was selected with a global
 `--scope-column=COLUMN` flag (every table filtered by that one column, `--ids` its
 values, no `--target-table`). The flag still works — SQL-only and mutually
 exclusive with `--target-table` — but is **deprecated** and emits a warning; prefer
-declaring a per-table `scope_column` and running with `--target-table`. A per-table
+declaring a per-table `scope_column` and dropping the flag. A per-table
 `scope_column` takes precedence over the flag for any table that sets both.
 
 ### Config file (`exwiw.yml`)
@@ -308,6 +359,7 @@ output_format: insert        # insert | copy
 after_insert_hook: hooks/seed.rb
 log_level: info              # debug | info
 # target_table / ids / ids_field / scope_column may also be set here
+# (ids may map ID spaces to values; see "Per-table scope_column and ID spaces")
 # mongodb_query_timeout_ms: 30000   # global query timeout (mongodb only)
 ```
 
@@ -637,7 +689,8 @@ psql -d app_dev -f dump/insert-001-shops.sql
 
 **Ruby hook (`.rb`)**: provides a tiny DSL with these builtins:
 
-- `cli_options` — Hash of all parsed CLI options (e.g. `cli_options.fetch(:ids)` returns the `--ids` array).
+- `cli_options` — Hash of all parsed CLI options (e.g. `cli_options.fetch(:ids)` returns the `--ids` array of the `default` ID space)
+- `ids_for(id_space = "default")` — the values the run was scoped by for that ID space (e.g. `ids_for("org")` for `--ids=org=...`)
 - `insert_sql(template)` — appends an ERB-rendered string to a buffer. After the hook finishes, the buffer is concatenated and written to `insert-{N+1}-after_insert.{ext}` where `{N+1}` is one past the last per-table insert file. For the MongoDB adapter the equivalent alias `insert_jsonl(template)` is available; output goes to `insert-{N+1}-after_insert.jsonl`. Multiple `insert_sql` calls in a single hook are joined with `"\n"` into the same file. If no `insert_sql` call is made, no file is created.
 - `insert_jsonl(collection, template)` — **MongoDB adapter only**. SQL statements name their table in-band, but JSONL documents do not — the import convention derives the target collection from the filename — so the two-argument form writes the ERB-rendered extended-JSON lines to the named collection's own `insert-NNN-<collection>.jsonl` file, importable with the same `mongoimport --collection <collection>` convention as the per-collection dump files. Multiple calls targeting the same collection are appended (joined with `"\n"`) into that collection's file; distinct collections get one file each, numbered sequentially after the last per-collection dump file (the collection-less `after_insert` buffer, when also used, keeps `{N+1}` and the collection files follow it). Calling this form with a SQL adapter raises an error.
 
@@ -667,7 +720,8 @@ insert_jsonl 'posts', '{"title":"welcome"}'
 
 - `EXWIW_OUTPUT_DIR`, `EXWIW_SCHEMA_DIR`
 - `EXWIW_DATABASE_ADAPTER`, `EXWIW_DATABASE_HOST`, `EXWIW_DATABASE_PORT`, `EXWIW_DATABASE_USER`, `EXWIW_DATABASE_NAME`
-- `EXWIW_TARGET_TABLE`, `EXWIW_IDS` (comma-separated), `EXWIW_OUTPUT_FORMAT`
+- `EXWIW_TARGET_TABLE`, `EXWIW_IDS` (comma-separated, the `default` ID space), `EXWIW_OUTPUT_FORMAT`
+- `EXWIW_IDS_<ID_SPACE>` for each ID space given values, with the name uppercased (`--ids=org=...` becomes `EXWIW_IDS_ORG`). `EXWIW_IDS_DEFAULT` is set only when the `default` space is given, while `EXWIW_IDS` is always set
 
 A non-zero exit code from the shell hook aborts exwiw.
 

data/lib/exwiw/adapter/mongodb_adapter.rb

@@ -164,9 +164,9 @@ module Exwiw
             # left as Strings rather than guessed at — the caller passes values
             # matching the field's actual type.
             if dump_target.ids_field
-              { dump_target.ids_field => { "$in" => dump_target.ids } }
+              { dump_target.ids_field => { "$in" => dump_target.default_ids } }
             else
-              { config.primary_key => { "$in" => coerce_ids(dump_target.ids) } }
+              { config.primary_key => { "$in" => coerce_ids(dump_target.default_ids) } }
             end
           else
             related_collection_filter(config, config_by_name, dump_target)

data/lib/exwiw/after_insert_hook.rb

@@ -4,25 +4,26 @@ require 'erb'
 
 module Exwiw
   class AfterInsertHook
-    def self.run(path:, cli_options:, output_dir:, next_idx:, output_extension:, logger:)
+    def self.run(path:, cli_options:, dump_target:, output_dir:, next_idx:, output_extension:, logger:)
       ext = File.extname(path)
 
       if ext == '.rb'
         run_ruby(
           path: path,
           cli_options: cli_options,
+          dump_target: dump_target,
           output_dir: output_dir,
           next_idx: next_idx,
           output_extension: output_extension,
           logger: logger,
         )
       else
-        run_shell(path: path, cli_options: cli_options, output_dir: output_dir, logger: logger)
+        run_shell(path: path, cli_options: cli_options, dump_target: dump_target, output_dir: output_dir, logger: logger)
       end
     end
 
-    def self.run_ruby(path:, cli_options:, output_dir:, next_idx:, output_extension:, logger:)
-      ctx = Context.new(cli_options, output_extension: output_extension)
+    def self.run_ruby(path:, cli_options:, dump_target:, output_dir:, next_idx:, output_extension:, logger:)
+      ctx = Context.new(cli_options, dump_target: dump_target, output_extension: output_extension)
       ctx.instance_eval(File.read(path), path)
 
       # One output file per buffer, numbered sequentially from next_idx: first
@@ -53,7 +54,7 @@ module Exwiw
       end
     end
 
-    def self.run_shell(path:, cli_options:, output_dir:, logger:)
+    def self.run_shell(path:, cli_options:, dump_target:, output_dir:, logger:)
       env = {
         'EXWIW_OUTPUT_DIR'       => output_dir,
         'EXWIW_SCHEMA_DIR'       => cli_options[:schema_dir].to_s,
@@ -64,9 +65,12 @@ module Exwiw
         'EXWIW_DATABASE_NAME'    => cli_options[:database_name].to_s,
         'EXWIW_TARGET_TABLE'     => cli_options[:target_table].to_s,
         'EXWIW_SCOPE_COLUMN'     => cli_options[:scope_column].to_s,
-        'EXWIW_IDS'              => Array(cli_options[:ids]).join(','),
+        'EXWIW_IDS'              => dump_target.default_ids.join(','),
         'EXWIW_OUTPUT_FORMAT'    => cli_options[:output_format].to_s,
       }
+      dump_target.ids.each do |space, values|
+        env["EXWIW_IDS_#{space.upcase}"] = Array(values).join(',')
+      end
       logger.info("Running after-insert shell hook: #{path}")
       ok = system(env, path)
       raise "after-insert shell hook failed: #{path}" unless ok
@@ -79,13 +83,19 @@ module Exwiw
 
       attr_reader :cli_options, :collected, :collected_by_collection
 
-      def initialize(cli_options, output_extension: nil)
+      def initialize(cli_options, dump_target:, output_extension: nil)
         @cli_options = cli_options
+        @dump_target = dump_target
         @output_extension = output_extension
         @collected = []
         @collected_by_collection = {}
       end
 
+      # The values the run was scoped by for the given ID space (`default` when omitted).
+      def ids_for(id_space = DEFAULT_ID_SPACE)
+        @dump_target.ids_for(id_space.to_s)
+      end
+
       def insert_sql(template)
         @collected << render(template)
       end

data/lib/exwiw/cli.rb

@@ -111,7 +111,8 @@ module Exwiw
       @database_name = nil
       @target_table_name = nil
       @target_collection_name = nil
-      @ids = []
+      @ids_by_id_space = {}
+      @ids_errors = []
       @ids_field = nil
       @scope_column = nil
       @output_format = nil
@@ -151,7 +152,7 @@ module Exwiw
 
       dump_target = DumpTarget.new(
         table_name: @target_table_name,
-        ids: @ids,
+        ids: @ids_by_id_space,
         ids_field: @ids_field,
         scope_column: @scope_column,
       )
@@ -297,6 +298,9 @@ module Exwiw
       # Fill in any options not given on the CLI from the config file. Done first
       # so a config-provided `adapter` is in place before normalization below.
       # CLI values always win (the merge only fills nil/empty ivars).
+      @ids_errors.each { |message| $stderr.puts message }
+      exit 1 if @ids_errors.any?
+
       apply_config_file!
 
       # Default log level once CLI and config have both had their say.
@@ -316,6 +320,7 @@ module Exwiw
       end
 
       resolve_target_collection_alias!
+      resolve_named_ids!
       resolve_scope_column!
       resolve_ids_field!
       resolve_uri_option!
@@ -391,18 +396,22 @@ module Exwiw
         exit 1
       end
 
-      if @target_table_name && @ids.empty?
+      if @target_table_name && !ids_given?
         $stderr.puts "--ids is required when --target-table is specified"
         exit 1
       end
 
-      if @scope_column && @ids.empty?
+      if @scope_column && !ids_given?
         $stderr.puts "--ids is required when --scope-column is specified"
         exit 1
       end
 
-      if !@target_table_name && !@scope_column && @ids.any?
-        $stderr.puts "--target-table or --scope-column is required when --ids is specified"
+      # For the SQL adapters, --ids without a target is scope-column mode; whether
+      # the schema supports it is checked once it is loaded
+      # (QueryAstBuilder.validate_scope!). MongoDB has no scope-column mode.
+      if @database_adapter == "mongodb" && !@target_table_name && default_ids.any?
+        $stderr.puts "--target-collection (or --target-table) is required when --ids is specified " \
+                     "with the mongodb adapter"
         exit 1
       end
 
@@ -479,11 +488,23 @@ module Exwiw
       @log_level ||= config["log_level"]&.to_sym
       @target_table_name ||= config["target_table"]
       @target_collection_name ||= config["target_collection"]
-      if @ids.empty? && config.key?("ids")
+      # Like the other keys, `ids` is taken from the config only when the CLI
+      # passed no --ids at all; the two are never merged per ID space.
+      if !ids_given? && config.key?("ids")
         raw = config["ids"]
-        # Accept either a YAML list or a "1,2" string; coerce to strings to match
-        # the CLI's `--ids=1,2` -> ["1", "2"] shape.
-        @ids = (raw.is_a?(String) ? raw.split(",") : Array(raw)).map(&:to_s)
+        if raw.is_a?(Hash)
+          raw.each do |space, values|
+            space = space.to_s
+            unless space.match?(ID_SPACE_NAME_PATTERN)
+              $stderr.puts "config 'ids' keys must be ID space names (a lowercase letter followed by " \
+                           "lowercase letters, digits or underscores; got #{space.inspect}) in #{path}"
+              exit 1
+            end
+            @ids_by_id_space[space] = parse_config_ids(values)
+          end
+        else
+          @ids_by_id_space[DEFAULT_ID_SPACE] = parse_config_ids(raw)
+        end
       end
       @ids_field ||= config["ids_field"]
       @scope_column ||= config["scope_column"]
@@ -492,6 +513,56 @@ module Exwiw
       @explain_verbosity ||= config["explain_verbosity"]
     end
 
+    # Accept either a YAML list or a "1,2" string; coerce to strings to match
+    # the CLI's `--ids=1,2` -> ["1", "2"] shape.
+    private def parse_config_ids(raw)
+      (raw.is_a?(String) ? raw.split(",") : Array(raw)).map(&:to_s)
+    end
+
+    # `--ids=<id_space>=1,2` gives the values of a named ID space; without the
+    # prefix the values are the default space's. The prefix is recognized only
+    # when it is shaped like an ID space name, so an id containing `=` is passed
+    # as `--ids=default=<ids>`. Each ID space may be given once: a second --ids
+    # for it would otherwise replace the first without notice.
+    private def add_ids_option(value)
+      prefix, rest = value.to_s.split("=", 2)
+      space, list =
+        if rest && prefix.match?(ID_SPACE_NAME_PATTERN)
+          [prefix, rest]
+        else
+          [DEFAULT_ID_SPACE, value.to_s]
+        end
+
+      if @ids_by_id_space.key?(space)
+        @ids_errors << "--ids was given more than once for the ID space '#{space}'; " \
+                       "pass all of its ids in one comma-separated --ids."
+      end
+      @ids_by_id_space[space] = list.split(",")
+    end
+
+    private def ids_given?
+      @ids_by_id_space.each_value.any?(&:any?)
+    end
+
+    private def default_ids
+      @ids_by_id_space.fetch(DEFAULT_ID_SPACE, [])
+    end
+
+    # Named ID spaces exist only for scope-column mode, which the mongodb
+    # adapter does not have. Whether a SQL run is in scope-column mode depends on
+    # the schema, so that is checked once it is loaded (QueryAstBuilder.validate_scope!).
+    private def resolve_named_ids!
+      named = @ids_by_id_space.keys - [DEFAULT_ID_SPACE]
+      return if named.empty?
+
+      if @database_adapter == "mongodb"
+        $stderr.puts "--ids for a named ID space (#{named.map { |space| "'#{space}'" }.join(', ')}) " \
+                     "is not supported by the mongodb adapter, which uses only the default ID space " \
+                     "(--ids without an `<id_space>=` prefix)."
+        exit 1
+      end
+    end
+
     # Strip a trailing slash (like the CLI's dir options) and expand relative to
     # `base` (the config file's directory). Returns nil for a nil value.
     private def expand_dir(value, base)
@@ -542,8 +613,8 @@ module Exwiw
 
     # `--scope-column` is **deprecated**: it selected scope-column mode with a
     # single global column for every table. The preferred way is to declare a
-    # per-table `scope_column:` in the schema config and pass `--target-table`
-    # (the target is then scoped like any other table). The flag still works as
+    # per-table `scope_column:` in the schema config (`--target-table` is then
+    # optional). The flag still works as
     # before — SQL-only and mutually exclusive with `--target-table` — but emits a
     # deprecation warning. Runs after resolve_target_collection_alias! so
     # --target-collection is already folded into @target_table_name.
@@ -567,7 +638,7 @@ module Exwiw
       end
 
       $stderr.puts "warning: --scope-column is deprecated; declare a per-table `scope_column:` " \
-                   "in the schema config and run with --target-table instead."
+                   "in the schema config and drop the flag instead."
     end
 
     # `--uri` supplies a full connection string (e.g. `mongodb+srv://...`) and is
@@ -780,7 +851,7 @@ module Exwiw
         database_adapter: @database_adapter,
         database_name: @database_name,
         target_table: @target_table_name,
-        ids: @ids.dup.freeze,
+        ids: default_ids.dup.freeze,
         ids_field: @ids_field,
         scope_column: @scope_column,
         output_format: @output_format,
@@ -849,11 +920,11 @@ module Exwiw
         opts.on("-a", "--adapter=ADAPTER", "Database adapter: mysql, sqlite, postgresql, mongodb (aliases: mysql2, sqlite3)") { |v| @database_adapter = v }
         opts.on("--uri=URI", "Full MongoDB connection URI (mongodb:// or mongodb+srv://). mongodb adapter only; takes precedence over --host/--port/--user. TLS, replicaSet, authSource and credentials are read from the URI.") { |v| @connection_uri = v }
         opts.on("--database=DATABASE", "Target database name") { |v| @database_name = v }
-        opts.on("--target-table=[TABLE]", "Target table for extraction. If omitted, dump all tables.") { |v| @target_table_name = v }
+        opts.on("--target-table=[TABLE]", "Target table for extraction. If omitted with --ids, run in scope-column mode (SQL adapters; the schema must declare `scope_column:`). If omitted without --ids, dump all tables.") { |v| @target_table_name = v }
         opts.on("--target-collection=[COLLECTION]", "Alias of --target-table for the mongodb adapter.") { |v| @target_collection_name = v }
-        opts.on("--ids=[IDS]", "Comma-separated list of identifiers. Required when --target-table is given.") { |v| @ids = v.split(',') }
+        opts.on("--ids=[IDS]", "Comma-separated list of identifiers. Required when --target-table is given. Prefix with `<id_space>=` (e.g. --ids=org=a,b) to give the values of a named ID space; may be repeated, once per ID space. An id containing `=` is passed as --ids=default=<ids>.") { |v| add_ids_option(v) }
         opts.on("--ids-field=[FIELD]", "Field on the target collection that --ids is matched against. Defaults to the primary key. (mongodb adapter only)") { |v| @ids_field = v }
-        opts.on("--scope-column=[COLUMN]", "DEPRECATED. Filter every table by this shared global column (--ids are its values) instead of a single --target-table. SQL adapters only; mutually exclusive with --target-table. Prefer declaring a per-table `scope_column:` in the schema config and running with --target-table.") { |v| @scope_column = v }
+        opts.on("--scope-column=[COLUMN]", "DEPRECATED. Filter every table by this shared global column (--ids are its values) instead of a single --target-table. SQL adapters only; mutually exclusive with --target-table. Prefer declaring a per-table `scope_column:` in the schema config.") { |v| @scope_column = v }
         opts.on("--output-format=[FORMAT]", "Output format: insert (default) or copy (PostgreSQL only, export subcommand only)") { |v| @output_format = v }
         opts.on("--insert-only", "DEPRECATED: ignored. exwiw no longer generates delete-*.sql files, so every export is insert-only.") do
           $stderr.puts "warning: --insert-only is obsolete and ignored (exwiw no longer generates delete-*.sql files); remove the flag"

data/lib/exwiw/query_ast.rb

@@ -26,6 +26,25 @@ module Exwiw
       end
     end
 
+    # The ID spaces whose values `query` filters by, anywhere in it.
+    def self.id_spaces(query)
+      where_clauses = query.where_clauses +
+        query.join_clauses.flat_map { |j| j.where_clauses + j.base_where_clauses }
+
+      where_clauses.flat_map do |where_clause|
+        next [] unless where_clause.is_a?(WhereClause)
+
+        own = where_clause.is_a?(ScopeWhereClause) ? [where_clause.id_space] : []
+        nested =
+          case where_clause.value
+          when SelectSubquery then id_spaces(where_clause.value.query)
+          when UnionSubquery then where_clause.value.queries.flat_map { |q| id_spaces(q) }
+          else []
+          end
+        own + nested
+      end.uniq
+    end
+
     class JoinClause
       # `where_clauses` is compiled against this join's join_table_name (the
       # joined-to table). `base_where_clauses`, on the other hand, is compiled
@@ -70,6 +89,12 @@ module Exwiw
       end
     end
 
+    # A scope-column filter. It compiles like any WhereClause; `id_space` records
+    # which ID space its values come from, so a query mixing spaces can be found.
+    ScopeWhereClause = Class.new(WhereClause) do
+      attr_accessor :id_space
+    end
+
     # Resolves a set of values on `where_column` to the rows' `select_column`
     # via a nested SELECT. Used as the `value` of a WhereClause whose operator
     # is `:in_subquery`, so a non primary-key `ids_field` can filter related

data/lib/exwiw/query_ast_builder.rb

@@ -12,25 +12,57 @@ module Exwiw
       new(table_name, table_by_name, dump_target, logger).scope_category
     end
 
-    # Scope-column mode is active when EITHER the named `--target-table` declares a
-    # per-table `scope_column` (the preferred trigger: the target is then scoped
-    # like any other table — its `--ids` are scope-column values, not primary
-    # keys), OR the deprecated `--scope-column` flag is set (a global column with no
-    # target). In both cases every table is filtered by a shared column instead of
-    # being anchored on one named target's primary key.
+    # Scope-column mode is active when the named `--target-table` declares a
+    # per-table `scope_column` (the target is then scoped like any other table,
+    # and its `--ids` are scope-column values, not primary keys); when no target is
+    # named but `--ids` are given and some table declares a `scope_column`; or when
+    # the deprecated `--scope-column` flag is set (a global column with no target).
+    # In each case every table is filtered by a shared column instead of being
+    # anchored on one named target's primary key.
     def self.scope_mode?(table_by_name, dump_target)
       return true unless dump_target.scope_column.nil?
-      return false if dump_target.table_name.nil?
+      if dump_target.table_name.nil?
+        return ids_given?(dump_target) && scope_column_declared?(table_by_name)
+      end
 
       target = table_by_name[dump_target.table_name]
       !!(target && target.respond_to?(:scope_column) && target.scope_column)
     end
 
+    def self.ids_given?(dump_target)
+      dump_target.ids.each_value.any?(&:any?)
+    end
+
+    # An ignore:true table is not extracted, so its declaration cannot anchor the
+    # scope.
+    def self.scope_column_declared?(table_by_name)
+      table_by_name.each_value.any? do |table|
+        !table.ignore && table.respond_to?(:scope_column) && !table.scope_column.nil?
+      end
+    end
+
+    # `--ids` without `--target-table` (and without the deprecated flag) selects
+    # scope-column mode, which needs a declared `scope_column` to start from.
+    def self.validate_ids_anchor!(table_by_name, dump_target)
+      return unless dump_target.table_name.nil? && dump_target.scope_column.nil?
+      return unless ids_given?(dump_target)
+      return if scope_column_declared?(table_by_name)
+
+      raise ArgumentError,
+            "--ids was given without --target-table, but no table in the schema declares " \
+            "`scope_column` (tables marked `ignore: true` are not counted). Either name the table " \
+            "the ids belong to with --target-table (single-target mode), or declare " \
+            "`scope_column: <column>` on the tables to filter by it (scope-column mode)."
+    end
+
     # Strict pre-flight: abort if any extractable table cannot be scoped (scope
     # mode), or declares a `batch_scope` its scoping shape cannot be sliced by
     # (both modes) — before any output is written. `tables` is the set of
     # dumpable configs (ignore:true tables are skipped — they are not extracted).
     def self.validate_scope!(tables, table_by_name, dump_target, logger)
+      validate_ids_anchor!(table_by_name, dump_target)
+      validate_named_id_spaces_allowed!(tables, table_by_name, dump_target)
+
       # Unscopable is reported before a bad batch_scope shape — it is the more
       # fundamental problem.
       if scope_mode?(table_by_name, dump_target)
@@ -57,6 +89,9 @@ module Exwiw
                 "add a belongs_to path to a table that carries the scope column, mark it " \
                 "`scope_exempt: true` to export it in full, or set `ignore: true` to skip it."
         end
+
+        validate_id_space_values!(tables, table_by_name, dump_target, logger)
+        validate_single_id_space_per_table!(tables, table_by_name, dump_target, logger)
       end
 
       tables.reject(&:ignore).each do |table|
@@ -66,6 +101,77 @@ module Exwiw
       end
     end
 
+    # Only scope-column mode on the SQL adapters can filter by a named ID space.
+    def self.validate_named_id_spaces_allowed!(tables, table_by_name, dump_target)
+      named = dump_target.named_id_spaces
+      return if named.empty?
+
+      given = named.map { |space| "'#{space}'" }.join(', ')
+      # The CLI rejects this up front; this covers callers that build a Runner directly.
+      if tables.any? { |table| table.is_a?(MongodbCollectionConfig) }
+        raise ArgumentError,
+              "--ids were given for the ID space(s) #{given}, but the mongodb adapter supports " \
+              "only the default ID space (--ids without a `<id_space>=` prefix)."
+      end
+
+      return if scope_mode?(table_by_name, dump_target)
+
+      raise ArgumentError,
+            "--ids were given for the ID space(s) #{given}, but --target-table " \
+            "'#{dump_target.table_name}' declares no scope_column, so the run is single-target mode, " \
+            "which uses only the default ID space. Name a table that declares a scope_column as " \
+            "--target-table, or drop the named --ids."
+    end
+
+    # Every ID space a scope_column belongs to must be given values,
+    # and every ID space given values must be used by some table.
+    def self.validate_id_space_values!(tables, table_by_name, dump_target, logger)
+      tables_by_space =
+        tables.reject(&:ignore).each_with_object({}) do |table, hash|
+          space = new(table.name, table_by_name, dump_target, logger).declared_id_space
+          (hash[space] ||= []) << table.name if space
+        end
+
+      missing = tables_by_space.keys.select { |space| dump_target.ids_for(space).empty? }.sort
+      if missing.any?
+        details = missing.map do |space|
+          flag = space == DEFAULT_ID_SPACE ? "--ids=<ids>" : "--ids=#{space}=<ids>"
+          "'#{space}' (#{tables_by_space[space].sort.join(', ')}; pass #{flag})"
+        end
+        raise ArgumentError,
+              "scope-column mode: no ids were given for the ID space(s) that these tables are " \
+              "filtered by: #{details.join('; ')}."
+      end
+
+      unused = dump_target.ids.keys.reject { |space| tables_by_space.key?(space) }.sort
+      if unused.any?
+        raise ArgumentError,
+              "scope-column mode: ids were given for the ID space(s) #{unused.map { |space| "'#{space}'" }.join(', ')}, " \
+              "but no table declares a scope_column in them (a table without `id_space` is in " \
+              "'#{DEFAULT_ID_SPACE}'). Check the ID space name, or declare `id_space` on the tables it scopes."
+      end
+    end
+
+    # A table is filtered by the values of one ID space only. Without this check
+    # the belongs_to walk would silently settle on the nearest scoped table and
+    # ignore a route into another space.
+    def self.validate_single_id_space_per_table!(tables, table_by_name, dump_target, logger)
+      return if dump_target.ids.size < 2
+
+      mixed =
+        tables.reject(&:ignore).filter_map do |table|
+          spaces = new(table.name, table_by_name, dump_target, logger).reached_id_spaces
+          "#{table.name} (#{spaces.join(', ')})" if spaces.size > 1
+        end
+      return if mixed.empty?
+
+      raise ArgumentError,
+            "scope-column mode: #{mixed.size} table(s) reach more than one ID space: #{mixed.sort.join(', ')}. " \
+            "A table must be scoped by the values of a single ID space; set `ignore: true` on the " \
+            "belongs_to (or remove the reverse_scope arm) that leads into the other space, or declare " \
+            "`scope_column:` on the table to filter it directly."
+    end
+
     attr_reader :table_name, :table_by_name, :dump_target
 
     # The multiple constrained referencers (comma-joined) that made the
@@ -530,7 +636,7 @@ module Exwiw
         clauses.push Exwiw::QueryAst::WhereClause.new(
           column_name: dump_target.ids_field || table.primary_key,
           operator: :eq,
-          value: dump_target.ids
+          value: dump_target.default_ids
         )
 
         return clauses
@@ -572,7 +678,7 @@ module Exwiw
         return Exwiw::QueryAst::WhereClause.new(
           column_name: foreign_key,
           operator: :eq,
-          value: dump_target.ids
+          value: dump_target.default_ids
         )
       end
 
@@ -584,7 +690,7 @@ module Exwiw
           table_name: target.name,
           select_column: target.primary_key,
           where_column: dump_target.ids_field,
-          where_values: dump_target.ids
+          where_values: dump_target.default_ids
         )
       )
     end
@@ -672,7 +778,9 @@ module Exwiw
     # ------------------------------------------------------------------
 
     private def scope_mode?
-      self.class.scope_mode?(table_by_name, dump_target)
+      return @scope_mode if defined?(@scope_mode)
+
+      @scope_mode = self.class.scope_mode?(table_by_name, dump_target)
     end
 
     # Classifier used by validate_scope! and mirrored by build_scoped below.
@@ -687,6 +795,27 @@ module Exwiw
       :unscopable
     end
 
+    # The ID space this table's scope column belongs to, or nil when it has none.
+    # A scope_exempt table is dumped in full, but tables reaching it through
+    # belongs_to are still filtered by its scope column, so its ID space counts.
+    def declared_id_space
+      table = table_by_name.fetch(table_name)
+      return nil unless directly_scoped?(table)
+
+      id_space_of(table)
+    end
+
+    # The ID spaces a table that is not directly scoped can be scoped through:
+    # those of the scoped tables its belongs_to walk can stop at, and those its
+    # built query filters by (polymorphic arms, reverse_scope, referenced-by and
+    # the parent cascade).
+    def reached_id_spaces
+      table = table_by_name.fetch(table_name)
+      return [] if scope_exempt?(table) || directly_scoped?(table)
+
+      (belongs_to_reachable_id_spaces(table) | QueryAst.id_spaces(run)).sort
+    end
+
     # True when this table may still attempt the forward "scope via a scoped
     # belongs_to parent" rescue: it is not already on the forward-resolution
     # path, so descending into its parent cannot revisit it (a belongs_to cycle).
@@ -791,15 +920,43 @@ module Exwiw
       table.columns.any? { |c| c.name == column }
     end
 
+    private def id_space_of(table)
+      table.id_space || DEFAULT_ID_SPACE
+    end
+
     private def scope_where_clause(table)
       batch_clause = batch_ids_clause(table)
       return batch_clause if batch_clause
 
-      Exwiw::QueryAst::WhereClause.new(
+      Exwiw::QueryAst::ScopeWhereClause.new(
         column_name: resolved_scope_column(table),
         operator: :eq,
-        value: dump_target.ids
-      )
+        value: dump_target.ids_for(id_space_of(table))
+      ).tap { |clause| clause.id_space = id_space_of(table) }
+    end
+
+    # The ID spaces of every directly scoped table the belongs_to walk of
+    # #find_path_to_scoped could stop at, not only the nearest one.
+    private def belongs_to_reachable_id_spaces(table)
+      visited = Set[table.name]
+      queue = [table]
+      spaces = Set.new
+
+      until queue.empty?
+        current = queue.shift
+        current.belongs_tos.each do |relation|
+          next_table = table_by_name[relation.table_name]
+          next if next_table.nil? || !visited.add?(next_table.name)
+
+          if directly_scoped?(next_table)
+            spaces << id_space_of(next_table)
+          else
+            queue.push(next_table)
+          end
+        end
+      end
+
+      spaces
     end
 
     # This batch's ids, in place of the batch table's scope filter. nil when the

data/lib/exwiw/runner.rb

@@ -203,6 +203,7 @@ module Exwiw
       AfterInsertHook.run(
         path: @after_insert_hook_path,
         cli_options: @cli_options,
+        dump_target: @dump_target,
         output_dir: @output_dir,
         next_idx: total_size + 1,
         output_extension: adapter.output_extension,

data/lib/exwiw/table_config.rb

@@ -38,6 +38,9 @@ module Exwiw
     # different column name on this table).
     attribute :scope_exempt, Serdes::OptionalType.new(Serdes::ConcreteType.new(Boolean)), skip_serializing_if_nil: true
     attribute :scope_column, optional(String), skip_serializing_if_nil: true
+    # The ID space (Exwiw::DumpTarget#ids) the `scope_column` values belong to.
+    # nil means `default`. User-configured, never generated.
+    attribute :id_space, optional(String), skip_serializing_if_nil: true
 
     # `reverse_scope` opts a table into multi-referencer reverse scoping (see
     # Exwiw::ReverseScope and QueryAstBuilder#build_referenced_by_clause): a
@@ -175,6 +178,7 @@ module Exwiw
         merged_table.bulk_insert_chunk_size = bulk_insert_chunk_size
         merged_table.scope_exempt = scope_exempt
         merged_table.scope_column = scope_column
+        merged_table.id_space = id_space
         merged_table.reverse_scope = reverse_scope
         merged_table.batch_scope = batch_scope
 
@@ -250,6 +254,19 @@ module Exwiw
                 "Table '#{name}': reverse_scope.column '#{reverse_scope.column}' is not a declared column."
         end
 
+        if id_space
+          unless id_space.match?(ID_SPACE_NAME_PATTERN)
+            raise ArgumentError,
+                  "Table '#{name}': id_space '#{id_space}' must start with a lowercase letter and " \
+                  "contain only lowercase letters, digits and underscores."
+          end
+          if scope_column.nil?
+            raise ArgumentError,
+                  "Table '#{name}': id_space names the ID space of the scope_column values, " \
+                  "so it requires scope_column."
+          end
+        end
+
         columns.each { |column| validate_ruby_side_masking!(column) }
       end
     end

data/lib/exwiw/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Exwiw
-  VERSION = "1.1.5"
+  VERSION = "1.1.6"
 end

data/lib/exwiw.rb

@@ -64,7 +64,46 @@ module Exwiw
   # (`scope_column IN ids`) and tables lacking it are reached by walking
   # belongs_to up to the nearest table that has it. When set, `table_name` is
   # nil. SQL adapters only.
-  DumpTarget = Struct.new(:table_name, :ids, :ids_field, :scope_column, keyword_init: true)
+  #
+  # `ids` maps an ID space name to its values. A table's `scope_column` values
+  # belong to the ID space it declares (`id_space`, `default` when omitted), so
+  # groups of tables keyed by unrelated kinds of id can be scoped in one run. An
+  # Array is taken as the values of the `default` space. Single-target mode and
+  # the mongodb adapter use the `default` space only.
+  DEFAULT_ID_SPACE = "default"
+  ID_SPACE_NAME_PATTERN = /\A[a-z][a-z0-9_]*\z/
+
+  DumpTarget = Struct.new(:table_name, :ids, :ids_field, :scope_column, keyword_init: true) do
+    def self.normalize_ids(ids)
+      case ids
+      when nil then {}
+      when Hash then ids.to_h { |space, values| [space.to_s, Array(values)] }
+      else { DEFAULT_ID_SPACE => Array(ids) }
+      end
+    end
+
+    def initialize(**kwargs)
+      super
+      self.ids = ids
+    end
+
+    def ids=(value)
+      self[:ids] = self.class.normalize_ids(value)
+    end
+
+    def ids_for(id_space)
+      ids.fetch(id_space, [])
+    end
+
+    def default_ids
+      ids_for(DEFAULT_ID_SPACE)
+    end
+
+    # ID spaces other than `default` that were given, with or without values.
+    def named_id_spaces
+      ids.keys - [DEFAULT_ID_SPACE]
+    end
+  end
   # `uri` is an optional full connection string (currently only honored by the
   # mongodb adapter, e.g. `mongodb+srv://...`). When present it is the source of
   # truth for the connection — host/port/user/password are ignored — so TLS,

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: exwiw
 version: !ruby/object:Gem::Version
-  version: 1.1.5
+  version: 1.1.6
 platform: ruby
 authors:
 - Shia