exwiw 0.3.7 → 0.3.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 91f070d56a04f45680e1c858109fd6828d69220373ad725d07c248d4c8ed2f10
-  data.tar.gz: a2dc34318e0c0a15ec8af84d75330b92ce5278c14a5b88d6e824e26bc5b1f6a7
+  metadata.gz: 90a440a039497562ccc9a4ab3045342953ba23faf0fdff48cfd633771a4688a4
+  data.tar.gz: 5f8ee4853cbbe39dedba4176e5db61d91572ee3766d1d76e3eda6977ea373e6b
 SHA512:
-  metadata.gz: 5700d2fd7287c365ef6c9759fd7f5e85188db6e568b32f8c29a4a0f9f3479d967014a1f77bbf64e1d0e20fdfd75004d05e86e5152d781d6fcae8a5cbf82bca1b
-  data.tar.gz: '09d1f99b70a0f7d0a8e12fbb0f69de776b9932db9fdc386b71fb3822ee118eda786be79737e3a2e85c340d8c6ce15e5bca472e61d4f4f6517f0bc34d133db9d1'
+  metadata.gz: aa34168f0c3b6137eca95bf8ebeea625ea106559a779bd8739c09e5ab8ffeae0a9a84f9ca188809e579ab3d1c3f8c77b68d57d006b156de30b1d01e100435ecb
+  data.tar.gz: c7ad4547852032fcb4c356883232c8a34d6418baf498686ed6f25a80279d1cf61379b566ee29267f989d9de80f4b2490022d03695e686dedf5c506b2c01871f8

data/CHANGELOG.md

@@ -2,6 +2,12 @@
 
 ## [Unreleased]
 
+## [0.3.8] - 2026-06-02
+
+### Added
+
+- ActiveStorage support is now complete. `schema:generate` follows `has_one_attached` / `has_many_attached` macros so `active_storage_attachments` is extracted correctly via its polymorphic `belongs_to`. `active_storage_blobs` is no longer dumped wholesale: a reverse "referenced_by" extraction (a `SELECT` subquery) narrows it to only the blobs referenced by the extracted attachments. `active_storage_variant_records` (derivative, regenerable variant-tracking rows) is now emitted with `ignore: true` instead of being dumped in full — it has no path to a dump target and its `blob_id` could otherwise reference blobs outside the narrowed set, causing a foreign-key violation on import. It is also dropped from the attachments `record` polymorphic expansion so the non-ignored attachments table carries no dangling belongs_to to it. ([#69](https://github.com/heyinc/exwiw/pull/69))
+
 ## [0.3.7] - 2026-06-01
 
 - bug fix https://github.com/heyinc/exwiw/pull/67

data/README.md

@@ -341,6 +341,25 @@ WHERE reviews.reviewable_id IN (/* products subquery */)
 
 The same type filter is applied on the join path — and in the matching `delete-*.sql` bulk-delete subquery — when the polymorphic table is an intermediate hop rather than the directly-dumped table.
 
+### ActiveStorage (`has_one_attached` / `has_many_attached`)
+
+ActiveStorage is handled automatically — no ActiveStorage-specific configuration is required. The `has_one_attached` / `has_many_attached` macros don't add a column to the owning model; they generate ordinary associations that exwiw already understands:
+
+- **`active_storage_attachments`** is the polymorphic join row (`belongs_to :record, polymorphic: true` + `belongs_to :blob`). `exwiw:schema:generate` expands the polymorphic `record` into one `belongs_to` per model that declared `has_*_attached` (found via the generated `has_* ..., as: :record` reflections), exactly like any other [polymorphic `belongs_to`](#polymorphic-belongs_to). So only the attachments whose owner is among the dumped rows are extracted.
+- **`active_storage_blobs`** has no `belongs_to` of its own (attachments point *at* it), so it has no path to the dump target. exwiw narrows it via **reverse / "referenced_by" extraction**: a parent table referenced by exactly one constrained, non-polymorphic child is constrained to just the referenced ids instead of dumping every row:
+
+  ```sql
+  SELECT active_storage_blobs.* FROM active_storage_blobs
+  WHERE active_storage_blobs.id IN (
+    SELECT active_storage_attachments.blob_id FROM active_storage_attachments
+    WHERE active_storage_attachments.record_id IN (/* owner subquery */)
+      AND active_storage_attachments.record_type = '...'
+  )
+  ```
+
+  `active_storage_variant_records` also references blobs, but since it has no path of its own to the dump target it doesn't constrain anything and is ignored as a referencer — blobs stays narrowed to the attachment-referenced ids. (A parent referenced by *multiple* constrained children currently falls back to dumping all of its rows.)
+- **`active_storage_variant_records`** holds derivative variant-tracking rows that ActiveStorage regenerates lazily, and it too has no path to the dump target — left alone it would land in the "no relation → dump all" branch and, worse, its `blob_id` could point at blobs outside the narrowed set above (a foreign-key violation on import). `exwiw:schema:generate` therefore emits it with **`ignore: true`** (and drops it from the attachments `record` polymorphic expansion so nothing carries a dangling reference to it), so its data is skipped while the DDL is still written. Remove `ignore` from the generated config if you really need to export it.
+
 ### Rails-managed tables (special `type` values)
 
 Some tables are owned by Rails itself rather than the application — they have no ActiveRecord model and Rails reserves the right to evolve their column shape between versions (e.g. `schema_migrations`, `ar_internal_metadata`). exwiw treats them as a distinct category via the `type` field on a table config:

data/lib/exwiw/adapter/mysql_adapter.rb

@@ -226,6 +226,10 @@ module Exwiw
       end
 
       private def compile_subquery(subquery)
+        # A SelectSubquery wraps a full Select (the referencing table's
+        # extraction query, projected to a foreign key); compile it as-is.
+        return compile_ast(subquery.query) if subquery.is_a?(Exwiw::QueryAst::SelectSubquery)
+
         inner_values = subquery.where_values.map { |v| escape_value(v) }
         "SELECT #{subquery.table_name}.#{subquery.select_column} " \
           "FROM #{subquery.table_name} " \

data/lib/exwiw/adapter/postgresql_adapter.rb

@@ -253,6 +253,10 @@ module Exwiw
       end
 
       private def compile_subquery(subquery)
+        # A SelectSubquery wraps a full Select (the referencing table's
+        # extraction query, projected to a foreign key); compile it as-is.
+        return compile_ast(subquery.query) if subquery.is_a?(Exwiw::QueryAst::SelectSubquery)
+
         inner_values = subquery.where_values.map { |v| escape_value(v) }
         "SELECT #{subquery.table_name}.#{subquery.select_column} " \
           "FROM #{subquery.table_name} " \

data/lib/exwiw/adapter/sqlite_adapter.rb

@@ -198,6 +198,10 @@ module Exwiw
       end
 
       private def compile_subquery(subquery)
+        # A SelectSubquery wraps a full Select (the referencing table's
+        # extraction query, projected to a foreign key); compile it as-is.
+        return compile_ast(subquery.query) if subquery.is_a?(Exwiw::QueryAst::SelectSubquery)
+
         inner_values = subquery.where_values.map { |v| escape_value(v) }
         "SELECT #{subquery.table_name}.#{subquery.select_column} " \
           "FROM #{subquery.table_name} " \

data/lib/exwiw/query_ast.rb

@@ -41,7 +41,7 @@ module Exwiw
         {
           column_name: column_name,
           operator: operator,
-          value: value.is_a?(Subquery) ? value.to_h : value,
+          value: value.is_a?(Subquery) || value.is_a?(SelectSubquery) ? value.to_h : value,
         }
       end
     end
@@ -65,6 +65,24 @@ module Exwiw
       end
     end
 
+    # A subquery whose body is a full `Select`, projected down to a single
+    # column. Unlike the flat `Subquery` above (one column = one IN-list), this
+    # carries the referencing table's complete extraction query — joins,
+    # multiple where conditions, polymorphic type filters and all. Used by the
+    # reverse / "referenced_by" extraction so a parent table with no belongs_to
+    # path to the dump target (e.g. active_storage_blobs) is constrained to only
+    # the rows referenced by an extractable child table:
+    #
+    #   <parent>.<pk> IN (SELECT <child>.<fk> FROM <child> WHERE <child filters>)
+    #
+    # `query` is the child's `Select` already projected to the foreign-key
+    # column that points at the parent.
+    SelectSubquery = Struct.new(:query, keyword_init: true) do
+      def to_h
+        { query: query.to_h }
+      end
+    end
+
     module ColumnValue
       Base = Struct.new(:name, :value, keyword_init: true)
       Plain = Class.new(Base)
@@ -103,6 +121,15 @@ module Exwiw
         @join_clauses << join_clause
       end
 
+      def to_h
+        {
+          from: from_table_name,
+          columns: select_all ? "*" : columns.map { |c| { name: c.name, value: c.value } },
+          joins: join_clauses.map(&:to_h),
+          where: where_clauses.map { |w| w.is_a?(String) ? w : w.to_h },
+        }
+      end
+
       private def map_column_value(columns)
         columns.map do |c|
           if c.raw_sql

data/lib/exwiw/query_ast_builder.rb

@@ -2,17 +2,18 @@
 
 module Exwiw
   class QueryAstBuilder
-    def self.run(table_name, table_by_name, dump_target, logger)
-      new(table_name, table_by_name, dump_target, logger).run
+    def self.run(table_name, table_by_name, dump_target, logger, allow_reverse: true)
+      new(table_name, table_by_name, dump_target, logger, allow_reverse: allow_reverse).run
     end
 
     attr_reader :table_name, :table_by_name, :dump_target
 
-    def initialize(table_name, table_by_name, dump_target, logger)
+    def initialize(table_name, table_by_name, dump_target, logger, allow_reverse: true)
       @table_name = table_name
       @table_by_name = table_by_name
       @dump_target = dump_target
       @logger = logger
+      @allow_reverse = allow_reverse
     end
 
     def run
@@ -21,6 +22,19 @@ module Exwiw
       where_clauses = build_where_clauses(table, dump_target)
       join_clauses = build_join_clauses(table, table_by_name, dump_target)
 
+      # Reverse / "referenced_by" extraction. A table with no belongs_to path to
+      # the dump target produces no where/join clauses and would otherwise dump
+      # every row (see the "no relation -> dump all" case). If an extractable
+      # child table references it via a foreign key (e.g. active_storage_blobs is
+      # referenced by active_storage_attachments.blob_id), constrain it to just
+      # the referenced ids instead. Disabled (@allow_reverse=false) while building
+      # a child's subquery, so this never recurses.
+      if @allow_reverse && table.name != dump_target.table_name &&
+         where_clauses.empty? && join_clauses.empty?
+        reverse_clause = build_referenced_by_clause(table)
+        where_clauses.push(reverse_clause) if reverse_clause
+      end
+
       QueryAst::Select.new.tap do |ast|
         ast.from(table.name)
         if table.rails_managed?
@@ -100,6 +114,61 @@ module Exwiw
       join_clauses
     end
 
+    # Builds a `pk IN (SELECT child.fk FROM <child extraction query>)` clause
+    # for a table that is referenced by an extractable child table but has no
+    # belongs_to of its own toward the dump target. Returns nil when there is no
+    # such (single, unambiguous) referencer, leaving the caller to fall back to
+    # the dump-all behavior.
+    private def build_referenced_by_clause(table)
+      candidates = table_by_name.each_value.filter_map do |other|
+        next if other.name == table.name
+
+        relation = other.belongs_to(table.name)
+        # A polymorphic foreign key stores ids of several models in one column,
+        # so projecting it would pull in unrelated parents. Skip it here; the
+        # non-polymorphic blob_id on active_storage_attachments is what we want.
+        next if relation.nil? || relation.polymorphic?
+
+        # Build the child's own extraction query. allow_reverse:false stops a
+        # chain of FK-less tables from recursing back into each other.
+        child_query = self.class.run(other.name, table_by_name, dump_target, @logger, allow_reverse: false)
+
+        # Only an *already constrained* child narrows anything; an unconstrained
+        # child would select every fk value (i.e. dump all) and not help.
+        next unless child_query.where_clauses.any? || child_query.join_clauses.any?
+
+        [relation, child_query]
+      end
+
+      # Scope: only the unambiguous single-referencer case. Multiple referencers
+      # would need their subqueries OR'd together (not yet supported); falling
+      # back to dump-all preserves today's behavior for those.
+      if candidates.size != 1
+        if candidates.size > 1
+          @logger.debug("  #{table.name} has multiple referencing tables; skipping reverse extraction (dump all).")
+        end
+        return nil
+      end
+
+      relation, child_query = candidates.first
+
+      # Project the child's extraction query down to just the foreign key that
+      # points at `table`. Force a plain column so any masking/raw_sql configured
+      # on that column does not corrupt the id comparison.
+      fk_column = TableColumn.from_symbol_keys(name: relation.foreign_key)
+      projected = QueryAst::Select.new
+      projected.from(child_query.from_table_name)
+      projected.select([fk_column])
+      child_query.join_clauses.each { |j| projected.join(j) }
+      child_query.where_clauses.each { |w| projected.where(w) }
+
+      QueryAst::WhereClause.new(
+        column_name: table.primary_key,
+        operator: :in_subquery,
+        value: QueryAst::SelectSubquery.new(query: projected)
+      )
+    end
+
     private def build_where_clauses(table, dump_target)
       clauses = []
 

data/lib/exwiw/schema_generator.rb

@@ -5,6 +5,20 @@ require "json"
 
 module Exwiw
   class SchemaGenerator
+    # ActiveStorage tracks generated image variants in this table. Its rows are
+    # derivative and regenerable — ActiveStorage lazily (re)creates a variant the
+    # next time it is requested — so there is little value in exporting them. More
+    # importantly, the table has no belongs_to path to any dump target, which
+    # would land it in QueryAstBuilder's "no relation -> dump all" branch, while
+    # its `blob_id` references active_storage_blobs, which the reverse
+    # "referenced_by" extraction narrows to only the attachment-referenced blobs.
+    # A full variant_records dump can therefore reference blobs that were never
+    # exported (a foreign-key violation on import). So the table is emitted with
+    # `ignore: true` (data extraction skipped) and excluded as a polymorphic
+    # `record` target so the non-ignored attachments table carries no dangling
+    # belongs_to to it.
+    ACTIVE_STORAGE_VARIANT_RECORDS_TABLE = "active_storage_variant_records"
+
     def self.from_rails_application(output_dir:)
       Rails.application.eager_load!
       new(models: ActiveRecord::Base.descendants, output_dir: output_dir)
@@ -94,6 +108,20 @@ module Exwiw
             belongs_tos: aggregate_belongs_tos(model_group),
             columns: representative.column_names.map { |name| { name: name } },
           )
+        elsif table_name == ACTIVE_STORAGE_VARIANT_RECORDS_TABLE
+          # See ACTIVE_STORAGE_VARIANT_RECORDS_TABLE. Emitted with ignore:true so
+          # the derivative variant rows are not dumped; primary_key/columns are
+          # kept so a user can remove `ignore` to opt back in if they really want
+          # to export them.
+          TableConfig.from_symbol_keys(
+            name: table_name,
+            primary_key: primary_key,
+            ignore: true,
+            comment: "ActiveStorage variant tracking records are derivative and " \
+                     "regenerable; data extraction is skipped. Remove `ignore` to export them.",
+            belongs_tos: aggregate_belongs_tos(model_group),
+            columns: representative.column_names.map { |name| { name: name } },
+          )
         else
           TableConfig.from_symbol_keys(
             name: table_name,
@@ -203,6 +231,12 @@ module Exwiw
     # belongs_to ordering stable.
     private def polymorphic_target_models(association_name)
       concrete_models.select do |model|
+        # active_storage_variant_records is ignored (see the constant), so it must
+        # not be expanded as a polymorphic target — otherwise the non-ignored
+        # attachments table would carry a dangling belongs_to to an ignored table,
+        # which is rejected at load time.
+        next false if model.table_name == ACTIVE_STORAGE_VARIANT_RECORDS_TABLE
+
         (model.reflect_on_all_associations(:has_many) +
          model.reflect_on_all_associations(:has_one))
           .any? { |reflection| reflection.options[:as] == association_name }

data/lib/exwiw/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Exwiw
-  VERSION = "0.3.7"
+  VERSION = "0.3.8"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: exwiw
 version: !ruby/object:Gem::Version
-  version: 0.3.7
+  version: 0.3.8
 platform: ruby
 authors:
 - Shia