dumped_railers 0.1.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: e71ec228651de3ad8d735b37d3d091b1ac453df4e021a0545476a456eff5f7cb
+  data.tar.gz: da03643cbd834f41f59878ea1d7bf1411f2b739fc26a87e660e7b1f8d30c9fcd
+SHA512:
+  metadata.gz: f53cd73227a90f14269c0dad160e3105bb3307c28274f45b7e2a96b44a22c7e9765cca115e23cc3a62dd0678afd17b0e4135f2d9c5035eb03a93a9e0e34b8fbb
+  data.tar.gz: f0d1cfad86c548302527028ccfa20d5ef10a16d24206ece6bdd8318f14768fa23aa7df06f68115ec27869fda3fa1297f3855e407d6b8cb67d3b4fd523bcb439a

data/.gitignore

@@ -0,0 +1,16 @@
+/.bundle/
+/.yardoc
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+**/*/tmp/
+.rubocop-*
+
+Gemfile.lock
+*.sqlite3
+*.log
+
+# rspec failure tracking
+.rspec_status

data/.rspec

@@ -0,0 +1,3 @@
+--format documentation
+--color
+--require spec_helper

data/.travis.yml

@@ -0,0 +1,12 @@
+---
+language: ruby
+cache: bundler
+rvm:
+  - 2.7.2
+  - 2.6.6
+  - 2.5.8
+gemfile:
+  - gemfiles/Gemfile.rails_6.1.0
+  - gemfiles/Gemfile.rails_6.0.3.4
+  - gemfiles/Gemfile.rails_5.2.4
+before_install: gem install bundler -v 2.1.4

data/CHANGELOG.md

@@ -0,0 +1,10 @@
+# Change Log
+
+## [0.1.0]
+### Added
+- Implement method that dumps specified model data in fixture (YAML) format
+- Implement import method that transfer fixture into database
+- Add test helpers to make it easy to implement database related tests
+- Add config options to be able to ignore specific columns
+- Accept preprocessors to make filtering behavior pluggable and customizable
+- Add Readme

data/Gemfile

@@ -0,0 +1,4 @@
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in dumped_railers.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2020 Koji Onishi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,165 @@
+# DumpedRailers    <img src='https://user-images.githubusercontent.com/23026542/101830310-aaf10000-3b77-11eb-9e0a-d14e45b27760.png' width=40>
+[![Build Status](https://travis-ci.com/fursich/dumped_railers.svg?branch=main)](https://travis-ci.com/fursich/dumped_railers) [![Gem Version](https://badge.fury.io/rb/dumped_railers.svg)](https://badge.fury.io/rb/dumped_railers)
+
+Helping you take a snapshot of ActiveRecord models in Rails-compatible fixture format, and re-importing them wherever necessary without destroying current data you have.
+
+With Rails, you can import any fixture data using `rails db:fixtures:load` - however, this let's you remove all the existing data in your database before importing the fixtures. This is good for clean seeding, typically when running your tests, but there are other senarios where you want to merely ADD data, without damaging the current data you are woking with.
+
+This is a bit trickey puzzle, though. To add the imported records, you cannot dump and re-import the primary key, as they are already taken by the original records. But usually your records involve reference to associated records, where the associations are guaranteed by the very primary keys.
+In other words, the fixures have to be stored and re-imported, so as to maintain their original inter-dependencies, but their primary keys have to be re-assigned (headache)
+
+DumpedRailers can add (not replace) the fixture without removing the current records, while restoring **all associations** among the original records.
+Additionally, it can ignore, mask, or tweak any attributes when dumping the records into fixture files, which is convenient to export sensitive data (typically in your production environment) into, for instance, your staging environment.
+
+This feature can particularily help you in the following senarios:
+- you want to copy a group of records without breaking the other data sets you are working on
+- you want to transfer some production data into dev environment, to reproduce some errors you encontered.
+- you work for a multi-tenancy service, where you want to duplicate interdependent set of records from tenant A to tenant B.
+- you simply want to populate interdependent record sets, which is not easy to re-build with FactoryBot
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'dumped_railers'
+```
+
+And then execute:
+
+    $ bundle install
+
+Or install it yourself as:
+
+    $ gem install dumped_railers
+
+## Usage
+
+### Getting Started
+
+* if you want to dump (let's say) User, Item, and Tag models, just run the following.
+
+```ruby
+DumpedRailers.dump!(User, Item, Tag, base_dir: 'tmp/fixtures/')
+```
+this will generate three fixture files under tmp/fixtures/ folder.
+
+* if you want to import the records you just saved, run:
+
+```ruby
+DumpedRailers.import!('tmp/fixtures')
+```
+
+* you can also specify individual model(s) for selective import.
+
+```ruby
+DumpedRailers.import!('tmp/fixtures/users.yml', 'tmp/fixtures/items.yml')
+```
+
+NOTE: you at least have to provide all the dependent records, so that DumpedRailers can resolve dependencies among the fixtures provided.
+
+### Ignored Columns
+
+* By default, DumpedRailers ignore three columns - `id`, `created_at`, `updated_at`. You can always update/change this settings as follows.
+
+```ruby
+DumpedRailers.configure do |config|
+  config.ignorable_columns += [:published_on] # published_on will be ignored on top of default settings.
+end
+```
+
+* of course you can totally replace the settings with your own.
+```ruby
+DumpedRailers.configure do |config|
+  config.ignorable_columns = %i[uuid created_on updated_on] # uuid and created_on will be ignored instead of id, created_at, updated_at
+end
+```
+
+### Masking, filtering
+
+* you can pass `preprocessors` to DumpedRailers before it starts dump. All the attributes are filtered through preprocessors in order of registration.
+
+```ruby
+DumpedRailers.dump!(User, Item, base_dir: 'tmp/', preprocessors: [MaskingPreprocessor.new])
+```
+
+* "Preprocessors" can be lambda, or module, or any objects that can repond to #call(atrs, model).
+
+
+```ruby
+class MaskingPreprocessor
+  def call(attrs, model)
+    attrs.map { |col, value|
+      col.match?(/password/) ? [col, '<MASKED>'] : [col, value]
+    }.to_h
+  end
+end
+```
+
+* a lambda object can be accepted as well
+
+```ruby
+masking_preprocessor = -> (attrs, model) { attrs.transform_values(&:upcase) }
+```
+
+NOTE: The proprocessors must return attributes in the same format `{ attributes_name: value }` so that preprocessors and dump handlers can preprocessors in nested manner.
+
+### pseudo multi-tenancy (such as ActsAsTenant)
+
+* Such library builds multi-tenancy environment on one single database, using default_scope to switch over database access rights between tenants. You can incorporate data from Tenant A to Tenant B as follows. let's say we use [ActsAsTenant](https://github.com/ErwinM/acts_as_tenant)
+
+```ruby
+# make sure to delete old fixtures in the folder
+File.delete(Dir.glob('tmp/fixtures/{**,*}/*.yml'))
+
+# let DumpedRailers ignore tenant column, as it will be overwritten by ActsAsTenant
+DumpedRailers.configure do |config|
+  config.ignorable_columns += [:account_id]
+end
+
+# dump from tenant_a
+ActsAsTenant.with_tenant(tenant_a) do
+  DumpedRailers.dump!(Item, Tag, base_dir: 'tmp/fixtures/')
+end
+
+# import into tenant_b
+ActsAsTenant.with_tenant(tenant_b) do
+  DumpedRailers.import!('tmp/fixtures')
+end
+```
+
+## Troubleshooting
+
+When DumpedRailers fail to resolve dependencies, please check the following.
+
+* DumpedRailers uses ActiveRecord reflection methods to sort out model dependencies. You might have to check wheather your relation is defined on the model (especially, `belongs_to` cannot be omitted).
+
+* Dependencies cannot be resolved when cyclic dependencies are detected. For instance, the case like below cannot be resolved.
+
+```ruby
+class Chicken < ActiveRecord::Base
+  belongs_to :egg, optional: true
+end
+
+class Egg < ActiveRecord::Base
+  belongs_to :chicken, optional: true
+end
+```
+
+* When exception raised, checking your log might give you a good hint (desperately staring at the backtrace won't give much information)
+  consider displaying `tail -f logs/development.log` while executing your script.
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/[fursich]/dumped_railers.
+
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "dumped_railers"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/dumped_railers.gemspec

@@ -0,0 +1,37 @@
+require_relative 'lib/dumped_railers/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = 'dumped_railers'
+  spec.version       = DumpedRailers::VERSION
+  spec.authors       = ['Koji Onishi']
+  spec.email         = ['fursich0@gmail.com']
+
+  spec.summary       = %q{A flexible fixture importer/exporter, that can transport ActiveRecord data in fixture format}
+  spec.description   = %q{DumpedRailers helps you take a snapshot of ActiveRecord models in Rails-compatible fixture format, and re-import them wherever necessary without destroying current data you have.}
+  spec.homepage      = 'https://github.com/fursich/dumped_railers'
+  spec.license       = 'MIT'
+  spec.required_ruby_version = Gem::Requirement.new('>= 2.3.0')
+
+  spec.metadata['homepage_uri'] = spec.homepage
+  spec.metadata['source_code_uri'] = 'https://github.com/fursich/dumped_railers'
+  spec.metadata['changelog_uri'] = 'https://github.com/fursich/dumped_railers/CHANGELOG.md'
+
+  # Specify which files should be added to the gem when it is released.
+  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
+  spec.files         = Dir.chdir(File.expand_path('..', __FILE__)) do
+    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  end
+  spec.bindir        = 'exe'
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ['lib']
+
+  spec.add_development_dependency 'bundler', '~> 2.0'
+  spec.add_development_dependency 'rake', '~> 12.3.3'
+  spec.add_development_dependency 'rspec', '~> 3.0'
+  spec.add_development_dependency 'sqlite3'
+  spec.add_development_dependency 'activerecord', '~> 5.2'
+  spec.add_development_dependency 'database_cleaner-active_record', '~> 1.8'
+  spec.add_development_dependency 'pry'
+  spec.add_development_dependency 'pry-byebug'
+  spec.add_development_dependency 'pry-doc'
+end

data/gemfiles/Gemfile.rails_5.2.4

@@ -0,0 +1,6 @@
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in dumped_railers.gemspec
+gem 'activerecord', '5.2.4'
+
+gemspec path: '../'

data/gemfiles/Gemfile.rails_6.0.3.4

@@ -0,0 +1,6 @@
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in dumped_railers.gemspec
+gem 'activerecord', '6.1.0'
+
+gemspec path: '../'

data/gemfiles/Gemfile.rails_6.1.0

@@ -0,0 +1,6 @@
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in dumped_railers.gemspec
+gem 'activerecord', '6.0.3.4'
+
+gemspec path: '../'

data/lib/dumped_railers.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+require 'dumped_railers/version'
+require 'dumped_railers/file_helper.rb'
+require 'dumped_railers/dump'
+require 'dumped_railers/import'
+
+module DumpedRailers
+  class << self
+
+    def dump!(*models, base_dir: './', preprocessors: nil)
+      preprocessors = [Preprocessor::StripIgnorables.new, *preprocessors].compact.uniq
+
+      fixture_handler = Dump.new(*models, preprocessors: preprocessors)
+      fixture_handler.build_fixtures!
+      fixture_handler.persist_all!(base_dir)
+    end
+
+    def import!(*paths)
+      # make sure class-baseed caches starts with clean state
+      DumpedRailers::RecordBuilder::FixtureRow::RecordStore.clear!
+      DumpedRailers::RecordBuilder::DependencyTracker.clear!
+
+      fixture_handler = Import.new(*paths)
+      fixture_handler.import_all!
+    end
+
+    class Configuration < ::OpenStruct; end
+
+    def config
+      @_config ||= Configuration.new
+    end
+
+    def configure
+      yield config
+    end
+
+    # FIXME: make it minimum
+    IGNORABLE_COLUMNS = %w[id created_at updated_at]
+    def configure_defaults!
+      configure do |config|
+        config.ignorable_columns = IGNORABLE_COLUMNS
+      end
+    end
+  end
+
+  configure_defaults!
+end

data/lib/dumped_railers/dump.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+require_relative 'fixture_builder/model'
+require_relative 'preprocessor/strip_ignorables'
+
+module DumpedRailers
+  class Dump
+    def initialize(*models, preprocessors: [])
+      @fixture_tables = models.map { |model|
+        FixtureBuilder::Model.new(model, preprocessors: preprocessors)
+      }
+    end
+
+    def build_fixtures!
+      @fixtures = @fixture_tables.map(&:build!).to_h
+    end
+
+    def persist_all!(base_dir)
+      FileUtils.mkdir_p(base_dir)
+      FileHelper.write(*@fixtures, base_dir: base_dir)
+    end
+  end
+end

data/lib/dumped_railers/file_helper.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+require "yaml"
+
+module DumpedRailers
+  module FileHelper
+    class << self
+      def read_fixtures(*paths)
+        yaml_files = paths.flat_map { |path|
+          if File.file?(path)
+            path
+          else
+            [*Dir["#{path}/{**,*}/*.yml"], "#{path}.yml"].select { |f|
+              ::File.file?(f)
+            }
+          end
+        }.uniq.compact
+
+        yaml_files.map { |file| 
+          raw_data = ::File.read(file)
+          YAML.load(raw_data)
+        }
+      end
+
+      def write(*fixtures, base_dir:)
+        fixtures.each do |table_name, fixture|
+          pathname = 
+            if defined?(Rails)
+              Rails.root.join("#{base_dir}/#{table_name}.yml")
+            else
+              Pathname.new("#{base_dir}/#{table_name}.yml")
+            end
+
+          pathname.open('w') do |f|
+            f.write fixture.to_yaml
+          end
+        end
+      end
+    end
+  end
+end

data/lib/dumped_railers/fixture_builder/model.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+require_relative 'record'
+
+module DumpedRailers
+  module FixtureBuilder
+    class Model
+      def initialize(model, preprocessors:)
+        @model = model
+        @fixture_records = model.order(:id).map { |record|
+          Record.new(record, model, preprocessors: preprocessors)
+        }
+      end
+  
+      def build!
+        fixture_body = @fixture_records.map(&:build!).to_h
+        fixture = fixture_body.reverse_merge build_fixture_header_for(@model)
+  
+        [@model.table_name, fixture]
+      end
+  
+      private
+
+      def build_fixture_header_for(model)
+        { '_fixture' =>
+          {
+            'model_class'          => model.name,
+            'fixture_generated_by' => 'DumpedRailers',
+          }
+        }
+      end
+    end
+  end
+end

data/lib/dumped_railers/fixture_builder/record.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module DumpedRailers
+  module FixtureBuilder
+    class Record
+      def initialize(record, model, preprocessors:)
+        @record = record
+        @model = model
+        @preprocessors = preprocessors
+      end
+
+      def build!
+        id = @record.id
+        attributes = 
+          @preprocessors.inject(@record.attributes) { |attrs, preprocessor|
+            preprocessor.call(attrs, @model)
+          }
+  
+        # convert "belong_to association" foreign keys into record-unique labels
+        @model.reflect_on_all_associations.select(&:belongs_to?).each do |rel|
+          # skip ignorables
+          next unless attributes.has_key? rel.foreign_key.to_s
+  
+          if rel.polymorphic?
+            model_name = attributes[rel.foreign_type.to_s]
+  
+            attributes[rel.name.to_s] = record_label_for(
+              model_name,
+              attributes.delete(rel.foreign_key.to_s),
+              attributes.delete(rel.foreign_type.to_s)
+            )
+          else
+            attributes[rel.name.to_s] = record_label_for(
+              rel.name,
+              attributes.delete(rel.foreign_key.to_s)
+            )
+          end
+        end
+
+        [record_label_for(@model.name, id), attributes]
+      end
+    
+      private
+  
+      def record_label_for(model_name, id, type=nil)
+        return nil unless id
+
+        identifier = "#{model_name.to_s.underscore}_#{id}"
+        type_specifier = "(#{type})" if type
+  
+        "__#{identifier}#{type_specifier}"
+      end
+    end
+  end
+end

data/lib/dumped_railers/import.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+require_relative 'record_builder/fixture_set'
+
+module DumpedRailers
+  class Import
+    attr_reader :fixture_set
+
+    def initialize(*paths)
+      @raw_fixtures = FileHelper.read_fixtures(*paths)
+      @fixture_set = RecordBuilder::FixtureSet.new(@raw_fixtures)
+    end
+
+    def import_all!
+      fixture_set.sort_by_table_dependencies!
+      @record_sets = fixture_set.build_record_sets!
+
+      ActiveRecord::Base.transaction(joinable: false, requires_new: true) do
+        # models have to be persisted one-by-one so that dependent models are able to 
+        # resolve "belongs_to" (parent) association
+        @record_sets.each do |_model, records|
+          # FIXME: faster implementation wanted, parhaps with activerocord-import
+          # (objects needs to be reloaded somehow when using buik insert)
+          records.each(&:save!)
+        end
+      end
+    end
+  end
+end
+

data/lib/dumped_railers/preprocessor/strip_ignorables.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module DumpedRailers
+  module Preprocessor
+    class StripIgnorables
+      def call(attributes, _model)
+        attributes.reject { |column_name, _v|
+          DumpedRailers.config.ignorable_columns.map(&:to_s).include?(column_name)
+        }
+      end
+    end
+  end
+end

data/lib/dumped_railers/record_builder/dependency_tracker.rb

@@ -0,0 +1,82 @@
+# frozen_string_literal: true
+
+module DumpedRailers
+  module RecordBuilder
+    class DependencyTracker
+      class <<  self
+        def for(model)
+          trackers[model] ||= new
+        end
+
+        def clear!
+          @trackers = {}
+        end
+
+        private
+
+        def trackers
+          @trackers ||= {}
+        end
+      end
+
+      def on(record)
+        dependencies[record] ||= RecordDependency.new
+      end
+
+      def list_all_record_labels_with(attr)
+        list_all_dependencies_with(attr)
+          .map { |dependent| dependent.record_label }
+          .compact
+      end
+
+      def list_all_model_names_with(attr)
+        list_all_dependencies_with(attr)
+          .map { |dependent| dependent.model_name }
+          .compact
+      end
+
+      private
+
+      def dependencies
+        @dependencies ||= {}
+      end
+
+      def list_all_dependencies_with(attr)
+        dependencies
+          .values
+          .map { |record_dependency|
+            record_dependency.with(attr)
+          }
+      end
+
+      class RecordDependency
+        def with(attr)
+          record_dependency[attr.to_sym] ||= DependentObject.new
+        end
+
+        def each_dependent_record_label(&block)
+          return enum_for(:each_dependent_record_label) unless block_given?
+
+          record_dependency.each { |attr, dependent_object|
+            block.call(attr, dependent_object.record_label)
+          }
+        end
+
+        def dependent_record_labels
+          record_dependency.values.map(&:record_label).compact
+        end
+
+        private
+
+        def record_dependency
+          @record_dependency ||= {}
+        end
+
+        class DependentObject
+          attr_accessor :record_label, :model_name
+        end
+      end
+
+    end
+  end
+end

data/lib/dumped_railers/record_builder/fixture_row.rb

@@ -0,0 +1,114 @@
+# frozen_string_literal: true
+
+module DumpedRailers
+  module RecordBuilder
+    class FixtureRow
+      attr_reader :label, :attrs
+  
+      def initialize(label, attrs)
+        @label = label
+        @attrs = attrs
+      end
+
+      def analyze_dependencies!(dependency_tracker)
+        raise RuntimeError, 'Can\'t execute the dependency analysis twice. This has been done already' if @dependency
+
+        @dependency =  dependency_tracker
+
+        attrs.each do |attr, value|
+          ref, model_name = parse_reference_from(value)
+          next unless ref
+  
+          attrs[attr] = ref.to_sym
+          @dependency.with(attr).record_label = ref.to_sym
+          @dependency.with(attr).model_name   = model_name&.to_sym
+        end
+      end
+
+      def instantiate_as!(model)
+        raise RuntimeError, 'Could not find the dependency tracker. Run #analyze_dependencies to instaitiate the records' unless @dependency
+
+        @model = model
+        resolve_reference!
+        object = model.new(attrs)
+        RecordStore.register(label, object: object)
+  
+        object
+      end
+  
+      private
+  
+      def parse_reference_from(val)
+        # NOTE: make sure its object is a string (can be a json object)
+        return unless val.is_a? String
+        return unless val.start_with? '__'
+  
+        # format convention
+        # for non-polymorphic association: __[identifier]
+        # for polymorphic association:     __[identifier]([model_name])
+        ref, _, model_name = val.scan(/\A(__[^(\s]+)(\(([^)]+)\))?\z/).first
+  
+        [ref, model_name]
+      end
+  
+      def resolve_reference!
+        raise RuntimeError, <<~"ERROR_MESSAGE" unless resolvable?
+          cannot resolve dependencies. (some fixtures might be missing)
+            model:  #{@model}
+            record: #{label}
+          ERROR_MESSAGE
+  
+        @dependency.each_dependent_record_label { |attr, record_label|
+          attrs[attr] = RecordStore.retrieve!(record_label)
+        }
+      end
+  
+      def resolvable?
+        @dependency.dependent_record_labels.all? { |label|
+          RecordStore.registered?(label)
+        }
+      end
+
+      class RecordStore
+        class << self
+          def register(label, object:)
+            set_object(label, object)
+          end
+  
+          def registered?(label)
+            !object_for(label).nil?
+          end
+  
+          def retrieve!(label)
+            raise RuntimeError, "couldn't resolve dependent record: #{label}" unless registered?(label)
+  
+            object_for(label)
+          end
+
+          def clear!
+            @repository = {}
+          end
+  
+          private
+  
+          def repository
+            @repository ||= {}
+          end
+  
+          def object_for(label)
+            repository[label.to_sym]
+          end
+
+          def set_object(label, object)
+            repository[label.to_sym] = object
+          end
+        end
+
+        def initialize
+          # use this class as (some sort of) Singleton
+          raise NotImplementedError
+        end
+      end
+    end
+  end
+end

data/lib/dumped_railers/record_builder/fixture_set.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+require_relative 'fixture_table'
+require 'tsort'
+
+module DumpedRailers
+  module RecordBuilder
+    class FixtureSet
+      include TSort
+      attr_reader :fixture_tables, :record_sets
+  
+      def initialize(raw_fixtures)
+        @fixture_tables = raw_fixtures.map { |raw_records| build_fixture_table(raw_records) }
+      end
+  
+      def sort_by_table_dependencies!
+        @fixture_tables.each(&:analyze_metadata_dependencies!)
+        # dependency are sorted in topological order using Active Record reflection
+        @fixture_tables = tsort
+  
+        self
+      end
+  
+      def build_record_sets!
+        @record_sets = @fixture_tables.map { |table|
+          [table.model, table.build_records!]
+        }.to_h
+      end
+  
+      private
+  
+      def build_fixture_table(raw_records)
+        FixtureTable.new(raw_records)
+      end
+  
+      def tsort_each_node(&block)
+        @fixture_tables.each { |table| block.call(table) }
+      end
+  
+      def tsort_each_child(node, &block)
+        dependent_nodes = @fixture_tables.select { |table| node.dependencies.include? table.model_name }
+        dependent_nodes.each &block
+      end
+    end
+  end
+end

data/lib/dumped_railers/record_builder/fixture_table.rb

@@ -0,0 +1,74 @@
+# frozen_string_literal: true
+
+require_relative 'fixture_row'
+require_relative 'dependency_tracker'
+
+module DumpedRailers
+  module RecordBuilder
+    class FixtureTable
+      attr_reader  :model, :model_name, :rows, :objects, :dependencies
+  
+      def initialize(raw_records)
+        config = raw_records.delete('_fixture')
+  
+        @model = identify_model!(config)
+        @model_name = model.name.to_sym
+        @dependency_tracker = DependencyTracker.for(model)
+  
+        @rows = raw_records.map { |label, attrs|
+          build_fixture_row(label, attrs)
+        }
+      end
+
+      def analyze_metadata_dependencies!
+        raise RuntimeError, "Dependency Analysis has already been done with the fixture for #{model_name}" if @dependencies
+
+        rows.map { |row| row.analyze_dependencies!(@dependency_tracker.on(row)) }
+
+        @dependencies = model.reflect_on_all_associations.select(&:belongs_to?).flat_map { |rel|
+          if rel.polymorphic?
+            @dependency_tracker.list_all_model_names_with(rel.name)
+          else
+            rel.class_name.to_sym
+          end
+        }.uniq
+      end
+  
+      def build_records!
+        raise RuntimeError, "The records in this fixture for #{model_name} have been built already" if @instantiated
+  
+        @objects = rows.map { |row| row.instantiate_as!(model) }
+        @instantiated = true
+  
+        objects
+      end
+  
+      private
+
+      def build_fixture_row(label, attrs)
+        FixtureRow.new(
+          label.to_sym,
+          attrs.symbolize_keys,
+        )
+      end
+
+      def identify_model!(config)
+        model_name = config&.dig('model_class')
+        raise RuntimeError, <<~"ERROR_MESSAGE" unless model_name
+          couldn't find `_fixture: model_class` label in the fixture.
+          (possibly not an auto-generated one?)
+        ERROR_MESSAGE
+  
+        model = model_name.safe_constantize
+        return model if model && model < ActiveRecord::Base
+
+        raise RuntimeError, <<~"ERROR_MESSAGE"
+          couldn't find a model named #{model_name} specified with `_fixture: model_class` label in the fixture.
+          you might want to check whether:
+            - this task runs in the same application that the fixtures were generated in
+            - relevant tables have not been altered or dropped since the fixtures were generated
+        ERROR_MESSAGE
+      end
+    end
+  end
+end

data/lib/dumped_railers/version.rb

@@ -0,0 +1,3 @@
+module DumpedRailers
+  VERSION = "0.1.1"
+end

metadata

@@ -0,0 +1,201 @@
+--- !ruby/object:Gem::Specification
+name: dumped_railers
+version: !ruby/object:Gem::Version
+  version: 0.1.1
+platform: ruby
+authors:
+- Koji Onishi
+autorequire: 
+bindir: exe
+cert_chain: []
+date: 2020-12-10 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 12.3.3
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 12.3.3
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.2'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.2'
+- !ruby/object:Gem::Dependency
+  name: database_cleaner-active_record
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.8'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.8'
+- !ruby/object:Gem::Dependency
+  name: pry
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: pry-byebug
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: pry-doc
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: DumpedRailers helps you take a snapshot of ActiveRecord models in Rails-compatible
+  fixture format, and re-import them wherever necessary without destroying current
+  data you have.
+email:
+- fursich0@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".rspec"
+- ".travis.yml"
+- CHANGELOG.md
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- bin/console
+- bin/setup
+- dumped_railers.gemspec
+- gemfiles/Gemfile.rails_5.2.4
+- gemfiles/Gemfile.rails_6.0.3.4
+- gemfiles/Gemfile.rails_6.1.0
+- lib/dumped_railers.rb
+- lib/dumped_railers/dump.rb
+- lib/dumped_railers/file_helper.rb
+- lib/dumped_railers/fixture_builder/model.rb
+- lib/dumped_railers/fixture_builder/record.rb
+- lib/dumped_railers/import.rb
+- lib/dumped_railers/preprocessor/strip_ignorables.rb
+- lib/dumped_railers/record_builder/dependency_tracker.rb
+- lib/dumped_railers/record_builder/fixture_row.rb
+- lib/dumped_railers/record_builder/fixture_set.rb
+- lib/dumped_railers/record_builder/fixture_table.rb
+- lib/dumped_railers/version.rb
+homepage: https://github.com/fursich/dumped_railers
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/fursich/dumped_railers
+  source_code_uri: https://github.com/fursich/dumped_railers
+  changelog_uri: https://github.com/fursich/dumped_railers/CHANGELOG.md
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 2.3.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.0.3
+signing_key: 
+specification_version: 4
+summary: A flexible fixture importer/exporter, that can transport ActiveRecord data
+  in fixture format
+test_files: []