database_flusher 0.1.1 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 43b8589940f4be6cf29476431a5a59fefce4b3e5
-  data.tar.gz: a1d467168f8d33c47b0681d04b57fd58cc51f2a4
+  metadata.gz: 4ae582039a16a7f6b25c12b09683663d3061c9fb
+  data.tar.gz: ee031d973bf006c2640d8bae58b8aee97457c617
 SHA512:
-  metadata.gz: b7d0f5e9c7375dc85199d07f063b11744a3395e3cfbc0d432126ac84996e23600f68aae2673bd68d237942b242a393cf9d16f3085912cad3fbe67dee33f6dd08
-  data.tar.gz: d9a54465da056726d13f7800f79d9af1f8abf81b148d2d1b09532ba2dadb1256d8e0fddcdf0e1dc371489cc1efbd47a65bbdc666a65b2d1ca89985e7a8ac0202
+  metadata.gz: 489db40d248249811e8f16420dfda4c02a2e3b42bd2c709028061b371c8efcc3b2ab4c8e584a9f0c0f687396ce3aeb881fb5195933c9954409fb3d72f2774e5c
+  data.tar.gz: 9b039b7933fa803c298719792e74f5255c4b4f13e6310207777fe35c32dcd7580b833430e13763b3a5891673abee6fd4fe603219e4ed6ec90a90c12949fcf73a

data/.gitignore

@@ -7,3 +7,5 @@
 /pkg/
 /spec/reports/
 /tmp/
+/*.gem
+/.byebug_history

data/.travis.yml

@@ -1,5 +1,17 @@
 sudo: false
 language: ruby
+cache: bundler
+
+services:
+  - mongodb
+
 rvm:
   - 2.2.4
+  - 2.3.3
+
+env:
+  - DB=sqlite3
+  - DB=mysql2
+  - DB=postgresql
+
 before_install: gem install bundler -v 1.13.3

data/README.md

@@ -1,8 +1,23 @@
 # DatabaseFlusher
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/database_flusher`. To experiment with that code, run `bin/console` for an interactive prompt.
+[![Build Status](https://travis-ci.org/ebeigarts/database_flusher.svg?branch=master)](https://travis-ci.org/ebeigarts/database_flusher)
+
+database_flusher is a tiny and fast database cleaner inspired by [database_cleaner](https://github.com/DatabaseCleaner/database_cleaner) and [database_rewinder](https://github.com/amatsuda/database_rewinder).
+
+## Features
+
+* No monkey patching - uses `ActiveSupport::Notifications` and `Mongo::Monitoring::Global` to catch `INSERT` statements
+* Fast `:deletion` strategy that cleans only tables/collections where `INSERT` statements were performed
+* Faster `disable_referential_integrity` for PostgreSQL
+* Executes multiple `DELETE` statements as one query with ActiveRecord
+
+## Supported ORMs and strategies
+
+| ORM          | Deletion | Transaction |
+|:-------------|:---------|:------------|
+| ActiveRecord | Yes      | Yes         |
+| Mongoid      | Yes      | No          |
 
-TODO: Delete this and the text above, and describe your gem
 
 ## Installation
 
@@ -14,28 +29,68 @@ gem 'database_flusher'
 
 And then execute:
 
-    $ bundle
-
-Or install it yourself as:
-
-    $ gem install database_flusher
+```bash
+$ bundle
+```
 
 ## Usage
 
-TODO: Write usage instructions here
+RSpec:
 
-## Development
+```ruby
+RSpec.configure do |config|
+  config.use_transactional_fixtures = false
+
+  config.before :suite do
+    DatabaseFlusher[:active_record].strategy = :transaction
+    DatabaseFlusher[:mongoid].strategy = :deletion
+  end
+
+  config.before :each do
+    DatabaseFlusher[:active_record].strategy = :transaction
+  end
+
+  config.before :each, type: :feature do
+    if Capybara.current_driver != :rack_test
+      DatabaseFlusher[:active_record].strategy = :deletion
+    end
+  end
+
+  config.before :each do
+    DatabaseFlusher.start
+  end
+
+  config.append_after :each do
+    DatabaseFlusher.clean
+  end
+end
+```
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+Cucumber:
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+```ruby
+DatabaseFlusher[:active_record].strategy = :transaction
+DatabaseFlusher[:mongoid].strategy = :deletion
+
+Before('~@javascript') do
+  DatabaseFlusher[:active_record].strategy = :transaction
+  DatabaseFlusher.start
+end
+
+Before('@javascript') do
+  DatabaseFlusher[:active_record].strategy = :deletion
+  DatabaseFlusher.start
+end
+
+After do
+  DatabaseFlusher.clean
+end
+```
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/database_flusher. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
-
+Bug reports and pull requests are welcome on GitHub at https://github.com/ebeigarts/database_flusher. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
 
 ## License
 
 The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
-

data/database_flusher.gemspec

@@ -4,22 +4,32 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 require 'database_flusher/version'
 
 Gem::Specification.new do |spec|
-  spec.name          = "database_flusher"
+  spec.name          = 'database_flusher'
   spec.version       = DatabaseFlusher::VERSION
-  spec.authors       = ["Edgars Beigarts"]
-  spec.email         = ["edgars.beigarts@gmail.com"]
+  spec.authors       = ['Edgars Beigarts']
+  spec.email         = ['edgars.beigarts@gmail.com']
 
   spec.summary       = %q{super-fast database cleaner}
   spec.description   = spec.summary
-  spec.homepage      = "https://github.com/ebeigarts/database_flusher"
-  spec.license       = "MIT"
+  spec.homepage      = 'https://github.com/ebeigarts/database_flusher'
+  spec.license       = 'MIT'
 
   spec.files         = `git ls-files -z`.split("\x0").reject do |f|
     f.match(%r{^(test|spec|features)/})
   end
-  spec.require_paths = ["lib"]
+  spec.require_paths = ['lib']
 
-  spec.add_development_dependency "bundler", "~> 1.13"
-  spec.add_development_dependency "rake", "~> 10.0"
-  spec.add_development_dependency "rspec", "~> 3.0"
+  spec.required_ruby_version = '>= 2.2.1'
+
+  spec.add_development_dependency 'bundler', '~> 1.13'
+  spec.add_development_dependency 'rake', '~> 10.0'
+  spec.add_development_dependency 'rspec', '~> 3.0'
+  spec.add_development_dependency 'byebug'
+
+  spec.add_development_dependency 'activerecord'
+  spec.add_development_dependency 'sqlite3'
+  spec.add_development_dependency 'mysql2'
+  spec.add_development_dependency 'pg'
+
+  spec.add_development_dependency 'mongoid'
 end

data/lib/database_flusher/active_record/adapters/abstract_adapter.rb

@@ -0,0 +1,40 @@
+module DatabaseFlusher
+  module ActiveRecord
+    class AbstractAdapter
+      attr_reader :connection, :raw_connection
+
+      def initialize(connection)
+        @connection = connection
+        @raw_connection = connection.raw_connection
+      end
+
+      def delete(*tables)
+        disable_referential_integrity(*tables) do
+          stmts = tables.map do |name|
+            "DELETE FROM #{quote_table_name(name)}"
+          end
+          sql = stmts.join(';')
+          execute_multi sql
+        end
+      end
+
+      private
+
+      def execute_multi(sql)
+        connection.execute sql
+      end
+
+      def execute(sql)
+        connection.execute sql
+      end
+
+      def quote_table_name(name)
+        connection.quote_table_name(name)
+      end
+
+      def disable_referential_integrity(*tables, &block)
+        connection.disable_referential_integrity(&block)
+      end
+    end
+  end
+end

data/lib/database_flusher/active_record/adapters/mysql2_adapter.rb

@@ -0,0 +1,25 @@
+module DatabaseFlusher
+  module ActiveRecord
+    class Mysql2Adapter < AbstractAdapter
+      def initialize(connection)
+        super
+        flags = raw_connection.query_options[:flags]
+        unless flags.include?('MULTI_STATEMENTS'.freeze)
+          raise 'MULTI_STATEMENTS flag is not enabled'
+        end
+      end
+
+      private
+
+      def execute_multi(sql)
+        connection.disable_referential_integrity do
+          _result = raw_connection.query sql
+          while raw_connection.next_result
+            # just to make sure that all queries are finished
+            _result = raw_connection.store_result
+          end
+        end
+      end
+    end
+  end
+end

data/lib/database_flusher/active_record/adapters/postgresql_adapter.rb

@@ -0,0 +1,22 @@
+module DatabaseFlusher
+  module ActiveRecord
+    class PostgreSQLAdapter < AbstractAdapter
+      private
+
+      def disable_referential_integrity(*tables, &block)
+        begin
+          execute(tables.collect { |name| "ALTER TABLE #{quote_table_name(name)} DISABLE TRIGGER ALL" }.join(";"))
+        rescue
+          execute(tables.collect { |name| "ALTER TABLE #{quote_table_name(name)} DISABLE TRIGGER USER" }.join(";"))
+        end
+        yield
+      ensure
+        begin
+          execute(tables.collect { |name| "ALTER TABLE #{quote_table_name(name)} ENABLE TRIGGER ALL" }.join(";"))
+        rescue
+          execute(tables.collect { |name| "ALTER TABLE #{quote_table_name(name)} ENABLE TRIGGER USER" }.join(";"))
+        end
+      end
+    end
+  end
+end

data/lib/database_flusher/active_record/adapters/sqlite_adapter.rb

@@ -0,0 +1,11 @@
+module DatabaseFlusher
+  module ActiveRecord
+    class SQLiteAdapter < AbstractAdapter
+      private
+
+      def execute_multi(sql)
+        raw_connection.execute_batch sql
+      end
+    end
+  end
+end

data/lib/database_flusher/active_record/deletion_strategy.rb

@@ -0,0 +1,59 @@
+require 'database_flusher/active_record/adapters/abstract_adapter'
+require 'database_flusher/active_record/adapters/mysql2_adapter'
+require 'database_flusher/active_record/adapters/postgresql_adapter'
+require 'database_flusher/active_record/adapters/sqlite_adapter'
+
+module DatabaseFlusher
+  module ActiveRecord
+    class DeletionStrategy
+      attr_reader :tables, :adapter
+
+      class Subscriber
+        def initialize(strategy)
+          @strategy = strategy
+        end
+
+        def call(_, _, _, _, payload)
+          sql = payload[:sql]
+          match = sql.match(/\A\s*INSERT(?:\s+IGNORE)?(?:\s+INTO)?\s+(?:\.*[`"]?([^.\s`"]+)[`"]?)*/i)
+          return unless match
+          table = match[1]
+          if table
+            @strategy.tables << table
+          end
+        end
+      end
+
+      def initialize
+        @tables = Set.new
+      end
+
+      def start
+        @subscriber ||= ActiveSupport::Notifications.subscribe(
+          'sql.active_record',
+          Subscriber.new(self)
+        )
+        connection = ::ActiveRecord::Base.connection
+        @adapter = DatabaseFlusher::ActiveRecord.
+          const_get("#{connection.adapter_name}Adapter").
+          new(connection)
+      end
+
+      def stop
+        if @subscriber
+          ActiveSupport::Notifications.unsubscribe(@subscriber)
+          @subscriber = nil
+        end
+      end
+
+      def clean
+        return if tables.empty?
+
+        # puts "Cleaning #{tables.inspect}"
+        adapter.delete(*tables)
+
+        tables.clear
+      end
+    end
+  end
+end

data/lib/database_flusher/active_record/transaction_strategy.rb

@@ -0,0 +1,24 @@
+module DatabaseFlusher
+  module ActiveRecord
+    class TransactionStrategy
+      def start
+        # Hack to make sure that the connection is properly setup for
+        # the clean code.
+        ::ActiveRecord::Base.connection.transaction{ }
+
+        ::ActiveRecord::Base.connection.begin_transaction joinable: false
+      end
+
+      def stop
+        ::ActiveRecord::Base.connection_pool.connections.each do |connection|
+          next unless connection.open_transactions > 0
+          connection.rollback_transaction
+        end
+      end
+
+      def clean
+        stop
+      end
+    end
+  end
+end

data/lib/database_flusher/cleaner.rb

@@ -0,0 +1,57 @@
+module DatabaseFlusher
+  class Cleaner
+    attr_reader :strategy
+
+    def initialize(orm)
+      @orm = orm
+      reset_strategy
+    end
+
+    def strategy=(name)
+      strategy_changed = name != @strategy_name
+
+      stop if strategy_changed
+
+      if name
+        create_strategy(name) if strategy_changed
+      else
+        reset_strategy
+      end
+    end
+
+    def clean_with(name)
+      self.strategy = name
+      strategy.clean
+    end
+
+    def start
+      strategy.start
+    end
+
+    def stop
+      strategy.stop
+    end
+
+    def clean
+      strategy.clean
+    end
+
+    private
+
+    def create_strategy(name)
+      @strategy_name = name
+      @strategy = DatabaseFlusher.
+        const_get(classify(@orm)).
+        const_get("#{classify(name)}Strategy").new
+    end
+
+    def reset_strategy
+      @strategy_name = nil
+      @strategy = DatabaseFlusher::NullStrategy.new
+    end
+
+    def classify(name)
+      name.to_s.split('_').collect(&:capitalize).join
+    end
+  end
+end

data/lib/database_flusher/mongoid/deletion_strategy.rb

@@ -0,0 +1,52 @@
+module DatabaseFlusher
+  module Mongoid
+    class DeletionStrategy
+      attr_reader :collections
+
+      class Subscriber
+        def initialize(strategy)
+          @strategy = strategy
+        end
+
+        def started(event)
+          if event.command_name == :insert || event.command_name == 'insert'.freeze
+            collection = event.command['insert'.freeze]
+            if collection
+              @strategy.collections << collection
+            end
+          end
+        end
+
+        private
+
+        def method_missing(*args, &block)
+        end
+      end
+
+      def initialize
+        @collections = Set.new
+      end
+
+      def start
+        @subscriber ||= Mongo::Monitoring::Global.subscribe(
+          Mongo::Monitoring::COMMAND,
+          Subscriber.new(self)
+        )
+      end
+
+      def stop
+        raise NotImplementedError, "Mongo doesn't provide unsubscribe"
+      end
+
+      def clean
+        return if collections.empty?
+        # puts "Cleaning #{collections.inspect}"
+        collections.each do |name|
+          client = ::Mongoid::Clients.default
+          client[name].delete_many
+        end
+        collections.clear
+      end
+    end
+  end
+end

data/lib/database_flusher/null_strategy.rb

@@ -0,0 +1,7 @@
+module DatabaseFlusher
+  class NullStrategy
+    def start; end
+    def stop; end
+    def clean; end
+  end
+end

data/lib/database_flusher/version.rb

@@ -1,3 +1,3 @@
 module DatabaseFlusher
-  VERSION = "0.1.1"
+  VERSION = '0.2.0'.freeze
 end

data/lib/database_flusher.rb

@@ -1,5 +1,12 @@
-require "database_flusher/version"
-require "set"
+require 'set'
+
+require 'database_flusher/version'
+require 'database_flusher/null_strategy'
+require 'database_flusher/cleaner'
+
+require 'database_flusher/active_record/deletion_strategy'
+require 'database_flusher/active_record/transaction_strategy'
+require 'database_flusher/mongoid/deletion_strategy'
 
 module DatabaseFlusher
   extend self
@@ -8,96 +15,19 @@ module DatabaseFlusher
     @cleaners ||= {}
   end
 
-  # TODO: classify
   def [](name)
-    cleaners[name] ||= DatabaseFlusher.const_get("#{name.to_s.classify}Cleaner").new
-  end
-
-  def clean
-    cleaners.values.each { |cleaner| cleaner.clean }
-  end
-
-  class MongoSubscriber
-    def initialize(cleaner)
-      @cleaner = cleaner
-    end
-
-    def started(event)
-      if event.command_name == :insert || event.command_name == 'insert'.freeze
-        collection = event.command['insert'.freeze]
-        if collection
-          @cleaner.collections << collection
-        end
-      end
-    end
-
-    private
-
-    def method_missing(*args, &block)
-      # puts args.inspect
-    end
+    cleaners[name] ||= DatabaseFlusher::Cleaner.new(name)
   end
 
-  class MongoCleaner
-    attr_accessor :strategy
-    attr_reader :collections
-
-    def initialize
-      @collections = Set.new
-      Mongo::Monitoring::Global.subscribe(
-        Mongo::Monitoring::COMMAND,
-        MongoSubscriber.new(self)
-      )
-    end
-
-    def clean
-      return unless strategy
-      client = Mongoid::Clients.default
-      # puts "Cleaning #{collections.inspect}"
-      collections.each do |name|
-        client[name].delete_many
-      end
-      collections.clear
-    end
+  def start
+    cleaners.values.each(&:start)
   end
 
-  class ActiveRecordSubscriber
-    def initialize(cleaner)
-      @cleaner = cleaner
-    end
-
-    def call(_, _, _, _, payload)
-      sql = payload[:sql]
-      match = sql.match(/\A\s*INSERT(?:\s+IGNORE)?(?:\s+INTO)?\s+(?:\.*[`"]?([^.\s`"]+)[`"]?)*/i)
-      return unless match
-      table = match[1]
-      if table
-        @cleaner.tables << table
-      end
-    end
+  def stop
+    cleaners.values.each(&:stop)
   end
 
-  class ActiveRecordCleaner
-    attr_accessor :strategy
-    attr_reader :tables
-
-    def initialize
-      @tables = Set.new
-      ActiveSupport::Notifications.subscribe(
-        'sql.active_record',
-        ActiveRecordSubscriber.new(self)
-      )
-    end
-
-    def clean
-      return unless strategy
-      connection = ActiveRecord::Base.connection
-      # puts "Cleaning #{tables.inspect}"
-      connection.disable_referential_integrity do
-        stmts = tables.map { |t| "DELETE FROM #{connection.quote_table_name(t)}" }
-        connection.execute stmts.join('; ')
-      end
-      tables.clear
-    end
+  def clean
+    cleaners.values.each(&:clean)
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: database_flusher
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.2.0
 platform: ruby
 authors:
 - Edgars Beigarts
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-12-09 00:00:00.000000000 Z
+date: 2016-12-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -52,6 +52,90 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: byebug
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: mysql2
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: pg
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: mongoid
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: super-fast database cleaner
 email:
 - edgars.beigarts@gmail.com
@@ -71,6 +155,15 @@ files:
 - bin/setup
 - database_flusher.gemspec
 - lib/database_flusher.rb
+- lib/database_flusher/active_record/adapters/abstract_adapter.rb
+- lib/database_flusher/active_record/adapters/mysql2_adapter.rb
+- lib/database_flusher/active_record/adapters/postgresql_adapter.rb
+- lib/database_flusher/active_record/adapters/sqlite_adapter.rb
+- lib/database_flusher/active_record/deletion_strategy.rb
+- lib/database_flusher/active_record/transaction_strategy.rb
+- lib/database_flusher/cleaner.rb
+- lib/database_flusher/mongoid/deletion_strategy.rb
+- lib/database_flusher/null_strategy.rb
 - lib/database_flusher/version.rb
 homepage: https://github.com/ebeigarts/database_flusher
 licenses:
@@ -84,7 +177,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 2.2.1
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="