quickening 0.1.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 8f984164b807e1a51e7096822c12774f45251ccf
+  data.tar.gz: 58be26b882800907ef800820f0d6a6d003177efc
+SHA512:
+  metadata.gz: b434a24e36457c65d4649bbd0ef80a67911d65486508581495e767623e97b90d23efe0b90dbc88eea20a910a323c5fac94b7b2602832fb0dc7f49710513f2ea7
+  data.tar.gz: c51609c7df98bd8772ef0bd3bb8002330518175f151f68e2b005eaea6694a16b529fc69b9ef978de37e75cd5b50f20ed8a8cf6ea3ac2e8a5c111a839365eaf25

data/CHANGELOG.rdoc

@@ -0,0 +1,54 @@
+== 0.1.1
+
+* Just commemorating moving away from a stupid name to a less silly one.
+
+== 0.1.0
+
+* Bugfix
+  * Results were looking correct for the total set <tt>User.duplicate(force: true)</tt> and <tt>User.duplicate.copies</tt> but there were no results returned for <tt>User.duplicate.originals</tt>. Re-oriented the direction of the joins to be more straightforward and now the results have been consistent from my tests against data.
+
+* Todo
+  * Come up with a less silly name?
+  * Update: Changed name of gem
+
+== 0.0.3
+
+* Maintenance
+  * RDoc inline documentation in code
+  * Removing various extraneous development dependencies in gemspec
+  * Cleaning out commented-out code
+  * Separating ActiveRecord extension to ORM file
+  * Extracting shared spec behaviors to spec/support
+  * Correctly requiring the spec/support files
+  * UTF-8 encoding designations
+  * Preparing Rake task namespace
+  * Preparing base-module level setups
+  * Corrected mislinked script/rails
+  * Preparing integration with an actual app
+
+== 0.0.2
+
+* Features
+  * No longer need to require/include/set things to integrate with your model. Instead you can call the class method <tt>clone_wars(..)</tt>
+  * Setting up Rails Engine hooks into the Rails load process
+  * Significant improvement of SQL logic for determining whether or not records are duplicates
+  * Much safer way of determining subsets of the overarching query
+
+* Maintenance
+  * Improved spec coverage over core queries
+  * Setup for upcoming usage of simplecov
+  * Some adjustments to spec_helper and Guardfile for Spork (still need to finish defining reload-hooks for Spork-Guard)
+  * Push to GitHub
+  * Integration with FactoryGirl for testing
+  * Dummy app linked for better coverage with an actual app
+
+* Bugfixes
+  * Typos and minutiae in documentation
+  * Query for selecting just the originals of duplicates was flawed and unreliable
+
+== 0.0.1
+
+* Initial release
+  * Set up of the engine structure, hierarchy of folders, etc.
+  * Ironing out gemspec having given up integrating Jeweler.
+  * Integration of RSpec, Guard, Spork, testing frameworks in general.

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright 2013 caleon
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,145 @@
+# the Quickening
+
+Quickening is a Rails gem for adding to your model a facility to query and manage duplicate records. It's been written to remain relatively abstract and adaptable to various models, and so the library should be an easy plugin for models you may have set up already (barring name clashes).
+
+Beyond the abstracted query methods for searching efficiently throughout the table (but only tested against MySQL 5.5, sorry), your models gain access to methods for dispending with duplicates, chores varying in complexity ranging from the trivial deletes to customizable merges (future feature).
+
+This gem was built against Rails 3.2.12 on Ruby 2.0.0-p0 (although only using things available as of 1.9.3). Rails 3.1 doesn't handle the `uniq` relational
+query method, but that should not matter. I don't believe `from` is handled in either 3.0 or 3.1, so that might be a showstopper with pre-3.2 Rails setups. Ruby 1.9 syntaxes are prevalent, so this will not be compatible with Ruby 1.8, either.
+
+But if you see the utility of this sort of gem, please feel free to contribute and help out.
+
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'quickening'
+# or for edge
+gem 'quickening', github: 'caleon/quickening'
+```
+
+And then execute:
+
+```bash
+$ bundle
+```
+
+Setup your model one way (the old way):
+
+```ruby
+require 'quickening'
+
+class User < ActiveRecord::Base
+  include Quickening::Model
+  self.duplicate_matchers = %w(name code).map(&:to_sym)
+  ..
+end
+```
+
+...or the other way (new way, using Rails Engines):
+
+```ruby
+class User < ActiveRecord::Base
+  quickening :name, :code
+  ..
+end
+```
+
+And in your migration:
+
+```ruby
+add_index :users, [:name, :code]
+```
+
+The order by which your composite index is defined should match that of your class attribute value.
+
+## Usage
+
+### Retrieve all non-unique records
+
+```ruby
+> User.duplicate
+# => []
+
+> User.duplicate(force: true)
+# => [#<User id: 1 name: 'Bruce Wayne', code: nil ..>,
+      #<User id: 2 name: 'Bruce Wayne', code: nil ..>,
+      #<User id: 3 name: 'Syrio Forel', code: 50, died_on: "2013-03-01" ..>,
+      #<User id: 4 name: 'syrio forel', code: 50, died_on: nil ..>,
+      #<User id: 5 name: 'Marla Singer', code: 32 ..>,
+      #<User id: 7 name: 'tylerdurden', code: 1 ..>,
+      #<User id: 8 name: 'tyler durden', code: 1 ..>]
+```
+
+The conditions of their non-uniqueness is determined by checking the rest of the table using the `User.duplicate_matches` setting (which works with an Array only, with no special provisions yet for Procs or anything). If a record exists which is identical on all those fields, it is a part of the returned value. This means that both the "original" record as well as its (potentially multiple) copies will be included.
+
+#### Force: true
+
+Note that because tables requiring these operations could get potentially large, and because this library does not assume that you have properly applied indexes to the columns used for operational queries, the computation of this finder may be too strenuous. To prevent accidental triggers of this method which, on its own, provides less utility than the chained methods described below, it is initially restricted with `limit(0)` which will then be unrestricted when the follow-up methods are called.
+
+Obviously you can override this yourself with something like `except(:limit)` or by calling another limit. Alternatively, you can pass the option for `force: true` to the method.
+
+
+### Retrieve just the originals
+
+```ruby
+> User.duplicate.originals
+# => [#<User id: 1 name: 'Bruce Wayne', code: nil ..>,
+      #<User id: 3 name: 'Syrio Forel', code: 50, died_on: "2013-03-01" ..>]
+```
+
+So far as this initial version is concerned, the "originality" is determined by returning the record with the lowest ID among the matches. Also, it is not concerned a duplicate-original if it is a unique record to begin with.
+
+Note that this does not return *the* original, since among many sets of matches, there is no single "original". Thus, to act on one original record out of a particular set of duplicates, you would need to scope down the returned set as follows:
+
+```ruby
+> User.duplicate.originals.where(name: 'Bruce Wayne').first
+# => [#<User id: 1 name: 'Bruce Wayne', code: nil ..>]
+```
+
+For the sake of clarity, there should later be an aptly-named method for such individual cases.
+
+
+### Retrieve just the copies
+
+```ruby
+> User.duplicate.copies
+# => [#<User id: 2 name: 'Bruce Wayne', code: nil ..>,
+      #<User id: 4 name: 'syrio forel', code: 50, died_on: nil ..>]
+```
+
+This can otherwise be described as the set of all duplicated records without the original records:
+
+```ruby
+> User.duplicate.copies == User.duplicate(force: true) - User.duplicate.originals
+# => true
+```
+
+## Future
+
+1. ~~Consider a class method as an alternative to requiring-including-setting.~~
+2. Lower the version dependency for Rails (indirectly via ActiveRecord/ActiveSupport)
+3. *Perhaps* rewrite hash syntaxes to allow Ruby 1.8 compatibility...
+4. Write more utility functions for dealing with duplicates.
+5. Allow customization of how to determine a record's "originality".
+6. Create generators for automatically inputting the required lines into a model file as well as a new migration for adding indices to the appropriate columns.
+7. Setup faux model classes to allow an instance of a returned set to behave in a special way, distinguishing it from normal records.
+8. Further avoid MySQL-specific code and test against Postgres, SQLite, etc.
+9. Controller at the engine- or Rack- level for pre-made administrative interface for managing and reporting duplicates.
+10. Ability to turn on caching of duplicates per model instance.
+
+## Contributing to the Quickening
+
+* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
+* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
+* Fork the project.
+* Start a feature/bugfix branch.
+* Commit and push until you are happy with your contribution.
+* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
+* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
+
+## Copyright
+
+Copyright (c) 2013 caleon. See MIT-LICENSE for further details.

data/Rakefile

@@ -0,0 +1,36 @@
+#!/usr/bin/env rake
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+begin
+  require 'rdoc/task'
+rescue LoadError
+  require 'rdoc/rdoc'
+  require 'rake/rdoctask'
+  RDoc::Task = Rake::RDocTask
+end
+
+RDoc::Task.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'the Quickening'
+  rdoc.options << '--line-numbers'
+  rdoc.rdoc_files.include('CHANGELOG.rdoc', 'README.md')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+APP_RAKEFILE = File.expand_path('../spec/dummy/Rakefile', __FILE__)
+load 'rails/tasks/engine.rake'
+
+Bundler::GemHelper.install_tasks
+
+Dir[File.join(File.dirname(__FILE__), 'tasks/**/*.rake')].each { |f| load f }
+
+require 'rspec/core'
+require 'rspec/core/rake_task'
+
+desc 'Run all specs in spec directory (excluding plugin specs)'
+RSpec::Core::RakeTask.new(:spec => 'app:db:test:prepare')
+
+task :default => :spec

data/app/assets/javascripts/quickening/application.js

@@ -0,0 +1,15 @@
+// This is a manifest file that'll be compiled into application.js, which will include all the files
+// listed below.
+//
+// Any JavaScript/Coffee file within this directory, lib/assets/javascripts, vendor/assets/javascripts,
+// or vendor/assets/javascripts of plugins, if any, can be referenced here using a relative path.
+//
+// It's not advisable to add code directly here, but if you do, it'll appear at the bottom of the
+// the compiled file.
+//
+// WARNING: THE FIRST BLANK LINE MARKS THE END OF WHAT'S TO BE PROCESSED, ANY BLANK LINE SHOULD
+// GO AFTER THE REQUIRES BELOW.
+//
+//= require jquery
+//= require jquery_ujs
+//= require_tree .

data/app/assets/stylesheets/quickening/application.css

@@ -0,0 +1,13 @@
+/*
+ * This is a manifest file that'll be compiled into application.css, which will include all the files
+ * listed below.
+ *
+ * Any CSS and SCSS file within this directory, lib/assets/stylesheets, vendor/assets/stylesheets,
+ * or vendor/assets/stylesheets of plugins, if any, can be referenced here using a relative path.
+ *
+ * You're free to add application-wide styles to this file and they'll appear at the top of the
+ * compiled file, but it's generally better to create a new file per style scope.
+ *
+ *= require_self
+ *= require_tree .
+ */

data/app/controllers/quickening/application_controller.rb

@@ -0,0 +1,4 @@
+module Quickening
+  class ApplicationController < ActionController::Base
+  end
+end

data/app/helpers/quickening/application_helper.rb

@@ -0,0 +1,4 @@
+module Quickening
+  module ApplicationHelper
+  end
+end

data/app/views/layouts/quickening/application.html.erb

@@ -0,0 +1,14 @@
+<!DOCTYPE html>
+<html>
+<head>
+  <title>Quickening</title>
+  <%= stylesheet_link_tag    "quickening/application", :media => "all" %>
+  <%= javascript_include_tag "quickening/application" %>
+  <%= csrf_meta_tags %>
+</head>
+<body>
+
+<%= yield %>
+
+</body>
+</html>

data/config/routes.rb

@@ -0,0 +1,2 @@
+Quickening::Engine.routes.draw do
+end

data/lib/quickening/engine.rb

@@ -0,0 +1,16 @@
+# encoding: utf-8
+
+module Quickening # :nodoc:
+
+  class Engine < ::Rails::Engine # :nodoc:
+    isolate_namespace Quickening
+
+    config.quickening = Quickening
+
+    initializer 'quickening.active_record' do
+      ActiveSupport.on_load :active_record do
+        require 'quickening/orm/active_record'
+      end
+    end
+  end
+end

data/lib/quickening/model.rb

@@ -0,0 +1,178 @@
+# encoding: utf-8
+
+require 'active_support/concern'
+
+module Quickening
+  # == Quickening::Model
+  #
+  # Module to include within your ActiveRecord::Base class definitions, either
+  # manually or via the +quickening+ class method.
+  module Model
+    extend ActiveSupport::Concern
+
+    included do
+      # The <tt>limit(0)</tt> default exists now to prevent any inadvertent calls
+      # to what would amount to a very taxing call, for those on an app hooked
+      # to a large database. Besides, there is more utility to be had when
+      # following the scope call with one of the extensions.
+      #
+      #--
+      # I have avoided the practice of using a join table to create a self-
+      # referential association. Barring significant reasons to do so, a simple
+      # alias to itself appears to hold more promise of an elegant solution.
+      # Obviously there is a limit to how far SQL alone can go in terms of
+      # providing us an efficient way to query these things, and those options
+      # will be explored over time.
+      #
+      # In v0.0.1 the ARel methods and objects were directly utilized to avoid
+      # needing to hardcode table alias names. But even if that worked
+      # swimmingly as far as ARel was concerned, the interplay between it and
+      # ActiveRecord rendered these methods ineffective or even broken. Ended
+      # up interpolating code directly into strings, for a later time when a
+      # better solution is pursued.
+      #
+      # Also it's worth considering making +duplicate_matchers+ unwritable even
+      # at the class level once it's been set via +quickening+. The fact that it
+      # would be inserted as a raw string in the midst of a query is undesirable
+      # from a security standpoint.
+      #++
+      #
+      # Note the scope name is not pluralized ("duplicate", not "duplicates"),
+      # and a good way to think of this is to think of the word as an adjective
+      # to deocorate either of the follow-up methods.
+      #
+      # There is a degree of caution required when depending on these scopes and
+      # methods. Note that the <tt>#originals</tt> method performs a grouping
+      # query, and depending on your usage, it may interject its own overriding
+      # SELECT statements or table/column aliases.
+      #
+      # === Examples
+      #
+      #   User.duplicate # => []
+      #
+      #   User.duplicate(force: true)
+      #   => [#<User id: 1 name: 'Bruce Wayne', code: nil ..>,
+      #       #<User id: 2 name: 'Bruce Wayne', code: nil ..>,
+      #       #<User id: 3 name: 'Syrio Forel', code: 50 died_on: "2013-03-01" ..>,
+      #       #<User id: 4 name: 'syrio forel', code: 50 died_on: nil ..>,
+      #       #<User id: 5 name: 'Marla Singer', code: 32 ..>,
+      #       #<User id: 7 name: 'tylerdurden', code: 1 ..>,
+      #       #<User id: 8 name: 'tyler durden', code: 1 ..>]
+      # scope :duplicate, ->(opts = {}) {
+      #   select("`#{table_name}`.*").uniq.from("`#{table_name}` a2").
+      #   joins("INNER JOIN `#{table_name}` USING (#{duplicate_matchers * ', '})").
+      #   where("`#{table_name}`.`id` != `a2`.`id`").
+      #   order("`#{table_name}`.`id`").
+      #   limit(opts[:force] ? nil : 0)
+      # } do
+
+      scope :duplicate, ->(opts = {}) {
+        select("`#{table_name}`.*").
+        uniq.
+        joins("INNER JOIN `#{table_name}` a2 USING (#{duplicate_matchers * ', '})").
+        where("`#{table_name}`.`id` != `a2`.`id`").
+        order("`#{table_name}`.`id`").
+        limit(opts[:force] ? nil : 0)
+      } do
+
+
+        ##
+        # Returns a collection of all originals within each respective set of
+        # duplicates. Make sure that part was clear. A read of the RSpec tests
+        # with the "documentation" formatter may be assistive in clarifying the
+        # intend of these methods.
+        #
+        #   User.duplicate.originals
+        #   # => [#<User id: 1 name: 'Bruce Wayne', code: nil ..>,
+        #         #<User id: 3 name: 'Syrio Forel', code: 50 died_on: "2013-03-01" ..>]
+        def originals
+          except(:limit).group(duplicate_matchers).
+            having("`#{table_name}`.`id` = MIN(`#{table_name}`.`id`)")
+        end
+
+        ##
+        #   User.duplicate.copies
+        #   # => [#<User id: 2 name: 'Bruce Wayne', code: nil ..>,
+        #         #<User id: 4 name: 'syrio forel', code: 50 died_on: nil ..>]
+        def copies
+          # except(:limit).where("`a2`.`id` < `#{table_name}`.`id`")
+          except(:limit).where("`#{table_name}`.`id` > `a2`.`id`")
+        end
+      end
+    end
+
+    module ClassMethods #:nodoc:
+      # Looks for other records in the same table for items matching on all
+      # pre-defined columns. This has little benefit of usage except to act as
+      # a proxy for the instance-level methods, such as <tt>#duplicates</tt>.
+      #
+      #   User.find_duplicates_for(user)
+      #   # => [#<User id: 2 ..>]
+      def find_duplicates_for(item)
+        where(item._duplicate_conditions).
+        where("`#{table_name}`.`id` != ?", item.id)
+      end
+    end
+
+    # Returns a collection of records belonging to the same class/table which
+    # matches on the designated columns.
+    #
+    #   <%= render partial: 'user/duplicate', collection: @user.duplicates %>
+    def duplicates
+      self.class.find_duplicates_for(self)
+    end
+
+    # Returns a hash meant to be used as a parameter to the query method
+    # <tt>where(..)</tt>. To clarify the return value, if your model was set up like
+    # this:
+    #
+    #   class User < ActiveRecord::Base
+    #     quickening :last_name, :ssn
+    #     ..
+    #   end
+    #
+    # Then when you call a method utilizing this helper, such as
+    # <tt>User#duplicates</tt>:
+    #
+    #   @user.last_name   # => 'Wayne'
+    #   @user.ssn         # => '987-65-321'
+    #   @user.duplicates  # => []
+    #
+    # ...the <tt>where(..)</tt> condition will receive the output of this method
+    # and end up with the following:
+    #
+    #   where({ last_name: 'Wayne', ssn: '987-65-321' })
+    #
+    # Since the return of this method is simply injected into the +where+ method,
+    # you could override this method and, theoretically, do something as follows:
+    #
+    #   def _duplicate_conditions
+    #     ["first_name = ?, middle_name = ?, last_name = ?", *full_name.split]
+    #   end
+    #
+    # However, as this breaks the unified definition of which columns are
+    # expected to match, be sure you won't be breaking other aspects of the
+    # integration of this library.
+    #
+    # ==== Alternate override
+    #
+    # If you want to maintain the library's method behavior but extend it a bit,
+    # you might just want to follow the module extension scheme instead:
+    #
+    #   class User < ActiveRecord::Base
+    #     quickening [..]
+    #
+    #     # Custom overrides:
+    #     module DuncanMacLeod
+    #       def _duplicate_conditions
+    #         @temporary_conditions || super
+    #       end
+    #     end
+    #     include DuncanMacLeod
+    #     ..
+    #   end
+    def _duplicate_conditions
+      Hash[duplicate_matchers.map { |col| [col, send(col)] }]
+    end
+  end
+end

data/lib/quickening/orm/active_record.rb

@@ -0,0 +1,36 @@
+# encoding: utf-8
+
+module Quickening::ORM # :nodoc:
+
+  module ActiveRecord # :nodoc:
+    # In your model file, call +quickening+ method at the class-level, providing
+    # it the columns you want the library to use as the basis for determining
+    # whether or not records are "duplicates."
+    #
+    # In its present form, this comparison/matching is handled in a decidedly
+    # black-and-white manner (although it's likely that many MySQL setups will
+    # forgive case-insentivity).
+    #
+    #   class User < ActiveRecord::Base
+    #     quickening %w(first_name last_name birthdate).map(&:to_sym)
+    #     ..
+    #   end
+    #
+    # ==== Parameters
+    # * +attr_list+ - a list of symbolized attributes referencing column names
+    #
+    # It is only expecting a list, not an Array which could get flattened. For
+    # now please remember this.
+    def quickening(*attr_list)
+      include Quickening::Model
+      class_attribute :duplicate_matchers, instance_writer: false
+      self.duplicate_matchers = attr_list.map(&:to_sym) # Replace me in your models.
+    end
+  end
+
+  # :singleton-method: duplicate_matchers
+  # :singleton-method: duplicate_matchers=
+  # :method: duplicate_matchers
+end
+
+ActiveRecord::Base.extend Quickening::ORM::ActiveRecord

data/lib/quickening/version.rb

@@ -0,0 +1,5 @@
+# encoding: utf-8
+
+module Quickening # :nodoc:
+  VERSION = "0.1.1"
+end

data/lib/quickening.rb

@@ -0,0 +1,17 @@
+# encoding: utf-8
+
+require 'quickening/engine'
+require 'quickening/version'
+require 'active_support/dependencies'
+
+##
+# = the Quickening
+#
+# Please refer to the README file for documentation that is more likely to be
+# up-to-date.
+module Quickening
+
+  autoload :Model, 'quickening/model'
+
+  # Settings forthcoming
+end

data/lib/tasks/quickening_tasks.rake

@@ -0,0 +1,8 @@
+# encoding: utf-8
+
+namespace :quickening do
+  # desc "Explaining what the task does"
+  # task :quickening do
+  #   # Task goes here
+  # end
+end