seed-fu 1.2.3 → 2.0.0

data/CHANGELOG.md

@@ -0,0 +1,38 @@
+Version 2.0.0 (In Development)
+------------------------------
+
+Features:
+
+* Depends only on Active Record, not the whole of Rails
+
+* The `Model.seed_many` syntax is now supported by `Model.seed`, and `Model.seed_many` is deprecated
+
+* `Model.seed` supports adding multiple records without an explicit array argument. I.e. the following are equivalent:
+
+      Model.seed([
+        { :name => "Jon" },
+        { :name => "Emily" }
+      ])
+
+      Model.seed(
+        { :name => "Jon" },
+        { :name => "Emily }
+      )
+
+* A side-effect of the above is another option for single seeds:
+
+      Model.seed(:name => "Jon")
+
+* The `SEED` option to `rake db:seed_fu` is deprecated, and replaced by `FILTER` which works the same way.
+
+* Added `SeedFu.quiet` boolean option, set to `true` if you don't want any output from Seed Fu.
+
+* Added `SeedFu.fixture_paths`. Set to an array of paths to look for seed files in. Defaults to `["db/fixtures"]` in general, or `["#{Rails.root}/db/fixtures", "#{Rails.root}/db/fixtures/#{Rails.env}"]` when Seed Fu is installed as a Rails plugin.
+
+* Added `SeedFu.seed` method which is basically a method equivalent of running `rake db:seed_fu` (the rake task now just basically called `SeedFu.seed`)
+
+* Simplified and changed the `SeedFu::Writer` API, see docs for details
+
+Bug fixes:
+
+* Fix Rails 3 deprecation warnings and make seed-fu fully compatible with being installed as a gem

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2008-2010 Michael Bleigh
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,149 @@
+Seed Fu
+=======
+
+Seed Fu is an attempt to once and for all solve the problem of inserting and maintaining seed data in a database. It uses a variety of techniques gathered from various places around the web and combines them to create what is hopefully the most robust seed data system around.
+
+Warning: API Changes
+--------------------
+
+Version 2.0.0 of Seed Fu introduces API changes. `Seed::Writer` has been completely overhauled and will require you to update your scripts. Some other deprecations have been introduced, but the methods will not be removed until version 2.1.0. Please see the [CHANGELOG](CHANGELOG.md) for details.
+
+The API documentation is available in full at [http://rubydoc.info/github/mbleigh/seed-fu/master/frames](http://rubydoc.info/github/mbleigh/seed-fu/master/frames).
+
+Basic Example
+-------------
+
+### In `db/fixtures/users.rb`
+
+    User.seed do |s|
+      s.id    = 1
+      s.login = "jon"
+      s.email = "jon@example.com"
+      s.name  = "Jon"
+    end
+
+    User.seed do |s|
+      s.id    = 2
+      s.login = "emily"
+      s.email = "emily@example.com"
+      s.name  = "Emily"
+    end
+
+### To load the data:
+
+    $ rake db:seed_fu
+    == Seed from /path/to/app/db/fixtures/users.rb
+     - User {:id=>1, :login=>"jon", :email=>"jon@example.com", :name=>"Jon"}
+     - User {:id=>2, :login=>"emily", :email=>"emily@example.com", :name=>"Emily"}
+
+Installation
+------------
+
+### Rails 3
+
+Just add `gem 'seed-fu'` to your `Gemfile`
+
+### Active Record 3
+
+Seed Fu depends on Active Record, but doesn't have to be used with a full Rails app. Simply load and require the `seed-fu` gem and you're set.
+
+### Rails 2.3
+
+The current version is not backwards compatible with Rails 2.3. Please use Seed Fu [version 1.2.3](https://github.com/mbleigh/seed-fu/tree/v1.2.3).
+
+Constraints
+-----------
+
+Constraints are used to identify seeds, so that they can be updated if necessary. For example:
+
+    Point.seed(:x, :y) do |s|
+      s.x = 4
+      s.y = 7
+      s.name = "Home"
+    end
+
+The first time this seed is loaded, a `Point` record will be created. No suppose the name is changed:
+
+    Point.seed(:x, :y) do |s|
+      s.x = 4
+      s.y = 7
+      s.name = "Work"
+    end
+
+When this is run, Seed Fu will look for a `Point` based on the `:x` and `:y` constraints provided. It will see that a matching `Point` already exists and so update its attributes rather than create a new record.
+
+If you do not want seeds to be updated after they have been created, use `seed_once`:
+
+    Point.seed_once(:x, :y) do |s|
+      s.x = 4
+      s.y = 7
+      s.name = "Home"
+    end
+
+The default constraint just checks the `id` of the record.
+
+Where to put seed files
+-----------------------
+
+By default, seed files are looked for in the following locations:
+
+* `Rails.root/db/fixtures` and `Rails.root/db/fixtures/Rails.env` in a Rails app
+* `db/fixtures` when loaded without Rails
+
+You can change these defaults by modifying the `SeedFu.fixture_paths` array.
+
+Seed files can be named whatever you like, and are loaded in alphabetical order.
+
+Terser syntax
+-------------
+
+When loading lots of records, the above block-based syntax can be quite verbose. You can use the following instead:
+
+    User.seed(:id,
+      { :id => 1, :login => "jon",   :email => "jon@example.com",   :name => "Jon"   },
+      { :id => 2, :login => "emily", :email => "emily@example.com", :name => "Emily" }
+    )
+
+Rake task
+---------
+
+Seed files can be run automatically using `rake db:seed_fu`. There are two options which you can pass:
+
+* `rake db:seed_fu FIXTURE_PATH=path/to/fixtures` -- Where to find the fixtures
+* `rake db:seed_fu FILTER=users,articles` -- Only run seed files with a filename matching the `FILTER`
+
+You can also do a similar thing in your code by calling `SeedFu.seed(fixture_paths, filter)`.
+
+Disable output
+--------------
+
+To disable output from Seed Fu, set `SeedFu.quiet = true`.
+
+Handling large seed files
+-------------------------
+
+Seed files can be huge.  To handle large files (over a million rows), try these tricks:
+
+* Gzip your fixtures.  Seed Fu will read .rb.gz files happily.  `gzip -9` gives the   best compression, and with Seed Fu's repetitive syntax, a 160M file can shrink to 16M.
+* Add lines reading `# BREAK EVAL` in your big fixtures, and Seed Fu will avoid loading the whole file into memory.  If you use `SeedFu::Writer`, these breaks are built into your generated fixtures.
+* Load a single fixture at a time with the `FILTER` environment variable
+* If you don't need Seed Fu's ability to update seed with new data, then you may find that [activerecord-import](https://github.com/zdennis/activerecord-import) is faster
+
+Generating seed files
+---------------------
+
+If you need to programmatically generate seed files, for example to convert a CSV file into a seed file, then you can use [`SeedFu::Writer`](../SeedFu/Writer).
+
+Bugs / Feature requests
+-----------------------
+
+Please report them [on Lighthouse](http://mbleigh.lighthouseapp.com/projects/10223-seed-fu).
+
+Contributors
+------------
+
+* [Michael Bleigh](http://www.mbleigh.com/) is the original author
+* [Jon Leighton](http://jonathanleighton.com/) is the current maintainer
+* Thanks to [Matthew Beale](https://github.com/mixonic) for his great work in adding the writer, making it faster and better.
+
+Copyright © 2008-2010 Michael Bleigh, released under the MIT license

data/lib/seed-fu.rb

@@ -1,92 +1,35 @@
-module SeedFu
-  class Seeder
-    def self.plant(model_class, *constraints, &block)
-      constraints = [:id] if constraints.empty?
-      seed = Seeder.new(model_class)
-      insert_only = constraints.last.is_a? TrueClass
-      constraints.delete_at(*constraints.length-1) if (constraints.last.is_a? TrueClass or constraints.last.is_a? FalseClass)
-      seed.set_constraints(*constraints)
-      yield seed
-      seed.plant!(insert_only)
-    end
-
-    def initialize(model_class)
-      @model_class = model_class
-      @constraints = [:id]
-      @data = {}
-    end
-
-    def set_constraints(*constraints)
-      raise "You must set at least one constraint." if constraints.empty?
-      @constraints = []
-      constraints.each do |constraint|
-        raise "Your constraint `#{constraint}` is not a column in #{@model_class}. Valid columns are `#{@model_class.column_names.join("`, `")}`." unless @model_class.column_names.include?(constraint.to_s)
-        @constraints << constraint.to_sym
-      end
-    end
-
-    def plant! insert_only=false
-      record = get
-      return if !record.new_record? and insert_only
-      @data.each do |k, v|
-        record.send("#{k}=", v)
-      end
-      raise "Error Saving: #{record.inspect}" unless record.save(false)
-      puts " - #{@model_class} #{condition_hash.inspect}"      
-      record
-    end
-
-    def method_missing(method_name, *args) #:nodoc:
-      if args.size == 1 and (match = method_name.to_s.match(/(.*)=$/))
-        self.class.class_eval "def #{method_name} arg; @data[:#{match[1]}] = arg; end"
-        send(method_name, args[0])
-      else
-        super
-      end
-    end
-
-    protected
-
-    def get
-      records = @model_class.find(:all, :conditions => condition_hash)
-      raise "Given constraints yielded multiple records." unless records.size < 2
-      if records.any?
-        return records.first
-      else
-        return @model_class.new
-      end
-    end
+require 'active_record'
+require 'seed-fu/railtie' if defined?(Rails) && Rails.version >= "3"
 
-    def condition_hash
-      @constraints.inject({}) {|a,c| a[c] = @data[c]; a }
-    end
+module SeedFu
+  autoload :VERSION,               'seed-fu/version'
+  autoload :Seeder,                'seed-fu/seeder'
+  autoload :ActiveRecordExtension, 'seed-fu/active_record_extension'
+  autoload :BlockHash,             'seed-fu/block_hash'
+  autoload :Runner,                'seed-fu/runner'
+  autoload :Writer,                'seed-fu/writer'
+
+  mattr_accessor :quiet
+
+  # Set `SeedFu.quiet = true` to silence all output
+  @@quiet = false
+
+  mattr_accessor :fixture_paths
+
+  # Set this to be an array of paths to directories containing your seed files. If used as a Rails
+  # plugin, SeedFu will set to to contain `Rails.root/db/fixtures` and
+  # `Rails.root/db/fixtures/Rails.env`
+  @@fixture_paths = ['db/fixtures']
+
+  # Load seed data from files
+  # @param [Array] fixture_paths The paths to look for seed files in
+  # @param [Regexp] filter If given, only filenames matching this expression will be loaded
+  def self.seed(fixture_paths = SeedFu.fixture_paths, filter = nil)
+    Runner.new(fixture_paths, filter).run
   end
 end
 
-
+# @public
 class ActiveRecord::Base
-  # Creates a single record of seed data for use
-  # with the db:seed rake task. 
-  # 
-  # === Parameters
-  # constraints :: Immutable reference attributes. Defaults to :id
-  def self.seed(*constraints, &block)
-    SeedFu::Seeder.plant(self, *constraints, &block)
-  end
-  
-  def self.seed_once(*constraints, &block)
-    constraints << true
-    SeedFu::Seeder.plant(self, *constraints, &block)
-  end
-
-  def self.seed_many(*constraints)
-    seeds = constraints.pop
-    seeds.each do |seed_data|
-      seed(*constraints) do |s|
-        seed_data.each_pair do |k,v|
-          s.send "#{k}=", v
-        end
-      end
-    end
-  end
+  extend SeedFu::ActiveRecordExtension
 end

data/lib/seed-fu/active_record_extension.rb

@@ -0,0 +1,71 @@
+module SeedFu
+  module ActiveRecordExtension
+    # Load some seed data. There are two ways to do this.
+    #
+    # Verbose syntax
+    # --------------
+    #
+    # This will seed a single record. The `:id` parameter ensures that if a record already exists
+    # in the database with the same id, then it will be updated with the name and age, rather
+    # than created from scratch.
+    #
+    #     Person.seed(:id) do |s|
+    #       s.id   = 1
+    #       s.name = "Jon"
+    #       s.age  = 21
+    #     end
+    #
+    # Note that `:id` is the default attribute used to identify a seed, so it need not be
+    # specified.
+    #
+    # Terse syntax
+    # ------------
+    #
+    # This is a more succinct way to load multiple records. Note that both `:x` and `:y` are being
+    # used to identify a seed here.
+    #
+    #     Point.seed(:x, :y,
+    #       { :x => 3, :y => 10, :name => "Home" },
+    #       { :x => 5, :y => 9,  :name => "Office" }
+    #     )
+    def seed(*args, &block)
+      SeedFu::Seeder.new(self, *parse_seed_fu_args(args, block)).seed
+    end
+
+    # Has the same syntax as {#seed}, but if a record already exists with the same values for
+    # constraining attributes, it will not be updated.
+    #
+    # @example
+    #   Person.seed(:id, :id => 1, :name => "Jon") # => Record created
+    #   Person.seed(:id, :id => 1, :name => "Bob") # => Name changed
+    #   Person.seed(:id, :id => 1, :name => "Harry") # => Name *not* changed
+    def seed_once(*args, &block)
+      constraints, data = parse_seed_fu_args(args, block)
+      SeedFu::Seeder.new(self, constraints, data, :insert_only => true).seed
+    end
+
+    # @deprecated Use {#seed} instead
+    def seed_many(*args, &block)
+      puts "DEPRECATED: Model.seed_many is deprecated. You can now use Model.seed in exactly the same way." unless SeedFu.quiet
+      seed(*args, &block)
+    end
+
+    private
+
+      def parse_seed_fu_args(args, block)
+        if block.nil?
+          if args.last.is_a?(Array)
+            # Last arg is an array of data, so assume the rest of the args are constraints
+            data = args.pop
+            [args, data]
+          else
+            # Partition the args, assuming the first hash is the start of the data
+            args.partition { |arg| !arg.is_a?(Hash) }
+          end
+        else
+          # We have a block, so assume the args are all constraints
+          [args, [SeedFu::BlockHash.new(block).to_hash]]
+        end
+      end
+  end
+end

data/lib/seed-fu/block_hash.rb

@@ -0,0 +1,21 @@
+module SeedFu
+  # @private
+  class BlockHash
+    def initialize(proc)
+      @hash = {}
+      proc.call(self)
+    end
+
+    def to_hash
+      @hash
+    end
+
+    def method_missing(method_name, *args, &block)
+      if method_name.to_s =~ /^(.*)=$/ && args.length == 1 && block.nil?
+        @hash[$1] = args.first
+      else
+        super
+      end
+    end
+  end
+end

data/lib/seed-fu/railtie.rb

@@ -0,0 +1,14 @@
+module SeedFu
+  class Railtie < Rails::Railtie
+    rake_tasks do
+      load "tasks/seed_fu.rake"
+    end
+
+    initializer 'seed_fu.set_fixture_paths' do
+      SeedFu.fixture_paths = [
+        Rails.root.join('db/fixtures').to_s,
+        Rails.root.join('db/fixtures/' + Rails.env).to_s
+      ]
+    end
+  end
+end

data/lib/seed-fu/runner.rb

@@ -0,0 +1,73 @@
+require 'zlib'
+require 'active_support/core_ext/array/wrap'
+
+module SeedFu
+  # Runs seed files.
+  #
+  # It is not recommended to use this class directly. Instead, use {SeedFu.seed SeedFu.seed}, which creates
+  # an instead of {Runner} and calls {#run #run}.
+  #
+  # @see SeedFu.seed SeedFu.seed
+  class Runner
+    # @param [Array<String>] fixture_paths The paths where fixtures are located. Will use
+    #   `SeedFu.fixture_paths` if {nil}. If the argument is not an array, it will be wrapped by one.
+    # @param [Regexp] filter If given, only seed files with a file name matching this pattern will
+    #   be used
+    def initialize(fixture_paths = nil, filter = nil)
+      @fixture_paths = Array.wrap(fixture_paths || SeedFu.fixture_paths)
+      @filter        = filter
+    end
+
+    # Run the seed files.
+    def run
+      puts "\n== Filtering seed files against regexp: #{@filter.inspect}" if @filter && !SeedFu.quiet
+
+      filenames.each do |filename|
+        run_file(filename)
+      end
+    end
+
+    private
+
+      def run_file(filename)
+        puts "\n== Seed from #{filename}" unless SeedFu.quiet
+
+        ActiveRecord::Base.transaction do
+          open(filename) do |file|
+            chunked_ruby = ''
+            file.each_line do |line|
+              if line == "# BREAK EVAL\n"
+                eval(chunked_ruby)
+                chunked_ruby = ''
+              else
+                chunked_ruby << line
+              end
+            end
+            eval(chunked_ruby) unless chunked_ruby == ''
+          end
+        end
+      end
+
+      def open(filename)
+        if filename[-3..-1] == '.gz'
+          Zlib::GzipReader.open(filename) do |file|
+            yield file
+          end
+        else
+          File.open(filename) do |file|
+            yield file
+          end
+        end
+      end
+
+      def filenames
+        filenames = []
+        @fixture_paths.each do |path|
+          filenames += (Dir[File.join(path, '*.rb')] + Dir[File.join(path, '*.rb.gz')]).sort
+        end
+        filenames.uniq!
+        filenames = filenames.find_all { |filename| filename =~ @filter } if @filter
+        filenames
+      end
+  end
+end