sc-big_sitemap 0.8.3.4

data/Gemfile

@@ -0,0 +1,4 @@
+source "http://rubygems.org"
+
+# Specify your gem's dependencies in big_sitemap.gemspec
+gemspec

data/Gemfile.lock

@@ -0,0 +1,23 @@
+PATH
+  remote: .
+  specs:
+    big_sitemap (0.8.3.3)
+
+GEM
+  remote: http://rubygems.org/
+  specs:
+    mocha (0.9.10)
+      rake
+    nokogiri (1.4.4)
+    rake (0.8.7)
+    shoulda (2.11.3)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  big_sitemap!
+  bundler
+  mocha
+  nokogiri
+  shoulda

data/History.txt

@@ -0,0 +1,88 @@
+=== 0.8.3.1 / 2011-05-26
+
+* added support for absolute :url_path
+* added first basic mobile sitemap support
+
+=== 0.8.3 / 2011-03-08
+
+* Separate URL and file paths are now supported via the :document_path
+  and :url_path options
+* Fixes an issue when initializing in Rails 3
+
+=== 0.8.2 / 2011-01-25
+
+* Fixes an issue where sitemap files were not being generated if the same model
+  was added more than once (fixes issue #5: https://github.com/alexrabarts/big_sitemap/issues/#issue/5)
+
+=== 0.8.1 / 2011-01-25
+
+* API change: Rails/Merb are no longer automatically detected - use BigSitemapRails and BigSitemapMerb instead
+* API change: Rails' polymorphic_url helper is no longer used to generate URLs (use a lambda with the new :location option instead)
+* Static resources can now be added using the add_static method
+* Incremental updates are now available via the :partial_update option
+* "loc" URL values can now be generated with lambdas
+* Sitemap files can now be locked while being generated using the with_lock method
+* Several bug fixes
+
+=== 0.5.1 / 2009-09-07
+
+* Fixes an issue with the :last_modified key being passed into the find method options
+
+=== 0.5.0 / 2009-09-07
+
+* Add support for lambdas when specifying lastmod
+
+=== 0.4.0 / 2009-08-09
+
+* Use Bing instead of Live/MSN.  Note, this breaks backwards compatibility as
+  the old :ping_msn option is now :ping_bing.
+
+=== 0.3.5 / 2009-08-05
+
+* Fixed bugs in root_url generation and url_for_sitemap generation
+
+=== 0.3.4 / 2009-07-02
+
+* BigSitemap-specific options are no longer passed through to the ORM's find method
+
+=== 0.3.2 / 2009-06-09
+
+* Better handling of URLs when Rails' polymorphic_url isn't available in the model
+
+=== 0.3.2 / 2009-06-09
+
+* Fixes "uninitialized constant ActionController" error
+* Fixes "Unknown key(s): path" error
+
+=== 0.3.1 / 2009-04-18
+
+* Fixes broken gemspec
+
+=== 0.3.0 / 2009-04-06
+
+* API change: Pass model through as first argument to add method, e.g.sitemap.add(Posts, {:path => 'articles'})
+* API change: Use Rails' polymorphic_url helper to generate URLs if Rails is being used
+* API change: Only ping search engines when ping_search_engines is explicitly called
+* Add support for passing options through to the model's find method, e.g. :conditions
+* Allow base URL to be specified as a hash as well as a string
+* Add support for changefreq and priority
+* Pluralize sitemap model filenames
+* GZipping may optionally be turned off
+
+=== 0.2.1 / 2009-03-12
+
+* Normalize path arguments so it no longer matters whether a leading slash is used or not
+
+=== 0.2.0 / 2009-03-11
+
+* Methods are now chainable
+
+=== 0.1.4 / 2009-03-11
+
+* Add clean method to clear out Sitemaps directory
+* Make methods chainable
+
+=== 0.1.3 / 2009-03-10
+
+* Initial release
+

data/LICENSE

@@ -0,0 +1,22 @@
+(The MIT License)
+
+Copyright (c) 2009 Stateless Systems (http://statelesssystems.com)
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,179 @@
+= BigSitemap
+
+BigSitemap is a {Sitemap}[http://sitemaps.org] generator suitable for applications with greater than 50,000 URLs.  It splits large Sitemaps into multiple files, gzips the files to minimize bandwidth usage, batches database queries to minimize memory usage, supports increment updates, can be set up with just a few lines of code and is compatible with just about any framework.
+
+BigSitemap is best run periodically through a Rake/Thor task.
+
+  require 'big_sitemap'
+
+  sitemap = BigSitemap.new(
+    :url_options   => {:host => 'example.com'},
+    :document_root => "#{APP_ROOT}/public"
+  )
+
+  # Add a model
+  sitemap.add Product
+
+  # Add another model with some options
+  sitemap.add(Post,
+    :conditions       => {:published => true},
+    :path             => 'articles',
+    :change_frequency => 'daily',
+    :priority         => 0.5
+  )
+
+  # Add a static resource
+  sitemap.add_static('http://example.com/about', Time.now, 'monthly', 0.1)
+
+  # Generate the files
+  sitemap.generate
+
+The code above will create a minimum of four files:
+
+1. public/sitemaps/sitemap_index.xml.gz
+2. public/sitemaps/sitemap_products.xml.gz
+3. public/sitemaps/sitemap_posts.xml.gz
+4. public/sitemaps/sitemap_static.xml.gz
+
+If your sitemaps grow beyond 50,000 URLs (this limit can be overridden with the <code>:max_per_sitemap</code> option), the sitemap files will be partitioned into multiple files (<code>sitemap_products_1.xml.gz</code>, <code>sitemap_products_2.xml.gz</code>, ...).
+
+=== Framework-specific Classes
+
+Use the framework-specific classes to take advantage of built-in shortcuts.
+
+==== Rails
+
+<code>BigSiteMapRails</code> includes <code>UrlWriter</code> (useful for making use of your Rails routes - see the Location URLs section) and deals with setting the <code>:document_root</code> and <code>:url_options</code> initialization options.
+
+==== Merb
+
+<code>BigSitemapMerb</code> deals with setting the <code>:document_root</code> initialization option.
+
+== Install
+
+Via gem:
+
+  sudo gem install big_sitemap
+
+== Advanced
+
+=== Initialization Options
+
+* <code>:url_options</code> -- hash with <code>:host</code>, optionally <code>:port</code> and <code>:protocol</code>
+* <code>:base_url</code> -- string alternative to <code>:url_options</code>, e.g. <code>'https://example.com:8080/'</code>
+* <code>:url_path</code> -- string path_name to sitemaps folder, defaults to <code>:document_path</code>
+* <code>:document_root</code> -- string
+* <code>:document_path</code> -- string document path to generation folder, relative to :document_root, defaults to <code>'sitemaps/'</code>
+* <code>:path</code> -- string, alias for ":document_path" for legacy reasons
+* <code>:document_full</code> -- string absolute document path to generation folder - defaults to <code>:document_root/:document_path</code>
+* <code>:max_per_sitemap</code> -- <code>50000</code>, which is the limit dictated by Google but can be less
+* <code>:batch_size</code> -- <code>1001</code> (not <code>1000</code> due to a bug in DataMapper)
+* <code>:gzip</code> -- <code>true</code>
+* <code>:ping_google</code> -- <code>true</code>
+* <code>:ping_yahoo</code> -- <code>false</code>, needs <code>:yahoo_app_id</code>
+* <code>:ping_bing</code> -- <code>false</code>
+* <code>:ping_ask</code> -- <code>false</code>
+* <code>:partial_update</code> -- <code>false</code>
+
+=== Chaining
+
+You can chain methods together:
+
+  BigSitemap.new(:url_options => {:host => 'example.com'}).add(Post).generate
+
+With the Rails-specific class, you could even get away with as little code as:
+
+  BigSitemapRails.new.add(Post).generate
+
+=== Pinging Search Engines
+
+To ping search engines, call <code>ping_search_engines</code> after you generate the sitemap:
+
+  sitemap.generate.ping_search_engines
+
+=== Location URLs
+
+By default, URLs for the "loc" values are generated in the form:
+
+  :base_url/:path|<table_name>/<to_param>|<id>
+
+Alternatively, you can pass a lambda.  For example, to make use of your Rails route helper:
+
+  sitemap.add(Post,
+    :location => lambda { |post| post_url(post) }
+  )
+
+=== Mobile Sitemap support
+
+To create a mobile Sitemap for a model, just pass <code>:mobile => true</code>
+
+  sitemap.add(Post,
+    :location => lambda { |post| post_url(post) },
+    :mobile => true
+  )
+
+=== Change Frequency, Priority and Last Modified
+
+You can control "changefreq", "priority" and "lastmod" values for each record individually by passing lambdas instead of fixed values:
+
+  sitemap.add(Post,
+    :change_frequency => lambda { |post| ... },
+    :priority         => lambda { |post| ... },
+    :last_modified    => lambda { |post| ... }
+  )
+
+=== Find Methods
+
+Your models must provide either a <code>find_for_sitemap</code> or <code>all</code> class method that returns the instances that are to be included in the sitemap.
+
+Additionally, you models must provide a <code>count_for_sitemap</code> or <code>count</code> class method that returns a count of the instances to be included.
+
+If you're using ActiveRecord (Rails) or DataMapper then <code>all</code> and <code>count</code> are already provided and you can make use of any supported parameter: (:conditions, :limit, :joins, :select, :order, :include, :group)
+
+  sitemap.add(Track,
+    :select     => "id, permalink, user_id, updated_at",
+    :include    => :user,
+    :conditions => "public = 1 AND state = 'finished' AND user_id IS NOT NULL",
+    :order      => "id ASC"
+  )
+
+If you provide your own <code>find_for_sitemap</code> or <code>all</code> method then it should be able to handle the <code>:offset</code> and <code>:limit</code> options, in the same way that ActiveRecord and DataMapper handle them.  This is especially important if you have more than 50,000 URLs.
+
+=== Partial Update
+
+If you enable <code>:partial_update</code>, the filename will include an id smaller than the id of the first entry. This is perfect to update just the last file with new entries without the need to re-generate files being already there.
+
+=== Lock Generation Process
+
+To prevent another process overwriting from the generated files, use the <code>with_lock</code> method:
+
+  sitemap.with_lock do
+    sitemap.generate
+  end
+
+=== Cleaning the Sitemaps Directory
+
+Calling the <code>clean</code> method will remove all files from the Sitemaps directory.
+
+== Limitations
+
+If your database is likely to shrink during the time it takes to create the sitemap then you might run into problems (the final, batched SQL select will overrun by setting a limit that is too large since it is calculated from the count, which is queried at the very beginning).  In this case and your database uses incremental primary IDs then you might want to use the <code>:partial_update</code> option, which looks at the last ID instead of paginating.
+
+== TODO
+
+Tests for framework-specific components.
+
+== Credits
+
+Thanks to Alastair Brunton and Harry Love, who's work provided a starting point for this library.
+
+Thanks also to those who have contributed patches:
+
+* Mislav Marohnić
+* Jeff Schoolcraft
+* Dalibor Nasevic
+* Tobias Bielohlawek, SoundCloud Ltd. {http://www.rngtng.com}[http://www.rngtng.com]
+
+== Copyright
+
+Copyright (c) 2010 Stateless Systems (http://statelesssystems.com). See LICENSE for details.

data/Rakefile

@@ -0,0 +1,69 @@
+
+#############################################################################
+#
+# Helper functions
+#
+#############################################################################
+
+def name
+  @name ||= Dir['*.gemspec'].first.split('.').first
+end
+
+def gemspec_file
+  "#{name}.gemspec"
+end
+
+def gemspec
+  @gemspec ||= eval(IO.read(gemspec_file))
+end
+
+def gem_file
+  gemspec.file_name
+end
+
+#############################################################################
+#
+# Standard tasks
+#
+#############################################################################
+
+desc "Deploys the builded gem to the soundcloud gem repository: gems.soundcloud.com"
+task :release => :build do
+  remote_gem_host = 'soundcloud@gems.int.s-cloud.net'
+  remote_gem_path = '/srv/www/gems'
+  Dir.chdir File.dirname(__FILE__)
+  unless `git branch` =~ /^\* master$/
+    puts "You must be on the master branch to release!"
+    exit!
+  end
+  if `git fetch --tags && git tag`.split(/\n/).include?(gem_file)
+    raise "Version #{gem_file} already deployed"
+  end
+  sh <<-END
+    git commit -a --allow-empty -m 'Release #{gem_file}'
+    git tag -a #{gem_file} -m 'Version #{gem_file}'
+    git push origin master
+    git push origin --tags
+    scp pkg/#{gem_file} #{remote_gem_host}:#{remote_gem_path}/gems && \
+    ssh #{remote_gem_host} 'gem generate_index -d #{remote_gem_path}'
+  END
+end
+
+require 'bundler/gem_tasks'
+
+require 'rdoc/task'
+RDoc::Task.new do |rd|
+  rd.main = "README.rdoc"
+  rd.title = 'big_sitemap'
+  rd.options << '--line-numbers' << '--inline-source'
+  rd.rdoc_files.include("README.rdoc", "lib/**/*.rb")
+end
+
+require 'rake/testtask'
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib' << 'test' << Rake.original_dir
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = false
+end
+
+task :default => :test

data/VERSION

@@ -0,0 +1 @@
+0.8.3.4

data/lib/big_sitemap.rb

@@ -0,0 +1,367 @@
+require 'uri'
+require 'fileutils'
+
+require 'big_sitemap/builder'
+
+class BigSitemap
+  DEFAULTS = {
+    :max_per_sitemap => Builder::MAX_URLS,
+    :batch_size      => 1001,
+    :document_path   => 'sitemaps/',
+    :gzip            => true,
+
+    # opinionated
+    :ping_google => true,
+    :ping_yahoo  => false, # needs :yahoo_app_id
+    :ping_bing   => false,
+    :ping_ask    => false
+  }
+
+  COUNT_METHODS     = [:count_for_sitemap, :count]
+  FIND_METHODS      = [:find_for_sitemap, :all]
+  TIMESTAMP_METHODS = [:updated_at, :updated_on, :updated, :created_at, :created_on, :created]
+  PARAM_METHODS     = [:to_param, :id]
+
+  def initialize(options={})
+    @options = DEFAULTS.merge options
+    @options[:document_path] ||= @options[:path] #for legacy reasons
+
+    if @options[:max_per_sitemap] <= 1
+      raise ArgumentError, '":max_per_sitemap" must be greater than 1'
+    end
+
+    if @options[:url_options]
+      @options[:base_url] = URI::Generic.build( {:scheme => "http"}.merge(@options.delete(:url_options)) ).to_s
+    end
+
+    unless @options[:base_url]
+      raise ArgumentError, 'you must specify either ":url_options" hash or ":base_url" string'
+    end
+    @options[:url_path] ||= @options[:document_path]
+    @options[:url_path]   = File.join(@options[:base_url], @options[:url_path]) unless @options[:url_path].include?('http')
+
+    if @options[:batch_size] > @options[:max_per_sitemap]
+      raise ArgumentError, '":batch_size" must be less than ":max_per_sitemap"'
+    end
+
+    @options[:document_full] ||= File.join(@options[:document_root], @options[:document_path])
+    unless @options[:document_full]
+      raise ArgumentError, 'Document root must be specified with the ":document_root" option, the full path with ":document_full"'
+    end
+
+    Dir.mkdir(@options[:document_full]) unless File.exists?(@options[:document_full])
+
+    @sources       = []
+    @models        = []
+    @sitemap_files = []
+  end
+
+  def add(model, options={})
+    @models << model
+
+    filename_suffix = @models.count(model) - 1
+
+    options[:path]           ||= table_name(model)
+    options[:filename]       ||= file_name(model)
+    options[:primary_column] ||= 'id' if model.new.respond_to?('id')
+    options[:partial_update]   = @options[:partial_update] && options[:partial_update] != false
+
+    options[:filename] << "_#{filename_suffix}" unless filename_suffix == 0
+
+    @sources << [model, options.dup]
+
+    self
+  end
+
+  def add_static(url, time = nil, frequency = nil, priority = nil)
+    @static_pages ||= []
+    @static_pages << [url, time, frequency, priority]
+    self
+  end
+
+  def with_lock
+    lock!
+    begin
+      yield
+    ensure
+      unlock!
+    end
+  rescue Errno::EACCES => e
+    STDERR.puts 'Lockfile exists' if $VERBOSE
+  end
+
+  def table_name(model)
+    model.table_name
+  end
+
+  def file_name(name)
+    name = table_name(name) unless name.is_a? String
+    File.join(@options[:document_full], "sitemap_#{name}")
+  end
+
+  def dir_files
+    File.join(@options[:document_full], "sitemap_*.{xml,xml.gz}")
+  end
+
+  def clean
+    Dir[dir_files].each do |file|
+      FileUtils.rm file
+    end
+    self
+  end
+
+  def generate
+    prepare_update
+
+    generate_models
+    generate_static
+    generate_sitemap_index
+    self
+  end
+
+  def generate_models
+    for model, options in @sources
+      with_sitemap(model, options.dup) do |sitemap|
+        last_id = nil #id of last processed item
+        count_method = pick_method(model, COUNT_METHODS)
+        find_method  = pick_method(model, FIND_METHODS)
+        raise ArgumentError, "#{model} must provide a count_for_sitemap class method" if count_method.nil?
+        raise ArgumentError, "#{model} must provide a find_for_sitemap class method" if find_method.nil?
+
+        find_options = {}
+        [:conditions, :limit, :joins, :select, :order, :include, :group].each do |key|
+          find_options[key] = options.delete(key)
+        end
+        
+        # Keep the initial conditions for later use
+        conditions = find_options[:conditions]
+
+        primary_column   = options.delete(:primary_column)
+
+        count = model.send(count_method, find_options.merge(:select => (primary_column || '*'), :include => nil))
+        count = find_options[:limit].to_i if find_options[:limit] && find_options[:limit].to_i < count
+        num_sitemaps = 1
+        num_batches  = 1
+
+        if count > @options[:batch_size]
+          num_batches  = (count.to_f / @options[:batch_size].to_f).ceil
+          num_sitemaps = (count.to_f / @options[:max_per_sitemap].to_f).ceil
+        end
+        batches_per_sitemap = num_batches.to_f / num_sitemaps.to_f
+
+        for sitemap_num in 1..num_sitemaps
+          # Work out the start and end batch numbers for this sitemap
+          batch_num_start = sitemap_num == 1 ? 1 : ((sitemap_num * batches_per_sitemap).ceil - batches_per_sitemap + 1).to_i
+          batch_num_end   = (batch_num_start + [batches_per_sitemap, num_batches].min).floor - 1
+
+          for batch_num in batch_num_start..batch_num_end
+            offset        = (batch_num - 1) * @options[:batch_size]
+            limit         = (count - offset) < @options[:batch_size] ? (count - offset) : @options[:batch_size]
+            find_options.update(:limit => limit, :offset => offset) if num_batches > 1
+
+            if last_id && primary_column
+              find_options.update(:limit => limit, :offset => nil)
+              primary_column_value = escape_if_string last_id #escape '
+              find_options[:conditions] = [conditions, "(#{primary_column} > #{primary_column_value})"].compact.join(' AND ')
+            end
+
+            model.send(find_method, find_options).each do |record|
+              last_mod = options[:last_modified]
+              if last_mod.is_a?(Proc)
+                last_mod = last_mod.call(record)
+              elsif last_mod.nil?
+                last_mod_method = pick_method(record, TIMESTAMP_METHODS)
+                last_mod = last_mod_method.nil? ? Time.now : record.send(last_mod_method)
+              end
+
+              param_method = pick_method(record, PARAM_METHODS)
+
+              location =
+                if options[:location].is_a?(Proc)
+                  options[:location].call(record)
+                else
+                  File.join @options[:base_url], options[:path], record.send(param_method).to_s
+                end
+
+              change_frequency = options[:change_frequency] || 'weekly'
+              freq = change_frequency.is_a?(Proc) ? change_frequency.call(record) : change_frequency
+
+              priority = options[:priority]
+              pri = priority.is_a?(Proc) ? priority.call(record) : priority
+
+              last_id = primary_column ? record.send(primary_column) : nil
+              sitemap.add_url!(location, last_mod, freq, pri, last_id)
+            end
+          end
+        end
+      end
+    end
+    self
+  end
+
+  def generate_static
+    return self if Array(@static_pages).empty?
+    with_sitemap('static', :type => 'static') do |sitemap|
+      @static_pages.each do |location, last_mod, freq, pri|
+        sitemap.add_url!(location, last_mod, freq, pri)
+      end
+    end
+    self
+  end
+
+  # Create a sitemap index document
+  def generate_sitemap_index(files = nil)
+    files ||= Dir[dir_files]
+    with_sitemap 'index', :type => 'index' do |sitemap|
+      for path in files
+        next if path =~ /index/
+        sitemap.add_url!(url_for_sitemap(path), File.stat(path).mtime)
+      end
+    end
+    self
+  end
+
+  def ping_search_engines(index_file)
+    require 'net/http'
+    require 'cgi'
+
+    index_file ||= @sitemap_files.last
+    sitemap_uri = CGI::escape(url_for_sitemap(index_file))
+
+    if @options[:ping_google]
+      Net::HTTP.get('www.google.com', "/webmasters/tools/ping?sitemap=#{sitemap_uri}")
+    end
+
+    if @options[:ping_yahoo]
+      if @options[:yahoo_app_id]
+        Net::HTTP.get(
+          'search.yahooapis.com', "/SiteExplorerService/V1/updateNotification?" +
+            "appid=#{@options[:yahoo_app_id]}&url=#{sitemap_uri}"
+        )
+      else
+        STDERR.puts 'unable to ping Yahoo: no ":yahoo_app_id" provided'
+      end
+    end
+
+    if @options[:ping_bing]
+      Net::HTTP.get('www.bing.com', "/webmaster/ping.aspx?siteMap=#{sitemap_uri}")
+    end
+
+    if @options[:ping_ask]
+      Net::HTTP.get('submissions.ask.com', "/ping?sitemap=#{sitemap_uri}")
+    end
+  end
+
+  private
+
+  def prepare_update
+    @files_to_move = []
+    @sources.each do |model, options|
+      if options[:partial_update] && (primary_column = options[:primary_column]) && (last_id = get_last_id(options[:filename]))
+        primary_column_value       = escape_if_string last_id #escape '
+        options[:conditions]       = [options[:conditions], "(#{primary_column} >= #{primary_column_value})"].compact.join(' AND ')
+        options[:start_part_id]    = last_id
+      end
+    end
+  end
+
+  def lock!(lock_file = 'generator.lock')
+    lock_file = File.join(@options[:document_full], lock_file)
+    File.open(lock_file, 'w', File::EXCL)
+  end
+
+  def unlock!(lock_file = 'generator.lock')
+    lock_file = File.join(@options[:document_full], lock_file)
+    FileUtils.rm lock_file
+  end
+
+  def with_sitemap(name, options={})
+    options[:filename] ||= file_name(name)
+    options[:type]     ||= 'sitemap'
+    options[:max_urls] ||= @options["max_per_#{options[:type]}".to_sym]
+    options[:gzip]     ||= @options[:gzip]
+    options[:indent]     = options[:gzip] ? 0 : 2
+
+    sitemap = if options[:type] == 'index'
+      IndexBuilder.new(options)
+    elsif options[:geo]
+      options[:filename] << '_kml'
+      GeoBuilder.new(options)
+    elsif options[:mobile]
+      MobileBuilder.new(options)
+    else
+      Builder.new(options)
+    end
+
+    begin
+      yield sitemap
+    ensure
+      sitemap.close!
+      @sitemap_files.concat sitemap.paths!
+    end
+  end
+
+  def get_last_id(filename)
+    Dir["#{filename}*.{xml,xml.gz}"].map do |file|
+      file.to_s.scan(/#{filename}_(.+).xml/).flatten.last.to_i
+    end.sort.last
+  end
+
+  def pick_method(model, candidates)
+    method = nil
+    candidates.each do |candidate|
+      if model.respond_to? candidate
+        method = candidate
+        break
+      end
+    end
+    method
+  end
+
+  def escape_if_string(value)
+    (value.to_i.to_s == value.to_s) ?  value.to_i : "'#{value.gsub("'", %q(\\\'))}'"
+  end
+
+  def url_for_sitemap(path)
+    File.join @options[:url_path], File.basename(path)
+  end
+
+end
+
+
+class BigSitemapRails < BigSitemap
+
+  if defined?(Rails) && Rails.version < "3"
+    include ActionController::UrlWriter
+  end
+
+  def initialize(options={})
+    raise "No Rails Environment loaded" unless defined? Rails
+    require 'action_controller'
+
+    if Rails.version >= "3"
+      self.class.send(:include, Rails.application.routes.url_helpers)
+    end
+
+    DEFAULTS.merge!(:document_root => "#{Rails.root}/public", :url_options => default_url_options)
+    super(options)
+  end
+
+end
+
+
+class BigSitemapMerb < BigSitemap
+
+  def initialize(options={})
+    raise "No Merb Environment loaded" unless defined? Merb
+    require 'extlib'
+
+    DEFAULTS.merge!(:document_root => "#{Merb.root}/public")
+    super(options)
+  end
+
+  def table_name(model)
+    Extlib::Inflection.tableize(model.to_s)
+  end
+
+end