massive_sitemap 2.0.0.rc7 → 2.0.0.rc8

data/README.md

@@ -3,10 +3,14 @@
 Build painfree sitemaps for websites with millions of pages
 
 MassiveSitemap is a successor project of [BigSitemap](https://github.com/alexrabarts/big_sitemap), a [Sitemap](http://sitemaps.org) generator for websites with millions of pages.
-It implements various generation stategies, e.g. to split large Sitemaps into multiple files, gzip files to minimize bandwidth usage, or incremental updates. It offers API is very similar to _BigSitemap_ and therefor can be set up with just a few lines of code and is compatible with just about any framework.
+It implements various generation stategies, e.g. to split large Sitemaps into multiple files, gzip files to minimize bandwidth usage, or incremental updates. Its API is very similar to _BigSitemap_, can be set up with just a few lines of code and is compatible with just about any framework.
+
 
 ## Usage
 
+A simple usecase which fits most of the standard scenarios:
+
+
 ```ruby
 require 'massive_sitemap'
 
@@ -17,15 +21,19 @@ MassiveSitemap.ping(index_url)
 
 ```
 
-* clear structure
-* allows extension (S3)
+## Structure
+
+MassiveSitemap is structured in two major parts: `Builder` and `Writer`. Both offer an abstract interface which is tailored to the specific needs.
+
+### Builder
+`Builder` keeps all the sitemap structure related logic to build the XML data. `Builder::Index` does the similar for the index structure. `Builder::Rotation` is an extension to make sure no more than 50k urls are written per files, according to sitemap specs.
 
-MassiveSitemap - build huge sitemaps painfree. Differential updates keeps generation time short and reduces load on DB. It's heavealy inspired by BigSitemaps and offers compatiable API
 
-## Dependencies
+### Writer
+The `Writer` takes care of the storage. At top level, that's just a string (`Writer::String`), however `Writer::File` stores to files, `Writer::GzipFile` gzips it as well. `Writer` keeps the state of the files and implements various strategies how to update the files.
 
-Obviously depends on a S3 library which [S3 gem](https://github.com/qoobaa/s3)
 
+Further extension and customization can easily be done, e.g. a `Writer::S3` [extenstion](https://github.com/rngtng/massive_sitemap-writer-s3) stores the sitemap files to Amazon S3 .
 
 ## Contributing
 

data/VERSION

@@ -1 +1 @@
-2.0.0.rc7
+2.0.0.rc8

data/lib/massive_sitemap/writer/file.rb

@@ -85,10 +85,10 @@ module MassiveSitemap
 
       def chaos_monkey_stream_ids(stream_id_keys, days)
         return [] if days < 1
-        offset = Time.now.to_i / (24 * 60 * 60)
-        (0...stream_id_keys.size).step(days).map do |index|
-          stream_id_keys.at((offset % days) + index)
-        end.compact
+        offset = 1 + Time.now.to_i / (24 * 60 * 60)
+        stream_id_keys.select do |stream_id_key|
+          (stream_id_key.scan(/\d+/).first.to_i + offset) % days == 0
+        end
       end
 
       def delete_stream_ids(to_delete)

data/spec/writer/file_spec.rb

@@ -179,7 +179,7 @@ describe MassiveSitemap::Writer::File do
   end
 
   describe "chaos_monkey_stream_ids" do
-    let(:writer) {  MassiveSitemap::Writer::File.new }
+    let(:writer) { MassiveSitemap::Writer::File.new }
     let(:day) { 60 * 60 * 24}
 
     context "one file" do
@@ -213,6 +213,11 @@ describe MassiveSitemap::Writer::File do
         Time.stub!(:now).and_return(3 * day)
         writer.send(:chaos_monkey_stream_ids, %w(sitemap-1.xml sitemap-2.xml sitemap-3.xml), 2).should == %w(sitemap-2.xml)
       end
+
+      it "deletes file on 3rd day even when new file" do
+        Time.stub!(:now).and_return(3 * day)
+        writer.send(:chaos_monkey_stream_ids, %w(sitemap_tracks.xml sitemap-1.xml sitemap-2.xml sitemap-3.xml), 2).should == %w(sitemap_tracks.xml sitemap-2.xml)
+      end
     end
   end
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: massive_sitemap
 version: !ruby/object:Gem::Version
-  version: 2.0.0.rc7
+  version: 2.0.0.rc8
   prerelease: 6
 platform: ruby
 authors:
@@ -9,11 +9,11 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-02-16 00:00:00.000000000Z
+date: 2012-03-14 00:00:00.000000000Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
-  requirement: &70312466352360 !ruby/object:Gem::Requirement
+  requirement: &70360986010320 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -21,10 +21,10 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70312466352360
+  version_requirements: *70360986010320
 - !ruby/object:Gem::Dependency
   name: rspec
-  requirement: &70312466351900 !ruby/object:Gem::Requirement
+  requirement: &70360986009860 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -32,7 +32,7 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70312466351900
+  version_requirements: *70360986009860
 description: MassiveSitemap - build huge sitemaps painfree. Differential updates keeps
   generation time short and reduces load on DB. It's heavealy inspired by BigSitemaps
   and offers compatiable API