sitemap_generator 4.0.alpha → 4.0

data/Gemfile

@@ -7,7 +7,7 @@ group :development, :test do
   gem 'mocha'
   gem 'nokogiri'
   gem 'rake'
-  gem 'rcov'
   gem 'rspec'
-  # gem 'ruby-debug19', :require => 'ruby-debug'
+  #gem 'ruby-debug19', :require => 'ruby-debug'
+  #gem 'simplecov', :require => false
 end

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: ./
   specs:
-    sitemap_generator (4.0.alpha)
+    sitemap_generator (4.0)
       builder
 
 GEM
@@ -14,7 +14,6 @@ GEM
       metaclass (~> 0.0.1)
     nokogiri (1.5.0)
     rake (0.9.2.2)
-    rcov (0.9.11)
     rspec (2.8.0)
       rspec-core (~> 2.8.0)
       rspec-expectations (~> 2.8.0)
@@ -32,6 +31,5 @@ DEPENDENCIES
   mocha
   nokogiri
   rake
-  rcov
   rspec
   sitemap_generator!

data/README.md

@@ -9,13 +9,14 @@ Sitemaps adhere to the [Sitemap 0.9 protocol][sitemap_protocol] specification.
 * Framework agnostic
 * Supports [News sitemaps][sitemap_news], [Video sitemaps][sitemap_video], [Image sitemaps][sitemap_images], [Geo sitemaps][sitemap_geo], [Mobile sitemaps][sitemap_mobile] and [Alternate Links][alternate_links]
 * Supports read-only filesystems like Heroku via uploading to a remote host like Amazon S3
-* Compatible with Rails 2 & 3
+* Compatible with Rails 2 & 3 and tested with Ruby REE, 1.9.2 & 1.9.3
 * Adheres to the [Sitemap 0.9 protocol][sitemap_protocol]
 * Handles millions of links
 * Automatically compresses your sitemaps
 * Notifies search engines (Google, Bing, SitemapWriter) of new sitemaps
 * Ensures your old sitemaps stay in place if the new sitemap fails to generate
-* Gives you complete control over your sitemaps and their content
+* Gives you complete control over your sitemap contents and naming scheme
+* Intelligent sitemap indexing
 
 ### Show Me
 
@@ -49,8 +50,7 @@ Output:
 
 ```
 In /Users/karl/projects/sitemap_generator-test/public/
-+ sitemap1.xml.gz                                          3 links /  357 Bytes
-+ sitemap_index.xml.gz                                  1 sitemaps /  228 Bytes
++ sitemap.xml.gz                                           3 links /  364 Bytes
 Sitemap stats: 3 links / 1 sitemaps / 0m00s
 
 Successful ping of Google
@@ -65,9 +65,45 @@ Does your website use SitemapGenerator to generate Sitemaps?  Where would you be
 
 <a href='http://www.pledgie.com/campaigns/15267'><img alt='Click here to lend your support to: SitemapGenerator and make a donation at www.pledgie.com !' src='http://pledgie.com/campaigns/15267.png?skin_name=chrome' border='0' /></a>
 
+## Important changes in version 4!
+
+Version 4.0 introduces a new **non-backwards compatible** naming scheme.  **If you are running version 3 or earlier and you upgrade to version 4, you need to make a couple small changes to ensure that search engines can still find your sitemaps!**  Your sitemaps will still work fine, but the name of the index file has changed.
+
+### So what has changed?
+
+* **The index is generated intelligently**.  SitemapGenerator now detects whether you need an index or not, and only generates one if you need it or have requested it.  So small sites (less than 50,000 links) won't have one, large sites will.  You don't have to worry about anything.  And with the `create_index` option, it's easier than ever to control index creation to suit your needs.
+
+* **The default index file name has changed** from `sitemap_index.xml.gz` to just `sitemap.xml.gz`.  So the `_index` part has been removed.  This is a more standard naming scheme for the sitemaps. Any further sitemaps are named `sitemap1.xml.gz`, `sitemap2.xml.gz`, `sitemap3.xml.gz` etc, just as before.
+
+* **Everyone now points search engines to the `sitemap.xml.gz` file**.  It doesn't matter whether your site has 10 links or a million links, just point to `sitemap.xml.gz`.  If your site needs an index, that is the index.  If it doesn't, then that's your sitemap.  Simple.
+
+* **It's easier to write custom namers** because the index and the sitemaps share the same namer instance (which is now a `SitemapGenerator::SimpleNamer` instance).
+
+* **Groups share the new naming convention**.  So the files in your `geo` group will be named `geo.xml.gz`, `geo1.xml.gz`, `geo2.xml.gz` etc.  Pre-version 4 these files would have been named `geo1.xml.gz`, `geo2.xml.gz`, `geo3.xml.gz` etc.
+
+### I don't want it!  How can I keep everything as it was?
+
+You don't care, you just want to get on with your day.  To resort to pre-version 4 behaviour add the following to your sitemap config:
+
+```ruby
+SitemapGenerator::Sitemap.create_index = true
+SitemapGenerator::Sitemap.namer = SitemapGenerator::SimpleNamer.new(:sitemap, :zero => '_index')
+```
+
+This tells SitemapGenerator to always create an index file and to name it `sitemap_index.xml.gz`.  If you are already using custom namers, you don't need to set `namer`; your old namers should still work as before.  If you are using named groups, setting the sitemap namer in this way won't affect your groups, which will still be using the new naming scheme.  If this is an issue for you, you may have to create namers for your groups.
+
+### I want it!  What do I need to do?
+
+1. Update your `robots.txt` file and make sure it points to `sitemap.xml.gz`.
+2. Generate your sitemaps to create the new `sitemap.xml.gz` file.
+3. Optionally remove the old `sitemap_index.xml.gz` file (or link it to the new file if you want to make sure that search engines can find it while you update them.)
+4. Go to your Google Webmaster tools and other places where you've pointed search engines to your sitemaps and point them to your new `sitemap.xml.gz` file.
+
+That's it!  Welcome to the future!
 
 ## Changelog
 
+* **v4.0: NEW, NON-BACKWARDS COMPATIBLE CHANGES.**  See above for more info. `create_index` defaults to `:auto`.  Define `SitemapGenerator::SimpleNamer` class for simpler custom namers compatible with the new naming conventions.  Deprecate `sitemaps_namer`, `sitemap_index_namer` and their respective namer classes.  It's more just that their usage is discouraged.  Support `nofollow` option on alternate links.  Fix formatting of `publication_date` in News sitemaps.
 * v3.4: Support [alternate links][alternate_links] for urls; Support configurable options in the `SitemapGenerator::S3Adapter`
 * v3.3: **Support creating sitemaps with no index file**.  A big thank-you to [Eric Hochberger][ehoch] for generously paying for this feature.
 * v3.2.1: Fix syntax error in SitemapGenerator::S3Adapter
@@ -203,7 +239,7 @@ SitemapGenerator::Sitemap.ping_search_engines
 Alternatively you can pass in the full URL to your sitemap index in which case we would have just the following:
 
 ```ruby
-SitemapGenerator::Sitemap.ping_search_engines('http://example.com/sitemap_index.xml.gz')
+SitemapGenerator::Sitemap.ping_search_engines('http://example.com/sitemap.xml.gz')
 ```
 
 ### Crontab
@@ -225,7 +261,7 @@ end
 You should add the URL of the sitemap index file to `public/robots.txt` to help search engines find your sitemaps.  The URL should be the complete URL to the sitemap index.  For example:
 
 ```
-Sitemap: http://www.example.com/sitemap_index.xml.gz
+Sitemap: http://www.example.com/sitemap.xml.gz
 ```
 
 ## Deployments & Capistrano
@@ -233,40 +269,52 @@ Sitemap: http://www.example.com/sitemap_index.xml.gz
 To ensure that your application's sitemaps are available after a deployment you can do one of the following:
 
 1.  **Generate sitemaps into a directory which is shared by all deployments.**
-
     You can set your sitemaps path to your shared directory using the `sitemaps_path` option.  For example if we have a directory `public/shared/` that is shared by all deployments we can have our sitemaps generated into that directory by setting:
 
-```ruby
-SitemapGenerator::Sitemap.sitemaps_path = 'shared/'
-```
-
+    ```ruby
+    SitemapGenerator::Sitemap.sitemaps_path = 'shared/'
+    ```
 2.  **Copy the sitemaps from the previous deploy over to the new deploy:**
-
     (You will need to customize the task if you are using custom sitemap filenames or locations.)
 
-```ruby
-after "deploy:update_code", "deploy:copy_old_sitemap"
-namespace :deploy do
-  task :copy_old_sitemap do
-    run "if [ -e #{previous_release}/public/sitemap_index.xml.gz ]; then cp #{previous_release}/public/sitemap* #{current_release}/public/; fi"
-  end
-end
-```
-
+    ```ruby
+    after "deploy:update_code", "deploy:copy_old_sitemap"
+    namespace :deploy do
+      task :copy_old_sitemap do
+        run "if [ -e #{previous_release}/public/sitemap.xml.gz ]; then cp #{previous_release}/public/sitemap* #{current_release}/public/; fi"
+      end
+    end
+    ```
 3.  **Regenerate your sitemaps after each deployment:**
 
-```ruby
-after "deploy", "refresh_sitemaps"
-task :refresh_sitemaps do
-  run "cd #{latest_release} && RAILS_ENV=#{rails_env} rake sitemap:refresh"
-end
-```
+    ```ruby
+    after "deploy", "refresh_sitemaps"
+    task :refresh_sitemaps do
+      run "cd #{latest_release} && RAILS_ENV=#{rails_env} rake sitemap:refresh"
+    end
+    ```
 
 ### Sitemaps with no Index File
 
-Sometimes you may not want the sitemap index file to be automatically created, for example when you have a small site with only one sitemap file.  Or you may only want an index file created if you have more than one sitemap file.  Or you may never want the index file to be created.
+The sitemap index file is created for you on-demand, meaning that if you have a large site with more than one sitemap file, you will have a sitemap index file to reference those sitemap files.  If however you have a small site with only one sitemap file, you don't require an index and so no index will be created.  In both cases the index and sitemap file's name, respectively, is `sitemap.xml.gz`.
+
+You may want to always create an index, even if you only have a small site.  Or you may never want to create an index.  For these cases, you can use the `create_index` option to control index creation.  You can read about this option in the Sitemap Options section below.
+
+To always create an index:
+```ruby
+SitemapGenerator::Sitemap.create_index = true
+```
+
+To never create an index:
+```ruby
+SitemapGenerator::Sitemap.create_index = false
+```
+Your sitemaps will still be called `sitemap.xml.gz`, `sitemap1.xml.gz`, `sitemap2.xml.gz`, etc.
 
-To handle these cases, take a look at the `create_index` option in the Sitemap Options section below.
+And the default "intelligent" behaviour:
+```ruby
+SitemapGenerator::Sitemap.create_index = :auto
+```
 
 ### Upload Sitemaps to a Remote Host
 
@@ -287,42 +335,46 @@ Sitemap Generator uses CarrierWave to support uploading to Amazon S3 store, Rack
 
 2. Once you have CarrierWave setup and configured all you need to do is set some options in your sitemap config, such as:
 
-   * `default_host` - your website host name
-   * `sitemaps_host` - the remote host where your sitemaps will be hosted
-   * `public_path` - the directory to write sitemaps to locally e.g. `tmp/`
-   * `sitemaps_path` - set to a directory/path if you don't want to upload to the root of your `sitemaps_host`
-   * `adapter` - instance of `SitemapGenerator::WaveAdapter`
+* `default_host` - your website host name
+* `sitemaps_host` - the remote host where your sitemaps will be hosted
+* `public_path` - the directory to write sitemaps to locally e.g. `tmp/`
+* `sitemaps_path` - set to a directory/path if you don't want to upload to the root of your `sitemaps_host`
+* `adapter` - instance of `SitemapGenerator::WaveAdapter`
 
-   For Example:
+    For Example:
 
-```ruby
-SitemapGenerator::Sitemap.default_host = "http://www.example.com"
-SitemapGenerator::Sitemap.sitemaps_host = "http://s3.amazonaws.com/sitemap-generator/"
-SitemapGenerator::Sitemap.public_path = 'tmp/'
-SitemapGenerator::Sitemap.sitemaps_path = 'sitemaps/'
-SitemapGenerator::Sitemap.adapter = SitemapGenerator::WaveAdapter.new
-```
+     ```ruby
+     SitemapGenerator::Sitemap.default_host = "http://www.example.com"
+     SitemapGenerator::Sitemap.sitemaps_host = "http://s3.amazonaws.com/sitemap-generator/"
+     SitemapGenerator::Sitemap.public_path = 'tmp/'
+     SitemapGenerator::Sitemap.sitemaps_path = 'sitemaps/'
+     SitemapGenerator::Sitemap.adapter = SitemapGenerator::WaveAdapter.new
+     ```
 
 3. Update your `robots.txt` file to point robots to the remote sitemap index file, e.g:
 
-```
-Sitemap: http://s3.amazonaws.com/sitemap-generator/sitemaps/sitemap_index.xml.gz
-```
+    ```
+    Sitemap: http://s3.amazonaws.com/sitemap-generator/sitemaps/sitemap_index.xml.gz
+    ```
+
+    You generate your sitemaps as usual using `rake sitemap:refresh`.
 
-You generate your sitemaps as usual using `rake sitemap:refresh`.
+    Note that SitemapGenerator will automatically turn off `include_index` in this case because
+    the `sitemaps_host` does not match the `default_host`.  The link to the sitemap index file
+    that would otherwise be included would point to a different host than the rest of the links
+    in the sitemap, something that the sitemap rules forbid.  (Since version 3.2 this is no
+    longer an issue because [`include_index` is off by default][include_index_change].)
 
-Note that SitemapGenerator will automatically turn off `include_index` in this case because
-the `sitemaps_host` does not match the `default_host`.  The link to the sitemap index file
-that would otherwise be included would point to a different host than the rest of the links
-in the sitemap, something that the sitemap rules forbid.  (Since version 3.2 this is no
-longer an issue because [`include_index` is off by default][include_index_change].)
+4. Verify to google that you own the s3 url
+
+    In order for Google to use your sitemap, you need to prove you own the s3 bucket through [google webmaster tools](https://www.google.com/webmasters/tools/home?hl=en).  In the example above, you would add the site `http://s3.amazonaws.com/sitemap-generator/sitemaps`.  Once you have verified you own the directory then add your `sitemap.xml.gz` to this list of sitemaps for the site.
 
 ### Generating Multiple Sitemaps
 
 Each call to `create` creates a new sitemap index and associated sitemaps.  You can call `create` as many times as you want within your sitemap configuration.
 
 You must remember to use a different filename or location for each set of sitemaps, otherwise they will
-overwrite each other.  You can use the `filename`, `sitemaps_namer` and `sitemaps_path` options for this.
+overwrite each other.  You can use the `filename`, `namer` and `sitemaps_path` options for this.
 
 In the following example we generate three sitemaps each in its own subdirectory:
 
@@ -340,13 +392,13 @@ Outputs:
 
 ```
 + sitemaps/google/sitemap1.xml.gz             2 links /  822 Bytes /  328 Bytes gzipped
-+ sitemaps/google/sitemap_index.xml.gz          1 sitemaps /  389 Bytes /  217 Bytes gzipped
++ sitemaps/google/sitemap.xml.gz           1 sitemaps /  389 Bytes /  217 Bytes gzipped
 Sitemap stats: 2 links / 1 sitemaps / 0m00s
-+ sitemaps/bing/sitemap1.xml.gz             2 links /  820 Bytes /  330 Bytes gzipped
-+ sitemaps/bing/sitemap_index.xml.gz          1 sitemaps /  388 Bytes /  217 Bytes gzipped
++ sitemaps/bing/sitemap1.xml.gz               2 links /  820 Bytes /  330 Bytes gzipped
++ sitemaps/bing/sitemap.xml.gz             1 sitemaps /  388 Bytes /  217 Bytes gzipped
 Sitemap stats: 2 links / 1 sitemaps / 0m00s
-+ sitemaps/apple/sitemap1.xml.gz             2 links /  820 Bytes /  330 Bytes gzipped
-+ sitemaps/apple/sitemap_index.xml.gz          1 sitemaps /  388 Bytes /  214 Bytes gzipped
++ sitemaps/apple/sitemap1.xml.gz              2 links /  820 Bytes /  330 Bytes gzipped
++ sitemaps/apple/sitemap.xml.gz            1 sitemaps /  388 Bytes /  214 Bytes gzipped
 Sitemap stats: 2 links / 1 sitemaps / 0m00s
 ```
 
@@ -409,34 +461,24 @@ end
 A few things to note:
 
 * `SitemapGenerator::Sitemap` is a lazy-initialized sitemap object provided for your convenience.
-* Every sitemap must set `default_host`.  This is the hostname that is used when building links to add to the sitemap.
+* Every sitemap must set `default_host`.  This is the hostname that is used when building links to add to the sitemap (and all links in a sitemap must belong to the same host).
 * The `create` method takes a block with calls to `add` to add links to the sitemap.
-* The sitemaps are written to the `public/` directory, which is the default location.  You can specify a custom location using the `public_path` or `sitemaps_path` option.
+* The sitemaps are written to the `public/` directory in the directory from which the script is run.  You can specify a custom location using the `public_path` or `sitemaps_path` option.
 
 Now let's see what is output when we run this configuration with `rake sitemap:refresh:no_ping`:
 
 ```
-+ sitemap1.xml.gz                   2 links /  923 Bytes /  329 Bytes gzipped
-+ sitemap_index.xml.gz           1 sitemaps /  364 Bytes /  199 Bytes gzipped
+In /Users/karl/projects/sitemap_generator-test/public/
++ sitemap.xml.gz                                           2 links /  347 Bytes
 Sitemap stats: 2 links / 1 sitemaps / 0m00s
 ```
 
-Weird!  The sitemap has two links, even though only added one!  This is because SitemapGenerator adds the root URL `/` by default.  (Note that prior to version 3.2 the  URL of the sitemap index file was also added to the sitemap by default but [this behaviour has been changed][include_index_change] because of Google complaining about nested indexing.)  You can change the default behaviour by setting the `include_root` or `include_index` option.
-
-Now let's take a look at the files that were created.  After uncompressing and XML-tidying the contents we have:
+Weird!  The sitemap has two links, even though we only added one!  This is because SitemapGenerator adds the root URL `/` for you by default.  (Note that prior to version 3.2 the  URL of the sitemap index file was also added to the sitemap by default but [this behaviour has been changed][include_index_change] because of Google complaining about nested indexing.  This also doesn't make sense anymore because indexes are not always needed.)  You can change the default behaviour by setting the `include_root` or `include_index` option.
 
-* `public/sitemap_index.xml.gz`
+Now let's take a look at the file that was created.  After uncompressing and XML-tidying the contents we have:
 
-```xml
-<?xml version="1.0" encoding="UTF-8"?>
-<sitemapindex xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd">
-  <sitemap>
-    <loc>http://www.example.com/sitemap1.xml.gz</loc>
-  </sitemap>
-</sitemapindex>
-```
 
-* `public/sitemap1.xml.gz`
+* `public/sitemap.xml.gz`
 
 ```xml
 <?xml version="1.0" encoding="UTF-8"?>
@@ -458,6 +500,39 @@ Now let's take a look at the files that were created.  After uncompressing and X
 
 The sitemaps conform to the [Sitemap 0.9 protocol][sitemap_protocol].  Notice the value for `priority` and `changefreq` on the root link, the one that was added for us?  The values tell us that this link is the highest priority and should be checked regularly because it are constantly changing.  You can specify your own values for these options in your call to `add`.
 
+In this example no sitemap index was created because we have so few links, so none was needed.  If we run the same example above and set `create_index = true` we can take a look at what an index file looks like:
+
+```ruby
+SitemapGenerator::Sitemap.default_host = "http://www.example.com"
+SitemapGenerator::Sitemap.create_index = true
+SitemapGenerator::Sitemap.create do
+  add '/welcome'
+end
+```
+
+And the output:
+
+```
+In /Users/karl/projects/sitemap_generator-test/public/
++ sitemap1.xml.gz                                          2 links /  347 Bytes
++ sitemap.xml.gz                                        1 sitemaps /  228 Bytes
+Sitemap stats: 2 links / 1 sitemaps / 0m00s
+```
+
+Now if we look at the uncompressed and formatted contents of `sitemap.xml.gz` we can see that it is a sitemap index and `sitemap1.xml.gz` is a sitemap:
+
+* `public/sitemap.xml.gz`
+
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<sitemapindex xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd">
+  <sitemap>
+    <loc>http://www.example.com/sitemap1.xml.gz</loc>
+    <lastmod>2013-05-01T18:10:26-07:00</lastmod>
+  </sitemap>
+</sitemapindex>
+```
+
 ### Adding Links
 
 You call `add` in the block passed to `create` to add a **path** to your sitemap.  `add` takes a string path and optional hash of options, generates the URL and adds it to the sitemap.  You only need to pass a **path** because the URL will be built for us using the `default_host` we specified.  However, if we want to use a different host for a particular link, we can pass the `:host` option to `add`.
@@ -483,8 +558,7 @@ In the example about we pass a `lastmod` (last modified) option with the value o
 Looking at the output from running this sitemap, we see that we have a few more links than before:
 
 ```
-+ sitemap1.xml.gz                  12 links /     2.3 KB /  365 Bytes gzipped
-+ sitemap_index.xml.gz           1 sitemaps /  364 Bytes /  199 Bytes gzipped
++ sitemap.xml.gz                   12 links /     2.3 KB /  365 Bytes gzipped
 Sitemap stats: 12 links / 1 sitemaps / 0m00s
 ```
 
@@ -518,7 +592,7 @@ add content_path(content), :lastmod => content.updated_at
 
 * `host` - Default: `default_host` (String).
 
-  Host to use when building the URL.  Example:
+  Host to use when building the URL.  It's not technically valid to specify a different host for a link in a sitemap according to the spec, but this facility exists in case you have a need.  Example:
 
 ```ruby
 add '/login', :host => 'https://securehost.com'
@@ -562,6 +636,8 @@ SitemapGenerator::Sitemap.create do
 end
 ```
 
+When you add links in this way, an index is always created, unless you've explicitly set `create_index` to `false`.
+
 ### Accessing the LinkSet instance
 
 Sometimes you need to mess with the internals to do custom stuff.  If you need access to the LinkSet instance from within `create()` you can use the `sitemap` method to do so.
@@ -570,10 +646,10 @@ In this example, say we have already pre-generated three sitemap files: `sitemap
 
 ```ruby
 SitemapGenerator::Sitemap.default_host = "http://www.example.com"
+SitemapGenerator::Sitemap.namer = SitemapGenerator::SimpleNamer.new(:sitemap, :start => 4)
 SitemapGenerator::Sitemap.create do
-  3.times do |i|
-    add_to_index sitemap.sitemaps_namer.to_s
-    sitemap.sitemaps_namer.next
+  (1..3).each do |i|
+    add_to_index "sitemap#{i}.xml.gz"
   end
   add '/home'
   add '/another'
@@ -584,9 +660,9 @@ The output looks something like this:
 
 ```
 In /Users/karl/projects/sitemap_generator-test/public/
-+ sitemap4.xml.gz                                          4 links /  347 Bytes
-+ sitemap_index.xml.gz                                  4 sitemaps /  242 Bytes
-Sitemap stats: 4 links / 4 sitemaps / 0m00s
++ sitemap4.xml.gz                                          3 links /  355 Bytes
++ sitemap.xml.gz                                        4 sitemaps /  242 Bytes
+Sitemap stats: 3 links / 4 sitemaps / 0m00s
 ```
 
 ### Speeding Things Up
@@ -624,9 +700,8 @@ This is useful if you are setting a lot of options.
 Finally, passed as options in a call to `group`:
 
 ```ruby
-SitemapGenerator::Sitemap.create do
-  group(:default_host => 'http://example.com',
-        :sitemaps_path => 'sitemaps/') do
+SitemapGenerator::Sitemap.create(:default_host => 'http://example.com') do
+  group(:filename => :somegroup, :sitemaps_path => 'sitemaps/') do
     add '/home'
   end
 end
@@ -642,9 +717,9 @@ The following options are supported:
 
 * `default_host` - String.  Required.  **Host including protocol** to use when building a link to add to your sitemap.  For example `http://example.com`.  Calling `add '/home'` would then generate the URL `http://example.com/home` and add that to the sitemap.  You can pass a `:host` option in your call to `add` to override this value on a per-link basis.  For example calling `add '/home', :host => 'https://example.com'` would generate the URL `https://example.com/home`, for that link only.
 
-* `filename` - Symbol.  The **base name for the files** that will be generated.  The default value is `:sitemap`.  This yields sitemaps with names like `sitemap1.xml.gz`, `sitemap2.xml.gz`, `sitemap3.xml.gz` etc, and a sitemap index named `sitemap_index.xml.gz`.  If we now set the value to `:geo` the sitemaps would be named `geo1.xml.gz`, `geo2.xml.gz`, `geo3.xml.gz` etc, and the sitemap index would be named `geo_index.xml.gz`.
+* `filename` - Symbol.  The **base name for the files** that will be generated.  The default value is `:sitemap`.  This yields files with names like `sitemap.xml.gz`, `sitemap1.xml.gz`, `sitemap2.xml.gz`, `sitemap3.xml.gz` etc.  If we now set the value to `:geo` the files would be named `geo.xml.gz`, `geo1.xml.gz`, `geo2.xml.gz`, `geo3.xml.gz` etc.
 
-* `include_index` - Boolean.  Whether to **add a link to the sitemap index** to the current sitemap.  This points search engines to your Sitemap Index to include it in the indexing of your site.  2012-07: This is now turned off by default because Google may complain about there being 'Nested Sitemap indexes'.  Default is `false`.  Turned off when `sitemaps_host` is set or within a `group()` block.
+* `include_index` - Boolean.  Whether to **add a link pointing to the sitemap index** to the current sitemap.  This points search engines to your Sitemap Index to include it in the indexing of your site.  2012-07: This is now turned off by default because Google may complain about there being 'Nested Sitemap indexes'.  Default is `false`.  Turned off when `sitemaps_host` is set or within a `group()` block.
 
 * `include_root` - Boolean.  Whether to **add the root** url i.e. '/' to the current sitemap.  Default is `true`.  Turned off within a `group()` block.
 
@@ -652,12 +727,11 @@ The following options are supported:
 
 * `sitemaps_host` - String.  **Host including protocol** to use when generating a link to a sitemap file i.e. the hostname of the server where the sitemaps are hosted.  The value will differ from the hostname in your sitemap links.  For example: `'http://amazon.aws.com/'`.  Note that `include_index` is
 automatically turned off when the `sitemaps_host` does not match `default_host`.
-Because the link to the sitemap index file that would otherwise be added would point to a
-different host than the rest of the links in the sitemap.  Something that the sitemap rules forbid.
+Because the link to the sitemap index file that would otherwise be added would point to a different host than the rest of the links in the sitemap.  Something that the sitemap rules forbid.
 
-* `sitemaps_namer` - A `SitemapGenerator::SitemapNamer` instance **for generating sitemap names**.  You can read about Sitemap Namers by reading the API docs.  Sitemap Namers don't apply to the sitemap index.  You can only modify the name of the index file using the `filename` option.  Sitemap Namers allow you to set the name, extension and number sequence for sitemap files.
+* `namer` - A `SitemapGenerator::SimpleNamer` instance **for generating sitemap names**.  You can read about Sitemap Namers by reading the API docs.  Allows you to set the name, extension and number sequence for sitemap files, as well as modify the name of the first file in the sequence, which is often the index file.  A simple example if we want to generate files like 'newname.xml.gz', 'newname1.xml.gz', etc is `SitemapGenerator::SimpleNamer.new(:newname)`.  I've deprecated the old namer options `sitemaps_namer` and `sitemap_index_namer` in favour of this integrated approach, however those should still work.
 
-* `sitemaps_path` - String. A **relative path** giving a directory under your `public_path` at which to write sitemaps.  The difference between the two options is that the `sitemaps_path` is used when generating a link to a sitemap file.  For example, if we set `SitemapGenerator::Sitemap.sitemaps_path = 'en/'` and use the default `public_path` sitemaps will be written to `public/en/`.  And when the sitemap index is added to our sitemap it would have a URL like `http://example.com/en/sitemap_index.xml.gz`.
+* `sitemaps_path` - String. A **relative path** giving a directory under your `public_path` at which to write sitemaps.  The difference between the two options is that the `sitemaps_path` is used when generating a link to a sitemap file.  For example, if we set `SitemapGenerator::Sitemap.sitemaps_path = 'en/'` and use the default `public_path` sitemaps will be written to `public/en/`.  The URL to the sitemap index would then be `http://example.com/en/sitemap.xml.gz`.
 
 * `verbose` - Boolean.  Whether to **output a sitemap summary** describing the sitemap files and giving statistics about your sitemap.  Default is `false`.  When using the Rake tasks `verbose` will be `true` unless you pass the `-s` option.
 
@@ -678,6 +752,7 @@ Sitemap Groups is a powerful feature that is also very simple to use.
 * The sitemap index file is shared by all groups.
 * Groups can handle any number of links.
 * Group sitemaps are finalized (written out) as they get full and at the end of each group.
+* It's a good idea to name your groups
 
 ### A Groups Example
 
@@ -703,16 +778,17 @@ end
 And the output from running the above:
 
 ```
-+ en/english1.xml.gz                1 links /  612 Bytes /  296 Bytes gzipped
-+ fr/french1.xml.gz                 1 links /  614 Bytes /  298 Bytes gzipped
-+ sitemap1.xml.gz                   3 links /  919 Bytes /  328 Bytes gzipped
-+ sitemap_index.xml.gz           3 sitemaps /  505 Bytes /  221 Bytes gzipped
-Sitemap stats: 5 links / 3 sitemaps / 0m00s
+In /Users/karl/projects/sitemap_generator-test/public/
++ en/english.xml.gz                                        1 links /  328 Bytes
++ fr/french.xml.gz                                         1 links /  329 Bytes
++ sitemap1.xml.gz                                          2 links /  346 Bytes
++ sitemap.xml.gz                                        3 sitemaps /  252 Bytes
+Sitemap stats: 4 links / 3 sitemaps / 0m00s
 ```
 
-So we have two sitemaps with one link each and one sitemap with three links.  The sitemaps from the groups are easy to spot by their filenames.  They are `english1.xml.gz` and `french1.xml.gz`.  They contain only one link each because **`include_index` and `include_root` are set to `false` by default** in a group.
+So we have two sitemaps with one link each and one sitemap with two links.  The sitemaps from the groups are easy to spot by their filenames.  They are `english.xml.gz` and `french.xml.gz`.  They contain only one link each because **`include_index` and `include_root` are set to `false` by default** in a group.
 
-On the other hand, the default sitemap which we added `/rss` to has three links.  The sitemap index and root url were added to it when we added `/rss`.  If we hadn't added that link `sitemap1.xml.gz` would not have been created.  So **when we are using groups, the default sitemap will only be created if we add links to it**.
+On the other hand, the default sitemap which we added `/rss` to has two links.  The root url was added to it when we added `/rss`.  If we hadn't added that link `sitemap1.xml.gz` would not have been created.  So **when we are using groups, the default sitemap will only be created if we add links to it**.
 
 **The sitemap index file is shared by all groups**.  You can change its filename by setting `SitemapGenerator::Sitemap.filename` or by passing the `:filename` option to `create`.
 
@@ -730,6 +806,7 @@ A news item can be added to a sitemap URL by passing a `:news` hash to `add`.  T
 #### Example
 
 ```ruby
+SitemapGenerator::Sitemap.default_host = "http://www.example.com"
 SitemapGenerator::Sitemap.create do
   add('/index.html', :news => {
       :publication_name => "Example",
@@ -763,6 +840,7 @@ Images can be added to a sitemap URL by passing an `:images` array to `add`.  Ea
 #### Example
 
 ```ruby
+SitemapGenerator::Sitemap.default_host = "http://www.example.com"
 SitemapGenerator::Sitemap.create do
   add('/index.html', :images => [{
     :loc => 'http://www.example.com/image.png',
@@ -788,14 +866,17 @@ To add more than one video to a url, pass an array of video hashes using the `:v
 #### Example
 
 ```ruby
-add('/index.html', :video => {
-  :thumbnail_loc => 'http://www.example.com/video1_thumbnail.png',
-  :title => 'Title',
-  :description => 'Description',
-  :content_loc => 'http://www.example.com/cool_video.mpg',
-  :tags => %w[one two three],
-  :category => 'Category'
-})
+SitemapGenerator::Sitemap.default_host = "http://www.example.com"
+SitemapGenerator::Sitemap.create do
+  add('/index.html', :video => {
+    :thumbnail_loc => 'http://www.example.com/video1_thumbnail.png',
+    :title => 'Title',
+    :description => 'Description',
+    :content_loc => 'http://www.example.com/cool_video.mpg',
+    :tags => %w[one two three],
+    :category => 'Category'
+  })
+end
 ```
 
 #### Supported options
@@ -811,6 +892,7 @@ Pages with geo data can be added by passing a `:geo` Hash to `add`.  The Hash on
 #### Example:
 
 ```ruby
+SitemapGenerator::Sitemap.default_host = "http://www.example.com"
 SitemapGenerator::Sitemap.create do
   add('/stores/1234.xml', :geo => { :format => 'kml' })
 end
@@ -832,10 +914,12 @@ Check out the Google specification [here][alternate_links].
 #### Example
 
 ```ruby
+SitemapGenerator::Sitemap.default_host = "http://www.example.com"
 SitemapGenerator::Sitemap.create do
   add('/index.html', :alternate => {
     :href => 'http://www.example.de/index.html',
-    :lang => 'de'
+    :lang => 'de',
+    :nofollow => true
   })
 end
 ```
@@ -844,7 +928,7 @@ end
 
 * `:href` - Required, string.
 * `:lang`  - Required, string.
-
+* `:nofollow` - Optional, boolean. Used to mark link as "nofollow".
 
 ## Raison d'être
 
@@ -891,11 +975,13 @@ Tested and working on:
 
 ## Wishlist & Coming Soon
 
-* Rails framework agnosticism; support for other frameworks like Merb
-
 
 ## Thanks (in no particular order)
 
+I've kind of stopped maintaining the list of contributors.  To all those who have contributed code or a donation, many thanks!
+
+Some past contributors:
+
 * [Eric Hochberger][ehoch]
 * [Rodrigo Flores](https://github.com/rodrigoflores) for News sitemaps
 * [Alex Soto](http://github.com/apsoto) for Video sitemaps