bcms_sitemap 0.9.0

data/README.markdown

@@ -0,0 +1,167 @@
+**WARNING: This module has not yet been tested in a live production environment** 
+
+**NOTE: Gem has not been built yet.**
+
+# BCMS\_sitemap BrowserCMS sitemap submission module
+
+This is a general purpose sitemap submission tool for BrowserCMS. It will automatically submit your site's sitemap to the search engines of
+your preference. Once setup properly, it will submit the following url to your search engine: http://..yoursite../sitemaps.xml. This document
+contains information about when your models were changed last time and urls for fetching your models sitemaps, e.g. the search engine will be
+instructed to fetch http://..yoursite../sitemaps/pages.xml for your pages model.
+
+Search engines wants sitemaps submitted no more often than once an hour. You will need to setup a cron or a backgroundrb job to submit your
+sitemaps.
+
+If you deploy remotely with capistrano you will want to change your deployment routines so that signatory (verification) files are 
+re-created after deployment.
+
+**Limitations:** Search engines do not accept more than 50,000 page references in a sitemap file. If you expect to have more than 50,000
+active records in your models, then, you will need to implement paging in the module.
+
+
+## Installation instructions
+
+**Install the gem**
+
+    sudo gem install bcms_sitemap
+
+**Add it to your project**
+
+Edit the config/environment.rb file and add the following after config.gem 'browsercms':
+
+    config.gem 'bcms_sitemap'
+
+**Install the new module**
+
+From the root directory of your app, run the following:
+  
+    $ script/generate browser_cms
+  
+This will add the necessary files to you application. In particular, you may want to change the settings in
+the file config/initializers/bcms\_sitemap.rb. (See also customization below)
+
+Edit routes.rb file - add the following *before* map.routes\_for\_browser\_cms:
+
+    map.routes_for_bcms_sitemap
+  
+**Run the migrations**
+
+    $ rake db:migrate
+  
+**Register sitemap in robots.txt (optional)**  
+
+Add the following line to your robots.txt
+
+    Sitemap: http://...yoursite.../sitemaps.xml
+  
+**Additional setup**
+
+In order for your sitemap to be submitted
+
+* you will need to register your the search engine you would like to submit to
+* You will need to setup a job that submits your sitemap.
+
+See next section - configuration.
+
+
+## Configuration
+
+### Registering your search engines
+
+To submit sitemap to search engines, you will need to enable the search engines in the search engines administrative pages. Some search
+engines requires that you authenticate with the site. Where authentication is required, such as for Google, Yahoo, and Bing, you will have to
+register some kind of verification, normally specified, using a verification file.
+
+The search engine section in the admin pages will allow you to enter the required name and content for the verification files, 
+and automatically create these upon saving the record for the individual search engines.
+
+For more information see the individual sites:
+
+[Ask](http://about.ask.com/en/docs/about/webmasters.shtml),
+[Google](http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156184),
+[Yahoo](http://help.yahoo.com/l/us/yahoo/smallbusiness/store/promote/sitemap/sitemap-16.html) and 
+[yahoo submissions](https://siteexplorer.search.yahoo.com/submit), [Live/Bing](http://www.bing.com/webmaster)
+
+[Moreover](http://moreover.com) does not require authentication. (Not really a search engine, but provides news many to companies)
+  
+[See also](http://www.hariesdesign.com/tips-and-trick/6-seo/46-submit-a-sitemap-to-top-5-search-engine) http://www.hariesdesign.com/tips-and-trick/6-seo/46-submit-a-sitemap-to-top-5-search-engine
+  
+### Setting up your submission job
+You can use either cron or backgroundrb to submit your sitemap. My preference is cron.
+On a Ubuntu server you would do the following:
+
+    sudo vi /etc/cron.hourly/submit_sitemaps
+
+Add the following:
+
+    #!/usr/bin/env ruby
+    require 'fileutils'
+    
+    APP_ROOT="/...yoursite..."
+    RAILS_ENV="production"
+    RAKE_OPTS='-q -s'
+    Dir.chdir "#{APP_ROOT}/current"
+    `RAILS_ENV=#{RAILS_ENV} rake #{RAKE_OPTS} sitemap:submit`
+
+Make it executable:
+
+    sudo chmod 755 /etc/cron.hourly/submit_sitemaps
+
+## Customizations (adding new models)
+
+The sitemap submission facility has been setup with **pages** and **article\_news**. If you have more models that you want to add 
+or remove, you need to change the bcms_sitemap.rb file in your config/initializers directory:
+
+1. State what models to publish as a hash. 
+2. The keys are plural name of the model.
+3. The values should be the scope to be used, formed as string
+
+    Cms::SitemapSubmitter.publish_models = {:pages => 'published.not_hidden' :news_articles => 'released'}
+
+The bcms\_sitemap module will backport your model to access the scopes you define. 
+It will also automatically respond to a defined route for your model, e.g. if you add 'feeds', it will publish the feeds sitemap at
+http://...yoursite.../sitemaps/feeds.xml
+
+By default the view is generated with the view app/views/sitemaps/models.builder (located with the gem). If you want differenciated
+priorities or change modification frequencies, you can create your own builder. If so, you will have to give it a corresponding name, e.g. for 
+feeds you would name the view feeds.builder. Below is the source code for the model.builder. 
+
+    xml.instruct! :xml, :version => "1.0", :encoding => "UTF-8" 
+    xml.urlset :xmlns => "http://www.sitemaps.org/schemas/sitemap/0.9" do
+      @objects.each do |object|
+        xml.url do
+          xml.loc xml_url(object)
+          xml.lastmod xml_lastmod(object)
+          xml.changefreq 'weekly'
+          xml.priority '0.5'
+        end
+      end
+    end
+    
+The controller instanciates a variable with the model name pluralized, if you would prefere to use that, i.e. if the builder you are 
+creating is for feeds, you may use @feeds instead of @objects.
+
+## Rake tasks
+
+There are three rake tasks:
+
+* **sitemap:submit** which submits the sitemaps
+* **sitemap:status** which lists the status of submitted sitemaps to the search engines
+* **sitemap:verify_signatories** which re-creates signatory (verification) files after deployment, if necessary.
+
+If you deploy using capistrano, you may want to add the sitemap:verify_signatories task deployment.rb
+
+    namespace :sitemap, :roles => :app do
+      task :verify_signatories do
+        rake = fetch(:rake, "rake")
+        rails_env = fetch(:rails_env, "production")
+        run "cd \#{current_path}; \#{rake} RAILS_ENV=\#{rails_env} sitemap:verify_signatories"
+      end
+    end
+    after "deploy:finalize_update", "sitemap:verify_signatories"
+
+
+# Considerations for the future
+
+Currently the module is placing a template for rendering the search engines in the administration interface, due to inflexible module
+facilities in BrowserCMS. (Should be on the backlog for the browser cms team). Meanwhile, that view has to be stored locally.

data/app/controllers/cms/search_engines_controller.rb

@@ -0,0 +1,19 @@
+class Cms::SearchEnginesController < Cms::ResourceController 
+  layout 'cms/administration'
+  check_permissions :administrate  
+  before_filter :set_menu_section
+  protected
+    def show_url
+      index_url
+    end
+    
+    def order_by_column
+      "name"
+    end
+    
+  private
+    def set_menu_section
+      @menu_section = 'search_engines'
+    end
+
+end

data/app/controllers/sitemaps_controller.rb

@@ -0,0 +1,22 @@
+class SitemapsController < ApplicationController
+  caches_page :index, :model
+  
+  def index
+    @entries = {}
+    Cms::SitemapSubmitter.models.each do |model|
+      last_update = model.classify.constantize.bcms_sitemap_last_update
+      @entries[model.pluralize] = last_update if last_update
+    end
+  end
+  
+  def model
+    model = params[:model]
+    @objects = model.classify.constantize.bcms_sitemap_scope
+    instance_variable_set("@#{model}", @objects) # Template usually wants @pages, @news_articles etc
+    if Rails.root.join('app','views','sitemaps',"#{model}.builder").exist?
+      render "#{model}.builder"
+    else
+      render 'model.builder'
+    end
+  end  
+end

data/app/helpers/sitemaps_helper.rb

@@ -0,0 +1,13 @@
+module SitemapsHelper
+  def xml_lastmod(object)
+    object.updated_at.xmlschema
+  end
+  
+  def xml_url(object)
+    "http://#{SITE_DOMAIN}#{object.path}"
+  end
+  
+  def xml_article_url(object)
+    news_article_url(object.route_params)
+  end
+end

data/app/models/search_engine.rb

@@ -0,0 +1,69 @@
+class SearchEngine < ActiveRecord::Base
+  validates_presence_of :name
+  validates_uniqueness_of :name, :case_sensitive => false
+  validates_presence_of :url, :if => :enabled?
+  
+  after_save :manage_signatory_file
+  after_destroy :remove_signatory_file
+  
+  named_scope :enabled, :conditions => ['enabled = ?', true]
+
+  class << self
+    def last_submit
+      minimum(:submitted_at)
+    end
+    
+    def verify_signatories!
+      all.each do |search_engine|
+        search_engine.send(:create_signatory_file) unless search_engine.has_signatory_file?
+      end
+    end
+    
+    def signatory_folder
+      File.join(Rails.root, 'public')
+    end
+  end
+    
+  def submit
+    if status = Cms::SitemapSubmitter.submit(self)
+      self.update_attributes(:submitted_at => Time.now, :last_status => status)
+    end
+  end
+  
+  def signatory_folder
+    self.class.signatory_folder
+  end
+  
+  def signatory_file_name
+    verification_file && !verification_file.empty? && File.join(signatory_folder, verification_file )
+  end
+  
+  def has_signatory_file?
+    signatory_file_name && File.exists?(signatory_file_name)
+  end
+  
+protected
+  def manage_signatory_file
+    if verification_file_changed? || verification_content_changed?
+      unless verification_file_was.nil?
+        # new file_name - just remove the old one 
+        File.delete File.join(signatory_folder, verification_file_was )
+      end
+      create_signatory_file
+    end
+  end
+  
+  def create_signatory_file
+    if verification_file
+      File.atomic_write(signatory_file_name) do |file|
+        file.write(verification_content || '')
+      end
+    end
+  end
+
+  def remove_signatory_file
+    if signatory_file_name
+      File.delete signatory_file_name
+    end
+  end
+end

data/app/views/cms/search_engines/_form.html.erb

@@ -0,0 +1,17 @@
+<%= content_for :html_head, stylesheet_link_tag('cms/form_layout') %>
+<% form_for([:cms, @search_engine]) do |f| %>
+  <%= f.error_messages %>
+  <%= f.cms_text_field :name %>
+  <div class="fields checkbox_group">
+    <%= f.label :enabled, 'Enabled?' %>
+    <%= f.check_box :enabled %>
+  </div>
+  <%= f.cms_text_field :url, :label => 'Submission url', 
+    :instructions => 'This should be the complete submission url, ending with "=". The local sitemap url will be added to the end of this url' %>
+  <%= f.cms_text_field :verification_file, 
+    :instructions => 'Input filename. The system assumes that the file should reside in the public folder. Upon save, the file will be stored in that location with thw content defined below.' %>
+  <%= f.cms_text_area :verification_content, :label => "Verification file content" %>
+  <div class="buttons">
+    <%= lt_button_wrapper(f.submit("Save", :class => "submit")) %>
+  </div>
+<% end %>

data/app/views/cms/search_engines/edit.html.erb

@@ -0,0 +1,3 @@
+<% @page_title = @toolbar_title = "Edit Search Engine" %>
+<%= content_for :functions, link_to(span_tag("List All"), cms_search_engines_path, :class => "button") %>
+<%= render :partial => 'form' %>

data/app/views/cms/search_engines/index.html.erb

@@ -0,0 +1,51 @@
+<% @page_title = @toolbar_title = "List Search Engines" %>
+<% content_for(:html_head) do %>
+  <% javascript_tag do %>
+    jQuery(function($){
+      $('table.data tbody tr').hover(function(){
+        $(this).addClass('hover')
+      }, function(){
+        $(this).removeClass('hover')
+      }).click(function(){
+        var match = this.id.match(/(.*)_(\d+)/) 
+        var type = match[1];
+        var id = match[2];
+        $('table.data tbody tr').removeClass('selected');
+        $(this).addClass('selected');
+        $('#edit_button').removeClass('disabled').attr('href', '/cms/search_engines/'+id+'/edit');
+        $('#delete_button').removeClass('disabled').attr('href', '/cms/search_engines/'+id)
+      })
+    })
+  <% end %>
+<% end %>
+<% content_for :functions do %>
+<%= link_to(span_tag("&nbsp;Add&nbsp;"), new_cms_search_engine_path, :class => "button") %>
+<%= link_to(span_tag("&nbsp;Edit&nbsp;"), '#', :id => 'edit_button', :class => "button disabled") %>
+<%= link_to span_tag("<span class=\"delete_img\">&nbsp;</span>Delete"), "#", :id => "delete_button", :class => "button disabled http_delete", :onclick => "return confirm('Are you sure you want to delete this search engine?')" %>
+<% end %>
+<div class="roundedcorners">
+<table class="data">
+  <thead>
+  <tr>
+    <th class="first"></th>
+    <th><div class="dividers">Name</div></th>
+    <th><div class="dividers">Enabled</div></th>
+    <th><div>Submitted At</div></th>
+    <th class="last"></th>
+  </tr>
+  </thead>
+  <% @search_engines.each do |search_engine| %>
+    <tr id="search_engine_<%= search_engine.id %>">
+      <td class="first"></td>
+      <td><div class="dividers"><%=h search_engine.name %></div></td>
+      <td><div class="dividers"><%= search_engine.enabled? ? 'Enabled' : '' %></div></td>
+      <td><div><%= search_engine.submitted_at && search_engine.submitted_at.to_s(:date) %></div></td>
+      <td class="last"></td>
+    </tr>
+  <% end %>
+</table>
+<div class="tl"></div>
+<div class="tr"></div>
+<div class="bl"></div>
+<div class="br"></div>
+</div>

data/app/views/cms/search_engines/new.html.erb

@@ -0,0 +1,3 @@
+<% @page_title = @toolbar_title = "New Search Engine" %>
+<%= content_for :functions, link_to(span_tag("List All"), cms_search_engines_path, :class => "button") %>
+<%= render :partial => 'form' %>

data/app/views/cms/shared/_admin_sidebar.html.erb

@@ -0,0 +1,31 @@
+<div class="first">
+  <h3>Users</h3>
+  <ul>
+    <li<%= ' class="open"' if @menu_section == 'users' %>><%= link_to "Users", cms_users_path %></li>
+    <li<%= ' class="open"' if @menu_section == 'groups' %>><%= link_to "Groups", cms_groups_path %></li>
+  </ul>
+</div>
+<div class="bottom_cap bottom_pad"></div>
+
+<div class="top_cap"></div>
+<div>
+  <h3>Templates</h3>
+  <ul>  
+    <li<%= ' class="open"' if @menu_section == 'page_partials' %>><%= link_to "Page Partials", cms_page_partials_path %></li>
+    <li<%= ' class="open"' if @menu_section == 'page_templates' %>><%= link_to "Page Templates", cms_page_templates_path %></li>
+    <li<%= ' class="open"' if @menu_section == 'page_routes' %>><%= link_to "Page Routes", cms_page_routes_path %></li>
+  </ul>
+</div>
+<div class="bottom_cap bottom_pad"></div>
+
+<div class="top_cap"></div>
+<div>
+  <h3>Tools</h3>
+  <ul>  
+    <li<%= ' class="open"' if @menu_section == 'caching' %>><%= link_to "Page Caching", cms_cache_path %></li>
+    <li<%= ' class="open"' if @menu_section == 'redirects' %>><%= link_to "Redirects", cms_redirects_path %></li>
+    <li<%= ' class="open"' if @menu_section == 'email_messages' %>><%= link_to "Email Messages", cms_email_messages_path %></li>
+    <li<%= ' class="open"' if @menu_section == 'search_engines' %>><%= link_to "Search Engine Submission", cms_search_engines_path %></li>
+  </ul>
+</div>
+<div class="bottom_cap"></div>

data/app/views/sitemaps/index.builder

@@ -0,0 +1,10 @@
+xml.instruct! :xml, :version => "1.0", :encoding => "UTF-8" 
+xml.sitemapindex :xmlns => "http://www.sitemaps.org/schemas/sitemap/0.9" do
+  @entries.sort { |a,b| a[0] <=> b[0] }.each do |model,date|
+    xml.sitemap do
+      # xml.loc "http://#{SITE_DOMAIN}/sitemaps/#{path}.xml"
+      xml.loc "#{sitemap_for_model_url(model)}"
+      xml.lastmod date.xmlschema
+    end
+  end
+end

data/app/views/sitemaps/model.builder

@@ -0,0 +1,11 @@
+xml.instruct! :xml, :version => "1.0", :encoding => "UTF-8" 
+xml.urlset :xmlns => "http://www.sitemaps.org/schemas/sitemap/0.9" do
+  @objects.each do |object|
+    xml.url do
+      xml.loc xml_url(object)
+      xml.lastmod xml_lastmod(object)
+      xml.changefreq 'weekly'
+      xml.priority '0.5'
+    end
+  end
+end

data/app/views/sitemaps/news_articles.builder

@@ -0,0 +1,11 @@
+xml.instruct! :xml, :version => "1.0", :encoding => "UTF-8" 
+xml.urlset :xmlns => "http://www.sitemaps.org/schemas/sitemap/0.9" do
+  @articles.each do |object|
+    xml.url do
+      xml.loc news_article_url(object.route_params)
+      xml.lastmod xml_lastmod(object)
+      xml.changefreq object.archived? ? 'never' : 'weekly'
+      xml.priority object.archived? ? 0.4 : 0.7
+    end
+  end
+end