mine_shaft 0.1.0

data/CHANGELOG

@@ -0,0 +1,3 @@
+== 0.1.0
+
+* Initial release

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2010 Tom Kersten, http://tomkersten.com/
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.textile

@@ -0,0 +1,93 @@
+h1. MineShaft: Seed data from your stakeholders
+
+h2. Overview
+
+MineShaft uses @mechanize@ to walk through a Remine project site and parse content
+which can't be easily accessed via the REST API (yet). The gem also provides a
+simple way to deserialize an HTML table into an Array of Hash objects which are
+key-value pairs of the table heading(s) and the corresponding value for each row
+in the table.
+
+h2. Motivation
+
+We were looking for some domain-specific information to work with in an
+application and wanted to feed off of the knowledge of some of our stakeholders.
+We didn't want to build out an admin interface which would be replaced with an
+automated system down the road and having our users edit text files and send
+them to us was not going to work.  We created a @Seed_Data@ wiki page and threw
+in a few tables with ID's and wrote MineShaft to parse those tables and convert
+them into Hashes we could use to add/update content in the application database.
+Using this method allow(s) us to spread data entry & review requests for to a
+wide variety of people in an efficient manner.
+
+h2. Usage
+
+Assuming you have a Redmine installation set up at @http://your.rm-install.com@
+and a project named @twitter4biz@ set up.
+
+Given a wiki page titled "Seed_Data" with the following table definition
+_somewhere_ in that page (note: the table ID must be present):
+
+bc. table(#companies).
+|Ticker|Name     |
+|AAPL  |Apple    |
+|MSFT  |Microsoft|
+|GOOG  |Google   |
+|YHOO  |Yahoo    |
+
+Then in IRB (or whatever):
+
+bc. require 'mine_shaft'
+include MineShaft
+shaft = Shaft.new('rm-username', 'rm-password', 'http://your.rm-install.com')
+companies = shaft.grab("companies", '/projects/twitter4biz/wiki/Seed_Data')
+=> [{:name => 'Apple', :ticker => 'AAPL'}, {:name => 'Microsoft', :ticker => 'MSFT'},...]
+
+So, in the db/seeds.rb file of your Rails app, you could put something like the
+following (assuming you have also included the previous code example):
+
+bc. companies.each do |attributes|
+  company = Company.find_by_ticker(attributes[:ticker])
+  if company.nil?
+    Company.create!(attributes)
+    puts "Added '#{attributes[:ticker]}'"
+  else
+    ticker = attributes.delete(:ticker)
+    company.update(attributes)
+    puts "Updated '#{ticker}'"
+  end
+end
+
+...and then run:
+
+bc. rake db:seed
+
+h2. Installation
+
+bc. gem install mine_shaft
+
+h2. Contributing
+
+# Fork it...
+# bundle install
+# ...make awesomeness
+# Commit (ideally) to a feature-branch
+# Send a pull request
+
+h2. Found a bug?
+
+File an issue on the project's "issues page":https://github.com/gn-research/mine_shaft/issues
+
+h2. Dependencies
+
+* mechanize
+
+h2. License
+
+Refer to LICENSE file (hint: MIT)
+
+h2. Future Plans
+
+The gem is meeting our needs at the moment, so we don't have any plans to add
+significant functionality at the moment.  However, it has come in quite handy
+so far, so we may end up expanding it further if a new need arises.

data/Rakefile

@@ -0,0 +1,9 @@
+require 'bundler'
+require 'rspec/core/rake_task'
+
+Bundler::GemHelper.install_tasks
+RSpec::Core::RakeTask.new do |t|
+  t.rspec_opts = ["--color"]
+end
+
+task :default => :spec

data/lib/mine_shaft.rb

@@ -0,0 +1,13 @@
+# Redmine doesn't support XML auth to wiki pages...we need to get dirty w/ it
+
+require 'mechanize'
+require 'mine_shaft/errors'
+require 'mine_shaft/html_table'
+require 'mine_shaft/login_page'
+require 'mine_shaft/shaft'
+require 'mine_shaft/user_agent'
+require 'mine_shaft/web_page'
+
+module MineShaft
+  include Errors
+end

data/lib/mine_shaft/errors.rb

@@ -0,0 +1,5 @@
+module Errors
+  class FailedLogin  < StandardError; end
+  class InvalidPage < StandardError; end
+  class AuthenticityTokenNotFound < StandardError; end
+end

data/lib/mine_shaft/html_table.rb

@@ -0,0 +1,101 @@
+module MineShaft
+  # Provides several convenience methods for translating a (machinist-) parsed
+  # HTML table into standard Ruby data structures.  All tables are assumed to
+  # have a "heading" row as the first row, and that header uses <td> elements
+  # (instead of <th>).
+  class HTMLTable
+    # Public: Initialize a new HTMLTable with the specified table-data as parse
+    #         by machinist (or Nokogiri).
+    #
+    # parsed_table - A Nokogiri::HTML::Document or Nokogiri::XML::Element scoped
+    #                to only the HTML table you are interested in.  Technically
+    #                speaking, you could pass in more content than just the
+    #                <table> element and it would likely work fine, but that is
+    #                the anticipated content structure.
+    #
+    # Returns an instance of HTMLTable
+    def initialize(parsed_table)
+      @table = parsed_table
+    end
+
+    # Public: Retrieve the content of all the <td> elements from the table,
+    #         except for the first row.
+    #
+    # Returns an Array of Array elements, each one being the content from one
+    #   row of the table.  The returned content does NOT include the first row,
+    #   as it is assumed to be the heading of the table.
+    def content_rows
+      table_content = td_elements[column_count, td_elements.size]
+      table_content.enum_slice(column_count).to_a
+    end
+
+    # Public: Converts HTML table to an Array of Hash objects, using the column
+    #         headings as keys for each Hash element.
+    #
+    # Examples
+    #
+    #   Given 'names' was initialized with the following table:
+    #
+    #   ---------------------
+    #   |Name  |Number      |
+    #   ---------------------
+    #   |John  |123-456-7890|
+    #   ---------------------
+    #
+    #   names.deserialize
+    #   # => [{:name => "John", :number => "123-456-7890"}]
+    #
+    # Returns an Array of Hash objects.  Each Hash element is a
+    #   key-value mapping of "table header"-"row content". (Note that the
+    #   the key is a downcased-symbol of the heading value).
+    def deserialize
+      content_rows.map do |row_cells|
+        symbolized_headings.inject({}) do |all_attributes, current_attribute|
+          index_of_header = symbolized_headings.index(current_attribute)
+          value = row_cells[index_of_header]
+          all_attributes.merge({current_attribute.to_sym => value})
+        end
+      end
+    end
+
+    # Public: Retrieves the content from all <td> elements in the table.
+    #
+    # Returns an Array of the content contained in each <td> element.
+    def td_elements
+      @table.search("td").map(&:content)
+    end
+
+    # Public: Retrieves the content from the <td> elements of the first row of
+    #         the table.
+    #
+    # Returns an Array of the content contained in each <td> element of the
+    #   first row.
+    def headings
+      td_elements.slice(0,column_count)
+    end
+    alias :headers :headings
+
+    # Public: Converts the return value of #headings to an Array of
+    #         lower-cased Symbol elements.
+    #
+    # Returns an Array of Symbol elements.
+    def symbolized_headings
+      headings.map {|header| header.downcase.to_sym}
+    end
+
+    private
+      # Counts the number of columns in the table.
+      #
+      # Returns the number of columns.
+      def column_count
+        td_elements.count / row_count
+      end
+
+      # Counts the number of rows in the table.
+      #
+      # Returns the number of rows.
+      def row_count
+        @table.search("tr").count
+      end
+  end
+end

data/lib/mine_shaft/login_page.rb

@@ -0,0 +1,51 @@
+module MineShaft
+  # Collection of methods applicable to the login page, essentially simplifying
+  # the process of interacting with the login form for signing in.
+  class LoginPage
+    # The relative URL for the login page on a Redmine site
+    LOGIN_FORM_ACTION = '/login'
+
+    # Public: Instantiates a new LoginPage object.
+    #
+    # page  - A Nokogiri::HTML::Document.
+    #
+    # Returns a new instance of a LoginPage.
+    # Raises InvalidPage if the specified page does not contain the login form.
+    def initialize(page)
+      @page = page
+      raise InvalidPage, "Page specified does not appear to be the login page" if !login_page?
+    end
+
+    # Public: Confirms whether the specified page is the Redmine login page or
+    #         not.
+    #
+    # page  - A Nokogiri::HTML::Document.
+    #
+    # Returns true if the specified page is the login page.
+    # Returns false if the specified page is not the login page.
+    def self.valid?(page)
+      begin
+        new(page)
+        return true
+      rescue InvalidPage
+        return false
+      end
+    end
+
+    # Public: Retrieves the login form from the page.
+    #
+    # Returns an instance of the Mechanize::Form class.
+    def login_form
+      @login_form ||= @page.forms.find {|f| f.action == LOGIN_FORM_ACTION}
+    end
+
+    private
+      # Confirms whether the page is the login page or not
+      #
+      # Returns true if the login form was found
+      # Returns false if the login form was not found
+      def login_page?
+        !!login_form
+      end
+  end
+end

data/lib/mine_shaft/shaft.rb

@@ -0,0 +1,49 @@
+module MineShaft
+  # Provides simple interface for deserializing an id'd HTML table from a
+  # specific page of a Redmine project site.
+  class Shaft
+    # The relative URL for the login page on a Redmine site
+    LOGIN_PAGE_URL = "/login"
+
+    # Public: Initializes new instance of Shaft class.
+    #
+    # username - The username to log in with on the specified Redmine site.
+    # password - The password to log in with on the specified Redmine site.
+    # base_uri - The URL of the Redmine installation.
+    #
+    # Examples
+    #
+    #   shaft = Shaft.new('uname', 'password', 'http://myredmineinstall.com')
+    #
+    # Returns a new instance of the Shaft class.
+    def initialize(username, password, base_uri)
+      @agent    = UserAgent.new(username, password, base_uri)
+      @login_action = @login_page = LOGIN_PAGE_URL
+    end
+
+    # Public: Logs in and parses the specified page for a <table> with the
+    #         specified ID.
+    #
+    # table_id               - The HTML id of the desired table as a String.
+    # relative_wiki_page_url - The relative URL of the page within the Redmine
+    #                          site.
+    #
+    # Examples
+    #
+    #   shaft.grab('names', '/projects/name-parser/Wiki/Name_Data')
+    #
+    # Returns an Array of Hash objects.  Each Hash element is a
+    #   key-value mapping of "table header"-"row content". (Note that the
+    #   the key is a downcased-symbol of the heading value).
+    # Raises FailedLogin if the login failed
+    # Raises InvalidPage if the login page of the site renders a 404
+    #        OR if a table with the supplied ID is not found on the
+    #        specified page.
+    def grab(table_id, relative_wiki_page_url)
+      @agent.log_in
+      wiki_page = WebPage.new(@agent.get(relative_wiki_page_url))
+      requested_table = wiki_page.find_table(table_id)
+      requested_table.deserialize
+    end
+  end
+end

data/lib/mine_shaft/user_agent.rb

@@ -0,0 +1,109 @@
+module MineShaft
+  # Acts as a headless browser to log in and interact with a Redmine site.
+  class UserAgent
+    # The action on relative URL to hit for the login page on a Redmine site.
+    LOGIN_ACTION = '/login'
+
+    # Public: Creates a new instance of the UserAgent class.
+    #
+    # username - The username to use for logging into the Redmine site.
+    # password - The password to use for logging into the Redmine site.
+    # base_uri - The URL of the Redmine site.
+    #
+    # Returns a new instance of the UserAgent class.
+    def initialize(username, password, base_uri)
+      @username = username
+      @password = password
+      @base_uri = base_uri
+      @agent = Mechanize.new
+      @logged_in = false
+    end
+
+    # Public: Retrieves the specified page from the Redmine site.
+    #
+    # page - The relative URL of the page to retrieve as a String.
+    #
+    # Returns the page pased into a Mechanize::Page object.
+    def get(page)
+      @current_page = @agent.get("#{@base_uri}#{page}")
+    end
+
+    # Public: Logs into the Redmine site using credentials specified on object
+    #         instantiation.
+    #
+    # Returns true if login process was successful.
+    # Raises FailedLogin if the login was not successful.
+    # Raises InvalidPage if the specified site returns a 404 response code.
+    def log_in
+      return true if logged_in?
+      fill_out_login_form
+      submit(login_form, login_form.buttons.first)
+
+      if back_on_login_page?
+        raise FailedLogin, "Login failed. Please verify username & password"
+      end
+      @logged_in = true
+    rescue Mechanize::ResponseCodeError
+      raise InvalidPage, "'#{@base_uri}' is returning a 404. Please verify the URL is a functioning Redmine installation"
+    end
+
+    # Public: Submits the specified form by clicking the specified button on
+    #         said form.
+    #
+    # form   - A Mechanize::Form object.
+    # button - A Mechanize::Form::Submit object.
+    #
+    # Returns the page resulting from the submission process as a
+    #   Mechanize::Page object.
+    def submit(form, button)
+      @current_page = @agent.submit(form, button)
+    end
+
+    # Public: Confirms whether or not the UserAgent instance is logged in.
+    #
+    # Returns true if the UserAgent is currently logged in.
+    # Returns false if the UserAgent is not currently logged in.
+    def logged_in?
+      @logged_in
+    end
+
+    private
+      # Enters the username & password specified during instantiation into the
+      # username & password text fields of the login form.
+      #
+      # Returns nothing.
+      def fill_out_login_form
+        login_form.username = @username
+        login_form.password = @password
+        nil
+      end
+
+      # Convenience method to retrieve the login form ffrom the Redmine
+      # installation's login page.
+      #
+      # Returns the login form (caches form object, so you can interact with the
+      #   return value directly on the method).
+      def login_form
+        @login_form ||= load_login_page && @login_page.login_form
+      end
+
+      # Retrieves and caches the login page of the Redmine installation.
+      #
+      # Returns a LoginPage object (object is cached, so you can interact with
+      # the return value directly on the method).
+      def load_login_page
+        @login_page ||= LoginPage.new(get(LOGIN_ACTION))
+      end
+
+      # Confirms whether the current page is the login page...used to detect if
+      # a login failed.
+      #
+      # Returns true if the current page is the Redmine installation's login
+      #   page
+      # Returns false if the current page is not the Redmine installation's login
+      #   page
+      def back_on_login_page?
+        return LoginPage.valid?(@current_page)
+      end
+  end
+end