diffbot 0.1.0

data/.gitignore

@@ -0,0 +1 @@
+/pkg

data/LICENSE

@@ -0,0 +1,19 @@
+Copyright (c) 2012 Nicolás Sanguinetti for Tinder Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,125 @@
+# Diffbot
+
+This is a ruby client for the [Diffbot](http://diffbot.com) API.
+
+## Global Options
+
+You can pass some settings to Diffbot like this:
+
+``` ruby
+Diffbot.configure do |config|
+  config.token = ENV["DIFFBOT_TOKEN"]
+  config.instrumentor = ActiveSupport::Notifications
+end
+```
+
+The list of supported settings is:
+
+* `token`: Your Diffbot API token. This will be used for all requests in which
+  you don't specify it manually (see below).
+* `instrumentor`: An object that matches the [ActiveSupport::Notifications][1]
+  API, which will be used to trace network events. None is used by default.
+* `article_defaults`: Pass a block to this method to configure the global
+  request settings used for Diffbot::Article requests. See below the options
+  supported.
+
+[1]: http://api.rubyonrails.org/classes/ActiveSupport/Notifications.html
+
+## Articles
+
+In order to fetch an article, do this:
+
+``` ruby
+require "diffbot/article"
+
+article = Diffbot::Article.fetch(article_url, diffbot_token)
+
+# Now you can inspect the result:
+article.title
+article.author
+article.date
+article.text
+# etc. See below for the full list of available response attributes.
+```
+
+This is a list of all the fields returned by the `Diffbot::Article.fetch` call:
+
+* `url`: The URL of the article.
+* `title`: The title of the article.
+* `author`: The author of the article.
+* `date`: The date in which this article was published.
+* `media`: A list of media items attached to this article.
+* `text`: The body of the article. This will be plain text unless you specify
+  the HTML option in the request.
+* `tags`: A list of tags/keywords extracted from the article.
+* `xpath`: The XPath at which this article was found in the page.
+
+### Options
+
+You can customize your request like this:
+
+``` ruby
+article = Diffbot::Article.fetch(article_url, diffbot_token) do |request|
+  request.html = true # Return HTML instead of plain text.
+  request.dont_strip_ads = true # Leave any inline ads within the article.
+  request.tags = true # Generate ads for the article.
+  request.comments = true # Extract the comments from the article as well.
+  request.summary = true # Return a summary text instead of the full text.
+  request.stats = true # Return performance, probabilistic scoring stats.
+end
+```
+
+## Frontpages
+
+In order to fetch and analyze a front page, do this:
+
+``` ruby
+require "diffbot/frontpage"
+
+frontpage = Diffbot::Frontpage.fetch(url, diffbot_token)
+
+# Results are available in the returned object:
+frontpage.title
+frontpage.icon
+frontpage.items #=> An array of Diffbot::Item instances
+```
+
+The fields you can extract from a Frontpage are:
+
+* `title`: The title of the page.
+* `icon`: The favicon of the page.
+* `source_type`: What kind of page this is.
+* `source_url`: The URL of the page.
+* `items`: The list of `Diffbot::Item` representing each item on the page.
+
+The instances of `Diffbot::Item` have the following fields:
+
+* `id`: Unique identifier for this item.
+* `title`: Title of the item.
+* `link`: Extracted permalink of the item (if applicable).
+* `description`: innerHTML content of the item.
+* `summary`: A plain-text summary of the item.
+* `pub_date`: Date when item was detected on page.
+* `type`: The type of item, according to Diffbot. One of: `IMAGE`, `LINK`,
+  `STORY`, `CHUNK`.
+* `img`: The main image extracted from this item.
+* `xroot`: XPath of where the item was found on the page.
+* `cluster`: XPath of the cluster of items where this item was found.
+* `stats`: An object with the following attributes:
+  * `spam_score`: A Float between 0.0 and 1.0 indicating the probability this
+    item is spam/an advertisement.
+  * `static_rank`: A Float between 1.0 and 5.0 indicating the quality score of
+    the item.
+  * `fresh`: The percentage of the item that has changed compared to the
+    previous crawl.
+
+## TODO
+
+* Implement the Follow API.
+* Add tests for Article and Frontpage requests.
+* Add a Frontpage.crawl method that given the URL of a frontpage, it will fetch
+  the article for each item in the page.
+
+## License
+
+This is published under an MIT License, see LICENSE for further details.

data/Rakefile

@@ -0,0 +1,15 @@
+require "rake/testtask"
+require "rubygems/package_task"
+
+gem_spec = eval(File.read("./diffbot.gemspec")) rescue nil
+Gem::PackageTask.new(gem_spec) do |pkg|
+  pkg.need_zip = false
+  pkg.need_tar = false
+end
+
+Rake::TestTask.new do |t|
+  t.pattern = "test/*_test.rb"
+  t.verbose = true
+end
+
+task default: :test

data/diffbot.gemspec

@@ -0,0 +1,19 @@
+Gem::Specification.new do |s|
+  s.name        = "diffbot"
+  s.version     = "0.1.0"
+  s.description = "Diffbot provides a concise API for analyzing and extracting semantic information from web pages using Diffbot (http://www.diffbot.com)."
+  s.summary     = "Ruby interface to the Diffbot API "
+  s.authors     = ["Nicolas Sanguinetti"]
+  s.email       = "hi@nicolassanguinetti.info"
+  s.homepage    = "http://github.com/tinder/diffbot"
+  s.has_rdoc    = false
+  s.files       = `git ls-files`.split "\n"
+  s.platform    = Gem::Platform::RUBY
+
+  s.add_dependency("excon")
+  s.add_dependency("yajl-ruby")
+  s.add_dependency("nokogiri")
+  s.add_dependency("hashie")
+
+  s.add_development_dependency("minitest")
+end

data/lib/diffbot.rb

@@ -0,0 +1,45 @@
+require "hashie/trash"
+require "diffbot/coercible_hash"
+require "diffbot/request"
+require "diffbot/article"
+require "diffbot/frontpage"
+
+module Diffbot
+  # Public: Set global options. This is a nice API to group calls to the Diffbot
+  # module.
+  #
+  # Yields the Diffbot module so you can set options on it.
+  #
+  # Returns self.
+  def self.configure
+    yield self
+    self
+  end
+
+  # Public: Configure the default request parameters for Article requests. See
+  # Article::RequestParams documentation for the specific configuration values
+  # you can set.
+  #
+  # Yields the default Article::RequestParams object.
+  #
+  # Returns the default Article::RequestParams object.
+  def self.article_defaults
+    if block_given?
+      @article_defaults = Article::RequestParams.new
+      yield @article_defaults
+    else
+      @article_defaults ||= Article::RequestParams.new
+    end
+
+    @article_defaults
+  end
+
+  class << self
+    # Public: Your Diffbot API token.
+    attr_accessor :token
+
+    # Public: The object used for network instrumentation. Must match
+    # ActiveSupport::Notifications API.
+    attr_accessor :instrumentor
+  end
+end

data/lib/diffbot/article.rb

@@ -0,0 +1,194 @@
+require "yajl"
+require "diffbot"
+require "diffbot/coercible_hash"
+
+module Diffbot
+  # Representation of an article (ie a blog post or similar). This class offers
+  # a single entry point: the `.fetch` method, that, given a URL, will return
+  # the article as analyzed by Diffbot.
+  class Article < Hashie::Trash
+    extend CoercibleHash
+
+    # Public: Fetch an article from a URL.
+    #
+    # url      - The article URL.
+    # token    - The API token for Diffbot.
+    # parser   - The callable object that will parse the raw output from the
+    #            API. Defaults to Yajl::Parser.method(:parse).
+    # defaults - The default request options. See Diffbot.article_defaults.
+    #
+    # Yields the request configuration.
+    #
+    # Examples
+    #
+    #   # Request an article with the default options.
+    #   article = Diffbot::Article.fetch(url, api_token)
+    #
+    #   # Pass options to the request. See Diffbot::Article::RequestParams to
+    #   # see the available configuration options.
+    #   article = Diffbot::Article.fetch(url, api_token) do |req|
+    #     req.html = true
+    #   end
+    #
+    # Returns a Diffbot::Article.
+    def self.fetch(url, token=Diffbot.token, parser=Yajl::Parser.method(:parse), defaults=Diffbot.article_defaults)
+      params = defaults.dup
+      yield params if block_given?
+
+      request = Diffbot::Request.new(token)
+      response = request.perform(:get, endpoint, params) do |req|
+        req[:query][:url] = url
+      end
+
+      new(parser.call(response.body))
+    end
+
+    # The API endpoint where requests should be made.
+    #
+    # Returns a URL.
+    def self.endpoint
+      "http://www.diffbot.com/api/article"
+    end
+
+    # Public: URL of the article.
+    property :url
+
+    # Public: Title of the article.
+    property :title
+
+    # Public: Author (or Authors) ofthe article.
+    property :author
+
+    # Public: Date of the article (as a string).
+    property :date
+
+    class MediaItem < Hashie::Trash
+      property :type
+      property :link
+      property :primary, default: false
+    end
+
+    # Public: List of media items related to the articles. Each item is an
+    # object with the following attributes:
+    #
+    # type    - Either `"image"` or `"video"`.
+    # link    - The URL of the given media resource.
+    # primary - Only present in one of the items. This is assumed to be the most
+    #           representative media for this article.
+    property :media
+    coerce_property :media, collection: MediaItem
+
+    # Public: The raw text of the article, without formatting.
+    property :text
+
+    # Public: The contents of the article in HTML, stripped of any ads or other
+    # chunks of HTML which are considered unrelated by Diffbot, unless you set
+    # the `dont_strip_ads` option in the request.
+    #
+    # Only present if you set `html` to true in the request.
+    property :html
+
+    # Public: A summary line for this article.
+    #
+    # Only present if you set `summary` to true in the request.
+    property :summary
+
+    # Public: A list of tags related to this article.
+    #
+    # Only present if you set `tags` to true in the request.
+    property :tags
+
+    # Public: The favicon of the page where this article was extracted from.
+    property :icon
+
+    class Stats < Hashie::Trash
+      property :fetch_time, from: :fetchTime
+      property :confidence
+    end
+
+    # Public: Returns an object with the following attributes:
+    #
+    # fetch_time - The time of the request, in ms.
+    # confidence - The confidence of Diffbot that the returned text is really
+    #              the text of the article. Between 0.0 and 1.0.
+    #
+    # Only present if you set `stats` to true in the request.
+    property :stats
+    coerce_property :stats, class: Stats
+
+    # Public: The XPath selector at which the body of the article was found in
+    # the page.
+    property :xpath
+
+    # Public: If there was an error in the request, this will contain the error
+    # message.
+    property :error
+
+    # Public: If there was an error in the request, this will contain the error
+    # code.
+    property :error_code, from: :errorCode
+
+    # This represents the parameters you can pass to Diffbot to configure a
+    # given request. These are either set globally with Diffbot.article_defaults
+    # or on a request basis by passing a block to Diffbot::Article.fetch.
+    #
+    # Example:
+    #
+    #   # All article requests will include the HTML and tags.
+    #   Diffbot.configure do |config|
+    #     config.article_defaults do |defaults|
+    #       defaults.html = true
+    #       defaults.tags = true
+    #     end
+    #   end
+    #
+    #   # This article request will *also* include the summary.
+    #   Diffbot::Article.fetch(url, token) do |req|
+    #     req.summary = true
+    #   end
+    class RequestParams < Hashie::Trash
+      # Public: Set to true to return HTML instead of plain-text.
+      #
+      # Defaults to nil.
+      #
+      # If enabled, sets the `html` key in the `Diffbot::Article`.
+      property :html
+
+      # Public: Set to true to keep any inline ads in the generated story.
+      #
+      # Defaults to nil.
+      #
+      # If enabled, it will change the `html` key in the `Diffbot::Article`.
+      property :dontStripAds, from: :dont_strip_ads
+
+      # Public: Set to true to generate tags for the extracted story.
+      #
+      # Defaults to nil.
+      #
+      # If enabled, sets the `tags` key in the `Diffbot::Article`.
+      property :tags
+
+      # Public: Set to true to find the comments and identify count, link, etc.
+      #
+      # Defaults to nil.
+      #
+      # If enabled, sets the `comments` key in the `Diffbot::Article`.
+      property :comments
+
+      # Public: Set to true to return a summary text.
+      #
+      # Defaults to nil.
+      #
+      # If enabled, sets the `summary` key in the `Diffbot::Article`.
+      property :summary
+
+      # Public: Set to true to include performance and probabilistic scoring
+      # stats.
+      #
+      # Defaults to nil.
+      #
+      # If enabled, sets the `stats` key in the `Diffbot::Article`.
+      property :stats
+    end
+  end
+end

data/lib/diffbot/coercible_hash.rb

@@ -0,0 +1,113 @@
+module Diffbot
+  # Public: Extend a hash with this mixin to make keys coercible to certain
+  # classes. These keys, when assigned to the hash, will be transformed into the
+  # specified classes.
+  #
+  # The object you pass as coercion types should implement either a `coerce` or
+  # a `new` method.
+  #
+  # You can define rules to coerce properties into classes or collections of
+  # classes. In the latter case, CoercibleHash will just map over whatever value
+  # is passed and attempt to coerce each item individually to the given class.
+  #
+  # Examples
+  #
+  #   class Address < Struct.new(:street, :zipcode, :state)
+  #     def self.coerce(address)
+  #       new(address[:street], address[:zipcode], address[:state])
+  #     end
+  #   end
+  #
+  #   class Person < Hash
+  #     extend Diffbot::CoercibleHash
+  #
+  #     coerce_property :address, Address
+  #     coerce_property :children, collection: Person
+  #
+  #     def name
+  #       self["name"]
+  #     end
+  #   end
+  #
+  #   person = Person.new(address: {
+  #     street: "123 Example St.", zipcode: "12345", state: "XX"
+  #   })
+  #
+  #   person.address.street #=> "123 Example St."
+  #   # etc.
+  #
+  #   father = Person.new(name: "John", children: [
+  #     { name: "Tim" }, { name: "Sarah" }
+  #   ])
+  #
+  #   father.name                #=> "John"
+  #   father.children.first.name #=> "Tim"
+  #   father.children.last.name  #=> "Sarah"
+  module CoercibleHash
+    # The coercion rules defined for this hash.
+    attr_reader :coercions
+
+    # Adds a #[]= that checks for coercion on the property and delegates to super.
+    def self.extended(base)
+      base.instance_variable_set("@coercions", {})
+      base.class_eval do
+        def []=(property, value)
+          if self.class.coercions.key?(property.to_s)
+            super property, self.class.coercions[property.to_s].(value)
+          else
+            super
+          end
+        end
+      end
+    end
+
+    # Public: Coerce a property of this hash into a given type. We will try to
+    # call .coerce on the object you pass as the class, and if that fails, we will
+    # call .new.
+    #
+    # property - The name of the property to coerce.
+    # class_or_options  - Either a class to which coerce, or a hash with options:
+    #                     * class:      The class to which coerce
+    #                     * collection: Coerce the key into an array of members of
+    #                                   this class.
+    #
+    # Examples
+    #
+    #   class Person < Hash
+    #     extend Diffbot::CoercibleHash
+    #
+    #     coerce_property :address, Address
+    #
+    #     coerce_property :children, collection: Person
+    #
+    #     coerce_property :dob, class: Date
+    #   end
+    def coerce_property(property, options)
+      unless options.is_a?(Hash)
+        options = { class: options }
+      end
+
+      coercion_method = ->(obj) do
+        if obj.respond_to?(:coerce)
+          obj.method(:coerce)
+        elsif obj.respond_to?(:new)
+          obj.method(:new)
+        else
+          raise ArgumentError, "#{obj.inspect} does not implement neither .coerce nor .new"
+        end
+      end
+
+      if options.has_key?(:collection)
+        klass = options[:collection]
+        coercion = ->(value) { value.map { |el| coercion_method[klass][el] } }
+      elsif options.has_key?(:class)
+        klass = options[:class]
+        coercion = ->(value) { coercion_method[klass][value] }
+      else
+        raise ArgumentError, "You need to specify either :class or :collection"
+      end
+
+      coercions[property.to_s] = coercion
+    end
+  end
+end

data/lib/diffbot/frontpage.rb

@@ -0,0 +1,60 @@
+require "nokogiri"
+require "diffbot"
+require "diffbot/item"
+
+module Diffbot
+  # Representation of an front page. This class offers a single entry point: the
+  # `.fetch` method, that, given a URL, will return the front page as analyzed
+  # by Diffbot.
+  class Frontpage < Hashie::Trash
+    extend CoercibleHash
+
+    # Public: Fetch a frontpage's information from a URL.
+    #
+    # url      - The frontpage URL.
+    # token    - The API token for Diffbot.
+    # parser   - The callable object that will parse the raw output from the
+    #            API. Defaults to Diffbot::Frontpage::DmlParser.method(:parse).
+    #
+    # Examples
+    #
+    #   # Request a frontpage with the default options.
+    #   frontpage = Diffbot::Frontpage.fetch(url, api_token)
+    #
+    # Returns a Diffbot::Frontpage.
+    def self.fetch(url, token=Diffbot.token, parser=Diffbot::Frontpage::DmlParser.method(:parse))
+      request = Diffbot::Request.new(token)
+      response = request.perform(:get, endpoint) do |req|
+        req[:query][:url] = url
+      end
+
+      new(parser.call(response.body))
+    end
+
+    # The API endpoint where requests should be made.
+    #
+    # Returns a URL.
+    def self.endpoint
+      "http://www.diffbot.com/api/frontpage"
+    end
+
+    # Public: The title of the page.
+    property :title
+
+    # Public: The favicon of the page.
+    property :icon
+
+    # Public: The favicon of the page.
+    property :source_type, from: :sourceType
+
+    # Public: The URL where this page was extracted from.
+    property :source_url, from: :sourceURL
+
+    # Public: The items extracted from the page. These are instances of
+    # Diffbot::Item.
+    property :items
+    coerce_property :items, collection: Item
+  end
+end
+
+require "diffbot/frontpage/dml_parser"

data/lib/diffbot/frontpage/dml_parser.rb

@@ -0,0 +1,83 @@
+# Parser that takes the XML generated from Diffbot's Frontpage API call and
+# returns a hash suitable for Diffbot::Frontpage.
+class Diffbot::Frontpage::DmlParser
+  # Take the string of DML and convert it into a nice little hash we can pass to
+  # Diffbot::Frontpage.
+  #
+  # dml - A string of DML.
+  #
+  # Returns a Hash.
+  def self.parse(dml)
+    node = Nokogiri(dml).root
+    parser = new(node)
+    parser.parse
+  end
+
+  # Initialize the parser with a DML node.
+  #
+  # dml - The root XML::Element 
+  def initialize(node)
+    @dml = node
+  end
+
+  # The root element of the DML document.
+  attr_reader :dml
+
+  # Parses the Diffbot Markup Language and generates a Hash that we can pass to
+  # Frontpage.new.
+  #
+  # Returns a Hash.
+  def parse
+    attrs = {}
+
+    info = dml % "info"
+    attrs["title"]      = (info % "title").text
+    attrs["icon"]       = (info % "icon").text
+    attrs["sourceType"] = (info % "sourceType").text
+    attrs["sourceURL"]  = (info % "sourceURL").text
+
+    items = dml / "item"
+    attrs["items"] = items.map do |item|
+      ItemParser.new(item).parse
+    end
+
+    attrs
+  end
+
+  # Parser that takes the XML from a particular item from the XML returned from
+  # the frontpage API.
+  class ItemParser
+    # The root element of each item.
+    attr_reader :item
+
+    # Initialize the parser with an Item node.
+    #
+    # item_node - The root node of the item.
+    def initialize(item_node)
+      @item = item_node
+    end
+
+    # Parses the item's DML and generates a Hash that we can add to the DML
+    # parser's parser's "items" key together with the other items.
+    #
+    # Returns a Hash.
+    def parse
+      attrs = {}
+
+      %w(title link pubDate description textSummary).each do |attr|
+        node = item % attr
+        attrs[attr] = node && node.text
+      end
+
+      %w(type img id xroot cluster).each do |attr|
+        attrs[attr] = item[attr]
+      end
+
+      attrs["stats"] = %w(fresh sp sr).each_with_object({}) do |attr, hash|
+        hash[attr] = item[attr].to_f
+      end
+
+      attrs
+    end
+  end
+end

data/lib/diffbot/item.rb

@@ -0,0 +1,55 @@
+module Diffbot
+  class Item < Hashie::Trash
+    extend CoercibleHash
+
+    class Stats < Hashie::Trash
+      property :fresh
+      property :static_rank, from: :sr
+      property :spam_score, from: :sp
+    end
+
+    # Public: The identifier of this item.
+    property :id
+
+    # Public: The title of this item.
+    property :title
+
+    # Public: The permalink/URL for this item.
+    property :link
+
+    # Public: A string with the date of the item.
+    property :pub_date, from: :pubDate
+
+    # Public: The HTML from the item.
+    property :description
+
+    # Public: A summary line with text from the item.
+    property :summary, from: :textSummary
+
+    # Public: The type of the item. Can be either `IMAGE`, `LINK`, `STORY`, or
+    # `CHUNK` (a chunk of HTML).
+    property :type
+
+    # Public: The URL for the image of this item.
+    property :img
+
+    # Public: The XPath where this item is located at.
+    property :xroot
+
+    # Public: The XPath for the cluster of items where this item comes from. If
+    # a frontpage has, for example, a main list of articles and a sidebar with
+    # "Top Articles", for example, both will be separate clusters, each with
+    # their own articles.
+    property :cluster
+
+    # Public: Stats extracted from this item. This is an object with the
+    # following attributes:
+    #
+    # fresh       - The percentage of the item that has changed compared to the
+    #               previous crawl.
+    # static_rank - The quality score of the item on a 1 to 5 scale.
+    # spam_score  - The probability this item is spam/an advertisement.
+    property :stats
+    coerce_property :stats, class: Stats
+  end
+end

data/lib/diffbot/request.rb

@@ -0,0 +1,54 @@
+require "excon"
+
+module Diffbot
+  class Request
+    # The API token for Diffbot.
+    attr_reader :token
+
+    # Public: Initialize a new request to the API.
+    #
+    # token - The API token for Diffbot.
+    def initialize(token)
+      @token = token
+    end
+
+    # Public: Perform an HTTP request against Diffbot's API.
+    #
+    # method   - The request method, one of :get, :head, :post, :put, or
+    #            :delete.
+    # endpoint - The URL to which we'll make the request, as a String.
+    # query    - A hash of query string params we want to pass along.
+    #
+    # Yields the request hash before making the request.
+    #
+    # Returns the response.
+    def perform(method, endpoint, query={})
+      request_options = build_request(method, query)
+      yield request_options if block_given?
+
+      request  = Excon.new(endpoint)
+
+      request.request(request_options)
+    end
+
+    # Build the hash of options that Excon requires for an HTTP request.
+    #
+    # method       - A Symbol with the HTTP method (:get, :post, etc).
+    # query_params - Any query parameters to add to the request.
+    #
+    # Returns a Hash.
+    def build_request(method, query_params={})
+      query   = { token: token }.merge(query_params)
+      request = { query: query, method: method, headers: {} }
+
+      if Diffbot.instrumentor
+        request.update(
+          instrumentor: Diffbot.instrumentor,
+          instrumentor_name: "diffbot"
+        )
+      end
+
+      request
+    end
+  end
+end

data/test/coercible_hash_test.rb

@@ -0,0 +1,66 @@
+require "test_helper"
+require "diffbot/coercible_hash"
+
+describe Diffbot::CoercibleHash do
+  module Foo
+    def self.coerce(value)
+      "coerced #{value}"
+    end
+  end
+
+  module Bar
+    def self.new(value)
+      "initialized #{value}"
+    end
+  end
+
+  module Baz
+    def self.coerce(value)
+      "coerced #{value}"
+    end
+
+    def self.new(value)
+      "initialized #{value}"
+    end
+  end
+
+  class TestHash < Hash
+    extend Diffbot::CoercibleHash
+
+    coerce_property :foo, Foo
+    coerce_property :foos, collection: Foo
+
+    coerce_property :bar, Bar
+
+    coerce_property :baz, Baz
+  end
+
+  subject do
+    TestHash.new
+  end
+
+  it "coerces keys using the .coerce method" do
+    subject["foo"] = 1
+    subject["foo"].must_equal("coerced 1")
+  end
+
+  it "coerces collections" do
+    subject["foos"] = [1, 2, 3]
+    subject["foos"].must_equal(["coerced 1", "coerced 2", "coerced 3"])
+  end
+
+  it "coerces keys using the .new method" do
+    subject["bar"] = 2
+    subject["bar"].must_equal("initialized 2")
+  end
+
+  it "when both are present, prefers .coerce" do
+    subject["baz"] = 3
+    subject["baz"].must_equal("coerced 3")
+  end
+
+  it "coerces symbols as well" do
+    subject[:foo] = 2
+    subject[:foo].must_equal("coerced 2")
+  end
+end

data/test/test_helper.rb

@@ -0,0 +1,2 @@
+require "minitest/spec"
+require "minitest/autorun"

metadata

@@ -0,0 +1,114 @@
+--- !ruby/object:Gem::Specification
+name: diffbot
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+  prerelease: 
+platform: ruby
+authors:
+- Nicolas Sanguinetti
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2012-02-06 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: excon
+  requirement: &70280593864880 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: *70280593864880
+- !ruby/object:Gem::Dependency
+  name: yajl-ruby
+  requirement: &70280593864420 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: *70280593864420
+- !ruby/object:Gem::Dependency
+  name: nokogiri
+  requirement: &70280593864000 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: *70280593864000
+- !ruby/object:Gem::Dependency
+  name: hashie
+  requirement: &70280593863580 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: *70280593863580
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: &70280593863160 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: *70280593863160
+description: Diffbot provides a concise API for analyzing and extracting semantic
+  information from web pages using Diffbot (http://www.diffbot.com).
+email: hi@nicolassanguinetti.info
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- .gitignore
+- LICENSE
+- README.md
+- Rakefile
+- diffbot.gemspec
+- lib/diffbot.rb
+- lib/diffbot/article.rb
+- lib/diffbot/coercible_hash.rb
+- lib/diffbot/frontpage.rb
+- lib/diffbot/frontpage/dml_parser.rb
+- lib/diffbot/item.rb
+- lib/diffbot/request.rb
+- test/coercible_hash_test.rb
+- test/test_helper.rb
+homepage: http://github.com/tinder/diffbot
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.11
+signing_key: 
+specification_version: 3
+summary: Ruby interface to the Diffbot API
+test_files: []