documentcloud-calais 0.0.7

data/CHANGELOG.markdown

@@ -0,0 +1,33 @@
+# Changes
+
+## 0.0.7
+* verified 4.0 API
+* moved gem packaging to `jeweler` and documentation to `yard`
+
+## 0.0.6
+* fully implemented 3.1 API
+
+## 0.0.5
+* fixed error where classes weren't being required in the proper order on Ubuntu (reported by Jon Moses)
+* New things coming back from the API. Fixing in tests.
+
+## 0.0.4
+* changed dependency from `hpricot` to `libxml`
+* unicode fun
+* cleanup all around
+
+## 0.0.3
+* pluginized the library for Rails (thanks [pius](http://gitorious.org/projects/calais-au-rails))
+* added helper methods name entity types from a response
+
+## 0.0.2
+* cleanup in the specs
+* cleaner parsing
+* location of named entities
+* more data in relationships
+* moved Names and Relationships
+
+## 0.0.1
+* Access to OpenCalais's Enlighten action
+* Single method to process a document
+    * Get relationships and names from a document

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2008 Abhay Kumar info@opensynapse.net
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.markdown

@@ -0,0 +1,49 @@
+# Calais #
+A Ruby interface to the [Open Calais Web Service](http://opencalais.com)
+
+## Features ##
+* Accepts documents in text/plain, text/xml and text/html format.
+* Basic access to the Open Calais API's Enlighten action.
+    * Output is RDF representation of input document.
+* Single function ability to extract names, entities and geographies from given text.
+  
+## Synopsis ##
+
+This is a very basic wrapper to the Open Calais API. It uses the POST endpoint and currently supports the Enlighten action. Here's a simple call:
+
+    Calais.enlighten(
+        :content => "The government of the United Kingdom has given corporations like fast food chain McDonald's the right to award high school qualifications to employees who complete a company training program."
+        :content_type => :text, 
+        :license_id => 'your license id'
+    )
+
+This is the easiest way to get the RDF-formated response from the OpenCalais service.
+
+If you want to do something more fun like getting all sorts of fun information about a document, you can try this:
+
+    Calais.process_document(
+        :content => "The government of the United Kingdom has given corporations like fast food chain McDonald's the right to award high school qualifications to employees who complete a company training program.",
+        :content_type => :text,
+        :license_id => 'your license id'
+    )
+
+This will return an object containing information extracted from the RDF response.
+
+## Requirements ##
+
+* [Ruby 1.8.5 or better](http://ruby-lang.org)
+* [nokogiri](http://nokogiri.rubyforge.org/nokogiri/), [libxml2](http://xmlsoft.org/)
+* [curb](http://curb.rubyforge.org/), [libcurl](http://curl.haxx.se/)
+* [json](http://json.rubyforge.org/)
+
+## Install ##
+
+You can install the Calais gem via Rubygems (`gem install calais`) or by building from source.
+
+## Authors ##
+
+* [Abhay Kumar](http://opensynapse.net) 
+
+## Acknowledgements ##
+
+* [Paul Legato](http://www.economaton.com/): Help all around with the new response processor and implementation of the 3.1 API.

data/Rakefile

@@ -0,0 +1,97 @@
+# -*- ruby -*-
+
+require 'rake'
+require 'rake/clean'
+
+require './lib/calais.rb'
+
+begin
+  gem 'jeweler', '>= 1.0.1'
+  require 'jeweler'
+
+  Jeweler::Tasks.new do |s|
+    s.name = 'calais'
+    s.summary = 'A Ruby interface to the Calais Web Service'
+    s.email = 'info@opensynapse.net'
+    s.homepage = 'http://github.com/abhay/calais'
+    s.description = 'A Ruby interface to the Calais Web Service'
+    s.authors = ['Abhay Kumar']
+    s.files = FileList["[A-Z]*", "{bin,generators,lib,test}/**/*"]
+    s.rubyforge_project = 'calais'
+    s.add_dependency 'nokogiri', '>= 1.3.3'
+    s.add_dependency 'json', '>= 1.1.3'
+    s.add_dependency 'curb', '>= 0.1.4'
+  end
+rescue LoadError
+  puts "Jeweler, or one of its dependencies, is not available. Please install it."
+  exit(1)
+end
+
+begin
+  require 'spec/rake/spectask'
+
+  desc "Run all specs"
+  Spec::Rake::SpecTask.new do |t|
+    t.spec_files = FileList["spec/**/*_spec.rb"].sort
+    t.spec_opts = ["--options", "spec/spec.opts"]
+  end
+
+  desc "Run all specs and get coverage statistics"
+  Spec::Rake::SpecTask.new('coverage') do |t|
+    t.spec_opts = ["--options", "spec/spec.opts"]
+    t.spec_files = FileList["spec/*_spec.rb"].sort
+    t.rcov_opts = ["--exclude", "spec", "--exclude", "gems"]
+    t.rcov = true
+  end
+
+  task :default => :spec
+rescue LoadError
+  puts "RSpec, or one of its dependencies, is not available. Please install it."
+  exit(1)
+end
+
+begin
+  require 'yard'
+  require 'yard/rake/yardoc_task'
+
+  YARD::Rake::YardocTask.new do |t|
+    t.options = ["--verbose", "--markup=markdown", "--files=CHANGELOG.markdown,MIT-LICENSE"]
+  end
+  
+  task :rdoc => :yardoc
+  
+  CLOBBER.include 'doc'
+  CLOBBER.include '.yardoc'
+rescue LoadError
+  puts "Yard, or one of its dependencies is not available. Please install it."
+  exit(1)
+end
+
+begin
+  require 'rake/contrib/sshpublisher'
+  namespace :rubyforge do
+
+    desc "Release gem and RDoc documentation to RubyForge"
+    task :release => ["rubyforge:release:gem", "rubyforge:release:docs"]
+
+    namespace :release do
+      desc "Publish RDoc to RubyForge."
+      task :docs => [:yardoc] do
+        config = YAML.load(
+            File.read(File.expand_path('~/.rubyforge/user-config.yml'))
+        )
+
+        host = "#{config['username']}@rubyforge.org"
+        remote_dir = "/var/www/gforge-projects/calais/"
+        local_dir = 'doc'
+
+        Rake::SshDirPublisher.new(host, remote_dir, local_dir).upload
+      end
+    end
+  end
+rescue LoadError
+  puts "Rake SshDirPublisher is unavailable or your rubyforge environment is not configured."
+  exit(1)
+end
+
+# vim: syntax=Ruby

data/VERSION.yml

@@ -0,0 +1,4 @@
+--- 
+:patch: 7
+:major: 0
+:minor: 0

data/lib/calais.rb

@@ -0,0 +1,56 @@
+require 'digest/sha1'
+require 'net/http'
+require 'cgi'
+require 'iconv'
+require 'set'
+
+require 'rubygems'
+require 'nokogiri'
+require 'json'
+require 'curb'
+
+$KCODE = "UTF8"
+require 'jcode'
+
+$:.unshift File.expand_path(File.dirname(__FILE__)) + '/calais'
+
+require 'client'
+require 'response'
+require 'error'
+
+module Calais
+  REST_ENDPOINT = "http://api.opencalais.com/enlighten/rest/"
+  BETA_REST_ENDPOINT = "http://beta.opencalais.com/enlighten/rest/"
+
+  AVAILABLE_CONTENT_TYPES = {
+    :xml => 'text/xml',
+    :text => 'text/txt',
+    :html => 'text/html',
+    :raw => 'text/raw'
+  }
+
+  AVAILABLE_OUTPUT_FORMATS = {
+    :rdf => 'xml/rdf',
+    :simple => 'text/simple',
+    :microformats => 'text/microformats',
+    :json => 'application/json'
+  }
+
+  KNOWN_ENABLES = ['GenericRelations']
+  KNOWN_DISCARDS = ['er/Company', 'er/Geo']
+
+  MAX_RETRIES = 5
+  HTTP_TIMEOUT = 60
+  MIN_CONTENT_SIZE = 1
+  MAX_CONTENT_SIZE = 100_000
+
+  class << self
+    def enlighten(*args, &block); Client.new(*args, &block).enlighten; end
+
+    def process_document(*args, &block)
+      client = Client.new(*args, &block)
+      client.output_format = :rdf
+      Response.new(client.enlighten)
+    end
+  end
+end

data/lib/calais/client.rb

@@ -0,0 +1,111 @@
+module Calais
+  class Client
+    # base attributes of the call
+    attr_accessor :content
+    attr_accessor :license_id
+
+    # processing directives
+    attr_accessor :content_type, :output_format, :reltag_base_url, :calculate_relevance, :omit_outputting_original_text
+    attr_accessor :metadata_enables, :metadata_discards
+
+    # user directives
+    attr_accessor :allow_distribution, :allow_search, :external_id, :submitter
+
+    attr_accessor :external_metadata
+
+    attr_accessor :use_beta
+
+    def initialize(options={}, &block)
+      options.each {|k,v| send("#{k}=", v)}
+      yield(self) if block_given?
+    end
+
+    def enlighten
+      post_args = {
+        "licenseID" => @license_id,
+        "content" => Iconv.iconv('UTF-8//IGNORE', 'UTF-8',  "#{@content} ").first[0..-2],
+        "paramsXML" => params_xml
+      }
+
+      @client ||= Curl::Easy.new
+      @client.url = @use_beta ? BETA_REST_ENDPOINT : REST_ENDPOINT
+      @client.timeout = HTTP_TIMEOUT
+
+      post_fields = post_args.map {|k,v| Curl::PostField.content(k, v) }
+
+      do_request(post_fields)
+    end
+
+    def params_xml
+      check_params
+      document = Nokogiri::XML::Document.new
+      
+      params_node = Nokogiri::XML::Node.new('c:params', document)
+      params_node['xmlns:c'] = 'http://s.opencalais.com/1/pred/'
+      params_node['xmlns:rdf'] = 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'
+      
+      processing_node = Nokogiri::XML::Node.new('c:processingDirectives', document)
+      processing_node['c:contentType'] = AVAILABLE_CONTENT_TYPES[@content_type] if @content_type
+      processing_node['c:outputFormat'] = AVAILABLE_OUTPUT_FORMATS[@output_format] if @output_format
+      processing_node['c:reltagBaseURL'] = @reltag_base_url.to_s if @reltag_base_url
+      
+      processing_node['c:enableMetadataType'] = @metadata_enables.join(';') unless @metadata_enables.empty?
+      processing_node['c:discardMetadata'] = @metadata_discards.join(';') unless @metadata_discards.empty?
+      processing_node['c:omitOutputtingOriginalText'] = 'true' if @omit_outputting_original_text
+      
+      user_node = Nokogiri::XML::Node.new('c:userDirectives', document)
+      user_node['c:allowDistribution'] = @allow_distribution.to_s unless @allow_distribution.nil?
+      user_node['c:allowSearch'] = @allow_search.to_s unless @allow_search.nil?
+      user_node['c:externalID'] = @external_id.to_s if @external_id
+      user_node['c:submitter'] = @submitter.to_s if @submitter
+      
+      params_node << processing_node
+      params_node << user_node
+      
+      if @external_metadata
+        external_node = Nokogiri::XML::Node.new('c:externalMetadata', document)
+        external_node << @external_metadata
+        params_node << external_node
+      end
+      
+      params_node.to_xml(:indent => 2)
+    end
+
+    private
+      def check_params
+        raise 'missing content' if @content.nil? || @content.empty?
+
+        content_length = @content.length
+        raise 'content is too small' if content_length < MIN_CONTENT_SIZE
+        raise 'content is too large' if content_length > MAX_CONTENT_SIZE
+
+        raise 'missing license id' if @license_id.nil? || @license_id.empty?
+
+        raise 'unknown content type' unless AVAILABLE_CONTENT_TYPES.keys.include?(@content_type) if @content_type
+        raise 'unknown output format' unless AVAILABLE_OUTPUT_FORMATS.keys.include?(@output_format) if @output_format
+
+        %w[calculate_relevance allow_distribution allow_search].each do |variable|
+          value = self.send(variable)
+          unless NilClass === value || TrueClass === value || FalseClass === value
+            raise "expected a boolean value for #{variable} but got #{value}"
+          end
+        end
+
+        @metadata_enables ||= []
+        unknown_enables = Set.new(@metadata_enables) - KNOWN_ENABLES
+        raise "unknown metadata enables: #{unknown_enables.to_ainspect}" unless unknown_enables.empty?
+
+        @metadata_discards ||= []
+        unknown_discards = Set.new(@metadata_discards) - KNOWN_DISCARDS
+        raise "unknown metadata discards: #{unknown_discards.to_ainspect}" unless unknown_discards.empty?
+      end
+
+      def do_request(post_fields)
+        unless @client.http_post(post_fields)
+          raise 'unable to post to api endpoint'
+        end
+
+        @client.body_str
+      end
+  end
+end

data/lib/calais/error.rb

@@ -0,0 +1,3 @@
+class Calais::Error < StandardError
+  
+end

data/lib/calais/response.rb

@@ -0,0 +1,201 @@
+module Calais
+  class Response
+    MATCHERS = {
+      :docinfo => 'DocInfo',
+      :docinfometa => 'DocInfoMeta',
+      :defaultlangid => 'DefaultLangId',
+      :doccat => 'DocCat',
+      :entities => 'type/em/e',
+      :relations => 'type/em/r',
+      :geographies => 'type/er',
+      :instances => 'type/sys/InstanceInfo',
+      :relevances => 'type/sys/RelevanceInfo',
+    }
+
+    attr_accessor :submitter_code, :signature, :language, :submission_date, :request_id, :doc_title, :doc_date
+    attr_accessor :hashes, :entities, :relations, :geographies, :categories
+
+    def initialize(rdf_string)
+      @raw_response = rdf_string
+
+      @hashes = []
+      @entities = []
+      @relations = []
+      @geographies = []
+      @relevances = {} # key = String hash, val = Float relevance
+      @categories = []
+
+      extract_data
+    end
+
+    class Entity
+      attr_accessor :calais_hash, :type, :attributes, :relevance, :instances
+    end
+
+    class Relation
+      attr_accessor :calais_hash, :type, :attributes, :instances
+    end
+
+    class Geography
+      attr_accessor :name, :calais_hash, :attributes
+    end
+
+    class Category
+      attr_accessor :name, :score
+    end
+
+    class Instance
+      attr_accessor :prefix, :exact, :suffix, :offset, :length
+
+      # Makes a new Instance object from an appropriate Nokogiri::XML::Node.
+      def self.from_node(node)
+        instance = self.new
+        instance.prefix = node.xpath("c:prefix[1]").first.content
+        instance.exact  = node.xpath("c:exact[1]").first.content
+        instance.suffix = node.xpath("c:suffix[1]").first.content
+        instance.offset = node.xpath("c:offset[1]").first.content.to_i
+        instance.length = node.xpath("c:length[1]").first.content.to_i
+
+        instance
+      end
+    end
+
+    class CalaisHash
+      attr_accessor :value
+
+      def self.find_or_create(hash, hashes)
+        if !selected = hashes.select {|h| h.value == hash }.first
+          selected = self.new
+          selected.value = hash
+          hashes << selected
+        end
+
+        selected
+      end
+    end
+
+    private
+      def extract_data
+        doc = Nokogiri::XML(@raw_response)
+
+        if doc.root.xpath("/Error[1]").first
+          raise Calais::Error, doc.root.xpath("/Error/Exception").first.content
+        end        
+
+        doc.root.xpath("rdf:Description/rdf:type[contains(@rdf:resource, '#{MATCHERS[:docinfometa]}')]/..").each do |node|
+          @language = node['language']
+          @submission_date =  DateTime.parse node['submissionDate']
+
+          attributes = extract_attributes(node.xpath("*[contains(name(), 'c:')]"))
+
+          @signature = attributes.delete('signature')
+          @submitter_code = attributes.delete('submitterCode')
+
+          node.remove
+        end
+
+        doc.root.xpath("rdf:Description/rdf:type[contains(@rdf:resource, '#{MATCHERS[:docinfo]}')]/..").each do |node|
+          @request_id = node['calaisRequestID']
+
+          attributes = extract_attributes(node.xpath("*[contains(name(), 'c:')]"))
+
+          @doc_title = attributes.delete('docTitle')
+          @doc_date = Date.parse(attributes.delete('docDate')) 
+
+          node.remove
+        end
+
+        @categories = doc.root.xpath("rdf:Description/rdf:type[contains(@rdf:resource, '#{MATCHERS[:doccat]}')]/..").map do |node|
+          category = Category.new
+          category.name = node.xpath("c:categoryName[1]").first.content
+          score = node.xpath("c:score[1]").first
+          category.score = score.content.to_f unless score.nil?
+
+          node.remove
+          category
+        end
+
+        @relevances = doc.root.xpath("rdf:Description/rdf:type[contains(@rdf:resource, '#{MATCHERS[:relevances]}')]/..").inject({}) do |acc, node|
+          subject_hash = node.xpath("c:subject[1]").first[:resource].split('/')[-1]
+          acc[subject_hash] = node.xpath("c:relevance[1]").first.content.to_f
+
+          node.remove
+          acc
+        end
+
+        @entities = doc.root.xpath("rdf:Description/rdf:type[contains(@rdf:resource, '#{MATCHERS[:entities]}')]/..").map do |node|
+          extracted_hash = node['about'].split('/')[-1] rescue nil
+
+          entity = Entity.new
+          entity.calais_hash = CalaisHash.find_or_create(extracted_hash, @hashes)
+          entity.type = extract_type(node)
+          entity.attributes = extract_attributes(node.xpath("*[contains(name(), 'c:')]"))
+
+          entity.relevance = @relevances[extracted_hash]
+          entity.instances = extract_instances(doc, extracted_hash)
+
+          node.remove
+          entity
+        end
+
+        @relations = doc.root.xpath("rdf:Description/rdf:type[contains(@rdf:resource, '#{MATCHERS[:relations]}')]/..").map do |node|
+          extracted_hash = node['about'].split('/')[-1] rescue nil
+
+          relation = Relation.new
+          relation.calais_hash = CalaisHash.find_or_create(extracted_hash, @hashes)
+          relation.type = extract_type(node)
+          relation.attributes = extract_attributes(node.xpath("*[contains(name(), 'c:')]"))
+          relation.instances = extract_instances(doc, extracted_hash)
+
+          node.remove
+          relation
+        end
+
+        @geographies = doc.root.xpath("rdf:Description/rdf:type[contains(@rdf:resource, '#{MATCHERS[:geographies]}')]/..").map do |node|
+          attributes = extract_attributes(node.xpath("*[contains(name(), 'c:')]"))
+
+          geography = Geography.new
+          geography.name = attributes.delete('name')
+          geography.calais_hash = attributes.delete('subject')
+          geography.attributes = attributes
+
+          node.remove
+          geography
+        end
+
+        doc.root.xpath("rdf:Description/rdf:type[contains(@rdf:resource, '#{MATCHERS[:defaultlangid]}')]/..").each { |node| node.remove }
+        doc.root.xpath("./*").each { |node| node.remove }
+
+        return
+      end
+
+      def extract_instances(doc, hash)
+        doc.root.xpath("rdf:Description/rdf:type[contains(@rdf:resource, '#{MATCHERS[:instances]}')]/..").select do |instance_node|
+          instance_node.xpath("c:subject[1]").first[:resource].split("/")[-1] == hash
+        end.map do |instance_node|
+          instance = Instance.from_node(instance_node)
+          instance_node.remove
+
+          instance
+        end
+      end
+
+      def extract_type(node)
+        node.xpath("*[name()='rdf:type']")[0]['resource'].split('/')[-1]
+      rescue
+        nil
+      end
+
+      def extract_attributes(nodes)
+        nodes.inject({}) do |hsh, node|
+          value = if node['resource']
+              extracted_hash = node['resource'].split('/')[-1] rescue nil
+              CalaisHash.find_or_create(extracted_hash, @hashes)
+            else
+              node.content
+            end
+          hsh.merge(node.name => value)
+        end
+      end
+  end
+end

data/spec/calais/client_spec.rb

@@ -0,0 +1,79 @@
+require File.join(File.dirname(__FILE__), %w[.. helper])
+
+describe Calais::Client, :new do
+  it 'accepts arguments as a hash' do
+    client = nil
+
+    lambda { client = Calais::Client.new(:content => SAMPLE_DOCUMENT, :license_id => LICENSE_ID) }.should_not raise_error
+
+    client.license_id.should == LICENSE_ID
+    client.content.should == SAMPLE_DOCUMENT
+  end
+
+  it 'accepts arguments as a block' do
+    client = nil
+
+    lambda {
+      client = Calais::Client.new do |c|
+        c.content = SAMPLE_DOCUMENT
+        c.license_id = LICENSE_ID
+      end
+    }.should_not raise_error
+
+    client.license_id.should == LICENSE_ID
+    client.content.should == SAMPLE_DOCUMENT
+  end
+
+  it 'should not accept unknown attributes' do
+    lambda { Calais::Client.new(:monkey => 'monkey', :license_id => LICENSE_ID) }.should raise_error(NoMethodError)
+  end
+end
+
+describe Calais::Client, :params_xml do
+  it 'returns an xml encoded string' do
+    client = Calais::Client.new(:content => SAMPLE_DOCUMENT, :license_id => LICENSE_ID)
+    client.params_xml.should == %[<c:params xmlns:c=\"http://s.opencalais.com/1/pred/\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\"><c:processingDirectives/><c:userDirectives/></c:params>]
+
+    client.content_type = :xml
+    client.output_format = :json
+    client.reltag_base_url = 'http://opencalais.com'
+    client.calculate_relevance = true
+    client.metadata_enables = Calais::KNOWN_ENABLES
+    client.metadata_discards = Calais::KNOWN_DISCARDS
+    client.allow_distribution = true
+    client.allow_search = true
+    client.external_id = Digest::SHA1.hexdigest(client.content)
+    client.submitter = 'calais.rb'
+
+    client.params_xml.should == %[<c:params xmlns:c="http://s.opencalais.com/1/pred/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><c:processingDirectives c:contentType="text/xml" c:outputFormat="application/json" c:reltagBaseURL="http://opencalais.com" c:enableMetadataType="GenericRelations" c:discardMetadata="er/Company;er/Geo"/><c:userDirectives c:allowDistribution="true" c:allowSearch="true" c:externalID="1a008b91e7d21962e132bc1d6cb252532116a606" c:submitter="calais.rb"/></c:params>]
+  end
+end
+
+describe Calais::Client, :enlighten do
+  before do
+    @client = Calais::Client.new do |c|
+      c.content = SAMPLE_DOCUMENT
+      c.license_id = LICENSE_ID
+      c.content_type = :xml
+      c.output_format = :json
+      c.calculate_relevance = true
+      c.metadata_enables = Calais::KNOWN_ENABLES
+      c.allow_distribution = true
+      c.allow_search = true
+    end
+  end
+
+  it 'provides access to the enlighten command on the generic rest endpoint' do
+    @client.should_receive(:do_request).with(anything).and_return(SAMPLE_RESPONSE)
+    @client.enlighten
+    @client.instance_variable_get(:@client).url.should == Calais::REST_ENDPOINT
+  end
+
+  it 'provides access to the enlighten command on the beta rest endpoint' do
+    @client.use_beta = true
+
+    @client.should_receive(:do_request).with(anything).and_return(SAMPLE_RESPONSE)
+    @client.enlighten
+    @client.instance_variable_get(:@client).url.should == Calais::BETA_REST_ENDPOINT
+  end
+end

data/spec/calais/response_spec.rb

@@ -0,0 +1,128 @@
+require File.join(File.dirname(__FILE__), %w[.. helper])
+
+describe Calais::Response, :new do
+  it 'accepts an rdf string to generate the response object' do
+    lambda { Calais::Response.new(SAMPLE_RESPONSE) }.should_not raise_error
+  end
+end
+
+describe Calais::Response, :new do
+  it "should return error message in runtime error" do
+    lambda {
+      @response = Calais::Response.new(RESPONSE_WITH_EXCEPTION)
+    }.should raise_error(Calais::Error, "My Error Message") 
+  end
+end
+
+describe Calais::Response, :new do
+  before :all do
+    @response = Calais::Response.new(SAMPLE_RESPONSE)
+  end
+
+  it 'should extract document information' do
+    @response.language.should == 'English'
+    @response.submission_date.should be_a_kind_of(DateTime)
+    @response.signature.should be_a_kind_of(String)
+    @response.submitter_code.should be_a_kind_of(String)
+    @response.request_id.should be_a_kind_of(String)
+    @response.doc_title.should == 'Record number of bicycles sold in Australia in 2006'
+    @response.doc_date.should be_a_kind_of(Date)
+  end
+
+  it 'should extract entities' do
+    entities = @response.entities
+    entities.map { |e| e.type }.sort.uniq.should == %w[City Continent Country IndustryTerm Organization Person Position ProvinceOrState]
+  end
+
+  it 'should extract relations' do
+    relations = @response.relations
+    relations.map { |e| e.type }.sort.uniq.should == %w[GenericRelations PersonAttributes PersonCareer Quotation]
+  end
+
+  it 'should extract geographies' do
+    geographies = @response.geographies
+    geographies.map { |e| e.name }.sort.uniq.should == %w[Australia Hobart,Tasmania,Australia Tasmania,Australia]
+  end
+
+  it 'should extract relevances' do
+    @response.instance_variable_get(:@relevances).should be_a_kind_of(Hash)
+  end
+
+  it 'should assign a floating-point relevance to each entity' do
+    @response.entities.each {|e| e.relevance.should be_a_kind_of(Float) }
+  end
+
+  it 'should find the correct document categories returned by OpenCalais' do
+    @response.categories.map {|c| c.name }.sort.should == %w[Business_Finance Technology_Internet]
+  end
+
+  it 'should find the correct document category scores returned by OpenCalais' do
+    @response.categories.map {|c| c.score.should be_a_kind_of(Float) }
+  end
+  
+  it "should not raise an error if no score is given by OpenCalais" do
+    lambda {Calais::Response.new(SAMPLE_RESPONSE_WITH_NO_SCORE)}.should_not raise_error
+  end
+  
+  it "should not raise an error if no score is given by OpenCalais" do
+    response = Calais::Response.new(SAMPLE_RESPONSE_WITH_NO_SCORE)
+    response.categories.map {|c| c.score }.should == [nil]
+  end
+  
+  it 'should find instances for each entity' do
+    @response.entities.each {|e|
+      e.instances.size.should > 0
+    }
+  end
+
+
+  it 'should find instances for each relation' do
+    @response.relations.each {|r|
+      r.instances.size.should > 0
+    }
+  end
+
+  it 'should find the correct instances for each entity' do
+    ## This currently tests only for the "Australia" entity's
+    ## instances.  A more thorough test that tests for the instances
+    ## of each of the many entities in the sample doc is desirable in
+    ## the future.
+
+    australia = @response.entities.select {|e| e.attributes["name"] == "Australia" }.first
+    australia.instances.size.should == 3
+    instances = australia.instances.sort{|a,b| a.offset <=> b.offset }
+
+    instances[0].prefix.should == "number of bicycles sold in "
+    instances[0].exact.should == "Australia"
+    instances[0].suffix.should == " in 2006<\/title>\n<date>January 4,"
+    instances[0].offset.should == 67
+    instances[0].length.should == 9
+
+    instances[1].prefix.should == "4, 2007<\/date>\n<body>\nBicycle sales in "
+    instances[1].exact.should == "Australia"
+    instances[1].suffix.should == " have recorded record sales of 1,273,781 units"
+    instances[1].offset.should == 146
+    instances[1].length.should == 9
+
+    instances[2].prefix.should == " the traditional company car,\" he said.\n\n\"Some of "
+    instances[2].exact.should == "Australia"
+    instances[2].suffix.should == "'s biggest corporations now have bicycle fleets,"
+    instances[2].offset.should == 952
+    instances[2].length.should == 9
+  end
+
+  it 'should find the correct instances for each relation' do
+    ## This currently tests only for one relation's instances.  A more
+    ## thorough test that tests for the instances of each of the many other
+    ## relations in the sample doc is desirable in the future.
+
+    rel = @response.relations.select {|e| e.calais_hash.value == "8f3936d9-cf6b-37fc-ae0d-a145959ae3b5" }.first
+    rel.instances.size.should == 1
+
+    rel.instances.first.prefix.should == " manufacturers.\n\nThe Cycling Promotion Fund (CPF) "
+    rel.instances.first.exact.should == "spokesman Ian Christie said Australians were increasingly using bicycles as an alternative to cars."
+    rel.instances.first.suffix.should == " Sales rose nine percent in 2006 while the car"
+    rel.instances.first.offset.should == 425
+    rel.instances.first.length.should == 99
+  end
+end

data/spec/helper.rb

@@ -0,0 +1,12 @@
+require 'rubygems'
+require 'spec'
+require 'yaml'
+
+require File.dirname(__FILE__) + '/../lib/calais'
+
+FIXTURES_DIR = File.join File.dirname(__FILE__), %[fixtures]
+SAMPLE_DOCUMENT = File.read(File.join(FIXTURES_DIR, %[bicycles_australia.xml]))
+SAMPLE_RESPONSE = File.read(File.join(FIXTURES_DIR, %[bicycles_australia.response.rdf]))
+SAMPLE_RESPONSE_WITH_NO_SCORE = File.read(File.join(FIXTURES_DIR, %[twitter_tweet_without_score.response.rdf]))
+RESPONSE_WITH_EXCEPTION = File.read(File.join(FIXTURES_DIR, %[error.response.xml]))
+LICENSE_ID = YAML.load(File.read(File.join(FIXTURES_DIR, %[calais.yml])))['key']

metadata

@@ -0,0 +1,93 @@
+--- !ruby/object:Gem::Specification 
+name: documentcloud-calais
+version: !ruby/object:Gem::Version 
+  version: 0.0.7
+platform: ruby
+authors: 
+- Abhay Kumar
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2009-06-08 00:00:00 -07:00
+default_executable: 
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: nokogiri
+  type: :runtime
+  version_requirement: 
+  version_requirements: !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        version: 1.3.3
+    version: 
+- !ruby/object:Gem::Dependency 
+  name: json
+  type: :runtime
+  version_requirement: 
+  version_requirements: !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        version: 1.1.3
+    version: 
+- !ruby/object:Gem::Dependency 
+  name: curb
+  type: :runtime
+  version_requirement: 
+  version_requirements: !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        version: 0.1.4
+    version: 
+description: A Ruby interface to the Calais Web Service
+email: info@opensynapse.net
+executables: []
+
+extensions: []
+
+extra_rdoc_files: 
+- README.markdown
+files: 
+- CHANGELOG.markdown
+- MIT-LICENSE
+- README.markdown
+- Rakefile
+- VERSION.yml
+- lib/calais.rb
+- lib/calais/client.rb
+- lib/calais/error.rb
+- lib/calais/response.rb
+has_rdoc: true
+homepage: http://github.com/abhay/calais
+licenses: 
+post_install_message: 
+rdoc_options: 
+- --charset=UTF-8
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+  version: 
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: "0"
+  version: 
+requirements: []
+
+rubyforge_project: calais
+rubygems_version: 1.3.5
+signing_key: 
+specification_version: 2
+summary: A Ruby interface to the Calais Web Service
+test_files: 
+- spec/helper.rb
+- spec/calais/response_spec.rb
+- spec/calais/client_spec.rb