michael-ken 0.0.3 → 0.1.0

data/README.textile

@@ -5,18 +5,15 @@ h2. Introduction
 Ken is a Data Layer for Knowledge Representation.
 
 It's being built to access the Metaweb Services supplied by Freebase.com.
-Just born, the project’s goals are the provision of a concise API for querying and writing.
+The project’s goals are the provision of a concise API for querying and writing.
 Therefore it wraps the Metaweb Architecture to smart Ruby Objects.
 
 You can navigate the Freebase Graph using a rubyish syntax.
-
-If things go right, you should be able to use this library as a Data Layer (instead of or in addition to 
-ActiveRecord/DataMapper) for your Web Framework of choice (Merb, Rails).
-
+Also you can use this library as a Data Layer (instead of or in addition to ActiveRecord/DataMapper) for your Web Framework of choice (Merb, Rails).
 
 h2. Installation
 
-Use GitHub RubyGems.
+Use GitHub RubyGems:
 
 <pre>
 <code>
@@ -25,8 +22,15 @@ Use GitHub RubyGems.
 </code>
 </pre>
 
-Or even better, stay on The Edge.
-  
+In your Ruby files add:
+<pre>
+<code>
+  require 'rubygems'
+  gem 'michael-ken'
+  require 'ken'
+</code>
+</pre>
+
 
 h2. Getting started
 
@@ -159,7 +163,7 @@ So asking for values in a nested level does not make sense. Use nested statement
 lowering the top level result.
 
 However you can instead navigate the normal way to figure out that values.
-_But won't that require another query to triggered? Doubtful._
+_But won't that require another query to triggered? Certainly._
 
 Let's look at a nested query:
 
@@ -182,32 +186,76 @@ Let's look at a nested query:
 </code>
 </pre>
 
-h2. Status
+h3. Access properties attributes directly
+
+Ken is primarily designed for inspecting resources in a generic way, what's ideal for domain independent browsing applications.
+However, there are legitimate situations where you already know what you want to access.
+
+That's why I now added direct Property/Attribute access, but only on a Type/View level:
+
+<pre>
+<code>
+  resource = Ken.get('/en/new_order')
+  type = resource.types[1] # => #<Type id="/music/artist" name="Musical Artist">
+  # because we know _/music/artist_ has a _genre_ property we can access that directly
+  type.genre # => #<Property id="/music/artist/genre" expected_type="/music/genre" unique="false" object_type="true">
+</code>
+</pre>
+The same works for views:
+<pre>
+<code>
+  resource = Ken.get('/en/new_order')
+  view = resource.views[1] # => #<View type="/music/artist">
+  # because we know _/music/artist_ has a _genre_ property we can access attribute directly as well
+  view.genre # => #<Attribute property="/music/artist/genre">
+</code>
+</pre>
+
+If you rather want to query based on Types and access Properties/Attributes directly you can consider using
+Chris "Eppsteins Freebase Library":http://github.com/chriseppstein/freebase/tree/master as an alternative. 
+
 
-Currently, the focus lies on inspecting Freebase Resources in a generic way. That's why
-there is no support for accessing Properties or Attributes directly when you know them already.
-That keeps me focussed and prevents me from adding too much sugar.
+h3. Low Level API
 
-Therefore I would say that the first application using Ken will definitely be a Browser Application. :)
+Sometimes you may want to do specific queries instead of inspecting Resources as a whole.
+In such a case you would want to use Ken's low level API.
+
+_mqlread_ works like the regular _mqlread service_, except that you are able to pass ruby hashes instead of JSON.
+And you don't have to deal with HTTP, parameter encoding and parsing JSON.
+
+<pre>
+<code>
+  artists = Ken.session.mqlread([{
+    :type => "/music/artist",
+    :id => nil, 
+    :"/common/topic/webpage" => [{:uri => nil}], 
+    :home_page => [{:uri => nil}], 
+    :limit => 2
+  }])  
+  
+  # => [
+         {"type"=>"/music/artist", "home_page"=>[{"uri"=>"http://www.massiveattack.co.uk/"}], "id"=>"/en/massive_attack", "/common/topic/webpage"=>[{"uri"=>"http://musicmoz.org/Bands_and_Artists/M/Massive_Attack/"}, {"uri"=>"http://www.discogs.com/artist/Massive+Attack"}, {"uri"=>"http://www.massiveattackarea.com/"}, {"uri"=>"http://www.massiveattack.co.uk/"}, {"uri"=>"http://www.massiveattack.com/"}]},
+         {"type"=>"/music/artist", "home_page"=>[{"uri"=>"http://www.apartment26.com/"}], "id"=>"/en/apartment_26", "/common/topic/webpage"=>[{"uri"=>"http://www.discogs.com/artist/Apartment+26"}, {"uri"=>"http://musicmoz.org/Bands_and_Artists/A/Apartment_26/"}, {"uri"=>"http://www.apartment26.com/"}]}
+       ]
+</code>
+</pre>
 
-If you rather want to query based on Types and access Properties/Attributes directly you should consider using
-Chris "Eppsteins Freebase Library":http://github.com/chriseppstein/freebase/tree/master instead. 
+h2. Project Status
 
 h3. Features
 
 * Fetching of single Resources
 * Fetching of multiple Resources by specifying a query
+* Accessing Properties/Attributes directly (on a type/view level)
 * Type inspection
-* Attributes inspection
+* Attribute inspection
+* Low Level API (mqlread)
+* Rails and Merb support
 * Views on Resources to group Attributes based on the Resource's types
-* Some specs
 
 h3. Roadmap
 
-# Much more specs
-# Better Type Support
-# API for Set Based Browsing (see http://mqlx.com/~david/parallax/)
-# Accessing Properties/Attributes directly (e.g. resource.genres )
+# More tests
 # Write-Support
 
 Initial thoughts, obviously not up-to-date and not conforming to the current version, are available at "http://wiki.github.com/michael/ken":http://wiki.github.com/michael/ken.

data/Rakefile

@@ -9,6 +9,7 @@ begin
     gem.email = "ma[at]zive[dot]at"
     gem.homepage = "http://github.com/michael/ken"
     gem.authors = ["michael"]
+    gem.add_dependency('extlib')
     # gem.files = FileList["[A-Z]*.*"]
     # gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
   end
@@ -37,7 +38,6 @@ rescue LoadError
   end
 end
 
-
 task :default => :test
 
 require 'rake/rdoctask'

data/VERSION

@@ -1 +1 @@
-0.0.3
+0.1.0

data/examples/artist.rb

@@ -10,10 +10,10 @@ resource = Ken.get('/en/the_police')
 
 resource.views.each do |view|
   puts view
-  puts "==============================="
+  puts "="*20
   view.attributes.each do |a|
     puts a.property
-    puts "----------------"
+    puts "-"*20
     puts a
     puts # newline
   end

data/examples/artist_links.rb

@@ -0,0 +1,28 @@
+require 'pathname'
+require 'rubygems'
+
+# displays all links related to music/artists
+# low level api through Ken.session.mqlread is used here
+
+EXAMPLES_ROOT = Pathname(__FILE__).dirname.expand_path
+require EXAMPLES_ROOT.parent + 'lib/ken'
+
+Ken::Session.new('http://www.freebase.com', 'ma', '*****')
+
+artists = Ken.session.mqlread([{
+  :type => "/music/artist",
+  :id => nil, 
+  :"/common/topic/webpage" => [{:uri => nil}], 
+  :home_page => [{:uri => nil}], 
+  :limit => 50
+}])
+
+artists.each do |artist|
+  artist["/common/topic/webpage"].each do |webpage|
+    puts webpage["uri"]
+  end
+  
+  artist["home_page"].each do |homepage|
+    puts homepage["uri"]
+  end
+end

data/ken.gemspec

@@ -0,0 +1,82 @@
+# -*- encoding: utf-8 -*-
+
+Gem::Specification.new do |s|
+  s.name = %q{ken}
+  s.version = "0.1.0"
+
+  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+  s.authors = ["michael"]
+  s.date = %q{2009-07-08}
+  s.email = %q{ma[at]zive[dot]at}
+  s.extra_rdoc_files = [
+    "LICENSE",
+     "README.textile",
+     "README.txt"
+  ]
+  s.files = [
+    ".gitignore",
+     "History.txt",
+     "LICENSE",
+     "README.textile",
+     "README.txt",
+     "Rakefile",
+     "TODO",
+     "VERSION",
+     "examples/artist.rb",
+     "examples/artist_links.rb",
+     "ken.gemspec",
+     "lib/ken.rb",
+     "lib/ken/attribute.rb",
+     "lib/ken/collection.rb",
+     "lib/ken/logger.rb",
+     "lib/ken/property.rb",
+     "lib/ken/resource.rb",
+     "lib/ken/session.rb",
+     "lib/ken/type.rb",
+     "lib/ken/util.rb",
+     "lib/ken/view.rb",
+     "rails/init.rb",
+     "tasks/ken.rb",
+     "tasks/spec.rb",
+     "test/fixtures/music_artist.json",
+     "test/fixtures/the_police.json",
+     "test/integration/ken_test.rb",
+     "test/test_helper.rb",
+     "test/unit/attribute_test.rb",
+     "test/unit/property_test.rb",
+     "test/unit/resource_test.rb",
+     "test/unit/session_test.rb",
+     "test/unit/type_test.rb",
+     "test/unit/view_test.rb"
+  ]
+  s.homepage = %q{http://github.com/michael/ken}
+  s.rdoc_options = ["--charset=UTF-8"]
+  s.require_paths = ["lib"]
+  s.rubygems_version = %q{1.3.4}
+  s.summary = %q{Ruby API for Accessing the Freebase}
+  s.test_files = [
+    "test/integration/ken_test.rb",
+     "test/test_helper.rb",
+     "test/unit/attribute_test.rb",
+     "test/unit/property_test.rb",
+     "test/unit/resource_test.rb",
+     "test/unit/session_test.rb",
+     "test/unit/type_test.rb",
+     "test/unit/view_test.rb",
+     "examples/artist.rb",
+     "examples/artist_links.rb"
+  ]
+
+  if s.respond_to? :specification_version then
+    current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
+    s.specification_version = 3
+
+    if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
+      s.add_runtime_dependency(%q<extlib>, [">= 0"])
+    else
+      s.add_dependency(%q<extlib>, [">= 0"])
+    end
+  else
+    s.add_dependency(%q<extlib>, [">= 0"])
+  end
+end

data/lib/ken/attribute.rb

@@ -2,7 +2,6 @@ module Ken
   class Attribute
     
     include Extlib::Assertions
-    
     attr_reader :property
     
     # initializes a resource by json result
@@ -12,6 +11,8 @@ module Ken
       @data, @property = data, property
     end
     
+    # factory method for creating an attribute instance
+    # @api semipublic
     def self.create(data, property)
       Ken::Attribute.new(data, property)
     end
@@ -26,26 +27,34 @@ module Ken
       result = "#<Attribute property=\"#{property.id || "nil"}\">"
     end
     
-    # returns just the subject
+    # returns a collection of values
+    # in case of a unique property the array holds just one value
+    # @api public
     def values
       subject
     end
     
+    # unique properties can have at least one value
+    # @api public
     def unique?
       @property.unique?
     end
     
+    # object type properties always link to resources
+    # @api public
     def object_type?
       @property.object_type?
     end
     
-    # is this a good idea? using the values method seems more natural to me
-    # def method_missing(name, *args, &block)
-    #   subject.send(name, *args, &block)
-    # end
+    # returns true if the property is a value type
+    # value type properties refer to simple values like /type/text
+    # @api public
+    def value_type?
+      @property.value_type?
+    end
     
-    # initializes the subject if used the first time
     private
+    # initializes the subject if used for the first time
     def subject
       @subject ||= Ken::Collection.new(@data.map { |r| object_type? ? Ken::Resource.new(r) : r["value"] })
     end

data/lib/ken/collection.rb

@@ -1,13 +1,5 @@
 module Ken
   class Collection < Array
-    # def initialize(data = [])
-    #   @data = data
-    # end
-    
-    # def method_missing(name, *args, &blk)
-    #   @data.send name, *args, &blk
-    # end
-    
     # add a linebreak after each entry
     def to_s
       self.inject("") { |m,i| "#{m}#{i.to_s}\n"}

data/lib/ken/property.rb

@@ -22,11 +22,13 @@ module Ken
       @data, @type = data, type
     end
     
+    # property id
     # @api public
     def id
       @data["id"]
     end
     
+    # property name
     # @api public
     def name
       @data["name"]
@@ -50,11 +52,13 @@ module Ken
       @type
     end
     
+    # reverse property, which represent incoming links
     # @api public
     def reverse_property
       @data["reverse_property"]
     end
     
+    # master property, which represent an outgoing link (or primitive value)
     # @api public
     def master_property
       @data["master_property"]
@@ -78,6 +82,7 @@ module Ken
       VALUE_TYPES.include?(expected_type)
     end
     
+    # type, which attribute values of that property are expected to have
     # @api public
     def expected_type
       @data["expected_type"]

data/lib/ken/resource.rb

@@ -1,6 +1,5 @@
 module Ken
   class Resource
-    
     include Extlib::Assertions
     
     FETCH_SCHEMA_QUERY = {
@@ -19,6 +18,7 @@ module Ken
     }
     
     FETCH_ATTRIBUTES_QUERY = {
+      # :id => id # needs to be merg!d in instance method
       :"/type/reflect/any_master" => [
         {
           :id => nil,
@@ -37,27 +37,27 @@ module Ken
         {
           :link => nil,
           :value => nil
+          # TODO: support multiple language
           # :lang => "/lang/en",
           # :type => "/type/text"
         }
-      ],
-      
-      # :id => id # needs to be merg!d in instance method
+      ]
     }
     
-    
-    # initializes a resource by json result
+    # initializes a resource using a json result
     def initialize(data)
       assert_kind_of 'data', data, Hash
       # intialize lazy if there is no type supplied
       @schema_loaded, @attributes_loaded, @data = false, false, data
     end
     
+    # resource id
     # @api public
     def id
       @data["id"] || ""
     end
     
+    # resource name
     # @api public
     def name
       @data["name"] || ""
@@ -80,7 +80,7 @@ module Ken
       @types
     end
     
-    # returns all available vies based on the assigned types
+    # returns all available views based on the assigned types
     # @api public
     def views
       @views ||= Ken::Collection.new(types.map { |type| Ken::View.new(self, type) })
@@ -96,7 +96,7 @@ module Ken
       @properties
     end
     
-    # returns all attributes for every type the resource is an instance from
+    # returns all attributes for every type the resource is an instance of
     # @api public
     def attributes
       load_attributes! unless attributes_loaded?
@@ -116,15 +116,18 @@ module Ken
     end
     
     private
+    # executes the fetch attributes query in order to load the full set if attributes
+    # more info at http://lists.freebase.com/pipermail/developers/2007-December/001022.html
+    # @api private
     def fetch_attributes
-      # fetching all objects regardless of the type
-      # check this http://lists.freebase.com/pipermail/developers/2007-December/001022.html
       Ken.session.mqlread(FETCH_ATTRIBUTES_QUERY.merge!(:id => id))
     end
     
+    # loads the full set of attributes using reflection
+    # information is extracted from master, value and reverse attributes
+    # @api private
     def load_attributes!
       data = @data["ken:attribute"] || fetch_attributes
-      
       # master & value attributes
       raw_attributes = Ken::Util.convert_hash(data["/type/reflect/any_master"])
       raw_attributes.merge!(Ken::Util.convert_hash(data["/type/reflect/any_value"]))
@@ -146,18 +149,18 @@ module Ken
       @attributes_loaded = true
     end
     
+    # executes the fetch schema query in order to load all schema information
+    # @api private
     def fetch_schema
       Ken.session.mqlread(FETCH_SCHEMA_QUERY.merge!(:id => id))["ken:type"]
     end
     
-    # loads the resources metainfo
+    # loads the resource's metainfo
     # @api private
     def load_schema!
       @data["ken:type"] ||= fetch_schema
       @types = Ken::Collection.new(@data["ken:type"].map { |type| Ken::Type.new(type) })
       @schema_loaded = true
     end
-    
-
   end # class Resource
 end # module Ken

data/lib/ken/session.rb

@@ -16,10 +16,13 @@ module Ken
     end
   end
   
+  class AttributeNotFound < StandardError; end
+  class PropertyNotFound < StandardError; end
+  class ResourceNotFound < StandardError; end
+  
   # partially taken from chris eppstein's freebase api
   # http://github.com/chriseppstein/freebase/tree
   class Session
-    # include Singleton
     
     public
     # Initialize a new Ken Session
@@ -48,7 +51,6 @@ module Ken
 
     # get the service url for the specified service.
     def service_url(svc)
-      #"http://#{Configuration.instance[:host]}#{SERVICES[svc]}"
       "#{@host}#{SERVICES[svc]}"
     end
 
@@ -67,30 +69,51 @@ module Ken
       end
     end # handle_read_error
 
+
     # perform a mqlread and return the results
     # TODO: should support multiple queries
     #       you should be able to pass an array of queries
+    # Specify :cursor => true to batch the results of a query, sending multiple requests if necessary.
     def mqlread(query, options = {})
       Ken.logger.info ">>> Sending Query: #{query.to_json}"
-      
+      cursor = options[:cursor]
+      if cursor
+        query_result = []
+        while cursor
+          response = get_query_response(query, cursor)
+          query_result += response['result']
+          cursor = response['cursor']
+        end
+      else
+        response = get_query_response(query, cursor)
+        cursor = response['cursor']
+        query_result = response['result']
+      end
+      query_result
+    end
+
+    protected
+    # returns parsed json response from freebase mqlread service
+    def get_query_response(query, cursor=nil)
       envelope = { :qname => {:query => query }}
+      envelope[:qname][:cursor] = cursor if cursor
       
       response = http_request mqlread_service_url, :queries => envelope.to_json
+      
       result = JSON.parse response
+      
       inner = result['qname']
       handle_read_error(inner)
-      
       Ken.logger.info "<<< Received Response: #{inner['result'].inspect}"
-      
-      # will always return the converted ruby hash (from json)
-      inner['result']
-    end # mqlread
-
-    protected
+      inner
+    end
+    
+    # encode parameters
     def params_to_string(parameters)
       parameters.keys.map {|k| "#{URI.encode(k.to_s)}=#{URI.encode(parameters[k])}" }.join('&')
     end
-
+    
+    # does the dirty work
     def http_request(url, parameters = {})
       params = params_to_string(parameters)
       url << '?'+params unless params !~ /\S/

data/lib/ken/type.rb

@@ -3,7 +3,7 @@ module Ken
     
     include Extlib::Assertions
     
-    # initializes a resource by json result
+    # initializes a resource using a json result
     def initialize(data)
       assert_kind_of 'data', data, Hash
       @data = data
@@ -12,14 +12,16 @@ module Ken
     # access property info
     # @api public
     def properties
-      Ken::Collection.new(@data["properties"].map { |property| Ken::Property.new(property, self) })
+      @properties ||= Ken::Collection.new(@data["properties"].map { |property| Ken::Property.new(property, self) })
     end
     
+    # type id
     # @api public
     def id
       @data["id"]
     end
     
+    # type name
     # @api public
     def name
       @data["name"]
@@ -34,5 +36,18 @@ module Ken
     def inspect
       result = "#<Type id=\"#{id}\" name=\"#{name || "nil"}\">"
     end
+    
+    # delegate to property_get
+    def method_missing sym
+      property_get(sym.to_s)
+    end
+    
+    private
+    # @api private
+    # search for a property by name and return it
+    def property_get(name)
+      properties.each { |p| return p if p.id =~ /\/#{name}$/ }
+      raise PropertyNotFound
+    end
   end
 end

data/lib/ken/view.rb

@@ -17,6 +17,7 @@ module Ken
       @type.to_s
     end
     
+    # return correspondent type
     # @api public
     def type
       @type
@@ -39,5 +40,17 @@ module Ken
       @resource.properties.select { |p| p.type == @type}
     end
     
+    # delegate to attribute_get
+    def method_missing sym
+      attribute_get(sym.to_s)
+    end
+    
+    private
+    # search for an attribute by name and return it
+    # @api private
+    def attribute_get(name)
+      attributes.each { |a| return a if a.property.id =~ /\/#{name}$/ }
+      raise AttributeNotFound
+    end
   end
 end