xapian_db 1.0 → 1.1

data/CHANGELOG.md

@@ -1,3 +1,22 @@
+##1.1 (September 7th, 2011)
+
+Fixes:
+
+  - better handling of the beanstalk-client dependency
+  - recreate the xapian index database if the configured path exists but does not contain a valid xapian index
+  - support for non-integer primary keys (removed unneccesary to_i conversion)
+
+Features:
+
+  - rails sample app upgraded to 3.1
+  - support for value range queries (strings, dates, numbers)
+  - sorting now works on a global query, too (XapianDb.search...)
+  - global factes queries have now the same options like class scoped facet queries
+  - Support for custom serialization into xapian documents; overwrite the serialization implementation in type_codec.rb or implement your own serialization for specific types (see examples/custom_serialization.rb)
+  - support to reindex a single object while evaluation an ignore_if block (if present)
+
+IMPORTANT: YOU MUST REBUILD YOUR XAPIAN INDEX DATABASE SINCE THE INDEX STRUCTURE HAS CHANGED!
+
 ##1.0 (August 17th, 2011)
 
 Features:

data/README.rdoc

@@ -1,5 +1,9 @@
 = XapianDb
 
+== Important Information
+
+If you upgrade from an earlier version of xapian_db to 1.1, you MUST rebuild your entire index (XapianDb.rebuild_xapian_index)!
+
 == What's in the box?
 
 XapianDb is a ruby gem that combines features of nosql databases and fulltext indexing into one piece. The result: Rich documents and very fast queries. It is based on {Xapian}[http://xapian.org/], an efficient and powerful indexing library.
@@ -116,6 +120,14 @@ You may add a filter expression to exclude objects from the index. This is handy
       blueprint.ignore_if {active == false}
     end
 
+You can add a type information to an attribute. As of now the special types :string, :date and :number are supported (and required for range queries):
+
+    XapianDb::DocumentBlueprint.setup(Person) do |blueprint|
+      blueprint.attribute :age,           :as => :number
+      blueprint.attribute :date_of_birth, :as => :date
+      blueprint.attribute :name,          :as => :string
+    end
+
 You can override the global adapter configuration in a specific blueprint. Let's say you use ActiveRecord, but you have
 one more class that is not stored in the database, but you want it to be indexed:
 
@@ -145,6 +157,10 @@ To rebuild the index for all blueprints, use
 
   XapianDb.rebuild_xapian_index
 
+You can update the index for a single object, too (e.g. to reevaluate an ignore_if block without modifying and saving the object):
+
+  XapianDb.reindex object
+
 === Query the index
 
 A simple query looks like this:
@@ -180,7 +196,26 @@ On class queries you can specifiy order options:
   results = Person.search "name:Foo", :order => :first_name
   results = Person.search "Fo*", :order => [:name, :first_name], :sort_decending => true
 
-Please note that the order option is not available for global searches (XapianDb.search...)
+If you define an attribute with a supported type, you can do range searches:
+
+  XapianDb::DocumentBlueprint.setup(Person) do |blueprint|
+    blueprint.attribute :age,           :as => :number
+    blueprint.attribute :date_of_birth, :as => :date
+    blueprint.attribute :name,          :as => :string
+  end
+
+  result = XapianDb.search("date_of_birth:2011-01-01..2011-12-31")
+  result = XapianDb.search("age:30..40")
+  result = XapianDb.search("name:Adam..Chris")
+
+Open Ranges are supported, too:
+
+  result = XapianDb.search("age:..40")
+  result = XapianDb.search("age:30..")
+
+You can combine range query expressions with other expressions:
+
+  result = XapianDb.search("age:30..40 AND city:Aarau")
 
 === Process the results
 
@@ -216,7 +251,15 @@ Or with kaminari:
 If you want to implement a simple drilldown for your searches, you can use a global facets query:
 
   search_expression = "Foo"
-  facets = XapianDb.facets(search_expression)
+  facets = XapianDb.facets(:name, search_expression)
+  facets.each do |name, count|
+    puts "#{name}: #{count} hits"
+  end
+
+If you want the facets based on the indexed class, use the special attribute :indexed_class:
+
+  search_expression = "Foo"
+  facets = XapianDb.facets(:indexed_class, search_expression)
   facets.each do |klass, count|
     puts "#{klass.name}: #{count} hits"
 
@@ -224,7 +267,7 @@ If you want to implement a simple drilldown for your searches, you can use a glo
     # doc = klass.search search_expression
   end
 
-A global facet search always groups the results by the class of the indexed objects. There is a class level facet query syntax available, too:
+A class level facet query is possible, too:
 
   search_expression = "Foo"
   facets = Person.facets(:name, search_expression)
@@ -232,7 +275,7 @@ A global facet search always groups the results by the class of the indexed obje
     puts "#{name}: #{count} hits"
   end
 
-At the class level, any attribute can be used for a facet query. Use facet queries on attributes that store atomic values like strings, numbers or dates.
+Any attribute declared in a blueprint can be used for a facet query. Use facet queries on attributes that store atomic values like strings, numbers or dates.
 If you use it on attributes that contain collections (like an array of strings), you might get unexpected results.
 
 === Find similar documents
@@ -269,6 +312,12 @@ you can use the auto_indexing_disabled method with a block and rebuild the whole
   end
   Person.rebuild_xapian_index
 
+== Add your own serializers for special objects
+
+XapianDb serializes objects to xapian documents using YAML by default. This way, type information is preserved und you get back what you put into a xapian document, not just a string.
+
+However, dates need special handling to support date range queries. To support date range queries and allow the addition of other custom data types in the future, XapianDb uses a simple, extensible mechanism to serialize / deserialize your objects. An example on how to extend this mechanism is provided in examples/custom_serialization.rb.
+
 == Production setup
 
 Since Xapian allows only one database instance to write to the index, the default setup of XapianDb will not work

data/lib/type_codec.rb

@@ -0,0 +1,124 @@
+# encoding: utf-8
+
+# This class is responsible for encoding and decoding values depending on their
+# type
+
+require "bigdecimal"
+
+module XapianDb
+
+  class TypeCodec
+
+    extend XapianDb::Utilities
+
+    # Get the codec for a type
+    # @param [Symbol] type a supported type as a string or symbol.
+    #   The following types are supported:
+    #     - :date
+    # @return [DateCodec]
+    def self.codec_for(type)
+      begin
+        constantize "XapianDb::TypeCodec::#{camelize("#{type}_codec")}"
+      rescue NameError
+        raise ArgumentError.new "no codec defined for type #{type}"
+      end
+    end
+
+    class GenericCodec
+
+      # Encode an object to its yaml representation
+      # @param [Object] object an object to encode
+      # @return [String] the yaml string
+      def self.encode(object)
+        begin
+          if object.respond_to?(:attributes)
+            object.attributes.to_yaml
+          else
+            object.to_yaml
+          end
+        rescue NoMethodError
+          raise ArgumentError.new "#{object} does not support yaml serialization"
+        end
+      end
+
+      # Decode an object from a yaml string
+      # @param [String] yaml_string a yaml string representing the object
+      # @return [Object] the parsed object
+      def self.decode(yaml_string)
+        begin
+          YAML::load yaml_string
+        rescue TypeError
+          raise ArgumentError.new "'#{yaml_string}' cannot be loaded by YAML"
+        end
+      end
+    end
+
+    class StringCodec
+
+      # Encode an object to a string
+      # @param [Object] object an object to encode
+      # @return [String] the string
+      def self.encode(object)
+        object.to_s
+      end
+
+      # Decode a string
+      # @param [String] string a string
+      # @return [String] the string
+      def self.decode(string)
+        string
+      end
+    end
+
+    class DateCodec
+
+      # Encode a date to a string in the format 'yyyymmdd'
+      # @param [Date] date a date object to encode
+      # @return [String] the encoded date
+      def self.encode(date)
+        begin
+          date.strftime "%Y%m%d"
+        rescue NoMethodError
+          raise ArgumentError.new "#{date} was expected to be a date"
+        end
+      end
+
+      # Decode a string to a date
+      # @param [String] date_as_string a string representing a date
+      # @return [Date] the parsed date
+      def self.decode(date_as_string)
+        begin
+          Date.parse date_as_string
+        rescue ArgumentError
+          raise ArgumentError.new "'#{date_as_string}' cannot be converted to a date"
+        end
+      end
+    end
+
+    class NumberCodec
+
+      # Encode a number to a sortable string
+      # @param [Integer, BigDecimal, Float] number a number object to encode
+      # @return [String] the encoded number
+      def self.encode(number)
+        begin
+          Xapian::sortable_serialise number
+        rescue TypeError
+          raise ArgumentError.new "#{number} was expected to be a number"
+        end
+      end
+
+      # Decode a string to a BigDecimal
+      # @param [String] number_as_string a string representing a number
+      # @return [BigDecimal] the decoded number
+      def self.decode(encoded_number)
+        begin
+          BigDecimal.new(Xapian::sortable_unserialise(encoded_number).to_s)
+        rescue TypeError
+          raise ArgumentError.new "#{encoded_number} cannot be unserialized"
+        end
+      end
+    end
+
+  end
+end

data/lib/xapian_db/adapters/active_record_adapter.rb

@@ -37,14 +37,9 @@ module XapianDb
 
            klass.class_eval do
 
-             # add the after save logic
+             # add the after commit logic
              after_commit do
-               blueprint = XapianDb::DocumentBlueprint.blueprint_for klass
-               if blueprint.should_index?(self)
-                 XapianDb.index(self)
-               else
-                 XapianDb.delete_doc_with(self.xapian_id)
-               end
+               XapianDb.reindex(self)
              end
 
              # add the after destroy logic

data/lib/xapian_db/adapters/base_adapter.rb

@@ -33,9 +33,9 @@ module XapianDb
                order = options.delete :order
                if order
                  attr_names             = [order].flatten
-                 blueprint              = XapianDb::DocumentBlueprint.blueprint_for klass
-                 sort_indices           = attr_names.map {|attr_name| blueprint.value_index_for(attr_name)}
-                 options[:sort_indices] = attr_names.map {|attr_name| blueprint.value_index_for(attr_name)}
+                 undefined_attrs        = attr_names - XapianDb::DocumentBlueprint.attributes
+                 raise ArgumentError.new "invalid order clause: attributes #{undefined_attrs.inspect} are not defined" unless undefined_attrs.empty?
+                 options[:sort_indices] = attr_names.map {|attr_name| XapianDb::DocumentBlueprint.value_number_for(attr_name) }
                end
                result = XapianDb.database.search "#{class_scope} and (#{expression})", options
 
@@ -52,29 +52,9 @@ module XapianDb
              end
 
              # Add a method to search atribute facets of this class
-             define_singleton_method(:facets) do |attr_name, expression|
-
-               # return an empty hash if no search expression is given
-               return {} if expression.nil? || expression.strip.empty?
-
-               class_scope = "indexed_class:#{klass.name.downcase}"
-               blueprint   = XapianDb::DocumentBlueprint.blueprint_for klass
-               value_index = blueprint.value_index_for attr_name.to_sym
-
-               query_parser        = QueryParser.new(XapianDb.database)
-               query                = query_parser.parse("#{class_scope} and (#{expression})")
-               enquiry              = Xapian::Enquire.new(XapianDb.database.reader)
-               enquiry.query        = query
-               enquiry.collapse_key = value_index
-               facets = {}
-               enquiry.mset(0, XapianDb.database.size).matches.each do |match|
-                 facet_value = YAML::load match.document.values[value_index].value
-                 # We must add 1 to the collapse_count since collapse_count means
-                 # "how many other matches are there?"
-                 facets[facet_value] = match.collapse_count + 1
-               end
-               facets
-
+             define_singleton_method(:facets) do |attribute, expression|
+               class_scope  = "indexed_class:#{klass.name.downcase}"
+               XapianDb.database.facets attribute, "#{class_scope} and (#{expression})"
              end
 
            end

data/lib/xapian_db/config.rb

@@ -67,10 +67,9 @@ module XapianDb
       if path.to_sym == :memory
         @_database = XapianDb.create_db
       else
-        if File.exist?(path)
+        begin
           @_database = XapianDb.open_db :path => path
-        else
-          # Database does not exist; create it
+        rescue IOError
           @_database = XapianDb.create_db :path => path
         end
       end

data/lib/xapian_db/database.rb

@@ -75,13 +75,9 @@ module XapianDb
       sort_decending = opts.delete :sort_decending
 
       if sort_indices
-        raise ArgumentError.new("Sorting is available for class scoped searches only") unless expression =~ /^indexed_class:/
-        sorter = Xapian::MultiValueSorter.new
-
-        sort_indices.each do |index|
-          sorter.add(index, sort_decending)
-        end
-        enquiry.set_sort_by_key_then_relevance(sorter)
+        sorter = Xapian::MultiValueKeyMaker.new
+        sort_indices.each { |index| sorter.add_value index }
+        enquiry.set_sort_by_key_then_relevance(sorter, sort_decending)
       end
 
       opts[:spelling_suggestion] = @query_parser.spelling_suggestion
@@ -123,25 +119,28 @@ module XapianDb
       Resultset.new(enquiry, :db_size => self.size)
     end
 
-    # A very simple implementation of facets limited to the class facets.
+    # A very simple implementation of facets using Xapian collapse key.
+    # @param [Symbol, String] attribute the name of an attribute declared in one ore more blueprints
     # @param [String] expression A valid search expression (see {#search} for examples).
     # @return [Hash<Class, Integer>] A hash containing the classes and the hits per class
-    def facets(expression)
-      @query_parser        ||= QueryParser.new(self)
-      query                = @query_parser.parse(expression)
-      enquiry              = Xapian::Enquire.new(reader)
+    def facets(attribute, expression)
+     # return an empty hash if no search expression is given
+      return {} if expression.nil? || expression.strip.empty?
+      value_number         = XapianDb::DocumentBlueprint.value_number_for(attribute)
+      query_parser         = QueryParser.new(XapianDb.database)
+      query                = query_parser.parse(expression)
+      enquiry              = Xapian::Enquire.new(XapianDb.database.reader)
       enquiry.query        = query
-      enquiry.collapse_key = 0 # Value 0 always contains the class name
+      enquiry.collapse_key = value_number
       facets = {}
-      enquiry.mset(0, self.size).matches.each do |match|
-        class_name = match.document.values[0].value
+      enquiry.mset(0, XapianDb.database.size).matches.each do |match|
+        facet_value = YAML::load match.document.value(value_number)
         # We must add 1 to the collapse_count since collapse_count means
         # "how many other matches are there?"
-        facets[constantize class_name] = match.collapse_count + 1
+        facets[facet_value] = match.collapse_count + 1
       end
       facets
     end
-
   end
 
   # In Memory database

data/lib/xapian_db/document_blueprint.rb

@@ -35,12 +35,17 @@ module XapianDb
         @blueprints ||= {}
         blueprint = DocumentBlueprint.new
         yield blueprint if block_given? # configure the blueprint through the block
+        validate_type_consistency_on blueprint
         # Remove a previously loaded blueprint for this class to avoid stale blueprint definitions
-        @blueprints.delete_if { |key, blueprint| key.name == klass.name }
+        @blueprints.delete_if { |indexed_class, blueprint| indexed_class.name == klass.name }
         @blueprints[klass] = blueprint
         @_adapter = blueprint._adapter || XapianDb::Config.adapter || Adapters::GenericAdapter
         @_adapter.add_class_helper_methods_to klass
-        @searchable_prefixes = nil # force rebuild of the searchable prefixes
+
+        @searchable_prefixes = @blueprints.values.map { |blueprint| blueprint.searchable_prefixes }.flatten.compact.uniq || []
+        # We can always do a field search on the name of the indexed class
+        @searchable_prefixes << "indexed_class"
+        @attributes = @blueprints.values.map { |blueprint| blueprint.attribute_names}.flatten.compact.uniq.sort || []
       end
 
       # Get all configured classes
@@ -63,14 +68,54 @@ module XapianDb
         raise "Blueprint for class #{klass} is not defined"
       end
 
+      # Get the value number for an attribute. Please note that this is not the index in the values
+      # array of a xapian document but the valueno. Therefore, document.values[value_number] returns
+      # the wrong data, use document.value(value_number) instead.
+      # @param [attribute] The name of an attribute
+      # @return [Integer] The value number
+      def value_number_for(attribute)
+        raise ArgumentError.new "attribute #{attribute} is not configured in any blueprint" if @attributes.nil?
+        return 0 if attribute.to_sym == :indexed_class
+        position = @attributes.index attribute.to_sym
+        if position
+          # We add 1 because value slot 0 is reserved for the class name
+          return position + 1
+        else
+          raise ArgumentError.new "attribute #{attribute} is not configured in any blueprint"
+        end
+      end
+
+      # Get the type info of an attribute
+      # @param [attribute] The name of an indexed method
+      # @return [Symbol] The defined type or :untyped if no type is defined
+      def type_info_for(attribute)
+        return nil if @blueprints.nil?
+        @blueprints.values.each do |blueprint|
+          return blueprint.type_map[attribute] if blueprint.type_map.has_key?(attribute)
+        end
+        nil
+      end
+
       # Return an array of all configured text methods in any blueprint
       # @return [Array<String>] All searchable prefixes
       def searchable_prefixes
-        return [] unless @blueprints
-        @searchable_prefixes ||= @blueprints.values.map { |blueprint| blueprint.searchable_prefixes }.flatten.compact.uniq
-        # We can always do a field search on the name of the indexed class
-        @searchable_prefixes << "indexed_class"
-        @searchable_prefixes
+        @searchable_prefixes || []
+      end
+
+      # Return an array of all defined attributes
+      # @return [Array<Symbol>] All defined attributes
+      def attributes
+        @attributes || []
+      end
+
+      private
+
+      def validate_type_consistency_on(blueprint)
+        blueprint.type_map.each do |method_name, type|
+          if type_info_for(method_name) && type_info_for(method_name) != type
+            raise ArgumentError.new "ambigous type definition for #{method_name} detected (#{type_info_for(method_name)}, #{type})"
+          end
+        end
       end
 
     end
@@ -79,6 +124,8 @@ module XapianDb
     # Instance methods
     # ---------------------------------------------------------------------------------
 
+    attr_reader :type_map
+
     # Get the names of all configured attributes sorted alphabetically
     # @return [Array<Symbol>] The names of the attributes
     def attribute_names
@@ -89,7 +136,7 @@ module XapianDb
     # @param [Symbol] attribute The name of the attribute
     # @return [Block] The block
     def block_for_attribute(attribute)
-      @attributes_hash[attribute]
+      @attributes_hash[attribute][:block]
     end
 
     # Get the names of all configured index methods sorted alphabetically
@@ -105,22 +152,10 @@ module XapianDb
       @indexed_methods_hash[method]
     end
 
-    # Return the value index of an attribute. Needed to access the value of an attribute
-    # from a Xapian document.
-    # @param [String, Symbol] attribute_name The name of the attribute
-    # @return [Integer] The value index of the attribute
-    # @raise ArgumentError if the attribute name is unknown
-    def value_index_for(attribute_name)
-      index = attribute_names.index attribute_name.to_sym
-      raise ArgumentError.new("Attribute #{attribute_name} unknown") unless index
-      # We add 1 because value slot 0 is reserved for the class name
-      index + 1
-    end
-
     # Return an array of all configured text methods in this blueprint
     # @return [Array<String>] All searchable prefixes
     def searchable_prefixes
-      @prefixes ||= @indexed_methods_hash.keys
+      @searchable_prefixes ||= indexed_method_names
     end
 
     # Should the object go into the index? Evaluates an ignore expression,
@@ -151,10 +186,11 @@ module XapianDb
 
       # Add an accessor for each attribute
       attribute_names.each do |attribute|
-        index = value_index_for(attribute)
+        index = DocumentBlueprint.value_number_for(attribute)
+        codec = XapianDb::TypeCodec.codec_for @type_map[attribute]
         @accessors_module.instance_eval do
           define_method attribute do
-            YAML::load(self.values[index].value)
+            codec.decode self.value(index)
           end
         end
       end
@@ -174,8 +210,9 @@ module XapianDb
 
     # Construct the blueprint
     def initialize
-      @attributes_hash = {}
+      @attributes_hash =      {}
       @indexed_methods_hash = {}
+      @type_map =             {}
     end
 
     # Set the adapter
@@ -194,6 +231,7 @@ module XapianDb
     # @param [Hash] options
     # @option options [Integer] :weight (1) The weight for this attribute.
     # @option options [Boolean] :index (true) Should the attribute be indexed?
+    # @option options [Symbol] :as should add type info for range queries (:date, :numeric)
     # @example For complex attribute configurations you may pass a block:
     #   XapianDb::DocumentBlueprint.setup(IndexedObject) do |blueprint|
     #     blueprint.attribute :complex do
@@ -206,13 +244,15 @@ module XapianDb
     #   end
     def attribute(name, options={}, &block)
       raise ArgumentError.new("You cannot use #{name} as an attribute name since it is a reserved method name of Xapian::Document") if reserved_method_name?(name)
-      opts = {:index => true}.merge(options)
+      do_not_index    = options.delete(:index) == false
+      @type_map[name] = (options.delete(:as) || :generic)
+
       if block_given?
-        @attributes_hash[name] = block
+        @attributes_hash[name] = {:block => block}.merge(options)
       else
-        @attributes_hash[name] = nil
+        @attributes_hash[name] = options
       end
-      self.index(name, opts, &block) if opts[:index]
+      self.index(name, options, &block) unless do_not_index
     end
 
     # Add a list of attributes to the blueprint. Attributes will be stored in the xapian documents ans
@@ -221,7 +261,8 @@ module XapianDb
     def attributes(*attributes)
       attributes.each do |attr|
         raise ArgumentError.new("You cannot use #{attr} as an attribute name since it is a reserved method name of Xapian::Document") if reserved_method_name?(attr)
-        @attributes_hash[attr] = nil
+        @attributes_hash[attr] = {}
+        @type_map[attr] = :generic
         self.index attr
       end
     end
@@ -249,7 +290,9 @@ module XapianDb
         when 2
           # Is it a method name with options?
           if args.last.is_a? Hash
-            @indexed_methods_hash[args.first] = IndexOptions.new(args.last.merge(:block => block))
+            options = args.last
+            assert_valid_keys options, :weight
+            @indexed_methods_hash[args.first] = IndexOptions.new(options.merge(:block => block))
           else
             add_indexes_from args
           end
@@ -266,16 +309,16 @@ module XapianDb
     # Options for an indexed method
     class IndexOptions
 
-      # The weight for the indexed value
-      attr_accessor :weight, :block
+      attr_reader :weight, :block
 
       # Constructor
       # @param [Hash] options
       # @option options [Integer] :weight (1) The weight for the indexed value
-      def initialize(options)
+      def initialize(options = {})
         @weight = options[:weight] || 1
         @block  = options[:block]
       end
+
     end
 
     private
@@ -295,4 +338,4 @@ module XapianDb
 
   end
 
-end
+end

data/lib/xapian_db/index_writers/beanstalk_worker.rb

@@ -3,7 +3,7 @@
 module XapianDb
   module IndexWriters
 
-    # Worker to update the Xapian index; the worker is used in the beanstalk worker rake task
+    # Worker to update the Xapian index; the worker is used in the beanstalk worker script
     # and uses the DirectWriter to do the real work
     # @author Gernot Kogler
     class BeanstalkWorker
@@ -12,7 +12,7 @@ module XapianDb
 
       def index_task(options)
         klass = constantize options[:class]
-        obj   = klass.respond_to?(:get) ? klass.get(options[:id].to_i) : klass.find(options[:id].to_i)
+        obj   = klass.respond_to?(:get) ? klass.get(options[:id]) : klass.find(options[:id])
         DirectWriter.index obj
       end
 

data/lib/xapian_db/indexer.rb

@@ -43,10 +43,9 @@ module XapianDb
           value = @obj.send(attribute)
         end
 
-        # If we have an object that responds to attributes (e.g. an Active Record
-        # or a Datamapper model), we serialize only the attributes
-        yaml = value.respond_to?(:attributes) ? value.attributes.to_yaml : value.to_yaml
-        @xapian_doc.add_value(@blueprint.value_index_for(attribute), yaml)
+        codec          = XapianDb::TypeCodec.codec_for @blueprint.type_map[attribute]
+        encoded_string = codec.encode value
+        @xapian_doc.add_value DocumentBlueprint.value_number_for(attribute), encoded_string
       end
     end
 
@@ -105,4 +104,4 @@ module XapianDb
 
   end
 
-end
+end

data/lib/xapian_db/query_parser.rb

@@ -39,7 +39,20 @@ module XapianDb
 
       # Add the searchable prefixes to allow searches by field
       # (like "name:Kogler")
-      XapianDb::DocumentBlueprint.searchable_prefixes.each{|prefix| parser.add_prefix(prefix.to_s.downcase, "X#{prefix.to_s.upcase}") }
+      XapianDb::DocumentBlueprint.searchable_prefixes.each do |prefix|
+        parser.add_prefix(prefix.to_s.downcase, "X#{prefix.to_s.upcase}")
+        type_info = XapianDb::DocumentBlueprint.type_info_for(prefix)
+        next if type_info.nil? || type_info == :generic
+        value_number = XapianDb::DocumentBlueprint.value_number_for(prefix)
+        case type_info
+          when :date
+            parser.add_valuerangeprocessor Xapian::DateValueRangeProcessor.new(value_number, "#{prefix}:")
+          when :number
+            parser.add_valuerangeprocessor Xapian::NumberValueRangeProcessor.new(value_number, "#{prefix}:")
+          when :string
+            parser.add_valuerangeprocessor Xapian::StringValueRangeProcessor.new(value_number, "#{prefix}:")
+        end
+      end
       query = parser.parse_query(expression, @query_flags)
       @spelling_suggestion = parser.get_corrected_query_string.force_encoding("UTF-8")
       @spelling_suggestion = nil if @spelling_suggestion.empty?
@@ -48,4 +61,4 @@ module XapianDb
 
   end
 
-end
+end

data/lib/xapian_db/utilities.rb

@@ -24,5 +24,11 @@ module XapianDb
       constant
     end
 
+    # Taken from Rails
+    def assert_valid_keys(hash, *valid_keys)
+      unknown_keys = hash.keys - [valid_keys].flatten
+      raise(ArgumentError, "Unsupported option(s) detected: #{unknown_keys.join(", ")}") unless unknown_keys.empty?
+    end
+
   end
 end

data/lib/xapian_db.rb

@@ -9,6 +9,22 @@
 require 'xapian'
 require 'yaml'
 
+do_not_require = %w(update_stopwords.rb railtie.rb base_adapter.rb beanstalk_writer.rb utilities.rb install_generator.rb)
+files = Dir.glob("#{File.dirname(__FILE__)}/**/*.rb").reject{|path| do_not_require.include?(File.basename(path))}
+# Require these first
+require "#{File.dirname(__FILE__)}/xapian_db/utilities"
+require "#{File.dirname(__FILE__)}/xapian_db/adapters/base_adapter"
+files.each {|file| require file}
+
+# Configure XapianDB if we are in a Rails app
+require File.dirname(__FILE__) + '/xapian_db/railtie' if defined?(Rails)
+
+# Try to require the beanstalk writer (depends on beanstalk-client)
+begin
+  require File.dirname(__FILE__) + '/xapian_db/index_writers/beanstalk_writer'
+rescue LoadError
+end
+
 module XapianDb
 
   # Supported languages
@@ -75,14 +91,21 @@ module XapianDb
   # See {XapianDb::Database#search} for options
   # @return [XapianDb::Resultset]
   def self.search(expression, options={})
+    order = options.delete :order
+    if order
+      attr_names             = [order].flatten
+      undefined_attrs        = attr_names - XapianDb::DocumentBlueprint.attributes
+      raise ArgumentError.new "invalid order clause: attributes #{undefined_attrs.inspect} are not defined" unless undefined_attrs.empty?
+      options[:sort_indices] = attr_names.map {|attr_name| XapianDb::DocumentBlueprint.value_number_for(attr_name) }
+    end
     XapianDb::Config.database.search(expression, options)
   end
 
   # Get facets from the configured database.
   # See {XapianDb::Database#facets} for options
   # @return [Hash<Class, Integer>] A hash containing the classes and the hits per class
-  def self.facets(expression)
-    XapianDb::Config.database.facets(expression)
+  def self.facets(attribute, expression)
+    XapianDb::Config.database.facets attribute, expression
   end
 
   # Update an object in the index
@@ -99,6 +122,17 @@ module XapianDb
     writer.delete_doc_with xapian_id
   end
 
+  # Update or delete a xapian document belonging to an object depending on the ignore_if logic(if present)
+  # @param [Object] object An instance of a class with a blueprint configuration
+  def self.reindex(object)
+    blueprint = XapianDb::DocumentBlueprint.blueprint_for object.class
+    if blueprint.should_index?(object)
+      XapianDb.index object
+    else
+      XapianDb.delete_doc_with object.xapian_id
+    end
+  end
+
   # Reindex all objects of a given class
   # @param [Class] klass The class to reindex
   # @param [Hash] options Options for reindexing
@@ -161,16 +195,3 @@ module XapianDb
   end
 
 end
-
-do_not_require = %w(update_stopwords.rb railtie.rb base_adapter.rb beanstalk_writer.rb utilities.rb install_generator.rb)
-files = Dir.glob("#{File.dirname(__FILE__)}/**/*.rb").reject{|path| do_not_require.include?(File.basename(path))}
-# Require these first
-require "#{File.dirname(__FILE__)}/xapian_db/utilities"
-require "#{File.dirname(__FILE__)}/xapian_db/adapters/base_adapter"
-files.each {|file| require file}
-
-# Configure XapianDB if we are in a Rails app
-require File.dirname(__FILE__) + '/xapian_db/railtie' if defined?(Rails)
-
-# Require the beanstalk writer is beanstalk-client is installed
-require File.dirname(__FILE__) + '/xapian_db/index_writers/beanstalk_writer' if Gem.available?('beanstalk-client')

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: xapian_db
 version: !ruby/object:Gem::Version
-  version: '1.0'
+  version: '1.1'
   prerelease: 
 platform: ruby
 authors:
@@ -9,12 +9,12 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2011-08-17 00:00:00.000000000 +02:00
+date: 2011-09-07 00:00:00.000000000 +02:00
 default_executable: 
 dependencies:
 - !ruby/object:Gem::Dependency
   name: daemons
-  requirement: &70169658679640 !ruby/object:Gem::Requirement
+  requirement: &70318028242380 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -22,10 +22,10 @@ dependencies:
         version: 1.0.10
   type: :runtime
   prerelease: false
-  version_requirements: *70169658679640
+  version_requirements: *70318028242380
 - !ruby/object:Gem::Dependency
   name: xapian-ruby
-  requirement: &70169658679180 !ruby/object:Gem::Requirement
+  requirement: &70318028241920 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -33,10 +33,10 @@ dependencies:
         version: 1.2.6
   type: :runtime
   prerelease: false
-  version_requirements: *70169658679180
+  version_requirements: *70318028241920
 - !ruby/object:Gem::Dependency
   name: rspec
-  requirement: &70169658678720 !ruby/object:Gem::Requirement
+  requirement: &70318028241460 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -44,10 +44,10 @@ dependencies:
         version: 2.3.1
   type: :development
   prerelease: false
-  version_requirements: *70169658678720
+  version_requirements: *70318028241460
 - !ruby/object:Gem::Dependency
   name: simplecov
-  requirement: &70169658678120 !ruby/object:Gem::Requirement
+  requirement: &70318028241000 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -55,10 +55,10 @@ dependencies:
         version: 0.3.7
   type: :development
   prerelease: false
-  version_requirements: *70169658678120
+  version_requirements: *70318028241000
 - !ruby/object:Gem::Dependency
   name: beanstalk-client
-  requirement: &70169658677520 !ruby/object:Gem::Requirement
+  requirement: &70318028240520 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -66,7 +66,7 @@ dependencies:
         version: 1.1.0
   type: :development
   prerelease: false
-  version_requirements: *70169658677520
+  version_requirements: *70318028240520
 description: XapianDb is a ruby gem that combines features of nosql databases and
   fulltext indexing. It is based on Xapian, an efficient and powerful indexing library
 email: gernot.kogler (at) garaio (dot) com
@@ -76,6 +76,7 @@ extra_rdoc_files: []
 files:
 - lib/generators/install_generator.rb
 - lib/generators/templates/beanstalk_worker
+- lib/type_codec.rb
 - lib/xapian_db/adapters/active_record_adapter.rb
 - lib/xapian_db/adapters/base_adapter.rb
 - lib/xapian_db/adapters/datamapper_adapter.rb