oedipus 0.0.5 → 0.0.6

data/README.md

@@ -1,7 +1,7 @@
 # Oedipus: Sphinx 2 Search Client for Ruby
 
 Oedipus is a client for the Sphinx search engine (>= 2.0.2), with support for
-real-time indexes and multi and/or faceted searches.
+real-time indexes and multi and/or multi-dimensional faceted searches.
 
 It is not a clone of the PHP API, rather it is written from the ground up,
 wrapping the SphinxQL API offered by searchd.  Nor is it a plugin for
@@ -14,6 +14,11 @@ search may be implemented, while remaining light and simple.
 Data structures are managed using core ruby data types (Array and Hash), ensuring
 simplicity and flexibilty.
 
+The current development focus is on supporting realtime indexes, where data is
+indexed from your application, rather than by running the indexer tool that comes
+with Sphinx.  You may use indexes that are indexed with the indexer tool, but
+Oedipus does not (yet) provide wrappers for indexing that data via ruby [1].
+
 ## Dependencies
 
   * ruby (>= 1.9)
@@ -242,13 +247,16 @@ from the base query.
 
 Oedipus allows you to replace '%{query}' in your facets with whatever was in the
 original query.  This can be useful if you want to provide facets that only
-perform the search in the title of the document (`"@title (%{query})"`) for example.
+perform the search in the title of the document (`"@title (%{query})"`) for
+example.
 
-Each facet is given a name, which is used to reference them in the results.
+Each facet is given a key, which is used to reference it in the results.  This
+key is any arbitrary object that can be used as a key in a ruby Hash.  You may,
+for example, use domain-specific objects as keys.
 
 Sphinx optimizes the queries by figuring out what the common parts are.  Currently
 it does two optimizations, though in future this will likely improve further, so
-using this technique to do your faceted searches is a good idea.
+using this technique to do your faceted searches is the correct approach.
 
 ``` ruby
 results = sphinx[:articles].search(
@@ -283,12 +291,57 @@ results = sphinx[:articles].search(
 # }
 ```
 
+#### Multi-dimensional faceted search
+
+If you can add facets to the root query, how about adding facets to the facets
+themselves?  Easy:
+
+``` ruby
+results = sphinx[:articles].search(
+  "badgers",
+  facets: {
+    popular: {
+      views:  100..10000,
+      facets: {
+        in_title: "@title (%{query})"
+      }
+    }
+  }
+)
+# => {
+#   total_found: 987,
+#   time: 0.000,
+#   records: [ ... ],
+#   facets: {
+#     popular: {
+#       total_found: 25,
+#       time: 0.000,
+#       records: [ ... ],
+#       facets: {
+#         in_title: {
+#           total_found: 24,
+#           time: 0.000,
+#           records: [ ... ]
+#         }
+#       }
+#     }
+#   }
+# }
+```
+
+In the above example, the nested facet `:in_title` inherits the default
+parameters from the facet `:popular`, which inherits its parameters from
+the root query.  The result is a search for "badgers" limited only to the
+title, with views between 100 and 10000.
+
+There is no limit imposed in Oedipus for how deeply facets can be nested.
+
 ### General purpose multi-search
 
 If you want to execute multiple queries in a batch that are not related to each
 other (which would be a faceted search), then you can use `#multi_search`.
 
-You pass a Hash of named queries and get a Hash of named resultsets.
+You pass a Hash of keyed-queries and get a Hash of keyed-resultsets.
 
 ``` ruby
 results = sphinx[:articles].multi_search(
@@ -313,7 +366,7 @@ results = sphinx[:articles].multi_search(
 
 There are both unit tests and integration tests in the specs/ directory.  By default they
 will both run, but in order for the integration specs to work, you need a locally
-installed copy of [Sphinx] [1].  You then execute the specs as follows:
+installed copy of [Sphinx] [2].  You then execute the specs as follows:
 
     SEARCHD=/path/to/bin/searchd bundle exec rake spec
 
@@ -334,7 +387,9 @@ You may also compile the C extension and run the specs separately, if you prefer
 
 ### Footnotes
 
-  [1]: You can build a local copy of sphinx without installing it on the system:
+  [1]: In practice I find such an abstraction not to be very useful, as it assumes a single-server setup
+
+  [2]: You can build a local copy of sphinx without installing it on the system:
 
     cd sphinx-2.0.4/
     ./configure
@@ -344,11 +399,10 @@ You may also compile the C extension and run the specs separately, if you prefer
 
 ## Future Plans
 
-  * Integration with DataMapper and ActiveRecord (DataMapper first)
+  * Integration ActiveRecord (DataMapper support has already been added)
   * Support for re-indexing non-realtime indexes from ruby code
   * Distributed index support (sharding writes between indexes)
   * Make C extension optional and provide an implementation in pure-ruby
-  * N-dimensional faceted search (facets inside of facets)
   * Query translation layer for Lucene-style AND/OR/NOT and attribute:value interpretation
   * Fulltext query sanitization for unsafe user input (e.g. @@missing field)
 

data/lib/oedipus/index.rb

@@ -123,7 +123,7 @@ module Oedipus
     # The results returned include a :facets key, containing the results for each facet.
     #
     # @example Performing a faceted search
-    #   index.faceted_search(
+    #   index.search(
     #     "cats | dogs",
     #     category_id: 7,
     #     facets: {
@@ -132,6 +132,26 @@ module Oedipus
     #     }
     #   )
     #
+    # To perform an n-dimensional faceted search, add a :facets option to each
+    # facet. Each facet will inherit from its immediate parent, which inerits
+    # from its parent, up to the root query.
+    #
+    # @example Performing a n-dimensional faceted search
+    #   index.search(
+    #     "cats | dogs",
+    #     facets: {
+    #       popular: {
+    #         views: Oedipus.gte(1000),
+    #         facets: {
+    #           in_title: "@title (%{query})"
+    #         }
+    #       }
+    #     }
+    #   )
+    #
+    # The results in a n-dimensional faceted search are returned with each set
+    # of facet results in turn containing a :facets element.
+    #
     # @param [String] query
     #   a fulltext query
     #
@@ -160,15 +180,7 @@ module Oedipus
     #   a Hash containing meta data, with the records in :records, and if any
     #   facets were included, the facets inside the :facets Hash
     def search(*args)
-      query, options = extract_query_data(args)
-      main_query     = [query, options.reject { |k, _| k == :facets }]
-      facets         = merge_queries(main_query, options.fetch(:facets, {}))
-
-      { facets: {} }.tap do |results|
-        multi_search({ _main_: main_query }.merge(facets)).each do |k, v|
-          k == :_main_ ? results.merge!(v) : results[:facets].merge!(k => v)
-        end
-      end
+      expand_facet_tree(multi_search(deep_merge_facets(args)))
     end
 
     # Perform a faceted search on the index, using a base query and one or more facets.
@@ -257,19 +269,51 @@ module Oedipus
       end
 
       case args[0]
-      when String then [args[0],       args.fetch(1, {})]
-      when Hash   then [default_query, args[0]          ]
+      when String then [args[0],       args.fetch(1, {}).dup]
+      when Hash   then [default_query, args[0].dup          ]
       else raise ArgumentError, "Invalid query argument type #{args.first.class}"
       end
     end
 
-    def merge_queries(base, others)
-      base_query, base_filters = base
+    def expand_facet_tree(result)
+      Hash[].tap do |tree|
+        result.each do |k, v|
+          t = tree
+
+          k.each do |name|
+            f = t[:facets] ||= {}
+            t = f[name]    ||= {}
+          end
+
+          t.merge!(v)
+        end
+      end
+    end
+
+    def deep_merge_facets(base_args, list = {}, path = [])
+      # FIXME: Try and make this shorter and more functional in style
+      base_query, base_options = extract_query_data(base_args)
+
+      facets = base_options.delete(:facets)
+
+      list.merge!(path => [base_query, base_options])
+
+      unless facets.nil?
+        facets.each do |k, q|
+          facet_query, facet_options = extract_query_data(q, base_query)
+
+          deep_merge_facets(
+            [
+              facet_query.gsub("%{query}", base_query),
+              base_options.merge(facet_options)
+            ],
+            list,
+            path.dup << k
+          )
+        end
+      end
 
-      Hash[others.map { |k, q|
-        query, filters = extract_query_data(q, base_query)
-        [k, [query.gsub("%{query}", base_query), base_filters.merge(filters)]]
-      }]
+      list
     end
   end
 end

data/lib/oedipus/rspec/test_harness.rb

@@ -70,7 +70,7 @@ module Oedipus
 
               rt_attr_uint   = user_id
               rt_attr_uint   = views
-              rt_attr_string = status
+              rt_attr_string = state
             }
 
             searchd

data/lib/oedipus/version.rb

@@ -8,5 +8,5 @@
 ##
 
 module Oedipus
-  VERSION = "0.0.5"
+  VERSION = "0.0.6"
 end

data/spec/integration/index_spec.rb

@@ -63,7 +63,7 @@ describe Oedipus::Index do
 
     context "with a valid document ID" do
       it "returns the matched document" do
-        index.fetch(2).should == { id: 2, views: 73,  user_id: 0, status: "" }
+        index.fetch(2).should == { id: 2, views: 73,  user_id: 0, state: "" }
       end
     end
 
@@ -86,7 +86,7 @@ describe Oedipus::Index do
 
       it "modifies the data" do
         index.update(1, views: 721)
-        index.fetch(1).should == { id: 1, views: 721, user_id: 7, status: "" }
+        index.fetch(1).should == { id: 1, views: 721, user_id: 7, state: "" }
       end
     end
 
@@ -123,7 +123,7 @@ describe Oedipus::Index do
 
       it "entirely replaces the record" do
         index.replace(1, title: "Badgers and foxes", views: 150)
-        index.fetch(1).should == { id: 1, views: 150, user_id: 0, status: "" }
+        index.fetch(1).should == { id: 1, views: 150, user_id: 0, state: "" }
       end
     end
 
@@ -134,7 +134,7 @@ describe Oedipus::Index do
 
       it "entirely replaces the record" do
         index.replace(2, title: "Beer and wine", views: 15)
-        index.fetch(2).should == { id: 2, views: 15, user_id: 0, status: "" }
+        index.fetch(2).should == { id: 2, views: 15, user_id: 0, state: "" }
       end
     end
 
@@ -181,9 +181,9 @@ describe Oedipus::Index do
 
       it "includes the matches records" do
         index.search("badgers")[:records].should == [
-          { id: 1, views: 150,  user_id: 0, status: "" },
-          { id: 3, views: 41,   user_id: 0, status: "" },
-          { id: 4, views: 3003, user_id: 0, status: "" }
+          { id: 1, views: 150,  user_id: 0, state: "" },
+          { id: 3, views: 41,   user_id: 0, state: "" },
+          { id: 4, views: 3003, user_id: 0, state: "" }
         ]
       end
     end
@@ -195,19 +195,19 @@ describe Oedipus::Index do
 
       it "includes the matches records" do
         index.search(views: 40..90)[:records].should == [
-          { id: 2, views: 87,   user_id: 0, status: "" },
-          { id: 3, views: 41,   user_id: 0, status: "" }
+          { id: 2, views: 87,   user_id: 0, state: "" },
+          { id: 3, views: 41,   user_id: 0, state: "" }
         ]
       end
 
       pending "the sphinxql grammar does not currently support this, though I'm patching it" do
         context "with string attributes" do
           before(:each) do
-            index.insert(5, title: "No more badgers, please", views: 0, status: "new")
+            index.insert(5, title: "No more badgers, please", views: 0, state: "new")
           end
 
           it "filters by the string attribute" do
-            index.search(status: "new")[:records].should == [{ id: 5, views: 0, user_id: 0, status: "new" }]
+            index.search(state: "new")[:records].should == [{ id: 5, views: 0, user_id: 0, state: "new" }]
           end
         end
       end
@@ -220,8 +220,8 @@ describe Oedipus::Index do
 
       it "includes the matches records" do
         index.search("badgers", views: Oedipus.gt(100))[:records].should == [
-          { id: 1, views: 150,  user_id: 0, status: "" },
-          { id: 4, views: 3003, user_id: 0, status: "" }
+          { id: 1, views: 150,  user_id: 0, state: "" },
+          { id: 4, views: 3003, user_id: 0, state: "" }
         ]
       end
     end
@@ -233,14 +233,14 @@ describe Oedipus::Index do
 
       it "returns the limited subset of the results" do
         index.search("badgers", limit: 2)[:records].should == [
-          { id: 1, views: 150,  user_id: 0, status: "" },
-          { id: 3, views: 41,   user_id: 0, status: "" }
+          { id: 1, views: 150,  user_id: 0, state: "" },
+          { id: 3, views: 41,   user_id: 0, state: "" }
         ]
       end
 
       it "can use an offset" do
         index.search("badgers", limit: 1, offset: 1)[:records].should == [
-          { id: 3, views: 41,   user_id: 0, status: "" }
+          { id: 3, views: 41,   user_id: 0, state: "" }
         ]
       end
     end
@@ -248,9 +248,9 @@ describe Oedipus::Index do
     context "with ordering" do
       it "returns the results ordered accordingly" do
         index.search("badgers", order: {views: :desc})[:records].should == [
-          { id: 4, views: 3003, user_id: 0, status: "" },
-          { id: 1, views: 150,  user_id: 0, status: "" },
-          { id: 3, views: 41,   user_id: 0, status: "" },
+          { id: 4, views: 3003, user_id: 0, state: "" },
+          { id: 1, views: 150,  user_id: 0, state: "" },
+          { id: 3, views: 41,   user_id: 0, state: "" },
         ]
       end
 
@@ -265,9 +265,9 @@ describe Oedipus::Index do
     context "with attribute additions" do
       it "fetches the additional attributes" do
         index.search("badgers", attrs: [:*, "7 AS x"])[:records].should == [
-          { id: 1, views: 150,  user_id: 0, status: "", x: 7 },
-          { id: 3, views: 41,   user_id: 0, status: "", x: 7 },
-          { id: 4, views: 3003, user_id: 0, status: "", x: 7 },
+          { id: 1, views: 150,  user_id: 0, state: "", x: 7 },
+          { id: 3, views: 41,   user_id: 0, state: "", x: 7 },
+          { id: 4, views: 3003, user_id: 0, state: "", x: 7 },
         ]
       end
     end
@@ -304,19 +304,19 @@ describe Oedipus::Index do
 
       it "returns the main results in the top-level" do
         results[:records].should == [
-          { id: 1, views: 150,  user_id: 1, status: "" },
-          { id: 3, views: 41,   user_id: 2, status: "" },
-          { id: 4, views: 3003, user_id: 1, status: "" }
+          { id: 1, views: 150,  user_id: 1, state: "" },
+          { id: 3, views: 41,   user_id: 2, state: "" },
+          { id: 4, views: 3003, user_id: 1, state: "" }
         ]
       end
 
       it "applies the filters on top of the base query" do
         results[:facets][:popular][:records].should == [
-          { id: 1, views: 150,  user_id: 1, status: "" },
-          { id: 4, views: 3003, user_id: 1, status: "" }
+          { id: 1, views: 150,  user_id: 1, state: "" },
+          { id: 4, views: 3003, user_id: 1, state: "" }
         ]
         results[:facets][:di_carla][:records].should == [
-          { id: 3, views: 41, user_id: 2, status: "" }
+          { id: 3, views: 41, user_id: 2, state: "" }
         ]
       end
     end
@@ -334,7 +334,7 @@ describe Oedipus::Index do
 
       it "applies the filters on top of the base query" do
         results[:facets][:di_carla][:records].should == [
-          { id: 3, views: 41, user_id: 2, status: "" }
+          { id: 3, views: 41, user_id: 2, state: "" }
         ]
       end
     end
@@ -351,7 +351,7 @@ describe Oedipus::Index do
 
       it "entirely replaces the base query" do
         results[:facets][:rabbits][:records].should == [
-          { id: 2, views: 87, user_id: 1, status: "" }
+          { id: 2, views: 87, user_id: 1, state: "" }
         ]
       end
     end
@@ -368,7 +368,36 @@ describe Oedipus::Index do
 
       it "merges the queries" do
         results[:facets][:in_body][:records].should == [
-          { id: 1, views: 150,  user_id: 1, status: "" },
+          { id: 1, views: 150,  user_id: 1, state: "" },
+        ]
+      end
+    end
+
+    context "with multi-dimensional facets" do
+      let(:results) do
+        index.search(
+          "badgers",
+          facets: {
+            popular: {
+              views: Oedipus.gte(50),
+              facets: {
+                with_foxes: "%{query} & foxes"
+              }
+            },
+          }
+        )
+      end
+
+      it "merges the results in the outer facets" do
+        results[:facets][:popular][:records].should == [
+          { id: 1, views: 150,  user_id: 1, state: "" },
+          { id: 4, views: 3003, user_id: 1, state: "" }
+        ]
+      end
+
+      it "merges the results in the inner facets" do
+        results[:facets][:popular][:facets][:with_foxes][:records].should == [
+          { id: 1, views: 150,  user_id: 1, state: "" }
         ]
       end
     end
@@ -398,12 +427,12 @@ describe Oedipus::Index do
           rabbits: "rabbits"
         )
         results[:badgers][:records].should == [
-          { id: 1, views: 150,  user_id: 1, status: "" },
-          { id: 3, views: 41,   user_id: 2, status: "" },
-          { id: 4, views: 3003, user_id: 1, status: "" }
+          { id: 1, views: 150,  user_id: 1, state: "" },
+          { id: 3, views: 41,   user_id: 2, state: "" },
+          { id: 4, views: 3003, user_id: 1, state: "" }
         ]
         results[:rabbits][:records].should == [
-          { id: 2, views: 87, user_id: 1, status: "" }
+          { id: 2, views: 87, user_id: 1, state: "" }
         ]
       end
     end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: oedipus
 version: !ruby/object:Gem::Version
-  version: 0.0.5
+  version: 0.0.6
   prerelease: 
 platform: ruby
 authors:
@@ -9,11 +9,11 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-04-29 00:00:00.000000000 Z
+date: 2012-05-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rspec
-  requirement: &14417400 !ruby/object:Gem::Requirement
+  requirement: &20757600 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -21,10 +21,10 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *14417400
+  version_requirements: *20757600
 - !ruby/object:Gem::Dependency
   name: rake-compiler
-  requirement: &14717600 !ruby/object:Gem::Requirement
+  requirement: &20756700 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -32,7 +32,7 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *14717600
+  version_requirements: *20756700
 description: ! "== Sphinx 2 Comes to Ruby\n\nOedipus brings full support for Sphinx
   2 to Ruby:\n\n  - real-time indexes (insert, replace, update, delete)\n  - faceted
   search (variations on a base query)\n  - multi-queries (multiple queries executed
@@ -109,7 +109,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: 4166348847258820467
+      hash: -1569263709563515069
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
@@ -118,7 +118,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: 4166348847258820467
+      hash: -1569263709563515069
 requirements: []
 rubyforge_project: oedipus
 rubygems_version: 1.8.11