slingshot-rb 0.0.7 → 0.0.8

data/README.markdown

@@ -3,27 +3,21 @@ Slingshot
 
 ![Slingshot](https://github.com/karmi/slingshot/raw/master/slingshot.png)
 
-_Slingshot_ aims to provide rich and comfortable Ruby API and DSL for the
-[ElasticSearch](http://www.elasticsearch.org/) search engine/database.
+_Slingshot_ is a Ruby client for the [ElasticSearch](http://www.elasticsearch.org/) search engine/database.
+It aims to provide rich and comfortable Ruby API in the form of a simple domain-specific language.
 
-_ElasticSearch_ is a scalable, distributed, highly-available,
+_ElasticSearch_ is a scalable, distributed, cloud-ready, highly-available,
 RESTful database communicating by JSON over HTTP, based on [Lucene](http://lucene.apache.org/),
-written in Java. It manages to be very simple and very powerful at the same time.
-You should seriously consider it to power search in your Ruby applications:
-it will deliver all the features you want — and many more you may have not
-imagined yet (native geo search? date histogram facets? _percolator_?)
-
-_Slingshot_ currently allows basic operation with the index and searching. More is planned.
-
+written in Java. It manages to be very simple to use and very powerful at the same time.
 
 Installation
 ------------
 
 First, you need a running _ElasticSearch_ server. Thankfully, it's easy. Let's define easy:
 
-    $ curl -k -L -o elasticsearch-0.15.0.tar.gz http://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.15.0.tar.gz
-    $ tar -zxvf elasticsearch-0.15.0.tar.gz
-    $ ./elasticsearch-0.15.0/bin/elasticsearch -f
+    $ curl -k -L -o elasticsearch-0.15.2.tar.gz http://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.15.2.tar.gz
+    $ tar -zxvf elasticsearch-0.15.2.tar.gz
+    $ ./elasticsearch-0.15.2/bin/elasticsearch -f
 
 OK, easy. Now, install the gem via Rubygems:
 
@@ -63,8 +57,8 @@ First, let's create an index named `articles` and store/index some documents:
       refresh
     end
 
-You can also create the
-index with specific [mappings](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html), such as:
+We can also create the
+index with specific [mapping](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html):
 
     Slingshot.index 'articles' do
       create :mappings => {
@@ -82,7 +76,7 @@ index with specific [mappings](http://www.elasticsearch.org/guide/reference/api/
 Now, let's query the database.
 
 We are searching for articles whose `title` begins with letter “T”, sorted by `title` in `descending` order,
-filtering them for ones tagged “ruby”, and also retrieving some [_facets_](http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Faceted-Search-Solr)
+filtering them for ones tagged “ruby”, and also retrieving some [_facets_](http://www.elasticsearch.org/guide/reference/api/search/facets/)
 from the database:
 
     s = Slingshot.search 'articles' do
@@ -139,56 +133,65 @@ We can display the full query JSON:
     puts s.to_json
     # {"facets":{"current-tags":{"terms":{"field":"tags"}},"global-tags":{"global":true,"terms":{"field":"tags"}}},"query":{"query_string":{"query":"title:T*"}},"filter":{"terms":{"tags":["ruby"]}},"sort":[{"title":"desc"}]}
 
-Or, we can display the corresponding `curl` command for easy debugging:
+Or, better, we can display the corresponding `curl` command for easy debugging:
 
     puts s.to_curl
     # curl -X POST "http://localhost:9200/articles/_search?pretty=true" -d '{"facets":{"current-tags":{"terms":{"field":"tags"}},"global-tags":{"global":true,"terms":{"field":"tags"}}},"query":{"query_string":{"query":"title:T*"}},"filter":{"terms":{"tags":["ruby"]}},"sort":[{"title":"desc"}]}'
 
+Since `curl` is the crucial debugging tool in _ElasticSearch_ land, we can log every search query in `curl` format:
+
+    Slingshot.configure { logger 'elasticsearch.log' }
+
 
 Features
 --------
 
-Currently, _Slingshot_ supports only a limited subset of vast _ElasticSearch_ [Search API](http://www.elasticsearch.org/guide/reference/api/search/request-body.html) and it's [Query DSL](http://www.elasticsearch.org/guide/reference/query-dsl/):
+Currently, _Slingshot_ supports main features of the _ElasticSearch_ [Search API](http://www.elasticsearch.org/guide/reference/api/search/request-body.html) and it's [Query DSL](http://www.elasticsearch.org/guide/reference/query-dsl/). In present, it allows you to:
 
-* Creating, deleting and refreshing the index
-* Creating the index with specific [mapping](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html)
-* Storing a document in the index
-* [Querying](https://github.com/karmi/slingshot/blob/master/examples/dsl.rb) the index with the `query_string`, `term` and `terms` types of queries
-* [Sorting](http://elasticsearch.org/guide/reference/api/search/sort.html) the results by `fields`
-* [Filtering](http://elasticsearch.org/guide/reference/query-dsl/) the results
-* Retrieving a _terms_ type of [facets](http://www.elasticsearch.org/guide/reference/api/search/facets/index.html) -- other types are high priority
-* Returning just specific `fields` from documents
-* Paging with `from` and `size` query options
+* Create, delete and refresh the index
+* Create the index with specific [mapping](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html)
+* Store a document in the index
+* [Query](https://github.com/karmi/slingshot/blob/master/examples/dsl.rb) the index with the `query_string`, `term`, `terms` and `match_all` types of queries
+* [Sort](http://elasticsearch.org/guide/reference/api/search/sort.html) the results by `fields`
+* [Filter](http://elasticsearch.org/guide/reference/query-dsl/) the results
+* Retrieve the _terms_ and _date histogram_ types of [facets](http://www.elasticsearch.org/guide/reference/api/search/facets/index.html) (other types are high priority)
+* [Highlight](http://www.elasticsearch.org/guide/reference/api/search/highlighting.html) matching fields
+* Return just specific `fields` from documents
+* Page the results with `from` and `size` query options
+* Log the `curl`-equivalent of requests and response JSON
 
-See the [`examples/dsl.rb`](blob/master/examples/dsl.rb).
+See the [`examples/slingshot-dsl.rb`](blob/master/examples/slingshot-dsl.rb) file for the full, working examples.
 
 _Slingshot_ wraps the results in a enumerable `Results::Collection` class, and every result in a `Results::Item` class,
-which looks like a child of `Hash` and `Openstruct`.
+which looks like a child of `Hash` and `Openstruct`, for smooth iterating and displaying the results.
 
-You may wrap the result items in your own class by setting the `Configuration.wrapper` property.
-Check out file `test/unit/results_collection_test.rb` to see how to do that.
+You may wrap the result items in your own class just by setting the `Configuration.wrapper` property,
+supposed your class takes a hash of attributes upon initialization, in ActiveModel/ActiveRecord manner.
+Please see the files `test/models/article.rb` and `test/unit/results_collection_test.rb` for details.
 
 
-Todo & Plans
-------------
+Todo, Plans & Ideas
+-------------------
+
+_Slingshot_ is already used in production by its authors. Nevertheless, it's not finished yet.
 
-In order of importance:
+The todos and plans are vast, and the most important are listed below, in the order of importance:
 
-* Seamless _ActiveModel_ compatibility for easy usage in _Rails_ applications (this also means nearly full _ActiveRecord_ compatibility). See the [`activemodel`](https://github.com/karmi/slingshot/compare/activemodel) branch
-* Seamless [will_paginate](https://github.com/mislav/will_paginate) compatibility for easy pagination
+* Seamless _ActiveModel_ compatibility for easy usage in _Rails_ applications (this also means nearly full _ActiveRecord_ compatibility). See the ongoing work in the [`activemodel`](https://github.com/karmi/slingshot/compare/activemodel) branch
+* Seamless [will_paginate](https://github.com/mislav/will_paginate) compatibility for easy pagination. Already [implemented](https://github.com/karmi/slingshot/commit/e1351f6) on the `activemodel` branch
 * [Mapping](http://www.elasticsearch.org/guide/reference/mapping/) definition for models
 * Proper RDoc annotations for the source code
 * Dual interface: allow to simply pass queries/options for _ElasticSearch_ as a Hash in any method
 * [Histogram](http://www.elasticsearch.org/guide/reference/api/search/facets/histogram-facet.html) facets
-* Seamless support for [auto-updating _river_ index](http://www.elasticsearch.org/guide/reference/river/couchdb.html) for _CouchDB_ `_changes` feed
 * [Statistical](http://www.elasticsearch.org/guide/reference/api/search/facets/statistical-facet.html) facets
 * [Geo Distance](http://www.elasticsearch.org/guide/reference/api/search/facets/geo-distance-facet.html) facets
 * [Index aliases](http://www.elasticsearch.org/guide/reference/api/admin-indices-aliases.html) management
 * [Analyze](http://www.elasticsearch.org/guide/reference/api/admin-indices-analyze.html) API support
-* [Highligting](http://www.elasticsearch.org/guide/reference/api/search/highlighting.html) support
 * [Bulk](http://www.elasticsearch.org/guide/reference/api/bulk.html) API
 * Embedded webserver to display statistics and to allow easy searches
+* Seamless support for [auto-updating _river_ index](http://www.elasticsearch.org/guide/reference/river/couchdb.html) for _CouchDB_ `_changes` feed
 
+The full ActiveModel integration is planned for the 1.0 release.
 
 Other Clients
 -------------

data/Rakefile

@@ -6,8 +6,9 @@ task :default => :test
 require 'rake/testtask'
 Rake::TestTask.new(:test) do |test|
   test.libs << 'lib' << 'test'
-  test.pattern = 'test/**/*_test.rb'
+  test.test_files = FileList['test/unit/*_test.rb', 'test/integration/*_test.rb']
   test.verbose = true
+  # test.warning = true
 end
 
 namespace :test do

data/examples/slingshot-dsl.rb

@@ -1,6 +1,13 @@
-# **Slingshot** is a rich and comfortable Ruby API and DSL for the
+# **Slingshot** is a rich and comfortable Ruby client for the
 # [_ElasticSearch_](http://www.elasticsearch.org/) search engine/database.
 #
+# It provides the API for the main _ElasticSearch_ features, and this document
+# will walk you through them.
+#
+# <img src="http://github.com/favicon.ico" style="position:relative; top:2px">
+# _Slingshot_ is open source, and you can download or clone the source code
+# from <https://github.com/karmi/slingshot>.
+#
 # _ElasticSearch_ is a scalable, distributed, highly-available,
 # RESTful database communicating by JSON over HTTP, based on [Lucene](http://lucene.apache.org/),
 # written in Java. It manages to be very simple and very powerful at the same time.
@@ -22,9 +29,9 @@ require 'slingshot'
 # You'll need a working and running _ElasticSearch_ server. Thankfully, that's easy.
 ( puts <<-"INSTALL" ; exit(1) ) unless RestClient.get('http://localhost:9200') rescue false
  [!] You don’t appear to have ElasticSearch installed. Please install and launch it with the following commands.
- curl -k -L -o elasticsearch-0.15.0.tar.gz http://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.15.0.tar.gz
- tar -zxvf elasticsearch-0.15.0.tar.gz
- ./elasticsearch-0.15.0/bin/elasticsearch -f
+ curl -k -L -o elasticsearch-0.15.2.tar.gz http://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.15.2.tar.gz
+ tar -zxvf elasticsearch-0.15.2.tar.gz
+ ./elasticsearch-0.15.2/bin/elasticsearch -f
 INSTALL
 
 ### Simple Usage
@@ -124,11 +131,41 @@ puts s.to_json
 puts "", "Try the query in Curl:", "-"*80
 puts s.to_curl
 
+# For debugging more complex situations, you can enable logging, so requests and responses
+# will be logged using the `curl`-based format.
+
+Slingshot.configure do
+
+  # By default, at the _info_ level, only the `curl`-format of request and
+  # basic information about the response will be logged:
+  #
+  #     # 2011-04-24 11:34:01:150 [CREATE] ("articles")
+  #     #
+  #     curl -X POST "http://localhost:9200/articles" -d '{
+  #     
+  #     }'
+  #     
+  #     # 2011-04-24 11:34:01:152 [200]
+  #
+  logger 'elasticsearch.log'
+
+  # For debugging, you can switch to the _debug_ level, which will log the complete JSON responses.
+  #
+  # That's very convenient if you want to post a recreation of some problem or solution to the mailing list, IRC, etc.
+  #
+  logger 'elasticsearch.log', :level => 'debug'
+
+  # Note that you can pass any [`IO`](http://www.ruby-doc.org/core/classes/IO.html)-compatible Ruby object as a logging device.
+  #
+  logger STDERR
+end
 
 ##### Other Types of Queries
 
-# Of course, we may want to define our queries more expressively, for instance
-# when we're searching for articles with specific _tags_.
+# Well, query strings are convenient for simple searches, but
+# we may want to define our queries more expressively.
+#
+# For instance when we're searching for articles with specific _tags_.
 
 # Let's suppose we want to search for articles tagged “ruby” _or_ “python”.
 # That's a great excuse to use a [_terms_](http://elasticsearch.org/guide/reference/query-dsl/terms-query.html)
@@ -163,17 +200,20 @@ s.results.each do |document|
   puts "* #{ document.title } [tags: #{document.tags.join(', ')}]"
 end
 
-# _ElasticSearch_ allows us to do many more types of queries.
+# _ElasticSearch_ supports many types of [queries](http://www.elasticsearch.org/guide/reference/query-dsl/).
+#
 # Eventually, _Slingshot_ will support all of them.
 # So far, only these are supported:
 #
+# * [string](http://www.elasticsearch.org/guide/reference/query-dsl/query-string-query.html)
 # * [term](http://elasticsearch.org/guide/reference/query-dsl/term-query.html)
 # * [terms](http://elasticsearch.org/guide/reference/query-dsl/terms-query.html)
+# * [all](http://www.elasticsearch.org/guide/reference/query-dsl/match-all-query.html)
 
 ##### Faceted Search
 
 # _ElasticSearch_ makes it trivial to retrieve complex aggregated data from our index/database,
-# so called [_facets_](http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Faceted-Search-Solr).
+# so called [_facets_](http://www.elasticsearch.org/guide/reference/api/search/facets/index.html).
 
 # Let's say we want to display article counts for every tag in the database.
 # For that, we'll use a _terms_ facet.
@@ -183,7 +223,7 @@ s = Slingshot.search 'articles' do
   # We will search for articles whose title begins with letter “T”,
   query { string 'title:T*' }
 
-  # and retrieve their counts “bucketed” by their `tags`.
+  # and retrieve their counts, “bucketed” by their `tags`.
   facet 'tags' do
     terms :tags
   end
@@ -216,8 +256,8 @@ s = Slingshot.search 'articles' do
     terms :tags, :global => true
   end
 
-  # As you can see, we can even combine facets scoped
-  # to the current query with global facets.
+  # As you can see, we can even combine facets scoped to the current query
+  # with global facets — we'll just use a different name.
   facet 'current-tags' do
     terms :tags
   end
@@ -247,18 +287,25 @@ s.results.facets['global-tags']['terms'].each do |f|
   puts "#{f['term'].ljust(10)} #{f['count']}"
 end
 
+# _ElasticSearch_ supports many types of advanced facets.
+#
+# Eventually, _Slingshot_ will support all of them.
+# So far, only these are supported:
+#
+# * [terms](http://www.elasticsearch.org/guide/reference/api/search/facets/terms-facet.html)
+# * [date](http://www.elasticsearch.org/guide/reference/api/search/facets/date-histogram-facet.html)
+
 # The real power of facets lies in their combination with
-# [filters](http://elasticsearch.karmi.cz/guide/reference/api/search/filter.html),
-# though:
+# [filters](http://elasticsearch.org/guide/reference/api/search/filter.html).
 
-# > When doing things like facet navigation,
-# > sometimes only the hits are needed to be filtered by the chosen facet,
-# > and all the facets should continue to be calculated based on the original query.
+# Filters enable us to restrict the returned documents, for example to a specific category,
+# but to calculate the facet counts based on the original query.
+# The immediate use case for that is the „faceted navigation“.
 
 
 ##### Filtered Search
 
-# So, let's make out search a bit complex. Let's search for articles whose titles begin
+# So, let's make our search a bit more complex. Let's search for articles whose titles begin
 # with letter “T”, again, but filter the results, so only the articles tagged “ruby”
 # are returned.
 s = Slingshot.search 'articles' do
@@ -301,9 +348,9 @@ end
 # (available as the `_score` property).
 
 # But, what if we want to sort the results based on some other criteria,
-# such as published date, price, etc? We can do that.
+# such as published date or product price? We can do that.
 s = Slingshot.search 'articles' do
-  # We search for articles tagged “ruby”
+  # We search for articles tagged “ruby”.
   query { string 'tags:ruby' } 
 
    # And sort them by their `title`, in descending order.
@@ -322,7 +369,7 @@ end
 
 s = Slingshot.search 'articles' do
   # We will just get all articles for this case.
-  query { string '*' } 
+  query { all } 
 
   sort do
     # We will sort the results by their `published_on` property in ascending (default) order,
@@ -343,15 +390,19 @@ end
 
 ##### Highlighting
 
-# Often, you want to highlight the matched snippets in your text.
-# _ElasticSearch_ provides many features for
-# [highlighting](http://www.elasticsearch.org/guide/reference/api/search/highlighting.html),
-# 
+# Often, you want to highlight the snippets matching your query in the
+# displayed results.
+# _ElasticSearch_ provides rich
+# [highlighting](http://www.elasticsearch.org/guide/reference/api/search/highlighting.html)
+# features, and Slingshot makes them trivial to use.
+#
+# Let's suppose that we want to highlight terms of our query.
+#
 s = Slingshot.search 'articles' do
-  # Let's search for documents containing “Two” in their titles.
+  # Let's search for documents containing word “Two” in their titles,
   query { string 'title:Two' } 
 
-   # And use the `highlight` method.
+   # and instruct _ElasticSearch_ to highlight relevant snippets.
   highlight :title
 end
 
@@ -361,7 +412,7 @@ s.results.each do |document|
   puts "Title: #{ document.title }, highlighted title: #{document.highlight.title}"
 end
 
-# Slingshot allows you to specify options for the highlighting, such as:
+# We can configure many options for highlighting, such as:
 #
 s = Slingshot.search 'articles' do
   query { string 'title:Two' }
@@ -372,7 +423,25 @@ s = Slingshot.search 'articles' do
   # • specifying their options
   highlight :title, :body => { :number_of_fragments => 0 }
 
-  # • or specifying highlighting options, such as the wrapper tag
+  # • or specifying global highlighting options, such as the wrapper tag
   highlight :title, :body, :options => { :tag => '<strong class="highlight">' }
 end
 
+
+
+#### What's next?
+
+# As you can see, [_Slingshot_](https://github.com/karmi/slingshot) supports the
+# main features of _ElasticSearch_ in Ruby.
+#
+# It allows you to create and delete indices, add documents, search them, retrieve facets, highlight the results,
+# and comes with usable logging facility.
+#
+# Of course, the holy grail of any search library is easy, painless integration with your Ruby classes, and,
+# most importantly, with ActiveRecord/ActiveModel classes.
+#
+# _Slingshot_ already provides such integration, though in experimental mode.
+# Check out the tests in the
+# [`activemodel`](https://github.com/karmi/slingshot/blob/activemodel/test/integration/active_model_searchable_test.rb) branch.
+#
+# Send any feedback via Github issues, or ask questions in the [#elasticsearch](irc://irc.freenode.net/#elasticsearch) IRC channel.

data/lib/slingshot/index.rb

@@ -7,17 +7,24 @@ module Slingshot
     end
 
     def delete
-      response = Configuration.client.delete "#{Configuration.url}/#{@name}"
-      return response =~ /error/ ? false : true
-    rescue
+      # FIXME: RestClient does not return response for DELETE requests?
+      @response = Configuration.client.delete "#{Configuration.url}/#{@name}"
+      return @response =~ /error/ ? false : true
+    rescue Exception => error
       false
+    ensure
+      curl = %Q|curl -X DELETE "#{Configuration.url}/#{@name}"|
+      logged(error, 'DELETE', curl)
     end
 
     def create(options={})
-      # http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html
-      Configuration.client.post "#{Configuration.url}/#{@name}", Yajl::Encoder.encode(options)
-    rescue
+      @options = options
+      @response = Configuration.client.post "#{Configuration.url}/#{@name}", Yajl::Encoder.encode(options)
+    rescue Exception => error
       false
+    ensure
+      curl = %Q|curl -X POST "#{Configuration.url}/#{@name}" -d '#{Yajl::Encoder.encode(options, :pretty => true)}'|
+      logged(error, 'CREATE', curl)
     end
 
     def mapping
@@ -46,17 +53,21 @@ module Slingshot
         else raise ArgumentError, "Please pass a JSON string or object with a 'to_indexed_json' method"
       end
 
-      if id
-        result = Configuration.client.post "#{Configuration.url}/#{@name}/#{type}/#{id}", document
-      else
-        result = Configuration.client.post "#{Configuration.url}/#{@name}/#{type}/", document
-      end
-      JSON.parse(result)
+      url = id ? "#{Configuration.url}/#{@name}/#{type}/#{id}" : "#{Configuration.url}/#{@name}/#{type}/"
+
+      @response = Configuration.client.post url, document
+      JSON.parse(@response)
+
+    rescue Exception => error
+      raise
+    ensure
+      curl = %Q|curl -X POST "#{url}" -d '#{document}'|
+      logged(error, "/#{@name}/#{type}/", curl)
     end
 
     def retrieve(type, id)
-      result = Configuration.client.get "#{Configuration.url}/#{@name}/#{type}/#{id}"
-      h = JSON.parse(result)
+      @response = Configuration.client.get "#{Configuration.url}/#{@name}/#{type}/#{id}"
+      h = JSON.parse(@response)
       if Configuration.wrapper == Hash then h
       else
         document = h['_source'] ? h['_source'] : h['fields']
@@ -66,7 +77,30 @@ module Slingshot
     end
 
     def refresh
-      Configuration.client.post "#{Configuration.url}/#{@name}/_refresh", ''
+      @response = Configuration.client.post "#{Configuration.url}/#{@name}/_refresh", ''
+    rescue Exception => error
+      raise
+    ensure
+      curl = %Q|curl -X POST "#{Configuration.url}/#{@name}/_refresh"|
+      logged(error, '_refresh', curl)
+    end
+
+    def logged(error=nil, endpoint='/', curl='')
+      if Configuration.logger
+
+        Configuration.logger.log_request endpoint, @name, curl
+
+        code = @response ? @response.code : error.message rescue 200
+
+        if Configuration.logger.level.to_s == 'debug'
+          # FIXME: Depends on RestClient implementation
+          body = @response ? Yajl::Encoder.encode(@response.body, :pretty => true) : error.http_body rescue ''
+        else
+          body = ''
+        end
+
+        Configuration.logger.log_response code, nil, body
+      end
     end
 
   end

data/lib/slingshot/logger.rb

@@ -21,21 +21,21 @@ module Slingshot
     end
 
     def log_request(endpoint, params=nil, curl='')
-      # [_search] (articles,users) 2001-02-12 18:20:42:32
+      # 2001-02-12 18:20:42:32 [_search] (articles,users)
       #
       # curl -X POST ....
       #
-      content  = "# [#{endpoint}] "
-      content += "(#{params.inspect}) " if params
-      content += time
+      content  = "# #{time}"
+      content += " [#{endpoint}]"
+      content += " (#{params.inspect})" if params
       content += "\n#\n"
       content += curl
       content += "\n\n"
       write content
     end
 
-    def log_response(status, json)
-      # [200 OK] (4 msec) Sat Feb 12 19:20:47 2011
+    def log_response(status, took=nil, json='')
+      # 2001-02-12 18:20:42:32 [200] (4 msec)
       #
       # {
       #   "took" : 4,
@@ -43,12 +43,11 @@ module Slingshot
       #   ...
       # }
       #
-      took = JSON.parse(json)['took'] rescue nil
-      content  = "# [#{status}] "
-      content += "(#{took} msec) " if took
-      content += time
-      content += "\n#\n"
-      json.each_line { |line| content += "# #{line}" }
+      content  = "# #{time}"
+      content += " [#{status}]"
+      content += " (#{took} msec)" if took
+      content += "\n#\n" unless json == ''
+      json.each_line { |line| content += "# #{line}" } unless json == ''
       content += "\n\n"
       write content
     end

data/lib/slingshot/search.rb

@@ -64,31 +64,18 @@ module Slingshot
       def perform
         @url      = "#{Configuration.url}/#{indices.join(',')}/_search"
         @response = Configuration.client.post(@url, self.to_json)
-        @json     = Yajl::Parser.parse(@response)
+        @json     = Yajl::Parser.parse(@response.body)
         @results  = Results::Collection.new(@json)
         self
-      rescue Exception => e
-        STDERR.puts "[REQUEST FAILED]\n#{self.to_curl}\n"
+      rescue Exception => error
+        STDERR.puts "[REQUEST FAILED] #{self.to_curl}\n"
         raise
       ensure
-        if Configuration.logger
-          Configuration.logger.log_request '_search', indices, to_curl
-          if Configuration.logger.level == 'debug'
-            # FIXME: Depends on RestClient implementation
-            if @response
-              code = @response.code
-              body = Yajl::Encoder.encode(@json, :pretty => true)
-            else
-              code = e.message
-              body = e.http_body
-            end
-            Configuration.logger.log_response code, body
-          end
-        end
+        logged(error)
       end
 
       def to_curl
-        %Q|curl -X POST "#{Configuration.url}/#{indices}/_search?pretty=true" -d '#{self.to_json}'|
+        %Q|curl -X POST "#{Configuration.url}/#{indices.join(',')}/_search?pretty=true" -d '#{self.to_json}'|
       end
 
       def to_json
@@ -104,6 +91,25 @@ module Slingshot
         Yajl::Encoder.encode(request)
       end
 
+      def logged(error=nil)
+        if Configuration.logger
+
+          Configuration.logger.log_request '_search', indices, to_curl
+
+          code = @response ? @response.code : error.message
+          took = @json['took'] rescue nil
+
+          if Configuration.logger.level.to_s == 'debug'
+            # FIXME: Depends on RestClient implementation
+            body = @response ? Yajl::Encoder.encode(@json, :pretty => true) : body = error.http_body
+          else
+            body = ''
+          end
+
+          Configuration.logger.log_response code, took, body
+        end
+      end
+
     end
 
   end

data/lib/slingshot/version.rb

@@ -1,3 +1,3 @@
 module Slingshot
-  VERSION = "0.0.7"
+  VERSION = "0.0.8"
 end

data/test/integration/highlight_test.rb

@@ -5,12 +5,13 @@ module Slingshot
   class HighlightIntegrationTest < Test::Unit::TestCase
     include Test::Integration
 
-    context "Highlight" do 
+    context "Highlight" do
+      teardown { Slingshot.index('highlight-test').delete }
 
       should "add 'highlight' field to the result item" do
-        Slingshot::Configuration.logger STDERR, :level => 'debug'
+        # Slingshot::Configuration.logger STDERR, :level => 'debug'
         s = Slingshot.search('articles-test') do
-          # query { string '-w' }
+          query { string 'Two' }
           highlight :title
         end
 
@@ -20,6 +21,31 @@ module Slingshot
         assert doc.highlight.title.to_s.include?('<em>'), "Highlight does not include default highlight tag"
       end
 
+      should "return entire content with highlighted fragments in ElasticSearch >= 0.16.0" do
+        # Slingshot::Configuration.logger STDERR, :level => 'debug'
+
+        content = "A Fox one day fell into a deep well and could find no means of escape. A Goat, overcome with thirst, came to the same well, and seeing the Fox, inquired if the water was good. Concealing his sad plight under a merry guise, the Fox indulged in a lavish praise of the water, saying it was excellent beyond measure, and encouraging him to descend. The Goat, mindful only of his thirst, thoughtlessly jumped down, but just as he drank, the Fox informed him of the difficulty they were both in and suggested a scheme for their common escape. \"If,\" said he, \"you will place your forefeet upon the wall and bend your head, I will run up your back and escape, and will help you out afterwards.\" The Goat readily assented and the Fox leaped upon his back. Steadying himself with the Goat horns, he safely reached the mouth of the well and made off as fast as he could. When the Goat upbraided him for breaking his promise, he turned around and cried out, \"You foolish old fellow! If you had as many brains in your head as you have hairs in your beard, you would never have gone down before you had inspected the way up, nor have exposed yourself to dangers from which you had no means of escape.\" Look before you leap."
+
+        Slingshot.index 'highlight-test' do
+          delete
+          create
+          store   :id => 1, :content => content
+          refresh
+        end
+        sleep(0.5)
+
+        s = Slingshot.search('highlight-test') do
+          query { string 'fox' }
+          highlight :content => { :number_of_fragments => 0 }
+        end
+
+        doc = s.results.first
+        assert_not_nil doc.highlight.content
+
+        highlight = doc.highlight.content
+        assert highlight.to_s.include?('<em>'), "Highlight does not include default highlight tag"
+      end
+
     end
 
   end

data/test/integration/index_mapping_test.rb

@@ -2,7 +2,7 @@ require 'test_helper'
 
 module Slingshot
 
-  class IndexMappingTest < Test::Unit::TestCase
+  class IndexMappingIntegrationTest < Test::Unit::TestCase
     include Test::Integration
 
     context "Default mapping" do

data/test/integration/index_store_test.rb

@@ -2,7 +2,7 @@ require 'test_helper'
 
 module Slingshot
 
-  class IndexStoreTest < Test::Unit::TestCase
+  class IndexStoreIntegrationTest < Test::Unit::TestCase
     include Test::Integration
 
     context "Storing the documents in index" do

data/test/unit/logger_test.rb

@@ -46,12 +46,12 @@ module Slingshot
 
     context "levels" do
 
-      should "have default level" do
+      should "have the default level" do
         logger = Logger.new STDERR
         assert_equal 'info', logger.level
       end
 
-      should "have set the level" do
+      should "set the level" do
         logger = Logger.new STDERR, :level => 'debug'
         assert_equal 'debug', logger.level
       end
@@ -66,7 +66,7 @@ module Slingshot
 
       should "log request in correct format" do
         log = (<<-"log;").gsub(/^ +/, '')
-          # [_search] (["articles", "users"]) 2011-03-19 11:00:00:L
+          # 2011-03-19 11:00:00:L [_search] (["articles", "users"])
           #
           curl -X GET http://...
 
@@ -98,14 +98,14 @@ module Slingshot
         }
         json;
         log  = (<<-"log;").gsub(/^\s*/, '')
-          # [200 OK] (4 msec) 2011-03-19 11:00:00:L
+          # 2011-03-19 11:00:00:L [200 OK] (4 msec)
           #
         log;
         # log += json.split.map { |line| "# #{line}" }.join("\n")
         json.each_line { |line| log += "# #{line}" }
         log += "\n\n"
         @logger.expects(:write).with(log)
-        @logger.log_response('200 OK', json)
+        @logger.log_response('200 OK', 4, json)
       end
 
     end

data/test/unit/search_test.rb

@@ -32,6 +32,13 @@ module Slingshot
                      s.to_curl
       end
 
+      should "return curl snippet with multiple indices for debugging" do
+        s = Search::Search.new('index_1', 'index_2') do
+          query { string 'title:foo' }
+        end
+        assert_match /index_1,index_2/, s.to_curl
+      end
+
       should "allow chaining" do
         assert_nothing_raised do
           Search::Search.new('index').query { }.sort { title 'desc' }.size(5).sort { name 'asc' }.from(1)
@@ -39,8 +46,10 @@ module Slingshot
       end
 
       should "perform the search" do
-        Configuration.client.expects(:post).returns('{"hits":[]}')
+        response = stub(:body => '{"took":1,"hits":[]}', :code => 200)
+        Configuration.client.expects(:post).returns(response)
         Results::Collection.expects(:new).returns([])
+
         s = Search::Search.new('index')
         s.perform
         assert_not_nil s.results
@@ -58,10 +67,12 @@ module Slingshot
       should "log request, but not response, when logger is set" do
         Configuration.logger STDERR
 
-        Configuration.client.expects(:post).returns('{"hits":[]}')
+        response = stub(:body => '{"took":1,"hits":[]}', :code => 200)
+        Configuration.client.expects(:post).returns(response)
+
         Results::Collection.expects(:new).returns([])
         Configuration.logger.expects(:log_request).returns(true)
-        Configuration.logger.expects(:log_response).never
+        Configuration.logger.expects(:log_response).with(200, 1, '')
 
         Search::Search.new('index').perform
       end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: slingshot-rb
 version: !ruby/object:Gem::Version 
-  hash: 17
+  hash: 15
   prerelease: 
   segments: 
   - 0
   - 0
-  - 7
-  version: 0.0.7
+  - 8
+  version: 0.0.8
 platform: ruby
 authors: 
 - Karel Minarik
@@ -15,7 +15,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2011-04-02 00:00:00 +02:00
+date: 2011-04-24 00:00:00 +02:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency