elasticsearch-persistence 0.0.0 → 0.0.1

checksums.yaml

@@ -0,0 +1,15 @@
+---
+!binary "U0hBMQ==":
+  metadata.gz: !binary |-
+    YTFjMGYyOGMzYTRjNWY3YjNjOWJhNjA5MWI1Nzc5Y2MyNTJjZmU4MA==
+  data.tar.gz: !binary |-
+    MDQwNzM2ZGJiZTQ4Yjk2NWZlMDZkM2Y5ODE4ODkzZTMzMzU5NWJmMQ==
+SHA512:
+  metadata.gz: !binary |-
+    MDY0NjExYzBlNTQ3NzM2YjQzNTUyMTRhYzk3ODk5Njk5MzdlZDBlMzE1ZGM3
+    YjYxMmVmM2QxZmM1YzY0N2RkYTZkMjZiYWNiOGFlM2NlOGQzMmZjN2ZhYmFm
+    YTI4NjFlMWZhNjE3ZjJhMGFlNjZhYzkyOTA4Y2Q5OGFmNzk3ZWI=
+  data.tar.gz: !binary |-
+    YmJhMzFhZDBjNTQ4ZjU1ZTA5ZGM3MzI2MmY2MzhlNGIyNTBkNDc3Nzk5ZDAy
+    MGE4MGU4MTc4ZjFjMjJiMGExM2JhOTRhMWM0NmVkZTEzYTkzOWExNjQ3ODIx
+    NTYxNmQ0NDQ1ODA2NTIzNWJhMTEyYjZkNDhlYmZjZGZkNjkyN2Y=

data/LICENSE.txt

@@ -1,22 +1,13 @@
-Copyright (c) 2013 Karel Minarik
+Copyright (c) 2014 Elasticsearch <http://www.elasticsearch.org>
 
-MIT License
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
 
-Permission is hereby granted, free of charge, to any person obtaining
-a copy of this software and associated documentation files (the
-"Software"), to deal in the Software without restriction, including
-without limitation the rights to use, copy, modify, merge, publish,
-distribute, sublicense, and/or sell copies of the Software, and to
-permit persons to whom the Software is furnished to do so, subject to
-the following conditions:
+    http://www.apache.org/licenses/LICENSE-2.0
 
-The above copyright notice and this permission notice shall be
-included in all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
-NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
-LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
-OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
-WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/README.md

@@ -1,29 +1,447 @@
 # Elasticsearch::Persistence
 
-TODO: Write a gem description
+Persistence layer for Ruby domain objects in Elasticsearch, using the Repository and ActiveRecord patterns.
+
+The library is compatible with Ruby 1.9.3 (or higher) and Elasticsearch 1.0 (or higher).
 
 ## Installation
 
-Add this line to your application's Gemfile:
+Install the package from [Rubygems](https://rubygems.org):
 
-    gem 'elasticsearch-persistence'
+    gem install elasticsearch-persistence
 
-And then execute:
+To use an unreleased version, either add it to your `Gemfile` for [Bundler](http://bundler.io):
 
-    $ bundle
+    gem 'elasticsearch-persistence', git: 'git://github.com/elasticsearch/elasticsearch-rails.git'
 
-Or install it yourself as:
+or install it from a source code checkout:
 
-    $ gem install elasticsearch-persistence
+    git clone https://github.com/elasticsearch/elasticsearch-rails.git
+    cd elasticsearch-rails/elasticsearch-persistence
+    bundle install
+    rake install
 
 ## Usage
 
-TODO: Write usage instructions here
+### The Repository Pattern
+
+The `Elasticsearch::Persistence::Repository` module provides an implementation of the
+[repository pattern](http://martinfowler.com/eaaCatalog/repository.html) and allows
+to save, delete, find and search objects stored in Elasticsearch, as well as configure
+mappings and settings for the index.
+
+Let's have a simple plain old Ruby object (PORO):
+
+```ruby
+class Note
+  attr_reader :attributes
+
+  def initialize(attributes={})
+    @attributes = attributes
+  end
+
+  def to_hash
+    @attributes
+  end
+end
+```
+
+Let's create a default, "dumb" repository, as a first step:
+
+```ruby
+require 'elasticsearch/persistence'
+repository = Elasticsearch::Persistence::Repository.new
+```
+
+We can save a `Note` instance into the repository...
+
+```ruby
+note = Note.new id: 1, text: 'Test'
+
+repository.save(note)
+# PUT http://localhost:9200/repository/note/1 [status:201, request:0.210s, query:n/a]
+# > {"id":1,"text":"Test"}
+# < {"_index":"repository","_type":"note","_id":"1","_version":1,"created":true}
+```
+
+...find it...
+
+```ruby
+n = repository.find(1)
+# GET http://localhost:9200/repository/_all/1 [status:200, request:0.003s, query:n/a]
+# < {"_index":"repository","_type":"note","_id":"1","_version":2,"found":true, "_source" : {"id":1,"text":"Test"}}
+=> <Note:0x007fcbfc0c4980 @attributes={"id"=>1, "text"=>"Test"}>
+```
+
+...search for it...
+
+```ruby
+repository.search(query: { match: { text: 'test' } }).first
+# GET http://localhost:9200/repository/_search [status:200, request:0.005s, query:0.002s]
+# > {"query":{"match":{"text":"test"}}}
+# < {"took":2, ... "hits":{"total":1, ... "hits":[{ ... "_source" : {"id":1,"text":"Test"}}]}}
+=> <Note:0x007fcbfc1c7b70 @attributes={"id"=>1, "text"=>"Test"}>
+```
+
+...or delete it:
+
+```ruby
+repository.delete(note)
+# DELETE http://localhost:9200/repository/note/1 [status:200, request:0.014s, query:n/a]
+# < {"found":true,"_index":"repository","_type":"note","_id":"1","_version":3}
+=> {"found"=>true, "_index"=>"repository", "_type"=>"note", "_id"=>"1", "_version"=>2}
+```
+
+The repository module provides a number of features and facilities to configure and customize the behaviour:
+
+* Configuring the Elasticsearch [client](https://github.com/elasticsearch/elasticsearch-ruby#usage) being used
+* Setting the index name, document type, and object class for deserialization
+* Composing mappings and settings for the index
+* Creating, deleting or refreshing the index
+* Finding or searching for documents
+* Providing access both to domain objects and hits for search results
+* Providing access to the Elasticsearch response for search results (aggregations, total, ...)
+* Defining the methods for serialization and deserialization
+
+You can use the default repository class, or include the module in your own. Let's review it in detail.
+
+#### The Default Class
+
+For simple cases, you can use the default, bundled repository class, and configure/customize it:
+
+```ruby
+repository = Elasticsearch::Persistence::Repository.new do
+  # Configure the Elasticsearch client
+  client Elasticsearch::Client.new url: ENV['ELASTICSEARCH_URL'], log: true
+
+  # Set a custom index name
+  index :my_notes
+
+  # Set a custom document type
+  type  :my_note
+
+  # Specify the class to inicialize when deserializing documents
+  klass Note
+
+  # Configure the settings and mappings for the Elasticsearch index
+  settings number_of_shards: 1 do
+    mapping do
+      indexes :text, analyzer: 'snowball'
+    end
+  end
+
+  # Customize the serialization logic
+  def serialize(document)
+    super.merge(my_special_key: 'my_special_stuff')
+  end
+
+  # Customize the de-serialization logic
+  def deserialize(document)
+    puts "# ***** CUSTOM DESERIALIZE LOGIC KICKING IN... *****"
+    super
+  end
+end
+```
+
+The custom Elasticsearch client will be used now, with a custom index and type names,
+as well as the custom serialization and de-serialization logic.
+
+We can create the index with the desired settings and mappings:
+
+```ruby
+repository.create_index! force: true
+# PUT http://localhost:9200/my_notes
+# > {"settings":{"number_of_shards":1},"mappings":{ ... {"text":{"analyzer":"snowball","type":"string"}}}}}
+```
+
+Save the document with extra properties added by the `serialize` method:
+
+```ruby
+repository.save(note)
+# PUT http://localhost:9200/my_notes/my_note/1
+# > {"id":1,"text":"Test","my_special_key":"my_special_stuff"}
+{"_index"=>"my_notes", "_type"=>"my_note", "_id"=>"1", "_version"=>4, ... }
+```
+
+And `deserialize` it:
+
+```ruby
+repository.find(1)
+# ***** CUSTOM DESERIALIZE LOGIC KICKING IN... *****
+<Note:0x007f9bd782b7a0 @attributes={... "my_special_key"=>"my_special_stuff"}>
+```
+
+#### A Custom Class
+
+In most cases, though, you'll want to use a custom class for the repository, so let's do that:
+
+```ruby
+require 'base64'
+
+class NoteRepository
+  include Elasticsearch::Persistence::Repository
+
+  def initialize(options={})
+    index  options[:index] || 'notes'
+    client Elasticsearch::Client.new url: options[:url], log: options[:log]
+  end
+
+  klass Note
+
+  settings number_of_shards: 1 do
+    mapping do
+      indexes :text,  analyzer: 'snowball'
+      # Do not index images
+      indexes :image, index: 'no'
+    end
+  end
+
+  # Base64 encode the "image" field in the document
+  #
+  def serialize(document)
+    hash = document.to_hash.clone
+    hash['image'] = Base64.encode64(hash['image']) if hash['image']
+    hash.to_hash
+  end
+
+  # Base64 decode the "image" field in the document
+  #
+  def deserialize(document)
+    hash = document['_source']
+    hash['image'] = Base64.decode64(hash['image']) if hash['image']
+    klass.new hash
+  end
+end
+```
+
+Include the `Elasticsearch::Persistence::Repository` module to add the repository methods into the class.
+
+You can customize the repository in the familiar way, by calling the DSL-like methods.
+
+You can implement a custom initializer for your repository, add complex logic in its
+class and instance methods -- in general, have all the freedom of a standard Ruby class.
+
+```ruby
+repository = NoteRepository.new url: 'http://localhost:9200', log: true
+
+# Configure the repository instance
+repository.index = 'notes_development'
+repository.client.transport.logger.formatter = proc { |s, d, p, m| "\e[2m# #{m}\n\e[0m" }
+
+repository.create_index! force: true
+
+note = Note.new 'id' => 1, 'text' => 'Document with image', 'image' => '... BINARY DATA ...'
+
+repository.save(note)
+# PUT http://localhost:9200/notes_development/note/1
+# > {"id":1,"text":"Document with image","image":"Li4uIEJJTkFSWSBEQVRBIC4uLg==\n"}
+puts repository.find(1).attributes['image']
+# GET http://localhost:9200/notes_development/note/1
+# < {... "_source" : { ... "image":"Li4uIEJJTkFSWSBEQVRBIC4uLg==\n"}}
+# => ... BINARY DATA ...
+```
+
+#### Methods Provided by the Repository
+
+##### Client
+
+The repository uses the standard Elasticsearch [client](https://github.com/elasticsearch/elasticsearch-ruby#usage),
+which is accessible with the `client` getter and setter methods:
+
+```ruby
+repository.client = Elasticsearch::Client.new url: 'http://search.server.org'
+repository.client.transport.logger = Logger.new(STDERR)
+```
+
+##### Naming
+
+The `index` method specifies the Elasticsearch index to use for storage, lookup and search
+(when not set, the value is inferred from the repository class name):
+
+```ruby
+repository.index = 'notes_development'
+```
+
+The `type` method specifies the Elasticsearch document type to use for storage, lookup and search
+(when not set, the value is inferred from the document class name, or `_all` is used):
+
+```ruby
+repository.type = 'my_note'
+```
+
+The `klass` method specifies the Ruby class name to use when initializing objects from
+documents retrieved from the repository (when not set, the value is inferred from the
+document `_type` as fetched from Elasticsearch):
+
+```ruby
+repository.klass = MyNote
+```
+
+##### Index Configuration
+
+The `settings` and `mappings` methods, provided by the
+[`elasticsearch-model`](http://rubydoc.info/gems/elasticsearch-model/Elasticsearch/Model/Indexing/ClassMethods)
+gem, allow to configure the index properties:
+
+```ruby
+repository.settings number_of_shards: 1
+repository.settings.to_hash
+# => {:number_of_shards=>1}
+
+repository.mappings { indexes :title, analyzer: 'snowball' }
+repository.mappings.to_hash
+# => { :note => {:properties=> ... }}
+```
+
+The convenience methods `create_index!`, `delete_index!` and `refresh_index!` allow you to manage the index lifecycle.
+
+##### Serialization
+
+The `serialize` and `deserialize` methods allow you to customize the serialization of the document when passing it
+to the storage, and the initialization procedure when loading it from the storage:
+
+```ruby
+class NoteRepository
+  def serialize(document)
+    Hash[document.to_hash.map() { |k,v|  v.upcase! if k == :title; [k,v] }]
+  end
+  def deserialize(document)
+    MyNote.new ActiveSupport::HashWithIndifferentAccess.new(document['_source']).deep_symbolize_keys
+  end
+end
+```
+
+##### Storage
+
+The `save` method allows you to store a domain object in the repository:
+
+```ruby
+note = Note.new id: 1, title: 'Quick Brown Fox'
+repository.save(note)
+# => {"_index"=>"notes_development", "_type"=>"my_note", "_id"=>"1", "_version"=>1, "created"=>true}
+```
+
+The `update` method allows you to perform a partial update of a document in the repository.
+Use either a partial document:
+
+```ruby
+repository.update id: 1, title: 'UPDATED',  tags: []
+# => {"_index"=>"notes_development", "_type"=>"note", "_id"=>"1", "_version"=>2}
+```
+
+Or a script (optionally with parameters):
+
+```ruby
+repository.update 1, script: 'if (!ctx._source.tags.contains(t)) { ctx._source.tags += t }', params: { t: 'foo' }
+# => {"_index"=>"notes_development", "_type"=>"note", "_id"=>"1", "_version"=>3}
+```
+
+
+The `delete` method allows to remove objects from the repository (pass either the object itself or its ID):
+
+```ruby
+repository.delete(note)
+repository.delete(1)
+```
+
+##### Finding
+
+The `find` method allows to find one or many documents in the storage and returns them as deserialized Ruby objects:
+
+```ruby
+repository.save Note.new(id: 2, title: 'Fast White Dog')
+
+note = repository.find(1)
+# => <MyNote ... QUICK BROWN FOX>
+
+notes = repository.find(1, 2)
+# => [<MyNote... QUICK BROWN FOX>, <MyNote ... FAST WHITE DOG>]
+```
+
+When the document with a specific ID isn't found, a `nil` is returned instead of the deserialized object:
+
+```ruby
+notes = repository.find(1, 3, 2)
+# => [<MyNote ...>, nil, <MyNote ...>]
+```
+
+Handle the missing objects in the application code, or call `compact` on the result.
+
+##### Search
+
+The `search` method to retrieve objects from the repository by a query string or definition in the Elasticsearch DSL:
+
+```ruby
+repository.search('fox or dog').to_a
+# GET http://localhost:9200/notes_development/my_note/_search?q=fox
+# => [<MyNote ... FOX ...>, <MyNote ... DOG ...>]
+
+repository.search(query: { match: { title: 'fox dog' } }).to_a
+# GET http://localhost:9200/notes_development/my_note/_search
+# > {"query":{"match":{"title":"fox dog"}}}
+# => [<MyNote ... FOX ...>, <MyNote ... DOG ...>]
+```
+
+The returned object is an instance of the `Elasticsearch::Persistence::Repository::Response::Results` class,
+which provides access to the results, the full returned response and hits.
+
+```ruby
+results = repository.search(query: { match: { title: 'fox dog' } })
+
+# Iterate over the objects
+#
+results.each do |note|
+  puts "* #{note.attributes[:title]}"
+end
+# * QUICK BROWN FOX
+# * FAST WHITE DOG
+
+# Iterate over the objects and hits
+#
+results.each_with_hit do |note, hit|
+  puts "* #{note.attributes[:title]}, score: #{hit._score}"
+end
+# * QUICK BROWN FOX, score: 0.29930896
+# * FAST WHITE DOG, score: 0.29930896
+
+# Get total results
+#
+results.total
+# => 2
+
+# Access the raw response as a Hashie::Mash instance
+results.response._shards.failed
+# => 0
+```
+
+#### Example Application
+
+An example Sinatra application is available in
+[`examples/sinatra/application.rb`](examples/sinatra/application.rb),
+and demonstrates a rich set of features of the repository.
+
+
+### The ActiveRecord Pattern
+
+[_Work in progress_](https://github.com/elasticsearch/elasticsearch-rails/pull/91).
+The ActiveRecord [pattern](http://www.martinfowler.com/eaaCatalog/activeRecord.html) will work
+in a very similar way as `Tire::Model::Persistence`, allowing a drop-in replacement of
+an Elasticsearch-backed model in Ruby on Rails applications.
+
+## License
+
+This software is licensed under the Apache 2 license, quoted below.
+
+    Copyright (c) 2014 Elasticsearch <http://www.elasticsearch.org>
+
+    Licensed under the Apache License, Version 2.0 (the "License");
+    you may not use this file except in compliance with the License.
+    You may obtain a copy of the License at
 
-## Contributing
+       http://www.apache.org/licenses/LICENSE-2.0
 
-1. Fork it
-2. Create your feature branch (`git checkout -b my-new-feature`)
-3. Commit your changes (`git commit -am 'Add some feature'`)
-4. Push to the branch (`git push origin my-new-feature`)
-5. Create new Pull Request
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.