elastomer-client 0.3.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 6cd17dd15348e159466cead916479b5bcf68b80b
+  data.tar.gz: b2f0b0e2455ad13ed6a3a1670cf20a58e6239501
+SHA512:
+  metadata.gz: c9c2a2798145fa90c3291459de33dcdf01c1ad806b64e19b719fb6f5679c3b4b165583cb3ee4101a1f3e17879d15da93c350e88703256c1fab9755df80eda187
+  data.tar.gz: ce53c9838c420f6932d7792a088bdcd6ea7d3735e12f3df51d52459cc849b934cc91ff0703317574ad01ab75d0ca91ad0ef839d55cf7988a7417b499e52fcaed

data/.gitignore

@@ -0,0 +1,8 @@
+/bin
+/vendor/gems
+/.bundle
+/.rbenv-version
+/vendor/cache/*.gem
+/coverage
+Gemfile.lock
+*.gem

data/.ruby-version

@@ -0,0 +1 @@
+2.1.0-github

data/CHANGELOG.md

@@ -0,0 +1,4 @@
+## 0.3.1 (2014-06-24)
+- First rubygems release
+- Make `update_aliases` more flexible
+- Add `Client#semantic_version` for ES version comparisons

data/Gemfile

@@ -0,0 +1,5 @@
+source 'https://rubygems.org'
+
+gemspec
+
+gem 'simplecov', :require => false, :group => :development, :platform => :ruby_19

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2013 GitHub Inc.
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,108 @@
+# Elastomer
+
+Making a stupid simple ElasticSearch client so your project can be smarter!
+
+## Getting Started
+
+Use [Boxen](https://setup.githubapp.com) to get your machine set up. Then:
+
+```
+$ git clone https://github.com/github/elastomer-client.git
+$ cd elastomer-client
+$ script/bootstrap
+$ script/testsuite
+```
+
+## Client
+
+The client provides a one-to-one mapping to the ElasticSearch [API
+endpoints](http://www.elasticsearch.org/guide/reference/api/). The API is
+decomposed into logical sections and accessed according to what you are trying
+to accomplish. Each logical section is represented as a [client
+class](lib/elastomer/client) and a top-level accessor is provided for each.
+
+#### Cluster
+
+API endpoints dealing with cluster level information and settings are found in
+the [Cluster](lib/elastomer/client/cluster.rb) class.
+
+```ruby
+require 'elastomer/client'
+client = Elastomer::Client.new
+
+# the current health summary
+client.cluster.health
+
+# detailed cluster state information
+client.cluster.state
+
+# the list of all index templates
+client.cluster.templates
+```
+
+#### Index
+
+The methods in the [Index](lib/elastomer/client/index.rb) class deal with the
+management of indexes in the cluster. This includes setting up type mappings
+and adjusting settings. The actual indexing and search of documents are
+handled by the Docs class (discussed next).
+
+```ruby
+require 'elastomer/client'
+client = Elastomer::Client.new
+
+index = client.index('twitter')
+index.create(
+  :settings => { 'index.number_of_shards' => 3 },
+  :mappings => {
+    :tweet => {
+      :_source => { :enabled => true },
+      :_all    => { :enabled => false },
+      :properties => {
+        :author => { :type => 'string', :index => 'not_analyzed' },
+        :tweet  => { :type => 'string', :analyze => 'standard' }
+      }
+    }
+  }
+)
+
+index.exists?
+
+index.exists? :type => 'tweet'
+
+index.delete
+```
+
+#### Docs
+
+This decomposition is the most questionable, but it's a starting point. The
+[Docs](lib/elastomer/client/docs.rb) class handles the indexing and searching
+of documents. Each instance is scoped to an index and optionally a document
+type.
+
+```ruby
+require 'elastomer/client'
+client = Elastomer::Client.new
+
+docs = client.docs('twitter')
+
+docs.index({
+  :_id    => 1,
+  :_type  => 'tweet',
+  :author => '@pea53',
+  :tweet  => 'announcing Elastomer, the stupid simple ElasticSearch client'
+})
+
+docs.search({:query => {:match_all => {}}}, :search_type => 'count')
+```
+
+#### Performance
+
+By default Elastomer uses Net::HTTP (via Faraday) to communicate with
+ElasticSearch. You may find that Excon performs better for your use. To enable
+Excon, add it to your bundle and then change your Elastomer initialization
+thusly:
+
+```
+Elastomer::Client.new(url: YOUR_ES_URL, adapter: :excon)
+```

data/Rakefile

@@ -0,0 +1,9 @@
+require "bundler/gem_tasks"
+
+require 'rake/testtask'
+
+Rake::TestTask.new do |t|
+    t.pattern = "test/**/*_test.rb"
+end
+
+task :default => :test

data/docs/notifications.md

@@ -0,0 +1,71 @@
+# Notifications Support
+
+Requiring `elastomer/notifications` enables support for broadcasting
+elastomer events through ActiveSupport::Notifications.
+
+The event namespace is `request.client.elastomer`.
+
+## Sample event payload
+
+```
+:index  => "index-test",
+:type   => nil,
+:action => "docs.search",
+:context=> nil,
+:body   => "{\"query\":{\"match_all\":{}}}",
+:url    => #<URI::HTTP:0x007fb6f3e98b60 URL:http://localhost:19200/index-test/_search?search_type=count>,
+:method => :get,
+:status => 200}
+```
+
+## Valid actions
+- bulk
+- cluster.get_settings
+- cluster.health
+- cluster.reroute
+- cluster.state
+- cluster.update_settings
+- cluster.available
+- cluster.get_aliases
+- cluster.info
+- cluster.shutdown
+- cluster.update_aliases
+- docs.delete
+- docs.delete_by_query
+- docs.explain
+- docs.get
+- docs.index
+- docs.more_like_this
+- docs.multi_get
+- docs.search
+- docs.source
+- docs.update
+- docs.validate
+- index.analyze
+- index.clear_cache
+- index.close
+- index.create
+- index.delete
+- index.delete_mapping
+- index.exists
+- index.flush
+- index.get_aliases
+- index.get_settings
+- index.mapping
+- index.open
+- index.optimize
+- index.refresh
+- index.segments
+- index.snapshot
+- index.stats
+- index.status
+- index.update_mapping
+- index.update_settings
+- nodes.hot_threads
+- nodes.info
+- nodes.shutdown
+- nodes.stats
+- search.scan
+- search.scroll
+- template.create
+- template.get

data/elastomer-client.gemspec

@@ -0,0 +1,30 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'elastomer/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "elastomer-client"
+  spec.version       = Elastomer::VERSION
+  spec.authors       = ["Tim Pease", "Grant Rodgers"]
+  spec.email         = ["tim.pease@github.com", "grant.rodgers@github.com"]
+  spec.summary       = %q{A library for interacting with the GitHub Search infrastructure}
+  spec.description   = %q{Elastomer is a low level API client for the
+                          Elasticsearch HTTP interface.}
+  spec.homepage      = "https://github.com/github/elastomer-client"
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0")
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "addressable", "~> 2.3"
+  spec.add_dependency "faraday",     "~> 0.8"
+  spec.add_dependency "multi_json",  "~> 1.7"
+  spec.add_dependency "semantic",    "~> 1.3"
+
+  spec.add_development_dependency "bundler", "~> 1.5"
+  spec.add_development_dependency "minitest","~> 4.7"
+  spec.add_development_dependency "rake"
+end

data/lib/elastomer/client.rb

@@ -0,0 +1,307 @@
+require 'addressable/template'
+require 'faraday'
+require 'multi_json'
+require 'semantic'
+
+require 'elastomer/version'
+
+module Elastomer
+
+  class Client
+
+    # Create a new client that can be used to make HTTP requests to the
+    # ElasticSearch server.
+    #
+    # opts - The options Hash
+    #   :host - the host as a String
+    #   :port - the port number of the server
+    #   :url  - the URL as a String (overrides :host and :port)
+    #   :read_timeout - the timeout in seconds when reading from an HTTP connection
+    #   :open_timeout - the timeout in seconds when opening an HTTP connection
+    #   :adapter      - the Faraday adapter to use (defaults to :excon)
+    #   :opaque_id    - set to `true` to use the 'X-Opaque-Id' request header
+    #
+    def initialize( opts = {} )
+      host = opts.fetch :host, 'localhost'
+      port = opts.fetch :port, 9200
+      @url = opts.fetch :url,  "http://#{host}:#{port}"
+
+      uri = Addressable::URI.parse @url
+      @host = uri.host
+      @port = uri.port
+
+      @read_timeout = opts.fetch :read_timeout, 5
+      @open_timeout = opts.fetch :open_timeout, 2
+      @adapter      = opts.fetch :adapter, Faraday.default_adapter
+      @opaque_id    = opts.fetch :opaque_id, false
+    end
+
+    attr_reader :host, :port, :url
+    attr_reader :read_timeout, :open_timeout
+
+    # Returns true if the server is available; returns false otherwise.
+    def available?
+      response = head '/', :action => 'cluster.available'
+      response.success?
+    rescue StandardError
+      false
+    end
+
+    # Returns the version String of the attached ElasticSearch instance.
+    def version
+      @version ||= info['version']['number']
+    end
+
+    # Returns a Semantic::Version for the attached ElasticSearch instance.
+    # See https://rubygems.org/gems/semantic
+    def semantic_version
+      Semantic::Version.new(version)
+    end
+
+    # Returns the information Hash from the attached ElasticSearch instance.
+    def info
+      response = get '/', :action => 'cluster.info'
+      response.body
+    end
+
+    # Internal: Provides access to the Faraday::Connection used by this client
+    # for all requests to the server.
+    #
+    # Returns a Faraday::Connection
+    def connection
+      @connection ||= Faraday.new(url) do |conn|
+        conn.request  :encode_json
+        conn.response :parse_json
+        conn.request  :opaque_id if @opaque_id
+
+        Array === @adapter ?
+          conn.adapter(*@adapter) :
+          conn.adapter(@adapter)
+
+        conn.options[:timeout]      = read_timeout
+        conn.options[:open_timeout] = open_timeout
+      end
+    end
+
+    # Internal: Sends an HTTP HEAD request to the server.
+    #
+    # path   - The path as a String
+    # params - Parameters Hash
+    #
+    # Returns a Faraday::Response
+    def head( path, params = {} )
+      request :head, path, params
+    end
+
+    # Internal: Sends an HTTP GET request to the server.
+    #
+    # path   - The path as a String
+    # params - Parameters Hash
+    #
+    # Returns a Faraday::Response
+    # Raises an Elastomer::Client::Error on 4XX and 5XX responses
+    def get( path, params = {} )
+      request :get, path, params
+    end
+
+    # Internal: Sends an HTTP PUT request to the server.
+    #
+    # path   - The path as a String
+    # params - Parameters Hash
+    #
+    # Returns a Faraday::Response
+    # Raises an Elastomer::Client::Error on 4XX and 5XX responses
+    def put( path, params = {} )
+      request :put, path, params
+    end
+
+    # Internal: Sends an HTTP POST request to the server.
+    #
+    # path   - The path as a String
+    # params - Parameters Hash
+    #
+    # Returns a Faraday::Response
+    # Raises an Elastomer::Client::Error on 4XX and 5XX responses
+    def post( path, params = {} )
+      request :post, path, params
+    end
+
+    # Internal: Sends an HTTP DELETE request to the server.
+    #
+    # path   - The path as a String
+    # params - Parameters Hash
+    #
+    # Returns a Faraday::Response
+    # Raises an Elastomer::Client::Error on 4XX and 5XX responses
+    def delete( path, params = {} )
+      request :delete, path, params
+    end
+
+    # Internal: Sends an HTTP request to the server. If the `params` Hash
+    # contains a :body key, it will be deleted from the Hash and the value
+    # will be used as the body of the request.
+    #
+    # method - The HTTP method to send [:head, :get, :put, :post, :delete]
+    # path   - The path as a String
+    # params - Parameters Hash
+    #   :body         - Will be used as the request body
+    #   :read_timeout - Optional read timeout (in seconds) for the request
+    #
+    # Returns a Faraday::Response
+    # Raises an Elastomer::Client::Error on 4XX and 5XX responses
+    def request( method, path, params )
+      body = params.delete :body
+      body = MultiJson.dump body if Hash === body
+
+      read_timeout = params.delete :read_timeout
+
+      path = expand_path path, params
+
+      response = instrument(path, body, params) do
+        case method
+        when :head
+          connection.head(path) { |req| req.options[:timeout] = read_timeout if read_timeout }
+
+        when :get
+          connection.get(path) { |req|
+            req.body = body if body
+            req.options[:timeout] = read_timeout if read_timeout
+          }
+
+        when :put
+          connection.put(path, body) { |req| req.options[:timeout] = read_timeout if read_timeout }
+
+        when :post
+          connection.post(path, body) { |req| req.options[:timeout] = read_timeout if read_timeout }
+
+        when :delete
+          connection.delete(path) { |req|
+            req.body = body if body
+            req.options[:timeout] = read_timeout if read_timeout
+          }
+
+        else
+          raise ArgumentError, "unknown HTTP request method: #{method.inspect}"
+        end
+      end
+
+      handle_errors response
+
+    rescue Faraday::Error::TimeoutError => boom
+      raise ::Elastomer::Client::TimeoutError.new(boom, path)
+
+    # ensure
+    #   # FIXME: this is here until we get a real logger in place
+    #   STDERR.puts "[#{response.status.inspect}] curl -X#{method.to_s.upcase} '#{url}#{path}'" unless response.nil?
+    end
+
+    # Internal: Apply path expansions to the `path` and append query
+    # parameters to the `path`. We are using an Addressable::Template to
+    # replace '{expansion}' fields found in the path with the values extracted
+    # from the `params` Hash. Any remaining elements in the `params` hash are
+    # treated as query parameters and appended to the end of the path.
+    #
+    # path   - The path as a String
+    # params - Parameters Hash
+    #
+    # Examples
+    #
+    #   expand_path('/foo{/bar}', {:bar => 'hello', :q => 'what', :p => 2})
+    #   #=> '/foo/hello?q=what&p=2'
+    #
+    #   expand_path('/foo{/bar}{/baz}', {:baz => 'no bar'}
+    #   #=> '/foo/no%20bar'
+    #
+    # Returns an Addressable::Uri
+    def expand_path( path, params )
+      template = Addressable::Template.new path
+
+      expansions = {}
+      query_values = params.dup
+      query_values.delete :action
+      query_values.delete :context
+
+      template.keys.map(&:to_sym).each do |key|
+        value = query_values.delete key
+        value = assert_param_presence(value, key) unless path =~ /{\/#{key}}/ && value.nil?
+        expansions[key] = value
+      end
+
+      uri = template.expand(expansions)
+      uri.query_values = query_values unless query_values.empty?
+      uri.to_s
+    end
+
+    # Internal: A noop method that simply yields to the block. This method
+    # will be replaced when the 'elastomer/notifications' module is included.
+    #
+    # path   - The full request path as a String
+    # body   - The request body as a String or `nil`
+    # params - The request params Hash
+    # block  - The block that will be instrumented
+    #
+    # Returns the response from the block
+    def instrument( path, body, params )
+      yield
+    end
+
+    # Internal: Inspect the Faraday::Response and raise an error if the status
+    # is in the 5XX range or if the response body contains an 'error' field.
+    # In the latter case, the value of the 'error' field becomes our exception
+    # message. In the absence of an 'error' field the response body is used
+    # as the exception message.
+    #
+    # The raised exception will contain the response object.
+    #
+    # response - The Faraday::Response object.
+    #
+    # Returns the response.
+    # Raises an Elastomer::Client::Error on 500 responses or responses
+    # containing and 'error' field.
+    def handle_errors( response )
+      raise Error, response if response.status >= 500
+      raise Error, response if Hash === response.body && response.body['error']
+
+      response
+    end
+
+    # Internal: Ensure that the parameter has a valid value. Things like `nil`
+    # and empty strings are right out. This method also performs a little
+    # formating on the parameter:
+    #
+    # * leading and trailing whitespace is removed
+    # * arrays are flattend
+    # * and then joined into a String
+    # * numerics are converted to their string equivalents
+    #
+    # param - The param Object to validate
+    # name  - Optional param name as a String (used in exception messages)
+    #
+    # Returns the validated param as a String.
+    # Raises an ArgumentError if the param is not valid.
+    def assert_param_presence( param, name = 'input value' )
+      case param
+      when String, Numeric
+        param = param.to_s.strip
+        raise ArgumentError, "#{name} cannot be blank: #{param.inspect}" if param =~ /\A\s*\z/
+        param
+
+      when Array
+        param.flatten.map { |item| assert_param_presence(item, name) }.join(',')
+
+      when nil
+        raise ArgumentError, "#{name} cannot be nil"
+
+      else
+        raise ArgumentError, "#{name} is invalid: #{param.inspect}"
+      end
+    end
+
+  end  # Client
+end  # Elastomer
+
+# require all files in the `client` sub-directory
+Dir.glob(File.expand_path('../client/*.rb', __FILE__)).each { |fn| require fn }
+
+# require all files in the `middleware` sub-directory
+Dir.glob(File.expand_path('../middleware/*.rb', __FILE__)).each { |fn| require fn }