elasticshell 0.0.2 → 0.0.4

data/README.rdoc

@@ -26,12 +26,12 @@ How about
 
 == Installation
 
-Installing Elasticshell should be as simple as installing the gem:
+Install the gem:
 
   $ sudo gem install elasticshell
 
-This should install a binary program 'es' that you can run from the
-command line to start Elasticshell.  Try
+which installs a program 'es' that you can run from the command line
+to start Elasticshell.  Try
 
   $ es --help
 
@@ -53,57 +53,115 @@ Within Elasticshell, there are three variables whose values affect
 behavior.  These variables are reflected in the default prompt, for
 example:
 
-  GET /my_index/my_type >
+  GET /my_index/my_type$ 
 
 This prompt tells us three things:
 
 1. The default HTTP verb we're using for requests is +GET+.
 
-2. The default API "scope" we're in is <tt>/my_index/my_type</tt>.
+2. The default API "scope" we're in is <tt>/my_index/my_type</tt>.  If the shell is connected to an Elasticsearch server and the scope exists, it will be colored green.  Otherwise it's yellow.
 
-3. Elasticshell will print raw responses from the database -- this is the <tt>></tt> at the end of the prompt.  If we were in pretty-print mode, this would become a <tt>$</tt>.
+3. Elasticshell will print raw responses from the database -- this is the <tt>$</tt> at the end of the prompt.  If we were in pretty-print mode, this would become a <tt>$$</tt>.
 
-=== Changing Scope
+=== Connecting to the Database
+
+Elasticshell will try to connect to the Elasticsearch hosts passed
+with the <tt>--servers</tt> option during startup.  At any other time,
+you can connect to servers by issuing the +connect+ command:
+
+  GET /$ connect http://192.168.1.10:9200 http://192.168.1.11:9200 http://192.168.1.12:9200 http://192.168.1.13:9200
+
+=== Understanding Scope
+
+Scopes are defined by the Elasticsearch REST API.  Some scopes like
+<tt>/_cluster</tt> or <tt>/nodes</tt> are static and present for all
+Elasticsearch clusters.  Other scopes like <tt>/my_index/my_type</tt>
+depend upon the particular cluster.
 
 Use the +cd+ built-in to move between scopes:
 
-  GET /my_index/my_type > cd /other_index/other_type
-  GET /other_index/other_type > cd ..
-  GET /other_index > cd
-  GET / >
+  GET /$ cd /blog/comments
+  GET /blog/comments$ cd ..
+  GET /blog$ cd /blog/entries
+  GET /blog/entries$ cd
+  GET /$
+
+==== Different kinds of scopes  
+
+The +ls+ command will show the contents of a given scope:
+
+  GET /$ ls
+  blog _cluster _nodes _status
+
+but the +ll+ command gives more output:
+
+  GET /$ ll
+  i      1/1/0      5  3.3kb blog
+  s                          _cluster
+  s                          _nodes
+  -                          _status
 
-Tab-complete within a scope after typing +cd+ to see what other scopes
-live under this one.
+Here you see that +blog+
+
+- is an index (the +i+ in the first column)
+- has 1 total shard, 1 successful shard, and 0 failed shards
+- has 5 documents
+- occupies 3.3kb of space on disk
+
+And, because of the +s+ in the first-column, +_cluster+ and +_nodes+
+are scopes -- you can +cd+ into them.
+
+Finally, +_status+ is a request, you can't +cd+ into it, but you can
+issue it.
+
+If you were to first +cd+ into the index and run +ll+ you'll see
+different output:
+
+  GET /$ cd /blog/ 
+  GET /blog$ ll
+  m                          comments
+  m                          entries
+  -                          _aliases
+  -                          _search
+  -                          _stats
+  -                          _status
+
++_aliases+ and so on are just more requests you can make but
++comments+ and +entries+ are mappings (they have an +m+ in the
+first-column).
 
 === Changing HTTP Verb
 
 You can change Elasticsearch's default HTTP verb by giving it one.
 Here's the same thing in two steps:
 
-  GET / > PUT
-  PUT / > /my_new_index
+  GET /$ PUT
+  PUT /$ /my_new_index
+
+You can also do this on a per-request basis
 
-Non-ambiguous shortcuts for HTTP verbs will also work, e.g. - +pu+ in
-this case for +PUT+.
+  GET /$ PUT /my_new_index
+  GET /$
 
 === Changing Prettiness
 
 Typing +pretty+ at any time will toggle Elasticsearch's
 pretty-printing format on or off.
 
-  GET / > pretty
-  GET / $
+  GET /$ pretty
+  GET /$$
 
-The <tt>$</tt>-sign means it's pretty...
+The extra <tt>$</tt>-sign means it's pretty...
 
-=== Running Commands
+== Making Requests
 
-There are a lot of different ways of telling Elasticsearch what you
-want done.
+Scopes are fine for organizing the API but to get anything done you'll
+have to send a request.
 
-==== Named commands
+=== Named requests
 
-Each scope has different commands, as per the {Elasticsearch API
+Each scope has different fixed, named requests, as per the
+{Elasticsearch API
 documentation}[http://www.elasticsearch.org/guide/reference/api/].
 Within a scope, tab-complete on the first word to see a list of
 possible commands.  Hit enter after a command to see output from
@@ -111,28 +169,32 @@ Elasticsearch.
 
 Here's a command to get the status for the cluster:
 
-  GET / > _status
+  GET /$ _status
 
 Here's a command to get the health of the cluster:
 
-  GET / > cd _cluster
-  GET /_cluster > health
+  GET /$ cd _cluster
+  GET /_cluster$ health
+
+which you could also have issued like this
+
+  GET /$ _cluster/health
 
-==== Commands with query strings  
+=== Using query strings
 
 Commands will also accept a query string, as in this example of a
 search through +my_index+:
 
-  GET /my_index > _search?q=foo+AND+bar
+  GET /my_index$ _search?q=foo+AND+bar
   
-==== Commands with query bodies  
+=== Using a body
 
-In this example the query <tt>foo AND bar</tt> was passed via the
+In the above example the query <tt>foo AND bar</tt> was passed via the
 query string part of a URL.  Passing a more complex query requires we
 put the query in the body of the request.  If you're willing to forego
 using spaces you can do this right on the same line:
 
-  GET /my_index > _search {"query":{"query_string":{"query":"foo"}}}
+  GET /my_index$ _search {"query":{"query_string":{"query":"foo"}}}
 
 But if you want more expressiveness you can either name a file (with
 tab-completion) that contains the body you want:
@@ -148,12 +210,12 @@ tab-completion) that contains the body you want:
 	  
 followed by
 
-  GET /my_index > _search /tmp/query.json
+  GET /my_index$ _search /tmp/query.json
 
 Or you can do +cat+-style, pasting the query into the shell, by using
 the <tt>-</tt> character:
 
-  GET /my_index > _search -
+  GET /my_index$ _search -
   {
     "query": {
       "query_string: {
@@ -165,38 +227,40 @@ the <tt>-</tt> character:
 Don't forget to use <tt>Ctrl-D</tt> to send an +EOF+ to flush the
 input of the query.
 
-==== Arbitrary commands
+=== Redirecting output
 
-You can send an arbitrary HTTP request to Elasticsearch, just spell
-the command with a leading slash:
+You can redirect the output from a request in a variety of ways.
 
-  GET / > /my_index/_search?q=foo+AND+bar
+Most simply is to redirect to a file:
 
-You can specify a different HTTP verb by prefixing it before the path
-you send the request to.  Here's how to create an index using a +PUT+
-request:
+ GET /my_index$ _search /tmp/query_with_lots_of_output.json > /data/output_file.json
 
-  GET / > PUT /my_new_index
+Or you can try sending it to Ruby itself:
 
-You can also change Elasticsearch's default HTTP verb by giving it
-one.  Here's the same thing in two steps:
+  GET /my_index$ _search /tmp/query_that_needs_filtering.json | puts response["hits"]["hits"].first["body"]
 
-  GET / > PUT
-  PUT / > /my_new_index
+Everything to the right of the +|+ is executing within a Ruby process
+that has the +response+ and +request+ variables in scope.  You can be
+even more free by just piping without any Ruby code, which will leave
+you in a Ruby (Pry) shell with the same binding.
 
-Non-ambiguous shortcuts for HTTP verbs will also work, e.g. - +pu+ in
-this case for +PUT+.
+GET /my_index$ _search /tmp/query_that_needs_interaction.json | 
+>> response
+=> {"took"=>1, "timed_out"=>false, ... }
+>> request
+=> {:verb=>"GET", :path=>"/_search", :query_options=>{}, :body=>""}
 
-=== Running just a single command
+=== Requests from the command line
 
 Instead of running Elasticshell interactively, you can exit after
-running only a single command by using the <tt>--only</tt> option on
-startup.  For example,
+running only a single command by feeding the request path directly to
+the +es+ script.  For example
 
-  $ es --only /_cluster/health
+  $ es /_cluster/health
+  $ es --scope=/_cluster health
+  $ es --verb=GET /_cluster/health
 
-will output the cluster health and exit immediately.  This can be
-combined with the <tt>--pretty</tt> option for readability.
+all work like you think they do.
 
 The <tt>--only</tt> option can also be passed a <tt>.</tt>-separated
 hierarchical list of keys to slice into the resulting object.  This is

data/VERSION

@@ -1 +1 @@
-0.0.2
+0.0.4

data/bin/es

@@ -1,6 +1,4 @@
 #!/usr/bin/env ruby
 
-$: << File.expand_path('../../lib', __FILE__) unless $:.include?($: << File.expand_path('../../lib', __FILE__))
 require 'elasticshell'
-
 Elasticshell.start

data/lib/elasticshell.rb

@@ -2,58 +2,63 @@ require 'rubygems'
 require 'json'
 require 'configliere'
 
-require 'elasticshell/shell'
-require 'elasticshell/scopes'
-require 'elasticshell/client'
+require 'elasticshell/utils'
 
 Settings.use(:commandline)
 
-Settings.define(:servers, :description => "A comma-separated list of Elasticsearch servers to connect to.", :type => Array, :default => Elasticshell::Client::DEFAULT_SERVERS)
+Settings.define(:servers, :description => "A comma-separated list of Elasticsearch servers to connect to.", :type => Array, :default => 'http://localhost:9200')
 Settings.define(:only,    :description => "A dot-separated hierarchical key to extract from the output scope.")
 Settings.define(:pretty,  :description => "Pretty-print all output. ", :default => false, :type => :boolean)
 Settings.define(:verb,    :description => "Set the default HTTP verb. ", :default => "GET")
 Settings.define(:version, :description => "Print Elasticshell version and exit. ", :default => false, :type => :boolean)
+Settings.define(:scope,   :description => "The default scope to start the shell in.", :default => "/")
 Settings.description = <<-DESC
 Elasticshell is a command-line shell for interacting with an
 Elasticsearch database.  It has the following start-up options.
 DESC
 
 def Settings.usage
-  "usage: #{File.basename($0)} [OPTIONS] [SCOPE]"
+  "usage: #{File.basename($0)} [OPTIONS] [REQUEST]"
 end
 Settings.resolve!
 
 module Elasticshell
 
+  autoload :Client,   'elasticshell/client'
+  autoload :Shell,    'elasticshell/shell'
+  autoload :Scope,    'elasticshell/scopes'
+  autoload :Scopes,   'elasticshell/scopes'
+  autoload :Command,  'elasticshell/command'
+  autoload :Commands, 'elasticshell/command'
+
   def self.version
     @version ||= begin
-      File.read(File.expand_path('../../VERSION', __FILE__)).chomp
-    rescue => e
-      'unknown'
-    end
+                   File.read(File.expand_path('../../VERSION', __FILE__)).chomp
+                 rescue => e
+                   'unknown'
+                 end
   end
   
   def self.start *args
-    if Settings[:version]
-      puts version
-      exit()
-    end
-    
-    es = Shell.new(Settings)
-    if Settings[:only]
-      if Settings.rest.length == 0
-        $stderr.puts "Starting with the --only option requires the first argument to name an API path (like `/_cluster/health')"
+    begin
+      case
+      when Settings[:version]
+        puts version
+        exit()
+      when Settings[:only] && Settings.rest.empty?
+        raise ArgumentError.new("Starting with the --only option requires a request argument (like `/_cluster/health')")
         exit(1)
-      else
+      when (! Settings.rest.empty?)
+        es = Shell.new(Settings.merge(:log_requests => false))
+        es.connect
         es.eval_line(Settings.rest.first)
         exit()
+      else
+        Shell.new(Settings).run
       end
-    else
-      if Settings.rest.length > 0
-        es.scope = Scopes.from_path(Settings.rest.first, :client => es.client)
-      end
-      es.run
+    rescue Elasticshell::Error => e
+      $stderr.puts e.message
+      exit(2)
     end
   end
-  
 end

data/lib/elasticshell/client.rb

@@ -1,23 +1,36 @@
 require 'rubberband'
-require 'elasticshell/error'
-require 'elasticshell/log'
 
 module Elasticshell
 
   class Client
 
-    DEFAULT_SERVERS = ['http://localhost:9200']
-
     def initialize options={}
-      @client ||= ::ElasticSearch::Client.new(options[:servers] || DEFAULT_SERVERS)
+      @options = options
+    end
+
+    def connect options={}
+      servers = (options.merge(@options)[:servers] || Settings[:servers].to_s.split(","))
+      begin
+        @client = ElasticSearch::Client.new(servers)
+      rescue ElasticSearch::ConnectionFailed => e
+        raise ClientError.new("Could not connect to Elasticsearch server(s) at #{servers.join(',')}")
+      end
+    end
+
+    def connected?
+      @client
     end
 
     def request verb, params={}, options={}, body=''
+      raise ClientError.new("Not connected to any Elasticsearch servers.") unless connected?
       safe        = options.delete(:safely)
       safe_return = options.delete(:return)
-      # log_request(verb, params, options)
+      
+      # Log by default
+      log_request(verb, params, options) unless options.delete(:log) == false
+      
       begin
-        @client.execute(:standard_request, verb, params, options, body)
+        perform_request(verb, params, options, body)
       rescue ElasticSearch::RequestError, ArgumentError => e
         if safe
           safe_return
@@ -27,13 +40,21 @@ module Elasticshell
       end
     end
 
+    def perform_request verb, params, options, body
+      @client.execute(:standard_request, verb.downcase.to_sym, params, options, body)
+    end
+
     def log_request verb, params, options={}
-      # FIXME digging way too deep into rubberband here...is it really
-      # necessary?
-      uri   = @client.instance_variable_get('@connection').send(:generate_uri, params)
-      query = @client.instance_variable_get('@connection').send(:generate_query_string, options)
-      path  = [uri, query].reject { |s| s.nil? || s.strip.empty? }.join("?")
-      Elasticshell.log("#{verb.to_s.upcase} #{path}")
+      begin
+        # FIXME digging way too deep into rubberband here...is it really
+        # necessary?
+        uri   = @client.instance_variable_get('@connection').send(:generate_uri, params)
+        query = @client.instance_variable_get('@connection').send(:generate_query_string, options)
+        path  = [uri, query].reject { |s| s.nil? || s.strip.empty? }.join("?").gsub(Regexp.new("^/+"), "/")
+        Elasticshell.log("#{verb.to_s.upcase} #{path}".strip)
+      rescue
+        Elasticshell.log("#{verb.to_s.upcase} #{params.inspect} #{options.inspect}".strip)
+      end
     end
 
     def safely verb, params={}, options={}, body=''

data/lib/elasticshell/command.rb

@@ -1,12 +1,7 @@
-require 'elasticshell/error'
-require 'uri'
-
 module Elasticshell
 
   class Command
 
-    HTTP_VERB_RE = "(?:G(?:ET?)?|PO(?:ST?)?|PUT?|D(?:E(?:L(?:E(?:TE?)?)?)?)?)"
-
     attr_accessor :shell, :input
 
     def initialize shell, input
@@ -14,178 +9,27 @@ module Elasticshell
       self.input = input
     end
 
-    def evaluate!
-      case
-      when setting_scope?          then set_scope!
-      when setting_http_verb?      then set_http_verb!
-      when making_explicit_req?    then make_explicit_req!
-      when pretty?                 then pretty!
-      when help?                   then help!
-      when ls?                     then ls!
-      when blank?                  then nil
-      when scope_command?          then run_scope_command!        
-      else
-        raise ArgumentError.new("Unknown command '#{input}' for scope '#{shell.scope.path}'.  Try typing 'help' for a list of available commands.")
-      end
-    end
-
-    def setting_scope?
-      input =~ /^cd/
-    end
-
-    def set_scope!
-      if input =~ /^cd$/
-        shell.scope = Scopes.global(:client => shell.client)
-        return
-      end
-      
-      return unless input =~ /^cd\s+(.+)$/
-      scope = $1
-      if scope =~ %r!^/!
-        shell.scope = Scopes.from_path(scope, :client => shell.client)
-      else
-        shell.scope = Scopes.from_path(File.expand_path(File.join(shell.scope.path, scope)), :client => shell.client)
-      end
-    end
-
-    def setting_http_verb?
-      input =~ Regexp.new("^" + HTTP_VERB_RE + "$", true)
-    end
-
-    def canonicalize_http_verb v
-      case v
-      when /^G/i  then "GET"
-      when /^PO/i then "POST"
-      when /^PU/i then "PUT"
-      when /^D/i  then "DELETE"
-      end
-    end
-
-    def set_http_verb!
-      shell.verb = canonicalize_http_verb(input)
-    end
-
-    def scope_command?
-      shell.scope.command?(input)
-    end
-
-    def run_scope_command!
-      shell.scope.execute(input, shell)
-    end
-
-    def blank?
-      input.empty?
-    end
-
-    def help?
-      input =~ /^help/i
-    end
-
-    def help!
-      shell.scope.refresh
-      shell.print <<HELP
-
-Globally available commands:
-
-  cd [PATH]
-    Change scope to the given path.  Current path is reflected in the
-    prompt (it's '#{shell.scope.path}' right now).
-
-    Ex:
-      GET / > cd /my_index
-      GET /my_index > cd /other_index/some_type
-      GET /other_index/some_type
-
-  [get|post|put|delete]
-    Set the default HTTP verb (can use a non-ambiguous shortcut like 'g'
-    for 'GET' or 'pu' for 'PUT').  Current default HTTP verb is '#{shell.verb}'.
-
-  ls
-    Show what indices or mappings are within the current scope.
-
-  help
-    Show contextual help.
-
-  [VERB] PATH
-    Send an HTTP request with the given VERB to the given PATH
-    (including query string if given).  If no verb is given, use the
-    default.
-
-    Ex: Simple search
-      GET / > /my_index/_search?q=query+string
-      {...}
-
-    Ex: Create an index
-      GET / > PUT /my_new_index
-      {...}
-
-      or
-
-      GET / > put
-      PUT / > /my_new_index
-      {...}
-
-#{shell.scope.help}
-HELP
+    def self.matches? input
+      raise NotImplementedError.new("Define the 'matches?' method on #{self.class}")
     end
 
-    def ls?
-      input =~ /^l(s|l|a)?$/i
+    def be_connected!
+      raise ClientError.new("Not connected to any Elasticsearch servers.") unless shell.connected?
     end
 
-    def ls!
-      shell.scope.refresh!
-      case
-      when input =~ /ll/
-        shell.print shell.scope.contents.join("\n")
-      else
-        shell.print shell.scope.contents.join(' ')
-      end
-    end
-
-    def pretty?
-      input =~ /pretty/i
-    end
-
-    def pretty!
-      if shell.pretty?
-        shell.not_pretty!
-      else
-        shell.pretty!
-      end
-    end
-
-    def making_explicit_req?
-      input =~ Regexp.new("^(" + HTTP_VERB_RE + "\s+)?/", true)
+    def evaluate!
+      raise NotImplementedError.new("Define the 'evaluate!' instance method in your subclass of #{self.class}")
     end
+  end
 
-    def make_explicit_req!
-      if input =~ Regexp.new("^(" + HTTP_VERB_RE + ")\s+(.+)$", true)
-        verb, path_and_query = canonicalize_http_verb($1), $2
-      else
-        verb, path_and_query = shell.verb, input
-      end
-      path, query = path_and_query.split('?')
-      
-      params = {}
-      keys  = [:index, :type, :id, :op]
-      parts = path.gsub(%r!^/!,'').gsub(%r!/$!,'').split('/')
-      while parts.size > 0
-        part = parts.shift
-        key  = (keys.shift or ArgumentError.new("The input '#{path}' has too many path components."))
-        params[key] = part
-      end
-
-      options = {}
-      URI.decode_www_form(query || '').each do |key, value|
-        options[key] = value
+  module Commands
+    PRIORITY = [].tap do |priority|
+      %w[cd pwd df connect help ls pretty set_verb blank request unknown].each do |command_name|
+        klass_name = command_name.split("_").map(&:capitalize).join("")
+        autoload klass_name.to_sym, "elasticshell/commands/#{command_name}"
+        priority << klass_name
       end
-      
-      shell.print(shell.client.request(verb.downcase.to_sym, params, options))
     end
-    
-    
   end
-  
-end
 
+end