rsql 0.1.3

data/LICENSE

@@ -0,0 +1,19 @@
+Copyright (C) 2011 by brad+rsql@gigglewax.com
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.txt

@@ -0,0 +1,119 @@
+= rsql
+
+https://github.com/bradrf/rsql
+
+== DESCRIPTION
+
+This is an application to make working with a SQL command line more
+convenient by allowing interaction with Ruby in addition to embedding
+the common operation of using a SSH connection to an intermediary host
+for access to the SQL server.
+
+== SYNOPSIS
+
+See rsql -help for usage.
+
+Aside from the standard MySQL command syntax, the following
+functionality allows for a little more expressive processing.
+
+Multiple commands can be issued in one set by separation with
+semicolons.
+
+Generating SQL
+--------------
+
+Ruby code may be called to generate the SQL that is to be executed.
+This is done by starting any command string with a period. If the
+final result of evaluating the command string is another string, it is
+executed as SQL. Any semicolons meant to be processed by Ruby must be
+escaped. Example:
+
+ rsql> . puts 'hello world!' \\; 'select * from Account'
+
+Utilizing Canned Methods
+------------------------
+
+Commands can be stored in the .rsqlrc file in your HOME directory to
+expose methods that may be invoked to generate SQL with variable
+interpolation. Use of the 'register' helper is recommended for this
+approach. These can then be called in the same way as above. Example:
+
+In the .rsqlrc file...
+
+ register :users_by_email, :email %q{
+   SELECT * FROM Users WHERE email = '\#\{email\}'
+ }
+
+...then from the prompt:
+
+ rsql> . users_by_email 'brad@gigglewax.com'
+
+If a block is provided to the registration, it will be called as a
+method. Example:
+
+In the .sqlrc file...
+
+ register :dumby, :hello do |*args|
+   p args
+ end
+
+ rsql> . dumby :world
+
+All registered methods can be listed using the built-in 'list'
+command.
+
+Changes to a sourced file can be reloaded using the built-in 'reload'
+command.
+
+Processing Column Data
+----------------------
+
+Ruby can be called to process any data on a per-column basis before a
+displayer is used to render the output. In this way, one can write
+Ruby to act like MySQL functions on all the data for a given column,
+converting it into a more readable value. A bang indicator (exlamation
+point: !) is used to demarcate a mapping of column names to Ruby
+methods that should be invoked to processes content. Example:
+
+ rsql> select IpAddress from Devices ; ! IpAddress => bin_to_str
+
+This will call 'bin_to_str' for each 'IpAddress' returned from the
+query. Mulitple mappings are separated by a comma. These mappings can
+also be utilized in a canned method. Example:
+
+ register :all_ips, 'select IpAddress from Devices', 'IpAddress' => :bin_to_str
+
+Redirection
+-----------
+
+Output from one or more queries may be post-processed dynamically. If
+any set of commands is follwed by a greater-than symbol, all results
+will be stored in a global $results array (with field information
+stored in $fields) and the final Ruby code will be evaluated with
+access to them. Any result of the evaluation that is a string is then
+executed as SQL. Example:
+
+ rsql> select * from Account; select * from Users; > $results.each {|r| p r}
+
+== LICENSE
+
+Copyright (C) 2011 by Brad Robel-Forrest <brad+rsql@gigglewax.com>
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/TODO

@@ -0,0 +1,30 @@
+* Support separation of semicolon content from registered sql.
+
+* Fix multiline dump to be columnar sensitive (i.e. all but the first
+  line will need a prefix of previous column length whitespace.
+
+* Add option to allow registration to include an automatic displayer.
+
+* Add ability to save passwords in local OS password storage.
+
+* Add ability to highlight lines (e.g. to indicate warning lines about
+  some data like LastHeartbeatTime not recent).
+
+* Add way to prevent this and just allow the raw string to be printed:
+  elsif HEX_RANGE.include?(field.type) && val =~ /[^[:print:]\s]/
+      val = @@eval_context.to_hexstr(val)
+  end
+
+* Fix overlap of functionality between getting input in rsql and
+  parsing input in commands.rb (e.g. they both duplicate the concepts
+  of looking for exit/quit as well as what the end character(s) should
+  be)
+
+* Wrap calling of mysql query in a thread to make it interruptable
+  (might need to re-establish a connection if it's stopped but
+  expected a reply...look at the source).
+
+* Consider using mysql's ping to determine if we need to reconnect.
+
+* Move last_command logic into mysql_results and remember last five
+  (or so).

data/bin/rsql

@@ -0,0 +1,357 @@
+#!/usr/bin/env ruby
+
+# Copyright (C) 2011 by Brad Robel-Forrest <brad+rsql@gigglewax.com>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+begin
+    # this isn't required unless that's how mysql and net/ssh have
+    # been installed
+    require 'rubygems'
+rescue LoadError
+end
+
+require 'thread'
+require 'timeout'
+require 'readline'
+require 'yaml'
+require 'net/ssh'
+
+# allow ourselves to be run from within a source tree
+if File.symlink?(__FILE__)
+    fn = File.readlink(__FILE__)
+else
+    fn = __FILE__
+end
+libdir = File.expand_path(File.join(File.dirname(fn),'..','lib'))
+$: << libdir if File.directory?(libdir)
+
+require 'rsql'
+include RSQL
+
+bn = File.basename($0, '.rb')
+
+eval_context = EvalContext.new
+
+if i = ARGV.index('-rc')
+    ARGV.delete_at(i)
+    rc_fn = ARGV.delete_at(i)
+else
+    rc_fn = File.join(ENV['HOME'], ".#{bn}rc")
+end
+
+eval_context.load(rc_fn) if File.exists?(rc_fn)
+
+def get_password(prompt)
+    iswin = nil != (RUBY_PLATFORM =~ /(win|w)32$/)
+    STDOUT.print(prompt)
+    STDOUT.flush
+    `stty -echo` unless iswin
+    password = STDIN.gets
+    password.chomp!
+ensure
+    `stty echo` unless iswin
+    STDOUT.puts
+    return password
+end
+
+# safely separate login credentials while preserving "emtpy" values--
+# anything of the form [<username>[:<password]@]<host>
+#
+def split_login(str)
+    login = []
+    if i = str.rindex(?@)
+        login << str[i+1..-1]
+        if 0 < i
+            str = str[0..i-1]
+            i = str.index(?:)
+            if 0 == i
+                login << '' << str[i+1..-1]
+            elsif i
+                login << str[0..i-1] << str[i+1..-1]
+            else
+                login << str
+            end
+        else
+            login << ''
+        end
+    else
+        login << str
+    end
+end
+
+if ARGV.delete('-help')
+    eval_context.help
+    exit
+end
+
+if ARGV.delete('-version')
+    puts "#{bn} v#{RSQL::VERSION}"
+    exit
+end
+
+if i = ARGV.index('-maxrows')
+    ARGV.delete_at(i)
+    MySQLResults.max_rows = ARGV.delete_at(i).to_i
+end
+
+if i = ARGV.index('-batch')
+    ARGV.delete_at(i)
+    MySQLResults.field_separator = ARGV.delete_at(i)
+    MySQLResults.field_separator = "\t" if MySQLResults.field_separator == '\t'
+    batch_output = true
+end
+
+if i = ARGV.index('-ssh')
+    ARGV.delete_at(i)
+    (ssh_host, ssh_user, ssh_password) = split_login(ARGV.delete_at(i))
+end
+
+if i = ARGV.index('-e')
+    ARGV.delete_at(i)
+    batch_input = ''
+    ARGV.delete_if do |arg|
+        arg_i = ARGV.index(arg)
+        if i <= arg_i
+            batch_input << ' ' << arg
+        end
+    end
+    batch_input.strip!
+end
+
+if ARGV.size < 1
+    prefix = '        ' << ' ' * bn.size
+    $stderr.puts <<USAGE
+
+usage: #{bn} [-version] [-help]
+#{prefix}[-rc <rcfile>] [-maxrows <max>] [-batch <field_separator>]
+#{prefix}[-ssh [<ssh_user>[:<ssh_password>]@]<ssh_host>]
+#{prefix}[<mysql_user>[:<mysql_password>]@]<mysql_host>
+#{prefix}[<database>] [-e <remaining_args_as_input>]
+
+  If -ssh is used, a SSH tunnel is established before trying to
+  connect to the MySQL server.
+
+  Commands may either be passed in non-interactively via the -e option
+  or by piping into the process' standard input (stdin). If using -e,
+  it _must_ be the last option following all other arguments.
+
+USAGE
+    exit 1
+end
+
+(mysql_host, mysql_user, mysql_password) = split_login(ARGV.shift)
+mysql_password = get_password('mysql password? ') unless mysql_password
+real_mysql_host = mysql_host
+
+if ssh_host
+    # randomly pick a tcp port above 1024
+    mysql_port = rand(0xffff-1025) + 1025
+else
+    mysql_port = 3306
+end
+
+db_name = ARGV.shift
+
+unless $stdin.tty?
+    batch_input = $stdin.read.gsub(/\r?\n/,';')
+end
+
+# make sure we remove any duplicates when we add to the history to
+# keep it clean
+#
+def add_to_history(item)
+    found = nil
+    Readline::HISTORY.each_with_index do |h,i|
+        if h == item
+            found = i
+            break
+        end
+    end
+    Readline::HISTORY.delete_at(found) if found
+    Readline::HISTORY.push(item)
+end
+
+# try closing but wrapped with a timer so we don't hang forever
+#
+def safe_timeout(conn, meth, name)
+    Timeout.timeout(5) { conn.send(meth) }
+    true
+rescue Timeout::Error
+    $stderr.puts "Timed out waiting to close #{name} connection"
+    false
+rescue Exception => ex
+    $stderr.puts(ex)
+    false
+end
+
+MySQLResults.max_rows ||= batch_output ? 5000 : 1000
+
+ssh_enabled = false
+
+if ssh_host
+
+    # might need to open an idle channel here so server doesn't close on
+    # us...or just loop reconnection here in the thread...
+
+    port_opened = false
+
+    puts "SSH #{ssh_user}#{ssh_user ? '@' : ''}#{ssh_host}..." unless batch_input
+    ssh = nil
+    ssh_thread = Thread.new do
+        opts = {:timeout => 15}
+        opts[:password] = ssh_password if ssh_password
+        ssh = Net::SSH.start(ssh_host, ssh_user, opts)
+        ssh_enabled = true
+        ssh.forward.local(mysql_port, mysql_host, 3306)
+        port_opened = true
+        ssh.loop(1) { ssh_enabled }
+    end
+
+    15.times do
+        break if ssh_enabled && port_opened
+        sleep(1)
+    end
+
+    unless ssh_enabled
+        $stderr.puts "failed to connect to #{ssh_host} ssh host in 15 seconds"
+        exit 1
+    end
+
+    unless port_opened
+        $stderr.puts("failed to forward #{mysql_port}:#{mysql_host}:3306 via #{ssh_host} " \
+                     "ssh host in 15 seconds")
+        exit 1
+    end
+
+    # now have our mysql connection use our port forward...
+    mysql_host = '127.0.0.1'
+end
+
+puts "MySQL #{mysql_user}@#{real_mysql_host}..." unless batch_input
+begin
+    MySQLResults.conn = Mysql.new(mysql_host, mysql_user, mysql_password, db_name, mysql_port)
+rescue Mysql::Error => ex
+    if ex.message.include?('Client does not support authentication')
+        $stderr.puts "failed to connect to #{mysql_host} mysql server: unknown credentials?"
+    else
+        $stderr.puts "failed to connect to #{mysql_host} mysql server: #{ex.message}"
+    end
+    exit 1
+end
+
+eval_context.call_init_registrations(MySQLResults.conn)
+
+history_fn = File.join(ENV['HOME'], ".#{bn}_history")
+if File.exists?(history_fn) && 0 < File.size(history_fn)
+    YAML.load_file(history_fn).each {|i| Readline::HISTORY.push(i)}
+end
+
+Readline.completion_proc = eval_context.method(:complete)
+
+cmd_thread = Thread.new do
+    me = Thread.current
+    me[:shutdown] = false
+    until me[:shutdown] do
+        default_displayer = :display_by_column
+        if batch_input
+            default_displayer = :display_by_batch if batch_output
+            me[:shutdown] = true         # only run once
+            input = batch_input
+        else
+            db_name = (MySQLResults.database_name || db_name)
+            puts '',"[#{mysql_user}@#{ssh_host||mysql_host}:#{db_name}]"
+            input = ''
+            prompt = bn + '> '
+            loop do
+                str = Readline.readline(prompt)
+                if str.nil?
+                    input = nil if input.empty?
+                    break
+                end
+                if str =~ /^\s*(exit|quit)\s*$/
+                    me[:shutdown] = true
+                    break
+                end
+                input << str
+                break if input =~ /([^\\];|\\G)\s*$/
+                # make sure we separate the lines with some whitespace if
+                # they didn't
+                input << ' ' unless str =~ /\s$/
+                prompt = ''
+            end
+            if input.nil? || me[:shutdown]
+                puts
+                break
+            end
+            input.strip!
+            next if input.empty?
+        end
+
+        add_to_history(input)
+        cmds = Commands.new(input, default_displayer)
+        if cmds.empty?
+            Readline::HISTORY.pop
+            next
+        end
+
+        me[:running] = true
+        break if cmds.run!(eval_context) == :done
+        me[:running] = false
+    end
+end
+
+Signal.trap('INT') do
+    if cmd_thread[:running] && MySQLResults.conn
+        $stderr.puts 'Interrupting MySQL query...'
+        safe_timeout(MySQLResults.conn, :close, 'MySQL')
+        MySQLResults.conn = nil
+    end
+    $stderr.puts 'Shutting down...'
+    cmd_thread[:shutdown] = true
+    sleep(0.3)
+    cmd_thread.kill
+end
+
+begin
+    cmd_thread.join
+rescue Exception => ex
+    $stderr.puts ex.message, ex.backtrace
+end
+
+unless MySQLResults.conn.nil?
+    safe_timeout(MySQLResults.conn, :close, 'MySQL')
+    MySQLResults.conn = nil
+end
+
+sleep(0.3)
+ssh_enabled = false
+
+if Readline::HISTORY.any?
+    if 100 < Readline::HISTORY.size
+        (Readline::HISTORY.size - 100).times do |i|
+            Readline::HISTORY.delete_at(i)
+        end
+    end
+    File.open(history_fn, 'w') {|f| YAML.dump(Readline::HISTORY.to_a, f)}
+end
+
+if ssh_thread
+    safe_timeout(ssh_thread, :join, 'SSH')
+end