silw 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 210ca5566b6b7422cbb3031c6b0d1c7d57c93663
+  data.tar.gz: 64a592b3f5ee4eebaf26fe26bb79c5eeaf16a832
+SHA512:
+  metadata.gz: 779b1cb7d06bdd6b7f0366499d923258e67457cf4d736f46168440856d8e54d9e2ae3adf26840e47490bf4ecbe8c20a52376eb3840b827e0b259741ac13b955d
+  data.tar.gz: 9b9af3e1a6877a20ae716c6466d681d43b106ca0dec4f0cdcf45bf177734ec1f04ab2c189d6dca3d79a4f3ff5887ccdd21ae3189a12af0091a6e2bde713be970

data/.gitignore

@@ -0,0 +1,44 @@
+# the local logs folder, if any
+log/
+
+# rcov generated
+coverage
+coverage.data
+
+# rdoc generated
+rdoc
+
+# yard generated
+doc
+.yardoc
+
+# bundler
+.bundle
+
+# jeweler generated
+pkg
+
+# Have editor/IDE/OS specific files you need to ignore? Consider using a global gitignore: 
+#
+# * Create a file at ~/.gitignore
+# * Include files you want ignored
+# * Run: git config --global core.excludesfile ~/.gitignore
+#
+# After doing this, these files will be ignored in all your git projects,
+# saving you from having to 'pollute' every project you touch with them
+#
+# Not sure what to needs to be ignored for particular editors/OSes? Here's some ideas to get you started. (Remember, remove the leading # of the line)
+#
+# For MacOS:
+#
+.DS_Store
+Gemfile.lock
+
+# For TextMate
+*.tmproj
+tmtags
+
+# Intellij
+.idea/
+*.iml
+

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in silw.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2014 Florin T.PATRASCU
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,119 @@
+# SILW
+This is a simple utility that allows one user to monitor several remote systems, provided he has the proper credentials and the authorization to execute remote commands. Inspired from [usagewatch](https://github.com/nethacker/usagewatch), SILW differs by targeting remote systems and exposing a simple and customizable Sinatra web app.
+
+### SILW Plugins available in this release
+
+  - **Meminfo** - report the memory stats collected from a remote system.
+  - **CPU info** - report the total CPU usage for a remote system.
+  - **Diskio** - report the disk usage on a remote system.
+
+In order to use SILW, you must be authorized to use the target systems, and make sure you can authenticate and that you are authorized to run various system commands i.e. `top`, `free`, read the content of the `/proc/meminfo` file.
+
+### Using the gem in command line mode
+First create a `silw.yaml` config file in your home directory. Excerpt from `~/silw.yaml`:
+
+    :monitoring:
+      router:
+        :freq: 10s
+        :plugins: cpu mem diskio
+
+    :authentication:
+      :username: username_with_remote_access
+      :password: ~/.ssh/id_dsa (or password!!)
+
+    :server: 
+      :port: 8080
+
+and use SILW for querying the remote servers defined under the `:monitoring:`:
+
+    silw exec cpu
+    silw exec mem cpu diskio
+
+Or specify a different remote server, for example:
+
+    silw exec cpu mem -s router
+
+    {"host":"router","cpu":{"cpu_usage":"12.5%"}}
+    {"host":"router","mem":{"total":2075624.0,"active":538692.0,"free":1135688.0,"usagepercentage":26}}
+
+Without a configuration file, you'll have to specify the remote server and a valid username and password combination, for example:
+
+    silw exec cpu -s honeypot -u florin -k ~/.ssh/xtraterrestrial_dsa
+
+### Using the gem as a local web service
+You can use SILW to monitor remote systems from your browser. For this you will have to create or modify the SILW configuration file: `silw.yaml`. Example:
+    
+    :monitoring:
+      triba:
+        :freq: 1min
+        :plugins: mem, diskio, cpu
+      zorius:
+        :freq: 30s
+        :plugins: cpu, diskio
+    authentication:
+      :username: johndoe
+      :password: ~/.ssh/id_dsa
+    :server: 
+      :port: 8080
+
+Say the config file above is called: `silw.yaml` and it is created in your home folder. Then start the SILW server like this:
+
+    silw server start -c ~/silw.yaml
+
+View the data collected in your web browser by pointing it to `http://0.0.0.0:8080`, if the `server.port` in the config file was `8080`. You will see a similar interface:
+
+![](SILW_server_page_example.png)
+
+Run: `silw server stop`, to stop the server.
+
+### Integration with [Logstash](http://logstash.net/)
+Because you are using a centralized logging system for collecting your metrics :) SILW can be configured to echo the stats collected to [Logstash](http://logstash.net/).
+
+		# Echo the stats directly to logstash
+		:logstash:
+		  :host: localhost
+			:port: 5228, udp
+			# or:
+			# :port: 5229, tcp
+
+Excerpt from a SILW-logstash output:
+
+			{
+             "message" => "{\"@tags\":[],\"@fields\":{\"host\":\"honeypot\",\"diskio\":[0,24]},
+               \"@timestamp\":\"2014-04-18T13:48:00.663-04:00\",\"@version\":\"1\",\"severity\":\"INFO\"}",
+            "@version" => "1",
+          "@timestamp" => "2014-04-18T17:48:00.663Z",
+                "host" => "127.0.0.1:50719",
+                "type" => "silw",
+               "@tags" => [],
+             "@fields" => {
+                "host" => "honeypot",
+              "diskio" => [
+                  [0] 0,
+                  [1] 24
+              ]
+          },
+            "severity" => "INFO"
+      }
+
+
+SILW is using the [logstash-logger](https://github.com/dwbutler/logstash-logger) gem
+
+### Testing
+
+    cd silw
+    bundle exec rake
+
+### Contributing
+
+* Fork it
+* Create your feature branch (``git checkout -b my-new-feature``)
+* Add some tests and please make sure they pass
+* Commit your changes (``git commit -am 'Added some feature'``)
+* Push to the branch (``git push origin my-new-feature``)
+* Create new Pull Request
+
+## License
+Copyright (c) 2014 Florin T.Pătraşcu
+
+[MIT License](LICENSE.txt)

data/Rakefile

@@ -0,0 +1,8 @@
+#!/usr/bin/env rake
+require 'rubygems' 
+require 'bundler/setup'
+require "bundler/gem_tasks"
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new('spec')
+task :default => :spec

data/bin/silw

@@ -0,0 +1,21 @@
+#!/usr/bin/env ruby
+$:.unshift File.expand_path(File.join(File.dirname(__FILE__), %w[.. lib]))
+require "silw/cli"
+require "logger"
+require 'logstash-logger'
+
+SILW_ENV        = ENV['SILW_ENV'] || 'development'
+LOGFILE         = File.expand_path("../../log", __FILE__) + "/request.#{SILW_ENV}.log"
+PLUGINS_LOGFILE = File.expand_path("../../log", __FILE__) + "/plugins.#{SILW_ENV}.log"
+LOGS_DIR        = File.dirname(LOGFILE)
+PIDFILE         = File.expand_path("../../log", __FILE__) + '/silw_server.pid'
+
+# check if the log/ folder exists and create it if otherwise
+FileUtils.mkdir(LOGS_DIR) unless File.directory?(LOGS_DIR)
+LOG         = Logger.new(LOGFILE, 'daily')
+PLUGINS_LOG = Logger.new(PLUGINS_LOGFILE, 'daily')
+
+# Suppress back-trace when exiting command
+Signal.trap("INT") {} 
+
+Silw::Cli.start ARGV

data/lib/silw.rb

@@ -0,0 +1,35 @@
+$:.unshift File.dirname(__FILE__)
+require "rubygems"
+require 'sinatra'
+require 'haml'
+require 'rack/contrib'
+require 'rack/cache'
+
+require "silw/version"
+require "silw/plugin"
+require "silw/command"
+require "silw/helpers"
+require "silw/server"
+
+module Silw
+  LOCALHOST   = "localhost"
+  MY_KEY      = '~/.ssh/id_dsa'
+  CONFIG_PATH = '~/silw.yaml'
+
+  Dir.chdir(File.expand_path(File.join("silw/plugins"), File.dirname(__FILE__))) do
+    Dir.entries(".").each do |f|
+      next if f[0..0].eql?(".")
+      require "silw/plugins/#{f}"
+    end
+  end
+
+  class Authenticate
+    def self.with(auth_options, &block)      
+      cmd = Command.new(auth_options)
+      if block_given?
+        block.call(cmd)
+      end
+      cmd
+    end
+  end
+end

data/lib/silw/agent.rb

@@ -0,0 +1,17 @@
+module Silw
+  class Agent
+    attr_accessor :host, :plugins, :time
+
+    def initialize(host='')
+      @host = host
+      @plugins = {}
+    end
+    
+    def collect_data(plugin_name, data={}, logger=nil)
+      logger.info data.to_json unless logger.nil?
+      @plugins[plugin_name]=data[plugin_name]
+      @time=Time.now
+      self
+    end
+  end
+end

data/lib/silw/cli.rb

@@ -0,0 +1,165 @@
+require 'eventmachine'
+require 'chronic_duration'
+require "yaml"
+require "thor"
+require "silw"
+require "silw/agent"
+
+Dir["./lib/**/*.rb"].each{|f| require f}
+
+module Silw
+  class Cli < Thor
+    include Thor::Actions
+
+    method_option :config, :default => Silw::CONFIG_PATH, :aliases => "-c", :desc => "The Silw config file"
+    method_option :server, :aliases => "-s", :desc => "host name of the server to be monitored"
+    method_option :user, :aliases => "-u", :desc => "user that can be used with a specified public key"
+    method_option :key, :aliases => "-k", :desc => "a public key that works for the user specified. Example: ~/.ssh/id_dsa"
+    method_option :port, :aliases => "-p", :desc => "port number for the server (overriding the silw.yaml config file)"
+    method_option :help, :aliases => "-h", :desc => "showing the available command line options"
+    def initialize(*args); super; end
+
+    desc "exec plugin1 plugin2 [plugin3]", "execute sequentially the plugins specified"
+    def exec(*plugins)
+      opts = Silw::Cli::load_config(options[:config])
+      opts[:authentication][:username] = options[:user] if options[:user]
+      opts[:authentication][:password] = options[:key] if options[:key]
+      host = options[:server]
+      logstash_config = opts[:logstash]
+      logger = PLUGINS_LOG
+
+      unless logstash_config.nil?
+        logstash_host = logstash_config[:server] || '0.0.0.0'
+        logstash_port = (logstash_config[:port] || '5229, tcp').split(/\W+/)
+        logger = LogStashLogger.new logstash_host, logstash_port[0], logstash_port[1].to_sym
+      end
+
+      Authenticate.with(opts) do |user|
+        plugins.each do |n|
+          info = user.run(n.to_sym, :at => host).to_json
+          logger.info info
+          puts info
+        end
+      end
+    end
+
+    desc "info", "server info"
+    def info
+      if pid = Cli.get_pid
+        puts "SILW process info:"
+        puts "  pid: #{Cli.get_pid}"
+      else
+        puts "SILW server is not running."
+      end
+    end
+
+    # start a simple server for publishing the stats collected. The stats
+    # are those specified by the silw.yaml config file.
+    desc "server start|stop", "start or stop a Sinatra-based SILW server"
+    def server(cmd)
+      if cmd =~ /^start/
+        fork do
+          EM.run do
+            app_config      = Silw::Cli::load_config(options[:config])
+            hosts           = app_config[:monitoring]
+            app_container   = 'thin'
+            host            = '0.0.0.0'
+            port            = ENV['PORT'] || app_config[:server][:port]
+            agents          = {}
+            logstash_config = app_config[:logstash]
+            logger          = PLUGINS_LOG
+
+            unless logstash_config.nil?
+              logstash_host = logstash_config[:server] || '0.0.0.0'
+              logstash_port = (logstash_config[:port] || '5229, tcp').split(/\W+/)
+              logger = LogStashLogger.new logstash_host, logstash_port[0], logstash_port[1].to_sym
+            end
+
+            app = Sinatra.new(Server)
+            app.set silw_config: app_config
+            app.set agents: agents
+            app.use ::Rack::CommonLogger, LOGFILE
+
+            $stdout.reopen(LOGFILE)
+            $stderr.reopen(LOGFILE)
+
+            begin
+              # Start the background monitoring tasks here ... if any defined?!
+              hosts.keys.each do |host|
+                EM.add_periodic_timer(ChronicDuration.parse(hosts[host][:freq])) do
+                  Authenticate.with(app_config) do |user|
+                    agent = Agent.new host                    
+                    hosts[host][:plugins].scan(/(\w+)/).flatten.each do |n|
+                      agent.collect_data n.to_sym, user.run(n.to_sym, :at => host), logger
+                    end
+                    agents[host] = agent
+                  end
+                end
+              end
+
+              dispatch = Rack::Builder.app do
+                map '/' do
+                  run app
+                end
+              end
+
+              Rack::Server.start({
+                app:    dispatch,
+                server: app_container,
+                Host:   host,
+                Port:   port
+              })
+              
+              File.open(PIDFILE, 'w') { |f| f << Process.pid }
+              # CTRL-C? Should we do something about that?
+              Signal.trap('INT') do
+                puts
+                Cli.stop
+              end
+
+              # stopped using the script itself?
+              Signal.trap('HUP') do
+                puts
+                Cli.stop
+              end
+
+            rescue => e
+              LOG.error "#{e} at #{e.backtrace.join("\n")}"
+              # $stderr.puts "Cannot load the silw server; error: #{$!.message}"
+              exit 1    
+            end        
+          end    
+        end # fork     
+      else
+        Cli.stop
+      end
+    end
+    
+    def self.stop
+      pid = self.get_pid
+      if pid
+        Process.kill('HUP', pid)
+        File.unlink(PIDFILE)
+      else
+        exit -1
+      end
+    end
+
+    def self.get_pid
+      if File.exists?(PIDFILE)
+        pid = File.read(PIDFILE).to_i
+        pid > 0 ? pid : nil
+      end
+    end
+
+    private    
+    def self.load_config(path=CONFIG_PATH)
+      config_file = File.expand_path(path)
+      if File.exists?(config_file)
+        ::YAML.load_file config_file
+      else
+        {:authentication => {:username => 'nobody', :password => '~/.ssh/id_dsa'}}
+      end
+    end
+  end
+end