silw 0.0.1

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 210ca5566b6b7422cbb3031c6b0d1c7d57c93663
4
+ data.tar.gz: 64a592b3f5ee4eebaf26fe26bb79c5eeaf16a832
5
+ SHA512:
6
+ metadata.gz: 779b1cb7d06bdd6b7f0366499d923258e67457cf4d736f46168440856d8e54d9e2ae3adf26840e47490bf4ecbe8c20a52376eb3840b827e0b259741ac13b955d
7
+ data.tar.gz: 9b9af3e1a6877a20ae716c6466d681d43b106ca0dec4f0cdcf45bf177734ec1f04ab2c189d6dca3d79a4f3ff5887ccdd21ae3189a12af0091a6e2bde713be970
@@ -0,0 +1,44 @@
1
+ # the local logs folder, if any
2
+ log/
3
+
4
+ # rcov generated
5
+ coverage
6
+ coverage.data
7
+
8
+ # rdoc generated
9
+ rdoc
10
+
11
+ # yard generated
12
+ doc
13
+ .yardoc
14
+
15
+ # bundler
16
+ .bundle
17
+
18
+ # jeweler generated
19
+ pkg
20
+
21
+ # Have editor/IDE/OS specific files you need to ignore? Consider using a global gitignore:
22
+ #
23
+ # * Create a file at ~/.gitignore
24
+ # * Include files you want ignored
25
+ # * Run: git config --global core.excludesfile ~/.gitignore
26
+ #
27
+ # After doing this, these files will be ignored in all your git projects,
28
+ # saving you from having to 'pollute' every project you touch with them
29
+ #
30
+ # Not sure what to needs to be ignored for particular editors/OSes? Here's some ideas to get you started. (Remember, remove the leading # of the line)
31
+ #
32
+ # For MacOS:
33
+ #
34
+ .DS_Store
35
+ Gemfile.lock
36
+
37
+ # For TextMate
38
+ *.tmproj
39
+ tmtags
40
+
41
+ # Intellij
42
+ .idea/
43
+ *.iml
44
+
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in silw.gemspec
4
+ gemspec
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2014 Florin T.PATRASCU
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,119 @@
1
+ # SILW
2
+ This is a simple utility that allows one user to monitor several remote systems, provided he has the proper credentials and the authorization to execute remote commands. Inspired from [usagewatch](https://github.com/nethacker/usagewatch), SILW differs by targeting remote systems and exposing a simple and customizable Sinatra web app.
3
+
4
+ ### SILW Plugins available in this release
5
+
6
+ - **Meminfo** - report the memory stats collected from a remote system.
7
+ - **CPU info** - report the total CPU usage for a remote system.
8
+ - **Diskio** - report the disk usage on a remote system.
9
+
10
+ In order to use SILW, you must be authorized to use the target systems, and make sure you can authenticate and that you are authorized to run various system commands i.e. `top`, `free`, read the content of the `/proc/meminfo` file.
11
+
12
+ ### Using the gem in command line mode
13
+ First create a `silw.yaml` config file in your home directory. Excerpt from `~/silw.yaml`:
14
+
15
+ :monitoring:
16
+ router:
17
+ :freq: 10s
18
+ :plugins: cpu mem diskio
19
+
20
+ :authentication:
21
+ :username: username_with_remote_access
22
+ :password: ~/.ssh/id_dsa (or password!!)
23
+
24
+ :server:
25
+ :port: 8080
26
+
27
+ and use SILW for querying the remote servers defined under the `:monitoring:`:
28
+
29
+ silw exec cpu
30
+ silw exec mem cpu diskio
31
+
32
+ Or specify a different remote server, for example:
33
+
34
+ silw exec cpu mem -s router
35
+
36
+ {"host":"router","cpu":{"cpu_usage":"12.5%"}}
37
+ {"host":"router","mem":{"total":2075624.0,"active":538692.0,"free":1135688.0,"usagepercentage":26}}
38
+
39
+ Without a configuration file, you'll have to specify the remote server and a valid username and password combination, for example:
40
+
41
+ silw exec cpu -s honeypot -u florin -k ~/.ssh/xtraterrestrial_dsa
42
+
43
+ ### Using the gem as a local web service
44
+ You can use SILW to monitor remote systems from your browser. For this you will have to create or modify the SILW configuration file: `silw.yaml`. Example:
45
+
46
+ :monitoring:
47
+ triba:
48
+ :freq: 1min
49
+ :plugins: mem, diskio, cpu
50
+ zorius:
51
+ :freq: 30s
52
+ :plugins: cpu, diskio
53
+ authentication:
54
+ :username: johndoe
55
+ :password: ~/.ssh/id_dsa
56
+ :server:
57
+ :port: 8080
58
+
59
+ Say the config file above is called: `silw.yaml` and it is created in your home folder. Then start the SILW server like this:
60
+
61
+ silw server start -c ~/silw.yaml
62
+
63
+ View the data collected in your web browser by pointing it to `http://0.0.0.0:8080`, if the `server.port` in the config file was `8080`. You will see a similar interface:
64
+
65
+ ![](SILW_server_page_example.png)
66
+
67
+ Run: `silw server stop`, to stop the server.
68
+
69
+ ### Integration with [Logstash](http://logstash.net/)
70
+ Because you are using a centralized logging system for collecting your metrics :) SILW can be configured to echo the stats collected to [Logstash](http://logstash.net/).
71
+
72
+ # Echo the stats directly to logstash
73
+ :logstash:
74
+ :host: localhost
75
+ :port: 5228, udp
76
+ # or:
77
+ # :port: 5229, tcp
78
+
79
+ Excerpt from a SILW-logstash output:
80
+
81
+ {
82
+ "message" => "{\"@tags\":[],\"@fields\":{\"host\":\"honeypot\",\"diskio\":[0,24]},
83
+ \"@timestamp\":\"2014-04-18T13:48:00.663-04:00\",\"@version\":\"1\",\"severity\":\"INFO\"}",
84
+ "@version" => "1",
85
+ "@timestamp" => "2014-04-18T17:48:00.663Z",
86
+ "host" => "127.0.0.1:50719",
87
+ "type" => "silw",
88
+ "@tags" => [],
89
+ "@fields" => {
90
+ "host" => "honeypot",
91
+ "diskio" => [
92
+ [0] 0,
93
+ [1] 24
94
+ ]
95
+ },
96
+ "severity" => "INFO"
97
+ }
98
+
99
+
100
+ SILW is using the [logstash-logger](https://github.com/dwbutler/logstash-logger) gem
101
+
102
+ ### Testing
103
+
104
+ cd silw
105
+ bundle exec rake
106
+
107
+ ### Contributing
108
+
109
+ * Fork it
110
+ * Create your feature branch (``git checkout -b my-new-feature``)
111
+ * Add some tests and please make sure they pass
112
+ * Commit your changes (``git commit -am 'Added some feature'``)
113
+ * Push to the branch (``git push origin my-new-feature``)
114
+ * Create new Pull Request
115
+
116
+ ## License
117
+ Copyright (c) 2014 Florin T.Pătraşcu
118
+
119
+ [MIT License](LICENSE.txt)
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env rake
2
+ require 'rubygems'
3
+ require 'bundler/setup'
4
+ require "bundler/gem_tasks"
5
+ require 'rspec/core/rake_task'
6
+
7
+ RSpec::Core::RakeTask.new('spec')
8
+ task :default => :spec
@@ -0,0 +1,21 @@
1
+ #!/usr/bin/env ruby
2
+ $:.unshift File.expand_path(File.join(File.dirname(__FILE__), %w[.. lib]))
3
+ require "silw/cli"
4
+ require "logger"
5
+ require 'logstash-logger'
6
+
7
+ SILW_ENV = ENV['SILW_ENV'] || 'development'
8
+ LOGFILE = File.expand_path("../../log", __FILE__) + "/request.#{SILW_ENV}.log"
9
+ PLUGINS_LOGFILE = File.expand_path("../../log", __FILE__) + "/plugins.#{SILW_ENV}.log"
10
+ LOGS_DIR = File.dirname(LOGFILE)
11
+ PIDFILE = File.expand_path("../../log", __FILE__) + '/silw_server.pid'
12
+
13
+ # check if the log/ folder exists and create it if otherwise
14
+ FileUtils.mkdir(LOGS_DIR) unless File.directory?(LOGS_DIR)
15
+ LOG = Logger.new(LOGFILE, 'daily')
16
+ PLUGINS_LOG = Logger.new(PLUGINS_LOGFILE, 'daily')
17
+
18
+ # Suppress back-trace when exiting command
19
+ Signal.trap("INT") {}
20
+
21
+ Silw::Cli.start ARGV
@@ -0,0 +1,35 @@
1
+ $:.unshift File.dirname(__FILE__)
2
+ require "rubygems"
3
+ require 'sinatra'
4
+ require 'haml'
5
+ require 'rack/contrib'
6
+ require 'rack/cache'
7
+
8
+ require "silw/version"
9
+ require "silw/plugin"
10
+ require "silw/command"
11
+ require "silw/helpers"
12
+ require "silw/server"
13
+
14
+ module Silw
15
+ LOCALHOST = "localhost"
16
+ MY_KEY = '~/.ssh/id_dsa'
17
+ CONFIG_PATH = '~/silw.yaml'
18
+
19
+ Dir.chdir(File.expand_path(File.join("silw/plugins"), File.dirname(__FILE__))) do
20
+ Dir.entries(".").each do |f|
21
+ next if f[0..0].eql?(".")
22
+ require "silw/plugins/#{f}"
23
+ end
24
+ end
25
+
26
+ class Authenticate
27
+ def self.with(auth_options, &block)
28
+ cmd = Command.new(auth_options)
29
+ if block_given?
30
+ block.call(cmd)
31
+ end
32
+ cmd
33
+ end
34
+ end
35
+ end
@@ -0,0 +1,17 @@
1
+ module Silw
2
+ class Agent
3
+ attr_accessor :host, :plugins, :time
4
+
5
+ def initialize(host='')
6
+ @host = host
7
+ @plugins = {}
8
+ end
9
+
10
+ def collect_data(plugin_name, data={}, logger=nil)
11
+ logger.info data.to_json unless logger.nil?
12
+ @plugins[plugin_name]=data[plugin_name]
13
+ @time=Time.now
14
+ self
15
+ end
16
+ end
17
+ end
@@ -0,0 +1,165 @@
1
+ require 'eventmachine'
2
+ require 'chronic_duration'
3
+ require "yaml"
4
+ require "thor"
5
+ require "silw"
6
+ require "silw/agent"
7
+
8
+ Dir["./lib/**/*.rb"].each{|f| require f}
9
+
10
+ module Silw
11
+ class Cli < Thor
12
+ include Thor::Actions
13
+
14
+ method_option :config, :default => Silw::CONFIG_PATH, :aliases => "-c", :desc => "The Silw config file"
15
+ method_option :server, :aliases => "-s", :desc => "host name of the server to be monitored"
16
+ method_option :user, :aliases => "-u", :desc => "user that can be used with a specified public key"
17
+ method_option :key, :aliases => "-k", :desc => "a public key that works for the user specified. Example: ~/.ssh/id_dsa"
18
+ method_option :port, :aliases => "-p", :desc => "port number for the server (overriding the silw.yaml config file)"
19
+ method_option :help, :aliases => "-h", :desc => "showing the available command line options"
20
+ def initialize(*args); super; end
21
+
22
+ desc "exec plugin1 plugin2 [plugin3]", "execute sequentially the plugins specified"
23
+ def exec(*plugins)
24
+ opts = Silw::Cli::load_config(options[:config])
25
+ opts[:authentication][:username] = options[:user] if options[:user]
26
+ opts[:authentication][:password] = options[:key] if options[:key]
27
+ host = options[:server]
28
+ logstash_config = opts[:logstash]
29
+ logger = PLUGINS_LOG
30
+
31
+ unless logstash_config.nil?
32
+ logstash_host = logstash_config[:server] || '0.0.0.0'
33
+ logstash_port = (logstash_config[:port] || '5229, tcp').split(/\W+/)
34
+ logger = LogStashLogger.new logstash_host, logstash_port[0], logstash_port[1].to_sym
35
+ end
36
+
37
+ Authenticate.with(opts) do |user|
38
+ plugins.each do |n|
39
+ info = user.run(n.to_sym, :at => host).to_json
40
+ logger.info info
41
+ puts info
42
+ end
43
+ end
44
+ end
45
+
46
+ desc "info", "server info"
47
+ def info
48
+ if pid = Cli.get_pid
49
+ puts "SILW process info:"
50
+ puts " pid: #{Cli.get_pid}"
51
+ else
52
+ puts "SILW server is not running."
53
+ end
54
+ end
55
+
56
+ # start a simple server for publishing the stats collected. The stats
57
+ # are those specified by the silw.yaml config file.
58
+ desc "server start|stop", "start or stop a Sinatra-based SILW server"
59
+ def server(cmd)
60
+ if cmd =~ /^start/
61
+ fork do
62
+ EM.run do
63
+ app_config = Silw::Cli::load_config(options[:config])
64
+ hosts = app_config[:monitoring]
65
+ app_container = 'thin'
66
+ host = '0.0.0.0'
67
+ port = ENV['PORT'] || app_config[:server][:port]
68
+ agents = {}
69
+ logstash_config = app_config[:logstash]
70
+ logger = PLUGINS_LOG
71
+
72
+ unless logstash_config.nil?
73
+ logstash_host = logstash_config[:server] || '0.0.0.0'
74
+ logstash_port = (logstash_config[:port] || '5229, tcp').split(/\W+/)
75
+ logger = LogStashLogger.new logstash_host, logstash_port[0], logstash_port[1].to_sym
76
+ end
77
+
78
+ app = Sinatra.new(Server)
79
+ app.set silw_config: app_config
80
+ app.set agents: agents
81
+ app.use ::Rack::CommonLogger, LOGFILE
82
+
83
+ $stdout.reopen(LOGFILE)
84
+ $stderr.reopen(LOGFILE)
85
+
86
+ begin
87
+ # Start the background monitoring tasks here ... if any defined?!
88
+ hosts.keys.each do |host|
89
+ EM.add_periodic_timer(ChronicDuration.parse(hosts[host][:freq])) do
90
+ Authenticate.with(app_config) do |user|
91
+ agent = Agent.new host
92
+ hosts[host][:plugins].scan(/(\w+)/).flatten.each do |n|
93
+ agent.collect_data n.to_sym, user.run(n.to_sym, :at => host), logger
94
+ end
95
+ agents[host] = agent
96
+ end
97
+ end
98
+ end
99
+
100
+ dispatch = Rack::Builder.app do
101
+ map '/' do
102
+ run app
103
+ end
104
+ end
105
+
106
+ Rack::Server.start({
107
+ app: dispatch,
108
+ server: app_container,
109
+ Host: host,
110
+ Port: port
111
+ })
112
+
113
+ File.open(PIDFILE, 'w') { |f| f << Process.pid }
114
+ # CTRL-C? Should we do something about that?
115
+ Signal.trap('INT') do
116
+ puts
117
+ Cli.stop
118
+ end
119
+
120
+ # stopped using the script itself?
121
+ Signal.trap('HUP') do
122
+ puts
123
+ Cli.stop
124
+ end
125
+
126
+ rescue => e
127
+ LOG.error "#{e} at #{e.backtrace.join("\n")}"
128
+ # $stderr.puts "Cannot load the silw server; error: #{$!.message}"
129
+ exit 1
130
+ end
131
+ end
132
+ end # fork
133
+ else
134
+ Cli.stop
135
+ end
136
+ end
137
+
138
+ def self.stop
139
+ pid = self.get_pid
140
+ if pid
141
+ Process.kill('HUP', pid)
142
+ File.unlink(PIDFILE)
143
+ else
144
+ exit -1
145
+ end
146
+ end
147
+
148
+ def self.get_pid
149
+ if File.exists?(PIDFILE)
150
+ pid = File.read(PIDFILE).to_i
151
+ pid > 0 ? pid : nil
152
+ end
153
+ end
154
+
155
+ private
156
+ def self.load_config(path=CONFIG_PATH)
157
+ config_file = File.expand_path(path)
158
+ if File.exists?(config_file)
159
+ ::YAML.load_file config_file
160
+ else
161
+ {:authentication => {:username => 'nobody', :password => '~/.ssh/id_dsa'}}
162
+ end
163
+ end
164
+ end
165
+ end