netuitived 0.10.1 → 1.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,20 +1,31 @@
1
- # netuitived
2
- a druby application that allows for the submission and aggregation of metrics through the use of DRbObjects
1
+ NetuitiveD
2
+ ===========
3
3
 
4
- How to install using gem:
4
+ NetuitiveD is a dRuby application submits and aggregates metrics through the use of DRbObjects. NetuitiveD is meant to work in conjunction with the [netuitive_ruby_api](https://rubygems.org/gems/netuitive_ruby_api) and [netuitive_rails_agent](https://rubygems.org/gems/netuitive_rails_agent) gems to help [Netuitive](https://www.netuitive.com) monitor your Ruby applications.
5
5
 
6
- run the command:
6
+ For more information on NetuitiveD, see our Ruby agent [help docs](https://help.netuitive.com/Content/Misc/Datasources/new_ruby_datasource.htm), or contact Netuitive support at [support@netuitive.com](mailto:support@netuitive.com).
7
+
8
+ Installing and Running NetuitiveD
9
+ ---------------------------------
10
+
11
+ ### Install
7
12
 
8
13
  gem install netuitived
9
14
 
10
- How to run:
15
+ ### Run
11
16
 
12
- Once the gem has been installed run the start script (make sure the user it's being run as has access rights to the gem's install directory):
17
+ 1. Run the start script (make sure the user using the command has access rights to the
18
+ gem's install directory):
13
19
 
14
- netuitived start
20
+ netuitived start
15
21
 
16
- The start script will prompt you for the API key (which is found from generating a datasource in the Netuitive ui) and the element name (if unsure what to put, just put the name of your ruby application)
22
+ 1. The start script will prompt you for an API key and an element name. You can get an API key from creating a Ruby datasource in [Netuitive](https://app.netuitive.com/auth/login).
17
23
 
18
- That's it, you're up and running. The extra config information is found in config/agent.yml. Each field in the config file can also be set using environment variables.
24
+ please enter an element name:
25
+ my_app_name
26
+ please enter an api key:
27
+ DEMOab681D46bf5616dba8337c85DEMO
28
+
29
+ If you're unsure about what element name you should use, just use the name of your Ruby application.
19
30
 
20
- This daemon is meant to be used in conjunction with the netuitive_ruby_api (https://rubygems.org/gems/netuitive_ruby_api) and netuitive_rails_agent (https://rubygems.org/gems/netuitive_rails_agent) gems.
31
+ >**Note:** The extra config information is found in config/agent.yml. Each field in the config file can also be set using environment variables.
data/bin/netuitived CHANGED
@@ -1,57 +1,16 @@
1
1
  #!/usr/bin/env ruby
2
+ require 'drb/drb'
3
+ require 'netuitived'
2
4
  require 'yaml'
3
- gem_root= File.expand_path("../..", __FILE__)
4
- data=YAML.load_file "#{gem_root}/config/agent.yml"
5
- if ARGV[0] == "start"
6
- written = false
7
- elementName=ENV["NETUITIVED_ELEMENT_NAME"]
8
- if(elementName == nil or elementName == "")
9
- elementName = data["elementName"]
10
- if(elementName == "elementName" or elementName == "")
11
- puts "please enter an element name: "
12
- elementName = STDIN.gets.chomp
13
- data["elementName"] = elementName
14
- written = true
15
- end
16
- end
17
- apiId=ENV["NETUITIVED_API_ID"]
18
- if(apiId == nil or apiId == "")
19
- apiId = data["apiId"]
20
- if(apiId == "apiId" or apiId == "")
21
- puts "please enter an api key: "
22
- apiId = STDIN.gets.chomp
23
- data["apiId"] = apiId
24
- written = true
25
- end
26
- end
27
- if written
28
- File.open("#{gem_root}/config/agent.yml", 'w') {|f| f.write data.to_yaml }
29
- end
30
- require 'drb/drb'
31
- netuitivedAddr=data["netuitivedAddr"]
32
- netuitivedPort=data["netuitivedPort"]
33
- SERVER_URI="druby://#{netuitivedAddr}:#{netuitivedPort}"
34
- DRb.start_service
35
- begin
36
- DRbObject.new_with_uri(SERVER_URI).stopServer
37
- sleep(2)
38
- rescue
39
-
40
- end
41
- require 'netuitived'
42
- puts "netuitived started"
43
- elsif ARGV[0] == "stop"
44
- require 'drb/drb'
45
- netuitivedAddr=data["netuitivedAddr"]
46
- netuitivedPort=data["netuitivedPort"]
47
- SERVER_URI="druby://#{netuitivedAddr}:#{netuitivedPort}"
48
- DRb.start_service
49
- begin
50
- DRbObject.new_with_uri(SERVER_URI).stopServer
51
- puts "netuitived stopped"
52
- rescue
53
- puts "netuitived isn't running"
54
- end
5
+
6
+ # Check command line arguments
7
+ case ARGV[0]
8
+ when 'run' # runs netuitived in the foreground
9
+ Netuitived.start(true)
10
+ when 'start' # runs netuitived in the background
11
+ Netuitived.start
12
+ when 'stop' # stops running netuitived
13
+ Netuitived.stop
55
14
  else
56
- puts "invalid option. options are: start, stop"
15
+ puts 'invalid option. options are: run, start, stop'
57
16
  end
data/config/agent.yml CHANGED
@@ -1,4 +1,7 @@
1
1
  #all are configurable using environment variables
2
+ logLocation: #absolute path of log file. leave blank for default location in the gem directory. environment variable: NETUITIVED_LOG_LOCATION
3
+ logAge: daily #Number of old log files to keep, or frequency of rotation (daily, weekly or monthly). environment variable: NETUITIVED_LOG_AGE
4
+ logSize: #Maximum logfile size (only applies when shift_age is a number). environment variable: NETUITIVED_LOG_SIZE
2
5
  apiId: apiId #apiId for the datasources. found in the datasources section in the netuitive ui. environment variable: NETUITIVED_API_ID
3
6
  baseAddr: "api.app.netuitive.com" #where to send the data. in most cases use the default. environment variable: NETUITIVED_BASE_ADDR
4
7
  port: nil #in most cases this will be nil. environment variable: NETUITIVED_PORT
data/lib/netuitived.rb CHANGED
@@ -1,12 +1,137 @@
1
- fork do
2
- require 'netuitive/netuitived_config_manager'
3
- require 'netuitive/scheduler'
4
- require 'drb/drb'
5
- require 'netuitive/netuitived_server'
6
- ConfigManager::setup
7
- Scheduler::startSchedule
8
- NETUITIVE_URI="druby://#{ConfigManager.netuitivedAddr}:#{ConfigManager.netuitivedPort}"
9
- FRONT_OBJECT=NetuitivedServer.new
10
- DRb.start_service(NETUITIVE_URI, FRONT_OBJECT)
11
- DRb.thread.join
12
- end
1
+ require 'drb/drb'
2
+ require 'net/http'
3
+ require 'json'
4
+ require 'yaml'
5
+ require 'drb/drb'
6
+ require 'logger'
7
+ require 'netuitived/config_manager'
8
+ require 'netuitived/netuitive_logger'
9
+ require 'netuitived/ingest_event'
10
+ require 'netuitived/ingest_tag'
11
+ require 'netuitived/ingest_metric'
12
+ require 'netuitived/ingest_sample'
13
+ require 'netuitived/ingest_element'
14
+ require 'netuitived/netuitived_server'
15
+ require 'netuitived/api_emissary'
16
+ require 'netuitived/metric_aggregator'
17
+ require 'netuitived/event_handler'
18
+ require 'netuitived/scheduler'
19
+
20
+ ##
21
+ # Provides facilities for running Netuitived
22
+ class Netuitived
23
+ class << self
24
+ attr_reader :server_uri
25
+
26
+ ##
27
+ # The DRb front object
28
+ #
29
+ # @return [NetuitivedServer] The NetuitivedServer front object
30
+ def front_object
31
+ @front_object ||= new_front_object
32
+ end
33
+
34
+ def new_front_object
35
+ apiEmissary = NetuitiveD::APIEmissary.new
36
+ NetuitiveD::NetuitivedServer.new(NetuitiveD::MetricAggregator.new(apiEmissary), NetuitiveD::EventHandler.new(apiEmissary))
37
+ end
38
+
39
+ ##
40
+ # Checks that config values are provided and prompts for them if not
41
+ def interactively_check_config
42
+ gem_root = File.expand_path('../..', __FILE__)
43
+ data = YAML.load_file("#{gem_root}/config/agent.yml")
44
+
45
+ written = false
46
+ element_name = ENV['NETUITIVED_ELEMENT_NAME'] || data['elementName'] || ''
47
+
48
+ if element_name == 'elementName' || element_name == ''
49
+ puts 'please enter an element name: '
50
+ element_name = STDIN.gets.chomp
51
+ data['elementName'] = element_name
52
+ written = true
53
+ end
54
+
55
+ api_id = ENV['NETUITIVED_API_ID'] || data['apiId'] || ''
56
+
57
+ if api_id == 'apiId' || api_id == ''
58
+ puts 'please enter an api key: '
59
+ api_id = STDIN.gets.chomp
60
+ data['apiId'] = api_id
61
+ written = true
62
+ end
63
+
64
+ return unless written
65
+
66
+ File.open("#{gem_root}/config/agent.yml", 'w') { |f| f.write data.to_yaml }
67
+ end
68
+
69
+ ##
70
+ # Starts netuitived
71
+ #
72
+ # @param foreground [true, false] If true, run netuitived in the foreground
73
+ def start(foreground = false)
74
+ # Maintain the initial setup behavior
75
+ interactively_check_config
76
+
77
+ # Load the config from disk into ConfigManager
78
+ load_config
79
+
80
+ # Stop the service if it's already running
81
+ stop(true)
82
+
83
+ # Create a proc to run netuitived
84
+ runner = proc do
85
+ NetuitiveD::NetuitiveLogger.log.debug 'starting scheduler'
86
+ NetuitiveD::Scheduler.startSchedule
87
+ NetuitiveD::NetuitiveLogger.log.debug 'starting drb service'
88
+ begin
89
+ DRb.start_service(server_uri, front_object)
90
+ DRb.thread.join
91
+ rescue => e
92
+ NetuitiveD::NetuitiveLogger.log.error "drb error: #{e.message} backtrace: #{e.backtrace}"
93
+ end
94
+ end
95
+
96
+ if foreground
97
+ puts 'netuitived running'
98
+ runner.call
99
+ else
100
+ fork(&runner)
101
+ puts 'netuitived started'
102
+ end
103
+ end
104
+
105
+ ##
106
+ # Stops netuitived if it's running
107
+ #
108
+ # @param suppress_not_running_message [true, false] suppresses the "netuitived isn't running" message
109
+ def stop(suppress_not_running_message = false)
110
+ load_config
111
+
112
+ DRb.start_service
113
+
114
+ begin
115
+ DRbObject.new_with_uri(@server_uri).stopServer
116
+ puts 'netuitived stopped'
117
+ rescue
118
+ puts "netuitived isn't running" unless suppress_not_running_message
119
+ end
120
+ end
121
+
122
+ private
123
+
124
+ ##
125
+ # Loads the configuration if necessary
126
+ def load_config
127
+ unless @config_manager_setup
128
+ NetuitiveD::ConfigManager.load_config
129
+ NetuitiveD::NetuitiveLogger.setup
130
+ NetuitiveD::ConfigManager.read_config
131
+ end
132
+
133
+ @config_manager_setup = true
134
+ @server_uri ||= "druby://#{NetuitiveD::ConfigManager.netuitivedAddr}:#{NetuitiveD::ConfigManager.netuitivedPort}".freeze
135
+ end
136
+ end
137
+ end
@@ -0,0 +1,51 @@
1
+ module NetuitiveD
2
+ class APIEmissary
3
+ def sendElements(elementString)
4
+ NetuitiveD::NetuitiveLogger.log.debug 'start sending elements'
5
+ send("/ingest/ruby/#{NetuitiveD::ConfigManager.apiId}", elementString)
6
+ NetuitiveD::NetuitiveLogger.log.debug 'finish sending elements'
7
+ end
8
+
9
+ def sendEvents(eventString)
10
+ NetuitiveD::NetuitiveLogger.log.debug 'finish sending events'
11
+ send("/ingest/events/ruby/#{NetuitiveD::ConfigManager.apiId}", eventString)
12
+ NetuitiveD::NetuitiveLogger.log.debug 'finish sending events'
13
+ end
14
+
15
+ def send(uri, body)
16
+ NetuitiveD::NetuitiveLogger.log.debug "post body: #{body}"
17
+ req = Net::HTTP::Post.new(uri.to_s, 'Content-Type' => 'application/json')
18
+ req.body = body
19
+ NetuitiveD::NetuitiveLogger.log.debug 'starting post'
20
+ port = if NetuitiveD::ConfigManager.port =~ /(.*)nil(.*)/
21
+ nil
22
+ else
23
+ NetuitiveD::ConfigManager.port.to_int
24
+ end
25
+ NetuitiveD::NetuitiveLogger.log.debug "port: #{port}"
26
+ NetuitiveD::NetuitiveLogger.log.debug "path: #{req.path}"
27
+ NetuitiveD::NetuitiveLogger.log.debug "addr: #{NetuitiveD::ConfigManager.baseAddr}"
28
+ response = Net::HTTP.start(NetuitiveD::ConfigManager.baseAddr.to_s, port, use_ssl: true, read_timeout: 30, open_timeout: 30) do |http|
29
+ http.verify_mode = OpenSSL::SSL::VERIFY_NONE
30
+ http.ssl_version = :SSLv3
31
+ http.request req
32
+ end
33
+ NetuitiveD::NetuitiveLogger.log.debug 'post finished'
34
+ if response.code.to_s != '202' && response.code.to_s != '200'
35
+ NetuitiveD::NetuitiveLogger.log.error "Response from submitting netuitive metrics to api
36
+ code: #{response.code}
37
+ message: #{response.message}
38
+ body: #{response.body}"
39
+ else
40
+ NetuitiveD::NetuitiveLogger.log.info "Response from submitting netuitive metrics to api
41
+ code: #{response.code}
42
+ message: #{response.message}
43
+ body: #{response.body}"
44
+ end
45
+ rescue => exception
46
+ NetuitiveD::NetuitiveLogger.log.error 'error with http post: '
47
+ NetuitiveD::NetuitiveLogger.log.error exception.message
48
+ NetuitiveD::NetuitiveLogger.log.error exception.backtrace
49
+ end
50
+ end
51
+ end
@@ -0,0 +1,91 @@
1
+ module NetuitiveD
2
+ class ConfigManager
3
+ class << self
4
+ attr_reader :apiId
5
+
6
+ attr_reader :baseAddr
7
+
8
+ attr_reader :port
9
+
10
+ attr_reader :elementName
11
+
12
+ attr_reader :netuitivedAddr
13
+
14
+ attr_reader :netuitivedPort
15
+
16
+ attr_reader :interval
17
+
18
+ attr_reader :data
19
+
20
+ def property(name, var, default = nil)
21
+ prop = ENV[var]
22
+ prop = data[name] if prop.nil? || (prop == '')
23
+ return prop unless prop.nil? || (prop == '')
24
+ default
25
+ end
26
+
27
+ def boolean_property(name, var)
28
+ prop = ENV[var].nil? ? nil : ENV[var].dup
29
+ if prop.nil? || (prop == '')
30
+ prop = data[name]
31
+ else
32
+ prop.strip!
33
+ prop = prop.casecmp('true').zero?
34
+ end
35
+ prop
36
+ end
37
+
38
+ def float_property(name, var)
39
+ prop = ENV[var].nil? ? nil : ENV[var]
40
+ if prop.nil? || (prop == '')
41
+ data[name].to_f
42
+ else
43
+ prop.to_f
44
+ end
45
+ end
46
+
47
+ def string_list_property(name, var)
48
+ list = []
49
+ prop = ENV[var].nil? ? nil : ENV[var].dup
50
+ if prop.nil? || (prop == '')
51
+ list = data[name] if !data[name].nil? && data[name].is_a?(Array)
52
+ else
53
+ list = prop.split(',')
54
+ end
55
+ list.each(&:strip!) unless list.empty?
56
+ list
57
+ end
58
+
59
+ def load_config
60
+ gem_root = File.expand_path('../../..', __FILE__)
61
+ @data = YAML.load_file "#{gem_root}/config/agent.yml"
62
+ end
63
+
64
+ def read_config
65
+ @apiId = property('apiId', 'NETUITIVED_API_ID')
66
+ @baseAddr = property('baseAddr', 'NETUITIVED_BASE_ADDR')
67
+ @port = property('port', 'NETUITIVED_PORT')
68
+ @elementName = property('elementName', 'NETUITIVED_ELEMENT_NAME')
69
+ @netuitivedAddr = property('netuitivedAddr', 'NETUITIVED_NETUITIVED_ADDR')
70
+ @netuitivedPort = property('netuitivedPort', 'NETUITIVED_NETUITIVED_PORT')
71
+ @interval = property('interval', 'NETUITIVED_INTERVAL')
72
+ debugLevelString = property('debugLevel', 'NETUITIVED_DEBUG_LEVEL')
73
+ NetuitiveD::NetuitiveLogger.log.level = if debugLevelString == 'error'
74
+ Logger::ERROR
75
+ elsif debugLevelString == 'info'
76
+ Logger::INFO
77
+ elsif debugLevelString == 'debug'
78
+ Logger::DEBUG
79
+ else
80
+ Logger::ERROR
81
+ end
82
+ NetuitiveD::NetuitiveLogger.log.debug "read config file. Results:
83
+ apiId: #{apiId}
84
+ baseAddr: #{baseAddr}
85
+ port: #{port}
86
+ elementName: #{elementName}
87
+ debugLevel: #{debugLevelString}"
88
+ end
89
+ end
90
+ end
91
+ end
@@ -0,0 +1,40 @@
1
+ module NetuitiveD
2
+ class EventHandler
3
+ def initialize(apiEmissary)
4
+ @apiEmissary = apiEmissary
5
+ end
6
+
7
+ def handleEvent(message, timestamp, title, level, source, type, tags)
8
+ NetuitiveD::NetuitiveLogger.log.debug "self: #{object_id}"
9
+ NetuitiveD::NetuitiveLogger.log.debug "Thread: #{Thread.current.object_id}"
10
+ NetuitiveD::NetuitiveLogger.log.debug "Received event: message:#{message}, timestamp:#{timestamp}, title:#{title}, level:#{level}, source:#{source}, type:#{type}, tags:#{tags}"
11
+ event = NetuitiveD::IngestEvent.new(NetuitiveD::ConfigManager.elementName, message, timestamp, title, level, source, type, tags)
12
+ events = [event]
13
+ eventString = events.to_json
14
+ @apiEmissary.sendEvents(eventString)
15
+ rescue => e
16
+ NetuitiveD::NetuitiveLogger.log.error "error in handling event: #{e.message}, backtrace: #{e.backtrace}"
17
+ end
18
+
19
+ def handleExceptionEvent(exception, klass, tags = {})
20
+ message = "Exception Message: #{exception[:message]}\n" if (defined? exception[:message]) && !exception[:message].nil?
21
+ message ||= ''
22
+ timestamp = Time.new
23
+ title = 'Ruby Exception'
24
+ level = 'Warning'
25
+ source = 'Ruby Agent'
26
+ type = 'INFO'
27
+ ingest_tags = []
28
+ tags [:Exception] = klass unless klass.nil?
29
+ tags.each do |key, value|
30
+ next if !(defined? value) || value.nil? || value == ''
31
+ ingest_tags << NetuitiveD::IngestTag.new(key, value)
32
+ message += "#{key}: #{value}\n"
33
+ end
34
+ message += "Backtrace:\n\t#{exception[:backtrace]}" if (defined? exception[:backtrace]) && !exception[:backtrace].nil?
35
+ handleEvent(message, timestamp, title, level, source, type, ingest_tags)
36
+ rescue => e
37
+ NetuitiveD::NetuitiveLogger.log.error "error in exception event: #{e.message}, backtrace: #{e.backtrace}"
38
+ end
39
+ end
40
+ end