resque_to_cloudwatch 1.0.3 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: d88766e5181718a770178730716c7d9ad96c7b0b
4
- data.tar.gz: 539d25384ef0cb4e83663950c0ffe56bd0e42eb5
3
+ metadata.gz: 11e5ee66485c3ad30e8ceb3c2c62918025b90433
4
+ data.tar.gz: c5cdef90cef5a6ac4dc60619add241626425707d
5
5
  SHA512:
6
- metadata.gz: ce13d441f2406006263875786c3f5a4bf1c0fe3ae4c28e5f30e9fc85739f79faad4b3461ac0ab0ff8a0cea688456d68e6e53e4de2cfcebd5d8e590b0a993a3a8
7
- data.tar.gz: 878cd5a3c2dfe5b9a30c97317042fe8c3c8b38115a87b29a6375115ec385e1b9959de3d84178c0a994138c44be545ec0c7304e7c4412796e1c9de3e74406fedf
6
+ metadata.gz: 0d66e4be36c5b12e0fa3084630c6bfd39accee9d5fbb983ae66aefd2536c99cea7a1199bfb1d4e621fa4490293892ba3437486b94a60c8bb57c3d350d8514e9e
7
+ data.tar.gz: 956431e69c343e0dabff5e28a128b322269c602862ba764b8b6014595a48ba0f931fe7ec2021e0caee77bd021a85628e58a36794eba8df8fa61cdaed2e25fc5b
data/README.md CHANGED
@@ -1,7 +1,7 @@
1
1
  # resque_to_cloudwatch
2
2
 
3
3
  This is a gem containing a daemon for submitting Resque queue lengths to AWS
4
- Cloudwatch.
4
+ Cloudwatch. In addition, it can submit this metric to Graphite.
5
5
 
6
6
  ## Usage
7
7
 
@@ -12,6 +12,8 @@ The daemon lives at `bin/resque_to_cloudwatch`, and takes only a single paramete
12
12
  If you don't supply this parameter, the daemon will look for a config file in the
13
13
  current directory.
14
14
 
15
+ ## Configuration
16
+
15
17
  The config file has the following format (in YAML):
16
18
 
17
19
  access_key_id: asdfasfasd
@@ -21,4 +23,42 @@ The config file has the following format (in YAML):
21
23
  hostname: laptop
22
24
  redis_host: some.redis.host
23
25
  redis_port: 6379
24
- period: 60
26
+ period: 60
27
+ namespace: F3D
28
+
29
+ #### `access_key_id`, `secret_access_key`
30
+
31
+ Your AWS access key/secret key pair.
32
+
33
+ #### `region`
34
+
35
+ Region of Cloudwatch the metric should be submitted to.
36
+
37
+ #### `project`, `namespace`, `hostname`
38
+
39
+ These can actually contain any text - they are submitted as "dimensions" along
40
+ with the metric value. A single metric in Cloudwatch is uniquely identified by
41
+ its name (hardcoded to `jobs_queued` in this gem), and any dimensions it has.
42
+ At Forward3D, we have a number of autoscaling "projects" (codebases), so we use
43
+ the project dimension to represent that. Namespace is more important, as that
44
+ determines how the metric is categorised in the Cloudwatch interface. Hostname
45
+ can be anything - we set it to the hostname of the submitting machine.
46
+
47
+ #### `redis_host`, `redis_port`
48
+
49
+ Redis host and port.
50
+
51
+ #### `enable_graphite`, `graphite_host`, `graphite_port`
52
+
53
+ This gem can also send the metric it collects to Graphite - set `enable_graphite`
54
+ to `true`, and set the Graphite host and port if you want to the daemon to do this.
55
+
56
+ #### `period`
57
+
58
+ The period of the EventMachine loop - how often stats are collected and sent.
59
+
60
+ ## Graphite metric
61
+
62
+ The Graphite metric name will look like this:
63
+
64
+ resque_to_cloudwatch.namespace.hostname.project
@@ -18,19 +18,26 @@ end.parse!
18
18
  options[:config] ||= "#{File.expand_path("../../config.yaml", __FILE__)}"
19
19
  config = ResqueToCloudwatch::Config.new(options[:config])
20
20
  collector = ResqueToCloudwatch::Collector.new(config)
21
- sender = ResqueToCloudwatch::Sender.new(config)
22
21
 
22
+ # Initialize senders we're going to use
23
+ senders = []
24
+ senders << ResqueToCloudwatch::CloudwatchSender.new(config)
25
+ senders << ResqueToCloudwatch::GraphiteSender.new(config) if config.enable_graphite
26
+
27
+ $log.info "Senders in use: #{senders.join(", ")}"
23
28
  $log.info "Entering EventMachine loop with period of #{config.period} seconds"
24
29
 
25
30
  EventMachine.run do
26
31
  EventMachine.add_periodic_timer(config.period) do
27
- begin
28
- sender.send_value(collector.get_queue_length)
29
- rescue => e
30
- $log.error "Exception sending or collecting data: #{e.message}"
31
- e.backtrace.each do |line|
32
- $log.error line
32
+ senders.each do |sender|
33
+ begin
34
+ sender.send_value(collector.get_queue_length)
35
+ rescue => e
36
+ $log.error "Exception sending or collecting data with sender #{sender}: #{e.message}"
37
+ e.backtrace.each do |line|
38
+ $log.error line
39
+ end
33
40
  end
34
- end
41
+ end
35
42
  end
36
43
  end
data/config.yaml CHANGED
@@ -6,3 +6,4 @@ hostname: andy-laptop
6
6
  redis_host: some.redis.host
7
7
  redis_port: 6379
8
8
  period: 60
9
+ namespace: F3D
@@ -1,7 +1,7 @@
1
1
  require 'aws-sdk'
2
2
 
3
3
  module ResqueToCloudwatch
4
- class Sender
4
+ class CloudwatchSender
5
5
 
6
6
  # Pass an instance of CloudwatchToResque::Config
7
7
  def initialize(config)
@@ -11,7 +11,7 @@ module ResqueToCloudwatch
11
11
  def send_value(value)
12
12
  cw = AWS::CloudWatch.new
13
13
  cw.client.put_metric_data({
14
- :namespace => 'F3D/resque_queues',
14
+ :namespace => "#{@config.namespace}/resque_queues",
15
15
  :metric_data => [
16
16
  :metric_name => "jobs_queued",
17
17
  :dimensions => [
@@ -28,7 +28,17 @@ module ResqueToCloudwatch
28
28
  :unit => 'Count'
29
29
  ]
30
30
  })
31
- $log.info "Sent metric value #{value}"
31
+ $log.info "CloudwatchSender: Sent metric value #{value}"
32
32
  end
33
+
34
+ def inspect
35
+ to_s
36
+ end
37
+
38
+ def to_s
39
+ "CloudwatchSender"
40
+ end
41
+
33
42
  end
43
+
34
44
  end
@@ -6,17 +6,19 @@ module ResqueToCloudwatch
6
6
 
7
7
  attr_reader :access_key_id, :secret_access_key, :project, :period, :region
8
8
  attr_reader :redis_host, :redis_port, :hostname
9
+ attr_reader :graphite_host, :graphite_port, :enable_graphite
10
+ attr_reader :namespace
9
11
 
10
12
  def initialize(path)
11
13
  $log.info "Loading configuration"
12
14
  raise "Config file #{path} not found or readable" unless File.exists?(path)
13
- @required_opts = %w{access_key_id secret_access_key project period region redis_host redis_port hostname}
15
+ @required_opts = %w{access_key_id secret_access_key project period region redis_host redis_port hostname namespace}
14
16
  @hash = YAML.load_file(path)
15
17
  raise "Config file #{path} is empty" unless @hash
16
18
  validate_config
17
- @required_opts.each do |opt|
18
- instance_variable_set("@#{opt}", @hash[opt])
19
- $log.info "Config parameter: #{opt} is #{@hash[opt]}"
19
+ @hash.each_pair do |opt,value|
20
+ instance_variable_set("@#{opt}", value)
21
+ $log.info "Config parameter: #{opt} is #{value}"
20
22
  end
21
23
 
22
24
  # Set up AWS credentials
@@ -34,6 +36,12 @@ module ResqueToCloudwatch
34
36
  @hash[opt].nil?
35
37
  end
36
38
  raise "Missing options: #{missing_opts.join(", ")}" unless missing_opts.empty?
39
+ if @hash["enable_graphite"]
40
+ raise "Graphite enabled but config missing graphite_host" if @hash["graphite_host"].nil?
41
+ @hash["graphite_port"] ||= 2003
42
+ else
43
+ @hash["enable_graphite"] = false
44
+ end
37
45
  end
38
46
 
39
47
  end
@@ -0,0 +1,28 @@
1
+ require 'simple-graphite'
2
+
3
+ module ResqueToCloudwatch
4
+ class GraphiteSender
5
+
6
+ # Pass an instance of ResqueToCloudwatch::Config
7
+ def initialize(config)
8
+ @config = config
9
+ end
10
+
11
+ def send_value(value)
12
+ graphite = Graphite.new({:host => @config.graphite_host, :port => @config.graphite_port})
13
+ graphite.send_metrics({
14
+ "resque_to_cloudwatch.#{@config.namespace}.#{@config.hostname}.#{@config.project}" => value
15
+ })
16
+ $log.info "GraphiteSender: sent metric value #{value}"
17
+ end
18
+
19
+ def inspect
20
+ to_s
21
+ end
22
+
23
+ def to_s
24
+ "GraphiteSender"
25
+ end
26
+
27
+ end
28
+ end
@@ -1,3 +1,3 @@
1
1
  module ResqueToCloudwatch
2
- VERSION = "1.0.3"
2
+ VERSION = "1.1.0"
3
3
  end
@@ -3,5 +3,6 @@ $stdout.sync = true
3
3
  $log = Logger.new($stdout)
4
4
 
5
5
  require_relative "resque_to_cloudwatch/config.rb"
6
- require_relative "resque_to_cloudwatch/sender.rb"
6
+ require_relative "resque_to_cloudwatch/cloudwatch_sender.rb"
7
+ require_relative "resque_to_cloudwatch/graphite_sender.rb"
7
8
  require_relative "resque_to_cloudwatch/collector.rb"
@@ -24,5 +24,6 @@ Gem::Specification.new do |spec|
24
24
  spec.add_dependency "aws-sdk", "= 1.24.0"
25
25
  spec.add_dependency "eventmachine", "= 1.0.3"
26
26
  spec.add_dependency "redis", "= 3.0.6"
27
+ spec.add_dependency "simple-graphite", "~> 2.1.0"
27
28
 
28
29
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: resque_to_cloudwatch
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.3
4
+ version: 1.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andy Sykes
@@ -80,6 +80,20 @@ dependencies:
80
80
  - - '='
81
81
  - !ruby/object:Gem::Version
82
82
  version: 3.0.6
83
+ - !ruby/object:Gem::Dependency
84
+ name: simple-graphite
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ~>
88
+ - !ruby/object:Gem::Version
89
+ version: 2.1.0
90
+ type: :runtime
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ~>
95
+ - !ruby/object:Gem::Version
96
+ version: 2.1.0
83
97
  description: Submit Resque queue lengths to Cloudwatch
84
98
  email:
85
99
  - github@tinycat.co.uk
@@ -98,9 +112,10 @@ files:
98
112
  - bin/resque_to_cloudwatch
99
113
  - config.yaml
100
114
  - lib/resque_to_cloudwatch.rb
115
+ - lib/resque_to_cloudwatch/cloudwatch_sender.rb
101
116
  - lib/resque_to_cloudwatch/collector.rb
102
117
  - lib/resque_to_cloudwatch/config.rb
103
- - lib/resque_to_cloudwatch/sender.rb
118
+ - lib/resque_to_cloudwatch/graphite_sender.rb
104
119
  - lib/resque_to_cloudwatch/version.rb
105
120
  - resque_to_cloudwatch.gemspec
106
121
  homepage: https://github.com/forward3d/resque_to_cloudwatch