resque-job-stats-promptcloud 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 5769ba71b39f54f345d77c40404fdee78785c473
4
+ data.tar.gz: c6ca1858ee91ba5b3d79e1724b70d406f87a80d6
5
+ SHA512:
6
+ metadata.gz: 3b9d4fb2b1ef9c452f245950af0ff1f53838432a0ab0c2b4c15cf745bc03e359c1a623a13ea19dacfbb6a63216c02e2232ba1442f0990cfd2cf894a87c181849
7
+ data.tar.gz: 517878d9328f92b898289d7a011f7ff1119b76b48d87bfb9d0ab8575f2f889f870f76538d749ac156151b0b785825aa050e1b91cf09fd0771a4dd31a6e5610aa
data/CHANGELOG.md ADDED
@@ -0,0 +1,20 @@
1
+ # Changelog
2
+
3
+ ## Master
4
+
5
+ * Add example app (#27, 158f466)
6
+
7
+ ## 0.4.2
8
+
9
+ * Fix missing ERB files in gemspec (e30e599)
10
+
11
+ ## 0.4.1
12
+
13
+ * Remove `date` from gemspec (31e054b)
14
+
15
+ ## 0.4.0
16
+
17
+ * Add stats and UI for individual job histories (#24, f10d9b6) -- @emilong
18
+ * Modernize the gem, and add Travis CI testing for newer rubies (#23, cbac34d)
19
+
20
+ Changelog started on Oct 20, 2016.
data/LICENSE.txt ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2011 Alan Peabody
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,154 @@
1
+ # resque-job-stats
2
+
3
+ [![Build Status](https://travis-ci.org/alanpeabody/resque-job-stats.svg)](http://travis-ci.org/alanpeabody/resque-job-stats)
4
+
5
+ Job centric stats for Resque.
6
+
7
+ Stats are tracked per Job type (class/module) in addition to the worker based stats Resque provides.
8
+
9
+ Stats tracked are:
10
+
11
+ * Jobs performed
12
+ * Jobs enqueued
13
+ * Jobs failed
14
+ * Duration of last x jobs completed
15
+ * Average job duration over last 100 jobs completed
16
+ * Longest job duration over last 100 jobs completed
17
+ * Jobs enqueued as timeseries data (minute, hourly)
18
+ * Jobs performed as timeseries data (minute, hourly)
19
+
20
+ This information can be used to help track performance and diagnose specific bottlenecks.
21
+
22
+ We are sending this information to Nagios for graphing and alerts (via a custom rake task).
23
+
24
+ ## Installation
25
+
26
+ Requires resque '~> 1.17.0'
27
+
28
+ In your Gemfile add:
29
+
30
+ ```ruby
31
+ gem 'resque-job-stats'
32
+ ```
33
+
34
+ ## Usage
35
+
36
+ Simply extend your class
37
+
38
+ ```ruby
39
+ class MyJob
40
+ extend Resque::Plugins::JobStats
41
+
42
+ @queue = :my_job
43
+ def self.perform(*args)
44
+ # ..
45
+ end
46
+ end
47
+ ```
48
+
49
+ And you will have a set of keys starting with `'stats:jobs:my_job'` inside your Resque redis namespace.
50
+
51
+ Alternatively you can include just the metric you wish to record.
52
+
53
+ ```ruby
54
+ class MyVerboseJob
55
+ extend Resque::Plugins::JobStats::Performed
56
+ extend Resque::Plugins::JobStats::Enqueued
57
+ extend Resque::Plugins::JobStats::Failed
58
+ extend Resque::Plugins::JobStats::Duration
59
+ extend Resque::Plugins::JobStats::Timeseries::Enqueued
60
+ extend Resque::Plugins::JobStats::Timeseries::Performed
61
+
62
+ @queue = :my_job
63
+ def self.perform(*args)
64
+ # ...
65
+ end
66
+ end
67
+ ```
68
+
69
+ ### Duration module
70
+
71
+ The duration module provides two metrics, the longest job and the job rolling avg.
72
+
73
+ These are accessible via two singleton methods, `MyJob.job_rolling_avg` and `MyJob.longest_job`.
74
+
75
+ By default the last 100 jobs durations are stored and used to provide the above metrics.
76
+
77
+ You may set the number of jobs to include by setting the `@durations_recorded` variable.
78
+
79
+
80
+ ```ruby
81
+ class MyJob
82
+ extend Resque::Plugins::JobStats::Duration
83
+
84
+ @queue = :my_job
85
+ @durations_recorded = 1000
86
+
87
+ def self.perform(*payload)
88
+ # ...
89
+ end
90
+ end
91
+ ```
92
+
93
+ ### Timeseries module
94
+
95
+ The timeseries module provides timeseries counts of jobs performed. The metrics are rolling and kept for a period of time before being expired.
96
+ The timestamp used for the timeseries data is UTC.
97
+
98
+ ## Resque Web Tab
99
+
100
+ The Resque web page for showing the statistics will only display jobs that extend Resque::Plugins::JobStats (in other words, just
101
+ the jobs that include all of the metrics):
102
+
103
+ ```ruby
104
+ class MyJob
105
+ extend Resque::Plugins::JobStats
106
+ ...
107
+ end
108
+ ```
109
+
110
+ The interface can be included in your app like this:
111
+
112
+ ```ruby
113
+ require 'resque-job-stats/server'
114
+ ```
115
+
116
+ If you wish to display only certain metrics, you can filter the metrics accordingly. The default metrics can be found in Resque::Plugins::JobStats::Statistic.
117
+
118
+ ```ruby
119
+ Resque::Server.job_stats_to_display = [:jobs_enqueued, :job_rolling_avg]
120
+ ```
121
+
122
+ ## Screenshots
123
+
124
+ ### Overview
125
+
126
+ ![overview stats](docs/images/stats-overview.png)
127
+
128
+ ### Individual Job Histories
129
+
130
+ ![individual stats](docs/images/stats-individual.png)
131
+
132
+ ## TODO
133
+
134
+ * Find clean way to add queue wait time stats.
135
+
136
+ ## Contributing to resque-job-stats
137
+
138
+ * Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
139
+ * Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it
140
+ * Fork the project
141
+ * Start a feature/bugfix branch
142
+ * Commit and push until you are happy with your contribution
143
+ * Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
144
+ * Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
145
+
146
+ ## Contributers
147
+
148
+ * [damonmorgan](https://github.com/damonmorgan)
149
+ * [unclebilly](https://github.com/unclebilly)
150
+
151
+ ## Copyright
152
+
153
+ Copyright (c) 2011-2012 Alan Peabody. See LICENSE.txt for further details.
154
+
@@ -0,0 +1 @@
1
+ require 'resque/plugins/job_stats'
@@ -0,0 +1,87 @@
1
+ require 'resque/server'
2
+
3
+ module Resque
4
+ module Plugins
5
+ module JobStats
6
+ module Server
7
+ VIEW_PATH = File.join(File.dirname(__FILE__), 'server', 'views')
8
+
9
+ def job_stats_to_display
10
+ @job_stats_to_display ||= Resque::Plugins::JobStats::Statistic::DEFAULT_STATS
11
+ end
12
+
13
+ # Set this to an array of the public accessor names in Resque::Plugins::JobStats::Statistic
14
+ # that you wish to display. The default is [:jobs_enqueued, :jobs_performed, :jobs_failed, :job_rolling_avg, :longest_job]
15
+ # Examples:
16
+ # Resque::Server.job_stats_to_display = [:jobs_performed, :job_rolling_avg]
17
+ def job_stats_to_display=(stats)
18
+ @job_stats_to_display = stats
19
+ end
20
+
21
+ module Helpers
22
+ def display_stat?(stat_name)
23
+ self.class.job_stats_to_display == :all ||
24
+ [self.class.job_stats_to_display].flatten.map(&:to_sym).include?(stat_name.to_sym)
25
+ end
26
+
27
+ def time_display(float)
28
+ float.zero? ? "" : ("%.2f" % float.to_s) + "s"
29
+ end
30
+
31
+ def number_display(num)
32
+ num.zero? ? "" : num
33
+ end
34
+
35
+ def stat_header(stat_name)
36
+ if(display_stat?(stat_name))
37
+ "<th>" + stat_name.to_s.gsub(/_/,' ').capitalize + "</th>"
38
+ end
39
+ end
40
+
41
+ def display_stat(stat, stat_name, format)
42
+ if(display_stat?(stat_name))
43
+ formatted_stat = self.send(format, stat.send(stat_name))
44
+ "<td>#{formatted_stat}</td>"
45
+ end
46
+ end
47
+
48
+ def check_or_cross_stat(value)
49
+ value ? "&#x2713;" : "&#x2717;"
50
+ end
51
+ end
52
+
53
+ class << self
54
+ def registered(app)
55
+ app.get '/job_stats' do
56
+ @jobs = Resque::Plugins::JobStats::Statistic.find_all(self.class.job_stats_to_display).sort
57
+ erb(File.read(File.join(VIEW_PATH, 'job_stats.erb')))
58
+ end
59
+ # We have little choice in using this funky name - Resque
60
+ # already has a "Stats" tab, and it doesn't like
61
+ # tab names with spaces in it (it translates the url as job%20stats)
62
+ app.tabs << "Job_Stats"
63
+
64
+ app.get '/job_history/:job_class' do
65
+ @job_class = Resque::Plugins::JobStats.measured_jobs.find { |j| j.to_s == params[:job_class] }
66
+ pass unless @job_class
67
+
68
+ @start = 0
69
+ @start = params[:start].to_i if params[:start]
70
+ @limit = 100
71
+ @limit = params[:limit].to_i if params[:limit]
72
+
73
+ @histories = @job_class.job_histories(@start,@limit)
74
+ @size = @job_class.histories_recorded
75
+
76
+ erb(File.read(File.join(VIEW_PATH, 'job_histories.erb')))
77
+ end
78
+
79
+ app.helpers(Helpers)
80
+ end
81
+ end
82
+ end
83
+ end
84
+ end
85
+ end
86
+
87
+ Resque::Server.register Resque::Plugins::JobStats::Server
@@ -0,0 +1,38 @@
1
+ <h1>Resque Job Histories</h1>
2
+
3
+ <p class="intro">
4
+ This page displays histories of jobs that have been executed.
5
+ </p>
6
+
7
+ <h2><%= @job_class %></h2>
8
+
9
+ <table>
10
+ <tr>
11
+ <th>Start time</th>
12
+ <th>Duration</th>
13
+ <th>Arguments</th>
14
+ <th>Success</th>
15
+ <th>Exception</th>
16
+ </tr>
17
+ <% @histories.each do |history| %>
18
+ <tr>
19
+ <td><span class="time"><%= history["run_at"] %></span></td>
20
+ <td><%= history["duration"] %></td>
21
+ <td><%= history["args"].inspect %></td>
22
+ <td><%= check_or_cross_stat(history["success"]) %></td>
23
+ <td><%= history["exception"]["name"] if history["exception"] %></td>
24
+ </tr>
25
+ <% end %>
26
+ </table>
27
+
28
+ <%if @start > 0 || @start + @limit <= @size %>
29
+ <p class='pagination'>
30
+ <% if @start - @limit >= 0 %>
31
+ <a href="<%= current_page %>?start=<%= [0, @start - @limit].max %>&limit=<%= @limit %>" class='less'>&laquo; less</a>
32
+ <% end %>
33
+ <% if @start + @limit <= @size %>
34
+ <a href="<%= current_page %>?start=<%= @start + @limit %>&limit=<%= @limit %>" class='more'>more &raquo;</a>
35
+ <% end %>
36
+ </p>
37
+ <%end%>
38
+
@@ -0,0 +1,32 @@
1
+ <h1>Resque Job Stats</h1>
2
+
3
+ <p class="intro">
4
+ This page displays statistics about jobs that have been executed.
5
+ </p>
6
+
7
+ <table>
8
+ <tr>
9
+ <th>Name</th>
10
+ <%= stat_header(:jobs_enqueued) %>
11
+ <%= stat_header(:jobs_performed) %>
12
+ <%= stat_header(:jobs_failed) %>
13
+ <%= stat_header(:job_rolling_avg) %>
14
+ <%= stat_header(:longest_job) %>
15
+ </tr>
16
+ <% @jobs.each do |job| %>
17
+ <tr>
18
+ <td>
19
+ <%= job.name %>
20
+ <% if job.job_class <= Resque::Plugins::JobStats::History ||
21
+ job.job_class.is_a?(Resque::Plugins::JobStats::History) %>
22
+ <a href='<%= url "/job_history/#{job.name}" %>'>[history]</a>
23
+ <% end %>
24
+ </td>
25
+ <%= display_stat(job, :jobs_enqueued, :number_display) %>
26
+ <%= display_stat(job, :jobs_performed, :number_display) %>
27
+ <%= display_stat(job, :jobs_failed, :number_display) %>
28
+ <%= display_stat(job, :job_rolling_avg, :time_display) %>
29
+ <%= display_stat(job, :longest_job, :time_display) %>
30
+ </tr>
31
+ <% end %>
32
+ </table>
@@ -0,0 +1,36 @@
1
+ require 'resque'
2
+ require 'resque/plugins/job_stats/measured_hook'
3
+ require 'resque/plugins/job_stats/performed'
4
+ require 'resque/plugins/job_stats/enqueued'
5
+ require 'resque/plugins/job_stats/failed'
6
+ require 'resque/plugins/job_stats/duration'
7
+ require 'resque/plugins/job_stats/timeseries'
8
+ require 'resque/plugins/job_stats/statistic'
9
+ require 'resque/plugins/job_stats/history'
10
+
11
+ module Resque
12
+ module Plugins
13
+ module JobStats
14
+ extend Resque::Plugins::JobStats::MeasuredHook
15
+ include Resque::Plugins::JobStats::Performed
16
+ include Resque::Plugins::JobStats::Enqueued
17
+ include Resque::Plugins::JobStats::Failed
18
+ include Resque::Plugins::JobStats::Duration
19
+ include Resque::Plugins::JobStats::Timeseries::Enqueued
20
+ include Resque::Plugins::JobStats::Timeseries::Performed
21
+ include Resque::Plugins::JobStats::History
22
+
23
+ def self.add_measured_job(name)
24
+ Resque.redis.sadd("stats:jobs", name)
25
+ end
26
+
27
+ def self.rem_measured_job(name)
28
+ Resque.redis.srem("stats:jobs", name)
29
+ end
30
+
31
+ def self.measured_jobs
32
+ Resque.redis.smembers("stats:jobs").collect { |c| Object.const_get(c) rescue nil }.compact
33
+ end
34
+ end
35
+ end
36
+ end
@@ -0,0 +1,49 @@
1
+ module Resque
2
+ module Plugins
3
+ module JobStats
4
+ module Duration
5
+ extend Resque::Plugins::JobStats::MeasuredHook
6
+
7
+ # Resets all job durations
8
+ def reset_job_durations
9
+ Resque.redis.del(jobs_duration_key)
10
+ end
11
+
12
+ # Returns the number of jobs failed
13
+ def job_durations
14
+ Resque.redis.lrange(jobs_duration_key,0,durations_recorded - 1).map(&:to_f)
15
+ end
16
+
17
+ # Returns the key used for tracking job durations
18
+ def jobs_duration_key
19
+ "stats:jobs:#{self.name}:duration"
20
+ end
21
+
22
+ # Increments the failed count when job is complete
23
+ def around_perform_job_stats_duration(*args)
24
+ start = Time.now
25
+ yield
26
+ duration = Time.now - start
27
+
28
+ Resque.redis.lpush(jobs_duration_key, duration)
29
+ Resque.redis.ltrim(jobs_duration_key, 0, durations_recorded)
30
+ end
31
+
32
+ def durations_recorded
33
+ @durations_recorded || 100
34
+ end
35
+
36
+ def job_rolling_avg
37
+ job_times = job_durations
38
+ return 0.0 if job_times.size == 0.0
39
+ job_times.inject(0.0) {|s,j| s + j} / job_times.size
40
+ end
41
+
42
+ def longest_job
43
+ job_durations.max.to_f
44
+ end
45
+
46
+ end
47
+ end
48
+ end
49
+ end
@@ -0,0 +1,34 @@
1
+ module Resque
2
+ module Plugins
3
+ module JobStats
4
+
5
+ # Extend your job with this module to track how many
6
+ # jobs are queued successfully
7
+ module Enqueued
8
+ extend Resque::Plugins::JobStats::MeasuredHook
9
+
10
+ # Sets the number of jobs queued
11
+ def jobs_enqueued=(int)
12
+ Resque.redis.set(jobs_enqueued_key,int)
13
+ end
14
+
15
+ # Returns the number of jobs enqueued
16
+ def jobs_enqueued
17
+ Resque.redis.get(jobs_enqueued_key).to_i
18
+ end
19
+
20
+ # Returns the key used for tracking jobs enqueued
21
+ def jobs_enqueued_key
22
+ "stats:jobs:#{self.name}:enqueued"
23
+ end
24
+
25
+ # Increments the enqueued count when job is queued
26
+ def after_enqueue_job_stats_enqueued(*args)
27
+ Resque.redis.incr(jobs_enqueued_key)
28
+ end
29
+
30
+ end
31
+ end
32
+ end
33
+ end
34
+
@@ -0,0 +1,35 @@
1
+ module Resque
2
+ module Plugins
3
+ module JobStats
4
+
5
+ # Extend your job with this module to track how many
6
+ # jobs fail
7
+ module Failed
8
+ extend Resque::Plugins::JobStats::MeasuredHook
9
+
10
+ # Sets the number of jobs failed
11
+ def jobs_failed=(int)
12
+ Resque.redis.set(jobs_failed_key,int)
13
+ end
14
+
15
+ # Returns the number of jobs failed
16
+ def jobs_failed
17
+ jobs_failed = Resque.redis.get(jobs_failed_key).to_i
18
+ return jobs_failed / 2 if Resque::VERSION == '1.20.0'
19
+ jobs_failed
20
+ end
21
+
22
+ # Returns the key used for tracking jobs failed
23
+ def jobs_failed_key
24
+ "stats:jobs:#{self.name}:failed"
25
+ end
26
+
27
+ # Increments the failed count when job is complete
28
+ def on_failure_job_stats_failed(e,*args)
29
+ Resque.redis.incr(jobs_failed_key)
30
+ end
31
+
32
+ end
33
+ end
34
+ end
35
+ end
@@ -0,0 +1,55 @@
1
+ module Resque
2
+ module Plugins
3
+ module JobStats
4
+ module History
5
+ include Resque::Helpers
6
+
7
+ def job_histories(start=0, limit=histories_recordable)
8
+ Resque.redis.lrange(jobs_history_key, start, start + limit - 1).map { |h| decode(h) }
9
+ end
10
+
11
+ # Returns the key used for tracking job histories
12
+ def jobs_history_key
13
+ "stats:jobs:#{self.name}:history"
14
+ end
15
+
16
+ def around_perform_job_stats_history(*args)
17
+ # we collect our own duration and start time rather
18
+ # than correlate with the duration stat to make sure
19
+ # we're associating them with the right job arguments
20
+ start = Time.now
21
+ begin
22
+ yield
23
+ duration = Time.now - start
24
+ push_history "success" => true, "args" => args, "run_at" => start, "duration" => duration
25
+ rescue Exception => e
26
+ duration = Time.now - start
27
+ exception = { "name" => e.to_s, "backtrace" => e.backtrace }
28
+ push_history "success" => false, "exception" => exception, "args" => args, "run_at" => start, "duration" => duration
29
+ raise e
30
+ end
31
+ end
32
+
33
+ def histories_recordable
34
+ @histories_recordable || 100
35
+ end
36
+
37
+ def histories_recorded
38
+ Resque.redis.llen(jobs_history_key)
39
+ end
40
+
41
+ def reset_job_histories
42
+ Resque.redis.del(jobs_history_key)
43
+ end
44
+
45
+ private
46
+
47
+ def push_history(history)
48
+ Resque.redis.lpush(jobs_history_key, encode(history))
49
+ Resque.redis.ltrim(jobs_history_key, 0, histories_recordable)
50
+ end
51
+ end
52
+ end
53
+ end
54
+ end
55
+
@@ -0,0 +1,13 @@
1
+ module Resque
2
+ module Plugins
3
+ module JobStats
4
+ module MeasuredHook
5
+
6
+ def extended(base)
7
+ Resque.redis.sadd("stats:jobs", base)
8
+ end
9
+
10
+ end
11
+ end
12
+ end
13
+ end
@@ -0,0 +1,33 @@
1
+ module Resque
2
+ module Plugins
3
+ module JobStats
4
+
5
+ # Extend your job with this module to track how many
6
+ # jobs are performed successfully
7
+ module Performed
8
+ extend Resque::Plugins::JobStats::MeasuredHook
9
+
10
+ # Sets the number of jobs performed
11
+ def jobs_performed=(int)
12
+ Resque.redis.set(jobs_performed_key,int)
13
+ end
14
+
15
+ # Returns the number of jobs performed
16
+ def jobs_performed
17
+ Resque.redis.get(jobs_performed_key).to_i
18
+ end
19
+
20
+ # Returns the key used for tracking jobs performed
21
+ def jobs_performed_key
22
+ "stats:jobs:#{self.name}:performed"
23
+ end
24
+
25
+ # Increments the performed count when job is complete
26
+ def after_perform_job_stats_performed(*args)
27
+ Resque.redis.incr(jobs_performed_key)
28
+ end
29
+
30
+ end
31
+ end
32
+ end
33
+ end
@@ -0,0 +1,43 @@
1
+ module Resque
2
+ module Plugins
3
+ module JobStats
4
+ # A class composed of a job class and the various job statistics
5
+ # collected for the given job.
6
+ class Statistic
7
+ include Comparable
8
+
9
+ # An array of the default statistics that will be displayed in the web tab
10
+ DEFAULT_STATS = [:jobs_enqueued, :jobs_performed, :jobs_failed, :job_rolling_avg, :longest_job]
11
+
12
+ attr_accessor *[:job_class].concat(DEFAULT_STATS)
13
+
14
+ class << self
15
+ # Find and load a Statistic for all resque jobs that are in the Resque::Plugins::JobStats.measured_jobs collection
16
+ def find_all(metrics)
17
+ Resque::Plugins::JobStats.measured_jobs.map{|j| new(j, metrics)}
18
+ end
19
+ end
20
+
21
+ # A series of metrics describing one job class.
22
+ def initialize(job_class, metrics)
23
+ self.job_class = job_class
24
+ self.load(metrics)
25
+ end
26
+
27
+ def load(metrics)
28
+ metrics.each do |metric|
29
+ self.send("#{metric}=", job_class.send(metric))
30
+ end
31
+ end
32
+
33
+ def name
34
+ self.job_class.name
35
+ end
36
+
37
+ def <=>(other)
38
+ self.name <=> other.name
39
+ end
40
+ end
41
+ end
42
+ end
43
+ end
@@ -0,0 +1,97 @@
1
+ require 'time'
2
+
3
+ module Resque
4
+ module Plugins
5
+ module JobStats
6
+
7
+ # Extend your job with this module to track how many
8
+ # jobs are performed over a period of time
9
+ module Timeseries
10
+
11
+ module Common
12
+ # A timestamp rounded to the lowest minute
13
+ def timestamp
14
+ time = Time.now.utc
15
+ Time.at(time.to_i - time.sec).utc # to_i removes usecs
16
+ end
17
+
18
+ private
19
+
20
+ TIME_FORMAT = {:minutes => "%d:%H:%M", :hours => "%d:%H"}
21
+ FACTOR = {:minutes => 1, :hours => 60}
22
+
23
+ def range(sample_size, time_unit, end_time) # :nodoc:
24
+ (0..sample_size).map { |n| end_time - (n * 60 * FACTOR[time_unit])}
25
+ end
26
+
27
+ def timeseries_data(type, sample_size, time_unit) # :nodoc:
28
+ timeseries_range = range(sample_size, time_unit, timestamp)
29
+ timeseries_keys = timeseries_range.map { |time| jobs_timeseries_key(type, time, time_unit)}
30
+ timeseries_data = Resque.redis.mget(*(timeseries_keys))
31
+
32
+ return Hash[(0..sample_size).map { |i| [timeseries_range[i], timeseries_data[i].to_i]}]
33
+ end
34
+
35
+ def jobs_timeseries_key(type, key_time, time_unit) # :nodoc:
36
+ "#{prefix}:#{type}:#{key_time.strftime(TIME_FORMAT[time_unit])}"
37
+ end
38
+
39
+ def prefix # :nodoc:
40
+ "stats:jobs:#{self.name}:timeseries"
41
+ end
42
+
43
+ def incr_timeseries(type) # :nodoc:
44
+ increx(jobs_timeseries_key(type, timestamp, :minutes), (60 * 61)) # 1h + 1m for some buffer
45
+ increx(jobs_timeseries_key(type, timestamp, :hours), (60 * 60 * 25)) # 24h + 60m for some buffer
46
+ end
47
+
48
+ # Increments a key and sets its expiry time
49
+ def increx(key, ttl)
50
+ Resque.redis.incr(key)
51
+ Resque.redis.expire(key, ttl)
52
+ end
53
+ end
54
+ end
55
+ end
56
+ end
57
+ end
58
+
59
+ module Resque::Plugins::JobStats::Timeseries::Enqueued
60
+ extend Resque::Plugins::JobStats::MeasuredHook
61
+ include Resque::Plugins::JobStats::Timeseries::Common
62
+
63
+ # Increments the enqueued count for the timestamp when job is queued
64
+ def after_enqueue_job_stats_timeseries(*args)
65
+ incr_timeseries(:enqueued)
66
+ end
67
+
68
+ # Hash of timeseries data over the last 60 minutes for queued jobs
69
+ def queued_per_minute
70
+ timeseries_data(:enqueued, 60, :minutes)
71
+ end
72
+
73
+ # Hash of timeseries data over the last 24 hours for queued jobs
74
+ def queued_per_hour
75
+ timeseries_data(:enqueued, 24, :hours)
76
+ end
77
+ end
78
+
79
+ module Resque::Plugins::JobStats::Timeseries::Performed
80
+ extend Resque::Plugins::JobStats::MeasuredHook
81
+ include Resque::Plugins::JobStats::Timeseries::Common
82
+
83
+ # Increments the performed count for the timestamp when job is complete
84
+ def after_perform_job_stats_timeseries(*args)
85
+ incr_timeseries(:performed)
86
+ end
87
+
88
+ # Hash of timeseries data over the last 60 minutes for completed jobs
89
+ def performed_per_minute
90
+ timeseries_data(:performed, 60, :minutes)
91
+ end
92
+
93
+ # Hash of timeseries data over the last 24 hours for completed jobs
94
+ def performed_per_hour
95
+ timeseries_data(:performed, 24, :hours)
96
+ end
97
+ end
@@ -0,0 +1,7 @@
1
+ module Resque
2
+ module Plugins
3
+ module JobStats
4
+ VERSION = '0.4.2'
5
+ end
6
+ end
7
+ end
metadata ADDED
@@ -0,0 +1,133 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: resque-job-stats-promptcloud
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - alanpeabody
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2018-11-29 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: resque
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '1.17'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '1.17'
27
+ - !ruby/object:Gem::Dependency
28
+ name: rake
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - ">="
32
+ - !ruby/object:Gem::Version
33
+ version: '0'
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ version: '0'
41
+ - !ruby/object:Gem::Dependency
42
+ name: minitest
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: '5.0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '5.0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: timecop
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '0.6'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '0.6'
69
+ - !ruby/object:Gem::Dependency
70
+ name: rack-test
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - ">="
74
+ - !ruby/object:Gem::Version
75
+ version: '0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ description: Tracks jobs performed, failed, and the duration of the last 100 jobs
84
+ for each job type.
85
+ email: gapeabody@gmail.com
86
+ executables: []
87
+ extensions: []
88
+ extra_rdoc_files:
89
+ - LICENSE.txt
90
+ - README.md
91
+ files:
92
+ - CHANGELOG.md
93
+ - LICENSE.txt
94
+ - README.md
95
+ - lib/resque-job-stats.rb
96
+ - lib/resque-job-stats/server.rb
97
+ - lib/resque-job-stats/server/views/job_histories.erb
98
+ - lib/resque-job-stats/server/views/job_stats.erb
99
+ - lib/resque/plugins/job_stats.rb
100
+ - lib/resque/plugins/job_stats/duration.rb
101
+ - lib/resque/plugins/job_stats/enqueued.rb
102
+ - lib/resque/plugins/job_stats/failed.rb
103
+ - lib/resque/plugins/job_stats/history.rb
104
+ - lib/resque/plugins/job_stats/measured_hook.rb
105
+ - lib/resque/plugins/job_stats/performed.rb
106
+ - lib/resque/plugins/job_stats/statistic.rb
107
+ - lib/resque/plugins/job_stats/timeseries.rb
108
+ - lib/resque/plugins/job_stats/version.rb
109
+ homepage: http://github.com/alanpeabody/resque-job-stats
110
+ licenses:
111
+ - MIT
112
+ metadata: {}
113
+ post_install_message:
114
+ rdoc_options: []
115
+ require_paths:
116
+ - lib
117
+ required_ruby_version: !ruby/object:Gem::Requirement
118
+ requirements:
119
+ - - ">="
120
+ - !ruby/object:Gem::Version
121
+ version: 1.9.2
122
+ required_rubygems_version: !ruby/object:Gem::Requirement
123
+ requirements:
124
+ - - ">="
125
+ - !ruby/object:Gem::Version
126
+ version: '0'
127
+ requirements: []
128
+ rubyforge_project:
129
+ rubygems_version: 2.6.10
130
+ signing_key:
131
+ specification_version: 4
132
+ summary: Job-centric stats for Resque
133
+ test_files: []