jobtick 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: abfce9cfddae0a798d132217b9376b130afe9446f85e279d3241e7016ed1a89f
4
- data.tar.gz: 36590511cdf6e5b07541cdb66589fc5efd7f2250aeb4d5e5e4b20a18976642de
3
+ metadata.gz: e04ffd3cc7f9e27dc57953e65230cc43d518e3c03413ad1a0f39ef07ac9cde73
4
+ data.tar.gz: ab9648e703ca1baa1397f07a397dad8a5bac19d4c9773a32c64a82e8ca1e1f92
5
5
  SHA512:
6
- metadata.gz: a81d1bf56a1e48ed5395bd5d913b8f49aafa163095261e7f066182f138c71f00259d7e428c5ee5a4e685a4320464aecd66aedde77a1f4c41cbb8431491999a39
7
- data.tar.gz: f1fc3adfece35b9253e59be4beab12c5b2e1292c8330f287a386f37a71715d9cda922ca216a5219052e7097e1db13404b23c8118a10203fdbbf4024f37651e51
6
+ metadata.gz: f6ed9a0f0486689336f4eccc6d0d7c75eb20038b999399240ddab1015b5c85d8411a4b7aa08e8ba21943998b5f5efcfa67a6b288a149684ffdd95942744df721
7
+ data.tar.gz: 4713a1d1f9ee0f4ee9e6e74de2dadac6f18b70fb85bb1f0036554a28feb487a35e1c2b01aea9fe3f7927a9beeb532adc1a87f3840a467cc660fe628fd5a98df8
data/CHANGELOG.md CHANGED
@@ -1,3 +1,18 @@
1
+ ## [0.3.0] - 2026-08-20
2
+
3
+ - **Fix: Whenever monitors never actually registered.** `Parsers::Whenever` called a `Whenever::JobList#jobs` API that doesn't exist on the real gem (only `attr_reader :roles`), so every discovery attempt raised and was silently swallowed — no Whenever job has ever been registered by prior versions. The parser now reads the real, public `generate_cron_output` instead, and `WheneverSetup` injects the monitor key as a literal job option so the registered key and the pinged key are always the same value by construction. `config/schedule.rb` needs `require 'jobtick/whenever_setup'` before `JobTick::WheneverSetup.install!(self)` now — see the README.
4
+ - **Fix: the Whenever shell wrapper could send a false failure alert.** `inner && ping_completed || ping_failed` fired the `failed` ping whenever the *completed* ping's own `curl` call failed — even though the job succeeded. Rewritten as `if <job> ; then ... completed ; else ... failed ; fi`, plus `curl --max-time 10` so a hung ping can't wedge a cron slot, and `exit $rc` so cron still sees the job's real exit status.
5
+ - **Fix: the persistent HTTPS connection was fork-unsafe.** A clustered app server (Puma with `preload_app!`, Passenger, Unicorn) forking after boot left every worker sharing one file descriptor, corrupting or dropping pings under load. The dispatcher now detects a pid change on every call from `.enqueue`/`.send_sync` and rebuilds its connection and background thread in the child without touching the parent's socket.
6
+ - **Fix: `Configuration#environment`-scoped `config/recurring.yml` registered environment names as monitors.** When the current environment had no matching top-level key, the parser fell back to treating the whole file as a task list — registering `solid_queue.production`, `solid_queue.staging`, etc. as permanently-down monitors. It now returns no monitors (with a warning) instead.
7
+ - **Fix:** HTTP responses were previously discarded entirely — a bad API key returned 401 forever in total silence. The dispatcher now logs a 401/403 once and opens a circuit breaker so pings stop being sent to an API that's rejecting them; other error statuses are logged at a throttled rate.
8
+ - **Fix:** `Dispatcher.reset!` could leave a stale connection open (e.g. after only `.send_sync` had ever run), so a subsequent endpoint change kept posting to the old host.
9
+ - Add a circuit breaker: after 3 consecutive network failures, the dispatcher stops attempting connections for a backed-off window (30s–300s) instead of paying a connect timeout per queued ping during an outage.
10
+ - Boot-time sync no longer blocks: it now goes through the async dispatcher, and is skipped entirely for `rails console`, `rails runner`, and rake tasks (configurable via `config.sync_on_boot`). `rake jobtick:sync` remains a blocking call for use as an explicit deploy step.
11
+ - Add `config.ping_started` (default `true`) to suppress the `started` ping for high-frequency jobs.
12
+ - `Parsers::Whenever`/`Parsers::SolidQueue` now resolve their config file paths against `Rails.root` (or `Dir.pwd` outside Rails) instead of the process's current working directory.
13
+ - Two monitors that resolve to the same job class now log a warning naming both, instead of one silently losing its pings to the other.
14
+ - `Configuration#enabled?` is now used consistently by `Client` and `Monitor` instead of each open-coding the same check.
15
+
1
16
  ## [0.2.0] - 2026-05-28
2
17
 
3
18
  - Performance: pings are now dispatched asynchronously on a single daemon thread, so job workers no longer block on network I/O. A persistent, keep-alive HTTPS connection is reused for all pings (no more TCP/TLS handshake per ping).
data/README.md CHANGED
@@ -103,6 +103,20 @@ end
103
103
 
104
104
  With `prune` enabled, a deploy acts as the single source of truth: any monitor whose key is not present in the current payload is permanently deleted. You can also remove individual monitors manually from the JobTick dashboard at any time.
105
105
 
106
+ ### Deploying
107
+
108
+ JobTick syncs your monitors automatically the first time a web or worker process boots after a deploy — this sync runs in the background and never blocks startup, even if the JobTick API is slow or unreachable.
109
+
110
+ To skip the wait until that first boot (or to sync from CI as an explicit deploy step), run it directly:
111
+
112
+ ```
113
+ bundle exec rake jobtick:sync
114
+ ```
115
+
116
+ This is the one case where the sync blocks — it prints the number of monitors registered, so its output is only trustworthy if it actually waited for the API to respond.
117
+
118
+ Boot-time sync is automatically skipped for `rails console`, `rails runner`, and any rake task — none of them should pay for a sync just to boot, and `rails runner` matters especially, since Whenever's cron wrapper shells out through it on every tick. Set `config.sync_on_boot = false` to disable the automatic sync everywhere and rely solely on `rake jobtick:sync`.
119
+
106
120
  ---
107
121
 
108
122
  ## What gets monitored
@@ -122,6 +136,8 @@ sync_exchange_rates:
122
136
 
123
137
  At boot, JobTick reads this file and registers a monitor for each entry. It then installs an `around_perform` hook into `ActiveJob::Base` so every job execution automatically sends `started`, `completed`, and `failed` pings. No changes to your job files.
124
138
 
139
+ Entries scheduled with `command:` instead of `class:` run a raw shell command rather than a Ruby job class, so there's no hook point for JobTick to instrument automatically — these are skipped (with a log line naming them) rather than registered as a monitor JobTick can never ping.
140
+
125
141
  ### Sidekiq periodic jobs
126
142
 
127
143
  Supports both **sidekiq-cron** and **Sidekiq::Periodic**:
@@ -143,15 +159,19 @@ JobTick installs a server middleware that wraps every job execution. For native
143
159
 
144
160
  ### Whenever (`config/schedule.rb`)
145
161
 
146
- Whenever schedules jobs as cron shell commands, so there is no Ruby hook point to instrument automatically. Add one line to `config/schedule.rb`:
162
+ Whenever schedules jobs as cron shell commands, so there is no Ruby hook point to instrument automatically. Add two lines to the top of `config/schedule.rb`:
147
163
 
148
164
  ```ruby
165
+ require 'jobtick/whenever_setup'
149
166
  JobTick::WheneverSetup.install!(self)
150
167
  ```
151
168
 
152
- This overrides the built-in `runner`, `rake`, and `command` job types to wrap every execution with `curl` pings. Your existing schedule entries need no changes:
169
+ `jobtick/whenever_setup` is a separate file from the main gem on purpose — `config/schedule.rb` is evaluated standalone (by `whenever --update-crontab`, and again by JobTick itself at boot to discover your jobs), not as part of a normal Rails boot, so it needs its own explicit require regardless of whether `gem 'jobtick'` is already in your Gemfile.
170
+
171
+ This overrides the built-in `runner`, `rake`, and `command` job types to wrap every execution with `curl` pings, gated by a real `if`/`then`/`else` so a network blip on the ping itself can never be mistaken for the job failing. Your existing schedule entries need no other changes:
153
172
 
154
173
  ```ruby
174
+ require 'jobtick/whenever_setup'
155
175
  JobTick::WheneverSetup.install!(self)
156
176
 
157
177
  every 1.day, at: '2:00 am' do
@@ -163,14 +183,7 @@ every :hour do
163
183
  end
164
184
  ```
165
185
 
166
- After adding the line, run `whenever --update-crontab` as normal and all jobs will start sending heartbeats.
167
-
168
- If jobtick is not already loaded via your Rails environment, require it first:
169
-
170
- ```ruby
171
- require 'jobtick/whenever_setup'
172
- JobTick::WheneverSetup.install!(self)
173
- ```
186
+ After adding these lines, run `whenever --update-crontab` as normal and all jobs will start sending heartbeats. JobTick discovers Whenever monitors by reading the same file back with the real `whenever` gem, so `whenever` must be a dependency of your app (it usually already is) — add it with `require: false` is fine, JobTick loads it itself when needed.
174
187
 
175
188
  ---
176
189
 
@@ -8,8 +8,7 @@ module JobTick
8
8
  SYNC_PATH = "/monitors/sync"
9
9
 
10
10
  def ping(monitor_key, status:, duration: nil, message: nil)
11
- config = JobTick.config
12
- return unless config.enabled && !config.api_key.nil?
11
+ return unless JobTick.config.enabled?
13
12
 
14
13
  payload = { status: status }
15
14
  payload[:duration] = duration.round(3) if duration
@@ -18,15 +17,18 @@ module JobTick
18
17
  Dispatcher.enqueue("#{PING_PREFIX}#{monitor_key}", payload)
19
18
  end
20
19
 
21
- def register(monitors, app_name: nil, prune: false)
22
- config = JobTick.config
23
- return unless config.enabled && !config.api_key.nil?
20
+ def register(monitors, app_name: nil, prune: false, sync: true)
21
+ return unless JobTick.config.enabled?
24
22
 
25
23
  payload = { monitors: monitors }
26
24
  payload[:app_name] = app_name if app_name && !app_name.empty?
27
25
  payload[:prune] = true if prune
28
26
 
29
- Dispatcher.send_sync(SYNC_PATH, payload)
27
+ if sync
28
+ Dispatcher.send_sync(SYNC_PATH, payload)
29
+ else
30
+ Dispatcher.enqueue(SYNC_PATH, payload)
31
+ end
30
32
  end
31
33
  end
32
34
  end
@@ -4,14 +4,17 @@ module JobTick
4
4
  class Configuration
5
5
  DEFAULT_QUEUE_LIMIT = 1000
6
6
 
7
- attr_accessor :api_key, :endpoint, :environment, :enabled, :prune, :queue_limit
7
+ attr_accessor :api_key, :endpoint, :environment, :enabled, :prune, :queue_limit,
8
+ :sync_on_boot, :ping_started
8
9
 
9
10
  def initialize
10
- @endpoint = "https://api.jobtick.app/v1"
11
- @environment = ENV["RAILS_ENV"] || ENV["RACK_ENV"] || "production"
12
- @enabled = @environment == "production"
13
- @prune = false
14
- @queue_limit = DEFAULT_QUEUE_LIMIT
11
+ @endpoint = "https://api.jobtick.app/v1"
12
+ @environment = ENV["RAILS_ENV"] || ENV["RACK_ENV"] || "production"
13
+ @enabled = @environment == "production"
14
+ @prune = false
15
+ @queue_limit = DEFAULT_QUEUE_LIMIT
16
+ @sync_on_boot = true
17
+ @ping_started = true
15
18
  end
16
19
 
17
20
  def enabled?
@@ -12,14 +12,30 @@ module JobTick
12
12
  #
13
13
  # All HTTP work (sync register + async pings) shares one Net::HTTP instance
14
14
  # serialized by @http_mutex. The connection is reopened lazily after errors.
15
+ #
16
+ # Fork safety: .enqueue and .send_sync are always called from the thread
17
+ # that survives a fork (the caller's thread — a job thread, or the process
18
+ # that just booted). Neither the background dispatcher thread nor its
19
+ # Net::HTTP socket survive a fork, even though the Ruby objects referencing
20
+ # them do (they're just inherited memory). guard_fork! runs first on both
21
+ # public entry points and drops those stale references — without closing
22
+ # the socket, which still belongs to the parent — so each process lazily
23
+ # builds its own connection and dispatcher thread.
15
24
  module Dispatcher
16
25
  SHUTDOWN_SIGNAL = :__shutdown__
26
+ FLUSH_SENTINEL = :__flush__
17
27
  HEADER_CONTENT_TYPE = "application/json"
18
28
  USER_AGENT = "jobtick-ruby/#{JobTick::VERSION}".freeze
19
29
  OPEN_TIMEOUT = 5
20
30
  READ_TIMEOUT = 5
21
31
  KEEP_ALIVE_TIMEOUT = 30
22
32
 
33
+ UNAUTHORIZED_CODES = %w[401 403].freeze
34
+ FAILURE_THRESHOLD = 3
35
+ CIRCUIT_INITIAL_BACKOFF = 30
36
+ CIRCUIT_MAX_BACKOFF = 300
37
+ STATUS_WARN_INTERVAL = 60
38
+
23
39
  NETWORK_ERRORS = [
24
40
  IOError, EOFError,
25
41
  Errno::ECONNRESET, Errno::ECONNREFUSED, Errno::ECONNABORTED,
@@ -34,26 +50,31 @@ module JobTick
34
50
  def enqueue(path, payload)
35
51
  return send_request(path, payload) if @synchronous
36
52
 
53
+ guard_fork!
37
54
  ensure_started
38
55
  @queue.push([path, payload], true)
39
56
  nil
40
57
  rescue ThreadError
41
- @dropped += 1
58
+ dropped_mutex.synchronize { @dropped = (@dropped || 0) + 1 }
42
59
  nil
43
60
  end
44
61
 
45
62
  def send_sync(path, payload)
63
+ guard_fork!
46
64
  send_request(path, payload)
47
65
  end
48
66
 
49
67
  def flush(timeout: 5)
50
- return unless @running && @queue
68
+ return unless @running && @queue && @thread&.alive?
51
69
 
52
- deadline = monotonic + timeout
53
- until @queue.empty? && @inflight.zero?
54
- sleep 0.001
55
- break if monotonic > deadline
70
+ ack = Queue.new
71
+ begin
72
+ @queue.push([FLUSH_SENTINEL, ack])
73
+ rescue ThreadError
74
+ return
56
75
  end
76
+ ack.pop(timeout: timeout)
77
+ nil
57
78
  end
58
79
 
59
80
  def shutdown(timeout: 2)
@@ -72,17 +93,46 @@ module JobTick
72
93
 
73
94
  def reset!
74
95
  shutdown(timeout: 1) if @running
96
+ close_http # shutdown is a no-op if only .send_sync ever ran, so close explicitly too
75
97
  @queue = nil
76
98
  @thread = nil
77
99
  @dropped = 0
78
- @inflight = 0
79
100
  @endpoint_uri = nil
80
101
  @at_exit_registered = false
81
102
  @synchronous = false
103
+ @pid = nil
104
+ @consecutive_failures = 0
105
+ @circuit_until = nil
106
+ @unauthorized_warned = false
107
+ @status_warned_at = nil
82
108
  end
83
109
 
84
110
  private
85
111
 
112
+ # Atomic under fork_mutex so a thread that loses the race can never null
113
+ # out @queue/@http *after* another thread has already rebuilt them for
114
+ # the new pid — once this returns, @pid always matches Process.pid, and
115
+ # that transition only ever happens once per fork.
116
+ def guard_fork!
117
+ return if @pid == Process.pid
118
+
119
+ fork_mutex.synchronize do
120
+ return if @pid == Process.pid
121
+
122
+ http_mutex.synchronize { @http = nil } # not #finish — the socket belongs to the parent
123
+ boot_mutex.synchronize do
124
+ @queue = nil
125
+ @thread = nil
126
+ @running = false
127
+ end
128
+ @pid = Process.pid
129
+ end
130
+ end
131
+
132
+ def fork_mutex
133
+ @fork_mutex ||= Mutex.new
134
+ end
135
+
86
136
  def ensure_started
87
137
  return if @running
88
138
 
@@ -91,7 +141,6 @@ module JobTick
91
141
 
92
142
  @queue = SizedQueue.new(queue_limit)
93
143
  @dropped = 0
94
- @inflight = 0
95
144
  @thread = Thread.new { run_loop }
96
145
  @thread.name = "jobtick-dispatcher" if @thread.respond_to?(:name=)
97
146
  @running = true
@@ -107,6 +156,10 @@ module JobTick
107
156
  @http_mutex ||= Mutex.new
108
157
  end
109
158
 
159
+ def dropped_mutex
160
+ @dropped_mutex ||= Mutex.new
161
+ end
162
+
110
163
  def queue_limit
111
164
  JobTick.config.queue_limit || Configuration::DEFAULT_QUEUE_LIMIT
112
165
  end
@@ -122,10 +175,13 @@ module JobTick
122
175
  while (item = @queue.pop)
123
176
  break if item == SHUTDOWN_SIGNAL
124
177
 
125
- @inflight = 1
126
- path, payload = item
127
- send_request(path, payload)
128
- @inflight = 0
178
+ key, payload = item
179
+ if key.equal?(FLUSH_SENTINEL)
180
+ payload.push(true)
181
+ next
182
+ end
183
+
184
+ send_request(key, payload)
129
185
  end
130
186
  rescue StandardError => e
131
187
  JobTick.logger.warn("[JobTick] Dispatcher thread crashed: #{e.message}")
@@ -134,26 +190,82 @@ module JobTick
134
190
  end
135
191
 
136
192
  def send_request(path, payload)
137
- body = JSON.generate(payload)
138
- full_path = "#{endpoint_uri.path}#{path}"
139
- http_mutex.synchronize do
140
- http = http_connection
141
- request = Net::HTTP::Post.new(full_path)
142
- request["Content-Type"] = HEADER_CONTENT_TYPE
143
- request["Authorization"] = "Bearer #{JobTick.config.api_key}"
144
- request["User-Agent"] = USER_AGENT
145
- request.body = body
146
- http.request(request)
147
- end
193
+ return nil if circuit_open?
194
+
195
+ response = http_mutex.synchronize { http_connection.request(build_request(path, payload)) }
196
+ handle_response(path, response)
197
+ response
148
198
  rescue *NETWORK_ERRORS => e
149
199
  JobTick.logger.warn("[JobTick] HTTP request failed (#{path}): #{e.message}")
150
200
  teardown_http
201
+ record_failure
151
202
  nil
152
203
  rescue StandardError => e
153
204
  JobTick.logger.warn("[JobTick] HTTP request failed (#{path}): #{e.message}")
154
205
  nil
155
206
  end
156
207
 
208
+ def build_request(path, payload)
209
+ request = Net::HTTP::Post.new("#{endpoint_uri.path}#{path}")
210
+ request["Content-Type"] = HEADER_CONTENT_TYPE
211
+ request["Authorization"] = "Bearer #{JobTick.config.api_key}"
212
+ request["User-Agent"] = USER_AGENT
213
+ request.body = JSON.generate(payload)
214
+ request
215
+ end
216
+
217
+ def handle_response(path, response)
218
+ code = response.code
219
+
220
+ if code.start_with?("2")
221
+ record_success
222
+ elsif UNAUTHORIZED_CODES.include?(code)
223
+ warn_unauthorized_once
224
+ open_circuit(CIRCUIT_MAX_BACKOFF)
225
+ else
226
+ record_failure
227
+ warn_status_throttled(code, path)
228
+ end
229
+ end
230
+
231
+ def circuit_open?
232
+ !@circuit_until.nil? && monotonic < @circuit_until
233
+ end
234
+
235
+ def open_circuit(seconds)
236
+ @circuit_until = monotonic + seconds + rand(5)
237
+ end
238
+
239
+ def record_failure
240
+ @consecutive_failures = (@consecutive_failures || 0) + 1
241
+ return if @consecutive_failures < FAILURE_THRESHOLD
242
+
243
+ backoff = CIRCUIT_INITIAL_BACKOFF * (2**(@consecutive_failures - FAILURE_THRESHOLD))
244
+ open_circuit([backoff, CIRCUIT_MAX_BACKOFF].min)
245
+ end
246
+
247
+ def record_success
248
+ @consecutive_failures = 0
249
+ @circuit_until = nil
250
+ end
251
+
252
+ def warn_unauthorized_once
253
+ return if @unauthorized_warned
254
+
255
+ @unauthorized_warned = true
256
+ JobTick.logger.warn("[JobTick] API rejected the API key (401/403); pings are being discarded")
257
+ end
258
+
259
+ def warn_status_throttled(code, path)
260
+ @status_warned_at ||= {}
261
+ last = @status_warned_at[code]
262
+ now = monotonic
263
+ return if last && (now - last) < STATUS_WARN_INTERVAL
264
+
265
+ @status_warned_at[code] = now
266
+ JobTick.logger.warn("[JobTick] API returned #{code} (#{path})")
267
+ end
268
+
157
269
  def endpoint_uri
158
270
  @endpoint_uri ||= URI(JobTick.config.endpoint)
159
271
  end
@@ -6,10 +6,10 @@ module JobTick
6
6
 
7
7
  def self.run(key)
8
8
  config = JobTick.config
9
- return yield unless config.enabled && !config.api_key.nil?
9
+ return yield unless config.enabled?
10
10
 
11
11
  client = JobTick.client
12
- client.ping(key, status: :started)
12
+ client.ping(key, status: :started) if config.ping_started
13
13
  started = Process.clock_gettime(MONOTONIC)
14
14
  result = yield
15
15
  duration = Process.clock_gettime(MONOTONIC) - started
@@ -1,14 +1,9 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require_relative "../parsers"
4
+
3
5
  module JobTick
4
6
  module Parsers
5
- SLUG_RE = /[^a-z0-9]+/
6
- SLUG_TRIM_RE = /\A_+|_+\z/
7
-
8
- def self.slugify(str)
9
- str.downcase.gsub(SLUG_RE, "_").gsub(SLUG_TRIM_RE, "")
10
- end
11
-
12
7
  class Sidekiq
13
8
  def self.parse
14
9
  return [] unless defined?(::Sidekiq)
@@ -8,28 +8,78 @@ module JobTick
8
8
  RECURRING_FILE = "config/recurring.yml"
9
9
 
10
10
  def self.parse
11
- return [] unless File.exist?(RECURRING_FILE)
11
+ return [] unless File.exist?(recurring_path)
12
12
 
13
- yaml = YAML.load_file(RECURRING_FILE, aliases: true)
14
- env = JobTick.config.environment
13
+ yaml = YAML.load_file(recurring_path, aliases: true)
14
+ return [] unless yaml.is_a?(Hash)
15
15
 
16
- tasks = yaml[env] || yaml["default"] || yaml
17
- return [] unless tasks.is_a?(Hash)
16
+ tasks = resolve_tasks(yaml)
17
+ return [] if tasks.empty?
18
18
 
19
- tasks.each_with_object([]) do |(key, config), out|
20
- next unless config.is_a?(Hash)
19
+ class_tasks, command_only = tasks.partition { |_, config| task_class?(config) }
20
+ warn_command_only(command_only) if command_only.any?
21
21
 
22
- out << {
23
- key: "solid_queue.#{key}",
24
- schedule: config["schedule"],
25
- source: "solid_queue",
26
- task: config["class"]
27
- }
28
- end
22
+ class_tasks.map { |key, config| monitor_for(key, config) }
29
23
  rescue StandardError => e
30
24
  JobTick.logger.warn("[JobTick] Solid Queue parser failed: #{e.message}")
31
25
  []
32
26
  end
27
+
28
+ # recurring.yml is either flat (task name => task config at the top
29
+ # level) or environment-scoped (environment name => { task name =>
30
+ # task config }). Distinguish by whether any top-level value actually
31
+ # looks like a task, rather than assuming the current environment is
32
+ # always present — a document scoped to "production"/"development"
33
+ # read under "staging" used to fall through to `yaml` itself and
34
+ # register every *environment name* as an unpingable monitor.
35
+ def self.resolve_tasks(yaml)
36
+ return yaml if yaml.any? { |_, v| task?(v) }
37
+
38
+ env = JobTick.config.environment
39
+ scoped = yaml[env] || yaml["default"]
40
+ return scoped if scoped.is_a?(Hash)
41
+
42
+ JobTick.logger.warn(
43
+ "[JobTick] #{RECURRING_FILE} is environment-scoped but has no entry for " \
44
+ "\"#{env}\" (or \"default\"); no Solid Queue monitors registered"
45
+ )
46
+ {}
47
+ end
48
+ private_class_method :resolve_tasks
49
+
50
+ def self.task?(value)
51
+ value.is_a?(Hash) && (value.key?("class") || value.key?("command"))
52
+ end
53
+ private_class_method :task?
54
+
55
+ def self.task_class?(value)
56
+ value.is_a?(Hash) && value.key?("class")
57
+ end
58
+ private_class_method :task_class?
59
+
60
+ def self.monitor_for(key, config)
61
+ { key: "solid_queue.#{key}", schedule: config["schedule"], source: "solid_queue", task: config["class"] }
62
+ end
63
+ private_class_method :monitor_for
64
+
65
+ # command: tasks run a raw shell command rather than a Ruby job class,
66
+ # so there's no ActiveJob/Sidekiq hook to ping them from — unlike
67
+ # Whenever, Solid Queue gives us no shell wrapping point either.
68
+ # Registering them anyway would create monitors that alert as
69
+ # permanently down, so we skip them and say why once.
70
+ def self.warn_command_only(command_only)
71
+ keys = command_only.map(&:first).join(", ")
72
+ JobTick.logger.warn(
73
+ "[JobTick] #{RECURRING_FILE} declares command-based task(s) (#{keys}) which have no " \
74
+ "Ruby class to hook into; JobTick cannot monitor them automatically"
75
+ )
76
+ end
77
+ private_class_method :warn_command_only
78
+
79
+ def self.recurring_path
80
+ File.expand_path(RECURRING_FILE, JobTick.root)
81
+ end
82
+ private_class_method :recurring_path
33
83
  end
34
84
  end
35
85
  end
@@ -1,34 +1,77 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require_relative "../parsers"
4
+
3
5
  module JobTick
4
6
  module Parsers
5
7
  class Whenever
6
8
  SCHEDULE_FILE = "config/schedule.rb"
9
+ JOBTICK_KEY_RE = /JOBTICK_KEY=(\S+)/
7
10
 
11
+ # Whenever::JobList has no public reader for its parsed jobs (only
12
+ # attr_reader :roles), and its internal @jobs is a private, nested
13
+ # [mailto][time_scope] => [Job] structure with no [] accessor on Job
14
+ # either. The only stable, public surface is #generate_cron_output —
15
+ # the same text Whenever writes to the crontab. We scan that instead
16
+ # of reaching into private internals.
17
+ #
18
+ # This only finds anything once JobTick::WheneverSetup.install!(self)
19
+ # has been added to config/schedule.rb: that's what stamps a literal
20
+ # JOBTICK_KEY=<key> into each job's command, which is what we key off
21
+ # of here. That also guarantees the registered key and the pinged key
22
+ # can never drift apart — they're the same literal.
8
23
  def self.parse
9
- return [] unless defined?(::Whenever)
10
- return [] unless File.exist?(SCHEDULE_FILE)
11
-
12
- schedule = ::Whenever::JobList.new(file: SCHEDULE_FILE)
13
- schedule.jobs.flat_map do |period, jobs|
14
- jobs.map do |job|
15
- {
16
- key: job_key(job),
17
- schedule: period.to_s,
18
- source: "whenever",
19
- task: job[:task].to_s.strip
20
- }
21
- end
22
- end
24
+ return [] unless whenever_available?
25
+ return [] unless File.exist?(schedule_path)
26
+
27
+ schedule = ::Whenever::JobList.new(file: schedule_path)
28
+ monitors = schedule.generate_cron_output.to_s.each_line.filter_map { |line| monitor_from_line(line) }
29
+
30
+ warn_if_not_installed if monitors.empty?
31
+ monitors
23
32
  rescue StandardError => e
24
33
  JobTick.logger.warn("[JobTick] Whenever parser failed: #{e.message}")
25
34
  []
26
35
  end
27
36
 
28
- def self.job_key(job)
29
- "whenever.#{Parsers.slugify(job[:task].to_s.strip)}"
37
+ def self.monitor_from_line(line)
38
+ key = line[JOBTICK_KEY_RE, 1]
39
+ return nil unless key
40
+
41
+ { key: key, schedule: cron_fields(line), source: "whenever", task: nil }
42
+ end
43
+ private_class_method :monitor_from_line
44
+
45
+ def self.cron_fields(line)
46
+ line = line.strip
47
+ return line[/\A@\S+/] if line.start_with?("@")
48
+
49
+ line.split(/\s+/, 6).first(5).join(" ")
50
+ end
51
+ private_class_method :cron_fields
52
+
53
+ def self.whenever_available?
54
+ return true if defined?(::Whenever::JobList)
55
+
56
+ require "whenever"
57
+ true
58
+ rescue LoadError
59
+ false
60
+ end
61
+ private_class_method :whenever_available?
62
+
63
+ def self.warn_if_not_installed
64
+ JobTick.logger.warn(
65
+ "[JobTick] #{SCHEDULE_FILE} found but JobTick::WheneverSetup.install!(self) " \
66
+ "is not installed there; no Whenever monitors registered"
67
+ )
68
+ end
69
+ private_class_method :warn_if_not_installed
70
+
71
+ def self.schedule_path
72
+ File.expand_path(SCHEDULE_FILE, JobTick.root)
30
73
  end
31
- private_class_method :job_key
74
+ private_class_method :schedule_path
32
75
  end
33
76
  end
34
77
  end
@@ -0,0 +1,20 @@
1
+ # frozen_string_literal: true
2
+
3
+ module JobTick
4
+ module Parsers
5
+ SLUG_RE = /[^a-z0-9]+/
6
+ SLUG_TRIM_RE = /\A_+|_+\z/
7
+
8
+ def self.slugify(str)
9
+ str.downcase.gsub(SLUG_RE, "_").gsub(SLUG_TRIM_RE, "")
10
+ end
11
+
12
+ # Shared by Parsers::Whenever (which reads it back out of the generated
13
+ # crontab) and WheneverSetup (which injects it as a job option). Keeping
14
+ # both sides derived from this one method is what guarantees the key a
15
+ # monitor is registered under matches the key its pings are sent to.
16
+ def self.whenever_key(task)
17
+ "whenever.#{slugify(task.to_s.strip)}"
18
+ end
19
+ end
20
+ end
@@ -2,6 +2,13 @@
2
2
 
3
3
  module JobTick
4
4
  class Railtie < Rails::Railtie
5
+ # Processes where a boot-time sync would be pure waste: `rails console`,
6
+ # any rake task, and `rails runner` — the latter matters most, since
7
+ # WheneverSetup shells out through `bundle exec rails runner` on every
8
+ # cron tick, so without this a minutely job would pay a full sync POST
9
+ # every minute just to boot.
10
+ ONE_OFF_PROCESSES = %w[rake].freeze
11
+
5
12
  initializer "jobtick.sync_registry" do
6
13
  ActiveSupport.on_load(:after_initialize) do
7
14
  next unless JobTick.config.enabled
@@ -13,7 +20,7 @@ module JobTick
13
20
  require_relative "hooks/active_job"
14
21
  require_relative "middleware/sidekiq"
15
22
 
16
- JobTick::Registry.sync
23
+ JobTick::Registry.sync(sync: false) if JobTick::Railtie.sync_on_boot?
17
24
 
18
25
  ::ActiveJob::Base.include(JobTick::Hooks::ActiveJob) if defined?(::ActiveJob::Base)
19
26
  JobTick::Middleware::Sidekiq.install if defined?(::Sidekiq)
@@ -23,5 +30,24 @@ module JobTick
23
30
  rake_tasks do
24
31
  load File.expand_path("../tasks/jobtick.rake", __dir__)
25
32
  end
33
+
34
+ def self.sync_on_boot?
35
+ return false unless JobTick.config.sync_on_boot
36
+
37
+ !one_off_process?
38
+ end
39
+
40
+ # `Rails::Console` / `Rails::Command::RunnerCommand` / `Rails::Command::RakeCommand`
41
+ # are only defined when boot was reached via `bin/rails console|runner|<task>` — the
42
+ # command file that defines each constant has to be loaded before boot can start.
43
+ # A plain `bundle exec rake <task>` never loads railties' command layer at all, so
44
+ # it's caught by the $PROGRAM_NAME basename check instead.
45
+ def self.one_off_process?
46
+ return true if defined?(Rails::Console)
47
+ return true if defined?(Rails::Command::RunnerCommand)
48
+ return true if defined?(Rails::Command::RakeCommand)
49
+
50
+ ONE_OFF_PROCESSES.include?(File.basename($PROGRAM_NAME.to_s, ".*"))
51
+ end
26
52
  end
27
53
  end
@@ -2,24 +2,54 @@
2
2
 
3
3
  module JobTick
4
4
  class Registry
5
- def self.sync
5
+ # sync: true blocks the caller until the register POST completes (used by
6
+ # `rake jobtick:sync`, whose printed count must be truthful). The railtie
7
+ # boot hook passes sync: false so a slow or unreachable API cannot delay
8
+ # every process boot.
9
+ def self.sync(sync: true)
6
10
  monitors = [
7
11
  Parsers::Whenever.parse,
8
12
  Parsers::SolidQueue.parse,
9
13
  Parsers::Sidekiq.parse
10
14
  ].flatten.compact
11
15
 
12
- map = {}
13
- monitors.each { |m| map[m[:task]] = m[:key] if m[:task] }
14
- JobTick.monitor_map = map.freeze
16
+ JobTick.monitor_map = build_monitor_map(monitors)
15
17
 
16
18
  return [] if monitors.empty?
17
19
 
18
20
  app_name = Rails.application.class.module_parent_name if defined?(Rails)
19
- options = { app_name: app_name }
21
+ options = { app_name: app_name, sync: sync }
20
22
  options[:prune] = true if JobTick.config.prune
21
23
  JobTick.client.register(monitors, **options)
22
24
  monitors
23
25
  end
26
+
27
+ # Builds the class-name => monitor-key map the ActiveJob hook and Sidekiq
28
+ # middleware use to find a job's monitor. Two monitors can legitimately
29
+ # target the same class (e.g. the same recurring job class scheduled
30
+ # twice with different arguments) — that's a real config ambiguity, not a
31
+ # crash, so we log it and keep the first mapping rather than silently
32
+ # letting the second overwrite it.
33
+ def self.build_monitor_map(monitors)
34
+ map = {}
35
+
36
+ monitors.each do |monitor|
37
+ task = monitor[:task]
38
+ next unless task
39
+
40
+ if map.key?(task)
41
+ JobTick.logger.warn(
42
+ "[JobTick] Multiple monitors target #{task} (#{map[task]} and #{monitor[:key]}); " \
43
+ "only #{map[task]} will receive pings for it"
44
+ )
45
+ next
46
+ end
47
+
48
+ map[task] = monitor[:key]
49
+ end
50
+
51
+ map.freeze
52
+ end
53
+ private_class_method :build_monitor_map
24
54
  end
25
55
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module JobTick
4
- VERSION = "0.2.0"
4
+ VERSION = "0.3.0"
5
5
  end
@@ -11,27 +11,48 @@ module JobTick
11
11
  #
12
12
  # This replaces the :runner, :rake, and :command job types so that every
13
13
  # scheduled job automatically sends started/completed/failed heartbeats without
14
- # any per-job configuration.
14
+ # any per-job configuration. The monitor key is injected as a literal job
15
+ # option (:jobtick_key) rather than recomputed in shell, so it always matches
16
+ # the key Parsers::Whenever reads back out of the generated crontab.
15
17
  module WheneverSetup
18
+ TEMPLATES = {
19
+ runner: "cd :path && bundle exec rails runner ':task' :output",
20
+ rake: "cd :path && bundle exec rake :task :output",
21
+ command: ":task :output"
22
+ }.freeze
23
+
24
+ KEY_INJECTOR = Module.new do
25
+ TEMPLATES.each_key do |type|
26
+ define_method(type) do |task, *args|
27
+ opts = args[0].is_a?(Hash) ? args[0].dup : {}
28
+ opts[:jobtick_key] = JobTick::Parsers.whenever_key(task)
29
+ super(task, opts)
30
+ end
31
+ end
32
+ end
33
+ private_constant :KEY_INJECTOR
34
+
16
35
  def self.install!(schedule)
17
36
  endpoint = JobTick.config.endpoint
18
37
 
19
- schedule.job_type :runner, wrap(endpoint, "cd :path && bundle exec rails runner ':task' :output")
20
- schedule.job_type :rake, wrap(endpoint, "cd :path && bundle exec rake :task :output")
21
- schedule.job_type :command, wrap(endpoint, "cd :path && :task :output")
38
+ TEMPLATES.each { |type, inner| schedule.job_type(type, wrap(endpoint, inner)) }
39
+
40
+ # Prepended after job_type defines the singleton methods above, but order
41
+ # doesn't matter for resolution: a prepended module always sits ahead of
42
+ # the singleton class in the ancestor chain, so this always wins and
43
+ # `super` always reaches the method job_type just defined.
44
+ schedule.singleton_class.prepend(KEY_INJECTOR)
22
45
  end
23
46
 
24
47
  def self.wrap(endpoint, inner_cmd)
25
- # Shell equivalent of Parsers.slugify: downcase, collapse non-alnum runs to _, strip leading/trailing _.
26
- sed = "sed 's/[^a-z0-9][^a-z0-9]*/_/g' | sed 's/^_*//;s/_*$//'"
27
- slug = "$(printf '%s' ':task' | tr '[:upper:]' '[:lower:]' | #{sed})"
28
- key_assign = %(JOBTICK_KEY="whenever.#{slug}")
29
-
30
- "#{key_assign} ; " \
31
- "curl -sf \"#{endpoint}/ping/$JOBTICK_KEY/started\" ; " \
32
- "#{inner_cmd} && " \
33
- "curl -sf \"#{endpoint}/ping/$JOBTICK_KEY/completed\" || " \
34
- "curl -sf \"#{endpoint}/ping/$JOBTICK_KEY/failed\""
48
+ key_assign = "JOBTICK_KEY=:jobtick_key"
49
+ started = %(curl -sf --max-time 10 "#{endpoint}/ping/$JOBTICK_KEY/started")
50
+ completed = %(curl -sf --max-time 10 "#{endpoint}/ping/$JOBTICK_KEY/completed")
51
+ failed = %(curl -sf --max-time 10 "#{endpoint}/ping/$JOBTICK_KEY/failed")
52
+
53
+ "#{key_assign} ; #{started} ; " \
54
+ "if #{inner_cmd} ; then rc=0 ; #{completed} ; " \
55
+ "else rc=$? ; #{failed} ; fi ; exit $rc"
35
56
  end
36
57
  private_class_method :wrap
37
58
  end
data/lib/jobtick.rb CHANGED
@@ -5,6 +5,7 @@ require_relative "jobtick/version"
5
5
  require_relative "jobtick/configuration"
6
6
  require_relative "jobtick/client"
7
7
  require_relative "jobtick/monitor"
8
+ require_relative "jobtick/parsers"
8
9
  require_relative "jobtick/railtie" if defined?(Rails::Railtie)
9
10
 
10
11
  module JobTick
@@ -25,8 +26,20 @@ module JobTick
25
26
  @client ||= Client.new
26
27
  end
27
28
 
29
+ # The app's root directory, used to resolve schedule/recurring config
30
+ # files. Falls back to the process's working directory when Rails isn't
31
+ # loaded (or hasn't set a root yet) so discovery doesn't silently depend
32
+ # on the caller's cwd.
33
+ def root
34
+ if defined?(Rails) && Rails.respond_to?(:root) && Rails.root
35
+ Rails.root.to_s
36
+ else
37
+ Dir.pwd
38
+ end
39
+ end
40
+
28
41
  def logger
29
- defined?(Rails) ? Rails.logger : Logger.new($stdout)
42
+ (defined?(Rails) && Rails.logger) || (@fallback_logger ||= Logger.new($stdout))
30
43
  end
31
44
 
32
45
  def monitor_map
data/sig/jobtick.rbs CHANGED
@@ -1,4 +1,4 @@
1
- module Jobtick
1
+ module JobTick
2
2
  VERSION: String
3
3
  # See the writing guide of rbs: https://github.com/ruby/rbs#guides
4
4
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: jobtick
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.0
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Clearstack Labs
@@ -42,6 +42,7 @@ files:
42
42
  - lib/jobtick/hooks/active_job.rb
43
43
  - lib/jobtick/middleware/sidekiq.rb
44
44
  - lib/jobtick/monitor.rb
45
+ - lib/jobtick/parsers.rb
45
46
  - lib/jobtick/parsers/sidekiq.rb
46
47
  - lib/jobtick/parsers/solid_queue.rb
47
48
  - lib/jobtick/parsers/whenever.rb