cronwatch 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +7 -7
- data/lib/cronwatch/alerts/provider.rb +3 -2
- data/lib/cronwatch/client.rb +4 -4
- data/lib/cronwatch/duration.rb +17 -0
- data/lib/cronwatch/evaluate.rb +28 -0
- data/lib/cronwatch/format.rb +4 -1
- data/lib/cronwatch/pg_cron.rb +1 -1
- data/lib/cronwatch/schedule.rb +53 -9
- data/lib/cronwatch/stores/active_record.rb +14 -2
- data/lib/cronwatch/stores/memory.rb +1 -1
- data/lib/cronwatch/triage/anthropic.rb +7 -2
- data/lib/cronwatch/version.rb +1 -1
- data/lib/cronwatch/web/html.rb +3 -1
- data/lib/cronwatch/web/timeline.rb +9 -2
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b8af9eac18f781dcf622c274e6f3e44370a4373d0721215ba2928f133d3b36bd
|
|
4
|
+
data.tar.gz: 20577e3909253de5b132c4fba48654b9176a593efc8c5c86caf85845fe16c1c7
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 3f29e87d9cd6b09776a1c744c147b29ac20f6311072e2f7eb7d16b5ba4eaf4f29b3cbb204567badb4f655af12224fb308923a7e6b1f8e4189fd9b42db2df2079
|
|
7
|
+
data.tar.gz: 8c3e6d1326f545a5fa453235a385814351c0575927a024ed0fc341cb6a2e639e996856ec4ca96eb2f4e674b0d90a70789a431514b9c4e7f140cb917efd12de0d
|
data/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Cron and scheduled-job monitoring that lives inside your Ruby or Rails app. Wrap a job once; every run is recorded in a database you already have, and you are told when a run is missed, fails, gets stuck, runs slow or goes over budget. No server to run, no account to make.
|
|
4
4
|
|
|
5
|
-
This is the Ruby port of [`@cronwatch/sdk`](https://www.npmjs.com/package/@cronwatch/sdk): the same rules, the same alert text and the same stored rows, so a Ruby process
|
|
5
|
+
This is the Ruby port of [`@cronwatch/sdk`](https://www.npmjs.com/package/@cronwatch/sdk): the same rules, the same alert text and the same stored rows, so a Ruby process can share one database with a Node, Python, PHP, Go or Rust process, and [`@cronwatch/mcp`](https://www.npmjs.com/package/@cronwatch/mcp) works against any of them.
|
|
6
6
|
|
|
7
7
|
Docs: [cronwatch.dev/docs/rails](https://cronwatch.dev/docs/rails/) and [cronwatch.dev/docs/ruby](https://cronwatch.dev/docs/ruby/)
|
|
8
8
|
|
|
@@ -20,13 +20,13 @@ Ruby 3.2 or newer; the Rails integration is tested on Rails 7.2, 8.0 and 8.1. Th
|
|
|
20
20
|
| `cronwatch` | the client, the memory store, and the Slack, Discord, webhook, console, email (Resend, Postmark, SendGrid, Mailgun, SES), Twilio and error tracker (Sentry, Honeybadger, Datadog, Rollbar, Bugsnag, New Relic) channels | |
|
|
21
21
|
| `cronwatch/pg_cron` | `Cronwatch::Sources::PgCron`, which watches pg_cron jobs through an ActiveRecord or pg connection | |
|
|
22
22
|
| `cronwatch/active_record` | the ActiveRecord store | `activerecord` |
|
|
23
|
-
| `cronwatch/rails` | the Railtie, `Cronwatch::ActiveJob`, `Cronwatch::CheckJob`, the `cronwatch:check` task, the install generator | `railties`, `activejob` |
|
|
23
|
+
| `cronwatch/rails` | the Railtie, `Cronwatch::ActiveJob`, `Cronwatch::CheckJob`, `Cronwatch::Web`, the `cronwatch:check` task, the install generator | `railties`, `activejob` |
|
|
24
24
|
| `cronwatch/sidekiq` | `Cronwatch::Sidekiq` for `Sidekiq::Job` classes, its server middleware, `Cronwatch::Sidekiq::CheckWorker` | `sidekiq` 7 or newer |
|
|
25
25
|
| `cronwatch/scheduler` | schedules read from Solid Queue's or sidekiq-cron's config, `Cronwatch.declare_from_scheduler!` | |
|
|
26
26
|
| `cronwatch/web` | the dashboard and JSON API as a Rack app | `rack` |
|
|
27
27
|
| `cronwatch/triage/anthropic` | Claude triage | `anthropic` |
|
|
28
28
|
|
|
29
|
-
In a Rails app, `gem "cronwatch"` also loads `cronwatch/rails` and `cronwatch/scheduler` (Bundler requires it after Rails), `cronwatch/sidekiq` when Sidekiq is in the bundle, and the ActiveRecord store on first use. `cronwatch/
|
|
29
|
+
In a Rails app, `gem "cronwatch"` also loads `cronwatch/rails`, `cronwatch/web` and `cronwatch/scheduler` (Bundler requires it after Rails), `cronwatch/sidekiq` when Sidekiq is in the bundle, and the ActiveRecord store on first use. `cronwatch/triage/anthropic` is always required by hand, and so is `cronwatch/web` outside Rails.
|
|
30
30
|
|
|
31
31
|
## Plain Ruby
|
|
32
32
|
|
|
@@ -64,7 +64,7 @@ run.finish # or run.fail(error); finish("text") and fin
|
|
|
64
64
|
|
|
65
65
|
Neither raises for the store: failures go to `on_error`, and a finish of a run already finished, not found, or of another job records nothing and returns nil. A run is judged once, however many processes finish it at once, and a store that fails during `finish` leaves the handle active to finish again. A run never finished is marked stuck after the job's `timeout`.
|
|
66
66
|
|
|
67
|
-
Client options: `store`, `alerts`, `triage`, `cron_secret`, `retention` (default `"30d"`), `defaults`, `redact` (default: blank values that look like secrets; `false` keeps output as logged, or pass a callable), `deliver` (`:now` by default; `:check` queues alerts for another process's check to send, for a worker that cannot reach Slack), `on_error` and `now`. Methods: `job`, `run`, `resume_run`, `check`, `start`/`stop`, `silence(name, for: "2h")`/`unsilence`, `forget`, `jobs`, `jobs_with_runs`, `job_summary`, `runs`, `get_run`, `defined_jobs`, `close`. [cronwatch.dev/docs/ruby](https://cronwatch.dev/docs/ruby/#api) has each one.
|
|
67
|
+
Client options: `store`, `alerts`, `triage`, `cron_secret`, `retention` (default `"30d"`), `defaults`, `sources` (where runs the process does not wrap come from, such as pg_cron), `redact` (default: blank values that look like secrets; `false` keeps output as logged, or pass a callable), `deliver` (`:now` by default; `:check` queues alerts for another process's check to send, for a worker that cannot reach Slack), `on_error` and `now`. Methods: `job`, `run`, `resume_run`, `record_run`, `check`, `start`/`stop`, `silence(name, for: "2h")`/`unsilence`, `forget`, `jobs`, `jobs_with_runs`, `job_summary`, `runs`, `get_run`, `defined_jobs`, `close`. [cronwatch.dev/docs/ruby](https://cronwatch.dev/docs/ruby/#api) has each one.
|
|
68
68
|
|
|
69
69
|
## Rails
|
|
70
70
|
|
|
@@ -130,7 +130,7 @@ reads the class's one entry in Solid Queue's `config/recurring.yml` (the section
|
|
|
130
130
|
|
|
131
131
|
### Scheduling the check
|
|
132
132
|
|
|
133
|
-
Failures are caught as they happen, but a run that never started or never finished can only be noticed by looking. `Cronwatch::CheckJob` looks: it loads `app/jobs` (and `app/workers`) when the app does not eager load, declares every monitored job and runs the check. Run it every five minutes.
|
|
133
|
+
Failures are caught as they happen, but a run that never started or never finished can only be noticed by looking. `Cronwatch::CheckJob` looks: it loads `app/jobs` (and `app/workers` and `app/sidekiq`) when the app does not eager load, declares every monitored job and runs the check. Run it every five minutes.
|
|
134
134
|
|
|
135
135
|
Solid Queue:
|
|
136
136
|
|
|
@@ -168,7 +168,7 @@ Rails.application.routes.draw do
|
|
|
168
168
|
end
|
|
169
169
|
```
|
|
170
170
|
|
|
171
|
-
Set `CRONWATCH_TOKEN` to a long random string and open `/cronwatch?token=<it>` once; the browser keeps a cookie. Without a token, while `RAILS_ENV` or `RACK_ENV` is `development` or `test`, it makes a token of its own and prints a sign-in link to standard output on its first request (open it once; unless `origin:` is set or the request's host is loopback, the link leaves the host out, since a client chooses it); anywhere else it answers 503. To rely on the app's own sign in, mount it behind that (Devise's `authenticate` block, or a routing constraint) and pass `token: nil`. The URLs, JSON shapes, headers and CSRF rules are the SDK's, so the MCP server reads it unchanged. The request's origin, which the CSRF check compares against, is what Rack reports and so already follows `X-Forwarded-Host` and `X-Forwarded-Proto`; pass `origin: "https://app.example.com"` to pin it instead. `GET /cronwatch/api/check` with a bearer (the token or `CRON_SECRET`) runs the check, for an outside cron.
|
|
171
|
+
Set `CRONWATCH_TOKEN` to a long random string and open `/cronwatch?token=<it>` once; the browser keeps a cookie. Without a token, while the environment (`Rails.env`, else `RAILS_ENV` or `RACK_ENV`) is `development` or `test`, it makes a token of its own and prints a sign-in link to standard output on its first request (open it once; unless `origin:` is set or the request's host is loopback, the link leaves the host out, since a client chooses it); anywhere else it answers 503. To rely on the app's own sign in, mount it behind that (Devise's `authenticate` block, or a routing constraint) and pass `token: nil`. The URLs, JSON shapes, headers and CSRF rules are the SDK's, so the MCP server reads it unchanged. The request's origin, which the CSRF check compares against, is what Rack reports and so already follows `X-Forwarded-Host` and `X-Forwarded-Proto`; pass `origin: "https://app.example.com"` to pin it instead. `GET /cronwatch/api/check` with a bearer (the token or `CRON_SECRET`) runs the check, for an outside cron.
|
|
172
172
|
|
|
173
173
|
There is no `handler()` as in the TypeScript SDK: for a job triggered over HTTP, wrap the controller action's body in `CW.job(...).run` (declared once, at boot) and check the bearer in the controller.
|
|
174
174
|
|
|
@@ -225,7 +225,7 @@ Adds two to four sentences from Claude (likely cause, first thing to check) to e
|
|
|
225
225
|
|
|
226
226
|
## Sharing a database with a Node app
|
|
227
227
|
|
|
228
|
-
The ActiveRecord store writes the same three tables as `@cronwatch/sdk/postgres` and `@cronwatch/sdk/sqlite`: same names, columns and indexes, epoch milliseconds in the time columns, the SDK's camelCase JSON in the JSON columns. [`test/active_record/node_compat_test.rb`](https://github.com/phillips-jon/cronwatch/blob/main/packages/ruby/test/active_record/node_compat_test.rb) runs the SDK's stores in Node beside this one, on SQLite and Postgres, and checks that each reads what the other wrote, that the tables are the same whoever creates them, and that the rows are the same bytes. Use the same prefix on both sides and give each job a name only one side uses. Then one dashboard, Rails or Node, shows every job, and one MCP server reads them all.
|
|
228
|
+
The ActiveRecord store writes the same three tables as `@cronwatch/sdk/postgres` and `@cronwatch/sdk/sqlite`: same names, columns and indexes, epoch milliseconds in the time columns, the SDK's camelCase JSON in the JSON columns. [`test/active_record/node_compat_test.rb`](https://github.com/phillips-jon/cronwatch/blob/main/packages/ruby/test/active_record/node_compat_test.rb) runs the SDK's stores in Node beside this one, on SQLite and Postgres, and checks that each reads what the other wrote, that the tables are the same whoever creates them, and that the rows are the same bytes. Use the same prefix on both sides and give each job a name only one side uses. Then one dashboard, Rails or Node, shows every job, and one MCP server reads them all. The Python, PHP, Go and Rust ports keep the same tables too, so a process in any of them can share the database the same way.
|
|
229
229
|
|
|
230
230
|
## Kept in step with the TypeScript SDK
|
|
231
231
|
|
|
@@ -93,13 +93,14 @@ module Cronwatch
|
|
|
93
93
|
JS.head16(text, max)
|
|
94
94
|
end
|
|
95
95
|
|
|
96
|
-
# The run fields worth attaching to a tracker event.
|
|
96
|
+
# The run fields worth attaching to a tracker event. A start before the
|
|
97
|
+
# year 1 or after 9999 is nil.
|
|
97
98
|
def run_summary(alert)
|
|
98
99
|
run = alert.run
|
|
99
100
|
return nil unless run
|
|
100
101
|
|
|
101
102
|
{
|
|
102
|
-
"id" => run.id, "status" => run.status.to_s, "startedAt" =>
|
|
103
|
+
"id" => run.id, "status" => run.status.to_s, "startedAt" => Duration.iso_time(run.started_at),
|
|
103
104
|
"durationMs" => run.duration_ms, "trigger" => run.trigger,
|
|
104
105
|
}
|
|
105
106
|
end
|
data/lib/cronwatch/client.rb
CHANGED
|
@@ -229,7 +229,7 @@ module Cronwatch
|
|
|
229
229
|
|
|
230
230
|
finished_at = now
|
|
231
231
|
run.finished_at = finished_at
|
|
232
|
-
run.duration_ms =
|
|
232
|
+
run.duration_ms = Evaluate.run_duration(started_at, finished_at)
|
|
233
233
|
run.metrics = recorder.metrics
|
|
234
234
|
returned = result.is_a?(String) ? Output.utf8(result) : nil
|
|
235
235
|
run.output = recorder.output || (returned && Output.cap(returned))
|
|
@@ -687,7 +687,7 @@ module Cronwatch
|
|
|
687
687
|
state, result = yield(current)
|
|
688
688
|
break [current, result] if same_state?(state, current)
|
|
689
689
|
|
|
690
|
-
version = current
|
|
690
|
+
version = Evaluate.state_version(current)
|
|
691
691
|
following = state.dup
|
|
692
692
|
following.version = version + 1
|
|
693
693
|
break [following, result] if write_state(following, version)
|
|
@@ -938,7 +938,7 @@ module Cronwatch
|
|
|
938
938
|
run = from.dup
|
|
939
939
|
run.status = :running
|
|
940
940
|
run.finished_at = finished_at
|
|
941
|
-
run.duration_ms =
|
|
941
|
+
run.duration_ms = Evaluate.run_duration(from.started_at, finished_at)
|
|
942
942
|
run.error = nil
|
|
943
943
|
run.output = join_output(from.output, added)
|
|
944
944
|
run.metrics = (from.metrics || {}).merge(recorder.metrics)
|
|
@@ -1063,7 +1063,7 @@ module Cronwatch
|
|
|
1063
1063
|
timeout = Evaluate.timeout_ms(definition)
|
|
1064
1064
|
run.status = :timeout
|
|
1065
1065
|
run.finished_at = at
|
|
1066
|
-
run.duration_ms =
|
|
1066
|
+
run.duration_ms = Evaluate.run_duration(run.started_at, at)
|
|
1067
1067
|
run.error = "Still running after #{Duration.format(timeout)}; marked as timed out"
|
|
1068
1068
|
# Only over a row still running: a finish that landed meanwhile wins.
|
|
1069
1069
|
next unless write_run_if(run, [:running])
|
data/lib/cronwatch/duration.rb
CHANGED
|
@@ -83,5 +83,22 @@ module Cronwatch
|
|
|
83
83
|
text = format(abs)
|
|
84
84
|
diff.negative? ? "#{text} ago" : "in #{text}"
|
|
85
85
|
end
|
|
86
|
+
|
|
87
|
+
# The first millisecond written as a date: 0001-01-01T00:00:00.000Z.
|
|
88
|
+
FIRST_DATE_MS = -62_135_596_800_000
|
|
89
|
+
# The last millisecond written as a date: 9999-12-31T23:59:59.999Z.
|
|
90
|
+
LAST_DATE_MS = 253_402_300_799_999
|
|
91
|
+
|
|
92
|
+
# "2026-01-05T09:30:00.000Z", or nil for a time before the year 1 or
|
|
93
|
+
# after 9999. A start read from another process's row, or a damaged one,
|
|
94
|
+
# can be any number; outside those years it is not written as a date.
|
|
95
|
+
def iso_time(at)
|
|
96
|
+
at >= FIRST_DATE_MS && at <= LAST_DATE_MS ? JS.iso(at) : nil
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
# The words that stand in for a time iso_time does not write.
|
|
100
|
+
def beyond_dates(at)
|
|
101
|
+
at > LAST_DATE_MS ? "after 9999-12-31 23:59:59 UTC" : "before 0001-01-01 00:00:00 UTC"
|
|
102
|
+
end
|
|
86
103
|
end
|
|
87
104
|
end
|
data/lib/cronwatch/evaluate.rb
CHANGED
|
@@ -18,8 +18,36 @@ module Cronwatch
|
|
|
18
18
|
CheckEvaluation = Struct.new(:state, :alerts, :next_expected_at, :due_at, keyword_init: true)
|
|
19
19
|
SlowThreshold = Struct.new(:threshold_ms, :basis, keyword_init: true)
|
|
20
20
|
|
|
21
|
+
# The longest duration written: the largest integer JavaScript holds
|
|
22
|
+
# exactly, which every port and store reads back unchanged.
|
|
23
|
+
MAX_DURATION_MS = 9_007_199_254_740_991
|
|
24
|
+
|
|
21
25
|
module_function
|
|
22
26
|
|
|
27
|
+
# How long a run took, from `started_at` to `finished_at`: 0 when it
|
|
28
|
+
# started later, and never more than MAX_DURATION_MS. A foreign row's
|
|
29
|
+
# start near a 64-bit limit must not make a duration no store can write.
|
|
30
|
+
def run_duration(started_at, finished_at)
|
|
31
|
+
ms = finished_at - started_at
|
|
32
|
+
return 0 unless ms.is_a?(Numeric) && ms.positive?
|
|
33
|
+
|
|
34
|
+
ms = ms.to_i if ms.is_a?(Float) && ms.finite? && ms == ms.floor
|
|
35
|
+
[ms, MAX_DURATION_MS].min
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
# The version a stored state counts as for compare_and_set_state: its
|
|
39
|
+
# `version` when that is a whole number from 0 to MAX_DURATION_MS
|
|
40
|
+
# (2**53 - 1), else 0, as when it is absent. The SQL stores read it the
|
|
41
|
+
# same way, so a foreign row's 1.5, "x" or -1 is written over by the next
|
|
42
|
+
# update instead of refusing every compare-and-set of its job for good.
|
|
43
|
+
def state_version(state)
|
|
44
|
+
version = state&.version
|
|
45
|
+
return 0 unless version.is_a?(Integer) || (version.is_a?(Float) && version.finite? && version == version.floor)
|
|
46
|
+
return 0 unless version >= 0 && version <= MAX_DURATION_MS
|
|
47
|
+
|
|
48
|
+
version.to_i
|
|
49
|
+
end
|
|
50
|
+
|
|
23
51
|
def empty_state(job)
|
|
24
52
|
JobState.new(job: job, open: {}, consecutive_failures: 0, silenced_until: nil, last_alert_at: nil,
|
|
25
53
|
pending_recovery: [], undelivered: [])
|
data/lib/cronwatch/format.rb
CHANGED
|
@@ -9,7 +9,10 @@ module Cronwatch
|
|
|
9
9
|
def at_time(at, now)
|
|
10
10
|
return "never" if at.nil?
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
iso = Duration.iso_time(at)
|
|
13
|
+
return Duration.beyond_dates(at) if iso.nil?
|
|
14
|
+
|
|
15
|
+
"#{iso.tr("T", " ")[0, 19]} UTC (#{Duration.relative(at, now)})"
|
|
13
16
|
end
|
|
14
17
|
|
|
15
18
|
def first_lines(text, n)
|
data/lib/cronwatch/pg_cron.rb
CHANGED
|
@@ -161,7 +161,7 @@ module Cronwatch
|
|
|
161
161
|
finish = done ? [started_at, finished_at || started_at].max : nil
|
|
162
162
|
Run.new(
|
|
163
163
|
id: "#{id_prefix}#{row["runid"]}", job: job, status: status, started_at: started_at, finished_at: finish,
|
|
164
|
-
duration_ms: finish.nil? ? nil :
|
|
164
|
+
duration_ms: finish.nil? ? nil : Evaluate.run_duration(started_at, finish),
|
|
165
165
|
error: status == :failed ? (message || "pg_cron reported the run as failed") : nil,
|
|
166
166
|
output: status == :ok ? message : nil, metrics: {}, trigger: "pg_cron",
|
|
167
167
|
)
|
data/lib/cronwatch/schedule.rb
CHANGED
|
@@ -15,6 +15,15 @@ module Cronwatch
|
|
|
15
15
|
# How early a run may start and still count for the fire it was meant for.
|
|
16
16
|
EARLY_SLACK_MS = 60_000
|
|
17
17
|
|
|
18
|
+
# Four hundred Gregorian years: 146,097 days, a whole number of weeks,
|
|
19
|
+
# after which the calendar repeats date for date and weekday for weekday.
|
|
20
|
+
CYCLE_MS = 146_097 * 86_400_000
|
|
21
|
+
# The years the walker searches in, as croner does: a time before 400 is
|
|
22
|
+
# asked a cycle or more later (croner misreads a year below 100), and one
|
|
23
|
+
# from 2800 a cycle or more earlier (croner finds no fire past 3000).
|
|
24
|
+
WALK_FIRST_MS = -49_544_438_400_000 # 0400-01-01T00:00:00Z
|
|
25
|
+
WALK_LAST_MS = 26_192_246_400_000 # 2800-01-01T00:00:00Z
|
|
26
|
+
|
|
18
27
|
Parsed = Struct.new(:kind, :source, :timezone, :every_ms, keyword_init: true) do
|
|
19
28
|
include Serializable
|
|
20
29
|
|
|
@@ -80,12 +89,15 @@ module Cronwatch
|
|
|
80
89
|
def fire_after(parsed, from)
|
|
81
90
|
raise ArgumentError, "schedule \"#{parsed.source}\" was not made by Schedule.parse" unless parsed.pattern
|
|
82
91
|
|
|
83
|
-
|
|
92
|
+
start = count_from(from)
|
|
93
|
+
return nil if start.nil?
|
|
94
|
+
|
|
95
|
+
probe = start
|
|
84
96
|
4.times do
|
|
85
97
|
runs = next_runs(parsed, 8, probe)
|
|
86
98
|
return nil if runs.empty?
|
|
87
99
|
|
|
88
|
-
found = runs.find { |t| t >
|
|
100
|
+
found = runs.find { |t| t > start }
|
|
89
101
|
return found if found
|
|
90
102
|
|
|
91
103
|
probe += 3_600_000
|
|
@@ -101,8 +113,11 @@ module Cronwatch
|
|
|
101
113
|
raise ArgumentError, "schedule \"#{parsed.source}\" was not made by Schedule.parse" unless parsed.pattern
|
|
102
114
|
|
|
103
115
|
out = []
|
|
104
|
-
|
|
105
|
-
|
|
116
|
+
start = count_from(from)
|
|
117
|
+
return out if start.nil?
|
|
118
|
+
|
|
119
|
+
probe = start
|
|
120
|
+
last = start
|
|
106
121
|
1000.times do
|
|
107
122
|
batch = next_runs(parsed, [limit + 1 - out.length, 24].min, probe)
|
|
108
123
|
return out if batch.empty?
|
|
@@ -121,15 +136,39 @@ module Cronwatch
|
|
|
121
136
|
out
|
|
122
137
|
end
|
|
123
138
|
|
|
124
|
-
#
|
|
139
|
+
# A stored time as a cron's fires are counted from it. A start read from
|
|
140
|
+
# a foreign or damaged row can be any number: one before the year 1
|
|
141
|
+
# counts from just before its first millisecond, so the first fire of the
|
|
142
|
+
# year 1 is the next one, and one at or after the last millisecond of 9999
|
|
143
|
+
# has no fire after it at all (nil). Ported from countFrom in schedule.ts.
|
|
144
|
+
def count_from(from)
|
|
145
|
+
return nil if from >= Duration::LAST_DATE_MS
|
|
146
|
+
|
|
147
|
+
from >= Duration::FIRST_DATE_MS ? from : Duration::FIRST_DATE_MS - 1
|
|
148
|
+
end
|
|
149
|
+
|
|
150
|
+
# Up to `count` fires, each found from the one before, as croner's
|
|
151
|
+
# nextRuns, and none after 9999. A time outside the years the walker
|
|
152
|
+
# searches is moved by whole 400-year cycles into them, and its fires
|
|
153
|
+
# moved back, as runsAfter in schedule.ts asks croner: a time before 400
|
|
154
|
+
# goes forward, into the same local mean time every zone kept then, and
|
|
155
|
+
# one from 2800 goes back, to where the zone's present rules already hold.
|
|
125
156
|
def next_runs(parsed, count, from)
|
|
157
|
+
from = count_from(from)
|
|
158
|
+
return [] if from.nil?
|
|
159
|
+
|
|
160
|
+
shift =
|
|
161
|
+
if from < WALK_FIRST_MS then -((from - WALK_FIRST_MS).div(CYCLE_MS)) * CYCLE_MS
|
|
162
|
+
elsif from >= WALK_LAST_MS then -((from - WALK_LAST_MS).div(CYCLE_MS) + 1) * CYCLE_MS
|
|
163
|
+
else 0
|
|
164
|
+
end
|
|
126
165
|
runs = []
|
|
127
|
-
at = from
|
|
166
|
+
at = from + shift
|
|
128
167
|
count.times do
|
|
129
168
|
at = parsed.pattern.next_after(at, parsed.timezone)
|
|
130
|
-
break if at.nil?
|
|
169
|
+
break if at.nil? || at - shift > Duration::LAST_DATE_MS
|
|
131
170
|
|
|
132
|
-
runs << at
|
|
171
|
+
runs << (at - shift)
|
|
133
172
|
end
|
|
134
173
|
runs
|
|
135
174
|
end
|
|
@@ -159,7 +198,9 @@ module Cronwatch
|
|
|
159
198
|
due_at.nil? ? nil : Expectation.new(due_at: due_at, deadline: due_at + grace_ms)
|
|
160
199
|
end
|
|
161
200
|
|
|
162
|
-
# The first fire that a run starting at `started_at` does not cover.
|
|
201
|
+
# The first fire that a run starting at `started_at` does not cover. A
|
|
202
|
+
# start before the year 1 covers none of them, so the first fire of the
|
|
203
|
+
# year 1 is due; after 9999 there is none (see count_from).
|
|
163
204
|
def due_after_run(parsed, started_at)
|
|
164
205
|
# A fire at or before the start is covered by the run itself.
|
|
165
206
|
following_fire = fire_after(parsed, started_at)
|
|
@@ -189,6 +230,9 @@ module Cronwatch
|
|
|
189
230
|
# ran early by up to an hour.
|
|
190
231
|
def in_spring_forward_gap?(parsed, started_at, fire_at)
|
|
191
232
|
lookback = 3 * 3_600_000
|
|
233
|
+
# Every zone kept its local mean time, with no clock change, in the year 1.
|
|
234
|
+
return false if fire_at - lookback < Duration::FIRST_DATE_MS
|
|
235
|
+
|
|
192
236
|
after = utc_offset(fire_at, parsed.timezone)
|
|
193
237
|
before = utc_offset(fire_at - lookback, parsed.timezone)
|
|
194
238
|
gap = after - before
|
|
@@ -344,8 +344,20 @@ module Cronwatch
|
|
|
344
344
|
seq = pg ? "seq" : "rowid"
|
|
345
345
|
# Byte order on both, so names sort the same whatever the database's collation.
|
|
346
346
|
by_name = pg ? 'name COLLATE "C"' : "name"
|
|
347
|
-
# The version inside a state's JSON,
|
|
348
|
-
|
|
347
|
+
# The version inside a state's JSON, as Evaluate.state_version reads
|
|
348
|
+
# it: a whole number from 0 to 2**53 - 1, else 0 (none, or a foreign
|
|
349
|
+
# row's 1.5 or "x", which must neither fail the statement nor refuse
|
|
350
|
+
# every write for good). Each CASE tests the JSON type before any cast.
|
|
351
|
+
version = lambda do |column|
|
|
352
|
+
if pg
|
|
353
|
+
v = "(#{column}->>'version')::numeric"
|
|
354
|
+
next "CASE WHEN jsonb_typeof(#{column}->'version') <> 'number' THEN 0 " \
|
|
355
|
+
"WHEN #{v} % 1 = 0 AND #{v} BETWEEN 0 AND 9007199254740991 THEN #{v}::bigint ELSE 0 END"
|
|
356
|
+
end
|
|
357
|
+
v = "json_extract(#{column}, '$.version')"
|
|
358
|
+
"CASE WHEN json_type(#{column}, '$.version') NOT IN ('integer', 'real') THEN 0 " \
|
|
359
|
+
"WHEN #{v} = CAST(#{v} AS INTEGER) AND #{v} BETWEEN 0 AND 9007199254740991 THEN CAST(#{v} AS INTEGER) ELSE 0 END"
|
|
360
|
+
end
|
|
349
361
|
sql = {
|
|
350
362
|
upsert_job: "INSERT INTO #{p}jobs (name, definition, created_at, updated_at) VALUES (?, ?, ?, ?)\n " \
|
|
351
363
|
"ON CONFLICT (name) DO UPDATE SET definition = excluded.definition, updated_at = excluded.updated_at",
|
|
@@ -134,7 +134,7 @@ module Cronwatch
|
|
|
134
134
|
# wrote. See JobState#version.
|
|
135
135
|
def compare_and_set_state(state, expected_version)
|
|
136
136
|
sync do
|
|
137
|
-
next false unless (@states[state.job]
|
|
137
|
+
next false unless Evaluate.state_version(@states[state.job]) == expected_version
|
|
138
138
|
|
|
139
139
|
@states[state.job] = clone(state, JobState)
|
|
140
140
|
true
|
|
@@ -92,6 +92,11 @@ module Cronwatch
|
|
|
92
92
|
"<job_data>\n#{text.gsub(TAG, "<_job_data")}\n</job_data>"
|
|
93
93
|
end
|
|
94
94
|
|
|
95
|
+
# "2026-01-05T09:30:00.000Z", or the words for a time before the year 1 or after 9999.
|
|
96
|
+
def self.stamp(at)
|
|
97
|
+
Duration.iso_time(at) || Duration.beyond_dates(at)
|
|
98
|
+
end
|
|
99
|
+
|
|
95
100
|
# The prompt: the alert, the definition, the triggering run and up to five earlier ones.
|
|
96
101
|
def self.describe(triage_context)
|
|
97
102
|
alert = triage_context.alert
|
|
@@ -104,7 +109,7 @@ module Cronwatch
|
|
|
104
109
|
lines << "Job definition: #{JS.json(definition.respond_to?(:to_h) ? definition.to_h : definition)}"
|
|
105
110
|
if run
|
|
106
111
|
lines << ""
|
|
107
|
-
lines << "Triggering run: status #{run.status}, started #{
|
|
112
|
+
lines << "Triggering run: status #{run.status}, started #{stamp(run.started_at)}, duration #{duration(run)}, trigger #{run.trigger}"
|
|
108
113
|
lines << "Metrics: #{JS.json(run.metrics)}" if run.metrics && !run.metrics.empty?
|
|
109
114
|
lines << "Error:\n#{data(JS.head16(run.error, 3000))}" if present?(run.error)
|
|
110
115
|
lines << "Output (tail):\n#{data(JS.tail16(run.output, 3000))}" if present?(run.output)
|
|
@@ -116,7 +121,7 @@ module Cronwatch
|
|
|
116
121
|
earlier.each do |r|
|
|
117
122
|
error = present?(r.error) ? ", error: #{data(JS.head16(r.error.split("\n", -1).first.to_s, 160))}" : ""
|
|
118
123
|
metrics = r.metrics && !r.metrics.empty? ? ", metrics #{JS.json(r.metrics)}" : ""
|
|
119
|
-
lines << "- #{r.status}, #{
|
|
124
|
+
lines << "- #{r.status}, #{stamp(r.started_at)}, #{duration(r)}#{error}#{metrics}"
|
|
120
125
|
end
|
|
121
126
|
end
|
|
122
127
|
lines.join("\n")
|
data/lib/cronwatch/version.rb
CHANGED
data/lib/cronwatch/web/html.rb
CHANGED
|
@@ -329,7 +329,9 @@ module Cronwatch
|
|
|
329
329
|
def stamp(at, now)
|
|
330
330
|
return %(<span class="muted">never</span>) if at.nil?
|
|
331
331
|
|
|
332
|
-
iso =
|
|
332
|
+
iso = Duration.iso_time(at.to_i)
|
|
333
|
+
return %(<span class="nowrap">#{Duration.beyond_dates(at.to_i)}</span>) if iso.nil?
|
|
334
|
+
|
|
333
335
|
%(<time class="nowrap" datetime="#{iso}" title="#{iso.sub("T", " ")[0, 19]} UTC">#{h(Duration.relative(at, now))}</time>)
|
|
334
336
|
end
|
|
335
337
|
|
|
@@ -86,12 +86,19 @@ module Cronwatch
|
|
|
86
86
|
"#{WEEKDAYS[d.wday]} #{d.day} #{MONTHS[d.month - 1]}"
|
|
87
87
|
end
|
|
88
88
|
|
|
89
|
-
# "22:42" on the same UTC day as `now`, otherwise "25 Sep 22:42"
|
|
89
|
+
# "22:42" on the same UTC day as `now`, otherwise "25 Sep 22:42", and
|
|
90
|
+
# "1 Jan 0001 02:00" in another UTC year. A time before the year 1 or
|
|
91
|
+
# after 9999 (a start read from a foreign or damaged row) is "before 1
|
|
92
|
+
# Jan 0001 00:00" or "after 31 Dec 9999 23:59".
|
|
90
93
|
def when_at(t, now)
|
|
94
|
+
return "after 31 Dec 9999 23:59" if t > Duration::LAST_DATE_MS
|
|
95
|
+
return "before 1 Jan 0001 00:00" if t < Duration::FIRST_DATE_MS
|
|
96
|
+
|
|
91
97
|
return clock(t) if t.div(DAY) == now.div(DAY)
|
|
92
98
|
|
|
93
99
|
d = utc(t)
|
|
94
|
-
|
|
100
|
+
year = d.year == utc(now).year ? "" : Kernel.format(" %04d", d.year)
|
|
101
|
+
"#{d.day} #{MONTHS[d.month - 1]}#{year} #{clock(t)}"
|
|
95
102
|
end
|
|
96
103
|
|
|
97
104
|
# The job's schedule, parsed, or nil when it has none or it no longer parses.
|