cronwatch 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE +21 -0
- data/README.md +268 -0
- data/lib/cronwatch/abort_signal.rb +45 -0
- data/lib/cronwatch/active_record.rb +11 -0
- data/lib/cronwatch/alerts/console.rb +22 -0
- data/lib/cronwatch/alerts/custom.rb +26 -0
- data/lib/cronwatch/alerts/discord.rb +52 -0
- data/lib/cronwatch/alerts/slack.rb +58 -0
- data/lib/cronwatch/alerts/webhook.rb +46 -0
- data/lib/cronwatch/client.rb +925 -0
- data/lib/cronwatch/cron_pattern.rb +277 -0
- data/lib/cronwatch/duration.rb +77 -0
- data/lib/cronwatch/environment.rb +29 -0
- data/lib/cronwatch/evaluate.rb +345 -0
- data/lib/cronwatch/flight.rb +42 -0
- data/lib/cronwatch/format.rb +89 -0
- data/lib/cronwatch/http.rb +52 -0
- data/lib/cronwatch/job.rb +144 -0
- data/lib/cronwatch/js.rb +188 -0
- data/lib/cronwatch/monitored.rb +259 -0
- data/lib/cronwatch/output.rb +199 -0
- data/lib/cronwatch/rails/active_job.rb +55 -0
- data/lib/cronwatch/rails/check_job.rb +32 -0
- data/lib/cronwatch/rails/railtie.rb +37 -0
- data/lib/cronwatch/rails/tasks.rb +12 -0
- data/lib/cronwatch/rails.rb +35 -0
- data/lib/cronwatch/schedule.rb +191 -0
- data/lib/cronwatch/scheduler.rb +763 -0
- data/lib/cronwatch/serialize.rb +51 -0
- data/lib/cronwatch/sidekiq.rb +129 -0
- data/lib/cronwatch/stats.rb +23 -0
- data/lib/cronwatch/stores/active_record.rb +397 -0
- data/lib/cronwatch/stores/memory.rb +163 -0
- data/lib/cronwatch/ticker.rb +59 -0
- data/lib/cronwatch/triage/anthropic.rb +134 -0
- data/lib/cronwatch/types.rb +341 -0
- data/lib/cronwatch/version.rb +6 -0
- data/lib/cronwatch/walker.rb +137 -0
- data/lib/cronwatch/web/app.rb +484 -0
- data/lib/cronwatch/web/html.rb +314 -0
- data/lib/cronwatch/web.rb +17 -0
- data/lib/cronwatch/zone.rb +72 -0
- data/lib/cronwatch.rb +96 -0
- data/lib/generators/cronwatch/install/install_generator.rb +176 -0
- metadata +104 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: cc6a450f111f1a0f65f0e92656784a215a37af11da63a723d76074f59fe21719
|
|
4
|
+
data.tar.gz: 7fadec6fdfa02a2b19bc39f32ee3a0e2942ed8077cc1cff4d8058ca0c9a1519f
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 5c02045a364c8d77dfaa6a7861d1e0c0ed9d9f33726f40b8dbc058f5ec0057f3eac073bce211f398f27751ecfff37b67023c8e5d1e07f04f5f3aee039a60fbd6
|
|
7
|
+
data.tar.gz: 8988406bf83d5685d3764f73b9a1af90999a8e1cfe2e967b0628705ba0c3d1b87cbf038733ad36279c1e3b65f1a98ba572ce69a9159be46bf28386584fc014e0
|
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Jon Phillips
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,268 @@
|
|
|
1
|
+
# cronwatch
|
|
2
|
+
|
|
3
|
+
Cron and scheduled-job monitoring that lives inside your Ruby or Rails app. Wrap a job once; every run is recorded in a database you already have, and you are told when a run is missed, fails, gets stuck, runs slow or goes over budget. No server to run, no account to make.
|
|
4
|
+
|
|
5
|
+
This is the Ruby port of [`@cronwatch/sdk`](https://www.npmjs.com/package/@cronwatch/sdk): the same rules, the same alert text and the same stored rows, so a Ruby process and a Node process can share one database, and [`@cronwatch/mcp`](https://www.npmjs.com/package/@cronwatch/mcp) works against either.
|
|
6
|
+
|
|
7
|
+
Docs: [cronwatch.dev/docs/rails](https://cronwatch.dev/docs/rails/) and [cronwatch.dev/docs/ruby](https://cronwatch.dev/docs/ruby/)
|
|
8
|
+
|
|
9
|
+
## Install
|
|
10
|
+
|
|
11
|
+
```ruby
|
|
12
|
+
# Gemfile
|
|
13
|
+
gem "cronwatch"
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Ruby 3.2 or newer; the Rails integration is tested on Rails 7.2, 8.0 and 8.1. The only dependency is `fugit`, the cron parser Solid Queue and sidekiq-cron already bring. Everything else loads only from its own file:
|
|
17
|
+
|
|
18
|
+
| Require | For | Needs |
|
|
19
|
+
|---|---|---|
|
|
20
|
+
| `cronwatch` | the client, the memory store, and the Slack, Discord, webhook and console channels | |
|
|
21
|
+
| `cronwatch/active_record` | the ActiveRecord store | `activerecord` |
|
|
22
|
+
| `cronwatch/rails` | the Railtie, `Cronwatch::ActiveJob`, `Cronwatch::CheckJob`, the `cronwatch:check` task, the install generator | `railties`, `activejob` |
|
|
23
|
+
| `cronwatch/sidekiq` | `Cronwatch::Sidekiq` for `Sidekiq::Job` classes, its server middleware, `Cronwatch::Sidekiq::CheckWorker` | `sidekiq` 7 or newer |
|
|
24
|
+
| `cronwatch/scheduler` | schedules read from Solid Queue's or sidekiq-cron's config, `Cronwatch.declare_from_scheduler!` | |
|
|
25
|
+
| `cronwatch/web` | the dashboard and JSON API as a Rack app | `rack` |
|
|
26
|
+
| `cronwatch/triage/anthropic` | Claude triage | `anthropic` |
|
|
27
|
+
|
|
28
|
+
In a Rails app, `gem "cronwatch"` also loads `cronwatch/rails` and `cronwatch/scheduler` (Bundler requires it after Rails), `cronwatch/sidekiq` when Sidekiq is in the bundle, and the ActiveRecord store on first use. `cronwatch/web` and `cronwatch/triage/anthropic` are always required by hand.
|
|
29
|
+
|
|
30
|
+
## Plain Ruby
|
|
31
|
+
|
|
32
|
+
```ruby
|
|
33
|
+
require "cronwatch"
|
|
34
|
+
|
|
35
|
+
CW = Cronwatch.new(
|
|
36
|
+
alerts: [Cronwatch::Alerts::Slack.new(webhook_url: ENV.fetch("SLACK_WEBHOOK_URL"))],
|
|
37
|
+
)
|
|
38
|
+
|
|
39
|
+
NIGHTLY = CW.job("nightly-report",
|
|
40
|
+
schedule: "0 2 * * *", timezone: "UTC", grace: "15m", timeout: "30m",
|
|
41
|
+
expect: "Report written", budget: { cost: 2 })
|
|
42
|
+
|
|
43
|
+
NIGHTLY.run do |job|
|
|
44
|
+
path = build_report
|
|
45
|
+
job.log("Report written:", path) # kept with the run, shown in alerts
|
|
46
|
+
job.metric(:cost, 1.2) # watched against budgets and baselines
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
CW.start # checks for missed and stuck runs every minute, in a background thread
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
`run` returns what the block returns and raises what it raises, after the run is recorded. A script run from crontab exits when it is done, so instead of `start`, add a second crontab line that declares the jobs and calls `CW.check` every five minutes.
|
|
53
|
+
|
|
54
|
+
Client options: `store`, `alerts`, `triage`, `cron_secret`, `retention` (default `"30d"`), `defaults`, `redact` (default: blank values that look like secrets; `false` keeps output as logged, or pass a callable), `deliver` (`:now` by default; `:check` queues alerts for another process's check to send, for a worker that cannot reach Slack), `on_error` and `now`. Methods: `job`, `run`, `check`, `start`/`stop`, `silence(name, for: "2h")`/`unsilence`, `forget`, `jobs`, `jobs_with_runs`, `job_summary`, `runs`, `get_run`, `defined_jobs`, `close`. [cronwatch.dev/docs/ruby](https://cronwatch.dev/docs/ruby/#api) has each one.
|
|
55
|
+
|
|
56
|
+
## Rails
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
bundle add cronwatch
|
|
60
|
+
bin/rails generate cronwatch:install # a migration and config/initializers/cronwatch.rb
|
|
61
|
+
bin/rails db:migrate
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
The generator takes `--prefix` (table prefix, default `cronwatch_`) and `--database` (the database whose migrations directory gets the migration). The initializer it writes sets the ActiveRecord store, which Rails needs so web, worker and check processes see the same runs:
|
|
65
|
+
|
|
66
|
+
```ruby
|
|
67
|
+
# config/initializers/cronwatch.rb
|
|
68
|
+
Cronwatch.configure do |c|
|
|
69
|
+
c.store = Cronwatch::Stores::ActiveRecord.new
|
|
70
|
+
c.alerts = [Cronwatch::Alerts::Slack.new(webhook_url: ENV["SLACK_WEBHOOK_URL"])]
|
|
71
|
+
end
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
`Cronwatch.configure` builds the app's client; `Cronwatch.client` returns it.
|
|
75
|
+
|
|
76
|
+
### ActiveJob
|
|
77
|
+
|
|
78
|
+
```ruby
|
|
79
|
+
class NightlyReportJob < ApplicationJob
|
|
80
|
+
include Cronwatch::ActiveJob
|
|
81
|
+
cronwatch schedule: "0 2 * * *", grace: "15m", expect: "Report written" # name: "nightly-report"
|
|
82
|
+
|
|
83
|
+
def perform
|
|
84
|
+
cronwatch.log("Report written")
|
|
85
|
+
cronwatch.metric(:cost, 1.2)
|
|
86
|
+
end
|
|
87
|
+
end
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Every `perform` is recorded as a run with the trigger `"active_job"`. The name defaults to the class name without `Job`, dasherized, with `::` as `:` (`Reports::NightlyJob` is `reports:nightly`); pass `name:` to choose another. Jobs are declared once the app has booted, so a check knows a job that has never run. A job that raises still raises after the run is recorded, so ActiveJob retries and your error reporter see it as before.
|
|
91
|
+
|
|
92
|
+
### Sidekiq
|
|
93
|
+
|
|
94
|
+
```ruby
|
|
95
|
+
class HardWorker
|
|
96
|
+
include Sidekiq::Job
|
|
97
|
+
include Cronwatch::Sidekiq
|
|
98
|
+
cronwatch schedule: "*/15 * * * *" # name: "hard-worker"
|
|
99
|
+
|
|
100
|
+
def perform
|
|
101
|
+
cronwatch.log("Done")
|
|
102
|
+
end
|
|
103
|
+
end
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
The same `cronwatch` as for ActiveJob, for classes that include `Sidekiq::Job` (or `Sidekiq::Worker`) directly. `Cronwatch::Sidekiq::ServerMiddleware` records each `perform` with the trigger `"sidekiq"` and raises the job's error on to Sidekiq, so retries and error handlers behave as before. In Rails the Railtie adds it to the Sidekiq server; elsewhere add it in `Sidekiq.configure_server` and call `Cronwatch::Sidekiq.ready!` once `Cronwatch.configure` has run. An ActiveJob class on Sidekiq's adapter keeps `Cronwatch::ActiveJob`; the middleware passes ActiveJob's wrapper through, so it is recorded once.
|
|
107
|
+
|
|
108
|
+
### Schedules from the scheduler's config
|
|
109
|
+
|
|
110
|
+
```ruby
|
|
111
|
+
cronwatch schedule: :from_scheduler, grace: "15m" # ActiveJob or Sidekiq
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
reads the class's one entry in Solid Queue's `config/recurring.yml` (the section for `Rails.env`) or sidekiq-cron's `config/schedule.yml`, so the schedule is written once. Fugit phrases such as `every day at 3am` or `every hour at minute 12` become the cron expression Fugit makes of them (`0 3 * * *`, `12 * * * *`), in the zone the scheduler reads them in. Each conversion is checked against Fugit's own next run times; anything CronWatch would not expect at exactly those times is refused at boot rather than approximated: every other week (`%`), days counted back from the month's end other than the last, random times (`~`), offsets instead of IANA zones, and a time that daylight saving skips (Fugit skips that run; CronWatch would report it missed). A class with no entry, or with two, also stops the boot.
|
|
115
|
+
|
|
116
|
+
`Cronwatch.declare_from_scheduler!(grace: "10m")` in the initializer watches every enabled entry once the app has booted: classes that call `cronwatch` declare themselves, other classes are named as `cronwatch` would name them and their runs are recorded, and Solid Queue `command:` tasks are named after their key. `except:` leaves keys out. `Cronwatch::Scheduler.sources` sets where to read, when the defaults (Solid Queue's file when Solid Queue is loaded, sidekiq-cron's when sidekiq-cron is) are not right.
|
|
117
|
+
|
|
118
|
+
### Scheduling the check
|
|
119
|
+
|
|
120
|
+
Failures are caught as they happen, but a run that never started or never finished can only be noticed by looking. `Cronwatch::CheckJob` looks: it loads `app/jobs` (and `app/workers`) when the app does not eager load, declares every monitored job and runs the check. Run it every five minutes.
|
|
121
|
+
|
|
122
|
+
Solid Queue:
|
|
123
|
+
|
|
124
|
+
```yaml
|
|
125
|
+
# config/recurring.yml
|
|
126
|
+
production:
|
|
127
|
+
nightly_report:
|
|
128
|
+
class: NightlyReportJob
|
|
129
|
+
schedule: every day at 2am
|
|
130
|
+
cronwatch_check:
|
|
131
|
+
class: Cronwatch::CheckJob
|
|
132
|
+
schedule: every 5 minutes
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
sidekiq-cron:
|
|
136
|
+
|
|
137
|
+
```yaml
|
|
138
|
+
# config/schedule.yml
|
|
139
|
+
nightly_report:
|
|
140
|
+
cron: "0 2 * * * UTC"
|
|
141
|
+
class: "NightlyReportJob"
|
|
142
|
+
cronwatch_check:
|
|
143
|
+
cron: "*/5 * * * *"
|
|
144
|
+
class: "Cronwatch::CheckJob" # Cronwatch::Sidekiq::CheckWorker when ActiveJob does not use Sidekiq
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
Or from a crontab: `bin/rails cronwatch:check`.
|
|
148
|
+
|
|
149
|
+
### Mounting the dashboard
|
|
150
|
+
|
|
151
|
+
```ruby
|
|
152
|
+
# config/routes.rb
|
|
153
|
+
Rails.application.routes.draw do
|
|
154
|
+
mount Cronwatch::Web.new(Cronwatch.client) => "/cronwatch"
|
|
155
|
+
end
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
Set `CRONWATCH_TOKEN` to a long random string and open `/cronwatch?token=<it>` once; the browser keeps a cookie. Without a token, while `RAILS_ENV` or `RACK_ENV` is `development` or `test`, it makes a token of its own and prints a sign-in link to standard output on its first request (open it once); anywhere else it answers 503. To rely on the app's own sign in, mount it behind that (Devise's `authenticate` block, or a routing constraint) and pass `token: nil`. The URLs, JSON shapes, headers and CSRF rules are the SDK's, so the MCP server reads it unchanged. `GET /cronwatch/api/check` with a bearer (the token or `CRON_SECRET`) runs the check, for an outside cron.
|
|
159
|
+
|
|
160
|
+
There is no `handler()` as in the TypeScript SDK: for a job triggered over HTTP, wrap the controller action's body in `CW.job(...).run` (declared once, at boot) and check the bearer in the controller.
|
|
161
|
+
|
|
162
|
+
In any other Rack app:
|
|
163
|
+
|
|
164
|
+
```ruby
|
|
165
|
+
# config.ru
|
|
166
|
+
require "cronwatch/web"
|
|
167
|
+
map("/cronwatch") { run Cronwatch::Web.new(CW) }
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
## Stores
|
|
171
|
+
|
|
172
|
+
- `Cronwatch::Stores::Memory.new`: the default. Nothing survives a restart.
|
|
173
|
+
- `Cronwatch::Stores::ActiveRecord.new(prefix: "cronwatch_", connection_class: nil)`: Postgres or SQLite through ActiveRecord, in `cronwatch_jobs`, `cronwatch_runs` and `cronwatch_state`. `connection_class` picks the pool (a class, or its name). In Rails the generator's migration creates the tables; elsewhere call `Cronwatch::Stores::ActiveRecord.create_tables!` (and `drop_tables!`). The store never creates them itself, and raises `MissingTables` when they are not there. On Postgres it writes through a connection pool of its own (up to the database config's `pool` more connections per process), so a run is recorded outside any transaction the app has open and stays recorded when that transaction rolls back; on SQLite it uses the app's pool, and inside an open transaction each call runs in a savepoint. It always writes on the writing role, even inside `connected_to(role: :reading)`. Other adapters, MySQL among them, are refused with `UnsupportedAdapter`.
|
|
174
|
+
|
|
175
|
+
Finished runs older than `retention` (default `"30d"`) are pruned by the check; each job's newest run is kept.
|
|
176
|
+
|
|
177
|
+
## Alerts
|
|
178
|
+
|
|
179
|
+
```ruby
|
|
180
|
+
Cronwatch::Alerts::Slack.new(webhook_url: ENV.fetch("SLACK_WEBHOOK_URL"))
|
|
181
|
+
Cronwatch::Alerts::Discord.new(webhook_url: ENV.fetch("DISCORD_WEBHOOK_URL"))
|
|
182
|
+
Cronwatch::Alerts::Webhook.new(url: "https://hooks.example.com/cronwatch", secret: ENV["CRONWATCH_WEBHOOK_SECRET"])
|
|
183
|
+
Cronwatch::Alerts::Console.new # the default
|
|
184
|
+
|
|
185
|
+
Cronwatch::Alerts::Custom.new("pagerduty") do |alert|
|
|
186
|
+
next if alert.type == :recovered
|
|
187
|
+
PagerDuty.trigger(summary: alert.title, details: alert.message)
|
|
188
|
+
end
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
Every alert goes to every channel. A channel that raises, or takes longer than 15 seconds, goes to `on_error` and never blocks the others. Each condition alerts once when it opens and once more when a clean run closes it; there are no repeat alerts. The webhook signs its body with `X-CronWatch-Signature: sha256=<hex>`, the HMAC-SHA256 of the raw body.
|
|
192
|
+
|
|
193
|
+
## Triage
|
|
194
|
+
|
|
195
|
+
```ruby
|
|
196
|
+
# Gemfile
|
|
197
|
+
gem "anthropic"
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
```ruby
|
|
201
|
+
require "cronwatch/triage/anthropic"
|
|
202
|
+
|
|
203
|
+
Cronwatch.configure do |c|
|
|
204
|
+
c.triage = Cronwatch::Triage::Anthropic.new(context: "A Rails app on Postgres, jobs on Solid Queue.")
|
|
205
|
+
end
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Adds two to four sentences from Claude (likely cause, first thing to check) to every alert except recoveries. Needs the `anthropic` gem and `ANTHROPIC_API_KEY`. It runs only when an alert is sent, never per run, and the alert goes out without it after 25 seconds. Options: `model` (default `"claude-opus-5"`), `effort` (`"medium"`), `max_tokens` (`800`), `context`, `fallbacks` (`true`), `api_key`, `client`.
|
|
209
|
+
|
|
210
|
+
## Sharing a database with a Node app
|
|
211
|
+
|
|
212
|
+
The ActiveRecord store writes the same three tables as `@cronwatch/sdk/postgres` and `@cronwatch/sdk/sqlite`: same names, columns and indexes, epoch milliseconds in the time columns, the SDK's camelCase JSON in the JSON columns. [`test/active_record/node_compat_test.rb`](https://github.com/phillips-jon/cronwatch/blob/main/packages/ruby/test/active_record/node_compat_test.rb) runs the SDK's stores in Node beside this one, on SQLite and Postgres, and checks that each reads what the other wrote, that the tables are the same whoever creates them, and that the rows are the same bytes. Use the same prefix on both sides and give each job a name only one side uses. Then one dashboard, Rails or Node, shows every job, and one MCP server reads them all.
|
|
213
|
+
|
|
214
|
+
## Kept in step with the TypeScript SDK
|
|
215
|
+
|
|
216
|
+
The TypeScript SDK is the source of truth. `npm run conformance` at the repository root runs it and writes JSON cases to `conformance/`: duration parsing and formatting, schedules including daylight saving, sequences of runs and checks with the alerts and state they must produce, alert titles and messages, stats and health. This gem's tests replay every case, and the SDK's own check fails when the files are stale. A change of behaviour lands in TypeScript first, the cases are regenerated, and the gem is fixed until its tests pass. When the two disagree, the Ruby side is wrong.
|
|
217
|
+
|
|
218
|
+
The dashboard is held to the SDK the same way: [`test/web/golden.json`](https://github.com/phillips-jon/cronwatch/blob/main/packages/ruby/test/web/golden.json) records what the SDK's routes answer to a fixed set of requests, and [`test/web_golden_test.rb`](https://github.com/phillips-jon/cronwatch/blob/main/packages/ruby/test/web_golden_test.rb) makes `Cronwatch::Web` answer them byte for byte.
|
|
219
|
+
|
|
220
|
+
The design of the port is in [DESIGN.md](https://github.com/phillips-jon/cronwatch/blob/main/packages/ruby/DESIGN.md).
|
|
221
|
+
|
|
222
|
+
## Testing
|
|
223
|
+
|
|
224
|
+
```sh
|
|
225
|
+
bundle install
|
|
226
|
+
bundle exec rake test
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
`rake test` runs four suites, each in its own process: `rake test:core` (the client, channels, conformance, the Rack app, Sidekiq without Rails, schedule conversion checked against Fugit), `rake test:slow` (every schedule conversion walked against Fugit across a year, some ten seconds), `rake test:active_record` (the store, on SQLite, and on Postgres when `CRONWATCH_TEST_PG` is set) and `rake test:rails` (a small Rails app: ActiveJob, Sidekiq, schedules from `recurring.yml` and `schedule.yml`, `CheckJob`, the rake task, the generator).
|
|
230
|
+
|
|
231
|
+
```sh
|
|
232
|
+
CRONWATCH_TEST_PG=postgres://postgres:pw@127.0.0.1:5432/cw bundle exec rake test
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
The node compatibility tests run the built SDK and its drivers, and skip themselves otherwise, so build it first at the repository root:
|
|
236
|
+
|
|
237
|
+
```sh
|
|
238
|
+
npm ci && npm run build
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
The default Gemfile tests Rails 8.1. Each supported Rails series has its own Gemfile, with its own lockfile, in [`test/rails/gemfiles`](https://github.com/phillips-jon/cronwatch/tree/main/packages/ruby/test/rails/gemfiles):
|
|
242
|
+
|
|
243
|
+
```sh
|
|
244
|
+
BUNDLE_GEMFILE=test/rails/gemfiles/rails_7_2.gemfile bundle install
|
|
245
|
+
BUNDLE_GEMFILE=test/rails/gemfiles/rails_7_2.gemfile bundle exec rake test
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
`rails_8_0.gemfile` and `rails_8_1.gemfile` work the same way. Sidekiq and sidekiq-cron are test gems too: Rails 8.0 and 8.1 resolve Sidekiq 8, and `rails_7_2.gemfile` pins Sidekiq 7 (`CRONWATCH_SIDEKIQ=7`), so both majors are tested. CI runs Ruby 3.2 with Rails 7.2, Ruby 3.4 with Rails 8.0 and 8.1, and Ruby 4.0 with Rails 8.1, all against Postgres.
|
|
249
|
+
|
|
250
|
+
When the SDK's routes or pages change, regenerate the dashboard fixture from the repository root:
|
|
251
|
+
|
|
252
|
+
```sh
|
|
253
|
+
npm run build --workspace packages/sdk && TZ=UTC node packages/ruby/test/web/golden.mjs
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
`npm run check` fails while `test/web/golden.json` or `conformance/` is stale.
|
|
257
|
+
|
|
258
|
+
The MCP server's tests can also drive this gem's `Cronwatch::Web` over HTTP ([`test/web/server.rb`](https://github.com/phillips-jon/cronwatch/blob/main/packages/ruby/test/web/server.rb)). They need `fugit`, `rack` and a server rackup can start (puma, or webrick, which the Gemfile's test group has) in the Ruby they run, and only run when asked, from `packages/mcp`:
|
|
259
|
+
|
|
260
|
+
```sh
|
|
261
|
+
CRONWATCH_TEST_RUBY=1 CRONWATCH_RUBY="rbenv exec ruby" npm test
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
`CRONWATCH_RUBY` is the command that runs Ruby; it defaults to `ruby`.
|
|
265
|
+
|
|
266
|
+
## License
|
|
267
|
+
|
|
268
|
+
MIT, see [LICENSE](LICENSE).
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Cronwatch
|
|
4
|
+
# Raised by AbortSignal#check! once a signal has aborted.
|
|
5
|
+
class AbortError < StandardError; end
|
|
6
|
+
|
|
7
|
+
# A cancellation flag, like JavaScript's AbortSignal. A job's signal aborts
|
|
8
|
+
# once the job's timeout has passed; triage gets one that aborts when the
|
|
9
|
+
# client stops waiting. Nothing is interrupted: code that can stop early
|
|
10
|
+
# checks it.
|
|
11
|
+
class AbortSignal
|
|
12
|
+
def initialize(timeout_ms = nil)
|
|
13
|
+
@deadline = timeout_ms && (AbortSignal.monotonic + (timeout_ms / 1000.0))
|
|
14
|
+
@aborted = false
|
|
15
|
+
@settled = false
|
|
16
|
+
@lock = Mutex.new
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def self.monotonic
|
|
20
|
+
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def aborted?
|
|
24
|
+
@lock.synchronize do
|
|
25
|
+
@aborted = true if !@aborted && !@settled && @deadline && AbortSignal.monotonic >= @deadline
|
|
26
|
+
@aborted
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def abort!
|
|
31
|
+
@lock.synchronize { @aborted = true unless @settled }
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
# Raises AbortError when aborted, for a loop that should stop there.
|
|
35
|
+
def check!
|
|
36
|
+
raise AbortError, "This operation was aborted" if aborted?
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
# Called when the work is over: a timeout that passes later no longer aborts it.
|
|
40
|
+
def settle!
|
|
41
|
+
aborted?
|
|
42
|
+
@lock.synchronize { @settled = true }
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
end
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# The ActiveRecord store. Needs the activerecord gem.
|
|
4
|
+
#
|
|
5
|
+
# require "cronwatch/active_record"
|
|
6
|
+
# CW = Cronwatch.new(store: Cronwatch::Stores::ActiveRecord.new)
|
|
7
|
+
#
|
|
8
|
+
# `require "cronwatch/rails"` loads it on first use, so a Rails app does not
|
|
9
|
+
# need this line.
|
|
10
|
+
require "cronwatch" unless defined?(Cronwatch::Client)
|
|
11
|
+
require_relative "stores/active_record"
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Cronwatch
|
|
4
|
+
module Alerts
|
|
5
|
+
# Writes alerts to the console: recoveries to standard output, the rest to
|
|
6
|
+
# standard error. The default channel.
|
|
7
|
+
class Console
|
|
8
|
+
attr_reader :name
|
|
9
|
+
|
|
10
|
+
def initialize(out: $stdout, err: $stderr)
|
|
11
|
+
@name = "console"
|
|
12
|
+
@out = out
|
|
13
|
+
@err = err
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def call(alert)
|
|
17
|
+
line = "[cronwatch] #{alert.title}\n#{alert.message}#{alert.triage ? "\nTriage: #{alert.triage}" : ""}"
|
|
18
|
+
(alert.type == :recovered ? @out : @err).puts(line)
|
|
19
|
+
end
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
end
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Cronwatch
|
|
4
|
+
module Alerts
|
|
5
|
+
# Wraps any block as an alert channel:
|
|
6
|
+
#
|
|
7
|
+
# Cronwatch::Alerts::Custom.new("pagerduty") do |alert|
|
|
8
|
+
# next if alert.type == :recovered
|
|
9
|
+
# PagerDuty.trigger(summary: alert.title, details: alert.message)
|
|
10
|
+
# end
|
|
11
|
+
class Custom
|
|
12
|
+
attr_reader :name
|
|
13
|
+
|
|
14
|
+
def initialize(name, callable = nil, &block)
|
|
15
|
+
@name = name.to_s
|
|
16
|
+
@send = callable || block
|
|
17
|
+
raise ArgumentError, "Cronwatch::Alerts::Custom needs a block" unless @send
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def call(alert)
|
|
21
|
+
@send.call(alert)
|
|
22
|
+
nil
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
end
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Cronwatch
|
|
4
|
+
module Alerts
|
|
5
|
+
# Sends alerts to a Discord channel through a webhook (Server Settings,
|
|
6
|
+
# Integrations, Webhooks).
|
|
7
|
+
class Discord
|
|
8
|
+
COLOR = {
|
|
9
|
+
missed: 0xb7791f, failed: 0xc62828, stuck: 0xc62828, slow: 0xb7791f, over_budget: 0xb7791f, recovered: 0x1f8a4c,
|
|
10
|
+
}.freeze
|
|
11
|
+
|
|
12
|
+
attr_reader :name
|
|
13
|
+
|
|
14
|
+
def initialize(webhook_url:, link: nil, http: HTTP.default)
|
|
15
|
+
raise ArgumentError, "Cronwatch::Alerts::Discord needs a webhook_url" if webhook_url.nil? || webhook_url.to_s.empty?
|
|
16
|
+
|
|
17
|
+
@name = "discord"
|
|
18
|
+
@webhook_url = webhook_url
|
|
19
|
+
@link = link
|
|
20
|
+
@http = http
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def call(alert)
|
|
24
|
+
url = @link&.call(alert)
|
|
25
|
+
embed = { "title" => alert.title }
|
|
26
|
+
embed["url"] = url if url && !url.empty?
|
|
27
|
+
triage = alert.triage && !alert.triage.empty? ? "\n**Triage:** #{Discord.escape_markdown(JS.head16(alert.triage, 1000))}" : ""
|
|
28
|
+
embed["description"] = "```\n#{Discord.code_block_safe(JS.head16(alert.message, 3800))}\n```#{triage}"
|
|
29
|
+
embed["color"] = COLOR[alert.type]
|
|
30
|
+
embed["timestamp"] = JS.iso(alert.at)
|
|
31
|
+
payload = {
|
|
32
|
+
"content" => alert.title,
|
|
33
|
+
# Job output can hold anything, "@everyone" included; ping no one.
|
|
34
|
+
"allowed_mentions" => { "parse" => [] },
|
|
35
|
+
"embeds" => [embed],
|
|
36
|
+
}
|
|
37
|
+
response = @http.post(@webhook_url, JS.json(payload), { "content-type" => "application/json" })
|
|
38
|
+
raise "Discord webhook answered #{response.status}: #{JS.head16(response.body.to_s, 200)}" unless response.ok?
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
# Breaks up ``` so text inside a code block cannot close it.
|
|
42
|
+
def self.code_block_safe(text)
|
|
43
|
+
text.gsub("```", "`\u200b`\u200b`")
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# Escapes the characters Discord reads as markdown, links included.
|
|
47
|
+
def self.escape_markdown(text)
|
|
48
|
+
text.gsub(/[\\`*_~|\[\]()<>]/) { |c| "\\#{c}" }
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
end
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Cronwatch
|
|
4
|
+
module Alerts
|
|
5
|
+
# Sends alerts to a Slack channel through an incoming webhook.
|
|
6
|
+
#
|
|
7
|
+
# Cronwatch::Alerts::Slack.new(webhook_url: ENV.fetch("SLACK_WEBHOOK_URL"),
|
|
8
|
+
# link: ->(alert) { "https://app.example.com/cronwatch/jobs/#{alert.job}" })
|
|
9
|
+
class Slack
|
|
10
|
+
EMOJI = {
|
|
11
|
+
missed: ":hourglass_flowing_sand:", failed: ":x:", stuck: ":no_entry:", slow: ":turtle:",
|
|
12
|
+
over_budget: ":moneybag:", recovered: ":white_check_mark:",
|
|
13
|
+
}.freeze
|
|
14
|
+
|
|
15
|
+
attr_reader :name
|
|
16
|
+
|
|
17
|
+
def initialize(webhook_url:, link: nil, http: HTTP.default)
|
|
18
|
+
raise ArgumentError, "Cronwatch::Alerts::Slack needs a webhook_url" if webhook_url.nil? || webhook_url.to_s.empty?
|
|
19
|
+
|
|
20
|
+
@name = "slack"
|
|
21
|
+
@webhook_url = webhook_url
|
|
22
|
+
@link = link
|
|
23
|
+
@http = http
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def call(alert)
|
|
27
|
+
url = @link&.call(alert)
|
|
28
|
+
title = "#{EMOJI[alert.type]} *#{Slack.escape(alert.title)}*#{url && !url.empty? ? " (<#{url}|open>)" : ""}"
|
|
29
|
+
body = JS.head16(Slack.code_block_safe(Slack.escape(alert.message)), 2900)
|
|
30
|
+
blocks = [
|
|
31
|
+
{ "type" => "section", "text" => { "type" => "mrkdwn", "text" => title } },
|
|
32
|
+
{ "type" => "section", "text" => { "type" => "mrkdwn", "text" => "```#{body}```" } },
|
|
33
|
+
]
|
|
34
|
+
if alert.triage && !alert.triage.empty?
|
|
35
|
+
# Its own block, so a long diagnosis cannot push a block past Slack's 3000 character limit.
|
|
36
|
+
blocks << { "type" => "section", "text" => { "type" => "mrkdwn", "text" => JS.head16("_Triage:_ #{Slack.escape(alert.triage)}", 3000) } }
|
|
37
|
+
end
|
|
38
|
+
payload = {
|
|
39
|
+
# The notification fallback is parsed as mrkdwn too, so it is escaped like the blocks.
|
|
40
|
+
"text" => Slack.escape("#{alert.title}\n#{alert.message}"),
|
|
41
|
+
"blocks" => blocks,
|
|
42
|
+
}
|
|
43
|
+
response = @http.post(@webhook_url, JS.json(payload), { "content-type" => "application/json" })
|
|
44
|
+
raise "Slack webhook answered #{response.status}: #{JS.head16(response.body.to_s, 200)}" unless response.ok?
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
# Slack's three control characters. Escaping < and > also stops <!channel> and <url|links>.
|
|
48
|
+
def self.escape(text)
|
|
49
|
+
text.gsub("&", "&").gsub("<", "<").gsub(">", ">")
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
# Breaks up ``` so text inside a code block cannot close it.
|
|
53
|
+
def self.code_block_safe(text)
|
|
54
|
+
text.gsub("```", "`\u200b`\u200b`")
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
|
58
|
+
end
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "openssl"
|
|
4
|
+
|
|
5
|
+
module Cronwatch
|
|
6
|
+
module Alerts
|
|
7
|
+
# POSTs the alert as JSON to any URL. The body is the alert's JSON:
|
|
8
|
+
# { type, run, details, job, definition, title, message, at, triage }.
|
|
9
|
+
# With a secret, each request carries `X-CronWatch-Signature: sha256=<hex>`,
|
|
10
|
+
# the HMAC-SHA256 of the raw body, so the receiver can verify it.
|
|
11
|
+
class Webhook
|
|
12
|
+
attr_reader :name
|
|
13
|
+
|
|
14
|
+
def initialize(url:, headers: {}, secret: nil, http: HTTP.default)
|
|
15
|
+
raise ArgumentError, "Cronwatch::Alerts::Webhook needs a url" if url.nil? || url.to_s.empty?
|
|
16
|
+
|
|
17
|
+
@name = "webhook"
|
|
18
|
+
@url = url
|
|
19
|
+
@headers = headers || {}
|
|
20
|
+
@secret = secret
|
|
21
|
+
@http = http
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def call(alert)
|
|
25
|
+
body = JS.json(alert.to_h)
|
|
26
|
+
headers = { "content-type" => "application/json", "user-agent" => "cronwatch" }.merge(@headers.transform_keys(&:to_s))
|
|
27
|
+
if @secret && !@secret.empty?
|
|
28
|
+
headers["x-cronwatch-signature"] = "sha256=#{OpenSSL::HMAC.hexdigest("SHA256", @secret, body)}"
|
|
29
|
+
end
|
|
30
|
+
response = @http.post(@url, body, headers)
|
|
31
|
+
# Only the origin: a webhook URL's path or query often is the credential.
|
|
32
|
+
raise "Webhook #{Webhook.origin(@url)} answered #{response.status}" unless response.ok?
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def self.origin(url)
|
|
36
|
+
uri = URI(url)
|
|
37
|
+
raise URI::InvalidURIError unless uri.scheme && uri.host
|
|
38
|
+
|
|
39
|
+
port = uri.port && uri.port != uri.default_port ? ":#{uri.port}" : ""
|
|
40
|
+
"#{uri.scheme.downcase}://#{uri.host.downcase}#{port}"
|
|
41
|
+
rescue URI::Error, ArgumentError
|
|
42
|
+
"(invalid URL)"
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
end
|
|
46
|
+
end
|