cronwatch 0.3.1 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: cc6a450f111f1a0f65f0e92656784a215a37af11da63a723d76074f59fe21719
-  data.tar.gz: 7fadec6fdfa02a2b19bc39f32ee3a0e2942ed8077cc1cff4d8058ca0c9a1519f
+  metadata.gz: e0e24280879ad52eeeaf138342e4266225aa702893875d07c1ae9e3f7728b1b5
+  data.tar.gz: e828a6fb7fd737df7100480129d986232f59e2adef2f41eed50a42ef7e71913a
 SHA512:
-  metadata.gz: 5c02045a364c8d77dfaa6a7861d1e0c0ed9d9f33726f40b8dbc058f5ec0057f3eac073bce211f398f27751ecfff37b67023c8e5d1e07f04f5f3aee039a60fbd6
-  data.tar.gz: 8988406bf83d5685d3764f73b9a1af90999a8e1cfe2e967b0628705ba0c3d1b87cbf038733ad36279c1e3b65f1a98ba572ce69a9159be46bf28386584fc014e0
+  metadata.gz: af2ec018d78e9277c24a81392e071a26aa31554c7fb3d60967e96761dc615b6aa0e3a501c97415fdf73b8db57320e9d1ecc2e2840e6e843690705424c18e2d04
+  data.tar.gz: 2ccf44bdaedd6d130d4f77daabd6030ffdaf690ef484dded57e74cf68c8b2f01f8587acda1bfd54732d9ada59f5a2e7e90ad2f92bc0ec33be0c08f3437779dc6

data/README.md

@@ -17,7 +17,8 @@ Ruby 3.2 or newer; the Rails integration is tested on Rails 7.2, 8.0 and 8.1. Th
 
 | Require | For | Needs |
 |---|---|---|
-| `cronwatch` | the client, the memory store, and the Slack, Discord, webhook and console channels | |
+| `cronwatch` | the client, the memory store, and the Slack, Discord, webhook, console, email (Resend, Postmark, SendGrid, Mailgun, SES), Twilio and error tracker (Sentry, Honeybadger, Datadog, Rollbar, Bugsnag, New Relic) channels | |
+| `cronwatch/pg_cron` | `Cronwatch::Sources::PgCron`, which watches pg_cron jobs through an ActiveRecord or pg connection | |
 | `cronwatch/active_record` | the ActiveRecord store | `activerecord` |
 | `cronwatch/rails` | the Railtie, `Cronwatch::ActiveJob`, `Cronwatch::CheckJob`, the `cronwatch:check` task, the install generator | `railties`, `activejob` |
 | `cronwatch/sidekiq` | `Cronwatch::Sidekiq` for `Sidekiq::Job` classes, its server middleware, `Cronwatch::Sidekiq::CheckWorker` | `sidekiq` 7 or newer |
@@ -51,7 +52,19 @@ CW.start # checks for missed and stuck runs every minute, in a background thread
 
 `run` returns what the block returns and raises what it raises, after the run is recorded. A script run from crontab exits when it is done, so instead of `start`, add a second crontab line that declares the jobs and calls `CW.check` every five minutes.
 
-Client options: `store`, `alerts`, `triage`, `cron_secret`, `retention` (default `"30d"`), `defaults`, `redact` (default: blank values that look like secrets; `false` keeps output as logged, or pass a callable), `deliver` (`:now` by default; `:check` queues alerts for another process's check to send, for a worker that cannot reach Slack), `on_error` and `now`. Methods: `job`, `run`, `check`, `start`/`stop`, `silence(name, for: "2h")`/`unsilence`, `forget`, `jobs`, `jobs_with_runs`, `job_summary`, `runs`, `get_run`, `defined_jobs`, `close`. [cronwatch.dev/docs/ruby](https://cronwatch.dev/docs/ruby/#api) has each one.
+A run that starts in one call and ends in another (a job that hands work to a queue, a webhook that reports back later) is one run too:
+
+```ruby
+run = NIGHTLY.start(id: batch_id)   # records a running run; a second start with this id finds it
+# later, perhaps in another process
+run = NIGHTLY.resume(batch_id)
+run.log("Report written:", path)
+run.finish                          # or run.fail(error); finish("text") and finish(result: x) work like run's return value
+```
+
+Neither raises for the store: failures go to `on_error`, and a finish of a run already finished, not found, or of another job records nothing and returns nil. A run is judged once, however many processes finish it at once, and a store that fails during `finish` leaves the handle active to finish again. A run never finished is marked stuck after the job's `timeout`.
+
+Client options: `store`, `alerts`, `triage`, `cron_secret`, `retention` (default `"30d"`), `defaults`, `redact` (default: blank values that look like secrets; `false` keeps output as logged, or pass a callable), `deliver` (`:now` by default; `:check` queues alerts for another process's check to send, for a worker that cannot reach Slack), `on_error` and `now`. Methods: `job`, `run`, `resume_run`, `check`, `start`/`stop`, `silence(name, for: "2h")`/`unsilence`, `forget`, `jobs`, `jobs_with_runs`, `job_summary`, `runs`, `get_run`, `defined_jobs`, `close`. [cronwatch.dev/docs/ruby](https://cronwatch.dev/docs/ruby/#api) has each one.
 
 ## Rails
 
@@ -155,7 +168,7 @@ Rails.application.routes.draw do
 end
 ```
 
-Set `CRONWATCH_TOKEN` to a long random string and open `/cronwatch?token=<it>` once; the browser keeps a cookie. Without a token, while `RAILS_ENV` or `RACK_ENV` is `development` or `test`, it makes a token of its own and prints a sign-in link to standard output on its first request (open it once); anywhere else it answers 503. To rely on the app's own sign in, mount it behind that (Devise's `authenticate` block, or a routing constraint) and pass `token: nil`. The URLs, JSON shapes, headers and CSRF rules are the SDK's, so the MCP server reads it unchanged. `GET /cronwatch/api/check` with a bearer (the token or `CRON_SECRET`) runs the check, for an outside cron.
+Set `CRONWATCH_TOKEN` to a long random string and open `/cronwatch?token=<it>` once; the browser keeps a cookie. Without a token, while `RAILS_ENV` or `RACK_ENV` is `development` or `test`, it makes a token of its own and prints a sign-in link to standard output on its first request (open it once); anywhere else it answers 503. To rely on the app's own sign in, mount it behind that (Devise's `authenticate` block, or a routing constraint) and pass `token: nil`. The URLs, JSON shapes, headers and CSRF rules are the SDK's, so the MCP server reads it unchanged. The request's origin, which the CSRF check compares against, is what Rack reports and so already follows `X-Forwarded-Host` and `X-Forwarded-Proto`; pass `origin: "https://app.example.com"` to pin it instead. `GET /cronwatch/api/check` with a bearer (the token or `CRON_SECRET`) runs the check, for an outside cron.
 
 There is no `handler()` as in the TypeScript SDK: for a job triggered over HTTP, wrap the controller action's body in `CW.job(...).run` (declared once, at boot) and check the bearer in the controller.
 
@@ -181,6 +194,9 @@ Cronwatch::Alerts::Slack.new(webhook_url: ENV.fetch("SLACK_WEBHOOK_URL"))
 Cronwatch::Alerts::Discord.new(webhook_url: ENV.fetch("DISCORD_WEBHOOK_URL"))
 Cronwatch::Alerts::Webhook.new(url: "https://hooks.example.com/cronwatch", secret: ENV["CRONWATCH_WEBHOOK_SECRET"])
 Cronwatch::Alerts::Console.new   # the default
+Cronwatch::Alerts::Resend.new(api_key: ENV.fetch("RESEND_API_KEY"), from: "alerts@example.com", to: "ops@example.com")
+Cronwatch::Alerts::Sentry.new(dsn: ENV.fetch("SENTRY_DSN"))
+# and Postmark, Sendgrid, Mailgun, Ses, Twilio, Honeybadger, Datadog, Rollbar, Bugsnag, NewRelic
 
 Cronwatch::Alerts::Custom.new("pagerduty") do |alert|
   next if alert.type == :recovered

data/lib/cronwatch/alerts/bugsnag.rb

@@ -0,0 +1,69 @@
+# frozen_string_literal: true
+
+module Cronwatch
+  module Alerts
+    # Reports alerts to Bugsnag (Error Reporting API, payload version 5),
+    # grouped per job and alert type
+    # (https://developer.smartbear.com/bugsnag/docs/reporting-events-and-sessions).
+    # Recoveries are not sent unless `recovered: true`, since each one is an
+    # event on an error.
+    class Bugsnag
+      attr_reader :name
+
+      # endpoint: another notify endpoint, for on-premise installs.
+      # now:      the clock for the Bugsnag-Sent-At header, a callable returning epoch milliseconds. For tests.
+      def initialize(api_key:, release_stage: nil, endpoint: nil, recovered: false, now: nil, link: nil, http: HTTP.default)
+        @api_key = Provider.require_credential(api_key, "Cronwatch::Alerts::Bugsnag needs an api_key")
+        @url = endpoint.nil? ? "https://notify.bugsnag.com/" : endpoint.to_s
+        @release_stage = release_stage
+        @recovered = recovered
+        @now = now || -> { Process.clock_gettime(Process::CLOCK_REALTIME, :millisecond) }
+        @link = link
+        @http = http
+        @name = "bugsnag"
+      end
+
+      def call(alert)
+        return if alert.type == :recovered && !@recovered
+
+        link = Provider.link_for(@link, alert)
+        type = alert.type.to_s
+        meta = { "job" => alert.job, "type" => type }
+        meta["triage"] = alert.triage if Provider.present?(alert.triage)
+        meta["link"] = link if link
+        meta["details"] = Naming.to_json_value(alert.details)
+        meta["run"] = Provider.run_summary(alert)
+        payload = {
+          "apiKey" => @api_key,
+          "payloadVersion" => "5",
+          # The notifier's own version, not the gem's; Bugsnag asks for one.
+          "notifier" => { "name" => "cronwatch", "version" => "1.0.0", "url" => "https://cronwatch.dev" },
+          "events" => [
+            {
+              "exceptions" => [{
+                "errorClass" => "CronWatch #{type}", "message" => Provider.cut("#{alert.title}\n#{alert.message}", 8000),
+                "stacktrace" => [], "type" => "nodejs",
+              }],
+              "severity" => Provider.severity(alert.type),
+              "unhandled" => false,
+              "severityReason" => { "type" => "handledException" },
+              "context" => alert.job,
+              "groupingHash" => "cronwatch:#{alert.job}:#{type}",
+              "metaData" => { "cronwatch" => meta },
+              "app" => { "releaseStage" => @release_stage.nil? ? "production" : @release_stage },
+              "device" => { "time" => JS.iso(alert.at) },
+            },
+          ],
+        }
+        headers = {
+          "content-type" => "application/json",
+          "bugsnag-api-key" => @api_key,
+          "bugsnag-payload-version" => "5",
+          "bugsnag-sent-at" => JS.iso(@now.call),
+        }
+        Provider.post(@http, "Bugsnag", @url, headers, JS.json(payload), [@api_key])
+        nil
+      end
+    end
+  end
+end

data/lib/cronwatch/alerts/custom.rb

@@ -17,8 +17,15 @@ module Cronwatch
         raise ArgumentError, "Cronwatch::Alerts::Custom needs a block" unless @send
       end
 
-      def call(alert)
-        @send.call(alert)
+      # `context` is the client's Client::ChannelContext: a block taking a
+      # second argument gets it, to report a problem that did not stop the
+      # alert going out (context.on_error(error)).
+      def call(alert, context = nil)
+        if Client.takes_context?(@send)
+          @send.call(alert, context)
+        else
+          @send.call(alert)
+        end
         nil
       end
     end

data/lib/cronwatch/alerts/datadog.rb

@@ -0,0 +1,54 @@
+# frozen_string_literal: true
+
+module Cronwatch
+  module Alerts
+    # Posts alerts to the Datadog event stream (Events API v1), aggregated per
+    # job and alert type. `site` is your Datadog site: "datadoghq.com" (the
+    # default), "datadoghq.eu", "us3.datadoghq.com", "us5.datadoghq.com",
+    # "ap1.datadoghq.com", "ddog-gov.com". Every event is tagged cronwatch,
+    # job:<name> and alert:<type>, then `tags`.
+    class Datadog
+      ALERT_TYPE = {
+        missed: "error", failed: "error", stuck: "error", slow: "warning", over_budget: "warning", recovered: "success",
+      }.freeze
+
+      attr_reader :name
+
+      def initialize(api_key:, site: nil, tags: nil, host: nil, link: nil, http: HTTP.default)
+        @api_key = Provider.require_credential(api_key, "Cronwatch::Alerts::Datadog needs an api_key")
+        site = (site.nil? ? "datadoghq.com" : site.to_s).sub(%r{\Ahttps?://}, "").sub(/\A(api|app)\./, "").sub(%r{/+\z}, "")
+        raise ArgumentError, "Cronwatch::Alerts::Datadog needs a site like datadoghq.com" unless /\A[a-z0-9.-]+\z/i.match?(site)
+
+        @url = "https://api.#{site}/api/v1/events"
+        @tags = tags || []
+        @host = host
+        @link = link
+        @http = http
+        @name = "datadog"
+      end
+
+      def call(alert)
+        link = Provider.link_for(@link, alert)
+        event = {
+          "title" => Provider.cut(alert.title, 500),
+          "text" => Provider.cut(Provider.plain_text(alert, link), 4000),
+          "alert_type" => ALERT_TYPE[alert.type.to_sym],
+          "aggregation_key" => Datadog.aggregation_key(alert),
+          "date_happened" => alert.at.div(1000),
+          "priority" => "normal",
+          "tags" => ["cronwatch", "job:#{alert.job}", "alert:#{alert.type}", *@tags],
+        }
+        event["host"] = @host if Provider.present?(@host)
+        headers = { "content-type" => "application/json", "accept" => "application/json", "dd-api-key" => @api_key }
+        Provider.post(@http, "Datadog", @url, headers, JS.json(event), [@api_key])
+        nil
+      end
+
+      # "cronwatch:<job>:<type>", or a hash of it when that passes Datadog's 100 characters.
+      def self.aggregation_key(alert)
+        key = "cronwatch:#{alert.job}:#{alert.type}"
+        JS.length16(key) <= 100 ? key : "cronwatch:#{Provider.sha256_hex(key)[0, 40]}"
+      end
+    end
+  end
+end

data/lib/cronwatch/alerts/email.rb

@@ -0,0 +1,78 @@
+# frozen_string_literal: true
+
+module Cronwatch
+  module Alerts
+    # What every email channel sends: one subject, a plain text body and a
+    # small HTML body, so an alert reads the same whichever provider carries
+    # it (the SDK's alerts/email.ts). Resend, Postmark, SendGrid, Mailgun and
+    # SES each take the same options:
+    #
+    #   from:           the sender, "alerts@example.com" or "CronWatch <alerts@example.com>"
+    #   to:             one address or several
+    #   subject_prefix: put in front of the title in the subject, "[prod]" say
+    #   link:           ->(alert) { "https://app.example.com/cronwatch/jobs/#{alert.job}" }
+    module Email
+      Message = Struct.new(:from, :to, :subject, :text, :html, keyword_init: true)
+
+      # Not a line terminator, as JavaScript's "." reads it.
+      LINE = "[^\\n\\r\\u2028\\u2029]"
+      ADDRESS = Regexp.new("\\A[#{JS::WHITESPACE}]*(#{LINE}*?)[#{JS::WHITESPACE}]*<([^<>]+)>[#{JS::WHITESPACE}]*\\z")
+      QUOTED = Regexp.new("\\A\"(#{LINE}*)\"\\z")
+
+      module_function
+
+      # Checks the shared options once, when the channel is made. Returns the recipients.
+      def recipients(name, from, to)
+        raise ArgumentError, "Cronwatch::Alerts::#{name} needs a from address" if from.nil? || from.to_s.empty?
+
+        list = (to.is_a?(Array) ? to : [to]).select { |a| a.is_a?(String) && JS.trim(a) != "" }
+        raise ArgumentError, "Cronwatch::Alerts::#{name} needs at least one to address" if list.empty?
+
+        list.map { |a| JS.trim(a) }
+      end
+
+      def compose(alert, from:, to:, subject_prefix: nil, link: nil)
+        url = safe_link(link&.call(alert))
+        # One line: a newline in a subject is a header injection or a rejected send.
+        prefix = Provider.present?(subject_prefix) ? "#{subject_prefix} " : ""
+        subject = JS.head16("#{prefix}#{alert.title}".gsub(/[\r\n]+/, " "), 250)
+        Message.new(from: from.to_s, to: to, subject: subject, text: Provider.plain_text(alert, url), html: html(alert, url))
+      end
+
+      # Escapes text for HTML content and double quoted attributes.
+      def escape_html(text)
+        text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;").gsub("'", "&#39;")
+      end
+
+      # Only http and https links are put in a mail; anything else is dropped.
+      def safe_link(link)
+        return nil unless Provider.present?(link)
+
+        %r{\Ahttps?://}i.match?(link.to_s) ? link.to_s : nil
+      end
+
+      def html(alert, link)
+        parts = [
+          "<!doctype html>",
+          '<html><body style="margin:0;padding:16px;font-family:Georgia,serif;color:#1d1b16;background:#ffffff">',
+          "<p style=\"margin:0 0 12px;font-size:18px\"><strong>#{escape_html(alert.title)}</strong></p>",
+          '<pre style="margin:0 0 12px;padding:12px;background:#f6f3ec;white-space:pre-wrap;word-break:break-word;' \
+          "font:13px/1.45 Menlo,Consolas,monospace\">#{escape_html(alert.message)}</pre>",
+        ]
+        parts << "<p style=\"margin:0 0 12px\"><em>Triage:</em> #{escape_html(alert.triage)}</p>" if Provider.present?(alert.triage)
+        parts << "<p style=\"margin:0\"><a href=\"#{escape_html(link)}\">Open #{escape_html(alert.job)}</a></p>" if link
+        parts << "</body></html>"
+        parts.join("\n")
+      end
+
+      # Splits "Name <a@b.c>" into its parts, as JSON-ready hashes; a bare address has no name.
+      def parse_address(address)
+        match = ADDRESS.match(address)
+        return { "email" => JS.trim(address) } unless match
+
+        name = match[1].sub(QUOTED, '\1')
+        name.empty? ? { "email" => JS.trim(match[2]) } : { "email" => JS.trim(match[2]), "name" => name }
+      end
+    end
+  end
+end

data/lib/cronwatch/alerts/honeybadger.rb

@@ -0,0 +1,59 @@
+# frozen_string_literal: true
+
+module Cronwatch
+  module Alerts
+    # Reports alerts to Honeybadger as error notices, one error per job and
+    # alert type (https://docs.honeybadger.io/api/reporting-exceptions/). Not
+    # Check-ins, a separate product. Recoveries are not sent unless
+    # `recovered: true`: Honeybadger has no levels, so one would read as an error.
+    class Honeybadger
+      CLASS = {
+        missed: "CronWatch::Missed", failed: "CronWatch::Failed", stuck: "CronWatch::Stuck", slow: "CronWatch::Slow",
+        over_budget: "CronWatch::OverBudget", recovered: "CronWatch::Recovered",
+      }.freeze
+
+      attr_reader :name
+
+      # endpoint: another API host, "https://eu-api.honeybadger.io" say.
+      def initialize(api_key:, environment: nil, endpoint: nil, recovered: false, link: nil, http: HTTP.default)
+        @api_key = Provider.require_credential(api_key, "Cronwatch::Alerts::Honeybadger needs an api_key")
+        @url = "#{(endpoint.nil? ? "https://api.honeybadger.io" : endpoint.to_s).sub(%r{/+\z}, "")}/v1/notices"
+        @environment = environment
+        @recovered = recovered
+        @link = link
+        @http = http
+        @name = "honeybadger"
+      end
+
+      def call(alert)
+        return if alert.type == :recovered && !@recovered
+
+        link = Provider.link_for(@link, alert)
+        type = alert.type.to_s
+        request = { "component" => "cronwatch", "action" => alert.job }
+        request["url"] = link if link
+        context = { "job" => alert.job, "type" => type }
+        context["triage"] = alert.triage if Provider.present?(alert.triage)
+        context["details"] = Naming.to_json_value(alert.details)
+        context["run"] = Provider.run_summary(alert)
+        request["context"] = context
+        notice = {
+          "notifier" => { "name" => "cronwatch", "url" => "https://cronwatch.dev" },
+          "error" => {
+            "class" => CLASS[alert.type.to_sym],
+            "message" => Provider.cut("#{alert.title}\n#{alert.message}", 8000),
+            # No code ran here; one frame naming the job keeps the notice well formed.
+            "backtrace" => [{ "number" => "0", "file" => "cronwatch/#{alert.job}", "method" => type }],
+            "fingerprint" => "cronwatch:#{alert.job}:#{type}",
+            "tags" => ["cronwatch", type],
+          },
+          "request" => request,
+          "server" => { "environment_name" => @environment.nil? ? "production" : @environment },
+        }
+        headers = { "content-type" => "application/json", "accept" => "application/json", "x-api-key" => @api_key }
+        Provider.post(@http, "Honeybadger", @url, headers, JS.json(notice), [@api_key])
+        nil
+      end
+    end
+  end
+end

data/lib/cronwatch/alerts/mailgun.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+module Cronwatch
+  module Alerts
+    # Sends alerts as email through Mailgun
+    # (https://documentation.mailgun.com/docs/mailgun/api-reference/send/mailgun/messages),
+    # form encoded with basic auth. `domain` is the sending domain,
+    # "mg.example.com"; `region: "eu"` for a domain in the EU region.
+    class Mailgun
+      attr_reader :name
+
+      def initialize(api_key:, domain:, from:, to:, region: nil, subject_prefix: nil, link: nil, http: HTTP.default)
+        @api_key = Provider.require_credential(api_key, "Cronwatch::Alerts::Mailgun needs an api_key")
+        domain = Provider.require_option(domain, "Cronwatch::Alerts::Mailgun needs a domain")
+        @to = Email.recipients("Mailgun", from, to)
+        @from = from
+        host = region.to_s == "eu" ? "https://api.eu.mailgun.net" : "https://api.mailgun.net"
+        @url = "#{host}/v3/#{Provider.encode_uri_component(domain)}/messages"
+        @subject_prefix = subject_prefix
+        @link = link
+        @http = http
+        @name = "mailgun"
+      end
+
+      def call(alert)
+        email = Email.compose(alert, from: @from, to: @to, subject_prefix: @subject_prefix, link: @link)
+        pairs = [["from", email.from]]
+        email.to.each { |address| pairs << ["to", address] }
+        pairs.push(["subject", email.subject], ["text", email.text], ["html", email.html], ["o:tag", "cronwatch"])
+        headers = { "content-type" => "application/x-www-form-urlencoded", "authorization" => Provider.basic_auth("api", @api_key) }
+        Provider.post(@http, "Mailgun", @url, headers, Provider.form(pairs), [@api_key])
+        nil
+      end
+    end
+  end
+end

data/lib/cronwatch/alerts/newrelic.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+module Cronwatch
+  module Alerts
+    # Records alerts as New Relic custom events through the Event API
+    # (https://docs.newrelic.com/docs/data-apis/ingest-apis/event-api/introduction-event-api/).
+    # Each alert is one event of type CronWatchAlert (or `event_type`),
+    # queryable with NRQL: SELECT * FROM CronWatchAlert WHERE job = 'nightly'.
+    # `api_key` is a license key; `region: "eu"` for an EU account.
+    class NewRelic
+      attr_reader :name
+
+      def initialize(account_id:, api_key:, region: nil, event_type: nil, link: nil, http: HTTP.default)
+        @api_key = Provider.require_credential(api_key, "Cronwatch::Alerts::NewRelic needs an api_key")
+        account = account_id.nil? ? "" : account_id.to_s
+        raise ArgumentError, "Cronwatch::Alerts::NewRelic needs a numeric account_id" unless /\A\d+\z/.match?(account)
+
+        host = region.to_s == "eu" ? "https://insights-collector.eu01.nr-data.net" : "https://insights-collector.newrelic.com"
+        @url = "#{host}/v1/accounts/#{account}/events"
+        @event_type = event_type.nil? ? "CronWatchAlert" : event_type.to_s
+        @link = link
+        @http = http
+        @name = "newrelic"
+      end
+
+      def call(alert)
+        link = Provider.link_for(@link, alert)
+        run = alert.run
+        # Flat attributes only, strings under 4096 characters.
+        event = {
+          "eventType" => @event_type,
+          "timestamp" => alert.at,
+          "job" => Provider.cut(alert.job, 4095),
+          "alertType" => alert.type.to_s,
+          "severity" => Provider.severity(alert.type),
+          "title" => Provider.cut(alert.title, 4095),
+          "message" => Provider.cut(alert.message, 4095),
+        }
+        event["triage"] = Provider.cut(alert.triage, 4095) if Provider.present?(alert.triage)
+        event["link"] = Provider.cut(link, 4095) if link
+        if run
+          event["runId"] = run.id
+          event["runStatus"] = run.status.to_s
+          event["durationMs"] = run.duration_ms unless run.duration_ms.nil?
+        end
+        headers = { "content-type" => "application/json", "api-key" => @api_key }
+        Provider.post(@http, "New Relic", @url, headers, JS.json([event]), [@api_key])
+        nil
+      end
+    end
+  end
+end

data/lib/cronwatch/alerts/postmark.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+module Cronwatch
+  module Alerts
+    # Sends alerts as email through Postmark
+    # (https://postmarkapp.com/developer/api/email-api). `message_stream`
+    # defaults to "outbound", the transactional stream.
+    class Postmark
+      ENDPOINT = "https://api.postmarkapp.com/email"
+
+      attr_reader :name
+
+      def initialize(server_token:, from:, to:, message_stream: nil, subject_prefix: nil, link: nil, http: HTTP.default)
+        @server_token = Provider.require_credential(server_token, "Cronwatch::Alerts::Postmark needs a server_token")
+        @to = Email.recipients("Postmark", from, to)
+        @from = from
+        @message_stream = message_stream
+        @subject_prefix = subject_prefix
+        @link = link
+        @http = http
+        @name = "postmark"
+      end
+
+      def call(alert)
+        email = Email.compose(alert, from: @from, to: @to, subject_prefix: @subject_prefix, link: @link)
+        headers = { "content-type" => "application/json", "accept" => "application/json", "x-postmark-server-token" => @server_token }
+        body = JS.json({
+          "From" => email.from,
+          "To" => email.to.join(", "),
+          "Subject" => email.subject,
+          "TextBody" => email.text,
+          "HtmlBody" => email.html,
+          "MessageStream" => @message_stream.nil? ? "outbound" : @message_stream,
+          "Tag" => "cronwatch",
+        })
+        Provider.post(@http, "Postmark", ENDPOINT, headers, body, [@server_token])
+        nil
+      end
+    end
+  end
+end

data/lib/cronwatch/alerts/provider.rb

@@ -0,0 +1,159 @@
+# frozen_string_literal: true
+
+require "digest"
+require "openssl"
+require "uri"
+
+module Cronwatch
+  module Alerts
+    # What the provider channels (email, SMS, error trackers) share, as the
+    # SDK's alerts/shared.ts: the POST that names the provider and the URL's
+    # origin on failure with every secret cut out, the stable alert id, and
+    # the run summary trackers attach. Standard library only.
+    module Provider
+      module_function
+
+      # Severity for trackers that have levels. Recovered is informational.
+      def severity(type)
+        case type.to_sym
+        when :recovered then "info"
+        when :slow, :over_budget then "warning"
+        else "error"
+        end
+      end
+
+      # The scheme, host and port only. A URL's path or query can hold a credential.
+      def origin(url)
+        Webhook.origin(url)
+      end
+
+      # How much of a provider's error body goes into the error message.
+      ERROR_BODY_MAX = 200
+
+      # POSTs and raises on a non-2xx answer. The error names the provider and
+      # the URL's origin, plus the start of the response body with every
+      # secret the channel holds cut out, in case a provider echoes one back
+      # (see error_body). A redirect is not followed (Net::HTTP never follows
+      # one): its 3xx is an error like any other answer outside 2xx, so the
+      # credential headers never go where it points.
+      def post(http, provider, url, headers, body, secrets = [])
+        response = http.post(url, body, headers)
+        return response if response.ok?
+
+        # response.text() drops a leading byte order mark.
+        text = Output.utf8(response.body.to_s).delete_prefix("\u{FEFF}")
+        raise "#{provider} #{origin(url)} answered #{response.status}#{text.empty? ? "" : ": #{error_body(text, secrets)}"}"
+      end
+
+      # The start of an error body: secrets are cut out of a prefix long
+      # enough to hold one that starts inside the first ERROR_BODY_MAX
+      # characters, and only then is it cut to that length, on a code point,
+      # so no part of a secret survives at the edge.
+      def error_body(text, secrets = [])
+        kept = secrets.select { |s| s.is_a?(String) && JS.length16(s) >= 4 }
+        longest = kept.map { |s| JS.length16(s) }.max || 0
+        head = JS.head16(text, ERROR_BODY_MAX + longest)
+        kept.each { |secret| head = head.split(secret, -1).join("[redacted]") }
+        JS.head16(head, ERROR_BODY_MAX)
+      end
+
+      # A credential with the spaces and newlines a paste leaves around it
+      # taken off, as it goes in a header. Anything not a String is "".
+      def trimmed(value)
+        value.is_a?(String) ? JS.trim(value) : ""
+      end
+
+      # Base64 of UTF-8, on one line.
+      def base64(text)
+        [text.to_s.b].pack("m0")
+      end
+
+      def basic_auth(user, password)
+        "Basic #{base64("#{user}:#{password}")}"
+      end
+
+      def sha256_hex(text)
+        Digest::SHA256.hexdigest(text.to_s)
+      end
+
+      # A stable 32 hex character id for one alert: the same job, type and
+      # time always give the same id, so a provider that deduplicates on it
+      # drops a resend of an alert it already took.
+      def alert_id(alert)
+        sha256_hex("#{alert.job}\n#{alert.type}\n#{JS.number(alert.at)}")[0, 32]
+      end
+
+      # The same id laid out as a UUID, for APIs that ask for one.
+      def as_uuid(id)
+        "#{id[0, 8]}-#{id[8, 4]}-#{id[12, 4]}-#{id[16, 4]}-#{id[20, 12]}"
+      end
+
+      # At most `max` UTF-16 units, without splitting a surrogate pair.
+      def cut(text, max)
+        JS.head16(text, max)
+      end
+
+      # The run fields worth attaching to a tracker event.
+      def run_summary(alert)
+        run = alert.run
+        return nil unless run
+
+        {
+          "id" => run.id, "status" => run.status.to_s, "startedAt" => JS.iso(run.started_at),
+          "durationMs" => run.duration_ms, "trigger" => run.trigger,
+        }
+      end
+
+      # Title, message, triage and link as one plain text block, the way every channel reads.
+      def plain_text(alert, link)
+        lines = [alert.title, "", alert.message]
+        lines.push("", "Triage: #{alert.triage}") if present?(alert.triage)
+        lines.push("", "Open: #{link}") if present?(link)
+        lines.join("\n")
+      end
+
+      # JavaScript's truthiness for an optional string: nil and "" are absent.
+      def present?(text)
+        !text.nil? && text != ""
+      end
+
+      # The link option's answer for this alert, or nil.
+      def link_for(link, alert)
+        value = link&.call(alert)
+        present?(value) ? value.to_s : nil
+      end
+
+      # encodeURIComponent.
+      def encode_uri_component(text)
+        text.to_s.b.gsub(/[^A-Za-z0-9\-_.!~*'()]/n) { |c| format("%%%02X", c.ord) }.force_encoding(Encoding::UTF_8)
+      end
+
+      # URLSearchParams#toString for these pairs: application/x-www-form-urlencoded.
+      def form(pairs)
+        pairs.map { |k, v| "#{URI.encode_www_form_component(k)}=#{URI.encode_www_form_component(v.to_s)}" }.join("&")
+      end
+
+      # A required option, as a string: raises when it is missing or empty.
+      def require_option(value, message)
+        raise ArgumentError, message if value.nil? || value.to_s.empty?
+
+        value.to_s
+      end
+
+      # A required credential, trimmed (see trimmed): raises when it is missing,
+      # not a String, or only whitespace. A pasted credential often carries a
+      # stray space or newline, which a header would refuse or send.
+      def require_credential(value, message)
+        credential = trimmed(value)
+        raise ArgumentError, message if credential.empty?
+
+        credential
+      end
+
+      # The recovered option. `default` is what the SDK does when it is not given.
+      def recovered?(option, default)
+        option.nil? ? default : option != false
+      end
+    end
+  end
+end