cronwatch 0.7.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4717ad1d0fcfb080755faed7d4f018257f1b382e3076ab3e16d6830d9f985732
-  data.tar.gz: a907c315c80b237b09eb40087cfa7f28f5ed261494686b00f4c6df3710afeaf1
+  metadata.gz: 6df945edfb266ca737de6343276860554a4d3d740331091e1167d00fe93888a4
+  data.tar.gz: ec33c3140fb5f453de2cd0263b74a894b590408394f9f1aade76f1be2485e190
 SHA512:
-  metadata.gz: 201500f44422be56f7c741a3dfd200ef7b0eabd893b3ea7466b8c3504f36d834f07d2a3a41e55d5035255abba46c0462498c9920c8c0d12e19843f9e3d8b1daa
-  data.tar.gz: c6fd81253765bd342f698cfa93ff66a87b13e20fee82d7d6c48972d8c25769b8cb48db1971bc082cfb0fc1660f1900535279135169514615724668b1aa21e0c4
+  metadata.gz: ba15e440f49e390715b9668a24c0ee2b1c0cf35dac907a461555d18181fc95572b0ba3fee50b00237893733ca5b864b9f35b6a4e30b0f0bee64e6eefa1f3c5b0
+  data.tar.gz: 71e579e3ebba4a279cfef5de79d611157dcc2fc8ad176fac77cc737e7e0750c9ecd3cfefed383b2ac51f2f7aef5a5a24a24a472b6b739e90c1bd007a5814f0f5

data/README.md

@@ -2,7 +2,7 @@
 
 Cron and scheduled-job monitoring that lives inside your Ruby or Rails app. Wrap a job once; every run is recorded in a database you already have, and you are told when a run is missed, fails, gets stuck, runs slow or goes over budget. No server to run, no account to make.
 
-This is the Ruby port of [`@cronwatch/sdk`](https://www.npmjs.com/package/@cronwatch/sdk): the same rules, the same alert text and the same stored rows, so a Ruby process and a Node process can share one database, and [`@cronwatch/mcp`](https://www.npmjs.com/package/@cronwatch/mcp) works against either.
+This is the Ruby port of [`@cronwatch/sdk`](https://www.npmjs.com/package/@cronwatch/sdk): the same rules, the same alert text and the same stored rows, so a Ruby process can share one database with a Node, Python, PHP, Go or Rust process, and [`@cronwatch/mcp`](https://www.npmjs.com/package/@cronwatch/mcp) works against any of them.
 
 Docs: [cronwatch.dev/docs/rails](https://cronwatch.dev/docs/rails/) and [cronwatch.dev/docs/ruby](https://cronwatch.dev/docs/ruby/)
 
@@ -20,13 +20,13 @@ Ruby 3.2 or newer; the Rails integration is tested on Rails 7.2, 8.0 and 8.1. Th
 | `cronwatch` | the client, the memory store, and the Slack, Discord, webhook, console, email (Resend, Postmark, SendGrid, Mailgun, SES), Twilio and error tracker (Sentry, Honeybadger, Datadog, Rollbar, Bugsnag, New Relic) channels | |
 | `cronwatch/pg_cron` | `Cronwatch::Sources::PgCron`, which watches pg_cron jobs through an ActiveRecord or pg connection | |
 | `cronwatch/active_record` | the ActiveRecord store | `activerecord` |
-| `cronwatch/rails` | the Railtie, `Cronwatch::ActiveJob`, `Cronwatch::CheckJob`, the `cronwatch:check` task, the install generator | `railties`, `activejob` |
+| `cronwatch/rails` | the Railtie, `Cronwatch::ActiveJob`, `Cronwatch::CheckJob`, `Cronwatch::Web`, the `cronwatch:check` task, the install generator | `railties`, `activejob` |
 | `cronwatch/sidekiq` | `Cronwatch::Sidekiq` for `Sidekiq::Job` classes, its server middleware, `Cronwatch::Sidekiq::CheckWorker` | `sidekiq` 7 or newer |
 | `cronwatch/scheduler` | schedules read from Solid Queue's or sidekiq-cron's config, `Cronwatch.declare_from_scheduler!` | |
 | `cronwatch/web` | the dashboard and JSON API as a Rack app | `rack` |
 | `cronwatch/triage/anthropic` | Claude triage | `anthropic` |
 
-In a Rails app, `gem "cronwatch"` also loads `cronwatch/rails` and `cronwatch/scheduler` (Bundler requires it after Rails), `cronwatch/sidekiq` when Sidekiq is in the bundle, and the ActiveRecord store on first use. `cronwatch/web` and `cronwatch/triage/anthropic` are always required by hand.
+In a Rails app, `gem "cronwatch"` also loads `cronwatch/rails`, `cronwatch/web` and `cronwatch/scheduler` (Bundler requires it after Rails), `cronwatch/sidekiq` when Sidekiq is in the bundle, and the ActiveRecord store on first use. `cronwatch/triage/anthropic` is always required by hand, and so is `cronwatch/web` outside Rails.
 
 ## Plain Ruby
 
@@ -64,7 +64,7 @@ run.finish                          # or run.fail(error); finish("text") and fin
 
 Neither raises for the store: failures go to `on_error`, and a finish of a run already finished, not found, or of another job records nothing and returns nil. A run is judged once, however many processes finish it at once, and a store that fails during `finish` leaves the handle active to finish again. A run never finished is marked stuck after the job's `timeout`.
 
-Client options: `store`, `alerts`, `triage`, `cron_secret`, `retention` (default `"30d"`), `defaults`, `redact` (default: blank values that look like secrets; `false` keeps output as logged, or pass a callable), `deliver` (`:now` by default; `:check` queues alerts for another process's check to send, for a worker that cannot reach Slack), `on_error` and `now`. Methods: `job`, `run`, `resume_run`, `check`, `start`/`stop`, `silence(name, for: "2h")`/`unsilence`, `forget`, `jobs`, `jobs_with_runs`, `job_summary`, `runs`, `get_run`, `defined_jobs`, `close`. [cronwatch.dev/docs/ruby](https://cronwatch.dev/docs/ruby/#api) has each one.
+Client options: `store`, `alerts`, `triage`, `cron_secret`, `retention` (default `"30d"`), `defaults`, `sources` (where runs the process does not wrap come from, such as pg_cron), `redact` (default: blank values that look like secrets; `false` keeps output as logged, or pass a callable), `deliver` (`:now` by default; `:check` queues alerts for another process's check to send, for a worker that cannot reach Slack), `on_error` and `now`. Methods: `job`, `run`, `resume_run`, `record_run`, `check`, `start`/`stop`, `silence(name, for: "2h")`/`unsilence`, `forget`, `jobs`, `jobs_with_runs`, `job_summary`, `runs`, `get_run`, `defined_jobs`, `close`. [cronwatch.dev/docs/ruby](https://cronwatch.dev/docs/ruby/#api) has each one.
 
 ## Rails
 
@@ -130,7 +130,7 @@ reads the class's one entry in Solid Queue's `config/recurring.yml` (the section
 
 ### Scheduling the check
 
-Failures are caught as they happen, but a run that never started or never finished can only be noticed by looking. `Cronwatch::CheckJob` looks: it loads `app/jobs` (and `app/workers`) when the app does not eager load, declares every monitored job and runs the check. Run it every five minutes.
+Failures are caught as they happen, but a run that never started or never finished can only be noticed by looking. `Cronwatch::CheckJob` looks: it loads `app/jobs` (and `app/workers` and `app/sidekiq`) when the app does not eager load, declares every monitored job and runs the check. Run it every five minutes.
 
 Solid Queue:
 
@@ -168,7 +168,7 @@ Rails.application.routes.draw do
 end
 ```
 
-Set `CRONWATCH_TOKEN` to a long random string and open `/cronwatch?token=<it>` once; the browser keeps a cookie. Without a token, while `RAILS_ENV` or `RACK_ENV` is `development` or `test`, it makes a token of its own and prints a sign-in link to standard output on its first request (open it once; unless `origin:` is set or the request's host is loopback, the link leaves the host out, since a client chooses it); anywhere else it answers 503. To rely on the app's own sign in, mount it behind that (Devise's `authenticate` block, or a routing constraint) and pass `token: nil`. The URLs, JSON shapes, headers and CSRF rules are the SDK's, so the MCP server reads it unchanged. The request's origin, which the CSRF check compares against, is what Rack reports and so already follows `X-Forwarded-Host` and `X-Forwarded-Proto`; pass `origin: "https://app.example.com"` to pin it instead. `GET /cronwatch/api/check` with a bearer (the token or `CRON_SECRET`) runs the check, for an outside cron.
+Set `CRONWATCH_TOKEN` to a long random string and open `/cronwatch?token=<it>` once; the browser keeps a cookie. Without a token, while the environment (`Rails.env`, else `RAILS_ENV` or `RACK_ENV`) is `development` or `test`, it makes a token of its own and prints a sign-in link to standard output on its first request (open it once; unless `origin:` is set or the request's host is loopback, the link leaves the host out, since a client chooses it); anywhere else it answers 503. To rely on the app's own sign in, mount it behind that (Devise's `authenticate` block, or a routing constraint) and pass `token: nil`. The URLs, JSON shapes, headers and CSRF rules are the SDK's, so the MCP server reads it unchanged. The request's origin, which the CSRF check compares against, is what Rack reports and so already follows `X-Forwarded-Host` and `X-Forwarded-Proto`; pass `origin: "https://app.example.com"` to pin it instead. `GET /cronwatch/api/check` with a bearer (the token or `CRON_SECRET`) runs the check, for an outside cron.
 
 There is no `handler()` as in the TypeScript SDK: for a job triggered over HTTP, wrap the controller action's body in `CW.job(...).run` (declared once, at boot) and check the bearer in the controller.
 
@@ -225,7 +225,7 @@ Adds two to four sentences from Claude (likely cause, first thing to check) to e
 
 ## Sharing a database with a Node app
 
-The ActiveRecord store writes the same three tables as `@cronwatch/sdk/postgres` and `@cronwatch/sdk/sqlite`: same names, columns and indexes, epoch milliseconds in the time columns, the SDK's camelCase JSON in the JSON columns. [`test/active_record/node_compat_test.rb`](https://github.com/phillips-jon/cronwatch/blob/main/packages/ruby/test/active_record/node_compat_test.rb) runs the SDK's stores in Node beside this one, on SQLite and Postgres, and checks that each reads what the other wrote, that the tables are the same whoever creates them, and that the rows are the same bytes. Use the same prefix on both sides and give each job a name only one side uses. Then one dashboard, Rails or Node, shows every job, and one MCP server reads them all.
+The ActiveRecord store writes the same three tables as `@cronwatch/sdk/postgres` and `@cronwatch/sdk/sqlite`: same names, columns and indexes, epoch milliseconds in the time columns, the SDK's camelCase JSON in the JSON columns. [`test/active_record/node_compat_test.rb`](https://github.com/phillips-jon/cronwatch/blob/main/packages/ruby/test/active_record/node_compat_test.rb) runs the SDK's stores in Node beside this one, on SQLite and Postgres, and checks that each reads what the other wrote, that the tables are the same whoever creates them, and that the rows are the same bytes. Use the same prefix on both sides and give each job a name only one side uses. Then one dashboard, Rails or Node, shows every job, and one MCP server reads them all. The Python, PHP, Go and Rust ports keep the same tables too, so a process in any of them can share the database the same way.
 
 ## Kept in step with the TypeScript SDK
 

data/lib/cronwatch/alerts/provider.rb

@@ -93,13 +93,14 @@ module Cronwatch
         JS.head16(text, max)
       end
 
-      # The run fields worth attaching to a tracker event.
+      # The run fields worth attaching to a tracker event. A start before the
+      # year 1 or after 9999 is nil.
       def run_summary(alert)
         run = alert.run
         return nil unless run
 
         {
-          "id" => run.id, "status" => run.status.to_s, "startedAt" => JS.iso(run.started_at),
+          "id" => run.id, "status" => run.status.to_s, "startedAt" => Duration.iso_time(run.started_at),
           "durationMs" => run.duration_ms, "trigger" => run.trigger,
         }
       end

data/lib/cronwatch/client.rb

@@ -229,7 +229,7 @@ module Cronwatch
 
       finished_at = now
       run.finished_at = finished_at
-      run.duration_ms = [0, finished_at - started_at].max
+      run.duration_ms = Evaluate.run_duration(started_at, finished_at)
       run.metrics = recorder.metrics
       returned = result.is_a?(String) ? Output.utf8(result) : nil
       run.output = recorder.output || (returned && Output.cap(returned))
@@ -326,6 +326,9 @@ module Cronwatch
       input = run.is_a?(Run) ? run : Run.from_h(run.is_a?(Hash) ? run.to_h { |k, v| [Naming.camel(k), v] } : run)
       declared = @registry.synchronize { @definitions[input.job] }
       raise ArgumentError, "record_run: job \"#{input.job}\" is not declared; call job first" unless declared
+      if input.id.to_s.include?("\0")
+        raise ArgumentError, "record_run: run ids cannot contain a NUL character (job \"#{input.job}\")"
+      end
 
       sync(declared)
       run = input.dup
@@ -537,6 +540,8 @@ module Cronwatch
     def reset_process_state
       @pid = Process.pid
       @locks = {}
+      # Each job's turn to write its declaration. See sync.
+      @syncing = {}
       @check_lock = Mutex.new
       @checking = nil
       @ticker_lock = Mutex.new
@@ -642,18 +647,34 @@ module Cronwatch
       end
     end
 
+    # Writes the declaration of `definition`'s name as it stands, unless the
+    # store has it. A handle kept from an earlier declaration writes the one
+    # that replaced it, never its own over it, and one forgotten since writes
+    # its own. The writes of one name take turns (the name's lock in
+    # @syncing, held across the write), so one still under way cannot land
+    # after a later one; and a name declared again while its write was under
+    # way is still to be written.
     def sync(definition)
       ensure_ready
-      return if @registry.synchronize { @synced.include?(definition.name) }
+      name = definition.name
+      return if @registry.synchronize { @synced.include?(name) }
 
-      @store.upsert_job(Serialize.to_stored(definition), now)
-      @registry.synchronize { @synced << definition.name }
+      after_fork_check
+      turn = @registry.synchronize { @syncing[name] ||= Mutex.new }
+      turn.synchronize do
+        standing = @registry.synchronize { @synced.include?(name) ? nil : @definitions.fetch(name, definition) }
+        next if standing.nil?
+
+        @store.upsert_job(Serialize.to_stored(standing), now)
+        @registry.synchronize { @synced << name if @definitions.fetch(name, standing).equal?(standing) }
+      end
     end
 
     # Runs the block while holding the job's lock, so two runs (or a run and a
     # check) in this process never read and write the job's state over each
     # other. Other processes are coordinated by update_state instead. Only
-    # store reads and writes happen inside; alerts are sent outside it.
+    # store reads and writes happen inside; alerts are sent outside it. sync
+    # gives a job's declaration writes turns the same way, on a lock of their own.
     def serial(job, &block)
       after_fork_check
       lock = @registry.synchronize { @locks[job] ||= Monitor.new }
@@ -687,7 +708,7 @@ module Cronwatch
           state, result = yield(current)
           break [current, result] if same_state?(state, current)
 
-          version = current.version || 0
+          version = Evaluate.state_version(current)
           following = state.dup
           following.version = version + 1
           break [following, result] if write_state(following, version)
@@ -938,7 +959,7 @@ module Cronwatch
       run = from.dup
       run.status = :running
       run.finished_at = finished_at
-      run.duration_ms = [0, finished_at - from.started_at].max
+      run.duration_ms = Evaluate.run_duration(from.started_at, finished_at)
       run.error = nil
       run.output = join_output(from.output, added)
       run.metrics = (from.metrics || {}).merge(recorder.metrics)
@@ -991,6 +1012,8 @@ module Cronwatch
         got = id.is_a?(String) ? "#{JS.length16(id)} characters" : id.class.to_s
         raise ArgumentError, "job \"#{job}\": #{method}() needs a run id of 1 to 200 characters (got #{got})"
       end
+      # Postgres refuses NUL in text, so no store could hold such an id.
+      raise ArgumentError, "job \"#{job}\": #{method}() cannot take a run id containing a NUL character" if id.include?("\0")
       return unless id.start_with?(RESERVED_RUN_ID_PREFIX)
 
       raise ArgumentError, "job \"#{job}\": #{method}() cannot take a run id starting with \"#{RESERVED_RUN_ID_PREFIX}\", " \
@@ -1063,7 +1086,7 @@ module Cronwatch
         timeout = Evaluate.timeout_ms(definition)
         run.status = :timeout
         run.finished_at = at
-        run.duration_ms = at - run.started_at
+        run.duration_ms = Evaluate.run_duration(run.started_at, at)
         run.error = "Still running after #{Duration.format(timeout)}; marked as timed out"
         # Only over a row still running: a finish that landed meanwhile wins.
         next unless write_run_if(run, [:running])

data/lib/cronwatch/duration.rb

@@ -83,5 +83,22 @@ module Cronwatch
       text = format(abs)
       diff.negative? ? "#{text} ago" : "in #{text}"
     end
+
+    # The first millisecond written as a date: 0001-01-01T00:00:00.000Z.
+    FIRST_DATE_MS = -62_135_596_800_000
+    # The last millisecond written as a date: 9999-12-31T23:59:59.999Z.
+    LAST_DATE_MS = 253_402_300_799_999
+
+    # "2026-01-05T09:30:00.000Z", or nil for a time before the year 1 or
+    # after 9999. A start read from another process's row, or a damaged one,
+    # can be any number; outside those years it is not written as a date.
+    def iso_time(at)
+      at >= FIRST_DATE_MS && at <= LAST_DATE_MS ? JS.iso(at) : nil
+    end
+
+    # The words that stand in for a time iso_time does not write.
+    def beyond_dates(at)
+      at > LAST_DATE_MS ? "after 9999-12-31 23:59:59 UTC" : "before 0001-01-01 00:00:00 UTC"
+    end
   end
 end

data/lib/cronwatch/evaluate.rb

@@ -18,8 +18,49 @@ module Cronwatch
     CheckEvaluation = Struct.new(:state, :alerts, :next_expected_at, :due_at, keyword_init: true)
     SlowThreshold = Struct.new(:threshold_ms, :basis, keyword_init: true)
 
+    # The longest duration written: the largest integer JavaScript holds
+    # exactly, which every port and store reads back unchanged.
+    MAX_DURATION_MS = 9_007_199_254_740_991
+
     module_function
 
+    # How long a run took, from `started_at` to `finished_at`: 0 when it
+    # started later, and never more than MAX_DURATION_MS. A foreign row's
+    # start near a 64-bit limit must not make a duration no store can write.
+    def run_duration(started_at, finished_at)
+      ms = finished_at - started_at
+      return 0 unless ms.is_a?(Numeric) && ms.positive?
+
+      ms = ms.to_i if ms.is_a?(Float) && ms.finite? && ms == ms.floor
+      [ms, MAX_DURATION_MS].min
+    end
+
+    # The version a stored state counts as for compare_and_set_state: its
+    # `version` when that is a whole number from 0 to MAX_DURATION_MS
+    # (2**53 - 1), else 0, as when it is absent. The SQL stores read it the
+    # same way, so a foreign row's 1.5, "x" or -1 is written over by the next
+    # update instead of refusing every compare-and-set of its job for good.
+    def state_version(state)
+      version = state&.version
+      return 0 unless version.is_a?(Integer) || (version.is_a?(Float) && version.finite? && version == version.floor)
+      return 0 unless version >= 0 && version <= MAX_DURATION_MS
+
+      version.to_i
+    end
+
+    # The failures in a row a stored state counts as: its `consecutive_failures`
+    # when that is a whole number, held at MAX_DURATION_MS (2**53 - 1), and 0
+    # when it is negative or not a whole number. A foreign row's count past
+    # 2**53 stays at the top, as the SDK holds it, and a 1.5, "3" or -1
+    # counts as none.
+    def failure_count(state)
+      count = state&.consecutive_failures
+      return 0 unless count.is_a?(Integer) || (count.is_a?(Float) && count.finite? && count == count.floor)
+      return 0 unless count.positive?
+
+      [count, MAX_DURATION_MS].min.to_i
+    end
+
     def empty_state(job)
       JobState.new(job: job, open: {}, consecutive_failures: 0, silenced_until: nil, last_alert_at: nil,
                    pending_recovery: [], undelivered: [])
@@ -33,7 +74,7 @@ module Cronwatch
       JobState.new(
         job: state.job.nil? ? job : state.job,
         open: (state.open || {}).dup,
-        consecutive_failures: state.consecutive_failures.nil? ? 0 : state.consecutive_failures,
+        consecutive_failures: failure_count(state),
         silenced_until: state.silenced_until,
         last_alert_at: state.last_alert_at,
         pending_recovery: (state.pending_recovery || []).dup,
@@ -205,7 +246,8 @@ module Cronwatch
       end
 
       # failed or timeout
-      next_state.consecutive_failures += 1
+      # Held at the top, as the SDK holds it: a count at the limit stays there.
+      next_state.consecutive_failures = [next_state.consecutive_failures + 1, MAX_DURATION_MS].min
       close_condition(next_state, :missed)
       threshold = [1, definition.failures_before_alert || 1].max
       condition = run.status == :timeout ? :stuck : :failed

data/lib/cronwatch/format.rb

@@ -9,7 +9,10 @@ module Cronwatch
     def at_time(at, now)
       return "never" if at.nil?
 
-      "#{JS.iso(at).tr("T", " ")[0, 19]} UTC (#{Duration.relative(at, now)})"
+      iso = Duration.iso_time(at)
+      return Duration.beyond_dates(at) if iso.nil?
+
+      "#{iso.tr("T", " ")[0, 19]} UTC (#{Duration.relative(at, now)})"
     end
 
     def first_lines(text, n)

data/lib/cronwatch/output.rb

@@ -137,6 +137,16 @@ module Cronwatch
       text.include?("\0") ? text.delete("\0") : text
     end
 
+    # Removes every U+0000 from JSON text, keys and strings alike, by dropping
+    # each \u0000 escape (a NUL can appear in JSON no other way). Escapes are
+    # read left to right in pairs, so an escaped backslash followed by "u0000"
+    # is left as it is.
+    def strip_json_nul(json)
+      return json unless json.include?("\\u0000")
+
+      json.gsub(/\\(u0000|.)/m) { Regexp.last_match(1) == "u0000" ? "" : Regexp.last_match(0) }
+    end
+
     # "Name: message" and the first five backtrace lines, each written as
     # "    at <line>" like the frames of a JavaScript stack, capped like output.
     # An exception that stops the thread rather than reporting a problem

data/lib/cronwatch/pg_cron.rb

@@ -41,8 +41,10 @@ module Cronwatch
     # prefix:   put before every job name, to keep them apart from your own ("db:"). Also keeps run ids apart.
     # job_name: a callable giving the CronWatch name for a Job. Default its jobname with anything other than
     #           letters, digits, ".", "_", ":" and "-" turned into "-", or "pg_cron:<jobid>" when it has none.
-    #           The prefix goes in front either way.
-    # options:  grace, timeout, max_duration, expect and the rest, for every job (a hash) or per job (a callable
+    #           The prefix goes in front either way. One that raises or returns no name, like a `jobs` or
+    #           `options` callable that raises, is reported once and fails only that job, which keeps its last
+    #           declaration until the callable works again.
+    # options: grace, timeout, max_duration, expect and the rest, for every job (a hash) or per job (a callable
     #           given a Job). The schedule and timezone always come from pg_cron.
     # timezone: the timezone pg_cron reads its cron expressions in. Default the server's cron.timezone, read from
     #           pg_settings, which shows it only to roles with pg_read_all_settings; UTC (pg_cron's default) is
@@ -110,6 +112,8 @@ module Cronwatch
         @retired = Set.new
         @scanned = false
         @warned = Set.new
+        # Jobids whose callback failed, reported once until it works again.
+        @failing = Set.new
         @name = "pg_cron"
       end
 
@@ -161,7 +165,7 @@ module Cronwatch
         finish = done ? [started_at, finished_at || started_at].max : nil
         Run.new(
           id: "#{id_prefix}#{row["runid"]}", job: job, status: status, started_at: started_at, finished_at: finish,
-          duration_ms: finish.nil? ? nil : finish - started_at,
+          duration_ms: finish.nil? ? nil : Evaluate.run_duration(started_at, finish),
           error: status == :failed ? (message || "pg_cron reported the run as failed") : nil,
           output: status == :ok ? message : nil, metrics: {}, trigger: "pg_cron",
         )
@@ -269,8 +273,7 @@ module Cronwatch
                                    "scheduled: connect as that role, or give this one BYPASSRLS.")
         end
         all = rows.map { |r| job_from(r) }
-        jobs = all.select { |job| picks?(job) }
-        names, definitions = declare(host, jobs, timezone, recording)
+        names, definitions = declare(host, all, timezone, recording)
         retire_unused(host, names, definitions, all)
         return [] if !recording || names.empty?
 
@@ -316,18 +319,60 @@ module Cronwatch
         /\A\d{1,15}\z/.match?(rest) ? rest.to_i : nil
       end
 
-      # Declares each job. A paused one (active = false) keeps its failures but loses its schedule, so it is not missed.
-      # Returns [{ jobid => name }, { jobid => definition as declared }].
-      def declare(host, jobs, timezone, recording)
+      # Declares each job picked. A paused one (active = false) keeps its failures but loses its schedule, so it
+      # is not missed. Returns [{ jobid => name }, { jobid => definition as declared }].
+      def declare(host, all, timezone, recording)
         names = {}
         definitions = {}
         used = Set.new
-        jobs.each do |job|
-          name = @prefix + (@job_name ? @job_name.call(job).to_s : PgCron.job_name(job))
+        # A callback of the app's (jobs, job_name, options) that raised, or a job_name that gave no name, fails
+        # only its job, as a bad row does: reported once until it works again, and the job carries on as last
+        # declared (skipped when it never was), so its runs are still copied.
+        trouble = lambda do |job, what|
+          unless @failing.include?(job.jobid)
+            @failing << job.jobid
+            host.on_error(RuntimeError.new("pg_cron job #{job.jobid}: #{what}; it keeps its last declaration until that works"),
+                          "source pg_cron")
+          end
+          last = @known[job.jobid]
+          next if last.nil? || used.include?(last[:name])
+
+          names[job.jobid] = last[:name]
+          definitions[job.jobid] = last[:definition]
+          used << last[:name]
+        end
+        all.each do |job|
+          begin
+            picked = picks?(job)
+          rescue StandardError => e
+            trouble.call(job, "the jobs callback raised #{e.class}: #{e.message}")
+            next
+          end
+          unless picked
+            @failing.delete(job.jobid)
+            next
+          end
+          begin
+            base = @job_name ? @job_name.call(job) : PgCron.job_name(job)
+          rescue StandardError => e
+            trouble.call(job, "job_name raised #{e.class}: #{e.message}")
+            next
+          end
+          unless base.is_a?(String) || base.is_a?(Symbol)
+            trouble.call(job, "job_name returned #{base.nil? ? "nil" : base.class}, not a name")
+            next
+          end
+          begin
+            extra = (@options.respond_to?(:call) ? @options.call(job) : @options) || {}
+            extra = extra.to_h.transform_keys(&:to_sym).except(*SCHEDULE_ONLY)
+          rescue StandardError => e
+            trouble.call(job, "the options callback raised #{e.class}: #{e.message}")
+            next
+          end
+          @failing.delete(job.jobid)
+          name = @prefix + base.to_s
           name = "#{name}:#{job.jobid}" if used.include?(name)
           used << name
-          extra = (@options.respond_to?(:call) ? @options.call(job) : @options) || {}
-          extra = extra.to_h.transform_keys(&:to_sym).except(*SCHEDULE_ONLY)
           schedule = job.active && recording ? PgCron.schedule(job.schedule) : nil
           definition = {
             description: "pg_cron job #{job.jobid} in #{job.database} as #{job.username}#{job.active ? "" : " (paused)"}",

data/lib/cronwatch/schedule.rb

@@ -15,6 +15,15 @@ module Cronwatch
     # How early a run may start and still count for the fire it was meant for.
     EARLY_SLACK_MS = 60_000
 
+    # Four hundred Gregorian years: 146,097 days, a whole number of weeks,
+    # after which the calendar repeats date for date and weekday for weekday.
+    CYCLE_MS = 146_097 * 86_400_000
+    # The years the walker searches in, as croner does: a time before 400 is
+    # asked a cycle or more later (croner misreads a year below 100), and one
+    # from 2800 a cycle or more earlier (croner finds no fire past 3000).
+    WALK_FIRST_MS = -49_544_438_400_000 # 0400-01-01T00:00:00Z
+    WALK_LAST_MS = 26_192_246_400_000 # 2800-01-01T00:00:00Z
+
     Parsed = Struct.new(:kind, :source, :timezone, :every_ms, keyword_init: true) do
       include Serializable
 
@@ -80,12 +89,15 @@ module Cronwatch
     def fire_after(parsed, from)
       raise ArgumentError, "schedule \"#{parsed.source}\" was not made by Schedule.parse" unless parsed.pattern
 
-      probe = from
+      start = count_from(from)
+      return nil if start.nil?
+
+      probe = start
       4.times do
         runs = next_runs(parsed, 8, probe)
         return nil if runs.empty?
 
-        found = runs.find { |t| t > from }
+        found = runs.find { |t| t > start }
         return found if found
 
         probe += 3_600_000
@@ -101,8 +113,11 @@ module Cronwatch
       raise ArgumentError, "schedule \"#{parsed.source}\" was not made by Schedule.parse" unless parsed.pattern
 
       out = []
-      probe = from
-      last = from
+      start = count_from(from)
+      return out if start.nil?
+
+      probe = start
+      last = start
       1000.times do
         batch = next_runs(parsed, [limit + 1 - out.length, 24].min, probe)
         return out if batch.empty?
@@ -121,15 +136,39 @@ module Cronwatch
       out
     end
 
-    # Up to `count` fires, each found from the one before, as croner's nextRuns.
+    # A stored time as a cron's fires are counted from it. A start read from
+    # a foreign or damaged row can be any number: one before the year 1
+    # counts from just before its first millisecond, so the first fire of the
+    # year 1 is the next one, and one at or after the last millisecond of 9999
+    # has no fire after it at all (nil). Ported from countFrom in schedule.ts.
+    def count_from(from)
+      return nil if from >= Duration::LAST_DATE_MS
+
+      from >= Duration::FIRST_DATE_MS ? from : Duration::FIRST_DATE_MS - 1
+    end
+
+    # Up to `count` fires, each found from the one before, as croner's
+    # nextRuns, and none after 9999. A time outside the years the walker
+    # searches is moved by whole 400-year cycles into them, and its fires
+    # moved back, as runsAfter in schedule.ts asks croner: a time before 400
+    # goes forward, into the same local mean time every zone kept then, and
+    # one from 2800 goes back, to where the zone's present rules already hold.
     def next_runs(parsed, count, from)
+      from = count_from(from)
+      return [] if from.nil?
+
+      shift =
+        if from < WALK_FIRST_MS then -((from - WALK_FIRST_MS).div(CYCLE_MS)) * CYCLE_MS
+        elsif from >= WALK_LAST_MS then -((from - WALK_LAST_MS).div(CYCLE_MS) + 1) * CYCLE_MS
+        else 0
+        end
       runs = []
-      at = from
+      at = from + shift
       count.times do
         at = parsed.pattern.next_after(at, parsed.timezone)
-        break if at.nil?
+        break if at.nil? || at - shift > Duration::LAST_DATE_MS
 
-        runs << at
+        runs << (at - shift)
       end
       runs
     end
@@ -159,7 +198,9 @@ module Cronwatch
       due_at.nil? ? nil : Expectation.new(due_at: due_at, deadline: due_at + grace_ms)
     end
 
-    # The first fire that a run starting at `started_at` does not cover.
+    # The first fire that a run starting at `started_at` does not cover. A
+    # start before the year 1 covers none of them, so the first fire of the
+    # year 1 is due; after 9999 there is none (see count_from).
     def due_after_run(parsed, started_at)
       # A fire at or before the start is covered by the run itself.
       following_fire = fire_after(parsed, started_at)
@@ -189,6 +230,9 @@ module Cronwatch
     # ran early by up to an hour.
     def in_spring_forward_gap?(parsed, started_at, fire_at)
       lookback = 3 * 3_600_000
+      # Every zone kept its local mean time, with no clock change, in the year 1.
+      return false if fire_at - lookback < Duration::FIRST_DATE_MS
+
       after = utc_offset(fire_at, parsed.timezone)
       before = utc_offset(fire_at - lookback, parsed.timezone)
       gap = after - before

data/lib/cronwatch/stores/active_record.rb

@@ -180,7 +180,7 @@ module Cronwatch
 
       def upsert_job(definition, now)
         definition = JobDefinition.from_h(definition)
-        write(:upsert_job, [definition.name, JS.json(definition.to_h), now, now])
+        write(:upsert_job, [definition.name, json_text(definition.to_h), now, now])
         nil
       end
 
@@ -205,14 +205,14 @@ module Cronwatch
 
       def insert_run(run)
         write(:insert_run, [run.id, run.job, run.status.to_s, run.started_at, run.finished_at, run.duration_ms,
-                            run.error, run.output, JS.json(run.metrics || {}), run.trigger])
+                            text(run.error), text(run.output), json_text(run.metrics || {}), text(run.trigger)])
         nil
       end
 
       # A run that is gone (its job was forgotten) stays gone.
       def update_run(run)
-        write(:update_run, [run.status.to_s, run.finished_at, run.duration_ms, run.error, run.output,
-                            JS.json(run.metrics || {}), run.id])
+        write(:update_run, [run.status.to_s, run.finished_at, run.duration_ms, text(run.error), text(run.output),
+                            json_text(run.metrics || {}), run.id])
         nil
       end
 
@@ -222,8 +222,8 @@ module Cronwatch
         statuses = Array(from_statuses).map(&:to_s)
         return false if statuses.empty?
 
-        binds = [run.status.to_s, run.finished_at, run.duration_ms, run.error, run.output,
-                 JS.json(run.metrics || {}), run.id, *statuses]
+        binds = [run.status.to_s, run.finished_at, run.duration_ms, text(run.error), text(run.output),
+                 json_text(run.metrics || {}), run.id, *statuses]
         changed = with_connection { |conn| conn.exec_update(update_run_if_sql(conn, statuses.length), NAME, binds) }
         changed.to_i.positive?
       end
@@ -253,7 +253,7 @@ module Cronwatch
       end
 
       def set_state(state)
-        write(:set_state, [state.job, JS.json(state.to_h)])
+        write(:set_state, [state.job, json_text(state.to_h)])
         nil
       end
 
@@ -262,10 +262,10 @@ module Cronwatch
       # Returns whether it wrote. This is what keeps two processes sharing the
       # database (Ruby or Node) from overwriting each other's updates.
       def compare_and_set_state(state, expected_version)
-        text = JS.json(state.to_h)
+        state_text = json_text(state.to_h)
         changed =
-          if expected_version.zero? then write(:cas_insert, [state.job, text])
-          else write(:cas_update, [text, state.job, expected_version])
+          if expected_version.zero? then write(:cas_insert, [state.job, state_text])
+          else write(:cas_update, [state_text, state.job, expected_version])
           end
         changed.to_i.positive?
       end
@@ -344,8 +344,20 @@ module Cronwatch
         seq = pg ? "seq" : "rowid"
         # Byte order on both, so names sort the same whatever the database's collation.
         by_name = pg ? 'name COLLATE "C"' : "name"
-        # The version inside a state's JSON, 0 when it has none.
-        version = ->(column) { pg ? "COALESCE((#{column}->>'version')::bigint, 0)" : "COALESCE(json_extract(#{column}, '$.version'), 0)" }
+        # The version inside a state's JSON, as Evaluate.state_version reads
+        # it: a whole number from 0 to 2**53 - 1, else 0 (none, or a foreign
+        # row's 1.5 or "x", which must neither fail the statement nor refuse
+        # every write for good). Each CASE tests the JSON type before any cast.
+        version = lambda do |column|
+          if pg
+            v = "(#{column}->>'version')::numeric"
+            next "CASE WHEN jsonb_typeof(#{column}->'version') <> 'number' THEN 0 " \
+                 "WHEN #{v} % 1 = 0 AND #{v} BETWEEN 0 AND 9007199254740991 THEN #{v}::bigint ELSE 0 END"
+          end
+          v = "json_extract(#{column}, '$.version')"
+          "CASE WHEN json_type(#{column}, '$.version') NOT IN ('integer', 'real') THEN 0 " \
+            "WHEN #{v} = CAST(#{v} AS INTEGER) AND #{v} BETWEEN 0 AND 9007199254740991 THEN CAST(#{v} AS INTEGER) ELSE 0 END"
+        end
         sql = {
           upsert_job: "INSERT INTO #{p}jobs (name, definition, created_at, updated_at) VALUES (?, ?, ?, ?)\n      " \
                       "ON CONFLICT (name) DO UPDATE SET definition = excluded.definition, updated_at = excluded.updated_at",
@@ -383,6 +395,19 @@ module Cronwatch
 
       # SQLite hands back JSON as TEXT; Postgres drivers may hand back JSONB
       # parsed or as text, and BIGINT as a string.
+      # Postgres refuses U+0000 in TEXT and JSONB, and a refused write loses
+      # the whole row, so every dialect writes text without it: a run's
+      # trigger, output, error and metric names, and every key and string of
+      # a definition and a state. Identifiers (a job's name, a run's id) are
+      # written as given; the client refuses one with a NUL before it gets here.
+      def text(value)
+        value.nil? ? nil : Output.strip_nul(value.to_s)
+      end
+
+      def json_text(value)
+        Output.strip_json_nul(JS.json(value))
+      end
+
       def json(value)
         js_numbers(value.is_a?(String) ? JS.parse(value) : value)
       end

data/lib/cronwatch/stores/memory.rb

@@ -28,7 +28,7 @@ module Cronwatch
           existing = @jobs[definition.name]
           @jobs[definition.name] = StoredJob.new(
             name: definition.name,
-            definition: clone(definition, JobDefinition),
+            definition: kept(definition, JobDefinition),
             created_at: existing ? existing.created_at : now,
             updated_at: now,
           )
@@ -62,7 +62,7 @@ module Cronwatch
         sync do
           raise "run #{run.id} already exists" if @runs.key?(run.id)
 
-          @runs[run.id] = clone(run, Run)
+          @runs[run.id] = kept_run(run)
           @order[run.id] = (@seq += 1)
         end
         nil
@@ -125,7 +125,7 @@ module Cronwatch
       end
 
       def set_state(state)
-        sync { @states[state.job] = clone(state, JobState) }
+        sync { @states[state.job] = kept(state, JobState) }
         nil
       end
 
@@ -134,9 +134,9 @@ module Cronwatch
       # wrote. See JobState#version.
       def compare_and_set_state(state, expected_version)
         sync do
-          next false unless (@states[state.job]&.version || 0) == expected_version
+          next false unless Evaluate.state_version(@states[state.job]) == expected_version
 
-          @states[state.job] = clone(state, JobState)
+          @states[state.job] = kept(state, JobState)
           true
         end
       end
@@ -161,7 +161,7 @@ module Cronwatch
 
       # `existing` with the fields an update writes taken from `run`.
       def finished_fields(existing, run)
-        copy = clone(run, Run)
+        copy = kept_run(run)
         existing.dup.tap do |r|
           r.status = copy.status
           r.finished_at = copy.finished_at
@@ -181,6 +181,21 @@ module Cronwatch
       def clone(value, type)
         type.from_h(JS.parse(JS.json(value.to_h)))
       end
+
+      # Text is held as the SQL stores write it, without U+0000, so every
+      # store reads back the same. Identifiers are held as given.
+      def kept(value, type)
+        type.from_h(JS.parse(Output.strip_json_nul(JS.json(value.to_h))))
+      end
+
+      def kept_run(run)
+        clone(run, Run).tap do |copy|
+          copy.trigger = Output.strip_nul(copy.trigger.to_s)
+          copy.output = copy.output && Output.strip_nul(copy.output)
+          copy.error = copy.error && Output.strip_nul(copy.error)
+          copy.metrics = JS.parse(Output.strip_json_nul(JS.json(copy.metrics || {})))
+        end
+      end
     end
   end
 end

data/lib/cronwatch/triage/anthropic.rb

@@ -92,6 +92,11 @@ module Cronwatch
         "<job_data>\n#{text.gsub(TAG, "<_job_data")}\n</job_data>"
       end
 
+      # "2026-01-05T09:30:00.000Z", or the words for a time before the year 1 or after 9999.
+      def self.stamp(at)
+        Duration.iso_time(at) || Duration.beyond_dates(at)
+      end
+
       # The prompt: the alert, the definition, the triggering run and up to five earlier ones.
       def self.describe(triage_context)
         alert = triage_context.alert
@@ -104,7 +109,7 @@ module Cronwatch
         lines << "Job definition: #{JS.json(definition.respond_to?(:to_h) ? definition.to_h : definition)}"
         if run
           lines << ""
-          lines << "Triggering run: status #{run.status}, started #{JS.iso(run.started_at)}, duration #{duration(run)}, trigger #{run.trigger}"
+          lines << "Triggering run: status #{run.status}, started #{stamp(run.started_at)}, duration #{duration(run)}, trigger #{run.trigger}"
           lines << "Metrics: #{JS.json(run.metrics)}" if run.metrics && !run.metrics.empty?
           lines << "Error:\n#{data(JS.head16(run.error, 3000))}" if present?(run.error)
           lines << "Output (tail):\n#{data(JS.tail16(run.output, 3000))}" if present?(run.output)
@@ -116,7 +121,7 @@ module Cronwatch
           earlier.each do |r|
             error = present?(r.error) ? ", error: #{data(JS.head16(r.error.split("\n", -1).first.to_s, 160))}" : ""
             metrics = r.metrics && !r.metrics.empty? ? ", metrics #{JS.json(r.metrics)}" : ""
-            lines << "- #{r.status}, #{JS.iso(r.started_at)}, #{duration(r)}#{error}#{metrics}"
+            lines << "- #{r.status}, #{stamp(r.started_at)}, #{duration(r)}#{error}#{metrics}"
           end
         end
         lines.join("\n")

data/lib/cronwatch/version.rb

@@ -2,5 +2,5 @@
 
 module Cronwatch
   # Kept in step with @cronwatch/sdk.
-  VERSION = "0.7.0"
+  VERSION = "0.9.0"
 end

data/lib/cronwatch/web/html.rb

@@ -329,7 +329,9 @@ module Cronwatch
       def stamp(at, now)
         return %(<span class="muted">never</span>) if at.nil?
 
-        iso = JS.iso(at.to_i)
+        iso = Duration.iso_time(at.to_i)
+        return %(<span class="nowrap">#{Duration.beyond_dates(at.to_i)}</span>) if iso.nil?
+
         %(<time class="nowrap" datetime="#{iso}" title="#{iso.sub("T", " ")[0, 19]} UTC">#{h(Duration.relative(at, now))}</time>)
       end
 

data/lib/cronwatch/web/timeline.rb

@@ -86,12 +86,19 @@ module Cronwatch
         "#{WEEKDAYS[d.wday]} #{d.day} #{MONTHS[d.month - 1]}"
       end
 
-      # "22:42" on the same UTC day as `now`, otherwise "25 Sep 22:42".
+      # "22:42" on the same UTC day as `now`, otherwise "25 Sep 22:42", and
+      # "1 Jan 0001 02:00" in another UTC year. A time before the year 1 or
+      # after 9999 (a start read from a foreign or damaged row) is "before 1
+      # Jan 0001 00:00" or "after 31 Dec 9999 23:59".
       def when_at(t, now)
+        return "after 31 Dec 9999 23:59" if t > Duration::LAST_DATE_MS
+        return "before 1 Jan 0001 00:00" if t < Duration::FIRST_DATE_MS
+
         return clock(t) if t.div(DAY) == now.div(DAY)
 
         d = utc(t)
-        "#{d.day} #{MONTHS[d.month - 1]} #{clock(t)}"
+        year = d.year == utc(now).year ? "" : Kernel.format(" %04d", d.year)
+        "#{d.day} #{MONTHS[d.month - 1]}#{year} #{clock(t)}"
       end
 
       # The job's schedule, parsed, or nil when it has none or it no longer parses.

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: cronwatch
 version: !ruby/object:Gem::Version
-  version: 0.7.0
+  version: 0.9.0
 platform: ruby
 authors:
 - Jon C. Phillips