shraga 0.1.32 → 0.1.34

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -140,8 +140,9 @@ Schedule: { id, name, enabled, trigger, task, scope, createdBy, nextRun?, lastRu
140
140
 
141
141
  ## Missed windows (`onMissed` / `missedRun`)
142
142
 
143
- If the process is down when a window should have fired (deploy, crash, power cut), the scheduler
144
- decides at the next boot whether to replay it. `onMissed` is settable on `POST`/`PUT`:
143
+ If a window elapses with nothing running (deploy, crash, power cut — or the host **suspending**
144
+ with the process frozen), the scheduler decides whether to replay it. `onMissed` is settable on
145
+ `POST`/`PUT`:
145
146
 
146
147
  | `onMissed` | Behaviour |
147
148
  |---|---|
@@ -157,10 +158,20 @@ curl -s -X PUT -H "Content-Type: application/json" -H "x-internal-token: $INTERN
157
158
  - **A staleness ceiling overrides every policy, `run` included**: a window more than
158
159
  `SCHEDULER_MAX_MISSED_AGE_MS` (default **6h**) late is never replayed. Finishing an 08:00 report
159
160
  at 22:00 is not the job the schedule describes.
160
- - This applies to **both** paths that could replay a window: the boot catch-up (cron), and the
161
- in-place resume of a run interrupted mid-flight (every trigger kind — `interval`/`once`/`event`
162
- have no catch-up, so resume is their only gate; their window is the one the interrupted run
163
- recorded when it started).
161
+ - One decision (`judgeMissed` in `scheduler/engine.ts`) governs **all three** paths that could
162
+ replay a window:
163
+ 1. **Boot catch-up** (cron only) the process was down when the window passed.
164
+ 2. **In-place resume** of a run interrupted mid-flight — every trigger kind
165
+ (`interval`/`once`/`event` have no catch-up, so resume is their only gate; their window is
166
+ the one the interrupted run recorded when it started).
167
+ 3. **A late timer fire** — the process was *frozen*, not killed (laptop lid/standby, NTP step,
168
+ VM pause): same pid, no restart, so catch-up never ran, and the overdue `setTimeout` fires on
169
+ wake. The window is the schedule's `nextRun` (when it *should* have fired), not the wake time.
170
+ A punctual fire is untouched — `onMissed` is a missed-window policy, never a mute — the gate
171
+ engages only once the fire is past the ceiling.
172
+ - After a refusal the schedule is still re-armed normally: an `interval` job loses exactly one
173
+ cycle and fires again next period; a `once` is retired (disabled, `nextRun` cleared) with its
174
+ `missedRun` left for an on-demand run. Nothing is left enabled-but-never-firing.
164
175
  - Whenever a window is not replayed, the schedule gets
165
176
  `missedRun: { at, reason: "skip"|"offer"|"stale", noticedAt }` — visible on
166
177
  `GET /api/schedules` and `GET /api/schedules/{id}`. **That is the `offer` affordance**: read it,
@@ -173,9 +184,32 @@ curl -s -X PUT -H "Content-Type: application/json" -H "x-internal-token: $INTERN
173
184
  ## "What did I miss?" — downtime recovery (`GET /api/downtime`)
174
185
 
175
186
  This box is deliberately not always-on. The server heartbeats to `data/state/heartbeat.json` every
176
- `HEARTBEAT_INTERVAL_MS` (default **60s**); at boot, a gap larger than `DOWNTIME_THRESHOLD_MS`
177
- (default **3 intervals = 3m**) is appended to `data/state/downtime.json` as
178
- `{ from, to, ms, slackCursors }` (last **20** kept). A clean restart records nothing.
187
+ `HEARTBEAT_INTERVAL_MS` (default **60s**). A gap larger than `DOWNTIME_THRESHOLD_MS` (default
188
+ **3 intervals = 3m**) is appended to `data/state/downtime.json` as
189
+ `{ from, to, ms, cause, slackCursors? }` (last **20** kept). A clean restart records nothing.
190
+
191
+ Two `cause`s, one code path:
192
+
193
+ - **`cause: 'boot'`** — the process died (crash, power cut). The gap is measured at startup, from
194
+ the last persisted heartbeat.
195
+ - **`cause: 'suspend'`** — the process never died: the **host slept** (lid closed, standby) and the
196
+ process was *frozen*. Nothing restarts, so a boot check would never see it — instead the next
197
+ heartbeat tick comes back late by more than the threshold and records the gap itself. One entry
198
+ per outage, however long (the tick writes the heartbeat forward before the next one compares).
199
+ Two consequences of a `suspend` gap, both flagged in the report's `note`:
200
+
201
+ - **`missedSchedules[]` is INCOMPLETE, not empty** — the boot-time catch-up scan never ran (nothing
202
+ restarted), so the only records are the ones the **late timer fire** wrote itself (see "Missed
203
+ windows" above: a fire past the ceiling is refused and recorded as `missedRun`, at a timestamp
204
+ *inside* the gap, which the report then joins). Windows still inside the ceiling on wake simply
205
+ ran; anything else that elapsed in the gap left **no record at all** — check those by hand. The
206
+ note is emitted for any suspend in the last `SUSPEND_NOTE_MAX_AGE_MS` (**24h**), not just the
207
+ newest entry: a restart afterwards appends a `boot` entry but rescans nothing.
208
+ - **No `slackCursors` snapshot** — the snapshot is only taken on the `boot` path, where it is
209
+ provably older than any post-recovery traffic. A suspend gap is recorded from a heartbeat tick up
210
+ to 60s *after* the wake, by which time the reconnected socket may already have pushed the cursor
211
+ past the whole backlog. So the entry carries none and the backfill floors at the gap's `from` —
212
+ it may re-report a few already-seen messages, never skip unseen ones.
179
213
 
180
214
  ```bash
181
215
  curl -s -H "x-internal-token: $INTERNAL_API_TOKEN" "http://localhost:$PORT/api/downtime" | jq .
@@ -192,7 +226,8 @@ Returns `{ heartbeat, downtime[], lastDowntime, missedSchedules[], slack }`:
192
226
  Events API retries a dropped delivery for only ~30 minutes then discards it forever, while
193
227
  `conversations.history` stays readable for days.
194
228
  - **Where it starts:** the outage's `from`, raised only by the last-seen cursor **as
195
- snapshotted into the downtime entry at boot** (`slackCursors`). The live cursors in
229
+ snapshotted into the downtime entry at boot** (`slackCursors`, `cause: 'boot'` only a
230
+ `suspend` entry has none, so it starts at `from`). The live cursors in
196
231
  `data/state/slack-cursors.json` are a tail pointer — one message after recovery pushes them
197
232
  past the whole backlog — so they are never used as the floor.
198
233
  - **Which channels:** every channel with a cursor (at the time of the outage or since), plus
@@ -123,12 +123,14 @@ Schedules sync across all envs but only **execute** where `DATA_SYNC_SCHEDULER_A
123
123
  Two layers cause you to run — know both, and which to reach for:
124
124
 
125
125
  **Schedules (`schedules.json`)** — a schedule is `trigger` + `task` (`prompt`/`bash`/`job`). One execution path, two trigger families:
126
- - **Time**: `cron` / `interval` / `once`. A window missed while you were down obeys `onMissed`
126
+ - **Time**: `cron` / `interval` / `once`. A window missed while you were down or while the host
127
+ was **suspended** and you were merely frozen — obeys `onMissed`
127
128
  (`run` default / `skip` / `offer`) plus a hard 6h staleness ceiling that overrides *every*
128
129
  policy — so an 08:00 job never silently replays at 22:00. Anything not replayed is recorded as
129
130
  `missedRun` on the schedule (visible in `GET /api/schedules`) for an on-demand run. See the
130
131
  **scheduler** skill.
131
- - **Asked "what did I miss?"** (after a power cut / long downtime): `GET /api/downtime`. It
132
+ - **Asked "what did I miss?"** (after a power cut, or a laptop **suspend** a slept host is
133
+ caught too, by a late heartbeat tick, `cause: 'suspend'`): `GET /api/downtime`. It
132
134
  joins the recorded outage ranges to those `missedRun` windows and to the Slack messages that
133
135
  arrived while you were down, and it **reports and proposes only** — run a missed window with
134
136
  an explicit `POST /api/schedules/{id}/run`. Never replay everything you missed; a 14h-late
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "shraga",
3
- "version": "0.1.32",
3
+ "version": "0.1.34",
4
4
  "description": "The teammate you delegate coding to — a self-hostable, multi-user AI coding agent web UI (Claude Code, with a pluggable engine seam).",
5
5
  "type": "module",
6
6
  "main": "./src/index.ts",
@@ -2016,6 +2016,20 @@ try {
2016
2016
 
2017
2017
  const PORT = Number(process.env.PORT) || 3032;
2018
2018
  await new Promise<void>((resolve) => {
2019
+ // A listen failure MUST kill the process. Without this the error reaches the `uncaughtException`
2020
+ // handler above, which by design keeps us alive — so the promise never settles and we sit there
2021
+ // forever: process running, port unbound, nothing served. The service manager sees a healthy job
2022
+ // and a port watchdog sees a dead one, so it kickstarts on a loop and shreds in-flight runs.
2023
+ // EADDRINUSE is the common case: `kickstart -k` starts the replacement while the old process is
2024
+ // still draining (up to 90s). Exiting non-zero is the correct answer — the manager restarts us,
2025
+ // and by then the port is free.
2026
+ server.once('error', (err: NodeJS.ErrnoException) => {
2027
+ const why = err.code === 'EADDRINUSE'
2028
+ ? `port ${PORT} is already in use (previous instance still draining?)`
2029
+ : (err.message ?? String(err));
2030
+ console.error(`[server] FATAL: cannot listen on ${PORT} — ${why}. Exiting so the service manager restarts us.`);
2031
+ process.exit(1);
2032
+ });
2019
2033
  server.listen(PORT, () => {
2020
2034
  console.log(`[server] Running on http://0.0.0.0:${PORT}`);
2021
2035
  resolve();
@@ -7,7 +7,8 @@
7
7
  * 22:00; this module supplies the other half — the ledger and the on-demand report.
8
8
  *
9
9
  * Deliberately inert: nothing here starts a run, answers a Slack message, or mutates a schedule.
10
- * The only writers are the heartbeat, the boot gap record, and the Slack last-seen cursor.
10
+ * The only writers are the heartbeat, the gap ledger (boot gap + late-tick/suspend gap), and the
11
+ * Slack last-seen cursor.
11
12
  * Acting on a finding is always an explicit follow-up (`POST /api/schedules/:id/run`).
12
13
  */
13
14
  import { readFileSync, writeFileSync, existsSync, mkdirSync, renameSync } from 'node:fs';
@@ -42,6 +43,18 @@ export const DOWNTIME_THRESHOLD_MS = Number(process.env.DOWNTIME_THRESHOLD_MS ??
42
43
  */
43
44
  export const DOWNTIME_HISTORY_MAX = 20;
44
45
 
46
+ /**
47
+ * How long a `suspend` gap keeps colouring the report's `note`.
48
+ *
49
+ * A suspend is never retroactively scanned — no restart happens, so no catch-up ever covers it, and
50
+ * a later `boot` entry does not make it honest. The warning therefore cannot be tied to "is the
51
+ * newest entry a suspend?"; it has to age out on its own. 24h is the horizon of the manual check
52
+ * the note asks for ("which schedule windows fell in the gap?"): nearly every schedule here is
53
+ * daily or tighter, so after a day the same window has come round again and run normally, and the
54
+ * gap is no longer the thing to look at.
55
+ */
56
+ export const SUSPEND_NOTE_MAX_AGE_MS = Number(process.env.SUSPEND_NOTE_MAX_AGE_MS ?? 24 * 60 * 60 * 1000);
57
+
45
58
  export interface DowntimeEntry {
46
59
  /** Last proven-alive moment (the final heartbeat before the gap). */
47
60
  from: number;
@@ -49,10 +62,26 @@ export interface DowntimeEntry {
49
62
  to: number;
50
63
  ms: number;
51
64
  /**
52
- * channelId last Slack ts we had seen when this outage was recorded, snapshotted at boot
53
- * BEFORE any live traffic can move the cursors. The live cursor is a tail pointer: one normal
54
- * message after recovery pushes it past the entire backlog. This frozen copy is the only thing
55
- * that still knows where the outage's unseen history begins.
65
+ * How the gap was noticed. `boot` = the process died (crash, power cut, deploy) and the gap was
66
+ * measured at startup. `suspend` = the process never died the host slept (lid closed, standby)
67
+ * and a heartbeat tick came back late by more than the threshold. Both are real outages; the
68
+ * distinction matters to the reader, because a `suspend` gap means the process was FROZEN, so
69
+ * nothing at all ran at boot afterwards (no scheduler catch-up scan — see `missedSchedules`).
70
+ * Absent on entries written before this field existed.
71
+ */
72
+ cause?: 'boot' | 'suspend';
73
+ /**
74
+ * channelId → last Slack ts we had seen when this outage was recorded. Present on `boot` entries
75
+ * ONLY, where it is snapshotted before any live traffic can move the cursors (`recordBootGap()`
76
+ * runs before the Slack ingress mounts — see boot.ts). The live cursor is a tail pointer: one
77
+ * normal message after recovery pushes it past the entire backlog, so a snapshot taken after
78
+ * traffic resumed would hide the outage rather than bound it.
79
+ *
80
+ * Absent on `suspend` entries by design: that gap is recorded from a heartbeat tick, up to
81
+ * HEARTBEAT_INTERVAL_MS after the wake — exactly when the reconnected socket delivers the
82
+ * backlog — so the cursor is no longer provably pre-gap. With no snapshot the backfill floors at
83
+ * the outage start (`from`), which can re-report a few already-seen messages but can never skip
84
+ * unseen ones.
56
85
  */
57
86
  slackCursors?: Record<string, string>;
58
87
  }
@@ -90,11 +119,19 @@ export function writeHeartbeat(at = Date.now()): void {
90
119
  writeJsonAtomic(HEARTBEAT_FILE, { at, pid: process.pid });
91
120
  }
92
121
 
93
- /** Stamp liveness every interval. Unref'd — a heartbeat must never hold the process open. */
122
+ /**
123
+ * Stamp liveness every interval, and NOTICE when a tick comes back late.
124
+ *
125
+ * A boot-time check only catches an outage that killed the process. The common failure on this box
126
+ * is the opposite: a MacBook that sleeps with the lid closed. The process is frozen, not killed —
127
+ * it never restarts, so `recordBootGap()` never runs, and a 2.5h outage was completely invisible.
128
+ * A late tick is the one signal that survives a suspend, because the timer resumes on wake and the
129
+ * wall clock has moved on. Unref'd — a heartbeat must never hold the process open.
130
+ */
94
131
  export function startHeartbeat(intervalMs = HEARTBEAT_INTERVAL_MS): () => void {
95
132
  writeHeartbeat();
96
133
  const timer = setInterval(() => {
97
- try { writeHeartbeat(); } catch (err) { console.error('[downtime] heartbeat write failed:', err); }
134
+ try { recordTickGap(); } catch (err) { console.error('[downtime] heartbeat write failed:', err); }
98
135
  }, intervalMs);
99
136
  timer.unref?.();
100
137
  return () => clearInterval(timer);
@@ -106,6 +143,37 @@ export function listDowntime(): DowntimeEntry[] {
106
143
  return readJson<DowntimeFile>(DOWNTIME_FILE, { entries: [] }).entries ?? [];
107
144
  }
108
145
 
146
+ /**
147
+ * The one gap-recording path, shared by the boot check and the late-tick check.
148
+ *
149
+ * Both ask the same question — "the last proven-alive moment is `last`, it is now `now`, is that a
150
+ * hole?" — so they share the same threshold, the same bounds, and the same Slack-cursor snapshot.
151
+ * Writing the heartbeat forward is the caller's job and happens either way (this process is alive
152
+ * NOW regardless of the verdict), which is also what makes an outage record exactly ONCE: the next
153
+ * comparison starts from `now`, not from the stale pre-gap stamp.
154
+ *
155
+ * Never fires anything. It appends a row to a JSON file and logs — that is the whole contract.
156
+ */
157
+ function recordGap(last: number, now: number, cause: 'boot' | 'suspend'): DowntimeEntry | null {
158
+ const ms = now - last;
159
+ // `<=` also covers a backwards clock jump (negative ms): degrade to "no outage", never invent one.
160
+ if (ms <= DOWNTIME_THRESHOLD_MS) return null;
161
+
162
+ // Freeze the Slack cursors into the entry — but only on the boot path, where they are still
163
+ // provably pre-gap (recordBootGap runs before the Slack ingress mounts). On the suspend path the
164
+ // socket has been back for up to a heartbeat interval and may already have pushed the cursor past
165
+ // the whole backlog; recording that would be worse than recording nothing, so the entry carries
166
+ // no snapshot and the backfill floors at the gap start instead. See DowntimeEntry.slackCursors.
167
+ const snapshot: Record<string, string> = {};
168
+ if (cause === 'boot') for (const [channel, cursor] of Object.entries(listSlackCursors())) snapshot[channel] = cursor.ts;
169
+
170
+ const entry: DowntimeEntry = { from: last, to: now, ms, cause, ...(Object.keys(snapshot).length ? { slackCursors: snapshot } : {}) };
171
+ const entries = [...listDowntime(), entry].slice(-DOWNTIME_HISTORY_MAX);
172
+ writeJsonAtomic(DOWNTIME_FILE, { entries } satisfies DowntimeFile);
173
+ console.log(`[downtime] ${cause} gap of ${Math.round(ms / 60_000)}m recorded — down from ${new Date(last).toISOString()} to ${new Date(now).toISOString()}`);
174
+ return entry;
175
+ }
176
+
109
177
  /**
110
178
  * Compare the last heartbeat against boot time and, if the gap is real, record it. Returns the
111
179
  * entry it recorded, or null — a clean restart (and a first-ever boot, which has no heartbeat to
@@ -116,19 +184,26 @@ export function recordBootGap(bootTime = Date.now()): DowntimeEntry | null {
116
184
  // Write the new heartbeat regardless: whatever we conclude, this process is alive now.
117
185
  writeHeartbeat(bootTime);
118
186
  if (last === null) return null;
119
- const ms = bootTime - last;
120
- if (ms <= DOWNTIME_THRESHOLD_MS) return null;
121
-
122
- // Freeze the Slack cursors into the entry now, at boot, before the first inbound message can
123
- // advance them past the backlog this outage left behind.
124
- const snapshot: Record<string, string> = {};
125
- for (const [channel, cursor] of Object.entries(listSlackCursors())) snapshot[channel] = cursor.ts;
187
+ return recordGap(last, bootTime, 'boot');
188
+ }
126
189
 
127
- const entry: DowntimeEntry = { from: last, to: bootTime, ms, ...(Object.keys(snapshot).length ? { slackCursors: snapshot } : {}) };
128
- const entries = [...listDowntime(), entry].slice(-DOWNTIME_HISTORY_MAX);
129
- writeJsonAtomic(DOWNTIME_FILE, { entries } satisfies DowntimeFile);
130
- console.log(`[downtime] gap of ${Math.round(ms / 60_000)}m recordeddown from ${new Date(last).toISOString()} to ${new Date(bootTime).toISOString()}`);
131
- return entry;
190
+ /**
191
+ * One heartbeat tick: stamp liveness, and record an outage if the tick is late past the threshold.
192
+ *
193
+ * The reference point is the PERSISTED heartbeat the same value `recordBootGap()` measures from,
194
+ * so boot and suspend can never disagree about where the last proven-alive moment was, and can
195
+ * never double-record one outage: whichever check sees the gap first advances the stamp, and the
196
+ * other then measures from the new one.
197
+ *
198
+ * A long suspend yields ONE entry, not one per missed interval, because a frozen process fires no
199
+ * timers while it sleeps — `setInterval` does not accumulate a backlog of missed ticks — and even
200
+ * if it did, this writes the heartbeat forward before the next tick can compare.
201
+ */
202
+ export function recordTickGap(now = Date.now()): DowntimeEntry | null {
203
+ const last = readHeartbeat();
204
+ writeHeartbeat(now);
205
+ if (last === null) return null;
206
+ return recordGap(last, now, 'suspend');
132
207
  }
133
208
 
134
209
  // ── Slack last-seen cursors ───────────────────────────────────────────────────────────────────
@@ -354,6 +429,16 @@ export async function buildReport(opts: { slack?: boolean; backfill?: BackfillOp
354
429
  missedSchedules: missedSchedules(entries),
355
430
  note: 'Report only — nothing here has been run, answered, or replayed. Act on an item explicitly.',
356
431
  };
432
+ // A suspend gap froze the process instead of killing it, so the scheduler's boot-time catch-up
433
+ // scan never ran. The late timer fire on wake DOES record the windows it refuses, so the list is
434
+ // not empty — it is INCOMPLETE: it holds only what that fire judged. Say that, and say it for
435
+ // every recent suspend, not just the newest entry: a restart afterwards appends a `boot` entry
436
+ // but scans nothing retroactively, so gating on `last` would silently drop the warning.
437
+ const suspends = entries.filter(e => e.cause === 'suspend' && now - e.to <= SUSPEND_NOTE_MAX_AGE_MS);
438
+ if (suspends.length) {
439
+ const when = suspends.map(e => `${new Date(e.from).toISOString()}→${new Date(e.to).toISOString()}`).join(', ');
440
+ report.note += ` NOTE: a recent outage was a host SUSPEND (the process was frozen, not restarted: ${when}), so no scheduler catch-up scan ran. missedSchedules is INCOMPLETE for that gap — it lists only the windows the late timer fire itself judged on wake; any other window that elapsed inside the gap left no record at all. Check schedules whose window falls in the gap by hand.`;
441
+ }
357
442
  if (opts.slack !== false) {
358
443
  report.slack = last
359
444
  // The snapshot frozen at boot, not the live cursors — see DowntimeEntry.slackCursors.
@@ -111,22 +111,10 @@ export function start(broadcast: Broadcast): void {
111
111
  continue;
112
112
  }
113
113
 
114
- const policy = missedPolicy(s);
115
- if (policy === 'skip') {
116
- console.log(`[scheduler] missed window ${new Date(prev).toISOString()} for ${s.id} — onMissed=skip, not replaying`);
117
- noteMissed(s, prev, 'skip');
118
- continue;
119
- }
120
- if (policy === 'offer') {
121
- console.log(`[scheduler] missed window ${new Date(prev).toISOString()} for ${s.id} — onMissed=offer, recorded for on-demand run`);
122
- noteMissed(s, prev, 'offer');
123
- continue;
124
- }
125
- const age = Date.now() - prev;
126
- const grace = maxMissedAgeMs();
127
- if (age > grace) {
128
- console.log(`[scheduler] missed window ${new Date(prev).toISOString()} for ${s.id} is ${Math.round(age / 60_000)}m stale (grace ${Math.round(grace / 60_000)}m) — not replaying`);
129
- noteMissed(s, prev, 'stale');
114
+ const verdict = judgeMissed(s, prev);
115
+ if (!verdict.replay) {
116
+ console.log(`[scheduler] not replaying ${s.id} ${verdict.message}`);
117
+ noteMissed(s, prev, verdict.reason);
130
118
  continue;
131
119
  }
132
120
  catchUps.push({ id: s.id, window: prev });
@@ -347,20 +335,11 @@ export function resumeRun(scheduleId: string, sessionId: string, prompt: string)
347
335
  console.log(`[scheduler] not resuming ${scheduleId} — ${msg}`);
348
336
  return refuse('already-completed', msg);
349
337
  }
350
- const policy = missedPolicy(s);
351
- if (policy !== 'run') {
352
- const msg = `window ${new Date(window).toISOString()} was missed and onMissed=${policy}`;
353
- console.log(`[scheduler] not resuming ${scheduleId} — ${msg}`);
354
- noteMissed(s, window, policy);
355
- return refuse(policy === 'skip' ? 'policy-skip' : 'policy-offer', msg);
356
- }
357
- const age = Date.now() - window;
358
- const grace = maxMissedAgeMs();
359
- if (age > grace) {
360
- const msg = `window ${new Date(window).toISOString()} is ${Math.round(age / 60_000)}m stale (ceiling ${Math.round(grace / 60_000)}m)`;
361
- console.log(`[scheduler] not resuming ${scheduleId} — ${msg}`);
362
- noteMissed(s, window, 'stale');
363
- return refuse('stale', msg);
338
+ const verdict = judgeMissed(s, window);
339
+ if (!verdict.replay) {
340
+ console.log(`[scheduler] not resuming ${scheduleId} ${verdict.message}`);
341
+ noteMissed(s, window, verdict.reason);
342
+ return refuse(verdict.refusal, verdict.message);
364
343
  }
365
344
  }
366
345
  console.log(`[scheduler] resuming ${scheduleId} in-place on session ${sessionId.slice(0, 30)}…`);
@@ -423,6 +402,32 @@ function missedPolicy(s: Schedule): MissedPolicy {
423
402
  return s.onMissed ?? 'run';
424
403
  }
425
404
 
405
+ type MissedVerdict =
406
+ | { replay: true }
407
+ | { replay: false; reason: 'skip' | 'offer' | 'stale'; refusal: 'policy-skip' | 'policy-offer' | 'stale'; message: string };
408
+
409
+ /**
410
+ * The one decision behind "this window already elapsed — may it still run?", shared by all three
411
+ * replay paths: the boot catch-up scan, the in-place resume, and the late timer fire.
412
+ *
413
+ * `onMissed` first (an explicit skip/offer is a standing instruction, not an age question), then
414
+ * the absolute staleness ceiling, which overrides even `run`. Callers do the recording, because
415
+ * only they know how to refuse (a return value, or `continue`).
416
+ */
417
+ function judgeMissed(s: Schedule, window: number, now: number = Date.now()): MissedVerdict {
418
+ const when = new Date(window).toISOString();
419
+ const policy = missedPolicy(s);
420
+ if (policy !== 'run') {
421
+ return { replay: false, reason: policy, refusal: policy === 'skip' ? 'policy-skip' : 'policy-offer', message: `window ${when} was missed and onMissed=${policy}` };
422
+ }
423
+ const age = now - window;
424
+ const grace = maxMissedAgeMs();
425
+ if (age > grace) {
426
+ return { replay: false, reason: 'stale', refusal: 'stale', message: `window ${when} is ${Math.round(age / 60_000)}m stale (ceiling ${Math.round(grace / 60_000)}m)` };
427
+ }
428
+ return { replay: true };
429
+ }
430
+
426
431
  function noteMissed(s: Schedule, at: number, reason: 'skip' | 'offer' | 'stale'): void {
427
432
  s.missedRun = { at, reason, noticedAt: Date.now() };
428
433
  saveSchedules(state.schedules);
@@ -455,7 +460,24 @@ function fireDue(): void {
455
460
  const now = Date.now();
456
461
  for (const s of state.schedules) {
457
462
  if (!s.enabled || s.nextRun === undefined) continue;
458
- if (s.nextRun <= now) enqueueFire(s, s.nextRun);
463
+ if (s.nextRun > now) continue;
464
+ // A timer fire is normally punctual, so it is NOT treated as a missed window — `onMissed`
465
+ // must never gate healthy operation. But a setTimeout does not survive a wall-clock jump:
466
+ // when the host sleeps (lid closed) or NTP steps the clock, the process is FROZEN, not
467
+ // killed — no restart, so the boot catch-up scan never runs — and the overdue timer fires
468
+ // the elapsed window the moment the machine wakes. Firing an 08:00 job at 21:36 is exactly
469
+ // what the catch-up ceiling exists to prevent, reached by the one path it didn't cover.
470
+ // Past the ceiling we hand the window to the same judgement the other two paths use.
471
+ // Past the ceiling `judgeMissed` can only refuse (it replays a `run` window only while it is
472
+ // within the ceiling), so there is no replay branch here — the shared decision is used for WHAT
473
+ // to record, not whether to fire.
474
+ if (now - s.nextRun > maxMissedAgeMs()) {
475
+ const verdict = judgeMissed(s, s.nextRun, now) as Extract<MissedVerdict, { replay: false }>;
476
+ console.log(`[scheduler] not firing ${s.id} — ${verdict.message} (timer fired late; host suspended or clock jumped)`);
477
+ noteMissed(s, s.nextRun, verdict.reason);
478
+ continue; // the advance loop below still re-arms / retires this schedule
479
+ }
480
+ enqueueFire(s, s.nextRun);
459
481
  }
460
482
  // Advance nextRun for recurring triggers; disable fired `once` triggers
461
483
  for (const s of state.schedules) {