@feigi/fleet-ctl 3.87.1 → 3.88.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
package/scripts/fleet-event.mjs
CHANGED
|
@@ -126,8 +126,8 @@
|
|
|
126
126
|
// heartbeat
|
|
127
127
|
// No write: the tick runs with `--fold-unchanged`, which prints one line
|
|
128
128
|
// instead of the rows when the tick asks for nothing and says what the
|
|
129
|
-
// last one said. The
|
|
130
|
-
//
|
|
129
|
+
// last one said. The alarm turn that runs it is started by
|
|
130
|
+
// `controller-alarm.mjs`, once the beat `fleet-heartbeat.mjs` marked falls due.
|
|
131
131
|
// drain --reason "<reason>"
|
|
132
132
|
// `closeout.mjs begin --reason "<reason>"`, its output printed as it
|
|
133
133
|
// stands, then the tick.
|
|
@@ -6,37 +6,39 @@
|
|
|
6
6
|
// every other invocation is a wake: a member report, a label, a CI run ending.
|
|
7
7
|
// A drained fleet emits no wakes at all — no implementer completions, no merge
|
|
8
8
|
// activity — so nothing calls the tick and the stall is silent again. This
|
|
9
|
-
// script is the level-check that needs no
|
|
10
|
-
//
|
|
11
|
-
//
|
|
9
|
+
// script is the level-check that needs no event: it marks when the controller
|
|
10
|
+
// went idle and how long the next level-check may wait, and the controller then
|
|
11
|
+
// ends its turn. The alarm (controller-alarm.mjs, an omp extension loaded into
|
|
12
|
+
// the controller's own session) starts the next turn once the mark has fallen
|
|
13
|
+
// due, and that turn runs the tick.
|
|
12
14
|
//
|
|
13
|
-
// WHY
|
|
15
|
+
// WHY AN ALARM AND NOT A WATCHER — the ruling, measured before it was made:
|
|
14
16
|
//
|
|
15
17
|
// A backgrounded watcher does not wake an idle agent. Measured twice in one
|
|
16
18
|
// run (run-team/references/member-lifecycle.md, "A member runs its test suite in the foreground"):
|
|
17
19
|
// `fix-pr-1184` backgrounded its suite plus a Monitor and sat idle for
|
|
18
20
|
// roughly two hours; neither ever woke it. run-merge-bot.md's CI gate states
|
|
19
21
|
// the same rule as a rule — "whatever wakes you is external and may never
|
|
20
|
-
// come"
|
|
21
|
-
// inside one blocking call.
|
|
22
|
+
// come".
|
|
22
23
|
//
|
|
23
|
-
// So the
|
|
24
|
+
// So the alarm is not a Monitor, not a cron entry and not a paragraph of
|
|
24
25
|
// prose. A cron entry that ran the reconcile would print its rows to nobody:
|
|
25
26
|
// the tick reads the run for itself, but only the controller can act
|
|
26
|
-
// on what it prints.
|
|
27
|
-
// which is the one thing the measurement above says
|
|
28
|
-
// do
|
|
27
|
+
// on what it prints. Nothing outside the omp process can start a turn in an
|
|
28
|
+
// idle session, which is the one thing the measurement above says an
|
|
29
|
+
// external watcher cannot be trusted to do — so the alarm is an in-process
|
|
30
|
+
// extension, which can.
|
|
29
31
|
//
|
|
30
|
-
// WHY IT TAKES NO CONTROLLER STATE — every flag below has a default.
|
|
31
|
-
//
|
|
32
|
-
// script never reads or reports fleet state: counts captured
|
|
33
|
-
//
|
|
32
|
+
// WHY IT TAKES NO CONTROLLER STATE — every flag below has a default. An alarm
|
|
33
|
+
// turn is a wake-up, never a verdict (run-team/SKILL.md, "A monitor event is a wake-up, never a verdict"), so this
|
|
34
|
+
// script never reads or reports fleet state: counts captured when the beat was
|
|
35
|
+
// marked and printed twenty minutes later are a stale verdict, and a stale
|
|
34
36
|
// verdict is how a reconcile prints an ACTION nobody can take. The controller
|
|
35
37
|
// runs `fleet-event.mjs heartbeat` itself at wake time, which ticks and so reads the run as it stands then.
|
|
36
|
-
// It also makes a late
|
|
37
|
-
// the controller acts, the
|
|
38
|
+
// It also makes a late alarm harmless: if a member reports at minute two and
|
|
39
|
+
// the controller acts, the alarm that fires afterwards is a free extra tick.
|
|
38
40
|
//
|
|
39
|
-
// The pure half is interval()
|
|
41
|
+
// The pure half is interval(); main() does the I/O.
|
|
40
42
|
|
|
41
43
|
// Back-off. The interval doubles while the fleet has nothing to act on, so a
|
|
42
44
|
// quiet night costs a few wakes instead of one every five minutes — and stops
|
|
@@ -58,20 +60,6 @@ export function interval({ quiet, base, ceiling, multiplier }) {
|
|
|
58
60
|
return Math.min(base * multiplier ** quiet, ceiling);
|
|
59
61
|
}
|
|
60
62
|
|
|
61
|
-
// How long THIS invocation may block, which is not the same as the interval.
|
|
62
|
-
//
|
|
63
|
-
// Nothing lets one command block for twenty minutes. Measured: omp
|
|
64
|
-
// auto-backgrounds any foreground command at 60s (`bash.autoBackground`), and
|
|
65
|
-
// its command deadline defaults to 300s (the bash tool's default
|
|
66
|
-
// `timeout`).
|
|
67
|
-
//
|
|
68
|
-
// So a long interval is served by SEVERAL holds, and the elapsed total is
|
|
69
|
-
// persisted rather than counted in prose: the script says how much remains, so
|
|
70
|
-
// the controller re-issues without tracking arithmetic across turns.
|
|
71
|
-
export function heldThisCall({ elapsed, target, hold }) {
|
|
72
|
-
return Math.min(hold, Math.max(0, target - elapsed));
|
|
73
|
-
}
|
|
74
|
-
|
|
75
63
|
// --------------------------------------------------------------------------
|
|
76
64
|
// I/O. Everything below runs only as a CLI — importing this file must never
|
|
77
65
|
// parse argv or touch the filesystem, or the pure half stops being unit-testable.
|
|
@@ -79,7 +67,7 @@ export function heldThisCall({ elapsed, target, hold }) {
|
|
|
79
67
|
import { isCLI } from "./is-cli.mjs";
|
|
80
68
|
import { parseArgs } from "node:util";
|
|
81
69
|
import { makeDie, isDigits } from "./arg.mjs";
|
|
82
|
-
import {
|
|
70
|
+
import { resolveStatePath, readState, writeState, DEFAULT_CEILING_S } from "./fleet-state.mjs";
|
|
83
71
|
|
|
84
72
|
const NAME = "fleet-heartbeat";
|
|
85
73
|
const die = makeDie(NAME);
|
|
@@ -95,11 +83,6 @@ const OPTIONS = {
|
|
|
95
83
|
// header's "never a copy of it" rule exists to close.
|
|
96
84
|
ceiling: { type: "string", default: String(DEFAULT_CEILING_S) },
|
|
97
85
|
multiplier: { type: "string", default: "2" },
|
|
98
|
-
// Per-invocation blocking budget, under omp's defaults. 240s leaves
|
|
99
|
-
// headroom below omp's 300s command deadline, which is itself shorter than
|
|
100
|
-
// the 5-6 minute CI cycle run-merge-bot.md's CI wait rule states. Raise
|
|
101
|
-
// it only together with the tool call's own timeout.
|
|
102
|
-
hold: { type: "string", default: "240" },
|
|
103
86
|
// The deliberate stop. No default and no boolean spelling: the flag
|
|
104
87
|
// IS the reason, and a `--stop` that recorded an empty string would be the
|
|
105
88
|
// abrupt-death case wearing a deliberate stop's clothes — a reader would
|
|
@@ -114,19 +97,8 @@ const OPTIONS = {
|
|
|
114
97
|
};
|
|
115
98
|
|
|
116
99
|
// State path, read and write all come from fleet-state.mjs, which owns the key
|
|
117
|
-
// ownership rule this script depends on: it writes `
|
|
118
|
-
// nothing else, so fleet-tick's `quiet` and `digest` survive every
|
|
119
|
-
// main() re-reads the file after its hold to keep that true across the hold
|
|
120
|
-
// itself, and treats a write that did not land as a fire rather than as
|
|
121
|
-
// progress.
|
|
122
|
-
|
|
123
|
-
// A blocking sleep, not a poll loop. Atomics.wait on a SharedArrayBuffer parks
|
|
124
|
-
// the thread; `while (Date.now() < end) {}` would burn a core for the whole
|
|
125
|
-
// hold, and setTimeout would need the event loop to stay alive and would let
|
|
126
|
-
// anything else queued run first.
|
|
127
|
-
function block(seconds) {
|
|
128
|
-
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, seconds * 1000);
|
|
129
|
-
}
|
|
100
|
+
// ownership rule this script depends on: it writes `beat` and
|
|
101
|
+
// nothing else, so fleet-tick's `quiet` and `digest` survive every call.
|
|
130
102
|
|
|
131
103
|
function args() {
|
|
132
104
|
let values;
|
|
@@ -142,7 +114,7 @@ function args() {
|
|
|
142
114
|
const raw = String(values[name]).trim();
|
|
143
115
|
// Digits, not Number(): `Number("")` is 0, so `--base ""` — the shape an
|
|
144
116
|
// unset shell variable produces — would read as a real zero and turn the
|
|
145
|
-
//
|
|
117
|
+
// interval into a no-op that looks like a working heartbeat.
|
|
146
118
|
if (!isDigits(raw)) die(`--${name} must be a non-negative integer, got '${values[name]}'`);
|
|
147
119
|
const n = Number(raw);
|
|
148
120
|
if (n < min) die(`--${name} must be at least ${min}, got ${n}`);
|
|
@@ -151,7 +123,6 @@ function args() {
|
|
|
151
123
|
const base = int("base", 1);
|
|
152
124
|
const ceiling = int("ceiling", 1);
|
|
153
125
|
const multiplier = int("multiplier", 1);
|
|
154
|
-
const hold = int("hold", 1);
|
|
155
126
|
// A ceiling under the base is not a slow heartbeat, it is a back-off that
|
|
156
127
|
// runs backwards — min() would clamp every interval to the ceiling and the
|
|
157
128
|
// base would never be honoured. Refuse rather than silently reinterpret.
|
|
@@ -167,7 +138,8 @@ function args() {
|
|
|
167
138
|
if (values.stop !== undefined && values.stop.trim() === "") {
|
|
168
139
|
die("--stop must carry the reason the run is stopping, e.g. --stop 'budget exhausted'");
|
|
169
140
|
}
|
|
170
|
-
|
|
141
|
+
const { path: state, fellBack } = values.state ? { path: values.state, fellBack: false } : resolveStatePath(NAME);
|
|
142
|
+
return { base, ceiling, multiplier, stop: values.stop?.trim(), state, fellBack };
|
|
171
143
|
}
|
|
172
144
|
|
|
173
145
|
// The mark: when this beat was seen and the interval that was in effect
|
|
@@ -176,26 +148,21 @@ function args() {
|
|
|
176
148
|
//
|
|
177
149
|
// The interval travels WITH the time because a reader comparing an age
|
|
178
150
|
// against a fixed threshold either cries wolf on a quiet night at the ceiling
|
|
179
|
-
// or misses a death during a busy one at the base. It is the
|
|
180
|
-
//
|
|
181
|
-
// implementation detail of how one interval gets served, while the interval
|
|
182
|
-
// is the beat the run actually promised.
|
|
151
|
+
// or misses a death during a busy one at the base. It is the beat the run
|
|
152
|
+
// actually promised: the alarm fires once `at` plus the interval has passed.
|
|
183
153
|
function mark(target, stopped) {
|
|
184
154
|
return { at: Date.now(), interval: target, stopped };
|
|
185
155
|
}
|
|
186
156
|
|
|
187
157
|
function main() {
|
|
188
|
-
const { base, ceiling, multiplier,
|
|
158
|
+
const { base, ceiling, multiplier, stop, state: path, fellBack } = args();
|
|
189
159
|
const state = readState(path, NAME);
|
|
190
160
|
const target = interval({ quiet: state.quiet, base, ceiling, multiplier });
|
|
191
161
|
|
|
192
162
|
if (stop !== undefined) {
|
|
193
|
-
//
|
|
194
|
-
//
|
|
195
|
-
//
|
|
196
|
-
// and knows the streak behind it is a dead run's. Leave degraded files
|
|
197
|
-
// byte-identical, so a same-session rotate still refuses rather than
|
|
198
|
-
// judges none.
|
|
163
|
+
// This invocation is not a beat, it is the run saying why there will not
|
|
164
|
+
// be another one. Leave degraded files byte-identical, so a same-session
|
|
165
|
+
// rotate still refuses rather than judges none.
|
|
199
166
|
if (state.degraded) {
|
|
200
167
|
console.log(`heartbeat: stop NOT recorded (${stop}) — the heartbeat file is degraded and was left as it is, so a same-session rotate refuses on it until it is repaired`);
|
|
201
168
|
return;
|
|
@@ -213,72 +180,44 @@ function main() {
|
|
|
213
180
|
return;
|
|
214
181
|
}
|
|
215
182
|
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
// Re-read AFTER the hold, and patch THAT rather than the pre-hold snapshot.
|
|
220
|
-
// fleet-tick can run while this call is blocked — up to --hold seconds, 240
|
|
221
|
-
// by default — and it owns both keys this script must not touch. Writing the
|
|
222
|
-
// snapshot back reverts them: a busy stretch that reset `quiet` to 0 would find
|
|
223
|
-
// the long interval re-armed the moment the hold ended, and a reverted
|
|
224
|
-
// `digest` reads as "the output changed" on the next tick, un-folding the
|
|
225
|
-
// quiet night the digest exists to fold. `elapsed` is still one of the keys
|
|
226
|
-
// this script writes; it is now read fresh rather than remembered across a
|
|
227
|
-
// hold long enough for the file to have moved underneath it.
|
|
228
|
-
const now = readState(path, NAME);
|
|
229
|
-
const elapsed = now.elapsed + held;
|
|
230
|
-
const reached = elapsed >= target;
|
|
231
|
-
// Reset on fire, so the next interval starts from zero rather than from a
|
|
232
|
-
// total that has already elapsed. Only `elapsed` and `beat` are written —
|
|
233
|
-
// fleet-tick owns `quiet` and `digest`, and fleet-state.mjs's patch write
|
|
234
|
-
// preserves them.
|
|
235
|
-
//
|
|
236
|
-
// The mark rides the SAME write as `elapsed`, deliberately, rather than
|
|
237
|
-
// taking one of its own. One write per invocation means a mark can never
|
|
238
|
-
// land while the progress it was taken beside did not, and the two can
|
|
239
|
-
// never disagree about which invocation they came from. It also keeps the
|
|
240
|
-
// failure story single: the warning below covers both keys at once. Leave
|
|
241
|
-
// degraded files byte-identical, so a same-session rotate still refuses
|
|
242
|
-
// rather than judges none — a skipped write reports a fire like a failed one.
|
|
183
|
+
// Leave degraded files byte-identical, so a same-session rotate still
|
|
184
|
+
// refuses rather than judges none — a skipped write is a mark that was not
|
|
185
|
+
// recorded, like a failed one.
|
|
243
186
|
//
|
|
244
187
|
// `stopped: ""` on every ordinary beat, never carried forward: a run that
|
|
245
188
|
// recorded a stop and then kept beating is a run that did not stop, and a
|
|
246
189
|
// stale reason surviving into a live beat would have the next reader
|
|
247
190
|
// announcing a death that already un-happened.
|
|
248
|
-
const persisted = !
|
|
249
|
-
// A failed write is itself a fire. With nothing persisting the remainder,
|
|
250
|
-
// every invocation reads the same elapsed total, holds the same seconds and
|
|
251
|
-
// prints the same remainder: the interval can never complete, so fleet-tick
|
|
252
|
-
// is never run at all — the drained-queue stall itself, reached through a line that reads
|
|
253
|
-
// like a working heartbeat. Beating too often is the harmless direction;
|
|
254
|
-
// never beating is the one this script exists to prevent.
|
|
255
|
-
//
|
|
256
|
-
// The mark fails the same way and on purpose: writeState has already said
|
|
257
|
-
// so on stderr, this process keeps beating, and a reader that never sees a
|
|
258
|
-
// mark reports a stall it cannot explain — loud and wrong in the safe
|
|
259
|
-
// direction — rather than this script dying to protect its own telemetry.
|
|
260
|
-
// A liveness mark that killed the beat it measures would be that same stall
|
|
261
|
-
// wearing a new hat.
|
|
262
|
-
const done = reached || !persisted;
|
|
191
|
+
const persisted = !state.degraded && writeState(path, NAME, state, { beat: mark(target, "") });
|
|
263
192
|
|
|
264
193
|
// One line either way, and never zero lines. Silence is the failure mode this
|
|
265
|
-
// whole
|
|
194
|
+
// whole script is about: a heartbeat that printed nothing would be
|
|
266
195
|
// indistinguishable from a heartbeat that died, which is the same
|
|
267
196
|
// "indistinguishable from a working one" the member-idle measurement names.
|
|
268
|
-
// The partial line says what to do next in the imperative, because the one
|
|
269
|
-
// thing that must not happen here is the controller ending its turn.
|
|
270
197
|
//
|
|
271
|
-
//
|
|
272
|
-
//
|
|
273
|
-
//
|
|
198
|
+
// The line that tells the controller to end its turn is printed ONLY when
|
|
199
|
+
// the mark is on disk where the alarm reads. The alarm's gate reads `beat`;
|
|
200
|
+
// with none recorded it never passes, so that line would send the controller
|
|
201
|
+
// idle into a session nothing will wake — the drained-queue stall, reached
|
|
202
|
+
// through a line that reads like a working heartbeat. A mark that failed to
|
|
203
|
+
// land says so, says no alarm is armed for it (an earlier mark still on disk
|
|
204
|
+
// is all the alarm can fire on), and exits 0 like the stop path: writeState
|
|
205
|
+
// has already said so on stderr, and the controller can repair the file and
|
|
206
|
+
// beat again. A mark written through statePath's cwd-relative fallback is
|
|
207
|
+
// treated the same way: the alarm resolves the run's `.fleet` for itself and
|
|
208
|
+
// may never read that file.
|
|
274
209
|
//
|
|
275
|
-
//
|
|
276
|
-
//
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
210
|
+
// `quiet=` is the streak the interval was DERIVED from, printed beside the
|
|
211
|
+
// number it explains.
|
|
212
|
+
if (persisted && fellBack) {
|
|
213
|
+
console.log(`heartbeat: WARNING beat NOT marked in a resolved heartbeat file — the run's .fleet directory could not be resolved, so the mark went to a cwd-relative .fleet/heartbeat.json the alarm may not read; fix the resolution and run this again`);
|
|
214
|
+
} else if (persisted) {
|
|
215
|
+
console.log(`heartbeat: beat marked, alarm due in ${target}s (quiet=${state.quiet}) → end your turn`);
|
|
216
|
+
} else if (state.degraded) {
|
|
217
|
+
console.log(`heartbeat: WARNING beat NOT marked — the heartbeat file is degraded and was left as it is, so no alarm is armed; repair the file and run this again`);
|
|
218
|
+
} else {
|
|
219
|
+
console.log(`heartbeat: WARNING beat NOT marked — the heartbeat file could not be written, so no alarm is armed for this beat; fix the write and run this again`);
|
|
220
|
+
}
|
|
282
221
|
}
|
|
283
222
|
|
|
284
223
|
if (isCLI(import.meta.url)) main();
|
package/scripts/fleet-state.mjs
CHANGED
|
@@ -11,9 +11,6 @@
|
|
|
11
11
|
// nothing. Drives the back-off interval.
|
|
12
12
|
// digest fleet-tick only. The last tick's own output, so an unchanged
|
|
13
13
|
// quiet tick can fold to one line instead of three.
|
|
14
|
-
// elapsed fleet-heartbeat only. Seconds already held toward the current
|
|
15
|
-
// interval, because no harness lets one command block long enough
|
|
16
|
-
// to serve a twenty-minute interval in a single call.
|
|
17
14
|
// beat fleet-heartbeat only. The liveness mark: when the beat
|
|
18
15
|
// was last seen, the interval that was in effect when it was, and
|
|
19
16
|
// a deliberate stop's reason if one was recorded. Written by the
|
|
@@ -87,20 +84,28 @@ export function stateFileIn(fleetDir) {
|
|
|
87
84
|
// There is ONE heartbeat per run and its state lives in the main checkout.
|
|
88
85
|
// Members run from their own worktrees, where a cwd-relative `.fleet/` does not
|
|
89
86
|
// exist, so a cwd-relative path would give every worktree a private back-off
|
|
90
|
-
// streak
|
|
87
|
+
// streak — the run's beat would be whichever
|
|
91
88
|
// worktree happened to call last. fleet-dir.mjs's fleetFile() resolves it the
|
|
92
89
|
// way it resolves the ledger, scrubbed and bounded: a `git` that never answers
|
|
93
90
|
// is killed rather than holding the beat. Only the warning and the fallback
|
|
94
91
|
// below are this caller's own.
|
|
95
92
|
export function statePath(name) {
|
|
93
|
+
return resolveStatePath(name).path;
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
// The same resolution, saying whether it fell back. fleet-heartbeat needs the
|
|
97
|
+
// flag: a beat marked in the cwd-relative file is one the alarm, which resolves
|
|
98
|
+
// the run's `.fleet` for itself, may never read, so that beat must not tell the
|
|
99
|
+
// controller to end its turn.
|
|
100
|
+
export function resolveStatePath(name) {
|
|
96
101
|
try {
|
|
97
|
-
return stateFileIn(fleetFile(null));
|
|
102
|
+
return { path: stateFileIn(fleetFile(null)), fellBack: false };
|
|
98
103
|
} catch (e) {
|
|
99
104
|
if (!(e instanceof FleetDirUnresolvable)) throw e;
|
|
100
105
|
// Announced, never silent: a cwd-relative fallback re-opens exactly the
|
|
101
106
|
// per-worktree split this resolution exists to close.
|
|
102
107
|
console.error(`${name}: WARNING ${e.message}; using cwd-relative .fleet/heartbeat.json`);
|
|
103
|
-
return stateFileIn(".fleet");
|
|
108
|
+
return { path: stateFileIn(".fleet"), fellBack: true };
|
|
104
109
|
}
|
|
105
110
|
}
|
|
106
111
|
|
|
@@ -134,7 +139,7 @@ export function readState(path, name, warn = console.error) {
|
|
|
134
139
|
// to judge and nothing to report. A zeroed mark would date the beat to the
|
|
135
140
|
// epoch and read as decades overdue on a run that has simply not started
|
|
136
141
|
// one — the cry-wolf direction this key must never fail in.
|
|
137
|
-
const fresh = { quiet: 0,
|
|
142
|
+
const fresh = { quiet: 0, digest: "", beat: null, ticked: null, acted: null, controller: null, seen: {}, applied: null, rest: {} };
|
|
138
143
|
// The same state, flagged: the file EXISTS and could not be used. A caller
|
|
139
144
|
// that acts on the absence of a record — ledger.mjs rotate — must tell this
|
|
140
145
|
// from `fresh`, which is a run that has simply not written one yet. The
|
|
@@ -149,7 +154,7 @@ export function readState(path, name, warn = console.error) {
|
|
|
149
154
|
// fault: a fresh run legitimately has no file yet. Every other errno is a
|
|
150
155
|
// file that EXISTS and could not be read — EACCES on a chmod'ed state
|
|
151
156
|
// file, EISDIR on a path something else claimed, EIO on a bad disk — and
|
|
152
|
-
// each of those discards a real back-off streak
|
|
157
|
+
// each of those discards a real back-off streak.
|
|
153
158
|
// Swallowing them with the same bare catch the absent case uses is a
|
|
154
159
|
// degraded read reported as a fresh run, which is the failure class this
|
|
155
160
|
// whole ticket exists to close.
|
|
@@ -171,13 +176,13 @@ export function readState(path, name, warn = console.error) {
|
|
|
171
176
|
return makeDegraded("is not a JSON object");
|
|
172
177
|
}
|
|
173
178
|
// Per-field validation, not all-or-nothing: a file carrying a good `quiet`
|
|
174
|
-
// and a junk `
|
|
179
|
+
// and a junk `digest` keeps the streak instead of losing both to one key.
|
|
175
180
|
// The exception is `controller`, below: a record that cannot be judged
|
|
176
181
|
// flags the whole file, because a write that dropped it is the loss.
|
|
177
182
|
const num = (v) => (typeof v === "number" && Number.isInteger(v) && v >= 0 ? v : 0);
|
|
178
183
|
// `rest` is strictly what lies OUTSIDE the schema, which is what makes the
|
|
179
184
|
// patch write below a patch rather than a laundering step. A raw copy of
|
|
180
|
-
// quiet/
|
|
185
|
+
// quiet/digest in here would be written straight back by whichever
|
|
181
186
|
// script does not own the key, so a junk value would survive every write
|
|
182
187
|
// that sanitized it in memory — validation that only ever holds inside one
|
|
183
188
|
// process is not validation of the file.
|
|
@@ -253,7 +258,7 @@ export function readState(path, name, warn = console.error) {
|
|
|
253
258
|
if (!(Number.isInteger(v.run) && v.run > 0)) return null;
|
|
254
259
|
return { run: v.run, reapAfter: v.reapAfter.filter((n) => Number.isInteger(n) && n > 0) };
|
|
255
260
|
};
|
|
256
|
-
const { quiet,
|
|
261
|
+
const { quiet, digest, beat, ticked, acted, controller, seen, applied, ...rest } = parsed;
|
|
257
262
|
// A controller key that ctl() rejects is a record someone wrote and nobody
|
|
258
263
|
// can judge, not the absence of one — announced like the other degraded
|
|
259
264
|
// causes, and flagged.
|
|
@@ -264,7 +269,6 @@ export function readState(path, name, warn = console.error) {
|
|
|
264
269
|
}
|
|
265
270
|
return {
|
|
266
271
|
quiet: num(quiet),
|
|
267
|
-
elapsed: num(elapsed),
|
|
268
272
|
digest: typeof digest === "string" ? digest : "",
|
|
269
273
|
beat: mark(beat),
|
|
270
274
|
ticked: tick(ticked),
|
|
@@ -284,11 +288,9 @@ export function readState(path, name, warn = console.error) {
|
|
|
284
288
|
// A failed write is reported and SURVIVED, never fatal, for the same reason a
|
|
285
289
|
// failed read is. But survived is not the same as ignored: it returns whether
|
|
286
290
|
// the write landed, because the two cases call for different behaviour from
|
|
287
|
-
// the caller and only the caller can act. An unpersisted `
|
|
288
|
-
//
|
|
289
|
-
//
|
|
290
|
-
// a failed write as a fire instead of counting progress that nothing is
|
|
291
|
-
// keeping.
|
|
291
|
+
// the caller and only the caller can act. An unpersisted `beat` is one the
|
|
292
|
+
// alarm's gate never passes on, so fleet-heartbeat says no alarm is armed for
|
|
293
|
+
// it instead of telling the controller to end its turn.
|
|
292
294
|
//
|
|
293
295
|
// The write is a sibling temp file renamed over the target, ledger.mjs save()'s
|
|
294
296
|
// convention, because the readers do not hold still for it. A plain
|
|
@@ -307,16 +309,16 @@ export function writeState(path, name, prev, patch) {
|
|
|
307
309
|
// from the raw parse: see `rest` in readState above.
|
|
308
310
|
const next = {
|
|
309
311
|
...(prev.rest ?? {}),
|
|
310
|
-
quiet: prev.quiet,
|
|
311
|
-
// Carried like the other
|
|
312
|
+
quiet: prev.quiet, digest: prev.digest,
|
|
313
|
+
// Carried like the other two, and omitted rather than written as
|
|
312
314
|
// `null` when there is none: fleet-tick patches `quiet`/`digest` on every
|
|
313
315
|
// tick and would otherwise erase the heartbeat's mark on the first tick
|
|
314
316
|
// after it was written — one script deleting another's key, which is the
|
|
315
317
|
// failure the ownership rule exists to prevent and which no reader could
|
|
316
318
|
// tell from a run that never beat.
|
|
317
319
|
...(prev.beat ? { beat: prev.beat } : {}),
|
|
318
|
-
// Same reason, same shape: fleet-heartbeat patches `
|
|
319
|
-
// every
|
|
320
|
+
// Same reason, same shape: fleet-heartbeat patches `beat` on
|
|
321
|
+
// every call and would otherwise erase fleet-tick's `ticked` the first
|
|
320
322
|
// time a heartbeat lands after a busy stretch.
|
|
321
323
|
...(prev.ticked ? { ticked: prev.ticked } : {}),
|
|
322
324
|
// And again for ledger.mjs's `controller` and `acted`, which both scripts
|
|
@@ -371,10 +373,10 @@ export function writeState(path, name, prev, patch) {
|
|
|
371
373
|
//
|
|
372
374
|
// The grace multiplier is the only constant, and it is a multiplier ON the
|
|
373
375
|
// recorded interval rather than a duration added to it, so it scales with the
|
|
374
|
-
// promise instead of drowning it. One whole extra interval of silence:
|
|
375
|
-
//
|
|
376
|
-
//
|
|
377
|
-
//
|
|
376
|
+
// promise instead of drowning it. One whole extra interval of silence: the
|
|
377
|
+
// alarm only fires once the controller is idle, so the controller's own turn
|
|
378
|
+
// sits between the beat falling due and the next invocation, and that turn is
|
|
379
|
+
// not bounded tightly enough to call a run dead the second one
|
|
378
380
|
// interval lapses. It is the same shape board.html already uses to call its
|
|
379
381
|
// own data stale (`interval * 1000 * 2`), kept deliberately: an operator
|
|
380
382
|
// reading two "stale" verdicts on one page should not have to learn two rules.
|
|
@@ -499,16 +501,16 @@ export function isReported(verdict) {
|
|
|
499
501
|
|
|
500
502
|
// Whole minutes, floored, because the numbers here are tens of minutes to
|
|
501
503
|
// hours and a seconds-precise age in a stall line is false precision: the
|
|
502
|
-
// mark is written once per
|
|
504
|
+
// mark is written once per idle, so its resolution is minutes anyway. That
|
|
503
505
|
// holds for production (--base 300, --ceiling 1200 — SKILL.md's only
|
|
504
506
|
// documented values) but not for every value the CLI guard actually accepts:
|
|
505
507
|
// fleet-heartbeat.mjs's own guard refuses `--ceiling < --base` the same way
|
|
506
508
|
// this file refuses other invalid shapes, and mirroring that refusal onto
|
|
507
509
|
// sub-60s `--base`/`--ceiling` was the first fix tried for the "0m" floor —
|
|
508
510
|
// until the heartbeat's own test suite turned out to lean on 2s/3s/8s/9s/64s intervals
|
|
509
|
-
// throughout, deliberately, to keep its
|
|
510
|
-
// CLI-level refusal would force that whole suite onto minute-plus
|
|
511
|
-
//
|
|
511
|
+
// throughout, deliberately, to keep its time-travelled marks small. A
|
|
512
|
+
// CLI-level refusal would force that whole suite onto minute-plus
|
|
513
|
+
// intervals for no behavioural gain, so the fix lives here instead: a value
|
|
512
514
|
// under a minute is still a real value, not an invalid one, and flooring it
|
|
513
515
|
// to "0m" is what read as a division-by-zero defect, not the sub-minute
|
|
514
516
|
// input itself.
|
package/skills/run-team/SKILL.md
CHANGED
|
@@ -111,7 +111,7 @@ phase, or in any later one, asks the maintainer which tickets to take.
|
|
|
111
111
|
|
|
112
112
|
## The wake table
|
|
113
113
|
|
|
114
|
-
A wake is done when its one `fleet-event.mjs` call has run and you have acted on every line it printed, the tick's last; with nothing left to act on,
|
|
114
|
+
A wake is done when its one `fleet-event.mjs` call has run and you have acted on every line it printed, the tick's last; with nothing left to act on, mark the beat and end your turn. What a wake owes beyond its call is under **What a wake owes beyond its call**, below. The loop has no exit but the run's own end: drain the run when it ends (**End of run**).
|
|
115
115
|
|
|
116
116
|
React to events; never block on one.
|
|
117
117
|
|
|
@@ -146,14 +146,14 @@ Run each as `~/.fleet/bin/fleet-run fleet-event.mjs <kind> <args>`, from the mai
|
|
|
146
146
|
| CI run terminal | `fleet-event.mjs ci <pr>` |
|
|
147
147
|
| Label seen (persistent Monitor) | `fleet-event.mjs label <pr>` |
|
|
148
148
|
| `CONFLICT` line | `fleet-event.mjs conflict <pr>` |
|
|
149
|
-
|
|
|
149
|
+
| Alarm (a [fleet-alarm] turn) | `fleet-event.mjs heartbeat` |
|
|
150
150
|
| Drain | `fleet-event.mjs drain --reason "<reason>"` |
|
|
151
151
|
| Undrain | `fleet-event.mjs undrain --reason "<reason>"` |
|
|
152
152
|
| Member confirmed dead or never dispatched | `fleet-event.mjs member-died <member>` |
|
|
153
153
|
|
|
154
154
|
### What a wake owes beyond its call
|
|
155
155
|
|
|
156
|
-
Act on every line the call prints, the tick's last, and
|
|
156
|
+
Act on every line the call prints, the tick's last, and mark the beat. What a wake
|
|
157
157
|
owes beyond its call:
|
|
158
158
|
|
|
159
159
|
- **Implementer report** → the PR is owed a review from here on:
|
|
@@ -305,25 +305,25 @@ it; never type a tier in over that. That append is the whole duty. See reference
|
|
|
305
305
|
|
|
306
306
|
**A verdict is a measurement only if it post-dates the run that produced it.** See references/reviewers.md.
|
|
307
307
|
|
|
308
|
-
### Nothing to do:
|
|
308
|
+
### Nothing to do: mark the beat and end your turn
|
|
309
309
|
|
|
310
|
-
- **Nothing to do right now** →
|
|
311
|
-
below: nothing external will wake you. See references/ci-and-staleness.md.
|
|
310
|
+
- **Nothing to do right now** → mark the beat, then end your turn.
|
|
312
311
|
|
|
313
|
-
**
|
|
312
|
+
**Mark the beat when there is nothing to do, and end your turn.** See references/ci-and-staleness.md.
|
|
314
313
|
|
|
315
314
|
```bash
|
|
316
315
|
~/.fleet/bin/fleet-run fleet-heartbeat.mjs \
|
|
317
|
-
[--base 300] [--ceiling 1200] [--multiplier 2]
|
|
316
|
+
[--base 300] [--ceiling 1200] [--multiplier 2]
|
|
318
317
|
```
|
|
319
318
|
|
|
320
|
-
It
|
|
319
|
+
It returns at once and prints one line. Which line it is, is the whole protocol:
|
|
321
320
|
|
|
322
|
-
- `…
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
321
|
+
- `… beat marked, alarm due in <N>s (quiet=<n>) → end your turn` — the mark is
|
|
322
|
+
recorded. The alarm's turn arrives as a user message that
|
|
323
|
+
starts `[fleet-alarm]` and names `fleet-event.mjs heartbeat`: run that
|
|
324
|
+
command and act on what the tick it ends in prints. Then mark the beat again.
|
|
325
|
+
- `… beat NOT marked …` — this beat arms no alarm you can count on, so ending
|
|
326
|
+
your turn could be the stall. Fix what it names; rerun.
|
|
327
327
|
|
|
328
328
|
**Do not raise `--ceiling` past 30 minutes.** See references/ci-and-staleness.md.
|
|
329
329
|
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
Why green not verdict, why absence is its own verdict, why conclusion not
|
|
4
4
|
stable, why behind-count only honest signal, why the tick runs on every wake
|
|
5
5
|
and why the beat is shaped as it is. Assertions live in SKILL.md's wake table,
|
|
6
|
-
**Nothing to do:
|
|
6
|
+
**Nothing to do: mark the beat and end your turn** and **Own the CI waits**; evidence here.
|
|
7
7
|
|
|
8
8
|
## Own the CI waits — members cannot hold across a run
|
|
9
9
|
|
|
@@ -96,14 +96,17 @@ time enough: reviewers label on own green without requiring currency, and
|
|
|
96
96
|
requiring it earlier costs full CI cycle per sibling merge — six wasted cycles
|
|
97
97
|
in one run. Any behind-count you hand bot expired on arrival.
|
|
98
98
|
|
|
99
|
-
##
|
|
99
|
+
## Mark the beat when there is nothing to do, and end your turn
|
|
100
100
|
|
|
101
|
-
**
|
|
102
|
-
|
|
101
|
+
**Mark the beat when there is nothing to do, and end your turn.** Nothing
|
|
102
|
+
outside your own process can start a turn in an idle session. A backgrounded
|
|
103
|
+
Monitor does not wake an idle
|
|
103
104
|
agent — measured twice, see **A member runs its test suite in the foreground** in references/member-lifecycle.md, where `fix-pr-<M>` backgrounded a suite plus a Monitor and sat idle for
|
|
104
105
|
roughly two hours — and `run-merge-bot.md`'s CI gate states the general rule:
|
|
105
|
-
"whatever wakes you is external and may never come". So
|
|
106
|
-
|
|
106
|
+
"whatever wakes you is external and may never come". So the wake-up is not a
|
|
107
|
+
watcher and not a cron entry. It is an omp extension loaded into your own
|
|
108
|
+
session, the alarm, which polls from inside the process and starts a turn once
|
|
109
|
+
the beat has fallen due.
|
|
107
110
|
|
|
108
111
|
## Tick on every wake
|
|
109
112
|
|
|
@@ -126,7 +129,7 @@ Do not raise it past 30 minutes.
|
|
|
126
129
|
|
|
127
130
|
## When you stop on purpose, say why
|
|
128
131
|
|
|
129
|
-
The beat leaves a dated mark on every
|
|
132
|
+
The beat leaves a dated mark on every call, so a run
|
|
130
133
|
that dies abruptly (a crashed harness, a closed terminal, an OOM kill) is
|
|
131
134
|
detected by its mark going stale. A run that stops KNOWINGLY can do better
|
|
132
135
|
than being detected, and nothing else can write the reason afterwards:
|
|
@@ -141,17 +144,20 @@ honestly say about a death that named nothing.
|
|
|
141
144
|
|
|
142
145
|
## Nothing to do: why the beat, and what its lines mean
|
|
143
146
|
|
|
144
|
-
- **Nothing to do right now** →
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
147
|
+
- **Nothing to do right now** → mark the beat and end your turn (SKILL.md's
|
|
148
|
+
**Nothing to do: mark the beat and end your turn**). The beat is what the
|
|
149
|
+
alarm reads: with no recorded mark its gate never passes, so a turn that
|
|
150
|
+
ends without one is the stall itself — the level-check you are one command
|
|
151
|
+
away from never runs. Listed on its own because it is the only wake that is
|
|
152
|
+
not an event — every other row of SKILL.md's wake list reports something
|
|
153
|
+
that happened; this one is the beat falling due.
|
|
154
|
+
|
|
155
|
+
- `… beat NOT marked …` — the write did not land, the file is degraded and
|
|
156
|
+
was left as it is, or the run's `.fleet` directory could not be resolved and
|
|
157
|
+
the mark went to a cwd-relative file the alarm may not read. No alarm is
|
|
158
|
+
armed for this beat (an earlier mark still on disk is all the alarm can fire
|
|
159
|
+
on), so do not take the line as a beat; fix what it names and run the
|
|
160
|
+
command again.
|
|
155
161
|
|
|
156
162
|
**The beat does not stop on idleness.** An empty shortlist and an empty supply are
|
|
157
163
|
not a reason to end the run; they are the state a newly triaged ticket arrives
|