cursedops 0.11.0 → 0.11.2

package/README.md

@@ -14,11 +14,11 @@ bun add cursedops
 | `cursedops/smoke` | the scaffolding of a deployed smoke — the ledger, the fetch, the DNS hint, the exit code — the one check no app owns (every address of a deployment serving the same built client), and since 0.5.0 its VERDICTS: the origin asked on loopback, the smoke's own environment, a network that lies about DNS, and a version that has settled |
 | `cursedops/worker-deploy` | the sequence that ships a Cloudflare Worker — clean tree, stage first, schema, `--var` stamp, secrets, smoke — and the readers it needs (0.5.0); `readWranglerFile(root)` since 0.9.0, lifted from four apps' `readWrangler` (task 2153); since 0.10.25 every run, red or green, appends a `kind: "deploy"` line with each leg's wall clock (`install`, `stage`, `walk`, `build`, `schema`, `upload`, `secrets`, `smoke`) to the generation's ledger (task 2246) |
 | `cursedops/worker-deploy` (CPU) | a `wrangler tail` around the traffic a deploy already sends, each request's BILLED CPU charged to its declared route, red on an undeclared, over-budget or uncapped route — and `runWorkerDeploy`'s `stageWalk` + `cpuTail` that wrap the walk and the smoke in it (0.7.0, task 2134, lifted from collections) |
-| `cursedops/worker-secrets` | a Worker holding EXACTLY its deployment's secrets, uploaded over a pipe, read back (0.5.0) |
+| `cursedops/worker-secrets` | a Worker holding EXACTLY its deployment's secrets, uploaded over a pipe, read back (0.5.0) — and read again, up to five times, before a set that is not exact is believed: the list lags the puts (0.11.1) |
 | `cursedops/worker-rollback` | a hostname back on its Mac origin: origin first, route second, the route found rather than typed (0.5.0) |
-| `cursedops/edge-fetch` | a request to a deployed Worker as a script must make it — curl pinned past the Mac's negative DNS cache (0.5.0) |
+| `cursedops/edge-fetch` | a request to a deployed Worker as a script must make it — curl pinned past the Mac's negative DNS cache (0.5.0); a request's own `maxTimeSec` over the instance's (0.11.1) |
 | `cursedops/d1-import` | a cutover's data proof — SQL literals and the row-for-row comparison, never a count (0.5.0); and `d1SchemaText`, the generated `db/schema.sql` that refuses a statement spanning lines, because D1's exec splits on newlines (0.9.0, lifted from five apps, task 2153) — and its `forge-d1-schema` bin (0.9.1), so no app carries its own schema script at all |
-| `cursedops/backups` | `BACKUP_KEEP`, the fleet's one retention for dated snapshots — six apps each exported `KEEP = 14` (0.9.0, task 2153) |
+| `cursedops/backups` | `BACKUP_KEEP`, the fleet's one retention for dated snapshots — six apps each exported `KEEP = 14` (0.9.0, task 2153); `BACKUP_STAMP_PATTERN`, `backupStamp` and its inverse `stampTime` — the local-time run name every backup script writes and station's audit reads (0.11.2) |
 | `cursedops/serve` | the static tier's four helpers — the path-traversal guard, the MIME table, the hashed-asset test, the crash handlers |
 | `cursedops/api-floor` | the rule that an unmatched `/api/...` is a phrase and never the app shell — the namespace predicates, the trailing-slash normaliser and the default 404 body. No `node:` import, so it mounts inside a Worker |
 | `cursedops/build-info` | which commit a checkout-served process is running and whether its tree was dirty — read once at load, and a `null` retried in the background rather than cached for the life of the process. `node:child_process`, so never in a Worker |
@@ -304,6 +304,12 @@ pins an address** — the deliverable is the distinction, not a way around the n
 `steadyHealth(ask, ready)` is the Worker smoke's wait: `ready` must hold for THREE reads in a
 row, because on `music`'s first deploy (2026-09-23, `a54ef71`) one good `/healthz` was followed
 by thirteen routes answering the pre-secrets version. It returns what answered and never judges.
+A read that THROWS (0.11.1) is a bad read asked again, bounded by `deadlineMs` (240 s), and only
+a LAST read that threw is thrown to the caller: the 2026-09-29 ship reported family and music red
+on a first `/healthz` that timed out twice while the edge was between versions (task 2291).
+`runWorkerSmoke` asks each of those reads with a 10 s curl `--max-time`
+(`HEALTH_READ_MAX_TIME_SEC`, passed as `EdgeRequestInit.maxTimeSec`), so the window costs 20 s a
+read instead of 120.
 
 `awaitConverged(probes)` (0.10.19) is the same wait over SEVERAL paths: every probe must agree
 three rounds in a row, bounded (120 s), before any leg is judged — a Worker's code and its static

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "cursedops",
-	"version": "0.11.0",
+	"version": "0.11.2",
 	"description": "The build-and-ops answers this generation's apps wrote independently and identically: finding a generation's roots — and printing a command that runs when pasted — without knowing a path, the generation's whole-tree laws run over one repo from a checkout or a worktree, macOS launchd agent install/replace/remove and the live port a job serves, the scaffolding and verdicts of a deployed smoke (origin probe, the smoke's own environment, a network that lies about DNS, a settled version), the static-serving helpers eight apps copied — the path-traversal guard among them — the API floor that keeps an unmatched /api/... from ever being answered with the app shell, the commit and dirty flag a checkout-served process reports, the Cloudflare Worker deploy toolkit four apps copied (the deploy sequence, exact-set secrets over a pipe, origin-first rollback, the curl edge fetch, the row-for-row D1 import proof, the billed-CPU tail check around a deploy's walk and smoke, and each app's worker:secrets and worker:smoke main as one function of its data), the relay a Worker fronts a Mac-bound app with (the Durable Object, the frames, the Mac's dialer and key rotation — lifted from station for roms — and the signed-in stage walk's skeleton and the relay app's whole worker:deploy), the Worker import-graph and await-port checks every Worker app's suite runs over its own source, and the public-surface ratchet three published libraries each carried a forked copy of. Mechanism only — no app knows its name from here. Bun, zero runtime dependencies (typescript is an optional peer, for public-surface only), ships source.",
 	"type": "module",
 	"scripts": {

package/src/backups.ts

@@ -7,3 +7,28 @@
  * whatever was decided. So it is one number here, and each app's prune takes it.
  */
 export const BACKUP_KEEP = 14;
+
+/**
+ * The name of one dated run: `YYYYMMDD-HHMMSS`, in the Mac's LOCAL time.
+ *
+ * 🔴 A contract with two sides, so it is spelled once. Every app's own backup script WRITES
+ * these directory names and prunes only what matches the pattern; station's backup audit READS
+ * them back into a time. Until cursedops 0.11.2 the writers held nine copies of the pattern and
+ * two of the stamp, and the reader held its own pair that wrote and read the stamp as UTC — so a
+ * run the apps named `20260929-100000` at 10:00 CDT was read as 10:00Z, five hours off.
+ */
+export const BACKUP_STAMP_PATTERN = /^\d{8}-\d{6}$/;
+
+const pad = (n: number): string => String(n).padStart(2, "0");
+
+/** A `Date` → its run name, in local time. {@link stampTime} is its inverse. */
+export const backupStamp = (d: Date): string =>
+	`${d.getFullYear()}${pad(d.getMonth() + 1)}${pad(d.getDate())}-${pad(d.getHours())}${pad(d.getMinutes())}${pad(d.getSeconds())}`;
+
+/** A run name → epoch ms, read as local time — or `null` for anything that is not one. */
+export function stampTime(stamp: string): number | null {
+	if (!BACKUP_STAMP_PATTERN.test(stamp)) return null;
+	const n = (from: number, to: number): number => Number(stamp.slice(from, to));
+	const at = new Date(n(0, 4), n(4, 6) - 1, n(6, 8), n(9, 11), n(11, 13), n(13, 15));
+	return Number.isNaN(at.getTime()) ? null : at.getTime();
+}

package/src/edgeFetch.ts

@@ -58,6 +58,12 @@ export interface EdgeFetchOptions {
 	retryDelayMs?: (retry: number) => number;
 }
 
+/**
+ * A `RequestInit` this fetch reads one more field from: `maxTimeSec`, curl's `--max-time` for
+ * THIS request, over the instance's {@link EdgeFetchOptions.maxTimeSec}. A plain `fetch` ignores it.
+ */
+export type EdgeRequestInit = RequestInit & { maxTimeSec?: number };
+
 /**
  * 🔴 How many MORE times a request is sent after a connection-level curl failure that carried
  * no answer. Measured 2026-09-24: vault's prod deploy of `953e19a` reported "smoke FAILED" on one
@@ -125,7 +131,7 @@ export function parseCurlResponse(raw: Buffer, origin = "the edge", exit: number
 }
 
 /** A `fetch`-shaped function over curl — see the header. Each host is resolved once per instance — once it has an answer. */
-export function createEdgeFetch(options: EdgeFetchOptions = {}): (url: string, init?: RequestInit) => Promise<Response> {
+export function createEdgeFetch(options: EdgeFetchOptions = {}): (url: string, init?: EdgeRequestInit) => Promise<Response> {
 	const pinned = new Map<string, string>();
 	const resolve = options.resolve === undefined ? digCloudflare : options.resolve;
 	const curl =
@@ -155,7 +161,7 @@ export function createEdgeFetch(options: EdgeFetchOptions = {}): (url: string, i
 		const port = target.port || (target.protocol === "http:" ? "80" : "443");
 		const head = () => {
 			const ip = pinFor(target.hostname);
-			return ["-s", "-i", "--max-time", String(options.maxTimeSec ?? 60), ...(ip ? ["--resolve", `${target.hostname}:${port}:${ip}`] : [])];
+			return ["-s", "-i", "--max-time", String(init.maxTimeSec ?? options.maxTimeSec ?? 60), ...(ip ? ["--resolve", `${target.hostname}:${port}:${ip}`] : [])];
 		};
 		const args: string[] = [];
 		for (const [k, v] of Object.entries(options.headersFor?.(target.hostname) ?? {})) args.push("-H", `${k}: ${v}`);

package/src/smoke.ts

@@ -83,6 +83,8 @@
 import { spawnSync } from "node:child_process";
 import { lookup } from "node:dns/promises";
 import { isIP } from "node:net";
+// The package's own subpath, never `./edgeFetch.ts` — see `publishShape.test.ts`.
+import type { EdgeRequestInit } from "cursedops/edge-fetch";
 
 export interface Failure {
 	check: string;
@@ -840,12 +842,18 @@ export interface SteadyOptions {
 	reads?: number;
 	/** Reads before giving up and judging whatever answered. Default 40. */
 	tries?: number;
+	/**
+	 * Stop asking after this long, whatever `tries` says. Default 240 s — a read that THROWS can
+	 * cost a whole curl `--max-time`, so a count alone does not bound the wait.
+	 */
+	deadlineMs?: number;
 	/** Pause after a GOOD read, before the next. Default 2 s. */
 	pauseMs?: number;
 	/** Pause after a BAD read. Default 5 s. */
 	retryMs?: number;
 	/** Injected for tests. */
 	sleep?: (ms: number) => Promise<void>;
+	now?: () => number;
 }
 
 export interface SteadyRead<T> {
@@ -868,6 +876,14 @@ export interface SteadyRead<T> {
  * answering `503` (the pre-secrets version) and `404`, because one isolate had the new version
  * and the next did not. The copies in `vault` and `collections` still judged after one read.
  *
+ * 🔴 **A read that THROWS is a bad read, not the end of the wait.** Measured 2026-09-29 (task
+ * 2291): family's and music's production smokes each went red on their FIRST `/healthz`, straight
+ * after the secrets were put — `no answer … (curl exit 28)`, twice 60 s — and both hosts answered
+ * in 0.2 s moments later. The edge was between versions; rethrowing that first silence charged a
+ * green deploy with a red smoke. A throw now resets the count and is asked again until `tries`
+ * or `deadlineMs`; only when the LAST read threw is that error thrown to the caller, so a Worker
+ * that never answers is still a red that names why.
+ *
  * It never decides pass or fail: it returns what answered, and the caller judges that — a wait
  * that swallowed its own timeout would be a check that cannot go red.
  */
@@ -878,22 +894,44 @@ export async function steadyHealth<T = Record<string, unknown>>(
 ): Promise<SteadyRead<T>> {
 	const reads = options.reads ?? 3;
 	const tries = options.tries ?? 40;
+	const deadlineMs = options.deadlineMs ?? 240_000;
 	const sleep = options.sleep ?? ((ms: number) => new Promise<void>((done) => setTimeout(done, ms)));
+	const now = options.now ?? Date.now;
+	const started = now();
 	let steady = 0;
 	let asked = 0;
 	for (;;) {
-		const response = await ask();
+		let read: { response: Response; body: T } | null = null;
+		let thrown: unknown = null;
+		try {
+			const response = await ask();
+			const body = (await response
+				.clone()
+				.json()
+				.catch(() => ({}))) as T;
+			read = { response, body };
+		} catch (error) {
+			thrown = error;
+		}
 		asked++;
-		const body = (await response
-			.clone()
-			.json()
-			.catch(() => ({}))) as T;
-		steady = ready(body, response) ? steady + 1 : 0;
-		if (steady >= reads || asked >= tries) return { response, body, steady, asked };
+		steady = read && ready(read.body, read.response) ? steady + 1 : 0;
+		const out = asked >= tries || now() - started >= deadlineMs;
+		if (read && (steady >= reads || out)) return { ...read, steady, asked };
+		if (!read && out) throw thrown;
 		await sleep(steady > 0 ? (options.pauseMs ?? 2_000) : (options.retryMs ?? 5_000));
 	}
 }
 
+/**
+ * curl's `--max-time` for ONE `/healthz` read of a settling Worker, in seconds. Short on purpose:
+ * {@link steadyHealth} asks again after a silence, so a long timeout only makes the edge's
+ * between-versions window cost more — the 2026-09-29 reds spent 120 s on two reads of it.
+ */
+export const HEALTH_READ_MAX_TIME_SEC = 10;
+
+/** The init a smoke's `/healthz` read carries — `cursedops/edge-fetch` honours it; a plain `fetch` ignores it. */
+const healthRead: EdgeRequestInit = { maxTimeSec: HEALTH_READ_MAX_TIME_SEC };
+
 /** One question a deploy must answer the NEW way — `null` when it does, else why not. */
 export interface ConvergenceProbe {
 	name: string;
@@ -1065,7 +1103,7 @@ export async function runWorkerSmoke(spec: WorkerSmokeSpec): Promise<number> {
 		let status: number;
 		try {
 			const read = await steadyHealth<Record<string, unknown>>(
-				() => ask("/healthz"),
+				() => ask("/healthz", healthRead),
 				(body) => body.commit === spec.head && body.apiServed === true,
 				spec.steady,
 			);
@@ -1155,7 +1193,7 @@ async function accessDoorSmoke(
 		let health: Record<string, unknown>;
 		let status: number;
 		try {
-			const read = await steadyHealth<Record<string, unknown>>(() => ask("/healthz"), (body) => body.commit === spec.head, spec.steady);
+			const read = await steadyHealth<Record<string, unknown>>(() => ask("/healthz", healthRead), (body) => body.commit === spec.head, spec.steady);
 			health = read.body;
 			status = read.response.status;
 		} catch (error) {

package/src/workerSecrets.ts

@@ -92,6 +92,12 @@ export interface SecretSync {
 /** Tries per put. A put is idempotent, so a retry can only turn a network blip into a success. */
 export const SECRET_PUT_ATTEMPTS = 3;
 
+/**
+ * Reads of the account after the puts before a set that is not exact is believed. A read changes
+ * nothing, so asking again can only replace a stale answer with a current one — see {@link syncSecrets}.
+ */
+export const SECRET_READBACK_ATTEMPTS = 5;
+
 /**
  * Apply `plan`: put every value, delete every extra, then READ THE ACCOUNT BACK and fail unless it
  * holds exactly the set. Refuses without touching anything when the plan has `absent` keys.
@@ -100,11 +106,17 @@ export const SECRET_PUT_ATTEMPTS = 3;
  * `wrangler secret put` of three hit `fetch failed` and the whole deploy stopped with
  * collections-stage holding a Worker and no secrets — serving nothing — for a blip the next
  * attempt would have cleared. The read-back is still the verdict.
+ *
+ * 🔴 **The read-back is asked again, up to {@link SECRET_READBACK_ATTEMPTS} times, before a set
+ * that is not exact is believed.** Measured 2026-09-29 on family-stage (task 2291): the account
+ * listed both secrets, both puts said ok, and the list read straight after answered NOTHING — a
+ * green deploy stopped as "secrets NOT attached". The same script run again a minute later read
+ * both. The LAST read is the verdict, so a set that is really wrong is still a red.
  */
 export async function syncSecrets(
 	plan: SecretPlan,
 	ops: SecretOps,
-	options: { attempts?: number; pause?: (ms: number) => Promise<void> } = {},
+	options: { attempts?: number; readbacks?: number; pause?: (ms: number) => Promise<void> } = {},
 ): Promise<SecretSync> {
 	const attempts = Math.max(1, options.attempts ?? SECRET_PUT_ATTEMPTS);
 	const pause = options.pause ?? ((ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms)));
@@ -130,9 +142,20 @@ export async function syncSecrets(
 		lines.push(`DELETE ${name} — not in the set — ${ok ? "gone" : "FAILED"}`);
 		if (!ok) bad++;
 	}
-	const after = await ops.held();
-	const exact = after.length === plan.wanted.length && plan.wanted.every((name) => after.includes(name));
-	lines.push(exact ? `holds exactly: ${after.join(", ")}` : `🔴 holds ${after.join(", ") || "(nothing)"} — not exactly ${plan.wanted.join(", ")}`);
+	const readbacks = Math.max(1, options.readbacks ?? SECRET_READBACK_ATTEMPTS);
+	const isExact = (held: readonly string[]) => held.length === plan.wanted.length && plan.wanted.every((name) => held.includes(name));
+	let after = await ops.held();
+	let reads = 1;
+	while (!isExact(after) && reads < readbacks) {
+		await pause(2_000 * reads);
+		after = await ops.held();
+		reads++;
+	}
+	const exact = isExact(after);
+	const reread = reads > 1 ? ` (read ${reads} times)` : "";
+	lines.push(
+		exact ? `holds exactly: ${after.join(", ")}${reread}` : `🔴 holds ${after.join(", ") || "(nothing)"} — not exactly ${plan.wanted.join(", ")}${reread}`,
+	);
 	return { ok: exact && bad === 0, lines, after };
 }