@mark1russell7/harness 0.11.0 → 0.11.2

package/README.md

@@ -107,10 +107,11 @@ A fault names a member by its role at the start of a run: `primary`, or `replica
   - or by itself, a gap after its first reply (`gap`), for the keep-alive race.
 - **The lines of an HTTP client.** It writes the lines of the TCP client, with the number `i` of the request on each line of a request. It also writes the connection of each try of a request (`request`, then `retry`), with its local address and a flag for a reused connection. Each connection gets a `conn` line, and each end of a connection a line (`eof`, `conn-error`, `closed`). The end of the second request is a `done` line.
 - **A target with a script.** A target can take files before it starts (`Target.files`), for example the script of a Node server on the stock Node image. A target that writes JSON lines (`Target.lines`) gives them to each run (`Run.server`). The manifest holds the sha256 of each file and the first line of the target.
-- **The WebSocket client** (`Cell.client` is `procedures-ws`) is the `WebSocketTransport` of `@procedures/client` 0.3.0. The cell gives each of its settings (`probe.ws`): the class of its sockets, the heartbeat, the reconnect, and the times of a call.
+- **The WebSocket client** (`Cell.client` is `procedures-ws`) is the `WebSocketTransport` of `@procedures/client` 0.7.0. The cell gives each of its settings (`probe.ws`): the class of its sockets, the heartbeat, the reconnect, and the times of a call.
   - Its first request is `harness.echo`. In flight, its request into the fault is `harness.echo`, or `harness.hold` for a held reply.
   - It writes the lines of the TCP client. It also writes a line for each act of the program: `ping`, `drop`, `reconnecting` and `dial`.
-  - The class of its sockets is the global `WebSocket` of Node, or the `WebSocket` of the `ws` package (the option `webSocket`). The transport gets a subclass of that class. The subclass writes a line, then gives the work to its class. Thus the transport, undici and ws operate as they do without it.
+  - The class of its sockets is the global `WebSocket` of Node, with the transport of the root entry. Or it is the `WebSocket` of the `ws` package, with the transport of the Node entry `@procedures/client/node`.
+  - The transport gets a subclass of that class. The subclass writes a line, then gives the work to its class. Thus the transport, undici and ws operate as they do without it.
   - The subclass has `terminate()` only when its class has it, because the transport drops a socket with `terminate()` when it exists.
 - **A target from an image of the harness** (`Target.harness`). The target operates an image of the harness in place of its stock image. The procedures server is a script in the image of the WebSocket client (`server-ws.mjs`). Thus the client and the server use one release of the package. `Target.image` is the stock base of that image, and the manifest names both.
 - **An agent** (`lab.agent(image, command, { pid })`) is a container in the network namespace of the target. With `pid`, it also shares the process namespace of the target, thus it can signal the server process. Its JSON lines and its `exec` stay available to the caller.
@@ -128,12 +129,13 @@ The harness builds six images. The tag of each image comes from the content of a
 - **The Node client image** (`image/node`) is a driver client: ioredis 6.0.0 on Node 24.21.0. The digest of the image and an npm lockfile pin them. `client-ioredis.mjs` prints the lines of the TCP client on the same clock, and it adds the status changes of ioredis as `status` lines. The same image holds the client of a group in Sentinel mode (`client-sentinel.mjs`) and the observer of its Sentinels (`observer.mjs`).
 - **The Go HTTP image** (`image/go-http`) is the `http.Transport` of Go 1.27.2 at its defaults (`client-go`). The program uses the standard library only, thus the digest of the Go builder pins all of its code. The build does `gofmt` and `go vet` first. A wrapper on each connection writes what the read loop of the transport reads. The trace of the transport writes each try of a request.
 - **The Node HTTP image** (`image/node-http`) holds the `http.Agent` of Node 24.21.0 (`client-agent.mjs`) and an HTTP/1.1 `Client` of undici 8.12.2 (`client-undici.mjs`). The digest of the image and an npm lockfile pin them. It is apart from the image of ioredis, thus the cells of ioredis keep their image.
+- **The PostgreSQL client image** (`image/pg`) is node-postgres: pg 8.23.1 with pg-pool 3.14.0 on Node 24.21.0. The digest of the image and an npm lockfile pin them. `client-pg.mjs` makes one `pg.Pool` with the options of the cell (`Cell.driverOptions`), and it prints the lines of the TCP client on the same clock. The first query makes the connection, and the reply gives the client back to the pool, where it is idle. An error of the pool for that idle client is the detection of an idle cell. The probe is `{ driver: { name: "pg", query } }`, with `SELECT 1` as the query when the cell gives none.
 - **The MongoDB client image** (`image/mongo`) is the driver mongodb 7.7.0 on Node 24.21.0. The digest of the image and an npm lockfile pin them. `client-mongo.mjs` prints the lines of the TCP client on the same clock:
   - Each socket of the driver is a `socket` line with its role: `monitor`, `rtt` or `pool`. The stack of the call that makes the socket gives the role.
   - The events of the driver are `sdam` lines (the heartbeats and the changes of each server), `cmap` lines (the pools) and `command` lines (each try of an operation).
   - Each operation is a `sent` line, then an `ack` line or a `fail` line. An `ack` line has the server that accepted it. A `fail` line has the name and the labels of its error.
   - The same image holds `stepdown-mongo.mjs`, the sender of `replSetStepDown`.
-- **The procedures image** (`image/procedures`) holds `@procedures/client` 0.3.0 and `ws` 8.22.0 on Node 24.21.0, whose WebSocket is undici 7.29.1. The digest of the image and an npm lockfile pin them. It holds the WebSocket client (`client-ws.mjs`) and the script of the procedures server (`server-ws.mjs`). Each takes each of its settings as a flag, and it has no default.
+- **The procedures image** (`image/procedures`) holds `@procedures/client` 0.7.0 and `ws` 8.22.0 on Node 24.21.0, whose WebSocket is undici 7.29.1. The digest of the image and an npm lockfile pin them. It holds the WebSocket client (`client-ws.mjs`) and the script of the procedures server (`server-ws.mjs`). Each takes each of its settings as a flag, and it has no default. The lockfile holds one copy of `ws`, and the client stops when `@procedures/client/node` resolves another one.
 - **The router image** (`image/router`) is a digest-pinned Alpine with conntrack-tools, nftables, iproute2 and Python. `mbx.sh` sets the form of the router in one nftables table, `ctwatch.py` writes the conntrack events, and `resetter.py` sends the resets of the `reset-both` form. Each run records its packages in the manifest.
 
 The build does the `cargo test` of the two crates. Thus the record decoder passes its tests before an image exists. Each amount that the native programs print is a quantity in a canonical unit. For example, an RTO is in `jiffy`, a round trip in `us`, and a segment in `B`.
@@ -202,6 +204,8 @@ Each run starts from kernels that keep nothing from the run before it (`lab.forg
   - A drop counts when it comes before each report of the kernel. A report of the kernel that comes first is the detection, and the drop after it is not.
   - The second view is the stamps of the program: from its `ping` line to its `drop` line. The steps are the later pings, each from the reference.
   - The lines of a client name the socket by its local address (`local`), and the lines of a server by its peer (`peer`). For the server, the lines are the lines of the target. A line whose address is null is about a try that did not open.
+  - The harness reads the peer of a server from the records of its socket. Behind a router, that peer is the address of the router, not the address of the client.
+  - A first ping without an answer can come late, for example at a beat of a server whose phase no document gives. Then the prediction gives its latest time after the reply (`startsBy`), and the run waits that much longer.
   - A prediction can say how the program drops (`drop`). A close frame (`frame`) is a write, and the socket stays ESTABLISHED after it. A close (`close`) closes the socket in δ_abs: a FIN or an abort. Another drop is a finding. A FIN of the peer that answers a close frame is no close by the program. The run gives the time from the `drop` line to the close by the program (`left`).
   - The client process ends at the end of each run, and its kernel then closes each socket that is still open. A prediction can say that the program is that client process (`exits`). Then a close in δ_abs of that end is no close by the program: `left` stays null, and `exited` gives the time.
   - The end of the process is the first change out of ESTABLISHED of another socket of the client after its last `dial` line (`exitOf`). The program dropped none of these sockets.

package/build/index.d.ts

@@ -456,6 +456,12 @@ type Prediction = {
   readonly low: Quantity;
   /** This time is the closed form, plus the most that the timer can fire late, plus each proxy time. */
   readonly high: Quantity;
+  /**
+   * This time is the latest reference of the band after the reply of the client. It is for a
+   * reference that can come late, for example the first ping without an answer of a server
+   * heartbeat. The run then waits that much longer for the detection.
+   */
+  readonly startsBy?: Quantity;
   readonly rtt?: RttRange;
   readonly through?: readonly ProxyTime[];
   readonly group?: GroupPrediction;
@@ -560,10 +566,10 @@ type HarnessImage = "procedures";
 /**
  * A client is what the runs of a cell use: the TCP client of the harness, or a driver on its
  * socket. The HTTP clients are the `http.Transport` of Go (`go-http`), the `http.Agent` of Node
- * (`http-agent`) and an HTTP/1.1 `Client` of undici (`undici`). The MongoDB driver is `mongodb`, and the WebSocket client of
- * `@procedures/client` is `procedures-ws`.
+ * (`http-agent`) and an HTTP/1.1 `Client` of undici (`undici`). The MongoDB driver is `mongodb`, node-postgres is `pg`, and
+ * the WebSocket client of `@procedures/client` is `procedures-ws`.
  */
-type Client = "tcp" | "ioredis" | "go-http" | "http-agent" | "undici" | "mongodb" | "procedures-ws";
+type Client = "tcp" | "ioredis" | "go-http" | "http-agent" | "undici" | "mongodb" | "pg" | "procedures-ws";
 /**
  * A WebSocket probe is the workload and the settings of the WebSocket client of
  * `@procedures/client` (`procedures-ws`). The cell gives each setting, thus the client takes
@@ -677,6 +683,8 @@ interface Cell {
      * - ioredis 6 on Node, whose keepAlive is the keepalive delay of the socket.
      * - the MongoDB driver on Node, against the replica set `set`. Its operation is a write, or
      *   the read of a long document (`read-long`). `Cell.driverOptions` are its options.
+     * - node-postgres on Node, as a `pg.Pool` against one server. Its operation is `query`, or
+     *   `SELECT 1`. `Cell.driverOptions` are the options of its pool.
      */
     readonly driver?: {
       readonly name: "ioredis";
@@ -685,6 +693,9 @@ interface Cell {
       readonly name: "mongodb";
       readonly set: string;
       readonly op?: "write" | "read-long";
+    } | {
+      readonly name: "pg";
+      readonly query?: string;
     };
     /** This workload is the workload of an HTTP client (`Cell.client` is an HTTP client). The client then ignores `keepalive` and `userTimeout`. */
     readonly http?: HttpProbe;
@@ -798,6 +809,13 @@ interface HopPlace {
 type FaultPlace = "target" | "client" | HopPlace | GroupPlace;
 /** This function tells if a fault place is a hop of a topology. */
 export declare const isHopPlace: (place: FaultPlace | undefined) => place is HopPlace;
+/**
+ * This function gives the latest end of a band in seconds, from the reply of the client. It is
+ * the high bound, after the latest reference of the band (`startsBy`).
+ */
+export declare const latestEndOf: (prediction: Extract<Prediction, {
+  readonly detects: true;
+}>) => number;
 /** A middlebox event is one line of the router: a change of a conntrack entry, or a reset that it sent. `t` is CLOCK_MONOTONIC in nanoseconds. */
 interface MiddleboxEvent {
   readonly ev: string;
@@ -1561,6 +1579,8 @@ export declare const goHttpImage: () => Promise<string>;
 export declare const nodeHttpImage: () => Promise<string>;
 /** This function gives the image of the MongoDB client. */
 export declare const mongoClientImage: () => Promise<string>;
+/** This function gives the image of the PostgreSQL client. */
+export declare const pgClientImage: () => Promise<string>;
 /** This function gives the image of `@procedures/client`: its WebSocket client and its server. */
 export declare const proceduresImage: () => Promise<string>;
 /** This function gives the image that a target operates: its stock image, or an image of the harness (`Target.harness`). */

package/build/index.js

@@ -8,6 +8,11 @@ import { fileURLToPath } from "node:url";
 //#region src/cell.ts
 /** This function tells if a fault place is a hop of a topology. */
 const isHopPlace = (place) => typeof place === "object" && "hop" in place;
+/**
+* This function gives the latest end of a band in seconds, from the reply of the client. It is
+* the high bound, after the latest reference of the band (`startsBy`).
+*/
+const latestEndOf = (prediction) => valueIn(prediction.high, "s") + (prediction.startsBy === void 0 ? 0 : valueIn(prediction.startsBy, "s"));
 //#endregion
 //#region src/verdict.ts
 /** This function gives the words of the outcome of a second request: its connection, a retry, and its end. */
@@ -1652,8 +1657,9 @@ const measureSocket = (cell, run) => {
 	const closeWait = socket.find((record) => record.ev === "state" && newState(record) === 8);
 	const own = cell.prediction.detects === true && cell.prediction.errno === "CLOSED" || cell.prediction.detects === null && cell.prediction.ends === "own-close" ? socket.find((record) => record.ev === "state" && oldState(record) === 1 && newState(record) === 4) : void 0;
 	const lines = programLines(cell, run);
+	const named = server ? socket.find((record) => record.dst !== void 0)?.dst ?? local : local;
 	const kernelEnded = reported?.mono ?? closeWait?.mono ?? own?.mono ?? seen?.t;
-	const first = dropOf(lines, socket, local, noiseOf(cell));
+	const first = dropOf(lines, socket, named, noiseOf(cell));
 	const drop = first !== void 0 && (kernelEnded === void 0 || first.at <= kernelEnded) ? first : void 0;
 	const dropped = drop !== void 0;
 	const outcome = dropped ? { kind: "dropped" } : reported !== void 0 ? {
@@ -1666,7 +1672,7 @@ const measureSocket = (cell, run) => {
 	const ended = drop?.at ?? kernelEnded;
 	const noticed = reported !== void 0 && seen !== void 0 && !dropped ? between(reported.mono, seen.t) : null;
 	const { prediction } = cell;
-	const pings = pingsOf(lines, socket, local, noiseOf(cell));
+	const pings = pingsOf(lines, socket, named, noiseOf(cell));
 	const before = run.fault?.t1 ?? ready.t;
 	const referenceAt = (from) => {
 		if (from === "request-sent") return request.t;
@@ -1947,7 +1953,13 @@ const MONGO = {
 	dockerfile: "harness/image/mongo/Dockerfile",
 	from: ["harness/image/mongo"]
 };
-/** This image holds `@procedures/client` 0.3.0 on Node 24.21.0: its WebSocket client as a client of the harness, and the script of its server. */
+/** This image holds the PostgreSQL client: node-postgres (pg 8.23.1 with pg-pool 3.14.0) on Node 24.21.0. */
+const PG = {
+	name: "loom-harness-pg",
+	dockerfile: "harness/image/pg/Dockerfile",
+	from: ["harness/image/pg"]
+};
+/** This image holds `@procedures/client` 0.7.0 on Node 24.21.0: its WebSocket client as a client of the harness, and the script of its server. */
 const PROCEDURES = {
 	name: "loom-harness-procedures",
 	dockerfile: "harness/image/procedures/Dockerfile",
@@ -2025,6 +2037,8 @@ const goHttpImage = () => build(GO_HTTP);
 const nodeHttpImage = () => build(NODE_HTTP);
 /** This function gives the image of the MongoDB client. */
 const mongoClientImage = () => build(MONGO);
+/** This function gives the image of the PostgreSQL client. */
+const pgClientImage = () => build(PG);
 /** This function gives the image of `@procedures/client`: its WebSocket client and its server. */
 const proceduresImage = () => build(PROCEDURES);
 /** This function gives the image that a target operates: its stock image, or an image of the harness (`Target.harness`). */
@@ -2222,6 +2236,11 @@ const clientOf = async (kind, image) => {
 		command: ["node", "/opt/harness/mongo/client-mongo.mjs"],
 		clientKind: kind
 	};
+	if (kind === "pg") return {
+		clientImage: await pgClientImage(),
+		command: ["node", "/opt/harness/pg/client-pg.mjs"],
+		clientKind: kind
+	};
 	if (kind === "procedures-ws") return {
 		clientImage: await proceduresImage(),
 		command: ["node", "/opt/harness/procedures/client-ws.mjs"],
@@ -4185,7 +4204,7 @@ const droppedSoFar = (records) => {
 /** This function gives the argv of the client for a cell, with a deadline far after all that it waits for. */
 const probeArgs = (cell, lab) => {
 	const { prediction, probe } = cell;
-	const longest = (prediction.detects === true && prediction.driver !== void 0 ? longestOf(cell) : prediction.detects ? valueIn(prediction.high, "s") : valueIn(prediction.window, "s")) + (cell.idleBefore === void 0 ? 0 : valueIn(cell.idleBefore, "s"));
+	const longest = (prediction.detects === true && prediction.driver !== void 0 ? longestOf(cell) : prediction.detects ? latestEndOf(prediction) : valueIn(prediction.window, "s")) + (cell.idleBefore === void 0 ? 0 : valueIn(cell.idleBefore, "s"));
 	const args = [
 		lab.targetIp,
 		String(lab.targetPort),
@@ -4230,6 +4249,13 @@ const probeArgs = (cell, lab) => {
 		args.push("--max-s", String(Math.ceil(longest * 2 + 10)));
 		return args;
 	}
+	if (probe.driver?.name === "pg") {
+		if (probe.driver.query !== void 0) args.push("--query", probe.driver.query);
+		if (cell.before === true) args.push("--before");
+		if (cell.driverOptions !== void 0) args.push("--options", JSON.stringify(cell.driverOptions));
+		args.push("--max-s", String(Math.ceil(longest * 2 + 10)));
+		return args;
+	}
 	if (probe.driver !== void 0) {
 		args.push("--keepalive-ms", String(valueIn(probe.driver.keepAlive, "ms")));
 		args.push("--max-s", String(Math.ceil(longest * 2 + 10)));
@@ -4555,7 +4581,7 @@ const once = async (lab, cell, environment, index, dry = false) => {
 						"deadline"
 					], window).catch(() => void 0);
 				} else if (prediction.detects) {
-					const within = valueIn(prediction.high, "s") * 2e3 + 1e4;
+					const within = latestEndOf(prediction) * 2e3 + 1e4;
 					if (cell.observer !== void 0 && cell.observer !== "client") await observerError(cell, partial, within, () => lab.server.slice(serverFrom).some((line) => line.ev === "drop"));
 					else if (http && cell.probe.mode !== "idle") await probe.until(HTTP_DONE, within).catch(() => void 0);
 					else await probe.until([
@@ -4757,6 +4783,6 @@ const runCell = async (cell, say = () => {}) => {
 	}
 };
 //#endregion
-export { CONVERGE_MS, INJECT, KEPT, LINGER_MS, LISTING, MEMBER_MARKS, NOISE, PUMP, QUIET_MS, RELATIVE, REPLICA_SET_STAGES, STAGES, STATUS_SCRIPT, STEPS, STEP_DOWN, callOf, clockOffsetOf, closeOrExit, crossingsOf, cutOf, deadlineOf, delayVoids, deviceOf, dropFinding, dropOf, endpointAfter, errorsOf, exitOf, fieldOf, fieldsOf, firstUnanswered, goHttpImage, groupOnce, groupVerdictOf, harnessImage, hzOf, infoOf, initiateScript, isHopPlace, judge, lagsOf, leftAfter, matches, measure, measureDriver, measureGroup, medianInterval, memberCommand, memberConfig, memberLineOf, mongoClientImage, nodeClientImage, nodeHttpImage, noiseOf, onMonotonic, openGroupLab, openLab, openReplicaSetLab, peersOf, pingsOf, proceduresImage, programLines, quantileOf, reconnectFinding, reconnectOf, referenceOf, reorderingOf, replicaSetArgs, replicaSetOnce, replicaSetStagesOf, requestOf, requestWords, routerImage, routesOf, runCell, sentinelConfig, socketAt, socketOf, socketsOf, spread, spreadOf, stagesOf, statusViewOf, subscriptionsOf, targetImageOf, templated, toxicBody, verdictOf, viewOf, writesOf };
+export { CONVERGE_MS, INJECT, KEPT, LINGER_MS, LISTING, MEMBER_MARKS, NOISE, PUMP, QUIET_MS, RELATIVE, REPLICA_SET_STAGES, STAGES, STATUS_SCRIPT, STEPS, STEP_DOWN, callOf, clockOffsetOf, closeOrExit, crossingsOf, cutOf, deadlineOf, delayVoids, deviceOf, dropFinding, dropOf, endpointAfter, errorsOf, exitOf, fieldOf, fieldsOf, firstUnanswered, goHttpImage, groupOnce, groupVerdictOf, harnessImage, hzOf, infoOf, initiateScript, isHopPlace, judge, lagsOf, latestEndOf, leftAfter, matches, measure, measureDriver, measureGroup, medianInterval, memberCommand, memberConfig, memberLineOf, mongoClientImage, nodeClientImage, nodeHttpImage, noiseOf, onMonotonic, openGroupLab, openLab, openReplicaSetLab, peersOf, pgClientImage, pingsOf, proceduresImage, programLines, quantileOf, reconnectFinding, reconnectOf, referenceOf, reorderingOf, replicaSetArgs, replicaSetOnce, replicaSetStagesOf, requestOf, requestWords, routerImage, routesOf, runCell, sentinelConfig, socketAt, socketOf, socketsOf, spread, spreadOf, stagesOf, statusViewOf, subscriptionsOf, targetImageOf, templated, toxicBody, verdictOf, viewOf, writesOf };
 
 //# sourceMappingURL=index.js.map