@mark1russell7/harness 0.11.6 → 0.11.8

package/README.md

@@ -51,7 +51,9 @@ The element before a router reaches the networks after the router through it. It
 
 A lookup treats an expired conntrack entry as gone, thus the router forgets a flow at its timeout, to the jiffy. The DESTROY event of the entry waits for the garbage collector or a list of the table. Thus `ctwatch.py` in the router lists the table each `pump` (`PUMP`, 0.1 s by default). A DESTROY then follows an expiry within the pump and two lists of the table.
 
-`LISTING` (20 ms) is the longest list of the table that a prediction assumes. A longer list writes a `pump-slow` line, and a run with one near a detection by a timer of conntrack is void. The `reset-both` form sends its resets on the DESTROY event. Each reset has the sequence number that its end expects. A raw sniffer reads it from the last acknowledgment of that end.
+`LISTING` (20 ms) is the longest list of the table that a prediction assumes. A longer list writes a `pump-slow` line, and a run with one near a detection by a timer of conntrack is void. `PUMP_SLACK` (5 ms) is how late the pump can be after its interval and two lists. Its sleep can end late, and the read of the event takes time. Thus a DESTROY follows an expiry within the interval of the pump, two lists and that slack.
+
+The `reset-both` form sends its resets on the DESTROY event. Each reset has the sequence number that its end expects. A raw sniffer reads it from the last acknowledgment of that end.
 
 The lines of each router (`Lab.middlebox`) are the conntrack events and the resets that it sent, on CLOCK_MONOTONIC. Each run keeps the lines from its start to its end.
 
@@ -158,11 +160,12 @@ The harness ships eight images, each as a shipped source (`ImageSource`). The ta
 - **The Node client image** (`image/node`) is a driver client: ioredis 6.0.0 on Node 24.21.0. The digest of the image and an npm lockfile pin them. `client-ioredis.mjs` prints the lines of the TCP client on the same clock, and it adds the status changes of ioredis as `status` lines. The same image holds the client of a group in Sentinel mode (`client-sentinel.mjs`) and the observer of its Sentinels (`observer.mjs`).
 - **The Go HTTP image** (`image/go-http`) is the `http.Transport` of Go 1.27.2 at its defaults (`client-go`). The program uses the standard library only, thus the digest of the Go builder pins all of its code. The build does `gofmt` and `go vet` first. A wrapper on each connection writes what the read loop of the transport reads. The trace of the transport writes each try of a request.
 - **The Node HTTP image** (`image/node-http`) holds the `http.Agent` of Node 24.21.0 (`client-agent.mjs`) and an HTTP/1.1 `Client` of undici 8.12.2 (`client-undici.mjs`). The digest of the image and an npm lockfile pin them. It is apart from the image of ioredis, thus the cells of ioredis keep their image.
-- **The PostgreSQL client image** (`image/pg`) is node-postgres: pg 8.23.1 with pg-pool 3.14.0 on Node 24.21.0. The digest of the image and an npm lockfile pin them. `client-pg.mjs` makes one `pg.Pool` with the options of the cell (`Cell.driverOptions`), and it prints the lines of the TCP client on the same clock. The first query makes the connection, and the reply gives the client back to the pool, where it is idle. An error of the pool for that idle client is the detection of an idle cell. The probe is `{ driver: { name: "pg", query } }`, with `SELECT 1` as the query when the cell gives none.
-- **The HikariCP client image** (`image/hikari`) is HikariCP 7.1.0 over pgjdbc 42.7.13 on Temurin 21.0.12.1. The digests of the two stages and the SHA-256 of each jar pin them. The probe is `{ driver: { name: "hikari", query } }`.
+- **The PostgreSQL client image** (`image/pg`) is node-postgres: pg 8.23.1 with pg-pool 3.14.0 on Node 24.21.0. The digest of the image and an npm lockfile pin them. `client-pg.mjs` makes one `pg.Pool` with the options of the cell (`Cell.driverOptions`), and it prints the lines of the TCP client on the same clock. The first query makes the connection, and the reply gives the client back to the pool, where it is idle. An error of the pool for that idle client is the detection of an idle cell. The probe is `{ driver: { name: "pg", query, inflight } }`, with `SELECT 1` as the query when the cell gives none.
+  - `inflight` is the text of the query in flight: the second query of an in-flight run. The first query then makes the pool ready at once, and only the query in flight is slow, for example `SELECT pg_sleep(20)`. The HikariCP client reads it too.
+- **The HikariCP client image** (`image/hikari`) is HikariCP 7.1.0 over pgjdbc 42.7.13 on Temurin 21.0.12.1. The digests of the two stages and the SHA-256 of each jar pin them. The probe is `{ driver: { name: "hikari", query, inflight } }`.
   - `ClientHikari.java` makes one `HikariDataSource` with the properties of the cell (`Cell.driverOptions`). A name with the prefix `dataSource.` is a property of pgjdbc.
   - It prints the lines of the pg client on the same clock. Each socket comes from the `PGStream` of its connection, and a `pool` line gives each change of the counts of the pool. The errno of a failure is the SQLState of pgjdbc.
-  - Before its first line, it loads the classes of a connect and of a pool on the loopback of its namespace. It also starts the SecureRandom that HikariCP reads for the UUID of each new connection. A new JVM loads each class at its first use. Thus a cell that starts into a fault does not time that load.
+  - Before its first line, it loads the classes of a connect and of a pool on the loopback of its namespace. It also starts the SecureRandom that HikariCP reads for the UUID of each new connection. Then it parses an ERROR of the server, as for the timeout of a statement. A new JVM loads each class at its first use. Thus a band does not time that load.
 - **The MongoDB client image** (`image/mongo`) is the driver mongodb 7.7.0 on Node 24.21.0. The digest of the image and an npm lockfile pin them. `client-mongo.mjs` prints the lines of the TCP client on the same clock:
   - Each socket of the driver is a `socket` line with its role: `monitor`, `rtt` or `pool`. The stack of the call that makes the socket gives the role.
   - The events of the driver are `sdam` lines (the heartbeats and the changes of each server), `cmap` lines (the pools) and `command` lines (each try of an operation).

package/build/{cell-lSCgFB54.d.ts → cell-7HFQIHlK.d.ts}

@@ -43,9 +43,10 @@ import { KernelRecord } from "@mark1russell7/prober";
  *   kernel stays up. `fin` stops the process with SIGKILL.
  * - `planned-failover` asks a Sentinel for a failover: `SENTINEL FAILOVER <group>`.
  *
- * This fault is of an HTTP server:
+ * This fault is of an HTTP server or of PostgreSQL:
  * - `app-stall` injects nothing. The server process holds the reply to the request, because the
- *   request asks for that (`HttpProbe.hold`), while its kernel stays up.
+ *   request asks for that, while its kernel stays up. An HTTP request asks with `HttpProbe.hold`,
+ *   and a query of PostgreSQL with its text, for example `SELECT pg_sleep(20)`.
  *
  * This fault is of a replica set (`Cell.replicaSet`):
  * - `step-down` sends `replSetStepDown` to the primary, as a client sends a command.
@@ -756,6 +757,10 @@ interface Cell {
      * - HikariCP over pgjdbc on Java, as a `HikariDataSource` against one server. Its operation is
      *   `query`, or `SELECT 1`. `Cell.driverOptions` are the properties of HikariConfig, and a name
      *   with the prefix `dataSource.` is a property of pgjdbc.
+     *
+     * A pg or hikari driver can send a different text in flight (`inflight`): the second query of
+     * an in-flight run. The first query then makes the pool ready at once, and only the query in
+     * flight is slow, for example `SELECT pg_sleep(20)`.
      */
     readonly driver?: {
       readonly name: "ioredis";
@@ -767,9 +772,11 @@ interface Cell {
     } | {
       readonly name: "pg";
       readonly query?: string;
+      readonly inflight?: string;
     } | {
       readonly name: "hikari";
       readonly query?: string;
+      readonly inflight?: string;
     };
     /** This workload is the workload of an HTTP client (`Cell.client` is an HTTP client). The client then ignores `keepalive` and `userTimeout`. */
     readonly http?: HttpProbe;
@@ -1000,4 +1007,4 @@ interface RunTopology {
 }
 //#endregion
 export { isHopPlace as $, Recovers as A, RunPart as B, Observer as C, ProxyHop as D, ProbeEvent as E, RequestPrediction as F, ShippedImage as G, SentinelEvent as H, RouterHop as I, Subject as J, Stage as K, RttRange as L, ReplicaSet as M, ReplicaSetStage as N, ProxyTime as O, Replies as P, WsProbe as Q, Run as R, MiddleboxForm as S, Prediction as T, SentinelStage as U, RunTopology as V, Shipped as W, Topology as X, Target as Y, Toxic as Z, ImageSource as _, DriverBand as a, MiddleboxEvent as b, FaultPlace as c, GroupPrediction as d, latestEndOf as et, GroupRole as f, HttpProbe as g, HopPlace as h, ClientImage as i, Reference as j, Reconnects as k, Group as l, Hop as m, Cell as n, DriverStep as o, HarnessImage as p, StageBand as q, Client as r, Fault as s, BuildImage as t, GroupPlace as u, KernelRecord as v, PinnedImage as w, MiddleboxExpectation as x, LineMatch as y, RunGroup as z };
-//# sourceMappingURL=cell-lSCgFB54.d.ts.map
+//# sourceMappingURL=cell-7HFQIHlK.d.ts.map

package/build/index.d.ts

@@ -1,4 +1,4 @@
-import { $ as isHopPlace, A as Recovers, B as RunPart, C as Observer, D as ProxyHop, E as ProbeEvent, F as RequestPrediction, G as ShippedImage, H as SentinelEvent, I as RouterHop, J as Subject, K as Stage, L as RttRange, M as ReplicaSet, N as ReplicaSetStage, O as ProxyTime, P as Replies, Q as WsProbe, R as Run, S as MiddleboxForm, T as Prediction, U as SentinelStage, V as RunTopology, W as Shipped, X as Topology, Y as Target, Z as Toxic, _ as ImageSource, a as DriverBand, b as MiddleboxEvent, c as FaultPlace, d as GroupPrediction, et as latestEndOf, f as GroupRole, g as HttpProbe, h as HopPlace, i as ClientImage, j as Reference, k as Reconnects, l as Group, m as Hop, n as Cell, o as DriverStep, p as HarnessImage, q as StageBand, r as Client, s as Fault, t as BuildImage, u as GroupPlace, v as KernelRecord, w as PinnedImage, x as MiddleboxExpectation, y as LineMatch, z as RunGroup } from "./cell-lSCgFB54.js";
+import { $ as isHopPlace, A as Recovers, B as RunPart, C as Observer, D as ProxyHop, E as ProbeEvent, F as RequestPrediction, G as ShippedImage, H as SentinelEvent, I as RouterHop, J as Subject, K as Stage, L as RttRange, M as ReplicaSet, N as ReplicaSetStage, O as ProxyTime, P as Replies, Q as WsProbe, R as Run, S as MiddleboxForm, T as Prediction, U as SentinelStage, V as RunTopology, W as Shipped, X as Topology, Y as Target, Z as Toxic, _ as ImageSource, a as DriverBand, b as MiddleboxEvent, c as FaultPlace, d as GroupPrediction, et as latestEndOf, f as GroupRole, g as HttpProbe, h as HopPlace, i as ClientImage, j as Reference, k as Reconnects, l as Group, m as Hop, n as Cell, o as DriverStep, p as HarnessImage, q as StageBand, r as Client, s as Fault, t as BuildImage, u as GroupPlace, v as KernelRecord, w as PinnedImage, x as MiddleboxExpectation, y as LineMatch, z as RunGroup } from "./cell-7HFQIHlK.js";
 import { Quantity } from "@mark1russell7/quantity";
 import "testcontainers";
 //#region src/application.d.ts
@@ -656,6 +656,14 @@ export declare const PUMP: Quantity;
  * longer listing writes a `pump-slow` line, and a run with one near its detection is void.
  */
 export declare const LISTING: Quantity;
+/**
+ * This time is how late the pump of a router can be beyond its interval and its two listings. The
+ * pump sleeps after each listing, and the sleep can end late. The watcher also reads and stamps each
+ * event within a millisecond. Thus a DESTROY follows an expiry within the interval of the pump, two
+ * listings and this slack. It is an assumption: in run 39116866471, two of 11 forgets came 1.6 ms
+ * and 2.2 ms after the pump and two listings.
+ */
+export declare const PUMP_SLACK: Quantity;
 /**
  * A network fault is a fault that the injector applies as a filter rule. It is a black hole (both
  * directions, or one), or a reset on the next segment. It can also be an ICMP unreachable for each

package/build/index.js

@@ -1916,6 +1916,17 @@ const LISTING = {
 	value: 20,
 	unit: "ms"
 };
+/**
+* This time is how late the pump of a router can be beyond its interval and its two listings. The
+* pump sleeps after each listing, and the sleep can end late. The watcher also reads and stamps each
+* event within a millisecond. Thus a DESTROY follows an expiry within the interval of the pump, two
+* listings and this slack. It is an assumption: in run 39116866471, two of 11 forgets came 1.6 ms
+* and 2.2 ms after the pump and two listings.
+*/
+const PUMP_SLACK = {
+	value: 5,
+	unit: "ms"
+};
 /** This function gives the arguments of the injector for the options of a fault. */
 const optionArgs = (options) => [
 	...options.direction === void 0 ? [] : ["--direction", options.direction],
@@ -4082,6 +4093,7 @@ const probeArgs = (cell, lab) => {
 	}
 	if (probe.driver?.name === "pg" || probe.driver?.name === "hikari") {
 		if (probe.driver.query !== void 0) args.push("--query", probe.driver.query);
+		if (probe.driver.inflight !== void 0) args.push("--inflight-query", probe.driver.inflight);
 		if (cell.before === true) args.push("--before");
 		if (cell.driverOptions !== void 0) args.push("--options", JSON.stringify(cell.driverOptions));
 		args.push("--max-s", String(Math.ceil(longest * 2 + 10)));
@@ -4614,6 +4626,6 @@ const runCell = async (cell, say = () => {}) => {
 	}
 };
 //#endregion
-export { CLIENT_NAMES, CONVERGE_MS, INJECT, KEPT, LINGER_MS, LISTING, MEMBER_MARKS, NOISE, PINNED, PUMP, QUIET_MS, RELATIVE, REPLICA_SET_STAGES, REPOSITORY, SHIPPED_CLIENTS, SHIPPED_NAMES, STAGES, STATUS_SCRIPT, STEPS, STEP_DOWN, SUBJECTS, callOf, clockOffsetOf, closeOrExit, crossingsOf, cutOf, deadlineOf, delayVoids, deviceOf, dropFinding, dropOf, endpointAfter, errorsOf, exitOf, fieldOf, fieldsOf, firstUnanswered, goHttpImage, groupOnce, groupVerdictOf, harnessImage, hikariClientImage, hzOf, imageOf, imageProblems, inContext, infoOf, initiateScript, isHopPlace, judge, lagsOf, latestEndOf, leftAfter, matches, measure, measureDriver, measureGroup, medianInterval, memberCommand, memberConfig, memberLineOf, mongoClientImage, nodeClientImage, nodeHttpImage, noiseOf, onMonotonic, openGroupLab, openLab, openReplicaSetLab, peersOf, pgClientImage, pingsOf, planOf, proceduresImage, programLines, programOf, quantileOf, reconnectFinding, reconnectOf, referenceOf, reorderingOf, replicaSetArgs, replicaSetOnce, replicaSetStagesOf, requestOf, requestWords, routerImage, routesOf, runCell, sentinelConfig, socketAt, socketOf, socketsOf, spread, spreadOf, stagesOf, statusViewOf, subjectClient, subjectProblems, subscriptionsOf, targetImageOf, targetSourceOf, templated, toxicBody, unlockedManifests, unpinnedBases, verdictOf, viewOf, writesOf };
+export { CLIENT_NAMES, CONVERGE_MS, INJECT, KEPT, LINGER_MS, LISTING, MEMBER_MARKS, NOISE, PINNED, PUMP, PUMP_SLACK, QUIET_MS, RELATIVE, REPLICA_SET_STAGES, REPOSITORY, SHIPPED_CLIENTS, SHIPPED_NAMES, STAGES, STATUS_SCRIPT, STEPS, STEP_DOWN, SUBJECTS, callOf, clockOffsetOf, closeOrExit, crossingsOf, cutOf, deadlineOf, delayVoids, deviceOf, dropFinding, dropOf, endpointAfter, errorsOf, exitOf, fieldOf, fieldsOf, firstUnanswered, goHttpImage, groupOnce, groupVerdictOf, harnessImage, hikariClientImage, hzOf, imageOf, imageProblems, inContext, infoOf, initiateScript, isHopPlace, judge, lagsOf, latestEndOf, leftAfter, matches, measure, measureDriver, measureGroup, medianInterval, memberCommand, memberConfig, memberLineOf, mongoClientImage, nodeClientImage, nodeHttpImage, noiseOf, onMonotonic, openGroupLab, openLab, openReplicaSetLab, peersOf, pgClientImage, pingsOf, planOf, proceduresImage, programLines, programOf, quantileOf, reconnectFinding, reconnectOf, referenceOf, reorderingOf, replicaSetArgs, replicaSetOnce, replicaSetStagesOf, requestOf, requestWords, routerImage, routesOf, runCell, sentinelConfig, socketAt, socketOf, socketsOf, spread, spreadOf, stagesOf, statusViewOf, subjectClient, subjectProblems, subscriptionsOf, targetImageOf, targetSourceOf, templated, toxicBody, unlockedManifests, unpinnedBases, verdictOf, viewOf, writesOf };
 
 //# sourceMappingURL=index.js.map