@mark1russell7/harness 0.12.0 → 0.12.1

package/README.md

@@ -293,13 +293,15 @@ Each run starts from kernels that keep nothing from the run before it (`lab.forg
 - **A detection by the program** (`application.ts`). The kernel has no report of it. Thus the program of the observer writes a line before each of its own acts, and the kernel stamps each act on the socket.
   - The program writes `ping` before each ping of its heartbeat. The kernel stamps the write of the ping (`tx`).
   - The reference is the first ping that got no answer (`ping-sent`): no segment with data reached the socket after its write, up to the detection. Its write starts the timer of the heartbeat that the fault met. The fault can come at any time, and its stamps are not necessary.
-  - The program writes `drop` before it drops the connection itself. The detection is the kernel's stamp of that drop: the write of its close frame, or the change of its socket out of ESTABLISHED. Without such a record in δ_abs, it is the stamp of the line.
+  - The program writes `drop` before it drops the connection itself. The detection is the kernel's stamp of that drop: the write of its close frame, or the change of its socket out of ESTABLISHED. Without such a record in the handling time of the program (`DROP_HANDLING`, 1.5 ms), it is the stamp of the line. Where δ_abs is longer, the window is δ_abs.
+  - The handling time is not δ_abs. δ_abs is the noise of the pipeline, from a reset. The handling is the time of the code of the program, from the firing of its timer to its act.
+  - In 585 drops with a close frame in CI, the write came a median of 0.652 ms after the `drop` line, and 1.017 ms at the most. Two of them came more than the δ_abs of 1 ms after it. Each failed its cell, with its time in the band. These are the runs 39390222333 and 39903020657 of the client repository. The bound is that maximum and a margin of 0.48 ms.
   - A drop counts when it comes before each report of the kernel. A report of the kernel that comes first is the detection, and the drop after it is not.
   - The second view is the stamps of the program: from its `ping` line to its `drop` line. The steps are the later pings, each from the reference.
   - The lines of a client name the socket by its local address (`local`), and the lines of a server by its peer (`peer`). For the server, the lines are the lines of the target. A line whose address is null is about a try that did not open.
   - The harness reads the peer of a server from the records of its socket. Behind a router, that peer is the address of the router, not the address of the client.
   - A first ping without an answer can come late, for example at a beat of a server whose phase no document gives. Then the prediction gives its latest time after the reply (`startsBy`), and the run waits that much longer.
-  - A prediction can say how the program drops (`drop`). A close frame (`frame`) is a write, and the socket stays ESTABLISHED after it. A close (`close`) closes the socket in δ_abs: a FIN or an abort. Another drop is a finding. A FIN of the peer that answers a close frame is no close by the program. The run gives the time from the `drop` line to the close by the program (`left`).
+  - A prediction can say how the program drops (`drop`). A close frame (`frame`) is a write, and the socket stays ESTABLISHED after it. A close (`close`) closes the socket at once, in the same window: a FIN or an abort. Another drop is a finding. A FIN of the peer that answers a close frame is no close by the program. The run gives the time from the `drop` line to the close by the program (`left`).
   - The client process ends at the end of each run, and its kernel then closes each socket that is still open. A prediction can say that the program is that client process (`exits`). Then a close in δ_abs of that end is no close by the program: `left` stays null, and `exited` gives the time.
   - The end of the process is the first change out of ESTABLISHED of another socket of the client after its last `dial` line (`exitOf`). The program dropped none of these sockets.
   - A heartbeat is a timer of the program. The retransmit timer of its ping fires in the same interval. Thus a run whose retransmit timer was two jiffies or more behind its wheel slot is void: the VM stopped a vCPU.
@@ -351,8 +353,9 @@ Each run starts from kernels that keep nothing from the run before it (`lab.forg
 `verdictOf(prediction, runs)` (`verdict.ts`) judges a cell:
 
 - **Containment.** Each run lands in [T_lo − δ_abs, T_hi + δ_abs]. `NOISE` (δ_abs) is 4 ms, from the calibration of the first environment: the worst reset took 1.141 ms, and δ_abs is one jiffy at the minimum.
-  - **δ_abs of an environment.** A cell can give the δ_abs of its environment (`Cell.noise`). Stage 0 measures it in each environment. On a hosted runner of GitHub at HZ 1000, the worst reset took 0.147 ms, thus δ_abs is one jiffy, 1 ms. These read it (`noiseOf`): the bands, the pairing of a program line with its kernel record, and the drop of a program.
+  - **δ_abs of an environment.** A cell can give the δ_abs of its environment (`Cell.noise`). Stage 0 measures it in each environment. On a hosted runner of GitHub at HZ 1000, the worst reset took 0.147 ms, thus δ_abs is one jiffy, 1 ms. These read it (`noiseOf`): the bands, the pairing of a program line with its kernel record, and the drop of a program (with `DROP_HANDLING`).
   - **The injector has its own bound.** A fault that the injector commits or lifts late makes a run void. That bound is `INJECT` (4 ms), not δ_abs. Stage 0 measures the commit apart. On a hosted runner it took up to 2.75 ms at 4 environments.
+  - **The drop of a program has its own bound.** The pairing of a `drop` line with its act uses `DROP_HANDLING` (1.5 ms), or δ_abs when δ_abs is longer. The band of a drop in the cells adds `DROP_HANDLING` to its high bound.
 - **Median equivalence.** The order-statistic interval of the median must be in the band, made 5 % wider.
 - **Extension.** A median across the edge of the band extends the cell from 11 runs to 22, then 44. Then the verdict is inconclusive.
 - **Recovery.** The recovery after a stall gets the same test as a detection, by its band. A detection is then a finding.

package/build/index.d.ts

@@ -2,10 +2,47 @@ import { $ as Subject, A as Prediction, B as RequestPrediction, C as KernelRecor
 import { Quantity } from "@mark1russell7/quantity";
 import "testcontainers";
 //#region src/application.d.ts
+/**
+ * This module reads the acts of the program of the observer: its heartbeat, its drop of the
+ * connection, its reconnect and its call in flight. The kernel has no report of a detection by
+ * a program. Thus the program writes a line before each of its own acts, on CLOCK_MONOTONIC:
+ * - `ping` before each ping of its heartbeat, and `drop` before it closes the connection by itself.
+ * - `reconnecting` when it starts its reconnect timer, and `dial` before each new connection.
+ *
+ * The kernel then stamps each write of the program on the socket (`tx`) and each change of the
+ * state of the socket. A line of the client names its socket by its local address (`local`), and
+ * a line of a server names it by its peer (`peer`).
+ */
+/**
+ * This time bounds the handling of a drop by the program of the observer. It starts when the timer
+ * of the program fires. It ends at the kernel's stamp of the act: the write of a close frame, or
+ * the close of the socket. The `drop` line comes in that time, before the act.
+ *
+ * This bound is not δ_abs. δ_abs is the noise of the pipeline, and stage 0 calibrates it from a
+ * reset. The handling is the time of the code of the program, and it is longer:
+ * - No source and no machine document bounds it (`http/timers.ts` of `@mark1russell7/machines`).
+ * - The handling time of the HTTP cells (`HANDLING` in `packages/cells/src/http/race.ts`) is
+ *   another act. It is the first reply of a new process, up to 11.182 ms.
+ * - A drop comes from a timer, long after the start of the process.
+ *
+ * The bound comes from the CI records of the WS and WST cells, from the `drop` line to the act:
+ * - 585 drops with a close frame had a median of 0.652 ms, a 99th percentile of 0.87 ms and a
+ *   maximum of 1.017 ms.
+ * - 559 drops with a close of the socket had a maximum of 0.914 ms.
+ * - Two drops took more than the δ_abs of 1 ms. Each failed its cell, and its time was in the band.
+ *   These are the runs 39390222333 and 39903020657 of the client repository.
+ *
+ * The bound is 1.5 ms: the maximum and a margin of 0.48 ms. The pairing of a drop with its act
+ * uses it, or δ_abs when δ_abs is longer. The band of a drop adds it to its high bound (the WS
+ * catalog of the cells). Read the records again after a change to the subject, to Node or to
+ * the runners.
+ */
+export declare const DROP_HANDLING: Quantity;
 /**
  * A program event is a line of the program of the observer, with the kernel's stamp of its act.
- * The act is a write, or a change of state of the socket. Without a record of the kernel in δ_abs
- * after the line, the instant is the stamp of the line.
+ * The act is a write, or a change of state of the socket. Without a record of the kernel in its
+ * window after the line, the instant is the stamp of the line. The window is δ_abs, and for a drop
+ * it is `DROP_HANDLING` when that is longer.
  */
 interface ProgramEvent {
   readonly line: ProbeEvent;
@@ -42,7 +79,8 @@ export declare const closeOrExit: (line: ProbeEvent, left: Quantity | null, exit
 };
 /**
  * This function tells how a drop differs from the way that its prediction gives, or gives null.
- * A close closes the socket in δ_abs. A close frame is a write, and the program does not close
+ * A close closes the socket at once: in the handling time of the program (`DROP_HANDLING`), or in
+ * δ_abs (`delta`) when δ_abs is longer. A close frame is a write, and the program does not close
  * the socket at the drop. A peer that answers the close frame can close it, and that is no finding.
  */
 export declare const dropFinding: (expected: "frame" | "close", drop: ProgramEvent, left: Quantity | null, delta?: Quantity, exited?: Quantity) => string | null;
@@ -56,7 +94,8 @@ export declare const firstUnanswered: (pings: readonly ProgramEvent[], socket: r
 /**
  * This function gives the first drop of the connection by its program. The kernel stamps it as the
  * write of a close frame. Or it stamps the change of the socket out of ESTABLISHED: a close or an
- * abort.
+ * abort. That stamp comes in the handling time of the program after the drop line
+ * (`DROP_HANDLING`), or in δ_abs (`delta`) when δ_abs is longer.
  */
 export declare const dropOf: (lines: readonly ProbeEvent[], socket: readonly KernelRecord[], local: unknown, delta?: Quantity) => ProgramEvent | undefined;
 /** A reconnect reading is what the client did after the detection, from its lines and from the records of the kernel. */

package/build/index.js

@@ -463,6 +463,33 @@ const verdictOfBand = (prediction, runs, delta = NOISE) => {
 * state of the socket. A line of the client names its socket by its local address (`local`), and
 * a line of a server names it by its peer (`peer`).
 */
+/**
+* This time bounds the handling of a drop by the program of the observer. It starts when the timer
+* of the program fires. It ends at the kernel's stamp of the act: the write of a close frame, or
+* the close of the socket. The `drop` line comes in that time, before the act.
+*
+* This bound is not δ_abs. δ_abs is the noise of the pipeline, and stage 0 calibrates it from a
+* reset. The handling is the time of the code of the program, and it is longer:
+* - No source and no machine document bounds it (`http/timers.ts` of `@mark1russell7/machines`).
+* - The handling time of the HTTP cells (`HANDLING` in `packages/cells/src/http/race.ts`) is
+*   another act. It is the first reply of a new process, up to 11.182 ms.
+* - A drop comes from a timer, long after the start of the process.
+*
+* The bound comes from the CI records of the WS and WST cells, from the `drop` line to the act:
+* - 585 drops with a close frame had a median of 0.652 ms, a 99th percentile of 0.87 ms and a
+*   maximum of 1.017 ms.
+* - 559 drops with a close of the socket had a maximum of 0.914 ms.
+* - Two drops took more than the δ_abs of 1 ms. Each failed its cell, and its time was in the band.
+*   These are the runs 39390222333 and 39903020657 of the client repository.
+*
+* The bound is 1.5 ms: the maximum and a margin of 0.48 ms. The pairing of a drop with its act
+* uses it, or δ_abs when δ_abs is longer. The band of a drop adds it to its high bound (the WS
+* catalog of the cells). Read the records again after a change to the subject, to Node or to
+* the runners.
+*/
+const DROP_HANDLING = quantity(1.5, "ms");
+/** This function gives the window of the act of a drop after its line, in seconds: `DROP_HANDLING`, or δ_abs when δ_abs is longer. */
+const dropWindow = (delta) => Math.max(valueIn(DROP_HANDLING, "s"), valueIn(delta, "s"));
 /** This function gives the interval between two instants on the one clock, in seconds to the microsecond. */
 const between$3 = (from, to) => quantity(Math.round((to - from) / 1e3) / 1e6, "s");
 /** This function gives the state of a state record after the change, and before it. TCP_ESTABLISHED is 1, TCP_SYN_SENT 2. */
@@ -485,7 +512,7 @@ const onConnection = (line, local) => {
 	if (!("local" in line) && !("peer" in line)) return true;
 	return (line["local"] ?? line["peer"]) === local;
 };
-/** This function gives the first record of a socket that matches, at the stamp of a line or in δ_abs after it. */
+/** This function gives the first record of a socket that matches, at the stamp of a line or in a window (`delta`) after it. */
 const kernelAt = (socket, line, matches, delta) => {
 	const noise = valueIn(delta, "s") * 1e9;
 	return socket.find((record) => record.mono >= line.t && record.mono <= line.t + noise && matches(record));
@@ -536,11 +563,12 @@ const closeOrExit = (line, left, exit, delta = NOISE) => {
 };
 /**
 * This function tells how a drop differs from the way that its prediction gives, or gives null.
-* A close closes the socket in δ_abs. A close frame is a write, and the program does not close
+* A close closes the socket at once: in the handling time of the program (`DROP_HANDLING`), or in
+* δ_abs (`delta`) when δ_abs is longer. A close frame is a write, and the program does not close
 * the socket at the drop. A peer that answers the close frame can close it, and that is no finding.
 */
 const dropFinding = (expected, drop, left, delta = NOISE, exited) => {
-	const noise = valueIn(delta, "s");
+	const noise = dropWindow(delta);
 	const soon = left !== null && valueIn(left, "s") <= noise;
 	if (expected === "close") {
 		if (left === null) return `the program did not close its socket after the drop, where a close is at once${exited === void 0 ? "" : ": it stayed open until the client process ended"}`;
@@ -560,11 +588,12 @@ const firstUnanswered = (pings, socket, until = Number.POSITIVE_INFINITY) => pin
 /**
 * This function gives the first drop of the connection by its program. The kernel stamps it as the
 * write of a close frame. Or it stamps the change of the socket out of ESTABLISHED: a close or an
-* abort.
+* abort. That stamp comes in the handling time of the program after the drop line
+* (`DROP_HANDLING`), or in δ_abs (`delta`) when δ_abs is longer.
 */
 const dropOf = (lines, socket, local, delta = NOISE) => {
 	const line = lines.find((each) => each.ev === "drop" && onConnection(each, local));
-	return line === void 0 ? void 0 : eventOf(socket, line, (record) => isWrite(record) || record.ev === "state" && oldState$2(record) === 1, delta);
+	return line === void 0 ? void 0 : eventOf(socket, line, (record) => isWrite(record) || record.ev === "state" && oldState$2(record) === 1, quantity(dropWindow(delta), "s"));
 };
 /**
 * This function reads the reconnect of the client after the detection at `detected`. The socket of
@@ -5699,6 +5728,6 @@ const runCell = async (cell, say = () => {}) => {
 	}
 };
 //#endregion
-export { CLIENT_NAMES, CLUSTER_STAGES, CONVERGE_MS, GIVES_AWAY, INJECT, KEPT, LINGER_MS, LISTING, MEMBER_MARKS, MONITOR_DOWN, NODE_MARKS, NOISE, PINNED, PUMP, PUMP_SLACK, QUIET_MS, RELATIVE, REPLICA_SET_STAGES, REPOSITORY, SHIPPED_CLIENTS, SHIPPED_NAMES, STAGES, STATUS_SCRIPT, STEPS, STEP_DOWN, amqpClientImage, callOf, clockOffsetOf, closeOrExit, clusterArgs, clusterConf, clusterCutOf, clusterHostsOf, clusterOnce, clusterStagesOf, clusterViewOf, crossingsOf, cutOf, deadlineOf, delayVoids, deviceOf, dropFinding, dropOf, endpointAfter, errorsOf, exitOf, fieldOf, fieldsOf, firstUnanswered, goHttpImage, groupOnce, groupVerdictOf, grpcClientImage, harnessImage, hikariClientImage, hzOf, imageOf, imageProblems, inContext, infoOf, initiateScript, isHopPlace, judge, lagsOf, latestEndOf, leftAfter, logNameOf, matches, measure, measureDriver, measureGroup, medianInterval, memberCommand, memberConfig, memberLineOf, mongoClientImage, monotonicOf, nodeClientImage, nodeCommand, nodeHttpImage, nodeLineOf, nodeNameOf, nodesOnMonotonic, noiseOf, onMonotonic, openClusterLab, openGroupLab, openLab, openReplicaSetLab, peersOf, pgClientImage, pingsOf, plainOf, planOf, programLines, programOf, quantileOf, reconnectFinding, reconnectOf, referenceOf, reorderingOf, replicaSetArgs, replicaSetOnce, replicaSetStagesOf, requestOf, requestWords, restamped, routerImage, routesOf, runCell, sentinelConfig, socketAt, socketOf, socketsOf, spread, spreadOf, stagesOf, statusViewOf, subjectClient, subjectProblems, subscriptionsOf, targetImageOf, targetSourceOf, templated, toolJsonOf, toxicBody, unlockedManifests, unpinnedBases, verdictOf, viewOf, writesOf };
+export { CLIENT_NAMES, CLUSTER_STAGES, CONVERGE_MS, DROP_HANDLING, GIVES_AWAY, INJECT, KEPT, LINGER_MS, LISTING, MEMBER_MARKS, MONITOR_DOWN, NODE_MARKS, NOISE, PINNED, PUMP, PUMP_SLACK, QUIET_MS, RELATIVE, REPLICA_SET_STAGES, REPOSITORY, SHIPPED_CLIENTS, SHIPPED_NAMES, STAGES, STATUS_SCRIPT, STEPS, STEP_DOWN, amqpClientImage, callOf, clockOffsetOf, closeOrExit, clusterArgs, clusterConf, clusterCutOf, clusterHostsOf, clusterOnce, clusterStagesOf, clusterViewOf, crossingsOf, cutOf, deadlineOf, delayVoids, deviceOf, dropFinding, dropOf, endpointAfter, errorsOf, exitOf, fieldOf, fieldsOf, firstUnanswered, goHttpImage, groupOnce, groupVerdictOf, grpcClientImage, harnessImage, hikariClientImage, hzOf, imageOf, imageProblems, inContext, infoOf, initiateScript, isHopPlace, judge, lagsOf, latestEndOf, leftAfter, logNameOf, matches, measure, measureDriver, measureGroup, medianInterval, memberCommand, memberConfig, memberLineOf, mongoClientImage, monotonicOf, nodeClientImage, nodeCommand, nodeHttpImage, nodeLineOf, nodeNameOf, nodesOnMonotonic, noiseOf, onMonotonic, openClusterLab, openGroupLab, openLab, openReplicaSetLab, peersOf, pgClientImage, pingsOf, plainOf, planOf, programLines, programOf, quantileOf, reconnectFinding, reconnectOf, referenceOf, reorderingOf, replicaSetArgs, replicaSetOnce, replicaSetStagesOf, requestOf, requestWords, restamped, routerImage, routesOf, runCell, sentinelConfig, socketAt, socketOf, socketsOf, spread, spreadOf, stagesOf, statusViewOf, subjectClient, subjectProblems, subscriptionsOf, targetImageOf, targetSourceOf, templated, toolJsonOf, toxicBody, unlockedManifests, unpinnedBases, verdictOf, viewOf, writesOf };
 
 //# sourceMappingURL=index.js.map