@mark1russell7/harness 0.0.0-stage → 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mark Russell
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,101 @@
-# Temporary Holding Version
+# @mark1russell7/harness
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+A fault-injection harness for connection timers. In a cell, a client connects to a stock target in new Docker environments. The harness injects a fault, and it measures what the kernel did. Then the harness judges the measurement against the prediction of the cell.
+
+The harness does not compute predictions. The author of a cell computes the prediction, for example from a timer algebra, and puts it in the cell.
+
+## The cell
+
+A **cell** (`cell.ts`) is all that one measurement needs, as data:
+
+- **The topology.** A stock target image, pinned by digest.
+- **The settings.** The sysctls of the client namespace, and the socket options of the client.
+- **The workload.** A connection that stays idle, or a connection with a request in flight.
+- **The fault.** A black hole, a reset, a frozen process, or a stall.
+- **The prediction.** An interval [T_lo, T_hi] after the event that arms the timer, or silence in a window. After a stall, the prediction also tells when the connection recovers.
+
+## The environment
+
+`openLab(target)` (`lab.ts`) builds one environment with Testcontainers:
+
+- A new network and the target.
+- A client container that holds its network namespace.
+- A privileged sidecar in the namespace of the client, for its sysctls. Docker Desktop keeps `/proc/sys` read-only in a container that operates.
+- An injector sidecar in the namespace of the target, with NET_ADMIN. Thus the image of the target stays stock.
+- The prober ([`@mark1russell7/prober`](https://www.npmjs.com/package/@mark1russell7/prober)): a privileged container that loads the kernel program and watches the two namespaces. Each run keeps its records from its start to its end. A run whose records the prober lost is void.
+
+Each command is an argv array.
+
+## The images
+
+The harness builds two images. The tag of each image comes from the content of all the files that its Dockerfile reads. The harness copies these files into a new build context, from its own package and from the prober package. Thus the build works in a workspace and from npm.
+
+- **The harness image** (`image/Dockerfile`) has two stages, each a digest-pinned Alpine. `Cargo.lock` pins the crates. The image holds:
+  - the kernel program of the prober, compiled against the BTF of the kernel that builds it, and its Rust collector.
+  - `client`, the TCP client of a cell. Rust's `std::net` sets no socket option by itself, thus the measurement gets the timers of the namespace. The client also samples `TCP_INFO` each 2 ms, a second vantage on its socket. It waits in `poll()`, which does not lock the socket. A keepalive timer that finds its socket held tries again HZ/20 later.
+  - `inject`, which applies a fault in one `iptables-restore` commit, with monotonic stamps around the commit. Thus the start of a fault is known in the VM. A `docker exec` from Windows comes 190 ms late at the median.
+- **The Node client image** (`image/node`) is a driver client: ioredis 6.0.0 on Node 24.21.0. The digest of the image and an npm lockfile pin them. `client-ioredis.mjs` prints the lines of the TCP client on the same clock, and it adds the status changes of ioredis as `status` lines.
+
+The build does the `cargo test` of the two crates. Thus the record decoder passes its tests before an image exists. Each amount that the native programs print is a quantity in a canonical unit. For example, an RTO is in `jiffy`, a round trip in `us`, and a segment in `B`.
+
+ioredis sets the keepalive of its socket itself, after it connects. Thus a cell sets the keepalive of a driver through the keepAlive option of ioredis (`probe.driver`). The keepalive sysctls of the namespace do not apply to it.
+
+## The faults
+
+- **A black hole** is DROP rules in the two directions, in the namespace of the peer of the observer. Thus the measurement gets the kernel timers of the observer. For the client, the rules go in the namespace of the target, through the injector. For the server, they go in the namespace of the client, through its admin sidecar. Rules in the namespace of the observer make its sends fail with EPERM, and the kernel records only a retransmission that goes out (`FaultSide`).
+- **A reset** is a `REJECT --reject-with tcp-reset` rule in the namespace of the target, for the segments of the client. The next send of the client gets a reset one round trip later. It is the reference fault of the calibration.
+- **A frozen process** is the cgroup freezer (`docker pause`), while the kernel stays up.
+- **A stall** is a black hole that stops by itself, `Cell.stall` after the send of the request. The run reads the stamp of the client on the send. Then `inject clear PEER --at NS` holds the end in the VM until that instant (`lab.heal`). Thus the latency of a docker exec does not change the length of the stall.
+
+Each run starts from kernels that keep nothing from the run before it (`lab.forget()`). When a connection closes, each kernel keeps TCP metrics for the address of its peer. These metrics set the first round trip of the next connection, and thus its RTO. The harness removes the `tcp_metrics` of the two namespaces before each run.
+
+## The measurement
+
+`measure(cell, run)` (`observe.ts`) reads a run from the records of the kernel:
+
+- **The socket.** The socket of the run is the socket that the client connected from, in its namespace.
+- **The time.** It times the detection from the reference event of the prediction to the error report of the kernel. eBPF stamps the two events. For keepalive, the reference is the last received segment. For the give-up, it is the first retransmission of the RTO timer, not the tail loss probe one RTO before it. For a reset, it is the send of the client, which no kernel record marks. The client stamps the send before its write.
+- **The other views.** It also gives the same interval from the `TCP_INFO` samples of the client, and the delay before the client saw the error. It also gives the steps of the kernel: each keepalive probe, or each expiry of the RTO timer.
+- **Void runs.** A run is void when the timer that drives the measurement was two jiffies or more behind its wheel slot in the timed interval. The wheel does a slot in the tick when it is due. Thus such a firing measured the host, which stopped a vCPU, not the kernel. A keepalive timer that found its socket held also makes the run void. A run whose fault came too late is void too.
+- **Findings.** A finding is a wrong errno, a detector that did not fire, or a detection where the prediction is silence.
+- **Recovery.** After a stall (`prediction.recovers`), it times the send to the first segment back after the stall stopped. That segment is the answer of the server to the retransmission that got through. A run whose stall stopped outside the window of its prediction is void. A run where nothing came back is a finding.
+
+## The verdict
+
+`verdictOf(prediction, runs)` (`verdict.ts`) judges a cell:
+
+- **Containment.** Each run lands in [T_lo − δ_abs, T_hi + δ_abs]. `NOISE` (δ_abs) is 4 ms, from the calibration: the worst reset took 1.141 ms, and δ_abs is one jiffy at the minimum.
+- **Median equivalence.** The order-statistic interval of the median must be in the band, made 5 % wider.
+- **Extension.** A median across the edge of the band extends the cell from 11 runs to 22, then 44. Then the verdict is inconclusive.
+- **Recovery.** The recovery after a stall gets the same test as a detection, by its band. A detection is then a finding.
+
+The distribution test needs a simulator, which does not exist at this time.
+
+## Runs
+
+- **`runCell(cell)`** (`run.ts`) divides the 11 runs of a cell across two new environments. It records the manifest of each environment, and it removes the fault after each run.
+  - The `concurrency` of a cell sets how many environments operate at the same time, each with its own prober.
+  - The manifest holds the kernel release, the architecture, the boot id and the Docker version. It also holds the image ids, the packages of the image, the client and the sysctls as read back.
+  - A cell whose probe names a `driver` opens its labs with the client of that driver (`openLab(target, "ioredis")`).
+  - The harness sets the `targetSettings` of a cell in the target before a client connects, with the `settings` commands of the target (`redis-cli CONFIG SET`, then `CONFIG GET`).
+- **The observer.** A cell observes the client, or the server (`observer`). A cell on the server side measures the socket of the target for the connection of the client. The run waits for the error of that socket. `freeze-process` freezes the peer of the observer: the target for the client, the client for the server.
+  - A cell on the server side sets the sysctls of the target through `targetSysctls`, with a privileged sidecar in the namespace of the target. That sidecar starts only when a cell asks for it.
+  - Its in-flight data is the reply of the server. Thus the client sends `probe.request` first, for example a BLPOP with a timeout, and the fault occurs while the server holds it.
+- **`judge(cell, runs)`** (`run.ts`) measures and judges the raw records of a cell with the code at this time. `runCell` uses it during a cell. A holder of raw records can judge them again after a change to `observe.ts` or to the verdict.
+- **Testcontainers loads when a lab first opens.** Code that only measures and judges does not load it.
+
+## Installation
+
+```sh
+pnpm add @mark1russell7/harness zod
+```
+
+A run needs Docker, and privileged containers for the prober and the sidecars. The kernel must have BTF (`/sys/kernel/btf/vmlinux`).
+
+## Tests
+
+`pnpm --filter @mark1russell7/harness test` starts the tests. They measure test records with the shape of real kernel records (keepalive, and the tail loss probe before the retransmissions of the RTO timer). They also test the containment, the median equivalence, the extension and the silence of the verdict.
+
+The cells themselves need Docker and privileged containers. CI does not start them.
+
+`pnpm --filter @mark1russell7/harness build` writes the package for npm into `build/`.

package/build/index.d.ts

@@ -0,0 +1,455 @@
+import { Quantity } from "@mark1russell7/quantity";
+import { KernelRecord } from "@mark1russell7/prober";
+//#region src/cell.d.ts
+/**
+ * A fault is a failure that the harness injects, by the name of its manifestation. Each name is
+ * the last segment of the identifier of the manifestation in the corpus:
+ * - `blackhole` is two DROP rules in the network namespace of the target. Thus the measurement
+ *   gets the kernel timers of the client.
+ * - `freeze-process` is the cgroup freezer on the target, while its kernel stays up.
+ * - `rst` answers the next segment of the client with a reset, from the namespace of the target.
+ *   It is the reference for the noise of the harness itself.
+ * - `stall` is a black hole that stops by itself, `Cell.stall` after the client sends its
+ *   request. The VM times it. It shows what recovers, and when.
+ */
+type Fault = "blackhole" | "freeze-process" | "rst" | "stall";
+/**
+ * A reference is the event that the time of a detector counts from: the event that arms the
+ * timer in its machine document. Keepalive counts from the last received segment. The give-up of
+ * the retransmissions counts from the first retransmission of the RTO timer, not from the tail
+ * loss probe one RTO before it. The request itself meets a reset, thus a reset counts from the
+ * send of the client.
+ */
+type Reference = "last-received" | "first-retransmission" | "request-sent";
+/**
+ * This prediction tells when a connection moves again after a stall. That occurs at the first
+ * firing of its retransmission timer after the stall stops, and its reply comes back one round
+ * trip later.
+ */
+interface Recovers {
+  /** This count identifies the firing that gets through: the number of firings before it. */
+  readonly firing: number;
+  /** This time is the earliest return of the reply, from the send of the request. */
+  readonly low: Quantity;
+  /** This time is the latest return of the reply, from the send of the request. */
+  readonly high: Quantity;
+  /**
+   * This range is where the stall must stop, from the send, so that this firing is the first
+   * after it. The range starts after the latest time of the firing before, and it stops before
+   * the earliest time of this firing.
+   */
+  readonly heal: {
+    readonly low: Quantity;
+    readonly high: Quantity;
+  };
+}
+/**
+ * A prediction is what a cell predicts. Either a detection occurs in an interval [T_lo, T_hi]
+ * after the reference event, or nothing detects the fault in a window. Nothing detects a stall
+ * that is shorter than the give-up, and the connection recovers from it. The prediction then also
+ * tells when.
+ */
+type Prediction = {
+  readonly detects: true;
+  /** This name is the timer of the machine that fires first, or the event that detects the fault, with its errno. */
+  readonly timer: string;
+  readonly errno: string;
+  readonly from: Reference;
+  /** This time is the closed form, less the most that the timer can fire early. */
+  readonly low: Quantity;
+  /** This time is the closed form, plus the most that the timer can fire late. */
+  readonly high: Quantity;
+} | {
+  readonly detects: false;
+  readonly window: Quantity;
+  readonly recovers?: Recovers;
+};
+/** A target is the server that a cell connects to: a stock image. */
+interface Target {
+  /** This image is a stock image, pinned by digest. */
+  readonly image: string;
+  readonly port: number;
+  /** This text is the log line that tells that the target serves. */
+  readonly ready: string;
+  /**
+   * These commands set and read one setting of the target, as argv in it. The name and the value
+   * come after them: `redis-cli CONFIG SET` and `CONFIG GET` for Redis. Without them, the target
+   * has no settings to set.
+   */
+  readonly settings?: {
+    readonly set: readonly string[];
+    readonly get: readonly string[];
+  };
+}
+/** A client is what the runs of a cell use: the TCP client of the harness, or a driver on its socket. */
+type Client = "tcp" | "ioredis";
+/**
+ * An observer is the side whose detection a cell measures. The client is one observer. The
+ * server is the other observer. It is the hold of the target on the connection of the client, at
+ * its server socket in its namespace. The fault is on the peer of the observer. Thus
+ * freeze-process freezes the target for the client, and the client for the server.
+ */
+type Observer = "client" | "server";
+/** A cell is one cell of the matrix: a topology, its settings, a workload, a fault and the prediction. */
+interface Cell {
+  readonly name: string;
+  readonly target: Target;
+  /** These are the sysctls of the network namespace of the client, as the kernel spells them. The harness sets them before the client connects. */
+  readonly sysctls: Readonly<Record<string, string>>;
+  readonly probe: {
+    /** This mode tells if the client is idle after its first reply, or sends a request into the fault. */
+    readonly mode: "idle" | "inflight";
+    /** This flag sets SO_KEEPALIVE without overrides for the socket: the keepalive sysctls of the namespace apply. */
+    readonly keepalive: boolean;
+    /** This time sets TCP_USER_TIMEOUT. Without it, the default of the kernel (off) applies. */
+    readonly userTimeout?: Quantity;
+    /**
+     * This text is the request that an in-flight run sends, as a Redis inline command. Without it,
+     * the request is PING. BLPOP with a timeout makes the server answer only at the timeout. Thus
+     * a cell on the server side has a reply in flight when the fault occurs.
+     */
+    readonly request?: string;
+    /**
+     * This driver replaces the TCP client of the harness, with its own options. It is ioredis 6 on
+     * Node, whose keepAlive is the keepalive delay of the socket. The driver then ignores `keepalive`
+     * and `userTimeout`.
+     */
+    readonly driver?: {
+      readonly name: "ioredis";
+      readonly keepAlive: Quantity;
+    };
+  };
+  readonly fault: Fault;
+  /** This time is the length of a stall, from the send of the request. */
+  readonly stall?: Quantity;
+  readonly prediction: Prediction;
+  /** This side is the side whose detection the cell measures. Without it, the side is the client. */
+  readonly observer?: Observer;
+  /** These are the settings of the target. The harness sets them in each environment before a client connects. */
+  readonly targetSettings?: Readonly<Record<string, string>>;
+  /** These are the sysctls of the network namespace of the target, for a cell on the server side: its tcp_retries2. */
+  readonly targetSysctls?: Readonly<Record<string, string>>;
+  /** This count is the number of runs of the cell (11), across new environments (two at least). */
+  readonly runs: number;
+  readonly environments: number;
+  /**
+   * This count is the number of environments at the same time, each with its own prober. Without
+   * it, the count is one. The calibration does its reference cells at 1, 4 and 8, to find how
+   * many environments the machine holds before its noise increases.
+   */
+  readonly concurrency?: number;
+}
+/** A probe event is one line that the client wrote. `t` is CLOCK_MONOTONIC in nanoseconds: an instant, not an amount. */
+interface ProbeEvent {
+  readonly ev: string;
+  readonly t: number;
+  readonly [field: string]: unknown;
+}
+/** A run is one run of a cell, as the harness records it. */
+interface Run {
+  readonly environment: number;
+  readonly index: number;
+  readonly events: readonly ProbeEvent[];
+  /** This bracket is the time of the injector around the commit of the fault, on the same clock. It is null for a freeze. */
+  readonly fault: {
+    readonly t0: number;
+    readonly t1: number;
+  } | null;
+  /** This bracket is the time around the commit that stopped a stall. */
+  readonly heal?: {
+    readonly t0: number;
+    readonly t1: number;
+  } | null;
+  /** These are the records of the prober from the start to the end of the run, in the two namespaces. */
+  readonly kernel: readonly KernelRecord[];
+  /** This inode identifies the network namespace of the client, thus the records of the client. */
+  readonly netns: number;
+  /** This inode identifies the namespace of the target, where the records of its server socket are. */
+  readonly targetNetns: number;
+  /**
+   * This count is the number of records that the prober lost in the run. One lost record makes
+   * the run void. Null also makes the run void: no clock line came after the run to tell the count.
+   */
+  readonly lost: number | null;
+}
+//#endregion
+//#region src/observe.d.ts
+/**
+ * An outcome tells how the wait of a run ended. The socket failed with an errno, or it read the
+ * end of the file. Or nothing came, or the reply came back after a stall.
+ */
+type Outcome = {
+  readonly kind: "error";
+  readonly errno: string;
+} | {
+  readonly kind: "eof";
+} | {
+  readonly kind: "silent";
+} | {
+  readonly kind: "answered";
+};
+/** A measurement is what one run measured, against the prediction of its cell. */
+interface Measured {
+  readonly environment: number;
+  readonly index: number;
+  readonly outcome: Outcome;
+  /**
+   * This time is from the reference event of the prediction to the detection, both as the kernel
+   * stamped them in the records of the prober. It is null when nothing detected the fault.
+   */
+  readonly elapsed: Quantity | null;
+  /** This time is the same interval from the TCP_INFO samples of the client, the second vantage. */
+  readonly socket: Quantity | null;
+  /** This time is how long the client took to see the error after the kernel reported it. */
+  readonly noticed: Quantity | null;
+  /**
+   * These are the steps of the kernel after the reference: each keepalive probe, or each firing of
+   * the RTO timer. After a stall, they are each retransmission from the send, the loss probe first.
+   */
+  readonly steps: readonly Quantity[];
+  /** This count is the number of keepalive probes that the kernel sent. */
+  readonly probes: number;
+  /** This count is the number of segments that the kernel sent again. */
+  readonly retransmissions: number;
+  /** This count is the number of segments that the socket received after the request. */
+  readonly acks: number;
+  /** This text is the reason that the run cannot count, or null. A void run is not a pass and not a finding. */
+  readonly void: string | null;
+  /** This text tells what in the run disagrees with the prediction, other than its timing, or null. */
+  readonly finding: string | null;
+}
+/**
+ * This function reads one run against its cell, from the records of the kernel first.
+ * - The socket of the run is the socket that the client connected from, in its namespace.
+ * - The fault must hold before the event that it must meet. That is before the request goes into
+ *   it, and before the first keepalive probe. A live peer answers that probe, and the answer
+ *   starts the idle time again. A run where the fault was late is void. A run whose records the
+ *   prober lost is also void.
+ */
+export declare const measure: (cell: Cell, run: Run) => Measured;
+//#endregion
+//#region src/verdict.d.ts
+/**
+ * This amount is the noise of the pipeline itself, δ_abs, as the calibration measured it. It is
+ * the 99.9th percentile of the whole latency of a reset, at the concurrency cap or less. Its
+ * minimum is one jiffy. On 2026-10-12 the worst of 60 resets at 1 and 4 environments took
+ * 1.141 ms, thus the minimum sets it to 4 ms. The calibration occurs again after each change to
+ * the harness, the kernel or Docker.
+ */
+export declare const NOISE: Quantity;
+/** This fraction, δ_rel, is how far the median of a cell can be outside the prediction and stay equivalent. */
+export declare const RELATIVE = .05;
+/** These are the run counts of a cell: 11, then 22 and 44 while its median crosses the edge of the band. */
+export declare const STEPS: readonly [11, 22, 44];
+/** A decision is the result of a verdict. */
+type Decision = "holds" | "fails" | "extend" | "inconclusive";
+/** An interval is a range of times. */
+interface Interval {
+  readonly low: Quantity;
+  readonly high: Quantity;
+}
+/** A verdict is the judgement of a cell on its runs. */
+interface Verdict {
+  readonly decision: Decision;
+  readonly counted: number;
+  readonly voided: number;
+  /** This band is where each run must land: [T_lo − δ_abs, T_hi + δ_abs]. It is null for a silent prediction. */
+  readonly band: Interval | null;
+  /** These are the runs outside the band, and the reason for each. Each is a finding. */
+  readonly outside: ReadonlyArray<{
+    readonly run: string;
+    readonly why: string;
+  }>;
+  readonly median: Quantity | null;
+  /** This interval is the order-statistic interval of the median at 95 %, or the whole range for fewer than six runs. */
+  readonly medianInterval: Interval | null;
+  readonly why: string;
+}
+/**
+ * This function gives the distribution-free interval of a median: [x_(k), x_(n+1−k)], for the
+ * largest k whose two tails each hold 2.5 % or less. For eleven runs, it gives [x_(2), x_(10)],
+ * at 98.8 %.
+ */
+export declare const medianInterval: (sorted: readonly number[]) => [number, number];
+/**
+ * This function gives the verdict of a cell, from two tests:
+ * - Containment: each run lands in [T_lo − δ_abs, T_hi + δ_abs]. A run outside is a finding.
+ * - Median equivalence: the interval of the median is in the band, made wider by δ_rel. An
+ *   interval outside the band fails. An interval across its edge extends the cell to 22 runs, then
+ *   44, and then the verdict is inconclusive.
+ *
+ * A silent prediction holds only if no run detected the fault in its window. After a stall, the
+ * return of the reply gets the same test as a detection, by its interval, and a detection is a
+ * finding. The distribution test (a DKW band) needs a predicted distribution from a simulator,
+ * which does not exist at this time.
+ */
+export declare const verdictOf: (prediction: Prediction, runs: readonly Measured[]) => Verdict;
+//#endregion
+//#region src/lab.d.ts
+/** This function gives the image of the harness. */
+export declare const harnessImage: () => Promise<string>;
+/** This function gives the image of the Node client. */
+export declare const nodeClientImage: () => Promise<string>;
+/** A network fault is a fault that the injector applies, by the name of its manifestation. */
+type NetworkFault = "blackhole" | "rst";
+/**
+ * A fault side is the namespace that gets the rules of a fault: the namespace of the peer of the
+ * observer. For the client, it is the namespace of the target (the injector). For the server, it
+ * is the namespace of the client (its privileged admin). Rules in the namespace of the observer
+ * make its sends fail locally, with EPERM, and the network does not lose them. Thus its
+ * retransmissions do not go out. The retransmission tracepoint of the kernel then records none,
+ * because it fires only for a retransmission that went out.
+ */
+type FaultSide = "target" | "client";
+/** A manifest is what a lab records about its machine, its images and its settings. */
+interface Manifest {
+  readonly kernel: string;
+  /** This text is the architecture of the machine, as uname -m gives it: x86_64. */
+  readonly architecture: string;
+  readonly hz: string | null;
+  readonly boot_id: string;
+  readonly docker: string;
+  /** This record is the target, and its settings as it read them back. */
+  readonly target: {
+    readonly image: string;
+    readonly id: string;
+    readonly settings: Readonly<Record<string, string>>;
+    /** These are the sysctls of its namespace, as read back after the harness set them. */
+    readonly sysctls: Readonly<Record<string, string>>;
+  };
+  readonly harness: {
+    readonly image: string;
+    readonly id: string;
+    readonly packages: string[];
+  };
+  /** This record is the client of each run: the TCP client of the harness, or a driver from its own image. */
+  readonly client: {
+    readonly kind: Client;
+    readonly image: string;
+  };
+  /** These are the namespaces that the prober watched, by inode. */
+  readonly netns: {
+    readonly client: number;
+    readonly target: number;
+  };
+  /** These are the sysctls of the client namespace, as read back after the harness set them. */
+  readonly sysctls: Readonly<Record<string, string>>;
+}
+/** A probe is the client container of one run. */
+interface Probe {
+  /** This method gives the first event of one of these kinds, or it rejects after `within` ms. */
+  until(kinds: readonly string[], within: number): Promise<ProbeEvent>;
+  /** This method lets an in-flight probe send its request. */
+  go(): Promise<void>;
+  /** This method freezes the client with the cgroup freezer: its process stops, and its kernel continues to answer. */
+  freeze(): Promise<void>;
+  thaw(): Promise<void>;
+  readonly events: readonly ProbeEvent[];
+  stop(): Promise<void>;
+}
+/** A lab is one environment: the target, the client, the injector and the prober. */
+interface Lab {
+  readonly targetIp: string;
+  readonly clientIp: string;
+  readonly netns: {
+    readonly client: number;
+    readonly target: number;
+  };
+  /** These are all the records that the prober wrote, in sequence. */
+  readonly kernel: readonly KernelRecord[];
+  /** This method sets the sysctls of the client namespace through its privileged sidecar, and reads them back. */
+  sysctl(values: Readonly<Record<string, string>>): Promise<Record<string, string>>;
+  /** This method sets the settings of the target in it before a client connects, and reads them back. */
+  configure(values: Readonly<Record<string, string>>): Promise<Record<string, string>>;
+  /** This method sets the sysctls of the target namespace through a privileged sidecar there, and reads them back. */
+  targetSysctl(values: Readonly<Record<string, string>>): Promise<Record<string, string>>;
+  /**
+   * This method applies a fault between the client and the target, from the namespace of the
+   * target. The fault is a black hole, or a reset that answers the next segment of the client. The stamps of
+   * the injector bracket it.
+   */
+  fault(kind: NetworkFault, at?: FaultSide): Promise<{
+    t0: number;
+    t1: number;
+  }>;
+  /** This method removes a fault. */
+  lift(kind: NetworkFault, at?: FaultSide): Promise<void>;
+  /**
+   * This method stops a black hole at an instant on the monotonic clock of the VM. The injector
+   * waits for that instant in the VM, thus the length of a stall does not include the latency of
+   * a docker exec. Stamps bracket it, as for a fault.
+   */
+  heal(at: FaultSide, when: number): Promise<{
+    t0: number;
+    t1: number;
+  }>;
+  /**
+   * This method removes what the two kernels learned from their earlier connections. These are
+   * the TCP metrics that each kernel keeps for a peer address when a connection closes. These metrics set the
+   * first round trip of the next connection, and thus its RTO. A run with these metrics is not
+   * independent of the run before it.
+   */
+  forget(): Promise<void>;
+  /** This method freezes the target with the cgroup freezer: its process stops, and its kernel continues to answer. */
+  freeze(): Promise<void>;
+  thaw(): Promise<void>;
+  probe(args: readonly string[]): Promise<Probe>;
+  /**
+   * This method waits for the next clock line of the prober, which gives the count of lost
+   * records. It gives null when no clock line comes in 5 s, because then nothing tells the count.
+   */
+  settle(): Promise<number | null>;
+  manifest(sysctls: Readonly<Record<string, string>>, settings?: Readonly<Record<string, string>>, targetSysctls?: Readonly<Record<string, string>>): Promise<Manifest>;
+  close(): Promise<void>;
+}
+/**
+ * This function opens one environment:
+ * - a new network and the stock target.
+ * - a client that holds its own network namespace, with a privileged sidecar in it for its
+ *   sysctls. Docker Desktop keeps /proc/sys read-only in a container that operates.
+ * - an injector sidecar in the namespace of the target.
+ * - the prober, a privileged container that loads the kernel program and watches the two
+ *   namespaces.
+ *
+ * The client of each run is its own container in the namespace of the client. It is the TCP
+ * client of the harness, or a driver from its own image (ioredis on Node). The two print the same
+ * lines.
+ */
+export declare const openLab: (target: Target, kind?: Client) => Promise<Lab>;
+//#endregion
+//#region src/run.d.ts
+/** A cell result is a cell with its manifests, its runs, their measurements and its verdict. */
+interface CellResult {
+  readonly cell: Cell;
+  readonly manifests: readonly Manifest[];
+  readonly runs: readonly Run[];
+  readonly measured: readonly Measured[];
+  readonly verdict: Verdict;
+  /** These times are when its first environment opened and its last environment closed, on the wall clock. */
+  readonly at: {
+    readonly started: string;
+    readonly ended: string;
+  };
+}
+/**
+ * This function measures and judges the raw records of a cell, with the code of the harness at
+ * this time. `runCell` uses it during a cell. A holder of the raw records of a cell can use it
+ * again later. Thus the records get a new analysis after a change to observe.ts or to the verdict.
+ */
+export declare const judge: (cell: Cell, runs: readonly Run[]) => {
+  measured: Measured[];
+  verdict: Verdict;
+};
+/** This function divides a count of runs across environments, as equally as possible. */
+export declare const spread: (runs: number, environments: number) => number[];
+/**
+ * This function does the runs of a cell. They go across new environments, and each environment records
+ * its manifest. Then the verdict judges the runs. A median across the edge of the band extends
+ * the cell from 11 runs to 22, and then to 44, each extension in new environments. With a
+ * concurrency of more than one, that many environments operate at the same time, each with its
+ * own prober. The runs then go across at least as many environments.
+ */
+export declare const runCell: (cell: Cell, say?: (line: string) => void) => Promise<CellResult>;
+//#endregion
+export type { Cell, CellResult, Client, Decision, Fault, FaultSide, Interval, KernelRecord, Lab, Manifest, Measured, NetworkFault, Observer, Outcome, Prediction, Probe, ProbeEvent, Recovers, Reference, Run, Target, Verdict };
+//# sourceMappingURL=index.d.ts.map