@mark1russell7/kind 0.6.0 → 0.8.1

package/README.md

@@ -137,6 +137,13 @@ From 0.5.0, the lab installs the flavor `ambient` too:
 - **The kernel.** istio-cni writes iptables rules in each pod and on each node. Thus `kind.yml` loads their modules first, and an ambient cell waits when one does not load.
 - **The log of ztunnel.** For a cell in the ambient mode, the watcher follows each ztunnel pod in istio-system (`followsZtunnel`). Each line of its log becomes a `ztunnel` line, stamped at its arrival, and each ztunnel pod is a `ztunnel-pod` line. No band reads them.
 
+From 0.7.0, a cell can restart ztunnel, and it can set the environment of ztunnel:
+
+- **The watcher.** It follows the ztunnel pods for each cell in the flavor `ambient`, also for a cell of sidecar pods there. The lines of the ztunnel pods are the evidence of a restart that no pod of the cell sees.
+- **The environment.** `mesh.ztunnel` gives variables of the DaemonSet of ztunnel for one cell, for example `CONNECTION_TERMINATION_DEADLINE`. Before the cell, `setZtunnelEnv` sets them with `kubectl set env` (`ztunnelEnvArgs`). Then it waits for the rollout, and until no old ztunnel pod is left. After the cell, it removes them in the same way.
+- **The reference `restart`.** It is the first deletion timestamp of a ztunnel pod in the run, as the watcher receives it (`restartOf`). A run of the trigger `ztunnel-restart` ends when each deleted ztunnel pod is gone, and each node has a ready new one (`ztunnelReplaced`).
+- **The DaemonSet.** The chart of ztunnel sets `maxSurge` 1. Then the controller of the DaemonSet of Kubernetes v1.37.0 skips a pod with a deletion timestamp. It makes a new pod on that node at once (`daemon_controller.go`, lines 863–924). Thus the new ztunnel runs beside the old one while the old one drains.
+
 A new member of the union `Template` breaks a reader that switches on it. That is a breaking change before 1.0.0, thus the version is 0.3.0. The record format stays `/2`, with the optional field `mesh`. A judge before 0.3.0 cannot read a record of T15.
 
 ## The image
@@ -154,6 +161,7 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
 - **The gRPC clients** (`client-grpc.mjs`) hold one channel of grpc-js 1.14.6 to the ClusterIP, and send a call of `grpc.health.v1.Health/Check` each interval on it. Each call gives the line of a request of `client.mjs`, with the pod that answered (the header `x-pod`). A call that fails at once names the errno of the last connect: refused, reset or unreachable. A call of grpc-js has no deadline at its defaults. Thus a call with no answer in the timeout is a timeout, and the client cancels it.
   - From 0.6.0, each socket of the channel also gives a `sock` line at its connect and at its close (`client-lines.ts`). Each line gives the local port of the socket (`lp`). The close line keeps the port of the connect, and it gives the code of the error of the socket (`err`), or null.
   - From 0.6.0, SIGUSR1 stops the calls of the client, and SIGUSR2 starts them again. Each signal gives a `gap` line. The channel and its socket stay open in the gap. A listener of SIGUSR1 replaces the start of the inspector of Node at that signal.
+  - From 0.8.0, SIGHUP stops the calls and closes the channel. Then the next SIGUSR2 opens a new channel before it starts the calls, and its `gap` line has `open: true`. The connect line of a socket also gives its local address (`la`): the address of the pod.
 
 ## The triggers
 
@@ -174,10 +182,18 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
 | `rollout` | no Redis pod, then a rolling update |
 | `cpu-throttle` | `cpu.max` in the cgroup of the container of the first replica |
 | `authorization-policy` | `kubectl apply` of a DENY AuthorizationPolicy on the server pods for the principal of an account (`account`, the account `default` of the clients when not given), then `kubectl delete` after the hold |
-| `idle` | SIGUSR1 to the gRPC client of each client pod, and SIGUSR2 after the hold: a gap of the calls (the target `clients`). The restore sends SIGUSR2 again |
+| `idle` | SIGUSR1 to the gRPC client of each client pod, and SIGUSR2 after the hold: a gap of the calls (the target `clients`). The restore sends SIGUSR2 again. With `middlebox`, a stateful middlebox in each replica from before the gap to the restore. With `close`, a new channel at the fault for 5 s: SIGHUP before the fault, then SIGUSR2, a wait and SIGHUP |
+| `ztunnel-restart` | `kubectl delete pod` of the ztunnel pod of each node, with its grace (`ZTUNNEL_PODS`). The flow of each client crosses the ztunnel of its own node. The restore waits for the rollout of the DaemonSet |
 
 A cgroup path that is not the scope of one container is refused. A write to the root of the node throttles or freezes each pod of the node.
 
+From 0.8.0, a plan can have steps before the fault (`prepare`). No reference comes from them, and a failed one makes the run void after the restore. The steps of the borders of T15:
+
+- **A sysctl of a namespace** (`sysctl`): the injector of the harness enters the network namespace of each client pod or each replica. There it sets one sysctl (`sysctlCommand`). Only a sysctl of the namespace (`net.`) and a whole number reach it. A black hole of T15 with `retries` sets `net.ipv4.tcp_retries2` of each client pod. The restore sets the default of a new namespace, 15 (`NETNS_DEFAULTS`).
+- **A stateful middlebox** (`middlebox`): two chains of its own (`MIDDLEBOX`) in each replica take the segments of each client pod on the HBONE port. A table of the match recent keeps the time of the last segment of each client pod. After the idle timeout with no segment, the chains drop each later segment of the flow: the silent-drop form (`middleboxCommands`). A SYN is a new flow. Conntrack stays as it is: istio-cni sends each answer of the app back to ztunnel through the connmark of its conntrack entry. The lab reads the filter table after the set and before the removal.
+- **A quiet black hole** (`quiet`): SIGUSR1 to each client 1 s before the fault, and SIGUSR2 after the clear. Thus the PING of ztunnel is the only data that gets no answer.
+- `KindLab` has the method `netns`: the commands of the injector in the namespace of a PID, in sequence, each word plain. Each one gives an `act` line with its output.
+
 The plan's `event:clock-jump` has no trigger. A step of CLOCK_REALTIME in the VM steps the clock of each container: the kubelet, the API server and the cells of other agents. A time namespace offsets only the monotonic clocks, and the runtime of the service reads only the monotonic clock.
 
 ## The predictions
@@ -229,6 +245,19 @@ From 0.6.0, the prediction `ends` is the end of the socket of each client, from
 - With a window (`window`), the failed requests must also agree with that window. The observation lasts until each socket can end, and until each request of the window ended.
 - The facts of a run give the first close of each client after the reference (`closes`).
 
+From 0.8.0, a prediction `ends` can be the end of the tunnel of each client (`of: "tunnel"`). The tunnel is the HBONE connection of its ztunnel to the replica of its last answer before the heal.
+
+- The prober gives each change of the state of a socket of the HBONE port, 15008. The lab keeps these records (`keptKernel`) as `kernel.state` lines (`tunnelsOf`). It drops the other changes of state, as before.
+- The tunnel of a client goes from the address of its pod (`la` of a `sock` line) to that replica at port 15008. Its first change out of ESTABLISHED after the reference must be in the band ± δ. It must be to FIN_WAIT1: the side of the client closes it, and the peer does not close first.
+- The first request of each client after the heal must be `ok`, on a new tunnel. The main value is the middle of the ends, in `s`.
+- From 0.8.1, `ends` has the optional band `answer`. A change to CLOSE_WAIT in that band ± δ also agrees: the peer answers the close of the client, and its FIN comes first. Before the band, the peer closed first.
+
+In 0.8.1, the middlebox uses the match recent, not the sysctls of conntrack, and `ends` has the optional field `answer`. `SYSCTL` and `NETNS_DEFAULTS` keep only `tcp_retries2`. These fields came in 0.8.0, which was not published. The fix and the optional field are a patch, thus the version is 0.8.1.
+
+New members of the union `Step` (`sysctl` and `middlebox`) and of the signals of a step (`SIGHUP`) break a reader that switches on them. `KindLab` also has a new method, `netns`. Each is a breaking change before 1.0.0, thus the version is 0.8.0. The record format stays `/2`. A trigger has the optional fields `middlebox`, `retries`, `quiet` and `close`, and `ends` has the optional field `of`. The lines `kernel.state` are new lines of a run.
+
+A new member of the union `TriggerKind`, `ztunnel-restart`, and a new member of the union `Reference`, `restart`, break a reader that switches on them. Each is a breaking change before 1.0.0, thus the version is 0.7.0. The record format stays `/2`, with the optional field `mesh.ztunnel`.
+
 A new member of the union `Prediction`, `ends`, and a new member of the union `TriggerKind`, `idle`, break a reader that switches on them. The union `Target` has the new member `clients` too. Each is a breaking change before 1.0.0, thus the version is 0.6.0. The record format stays `/2`, with the optional field `closes`: the lines `sock` and `gap` are new lines of a run.
 
 A new member of the union `Target`, `servers`, breaks a reader that switches on it. `KindLab` also has a new method, `tables`, which each implementation must have. Each is a breaking change before 1.0.0, thus the version is 0.5.0. The record format stays `/2`: the lines `tables`, `ztunnel` and `ztunnel-pod` are new lines of a run.

package/build/index.d.ts

@@ -1,4 +1,4 @@
-import { $ as updatesOf, $t as RedisDeclaration, A as TriggerV1, At as socketsOf, B as RunCheck, Bt as seconds, C as KindCellResult, Ct as restartsAt, D as PredictionV1, Dt as sliceDropOf, E as LoadSample, Et as signalsOf, F as KIND_NOISE, Ft as LOG_LABEL, G as judgeKind, Gt as KindCell, H as checkRun, Ht as showBand, I as KindDecision, It as PORT, J as propagationOf, Jt as Mesh, K as observationAfter, Kt as KindRun, L as KindVerdict, Lt as REDIS, M as upgrade, Mt as transitionOf, N as upgradeCell, Nt as CLIENT, O as RunCheckV1, Ot as sliceEmptyOf, P as Deleted, Pt as HEALTH_PORT, Q as unitOf, Qt as Prediction, R as LAST_SLACK, Rt as SERVER, S as KindCellRecordV1, Sn as scenarioProblems, St as requestsOf, T as KindVerdictV1, Tt as signalOf, U as deletionsOf, Ut as Band, V as RunFacts, Vt as secondsOf, W as factsOf, Wt as Declaration, X as referenceOf, Xt as MeshMode, Y as propagationSource, Yt as MeshFlavor, Z as reusesReset, Zt as PodSettings, _ as ClusterManifest, _n as goDuration, _t as kernelCounts, a as STAGES, an as Trigger, at as Request, b as FORMAT_V1, bn as nodeNames, bt as podsOf, c as readRecord, cn as Workload, ct as SocketLine, d as calibrationOf, dn as KindScenario, dt as byNode, en as Reference, et as Endpoint, f as cellsTable, fn as KubeProxyMode, ft as deletionOf, g as BandV1, gn as cidrRange, gt as intakeOf, h as timeSpread, hn as cidrOverlap, ht as failuresOf, i as quantileOf, in as Template, it as RecordLine, j as judge, jt as stoppingOf, k as RunFactsV1, kt as slicesOf, l as Judged, ln as KUBE_PROXY_MODES, lt as backendsAt, m as predictionText, mn as NodeImage, mt as emptyOf, n as medianInterval, nn as ResetReuse, nt as PodLine, o as kindCellRecordSchema, on as TriggerKind, ot as RulesLine, p as medianText, pn as NODE_IMAGES, pt as dropOf, q as observationEnd, qt as KubeletProbe, r as middle, rn as Stack, rt as PropagationSource, s as kindCellRecordV1Schema, sn as WindowPrediction, st as SliceLine, t as RELATIVE, tn as Replica, tt as Line, u as TimeSpread, un as KindConfig, ut as byClients, v as EnvironmentManifest, vn as kindConfig, vt as outcomesOf, w as KindCellV1, wt as rulesOf, x as KindCellRecord, xn as scenario, xt as recordsOf, y as FORMAT, yn as kubeProxyPatch, yt as podsAt, z as Propagated, zt as bandOf } from "./judge-B9kVpFE-.js";
+import { $ as updatesOf, $t as Declaration, A as TriggerV1, An as scenario, At as rulesOf, B as RunCheck, Bt as ztunnelPodsOf, C as KindCellResult, Cn as NodeImage, Ct as outcomesOf, D as PredictionV1, Dn as kindConfig, Dt as requestsOf, E as LoadSample, En as goDuration, Et as recordsOf, F as KIND_NOISE, Ft as slicesOf, G as judgeKind, Gt as PORT, H as checkRun, Ht as CLIENT, I as KindDecision, It as socketsOf, J as propagationOf, Jt as bandOf, K as observationAfter, Kt as REDIS, L as KindVerdict, Lt as stoppingOf, M as upgrade, Mt as signalsOf, N as upgradeCell, Nt as sliceDropOf, O as RunCheckV1, On as kubeProxyPatch, Ot as restartOf, P as Deleted, Pt as sliceEmptyOf, Q as unitOf, Qt as Band, R as LAST_SLACK, Rt as transitionOf, S as KindCellRecordV1, Sn as NODE_IMAGES, St as kernelCounts, T as KindVerdictV1, Tn as cidrRange, Tt as podsOf, U as deletionsOf, Ut as HEALTH_PORT, V as RunFacts, Vt as ztunnelReplaced, W as factsOf, Wt as LOG_LABEL, X as referenceOf, Xt as secondsOf, Y as propagationSource, Yt as seconds, Z as reusesReset, Zt as showBand, _ as ClusterManifest, _n as Workload, _t as dropOf, a as STAGES, an as MeshMode, at as RecordLine, b as FORMAT_V1, bn as KindScenario, bt as intakeOf, c as readRecord, cn as RedisDeclaration, ct as SliceLine, d as calibrationOf, dn as ResetReuse, dt as TunnelLine, en as KindCell, et as Endpoint, f as cellsTable, fn as Stack, ft as ZtunnelPodLine, g as BandV1, gn as WindowPrediction, gt as deletionOf, h as timeSpread, hn as TriggerKind, ht as byNode, i as quantileOf, in as MeshFlavor, it as PropagationSource, j as judge, jn as scenarioProblems, jt as signalOf, k as RunFactsV1, kn as nodeNames, kt as restartsAt, l as Judged, ln as Reference, lt as SocketLine, m as predictionText, mn as Trigger, mt as byClients, n as medianInterval, nn as KubeletProbe, nt as Line, o as kindCellRecordSchema, on as PodSettings, ot as Request, p as medianText, pn as Template, pt as backendsAt, q as observationEnd, qt as SERVER, r as middle, rn as Mesh, rt as PodLine, s as kindCellRecordV1Schema, sn as Prediction, st as RulesLine, t as RELATIVE, tn as KindRun, tt as HBONE_PORT, u as TimeSpread, un as Replica, ut as TCP_STATE, v as EnvironmentManifest, vn as KUBE_PROXY_MODES, vt as emptyOf, w as KindCellV1, wn as cidrOverlap, wt as podsAt, x as KindCellRecord, xn as KubeProxyMode, xt as isTunnelState, y as FORMAT, yn as KindConfig, yt as failuresOf, z as Propagated, zt as tunnelsOf } from "./judge-gpC49mVw.js";
 import { Quantity } from "@mark1russell7/quantity";
 import { CheckFn, Finding, HealthServer, Poll, Rule, Scenario, ServiceBuilder, ServiceDescription, Stage, Timing, TransportImpl, Verdict } from "@mark1russell7/service";
 import { RedisClient } from "@mark1russell7/service/redis";
@@ -111,10 +111,12 @@ export declare const hostsOf: (text: string, cluster: string) => ClusterHosts;
  * - `server` is the first replica of the service, in the sequence of its name.
  * - `redis` is the Redis pod.
  * - `kube-proxy` is kube-proxy on the first worker.
- * - `servers` is each replica of the service. Only the injector takes it: a rule in the network
- *   namespace of each replica, with each client pod as a peer. A REJECT resets only the sender of
- *   the segment that it matches, thus the rule goes where the segments of the clients arrive.
- * - `clients` is the process of each client pod. Only a signal takes it: the gap of the trigger `idle`.
+ * - `servers` is each replica of the service. Only the injector, a middlebox and a sysctl take it.
+ *   Each one acts in the network namespace of each replica, with each client pod as a peer. A REJECT
+ *   resets only the sender of the segment that it matches, thus the rule goes where the segments of
+ *   the clients arrive.
+ * - `clients` is the process of each client pod. Only a signal and a sysctl take it: the gap of the
+ *   trigger `idle`, and a sysctl of the namespace of each client pod.
  */
 type Target = "server" | "redis" | "kube-proxy" | "servers" | "clients";
 /** A step is one act of a fault, in the sequence of the plan. */
@@ -125,7 +127,7 @@ type Step = {
 } | {
   readonly do: "signal";
   readonly target: Target;
-  readonly signal: "SIGKILL" | "SIGSTOP" | "SIGCONT" | "SIGUSR1" | "SIGUSR2";
+  readonly signal: "SIGKILL" | "SIGSTOP" | "SIGCONT" | "SIGUSR1" | "SIGUSR2" | "SIGHUP";
 } | {
   readonly do: "pause";
   readonly target: Target;
@@ -152,14 +154,27 @@ type Step = {
 } | {
   readonly do: "apply";
   readonly object: Readonly<Record<string, unknown>>;
+} | {
+  readonly do: "sysctl";
+  readonly target: Target;
+  readonly key: string;
+  readonly value: string;
+} | {
+  readonly do: "middlebox";
+  readonly target: Target;
+  readonly idle: Quantity | null;
+  readonly port: number;
 };
 /**
  * A fault plan is the steps of a run:
+ * - `prepare` comes before the fault, and no reference comes from it: a sysctl, a middlebox, or a
+ *   stop of the calls. A plan without it has no such steps.
  * - `inject` starts the trigger. The stamp of its first act is the `fault` reference.
  * - `heal` ends the fault, `hold` after the fault. A trigger that ends by itself has no heal.
  * - `restore` brings the cell back to its steady state after the run.
  */
 interface FaultPlan {
+  readonly prepare?: readonly Step[] | undefined;
   readonly inject: readonly Step[];
   readonly hold: Quantity | null;
   readonly heal: readonly Step[];
@@ -169,6 +184,8 @@ interface FaultPlan {
 export declare const POLICY = "lab-deny";
 /** This service account is the account of the clients: their pods name no other account. */
 export declare const CLIENT_ACCOUNT = "default";
+/** This selector gives the ztunnel pods in istio-system: the label `app` of the chart of ztunnel. */
+export declare const ZTUNNEL_PODS = "app=ztunnel";
 /**
  * This function gives the AuthorizationPolicy of the trigger `authorization-policy`. It denies each
  * request to the server pods from the principal of the service account `account`. Istio gives a
@@ -176,6 +193,64 @@ export declare const CLIENT_ACCOUNT = "default";
  * RBAC filter of the server sidecar reads the principal of the peer from its mTLS certificate.
  */
 export declare const policyObject: (account: string) => Readonly<Record<string, unknown>>;
+/** These are the names of the sysctls that the steps of a plan set. Each one is a sysctl of a network namespace. */
+export declare const SYSCTL: {
+  readonly retries: "net.ipv4.tcp_retries2";
+};
+/**
+ * These are the values of those sysctls in a new network namespace of Linux v6.17. A restore puts
+ * them back. `tcp_retries2` is TCP_RETR2 (include/net/tcp.h, line 105, and net/ipv4/tcp_ipv4.c,
+ * line 3604).
+ */
+export declare const NETNS_DEFAULTS: {
+  readonly retries: "15";
+};
+/**
+ * These are the names of the middlebox in the filter table of a replica. One chain takes the
+ * segments from the peers, and one chain takes the segments to them. The table of xt_recent holds
+ * the time of the last segment of each peer.
+ */
+export declare const MIDDLEBOX: {
+  readonly input: "LAB-MBX-IN";
+  readonly output: "LAB-MBX-OUT";
+  readonly table: "LAB-MBX";
+};
+/** This time is between the stop of the calls and the fault of a quiet black hole. The last answers and their ACKs, at most 200 ms late, come in it. */
+export declare const QUIET_LEAD: Quantity;
+/** This time is between the close of the old channel of each client and the fault of a gap with `close`. The stream of the old channel ends in it. */
+export declare const CLOSE_LEAD: Quantity;
+/** This time is how long each client calls on its new channel at the fault of a gap with `close`, before it closes it. */
+export declare const CHANNEL_USE: Quantity;
+/**
+ * This function gives the command of the injector that sets one sysctl of its network namespace.
+ * Only a sysctl of the namespace (`net.`) and a whole number reach it.
+ */
+export declare const sysctlCommand: (key: string, value: string) => string[];
+/**
+ * This function gives the commands of the injector that put a stateful middlebox in its network
+ * namespace, with each peer, on the local port `port`. With `idle` null, they remove it.
+ *
+ * The middlebox forgets a flow of the port after `idle` with no segment. Then it drops each later
+ * segment of it: the silent-drop form of the router of the harness. Conntrack stays as it is. The
+ * local leg from ztunnel to the app has no PING, and istio-cni needs its conntrack entry. The
+ * connmark of that entry sends each answer of the app back to ztunnel (cni/pkg/iptables/iptables.go
+ * of Istio 1.31.3, lines 172–178, 278–284 and 336–347).
+ *
+ * The match recent keeps a table in each network namespace (xt_recent.c of Linux v6.17). Its key is
+ * the address of the peer: the source of a segment from it, or the destination of a segment to it
+ * (lines 251–257). The chain of each direction has these rules, in sequence:
+ * - In the input chain only, a SYN sets the time of its peer and passes: a new flow.
+ * - A segment of a peer with a time in the last `idle` seconds passes, and it sets the time again
+ *   (lines 298–321).
+ * - A segment of a peer with an older time is dropped, and the time stays old. Thus the flow stays forgotten.
+ * - A segment of a peer that the table does not hold sets its time and passes (lines 283–290). That
+ *   is the flow that was open before the set.
+ *
+ * Each rule that passes a segment goes back to INPUT or OUTPUT (RETURN), thus the other rules of the
+ * namespace still act on it. The key is an address, not a flow. Thus a new SYN of a peer also refreshes an
+ * old flow of the peer. A cell of the middlebox opens no new flow in its gap.
+ */
+export declare const middleboxCommands: (peers: readonly string[], idle: Quantity | null, port: number) => string[][];
 /** This function gives the fault plan of a cell. */
 export declare const planOf: (cell: KindCell) => FaultPlan;
 /** This function tells if a trigger stops a pod of the service with its grace: a deletion, an eviction, a scale-in or a rollout. */
@@ -268,8 +343,23 @@ export declare const serverCluster: (namespace: string) => string;
 export declare const EDS_EVERY: Quantity;
 /** This time is the interval of the LDS poll of each server sidecar, the same as the poll of EDS. */
 export declare const LDS_EVERY: Quantity;
-/** This function tells if the watcher follows the log of each ztunnel for a cell: a cell in ambient mode. */
+/**
+ * This function tells if the watcher follows each ztunnel pod and its log for a cell: a cell in the
+ * flavor `ambient`. A cell of sidecar pods in that flavor needs the lines of the ztunnel pods too.
+ * They show the restart of ztunnel, which its pods do not see.
+ */
 export declare const followsZtunnel: (cell: Pick<KindCell, "mesh">) => boolean;
+/**
+ * This function gives the arguments of kubectl that set the environment of ztunnel for a cell, or
+ * remove it (`undo`). The variables go in the sequence of their names.
+ */
+export declare const ztunnelEnvArgs: (env: Readonly<Record<string, string>>, undo: boolean) => string[];
+/**
+ * This function sets the environment of ztunnel for a cell, or removes it (`undo`). Then it waits for
+ * the rollout of the DaemonSet, and until no old ztunnel pod is left. An old pod drains for its
+ * deadline after the rollout replaces it. Thus no run starts beside a ztunnel pod that is going.
+ */
+export declare const setZtunnelEnv: (cluster: Cluster, env: Readonly<Record<string, string>>, undo: boolean) => Promise<void>;
 /** This function tells if the lab polls the LDS updates of the server sidecars of a cell: a policy in a sidecar mesh. */
 export declare const pollsListeners: (cell: Pick<KindCell, "mesh" | "trigger">) => boolean;
 //#endregion
@@ -279,8 +369,14 @@ interface Stamp {
   readonly t0: number;
   readonly t1: number;
 }
-/** These kinds of kernel record stay in the raw record of a run. The lab counts the other kinds and drops them. */
+/**
+ * These kinds of kernel record stay in the raw record of a run. The lab also keeps each change of the
+ * state of a socket of the HBONE port (`isTunnelState`): the tunnels of ztunnel. It counts the other
+ * records and drops them.
+ */
 export declare const KEPT_KERNEL: ReadonlySet<string>;
+/** This function tells if the lab keeps a kernel record of the prober. */
+export declare const keptKernel: (record: Readonly<Record<string, unknown>>) => boolean;
 /** An EDS spec is the cluster that the EDS pollers read, their interval, and each client pod with its node and a PID in it. */
 interface EdsSpec {
   readonly cluster: string;
@@ -347,6 +443,13 @@ interface KindLab {
    * lines as a line `tables`: evidence of which rules matched.
    */
   tables(node: string, pid: number, table: string): Promise<string>;
+  /**
+   * This method runs commands of the injector of the harness in the network namespace of a PID. Each
+   * command is a sysctl or a rule of iptables, in sequence. It stops at the first command that fails. Each word of
+   * a command is plain: letters, digits and `_ . , : = / + -`. The method adds the commands and their
+   * output to the lines as a line `act`, and gives the output.
+   */
+  netns(node: string, pid: number, commands: ReadonlyArray<readonly string[]>): Promise<string>;
   now(): Promise<number>;
   pidOf(node: string, namespace: string, pod: string, container: string): Promise<number>;
   netnsOf(node: string, pid: number): Promise<number>;
@@ -629,5 +732,5 @@ export declare const toolPath: (tool: Tool, key?: string, cache?: string) => str
  */
 export declare const ensureTool: (tool: Tool, key?: string, cache?: string) => Promise<string>;
 //#endregion
-export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, SocketLine, type Stack, type Stamp, type Step, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, type Verdicts, type WindowPrediction, type Workload, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stoppingOf, timeSpread, transitionOf, unitOf, updatesOf, upgrade, upgradeCell };
+export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HBONE_PORT, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, SocketLine, type Stack, type Stamp, type Step, TCP_STATE, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, TunnelLine, type Verdicts, type WindowPrediction, type Workload, ZtunnelPodLine, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, isTunnelState, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stoppingOf, timeSpread, transitionOf, tunnelsOf, unitOf, updatesOf, upgrade, upgradeCell, ztunnelPodsOf, ztunnelReplaced };
 //# sourceMappingURL=index.d.ts.map