@mark1russell7/kind 0.5.0 → 0.7.0

package/README.md

@@ -137,6 +137,13 @@ From 0.5.0, the lab installs the flavor `ambient` too:
 - **The kernel.** istio-cni writes iptables rules in each pod and on each node. Thus `kind.yml` loads their modules first, and an ambient cell waits when one does not load.
 - **The log of ztunnel.** For a cell in the ambient mode, the watcher follows each ztunnel pod in istio-system (`followsZtunnel`). Each line of its log becomes a `ztunnel` line, stamped at its arrival, and each ztunnel pod is a `ztunnel-pod` line. No band reads them.
 
+From 0.7.0, a cell can restart ztunnel, and it can set the environment of ztunnel:
+
+- **The watcher.** It follows the ztunnel pods for each cell in the flavor `ambient`, also for a cell of sidecar pods there. The lines of the ztunnel pods are the evidence of a restart that no pod of the cell sees.
+- **The environment.** `mesh.ztunnel` gives variables of the DaemonSet of ztunnel for one cell, for example `CONNECTION_TERMINATION_DEADLINE`. Before the cell, `setZtunnelEnv` sets them with `kubectl set env` (`ztunnelEnvArgs`). Then it waits for the rollout, and until no old ztunnel pod is left. After the cell, it removes them in the same way.
+- **The reference `restart`.** It is the first deletion timestamp of a ztunnel pod in the run, as the watcher receives it (`restartOf`). A run of the trigger `ztunnel-restart` ends when each deleted ztunnel pod is gone, and each node has a ready new one (`ztunnelReplaced`).
+- **The DaemonSet.** The chart of ztunnel sets `maxSurge` 1. Then the controller of the DaemonSet of Kubernetes v1.37.0 skips a pod with a deletion timestamp. It makes a new pod on that node at once (`daemon_controller.go`, lines 863–924). Thus the new ztunnel runs beside the old one while the old one drains.
+
 A new member of the union `Template` breaks a reader that switches on it. That is a breaking change before 1.0.0, thus the version is 0.3.0. The record format stays `/2`, with the optional field `mesh`. A judge before 0.3.0 cannot read a record of T15.
 
 ## The image
@@ -152,6 +159,8 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
 - **The seams** of the service are a flag file that fails its liveness check, and SIGUSR2. SIGUSR2 blocks its event loop for the time in a file.
 - **The clients** (`client.mjs`) send requests on new connections to the ClusterIP. Each request gives one line with its outcome: ok, 503, refused, reset, timeout or unreachable. From 0.1.3, each line also gives the local port of its connection (`lp`), a refused one's too. A conntrack entry is per port. Thus the requests on the port of a failure show the connection before it, and the pod that it went to.
 - **The gRPC clients** (`client-grpc.mjs`) hold one channel of grpc-js 1.14.6 to the ClusterIP, and send a call of `grpc.health.v1.Health/Check` each interval on it. Each call gives the line of a request of `client.mjs`, with the pod that answered (the header `x-pod`). A call that fails at once names the errno of the last connect: refused, reset or unreachable. A call of grpc-js has no deadline at its defaults. Thus a call with no answer in the timeout is a timeout, and the client cancels it.
+  - From 0.6.0, each socket of the channel also gives a `sock` line at its connect and at its close (`client-lines.ts`). Each line gives the local port of the socket (`lp`). The close line keeps the port of the connect, and it gives the code of the error of the socket (`err`), or null.
+  - From 0.6.0, SIGUSR1 stops the calls of the client, and SIGUSR2 starts them again. Each signal gives a `gap` line. The channel and its socket stay open in the gap. A listener of SIGUSR1 replaces the start of the inspector of Node at that signal.
 
 ## The triggers
 
@@ -172,6 +181,8 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
 | `rollout` | no Redis pod, then a rolling update |
 | `cpu-throttle` | `cpu.max` in the cgroup of the container of the first replica |
 | `authorization-policy` | `kubectl apply` of a DENY AuthorizationPolicy on the server pods for the principal of an account (`account`, the account `default` of the clients when not given), then `kubectl delete` after the hold |
+| `idle` | SIGUSR1 to the gRPC client of each client pod, and SIGUSR2 after the hold: a gap of the calls (the target `clients`). The restore sends SIGUSR2 again |
+| `ztunnel-restart` | `kubectl delete pod` of the ztunnel pod of each node, with its grace (`ZTUNNEL_PODS`). The flow of each client crosses the ztunnel of its own node. The restore waits for the rollout of the DaemonSet |
 
 A cgroup path that is not the scope of one container is refused. A write to the root of the node throttles or freezes each pod of the node.
 
@@ -219,6 +230,17 @@ From 0.4.0, a window can count from the heal (`heal`), the first act that ends t
 - A denial after the window, before the last update after the delete and one more call, is a `tail`. A denied call has no pod, because the sidecar answers it. Thus the judge reads the last update of each server sidecar.
 - `calibrationOf` gives the updates after the apply by node, and the updates after the delete (`heal`).
 
+From 0.6.0, the prediction `ends` is the end of the socket of each client, from the `sock` lines. A client of the run is a client with an answer before the reference.
+
+- With a band, the first close of a socket of each client after the reference must be in the band ± δ. It must be the socket of its last answer. The main value is the middle of these times, in `s`.
+- With no band, no socket of a client can close after the reference. The first request of each client after the heal must be `ok` on the socket of its last answer. The main value is the count of the closes, in `{socket}`.
+- With a window (`window`), the failed requests must also agree with that window. The observation lasts until each socket can end, and until each request of the window ended.
+- The facts of a run give the first close of each client after the reference (`closes`).
+
+A new member of the union `TriggerKind`, `ztunnel-restart`, and a new member of the union `Reference`, `restart`, break a reader that switches on them. Each is a breaking change before 1.0.0, thus the version is 0.7.0. The record format stays `/2`, with the optional field `mesh.ztunnel`.
+
+A new member of the union `Prediction`, `ends`, and a new member of the union `TriggerKind`, `idle`, break a reader that switches on them. The union `Target` has the new member `clients` too. Each is a breaking change before 1.0.0, thus the version is 0.6.0. The record format stays `/2`, with the optional field `closes`: the lines `sock` and `gap` are new lines of a run.
+
 A new member of the union `Target`, `servers`, breaks a reader that switches on it. `KindLab` also has a new method, `tables`, which each implementation must have. Each is a breaking change before 1.0.0, thus the version is 0.5.0. The record format stays `/2`: the lines `tables`, `ztunnel` and `ztunnel-pod` are new lines of a run.
 
 A new member of the union `TriggerKind` breaks a reader that switches on it. That is a breaking change before 1.0.0, thus the version is 0.4.0. The record format stays `/2`, with the optional fields `account`, `outcomes`, `updates` and `healUpdates`, and the reference `heal`.

package/build/index.d.ts

@@ -1,4 +1,4 @@
-import { $ as updatesOf, $t as Replica, A as TriggerV1, At as transitionOf, B as RunCheck, Bt as showBand, C as KindCellResult, Ct as rulesOf, D as PredictionV1, Dt as sliceEmptyOf, E as LoadSample, Et as sliceDropOf, F as KIND_NOISE, Ft as REDIS, G as judgeKind, Gt as KubeletProbe, H as checkRun, Ht as Declaration, I as KindDecision, It as SERVER, J as propagationOf, Jt as MeshMode, K as observationAfter, Kt as Mesh, L as KindVerdict, Lt as bandOf, M as upgrade, Mt as HEALTH_PORT, N as upgradeCell, Nt as LOG_LABEL, O as RunCheckV1, Ot as slicesOf, P as Deleted, Pt as PORT, Q as unitOf, Qt as Reference, R as LAST_SLACK, Rt as seconds, S as KindCellRecordV1, St as restartsAt, T as KindVerdictV1, Tt as signalsOf, U as deletionsOf, Ut as KindCell, V as RunFacts, Vt as Band, W as factsOf, Wt as KindRun, X as referenceOf, Xt as Prediction, Y as propagationSource, Yt as PodSettings, Z as reusesReset, Zt as RedisDeclaration, _ as ClusterManifest, _n as nodeNames, _t as outcomesOf, a as STAGES, an as Workload, at as Request, b as FORMAT_V1, bt as recordsOf, c as readRecord, cn as KindScenario, ct as backendsAt, d as calibrationOf, dn as NodeImage, dt as deletionOf, en as ResetReuse, et as Endpoint, f as cellsTable, fn as cidrOverlap, ft as dropOf, g as BandV1, gn as kubeProxyPatch, gt as kernelCounts, h as timeSpread, hn as kindConfig, ht as intakeOf, i as quantileOf, in as TriggerKind, it as RecordLine, j as judge, jt as CLIENT, k as RunFactsV1, kt as stoppingOf, l as Judged, ln as KubeProxyMode, lt as byClients, m as predictionText, mn as goDuration, mt as failuresOf, n as medianInterval, nn as Template, nt as PodLine, o as kindCellRecordSchema, on as KUBE_PROXY_MODES, ot as RulesLine, p as medianText, pn as cidrRange, pt as emptyOf, q as observationEnd, qt as MeshFlavor, r as middle, rn as Trigger, rt as PropagationSource, s as kindCellRecordV1Schema, sn as KindConfig, st as SliceLine, t as RELATIVE, tn as Stack, tt as Line, u as TimeSpread, un as NODE_IMAGES, ut as byNode, v as EnvironmentManifest, vn as scenario, vt as podsAt, w as KindCellV1, wt as signalOf, x as KindCellRecord, xt as requestsOf, y as FORMAT, yn as scenarioProblems, yt as podsOf, z as Propagated, zt as secondsOf } from "./judge-BS9pk96o.js";
+import { $ as updatesOf, $t as MeshFlavor, A as TriggerV1, At as sliceEmptyOf, B as RunCheck, Bt as PORT, C as KindCellResult, Cn as kubeProxyPatch, Ct as requestsOf, D as PredictionV1, Dt as signalOf, E as LoadSample, En as scenarioProblems, Et as rulesOf, F as KIND_NOISE, Ft as ztunnelPodsOf, G as judgeKind, Gt as secondsOf, H as checkRun, Ht as SERVER, I as KindDecision, It as ztunnelReplaced, J as propagationOf, Jt as Declaration, K as observationAfter, Kt as showBand, L as KindVerdict, Lt as CLIENT, M as upgrade, Mt as socketsOf, N as upgradeCell, Nt as stoppingOf, O as RunCheckV1, Ot as signalsOf, P as Deleted, Pt as transitionOf, Q as unitOf, Qt as Mesh, R as LAST_SLACK, Rt as HEALTH_PORT, S as KindCellRecordV1, Sn as kindConfig, St as recordsOf, T as KindVerdictV1, Tn as scenario, Tt as restartsAt, U as deletionsOf, Ut as bandOf, V as RunFacts, Vt as REDIS, W as factsOf, Wt as seconds, X as referenceOf, Xt as KindRun, Y as propagationSource, Yt as KindCell, Z as reusesReset, Zt as KubeletProbe, _ as ClusterManifest, _n as NODE_IMAGES, _t as intakeOf, a as STAGES, an as Replica, at as Request, b as FORMAT_V1, bn as cidrRange, bt as podsAt, c as readRecord, cn as Template, ct as SocketLine, d as calibrationOf, dn as WindowPrediction, dt as byClients, en as MeshMode, et as Endpoint, f as cellsTable, fn as Workload, ft as byNode, g as BandV1, gn as KubeProxyMode, gt as failuresOf, h as timeSpread, hn as KindScenario, ht as emptyOf, i as quantileOf, in as Reference, it as RecordLine, j as judge, jt as slicesOf, k as RunFactsV1, kt as sliceDropOf, l as Judged, ln as Trigger, lt as ZtunnelPodLine, m as predictionText, mn as KindConfig, mt as dropOf, n as medianInterval, nn as Prediction, nt as PodLine, o as kindCellRecordSchema, on as ResetReuse, ot as RulesLine, p as medianText, pn as KUBE_PROXY_MODES, pt as deletionOf, q as observationEnd, qt as Band, r as middle, rn as RedisDeclaration, rt as PropagationSource, s as kindCellRecordV1Schema, sn as Stack, st as SliceLine, t as RELATIVE, tn as PodSettings, tt as Line, u as TimeSpread, un as TriggerKind, ut as backendsAt, v as EnvironmentManifest, vn as NodeImage, vt as kernelCounts, w as KindCellV1, wn as nodeNames, wt as restartOf, x as KindCellRecord, xn as goDuration, xt as podsOf, y as FORMAT, yn as cidrOverlap, yt as outcomesOf, z as Propagated, zt as LOG_LABEL } from "./judge-B1nSdbS0.js";
 import { Quantity } from "@mark1russell7/quantity";
 import { CheckFn, Finding, HealthServer, Poll, Rule, Scenario, ServiceBuilder, ServiceDescription, Stage, Timing, TransportImpl, Verdict } from "@mark1russell7/service";
 import { RedisClient } from "@mark1russell7/service/redis";
@@ -114,8 +114,9 @@ export declare const hostsOf: (text: string, cluster: string) => ClusterHosts;
  * - `servers` is each replica of the service. Only the injector takes it: a rule in the network
  *   namespace of each replica, with each client pod as a peer. A REJECT resets only the sender of
  *   the segment that it matches, thus the rule goes where the segments of the clients arrive.
+ * - `clients` is the process of each client pod. Only a signal takes it: the gap of the trigger `idle`.
  */
-type Target = "server" | "redis" | "kube-proxy" | "servers";
+type Target = "server" | "redis" | "kube-proxy" | "servers" | "clients";
 /** A step is one act of a fault, in the sequence of the plan. */
 type Step = {
   readonly do: "kubectl";
@@ -124,7 +125,7 @@ type Step = {
 } | {
   readonly do: "signal";
   readonly target: Target;
-  readonly signal: "SIGKILL" | "SIGSTOP" | "SIGCONT";
+  readonly signal: "SIGKILL" | "SIGSTOP" | "SIGCONT" | "SIGUSR1" | "SIGUSR2";
 } | {
   readonly do: "pause";
   readonly target: Target;
@@ -168,6 +169,8 @@ interface FaultPlan {
 export declare const POLICY = "lab-deny";
 /** This service account is the account of the clients: their pods name no other account. */
 export declare const CLIENT_ACCOUNT = "default";
+/** This selector gives the ztunnel pods in istio-system: the label `app` of the chart of ztunnel. */
+export declare const ZTUNNEL_PODS = "app=ztunnel";
 /**
  * This function gives the AuthorizationPolicy of the trigger `authorization-policy`. It denies each
  * request to the server pods from the principal of the service account `account`. Istio gives a
@@ -267,8 +270,23 @@ export declare const serverCluster: (namespace: string) => string;
 export declare const EDS_EVERY: Quantity;
 /** This time is the interval of the LDS poll of each server sidecar, the same as the poll of EDS. */
 export declare const LDS_EVERY: Quantity;
-/** This function tells if the watcher follows the log of each ztunnel for a cell: a cell in ambient mode. */
+/**
+ * This function tells if the watcher follows each ztunnel pod and its log for a cell: a cell in the
+ * flavor `ambient`. A cell of sidecar pods in that flavor needs the lines of the ztunnel pods too.
+ * They show the restart of ztunnel, which its pods do not see.
+ */
 export declare const followsZtunnel: (cell: Pick<KindCell, "mesh">) => boolean;
+/**
+ * This function gives the arguments of kubectl that set the environment of ztunnel for a cell, or
+ * remove it (`undo`). The variables go in the sequence of their names.
+ */
+export declare const ztunnelEnvArgs: (env: Readonly<Record<string, string>>, undo: boolean) => string[];
+/**
+ * This function sets the environment of ztunnel for a cell, or removes it (`undo`). Then it waits for
+ * the rollout of the DaemonSet, and until no old ztunnel pod is left. An old pod drains for its
+ * deadline after the rollout replaces it. Thus no run starts beside a ztunnel pod that is going.
+ */
+export declare const setZtunnelEnv: (cluster: Cluster, env: Readonly<Record<string, string>>, undo: boolean) => Promise<void>;
 /** This function tells if the lab polls the LDS updates of the server sidecars of a cell: a policy in a sidecar mesh. */
 export declare const pollsListeners: (cell: Pick<KindCell, "mesh" | "trigger">) => boolean;
 //#endregion
@@ -628,5 +646,5 @@ export declare const toolPath: (tool: Tool, key?: string, cache?: string) => str
  */
 export declare const ensureTool: (tool: Tool, key?: string, cache?: string) => Promise<string>;
 //#endregion
-export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, type Stack, type Stamp, type Step, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, type Verdicts, type Workload, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stoppingOf, timeSpread, transitionOf, unitOf, updatesOf, upgrade, upgradeCell };
+export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, SocketLine, type Stack, type Stamp, type Step, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, type Verdicts, type WindowPrediction, type Workload, ZtunnelPodLine, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stoppingOf, timeSpread, transitionOf, unitOf, updatesOf, upgrade, upgradeCell, ztunnelPodsOf, ztunnelReplaced };
 //# sourceMappingURL=index.d.ts.map

package/build/index.js

@@ -1,4 +1,4 @@
-import { $ as slicesOf, A as medianInterval, B as intakeOf, C as propagationOf, D as unitOf, E as reusesReset, F as byNode, G as recordsOf, H as outcomesOf, I as deletionOf, J as rulesOf, K as requestsOf, L as dropOf, M as quantileOf, N as backendsAt, O as updatesOf, P as byClients, Q as sliceEmptyOf, R as emptyOf, S as observationEnd, T as referenceOf, U as podsAt, V as kernelCounts, W as podsOf, X as signalsOf, Y as signalOf, Z as sliceDropOf, _ as checkRun, a as calibrationOf, at as PORT, b as judgeKind, c as predictionText, ct as bandOf, d as FORMAT_V1, dt as showBand, et as stoppingOf, f as judge, g as LAST_SLACK, h as KIND_NOISE, i as readRecord, it as LOG_LABEL, j as middle, k as RELATIVE, l as timeSpread, lt as seconds, m as upgradeCell, n as kindCellRecordSchema, nt as CLIENT, o as cellsTable, ot as REDIS, p as upgrade, q as restartsAt, r as kindCellRecordV1Schema, rt as HEALTH_PORT, s as medianText, st as SERVER, t as STAGES, tt as transitionOf, u as FORMAT, ut as secondsOf, v as deletionsOf, w as propagationSource, x as observationAfter, y as factsOf, z as failuresOf } from "./judge-Bkbs7VVG.js";
+import { $ as sliceEmptyOf, A as medianInterval, B as intakeOf, C as propagationOf, D as unitOf, E as reusesReset, F as byNode, G as recordsOf, H as outcomesOf, I as deletionOf, J as restartsAt, K as requestsOf, L as dropOf, M as quantileOf, N as backendsAt, O as updatesOf, P as byClients, Q as sliceDropOf, R as emptyOf, S as observationEnd, T as referenceOf, U as podsAt, V as kernelCounts, W as podsOf, X as signalOf, Y as rulesOf, Z as signalsOf, _ as checkRun, a as calibrationOf, at as ztunnelReplaced, b as judgeKind, c as predictionText, ct as LOG_LABEL, d as FORMAT_V1, dt as SERVER, et as slicesOf, f as judge, ft as bandOf, g as LAST_SLACK, h as KIND_NOISE, ht as showBand, i as readRecord, it as ztunnelPodsOf, j as middle, k as RELATIVE, l as timeSpread, lt as PORT, m as upgradeCell, mt as secondsOf, n as kindCellRecordSchema, nt as stoppingOf, o as cellsTable, ot as CLIENT, p as upgrade, pt as seconds, q as restartOf, r as kindCellRecordV1Schema, rt as transitionOf, s as medianText, st as HEALTH_PORT, t as STAGES, tt as socketsOf, u as FORMAT, ut as REDIS, v as deletionsOf, w as propagationSource, x as observationAfter, y as factsOf, z as failuresOf } from "./judge-CmiF-Fcs.js";
 import { valueIn } from "@mark1russell7/quantity";
 import { execFile, execFileSync } from "node:child_process";
 import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, realpathSync, renameSync, rmSync, writeFileSync } from "node:fs";
@@ -637,6 +637,8 @@ const DELETION_COST = "controller.kubernetes.io/pod-deletion-cost";
 const POLICY = "lab-deny";
 /** This service account is the account of the clients: their pods name no other account. */
 const CLIENT_ACCOUNT = "default";
+/** This selector gives the ztunnel pods in istio-system: the label `app` of the chart of ztunnel. */
+const ZTUNNEL_PODS = "app=ztunnel";
 /**
 * This function gives the AuthorizationPolicy of the trigger `authorization-policy`. It denies each
 * request to the server pods from the principal of the service account `account`. Istio gives a
@@ -950,6 +952,51 @@ const planOf = (cell) => {
 				]
 			}]
 		};
+		case "idle": return {
+			inject: [{
+				do: "signal",
+				target: "clients",
+				signal: "SIGUSR1"
+			}],
+			hold: needs(trigger.hold, "a hold, the time of the gap"),
+			heal: [{
+				do: "signal",
+				target: "clients",
+				signal: "SIGUSR2"
+			}],
+			restore: [{
+				do: "signal",
+				target: "clients",
+				signal: "SIGUSR2"
+			}]
+		};
+		case "ztunnel-restart": return {
+			inject: [{
+				do: "kubectl",
+				args: [
+					"-n",
+					"istio-system",
+					"delete",
+					"pod",
+					"-l",
+					ZTUNNEL_PODS,
+					"--wait=false"
+				]
+			}],
+			hold: null,
+			heal: [],
+			restore: [{
+				do: "kubectl",
+				args: [
+					"-n",
+					"istio-system",
+					"rollout",
+					"status",
+					"daemonset/ztunnel",
+					"--timeout=300s"
+				]
+			}]
+		};
 	}
 };
 /** This function tells if a trigger stops a pod of the service with its grace: a deletion, an eviction, a scale-in or a rollout. */
@@ -986,7 +1033,8 @@ const FILES = [
 	["conntrack.ts", "src/conntrack.ts"],
 	["drops.ts", "src/drops.ts"],
 	["eds.ts", "src/eds.ts"],
-	["lds.ts", "src/lds.ts"]
+	["lds.ts", "src/lds.ts"],
+	["client-lines.ts", "src/client-lines.ts"]
 ];
 const shell = process.platform === "win32";
 /** This function gives each file of a staged context, with its path in the context, in a stable sequence. */
@@ -1277,8 +1325,55 @@ const LDS_EVERY = {
 	value: 50,
 	unit: "ms"
 };
-/** This function tells if the watcher follows the log of each ztunnel for a cell: a cell in ambient mode. */
-const followsZtunnel = (cell) => cell.mesh?.mode === "ambient";
+/**
+* This function tells if the watcher follows each ztunnel pod and its log for a cell: a cell in the
+* flavor `ambient`. A cell of sidecar pods in that flavor needs the lines of the ztunnel pods too.
+* They show the restart of ztunnel, which its pods do not see.
+*/
+const followsZtunnel = (cell) => cell.mesh?.flavor === "ambient";
+/**
+* This function gives the arguments of kubectl that set the environment of ztunnel for a cell, or
+* remove it (`undo`). The variables go in the sequence of their names.
+*/
+const ztunnelEnvArgs = (env, undo) => [
+	"-n",
+	"istio-system",
+	"set",
+	"env",
+	"daemonset/ztunnel",
+	...Object.keys(env).sort().map((key) => undo ? `${key}-` : `${key}=${env[key]}`)
+];
+/**
+* This function sets the environment of ztunnel for a cell, or removes it (`undo`). Then it waits for
+* the rollout of the DaemonSet, and until no old ztunnel pod is left. An old pod drains for its
+* deadline after the rollout replaces it. Thus no run starts beside a ztunnel pod that is going.
+*/
+const setZtunnelEnv = async (cluster, env, undo) => {
+	await cluster.kubectl(ztunnelEnvArgs(env, undo));
+	await cluster.kubectl([
+		"-n",
+		"istio-system",
+		"rollout",
+		"status",
+		"daemonset/ztunnel",
+		"--timeout=300s"
+	]);
+	const until = Date.now() + 12e4;
+	for (;;) {
+		if (JSON.parse(await cluster.kubectl([
+			"-n",
+			"istio-system",
+			"get",
+			"pods",
+			"-l",
+			"app=ztunnel",
+			"-o",
+			"json"
+		])).items.every((item) => item.metadata.deletionTimestamp === void 0)) return;
+		if (Date.now() > until) throw new Error("an old ztunnel pod is still going 120 s after the rollout");
+		await new Promise((resolve) => setTimeout(resolve, 2e3));
+	}
+};
 /** This function tells if the lab polls the LDS updates of the server sidecars of a cell: a policy in a sidecar mesh. */
 const pollsListeners = (cell) => cell.mesh?.mode === "sidecar" && cell.trigger.kind === "authorization-policy";
 //#endregion
@@ -2591,6 +2686,38 @@ const injectServers = async (ctx, action, replicas) => {
 		t1: Math.max(...all.map((stamp) => stamp.t1))
 	};
 };
+/**
+* This function sends a signal to the process of each client pod: the target `clients`. The nodes go
+* at the same time. SIGUSR1 finds the PID of each client, and SIGUSR2 uses it again, thus the heal of
+* a gap comes soon after its hold. When that PID fails, the client started again, and the lab finds
+* its PID. The stamp is the first act and the last end.
+*/
+const signalClients = async (ctx, signal) => {
+	const clients = [...ctx.tracker.pods.values()].filter((pod) => pod.labels["app"] === "client" && !pod.del && pod.node !== null);
+	if (clients.length === 0) throw new Error(`the signal ${signal} needs clients`);
+	const stamps = await Promise.all(clients.map(async (pod) => {
+		const node = pod.node;
+		const known = signal === "SIGUSR2" ? ctx.pids.get(pod.name) : void 0;
+		if (known !== void 0) try {
+			return await ctx.lab.act(node, {
+				op: "signal",
+				pid: known,
+				signal
+			});
+		} catch {}
+		const pid = await ctx.lab.pidOf(node, ctx.namespace, pod.name, CLIENT);
+		ctx.pids.set(pod.name, pid);
+		return ctx.lab.act(node, {
+			op: "signal",
+			pid,
+			signal
+		});
+	}));
+	return {
+		t0: Math.min(...stamps.map((stamp) => stamp.t0)),
+		t1: Math.max(...stamps.map((stamp) => stamp.t1))
+	};
+};
 /** This function does one step of a fault plan. It gives the stamp of its act, or null for an act without a stamp. */
 const doStep = async (ctx, step, replicas, background) => {
 	const { lab, cluster, namespace } = ctx;
@@ -2598,6 +2725,7 @@ const doStep = async (ctx, step, replicas, background) => {
 	const fill = (args) => args.map((arg) => arg.replaceAll("{namespace}", namespace).replaceAll(POD, target.pod).replaceAll(NODE, target.node).replaceAll(VERSION, String(ctx.version)));
 	const where = async (which) => {
 		if (which === "servers") throw new Error("the target servers is a target of the injector only");
+		if (which === "clients") throw new Error("the target clients is a target of a signal only");
 		if (which === "server") return {
 			node: target.node,
 			pid: await lab.pidOf(target.node, namespace, target.pod, "app"),
@@ -2649,6 +2777,7 @@ const doStep = async (ctx, step, replicas, background) => {
 			};
 		}
 		case "signal": {
+			if (step.target === "clients") return signalClients(ctx, step.signal);
 			const at = await where(step.target);
 			return lab.act(at.node, {
 				op: "signal",
@@ -2738,6 +2867,7 @@ const over = (ctx, run, lines) => {
 	].includes(kind) && (ctx.cell.template === "T07" || ctx.cell.template === "T15")) return kind === "scale-in" || kind === "node-drain" ? run.replicas.some((replica) => !ctx.tracker.pods.has(replica.pod)) : !ctx.tracker.pods.has(run.target.pod);
 	if (kind === "rolling-update" || kind === "rollout") return run.replicas.every((replica) => !ctx.tracker.pods.has(replica.pod));
 	if (kind === "app-stall") return lines.some((line) => line.ev === "stall" && line["phase"] === "end");
+	if (kind === "ztunnel-restart") return ztunnelReplaced(lines, ctx.cluster.nodes);
 	return true;
 };
 /** This function gives the replica that the trigger stopped. For a scale-in or a drain, the controller chooses it. */
@@ -2997,7 +3127,9 @@ const runKindCells = async (cells, options) => {
 					const tracker = new Tracker();
 					const off = lab.listen(tracker.see);
 					const loads = [];
+					const ztunnelEnv = cell.mesh?.ztunnel;
 					try {
+						if (ztunnelEnv !== void 0) await setZtunnelEnv(cluster, ztunnelEnv, false);
 						const clusterIp = await deploy(lab, cluster, cell, namespace, images);
 						for (const name of [...lab.own(), loader.getName().replace(/^\//, "")]) ours.add(name);
 						const ctx = {
@@ -3059,6 +3191,7 @@ const runKindCells = async (cells, options) => {
 							"--wait=true",
 							"--timeout=300s"
 						]).catch((error) => options.log(`  ${String(error)}`));
+						if (ztunnelEnv !== void 0) await setZtunnelEnv(cluster, ztunnelEnv, true).catch((error) => options.log(`  the undo of the environment of ztunnel failed: ${String(error)}`));
 					}
 				}
 			} finally {
@@ -3095,11 +3228,14 @@ const estimate = (cell, environments = 2) => {
 		"app-stall": hold + 40,
 		"cpu-throttle": hold + 40,
 		rollout: hold + 50,
-		"authorization-policy": hold + 20
+		"authorization-policy": hold + 20,
+		idle: hold + 5,
+		"ztunnel-restart": 40
 	};
 	const after = observationAfter(cell);
 	const observed = after === null ? 0 : hold + valueIn(after, "s");
-	const each = 25 + Math.max(observed, trigger[cell.trigger.kind] ?? (cell.template === "T18" ? hold + 35 : 15 + valueIn(cell.pod.preStop, "s")));
+	const closes = cell.prediction.kind === "ends" && cell.prediction.band !== null ? valueIn(cell.prediction.band.hi, "s") + 3 : 0;
+	const each = 25 + Math.max(observed, closes, trigger[cell.trigger.kind] ?? (cell.template === "T18" ? hold + 35 : 15 + valueIn(cell.pod.preStop, "s")));
 	return (cell.mesh === void 0 ? 60 : 180) * environments + (cell.runs + environments) * each;
 };
 /**
@@ -3131,6 +3267,6 @@ const shardOf = (cells, index, count) => {
 	return out;
 };
 //#endregion
-export { ACCESS_LOG_FORMAT, CLIENT, CLIENT_ACCOUNT, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LDS_EVERY, LDS_FILTER, LOG_LABEL, MESH_IMAGES, MESH_WORKLOADS, NODE_IMAGES, O04, O09, O10, O16, POLICY, PORT, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, WATCHDOG, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, followsZtunnel, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, istioOperator, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, lintCell, listenerCountsOf, localPin, medianInterval, medianText, meshKey, middle, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, policyObject, pollsListeners, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stackOf, stageLabImage, stallRemovalBand, stoppingOf, tarEntry, timeSpread, toolPath, transitionOf, unitOf, updatesOf, upgrade, upgradeCell, verdictsOf, writeResult };
+export { ACCESS_LOG_FORMAT, CLIENT, CLIENT_ACCOUNT, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LDS_EVERY, LDS_FILTER, LOG_LABEL, MESH_IMAGES, MESH_WORKLOADS, NODE_IMAGES, O04, O09, O10, O16, POLICY, PORT, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, WATCHDOG, ZTUNNEL_PODS, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, followsZtunnel, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, istioOperator, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, lintCell, listenerCountsOf, localPin, medianInterval, medianText, meshKey, middle, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, policyObject, pollsListeners, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartOf, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, setZtunnelEnv, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stackOf, stageLabImage, stallRemovalBand, stoppingOf, tarEntry, timeSpread, toolPath, transitionOf, unitOf, updatesOf, upgrade, upgradeCell, verdictsOf, writeResult, ztunnelEnvArgs, ztunnelPodsOf, ztunnelReplaced };
 
 //# sourceMappingURL=index.js.map