@mark1russell7/kind 0.8.2 → 0.10.0

package/README.md

@@ -184,6 +184,7 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
 | `authorization-policy` | `kubectl apply` of a DENY AuthorizationPolicy on the server pods for the principal of an account (`account`, the account `default` of the clients when not given), then `kubectl delete` after the hold |
 | `idle` | SIGUSR1 to the gRPC client of each client pod, and SIGUSR2 after the hold: a gap of the calls (the target `clients`). The restore sends SIGUSR2 again. With `middlebox`, a stateful middlebox in each replica from before the gap to the restore. With `close`, a new channel at the fault for 5 s: SIGHUP before the fault, then SIGUSR2, a wait and SIGHUP |
 | `ztunnel-restart` | `kubectl delete pod` of the ztunnel pod of each node, with its grace (`ZTUNNEL_PODS`). The flow of each client crosses the ztunnel of its own node. The restore waits for the rollout of the DaemonSet |
+| `certificate-expiry` | No act on a connection: the calls of each client go on for the hold. At the fault and at the heal, istioctl reads the leaf certificates of the ztunnel pod of each node (`ztunnelCertificates`). A line `certificates` keeps their validity, the evidence of their lifetime |
 
 A cgroup path that is not the scope of one container is refused. A write to the root of the node throttles or freezes each pod of the node.
 
@@ -201,6 +202,7 @@ The plan's `event:clock-jump` has no trigger. A step of CLOCK_REALTIME in the VM
 `predict.ts` gives each band from the declaration, the pod settings and the calibration of the propagation:
 
 - **A deletion.** The listener of a leaving pod stays open for its preStop sleep and the hold of `fail readiness`. A connection fails when the rules of its node select the pod after that. `rules.ts` gives O04 with the hold as data, and `lint()` decides it.
+- **The way that a plan stops a pod.** `stopOf(cell)` reads it from the trigger: a deletion, a liveness restart, a crash, or null when the plan stops no pod. `lintCell()` gives each cell only the drain rules that guard its way (`lintStop()` of the service). A plan that stops no pod gets each drain rule. A crash gets no drain rule, and its finding of `stop` depends.
 - **A pause of kube-proxy** keeps the rules of one node for its hold. The window on that node ends one propagation after the pause. A SYN_SENT entry of an earlier run lives up to 122 s, thus the window also opens up to 122 s before the intake. `predictionText` writes each end of a window with its band, for example `the intake + [-122, 4.605] s`.
 - **A reset from a leaving pod** leaves the conntrack entry of its flow in CLOSE for 10 s, with the NAT to that pod. From 0.1.6, a window can give that life (`afterReset`). Then a request outside the window can fail with the outcome of the rule (`reusesReset`). The condition is a reset of the same client on the same port, up to that life before the request. The catalog gives it to a deletion without a hold, in the modes with the NAT in conntrack.
 - **A fault of a shared dependency** trips the readiness latch of each replica (`failSpan`). Then the kubelet needs its threshold of probes, and the rules need one propagation.
@@ -252,6 +254,8 @@ From 0.8.0, a prediction `ends` can be the end of the tunnel of each client (`of
 - The first request of each client after the heal must be `ok`, on a new tunnel. The main value is the middle of the ends, in `s`.
 - From 0.8.1, `ends` has the optional band `answer`. A change to CLOSE_WAIT in that band ± δ also agrees: the peer answers the close of the client, and its FIN comes first. Before the band, the peer closed first.
 
+A new member of the union `TriggerKind`, `certificate-expiry`, and a new member of the union `Step`, `certificates`, break a reader that switches on them. `KindLab` also has a new method, `record`. Each is a breaking change before 1.0.0, thus the version is 0.9.0. The record format stays `/2`: the lines `certificates` are new lines of a run.
+
 In 0.8.1, the middlebox uses the match recent, not the sysctls of conntrack, and `ends` has the optional field `answer`. `SYSCTL` and `NETNS_DEFAULTS` keep only `tcp_retries2`. These fields came in 0.8.0, which was not published. The fix and the optional field are a patch, thus the version is 0.8.1.
 
 New members of the union `Step` (`sysctl` and `middlebox`) and of the signals of a step (`SIGHUP`) break a reader that switches on them. `KindLab` also has a new method, `netns`. Each is a breaking change before 1.0.0, thus the version is 0.8.0. The record format stays `/2`. A trigger has the optional fields `middlebox`, `retries`, `quiet` and `close`, and `ends` has the optional field `of`. The lines `kernel.state` are new lines of a run.

package/build/index.d.ts

@@ -1,6 +1,6 @@
-import { $ as updatesOf, $t as Declaration, A as TriggerV1, An as scenario, At as rulesOf, B as RunCheck, Bt as ztunnelPodsOf, C as KindCellResult, Cn as NodeImage, Ct as outcomesOf, D as PredictionV1, Dn as kindConfig, Dt as requestsOf, E as LoadSample, En as goDuration, Et as recordsOf, F as KIND_NOISE, Ft as slicesOf, G as judgeKind, Gt as PORT, H as checkRun, Ht as CLIENT, I as KindDecision, It as socketsOf, J as propagationOf, Jt as bandOf, K as observationAfter, Kt as REDIS, L as KindVerdict, Lt as stoppingOf, M as upgrade, Mt as signalsOf, N as upgradeCell, Nt as sliceDropOf, O as RunCheckV1, On as kubeProxyPatch, Ot as restartOf, P as Deleted, Pt as sliceEmptyOf, Q as unitOf, Qt as Band, R as LAST_SLACK, Rt as transitionOf, S as KindCellRecordV1, Sn as NODE_IMAGES, St as kernelCounts, T as KindVerdictV1, Tn as cidrRange, Tt as podsOf, U as deletionsOf, Ut as HEALTH_PORT, V as RunFacts, Vt as ztunnelReplaced, W as factsOf, Wt as LOG_LABEL, X as referenceOf, Xt as secondsOf, Y as propagationSource, Yt as seconds, Z as reusesReset, Zt as showBand, _ as ClusterManifest, _n as Workload, _t as dropOf, a as STAGES, an as MeshMode, at as RecordLine, b as FORMAT_V1, bn as KindScenario, bt as intakeOf, c as readRecord, cn as RedisDeclaration, ct as SliceLine, d as calibrationOf, dn as ResetReuse, dt as TunnelLine, en as KindCell, et as Endpoint, f as cellsTable, fn as Stack, ft as ZtunnelPodLine, g as BandV1, gn as WindowPrediction, gt as deletionOf, h as timeSpread, hn as TriggerKind, ht as byNode, i as quantileOf, in as MeshFlavor, it as PropagationSource, j as judge, jn as scenarioProblems, jt as signalOf, k as RunFactsV1, kn as nodeNames, kt as restartsAt, l as Judged, ln as Reference, lt as SocketLine, m as predictionText, mn as Trigger, mt as byClients, n as medianInterval, nn as KubeletProbe, nt as Line, o as kindCellRecordSchema, on as PodSettings, ot as Request, p as medianText, pn as Template, pt as backendsAt, q as observationEnd, qt as SERVER, r as middle, rn as Mesh, rt as PodLine, s as kindCellRecordV1Schema, sn as Prediction, st as RulesLine, t as RELATIVE, tn as KindRun, tt as HBONE_PORT, u as TimeSpread, un as Replica, ut as TCP_STATE, v as EnvironmentManifest, vn as KUBE_PROXY_MODES, vt as emptyOf, w as KindCellV1, wn as cidrOverlap, wt as podsAt, x as KindCellRecord, xn as KubeProxyMode, xt as isTunnelState, y as FORMAT, yn as KindConfig, yt as failuresOf, z as Propagated, zt as tunnelsOf } from "./judge-gpC49mVw.js";
+import { $ as updatesOf, $t as Declaration, A as TriggerV1, An as scenario, At as rulesOf, B as RunCheck, Bt as ztunnelPodsOf, C as KindCellResult, Cn as NodeImage, Ct as outcomesOf, D as PredictionV1, Dn as kindConfig, Dt as requestsOf, E as LoadSample, En as goDuration, Et as recordsOf, F as KIND_NOISE, Ft as slicesOf, G as judgeKind, Gt as PORT, H as checkRun, Ht as CLIENT, I as KindDecision, It as socketsOf, J as propagationOf, Jt as bandOf, K as observationAfter, Kt as REDIS, L as KindVerdict, Lt as stoppingOf, M as upgrade, Mt as signalsOf, N as upgradeCell, Nt as sliceDropOf, O as RunCheckV1, On as kubeProxyPatch, Ot as restartOf, P as Deleted, Pt as sliceEmptyOf, Q as unitOf, Qt as Band, R as LAST_SLACK, Rt as transitionOf, S as KindCellRecordV1, Sn as NODE_IMAGES, St as kernelCounts, T as KindVerdictV1, Tn as cidrRange, Tt as podsOf, U as deletionsOf, Ut as HEALTH_PORT, V as RunFacts, Vt as ztunnelReplaced, W as factsOf, Wt as LOG_LABEL, X as referenceOf, Xt as secondsOf, Y as propagationSource, Yt as seconds, Z as reusesReset, Zt as showBand, _ as ClusterManifest, _n as Workload, _t as dropOf, a as STAGES, an as MeshMode, at as RecordLine, b as FORMAT_V1, bn as KindScenario, bt as intakeOf, c as readRecord, cn as RedisDeclaration, ct as SliceLine, d as calibrationOf, dn as ResetReuse, dt as TunnelLine, en as KindCell, et as Endpoint, f as cellsTable, fn as Stack, ft as ZtunnelPodLine, g as BandV1, gn as WindowPrediction, gt as deletionOf, h as timeSpread, hn as TriggerKind, ht as byNode, i as quantileOf, in as MeshFlavor, it as PropagationSource, j as judge, jn as scenarioProblems, jt as signalOf, k as RunFactsV1, kn as nodeNames, kt as restartsAt, l as Judged, ln as Reference, lt as SocketLine, m as predictionText, mn as Trigger, mt as byClients, n as medianInterval, nn as KubeletProbe, nt as Line, o as kindCellRecordSchema, on as PodSettings, ot as Request, p as medianText, pn as Template, pt as backendsAt, q as observationEnd, qt as SERVER, r as middle, rn as Mesh, rt as PodLine, s as kindCellRecordV1Schema, sn as Prediction, st as RulesLine, t as RELATIVE, tn as KindRun, tt as HBONE_PORT, u as TimeSpread, un as Replica, ut as TCP_STATE, v as EnvironmentManifest, vn as KUBE_PROXY_MODES, vt as emptyOf, w as KindCellV1, wn as cidrOverlap, wt as podsAt, x as KindCellRecord, xn as KubeProxyMode, xt as isTunnelState, y as FORMAT, yn as KindConfig, yt as failuresOf, z as Propagated, zt as tunnelsOf } from "./judge-BD4UVQfy.js";
 import { Quantity } from "@mark1russell7/quantity";
-import { CheckFn, Finding, HealthServer, Poll, Rule, Scenario, ServiceBuilder, ServiceDescription, Stage, Timing, TransportImpl, Verdict } from "@mark1russell7/service";
+import { CheckFn, Finding, HealthServer, Poll, Rule, Scenario, ServiceBuilder, ServiceDescription, Stage, StopWay, Timing, TransportImpl, Verdict } from "@mark1russell7/service";
 import { RedisClient } from "@mark1russell7/service/redis";
 //#region src/backends.d.ts
 /** A service rule names the Service whose backends the agent reads: its namespace, its name, and its ClusterIP and port for IPVS. */
@@ -164,6 +164,8 @@ type Step = {
   readonly target: Target;
   readonly idle: Quantity | null;
   readonly port: number;
+} | {
+  readonly do: "certificates";
 };
 /**
  * A fault plan is the steps of a run:
@@ -362,6 +364,28 @@ export declare const ztunnelEnvArgs: (env: Readonly<Record<string, string>>, und
 export declare const setZtunnelEnv: (cluster: Cluster, env: Readonly<Record<string, string>>, undo: boolean) => Promise<void>;
 /** This function tells if the lab polls the LDS updates of the server sidecars of a cell: a policy in a sidecar mesh. */
 export declare const pollsListeners: (cell: Pick<KindCell, "mesh" | "trigger">) => boolean;
+/** This is the validity of one leaf certificate of ztunnel, as istioctl gives it: RFC 3339 times. */
+interface LeafCertificate {
+  readonly pod: string;
+  readonly identity: string;
+  readonly state: string;
+  readonly from: string | null;
+  readonly to: string | null;
+}
+/** This function gives the arguments of istioctl that read the certificates of one ztunnel pod as JSON. */
+export declare const certificateArgs: (pod: string, kubeconfig: string) => string[];
+/**
+ * This function gives the leaf of each identity in the JSON of `istioctl ztunnel-config certificate`.
+ * The leaf is the first certificate of the chain (certificates.go of istioctl 1.31.3, lines 68–71).
+ * An identity with no chain yet has no times.
+ */
+export declare const leafCertificates: (pod: string, json: string) => LeafCertificate[];
+/**
+ * This function reads the leaf certificates of each ztunnel pod that is not going, with istioctl.
+ * The CLI reads the admin port of ztunnel through a port-forward. Each ztunnel binds that port to
+ * localhost only (config.rs of ztunnel 1.31.3, lines 831–835).
+ */
+export declare const ztunnelCertificates: (cluster: Cluster) => Promise<LeafCertificate[]>;
 //#endregion
 //#region src/lab.d.ts
 /** A stamp brackets an act on CLOCK_MONOTONIC, in milliseconds. */
@@ -410,6 +434,8 @@ interface KindLab {
   take(): Line[];
   /** This method adds a listener of each line, and gives the function that removes it. */
   listen(listener: (line: Line) => void): () => void;
+  /** This method adds a line of the runner to the lines: for example, evidence that the runner reads with a tool. */
+  record(line: Line): void;
   /**
    * This method starts the watcher of the API on the namespaces. With `ztunnel`, it also follows the
    * log of each ztunnel in istio-system: lines `ztunnel-pod` and `ztunnel`, which no tracker of the
@@ -623,7 +649,7 @@ export declare const O09: Rule;
 export declare const O10: Rule;
 /** O16: the cooperative window is shorter than the drain, which is shorter than the grace less the preStop sleep. */
 export declare const O16: Rule;
-/** These are the rules of the drain that each kind cell gets. */
+/** These are the rules of the drain. A kind cell gets the rules that guard the way that its plan stops a pod. */
 export declare const DRAIN_RULES: readonly Rule[];
 /**
  * This function gives the scenario of a cell for `lint()`: the preStop sleep, the grace, the probe
@@ -632,7 +658,19 @@ export declare const DRAIN_RULES: readonly Rule[];
 export declare const scenarioOf: (cell: KindCell, propagation: Propagation) => Scenario;
 /** A verdict set is the verdict of each drain rule for one declaration, by the identifier of the rule. */
 type Verdicts = Readonly<Record<string, Verdict>>;
-/** This function applies the drain rules to the description of a cell, and gives each finding. */
+/**
+ * This function gives the way that the fault plan of a cell stops a pod, or null when the plan stops no pod.
+ * - A deletion, a rolling update, a node drain, a scale-in and a rollout delete a pod with its grace.
+ * - In T07 a pause of kube-proxy comes before a deletion. In T18 the pause is on Redis.
+ * - The crash of the lab is SIGKILL from the node. Like a crash, it has no preStop hook and no drain.
+ * - The other faults are on a link, on Redis or on the process. The kubelet can restart a pod after them.
+ */
+export declare const stopOf: (cell: KindCell) => StopWay | null;
+/**
+ * This function applies the drain rules to the description of a cell, and gives each finding. A cell
+ * whose plan stops a pod gets the rules of that way (`lintStop()`). A plan that stops no pod gets each
+ * drain rule, because a restart by the kubelet or a later deletion can follow its fault.
+ */
 export declare const lintCell: (description: ServiceDescription, cell: KindCell, propagation: Propagation) => Finding[];
 /** This function gives the verdict of each drain rule, by its identifier. */
 export declare const verdictsOf: (findings: readonly Finding[]) => Verdicts;
@@ -732,5 +770,5 @@ export declare const toolPath: (tool: Tool, key?: string, cache?: string) => str
  */
 export declare const ensureTool: (tool: Tool, key?: string, cache?: string) => Promise<string>;
 //#endregion
-export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HBONE_PORT, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, SocketLine, type Stack, type Stamp, type Step, TCP_STATE, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, TunnelLine, type Verdicts, type WindowPrediction, type Workload, ZtunnelPodLine, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, isTunnelState, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stoppingOf, timeSpread, transitionOf, tunnelsOf, unitOf, updatesOf, upgrade, upgradeCell, ztunnelPodsOf, ztunnelReplaced };
+export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HBONE_PORT, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, type LeafCertificate, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, SocketLine, type Stack, type Stamp, type Step, TCP_STATE, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, TunnelLine, type Verdicts, type WindowPrediction, type Workload, ZtunnelPodLine, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, isTunnelState, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stoppingOf, timeSpread, transitionOf, tunnelsOf, unitOf, updatesOf, upgrade, upgradeCell, ztunnelPodsOf, ztunnelReplaced };
 //# sourceMappingURL=index.d.ts.map

package/build/index.js

@@ -1,4 +1,4 @@
-import { $ as signalOf, A as medianInterval, B as emptyOf, C as propagationOf, D as unitOf, E as reusesReset, F as backendsAt, G as outcomesOf, H as intakeOf, I as byClients, J as recordsOf, K as podsAt, L as byNode, M as quantileOf, N as HBONE_PORT, O as updatesOf, P as TCP_STATE, Q as rulesOf, R as deletionOf, S as observationEnd, T as referenceOf, U as isTunnelState, V as failuresOf, W as kernelCounts, X as restartOf, Y as requestsOf, Z as restartsAt, _ as checkRun, _t as seconds, a as calibrationOf, at as stoppingOf, b as judgeKind, c as predictionText, ct as ztunnelPodsOf, d as FORMAT_V1, dt as HEALTH_PORT, et as signalsOf, f as judge, ft as LOG_LABEL, g as LAST_SLACK, gt as bandOf, h as KIND_NOISE, ht as SERVER, i as readRecord, it as socketsOf, j as middle, k as RELATIVE, l as timeSpread, lt as ztunnelReplaced, m as upgradeCell, mt as REDIS, n as kindCellRecordSchema, nt as sliceEmptyOf, o as cellsTable, ot as transitionOf, p as upgrade, pt as PORT, q as podsOf, r as kindCellRecordV1Schema, rt as slicesOf, s as medianText, st as tunnelsOf, t as STAGES, tt as sliceDropOf, u as FORMAT, ut as CLIENT, v as deletionsOf, vt as secondsOf, w as propagationSource, x as observationAfter, y as factsOf, yt as showBand, z as dropOf } from "./judge-B6W_gFv0.js";
+import { $ as signalOf, A as medianInterval, B as emptyOf, C as propagationOf, D as unitOf, E as reusesReset, F as backendsAt, G as outcomesOf, H as intakeOf, I as byClients, J as recordsOf, K as podsAt, L as byNode, M as quantileOf, N as HBONE_PORT, O as updatesOf, P as TCP_STATE, Q as rulesOf, R as deletionOf, S as observationEnd, T as referenceOf, U as isTunnelState, V as failuresOf, W as kernelCounts, X as restartOf, Y as requestsOf, Z as restartsAt, _ as checkRun, _t as seconds, a as calibrationOf, at as stoppingOf, b as judgeKind, c as predictionText, ct as ztunnelPodsOf, d as FORMAT_V1, dt as HEALTH_PORT, et as signalsOf, f as judge, ft as LOG_LABEL, g as LAST_SLACK, gt as bandOf, h as KIND_NOISE, ht as SERVER, i as readRecord, it as socketsOf, j as middle, k as RELATIVE, l as timeSpread, lt as ztunnelReplaced, m as upgradeCell, mt as REDIS, n as kindCellRecordSchema, nt as sliceEmptyOf, o as cellsTable, ot as transitionOf, p as upgrade, pt as PORT, q as podsOf, r as kindCellRecordV1Schema, rt as slicesOf, s as medianText, st as tunnelsOf, t as STAGES, tt as sliceDropOf, u as FORMAT, ut as CLIENT, v as deletionsOf, vt as secondsOf, w as propagationSource, x as observationAfter, y as factsOf, yt as showBand, z as dropOf } from "./judge-BbLWVTYj.js";
 import { valueIn } from "@mark1russell7/quantity";
 import { execFile, execFileSync } from "node:child_process";
 import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, realpathSync, renameSync, rmSync, writeFileSync } from "node:fs";
@@ -7,7 +7,7 @@ import { promisify } from "node:util";
 import { createHash } from "node:crypto";
 import { fileURLToPath } from "node:url";
 import { gunzipSync } from "node:zlib";
-import { failSpan, lint, service } from "@mark1russell7/service";
+import { failSpan, lint, lintStop, service } from "@mark1russell7/service";
 import { createRequire } from "node:module";
 import { tmpdir } from "node:os";
 import { harnessImage, imageOf } from "@mark1russell7/harness";
@@ -1242,6 +1242,12 @@ const planOf = (cell) => {
 				]
 			}]
 		};
+		case "certificate-expiry": return {
+			inject: [{ do: "certificates" }],
+			hold: needs(trigger.hold, "a hold, the time past the lifetime of the certificates"),
+			heal: [{ do: "certificates" }],
+			restore: []
+		};
 	}
 };
 /** This function tells if a trigger stops a pod of the service with its grace: a deletion, an eviction, a scale-in or a rollout. */
@@ -1621,6 +1627,57 @@ const setZtunnelEnv = async (cluster, env, undo) => {
 };
 /** This function tells if the lab polls the LDS updates of the server sidecars of a cell: a policy in a sidecar mesh. */
 const pollsListeners = (cell) => cell.mesh?.mode === "sidecar" && cell.trigger.kind === "authorization-policy";
+/** This function gives the arguments of istioctl that read the certificates of one ztunnel pod as JSON. */
+const certificateArgs = (pod, kubeconfig) => [
+	"ztunnel-config",
+	"certificate",
+	`${pod}.istio-system`,
+	"--kubeconfig",
+	kubeconfig,
+	"-o",
+	"json"
+];
+/**
+* This function gives the leaf of each identity in the JSON of `istioctl ztunnel-config certificate`.
+* The leaf is the first certificate of the chain (certificates.go of istioctl 1.31.3, lines 68–71).
+* An identity with no chain yet has no times.
+*/
+const leafCertificates = (pod, json) => {
+	return JSON.parse(json).map((each) => {
+		const leaf = each.certChain?.[0];
+		return {
+			pod,
+			identity: each.identity ?? "",
+			state: each.state ?? "",
+			from: leaf?.validFrom || null,
+			to: leaf?.expirationTime || null
+		};
+	});
+};
+/**
+* This function reads the leaf certificates of each ztunnel pod that is not going, with istioctl.
+* The CLI reads the admin port of ztunnel through a port-forward. Each ztunnel binds that port to
+* localhost only (config.rs of ztunnel 1.31.3, lines 831–835).
+*/
+const ztunnelCertificates = async (cluster) => {
+	const istioctl = await ensureTool(ISTIOCTL);
+	const listed = JSON.parse(await cluster.kubectl([
+		"-n",
+		"istio-system",
+		"get",
+		"pods",
+		"-l",
+		ZTUNNEL_PODS,
+		"-o",
+		"json"
+	]));
+	const out = [];
+	for (const item of listed.items.filter((each) => each.metadata.deletionTimestamp === void 0)) {
+		const json = await command(istioctl, certificateArgs(item.metadata.name, cluster.kubeconfig), { timeout: 6e4 });
+		out.push(...leafCertificates(item.metadata.name, json));
+	}
+	return out;
+};
 //#endregion
 //#region src/lds.ts
 /**
@@ -1808,6 +1865,9 @@ const openKindLab = async (cluster, image) => {
 			lines = [];
 			return out;
 		},
+		record(line) {
+			push(line);
+		},
 		listen(listener) {
 			listeners.add(listener);
 			return () => listeners.delete(listener);
@@ -2597,7 +2657,7 @@ const O16 = {
 		}] }
 	}]
 };
-/** These are the rules of the drain that each kind cell gets. */
+/** These are the rules of the drain. A kind cell gets the rules that guard the way that its plan stops a pod. */
 const DRAIN_RULES = [
 	O04,
 	O09,
@@ -2625,8 +2685,44 @@ const scenarioOf = (cell, propagation) => ({
 		liveness: s(seconds$1(cell.pod.liveness.period) * cell.pod.liveness.threshold)
 	}
 });
-/** This function applies the drain rules to the description of a cell, and gives each finding. */
-const lintCell = (description, cell, propagation) => lint(description, scenarioOf(cell, propagation), DRAIN_RULES);
+/**
+* This function gives the way that the fault plan of a cell stops a pod, or null when the plan stops no pod.
+* - A deletion, a rolling update, a node drain, a scale-in and a rollout delete a pod with its grace.
+* - In T07 a pause of kube-proxy comes before a deletion. In T18 the pause is on Redis.
+* - The crash of the lab is SIGKILL from the node. Like a crash, it has no preStop hook and no drain.
+* - The other faults are on a link, on Redis or on the process. The kubelet can restart a pod after them.
+*/
+const stopOf = (cell) => {
+	switch (cell.trigger.kind) {
+		case "pod-deletion":
+		case "rolling-update":
+		case "node-drain":
+		case "scale-in":
+		case "rollout": return "deletion";
+		case "freeze-process": return cell.template === "T07" ? "deletion" : null;
+		case "crash": return "crash";
+		case "liveness-restart": return "liveness-restart";
+		case "rst":
+		case "blackhole":
+		case "peer-gone-without-close":
+		case "app-stall":
+		case "cpu-throttle":
+		case "authorization-policy":
+		case "idle":
+		case "ztunnel-restart":
+		case "certificate-expiry": return null;
+	}
+};
+/**
+* This function applies the drain rules to the description of a cell, and gives each finding. A cell
+* whose plan stops a pod gets the rules of that way (`lintStop()`). A plan that stops no pod gets each
+* drain rule, because a restart by the kubelet or a later deletion can follow its fault.
+*/
+const lintCell = (description, cell, propagation) => {
+	const way = stopOf(cell);
+	const scenario = scenarioOf(cell, propagation);
+	return way === null ? lint(description, scenario, DRAIN_RULES) : lintStop(description, scenario, DRAIN_RULES, way);
+};
 /** This function gives the verdict of each drain rule, by its identifier. */
 const verdictsOf = (findings) => Object.fromEntries(findings.map((finding) => [finding.rule, finding.verdict]));
 /** This function gives the description of the service of a cell, as the app of the image composes it. Nothing connects. */
@@ -3176,6 +3272,26 @@ const doStep = async (ctx, step, replicas, background) => {
 				t1: await lab.now()
 			};
 		}
+		case "certificates": {
+			const t0 = await lab.now();
+			const certificates = await ztunnelCertificates(cluster).catch((error) => String(error).slice(0, 400));
+			const t1 = await lab.now();
+			lab.record(typeof certificates === "string" ? {
+				ev: "certificates",
+				t: t0,
+				t1,
+				error: certificates
+			} : {
+				ev: "certificates",
+				t: t0,
+				t1,
+				certificates
+			});
+			return {
+				t0,
+				t1
+			};
+		}
 		case "middlebox":
 			if (step.target !== "servers") throw new Error(`a middlebox takes the target servers, not ${step.target}`);
 			return middleboxServers(ctx, step.idle, step.port, replicas);
@@ -3568,7 +3684,8 @@ const estimate = (cell, environments = 2) => {
 		rollout: hold + 50,
 		"authorization-policy": hold + 20,
 		idle: hold + 5,
-		"ztunnel-restart": 40
+		"ztunnel-restart": 40,
+		"certificate-expiry": hold + 20
 	};
 	const after = observationAfter(cell);
 	const observed = after === null ? 0 : hold + valueIn(after, "s");
@@ -3606,6 +3723,6 @@ const shardOf = (cells, index, count) => {
 	return out;
 };
 //#endregion
-export { ACCESS_LOG_FORMAT, CHANNEL_USE, CLIENT, CLIENT_ACCOUNT, CLOSE_LEAD, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HBONE_PORT, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LDS_EVERY, LDS_FILTER, LOG_LABEL, MESH_IMAGES, MESH_WORKLOADS, MIDDLEBOX, NETNS_DEFAULTS, NODE_IMAGES, O04, O09, O10, O16, POLICY, PORT, QUIET_LEAD, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, SYSCTL, TCP_STATE, WATCHDOG, ZTUNNEL_PODS, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, followsZtunnel, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, isTunnelState, istioOperator, judge, judgeKind, keptKernel, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, lintCell, listenerCountsOf, localPin, medianInterval, medianText, meshKey, middle, middleboxCommands, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, policyObject, pollsListeners, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartOf, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, setZtunnelEnv, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stackOf, stageLabImage, stallRemovalBand, stoppingOf, sysctlCommand, tarEntry, timeSpread, toolPath, transitionOf, tunnelsOf, unitOf, updatesOf, upgrade, upgradeCell, verdictsOf, writeResult, ztunnelEnvArgs, ztunnelPodsOf, ztunnelReplaced };
+export { ACCESS_LOG_FORMAT, CHANNEL_USE, CLIENT, CLIENT_ACCOUNT, CLOSE_LEAD, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HBONE_PORT, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LDS_EVERY, LDS_FILTER, LOG_LABEL, MESH_IMAGES, MESH_WORKLOADS, MIDDLEBOX, NETNS_DEFAULTS, NODE_IMAGES, O04, O09, O10, O16, POLICY, PORT, QUIET_LEAD, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, SYSCTL, TCP_STATE, WATCHDOG, ZTUNNEL_PODS, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, certificateArgs, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, followsZtunnel, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, isTunnelState, istioOperator, judge, judgeKind, keptKernel, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, leafCertificates, lintCell, listenerCountsOf, localPin, medianInterval, medianText, meshKey, middle, middleboxCommands, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, policyObject, pollsListeners, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartOf, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, setZtunnelEnv, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stackOf, stageLabImage, stallRemovalBand, stopOf, stoppingOf, sysctlCommand, tarEntry, timeSpread, toolPath, transitionOf, tunnelsOf, unitOf, updatesOf, upgrade, upgradeCell, verdictsOf, writeResult, ztunnelCertificates, ztunnelEnvArgs, ztunnelPodsOf, ztunnelReplaced };
 
 //# sourceMappingURL=index.js.map