@mark1russell7/kind 0.3.0 → 0.4.1

package/README.md

@@ -85,6 +85,10 @@ The tests read three real records of `/1` (`test/records/`): a cell that holds,
 - **The endpoints of each client sidecar.** From 0.3.0, in a sidecar mesh, the agent also polls the Envoy of the client sidecar on its node each 50 ms (`eds`). The client sidecar picks the backend of each call, thus these endpoints and not the rules are the propagation of the cell (`propagationSource`).
   - A poller enters the network namespace of the client pod (`nsenter -n`). It reads `/clusters` on `127.0.0.1:15000` with a filter that selects the server cluster (`eds.ts`).
   - Each change of the healthy hosts gives an `eds` line with the start of the poll before it. That line has the form of a rules line. Each second, an `eds-alive` line counts the polls and gives their mean time.
+- **The listeners of each server sidecar.** From 0.4.0, the agent also polls the Envoy of each server sidecar on its node each 50 ms (`lds`). It does this for a policy in a sidecar mesh (`pollsListeners`).
+  - A poller enters the network namespace of the server pod through the PID of its app. It reads `/stats` on `127.0.0.1:15000` with a filter that selects two counters of the listener manager (`lds.ts`).
+  - `listener_modified` counts each update of a listener, and `listener_in_place_updated` counts each update of its filter chains only.
+  - Each change of the counters gives an `lds` line with the start of the poll before it. The first line of a poller has no poll before it, thus it is no update (`updatesOf`).
 - **The injector of each node** is the image of the harness, in the PID namespace of the node. `crictl` on the node gives the PID of a container, and `nsenter -n` enters the network namespace of its pod. The injector of the harness then applies its rules there.
 - **The watcher** is a container on the network of the cluster. It watches the pods, the EndpointSlices and the events through the API, and it follows the log of each pod of the cell.
   - From 0.3.0, it also follows the sidecar of Istio (`istio-proxy`), and a pod line gives the state of the sidecar (`sc`). Each line of the sidecar becomes a `proxy` line, stamped at its arrival. Its own stamps are of the wall clock, thus no band reads them.
@@ -157,6 +161,7 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
 | `app-stall` | the event loop of the first replica blocked |
 | `rollout` | no Redis pod, then a rolling update |
 | `cpu-throttle` | `cpu.max` in the cgroup of the container of the first replica |
+| `authorization-policy` | `kubectl apply` of a DENY AuthorizationPolicy on the server pods for the principal of an account (`account`, the account `default` of the clients when not given), then `kubectl delete` after the hold |
 
 A cgroup path that is not the scope of one container is refused. A write to the root of the node throttles or freezes each pod of the node.
 
@@ -197,6 +202,15 @@ From 0.2.3, a window can also give the propagation that it assumes (`propagation
 - An earlier drop only closes the window earlier, thus the judge does not test the start of the band.
 - `predictionText` adds `the rules drop the replica in` and the band.
 
+From 0.4.0, a window can count from the heal (`heal`), the first act that ends the fault. A window can also name its outcomes (`outcomes`). A failure in the window with another outcome then fails the run.
+
+- The client of gRPC gives the outcome `denied` to PERMISSION_DENIED: the local reply 403 of an RBAC deny of a sidecar.
+- For a policy, the propagation of a window is the LDS update of each server sidecar, after the apply and after the delete (`updatesOf`). An update after the band agrees with the law, and the cell is `extend`.
+- A denial after the window, before the last update after the delete and one more call, is a `tail`. A denied call has no pod, because the sidecar answers it. Thus the judge reads the last update of each server sidecar.
+- `calibrationOf` gives the updates after the apply by node, and the updates after the delete (`heal`).
+
+A new member of the union `TriggerKind` breaks a reader that switches on it. That is a breaking change before 1.0.0, thus the version is 0.4.0. The record format stays `/2`, with the optional fields `account`, `outcomes`, `updates` and `healUpdates`, and the reference `heal`.
+
 A preStop sleep that is longer than the grace is a command in the container (`sh -c "sleep 60"`). The API server refuses a sleep action that is longer than the grace.
 
 ## Tests

package/build/index.d.ts

@@ -1,4 +1,4 @@
-import { $ as Endpoint, $t as ResetReuse, A as TriggerV1, At as CLIENT, B as RunCheck, Bt as Band, C as KindCellResult, Ct as signalOf, D as PredictionV1, Dt as slicesOf, E as LoadSample, Et as sliceEmptyOf, F as KIND_NOISE, Ft as SERVER, G as judgeKind, Gt as Mesh, H as checkRun, Ht as KindCell, I as KindDecision, It as bandOf, J as propagationOf, Jt as PodSettings, K as observationAfter, Kt as MeshFlavor, L as KindVerdict, Lt as seconds, M as upgrade, Mt as LOG_LABEL, N as upgradeCell, Nt as PORT, O as RunCheckV1, Ot as stoppingOf, P as Deleted, Pt as REDIS, Q as unitOf, Qt as Replica, R as LAST_SLACK, Rt as secondsOf, S as KindCellRecordV1, St as rulesOf, T as KindVerdictV1, Tt as sliceDropOf, U as deletionsOf, Ut as KindRun, V as RunFacts, Vt as Declaration, W as factsOf, Wt as KubeletProbe, X as referenceOf, Xt as RedisDeclaration, Y as propagationSource, Yt as Prediction, Z as reusesReset, Zt as Reference, _ as ClusterManifest, _n as scenario, _t as podsAt, a as STAGES, an as KUBE_PROXY_MODES, at as RulesLine, b as FORMAT_V1, bt as requestsOf, c as readRecord, cn as KubeProxyMode, ct as byClients, d as calibrationOf, dn as cidrOverlap, dt as dropOf, en as Stack, et as Line, f as cellsTable, fn as cidrRange, ft as emptyOf, g as BandV1, gn as nodeNames, gt as outcomesOf, h as timeSpread, hn as kubeProxyPatch, ht as kernelCounts, i as quantileOf, in as Workload, it as Request, j as judge, jt as HEALTH_PORT, k as RunFactsV1, kt as transitionOf, l as Judged, ln as NODE_IMAGES, lt as byNode, m as predictionText, mn as kindConfig, mt as intakeOf, n as medianInterval, nn as Trigger, nt as PropagationSource, o as kindCellRecordSchema, on as KindConfig, ot as SliceLine, p as medianText, pn as goDuration, pt as failuresOf, q as observationEnd, qt as MeshMode, r as middle, rn as TriggerKind, rt as RecordLine, s as kindCellRecordV1Schema, sn as KindScenario, st as backendsAt, t as RELATIVE, tn as Template, tt as PodLine, u as TimeSpread, un as NodeImage, ut as deletionOf, v as EnvironmentManifest, vn as scenarioProblems, vt as podsOf, w as KindCellV1, wt as signalsOf, x as KindCellRecord, xt as restartsAt, y as FORMAT, yt as recordsOf, z as Propagated, zt as showBand } from "./judge-DWF4QdZ8.js";
+import { $ as updatesOf, $t as Replica, A as TriggerV1, At as transitionOf, B as RunCheck, Bt as showBand, C as KindCellResult, Ct as rulesOf, D as PredictionV1, Dt as sliceEmptyOf, E as LoadSample, Et as sliceDropOf, F as KIND_NOISE, Ft as REDIS, G as judgeKind, Gt as KubeletProbe, H as checkRun, Ht as Declaration, I as KindDecision, It as SERVER, J as propagationOf, Jt as MeshMode, K as observationAfter, Kt as Mesh, L as KindVerdict, Lt as bandOf, M as upgrade, Mt as HEALTH_PORT, N as upgradeCell, Nt as LOG_LABEL, O as RunCheckV1, Ot as slicesOf, P as Deleted, Pt as PORT, Q as unitOf, Qt as Reference, R as LAST_SLACK, Rt as seconds, S as KindCellRecordV1, St as restartsAt, T as KindVerdictV1, Tt as signalsOf, U as deletionsOf, Ut as KindCell, V as RunFacts, Vt as Band, W as factsOf, Wt as KindRun, X as referenceOf, Xt as Prediction, Y as propagationSource, Yt as PodSettings, Z as reusesReset, Zt as RedisDeclaration, _ as ClusterManifest, _n as nodeNames, _t as outcomesOf, a as STAGES, an as Workload, at as Request, b as FORMAT_V1, bt as recordsOf, c as readRecord, cn as KindScenario, ct as backendsAt, d as calibrationOf, dn as NodeImage, dt as deletionOf, en as ResetReuse, et as Endpoint, f as cellsTable, fn as cidrOverlap, ft as dropOf, g as BandV1, gn as kubeProxyPatch, gt as kernelCounts, h as timeSpread, hn as kindConfig, ht as intakeOf, i as quantileOf, in as TriggerKind, it as RecordLine, j as judge, jt as CLIENT, k as RunFactsV1, kt as stoppingOf, l as Judged, ln as KubeProxyMode, lt as byClients, m as predictionText, mn as goDuration, mt as failuresOf, n as medianInterval, nn as Template, nt as PodLine, o as kindCellRecordSchema, on as KUBE_PROXY_MODES, ot as RulesLine, p as medianText, pn as cidrRange, pt as emptyOf, q as observationEnd, qt as MeshFlavor, r as middle, rn as Trigger, rt as PropagationSource, s as kindCellRecordV1Schema, sn as KindConfig, st as SliceLine, t as RELATIVE, tn as Stack, tt as Line, u as TimeSpread, un as NODE_IMAGES, ut as byNode, v as EnvironmentManifest, vn as scenario, vt as podsAt, w as KindCellV1, wt as signalOf, x as KindCellRecord, xt as requestsOf, y as FORMAT, yn as scenarioProblems, yt as podsOf, z as Propagated, zt as secondsOf } from "./judge-DMuI3aDn.js";
 import { Quantity } from "@mark1russell7/quantity";
 import { CheckFn, Finding, HealthServer, Poll, Rule, Scenario, ServiceBuilder, ServiceDescription, Stage, Timing, TransportImpl, Verdict } from "@mark1russell7/service";
 import { RedisClient } from "@mark1russell7/service/redis";
@@ -145,6 +145,9 @@ type Step = {
 } | {
   readonly do: "wait";
   readonly time: Quantity;
+} | {
+  readonly do: "apply";
+  readonly object: Readonly<Record<string, unknown>>;
 };
 /**
  * A fault plan is the steps of a run:
@@ -158,6 +161,17 @@ interface FaultPlan {
   readonly heal: readonly Step[];
   readonly restore: readonly Step[];
 }
+/** This name is the name of the AuthorizationPolicy of the trigger `authorization-policy`. */
+export declare const POLICY = "lab-deny";
+/** This service account is the account of the clients: their pods name no other account. */
+export declare const CLIENT_ACCOUNT = "default";
+/**
+ * This function gives the AuthorizationPolicy of the trigger `authorization-policy`. It denies each
+ * request to the server pods from the principal of the service account `account`. Istio gives a
+ * pod of the mesh the principal `cluster.local/ns/<namespace>/sa/<account>` of its account. The
+ * RBAC filter of the server sidecar reads the principal of the peer from its mTLS certificate.
+ */
+export declare const policyObject: (account: string) => Readonly<Record<string, unknown>>;
 /** This function gives the fault plan of a cell. */
 export declare const planOf: (cell: KindCell) => FaultPlan;
 /** This function tells if a trigger stops a pod of the service with its grace: a deletion, an eviction, a scale-in or a rollout. */
@@ -170,7 +184,11 @@ export declare const deletes: (cell: KindCell) => boolean;
  * Then it copies the files of the image.
  */
 export declare const stageLabImage: () => string;
-/** This function builds the lab image one time for each process, and gives its tag. Docker keeps its layers between builds. */
+/**
+ * This function builds the lab image one time for each process, and gives its tag. Docker keeps its
+ * layers between builds. A build that fails leaves the cache. Thus a later call builds again after a
+ * fault of Docker or of the network.
+ */
 export declare const labImage: () => Promise<string>;
 //#endregion
 //#region src/istio.d.ts
@@ -228,6 +246,10 @@ export declare const meshKey: (cell: Pick<KindCell, "mesh">) => MeshFlavor | "no
 export declare const serverCluster: (namespace: string) => string;
 /** This time is the interval of the EDS poll of each client sidecar, the same as the poll of the rules. */
 export declare const EDS_EVERY: Quantity;
+/** This time is the interval of the LDS poll of each server sidecar, the same as the poll of EDS. */
+export declare const LDS_EVERY: Quantity;
+/** This function tells if the lab polls the LDS updates of the server sidecars of a cell: a policy in a sidecar mesh. */
+export declare const pollsListeners: (cell: Pick<KindCell, "mesh" | "trigger">) => boolean;
 //#endregion
 //#region src/lab.d.ts
 /** A stamp brackets an act on CLOCK_MONOTONIC, in milliseconds. */
@@ -247,6 +269,15 @@ interface EdsSpec {
     readonly pid: number;
   }>;
 }
+/** An LDS spec is the interval of the LDS pollers, and each server pod with its node and a PID in it. */
+interface LdsSpec {
+  readonly every: Quantity;
+  readonly pods: ReadonlyArray<{
+    readonly node: string;
+    readonly pod: string;
+    readonly pid: number;
+  }>;
+}
 /** A kind lab is the instruments of one cluster: a sidecar pair on each node, the watcher of the API and the prober. */
 interface KindLab {
   readonly cluster: Cluster;
@@ -274,6 +305,11 @@ interface KindLab {
    * of its processes, and the agent enters the network namespace of that PID.
    */
   eds(spec: EdsSpec | null): Promise<void>;
+  /**
+   * This method asks each agent to poll the sidecar of each server pod of its node, `every` apart.
+   * Each poll gives the counters of LDS updates of that sidecar (`lds.ts`). With null, the polls stop.
+   */
+  lds(spec: LdsSpec | null): Promise<void>;
   act(node: string, spec: Readonly<Record<string, unknown>>): Promise<Stamp>;
   inject(node: string, pid: number, args: readonly string[]): Promise<Stamp>;
   now(): Promise<number>;
@@ -299,6 +335,25 @@ export declare const openKindLab: (cluster: Cluster, image: string) => Promise<K
 /** This function pins a Docker image by digest under a local tag that holds the digest, for `kind load` and `imagePullPolicy: Never`. */
 export declare const localPin: (image: string) => Promise<string>;
 //#endregion
+//#region src/lds.d.ts
+/**
+ * This filter of RE2 selects the two counters of the listener manager of Envoy that an update of LDS
+ * moves (`/stats?filter=`, `source/server/admin/stats_params.cc` of Envoy v1.39.4). An update of a
+ * listener counts `listener_modified`. An update in place, of its filter chains only, also counts
+ * `listener_in_place_updated` (`listener_manager_impl.cc:644`, `:692`).
+ */
+export declare const LDS_FILTER = "^listener_manager\\.listener_(in_place_updated|modified)$";
+/** The listener counts are the two counters of LDS of one sidecar: each update (`mod`), and each update in place (`inPlace`). */
+interface ListenerCounts {
+  readonly mod: number;
+  readonly inPlace: number;
+}
+/**
+ * This function gives the listener counts of the text of `/stats`, one `name: value` on each line
+ * (`stats_render.cc:15`). It gives null when the text has no counter `listener_modified`.
+ */
+export declare const listenerCountsOf: (text: string) => ListenerCounts | null;
+//#endregion
 //#region src/manifests.d.ts
 /**
  * This module generates the Kubernetes objects of a cell. They are its namespace, its Redis, the
@@ -537,5 +592,5 @@ export declare const toolPath: (tool: Tool, key?: string, cache?: string) => str
  */
 export declare const ensureTool: (tool: Tool, key?: string, cache?: string) => Promise<string>;
 //#endregion
-export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, Line, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, type Stack, type Stamp, type Step, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, type Verdicts, type Workload, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stoppingOf, timeSpread, transitionOf, unitOf, upgrade, upgradeCell };
+export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, type Stack, type Stamp, type Step, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, type Verdicts, type Workload, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stoppingOf, timeSpread, transitionOf, unitOf, updatesOf, upgrade, upgradeCell };
 //# sourceMappingURL=index.d.ts.map

package/build/index.js

@@ -1,4 +1,4 @@
-import { $ as stoppingOf, A as middle, B as kernelCounts, C as propagationOf, D as unitOf, E as reusesReset, F as deletionOf, G as requestsOf, H as podsAt, I as dropOf, J as signalOf, K as restartsAt, L as emptyOf, M as backendsAt, N as byClients, O as RELATIVE, P as byNode, Q as slicesOf, R as failuresOf, S as observationEnd, T as referenceOf, U as podsOf, V as outcomesOf, W as recordsOf, X as sliceDropOf, Y as signalsOf, Z as sliceEmptyOf, _ as checkRun, a as calibrationOf, at as REDIS, b as judgeKind, c as predictionText, ct as seconds, d as FORMAT_V1, et as transitionOf, f as judge, g as LAST_SLACK, h as KIND_NOISE, i as readRecord, it as PORT, j as quantileOf, k as medianInterval, l as timeSpread, lt as secondsOf, m as upgradeCell, n as kindCellRecordSchema, nt as HEALTH_PORT, o as cellsTable, ot as SERVER, p as upgrade, q as rulesOf, r as kindCellRecordV1Schema, rt as LOG_LABEL, s as medianText, st as bandOf, t as STAGES, tt as CLIENT, u as FORMAT, ut as showBand, v as deletionsOf, w as propagationSource, x as observationAfter, y as factsOf, z as intakeOf } from "./judge-CRMYoEOw.js";
+import { $ as slicesOf, A as medianInterval, B as intakeOf, C as propagationOf, D as unitOf, E as reusesReset, F as byNode, G as recordsOf, H as outcomesOf, I as deletionOf, J as rulesOf, K as requestsOf, L as dropOf, M as quantileOf, N as backendsAt, O as updatesOf, P as byClients, Q as sliceEmptyOf, R as emptyOf, S as observationEnd, T as referenceOf, U as podsAt, V as kernelCounts, W as podsOf, X as signalsOf, Y as signalOf, Z as sliceDropOf, _ as checkRun, a as calibrationOf, at as PORT, b as judgeKind, c as predictionText, ct as bandOf, d as FORMAT_V1, dt as showBand, et as stoppingOf, f as judge, g as LAST_SLACK, h as KIND_NOISE, i as readRecord, it as LOG_LABEL, j as middle, k as RELATIVE, l as timeSpread, lt as seconds, m as upgradeCell, n as kindCellRecordSchema, nt as CLIENT, o as cellsTable, ot as REDIS, p as upgrade, q as restartsAt, r as kindCellRecordV1Schema, rt as HEALTH_PORT, s as medianText, st as SERVER, t as STAGES, tt as transitionOf, u as FORMAT, ut as secondsOf, v as deletionsOf, w as propagationSource, x as observationAfter, y as factsOf, z as failuresOf } from "./judge-Bkbs7VVG.js";
 import { valueIn } from "@mark1russell7/quantity";
 import { execFile, execFileSync } from "node:child_process";
 import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, realpathSync, renameSync, rmSync, writeFileSync } from "node:fs";
@@ -619,6 +619,29 @@ const NODE = "{node}";
 const VERSION = "{version}";
 /** This annotation of a pod is its deletion cost for its ReplicaSet: a lower cost goes first at a scale-in. */
 const DELETION_COST = "controller.kubernetes.io/pod-deletion-cost";
+/** This name is the name of the AuthorizationPolicy of the trigger `authorization-policy`. */
+const POLICY = "lab-deny";
+/** This service account is the account of the clients: their pods name no other account. */
+const CLIENT_ACCOUNT = "default";
+/**
+* This function gives the AuthorizationPolicy of the trigger `authorization-policy`. It denies each
+* request to the server pods from the principal of the service account `account`. Istio gives a
+* pod of the mesh the principal `cluster.local/ns/<namespace>/sa/<account>` of its account. The
+* RBAC filter of the server sidecar reads the principal of the peer from its mTLS certificate.
+*/
+const policyObject = (account) => ({
+	apiVersion: "security.istio.io/v1",
+	kind: "AuthorizationPolicy",
+	metadata: {
+		name: POLICY,
+		namespace: "{namespace}"
+	},
+	spec: {
+		selector: { matchLabels: { app: "lab" } },
+		action: "DENY",
+		rules: [{ from: [{ source: { principals: [`cluster.local/ns/{namespace}/sa/${account}`] } }] }]
+	}
+});
 /** This function gives the fault plan of a cell. */
 const planOf = (cell) => {
 	const ns = ["-n", "{namespace}"];
@@ -881,6 +904,32 @@ const planOf = (cell) => {
 				restore: []
 			};
 		}
+		case "authorization-policy": return {
+			inject: [{
+				do: "apply",
+				object: policyObject(trigger.account ?? "default")
+			}],
+			hold: needs(trigger.hold, "a hold"),
+			heal: [{
+				do: "kubectl",
+				args: [
+					...ns,
+					"delete",
+					"authorizationpolicy",
+					POLICY
+				]
+			}],
+			restore: [{
+				do: "kubectl",
+				args: [
+					...ns,
+					"delete",
+					"authorizationpolicy",
+					POLICY,
+					"--ignore-not-found"
+				]
+			}]
+		};
 	}
 };
 /** This function tells if a trigger stops a pod of the service with its grace: a deletion, an eviction, a scale-in or a rollout. */
@@ -916,7 +965,8 @@ const FILES = [
 	["backends.ts", "src/backends.ts"],
 	["conntrack.ts", "src/conntrack.ts"],
 	["drops.ts", "src/drops.ts"],
-	["eds.ts", "src/eds.ts"]
+	["eds.ts", "src/eds.ts"],
+	["lds.ts", "src/lds.ts"]
 ];
 const shell = process.platform === "win32";
 /** This function gives each file of a staged context, with its path in the context, in a stable sequence. */
@@ -991,9 +1041,14 @@ const stageLabImage = () => {
 	return context;
 };
 let built = null;
-/** This function builds the lab image one time for each process, and gives its tag. Docker keeps its layers between builds. */
+/**
+* This function builds the lab image one time for each process, and gives its tag. Docker keeps its
+* layers between builds. A build that fails leaves the cache. Thus a later call builds again after a
+* fault of Docker or of the network.
+*/
 const labImage = () => {
-	built ??= (async () => {
+	if (built !== null) return built;
+	const building = (async () => {
 		const context = stageLabImage();
 		try {
 			const tag = contentTag("systems-kind-lab", context);
@@ -1007,7 +1062,11 @@ const labImage = () => {
 			});
 		}
 	})();
-	return built;
+	building.catch(() => {
+		if (built === building) built = null;
+	});
+	built = building;
+	return building;
 };
 //#endregion
 //#region src/istio.ts
@@ -1135,6 +1194,40 @@ const EDS_EVERY = {
 	value: 50,
 	unit: "ms"
 };
+/** This time is the interval of the LDS poll of each server sidecar, the same as the poll of EDS. */
+const LDS_EVERY = {
+	value: 50,
+	unit: "ms"
+};
+/** This function tells if the lab polls the LDS updates of the server sidecars of a cell: a policy in a sidecar mesh. */
+const pollsListeners = (cell) => cell.mesh?.mode === "sidecar" && cell.trigger.kind === "authorization-policy";
+//#endregion
+//#region src/lds.ts
+/**
+* This filter of RE2 selects the two counters of the listener manager of Envoy that an update of LDS
+* moves (`/stats?filter=`, `source/server/admin/stats_params.cc` of Envoy v1.39.4). An update of a
+* listener counts `listener_modified`. An update in place, of its filter chains only, also counts
+* `listener_in_place_updated` (`listener_manager_impl.cc:644`, `:692`).
+*/
+const LDS_FILTER = "^listener_manager\\.listener_(in_place_updated|modified)$";
+/**
+* This function gives the listener counts of the text of `/stats`, one `name: value` on each line
+* (`stats_render.cc:15`). It gives null when the text has no counter `listener_modified`.
+*/
+const listenerCountsOf = (text) => {
+	let mod = null;
+	let inPlace = 0;
+	for (const line of text.split("\n")) {
+		const match = /^listener_manager\.listener_(in_place_updated|modified): (\d+)$/.exec(line.trim());
+		if (match === null) continue;
+		if (match[1] === "modified") mod = Number(match[2]);
+		else inPlace = Number(match[2]);
+	}
+	return mod === null ? null : {
+		mod,
+		inPlace
+	};
+};
 //#endregion
 //#region src/conntrack.ts
 /** This function gives the count of entries that `conntrack -D` deleted, from its last line: "N flow entries have been deleted". */
@@ -1243,6 +1336,33 @@ const openKindLab = async (cluster, image) => {
 		return found;
 	};
 	const firstNode = cluster.nodes[0];
+	/**
+	* This function writes the file of the pollers of one kind on each agent: `fields`, the interval
+	* and the pods of its node. A node without a pod, or a spec of null, gets no file.
+	*/
+	const pollersOf = async (kind, fields, spec) => {
+		for (const [node, { agent }] of sidecars) {
+			const pods = spec === null ? [] : spec.pods.filter((pod) => pod.node === node).map(({ pod, pid }) => ({
+				pod,
+				pid
+			}));
+			const file = spec === null || pods.length === 0 ? null : JSON.stringify({
+				...fields,
+				every: valueIn(spec.every, "ms"),
+				pods
+			});
+			if (file === null) await output(agent, [
+				"rm",
+				"-f",
+				`/tmp/${kind}.json`
+			]);
+			else await output(agent, [
+				"sh",
+				"-c",
+				`printf '%s' '${file}' > /tmp/${kind}.json.partial && mv /tmp/${kind}.json.partial /tmp/${kind}.json`
+			]);
+		}
+	};
 	const lab = {
 		cluster,
 		image,
@@ -1318,27 +1438,10 @@ const openKindLab = async (cluster, image) => {
 			]);
 		},
 		async eds(spec) {
-			for (const [node, { agent }] of sidecars) {
-				const pods = spec === null ? [] : spec.pods.filter((pod) => pod.node === node).map(({ pod, pid }) => ({
-					pod,
-					pid
-				}));
-				const file = spec === null || pods.length === 0 ? null : JSON.stringify({
-					cluster: spec.cluster,
-					every: valueIn(spec.every, "ms"),
-					pods
-				});
-				if (file === null) await output(agent, [
-					"rm",
-					"-f",
-					"/tmp/eds.json"
-				]);
-				else await output(agent, [
-					"sh",
-					"-c",
-					`printf '%s' '${file}' > /tmp/eds.json.partial && mv /tmp/eds.json.partial /tmp/eds.json`
-				]);
-			}
+			await pollersOf("eds", spec === null ? null : { cluster: spec.cluster }, spec);
+		},
+		async lds(spec) {
+			await pollersOf("lds", {}, spec);
 		},
 		async forgetHandshakes(ip, port) {
 			const out = {};
@@ -2278,6 +2381,28 @@ const pollEndpoints = async (lab, cluster, namespace) => {
 		pods
 	});
 };
+/**
+* This function starts the LDS poll of the sidecar of each server pod, each 50 ms, for a cell that
+* polls them (`pollsListeners`). It starts the pollers again only when the server pods changed. The
+* agent enters the network namespace of the pod through the PID of its app.
+*/
+const pollListeners = async (ctx) => {
+	if (!pollsListeners(ctx.cell)) return;
+	const servers = ctx.tracker.servers().filter((pod) => !pod.del && pod.node !== null);
+	const key = servers.map((pod) => `${pod.name}/${pod.cs.map((status) => status.id).join("+")}`).sort().join();
+	if (key === ctx.listened) return;
+	const pods = [];
+	for (const pod of servers) pods.push({
+		node: pod.node,
+		pod: pod.name,
+		pid: await ctx.lab.pidOf(pod.node, ctx.namespace, pod.name, "app")
+	});
+	await ctx.lab.lds({
+		every: LDS_EVERY,
+		pods
+	});
+	ctx.listened = key;
+};
 /** This function starts the prober again when the pods of the cell changed: it watches the namespace of each pod. */
 const probeThePods = async (ctx) => {
 	const pods = [...ctx.tracker.pods.values()].filter((pod) => !pod.del && pod.node !== null && pod.cs.every((status) => status.st === "running"));
@@ -2425,6 +2550,15 @@ const doStep = async (ctx, step, replicas, background) => {
 		case "wait":
 			await sleep(valueIn(step.time, "ms"));
 			return null;
+		case "apply": {
+			const object = JSON.parse(fill([JSON.stringify(step.object)])[0]);
+			const t0 = await lab.now();
+			await cluster.apply([object]);
+			return {
+				t0,
+				t1: await lab.now()
+			};
+		}
 	}
 };
 /** This function tells if a run is over: the trigger is done, and the cell is at rest again. */
@@ -2476,6 +2610,7 @@ const runOnce = async (ctx, env, index, warmup, ours) => {
 	});
 	if (atRest !== null) return empty(`the cell was not at rest: ${atRest}`, await lab.now());
 	await probeThePods(ctx);
+	await pollListeners(ctx);
 	await quiet(ctx, ours);
 	if (await steady(ctx, 3e5) !== null) return empty("the cell was not at rest after the wait for a quiet VM", await lab.now());
 	const replicas = replicasOf(ctx);
@@ -2708,6 +2843,7 @@ const runKindCells = async (cells, options) => {
 							log: options.log,
 							version: 1,
 							probed: "",
+							listened: "",
 							expected: cell.pod.replicas
 						};
 						const share = Math.ceil(cell.runs / environments) - (env < cell.runs % environments || cell.runs % environments === 0 ? 0 : 1);
@@ -2746,6 +2882,7 @@ const runKindCells = async (cells, options) => {
 						off();
 						await lab.rules(null).catch(() => void 0);
 						await lab.eds(null).catch(() => void 0);
+						await lab.lds(null).catch(() => void 0);
 						await lab.stopProbe();
 						await cluster.kubectl([
 							"delete",
@@ -2789,7 +2926,8 @@ const estimate = (cell, environments = 2) => {
 		crash: 30,
 		"app-stall": hold + 40,
 		"cpu-throttle": hold + 40,
-		rollout: hold + 50
+		rollout: hold + 50,
+		"authorization-policy": hold + 20
 	};
 	const after = observationAfter(cell);
 	const observed = after === null ? 0 : hold + valueIn(after, "s");
@@ -2825,6 +2963,6 @@ const shardOf = (cells, index, count) => {
 	return out;
 };
 //#endregion
-export { ACCESS_LOG_FORMAT, CLIENT, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LOG_LABEL, NODE_IMAGES, O04, O09, O10, O16, PORT, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, WATCHDOG, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, istioOperator, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, lintCell, localPin, medianInterval, medianText, meshKey, middle, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stackOf, stageLabImage, stallRemovalBand, stoppingOf, tarEntry, timeSpread, toolPath, transitionOf, unitOf, upgrade, upgradeCell, verdictsOf, writeResult };
+export { ACCESS_LOG_FORMAT, CLIENT, CLIENT_ACCOUNT, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LDS_EVERY, LDS_FILTER, LOG_LABEL, NODE_IMAGES, O04, O09, O10, O16, POLICY, PORT, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, WATCHDOG, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, istioOperator, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, lintCell, listenerCountsOf, localPin, medianInterval, medianText, meshKey, middle, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, policyObject, pollsListeners, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stackOf, stageLabImage, stallRemovalBand, stoppingOf, tarEntry, timeSpread, toolPath, transitionOf, unitOf, updatesOf, upgrade, upgradeCell, verdictsOf, writeResult };
 
 //# sourceMappingURL=index.js.map