@mark1russell7/kind 0.1.8 → 0.2.1

package/README.md

@@ -54,7 +54,7 @@ The unit of the main value of a run comes from the prediction of its cell:
 
 These values stay numbers in the two formats:
 
-- **A stamp** is a reading of CLOCK_MONOTONIC in milliseconds, a point on a clock, not an amount. The stamps are `start`, `end` and `acts` of a run, `t` of each line, and `first`, `last` and `deletion` of the facts.
+- **A stamp** is a reading of CLOCK_MONOTONIC in milliseconds, a point on a clock, not an amount. The stamps are `start`, `end` and `acts` of a run, `t` of each line, and `first`, `last`, `deletion` and `deletions` of the facts.
 - **A line** stays as its source wrote it. The image writes it, and the record keeps it.
 - **A count** or **a share**: the replicas, the thresholds, the runs and the counts of a verdict. The jitter of a poll and the idle share of a load sample are shares.
 
@@ -89,6 +89,20 @@ Each pod of a cluster shares the kernel of the Docker VM. Thus each stamp of the
 
 Before each run, the lab deletes the conntrack entries in SYN_SENT to the ClusterIP on each node (`forgetHandshakes`). A note of the run gives each count. Such an entry is a SYN with no answer, and it keeps its NAT for 120 s. A new SYN from the same port goes where the first one went, and it gets no answer either. The clients send requests between the runs too. Thus the restore of one run left such entries, and a client that took the port again in the next run timed out (T07, 2026-10-17).
 
+## The stacks
+
+From 0.2.0, a cell names its stack (`stack`): its server and its clients. A cell without a stack has the service.
+
+- **`service`** is the service of this repository with its declaration, and the HTTP clients. Each request opens a new connection, thus kube-proxy picks a backend for each request.
+- **`grpc`** is the server of grpc-go v1.84.0 from the image of the harness, and the gRPC clients of the lab image. A client holds one channel. Thus kube-proxy picks a backend only when grpc-js connects again: after a GOAWAY of the server, or after the end of the socket.
+  - The probes are the gRPC probes of the kubelet on the port of the server.
+  - The lines of grpc-go name their pod and stamp in milliseconds (`--name`, `--clock ms`). Its line `stopping` gives the intake and the SIGTERM of a pod (`stoppingOf`): then GracefulStop closes the listener and sends GOAWAY.
+  - The clients start after the server is ready. Each failed connect of a channel makes the backoff of grpc-js longer, up to 120 s.
+  - The target of a run is the replica that answers the most clients. A channel stays on its replica, and the channels go to the replica that stays after a run. Thus a target by name has no client in most runs.
+  - The cell has no declaration (`declaration` is null), thus `describeCell` refuses it. The server grpc-go has no seam for `liveness-restart`, thus its manifest refuses that trigger.
+
+A declaration that can be null changes the type `KindCell`. That is a breaking change before 1.0.0, thus the version is 0.2.0.
+
 ## The image
 
 The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`, its clients, the agent and the watcher, on Node 24.21.0. Its base is pinned by digest, and `npm-lock.json` pins the packages from npm. `stageLabImage()` stages its context, and `labImage()` tags the image by its content.
@@ -101,6 +115,7 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
 - **The knobs of the listener.** `server_idle_close` is 6 s, the real idle close of Node 24.21: `keepAliveTimeout` of 5 s and its buffer of 1 s. The sweep of idle connections comes each second after the stop of the intake.
 - **The seams** of the service are a flag file that fails its liveness check, and SIGUSR2. SIGUSR2 blocks its event loop for the time in a file.
 - **The clients** (`client.mjs`) send requests on new connections to the ClusterIP. Each request gives one line with its outcome: ok, 503, refused, reset, timeout or unreachable. From 0.1.3, each line also gives the local port of its connection (`lp`), a refused one's too. A conntrack entry is per port. Thus the requests on the port of a failure show the connection before it, and the pod that it went to.
+- **The gRPC clients** (`client-grpc.mjs`) hold one channel of grpc-js 1.14.6 to the ClusterIP, and send a call of `grpc.health.v1.Health/Check` each interval on it. Each call gives the line of a request of `client.mjs`, with the pod that answered (the header `x-pod`). A call that fails at once names the errno of the last connect: refused, reset or unreachable. A call of grpc-js has no deadline at its defaults. Thus a call with no answer in the timeout is a timeout, and the client cancels it.
 
 ## The triggers
 
@@ -111,7 +126,7 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
 | `pod-deletion` | `kubectl delete pod` of the first replica |
 | `rolling-update` | a new value of an environment variable of the Deployment |
 | `node-drain` | `kubectl drain` of the node of the first replica, then `uncordon` |
-| `scale-in` | one replica less, then back |
+| `scale-in` | one replica less, then back. For the stack `grpc`, the first replica first gets a deletion cost of -1, thus its ReplicaSet removes it |
 | `crash` | SIGKILL of the process of the service, from the node |
 | `liveness-restart` | the flag file that fails the liveness check of the first replica |
 | `freeze-process` | SIGSTOP and SIGCONT: of kube-proxy on one node across a deletion (T07), or of Redis (T18) |
@@ -139,6 +154,15 @@ The plan's `event:clock-jump` has no trigger. A step of CLOCK_REALTIME in the VM
 
 From 0.1.8, a run can agree with the law but be past its calibrated band (D-EXTEND). A request failed after the window, but before the rules of its node dropped the target. The check of that run tells it (`tail`), and its cell is `extend`. Such a cell does not hold. A request after the drop is still late, thus its run fails the cell.
 
+From 0.2.1, a window can be one window for each deleted replica (`each`). A rolling update deletes each old replica in turn, and a client of gRPC can see each of these deletions.
+
+- The judge makes one window for each replica of the run (`replicas`) with a deletion after the start of the run. Each window counts from the intake and the deletion of its own replica.
+- A request is late only when it is outside each of these windows. A request after a window and before the rules of its node dropped the replica of that window is a `tail`.
+- The facts of each run give these replicas, with the stamps of their intake and their deletion (`deletions`).
+- `predictionText` adds `of each deleted replica`. A window without `each` stays the window of the target.
+
+A preStop sleep that is longer than the grace is a command in the container (`sh -c "sleep 60"`). The API server refuses a sleep action that is longer than the grace.
+
 ## Tests
 
 `pnpm --filter @mark1russell7/kind test` starts the tests of the configuration, the tools, the parser of the rules and the fault mapping. It also starts the tests of the predictions, the verdict and the judge of the real records. They do not need Docker.

package/build/index.d.ts

@@ -1,4 +1,4 @@
-import { $ as RulesLine, $t as cidrOverlap, A as TriggerV1, At as secondsOf, B as RunFacts, Bt as Reference, C as KindCellResult, Ct as HEALTH_PORT, D as PredictionV1, Dt as SERVER, E as LoadSample, Et as REDIS, F as KindDecision, Ft as KindRun, G as referenceOf, Gt as TriggerKind, H as factsOf, Ht as ResetReuse, I as KindVerdict, It as KubeletProbe, J as Endpoint, Jt as KindConfig, K as reusesReset, Kt as Workload, L as LAST_SLACK, Lt as PodSettings, M as upgrade, Mt as Band, N as upgradeCell, Nt as Declaration, O as RunCheckV1, Ot as bandOf, P as KIND_NOISE, Pt as KindCell, Q as Request, Qt as NodeImage, R as Propagated, Rt as Prediction, S as KindCellRecordV1, St as CLIENT, T as KindVerdictV1, Tt as PORT, U as judgeKind, Ut as Template, V as checkRun, Vt as Replica, W as propagationOf, Wt as Trigger, X as PodLine, Xt as KubeProxyMode, Y as Line, Yt as KindScenario, Z as RecordLine, Zt as NODE_IMAGES, _ as ClusterManifest, _t as signalsOf, a as STAGES, an as scenario, at as emptyOf, b as FORMAT_V1, bt as slicesOf, c as readRecord, ct as kernelCounts, d as calibrationOf, dt as podsOf, en as cidrRange, et as SliceLine, f as cellsTable, ft as recordsOf, g as BandV1, gt as signalOf, h as timeSpread, ht as rulesOf, i as quantileOf, in as nodeNames, it as dropOf, j as judge, jt as showBand, k as RunFactsV1, kt as seconds, l as Judged, lt as outcomesOf, m as predictionText, mt as restartsAt, n as medianInterval, nn as kindConfig, nt as byNode, o as kindCellRecordSchema, on as scenarioProblems, ot as failuresOf, p as medianText, pt as requestsOf, q as unitOf, qt as KUBE_PROXY_MODES, r as middle, rn as kubeProxyPatch, rt as deletionOf, s as kindCellRecordV1Schema, st as intakeOf, t as RELATIVE, tn as goDuration, tt as backendsAt, u as TimeSpread, ut as podsAt, v as EnvironmentManifest, vt as sliceDropOf, w as KindCellV1, wt as LOG_LABEL, x as KindCellRecord, xt as transitionOf, y as FORMAT, yt as sliceEmptyOf, z as RunCheck, zt as RedisDeclaration } from "./judge-a7yCzwZc.js";
+import { $ as RecordLine, $t as KindConfig, A as TriggerV1, At as REDIS, B as RunCheck, Bt as KubeletProbe, C as KindCellResult, Ct as slicesOf, D as PredictionV1, Dt as HEALTH_PORT, E as LoadSample, Et as CLIENT, F as KIND_NOISE, Ft as showBand, G as judgeKind, Gt as Replica, H as checkRun, Ht as Prediction, I as KindDecision, It as Band, J as reusesReset, Jt as Template, K as propagationOf, Kt as ResetReuse, L as KindVerdict, Lt as Declaration, M as upgrade, Mt as bandOf, N as upgradeCell, Nt as seconds, O as RunCheckV1, Ot as LOG_LABEL, P as Deleted, Pt as secondsOf, Q as PodLine, Qt as KUBE_PROXY_MODES, R as LAST_SLACK, Rt as KindCell, S as KindCellRecordV1, St as sliceEmptyOf, T as KindVerdictV1, Tt as transitionOf, U as deletionsOf, Ut as RedisDeclaration, V as RunFacts, Vt as PodSettings, W as factsOf, Wt as Reference, X as Endpoint, Xt as TriggerKind, Y as unitOf, Yt as Trigger, Z as Line, Zt as Workload, _ as ClusterManifest, _t as restartsAt, a as STAGES, an as cidrRange, at as byNode, b as FORMAT_V1, bt as signalsOf, c as readRecord, cn as kubeProxyPatch, ct as emptyOf, d as calibrationOf, dn as scenarioProblems, dt as kernelCounts, en as KindScenario, et as Request, f as cellsTable, ft as outcomesOf, g as BandV1, gt as requestsOf, h as timeSpread, ht as recordsOf, i as quantileOf, in as cidrOverlap, it as byClients, j as judge, jt as SERVER, k as RunFactsV1, kt as PORT, l as Judged, ln as nodeNames, lt as failuresOf, m as predictionText, mt as podsOf, n as medianInterval, nn as NODE_IMAGES, nt as SliceLine, o as kindCellRecordSchema, on as goDuration, ot as deletionOf, p as medianText, pt as podsAt, q as referenceOf, qt as Stack, r as middle, rn as NodeImage, rt as backendsAt, s as kindCellRecordV1Schema, sn as kindConfig, st as dropOf, t as RELATIVE, tn as KubeProxyMode, tt as RulesLine, u as TimeSpread, un as scenario, ut as intakeOf, v as EnvironmentManifest, vt as rulesOf, w as KindCellV1, wt as stoppingOf, x as KindCellRecord, xt as sliceDropOf, y as FORMAT, yt as signalOf, z as Propagated, zt as KindRun } from "./judge-Dwa6MqP4.js";
 import { Quantity } from "@mark1russell7/quantity";
 import { CheckFn, Finding, HealthServer, Poll, Rule, Scenario, ServiceBuilder, ServiceDescription, Stage, Timing, TransportImpl, Verdict } from "@mark1russell7/service";
 import { RedisClient } from "@mark1russell7/service/redis";
@@ -205,17 +205,37 @@ export declare const openKindLab: (cluster: Cluster, image: string) => Promise<K
 export declare const localPin: (image: string) => Promise<string>;
 //#endregion
 //#region src/manifests.d.ts
+/**
+ * This module generates the Kubernetes objects of a cell. They are its namespace, its Redis, the
+ * server as a Deployment behind a Service, and one client on each node. The server is the service
+ * or grpc-go, by the stack of the cell. Each setting of the pod comes from the cell. The rollout
+ * strategy and the probe fields state the defaults of Kubernetes as values.
+ */
+/** This function gives the stack of a cell: the service, when the cell names no stack. */
+export declare const stackOf: (cell: Pick<KindCell, "stack">) => Stack;
 /** This image is Redis 7.4, the stock target of the harness, pinned by digest. */
 export declare const REDIS_IMAGE = "redis:7.4@sha256:c6eabf748fc7a61dbb5a705c78bcf3d6377b1127a97d0ce965c11c44ba46896f";
 /** This function gives the namespace of a cell: `lab-` and 10 digits of the sha256 of its name. */
 export declare const namespaceOf: (cell: Pick<KindCell, "name">) => string;
-/** This function gives a probe of the pod: an HTTP GET of the health server, with each field of the cell. */
-export declare const probeOf: (path: string, probe: KubeletProbe) => Record<string, unknown>;
+/**
+ * This function gives a probe of the pod, with each field of the cell. The probe of the service is an
+ * HTTP GET of its health server (`path`). The probe of grpc-go (`path` null) is the gRPC probe of the
+ * kubelet. It is a Check of the service "" on the port of the server, and the answer is SERVING until
+ * GracefulStop.
+ */
+export declare const probeOf: (path: string | null, probe: KubeletProbe) => Record<string, unknown>;
 /** These are the objects of Redis: one replica behind a Service, with no persistence. */
 export declare const redisObjects: (namespace: string, image: string) => unknown[];
-/** These are the objects of the service: a Service of type ClusterIP, and the Deployment with the settings of the cell. */
+/**
+ * These are the objects of the server: a Service of type ClusterIP, and the Deployment with the
+ * settings of the cell. The image is the lab image for the service, or the image of grpc-go.
+ */
 export declare const serverObjects: (cell: KindCell, namespace: string, image: string, version?: number) => unknown[];
-/** These are the objects of the clients: a DaemonSet with one client on each node, the control plane too. */
+/**
+ * These are the objects of the clients: a DaemonSet with one client on each node, the control plane
+ * too. The client of the service is `client.mjs`, and the client of grpc-go is `client-grpc.mjs`.
+ * Each one is in the lab image, and each one writes the same lines.
+ */
 export declare const clientObjects: (cell: KindCell, namespace: string, image: string, target: string) => unknown[];
 //#endregion
 //#region src/predict.d.ts
@@ -395,5 +415,5 @@ export declare const toolPath: (tool: Tool, key?: string, cache?: string) => str
  */
 export declare const ensureTool: (tool: Tool, key?: string, cache?: string) => Promise<string>;
 //#endregion
-export { type Band, type BandV1, CLIENT, type Cluster, type ClusterManifest, type Credentials, type Declaration, type DeletionWindow, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, Line, type LoadSample, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, type Stamp, type Step, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, type Verdicts, type Workload, backendsAt, bandOf, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, outcomesOf, podsAt, podsOf, predictionText, propagationOf, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, timeSpread, transitionOf, unitOf, upgrade, upgradeCell };
+export { type Band, type BandV1, CLIENT, type Cluster, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, Line, type LoadSample, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, type Stack, type Stamp, type Step, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, type Verdicts, type Workload, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, outcomesOf, podsAt, podsOf, predictionText, propagationOf, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stoppingOf, timeSpread, transitionOf, unitOf, upgrade, upgradeCell };
 //# sourceMappingURL=index.d.ts.map

package/build/index.js

@@ -1,4 +1,4 @@
-import { $ as REDIS, A as deletionOf, B as requestsOf, C as unitOf, D as quantileOf, E as middle, F as kernelCounts, G as sliceDropOf, H as rulesOf, I as outcomesOf, J as transitionOf, K as sliceEmptyOf, L as podsAt, M as emptyOf, N as failuresOf, O as backendsAt, P as intakeOf, Q as PORT, R as podsOf, S as reusesReset, T as medianInterval, U as signalOf, V as restartsAt, W as signalsOf, X as HEALTH_PORT, Y as CLIENT, Z as LOG_LABEL, _ as checkRun, a as calibrationOf, b as propagationOf, c as predictionText, d as FORMAT_V1, et as SERVER, f as judge, g as LAST_SLACK, h as KIND_NOISE, i as readRecord, it as showBand, j as dropOf, k as byNode, l as timeSpread, m as upgradeCell, n as kindCellRecordSchema, nt as seconds, o as cellsTable, p as upgrade, q as slicesOf, r as kindCellRecordV1Schema, rt as secondsOf, s as medianText, t as STAGES, tt as bandOf, u as FORMAT, v as factsOf, w as RELATIVE, x as referenceOf, y as judgeKind, z as recordsOf } from "./judge-BbTAxRSd.js";
+import { $ as HEALTH_PORT, A as byClients, B as podsOf, C as reusesReset, D as middle, E as medianInterval, F as failuresOf, G as signalOf, H as requestsOf, I as intakeOf, J as sliceEmptyOf, K as signalsOf, L as kernelCounts, M as deletionOf, N as dropOf, O as quantileOf, P as emptyOf, Q as CLIENT, R as outcomesOf, S as referenceOf, T as RELATIVE, U as restartsAt, V as recordsOf, W as rulesOf, X as stoppingOf, Y as slicesOf, Z as transitionOf, _ as checkRun, a as calibrationOf, at as seconds, b as judgeKind, c as predictionText, d as FORMAT_V1, et as LOG_LABEL, f as judge, g as LAST_SLACK, h as KIND_NOISE, i as readRecord, it as bandOf, j as byNode, k as backendsAt, l as timeSpread, m as upgradeCell, n as kindCellRecordSchema, nt as REDIS, o as cellsTable, ot as secondsOf, p as upgrade, q as sliceDropOf, r as kindCellRecordV1Schema, rt as SERVER, s as medianText, st as showBand, t as STAGES, tt as PORT, u as FORMAT, v as deletionsOf, w as unitOf, x as propagationOf, y as factsOf, z as podsAt } from "./judge-reWPMVr4.js";
 import { valueIn } from "@mark1russell7/quantity";
 import { execFile, execFileSync } from "node:child_process";
 import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, realpathSync, renameSync, rmSync, writeFileSync } from "node:fs";
@@ -9,7 +9,7 @@ import { fileURLToPath } from "node:url";
 import { failSpan, lint, service } from "@mark1russell7/service";
 import { createRequire } from "node:module";
 import { tmpdir } from "node:os";
-import { harnessImage } from "@mark1russell7/harness";
+import { harnessImage, imageOf } from "@mark1russell7/harness";
 import { redisClient } from "@mark1russell7/service/redis";
 //#region src/backends.ts
 const hexIp = (hex) => [
@@ -520,6 +520,8 @@ const POD = "{pod}";
 const NODE = "{node}";
 /** This value marks the next version of the Deployment, for a rollout. */
 const VERSION = "{version}";
+/** This annotation of a pod is its deletion cost for its ReplicaSet: a lower cost goes first at a scale-in. */
+const DELETION_COST = "controller.kubernetes.io/pod-deletion-cost";
 /** This function gives the fault plan of a cell. */
 const planOf = (cell) => {
 	const ns = ["-n", "{namespace}"];
@@ -577,7 +579,17 @@ const planOf = (cell) => {
 			}]
 		};
 		case "scale-in": return {
-			inject: [{
+			inject: [...cell.stack === "grpc" ? [{
+				do: "kubectl",
+				args: [
+					...ns,
+					"annotate",
+					"pod",
+					POD,
+					`${DELETION_COST}=-1`,
+					"--overwrite"
+				]
+			}] : [], {
 				do: "kubectl",
 				args: [
 					...ns,
@@ -800,6 +812,7 @@ const FILES = [
 	["npm-lock.json", "image/npm-lock.json"],
 	["app.mjs", "image/app.mjs"],
 	["client.mjs", "image/client.mjs"],
+	["client-grpc.mjs", "image/client-grpc.mjs"],
 	["agent.mjs", "image/agent.mjs"],
 	["watch.mjs", "image/watch.mjs"],
 	["compose.ts", "src/compose.ts"],
@@ -1220,6 +1233,14 @@ const localPin = async (image) => {
 };
 //#endregion
 //#region src/manifests.ts
+/**
+* This module generates the Kubernetes objects of a cell. They are its namespace, its Redis, the
+* server as a Deployment behind a Service, and one client on each node. The server is the service
+* or grpc-go, by the stack of the cell. Each setting of the pod comes from the cell. The rollout
+* strategy and the probe fields state the defaults of Kubernetes as values.
+*/
+/** This function gives the stack of a cell: the service, when the cell names no stack. */
+const stackOf = (cell) => cell.stack ?? "service";
 /** This image is Redis 7.4, the stock target of the harness, pinned by digest. */
 const REDIS_IMAGE = "redis:7.4@sha256:c6eabf748fc7a61dbb5a705c78bcf3d6377b1127a97d0ce965c11c44ba46896f";
 const whole = (amount, what) => {
@@ -1229,18 +1250,28 @@ const whole = (amount, what) => {
 };
 /** This function gives the namespace of a cell: `lab-` and 10 digits of the sha256 of its name. */
 const namespaceOf = (cell) => `lab-${createHash("sha256").update(cell.name).digest("hex").slice(0, 10)}`;
-/** This function gives a probe of the pod: an HTTP GET of the health server, with each field of the cell. */
+/**
+* This function gives a probe of the pod, with each field of the cell. The probe of the service is an
+* HTTP GET of its health server (`path`). The probe of grpc-go (`path` null) is the gRPC probe of the
+* kubelet. It is a Check of the service "" on the port of the server, and the answer is SERVING until
+* GracefulStop.
+*/
 const probeOf = (path, probe) => ({
-	httpGet: {
+	...path === null ? { grpc: { port: PORT } } : { httpGet: {
 		path,
 		port: HEALTH_PORT
-	},
+	} },
 	initialDelaySeconds: 0,
-	periodSeconds: whole(probe.period, `the period of ${path}`),
-	timeoutSeconds: whole(probe.timeout, `the timeout of ${path}`),
+	periodSeconds: whole(probe.period, `the period of ${path ?? "the gRPC probe"}`),
+	timeoutSeconds: whole(probe.timeout, `the timeout of ${path ?? "the gRPC probe"}`),
 	failureThreshold: probe.threshold,
 	successThreshold: 1
 });
+/** This variable is the name of the pod, from the API: the service and grpc-go put it in their lines. */
+const POD_ENV = {
+	name: "POD_NAME",
+	valueFrom: { fieldRef: { fieldPath: "metadata.name" } }
+};
 /** This function gives the environment of the app: its config as JSON, and the name of its pod. */
 const appEnv = (cell) => [{
 	name: "CONFIG",
@@ -1250,10 +1281,44 @@ const appEnv = (cell) => [{
 		healthPort: HEALTH_PORT,
 		declaration: cell.declaration
 	})
-}, {
-	name: "POD_NAME",
-	valueFrom: { fieldRef: { fieldPath: "metadata.name" } }
-}];
+}, POD_ENV];
+/**
+* This function gives the container of the server of a cell. The service runs `app.mjs` of the lab
+* image with its declaration. The server grpc-go runs from the image of the harness. Its lines name
+* the pod and stamp in milliseconds. Its replies name the pod in the header `x-pod`. Kubernetes
+* expands `$(POD_NAME)` in the command.
+*/
+const serverContainer = (cell, version) => {
+	const env = {
+		name: "LAB_VERSION",
+		value: String(version)
+	};
+	if (stackOf(cell) === "service") return {
+		command: ["node", "app.mjs"],
+		env: [...appEnv(cell), env],
+		ports: [{ containerPort: PORT }, { containerPort: HEALTH_PORT }],
+		...cell.pod.startup === null ? {} : { startupProbe: probeOf("/startup", cell.pod.startup) },
+		readinessProbe: probeOf("/readiness", cell.pod.readiness),
+		livenessProbe: probeOf("/liveness", cell.pod.liveness)
+	};
+	if (cell.trigger.kind === "liveness-restart") throw new Error(`${cell.name}: grpc-go has no seam for liveness-restart`);
+	return {
+		command: [
+			"grpc-go-server",
+			"--port",
+			String(PORT),
+			"--name",
+			"$(POD_NAME)",
+			"--clock",
+			"ms"
+		],
+		env: [POD_ENV, env],
+		ports: [{ containerPort: PORT }],
+		...cell.pod.startup === null ? {} : { startupProbe: probeOf(null, cell.pod.startup) },
+		readinessProbe: probeOf(null, cell.pod.readiness),
+		livenessProbe: probeOf(null, cell.pod.liveness)
+	};
+};
 /** These are the objects of Redis: one replica behind a Service, with no persistence. */
 const redisObjects = (namespace, image) => [{
 	apiVersion: "apps/v1",
@@ -1308,9 +1373,29 @@ const redisObjects = (namespace, image) => [{
 		}]
 	}
 }];
-/** These are the objects of the service: a Service of type ClusterIP, and the Deployment with the settings of the cell. */
+/**
+* This function gives the preStop hook of a sleep, or nothing for no sleep. The hook is the native
+* sleep action of the kubelet. The API server refuses a sleep action that is longer than the grace
+* (`validateSleepAction`, `pkg/apis/core/validation/validation.go` at v1.37.0, line 3483). Thus a
+* longer sleep is a command in the container, `sleep` from `/bin/sh`, as public manifests write it.
+* The kubelet starts that command with no timeout (`pkg/kubelet/lifecycle/handlers.go`, line 81).
+*/
+const preStopOf = (preStop, grace) => {
+	if (preStop === 0) return {};
+	if (preStop <= grace) return { lifecycle: { preStop: { sleep: { seconds: preStop } } } };
+	return { lifecycle: { preStop: { exec: { command: [
+		"/bin/sh",
+		"-c",
+		`sleep ${preStop}`
+	] } } } };
+};
+/**
+* These are the objects of the server: a Service of type ClusterIP, and the Deployment with the
+* settings of the cell. The image is the lab image for the service, or the image of grpc-go.
+*/
 const serverObjects = (cell, namespace, image, version = 1) => {
 	const preStop = whole(cell.pod.preStop, "the preStop sleep");
+	const grace = whole(cell.pod.grace, "the grace period");
 	return [{
 		apiVersion: "v1",
 		kind: "Service",
@@ -1321,7 +1406,7 @@ const serverObjects = (cell, namespace, image, version = 1) => {
 		spec: {
 			selector: { app: "lab" },
 			ports: [{
-				name: "http",
+				name: stackOf(cell) === "grpc" ? "grpc" : "http",
 				port: 80,
 				targetPort: PORT
 			}]
@@ -1352,7 +1437,7 @@ const serverObjects = (cell, namespace, image, version = 1) => {
 					"lab.systems/version": String(version)
 				} },
 				spec: {
-					terminationGracePeriodSeconds: whole(cell.pod.grace, "the grace period"),
+					terminationGracePeriodSeconds: grace,
 					enableServiceLinks: true,
 					topologySpreadConstraints: [{
 						maxSkew: 1,
@@ -1364,23 +1449,19 @@ const serverObjects = (cell, namespace, image, version = 1) => {
 						name: "app",
 						image,
 						imagePullPolicy: "Never",
-						command: ["node", "app.mjs"],
-						env: [...appEnv(cell), {
-							name: "LAB_VERSION",
-							value: String(version)
-						}],
-						ports: [{ containerPort: PORT }, { containerPort: HEALTH_PORT }],
-						...cell.pod.startup === null ? {} : { startupProbe: probeOf("/startup", cell.pod.startup) },
-						readinessProbe: probeOf("/readiness", cell.pod.readiness),
-						livenessProbe: probeOf("/liveness", cell.pod.liveness),
-						...preStop === 0 ? {} : { lifecycle: { preStop: { sleep: { seconds: preStop } } } }
+						...serverContainer(cell, version),
+						...preStopOf(preStop, grace)
 					}]
 				}
 			}
 		}
 	}];
 };
-/** These are the objects of the clients: a DaemonSet with one client on each node, the control plane too. */
+/**
+* These are the objects of the clients: a DaemonSet with one client on each node, the control plane
+* too. The client of the service is `client.mjs`, and the client of grpc-go is `client-grpc.mjs`.
+* Each one is in the lab image, and each one writes the same lines.
+*/
 const clientObjects = (cell, namespace, image, target) => [{
 	apiVersion: "apps/v1",
 	kind: "DaemonSet",
@@ -1407,7 +1488,7 @@ const clientObjects = (cell, namespace, image, target) => [{
 					name: CLIENT,
 					image,
 					imagePullPolicy: "Never",
-					command: ["node", "client.mjs"],
+					command: ["node", stackOf(cell) === "grpc" ? "client-grpc.mjs" : "client.mjs"],
 					env: [
 						{
 							name: "TARGET",
@@ -1694,6 +1775,7 @@ const lintCell = (description, cell, propagation) => lint(description, scenarioO
 const verdictsOf = (findings) => Object.fromEntries(findings.map((finding) => [finding.rule, finding.verdict]));
 /** This function gives the description of the service of a cell, as the app of the image composes it. Nothing connects. */
 const describeCell = (cell) => {
+	if (cell.declaration === null) throw new Error(`${cell.name}: the server is not the service, thus the cell has no declaration to describe`);
 	const redis = cell.declaration.redis;
 	const client = redis === null ? void 0 : redisClient("redis", {
 		host: "redis",
@@ -1713,12 +1795,16 @@ const RULES_EVERY = {
 };
 const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
 const slug = (name) => name.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-|-$/g, "");
+/** This function tells if a cell has Redis: only a declaration of the service can have it. */
+const hasRedis = (cell) => cell.declaration !== null && cell.declaration.redis !== null;
 /** This tracker keeps the latest state of the cell from the lines: each pod, each slice, the backends of each node and each client. */
 var Tracker = class {
 	pods = /* @__PURE__ */ new Map();
 	slices = /* @__PURE__ */ new Map();
 	backends = /* @__PURE__ */ new Map();
 	clients = /* @__PURE__ */ new Map();
+	/** This map is the backend of the last answer of each client, by client. */
+	answers = /* @__PURE__ */ new Map();
 	last = 0;
 	see = (line) => {
 		if (typeof line.t === "number" && line.t > this.last && line.ev === "req") this.last = line.t;
@@ -1740,6 +1826,7 @@ var Tracker = class {
 			if (request.o === "ok") client.ok = Math.max(client.ok, request.t);
 			else client.failed = Math.max(client.failed, request.t);
 			this.clients.set(request.p, client);
+			if (request.o === "ok" && request.b !== void 0) this.answers.set(request.p, request.b);
 		}
 	};
 	servers() {
@@ -1754,7 +1841,7 @@ const unsteady = (ctx) => {
 	const busy = servers.find((pod) => pod.del || !pod.ready);
 	if (busy !== void 0) return `${busy.name} is not ready`;
 	if (new Set(servers.map((pod) => pod.labels["pod-template-hash"])).size > 1) return "two versions of the replicas";
-	if (cell.declaration.redis !== null) {
+	if (hasRedis(cell)) {
 		const redis = [...tracker.pods.values()].filter((pod) => pod.labels["app"] === REDIS);
 		if (redis.length !== 1 || !redis[0].ready || redis[0].del) return "Redis is not ready";
 	}
@@ -1819,16 +1906,20 @@ const quiet = async (ctx, ours) => {
 		await sleep(3e4);
 	}
 };
-/** This function deploys a cell in its namespace, with the watcher, the poll of the rules and the clients. */
-const deploy = async (lab, cluster, cell, namespace, redisImage) => {
+/**
+* This function deploys a cell in its namespace, with the watcher, the poll of the rules and the
+* clients. A client of gRPC starts after the server is ready: each failed connect of a channel makes
+* the backoff of grpc-js longer, up to 120 s.
+*/
+const deploy = async (lab, cluster, cell, namespace, images) => {
 	await cluster.apply([{
 		apiVersion: "v1",
 		kind: "Namespace",
 		metadata: { name: namespace }
 	}]);
 	await lab.watch([namespace]);
-	if (cell.declaration.redis !== null) {
-		await cluster.apply(redisObjects(namespace, redisImage));
+	if (hasRedis(cell)) {
+		await cluster.apply(redisObjects(namespace, images.redis));
 		await cluster.kubectl([
 			"-n",
 			namespace,
@@ -1838,7 +1929,9 @@ const deploy = async (lab, cluster, cell, namespace, redisImage) => {
 			"--timeout=180s"
 		]);
 	}
-	await cluster.apply(serverObjects(cell, namespace, lab.image));
+	const grpc = stackOf(cell) === "grpc";
+	if (grpc && images.grpc === void 0) throw new Error(`${cell.name}: the cluster has no image of grpc-go`);
+	await cluster.apply(serverObjects(cell, namespace, grpc ? images.grpc : lab.image));
 	const clusterIp = (await cluster.kubectl([
 		"-n",
 		namespace,
@@ -1848,7 +1941,8 @@ const deploy = async (lab, cluster, cell, namespace, redisImage) => {
 		"-o",
 		"jsonpath={.spec.clusterIP}"
 	])).trim();
-	await cluster.apply(clientObjects(cell, namespace, lab.image, `${clusterIp}:80`));
+	const clients = clientObjects(cell, namespace, lab.image, `${clusterIp}:80`);
+	if (!grpc) await cluster.apply(clients);
 	await cluster.kubectl([
 		"-n",
 		namespace,
@@ -1857,6 +1951,7 @@ const deploy = async (lab, cluster, cell, namespace, redisImage) => {
 		`deployment/lab`,
 		"--timeout=300s"
 	]);
+	if (grpc) await cluster.apply(clients);
 	await cluster.kubectl([
 		"-n",
 		namespace,
@@ -1889,13 +1984,21 @@ const probeThePods = async (ctx) => {
 	await ctx.lab.probe([...new Set(netns)]);
 	ctx.probed = key;
 };
-/** This function gives the replicas of the service at this time, in the sequence of their names. */
-const replicasOf = (ctx) => ctx.tracker.servers().map((pod) => ({
-	pod: pod.name,
-	node: pod.node ?? "",
-	ip: pod.ip ?? "",
-	version: pod.labels["pod-template-hash"] ?? ""
-}));
+/**
+* This function gives the replicas of the server at this time, in the sequence of their names. The
+* first one is the target of a run. A client of gRPC holds its connection to one replica, and the
+* channels go to the replica that stays after a run. Thus for the stack `grpc`, the first replica
+* is the one that answers the most clients (`byClients`).
+*/
+const replicasOf = (ctx) => {
+	const replicas = ctx.tracker.servers().map((pod) => ({
+		pod: pod.name,
+		node: pod.node ?? "",
+		ip: pod.ip ?? "",
+		version: pod.labels["pod-template-hash"] ?? ""
+	}));
+	return stackOf(ctx.cell) === "grpc" ? byClients(replicas, ctx.tracker.answers) : replicas;
+};
 /** This function does one step of a fault plan. It gives the stamp of its act, or null for an act without a stamp. */
 const doStep = async (ctx, step, replicas, background) => {
 	const { lab, cluster, namespace } = ctx;
@@ -2063,7 +2166,7 @@ const runOnce = async (ctx, env, index, warmup, ours) => {
 	if (await steady(ctx, 3e5) !== null) return empty("the cell was not at rest after the wait for a quiet VM", await lab.now());
 	const replicas = replicasOf(ctx);
 	const restarts = new Map(ctx.tracker.servers().map((pod) => [pod.name, pod.cs.reduce((sum, status) => sum + status.rc, 0)]));
-	const target = cell.declaration.redis !== null && [
+	const target = hasRedis(cell) && [
 		"rst",
 		"blackhole",
 		"freeze-process",
@@ -2206,6 +2309,15 @@ const runKindCells = async (cells, options) => {
 	const environments = options.environments ?? 2;
 	const image = await labImage();
 	const redisImage = await localPin(REDIS_IMAGE);
+	const grpcImage = cells.some((cell) => stackOf(cell) === "grpc") ? await imageOf({
+		kind: "shipped",
+		name: "grpc-go"
+	}) : void 0;
+	const images = {
+		lab: image,
+		redis: redisImage,
+		grpc: grpcImage
+	};
 	const groups = /* @__PURE__ */ new Map();
 	for (const cell of cells) groups.set(`${cell.mode}-${cell.workers}`, [...groups.get(`${cell.mode}-${cell.workers}`) ?? [], cell]);
 	const results = [];
@@ -2221,6 +2333,7 @@ const runKindCells = async (cells, options) => {
 			try {
 				await cluster.load(image);
 				await cluster.load(redisImage);
+				if (grpcImage !== void 0 && group.some((cell) => stackOf(cell) === "grpc")) await cluster.load(grpcImage);
 				lab = await openKindLab(cluster, image);
 				const { GenericContainer } = await import("testcontainers");
 				const Shared = class extends GenericContainer {
@@ -2239,7 +2352,7 @@ const runKindCells = async (cells, options) => {
 					const off = lab.listen(tracker.see);
 					const loads = [];
 					try {
-						const clusterIp = await deploy(lab, cluster, cell, namespace, redisImage);
+						const clusterIp = await deploy(lab, cluster, cell, namespace, images);
 						for (const name of [...lab.own(), loader.getName().replace(/^\//, "")]) ours.add(name);
 						const ctx = {
 							lab,
@@ -2271,7 +2384,8 @@ const runKindCells = async (cells, options) => {
 							cluster: manifest,
 							images: {
 								lab: image,
-								redis: redisImage
+								redis: redisImage,
+								...stackOf(cell) === "grpc" ? { grpc: grpcImage } : {}
 							}
 						});
 						entry.loads.push(...loads);
@@ -2360,6 +2474,6 @@ const shardOf = (cells, index, count) => {
 	return out;
 };
 //#endregion
-export { CLIENT, CONTROLLER, DOCUMENTED, DRAIN_RULES, FLAG_CHECK, FORMAT, FORMAT_V1, HEALTH_PORT, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LOG_LABEL, NODE_IMAGES, O04, O09, O10, O16, PORT, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, WATCHDOG, backendsAt, backendsOf, bandOf, byNode, cacheFolder, calibrationOf, capOf, cellsTable, checkRun, cidrOverlap, cidrRange, clientObjects, compose, createCluster, deletes, deletionOf, deletionWindow, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, goDuration, intakeAfter, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, lintCell, localPin, medianInterval, medianText, middle, namespaceOf, nodeNames, openKindLab, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, predictionText, probeOf, propagationOf, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverObjects, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stageLabImage, stallRemovalBand, timeSpread, toolPath, transitionOf, unitOf, upgrade, upgradeCell, verdictsOf, writeResult };
+export { CLIENT, CONTROLLER, DOCUMENTED, DRAIN_RULES, FLAG_CHECK, FORMAT, FORMAT_V1, HEALTH_PORT, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LOG_LABEL, NODE_IMAGES, O04, O09, O10, O16, PORT, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, WATCHDOG, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, checkRun, cidrOverlap, cidrRange, clientObjects, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, goDuration, intakeAfter, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, lintCell, localPin, medianInterval, medianText, middle, namespaceOf, nodeNames, openKindLab, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, predictionText, probeOf, propagationOf, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverObjects, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stackOf, stageLabImage, stallRemovalBand, stoppingOf, timeSpread, toolPath, transitionOf, unitOf, upgrade, upgradeCell, verdictsOf, writeResult };
 
 //# sourceMappingURL=index.js.map