@mark1russell7/kind 0.10.1 → 0.11.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +13 -0
- package/build/index.d.ts +40 -3
- package/build/index.js +95 -8
- package/build/index.js.map +1 -1
- package/build/{judge-BbLWVTYj.js → judge-Bb4ebE9V.js} +8 -4
- package/build/{judge-BbLWVTYj.js.map → judge-Bb4ebE9V.js.map} +1 -1
- package/build/{judge-BD4UVQfy.d.ts → judge-C3mbgYxY.d.ts} +30 -4
- package/build/judge.d.ts +2 -2
- package/build/judge.js +1 -1
- package/image/app.mjs +11 -4
- package/image/client.mjs +148 -10
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -150,6 +150,13 @@ A new member of the union `Template` breaks a reader that switches on it. That i
|
|
|
150
150
|
|
|
151
151
|
The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`, its clients, the agent and the watcher, on Node 24.21.0. Its base is pinned by digest, and `npm-lock.json` pins the packages from npm. `stageLabImage()` stages its context, and `labImage()` gives the tag of the image.
|
|
152
152
|
|
|
153
|
+
- **The keep-alive clients.** A workload with `connections` gives each HTTP client that count of keep-alive connections. Each request goes on the next connection in turn. kube-proxy selects a backend only for a new connection. SIGHUP gives the client new connections, and its old connections close after the timeout. With no `connections`, each request has a new connection, as before.
|
|
154
|
+
- **The lame duck.** The declaration has the optional field `lameDuck`. With true or no value, the edge sends `Connection: close` while the service is not ready (`httpTransport` as it is). With false, the app gives the transport `keepAlive: () => true`, thus each connection stays open and a caller learns only from the endpoints.
|
|
155
|
+
- **HTTP/2.** The declaration has the optional field `protocol` (`EdgeProtocol`). With `h2c`, the app listens with `node:http2` through the same `httpTransport`, and the lame duck is a GOAWAY on each session. The manifests give each client `PROTOCOL=h2c`. The client then keeps `connections` sessions, and sends each request as a stream on the next session in turn.
|
|
156
|
+
- A GOAWAY, a close or an error of a session makes its slot open a new session at the next stream. Then kube-proxy selects a backend for that new connection.
|
|
157
|
+
- The client sends a stream that the server did not take again, once, on a new session (RFC 9113, section 8.7). Its line has `r: 1`.
|
|
158
|
+
- Each stream writes the same `req` line as a request of HTTP/1.1, with the local port of the socket of its session.
|
|
159
|
+
|
|
153
160
|
- **The resolver.** From 0.10.1, the lab image is a `build` source of the harness (`labSource()`). The resolver of the harness plans it and builds it, as it does each other image (`planOf`, `imageOf`). Thus its tag comes from its content, and each `FROM` has a digest. Each manifest of packages has a lockfile, and a failed build tries again after each wait. The build goes through the CLI of Docker, not through Testcontainers.
|
|
154
161
|
|
|
155
162
|
- **In the workspace** of this repository, the stage builds `@mark1russell7/quantity`, `@mark1russell7/algebra` and `@mark1russell7/service`, and packs each one as npm gets it.
|
|
@@ -186,6 +193,7 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
|
|
|
186
193
|
| `authorization-policy` | `kubectl apply` of a DENY AuthorizationPolicy on the server pods for the principal of an account (`account`, the account `default` of the clients when not given), then `kubectl delete` after the hold |
|
|
187
194
|
| `idle` | SIGUSR1 to the gRPC client of each client pod, and SIGUSR2 after the hold: a gap of the calls (the target `clients`). The restore sends SIGUSR2 again. With `middlebox`, a stateful middlebox in each replica from before the gap to the restore. With `close`, a new channel at the fault for 5 s: SIGHUP before the fault, then SIGUSR2, a wait and SIGHUP |
|
|
188
195
|
| `ztunnel-restart` | `kubectl delete pod` of the ztunnel pod of each node, with its grace (`ZTUNNEL_PODS`). The flow of each client crosses the ztunnel of its own node. The restore waits for the rollout of the DaemonSet |
|
|
196
|
+
| `cannot-serve` | SIGHUP to each client before the fault, thus the keep-alive connections spread over the replicas again in each run (`SPREAD_LEAD`, 3 s). Then the flag file of the first replica for the hold. While the flag is there, the route of the replica fails with 503, and the flag check of the declaration fails. The heal and the restore remove the flag |
|
|
189
197
|
| `certificate-expiry` | No act on a connection: the calls of each client go on for the hold. At the fault and at the heal, istioctl reads the leaf certificates of the ztunnel pod of each node (`ztunnelCertificates`). A line `certificates` keeps their validity, the evidence of their lifetime |
|
|
190
198
|
|
|
191
199
|
A cgroup path that is not the scope of one container is refused. A write to the root of the node throttles or freezes each pod of the node.
|
|
@@ -209,6 +217,7 @@ The plan's `event:clock-jump` has no trigger. A step of CLOCK_REALTIME in the VM
|
|
|
209
217
|
- **A reset from a leaving pod** leaves the conntrack entry of its flow in CLOSE for 10 s, with the NAT to that pod. From 0.1.6, a window can give that life (`afterReset`). Then a request outside the window can fail with the outcome of the rule (`reusesReset`). The condition is a reset of the same client on the same port, up to that life before the request. The catalog gives it to a deletion without a hold, in the modes with the NAT in conntrack.
|
|
210
218
|
- **A fault of a shared dependency** trips the readiness latch of each replica (`failSpan`). Then the kubelet needs its threshold of probes, and the rules need one propagation.
|
|
211
219
|
- **A restart through a coupling** adds the liveness probes of the kubelet and the preStop sleep to the latch.
|
|
220
|
+
- **A replica that cannot serve** (`learnWindow`) fails each request on a connection to it from the fault. A turn is the time between two requests on one connection. With the lame duck, the latch closes each connection (`Connection: close`, or GOAWAY on HTTP/2), and the rules select the replica until its removal. Thus the window ends from the fault, in the latch to the removal and one turn. Without the lame duck, no connection opens again, thus the removal changes nothing. The window ends from the heal, in the recovery of the readiness and one turn.
|
|
212
221
|
|
|
213
222
|
`verdict.ts` judges each run, and each cell on its counted runs. Each time lands in its band ± 100 ms: the poll of the rules and the delay of the watcher. A median in a band gets the test of the harness: its order-statistic interval in the band, made 5 % wider.
|
|
214
223
|
|
|
@@ -256,6 +265,10 @@ From 0.8.0, a prediction `ends` can be the end of the tunnel of each client (`of
|
|
|
256
265
|
- The first request of each client after the heal must be `ok`, on a new tunnel. The main value is the middle of the ends, in `s`.
|
|
257
266
|
- From 0.8.1, `ends` has the optional band `answer`. A change to CLOSE_WAIT in that band ± δ also agrees: the peer answers the close of the client, and its FIN comes first. Before the band, the peer closed first.
|
|
258
267
|
|
|
268
|
+
The optional field `protocol` of a declaration, the type `EdgeProtocol` and the mode h2c of the lab image are new features. They keep the old behavior. Before `1.0.0`, a feature is a patch, thus the next version is `0.11.1`. The record format stays `/2`.
|
|
269
|
+
|
|
270
|
+
A new member of the union `TriggerKind`, `cannot-serve`, breaks a reader that switches on it. That is a breaking change before 1.0.0, thus the next version is 0.11.0. The record format stays `/2`, with the optional fields `connections` of a workload and `lameDuck` of a declaration. The line `client` of the reconnect is a new line of a run.
|
|
271
|
+
|
|
259
272
|
A new member of the union `TriggerKind`, `certificate-expiry`, and a new member of the union `Step`, `certificates`, break a reader that switches on them. `KindLab` also has a new method, `record`. Each is a breaking change before 1.0.0, thus the version is 0.9.0. The record format stays `/2`: the lines `certificates` are new lines of a run.
|
|
260
273
|
|
|
261
274
|
In 0.8.1, the middlebox uses the match recent, not the sysctls of conntrack, and `ends` has the optional field `answer`. `SYSCTL` and `NETNS_DEFAULTS` keep only `tcp_retries2`. These fields came in 0.8.0, which was not published. The fix and the optional field are a patch, thus the version is 0.8.1.
|
package/build/index.d.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { $ as updatesOf, $t as Declaration, A as TriggerV1, An as
|
|
1
|
+
import { $ as updatesOf, $t as Declaration, A as TriggerV1, An as nodeNames, At as rulesOf, B as RunCheck, Bt as ztunnelPodsOf, C as KindCellResult, Cn as NODE_IMAGES, Ct as outcomesOf, D as PredictionV1, Dn as goDuration, Dt as requestsOf, E as LoadSample, En as cidrRange, Et as recordsOf, F as KIND_NOISE, Ft as slicesOf, G as judgeKind, Gt as PORT, H as checkRun, Ht as CLIENT, I as KindDecision, It as socketsOf, J as propagationOf, Jt as bandOf, K as observationAfter, Kt as REDIS, L as KindVerdict, Lt as stoppingOf, M as upgrade, Mn as scenarioProblems, Mt as signalsOf, N as upgradeCell, Nt as sliceDropOf, O as RunCheckV1, On as kindConfig, Ot as restartOf, P as Deleted, Pt as sliceEmptyOf, Q as unitOf, Qt as Band, R as LAST_SLACK, Rt as transitionOf, S as KindCellRecordV1, Sn as KubeProxyMode, St as kernelCounts, T as KindVerdictV1, Tn as cidrOverlap, Tt as podsOf, U as deletionsOf, Ut as HEALTH_PORT, V as RunFacts, Vt as ztunnelReplaced, W as factsOf, Wt as LOG_LABEL, X as referenceOf, Xt as secondsOf, Y as propagationSource, Yt as seconds, Z as reusesReset, Zt as showBand, _ as ClusterManifest, _n as WindowPrediction, _t as dropOf, a as STAGES, an as MeshFlavor, at as RecordLine, b as FORMAT_V1, bn as KindConfig, bt as intakeOf, c as readRecord, cn as Prediction, ct as SliceLine, d as calibrationOf, dn as Replica, dt as TunnelLine, en as EdgeProtocol, et as Endpoint, f as cellsTable, fn as ResetReuse, ft as ZtunnelPodLine, g as BandV1, gn as TriggerKind, gt as deletionOf, h as timeSpread, hn as Trigger, ht as byNode, i as quantileOf, in as Mesh, it as PropagationSource, j as judge, jn as scenario, jt as signalOf, k as RunFactsV1, kn as kubeProxyPatch, kt as restartsAt, l as Judged, ln as RedisDeclaration, lt as SocketLine, m as predictionText, mn as Template, mt as byClients, n as medianInterval, nn as KindRun, nt as Line, o as kindCellRecordSchema, on as MeshMode, ot as Request, p as medianText, pn as Stack, pt as backendsAt, q as observationEnd, qt as SERVER, r as middle, rn as KubeletProbe, rt as PodLine, s as kindCellRecordV1Schema, sn as PodSettings, st as RulesLine, t as RELATIVE, tn as KindCell, tt as HBONE_PORT, u as TimeSpread, un as Reference, ut as TCP_STATE, v as EnvironmentManifest, vn as Workload, vt as emptyOf, w as KindCellV1, wn as NodeImage, wt as podsAt, x as KindCellRecord, xn as KindScenario, xt as isTunnelState, y as FORMAT, yn as KUBE_PROXY_MODES, yt as failuresOf, z as Propagated, zt as tunnelsOf } from "./judge-C3mbgYxY.js";
|
|
2
2
|
import { Quantity } from "@mark1russell7/quantity";
|
|
3
3
|
import { CheckFn, Finding, HealthServer, Poll, Rule, Scenario, ServiceBuilder, ServiceDescription, Stage, StopWay, Timing, TransportImpl, Verdict } from "@mark1russell7/service";
|
|
4
4
|
import { BuildImage } from "@mark1russell7/harness";
|
|
@@ -224,6 +224,11 @@ export declare const QUIET_LEAD: Quantity;
|
|
|
224
224
|
export declare const CLOSE_LEAD: Quantity;
|
|
225
225
|
/** This time is how long each client calls on its new channel at the fault of a gap with `close`, before it closes it. */
|
|
226
226
|
export declare const CHANNEL_USE: Quantity;
|
|
227
|
+
/**
|
|
228
|
+
* This time is between the SIGHUP of `cannot-serve` and its fault. Each client opens new connections
|
|
229
|
+
* at the signal, and the requests on its old connections end in this time (the timeout and 100 ms).
|
|
230
|
+
*/
|
|
231
|
+
export declare const SPREAD_LEAD: Quantity;
|
|
227
232
|
/**
|
|
228
233
|
* This function gives the command of the injector that sets one sysctl of its network namespace.
|
|
229
234
|
* Only a sysctl of the namespace (`net.`) and a whole number reach it.
|
|
@@ -564,7 +569,8 @@ export declare const serverObjects: (cell: KindCell, namespace: string, image: s
|
|
|
564
569
|
/**
|
|
565
570
|
* These are the objects of the clients: a DaemonSet with one client on each node, the control plane
|
|
566
571
|
* too. The client of the service is `client.mjs`, and the client of grpc-go is `client-grpc.mjs`.
|
|
567
|
-
* Each one is in the lab image, and each one writes the same lines.
|
|
572
|
+
* Each one is in the lab image, and each one writes the same lines. A service with the protocol
|
|
573
|
+
* `h2c` gives its clients `PROTOCOL`, thus they speak HTTP/2 to it.
|
|
568
574
|
*/
|
|
569
575
|
export declare const clientObjects: (cell: KindCell, namespace: string, image: string, target: string) => unknown[];
|
|
570
576
|
//#endregion
|
|
@@ -639,6 +645,37 @@ export declare const deletionWindow: (declaration: Declaration, preStop: Quantit
|
|
|
639
645
|
export declare const pausedWindow: (declaration: Declaration, preStop: Quantity, lead: Quantity, hold: Quantity, propagation: Propagation) => DeletionWindow & {
|
|
640
646
|
readonly resume: Quantity;
|
|
641
647
|
};
|
|
648
|
+
/**
|
|
649
|
+
* A learn window is what `cannot-serve` predicts: when the callers stop getting errors from a
|
|
650
|
+
* replica that cannot serve. The window starts at the fault, and its end comes from `end.from`.
|
|
651
|
+
*/
|
|
652
|
+
interface LearnWindow {
|
|
653
|
+
/**
|
|
654
|
+
* This time is the interval between two requests on one connection of a client: `every` times the
|
|
655
|
+
* connections. A client of HTTP/2 sends its streams on fewer connections, thus its turn is shorter.
|
|
656
|
+
*/
|
|
657
|
+
readonly turn: Quantity;
|
|
658
|
+
/** This band is the trip of the readiness latch of the service after the fault. */
|
|
659
|
+
readonly latch: Band;
|
|
660
|
+
/** This band is the first failed request after the fault: within one turn. */
|
|
661
|
+
readonly start: Band;
|
|
662
|
+
readonly end: {
|
|
663
|
+
readonly from: Reference;
|
|
664
|
+
readonly band: Band;
|
|
665
|
+
};
|
|
666
|
+
}
|
|
667
|
+
/**
|
|
668
|
+
* This function gives the learn window of a declaration, its readiness probe, a propagation and a
|
|
669
|
+
* workload of keep-alive connections. A caller selects a replica only when it opens a connection:
|
|
670
|
+
* - The lame duck (`lameDuck` true or no value): after the latch, each answer closes its connection,
|
|
671
|
+
* thus each connection of the replica gets one more failure and opens again. The rules select the
|
|
672
|
+
* replica until the removal, thus the last failure comes in one turn after it.
|
|
673
|
+
* - The signal of the lame duck is `Connection: close` on HTTP/1.1, and GOAWAY on HTTP/2. The
|
|
674
|
+
* arithmetic is the same: the turn is `every` times the connections, which are sessions on HTTP/2.
|
|
675
|
+
* - No lame duck: no connection opens again, thus the rules do not matter. The failures end only
|
|
676
|
+
* when the readiness of the service recovers after the heal, and one turn after it.
|
|
677
|
+
*/
|
|
678
|
+
export declare const learnWindow: (declaration: Declaration, readiness: KubeletProbe, propagation: Propagation, workload: Workload) => LearnWindow;
|
|
642
679
|
//#endregion
|
|
643
680
|
//#region src/rules.d.ts
|
|
644
681
|
/**
|
|
@@ -778,5 +815,5 @@ export declare const toolPath: (tool: Tool, key?: string, cache?: string) => str
|
|
|
778
815
|
*/
|
|
779
816
|
export declare const ensureTool: (tool: Tool, key?: string, cache?: string) => Promise<string>;
|
|
780
817
|
//#endregion
|
|
781
|
-
export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HBONE_PORT, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, type LeafCertificate, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, SocketLine, type Stack, type Stamp, type Step, TCP_STATE, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, TunnelLine, type Verdicts, type WindowPrediction, type Workload, ZtunnelPodLine, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, isTunnelState, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stoppingOf, timeSpread, transitionOf, tunnelsOf, unitOf, updatesOf, upgrade, upgradeCell, ztunnelPodsOf, ztunnelReplaced };
|
|
818
|
+
export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdgeProtocol, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HBONE_PORT, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, type LeafCertificate, type LearnWindow, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, SocketLine, type Stack, type Stamp, type Step, TCP_STATE, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, TunnelLine, type Verdicts, type WindowPrediction, type Workload, ZtunnelPodLine, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, isTunnelState, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stoppingOf, timeSpread, transitionOf, tunnelsOf, unitOf, updatesOf, upgrade, upgradeCell, ztunnelPodsOf, ztunnelReplaced };
|
|
782
819
|
//# sourceMappingURL=index.d.ts.map
|
package/build/index.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { $ as signalOf, A as medianInterval, B as emptyOf, C as propagationOf, D as unitOf, E as reusesReset, F as backendsAt, G as outcomesOf, H as intakeOf, I as byClients, J as recordsOf, K as podsAt, L as byNode, M as quantileOf, N as HBONE_PORT, O as updatesOf, P as TCP_STATE, Q as rulesOf, R as deletionOf, S as observationEnd, T as referenceOf, U as isTunnelState, V as failuresOf, W as kernelCounts, X as restartOf, Y as requestsOf, Z as restartsAt, _ as checkRun, _t as seconds, a as calibrationOf, at as stoppingOf, b as judgeKind, c as predictionText, ct as ztunnelPodsOf, d as FORMAT_V1, dt as HEALTH_PORT, et as signalsOf, f as judge, ft as LOG_LABEL, g as LAST_SLACK, gt as bandOf, h as KIND_NOISE, ht as SERVER, i as readRecord, it as socketsOf, j as middle, k as RELATIVE, l as timeSpread, lt as ztunnelReplaced, m as upgradeCell, mt as REDIS, n as kindCellRecordSchema, nt as sliceEmptyOf, o as cellsTable, ot as transitionOf, p as upgrade, pt as PORT, q as podsOf, r as kindCellRecordV1Schema, rt as slicesOf, s as medianText, st as tunnelsOf, t as STAGES, tt as sliceDropOf, u as FORMAT, ut as CLIENT, v as deletionsOf, vt as secondsOf, w as propagationSource, x as observationAfter, y as factsOf, yt as showBand, z as dropOf } from "./judge-
|
|
1
|
+
import { $ as signalOf, A as medianInterval, B as emptyOf, C as propagationOf, D as unitOf, E as reusesReset, F as backendsAt, G as outcomesOf, H as intakeOf, I as byClients, J as recordsOf, K as podsAt, L as byNode, M as quantileOf, N as HBONE_PORT, O as updatesOf, P as TCP_STATE, Q as rulesOf, R as deletionOf, S as observationEnd, T as referenceOf, U as isTunnelState, V as failuresOf, W as kernelCounts, X as restartOf, Y as requestsOf, Z as restartsAt, _ as checkRun, _t as seconds, a as calibrationOf, at as stoppingOf, b as judgeKind, c as predictionText, ct as ztunnelPodsOf, d as FORMAT_V1, dt as HEALTH_PORT, et as signalsOf, f as judge, ft as LOG_LABEL, g as LAST_SLACK, gt as bandOf, h as KIND_NOISE, ht as SERVER, i as readRecord, it as socketsOf, j as middle, k as RELATIVE, l as timeSpread, lt as ztunnelReplaced, m as upgradeCell, mt as REDIS, n as kindCellRecordSchema, nt as sliceEmptyOf, o as cellsTable, ot as transitionOf, p as upgrade, pt as PORT, q as podsOf, r as kindCellRecordV1Schema, rt as slicesOf, s as medianText, st as tunnelsOf, t as STAGES, tt as sliceDropOf, u as FORMAT, ut as CLIENT, v as deletionsOf, vt as secondsOf, w as propagationSource, x as observationAfter, y as factsOf, yt as showBand, z as dropOf } from "./judge-Bb4ebE9V.js";
|
|
2
2
|
import { valueIn } from "@mark1russell7/quantity";
|
|
3
3
|
import { execFile, execFileSync } from "node:child_process";
|
|
4
4
|
import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, renameSync, rmSync, writeFileSync } from "node:fs";
|
|
@@ -7,7 +7,7 @@ import { promisify } from "node:util";
|
|
|
7
7
|
import { createHash } from "node:crypto";
|
|
8
8
|
import { fileURLToPath } from "node:url";
|
|
9
9
|
import { gunzipSync } from "node:zlib";
|
|
10
|
-
import { failSpan, lint, lintStop, service } from "@mark1russell7/service";
|
|
10
|
+
import { failSpan, lint, lintStop, recoverSpan, service } from "@mark1russell7/service";
|
|
11
11
|
import { createRequire } from "node:module";
|
|
12
12
|
import { tmpdir } from "node:os";
|
|
13
13
|
import { harnessImage, imageOf } from "@mark1russell7/harness";
|
|
@@ -691,6 +691,14 @@ const CHANNEL_USE = {
|
|
|
691
691
|
value: 5,
|
|
692
692
|
unit: "s"
|
|
693
693
|
};
|
|
694
|
+
/**
|
|
695
|
+
* This time is between the SIGHUP of `cannot-serve` and its fault. Each client opens new connections
|
|
696
|
+
* at the signal, and the requests on its old connections end in this time (the timeout and 100 ms).
|
|
697
|
+
*/
|
|
698
|
+
const SPREAD_LEAD = {
|
|
699
|
+
value: 3,
|
|
700
|
+
unit: "s"
|
|
701
|
+
};
|
|
694
702
|
const IPV4 = /^(25[0-5]|2[0-4]\d|1?\d?\d)(\.(25[0-5]|2[0-4]\d|1?\d?\d)){3}$/;
|
|
695
703
|
/**
|
|
696
704
|
* This function gives the command of the injector that sets one sysctl of its network namespace.
|
|
@@ -1248,6 +1256,32 @@ const planOf = (cell) => {
|
|
|
1248
1256
|
heal: [{ do: "certificates" }],
|
|
1249
1257
|
restore: []
|
|
1250
1258
|
};
|
|
1259
|
+
case "cannot-serve": return {
|
|
1260
|
+
prepare: [{
|
|
1261
|
+
do: "signal",
|
|
1262
|
+
target: "clients",
|
|
1263
|
+
signal: "SIGHUP"
|
|
1264
|
+
}, {
|
|
1265
|
+
do: "wait",
|
|
1266
|
+
time: SPREAD_LEAD
|
|
1267
|
+
}],
|
|
1268
|
+
inject: [{
|
|
1269
|
+
do: "flag",
|
|
1270
|
+
target: "server",
|
|
1271
|
+
on: true
|
|
1272
|
+
}],
|
|
1273
|
+
hold: needs(trigger.hold, "a hold, the time that the replica cannot serve"),
|
|
1274
|
+
heal: [{
|
|
1275
|
+
do: "flag",
|
|
1276
|
+
target: "server",
|
|
1277
|
+
on: false
|
|
1278
|
+
}],
|
|
1279
|
+
restore: [{
|
|
1280
|
+
do: "flag",
|
|
1281
|
+
target: "server",
|
|
1282
|
+
on: false
|
|
1283
|
+
}]
|
|
1284
|
+
};
|
|
1251
1285
|
}
|
|
1252
1286
|
};
|
|
1253
1287
|
/** This function tells if a trigger stops a pod of the service with its grace: a deletion, an eviction, a scale-in or a rollout. */
|
|
@@ -2181,14 +2215,18 @@ const POD_ENV = {
|
|
|
2181
2215
|
name: "POD_NAME",
|
|
2182
2216
|
valueFrom: { fieldRef: { fieldPath: "metadata.name" } }
|
|
2183
2217
|
};
|
|
2184
|
-
/**
|
|
2218
|
+
/**
|
|
2219
|
+
* This function gives the environment of the app: its config as JSON, and the name of its pod. With
|
|
2220
|
+
* `failsRoute`, the route fails while the flag file is there (`cannot-serve`).
|
|
2221
|
+
*/
|
|
2185
2222
|
const appEnv = (cell) => [{
|
|
2186
2223
|
name: "CONFIG",
|
|
2187
2224
|
value: JSON.stringify({
|
|
2188
2225
|
name: "lab",
|
|
2189
2226
|
port: PORT,
|
|
2190
2227
|
healthPort: HEALTH_PORT,
|
|
2191
|
-
declaration: cell.declaration
|
|
2228
|
+
declaration: cell.declaration,
|
|
2229
|
+
...cell.trigger.kind === "cannot-serve" ? { failsRoute: true } : {}
|
|
2192
2230
|
})
|
|
2193
2231
|
}, POD_ENV];
|
|
2194
2232
|
/** This file is the flag of the trigger `liveness-restart`, in the root of the server process (`run.ts`, the step `flag`). */
|
|
@@ -2376,7 +2414,8 @@ const serverObjects = (cell, namespace, image, version = 1) => {
|
|
|
2376
2414
|
/**
|
|
2377
2415
|
* These are the objects of the clients: a DaemonSet with one client on each node, the control plane
|
|
2378
2416
|
* too. The client of the service is `client.mjs`, and the client of grpc-go is `client-grpc.mjs`.
|
|
2379
|
-
* Each one is in the lab image, and each one writes the same lines.
|
|
2417
|
+
* Each one is in the lab image, and each one writes the same lines. A service with the protocol
|
|
2418
|
+
* `h2c` gives its clients `PROTOCOL`, thus they speak HTTP/2 to it.
|
|
2380
2419
|
*/
|
|
2381
2420
|
const clientObjects = (cell, namespace, image, target) => [{
|
|
2382
2421
|
apiVersion: "apps/v1",
|
|
@@ -2418,6 +2457,14 @@ const clientObjects = (cell, namespace, image, target) => [{
|
|
|
2418
2457
|
name: "TIMEOUT_MS",
|
|
2419
2458
|
value: String(valueIn(cell.workload.timeout, "ms"))
|
|
2420
2459
|
},
|
|
2460
|
+
...cell.workload.connections === void 0 ? [] : [{
|
|
2461
|
+
name: "CONNECTIONS",
|
|
2462
|
+
value: String(cell.workload.connections)
|
|
2463
|
+
}],
|
|
2464
|
+
...cell.declaration?.protocol === "h2c" ? [{
|
|
2465
|
+
name: "PROTOCOL",
|
|
2466
|
+
value: "h2c"
|
|
2467
|
+
}] : [],
|
|
2421
2468
|
{
|
|
2422
2469
|
name: "POD_NAME",
|
|
2423
2470
|
valueFrom: { fieldRef: { fieldPath: "metadata.name" } }
|
|
@@ -2538,6 +2585,44 @@ const pausedWindow = (declaration, preStop, lead, hold, propagation) => {
|
|
|
2538
2585
|
resume: seconds(resume)
|
|
2539
2586
|
};
|
|
2540
2587
|
};
|
|
2588
|
+
/**
|
|
2589
|
+
* This function gives the learn window of a declaration, its readiness probe, a propagation and a
|
|
2590
|
+
* workload of keep-alive connections. A caller selects a replica only when it opens a connection:
|
|
2591
|
+
* - The lame duck (`lameDuck` true or no value): after the latch, each answer closes its connection,
|
|
2592
|
+
* thus each connection of the replica gets one more failure and opens again. The rules select the
|
|
2593
|
+
* replica until the removal, thus the last failure comes in one turn after it.
|
|
2594
|
+
* - The signal of the lame duck is `Connection: close` on HTTP/1.1, and GOAWAY on HTTP/2. The
|
|
2595
|
+
* arithmetic is the same: the turn is `every` times the connections, which are sessions on HTTP/2.
|
|
2596
|
+
* - No lame duck: no connection opens again, thus the rules do not matter. The failures end only
|
|
2597
|
+
* when the readiness of the service recovers after the heal, and one turn after it.
|
|
2598
|
+
*/
|
|
2599
|
+
const learnWindow = (declaration, readiness, propagation, workload) => {
|
|
2600
|
+
const turn = us(s$1(workload.every) * (workload.connections ?? 1));
|
|
2601
|
+
const latch = latchBand(declaration.readiness, declaration.poll, "refuses");
|
|
2602
|
+
const start = band(0, turn);
|
|
2603
|
+
if (declaration.lameDuck === false) {
|
|
2604
|
+
const recover = recoverSpan(declaration.readiness, declaration.poll);
|
|
2605
|
+
return {
|
|
2606
|
+
turn: seconds(turn),
|
|
2607
|
+
latch,
|
|
2608
|
+
start,
|
|
2609
|
+
end: {
|
|
2610
|
+
from: "heal",
|
|
2611
|
+
band: band(0, s$1(recover.max) + turn)
|
|
2612
|
+
}
|
|
2613
|
+
};
|
|
2614
|
+
}
|
|
2615
|
+
const removal = secondsOf(removalBand(latch, readiness, propagation));
|
|
2616
|
+
return {
|
|
2617
|
+
turn: seconds(turn),
|
|
2618
|
+
latch,
|
|
2619
|
+
start,
|
|
2620
|
+
end: {
|
|
2621
|
+
from: "fault",
|
|
2622
|
+
band: band(secondsOf(latch).lo, removal.hi + turn)
|
|
2623
|
+
}
|
|
2624
|
+
};
|
|
2625
|
+
};
|
|
2541
2626
|
//#endregion
|
|
2542
2627
|
//#region src/rules.ts
|
|
2543
2628
|
/**
|
|
@@ -2710,7 +2795,8 @@ const stopOf = (cell) => {
|
|
|
2710
2795
|
case "authorization-policy":
|
|
2711
2796
|
case "idle":
|
|
2712
2797
|
case "ztunnel-restart":
|
|
2713
|
-
case "certificate-expiry":
|
|
2798
|
+
case "certificate-expiry":
|
|
2799
|
+
case "cannot-serve": return null;
|
|
2714
2800
|
}
|
|
2715
2801
|
};
|
|
2716
2802
|
/**
|
|
@@ -3685,7 +3771,8 @@ const estimate = (cell, environments = 2) => {
|
|
|
3685
3771
|
"authorization-policy": hold + 20,
|
|
3686
3772
|
idle: hold + 5,
|
|
3687
3773
|
"ztunnel-restart": 40,
|
|
3688
|
-
"certificate-expiry": hold + 20
|
|
3774
|
+
"certificate-expiry": hold + 20,
|
|
3775
|
+
"cannot-serve": hold + 20
|
|
3689
3776
|
};
|
|
3690
3777
|
const after = observationAfter(cell);
|
|
3691
3778
|
const observed = after === null ? 0 : hold + valueIn(after, "s");
|
|
@@ -3723,6 +3810,6 @@ const shardOf = (cells, index, count) => {
|
|
|
3723
3810
|
return out;
|
|
3724
3811
|
};
|
|
3725
3812
|
//#endregion
|
|
3726
|
-
export { ACCESS_LOG_FORMAT, CHANNEL_USE, CLIENT, CLIENT_ACCOUNT, CLOSE_LEAD, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HBONE_PORT, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LDS_EVERY, LDS_FILTER, LOG_LABEL, MESH_IMAGES, MESH_WORKLOADS, MIDDLEBOX, NETNS_DEFAULTS, NODE_IMAGES, O04, O09, O10, O16, POLICY, PORT, QUIET_LEAD, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, SYSCTL, TCP_STATE, WATCHDOG, ZTUNNEL_PODS, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, certificateArgs, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, followsZtunnel, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, isTunnelState, istioOperator, judge, judgeKind, keptKernel, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, labSource, latchBand, leafCertificates, lintCell, listenerCountsOf, localPin, medianInterval, medianText, meshKey, middle, middleboxCommands, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, policyObject, pollsListeners, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartOf, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, setZtunnelEnv, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stackOf, stageLabImage, stallRemovalBand, stopOf, stoppingOf, sysctlCommand, tarEntry, timeSpread, toolPath, transitionOf, tunnelsOf, unitOf, updatesOf, upgrade, upgradeCell, verdictsOf, writeResult, ztunnelCertificates, ztunnelEnvArgs, ztunnelPodsOf, ztunnelReplaced };
|
|
3813
|
+
export { ACCESS_LOG_FORMAT, CHANNEL_USE, CLIENT, CLIENT_ACCOUNT, CLOSE_LEAD, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HBONE_PORT, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LDS_EVERY, LDS_FILTER, LOG_LABEL, MESH_IMAGES, MESH_WORKLOADS, MIDDLEBOX, NETNS_DEFAULTS, NODE_IMAGES, O04, O09, O10, O16, POLICY, PORT, QUIET_LEAD, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, SPREAD_LEAD, STAGES, SYSCTL, TCP_STATE, WATCHDOG, ZTUNNEL_PODS, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, certificateArgs, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, followsZtunnel, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, isTunnelState, istioOperator, judge, judgeKind, keptKernel, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, labSource, latchBand, leafCertificates, learnWindow, lintCell, listenerCountsOf, localPin, medianInterval, medianText, meshKey, middle, middleboxCommands, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, policyObject, pollsListeners, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartOf, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, setZtunnelEnv, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stackOf, stageLabImage, stallRemovalBand, stopOf, stoppingOf, sysctlCommand, tarEntry, timeSpread, toolPath, transitionOf, tunnelsOf, unitOf, updatesOf, upgrade, upgradeCell, verdictsOf, writeResult, ztunnelCertificates, ztunnelEnvArgs, ztunnelPodsOf, ztunnelReplaced };
|
|
3727
3814
|
|
|
3728
3815
|
//# sourceMappingURL=index.js.map
|