@mark1russell7/kind 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -0
- package/build/index.d.ts +53 -2
- package/build/index.js +154 -25
- package/build/index.js.map +1 -1
- package/build/{judge-CRMYoEOw.js → judge-Bkbs7VVG.js} +121 -19
- package/build/judge-Bkbs7VVG.js.map +1 -0
- package/build/{judge-DWF4QdZ8.d.ts → judge-DMuI3aDn.d.ts} +31 -4
- package/build/judge.d.ts +2 -2
- package/build/judge.js +1 -1
- package/image/Dockerfile +1 -1
- package/image/agent.mjs +55 -24
- package/image/client-grpc.mjs +2 -0
- package/package.json +2 -1
- package/src/lds.ts +31 -0
- package/build/judge-CRMYoEOw.js.map +0 -1
package/README.md
CHANGED
|
@@ -85,6 +85,10 @@ The tests read three real records of `/1` (`test/records/`): a cell that holds,
|
|
|
85
85
|
- **The endpoints of each client sidecar.** From 0.3.0, in a sidecar mesh, the agent also polls the Envoy of the client sidecar on its node each 50 ms (`eds`). The client sidecar picks the backend of each call, thus these endpoints and not the rules are the propagation of the cell (`propagationSource`).
|
|
86
86
|
- A poller enters the network namespace of the client pod (`nsenter -n`). It reads `/clusters` on `127.0.0.1:15000` with a filter that selects the server cluster (`eds.ts`).
|
|
87
87
|
- Each change of the healthy hosts gives an `eds` line with the start of the poll before it. That line has the form of a rules line. Each second, an `eds-alive` line counts the polls and gives their mean time.
|
|
88
|
+
- **The listeners of each server sidecar.** From 0.4.0, the agent also polls the Envoy of each server sidecar on its node each 50 ms (`lds`). It does this for a policy in a sidecar mesh (`pollsListeners`).
|
|
89
|
+
- A poller enters the network namespace of the server pod through the PID of its app. It reads `/stats` on `127.0.0.1:15000` with a filter that selects two counters of the listener manager (`lds.ts`).
|
|
90
|
+
- `listener_modified` counts each update of a listener, and `listener_in_place_updated` counts each update of its filter chains only.
|
|
91
|
+
- Each change of the counters gives an `lds` line with the start of the poll before it. The first line of a poller has no poll before it, thus it is no update (`updatesOf`).
|
|
88
92
|
- **The injector of each node** is the image of the harness, in the PID namespace of the node. `crictl` on the node gives the PID of a container, and `nsenter -n` enters the network namespace of its pod. The injector of the harness then applies its rules there.
|
|
89
93
|
- **The watcher** is a container on the network of the cluster. It watches the pods, the EndpointSlices and the events through the API, and it follows the log of each pod of the cell.
|
|
90
94
|
- From 0.3.0, it also follows the sidecar of Istio (`istio-proxy`), and a pod line gives the state of the sidecar (`sc`). Each line of the sidecar becomes a `proxy` line, stamped at its arrival. Its own stamps are of the wall clock, thus no band reads them.
|
|
@@ -157,6 +161,7 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
|
|
|
157
161
|
| `app-stall` | the event loop of the first replica blocked |
|
|
158
162
|
| `rollout` | no Redis pod, then a rolling update |
|
|
159
163
|
| `cpu-throttle` | `cpu.max` in the cgroup of the container of the first replica |
|
|
164
|
+
| `authorization-policy` | `kubectl apply` of a DENY AuthorizationPolicy on the server pods for the principal of an account (`account`, the account `default` of the clients when not given), then `kubectl delete` after the hold |
|
|
160
165
|
|
|
161
166
|
A cgroup path that is not the scope of one container is refused. A write to the root of the node throttles or freezes each pod of the node.
|
|
162
167
|
|
|
@@ -197,6 +202,15 @@ From 0.2.3, a window can also give the propagation that it assumes (`propagation
|
|
|
197
202
|
- An earlier drop only closes the window earlier, thus the judge does not test the start of the band.
|
|
198
203
|
- `predictionText` adds `the rules drop the replica in` and the band.
|
|
199
204
|
|
|
205
|
+
From 0.4.0, a window can count from the heal (`heal`), the first act that ends the fault. A window can also name its outcomes (`outcomes`). A failure in the window with another outcome then fails the run.
|
|
206
|
+
|
|
207
|
+
- The client of gRPC gives the outcome `denied` to PERMISSION_DENIED: the local reply 403 of an RBAC deny of a sidecar.
|
|
208
|
+
- For a policy, the propagation of a window is the LDS update of each server sidecar, after the apply and after the delete (`updatesOf`). An update after the band agrees with the law, and the cell is `extend`.
|
|
209
|
+
- A denial after the window, before the last update after the delete and one more call, is a `tail`. A denied call has no pod, because the sidecar answers it. Thus the judge reads the last update of each server sidecar.
|
|
210
|
+
- `calibrationOf` gives the updates after the apply by node, and the updates after the delete (`heal`).
|
|
211
|
+
|
|
212
|
+
A new member of the union `TriggerKind` breaks a reader that switches on it. That is a breaking change before 1.0.0, thus the version is 0.4.0. The record format stays `/2`, with the optional fields `account`, `outcomes`, `updates` and `healUpdates`, and the reference `heal`.
|
|
213
|
+
|
|
200
214
|
A preStop sleep that is longer than the grace is a command in the container (`sh -c "sleep 60"`). The API server refuses a sleep action that is longer than the grace.
|
|
201
215
|
|
|
202
216
|
## Tests
|
package/build/index.d.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { $ as
|
|
1
|
+
import { $ as updatesOf, $t as Replica, A as TriggerV1, At as transitionOf, B as RunCheck, Bt as showBand, C as KindCellResult, Ct as rulesOf, D as PredictionV1, Dt as sliceEmptyOf, E as LoadSample, Et as sliceDropOf, F as KIND_NOISE, Ft as REDIS, G as judgeKind, Gt as KubeletProbe, H as checkRun, Ht as Declaration, I as KindDecision, It as SERVER, J as propagationOf, Jt as MeshMode, K as observationAfter, Kt as Mesh, L as KindVerdict, Lt as bandOf, M as upgrade, Mt as HEALTH_PORT, N as upgradeCell, Nt as LOG_LABEL, O as RunCheckV1, Ot as slicesOf, P as Deleted, Pt as PORT, Q as unitOf, Qt as Reference, R as LAST_SLACK, Rt as seconds, S as KindCellRecordV1, St as restartsAt, T as KindVerdictV1, Tt as signalsOf, U as deletionsOf, Ut as KindCell, V as RunFacts, Vt as Band, W as factsOf, Wt as KindRun, X as referenceOf, Xt as Prediction, Y as propagationSource, Yt as PodSettings, Z as reusesReset, Zt as RedisDeclaration, _ as ClusterManifest, _n as nodeNames, _t as outcomesOf, a as STAGES, an as Workload, at as Request, b as FORMAT_V1, bt as recordsOf, c as readRecord, cn as KindScenario, ct as backendsAt, d as calibrationOf, dn as NodeImage, dt as deletionOf, en as ResetReuse, et as Endpoint, f as cellsTable, fn as cidrOverlap, ft as dropOf, g as BandV1, gn as kubeProxyPatch, gt as kernelCounts, h as timeSpread, hn as kindConfig, ht as intakeOf, i as quantileOf, in as TriggerKind, it as RecordLine, j as judge, jt as CLIENT, k as RunFactsV1, kt as stoppingOf, l as Judged, ln as KubeProxyMode, lt as byClients, m as predictionText, mn as goDuration, mt as failuresOf, n as medianInterval, nn as Template, nt as PodLine, o as kindCellRecordSchema, on as KUBE_PROXY_MODES, ot as RulesLine, p as medianText, pn as cidrRange, pt as emptyOf, q as observationEnd, qt as MeshFlavor, r as middle, rn as Trigger, rt as PropagationSource, s as kindCellRecordV1Schema, sn as KindConfig, st as SliceLine, t as RELATIVE, tn as Stack, tt as Line, u as TimeSpread, un as NODE_IMAGES, ut as byNode, v as EnvironmentManifest, vn as scenario, vt as podsAt, w as KindCellV1, wt as signalOf, x as KindCellRecord, xt as requestsOf, y as FORMAT, yn as scenarioProblems, yt as podsOf, z as Propagated, zt as secondsOf } from "./judge-DMuI3aDn.js";
|
|
2
2
|
import { Quantity } from "@mark1russell7/quantity";
|
|
3
3
|
import { CheckFn, Finding, HealthServer, Poll, Rule, Scenario, ServiceBuilder, ServiceDescription, Stage, Timing, TransportImpl, Verdict } from "@mark1russell7/service";
|
|
4
4
|
import { RedisClient } from "@mark1russell7/service/redis";
|
|
@@ -145,6 +145,9 @@ type Step = {
|
|
|
145
145
|
} | {
|
|
146
146
|
readonly do: "wait";
|
|
147
147
|
readonly time: Quantity;
|
|
148
|
+
} | {
|
|
149
|
+
readonly do: "apply";
|
|
150
|
+
readonly object: Readonly<Record<string, unknown>>;
|
|
148
151
|
};
|
|
149
152
|
/**
|
|
150
153
|
* A fault plan is the steps of a run:
|
|
@@ -158,6 +161,17 @@ interface FaultPlan {
|
|
|
158
161
|
readonly heal: readonly Step[];
|
|
159
162
|
readonly restore: readonly Step[];
|
|
160
163
|
}
|
|
164
|
+
/** This name is the name of the AuthorizationPolicy of the trigger `authorization-policy`. */
|
|
165
|
+
export declare const POLICY = "lab-deny";
|
|
166
|
+
/** This service account is the account of the clients: their pods name no other account. */
|
|
167
|
+
export declare const CLIENT_ACCOUNT = "default";
|
|
168
|
+
/**
|
|
169
|
+
* This function gives the AuthorizationPolicy of the trigger `authorization-policy`. It denies each
|
|
170
|
+
* request to the server pods from the principal of the service account `account`. Istio gives a
|
|
171
|
+
* pod of the mesh the principal `cluster.local/ns/<namespace>/sa/<account>` of its account. The
|
|
172
|
+
* RBAC filter of the server sidecar reads the principal of the peer from its mTLS certificate.
|
|
173
|
+
*/
|
|
174
|
+
export declare const policyObject: (account: string) => Readonly<Record<string, unknown>>;
|
|
161
175
|
/** This function gives the fault plan of a cell. */
|
|
162
176
|
export declare const planOf: (cell: KindCell) => FaultPlan;
|
|
163
177
|
/** This function tells if a trigger stops a pod of the service with its grace: a deletion, an eviction, a scale-in or a rollout. */
|
|
@@ -228,6 +242,10 @@ export declare const meshKey: (cell: Pick<KindCell, "mesh">) => MeshFlavor | "no
|
|
|
228
242
|
export declare const serverCluster: (namespace: string) => string;
|
|
229
243
|
/** This time is the interval of the EDS poll of each client sidecar, the same as the poll of the rules. */
|
|
230
244
|
export declare const EDS_EVERY: Quantity;
|
|
245
|
+
/** This time is the interval of the LDS poll of each server sidecar, the same as the poll of EDS. */
|
|
246
|
+
export declare const LDS_EVERY: Quantity;
|
|
247
|
+
/** This function tells if the lab polls the LDS updates of the server sidecars of a cell: a policy in a sidecar mesh. */
|
|
248
|
+
export declare const pollsListeners: (cell: Pick<KindCell, "mesh" | "trigger">) => boolean;
|
|
231
249
|
//#endregion
|
|
232
250
|
//#region src/lab.d.ts
|
|
233
251
|
/** A stamp brackets an act on CLOCK_MONOTONIC, in milliseconds. */
|
|
@@ -247,6 +265,15 @@ interface EdsSpec {
|
|
|
247
265
|
readonly pid: number;
|
|
248
266
|
}>;
|
|
249
267
|
}
|
|
268
|
+
/** An LDS spec is the interval of the LDS pollers, and each server pod with its node and a PID in it. */
|
|
269
|
+
interface LdsSpec {
|
|
270
|
+
readonly every: Quantity;
|
|
271
|
+
readonly pods: ReadonlyArray<{
|
|
272
|
+
readonly node: string;
|
|
273
|
+
readonly pod: string;
|
|
274
|
+
readonly pid: number;
|
|
275
|
+
}>;
|
|
276
|
+
}
|
|
250
277
|
/** A kind lab is the instruments of one cluster: a sidecar pair on each node, the watcher of the API and the prober. */
|
|
251
278
|
interface KindLab {
|
|
252
279
|
readonly cluster: Cluster;
|
|
@@ -274,6 +301,11 @@ interface KindLab {
|
|
|
274
301
|
* of its processes, and the agent enters the network namespace of that PID.
|
|
275
302
|
*/
|
|
276
303
|
eds(spec: EdsSpec | null): Promise<void>;
|
|
304
|
+
/**
|
|
305
|
+
* This method asks each agent to poll the sidecar of each server pod of its node, `every` apart.
|
|
306
|
+
* Each poll gives the counters of LDS updates of that sidecar (`lds.ts`). With null, the polls stop.
|
|
307
|
+
*/
|
|
308
|
+
lds(spec: LdsSpec | null): Promise<void>;
|
|
277
309
|
act(node: string, spec: Readonly<Record<string, unknown>>): Promise<Stamp>;
|
|
278
310
|
inject(node: string, pid: number, args: readonly string[]): Promise<Stamp>;
|
|
279
311
|
now(): Promise<number>;
|
|
@@ -299,6 +331,25 @@ export declare const openKindLab: (cluster: Cluster, image: string) => Promise<K
|
|
|
299
331
|
/** This function pins a Docker image by digest under a local tag that holds the digest, for `kind load` and `imagePullPolicy: Never`. */
|
|
300
332
|
export declare const localPin: (image: string) => Promise<string>;
|
|
301
333
|
//#endregion
|
|
334
|
+
//#region src/lds.d.ts
|
|
335
|
+
/**
|
|
336
|
+
* This filter of RE2 selects the two counters of the listener manager of Envoy that an update of LDS
|
|
337
|
+
* moves (`/stats?filter=`, `source/server/admin/stats_params.cc` of Envoy v1.39.4). An update of a
|
|
338
|
+
* listener counts `listener_modified`. An update in place, of its filter chains only, also counts
|
|
339
|
+
* `listener_in_place_updated` (`listener_manager_impl.cc:644`, `:692`).
|
|
340
|
+
*/
|
|
341
|
+
export declare const LDS_FILTER = "^listener_manager\\.listener_(in_place_updated|modified)$";
|
|
342
|
+
/** The listener counts are the two counters of LDS of one sidecar: each update (`mod`), and each update in place (`inPlace`). */
|
|
343
|
+
interface ListenerCounts {
|
|
344
|
+
readonly mod: number;
|
|
345
|
+
readonly inPlace: number;
|
|
346
|
+
}
|
|
347
|
+
/**
|
|
348
|
+
* This function gives the listener counts of the text of `/stats`, one `name: value` on each line
|
|
349
|
+
* (`stats_render.cc:15`). It gives null when the text has no counter `listener_modified`.
|
|
350
|
+
*/
|
|
351
|
+
export declare const listenerCountsOf: (text: string) => ListenerCounts | null;
|
|
352
|
+
//#endregion
|
|
302
353
|
//#region src/manifests.d.ts
|
|
303
354
|
/**
|
|
304
355
|
* This module generates the Kubernetes objects of a cell. They are its namespace, its Redis, the
|
|
@@ -537,5 +588,5 @@ export declare const toolPath: (tool: Tool, key?: string, cache?: string) => str
|
|
|
537
588
|
*/
|
|
538
589
|
export declare const ensureTool: (tool: Tool, key?: string, cache?: string) => Promise<string>;
|
|
539
590
|
//#endregion
|
|
540
|
-
export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, Line, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, type Stack, type Stamp, type Step, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, type Verdicts, type Workload, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stoppingOf, timeSpread, transitionOf, unitOf, upgrade, upgradeCell };
|
|
591
|
+
export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, type Stack, type Stamp, type Step, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, type Verdicts, type Workload, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stoppingOf, timeSpread, transitionOf, unitOf, updatesOf, upgrade, upgradeCell };
|
|
541
592
|
//# sourceMappingURL=index.d.ts.map
|
package/build/index.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { $ as
|
|
1
|
+
import { $ as slicesOf, A as medianInterval, B as intakeOf, C as propagationOf, D as unitOf, E as reusesReset, F as byNode, G as recordsOf, H as outcomesOf, I as deletionOf, J as rulesOf, K as requestsOf, L as dropOf, M as quantileOf, N as backendsAt, O as updatesOf, P as byClients, Q as sliceEmptyOf, R as emptyOf, S as observationEnd, T as referenceOf, U as podsAt, V as kernelCounts, W as podsOf, X as signalsOf, Y as signalOf, Z as sliceDropOf, _ as checkRun, a as calibrationOf, at as PORT, b as judgeKind, c as predictionText, ct as bandOf, d as FORMAT_V1, dt as showBand, et as stoppingOf, f as judge, g as LAST_SLACK, h as KIND_NOISE, i as readRecord, it as LOG_LABEL, j as middle, k as RELATIVE, l as timeSpread, lt as seconds, m as upgradeCell, n as kindCellRecordSchema, nt as CLIENT, o as cellsTable, ot as REDIS, p as upgrade, q as restartsAt, r as kindCellRecordV1Schema, rt as HEALTH_PORT, s as medianText, st as SERVER, t as STAGES, tt as transitionOf, u as FORMAT, ut as secondsOf, v as deletionsOf, w as propagationSource, x as observationAfter, y as factsOf, z as failuresOf } from "./judge-Bkbs7VVG.js";
|
|
2
2
|
import { valueIn } from "@mark1russell7/quantity";
|
|
3
3
|
import { execFile, execFileSync } from "node:child_process";
|
|
4
4
|
import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, realpathSync, renameSync, rmSync, writeFileSync } from "node:fs";
|
|
@@ -619,6 +619,29 @@ const NODE = "{node}";
|
|
|
619
619
|
const VERSION = "{version}";
|
|
620
620
|
/** This annotation of a pod is its deletion cost for its ReplicaSet: a lower cost goes first at a scale-in. */
|
|
621
621
|
const DELETION_COST = "controller.kubernetes.io/pod-deletion-cost";
|
|
622
|
+
/** This name is the name of the AuthorizationPolicy of the trigger `authorization-policy`. */
|
|
623
|
+
const POLICY = "lab-deny";
|
|
624
|
+
/** This service account is the account of the clients: their pods name no other account. */
|
|
625
|
+
const CLIENT_ACCOUNT = "default";
|
|
626
|
+
/**
|
|
627
|
+
* This function gives the AuthorizationPolicy of the trigger `authorization-policy`. It denies each
|
|
628
|
+
* request to the server pods from the principal of the service account `account`. Istio gives a
|
|
629
|
+
* pod of the mesh the principal `cluster.local/ns/<namespace>/sa/<account>` of its account. The
|
|
630
|
+
* RBAC filter of the server sidecar reads the principal of the peer from its mTLS certificate.
|
|
631
|
+
*/
|
|
632
|
+
const policyObject = (account) => ({
|
|
633
|
+
apiVersion: "security.istio.io/v1",
|
|
634
|
+
kind: "AuthorizationPolicy",
|
|
635
|
+
metadata: {
|
|
636
|
+
name: POLICY,
|
|
637
|
+
namespace: "{namespace}"
|
|
638
|
+
},
|
|
639
|
+
spec: {
|
|
640
|
+
selector: { matchLabels: { app: "lab" } },
|
|
641
|
+
action: "DENY",
|
|
642
|
+
rules: [{ from: [{ source: { principals: [`cluster.local/ns/{namespace}/sa/${account}`] } }] }]
|
|
643
|
+
}
|
|
644
|
+
});
|
|
622
645
|
/** This function gives the fault plan of a cell. */
|
|
623
646
|
const planOf = (cell) => {
|
|
624
647
|
const ns = ["-n", "{namespace}"];
|
|
@@ -881,6 +904,32 @@ const planOf = (cell) => {
|
|
|
881
904
|
restore: []
|
|
882
905
|
};
|
|
883
906
|
}
|
|
907
|
+
case "authorization-policy": return {
|
|
908
|
+
inject: [{
|
|
909
|
+
do: "apply",
|
|
910
|
+
object: policyObject(trigger.account ?? "default")
|
|
911
|
+
}],
|
|
912
|
+
hold: needs(trigger.hold, "a hold"),
|
|
913
|
+
heal: [{
|
|
914
|
+
do: "kubectl",
|
|
915
|
+
args: [
|
|
916
|
+
...ns,
|
|
917
|
+
"delete",
|
|
918
|
+
"authorizationpolicy",
|
|
919
|
+
POLICY
|
|
920
|
+
]
|
|
921
|
+
}],
|
|
922
|
+
restore: [{
|
|
923
|
+
do: "kubectl",
|
|
924
|
+
args: [
|
|
925
|
+
...ns,
|
|
926
|
+
"delete",
|
|
927
|
+
"authorizationpolicy",
|
|
928
|
+
POLICY,
|
|
929
|
+
"--ignore-not-found"
|
|
930
|
+
]
|
|
931
|
+
}]
|
|
932
|
+
};
|
|
884
933
|
}
|
|
885
934
|
};
|
|
886
935
|
/** This function tells if a trigger stops a pod of the service with its grace: a deletion, an eviction, a scale-in or a rollout. */
|
|
@@ -916,7 +965,8 @@ const FILES = [
|
|
|
916
965
|
["backends.ts", "src/backends.ts"],
|
|
917
966
|
["conntrack.ts", "src/conntrack.ts"],
|
|
918
967
|
["drops.ts", "src/drops.ts"],
|
|
919
|
-
["eds.ts", "src/eds.ts"]
|
|
968
|
+
["eds.ts", "src/eds.ts"],
|
|
969
|
+
["lds.ts", "src/lds.ts"]
|
|
920
970
|
];
|
|
921
971
|
const shell = process.platform === "win32";
|
|
922
972
|
/** This function gives each file of a staged context, with its path in the context, in a stable sequence. */
|
|
@@ -1135,6 +1185,40 @@ const EDS_EVERY = {
|
|
|
1135
1185
|
value: 50,
|
|
1136
1186
|
unit: "ms"
|
|
1137
1187
|
};
|
|
1188
|
+
/** This time is the interval of the LDS poll of each server sidecar, the same as the poll of EDS. */
|
|
1189
|
+
const LDS_EVERY = {
|
|
1190
|
+
value: 50,
|
|
1191
|
+
unit: "ms"
|
|
1192
|
+
};
|
|
1193
|
+
/** This function tells if the lab polls the LDS updates of the server sidecars of a cell: a policy in a sidecar mesh. */
|
|
1194
|
+
const pollsListeners = (cell) => cell.mesh?.mode === "sidecar" && cell.trigger.kind === "authorization-policy";
|
|
1195
|
+
//#endregion
|
|
1196
|
+
//#region src/lds.ts
|
|
1197
|
+
/**
|
|
1198
|
+
* This filter of RE2 selects the two counters of the listener manager of Envoy that an update of LDS
|
|
1199
|
+
* moves (`/stats?filter=`, `source/server/admin/stats_params.cc` of Envoy v1.39.4). An update of a
|
|
1200
|
+
* listener counts `listener_modified`. An update in place, of its filter chains only, also counts
|
|
1201
|
+
* `listener_in_place_updated` (`listener_manager_impl.cc:644`, `:692`).
|
|
1202
|
+
*/
|
|
1203
|
+
const LDS_FILTER = "^listener_manager\\.listener_(in_place_updated|modified)$";
|
|
1204
|
+
/**
|
|
1205
|
+
* This function gives the listener counts of the text of `/stats`, one `name: value` on each line
|
|
1206
|
+
* (`stats_render.cc:15`). It gives null when the text has no counter `listener_modified`.
|
|
1207
|
+
*/
|
|
1208
|
+
const listenerCountsOf = (text) => {
|
|
1209
|
+
let mod = null;
|
|
1210
|
+
let inPlace = 0;
|
|
1211
|
+
for (const line of text.split("\n")) {
|
|
1212
|
+
const match = /^listener_manager\.listener_(in_place_updated|modified): (\d+)$/.exec(line.trim());
|
|
1213
|
+
if (match === null) continue;
|
|
1214
|
+
if (match[1] === "modified") mod = Number(match[2]);
|
|
1215
|
+
else inPlace = Number(match[2]);
|
|
1216
|
+
}
|
|
1217
|
+
return mod === null ? null : {
|
|
1218
|
+
mod,
|
|
1219
|
+
inPlace
|
|
1220
|
+
};
|
|
1221
|
+
};
|
|
1138
1222
|
//#endregion
|
|
1139
1223
|
//#region src/conntrack.ts
|
|
1140
1224
|
/** This function gives the count of entries that `conntrack -D` deleted, from its last line: "N flow entries have been deleted". */
|
|
@@ -1243,6 +1327,33 @@ const openKindLab = async (cluster, image) => {
|
|
|
1243
1327
|
return found;
|
|
1244
1328
|
};
|
|
1245
1329
|
const firstNode = cluster.nodes[0];
|
|
1330
|
+
/**
|
|
1331
|
+
* This function writes the file of the pollers of one kind on each agent: `fields`, the interval
|
|
1332
|
+
* and the pods of its node. A node without a pod, or a spec of null, gets no file.
|
|
1333
|
+
*/
|
|
1334
|
+
const pollersOf = async (kind, fields, spec) => {
|
|
1335
|
+
for (const [node, { agent }] of sidecars) {
|
|
1336
|
+
const pods = spec === null ? [] : spec.pods.filter((pod) => pod.node === node).map(({ pod, pid }) => ({
|
|
1337
|
+
pod,
|
|
1338
|
+
pid
|
|
1339
|
+
}));
|
|
1340
|
+
const file = spec === null || pods.length === 0 ? null : JSON.stringify({
|
|
1341
|
+
...fields,
|
|
1342
|
+
every: valueIn(spec.every, "ms"),
|
|
1343
|
+
pods
|
|
1344
|
+
});
|
|
1345
|
+
if (file === null) await output(agent, [
|
|
1346
|
+
"rm",
|
|
1347
|
+
"-f",
|
|
1348
|
+
`/tmp/${kind}.json`
|
|
1349
|
+
]);
|
|
1350
|
+
else await output(agent, [
|
|
1351
|
+
"sh",
|
|
1352
|
+
"-c",
|
|
1353
|
+
`printf '%s' '${file}' > /tmp/${kind}.json.partial && mv /tmp/${kind}.json.partial /tmp/${kind}.json`
|
|
1354
|
+
]);
|
|
1355
|
+
}
|
|
1356
|
+
};
|
|
1246
1357
|
const lab = {
|
|
1247
1358
|
cluster,
|
|
1248
1359
|
image,
|
|
@@ -1318,27 +1429,10 @@ const openKindLab = async (cluster, image) => {
|
|
|
1318
1429
|
]);
|
|
1319
1430
|
},
|
|
1320
1431
|
async eds(spec) {
|
|
1321
|
-
|
|
1322
|
-
|
|
1323
|
-
|
|
1324
|
-
|
|
1325
|
-
}));
|
|
1326
|
-
const file = spec === null || pods.length === 0 ? null : JSON.stringify({
|
|
1327
|
-
cluster: spec.cluster,
|
|
1328
|
-
every: valueIn(spec.every, "ms"),
|
|
1329
|
-
pods
|
|
1330
|
-
});
|
|
1331
|
-
if (file === null) await output(agent, [
|
|
1332
|
-
"rm",
|
|
1333
|
-
"-f",
|
|
1334
|
-
"/tmp/eds.json"
|
|
1335
|
-
]);
|
|
1336
|
-
else await output(agent, [
|
|
1337
|
-
"sh",
|
|
1338
|
-
"-c",
|
|
1339
|
-
`printf '%s' '${file}' > /tmp/eds.json.partial && mv /tmp/eds.json.partial /tmp/eds.json`
|
|
1340
|
-
]);
|
|
1341
|
-
}
|
|
1432
|
+
await pollersOf("eds", spec === null ? null : { cluster: spec.cluster }, spec);
|
|
1433
|
+
},
|
|
1434
|
+
async lds(spec) {
|
|
1435
|
+
await pollersOf("lds", {}, spec);
|
|
1342
1436
|
},
|
|
1343
1437
|
async forgetHandshakes(ip, port) {
|
|
1344
1438
|
const out = {};
|
|
@@ -2278,6 +2372,28 @@ const pollEndpoints = async (lab, cluster, namespace) => {
|
|
|
2278
2372
|
pods
|
|
2279
2373
|
});
|
|
2280
2374
|
};
|
|
2375
|
+
/**
|
|
2376
|
+
* This function starts the LDS poll of the sidecar of each server pod, each 50 ms, for a cell that
|
|
2377
|
+
* polls them (`pollsListeners`). It starts the pollers again only when the server pods changed. The
|
|
2378
|
+
* agent enters the network namespace of the pod through the PID of its app.
|
|
2379
|
+
*/
|
|
2380
|
+
const pollListeners = async (ctx) => {
|
|
2381
|
+
if (!pollsListeners(ctx.cell)) return;
|
|
2382
|
+
const servers = ctx.tracker.servers().filter((pod) => !pod.del && pod.node !== null);
|
|
2383
|
+
const key = servers.map((pod) => `${pod.name}/${pod.cs.map((status) => status.id).join("+")}`).sort().join();
|
|
2384
|
+
if (key === ctx.listened) return;
|
|
2385
|
+
const pods = [];
|
|
2386
|
+
for (const pod of servers) pods.push({
|
|
2387
|
+
node: pod.node,
|
|
2388
|
+
pod: pod.name,
|
|
2389
|
+
pid: await ctx.lab.pidOf(pod.node, ctx.namespace, pod.name, "app")
|
|
2390
|
+
});
|
|
2391
|
+
await ctx.lab.lds({
|
|
2392
|
+
every: LDS_EVERY,
|
|
2393
|
+
pods
|
|
2394
|
+
});
|
|
2395
|
+
ctx.listened = key;
|
|
2396
|
+
};
|
|
2281
2397
|
/** This function starts the prober again when the pods of the cell changed: it watches the namespace of each pod. */
|
|
2282
2398
|
const probeThePods = async (ctx) => {
|
|
2283
2399
|
const pods = [...ctx.tracker.pods.values()].filter((pod) => !pod.del && pod.node !== null && pod.cs.every((status) => status.st === "running"));
|
|
@@ -2425,6 +2541,15 @@ const doStep = async (ctx, step, replicas, background) => {
|
|
|
2425
2541
|
case "wait":
|
|
2426
2542
|
await sleep(valueIn(step.time, "ms"));
|
|
2427
2543
|
return null;
|
|
2544
|
+
case "apply": {
|
|
2545
|
+
const object = JSON.parse(fill([JSON.stringify(step.object)])[0]);
|
|
2546
|
+
const t0 = await lab.now();
|
|
2547
|
+
await cluster.apply([object]);
|
|
2548
|
+
return {
|
|
2549
|
+
t0,
|
|
2550
|
+
t1: await lab.now()
|
|
2551
|
+
};
|
|
2552
|
+
}
|
|
2428
2553
|
}
|
|
2429
2554
|
};
|
|
2430
2555
|
/** This function tells if a run is over: the trigger is done, and the cell is at rest again. */
|
|
@@ -2476,6 +2601,7 @@ const runOnce = async (ctx, env, index, warmup, ours) => {
|
|
|
2476
2601
|
});
|
|
2477
2602
|
if (atRest !== null) return empty(`the cell was not at rest: ${atRest}`, await lab.now());
|
|
2478
2603
|
await probeThePods(ctx);
|
|
2604
|
+
await pollListeners(ctx);
|
|
2479
2605
|
await quiet(ctx, ours);
|
|
2480
2606
|
if (await steady(ctx, 3e5) !== null) return empty("the cell was not at rest after the wait for a quiet VM", await lab.now());
|
|
2481
2607
|
const replicas = replicasOf(ctx);
|
|
@@ -2708,6 +2834,7 @@ const runKindCells = async (cells, options) => {
|
|
|
2708
2834
|
log: options.log,
|
|
2709
2835
|
version: 1,
|
|
2710
2836
|
probed: "",
|
|
2837
|
+
listened: "",
|
|
2711
2838
|
expected: cell.pod.replicas
|
|
2712
2839
|
};
|
|
2713
2840
|
const share = Math.ceil(cell.runs / environments) - (env < cell.runs % environments || cell.runs % environments === 0 ? 0 : 1);
|
|
@@ -2746,6 +2873,7 @@ const runKindCells = async (cells, options) => {
|
|
|
2746
2873
|
off();
|
|
2747
2874
|
await lab.rules(null).catch(() => void 0);
|
|
2748
2875
|
await lab.eds(null).catch(() => void 0);
|
|
2876
|
+
await lab.lds(null).catch(() => void 0);
|
|
2749
2877
|
await lab.stopProbe();
|
|
2750
2878
|
await cluster.kubectl([
|
|
2751
2879
|
"delete",
|
|
@@ -2789,7 +2917,8 @@ const estimate = (cell, environments = 2) => {
|
|
|
2789
2917
|
crash: 30,
|
|
2790
2918
|
"app-stall": hold + 40,
|
|
2791
2919
|
"cpu-throttle": hold + 40,
|
|
2792
|
-
rollout: hold + 50
|
|
2920
|
+
rollout: hold + 50,
|
|
2921
|
+
"authorization-policy": hold + 20
|
|
2793
2922
|
};
|
|
2794
2923
|
const after = observationAfter(cell);
|
|
2795
2924
|
const observed = after === null ? 0 : hold + valueIn(after, "s");
|
|
@@ -2825,6 +2954,6 @@ const shardOf = (cells, index, count) => {
|
|
|
2825
2954
|
return out;
|
|
2826
2955
|
};
|
|
2827
2956
|
//#endregion
|
|
2828
|
-
export { ACCESS_LOG_FORMAT, CLIENT, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LOG_LABEL, NODE_IMAGES, O04, O09, O10, O16, PORT, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, WATCHDOG, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, istioOperator, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, lintCell, localPin, medianInterval, medianText, meshKey, middle, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stackOf, stageLabImage, stallRemovalBand, stoppingOf, tarEntry, timeSpread, toolPath, transitionOf, unitOf, upgrade, upgradeCell, verdictsOf, writeResult };
|
|
2957
|
+
export { ACCESS_LOG_FORMAT, CLIENT, CLIENT_ACCOUNT, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LDS_EVERY, LDS_FILTER, LOG_LABEL, NODE_IMAGES, O04, O09, O10, O16, POLICY, PORT, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, WATCHDOG, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, istioOperator, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, lintCell, listenerCountsOf, localPin, medianInterval, medianText, meshKey, middle, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, policyObject, pollsListeners, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stackOf, stageLabImage, stallRemovalBand, stoppingOf, tarEntry, timeSpread, toolPath, transitionOf, unitOf, updatesOf, upgrade, upgradeCell, verdictsOf, writeResult };
|
|
2829
2958
|
|
|
2830
2959
|
//# sourceMappingURL=index.js.map
|