@mark1russell7/kind 0.5.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -152,6 +152,8 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
152
152
  - **The seams** of the service are a flag file that fails its liveness check, and SIGUSR2. SIGUSR2 blocks its event loop for the time in a file.
153
153
  - **The clients** (`client.mjs`) send requests on new connections to the ClusterIP. Each request gives one line with its outcome: ok, 503, refused, reset, timeout or unreachable. From 0.1.3, each line also gives the local port of its connection (`lp`), a refused one's too. A conntrack entry is per port. Thus the requests on the port of a failure show the connection before it, and the pod that it went to.
154
154
  - **The gRPC clients** (`client-grpc.mjs`) hold one channel of grpc-js 1.14.6 to the ClusterIP, and send a call of `grpc.health.v1.Health/Check` each interval on it. Each call gives the line of a request of `client.mjs`, with the pod that answered (the header `x-pod`). A call that fails at once names the errno of the last connect: refused, reset or unreachable. A call of grpc-js has no deadline at its defaults. Thus a call with no answer in the timeout is a timeout, and the client cancels it.
155
+ - From 0.6.0, each socket of the channel also gives a `sock` line at its connect and at its close (`client-lines.ts`). Each line gives the local port of the socket (`lp`). The close line keeps the port of the connect, and it gives the code of the error of the socket (`err`), or null.
156
+ - From 0.6.0, SIGUSR1 stops the calls of the client, and SIGUSR2 starts them again. Each signal gives a `gap` line. The channel and its socket stay open in the gap. A listener of SIGUSR1 replaces the start of the inspector of Node at that signal.
155
157
 
156
158
  ## The triggers
157
159
 
@@ -172,6 +174,7 @@ The lab image (`image/`) holds the service of a cell on `@mark1russell7/service`
172
174
  | `rollout` | no Redis pod, then a rolling update |
173
175
  | `cpu-throttle` | `cpu.max` in the cgroup of the container of the first replica |
174
176
  | `authorization-policy` | `kubectl apply` of a DENY AuthorizationPolicy on the server pods for the principal of an account (`account`, the account `default` of the clients when not given), then `kubectl delete` after the hold |
177
+ | `idle` | SIGUSR1 to the gRPC client of each client pod, and SIGUSR2 after the hold: a gap of the calls (the target `clients`). The restore sends SIGUSR2 again |
175
178
 
176
179
  A cgroup path that is not the scope of one container is refused. A write to the root of the node throttles or freezes each pod of the node.
177
180
 
@@ -219,6 +222,15 @@ From 0.4.0, a window can count from the heal (`heal`), the first act that ends t
219
222
  - A denial after the window, before the last update after the delete and one more call, is a `tail`. A denied call has no pod, because the sidecar answers it. Thus the judge reads the last update of each server sidecar.
220
223
  - `calibrationOf` gives the updates after the apply by node, and the updates after the delete (`heal`).
221
224
 
225
+ From 0.6.0, the prediction `ends` is the end of the socket of each client, from the `sock` lines. A client of the run is a client with an answer before the reference.
226
+
227
+ - With a band, the first close of a socket of each client after the reference must be in the band ± δ. It must be the socket of its last answer. The main value is the middle of these times, in `s`.
228
+ - With no band, no socket of a client can close after the reference. The first request of each client after the heal must be `ok` on the socket of its last answer. The main value is the count of the closes, in `{socket}`.
229
+ - With a window (`window`), the failed requests must also agree with that window. The observation lasts until each socket can end, and until each request of the window ended.
230
+ - The facts of a run give the first close of each client after the reference (`closes`).
231
+
232
+ A new member of the union `Prediction`, `ends`, and a new member of the union `TriggerKind`, `idle`, break a reader that switches on them. The union `Target` has the new member `clients` too. Each is a breaking change before 1.0.0, thus the version is 0.6.0. The record format stays `/2`, with the optional field `closes`: the lines `sock` and `gap` are new lines of a run.
233
+
222
234
  A new member of the union `Target`, `servers`, breaks a reader that switches on it. `KindLab` also has a new method, `tables`, which each implementation must have. Each is a breaking change before 1.0.0, thus the version is 0.5.0. The record format stays `/2`: the lines `tables`, `ztunnel` and `ztunnel-pod` are new lines of a run.
223
235
 
224
236
  A new member of the union `TriggerKind` breaks a reader that switches on it. That is a breaking change before 1.0.0, thus the version is 0.4.0. The record format stays `/2`, with the optional fields `account`, `outcomes`, `updates` and `healUpdates`, and the reference `heal`.
package/build/index.d.ts CHANGED
@@ -1,4 +1,4 @@
1
- import { $ as updatesOf, $t as Replica, A as TriggerV1, At as transitionOf, B as RunCheck, Bt as showBand, C as KindCellResult, Ct as rulesOf, D as PredictionV1, Dt as sliceEmptyOf, E as LoadSample, Et as sliceDropOf, F as KIND_NOISE, Ft as REDIS, G as judgeKind, Gt as KubeletProbe, H as checkRun, Ht as Declaration, I as KindDecision, It as SERVER, J as propagationOf, Jt as MeshMode, K as observationAfter, Kt as Mesh, L as KindVerdict, Lt as bandOf, M as upgrade, Mt as HEALTH_PORT, N as upgradeCell, Nt as LOG_LABEL, O as RunCheckV1, Ot as slicesOf, P as Deleted, Pt as PORT, Q as unitOf, Qt as Reference, R as LAST_SLACK, Rt as seconds, S as KindCellRecordV1, St as restartsAt, T as KindVerdictV1, Tt as signalsOf, U as deletionsOf, Ut as KindCell, V as RunFacts, Vt as Band, W as factsOf, Wt as KindRun, X as referenceOf, Xt as Prediction, Y as propagationSource, Yt as PodSettings, Z as reusesReset, Zt as RedisDeclaration, _ as ClusterManifest, _n as nodeNames, _t as outcomesOf, a as STAGES, an as Workload, at as Request, b as FORMAT_V1, bt as recordsOf, c as readRecord, cn as KindScenario, ct as backendsAt, d as calibrationOf, dn as NodeImage, dt as deletionOf, en as ResetReuse, et as Endpoint, f as cellsTable, fn as cidrOverlap, ft as dropOf, g as BandV1, gn as kubeProxyPatch, gt as kernelCounts, h as timeSpread, hn as kindConfig, ht as intakeOf, i as quantileOf, in as TriggerKind, it as RecordLine, j as judge, jt as CLIENT, k as RunFactsV1, kt as stoppingOf, l as Judged, ln as KubeProxyMode, lt as byClients, m as predictionText, mn as goDuration, mt as failuresOf, n as medianInterval, nn as Template, nt as PodLine, o as kindCellRecordSchema, on as KUBE_PROXY_MODES, ot as RulesLine, p as medianText, pn as cidrRange, pt as emptyOf, q as observationEnd, qt as MeshFlavor, r as middle, rn as Trigger, rt as PropagationSource, s as kindCellRecordV1Schema, sn as KindConfig, st as SliceLine, t as RELATIVE, tn as Stack, tt as Line, u as TimeSpread, un as NODE_IMAGES, ut as byNode, v as EnvironmentManifest, vn as scenario, vt as podsAt, w as KindCellV1, wt as signalOf, x as KindCellRecord, xt as requestsOf, y as FORMAT, yn as scenarioProblems, yt as podsOf, z as Propagated, zt as secondsOf } from "./judge-BS9pk96o.js";
1
+ import { $ as updatesOf, $t as RedisDeclaration, A as TriggerV1, At as socketsOf, B as RunCheck, Bt as seconds, C as KindCellResult, Ct as restartsAt, D as PredictionV1, Dt as sliceDropOf, E as LoadSample, Et as signalsOf, F as KIND_NOISE, Ft as LOG_LABEL, G as judgeKind, Gt as KindCell, H as checkRun, Ht as showBand, I as KindDecision, It as PORT, J as propagationOf, Jt as Mesh, K as observationAfter, Kt as KindRun, L as KindVerdict, Lt as REDIS, M as upgrade, Mt as transitionOf, N as upgradeCell, Nt as CLIENT, O as RunCheckV1, Ot as sliceEmptyOf, P as Deleted, Pt as HEALTH_PORT, Q as unitOf, Qt as Prediction, R as LAST_SLACK, Rt as SERVER, S as KindCellRecordV1, Sn as scenarioProblems, St as requestsOf, T as KindVerdictV1, Tt as signalOf, U as deletionsOf, Ut as Band, V as RunFacts, Vt as secondsOf, W as factsOf, Wt as Declaration, X as referenceOf, Xt as MeshMode, Y as propagationSource, Yt as MeshFlavor, Z as reusesReset, Zt as PodSettings, _ as ClusterManifest, _n as goDuration, _t as kernelCounts, a as STAGES, an as Trigger, at as Request, b as FORMAT_V1, bn as nodeNames, bt as podsOf, c as readRecord, cn as Workload, ct as SocketLine, d as calibrationOf, dn as KindScenario, dt as byNode, en as Reference, et as Endpoint, f as cellsTable, fn as KubeProxyMode, ft as deletionOf, g as BandV1, gn as cidrRange, gt as intakeOf, h as timeSpread, hn as cidrOverlap, ht as failuresOf, i as quantileOf, in as Template, it as RecordLine, j as judge, jt as stoppingOf, k as RunFactsV1, kt as slicesOf, l as Judged, ln as KUBE_PROXY_MODES, lt as backendsAt, m as predictionText, mn as NodeImage, mt as emptyOf, n as medianInterval, nn as ResetReuse, nt as PodLine, o as kindCellRecordSchema, on as TriggerKind, ot as RulesLine, p as medianText, pn as NODE_IMAGES, pt as dropOf, q as observationEnd, qt as KubeletProbe, r as middle, rn as Stack, rt as PropagationSource, s as kindCellRecordV1Schema, sn as WindowPrediction, st as SliceLine, t as RELATIVE, tn as Replica, tt as Line, u as TimeSpread, un as KindConfig, ut as byClients, v as EnvironmentManifest, vn as kindConfig, vt as outcomesOf, w as KindCellV1, wt as rulesOf, x as KindCellRecord, xn as scenario, xt as recordsOf, y as FORMAT, yn as kubeProxyPatch, yt as podsAt, z as Propagated, zt as bandOf } from "./judge-B9kVpFE-.js";
2
2
  import { Quantity } from "@mark1russell7/quantity";
3
3
  import { CheckFn, Finding, HealthServer, Poll, Rule, Scenario, ServiceBuilder, ServiceDescription, Stage, Timing, TransportImpl, Verdict } from "@mark1russell7/service";
4
4
  import { RedisClient } from "@mark1russell7/service/redis";
@@ -114,8 +114,9 @@ export declare const hostsOf: (text: string, cluster: string) => ClusterHosts;
114
114
  * - `servers` is each replica of the service. Only the injector takes it: a rule in the network
115
115
  * namespace of each replica, with each client pod as a peer. A REJECT resets only the sender of
116
116
  * the segment that it matches, thus the rule goes where the segments of the clients arrive.
117
+ * - `clients` is the process of each client pod. Only a signal takes it: the gap of the trigger `idle`.
117
118
  */
118
- type Target = "server" | "redis" | "kube-proxy" | "servers";
119
+ type Target = "server" | "redis" | "kube-proxy" | "servers" | "clients";
119
120
  /** A step is one act of a fault, in the sequence of the plan. */
120
121
  type Step = {
121
122
  readonly do: "kubectl";
@@ -124,7 +125,7 @@ type Step = {
124
125
  } | {
125
126
  readonly do: "signal";
126
127
  readonly target: Target;
127
- readonly signal: "SIGKILL" | "SIGSTOP" | "SIGCONT";
128
+ readonly signal: "SIGKILL" | "SIGSTOP" | "SIGCONT" | "SIGUSR1" | "SIGUSR2";
128
129
  } | {
129
130
  readonly do: "pause";
130
131
  readonly target: Target;
@@ -628,5 +629,5 @@ export declare const toolPath: (tool: Tool, key?: string, cache?: string) => str
628
629
  */
629
630
  export declare const ensureTool: (tool: Tool, key?: string, cache?: string) => Promise<string>;
630
631
  //#endregion
631
- export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, type Stack, type Stamp, type Step, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, type Verdicts, type Workload, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stoppingOf, timeSpread, transitionOf, unitOf, updatesOf, upgrade, upgradeCell };
632
+ export { type Band, type BandV1, CLIENT, type Cluster, type ClusterHosts, type ClusterManifest, type Credentials, type Declaration, type Deleted, type DeletionWindow, type EdsSpec, Endpoint, type EnvironmentManifest, FORMAT, FORMAT_V1, type FaultPlan, HEALTH_PORT, type Judged, KIND_NOISE, KUBE_PROXY_MODES, type KindCell, type KindCellRecord, type KindCellRecordV1, type KindCellResult, type KindCellV1, type KindConfig, type KindDecision, type KindLab, type KindRun, type KindScenario, type KindVerdict, type KindVerdictV1, type KubeProxyMode, type KubeletProbe, LAST_SLACK, LOG_LABEL, type LdsSpec, Line, type ListenerCounts, type LoadSample, type Mesh, type MeshFlavor, type MeshImages, type MeshMode, NODE_IMAGES, type NodeImage, PORT, type Parts, PodLine, type PodSettings, type Prediction, type PredictionV1, type Propagated, type Propagation, PropagationSource, REDIS, RELATIVE, RecordLine, type RedisDeclaration, type Reference, type Replica, Request, type ResetReuse, RulesLine, type RunCheck, type RunCheckV1, type RunFacts, type RunFactsV1, type RunOptions, SERVER, STAGES, type ServiceRule, SliceLine, SocketLine, type Stack, type Stamp, type Step, type Target, type Template, type TimeSpread, type Tool, type Trigger, type TriggerKind, type TriggerV1, type Verdicts, type WindowPrediction, type Workload, backendsAt, bandOf, byClients, byNode, calibrationOf, cellsTable, checkRun, cidrOverlap, cidrRange, deletionOf, deletionsOf, dropOf, emptyOf, factsOf, failuresOf, goDuration, intakeOf, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, medianInterval, medianText, middle, nodeNames, observationAfter, observationEnd, outcomesOf, podsAt, podsOf, predictionText, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, referenceOf, requestsOf, restartsAt, reusesReset, rulesOf, scenario, scenarioProblems, seconds, secondsOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stoppingOf, timeSpread, transitionOf, unitOf, updatesOf, upgrade, upgradeCell };
632
633
  //# sourceMappingURL=index.d.ts.map
package/build/index.js CHANGED
@@ -1,4 +1,4 @@
1
- import { $ as slicesOf, A as medianInterval, B as intakeOf, C as propagationOf, D as unitOf, E as reusesReset, F as byNode, G as recordsOf, H as outcomesOf, I as deletionOf, J as rulesOf, K as requestsOf, L as dropOf, M as quantileOf, N as backendsAt, O as updatesOf, P as byClients, Q as sliceEmptyOf, R as emptyOf, S as observationEnd, T as referenceOf, U as podsAt, V as kernelCounts, W as podsOf, X as signalsOf, Y as signalOf, Z as sliceDropOf, _ as checkRun, a as calibrationOf, at as PORT, b as judgeKind, c as predictionText, ct as bandOf, d as FORMAT_V1, dt as showBand, et as stoppingOf, f as judge, g as LAST_SLACK, h as KIND_NOISE, i as readRecord, it as LOG_LABEL, j as middle, k as RELATIVE, l as timeSpread, lt as seconds, m as upgradeCell, n as kindCellRecordSchema, nt as CLIENT, o as cellsTable, ot as REDIS, p as upgrade, q as restartsAt, r as kindCellRecordV1Schema, rt as HEALTH_PORT, s as medianText, st as SERVER, t as STAGES, tt as transitionOf, u as FORMAT, ut as secondsOf, v as deletionsOf, w as propagationSource, x as observationAfter, y as factsOf, z as failuresOf } from "./judge-Bkbs7VVG.js";
1
+ import { $ as slicesOf, A as medianInterval, B as intakeOf, C as propagationOf, D as unitOf, E as reusesReset, F as byNode, G as recordsOf, H as outcomesOf, I as deletionOf, J as rulesOf, K as requestsOf, L as dropOf, M as quantileOf, N as backendsAt, O as updatesOf, P as byClients, Q as sliceEmptyOf, R as emptyOf, S as observationEnd, T as referenceOf, U as podsAt, V as kernelCounts, W as podsOf, X as signalsOf, Y as signalOf, Z as sliceDropOf, _ as checkRun, a as calibrationOf, at as LOG_LABEL, b as judgeKind, c as predictionText, ct as SERVER, d as FORMAT_V1, dt as secondsOf, et as socketsOf, f as judge, ft as showBand, g as LAST_SLACK, h as KIND_NOISE, i as readRecord, it as HEALTH_PORT, j as middle, k as RELATIVE, l as timeSpread, lt as bandOf, m as upgradeCell, n as kindCellRecordSchema, nt as transitionOf, o as cellsTable, ot as PORT, p as upgrade, q as restartsAt, r as kindCellRecordV1Schema, rt as CLIENT, s as medianText, st as REDIS, t as STAGES, tt as stoppingOf, u as FORMAT, ut as seconds, v as deletionsOf, w as propagationSource, x as observationAfter, y as factsOf, z as failuresOf } from "./judge-DNCI5vvt.js";
2
2
  import { valueIn } from "@mark1russell7/quantity";
3
3
  import { execFile, execFileSync } from "node:child_process";
4
4
  import { chmodSync, copyFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, realpathSync, renameSync, rmSync, writeFileSync } from "node:fs";
@@ -950,6 +950,24 @@ const planOf = (cell) => {
950
950
  ]
951
951
  }]
952
952
  };
953
+ case "idle": return {
954
+ inject: [{
955
+ do: "signal",
956
+ target: "clients",
957
+ signal: "SIGUSR1"
958
+ }],
959
+ hold: needs(trigger.hold, "a hold, the time of the gap"),
960
+ heal: [{
961
+ do: "signal",
962
+ target: "clients",
963
+ signal: "SIGUSR2"
964
+ }],
965
+ restore: [{
966
+ do: "signal",
967
+ target: "clients",
968
+ signal: "SIGUSR2"
969
+ }]
970
+ };
953
971
  }
954
972
  };
955
973
  /** This function tells if a trigger stops a pod of the service with its grace: a deletion, an eviction, a scale-in or a rollout. */
@@ -986,7 +1004,8 @@ const FILES = [
986
1004
  ["conntrack.ts", "src/conntrack.ts"],
987
1005
  ["drops.ts", "src/drops.ts"],
988
1006
  ["eds.ts", "src/eds.ts"],
989
- ["lds.ts", "src/lds.ts"]
1007
+ ["lds.ts", "src/lds.ts"],
1008
+ ["client-lines.ts", "src/client-lines.ts"]
990
1009
  ];
991
1010
  const shell = process.platform === "win32";
992
1011
  /** This function gives each file of a staged context, with its path in the context, in a stable sequence. */
@@ -2591,6 +2610,38 @@ const injectServers = async (ctx, action, replicas) => {
2591
2610
  t1: Math.max(...all.map((stamp) => stamp.t1))
2592
2611
  };
2593
2612
  };
2613
+ /**
2614
+ * This function sends a signal to the process of each client pod: the target `clients`. The nodes go
2615
+ * at the same time. SIGUSR1 finds the PID of each client, and SIGUSR2 uses it again, thus the heal of
2616
+ * a gap comes soon after its hold. When that PID fails, the client started again, and the lab finds
2617
+ * its PID. The stamp is the first act and the last end.
2618
+ */
2619
+ const signalClients = async (ctx, signal) => {
2620
+ const clients = [...ctx.tracker.pods.values()].filter((pod) => pod.labels["app"] === "client" && !pod.del && pod.node !== null);
2621
+ if (clients.length === 0) throw new Error(`the signal ${signal} needs clients`);
2622
+ const stamps = await Promise.all(clients.map(async (pod) => {
2623
+ const node = pod.node;
2624
+ const known = signal === "SIGUSR2" ? ctx.pids.get(pod.name) : void 0;
2625
+ if (known !== void 0) try {
2626
+ return await ctx.lab.act(node, {
2627
+ op: "signal",
2628
+ pid: known,
2629
+ signal
2630
+ });
2631
+ } catch {}
2632
+ const pid = await ctx.lab.pidOf(node, ctx.namespace, pod.name, CLIENT);
2633
+ ctx.pids.set(pod.name, pid);
2634
+ return ctx.lab.act(node, {
2635
+ op: "signal",
2636
+ pid,
2637
+ signal
2638
+ });
2639
+ }));
2640
+ return {
2641
+ t0: Math.min(...stamps.map((stamp) => stamp.t0)),
2642
+ t1: Math.max(...stamps.map((stamp) => stamp.t1))
2643
+ };
2644
+ };
2594
2645
  /** This function does one step of a fault plan. It gives the stamp of its act, or null for an act without a stamp. */
2595
2646
  const doStep = async (ctx, step, replicas, background) => {
2596
2647
  const { lab, cluster, namespace } = ctx;
@@ -2598,6 +2649,7 @@ const doStep = async (ctx, step, replicas, background) => {
2598
2649
  const fill = (args) => args.map((arg) => arg.replaceAll("{namespace}", namespace).replaceAll(POD, target.pod).replaceAll(NODE, target.node).replaceAll(VERSION, String(ctx.version)));
2599
2650
  const where = async (which) => {
2600
2651
  if (which === "servers") throw new Error("the target servers is a target of the injector only");
2652
+ if (which === "clients") throw new Error("the target clients is a target of a signal only");
2601
2653
  if (which === "server") return {
2602
2654
  node: target.node,
2603
2655
  pid: await lab.pidOf(target.node, namespace, target.pod, "app"),
@@ -2649,6 +2701,7 @@ const doStep = async (ctx, step, replicas, background) => {
2649
2701
  };
2650
2702
  }
2651
2703
  case "signal": {
2704
+ if (step.target === "clients") return signalClients(ctx, step.signal);
2652
2705
  const at = await where(step.target);
2653
2706
  return lab.act(at.node, {
2654
2707
  op: "signal",
@@ -3095,11 +3148,13 @@ const estimate = (cell, environments = 2) => {
3095
3148
  "app-stall": hold + 40,
3096
3149
  "cpu-throttle": hold + 40,
3097
3150
  rollout: hold + 50,
3098
- "authorization-policy": hold + 20
3151
+ "authorization-policy": hold + 20,
3152
+ idle: hold + 5
3099
3153
  };
3100
3154
  const after = observationAfter(cell);
3101
3155
  const observed = after === null ? 0 : hold + valueIn(after, "s");
3102
- const each = 25 + Math.max(observed, trigger[cell.trigger.kind] ?? (cell.template === "T18" ? hold + 35 : 15 + valueIn(cell.pod.preStop, "s")));
3156
+ const closes = cell.prediction.kind === "ends" && cell.prediction.band !== null ? valueIn(cell.prediction.band.hi, "s") + 3 : 0;
3157
+ const each = 25 + Math.max(observed, closes, trigger[cell.trigger.kind] ?? (cell.template === "T18" ? hold + 35 : 15 + valueIn(cell.pod.preStop, "s")));
3103
3158
  return (cell.mesh === void 0 ? 60 : 180) * environments + (cell.runs + environments) * each;
3104
3159
  };
3105
3160
  /**
@@ -3131,6 +3186,6 @@ const shardOf = (cells, index, count) => {
3131
3186
  return out;
3132
3187
  };
3133
3188
  //#endregion
3134
- export { ACCESS_LOG_FORMAT, CLIENT, CLIENT_ACCOUNT, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LDS_EVERY, LDS_FILTER, LOG_LABEL, MESH_IMAGES, MESH_WORKLOADS, NODE_IMAGES, O04, O09, O10, O16, POLICY, PORT, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, WATCHDOG, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, followsZtunnel, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, istioOperator, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, lintCell, listenerCountsOf, localPin, medianInterval, medianText, meshKey, middle, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, policyObject, pollsListeners, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, stackOf, stageLabImage, stallRemovalBand, stoppingOf, tarEntry, timeSpread, toolPath, transitionOf, unitOf, updatesOf, upgrade, upgradeCell, verdictsOf, writeResult };
3189
+ export { ACCESS_LOG_FORMAT, CLIENT, CLIENT_ACCOUNT, CONTROLLER, DOCUMENTED, DRAIN_RULES, EDS_EVERY, FLAG_CHECK, FORMAT, FORMAT_V1, HEALTH_PORT, ISTIOCTL, ISTIO_IMAGES, ISTIO_VERSION, KEPT_KERNEL, KIND, KIND_NOISE, KUBECTL, KUBE_PROXY_MODES, LAST_SLACK, LDS_EVERY, LDS_FILTER, LOG_LABEL, MESH_IMAGES, MESH_WORKLOADS, NODE_IMAGES, O04, O09, O10, O16, POLICY, PORT, REDIS, REDIS_IMAGE, RELATIVE, ROUTE, RULES_EVERY, SERVER, STAGES, WATCHDOG, backendsAt, backendsOf, bandOf, byClients, byNode, cacheFolder, calibrationOf, capOf, cellsTable, checkRun, cidrOverlap, cidrRange, clientObjects, clusterFilter, compose, createCluster, deletes, deletionOf, deletionWindow, deletionsOf, describeCell, dropOf, emptyOf, ensureTool, estimatedTime, factsOf, failuresOf, followsZtunnel, goDuration, hostsOf, installMesh, intakeAfter, intakeOf, istioOperator, judge, judgeKind, kernelCounts, kindCellRecordSchema, kindCellRecordV1Schema, kindConfig, kubeProxyPatch, kubeletBand, labImage, latchBand, lintCell, listenerCountsOf, localPin, medianInterval, medianText, meshKey, middle, namespaceObject, namespaceOf, nodeNames, observationAfter, observationEnd, openKindLab, outboundCluster, outcomesOf, pausedWindow, planOf, platformKey, podsAt, podsOf, policyObject, pollsListeners, predictionText, probeOf, propagationOf, propagationSource, quantileOf, readRecord, recordsOf, redisObjects, referenceOf, removalBand, requestsOf, restartBand, restartsAt, reusesReset, rulesOf, runKindCells, scenario, scenarioOf, scenarioProblems, seconds, secondsOf, serverCluster, serverObjects, sha256, shardOf, showBand, signalOf, signalsOf, sliceDropOf, sliceEmptyOf, slicesOf, socketsOf, stackOf, stageLabImage, stallRemovalBand, stoppingOf, tarEntry, timeSpread, toolPath, transitionOf, unitOf, updatesOf, upgrade, upgradeCell, verdictsOf, writeResult };
3135
3190
 
3136
3191
  //# sourceMappingURL=index.js.map