@telorun/k8s-runner 0.10.2 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -66,6 +66,220 @@ service-account token, seccomp `RuntimeDefault`); a sandbox RuntimeClass
66
66
  image is single-tenant, so install scripts run normally inside the trusted build
67
67
  (native/postinstall controllers work) with no cross-tenant cache to poison.
68
68
 
69
+ ## Watch sessions
70
+
71
+ Everything above describes a **run** session: one pod per Run click, terminal when
72
+ the workload exits. `POST /v1/sessions` with `mode: "watch"` asks for something
73
+ different — a **workspace that runs continuously**. One pod holds a shared
74
+ `/workspace` volume, a `workspace` container serving the editor's file routes, one
75
+ `app-<name>` container per application running `telo run --watch`, and optionally
76
+ a co-resident `agent` drawn from the `RUNNER_APPS` catalog. An edit then costs a
77
+ kernel reload instead of a pod: no schedule, no pull, no build.
78
+
79
+ Off unless `RUNNER_WATCH_SESSIONS` is set. Run sessions are entirely unaffected.
80
+
81
+ ### The pod
82
+
83
+ | Container | Image | Present | Writes | Credential |
84
+ | --- | --- | --- | --- | --- |
85
+ | `workspace` | The kernel image, over a runner-supplied manifest | Always | `/workspace` | None |
86
+ | `agent` | Operator catalog image | When `agent` is requested | `/workspace` | Operator env |
87
+ | `app-<name>` | The kernel image | One per app | `/workspace`, `/telo-cache/<name>`, scratch | Session-declared env only |
88
+
89
+ **The env split is the credential boundary.** It used to be structural (two pods)
90
+ and is now a code invariant: the operator env goes on `agent` alone, every
91
+ `app-<name>` gets the session's declared env and nothing else, and `workspace`
92
+ gets neither — it serves files and holds no secrets.
93
+
94
+ **A shared `fsGroup` is required**, and its absence is the kind of thing that
95
+ fails as a confusing "file not found" one reload after a write: every container
96
+ reads and writes `/workspace`, so they must share a GID or the agent writes files
97
+ the app cannot read.
98
+
99
+ **The agent is routed like an app port, and reported like one.** Its catalog
100
+ entry declares the `port` its image listens on, and the session refuses an agent
101
+ without one (`400 agent_port_undeclared`) rather than starting a container
102
+ nothing can reach. The pod's containers share one network namespace, so two of
103
+ them cannot bind the same port at all — but where an application declares the
104
+ agent's port the **manifest wins**: the session starts without the agent and says
105
+ so on its stream, rather than refusing to run the user's app over a container
106
+ they never asked for and cannot decline. Nothing else is arranged: the pod's containers
107
+ share one network namespace, so the port simply joins the session's own Service
108
+ and Ingress and answers at `<agentPort>-<sessionId>.<base-domain>`. The `running`
109
+ status carries it as an `agent` endpoint beside `endpoints` — separate, because
110
+ `endpoints` are the ports the user's applications declared, and an
111
+ operator-launched container is not one of them. It is reachable without auth for
112
+ as long as the session lives, which is the exposure an app session of the same
113
+ image already has.
114
+
115
+ **Nothing verifies that the agent listens where `port` says.** `port` tells the
116
+ runner where to route; what the image actually binds is configured separately
117
+ (the authoring agent reads `PORT`, defaulting to 8080). If the two disagree the
118
+ runner publishes a port and routes a host with nothing behind it, and the agent
119
+ appears to start while every request to it fails — neither backend watches the
120
+ agent's port the way both watch an application's. Keep `port` and any `PORT` in
121
+ the entry's `env` in agreement.
122
+
123
+ **The workspace surface is runner infrastructure, not agent functionality.** It is
124
+ part of the `/v1` session contract, so the runner owns its manifest and its
125
+ routes; the agent is one more writer on the volume beside the app containers,
126
+ using its own filesystem tools. It runs the plain kernel image over a manifest the
127
+ runner reconciles into a content-addressed ConfigMap — there is no third image to
128
+ build, and a runner upgrade that changes the manifest leaves running sessions
129
+ mounting the one they booted with.
130
+
131
+ **Watch sessions never build an image.** The build path exists to put a dependency
132
+ closure on disk before boot; a watch session resolves its own into the workspace
133
+ volume, which lives as long as the pod, so the download happens once per session
134
+ and every later reload resolves from local disk.
135
+
136
+ **One cache for the whole session.** The runner seeds `telo-workspace.yaml` at
137
+ the workspace root when the session starts, and the kernel anchors its `.telo`
138
+ cache at the directory holding that marker — so two apps importing the same
139
+ module resolve it once between them, and an app in a subdirectory does not get a
140
+ cache of its own. The application containers therefore carry no `TELO_CACHE_DIR`:
141
+ that variable OUTRANKS the marker, so setting it per app is exactly what would
142
+ undo this. Only the `workspace` container keeps an explicit root, because its own
143
+ manifest lives outside the workspace and the walk-up would never reach the
144
+ marker. A workspace that brings its own marker keeps it — overwriting one with a
145
+ real `modules:` list would change what `telo release` discovers.
146
+
147
+ **Egress moves into the session namespace.** Module fetches used to be scoped to
148
+ the build namespace; a watch session resolves them itself, so the session
149
+ namespace's NetworkPolicy has to reach module registries as well as the model
150
+ provider. Core NetworkPolicy is CIDR-only and registries sit behind rotating-IP
151
+ CDNs, so a locked-down operator needs a CNI with FQDN policy or an egress proxy
152
+ here — the same stated dependency the build namespace already carries, one
153
+ namespace over.
154
+
155
+ ### Two nouns on one stream
156
+
157
+ *Session status* (`status` events) is `starting` / `running` / `suspended` /
158
+ `stopped` / `failed`. *Run outcome* (`run` events) is one per app per reload
159
+ generation:
160
+
161
+ ```json
162
+ { "type": "run", "app": "web", "generation": 3, "phase": "started", "trigger": "watch" }
163
+ { "type": "run", "app": "web", "generation": 3, "phase": "completed", "code": 0, "durationMs": 412 }
164
+ { "type": "run", "app": "worker", "generation": 4, "phase": "failed", "reason": "ERR_MANIFEST_VALIDATION_FAILED" }
165
+ ```
166
+
167
+ A one-shot Runnable finishing emits `run` with `phase: "completed"` and leaves
168
+ `status: running` — the session is alive and the next edit starts that app's next
169
+ generation. `generation` is monotonic per app and starts at 1.
170
+
171
+ Those events are **projected** from the kernel debug stream, not parsed out of the
172
+ terminal: a watch session always runs with `--inspect` on, and `Kernel.Starting` /
173
+ `Kernel.Stopped` bracket each generation on one debug connection that survives
174
+ reloads. The one thing that stream did not carry is a manifest that fails to load
175
+ at all, so the CLI now emits `Kernel.RunFailed` (`{ phase: "load" | "start", code?,
176
+ message }`) — a dotted event name inside an existing frame kind, so it obliges no
177
+ other runtime.
178
+
179
+ ### Routes
180
+
181
+ | Route | Purpose |
182
+ | --- | --- |
183
+ | `GET /v1/sessions/:id/workspace` | Content-hash tree the editor diffs against its own files |
184
+ | `POST /v1/sessions/:id/workspace` | Apply a `{ write: [{path, content, encoding?}], delete: [path] }` change set |
185
+ | `GET /v1/sessions/:id/workspace/file?path=` | One file's contents |
186
+ | `POST /v1/sessions/:id/reload?app=<name>` | Re-run one app with no file change (omit `app` for all) |
187
+ | `PUT /v1/sessions/:id/apps` | Change the running app set (checkpoint + pod recreate) |
188
+ | `POST /v1/sessions/:id/resume` | Bring a suspended session back under the same id |
189
+
190
+ ### A port set that changes on reload
191
+
192
+ Adding a `ports:` entry is as ordinary an edit as adding an import, and a
193
+ container may bind any port regardless of what the pod spec declares — so without
194
+ handling, the app listens and is simply unreachable: no ingress, no error, no
195
+ event.
196
+
197
+ The kernel re-resolves its `ports:` block on every load and says so on the stream
198
+ the runner is already reading (`Kernel.PortsResolved`), so nothing re-parses a
199
+ manifest on the reload path. The runner patches the Service and the Ingress live
200
+ and emits an `endpoints` event; a pod's `containerPort` list is documentation, so
201
+ this costs no pod recreate. A port another app in the session already declares
202
+ cannot be routed — session hosts carry no app name — and comes back on that same
203
+ event as `rejected` rather than being dropped.
204
+
205
+ It routes the DECLARED set, not what happened to bind. A port the manifest never
206
+ declared is not exposed, and "did anything actually bind it" is already answered
207
+ per port by the reachability watcher (`checking` → `reachable` / `unreachable`).
208
+ Keying on a listening event instead would rest on a per-module convention: a
209
+ transport whose kind does not emit one would silently get no routing.
210
+
211
+ ### Reload and the app set
212
+
213
+ `reload` exists because `--watch` reloads on change, and pressing Run again after a
214
+ one-shot app completed is not a change. It touches the named app's entry manifest
215
+ through the same path everything else uses, so it needs no signalling into the
216
+ container, no shared PID namespace and no `exec` — **RBAC gains only `configmaps:
217
+ get, create` and `update` on services/ingresses**, the latter so a reload that
218
+ changes an app's declared port set can re-patch its routing. Without that, adding a
219
+ `ports:` entry leaves the app bound to a port with no ingress, no error and no
220
+ event.
221
+
222
+ Changing the app set costs a pod recreate because a pod's container list is fixed
223
+ at creation. That is the only editing action in the design that costs a pod, and
224
+ it reuses suspend/resume rather than adding a second path.
225
+
226
+ ### Suspend and resume
227
+
228
+ A session is no longer a pod. With no SSE/WS subscriber for
229
+ `RUNNER_WATCH_IDLE_SECONDS`, the runner snapshots the workspace, deletes the pod
230
+ and keeps the session record — `status: suspended`, which is deliberately **not**
231
+ terminal. `POST /v1/sessions/:id/resume` creates a fresh pod seeded from that
232
+ checkpoint under the same session id. Aggressive reaping is what makes per-visitor
233
+ watch sessions affordable, and the checkpoint is what makes aggressive reaping
234
+ safe; they only work as a pair.
235
+
236
+ **The editor holds the authoritative workspace; the checkpoint is a cache.** The
237
+ runner is a single replica with an in-memory registry, so a redeploy, crash or
238
+ node move drops every suspended session and `resume` answers `404`. That is by
239
+ design and it buys less than it looks: a durable suspended workspace is only
240
+ meaningful when there is an identity to reattach it to, and accounts are an
241
+ explicit non-goal. A watch session exists because an editor is driving it, that
242
+ editor already holds every file and already diffs its own copy against
243
+ `GET /workspace`, so a `404` costs one change set. Two consequences, stated rather
244
+ than discovered: a suspended session is best-effort, and a watch session with no
245
+ editor attached and unsaved agent writes is the one place work can be lost —
246
+ bounded by `RUNNER_WORKSPACE_CHECKPOINT_SECONDS`.
247
+
248
+ **Capacity changes shape.** Concurrency becomes bounded by simultaneous *editors*
249
+ rather than simultaneous *runs*; the run-session ceilings were sized for the
250
+ opposite assumption, which is why watch has its own.
251
+
252
+ ### `io` — terminal or separated streams
253
+
254
+ Each app declares `io: "tty"` (default) or `io: "streams"`. The difference is
255
+ observable **to the application**, not just to the client: `isatty()` drives
256
+ colour, line-versus-block buffering, progress bars and prompts, so a loop that is
257
+ always a PTY systematically hides how the app behaves in production.
258
+
259
+ | | `tty` | `streams` |
260
+ | --- | --- | --- |
261
+ | Output | One merged stream, as a terminal produces | Separated at the source |
262
+ | `/io` resize | Yes | Rejected — meaningless without a PTY |
263
+ | `CLICOLOR_FORCE` | Injected | **Not** injected |
264
+
265
+ Nothing is invented at the transport layer: the Pod `attach` subresource without a
266
+ TTY already gives separate stdout and stderr channels. The TTY is what collapses
267
+ it. `streams` forces nothing *off* either — with no terminal the colour precedence
268
+ already resolves to no colour, and an explicit `NO_COLOR` would sit above an app's
269
+ own `color: always` and suppress a decision worth observing.
270
+
271
+ `GET /v1/sessions/:id/io?app=<name>` attaches to one app's terminal; `?app=` is
272
+ required whenever the session runs more than one, since there is no defensible
273
+ default among several. Every binary frame is `[seq:4 BE][stream:1][payload]`.
274
+
275
+ ### Ports are unique across the whole session
276
+
277
+ Session hosts are `<port>-<sessionId>.<base-domain>`, a single label, so two apps
278
+ both listening on 3000 would collide with nothing to distinguish them. That is a
279
+ `400 port_conflict` at session create rather than an app name added to the host
280
+ scheme: the user controls both manifests, and a rejected request is a better
281
+ outcome than a URL that silently reaches the wrong app.
282
+
69
283
  ## Configuration (env)
70
284
 
71
285
  | Env | Default | Purpose |
@@ -74,7 +288,7 @@ image is single-tenant, so install scripts run normally inside the trusted build
74
288
  | `PORT` | `8062` | HTTP listen port |
75
289
  | `RUNNER_DISPLAY_NAME` | `Telo Runner` | Display name advertised on `/v1/capabilities` (the editor's runner label) |
76
290
  | `RUNNER_DESCRIPTION` | `Runs the Telo application in a cloud environment` | Description advertised on `/v1/capabilities` |
77
- | `RUNNER_APPS` | _(unset → no apps)_ | JSON map of operator-predefined apps launchable by name (chart: inline `apps.catalog`, or `apps.catalogSecret` referencing a Secret holding the JSON — use the Secret whenever entries embed secrets in `env`); see the docker-runner README for the entry shape. App sessions run the catalog image directly as a pod — no on-cluster build |
291
+ | `RUNNER_APPS` | _(unset → no apps)_ | JSON map of operator-predefined apps launchable by name (chart: inline `apps.catalog`, or `apps.catalogSecret` referencing a Secret holding the JSON — use the Secret whenever entries embed secrets in `env`); see the docker-runner README for the entry shape, including the `port` an entry must declare to be usable as a session's co-resident `agent`. App sessions run the catalog image directly as a pod — no on-cluster build |
78
292
  | `RUNNER_APP_MAX_CPU` | `500m` | CPU ceiling for predefined-app pods (separate from the anonymous-session ceiling) |
79
293
  | `RUNNER_APP_MAX_MEMORY` | `512Mi` | Memory ceiling for predefined-app pods |
80
294
  | `RUNNER_APP_MAX_TTL_SECONDS` | `21600` | Wall-clock TTL for predefined-app pods (agent sessions are long-lived) |
@@ -90,6 +304,13 @@ image is single-tenant, so install scripts run normally inside the trusted build
90
304
  | `RUNNER_MAX_TTL_SECONDS` | `3600` | Wall-clock TTL (Pod `activeDeadlineSeconds`) |
91
305
  | `RUNNER_MAX_EPHEMERAL_STORAGE` | `512Mi` | Per-Pod ephemeral-storage ceiling |
92
306
  | `RUNNER_MAX_SESSIONS` | `32` | Global session backstop; at capacity the oldest exited session is evicted before a new run is rejected |
307
+ | `RUNNER_WATCH_SESSIONS` | `false` | Server-side gate. Watch sessions are never client-requestable when off |
308
+ | `RUNNER_WATCH_IDLE_SECONDS` | `300` | No SSE/WS subscriber for this long → suspend |
309
+ | `RUNNER_WATCH_MAX_TTL_SECONDS` | `21600` | Pod deadline for a watch session. One deadline covers the agent and the app containers, so it takes the longer ceiling and lets idleness do the real work |
310
+ | `RUNNER_WATCH_MAX_SESSIONS` | `8` | Concurrency ceiling for watch sessions, separate from `RUNNER_MAX_SESSIONS` |
311
+ | `RUNNER_WATCH_RELOAD_LIMIT` | `30` | Per-session reloads per minute |
312
+ | `RUNNER_WATCH_SUSPENDED_TTL_SECONDS` | `86400` | How long a suspended session record is retained before eviction. Deliberately not the pod deadline: that bounds a pod, so on its own nothing would ever evict a suspended record |
313
+ | `RUNNER_WORKSPACE_CHECKPOINT_SECONDS` | `30` | How often the runner pulls a whole-tree workspace snapshot |
93
314
  | `RUNNER_EXIT_TTL_MS` | `14400000` | How long exited sessions stay in the registry (so the editor can re-attach and replay their history after a reload) before eviction |
94
315
  | `RUNNER_TERMS_FILE` | _(unset)_ | Path to the agreement file (plain text / markdown), read at startup — mount it from a `ConfigMap` (e.g. `/etc/telo/terms.md`). Setting this (or `RUNNER_TERMS_BODY`) enables terms: the runner advertises them on `/v1/capabilities` and rejects `POST /v1/sessions` with `428` unless the client sends `x-telo-accepted-terms` matching the version. An unreadable path fails startup. The public cloud should set this |
95
316
  | `RUNNER_TERMS_BODY` | _(unset)_ | Inline agreement text, for short notes; ignored when `RUNNER_TERMS_FILE` is set |
@@ -7,6 +7,11 @@ export interface KubernetesRunnerCapabilitiesOptions {
7
7
  defaultImage: string;
8
8
  terms?: RunnerTerms;
9
9
  imageEnum?: string[];
10
+ /** Whether the operator enabled watch sessions. Advertised so a client knows
11
+ * before it asks — a runner with watch off rejects the field. */
12
+ watch: boolean;
13
+ /** Catalog names admissible as a session's co-resident `agent`. */
14
+ agents?: string[];
10
15
  }
11
16
  /** What k8s-runner advertises on `/v1/capabilities`. The runner serves
12
17
  * untrusted/anonymous code under a hard-ceiling policy. `image` is a base-image
@@ -1 +1 @@
1
- {"version":3,"file":"capabilities.d.ts","sourceRoot":"","sources":["../src/capabilities.ts"],"names":[],"mappings":"AAAA,OAAO,EAEL,KAAK,kBAAkB,EACvB,KAAK,WAAW,EACjB,MAAM,sBAAsB,CAAC;AAE9B,MAAM,WAAW,mCAAmC;IAClD;wDACoD;IACpD,WAAW,EAAE,MAAM,CAAC;IACpB,WAAW,EAAE,MAAM,CAAC;IACpB,YAAY,EAAE,MAAM,CAAC;IACrB,KAAK,CAAC,EAAE,WAAW,CAAC;IACpB,SAAS,CAAC,EAAE,MAAM,EAAE,CAAC;CACtB;AAED;;;;;;;;;;;6EAW6E;AAC7E,wBAAgB,4BAA4B,CAC1C,IAAI,EAAE,mCAAmC,GACxC,kBAAkB,CAiBpB"}
1
+ {"version":3,"file":"capabilities.d.ts","sourceRoot":"","sources":["../src/capabilities.ts"],"names":[],"mappings":"AAAA,OAAO,EAEL,KAAK,kBAAkB,EACvB,KAAK,WAAW,EACjB,MAAM,sBAAsB,CAAC;AAE9B,MAAM,WAAW,mCAAmC;IAClD;wDACoD;IACpD,WAAW,EAAE,MAAM,CAAC;IACpB,WAAW,EAAE,MAAM,CAAC;IACpB,YAAY,EAAE,MAAM,CAAC;IACrB,KAAK,CAAC,EAAE,WAAW,CAAC;IACpB,SAAS,CAAC,EAAE,MAAM,EAAE,CAAC;IACrB;sEACkE;IAClE,KAAK,EAAE,OAAO,CAAC;IACf,mEAAmE;IACnE,MAAM,CAAC,EAAE,MAAM,EAAE,CAAC;CACnB;AAED;;;;;;;;;;;6EAW6E;AAC7E,wBAAgB,4BAA4B,CAC1C,IAAI,EAAE,mCAAmC,GACxC,kBAAkB,CAyBpB"}
@@ -16,7 +16,15 @@ export function kubernetesRunnerCapabilities(opts) {
16
16
  return {
17
17
  displayName,
18
18
  description,
19
- features: { io: true, ports: true },
19
+ features: {
20
+ // Both attach modes: the kubernetes attach subresource without a TTY
21
+ // already gives separate stdout and stderr channels, so `streams` invents
22
+ // nothing at the transport layer.
23
+ io: ["tty", "streams"],
24
+ ports: true,
25
+ watch: opts.watch,
26
+ ...(opts.agents && opts.agents.length > 0 ? { agents: opts.agents } : {}),
27
+ },
20
28
  config: {
21
29
  schema: sessionConfigSchema({
22
30
  imageDefault: defaultImage,
@@ -1 +1 @@
1
- {"version":3,"file":"capabilities.js","sourceRoot":"","sources":["../src/capabilities.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,mBAAmB,GAGpB,MAAM,sBAAsB,CAAC;AAY9B;;;;;;;;;;;6EAW6E;AAC7E,MAAM,UAAU,4BAA4B,CAC1C,IAAyC;IAEzC,MAAM,EAAE,WAAW,EAAE,WAAW,EAAE,YAAY,EAAE,KAAK,EAAE,SAAS,EAAE,GAAG,IAAI,CAAC;IAC1E,OAAO;QACL,WAAW;QACX,WAAW;QACX,QAAQ,EAAE,EAAE,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE;QACnC,MAAM,EAAE;YACN,MAAM,EAAE,mBAAmB,CAAC;gBAC1B,YAAY,EAAE,YAAY;gBAC1B,QAAQ,EAAE,IAAI;gBACd,SAAS;gBACT,qBAAqB,EACnB,iKAAiK;aACpK,CAAC;SACH;QACD,KAAK;KACN,CAAC;AACJ,CAAC"}
1
+ {"version":3,"file":"capabilities.js","sourceRoot":"","sources":["../src/capabilities.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,mBAAmB,GAGpB,MAAM,sBAAsB,CAAC;AAiB9B;;;;;;;;;;;6EAW6E;AAC7E,MAAM,UAAU,4BAA4B,CAC1C,IAAyC;IAEzC,MAAM,EAAE,WAAW,EAAE,WAAW,EAAE,YAAY,EAAE,KAAK,EAAE,SAAS,EAAE,GAAG,IAAI,CAAC;IAC1E,OAAO;QACL,WAAW;QACX,WAAW;QACX,QAAQ,EAAE;YACR,qEAAqE;YACrE,0EAA0E;YAC1E,kCAAkC;YAClC,EAAE,EAAE,CAAC,KAAK,EAAE,SAAS,CAAC;YACtB,KAAK,EAAE,IAAI;YACX,KAAK,EAAE,IAAI,CAAC,KAAK;YACjB,GAAG,CAAC,IAAI,CAAC,MAAM,IAAI,IAAI,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;SAC1E;QACD,MAAM,EAAE;YACN,MAAM,EAAE,mBAAmB,CAAC;gBAC1B,YAAY,EAAE,YAAY;gBAC1B,QAAQ,EAAE,IAAI;gBACd,SAAS;gBACT,qBAAqB,EACnB,iKAAiK;aACpK,CAAC;SACH;QACD,KAAK;KACN,CAAC;AACJ,CAAC"}
@@ -2,18 +2,11 @@ import type { RunnerBackend } from "@telorun/runner-core";
2
2
  import type { BundleStore } from "../bundle-store.js";
3
3
  import type { K8sRunnerConfig } from "../config.js";
4
4
  import type { KubeClient } from "./client.js";
5
+ export { podFailureMessage } from "./pod-status.js";
5
6
  export interface K8sBackendDeps {
6
7
  kube: KubeClient;
7
8
  config: K8sRunnerConfig;
8
9
  bundleStore: BundleStore;
9
10
  }
10
11
  export declare function createKubernetesBackend(deps: K8sBackendDeps): RunnerBackend;
11
- /**
12
- * Builds an actionable failure message from a terminal Pod status. Init
13
- * containers are inspected first: a failed init container leaves the main
14
- * container unstarted, so reading only `containerStatuses` would fall through
15
- * to the bare "pod failed". For prebuilt session pods the common failure is the
16
- * main container itself (image pull, OOM, a non-zero exit).
17
- */
18
- export declare function podFailureMessage(obj: unknown): string;
19
12
  //# sourceMappingURL=backend.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"backend.d.ts","sourceRoot":"","sources":["../../src/k8s/backend.ts"],"names":[],"mappings":"AAGA,OAAO,KAAK,EAMV,aAAa,EACd,MAAM,sBAAsB,CAAC;AAG9B,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,oBAAoB,CAAC;AACtD,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,cAAc,CAAC;AAEpD,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAkB9C,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,UAAU,CAAC;IACjB,MAAM,EAAE,eAAe,CAAC;IACxB,WAAW,EAAE,WAAW,CAAC;CAC1B;AAED,wBAAgB,uBAAuB,CAAC,IAAI,EAAE,cAAc,GAAG,aAAa,CA0X3E;AAiGD;;;;;;GAMG;AACH,wBAAgB,iBAAiB,CAAC,GAAG,EAAE,OAAO,GAAG,MAAM,CAOtD"}
1
+ {"version":3,"file":"backend.d.ts","sourceRoot":"","sources":["../../src/k8s/backend.ts"],"names":[],"mappings":"AAGA,OAAO,KAAK,EAOV,aAAa,EACd,MAAM,sBAAsB,CAAC;AAG9B,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,oBAAoB,CAAC;AACtD,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,cAAc,CAAC;AAEpD,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAgB9C,OAAO,EAAE,iBAAiB,EAAE,MAAM,iBAAiB,CAAC;AAepD,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,UAAU,CAAC;IACjB,MAAM,EAAE,eAAe,CAAC;IACxB,WAAW,EAAE,WAAW,CAAC;CAC1B;AAED,wBAAgB,uBAAuB,CAAC,IAAI,EAAE,cAAc,GAAG,aAAa,CA0Y3E"}
@@ -4,6 +4,9 @@ import { clampLimits } from "../limits.js";
4
4
  import { ensureSessionImage } from "./image-build.js";
5
5
  import { buildSessionIngress, buildSessionService, endpointsFor } from "./ingress.js";
6
6
  import { buildAppPod, buildSessionPod, INSPECT_PORT } from "./pod-spec.js";
7
+ import { deletePod, is404, msg, podFailureMessage, podStatus, podPhase, provisionMessage, terminalStatus, } from "./pod-status.js";
8
+ import { startWatchSession } from "./watch-session.js";
9
+ export { podFailureMessage } from "./pod-status.js";
7
10
  const RESIZE_CHANNEL = 4;
8
11
  /** How long a Pod may take to reach Running before the start is abandoned.
9
12
  * activeDeadlineSeconds only bounds an already-running Pod, so a stuck
@@ -13,7 +16,7 @@ const WATCH_REARM_DELAY_MS = 2_000;
13
16
  export function createKubernetesBackend(deps) {
14
17
  const { kube, config, bundleStore } = deps;
15
18
  const ns = config.sessionNamespace;
16
- async function probe(_probe) {
19
+ async function probe(probeConfig) {
17
20
  try {
18
21
  await kube.core.readNamespace({ name: ns });
19
22
  }
@@ -35,7 +38,22 @@ export function createKubernetesBackend(deps) {
35
38
  return { status: "ready" };
36
39
  }
37
40
  async function start(spec) {
41
+ // A watch session is a different pod shape and a different lifetime — it
42
+ // outlives its runs — so it takes its own path rather than accreting
43
+ // branches through this one. It also never builds an image: the build path
44
+ // exists to put a dependency closure on disk before boot, and a watch
45
+ // session fetches its own and keeps it for the pod's life.
46
+ if (spec.mode === "watch") {
47
+ return startWatchSession({ kube, config }, spec);
48
+ }
49
+ return startRunSession(spec);
50
+ }
51
+ async function startRunSession(spec) {
38
52
  const podName = `telo-run-${spec.sessionId}`;
53
+ // A run session is one application by construction — `apps` carries exactly
54
+ // one entry, defaulted by core when the request declared none.
55
+ const app = spec.apps[0];
56
+ const appName = app.name;
39
57
  // The /v1 contract carries no per-request limits yet, so `requested` is
40
58
  // undefined and the configured ceiling is always the effective limit. When
41
59
  // a control plane begins passing limits, plumb them here — the clamp is
@@ -52,7 +70,7 @@ export function createKubernetesBackend(deps) {
52
70
  sessionId: spec.sessionId,
53
71
  podName,
54
72
  env: spec.env,
55
- ports: spec.ports,
73
+ ports: app.ports,
56
74
  limits,
57
75
  image: spec.config.image,
58
76
  pullPolicy: spec.config.pullPolicy,
@@ -72,10 +90,10 @@ export function createKubernetesBackend(deps) {
72
90
  managedByLabel: config.managedByLabel,
73
91
  }, {
74
92
  bundle: spec.bundle,
75
- entryRelativePath: spec.entryRelativePath,
93
+ entryRelativePath: app.entryRelativePath,
76
94
  baseImage: spec.config.image || config.defaultImage,
77
95
  pullPolicy: spec.config.pullPolicy,
78
- onProgress: (message, done) => spec.onProgress("build", message, done),
96
+ onProgress: (message, done) => spec.onProgress("build", message, done, appName),
79
97
  });
80
98
  // The image is keyed on the dependency closure only, so deliver the
81
99
  // per-session body to the Pod's /app at boot via a tokenized, single-use URL.
@@ -84,9 +102,9 @@ export function createKubernetesBackend(deps) {
84
102
  config,
85
103
  sessionId: spec.sessionId,
86
104
  podName,
87
- entryRelativePath: spec.entryRelativePath,
105
+ entryRelativePath: app.entryRelativePath,
88
106
  env: spec.env,
89
- ports: spec.ports,
107
+ ports: app.ports,
90
108
  limits,
91
109
  image,
92
110
  bundleUrl,
@@ -115,9 +133,9 @@ export function createKubernetesBackend(deps) {
115
133
  const reachAbort = new AbortController();
116
134
  const stdin = new PassThrough();
117
135
  const stdout = new Writable({
118
- write(chunk, _enc, cb) {
136
+ write(chunk, encoding, cb) {
119
137
  if (chunk?.byteLength)
120
- spec.onOutput(Buffer.from(chunk));
138
+ spec.onOutput(appName, Buffer.from(chunk), "tty");
121
139
  cb();
122
140
  },
123
141
  });
@@ -154,7 +172,7 @@ export function createKubernetesBackend(deps) {
154
172
  if (readyFlipped || finished)
155
173
  return;
156
174
  readyFlipped = true;
157
- spec.onStatus({ kind: "running", endpoints: endpointsFor(config, spec.sessionId, spec.ports) });
175
+ spec.onStatus({ kind: "running", endpoints: endpointsFor(config, spec.sessionId, app.ports) });
158
176
  };
159
177
  let resolveRunning;
160
178
  let rejectRunning;
@@ -171,7 +189,7 @@ export function createKubernetesBackend(deps) {
171
189
  const provision = provisionMessage(obj);
172
190
  if (provision && provision !== lastProvision) {
173
191
  lastProvision = provision;
174
- spec.onProgress("provision", provision);
192
+ spec.onProgress("provision", provision, undefined, appName);
175
193
  }
176
194
  }
177
195
  if (phase === "Running" && !runningSeen) {
@@ -219,7 +237,7 @@ export function createKubernetesBackend(deps) {
219
237
  if (finished)
220
238
  return;
221
239
  try {
222
- const req = await kube.watch.watch(`/api/v1/namespaces/${ns}/pods`, { fieldSelector: `metadata.name=${podName}` }, (_type, obj) => handlePhase(obj), () => {
240
+ const req = await kube.watch.watch(`/api/v1/namespaces/${ns}/pods`, { fieldSelector: `metadata.name=${podName}` }, (type, obj) => handlePhase(obj), () => {
223
241
  if (finished)
224
242
  return;
225
243
  void reconcileOnce().then(() => {
@@ -270,11 +288,11 @@ export function createKubernetesBackend(deps) {
270
288
  }
271
289
  catch (err) {
272
290
  // Attach failure isn't fatal — status still flows; surface the degraded PTY.
273
- spec.onOutput(Buffer.from(`\r\n[runner] failed to attach PTY: ${msg(err)}\r\n`));
291
+ spec.onOutput(appName, Buffer.from(`\r\n[runner] failed to attach PTY: ${msg(err)}\r\n`), "tty");
274
292
  }
275
- if (config.sessionIngressBaseDomain && spec.ports.length > 0) {
276
- await createIngress(deps, spec.sessionId, podName, podUid, spec.ports).catch((err) => {
277
- spec.onOutput(Buffer.from(`\r\n[runner] failed to create ingress: ${msg(err)}\r\n`));
293
+ if (config.sessionIngressBaseDomain && app.ports.length > 0) {
294
+ await createIngress(deps, spec.sessionId, podName, podUid, app.ports).catch((err) => {
295
+ spec.onOutput(appName, Buffer.from(`\r\n[runner] failed to create ingress: ${msg(err)}\r\n`), "tty");
278
296
  });
279
297
  }
280
298
  // The Running watch event usually carries `podIP`; if it lagged, read the
@@ -296,19 +314,19 @@ export function createKubernetesBackend(deps) {
296
314
  if (ip) {
297
315
  void relayDebugStream({
298
316
  url: `http://${ip}:${INSPECT_PORT}/events`,
299
- onFrame: spec.onDebug,
317
+ onFrame: (frame) => spec.onDebug(appName, frame),
300
318
  signal: debugAbort.signal,
301
319
  });
302
320
  }
303
321
  else {
304
- spec.onOutput(Buffer.from("\r\n[runner] debug stream unavailable: pod IP unknown\r\n"));
322
+ spec.onOutput(appName, Buffer.from("\r\n[runner] debug stream unavailable: pod IP unknown\r\n"), "tty");
305
323
  }
306
324
  }
307
325
  // Reachability check: a workload bound to 127.0.0.1 (or listening on the
308
326
  // wrong port) is unreachable on the pod network and surfaces only as a
309
327
  // downstream 502. Watch each advertised tcp port from the runner and report
310
328
  // per-port state to studio's endpoint badge. Background; finish() aborts it.
311
- const tcpPorts = spec.ports.filter((p) => p.protocol === "tcp").map((p) => p.port);
329
+ const tcpPorts = app.ports.filter((p) => p.protocol === "tcp").map((p) => p.port);
312
330
  if (tcpPorts.length > 0) {
313
331
  void (async () => {
314
332
  const ip = await resolvePodIP();
@@ -318,13 +336,13 @@ export function createKubernetesBackend(deps) {
318
336
  // Couldn't resolve the pod IP to probe — report unverified rather than
319
337
  // leaving the badge spinning forever.
320
338
  for (const port of tcpPorts)
321
- spec.onReachability(port, "unreachable");
339
+ spec.onReachability(appName, port, "unreachable");
322
340
  return;
323
341
  }
324
342
  await watchReachability({
325
343
  host: ip,
326
344
  ports: tcpPorts,
327
- onState: (port, state) => spec.onReachability(port, state),
345
+ onState: (port, state) => spec.onReachability(appName, port, state),
328
346
  signal: reachAbort.signal,
329
347
  });
330
348
  })();
@@ -332,7 +350,7 @@ export function createKubernetesBackend(deps) {
332
350
  // `flipRunning` already announced `running` from the watch's Running
333
351
  // transition (which `await running` above waited on).
334
352
  return {
335
- writeStdin(bytes) {
353
+ writeStdin(app, bytes) {
336
354
  try {
337
355
  stdin.write(Buffer.from(bytes));
338
356
  }
@@ -340,7 +358,7 @@ export function createKubernetesBackend(deps) {
340
358
  /* stream ended */
341
359
  }
342
360
  },
343
- resize(cols, rows) {
361
+ resize(app, cols, rows) {
344
362
  if (!socket)
345
363
  return;
346
364
  try {
@@ -397,16 +415,6 @@ async function createIngress(deps, sessionId, podName, podUid, ports) {
397
415
  return;
398
416
  await kube.networking.createNamespacedIngress({ namespace: ns, body: ingress });
399
417
  }
400
- async function deletePod(kube, ns, name) {
401
- try {
402
- await kube.core.deleteNamespacedPod({ name, namespace: ns, gracePeriodSeconds: 0 });
403
- }
404
- catch (err) {
405
- // 404 = already gone (natural exit + GC). Anything else is a real failure.
406
- if (!is404(err))
407
- throw err;
408
- }
409
- }
410
418
  async function clusterReachable(kube) {
411
419
  try {
412
420
  await kube.core.listNamespace();
@@ -416,124 +424,4 @@ async function clusterReachable(kube) {
416
424
  return false;
417
425
  }
418
426
  }
419
- function podStatus(obj) {
420
- return obj?.status;
421
- }
422
- function podPhase(obj) {
423
- return podStatus(obj)?.phase;
424
- }
425
- /** A coming-up message for the studio feed while the Pod is still scheduling /
426
- * pulling / delivering the body / creating the container; undefined once running. */
427
- function provisionMessage(obj) {
428
- const status = podStatus(obj);
429
- if (status?.phase !== "Pending")
430
- return undefined;
431
- const containers = [
432
- ...(status.initContainerStatuses ?? []),
433
- ...(status.containerStatuses ?? []),
434
- ];
435
- for (const cs of containers) {
436
- const reason = cs.state?.waiting?.reason;
437
- if (reason)
438
- return humanizeWaitReason(reason);
439
- }
440
- return "Scheduling";
441
- }
442
- function humanizeWaitReason(reason) {
443
- switch (reason) {
444
- case "ContainerCreating":
445
- return "Creating container";
446
- case "PodInitializing":
447
- return "Delivering application";
448
- case "ErrImagePull":
449
- case "ImagePullBackOff":
450
- return "Pulling image";
451
- default:
452
- return reason;
453
- }
454
- }
455
- function terminalStatus(obj, userStopped) {
456
- if (userStopped)
457
- return { kind: "stopped" };
458
- const phase = podPhase(obj);
459
- if (phase === "Succeeded")
460
- return { kind: "exited", code: containerExitCode(obj) ?? 0 };
461
- return { kind: "failed", message: podFailureMessage(obj) };
462
- }
463
- // Exit code of the main session container — used to report a clean exit.
464
- function containerExitCode(obj) {
465
- const term = podStatus(obj)?.containerStatuses?.[0]?.state?.terminated;
466
- return typeof term?.exitCode === "number" ? term.exitCode : null;
467
- }
468
- const MAX_FAILURE_DETAIL = 500;
469
- /**
470
- * Builds an actionable failure message from a terminal Pod status. Init
471
- * containers are inspected first: a failed init container leaves the main
472
- * container unstarted, so reading only `containerStatuses` would fall through
473
- * to the bare "pod failed". For prebuilt session pods the common failure is the
474
- * main container itself (image pull, OOM, a non-zero exit).
475
- */
476
- export function podFailureMessage(obj) {
477
- const status = podStatus(obj);
478
- const fromContainer = firstContainerProblem(status);
479
- if (fromContainer)
480
- return fromContainer;
481
- if (status?.message)
482
- return truncateDetail(status.message);
483
- if (status?.reason)
484
- return status.reason;
485
- return "pod failed";
486
- }
487
- function firstContainerProblem(status) {
488
- const groups = [
489
- ["init container", status?.initContainerStatuses],
490
- ["container", status?.containerStatuses],
491
- ];
492
- for (const [label, statuses] of groups) {
493
- for (const cs of statuses ?? []) {
494
- const problem = containerStateProblem(cs.state) ?? containerStateProblem(cs.lastState);
495
- if (problem)
496
- return `${label} "${cs.name}" ${problem}`;
497
- }
498
- }
499
- return undefined;
500
- }
501
- function containerStateProblem(state) {
502
- const term = state?.terminated;
503
- if (term && term.exitCode !== 0) {
504
- const reason = term.reason ? `${term.reason} ` : "";
505
- const detail = term.message ? `: ${truncateDetail(term.message)}` : "";
506
- return `failed: ${reason}(exit code ${term.exitCode ?? "unknown"})${detail}`;
507
- }
508
- const waiting = state?.waiting;
509
- if (waiting?.reason && isBlockingWaitReason(waiting.reason)) {
510
- const detail = waiting.message ? `: ${truncateDetail(waiting.message)}` : "";
511
- return `waiting: ${waiting.reason}${detail}`;
512
- }
513
- return undefined;
514
- }
515
- // Benign transient reasons the kubelet reports while a Pod is still coming up.
516
- function isBlockingWaitReason(reason) {
517
- return reason !== "PodInitializing" && reason !== "ContainerCreating";
518
- }
519
- function truncateDetail(text) {
520
- const trimmed = text.trim();
521
- return trimmed.length > MAX_FAILURE_DETAIL ? `${trimmed.slice(0, MAX_FAILURE_DETAIL)}…` : trimmed;
522
- }
523
- function is404(err) {
524
- const e = err;
525
- return e?.statusCode === 404 || e?.code === 404 || e?.response?.statusCode === 404;
526
- }
527
- function msg(err) {
528
- if (err instanceof Error)
529
- return err.message;
530
- if (typeof err === "string")
531
- return err;
532
- try {
533
- return JSON.stringify(err);
534
- }
535
- catch {
536
- return String(err);
537
- }
538
- }
539
427
  //# sourceMappingURL=backend.js.map