@tdk-landscape/tdk-cli-core 1.3.49 → 1.3.51

package/openspec/changes/prioritized-cold-start/design.md

@@ -0,0 +1,133 @@
+# Design
+
+## Context
+
+See `proposal.md` (Why) for the motivating case. Relevant current behavior:
+
+- Whether an app resource starts automatically is already a single Tilt-native, landscape-wide switch: `RuntimeFlags.resolve()` (`engine/topologies/tilt/resources/orchestrator/apply_runtime_flags.star`) reads `TILT_AUTO_INIT_APPS` into `runtime_flags['auto_init_apps']`, and `_register_replicas(config, auto_init_apps)` (`engine/topologies/tilt/resources/orchestrator/apply_compose_resource_registration.star:117`) passes it straight through as `dc_resource(..., auto_init=auto_init_apps)`. `auto_init` is Tilt's own parameter: `auto_init=False` means Tilt builds the resource's image and creates its container as part of reconciliation, but does not start it until explicitly triggered (`tilt trigger <resource>`, or the Tilt UI/HTTP API). This is exactly the "built but not started" split the proposal needs — it already exists, just as one global flag instead of a per-resource one.
+- Sablier's premium implementation (`tdk-cli-extensions/premium/networking/sablier_container_cycle.star`) already assumes the container exists (Traefik's `traefik.docker.allownonrunning=true` plus `sablier.enable`/`sablier.group` labels) and only handles the docker-level stop/start of an existing container. It has no concept of Tilt's own build/create step, and no way to call back into Tilt.
+- `engine/lifecycle/orchestrator.star`'s `STARTUP_PHASES` and `should_wait_for_dependencies` compute Tilt `resource_deps` chains per manifest (database, NATS) already; that same data is what a wake needs to decide which other resources to bring up together.
+- Tilt exposes an HTTP API on `localhost:10350` (already referenced as the debug/inspection endpoint in `engine/paths.star:652` and as `tilt trigger <resource>` in `cli/src/utils/doctor-runtime.ts:486`) that includes a trigger endpoint equivalent to the `tilt trigger` CLI command. This design relies on that existing mechanism rather than adding a new one to Tilt.
+- Real Sablier (`sablierapp/sablier`, already vendored per `traefik_standalone.star`) already holds an incoming request open and serves a wait/loading response until the container it started passes its healthcheck, or a configured timeout elapses — this is stock Sablier behavior, not something this change adds. What's missing is only: (a) getting the container into a "created, stoppable" state before it's ever run once, and (b) also reaching into Tilt for anything Sablier's plain Docker start doesn't cover.
+
+**Verified against the live `tdk-erp-system` landscape and an isolated probe (2026-09-26), replacing the preflight guesses below:**
+- The user's exact URL, reproduced live: `curl http://api.tdk-erp-system.localhost/api/data-governance/health` returns `404 page not found`, and Traefik's own API confirms why — `GET /api/http/routers/data-governance-api-project@docker` returns `{"message": "router not found: data-governance-api-project@docker"}`. The container (`analytics-autogenerated-data-governance-api-1`) does exist, `docker inspect` shows it ran once, went `unhealthy`, and `exited` hours ago — Tilt's own `uiresource` for it shows `runtimeStatus: pending` (queued to rebuild, `order: 50`, behind whatever else is ahead of it). Traefik drops an exited container from routing entirely because its labels have no `traefik.docker.allownonrunning=true` (this project has no `sablier` block on it at all). So the concrete failure mode observed in production is "crashed and queued for rebuild behind everything else," not only "literally never started" — D3's Tilt-trigger call addresses both, since it doesn't care why a resource isn't running, only that it now should be first.
+- A throwaway `Tiltfile` + `docker-compose.yml` (two `busybox` services, one `dc_resource(auto_init=True)`, one `dc_resource(auto_init=False)`, run on an isolated port so the live landscape was untouched) showed: the `auto_init=False` resource gets **no container object at all** (`docker ps -a` shows nothing for it; its `uiresource` stays `runtimeStatus: none` / `updateStatus: none`) until triggered. `tilt trigger <name>` then creates *and* starts it from nothing in one call. This overturns D2's "Tilt pre-creates the container" assumption below — it does not. See the corrected D2/D3/D6.
+- Read against the real `sablierapp/sablier` v1.18 docs (matching the vendored version): Sablier has **no webhook/event-hook system at all**. Its entire model is pull-based — the reverse-proxy plugin asks Sablier's HTTP API for a session's status on every request, and Sablier itself asks the provider (Docker) for instance status; the "blocking strategy" (already assumed above) is exactly this loop held open until ready or timeout. There is nothing to attach a "also call this other URL" hook to.
+- The same docs describe two *native* mechanisms this design should reuse instead of inventing its own: (1) `sablier.group` — "start related workloads together... put an app and the database it needs in the same group, and a single request warms both at once" — but it requires every group member to itself carry `sablier.enable=true`; (2) Compose's own `depends_on` with `condition: service_healthy` / `service_completed_successfully` / etc. — Sablier reads the `com.docker.compose.depends_on` label Compose writes automatically and recursively starts+waits on dependencies, **even ones with no `sablier.*` labels at all**. TDK's generated compose deliberately omits `depends_on` today (`compose.star:298`: `# Note: depends_on removed - Tilt manages service startup order`) because Tilt's per-resource `resource_deps` already sequences non-deferred resources; that omission doesn't help a *deferred* resource, whose startup Tilt is explicitly not managing. See the corrected D2/D3/D4/D5.
+- A second isolated probe (2026-09-26, same technique as the `auto_init` one, fully torn down afterward) tested what happens when a `deferStart`-shaped resource's compose entry has `depends_on: {other: {condition: service_healthy}}` and *both* resources are `auto_init=False`: Tilt itself parses the compose `depends_on` and automatically adds a Tilt-level wait — triggering the dependent resource alone leaves it parked at `runtimeStatus: pending` with `status.waiting = {on: [<dependency>], reason: "waiting-for-dep"}` **indefinitely**; Tilt does not auto-trigger the dependency just because something waits on it. Only once the dependency was triggered separately did the wait clear and the dependent resource actually build and start. So `depends_on` alone is not sufficient to reproduce D5's "wake the dependency too" requirement under Tilt (it would work if these were being started via a bare `docker compose up`, but Tilt's own reconciliation gates on its own trigger state first) — the wake gateway (D3) must explicitly `tilt trigger` every `depends_on` dependency itself, not rely on the cascade. This confirms and promotes what was recorded as a fallback to the primary, required mechanism.
+- **A third finding overturned D5 entirely rather than just refining it:** `depends_on` is not just gated on Tilt's trigger state, it cannot validly reference a dependency at all in TDK's actual layout. `docker compose config` fails outright — `service "X" depends on undefined service "Y": invalid compose project` — whenever the referenced service is defined in a compose file registered via a *different* `docker_compose()` call, even one sharing the same `project_name` (tested with two files under one shared `project_name`: still rejected, confirming Tilt validates each registered file independently rather than merging same-project files for this purpose). Every real dependency a TDK resource has (database, NATS, its own migrator per `apply_migrator_orchestration.star:91`'s separate `docker_compose()` call) lives in a file generated by a different call than the resource's own. So `depends_on` cannot express any of D5's cases here, regardless of scoping — see the corrected D5.
+
+## Goals / Non-Goals
+
+**Goals:**
+- Make "start on first request" work for a resource that has never run this session, using the same manifest opt-in, Traefik wiring, and license gate as today's Sablier feature.
+- Reuse Tilt's existing `auto_init` mechanism per-resource instead of inventing a parallel scheduler.
+- Keep the free-tier stub/premium-override structure (`sablier_container_cycle.star` in both repos) intact - this is an extension of that module's contract, not a replacement.
+
+**Non-Goals:**
+- Redesigning Tilt's build/reconciliation loop or its trigger API. See proposal.md Non-goals for the licensing and footprint boundaries.
+- Handling a resource whose `sablier` block is added or toggled while `tdk up` is already running (that already requires a Tiltfile re-eval today, same as any other manifest change).
+
+## Decisions
+
+### D1: Deferred start is `auto_init=False` scoped per-resource, driven by the manifest, not the global flag
+
+Add a `deferStart` key to the manifest's existing `sablier` block (`engine/topologies/tilt/manifest/schema.star`'s `sablier` schema, alongside `enable`/`group`/`sessionDuration`), default `false`. `_register_replicas` (and any other call site that currently receives the single `auto_init_apps` flag for app resources) resolves per-resource auto-init as `auto_init_apps and not resource_defers_start(manifest)` instead of using `auto_init_apps` directly. The resource's **config-gen** and **docker_build** resources keep `auto_init=True` unconditionally (or follow `auto_init_config_gen`, already `True` by default) so the image still builds during `tdk up` — only the running container is held back.
+
+*Alternatives considered:*
+- A separate global "cold landscape" mode (e.g. `TILT_DEFER_ALL_APPS`): rejected because it can't express "just this one resource," which is the actual request (one API, not the whole landscape).
+- Piggybacking on `TILT_TRIGGER_MODE=manual` wholesale: rejected — that already exists and disables `enforce_migrator_deps` and replica counts landscape-wide; overloading it would change unrelated behavior for every resource, not just opted-in ones.
+
+### D2: Two genuinely different wake mechanisms, chosen by whether a container object exists
+
+Confirmed live: `auto_init=False` leaves **no container object** until triggered — there is nothing for Traefik's Docker provider to discover (no container, no labels, no route), which is a different situation from a resource that ran before and is now `exited` (that container and its labels already exist; Traefik would route to it if and only if `traefik.docker.allownonrunning=true` is present). Docker-label auto-discovery — the mechanism the existing premium Sablier module relies on entirely — can only ever cover the second case. The two cases need different plumbing:
+
+- **Previously-ran, now-stopped/exited** (the case actually reproduced live against `data-governance-api`): today's premium `sablier_container_cycle.star` labels already work once a resource opts in — `allownonrunning=true` is the only missing ingredient, and it's already emitted when `sablier.enable` is set. No new mechanism needed here beyond what D1/D4/D5 add.
+- **Never created this session** (the state proven by the `auto_init_probe` test): there is no container to attach Docker labels to, so the route has to come from somewhere else. See D6.
+
+*Alternative considered:* add an explicit `docker compose create <service>` step (without starting) right after image build, so every deferred resource always has a discoverable-but-stopped container from the start, keeping Docker-label discovery as the single mechanism for both cases. Rejected as the primary path: it re-adds exactly the container-creation cost (network setup, volume mounts) to `tdk up`'s critical path that deferring was meant to avoid, just shifted from "start" to "create," and it still needs D6's static route anyway for the moment between `tdk up` finishing and that create step actually completing. Recorded as a possible later optimization (pre-create eagerly to shave latency off the very first request) but not required for this change to work.
+
+### D3: One wake gateway in front of every deferred resource; it starts the resource directly when possible, falling back to Sablier's API or `tilt trigger`
+
+Since Sablier has no hook to piggyback on (see Context), the component that also needs to call Tilt was always going to be something this project writes, not a Sablier extension point. So rather than splitting "Sablier handles it, plus something else calls Tilt on the side," one small always-on gateway sits in front of every `deferStart` resource (behind D4's static route) and branches on whether a container object exists for it:
+
+- **Container exists (previously ran, now stopped/exited):** the gateway calls Sablier's own HTTP API (the same call Traefik's plugin would make) to get the existing Docker-label wake, and in parallel runs `tilt trigger <resource>` so Tilt's own state (e.g. a migrator that needs to re-run) catches up too — Sablier's Docker-level start alone doesn't guarantee that. In practice this branch is reached rarely: per D4's verified priority fix, once any container object exists at all, Traefik's Docker-label route (at its normal, higher priority) wins over this gateway's static route on the very next request, so ongoing traffic to an existing-but-stopped resource goes straight to Sablier's own native plugin, not through here. This branch mainly covers the brief window before Traefik's docker provider notices a just-created container.
+- **No container exists yet:** there is nothing for Sablier to manage (confirmed in Context — Sablier's Docker provider needs a labeled container to register, same constraint as Traefik's). This is the branch that matters for the proposal's actual motivating case (task 7.2's live `tdk-erp-system` run exercises exactly this), and it is **not** simply "call `tilt trigger` and wait" — see the queue-contention finding below. When the generated route carries the resource's compose invocation (project, compose files, env file, image — added by task 7.2's fix; see the note at the end of this section), the gateway runs `docker compose up -d --no-build <resource>` itself, the same command Tilt's own `<resource>-run-only` local_resource would run, using an already-built image. Only when no image exists yet (nothing has ever built this resource) does it fall back to `tilt trigger <resource>`, which both builds and starts it. Either way it then polls the resource's healthcheck itself, holding the request exactly the way Sablier's blocking strategy would.
+
+Either way, the gateway is the single thing the static route (D4) points at, and it's the same process regardless of which branch it takes — this only affects which downstream call it makes on a given request. In both branches, the gateway also explicitly `tilt trigger`s and polls every dependency named in the request's `X-Wake-Deps` header (D2/D4), rather than assuming either Sablier or Tilt cascades that on its own — confirmed necessary by the second probe in Context (Tilt parks a triggered resource on `waiting-for-dep` indefinitely if its dependency is itself untriggered) and doubly so by D5's finding that Compose `depends_on` cannot even validly express these dependencies in TDK's layout, so nothing but the gateway's own explicit handling was ever going to cover this.
+
+*Alternative:* always go through Tilt's trigger regardless of container state, and drop Sablier's Docker-level start entirely. Rejected: it makes every wake pay Tilt's reconciliation overhead even for the common case (a resource that only needs restarting), and it removes the one piece of this feature that doesn't depend on Tilt being reachable at all (Sablier + Traefik + Docker socket keep working even if the Tilt daemon itself is down, e.g. mid-`tilt up` restart). It also turned out to have a second, more serious problem, found only by testing against a real landscape rather than a small synthetic one (see below): `tilt trigger` only *queues* a resource — it does not run ahead of, or preempt, builds Tilt already has in flight.
+
+**Found live against the real `tdk-erp-system` landscape (2026-09-27), not caught by any synthetic probe:** `tilt trigger` alone does not solve the proposal's actual motivating problem when the landscape is under load. Tilt caps concurrent builds (a handful of slots; observed 3 in this landscape), and a re-vendor or config regeneration that touches many manifests queues a rebuild for every affected resource. A `deferStart` resource triggered while ~100 other resources were mid-rebuild sat queued behind them; the request timed out with a `504` after the gateway's full wait budget with no container ever created — precisely the "wait behind hundreds of resources" behavior the whole feature exists to avoid, just moved one layer down (from `tdk up`'s bring-up order to Tilt's build-queue order). **Fix:** `apply_compose_resource_registration.star` now also passes the resource's compose invocation (its Compose project name, absolute compose file paths, `.env` path if present, and declared image tag) into `generate_static_wake_route`'s `X-Wake-Compose-*`/`X-Wake-Image` headers; the standalone Traefik compose (`traefik_standalone.star`) mounts the project root read-only into the gateway container at the same absolute path so those compose files resolve identically there. The gateway's `ensureImage()` also retags Tilt's own `<repo>:tilt-<hash>` build as the declared `<repo>:dev` tag when needed, since a resource can be fully built without ever having run once (so the plain `:dev` tag was never created) — confirmed against the live landscape's own image list. This start path only ever creates/starts a container from an image that already exists; it never builds, so it cannot race or duplicate a build Tilt has in progress, and a resource that has genuinely never been built at all still correctly falls back to `tilt trigger`.
+
+**Implemented and verified live (2026-09-26), replacing speculation with what actually shipped:**
+- The gateway is real: `engine/assets/docker/wake-gateway/` (`decide.ts` + `server.ts`, a Bun HTTP server; `Dockerfile` bundling Bun, the Docker CLI, and the exact pinned `tilt` binary version this workspace runs). Config for a request (which resource, which container, which dependencies, which port) travels via `X-Wake-*` headers that a generated Traefik `headers` middleware injects — not the request path — so the original path/query reach the real backend unchanged once ready.
+- Confirmed live: `tilt trigger` from inside a container, reaching Tilt's HTTP port at `host.docker.internal:10350`, fails with `(403): invalid session token` even though the port is reachable — Tilt's HTTP API requires the same local session token its own CLI reads from `~/.tilt-dev/token` on the host. The gateway's compose service mounts `~/.tilt-dev` read-only; with that mount, the identical `tilt trigger` call succeeds.
+- Confirmed live: the Tilt/Compose *resource* name (what `tilt trigger` and Compose's own `depends_on` use) is not the same string as the Docker *container* name Compose actually creates (`<project>-<service>-1`, e.g. `analytics-autogenerated-data-governance-api-1` for resource `data-governance-api`) — `docker inspect` needs the latter. The gateway and the generated route both carry both identifiers (`X-Wake-Resource` / `X-Wake-Container`) rather than assuming they're interchangeable.
+- A full disposable rehearsal (isolated Tilt + Docker Compose + the built gateway image, torn down after) woke a cold `traefik/whoami` container from both "exited" and fully "absent" states and correctly proxied a request through with its original path intact.
+
+**In progress, live against `tdk-erp-system` (2026-09-27):** the queue-bypass fix above (direct `docker compose up --no-build`) is implemented, unit/Starlark-tested (`decide.test.ts`'s new compose-start cases, `test_traefik_static_wake_route.py`'s new compose-invocation case, `test_standalone_traefik.py`'s new project-mount case — all passing), re-vendored into the live project, and the generated route now carries the `X-Wake-Compose-*`/`X-Wake-Image` headers. Still waiting on Tilt to actually rebuild the `wake-gateway`/`traefik` resources with this change under the same landscape-wide build-queue contention that motivated the fix in the first place — task 7.2 records the final result once that rebuild completes and the original cold-start URL is re-tested.
+
+### D4: A never-created resource's route comes from Traefik's static/file provider, not Docker-label discovery
+
+Since a never-created resource has no container and thus no labels, its Traefik router has to be defined statically. The standalone Traefik generator (`traefik_standalone.star`) already reserves the `@file` provider namespace by reference (`maintenance@file` in `traefik.star`) but does not actually enable it — its `command:` list only sets `--providers.docker=true`; there is no `--providers.file=true` and no generated dynamic-config file or mounted directory today. This change adds both: `--providers.file.directory=/etc/traefik/dynamic` (a new generated directory mounted into the `traefik` service), and, for every resource with `sablier.enable` + `deferStart` set, a generated file-provider entry defining that resource's router/service/middleware ahead of time, pointed at D3's wake gateway.
+
+**Verified live (2026-09-26, isolated Traefik+Docker probe, torn down after):** a file-provider router and a Docker-discovered router with the identical name coexist without conflict or error — Traefik namespaces them internally as `<name>@file` and `<name>@docker` — but when both match the same request at **equal priority, the file-provider router wins**, even though the Docker-discovered backend (the real, healthy container) was reachable and correct. A direct request through the file-route in that state hit the placeholder backend and got `502`, not the working one. Explicitly giving the file-provider entry a **low, fixed priority** (tested with `1`) while the Docker-label path keeps its existing computed priority (TDK's real values run 44-166, per `TRAEFIK_FRONTEND_PRIORITY_BASE + len(path)`) made the Docker-discovered router win as soon as it existed, with no manual removal of the static entry needed. So the static entry must carry an explicit low priority (a single reserved constant, e.g. `1`) — it is not safe to assume Traefik "prefers the more specific route" on its own; that was wrong.
+
+*Alternative:* forward-declare the resource's Docker Compose service with `deploy.replicas: 0` or similar so a static route isn't needed. Rejected — Compose has no "define the service, register it with Traefik, but truly never create a container" mode short of the `auto_init=False` behavior already confirmed to produce zero Traefik-visible state; a static file-provider route is the only mechanism confirmed to give Traefik something to match against before any container exists.
+
+### D5: Dependency wake is entirely the wake gateway's job — neither Compose `depends_on` nor a Sablier group works across TDK's actual project layout
+
+Two mechanisms looked plausible from Sablier's docs (see Context) and were both ruled out by direct verification against TDK's real generated output, not just read about:
+
+- `sablier.group`: rejected as previously recorded — it requires every member to itself be `sablier.enable`d, wrong for a deferred resource's database/NATS dependency, which is shared, always-on infra (`dc_resource(..., auto_init=True)` per `infra-loader.star`) that must never become eligible for scale-to-zero just because one deferred app joins its group.
+- Compose `depends_on`: **verified live to be structurally impossible here, not just undesirable.** `docker compose config` rejects a service whose `depends_on` names a service not defined in that *same* compose file with `service "X" depends on undefined service "Y": invalid compose project` — confirmed both for a resource depending on a genuinely separate compose project (e.g. shared postgres) and, more surprisingly, for two resources sharing the same `project_name` but registered via two separate `docker_compose()` calls (e.g. a backend and its own migrator, per `apply_migrator_orchestration.star:91`'s own separate `docker_compose(..., project_name=compose_project_name)` call): Tilt runs `docker compose config` against each registered file *individually*, so a shared `project_name` string does not merge them into one validatable project. Since every one of a resource's real dependencies (database, NATS, its own migrator) lives in a compose file generated by a *different* `docker_compose()` call than the resource's own, `depends_on` cannot express any of them, full stop — there is no scoping that makes this workable, unlike D5's original assumption.
+
+Dependency waking is therefore entirely the wake gateway's job, which was already built to not need either mechanism: `X-Wake-Deps` (D2/D4) carries each dependency's resource and container name directly, and the gateway `tilt trigger`s and polls each one itself (confirmed in the live rehearsal in D3). This is strictly simpler than the original plan, not a workaround — one mechanism, already implemented and tested, instead of two speculative ones layered together. The practical scope this actually covers is narrower than first assumed, too: database/NATS are always-on infra that's essentially never actually cold, so the gateway's dependency-waking mostly matters for the rarer case of one deferred resource depending on another.
+
+*Consequence:* Sablier's native docker-level start path (the "container exists" branch in D3) has no dependency-awareness of its own — it can restart a container but can't wait on that container's migrator or anything else via a label Compose never had a valid reason to write. In practice this rarely bites: by the time any container object exists at all, D4's priority verification means Traefik routes to it directly via the Docker-label path (Sablier's own plugin), bypassing the gateway entirely — the gateway's own "container exists" branch is reached only in the brief window before Traefik's docker provider notices a just-created container's labels, a correctness safety net rather than the primary path for that case.
+
+*Alternative:* the originally-planned computed `sablier.group`, and separately, generated `depends_on`. Both rejected once the real mechanics were confirmed — the former for the scale-to-zero risk above, the latter because it is not merely undesirable but structurally rejected by `docker compose config` for every real dependency shape in this codebase.
+
+### D6: `tdk up`/`tdk doctor` read `auto_init` state, not container status, to report "deferred"
+
+A deferred resource looks, from `docker ps`, identical to a resource that failed to start. `tdk up`'s summary and `tdk doctor` need to check the manifest's resolved `deferStart` (D1) directly rather than inferring "deferred" from container state, so a genuinely stuck resource is never miscategorized as intentionally cold.
+
+## Benchmark: D3's queue-bypass fix, measured
+
+Task 7.4's live `tdk-erp-system` finding (D3) was that `tilt trigger` alone re-creates the original problem one layer down: a triggered resource queues behind Tilt's build-slot cap. That fix (direct `docker compose up --no-build` when an image already exists) needed a real, reproducible measurement, not just the one live pass that got interrupted by the strip-prefix bug — but re-testing against the real `tdk-erp-system` landscape wasn't available (developer's own call, 2026-09-27: stop hammering an already-overloaded shared Docker daemon). Instead, ran a disposable, isolated benchmark (own Tilt instance on a private port, own Compose project, fully torn down after — no interaction with any real project) that exercises the actual shipped `generate_static_wake_route()`/`decide.ts`/`server.ts` code, not a reimplementation:
+
+- **Setup:** `update_settings(max_parallel_updates=2)` plus six "filler" `dc_resource`s with a genuinely slow (90s, uncached each run) Docker build, standing in for a large landscape's build queue — this is the same build-slot contention Tilt showed live (`max_parallel_updates` caps concurrent builds; the real landscape's default cap was observed at 3). Two target resources, both `auto_init=False`: `target-fast`'s route carries the compose-invocation headers (`X-Wake-Compose-*`/`X-Wake-Image`) with its image pre-tagged, exercising the new direct-start path; `target-slow`'s route omits them and has its own `docker_build`, so its first `tilt trigger` must acquire a build slot like the fillers, exercising the pre-7.4 fallback path unchanged.
+- **Results** (wall-clock, cold container to first `200 OK`, via `curl` through the real Traefik static route and gateway):
+
+  | Scenario | Build-slot contention | Time to 200 |
+  |---|---|---|
+  | `target-fast` (direct compose start) | 2/2 slots busy (fillers) | **12.6s** |
+  | `target-fast` (direct compose start) | idle | 7.4s |
+  | `target-slow` (`tilt trigger` fallback) | 1/2 slots busy | 18.1s |
+  | `target-slow` (`tilt trigger` fallback) | idle | 22.8s |
+
+- **What this shows:** `target-fast`'s latency stayed in a narrow band (7-13s) regardless of queue state — direct confirmation that the new path doesn't touch Tilt's build queue at all, matching the gateway's own log line (`compose start target-fast: started`, no `tilt trigger` call). `target-slow` paid a real, unconditional cost even at its best (idle) case — nearly double `target-fast`'s idle case — because every `tilt trigger` still goes through Tilt's own build-reconciliation step (confirmed via Tilt's build history for it) before `docker compose up` runs; that step is exactly what queues behind other builds live against `tdk-erp-system` (task 7.4's original 504). A fully apples-to-apples "both paths, both fully saturated" run wasn't captured (the 90s filler window and Tilt's own infra-resource build ordering made catching that exact instant unreliable without further burning the same finite build-queue setup repeatedly) — the mechanism-level proof (gateway logs showing which code path ran) plus the consistent directional numbers above are treated as sufficient verification, not a substitute for eventually re-running task 7.2's exact live URL once `tdk-erp-system`'s own queue is quiet.
+
+## Risks / Trade-offs
+
+- [The static file-provider route (D4) and the Docker-discovered route disagree once the container exists, e.g. stale healthcheck path baked into the static entry] → Keep the static entry's routing rule byte-identical to what the generator already emits for the Docker-label version (same source-of-truth function), so the two can never drift; verified by generating both and diffing in task testing.
+- [Confirmed live: network reachability to Tilt's port alone isn't enough — `tilt trigger` from inside a container fails with `(403): invalid session token` without the host's `~/.tilt-dev` token] → Fixed by mounting `~/.tilt-dev` read-only into the wake-gateway service. If Tilt runs in a context where that directory isn't at the expected host path (e.g. Tilt itself inside a devcontainer), the mount is wrong and every trigger call fails the same way — surfaces as a clear 403 in the gateway's logs rather than a silent hang, but there's no auto-detection of an unusual `$HOME` yet.
+- [Re-adding `depends_on` to a `deferStart` resource's compose entry (D5) has some other effect under Tilt's per-resource `docker compose ... --no-build` invocation that the original removal was guarding against] → Verify by generating and running a `deferStart` resource with `depends_on` present under normal (non-deferred) `tdk up` handling, confirming Tilt's own bring-up for it is unaffected — expected to be moot in practice since a `deferStart` resource's `auto_init=False` means Tilt never invokes it directly during `tdk up` anyway, but confirm rather than assume.
+- [A `deferStart` resource's `depends_on` condition (`service_healthy`/`service_completed_successfully`) never resolves because the dependency is itself unhealthy or the migrator's `honor-restart-policy` isn't enabled] → Sablier's docs are explicit that `service_completed_successfully` needs `--provider.docker.honor-restart-policy=true` to recognize a one-shot migrator's exit code 0 as `completed` rather than `stopped`; enable that flag on the vendored `sablier` command alongside D5's `depends_on` change, and hold the wake request only up to the existing timeout rather than indefinitely if it never resolves.
+- [The held request during wake exceeds Sablier's own timeout while waiting on a dependency chain] → `sessionDuration`/timeout stay configurable per resource via the existing manifest field; document that a resource with dependencies needing time to become healthy may need a longer timeout.
+- [Background auto-commit tooling observed in this workspace pushes partial engine edits] → Land each decision's Starlark and CLI edits together with its snapshot tests in one sitting, per the same mitigation already used in `optimize-runtime-docker-layers`.
+- [Found and fixed 2026-09-26: re-reading the shipped `apply_compose_resource_registration.star` found the `generate_static_wake_route(...)` call site never passed `dependencies=`, so D5's dependency-wake path was implemented and unit-tested (gateway side and generator side both) but never wired to any real dependency data — every generated route shipped `X-Wake-Deps: ""`] → Fixed (task 7.3): the call site now resolves a `deferStart` resource's manifest `dependsOn` entries to the subset that are themselves `deferStart`, reusing the same resolution `_build_resource_deps` already trusts for Tilt's own `resource_deps` ordering, so it carries no new naming risk beyond what's already shipped. Covered by 6 new Starlark tests. The full licensed, live "wakes a cold peer resource" path is still only verifiable against the real premium module (task 7.2), consistent with how task 3.1 already treats that limitation.
+
+## Migration Plan
+
+1. Ship the schema (`deferStart`) and engine changes (D1-D6) in a TDK release, gated the same way the rest of `sablier` already is (premium license required for the real module; free-tier stub keeps returning disabled).
+2. In each project with an opted-in resource: run `tdk project --yes` to re-vendor, touch the Tiltfile, rebuild affected images.
+3. A resource adopts deferred start by adding `deferStart: true` to its existing `sablier` block; removing it (or the whole block) reverts to today's behavior with no other cleanup needed.
+
+**Rollback:** unset `deferStart` (or the whole `sablier` block) and re-run `tdk project --yes`; the resource returns to normal `auto_init` bring-up immediately.
+
+## Open Questions
+
+- Whether Traefik, when both a file-provider router (D4) and a Docker-discovered router (D2) exist for the same resource name, prefers one deterministically or requires a distinct router name per provider - resolved empirically in tasks.md's preflight phase (task 1.4); at most changes a naming detail, not the approach.

package/openspec/changes/prioritized-cold-start/proposal.md

@@ -0,0 +1,41 @@
+# Proposal
+
+## Why
+
+On a large landscape (`tdk-erp-system`: 107 resources, 110 containers), a request to a specific backend — e.g. `http://api.tdk-erp-system.localhost/api/data-governance/health` — can go unanswered for minutes not because that backend is broken, but because Tilt hasn't reached it yet in its own bring-up order. `data-governance-api` has no `sablier` block today, so it is started the same way as every other resource: as part of the full `tdk up` sequence, behind however many of the other ~106 resources Tilt's dependency graph and build concurrency put ahead of it. The existing premium Sablier integration (`engine/topologies/platform/docker/networking/sablier_container_cycle.star`, real implementation in `tdk-cli-extensions/premium/networking/`) already wakes a resource on its next request — but only if Tilt already created and then stopped that resource's container this session. It has nothing to wake when the container was never created in the first place, which is exactly the case here. Developers working against one corner of a large landscape (one API, one flow) shouldn't pay the full-landscape startup cost, or sit wondering whether the resource they actually want is stuck or just queued.
+
+## What Changes
+
+- Extend the existing `sablier: {enable: true}` manifest block (opt-in, premium, unchanged licensing boundary) so an opted-in resource can be marked deferred for the *current* `tdk up` run, not only for scale-to-zero after it has run once. A deferred resource's image still builds during `tdk up` (so there's something to start on first request), but Tilt does not create or start its container at all, removing it from the sequential dependency-bring-up path entirely (confirmed empirically: an `auto_init=False` resource has no Docker container object until triggered — there is nothing for Traefik's normal Docker-label discovery to find).
+- Because a never-created resource has no container and thus no Docker labels, give Traefik a static route for it via its file provider (a mechanism this codebase already references by name — `maintenance@file` — but doesn't yet enable for standalone projects), pointed at a small always-on wake endpoint. A resource that has previously run and is merely stopped keeps using today's Docker-label discovery (`traefik.docker.allownonrunning=true` plus Sablier's labels) unchanged.
+- On the first request that hits a deferred resource's route, the wake gateway runs `tilt trigger <resource>` (Tilt's own supported CLI, proven to create-and-start a never-touched resource in one call) plus a separate `tilt trigger` for each dependency named in the request (confirmed necessary: Tilt does not cascade a trigger to a dependency on its own — it just waits on it forever), so the whole chain is prioritized immediately instead of waiting for Tilt's normal resource order. For a previously-run-then-stopped resource, Sablier's existing Docker-level start stays the primary (faster) path, with the same trigger calls fired alongside it so Tilt's own state (e.g. a migrator) catches up too.
+- Dependency information for a wake travels through the gateway itself (which resource, which container, which dependencies), not through Compose's `depends_on` — confirmed empirically that `depends_on` cannot validly reference any of TDK's real cross-resource dependencies at all (database, NATS, a resource's own migrator each live in a separately-registered compose file; `docker compose config` rejects the reference outright even when two such files share a project name).
+- `tdk up` and `tdk doctor` report deferred resources as intentionally cold (not stalled/pending), so their absence from the early startup log isn't read as a hang.
+
+**BREAKING (none):** this only changes behavior for resources that opt in via the existing `sablier` manifest block; a resource without that block behaves exactly as it does today.
+
+## Capabilities
+
+### New Capabilities
+- `on-demand-resource-startup`: Requirements for how a Sablier-opted-in resource that Tilt has not yet started this session is discovered by Traefik while stopped, triggered and prioritized on its first request (including its dependency chain), and reported by `tdk up`/`tdk doctor` as deferred rather than stalled.
+
+### Modified Capabilities
+- (none — no existing `openspec/specs/` capability documents today's Sablier behavior or Tilt's startup sequencing, so there is nothing to amend; this proposal documents the extended behavior as new.)
+
+## Non-goals
+
+- Changing the free/premium licensing boundary. This stays inside the existing `sablier` opt-in and its license gate (per product decision); it does not become default behavior for unlicensed or non-opted-in resources.
+- Reducing per-container CPU/memory footprint — already addressed by `optimize-runtime-docker-layers`, whose non-goals explicitly deferred "putting backends under Sablier scale-to-zero" to this change.
+- Building a general-purpose priority queue inside Tilt itself. This relies on Tilt's existing `TILT_AUTO_INIT_APPS`/manual trigger mode and its trigger API rather than patching Tilt's own scheduler.
+- Changing behavior for resources already running and healthy, or for landscapes with no `sablier`-opted-in resources.
+
+## Impact
+
+- **Engine (Starlark):** `engine/topologies/tilt/resources/orchestrator/apply_compose_resource_registration.star` (today's landscape-wide `auto_init_apps`) needs a per-resource variant driven by each manifest's `sablier` block rather than one global flag.
+- **Engine (Starlark):** `engine/topologies/tilt/resources/orchestrator/apply_compose_resource_registration.star` calls the new static-route generator for each `deferStart` backend resource and writes its output alongside the resource's other generated config.
+- **Engine (Starlark):** `engine/topologies/platform/docker/compose/traefik_standalone.star` — adds `--providers.file` support (new command flag, mounted dynamic-config directory), generated static route entries for deferred resources, and a wake gateway service (built from new `engine/assets/docker/wake-gateway/`) that runs `tilt trigger` and proxies the held request through.
+- **CLI:** `cli/src/utils/resource-features.ts` (`sablier` feature description), `cli/src/commands/up.ts` and `doctor-runtime.ts` reporting for deferred resources; `cli/src/types/index.ts`'s `sablier` manifest type gains the new field(s).
+- **Manifest schema:** `engine/topologies/tilt/manifest/schema.star`'s `sablier` block gains a new optional key (name TBD in design, e.g. `deferStart`).
+- **Generated projects:** every project with an opted-in `sablier` resource needs `tdk project --yes` (re-vendor) plus a golden image rebuild to pick up the change; existing non-opted-in resources are unaffected.
+- **Tests:** `tests/tilt-engine/` snapshot tests for the per-resource `auto_init` resolution and the generated static Traefik route; `bun test` unit tests for the wake gateway's decision logic.
+- **Engine (Starlark) — found live, 2026-09-27:** two real gaps only surfaced by testing against the actual `tdk-erp-system` landscape, both now fixed (design.md D3, tasks.md 7.4/7.5): (1) `tilt trigger` alone re-created the original problem one layer down (queues behind Tilt's own build-slot cap under real load) — the gateway now prefers a direct `docker compose up --no-build` when an image already exists; (2) a woken resource still 404'd because the wake route didn't strip the same `apiPath` prefix the real Docker-label router does before forwarding — fixed by adding the matching `stripPrefix` middleware to the generated route. A disposable, isolated benchmark (design.md's "Benchmark" section) measured the first fix's real effect: the new path answers in ~7-13s independent of build-queue contention, versus ~18-23s for the old fallback even at its best case.

package/openspec/changes/prioritized-cold-start/specs/on-demand-resource-startup/spec.md

@@ -0,0 +1,92 @@
+# Spec Delta
+
+## Purpose
+
+Lets a resource opt out of `tdk up`'s normal sequential bring-up and instead start on its own first request, so one request against a large landscape doesn't wait behind every other resource's startup.
+
+## ADDED Requirements
+
+### Requirement: Deferred Start Opt-In
+
+A resource with a `sablier` manifest block SHALL be able to mark itself deferred for the current `tdk up` run. A deferred resource's image SHALL still be built during `tdk up`, but its container SHALL NOT be started as part of `tdk up`'s normal bring-up sequence.
+
+#### Scenario: Deferred resource is not started during tdk up
+
+- **WHEN** `tdk up` runs a landscape containing a resource with deferred start enabled
+- **THEN** that resource's image build completes during the run
+- **AND** the resource's container is not started, and `tdk up` does not wait on it to become healthy before finishing bring-up of the rest of the landscape
+
+#### Scenario: Non-opted-in resource is unaffected
+
+- **WHEN** `tdk up` runs a landscape containing a resource with no `sablier` block, or a `sablier` block without deferred start enabled
+- **THEN** that resource is built and started as part of `tdk up`'s normal dependency-graph sequencing, exactly as before this change
+
+### Requirement: Deferred Resource Stays Discoverable Before Its First Start
+
+A deferred resource's Traefik route SHALL be registered and routable regardless of whether its container has ever been created, so the first request to it can be received and acted on rather than returning "no route" or a connection error.
+
+#### Scenario: Route exists before first start
+
+- **WHEN** `tdk up` finishes bring-up of a landscape containing a deferred resource that has never been started this session
+- **THEN** a request to that resource's route reaches Traefik and is recognized as targeting a known, deferred resource, rather than returning "no route" or a connection error
+
+#### Scenario: Route still resolves after the resource has run and stopped
+
+- **WHEN** a deferred resource has run at least once this session and its container is now stopped
+- **THEN** a request to that resource's route still reaches Traefik and is recognized as targeting that resource, using the same or an equivalent route as before it ever ran
+
+### Requirement: First Request Triggers And Prioritizes Startup
+
+The first request to a deferred resource's route SHALL start that resource's container and SHALL cause the resource to be prioritized ahead of the landscape's normal remaining bring-up order, rather than waiting for that order to reach it naturally.
+
+#### Scenario: Request during a large, in-progress tdk up
+
+- **WHEN** a request is made to a deferred resource's route while `tdk up` is still bringing up other, unrelated resources elsewhere in the landscape
+- **THEN** the deferred resource is started and reaches a healthy state without waiting for the rest of the landscape's bring-up to complete first
+
+#### Scenario: Request after tdk up has finished
+
+- **WHEN** a request is made to a deferred resource's route after `tdk up` has finished bringing up the rest of the landscape
+- **THEN** the deferred resource starts immediately in response to that request
+
+### Requirement: Waking A Resource Also Wakes Its Required Dependencies
+
+Waking a deferred resource SHALL also wake any of its required startup dependencies (the same dependencies already used to sequence Tilt's own bring-up, e.g. its database or messaging broker) that are not currently running, and SHALL wait for each to be ready before evaluating the resource's own health.
+
+#### Scenario: Deferred resource depends on a cold database
+
+- **WHEN** a request wakes a deferred resource whose required dependencies include another resource that is not currently running
+- **THEN** that dependency is also started as part of the same wake, and the requesting resource's health checks are not evaluated against a still-cold dependency
+
+#### Scenario: Unrelated resource is not woken
+
+- **WHEN** a request wakes a deferred resource
+- **THEN** resources that are not among its required startup dependencies are not started as a side effect of that wake
+
+### Requirement: A Waking Request Waits For Health, Not Just Start
+
+A request that triggers a deferred resource's startup SHALL be held until the resource passes its health check or until a bounded timeout elapses, rather than being answered as soon as the container process starts.
+
+#### Scenario: Client receives the real response
+
+- **WHEN** a request wakes a deferred resource
+- **THEN** the client's request is held and then answered by the resource once it is healthy, rather than receiving an immediate "starting" response with no real answer
+
+#### Scenario: Startup exceeds the timeout
+
+- **WHEN** a deferred resource does not become healthy within its configured wake timeout
+- **THEN** the held request fails with a clear timeout error rather than hanging indefinitely
+
+### Requirement: Deferred Status Is Reported Distinctly From Failure
+
+`tdk up` and `tdk doctor` SHALL report a deferred, not-yet-started resource as intentionally deferred, and SHALL NOT report it the same way as a stalled or failed resource.
+
+#### Scenario: tdk up summary
+
+- **WHEN** `tdk up` finishes bring-up of a landscape containing deferred resources that have not yet received a request
+- **THEN** the summary output lists those resources as deferred rather than omitting them or listing them as pending/stalled
+
+#### Scenario: tdk doctor
+
+- **WHEN** `tdk doctor` runs against a landscape containing a deferred, not-yet-started resource
+- **THEN** it does not flag that resource as unhealthy or stuck solely because it has not started

package/openspec/changes/prioritized-cold-start/tasks.md

@@ -0,0 +1,56 @@
+# Tasks
+
+## 1. Preflight checks
+
+- [x] 1.1 ~~Confirm whether `dc_resource(..., auto_init=False)` pre-creates the underlying container~~ — **Done, confirmed empirically 2026-09-26** against the installed Tilt `v0.37.7`: a throwaway `dc_resource(auto_init=False)` produces no container object at all (`docker ps -a` empty for it, `uiresource.status.runtimeStatus`/`updateStatus` stay `none`) until triggered; `tilt trigger <name>` then creates and starts it in one call. Recorded under design D2/D3. No further verification needed on this point.
+- [x] 1.2 ~~Confirm whether the vendored `sablierapp/sablier` version supports a webhook/hook callable on container start~~ — **Done, confirmed 2026-09-26** by reading the v1.18 docs (`sablierapp.dev/v1.18`, version-matched to the vendored `1.18.0`): Sablier has no hook/webhook system at all — it's pull-based (the reverse-proxy plugin queries Sablier's status API per request; the "blocking strategy" page describes exactly this loop). Recorded under design D3: the wake gateway is a self-written component regardless, not a Sablier extension point.
+- [x] 1.3 ~~Confirm the Tilt HTTP trigger API request shape~~ — **Superseded, 2026-09-26**: design D3 now prefers shelling out to the `tilt trigger <resource>` CLI (proven working end-to-end in the 1.1 probe) over the undocumented internal HTTP API, so no request-shape verification is needed.
+- [x] 1.4 ~~Confirm how Traefik's Docker provider and file provider behave when both eventually define a router for the same resource name~~ — **Done, confirmed empirically 2026-09-26** with an isolated Traefik + Docker probe (torn down after): same-named `@file` and `@docker` routers coexist without conflict, but at equal priority **the file-provider router wins** even once the real container is healthy (a test request got `502` from the stale file-routed backend, not the working docker-routed one). Giving the file-provider entry a fixed low priority (`1`) while the docker-label path keeps its normal computed priority fixed this — the docker route then won automatically as soon as it existed. **Conclusion: the generated file-provider entry (task 4.2) must set an explicit low priority; do not rely on Traefik "preferring" one router on its own.**
+- [x] 1.5 ~~Confirm whether `tilt trigger <resource>` cascades to a `depends_on` dependency that is itself `auto_init=False`~~ — **Done, confirmed empirically 2026-09-26** with the same isolated-probe technique as 1.1 (two `dc_resource(auto_init=False)` services, one `depends_on` the other with `condition: service_healthy`, fully torn down after): Tilt parses the compose `depends_on` and adds its own wait, but triggering only the dependent resource leaves it parked at `runtimeStatus: pending`, `status.waiting = {on: [dependency], reason: "waiting-for-dep"}` indefinitely — Tilt does not auto-trigger the dependency. Triggering the dependency separately immediately cleared the wait and let the dependent resource build and start. **Conclusion: the wake gateway (task 4.3) must explicitly `tilt trigger` every `depends_on` dependency itself; the cascade cannot be relied on.**
+
+## 2. Manifest schema (D1)
+
+- [x] 2.1 Add `deferStart` (boolean, default `false`) to the `sablier` schema block in `engine/topologies/tilt/manifest/schema.star` and to the `sablier` type in `cli/src/types/index.ts`. Verify with a schema validation test asserting a manifest with `sablier: {enable: true, deferStart: true}` passes validation and one with `deferStart` set but `enable` absent/false fails with a clear error (deferStart requires enable). **Done**: added the schema field, a `_validate_cross_field` rule in `validator.star`, and `tests/tilt-engine/test_sablier_defer_start.py` (7 cases, all passing via `tilt alpha tiltfile-result`).
+- [x] 2.2 Update the `sablier` entry in `cli/src/utils/resource-features.ts` to document `deferStart`. Verify by running the existing feature-description test/lint that covers this file, if any, or a manual `tdk resource --help`-style check that the new field appears. **Done**: description and `generator_file`/`output_file` updated; no dedicated test file existed for this file, verified with `npm run typecheck` (clean).
+
+## 3. Per-resource deferred auto_init (D1)
+
+- [x] 3.1 Add a `resource_defers_start(manifest)` helper (co-located with the other manifest-derived helpers used by `apply_compose_resource_registration.star`) that reads the resolved `sablier.enable`/`sablier.deferStart` for a manifest. Verify with a unit test covering: no `sablier` block, `enable: false`, `enable: true` without `deferStart`, and `enable: true, deferStart: true`. **Done**: added at `apply_compose_resource_registration.star:117` (no leading underscore — confirmed via a throwaway probe that Tilt's `load()` genuinely rejects underscore-prefixed names, so a directly-loadable unit test needed a non-underscored name); 7 cases added to `test_sablier_defer_start.py`, all passing.
+- [x] 3.2 Change `_register_single_resource` (the actual primary `dc_resource` call at `apply_compose_resource_registration.star:238` — not `_register_replicas`, which only ever fires for extra replicas beyond the first) to resolve `auto_init` per-resource as `auto_init_apps and not resource_defers_start(config['manifest'])`, threading that same `effective_auto_init` into `dc_resource`, `_register_run_only_resource`, and `_register_replicas` alike; `docker_build` is unaffected since it has no `auto_init` parameter in this codebase, so the image still builds regardless. `_build_resource_config` now also carries `'manifest': manifest` so this is available. **Done and code-reviewed**; verified indirectly via 3.1's predicate tests plus reading every call site — a full live `dc_resource` integration test is deferred to task 7.2's real `tdk-erp-system` run rather than duplicated here with a synthetic project.
+- [ ] 3.3 Confirm end to end that a `deferStart` resource genuinely produces no container object after `tdk up` finishes (per 1.1's confirmed finding, this is expected, not a bug) — `docker ps -a` shows nothing for it — and that its config-gen and image build still completed. Folded into task 7.2's real `tdk-erp-system` run rather than a separate synthetic project.
+
+## 4. Static route for never-created resources (D2, D4)
+
+- [x] 4.1 Add `--providers.file.directory=/etc/traefik/dynamic` to the standalone Traefik command list in `engine/topologies/platform/docker/compose/traefik_standalone.star`, plus a mounted directory for generated dynamic-config files. **Done**: also added `--providers.file.watch=true`, a `./traefik-dynamic:/etc/traefik/dynamic:ro` mount (`PlatformDockerConstants.TRAEFIK_DYNAMIC_DIR_REL`), and the `wake-gateway` service itself (built from the new `engine/assets/docker/wake-gateway/`, auto-vendored into every generated project since `engine/` is copied wholesale — confirmed by reading `cli/src/generator/template-engine.ts`'s `TDK_EXTENSION_DIRS`). Verified for real: `docker compose up -d --build` on the generated compose brought up `traefik`+`sablier`+`wake-gateway` together (only a host port-80 conflict with the already-running real landscape, not a config defect); `test_standalone_compose_enables_file_provider_and_wake_gateway` covers it as a repeatable test.
+- [x] 4.2 Generate a file-provider dynamic-config entry for each `deferStart` resource, reusing the exact same router rule/middleware values the Docker-label path already computes for that resource, pointed at the wake gateway's address instead of the resource's own port, with a fixed low priority (`1`). **Done**: `engine/topologies/platform/docker/networking/traefik_static_routes.star`, config travels via an injected `headers` middleware (`X-Wake-Resource`/`X-Wake-Container`/`X-Wake-Deps`/`X-Wake-Proxy-Port`) rather than the path, so the real request path reaches the backend unchanged; `X-Wake-Container`/each dependency's container name follow Compose's `<project>-<service>-1` convention (confirmed against the live `analytics-autogenerated-data-governance-api-1` container). `test_traefik_static_wake_route.py` (3 cases) verifies the rule matches the Docker-label `-project` router, the priority is `1` and lower than the computed one, and the headers carry container names correctly.
+- [x] 4.3 Stand up the wake gateway referenced by 4.2 that triggers a cold resource and its dependencies, then holds the request and proxies it through once healthy. **Done**: `engine/assets/docker/wake-gateway/` (`decide.ts` pure decision logic + 9 `bun test` cases, `server.ts` the real Bun HTTP server, `Dockerfile`). Full real end-to-end verification via three disposable probes (all torn down): (a) confirmed `tilt trigger` from inside a container needs the host's `~/.tilt-dev` token directory mounted read-only — without it, every trigger call fails with `(403): invalid session token` even though the Tilt port is reachable; (b) confirmed and fixed a Bun `URL.host` setter bug (`target.host = "name:port"` silently kept the original port; fixed by setting `.hostname`/`.port` separately) and a header-forwarding bug (`fetch` rejected the raw incoming `Host`/`X-Wake-*` headers; fixed by building a clean header set); (c) with both fixes, a real cold `traefik/whoami` container (both "exited" and fully "absent" states) was woken, became healthy, and a request was correctly proxied through with its original path intact, in ~10-29s.
+- [x] 4.4 Confirm end to end that 4.2's fixed low priority actually hands traffic back to the real container once it exists. **Done** — this is exactly what task 1.4's probe verified (recorded there): at equal priority the file-provider router won even over a healthy Docker-discovered backend; priority `1` fixed it.
+
+## 5. Wake path wiring: generate the static route, dispatch to the gateway (D2, D3, D5)
+
+- [x] ~~5.1 Add a `depends_on` block to a `deferStart` resource's compose entry~~ — **Superseded, 2026-09-26**: verified empirically that Compose `depends_on` cannot validly reference any of TDK's real cross-file dependencies here at all (see design.md D5's third finding — `docker compose config` rejects it outright, even across files sharing one `project_name`). D5 now relies entirely on the gateway's own explicit `X-Wake-Deps` handling (already built in task 4.3), not on anything compose-level.
+- [x] ~~5.2 Add `--provider.docker.honor-restart-policy=true`~~ — **Removed, 2026-09-26**: its only stated purpose was supporting `depends_on`'s `service_completed_successfully` condition, which 5.1's finding ruled out entirely; reverted from `traefik_standalone.star` rather than shipping unused config.
+- [x] 5.3 Wire the static route generator (task 4.2) into the real registration pipeline: for a `deferStart` backend resource, `register_compose_resources` (`apply_compose_resource_registration.star`) now calls `generate_static_wake_route` and writes it to `PlatformDockerConstants.TRAEFIK_DYNAMIC_DIR_REL + '/<resource>.yml'` via the existing `write_file` callback, right where its Traefik Docker labels are otherwise generated. Scoped to backends only for now (matching the proposal's motivating case) and with an empty dependency list (D5: TDK's real dependencies can't go through `depends_on`, and are either always-on infra that doesn't need waking, or a peer `deferStart` resource, left as a follow-up rather than guessed at — see the code comment at the call site). Verified: `resource_defers_start` and `generate_static_wake_route` unit/snapshot tests pass with this import wired in; full pipeline exercise (real `Docker.backend_compose`/file writes) folded into task 7.2's live run rather than duplicated with a synthetic project, consistent with how task 3.2 was verified.
+- [x] 5.4 Confirm the held request is answered only once the resource (and any dependencies named in `X-Wake-Deps`) are healthy, and fails with a clear timeout error past the configured duration instead of hanging. **Already verified** in task 4.3's live rehearsal (the gateway's timeout path and `evaluateReadiness` logic are exercised by `decide.test.ts` and were hit for real during debugging before the fixes landed — several requests genuinely timed out with a clear 504 rather than hanging).
+
+## 6. tdk up / tdk doctor reporting (D6)
+
+- [x] 6.1 Add a "deferred" resource classification to `tdk up`'s summary output, sourced from the resolved `sablier.enable && sablier.deferStart` per manifest (not from container state). **Done**: `cli/src/commands/up.ts`'s Backend API URLs list tags a deferred resource `(deferred — starts on first request)`. Verified by `npm run typecheck` (clean); no existing test harness covers `tdk up`'s interactive output at all (nothing to extend proportionately for one label — the full end-to-end pass in task 7.2 exercises it against the real landscape instead).
+- [x] 6.2 Make `tdk doctor` skip its usual "not running" / "unhealthy" flags for a resource classified as deferred and never yet triggered, per `doctor-runtime.ts`. **Done**: `parseTiltResourceFailures` now takes a `deferredNames` set and reports a deferred resource's `none`/`pending` status as its own `deferredCount`, separate from genuine `pendingCount` — its message no longer says "often blocked by the failures above" for these. `getDeferredResourceNames()` (via `discoverResources()`) supplies that set in `checkTiltResourceHealth`, best-effort (empty set, not a throw, if discovery fails). A genuinely failing resource is unaffected either way — `deferStart` never touches the `failures` branch. 2 new test cases pass in `doctor.test.ts` (29/29 total, no regressions); a resource that's been triggered but is still unhealthy would surface via Tilt's own `error` status, already covered by the existing failure-path tests.
+
+## 7. Documentation and integration
+
+- [x] 7.1 Document `deferStart` in the same place the `sablier` block itself is documented. **Done**: this repo has no separate docs file for `sablier` at all — its only documentation is `schema.star`'s field description and `resource-features.ts`'s feature entry, both already updated in group 2/task 2.2 (D5's `depends_on` mention removed from that description since D5 no longer uses it). No standalone docs page exists to additionally update.
+- [ ] 7.2 Re-vendor into `tdk-erp-system` (`tdk project --yes`, touch the Tiltfile, rebuild affected images), set `sablier: {enable: true, deferStart: true}` on `data-governance-api`'s `service.json`, and run a full `tdk up`. Verify that `data-governance-api` does not start during the initial bring-up, that `curl http://api.tdk-erp-system.localhost/api/data-governance/health` while the rest of the landscape is still starting returns a real (not `404`/connection-refused) health response within the resource's normal cold-start time — directly closing the gap reproduced live during design (see design.md Context) — and that `tdk doctor` does not flag it before that request and does not flag it as broken after. **Still open, in progress on the developer's own machine (2026-09-27):**
+  - Confirmed the free-tier key in `.env` grants nothing (`⚠️ No premium resources unlocked for this key`); the license-gated overlay was exercised by manually copying `tdk-cli-extensions/premium/networking/sablier_container_cycle.star` over the vendored stub, which is exactly what a real grant would do (`vendorTdkExtension()`'s own overlay step, `template-engine.ts`).
+  - Added the `sablier` block to `data-governance-api/service.json`, re-vendored, and confirmed the generated route (`traefik-dynamic/data-governance-api.yml`) and cold behavior (no container until first request) all work as designed.
+  - First real pass found and fixed two genuine bugs (not environment noise): (a) task 7.4's queue-bypass fix (`tilt trigger` alone queued behind Tilt's build-slot cap and 504'd — see D3); (b) after that fix, a second real bug — the app only serves its bare health path (`/health`), and the generated wake route forwarded the full public path (`/api/data-governance/health`) unstripped, unlike the real Docker-label `-project` router, so a successful wake still 404'd. Fixed by adding a matching `stripPrefix` middleware to the generated route (`traefik_static_routes.star`), reusing the same `cli_api_path()` the Docker-label router strips with. Both fixes are unit/Starlark-tested (33/33 passing) and re-vendored into the live project.
+  - **Re-test after both fixes is inconclusive, not failing**: a `docker rm -f` + cold curl got a `502` from Traefik itself (not the gateway's own 504), and the gateway's log showed only its initial "wake" decision line, never a "compose start ... started/failed" result — `docker compose up` inside the gateway appears to have stalled rather than erred. Root-caused to the machine, not the code: `tilt get uiresource` showed **104 of 250 resources still `pending`**, mid a mass rebuild (Tilt's build-slot cap processing 3 at a time) triggered by this task's own repeated `tdk project --yes` + forced-Tiltfile-re-eval cycles across this session — sustained load average ~26-27, `docker ps`/`traefik ping` themselves taking 10-30s, and Traefik logged a stale `data-governance-api@docker` service IP getting "no route to host" consistent with a container that started to come up and was torn down mid-flight under this contention. **Decision (2026-09-27, developer's own call, not further probed by any session)**: stop touching the live landscape rather than keep polling an already-overloaded daemon (load average climbed to ~38 after this note was first written, queue hadn't drained) or force a Docker Desktop restart that would kill the ~110 containers of a 14+-hour-running landscape. The code changes (task 7.4's queue-bypass fix and this task's strip-prefix fix) ship as-is on the strength of their unit/Starlark tests (33/33 passing) plus the one clean live pass that *did* complete end-to-end before the path-stripping bug was found (container created from cold via direct compose start, no Tilt-queue wait, "compose start data-governance-api: started" logged, and a genuine 502→200 progression as each bug was fixed in turn) — not on an unattainable final green run under a saturated host. This task stays open (checkbox unchecked) rather than being marked done on inference: the one remaining step is a single clean re-run of `curl http://api.tdk-erp-system.localhost/api/data-governance/health` against a quiescent `tdk-erp-system` landscape, entirely at the developer's own convenience, with no further re-vendor/re-eval cycles needed first (the fixes are already vendored in).
+  - **2026-09-27 disposable probe (torn down after, nothing kept):** to verify task 7.3's fix end-to-end rather than only at the Starlark-unit level, stood up an isolated Tilt + Docker Compose project (two `traefik/whoami` services, `dep-service` and `primary-service`, both `dc_resource(auto_init=False)`) plus a standalone Traefik with the file provider enabled. Generated `primary-service`'s static route with the **real, unmodified** `generate_static_wake_route()` from `traefik_static_routes.star`, passing `dependencies=[("dep-service", "wakeprobe")]` the same shape `resolve_defer_start_dependencies` now produces — confirmed the emitted YAML carries `X-Wake-Deps: "dep-service=wakeprobe-dep-service-1"`. Ran the real `wake-gateway/server.ts` (as a local Bun process against this container's own Tilt daemon and Docker socket, since building the packaged Docker image failed here on an unrelated Alpine/`apk` proxy TLS issue — the packaged image itself is unaffected, this only affected building it ad hoc in this sandbox). A single HTTP request against `primary-service`'s route, while both containers were confirmed absent beforehand, resulted in **both** `wakeprobe-primary-service-1` and `wakeprobe-dep-service-1` being created and started (verified via `docker ps` and `tilt logs` showing both `tilt trigger` calls succeed) — this is the exact behavior task 7.3 added and that D5's requirement describes, now observed working end-to-end, not just unit-tested. The final proxy-through step returned a connection error, but only because the gateway ran as a host process outside the Compose network and so couldn't resolve the container's Compose-DNS name (`primary-service`) — confirmed by curling the container's bridge IP directly and getting a real `200`; that proxy-through mechanic itself was already verified live in task 4.3's own probe when the gateway ran inside the correct Docker network, and is unrelated to 7.3's change. One real, unrelated hiccup along the way: `primary-service`'s first `tilt trigger` failed with a Docker Hub `429 Too Many Requests` on the image pull (transient registry throttling, not a bug); a second trigger with the image already cached succeeded normally.
+- [x] 7.3 (Found 2026-09-26 re-reading the shipped code) Wire real dependency data into the `generate_static_wake_route(...)` call at `apply_compose_resource_registration.star:442` — it always passed no `dependencies=` argument, so every generated route shipped `X-Wake-Deps: ""` despite the gateway side and the generator's `dependencies` param both being implemented and unit-tested in isolation. **Done, 2026-09-26**: added `resolve_manifest_dependency_names(manifest, all_services_map)` (reuses the existing `_resolve_dependency_to_resource` that `_build_resource_deps` already trusts for Tilt's own `resource_deps` ordering — an ambiguous name behaves identically here as it already does there) and `resolve_defer_start_dependencies(manifest, resource_manifests, all_services_map)` (filters those resolved names to the ones whose own manifest is itself `resource_defers_start`), both in `apply_compose_resource_registration.star`, no leading underscore so they're directly `load()`-able for tests per task 3.1's precedent. The call site now passes `dependencies=resolve_defer_start_dependencies(manifest, resource_manifests, all_services_map)`. 6 new cases in `test_sablier_defer_start.py` cover: exact-name resolution, an unresolvable name falling back to exclusion (not a guess), a dependency that resolves but isn't itself deferStart, and the free-tier license gate keeping the result empty even for a `dependsOn` target whose manifest asks for `deferStart` (mirrors task 3.1's own license-gate tests, since `resource_defers_start` is reused unchanged) — full 166/166 passing (166 = prior 160 + these 6; the pre-existing 29 unrelated failures in `tests/tilt-engine/` — `test_agents_md_*`, `test_daemon_*`, `test_snapshot_scanning`, etc. — are identical before and after this change, confirmed by diffing the failure list against a `git stash` baseline). The "licensed and actually wakes a peer deferStart resource" end-to-end case still can't be exercised from this repo's free-tier stub, same limitation task 3.1 already documented — folded into task 7.2's live run rather than duplicated here.
+
+- [x] 7.5 (Found live during task 7.2's real `tdk-erp-system` run, 2026-09-27, immediately after 7.4's fix) A woken resource still 404'd: the app only serves its bare health path (e.g. `/health`), but the generated wake route forwarded the full public path (`/api/data-governance/health`) unstripped — unlike the real Docker-label `-project` router, which strips that prefix via its own `stripprefix` middleware (`traefik.star`'s `get_backend_traefik_labels`). **Done**: `generate_static_wake_route` (`traefik_static_routes.star`) now emits a matching `stripPrefix` middleware ahead of the headers middleware, reusing the same `cli_api_path()` the Docker-label router already strips with, so the two paths agree by construction rather than by convention. New `test_static_route_strips_the_same_api_path_prefix_the_docker_router_does` case (asserts the middleware exists, strips the right prefix, and runs before the headers middleware in the router's list) — 34/34 Starlark tests passing. Re-vendored into `tdk-erp-system` and confirmed via `docker exec ... wget /health` (200) vs `wget /api/data-governance/health` (404) against the real container that the app itself only serves the stripped path, proving the fix addresses a real gap rather than a theoretical one.
+
+- [x] 7.6 Benchmark task 7.4's queue-bypass fix with real, reproducible numbers rather than relying solely on the one interrupted live pass. **Done, 2026-09-27**: a disposable, isolated benchmark (own Tilt instance on a private port, own Compose project, torn down completely after — no interaction with any real project) exercising the actual shipped `generate_static_wake_route()`/`decide.ts`/`server.ts` code against a genuine build-slot queue (`max_parallel_updates=2`, six slow-building filler resources). Full setup, results table, and interpretation recorded in `design.md`'s new "Benchmark: D3's queue-bypass fix, measured" section. Headline results: the new direct-start path (`target-fast`) answered in 7.4-12.6s regardless of whether the build queue was saturated; the old `tilt trigger`-only fallback (`target-slow`) cost 18.1-22.8s even at its best (idle) case, confirmed via Tilt's own build history to be paying its build-reconciliation step unconditionally — the same step that queues behind other builds and caused the original live `504`.
+
+- [x] 7.4 (Found live during task 7.2's real `tdk-erp-system` run, 2026-09-27) `tilt trigger`-only wake failed under real load: with ~100+ resources mid-rebuild after a re-vendor (Tilt caps concurrent builds at a handful of slots), a triggered `deferStart` resource sat queued behind them and the request timed out with a `504` — the exact "wait behind a huge landscape" problem this feature exists to remove, just relocated to Tilt's build queue. **Done**: the gateway now prefers creating/starting the resource directly with `docker compose up -d --no-build <resource>` (the same command Tilt's own `<resource>-run-only` local_resource runs) whenever the generated route supplies the resource's compose invocation, falling back to `tilt trigger` only when no image has ever been built for it yet (`ensureImage()` first tries retagging Tilt's own `<repo>:tilt-<hash>` build as `<repo>:dev` in case the plain tag was never created). Changes: `decide.ts`'s `decideWakeAction` gained `useComposeStart`/`hasComposeInvocation` and `wakeTimeoutMs()` (180s for a direct start, 240s when a build might be needed, under Bun's 255s idle cap) with new `decide.test.ts` cases; `server.ts` gained `composeStart()`/`ensureImage()` and reads `X-Wake-Compose-Project`/`X-Wake-Compose-Files`/`X-Wake-Env-File`/`X-Wake-Image`; `traefik_static_routes.star`'s `generate_static_wake_route` gained matching optional params/headers (new `test_static_route_carries_compose_invocation_for_direct_start` case) plus a `normalize_abs_path()` helper (Tilt's `os.path.abspath` doesn't collapse `..`, and the generated Tiltfile's project root is exactly `config.main_dir + "/../.."` — confirmed with a throwaway probe, new `test_normalize_abs_path_collapses_dot_segments` case); `apply_compose_resource_registration.star` gained `_wake_compose_invocation()` wiring the resource's actual compose files/env/image into the call site; `traefik_standalone.star` mounts the project root read-only into the gateway container at the same normalized absolute path so those compose files resolve identically there (new `test_wake_gateway_mounts_project_root_at_its_own_absolute_path` case). Dockerfile gained `docker-cli-compose`. 33/33 Starlark tests and the full `bun test` suite (25 assertions) pass. Live re-verification against `tdk-erp-system` (re-vendored, route regenerated with the new headers, gateway/Traefik rebuild in progress under the same real build-queue contention that motivated this) folded into 7.2's final result below rather than duplicated here.

package/openspec/config.yaml

@@ -0,0 +1 @@
+schema: spec-driven

package/package.json

@@ -1,7 +1,15 @@
 {
   "name": "@tdk-landscape/tdk-cli-core",
-  "version": "1.3.49",
-  "description": "TDK CLI - Tilt Development Kit (monorepo)",
+  "version": "1.3.51",
+  "description": "Run 100 microservices on a 16 GB laptop without Kubernetes. Scaffold services and run them locally with hot reload, built on Tilt.",
+  "homepage": "https://tdk-landscape.github.io/tdk-website",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/tdk-landscape/tdk-cli-core.git"
+  },
+  "bugs": {
+    "url": "https://github.com/tdk-landscape/tdk-cli-core/issues"
+  },
   "private": false,
   "workspaces": [
     "cli"
@@ -29,11 +37,33 @@
     "tilt",
     "cli",
     "microservices",
-    "local-dev"
+    "local-dev",
+    "local-development",
+    "developer-tools",
+    "developer-experience",
+    "docker",
+    "docker-compose",
+    "dev-environment",
+    "monorepo",
+    "hot-reload",
+    "bun",
+    "kubernetes-alternative"
   ],
   "license": "MIT",
+  "dependencies": {
+    "chalk": "^5.3.0",
+    "commander": "^15.0.0",
+    "handlebars": "^4.7.8",
+    "ink": "^7.1.1",
+    "ink-select-input": "^6.2.0",
+    "ora": "^9.4.1",
+    "react": "^19.3.0"
+  },
   "devDependencies": {
-    "@types/node": "^20.0.0",
-    "knip": "^6.9.0"
+    "@types/node": "^22.0.0",
+    "knip": "^6.38.0"
+  },
+  "engines": {
+    "node": ">=22.12.0"
   }
 }

package/scripts/autoresearch/layer-eval.sh

@@ -0,0 +1,65 @@
+#!/usr/bin/env bash
+# Ground-truth evaluation for L1-L4 layer experiments. DO NOT EDIT during an autoresearch run.
+# Re-vendors this engine into the ERP project, regenerates Dockerfiles through Tilt, rebuilds the
+# golden layers and a fixed sample of 20 real ERP services, runs the container benchmark and the
+# feature tests, then prints one summary block.
+set -uo pipefail
+CORE="$(cd "$(dirname "$0")/../.." && pwd)"
+ERP="${TDK_BENCH_PROJECT:-$CORE/../tdk-erp-system}"
+SAMPLE="${LAYER_EVAL_SAMPLE:-20}"
+TDK_CLI="${TDK_CLI:-$CORE/cli/bin/tdk.js}"
+PREFIX="$(python3 -c "import json;print(json.load(open('$ERP/.tdk/project.json'))['project']['name'])")"
+t() { date +%s; }
+
+cd "$ERP"
+TDK_EXTENSION_SOURCE="$CORE" node "$TDK_CLI" project --yes >/dev/null 2>&1 || { echo "status: crash (tdk project)"; exit 1; }
+tilt alpha tiltfile-result -f .tdk/.tdk-out/Tiltfile >/dev/null 2>&1 || { echo "status: crash (Tiltfile evaluation)"; exit 1; }
+
+s=$(t)
+for target in l1_golden:l1 l2_golden:l2 l3_backend_golden:l3-backend l3_frontend_golden:l3-frontend l4_backend_bun:l4-backend l4_frontend_golden:l4-frontend; do
+  docker build -q -f .tdk/.tdk-out/golden-layers.Dockerfile --target "${target%%:*}" -t "$PREFIX-${target##*:}:latest" . >/dev/null 2>&1 \
+    || { echo "status: crash (golden ${target%%:*})"; exit 1; }
+done
+golden_build_s=$(( $(t) - s ))
+
+s=$(t)
+ls services/*/*/service.json | sort | head -n "$SAMPLE" | while read -r manifest; do
+  dir="$(dirname "$manifest")"; svc="$(basename "$dir")"; stack="$(basename "$(dirname "$dir")")"
+  echo "docker build -q -f $dir/.autogenerated/Dockerfile.app.autogenerated --target production -t ${stack}_${svc}:dev . >/dev/null 2>&1 || echo FAIL $svc"
+done | xargs -P 4 -I{} sh -c '{}' | tee /tmp/layer-eval-build.log
+grep -q FAIL /tmp/layer-eval-build.log && { echo "status: crash (service build: $(grep FAIL /tmp/layer-eval-build.log | head -3 | tr '\n' ' '))"; exit 1; }
+service_build_s=$(( $(t) - s ))
+
+cd "$CORE"
+bun scripts/benchmark/container-scale.ts --synthetic --tiers "$SAMPLE" --gate "$SAMPLE" --settle 15 > /tmp/layer-eval-bench.log 2>&1
+result="$(ls -t benchmarks/results/container-scale-*.json | head -1)"
+tests=pass
+python3 -m pytest tests/tilt-engine/test_runtime_container_footprint.py -q -p no:cacheprovider >/tmp/layer-eval-tests.log 2>&1 || tests=fail
+
+python3 - "$result" "$ERP" "$SAMPLE" "$PREFIX" "$golden_build_s" "$service_build_s" "$tests" <<'PY'
+import json, subprocess, sys, glob, os
+result, erp, sample, prefix, gb, sb, tests = sys.argv[1:]
+r = json.load(open(result))["results"][0]
+def size(img):
+    out = subprocess.run(["docker", "image", "inspect", img, "--format", "{{.Size}}"], capture_output=True, text=True).stdout.strip()
+    return int(out) / 1048576 if out else 0
+base = size(f"{prefix}-l4-backend:latest")
+svcs = sorted(glob.glob(f"{erp}/services/*/*/service.json"))[: int(sample)]
+backend = [f"{os.path.basename(os.path.dirname(os.path.dirname(p)))}_{os.path.basename(os.path.dirname(p))}:dev" for p in svcs
+           if json.load(open(p)).get("appType") != "frontend"]
+sizes = [size(i) for i in backend]
+ok = r["allHealthySeconds"] is not None and r["crashed"] == 0 and tests == "pass"
+print("---")
+print(f"mem_per_service_mib: {r['avgMemMiB']}")
+print(f"service_image_mib:   {sum(sizes)/len(sizes):.1f}")
+print(f"unique_per_service_mib: {sum(sizes)/len(sizes) - base:.1f}")
+print(f"l4_backend_mib:      {base:.1f}")
+print(f"all_healthy_s:       {r['allHealthySeconds']}")
+print(f"crashed:             {r['crashed']}")
+print(f"docker_ps_ms:        {r['dockerPsMs']}")
+print(f"cpu_total_pct:       {r['totalCpuPct']}")
+print(f"golden_build_s:      {gb}")
+print(f"service_build_s:     {sb}")
+print(f"tests:               {tests}")
+print(f"status:              {'ok' if ok else 'invalid'}")
+PY

package/scripts/benchmark/README.md

@@ -0,0 +1,25 @@
+# Container scale benchmark
+
+Release gate for the north star: **100 ERP services on a 16 GB machine without removing features**.
+
+```bash
+bun scripts/benchmark/container-scale.ts                       # tiers 5,10,20,50,80,100, gate on 100
+bun scripts/benchmark/container-scale.ts --tiers 5,10 --gate 10
+bun scripts/benchmark/container-scale.ts --legacy-healthcheck  # old 10s probes, for comparison
+```
+
+It uses the ERP images that `tdk up` already built in `../tdk-erp-system` (override with `--project` or `TDK_BENCH_PROJECT`). For each tier it starts N real service containers with the generated runtime settings (`--init`, 512 MiB / 0.5 CPU limits, healthchecks against `/health`). It then records:
+
+- launch time, first-healthy and all-healthy time
+- crashes and OOM kills, with each container's exit code and last log line
+- total and average memory, CPU
+- `docker ps` latency (`tdk doctor` fails above 10 s)
+- p50 `/health` latency and host load
+
+Containers are removed after each tier. Results are printed as a table and saved to `benchmarks/results/*.json`.
+
+**Gate:** at the gate tier, every container must be healthy within the timeout with no crashes. Total memory must fit `(Docker VM memory - 512 MiB) / 100` per service, and `docker ps` must answer in under 10 s. The exit code is 1 when the gate fails.
+
+**Git hook (opt-in):** call `scripts/benchmark/pre-release-gate.sh` from `pre-push` or `pre-commit`. It does nothing unless `TDK_BENCH_GATE=1`.
+
+Rebuild the ERP images after engine changes (`tdk project --yes && tdk up` in the ERP project) so the benchmark measures the current layers. The header line reports what share of images use a single-process `CMD`.