docker-devkit 0.1.13__tar.gz → 0.1.15__tar.gz

docker_devkit-0.1.15/AGENTS.md

@@ -0,0 +1,62 @@
+# docker-devkit
+
+## What this is
+
+`docker-devkit` owns the Docker-stack lifecycle commands — `up`, `down`, `build` — plus the helpers they share (`detect_gpu`, `modes`, `context_sha`). It is a small, generic package that any repo shipping a compose graph can depend on to get the same bring-up/tear-down / cross-build flow, with the compose assembly declared per consumer in `[tool.docker-devkit.lifecycle]` so a repo that authors its own images and a repo that only OCI-includes an upstream artifact both work through the same commands.
+
+The package is `docker_devkit` (src-layout under `src/docker_devkit/`); all dependencies resolve from PyPI (`bashrun>=0.1.0`; git-source pins only in scratch branches testing unreleased changes). The repo and package renamed twice on the way here — `stack-lifecycle` → `stack-toolkit` (2026-09-20, before any publish) and `stack-toolkit` → `docker-devkit` (2026-09-21, member of the `-devkit` family). The PyPI identity is fresh: `docker-devkit` starts its own tag ledger at `0.1.0`; the terminal `stack-toolkit` distributions (≤0.1.1) are deprecation signposts pointing here, not this package's history.
+
+## Release flow
+
+Publishing rides `release.yml`, triggered by a successful CI run on a `main` push: the machinery — release-devkit's `publish-stable` (an inlined, version-pinned `uvx` step in the publish job, keeping OIDC identity local), never a project dependency (docker-devkit sits inside its own dependency graph; a project-level release-devkit edge is a resolver cycle) — computes the plan from the tag ledger and path-diff, patches the version ephemerally, and publishes to PyPI under OIDC trusted publishing (publisher bound to `release.yml`, no environment). The committed `pyproject.toml` version is permanently the `0.0.0.dev0` sentinel; the `docker-devkit-v*` tags are the version ledger (declared `major_minor` line in `publish-config.json`, patch-auto within the line). While pre-1.0, breaking changes ride the normal patch flow without a bump; from 1.0 on, API-breaking changes ship with a manually bumped `major_minor` — patch-auto assumes additive changes.
+
+## Shape
+
+Entry points (`[project.scripts]`): `up` → `up.py:app`, `down` → `down.py:app`, `build` → `build_docker.py:app`, `generate-score` → `score_generate.py:app`. All accept `--help`.
+
+| Module | Role |
+|---|---|
+| `up.py` | `docker compose up`. Flags: `--attached`/`-a`, `--quiet-pull`/`-q`, `--build`, `--gpu auto\|cuda\|rocm\|none`, `--dev`. |
+| `down.py` | `docker compose down`. Flags: `--volumes`/`-v` (also removes named volumes), `--gpu`. |
+| `build_docker.py` | `run_build()` + the `build` command — cross-builds all images per `compose.bake.yml`, writes `.env.lock` (one `NAME=path:tag@sha256:…` entry per `x-base-images` declaration; `@digest`-pinned declarations pass through unresolved) and `.env.shas` (locally-built `tree-<hash>` tags). `--mode ci` hard-requires `x-registry-cache` in the bake file. |
+| `score_generate.py` | `generate-score` — deterministic Score-manifest generation from a consumer's `score/` sources: `${ALL_CAPS}` placeholder expansion into tempdir-rendered workload copies, idempotent `score-compose`/`score-k8s` init (state preserved across runs), and a determinism suite (document sort, state-path normalisation, 0644 modes, LF newlines). Consumer policy — workloads, provisioners, patch templates, output paths, storage classes, publishes, tool versions — is declared in the consumer's root `pyproject.toml` under `[tool.docker-devkit.generate-score]` (`ScoreConfig`); `load_score_config()` reads it, `ensure_score_tools()` fetches the pinned score-compose/score-k8s release binaries (for CI staleness gates). `--local` writes the k8s artifact to the configured gitignored local path with the local storage class. |
+| `lifecycle.py` | `LifecycleConfig` — the `[tool.docker-devkit.lifecycle]` declaration (`files`: compose files in layering order with `{gpu}` templating; `dev_file`: optional dev overlay); `load_lifecycle_config()` (walks `tool → docker-devkit → lifecycle`, `None` when absent), `enforce_bake_declaration()` (a canonical `compose.bake.yml` requires a declaration), `expand_compose_files()` (assembles the `-f` list; a `{gpu}` entry is dropped when the GPU resolves to `none`; `dev_file` appended when a dev overlay applies). |
+| `detect_gpu.py` | `detect_gpu()` → `cuda`/`rocm`/`none` from host devices; the `Gpu` literal. |
+| `modes.py` | `resolve_auth_mode()` — validates `PUBLIC_URL` + `AUTH_MODE` from `.env`, rejecting `keycloak` over cleartext `http://`. |
+| `context_sha.py` | `compute_service_shas(repo_root, bake: BakeDocument)` — per-service `tree-<hash>` image tags over the `.dockerignore`-allowlisted git tree. |
+| `documents.py` | `BakeDocument` + `parse_bake(path)` — the typed bake-file model every consumer of bake data goes through (build, up, down, generate-score, the reference collectors). A field validator enforces at parse time that every `x-base-images` value is `path:tag` or `path@digest` (a `${…}` hole or a bare path is a validation error, killing the silent-unpinned `FROM` → `:latest` failure mode). `digest_pinned(reference)` is the shared tail test. |
+| `image_refs.py` | Image-reference collection, split by source. `declared_references(root)` enumerates bake `x-base-images` declarations (parsed and validated via `BakeDocument`) — the single source consumer mirroring consumes. `stray_references(root, …globs)` scans Dockerfile `FROM`, Dockerfile `COPY --from=` (slash-bearing captures only, so build-stage names are skipped), and yaml `image:` lines — hand-written literal refs that bypass the declaration system; `unpinned_references(root)` polices exactly those strays (every slash-bearing reference must carry a tag, a digest, or a trailing `${build-arg}` suffix — consumer CIs fail on its result). `bake_base_image_refs(bake: BakeDocument)` adapts declarations to `ImageReference` pairs; `version_coupling_violations(root, couplings)` (each `VersionCoupling` names a pyproject-declared version and the `VersionSite` regexes that must restate it — bake `x-base-images` values or raw file text — so non-derivable couplings like uv `required-version` ↔ uv-base-image tag can't drift silently; the mechanism lives here, the registry in consumer repos' preflight) is unchanged. Remote digest resolution goes through `imagetools inspect`. Compose files are parsed by nothing in this package. |
+
+## Constraints
+
+### Lifecycle declaration contract
+
+**Context.** A repo that authors its own stack builds its own images and assembles a multi-file compose graph (e.g. `compose.yml` + `compose.postgres.yml` + a GPU layer + `compose.dev.yml`), injecting per-service `${*_SHA}` image tags and a `.env.lock` of third-party digests. A consumer repo has none of that: it ships a single self-contained compose file that OCI-includes the upstream artifact already baked (image digests and internal vars frozen as literals) or includes a sibling checkout (which supplies its own `.env.lock`/`.env.shas` through the include's `env_file`).
+
+**Constraint.** `up`/`down` assemble the graph from `[tool.docker-devkit.lifecycle]` in the consumer's root pyproject — one code path, declared data. Keys: `files` (compose files in layering order, `{gpu}` templating, a `{gpu}` entry dropped when the GPU resolves to `none`; defaults to empty), `dev_file` (dev overlay, appended by `up --dev` and always by `down` so teardown removes containers from prior dev bring-ups). No table → single-file default (`-f compose.yml`). `--compose-file` does not exist; raw `docker compose` is the pressure valve. `--dev` is opt-in — production-shaped bring-up is the default. `.env.lock` is required unconditionally, no knob; consumers pin stock images via `${…_IMAGE:?err}` holes filled by a committed lock. A canonical `compose.bake.yml` present without a lifecycle declaration is a hard error (declare the table, or the file doesn't belong under that name) — guards are budgeted for silent failures only, and a missing lock already fails loudly. The bake file remains the build/up contract: SHA injection and `--build` acceptance key on `compose.bake.yml` existing, and `build` always reads exactly that name. An empty assembly (`files` empty/omitted, no dev overlay) is a loud `up`/`down` error — the honest state for a builds-only repo.
+
+**Consequences.** placeframe's shape lives in placeframe's pyproject, not in this package: `files = ["compose.yml", "compose.postgres.yml", "compose.{gpu}.yml"]`, `dev_file = "compose.dev.yml"`. A consumer repo declares a bare table (builds images, no repo-local runtime graph) and runs `uv run up` against its own `.env` + committed `.env.lock`.
+
+### File and env conventions the commands assume
+
+- `.env` (required) — carries `PUBLIC_URL` and `AUTH_MODE`; `modes.py` reads it and rejects `keycloak` over cleartext `http://`.
+- `.env.lock` (required in every repo, checked in) — third-party and base-image pins, written by `build`: one `NAME=path:tag@sha256:…` entry per bake declaration. `up`/`down` refuse without it; consumers satisfy it with a committed lock filling the `${…_IMAGE:?err}` holes.
+- `.env.shas` (where images are built locally, gitignored, per-build) — `tree-<hash>` tags of locally-built images, written by `build`. `up` injects the `${*_SHA}` values into the environment directly and reads neither file; `.env.shas` exists so a consumer running the compose files through raw `docker compose` can resolve those holes via `--env-file .env.shas`.
+- `compose.bake.yml` (where a local build graph exists, required) — the buildx-bake description; its top-level `x-cross-compile-targets` list (optional) is the set of services excluded from the default target list because they need per-arch treatment, opt-in via `--targets`. One bake file name, ever: `build` reads exactly this name.
+- the compose layers declared in `[tool.docker-devkit.lifecycle]` — e.g. placeframe's `compose.yml` / `compose.postgres.yml` / `compose.<gpu>.yml` / `compose.dev.yml`; the GPU one is picked by `detect_gpu()` (or the `--gpu` override).
+- `.dockerignore` (where images are built locally) — allowlist that `context_sha.py` hashes over to compute per-service `tree-<hash>` tags. See "Docker build context" in `AGENTS-SHARED.md` for the allowlist rationale.
+
+The build command writes to two files rather than one because a CI invariant forbids built-image digests in `.env.lock` (which is committed and reviewed) and because `.env.shas` changes on every source edit, so it stays out of version control.
+
+### Package split rationale
+
+Lifecycle (`up`/`down`/`build`) and codegen (client generation, datamodel generation, workspace locking) have disjoint dependency surfaces. Bundling them would force a consumer that only brings a stack up to install `sqlacodegen`, `psycopg`, `datamodel-code-generator`. Keeping the lifecycle in its own small package lets consumers git-reference just this without dragging the codegen surface behind it.
+
+### `--quiet-pull` logging and GPU auto-detection
+
+`--quiet-pull` suppresses compose's per-layer progress (CUDA images carry hundreds of layers) while still surfacing pull totals. `--gpu auto` resolves to `cuda`/`rocm`/`none` from host devices and selects the matching `compose.<gpu>.yml`; override only to reproduce a CI environment or test the CPU-only path.
+
+## See also
+
+- `README.md` — human-facing setup and usage.
+- [`bashrun`](https://github.com/outernet-foundation/bashrun) — the guardrailed subprocess wrapper the lifecycle commands shell out through.

{docker_devkit-0.1.13 → docker_devkit-0.1.15}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.5
 Name: docker-devkit
-Version: 0.1.13
-Summary: Docker-stack lifecycle commands (up/down/build) with a native/consumer split
+Version: 0.1.15
+Summary: Docker-stack lifecycle commands (up/down/build) driven by a pyproject-declared compose assembly
 License-File: LICENSE
 License-File: NOTICE
 Requires-Python: >=3.13

{docker_devkit-0.1.13 → docker_devkit-0.1.15}/README.md

@@ -1,8 +1,8 @@
 # docker-devkit
 
-Docker-stack lifecycle commands — `up`, `down`, `build` — with a native/consumer split. A repo that authors its own stack (Dockerfiles + `compose.bake.yml`) uses the native mode: multi-file compose assembly, per-service SHA injection, `.env.lock` resolution. A consumer repo that OCI-includes an already-baked upstream stack uses the same commands as a thin wrapper: single-graph, `.env`-only, no build path.
+Docker-stack lifecycle commands — `up`, `down`, `build` — with the compose assembly declared per repo in `[tool.docker-devkit.lifecycle]`. A repo that authors its own stack (Dockerfiles + `compose.bake.yml`) declares its file layers and gets multi-file assembly, per-service SHA injection, and `.env.lock` resolution; a consumer repo that OCI-includes an already-baked upstream stack declares a bare table and runs the same commands as a thin wrapper over its own `.env` + committed `.env.lock`.
 
-See [`AGENTS.md`](./AGENTS.md) for the file/layout conventions the commands assume (`compose.bake.yml`, `.env.lock`, `.env.shas`, `PUBLIC_URL` / `AUTH_MODE` in `.env`) and the mode-detection contract.
+See [`AGENTS.md`](./AGENTS.md) for the file/layout conventions the commands assume (`compose.bake.yml`, `.env.lock`, `.env.shas`, `PUBLIC_URL` / `AUTH_MODE` in `.env`) and the declaration contract.
 
 ## Setup
 
@@ -17,8 +17,9 @@ uv sync
 From a stack repo's root:
 
 ```bash
-uv run up                      # docker compose up (auto-detects GPU; native or consumer per compose.bake.yml)
-uv run up --build              # build images locally first (native only)
+uv run up                      # docker compose up (auto-detects GPU; assembles the declared lifecycle files)
+uv run up --dev                # layer the declared dev overlay (bind-mount/debug bring-up)
+uv run up --build              # build images locally first (requires compose.bake.yml)
 uv run up --gpu none           # override GPU auto-detection
 uv run down                    # docker compose down
 uv run down -v                 # also remove named volumes

docker_devkit-0.1.15/plan.md

@@ -0,0 +1,141 @@
+# One home for third-party image declarations; docker-devkit stops parsing compose
+
+Status note for pickup: the docker-devkit half is DONE — redesign executed in
+`2a1fe90` (compose parsing deleted, `declared_references`/`stray_references`
+split, `BakeDocument` base-images validator, typed bake plumbing through
+build/up/down/generate-score, ci-mode registry-cache guard). pytest 77 green,
+ruff + basedpyright clean — the checkpoint's intentional failures are gone.
+Still pending, in order: operator push + tag-ledger release of this repo;
+release-devkit 0.1.11 (`mirror_images.py` → `declared_references`); the
+placeframe PR; capture-tool renames. No consumer repo has been touched yet.
+
+Note on release numbering: the plan projects 0.1.15, but the local tag ledger
+sits at docker-devkit-v0.1.11 with the lifecycle rewrite also unreleased —
+patch-auto will compute the real number at publish time, and the placeframe
+repin must use whatever ships, not the projection.
+
+## Why this initiative exists
+
+docker-devkit parses consumer compose files for exactly one purpose: discovering
+`x-image-ref` labels (declarations of externally-pulled images) so `run_build`
+can digest-lock them and `mirror-images` can mirror them. That single need dragged
+in a pile of wrong-shaped machinery:
+
+- Compose's YAML dialect tags (`!reset`) require PyYAML constructor registration,
+  and PyYAML's constructor API is untyped — a type-checker fight with no clean
+  resolution (the only surviving form was a suppression-boundary debate).
+- The include-following logic in `run_build` serves zero org repos (the only
+  compose include anywhere is make-it-sing's `oci://` one, which contributes
+  nothing and was skipped anyway).
+- Two declaration mechanisms exist for one need: `x-image-ref` labels in compose
+  (placeframe: fluent-bit, cloudbeaver, keycloak, ngrok) and `x-base-images` in
+  the bake file (placeframe: loki, alloy, grafana, and every Dockerfile base).
+- The `_DIGEST` suffix lock shape (`image: path${X_DIGEST:?err}` in compose,
+  lock `X_DIGEST=@sha256:…`) authors the mirror path TWICE — bake declaration +
+  compose image line — a drift exposure (path drift fails loud at pull; tags
+  cannot drift, but the duplication is authored).
+- The Dockerfile suffix form (`FROM path${X_DIGEST}`) has a silent-unpinned
+  failure mode: an empty arg yields a tagless `FROM path`, and Docker silently
+  pulls `:latest`.
+
+## The design
+
+- **Single declaration home**: bake `x-base-images`. Keys use the `_IMAGE`
+  convention (`UV_BASE_IMAGE`, `KEYCLOAK_IMAGE`, …). One authored string
+  (`path:tag`) per third-party image, anywhere in the org.
+- **run_build single-emits** one lock entry per declaration:
+  `NAME=path:tag@sha256:…`. No dual shapes, no suffix entries.
+- **Compose image lines** are pure holes: `image: "${KEYCLOAK_IMAGE:?err}"`.
+- **Dockerfiles** use `ARG NAME` + `FROM ${NAME}`. An empty arg is a hard parse
+  error — this fixes the silent-`:latest` foot-gun.
+- **BakeDocument gains a field validator**: every `base_images` value must be
+  `path:tag` or `path@digest`. The declarations half of `unpinned_references`
+  moves to parse time (enforced on every command run, not a CI-only scan).
+- **`collect_repo_references` splits**: `declared_references(root)` (bake
+  declarations only — what mirror-images consumes) and a stray-ref scan
+  (Dockerfile `FROM`/`COPY --from` literals + score yaml `image:` lines — what
+  `unpinned_references` polices; it survives with that narrow job of catching
+  hand-written literal refs that bypass the declaration system).
+- **`version_coupling_violations` is unchanged** — it guards a genuinely
+  non-derivable agreement (pyproject `tool.uv.required-version` ↔ image tag; the
+  tag carries suffixes like `-python3.13-trixie-slim` that pyproject doesn't
+  hold). Two authored truths that must agree is a different problem from one
+  truth scattered; the check is correct, keep it.
+
+## Changes
+
+### docker-devkit (this repo) — release 0.1.15
+
+- `image_refs.py`: delete `compose_image_refs`, `ComposeDialectLoader`, both
+  `add_constructor` registrations, and the compose branch of
+  `collect_repo_references`; implement the `declared_references` / stray-scan
+  split. `bake_base_image_refs` and `version_coupling_violations` stay.
+- `documents.py`: add the `base_images` field validator.
+- `build_docker.py`: delete the third-party `x-image-ref` resolution block;
+  single-emit lock entries from `x-base-images`; wire all bake consumption
+  through `parse_bake`/`BakeDocument` (typed access replaces dict plumbing —
+  including `compute_service_shas(repo_root, bake: BakeDocument)` and
+  `compute_default_targets(bake: BakeDocument, …)` signatures, with `up.py`,
+  `down.py`, `score_generate.py` parsing at their call sites); loud guard:
+  `--mode ci` requires `x-registry-cache`.
+- Tests: delete the compose-dialect tests; add validator + split-function tests;
+  existing fixtures via `BakeDocument.model_validate`.
+- `types-pyyaml` stays as a dev dependency (it types `parse_bake`'s PyYAML
+  surface; with the dialect code gone, zero suppressions are needed anywhere).
+- AGENTS.md: update the `image_refs.py` row and any compose-parsing references
+  (separate prose commit from the code).
+- Release via the normal tag-ledger flow; operator push (App token cannot push).
+
+### release-devkit — next patch (0.1.11)
+
+- `mirror_images.py`: switch from
+  `collect_repo_references(Path.cwd(), dockerfile_glob=None)` to
+  `declared_references(root)` — same refs, declarations-only source.
+
+### placeframe — one PR
+
+- `compose.bake.yml`: add declarations for the 4 `x-image-ref` services
+  (`CLOUDBEAVER_IMAGE`, `KEYCLOAK_IMAGE`, `FLUENT_BIT_IMAGE`, `NGROK_IMAGE`,
+  values = today's label strings); rename every existing `_DIGEST` declaration
+  key to the `_IMAGE` convention; `x-base-args` keys follow the renames.
+- `compose.yml`: delete the 4 labels (their image lines are textually unchanged —
+  they were already `${X_IMAGE:?err}` holes); flip loki×2 / alloy / grafana from
+  `path${X_DIGEST:?err}` to `${X_IMAGE:?err}`; inventory every `${…_IMAGE}` hole
+  (at minimum `SEAWEEDFS_AUDIT_IMAGE`) and ensure a declaration exists for each.
+- Dockerfiles: convert every base `FROM` to `ARG NAME` + `FROM ${NAME}`.
+- Version-coupling site registry (consumer preflight): rename `base_image` site
+  names to the new declaration keys — locate the registry during execution.
+- Regenerate `.env.lock` (`uv run build --lock-only`); preflight green; repin
+  docker-devkit `>=0.1.15`.
+- AGENTS.md mentions of `x-image-ref` updated (separate prose commit).
+
+### capture-tool
+
+- 3 Dockerfiles + `compose.zed.bake.yml` `x-base-args` key renames; regen
+  `.env.lock`; repin docker-devkit.
+
+### make-it-sing — NOT this initiative
+
+Its own `plan.md` resumes AFTER this one lands (its Stage 3/5 consume the new
+pattern). Its committed Stage 3 checkpoint (`6b4120c`) still carries the suffix
+form (`ARG UV_BASE_DIGEST` + `FROM mirror/…${UV_BASE_DIGEST}`) — flip to
+`ARG UV_BASE_IMAGE` + `FROM ${UV_BASE_IMAGE}` when resuming that plan.
+
+## Ordering
+
+docker-devkit 0.1.15 → release-devkit 0.1.11 → placeframe PR → capture-tool.
+All releases are operator pushes; the sandbox App token has Contents:read only.
+
+## Verification
+
+- docker-devkit: pytest + ruff + basedpyright all green (the checkpoint's known
+  failures must be gone — their absence is the proof the deletion was complete).
+- placeframe / capture-tool: `uv run build --lock-only` round-trip, preflight
+  battery, `docker compose config` render.
+
+## Deliberately out of scope
+
+- Score yaml scanning changes.
+- Renaming the `x-base-images` key itself (it now covers runtime images too —
+  placeframe's loki already made that de facto; note it in a bake-file comment).
+- make-it-sing CI stages (owned by its own plan).

{docker_devkit-0.1.13 → docker_devkit-0.1.15}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "docker-devkit"
-version = "0.1.13"
-description = "Docker-stack lifecycle commands (up/down/build) with a native/consumer split"
+version = "0.1.15"
+description = "Docker-stack lifecycle commands (up/down/build) driven by a pyproject-declared compose assembly"
 requires-python = ">=3.13"
 dependencies = [
     "bashrun>=0.1.0",
@@ -13,7 +13,12 @@ dependencies = [
 ]
 
 [dependency-groups]
-dev = ["basedpyright>=1.39.10", "ruff>=0.14.11", "pytest>=8.0.0"]
+dev = [
+    "basedpyright>=1.39.10",
+    "ruff>=0.14.11",
+    "pytest>=8.0.0",
+    "types-pyyaml>=6.0.12.20260906",
+]
 
 [build-system]
 requires = ["hatchling"]

{docker_devkit-0.1.13 → docker_devkit-0.1.15}/src/docker_devkit/build_docker.py

@@ -9,13 +9,13 @@ from subprocess import CalledProcessError
 from typing import Annotated, Any, Literal
 
 import typer
-import yaml
 from bashrun.bash import bash, bash_output
 from .detect_gpu import GPU_TYPES, Gpu, detect_gpu
+from .documents import BakeDocument, digest_pinned, parse_bake
 from pydantic_settings import BaseSettings
 
 from .context_sha import compute_service_shas
-from .image_refs import bake_base_image_refs, compose_service_refs, resolve_remote_digest
+from .image_refs import bake_base_image_refs, resolve_remote_digest
 from .modes import parse_env_file
 
 
@@ -27,7 +27,6 @@ settings = Settings.model_validate({})
 
 LOCK_FILE = Path(".env.lock")
 ENV_SHAS_FILE = Path(".env.shas")
-COMPOSE_FILE = Path("compose.yml")
 DEFAULT_BAKE_FILE = Path("compose.bake.yml")
 METADATA_PATH = Path("metadata.json")
 
@@ -51,11 +50,8 @@ def build(
     no_cache: bool = typer.Option(False, "--no-cache", help="Force rebuild by disabling cache usage."),
     targets_opt: Annotated[
         list[str] | None,
-        typer.Option("--targets", "-t", help="Build only these services (from the selected bake file)."),
+        typer.Option("--targets", "-t", help="Build only these services (from compose.bake.yml)."),
     ] = None,
-    bake_file: Annotated[
-        Path, typer.Option("--bake-file", help="Bake file to load (e.g. compose.bake.yml or compose.zed.bake.yml).")
-    ] = DEFAULT_BAKE_FILE,
 ) -> None:
     run_build(
         upgrade=upgrade,
@@ -65,7 +61,6 @@ def build(
         gpu_only=gpu_only,
         no_cache=no_cache,
         targets_opt=targets_opt,
-        bake_file=bake_file,
     )
 
 
@@ -78,35 +73,27 @@ def run_build(
     gpu_only: bool = False,
     no_cache: bool = False,
     targets_opt: list[str] | None = None,
-    bake_file: Path = DEFAULT_BAKE_FILE,
 ) -> None:
-    service_shas = compute_service_shas(Path.cwd(), bake_file)
+    bake = parse_bake(DEFAULT_BAKE_FILE)
+
+    service_shas = compute_service_shas(Path.cwd(), bake)
     os.environ.update(service_shas)
 
-    # Tags of the images built this run — the local analog of .env.lock's pulled
-    # digests — so raw `docker compose` can resolve the compose graph's ${*_SHA} holes.
+    # Write .env.shas with the locally-built image tags
     ENV_SHAS_FILE.write_text(
         "".join(f"{key}={value}\n" for key, value in sorted(service_shas.items())), encoding="utf-8"
     )
 
-    # Read bake, compose, and lock files. A bake-only repo (cross-builds images
-    # from a bake file, no local compose graph) has no root compose file; the
-    # only downstream consumers of compose_data are the include merge and the
-    # third-party x-image-ref scan, both empty-safe.
-    bake_data: dict[str, Any] = yaml.safe_load(bake_file.read_text(encoding="utf-8"))
-    compose_data: dict[str, Any] = (
-        yaml.safe_load(COMPOSE_FILE.read_text(encoding="utf-8")) if COMPOSE_FILE.exists() else {}
-    )
-    for include in compose_data.pop("include", []):
-        include_path = COMPOSE_FILE.parent / (include if isinstance(include, str) else include["path"])
-        included: dict[str, Any] = yaml.safe_load(include_path.read_text(encoding="utf-8"))
-        compose_data.setdefault("services", {}).update(included.get("services", {}))
+    # Load any existing lock
     lock_data = parse_env_file(LOCK_FILE) if LOCK_FILE.exists() else {}
 
     # TOOD: Create separate commands for ci and local modes so typer can do this validation instead of us
     if mode == "ci" and gpu == "auto":
         raise typer.BadParameter("In CI mode, --gpu cannot be 'auto'; specify 'cuda' or 'rocm'.")
 
+    if mode == "ci" and bake.registry_cache is None:
+        raise typer.BadParameter("In CI mode, the bake file must declare x-registry-cache.")
+
     if gpu_only and gpu not in GPU_TYPES:
         raise typer.BadParameter("--gpu-only requires a concrete gpu (cuda or rocm), not 'auto' or 'none'.")
 
@@ -118,28 +105,18 @@ def run_build(
     if gpu == "auto" and not lock_only:
         gpu = detect_gpu()
 
-    # Resolve base image external dependencies
-    for occurrence in [ref for ref in bake_base_image_refs(bake_data) if upgrade or ref.name not in lock_data]:
-        lock_data[occurrence.name] = (
-            f"@{'' if '$' in occurrence.reference else resolve_remote_digest(occurrence.reference)}"
-        )
-
-    # Resolve third-party image external dependencies. x-image-ref marks services whose image
-    # is sourced externally (vs. built from a bake file), so it's the right discriminator
-    # regardless of how many bake files exist.
-    third_party_images: dict[str, str] = {
-        occurrence.name.upper().replace("-", "_") + "_IMAGE": occurrence.reference
-        for occurrence in compose_service_refs(compose_data)
-    }
-
-    # Re-resolve when explicitly requested, when unseen, or when the image name in compose.yml
-    # changed from what's in the lock file (e.g. postgres:16-alpine → postgis/postgres:16-3.4-alpine)
-    for image, ref in {
-        image: ref
-        for image, ref in third_party_images.items()
-        if upgrade or image not in lock_data or not lock_data[image].startswith(ref + "@")
-    }.items():
-        lock_data[image] = f"{ref}@{'' if '$' in ref else resolve_remote_digest(ref)}"
+    # One lock entry per declared third-party image: NAME=path:tag@sha256:…
+    for occurrence in bake_base_image_refs(bake):
+        if digest_pinned(occurrence.reference):
+            lock_data[occurrence.name] = occurrence.reference
+            continue
+        if (
+            not upgrade
+            and occurrence.name in lock_data
+            and lock_data[occurrence.name].startswith(occurrence.reference + "@")
+        ):
+            continue
+        lock_data[occurrence.name] = f"{occurrence.reference}@{resolve_remote_digest(occurrence.reference)}"
 
     # Update main lock file
     LOCK_FILE.write_text(
@@ -158,17 +135,17 @@ def run_build(
 
     # Determine bake targets
     if targets_opt:
-        unknown = set(targets_opt) - set(bake_data["services"])
+        unknown = set(targets_opt) - set(bake.services)
         if unknown:
-            raise typer.BadParameter(f"Unknown targets: {unknown}. Available: {sorted(bake_data['services'])}")
-        targets = [t for t in bake_data["services"] if t in set(targets_opt)]
+            raise typer.BadParameter(f"Unknown targets: {unknown}. Available: {sorted(bake.services)}")
+        targets = [t for t in bake.services if t in set(targets_opt)]
     else:
-        targets = compute_default_targets(bake_data, gpu, gpu_only)
+        targets = compute_default_targets(bake, gpu, gpu_only)
 
     # Configure registry caches in CI mode
     if mode == "ci":
         for target in targets:
-            target_cache = f"{bake_data['x-registry-cache']}:{target}"
+            target_cache = f"{bake.registry_cache}:{target}"
             command_arguments.append(
                 f"--set {target}.cache-to+=type=registry,ref={target_cache},mode=max,image-manifest=true,oci-mediatypes=true"
             )
@@ -187,8 +164,7 @@ def run_build(
         else:
             raise RuntimeError(f"Unsupported host architecture: {machine}")
         for target in targets:
-            target_platforms = bake_data["services"][target].get("build", {}).get("platforms", [])
-            if len(target_platforms) > 1:
+            if len(bake.services[target].build.platforms) > 1:
                 command_arguments.append(f"--set {target}.platform={host_platform}")
 
     # Load or push images based on mode
@@ -207,7 +183,7 @@ def run_build(
     # Bake images
     command = [
         "docker buildx bake",
-        f"-f {bake_file}",
+        f"-f {DEFAULT_BAKE_FILE}",
         f"--metadata-file {METADATA_PATH}",
         "--progress auto",
         "--provenance=false",
@@ -218,7 +194,7 @@ def run_build(
     # Sanity check
     baked_images: dict[str, Any] = json.loads(METADATA_PATH.read_text()) if METADATA_PATH.exists() else {}
     if not set(targets) <= baked_images.keys():
-        raise RuntimeError("Baked images do not match target images; something went wrong during the bake.")
+        raise RuntimeError("Bake output does not cover the requested targets")
 
 
 # Vibe code - Gemini 3
@@ -240,10 +216,7 @@ def _check_gc_limits(min_gb: int = 60):
     raw = data.get("builder", {}).get("gc", {}).get("defaultKeepStorage")
     if not raw:
         sample = json.dumps({"builder": {"gc": {"defaultKeepStorage": f"{min_gb}GB"}}}, indent=2)
-        raise RuntimeError(
-            f"Missing 'builder.gc.defaultKeepStorage' in {config}; Docker's default is too low for GPU builds "
-            f"(need >= {min_gb}GB). Add the following (merging into any existing keys) and restart Docker:\n\n{sample}"
-        )
+        raise RuntimeError(f"builder.gc.defaultKeepStorage missing in {config}; need >= {min_gb}GB:\n\n{sample}")
 
     m = re.match(r"^(\d+(?:\.\d+)?)\s*([TGMK]i?B)?$", str(raw), re.IGNORECASE)
     if not m:
@@ -254,9 +227,7 @@ def _check_gc_limits(min_gb: int = 60):
     val = float(m.group(1)) * mult.get(unit, 1 / 1024**3)
 
     if val < min_gb:
-        raise RuntimeError(
-            f"UNSAFE GC LIMIT: {val:.1f}GB in {config} (Required: {min_gb}GB). RESTART DOCKER AFTER FIXING."
-        )
+        raise RuntimeError(f"Docker GC limit {val:.1f}GB in {config} is below the required {min_gb}GB")
 
     print(f"    [OK] Docker GC Limit verified: {val:.1f}GB")
 
@@ -267,11 +238,9 @@ def _check_gc_limits(min_gb: int = 60):
 # Services without build.tags (e.g. neural-networks-base-*) stay out too: they are
 # build-only dependencies pulled in via additional_contexts, and bake rejects a
 # tagless target under --push.
-def compute_default_targets(bake_data: dict[str, Any], gpu: Gpu, gpu_only: bool = False) -> list[str]:
-    cross_compile_targets: set[str] = set(bake_data.get("x-cross-compile-targets", []))
-    tagged_services = [
-        service for service, config in bake_data["services"].items() if config.get("build", {}).get("tags")
-    ]
+def compute_default_targets(bake: BakeDocument, gpu: Gpu, gpu_only: bool = False) -> list[str]:
+    cross_compile_targets: set[str] = set(bake.cross_compile_targets)
+    tagged_services = [service for service, config in bake.services.items() if config.build.tags]
     if gpu_only:
         return [
             service

{docker_devkit-0.1.13 → docker_devkit-0.1.15}/src/docker_devkit/context_sha.py

@@ -3,23 +3,20 @@ from __future__ import annotations
 from pathlib import Path, PurePosixPath
 from tempfile import TemporaryDirectory
 
-from typing import Any
-
 import pathspec
-import yaml
 from bashrun.bash import bash, bash_output
 
+from .documents import BakeDocument
+
 
-def compute_service_shas(repo_root: Path, bake_file: Path) -> dict[str, str]:
+def compute_service_shas(repo_root: Path, bake: BakeDocument) -> dict[str, str]:
     repo_root = repo_root.resolve()
 
-    bake_data: dict[str, Any] = yaml.safe_load(bake_file.read_text(encoding="utf-8"))
     service_dirs: set[str] = set()
-    for config in bake_data["services"].values():
-        build: dict[str, Any] = config.get("build", {})
-        if not build.get("tags"):
+    for config in bake.services.values():
+        if not config.build.tags:
             continue
-        service_dirs.add(str(PurePosixPath(build.get("dockerfile", "")).parent))
+        service_dirs.add(str(PurePosixPath(config.build.dockerfile).parent))
 
     tree_entries = bash_output("git ls-tree -r HEAD", cwd=repo_root).splitlines()
 

docker_devkit-0.1.15/src/docker_devkit/documents.py

@@ -0,0 +1,64 @@
+from __future__ import annotations
+
+import re
+from pathlib import Path
+from typing import TypeVar
+
+import yaml
+from pydantic import BaseModel, ConfigDict, Field, field_validator
+
+ModelT = TypeVar("ModelT", bound=BaseModel)
+
+DECLARED_REFERENCE_PATTERN = re.compile(
+    r"^(?P<path>[^:@\s$]+)(?::(?P<tag>[^@\s$]+))?(?:@(?P<digest>sha256:[a-f0-9]{64}))?$"
+)
+DIGEST_TAIL_PATTERN = re.compile(r"@sha256:[a-f0-9]{64}$")
+
+
+class BakeBuild(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    context: str = "."
+    dockerfile: str
+    target: str | None = None
+    platforms: list[str] = Field(default_factory=list)
+    args: dict[str, str] = Field(default_factory=dict)
+    tags: list[str] = Field(default_factory=list)
+    additional_contexts: dict[str, str] = Field(default_factory=dict)
+
+
+class BakeService(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    build: BakeBuild
+
+
+class BakeDocument(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    base_images: dict[str, str] = Field(default_factory=dict, alias="x-base-images")
+    base_args: dict[str, str] = Field(default_factory=dict, alias="x-base-args")
+    registry_cache: str | None = Field(default=None, alias="x-registry-cache")
+    cross_compile_targets: list[str] = Field(default_factory=list, alias="x-cross-compile-targets")
+    services: dict[str, BakeService] = Field(default_factory=dict)
+
+    @field_validator("base_images")
+    @classmethod
+    def _require_tag_or_digest(cls, base_images: dict[str, str]) -> dict[str, str]:
+        for name, reference in base_images.items():
+            match = DECLARED_REFERENCE_PATTERN.fullmatch(reference)
+            if match is None or not (match.group("tag") or match.group("digest")):
+                raise ValueError(f"x-base-images.{name}: {reference!r} must be 'path:tag' or 'path@digest'")
+        return base_images
+
+
+def digest_pinned(reference: str) -> bool:
+    return DIGEST_TAIL_PATTERN.search(reference) is not None
+
+
+def parse_bake(path: Path) -> BakeDocument:
+    loader = yaml.SafeLoader(path.read_text(encoding="utf-8"))
+    try:
+        return BakeDocument.model_validate(loader.get_single_data())
+    finally:
+        loader.dispose()

docker_devkit-0.1.15/src/docker_devkit/down.py

@@ -0,0 +1,68 @@
+import os
+from pathlib import Path
+from typing import Annotated
+
+import typer
+from bashrun.bash import bash_handoff
+from .detect_gpu import Gpu, detect_gpu
+
+from .context_sha import compute_service_shas
+from .documents import parse_bake
+from .lifecycle import BAKE_FILE, enforce_bake_declaration, expand_compose_files, load_lifecycle_config
+from .modes import resolve_auth_mode
+
+ENV_FILE = Path(".env")
+LOCK_FILE = Path(".env.lock")
+
+app = typer.Typer(add_completion=False)
+
+
+@app.command()
+def down(
+    volumes: bool = typer.Option(False, "--volumes", "-v", help="Remove named volumes."),
+    gpu: Annotated[Gpu, typer.Option("--gpu", help="auto|cuda|rocm|none")] = "auto",
+) -> None:
+    config = load_lifecycle_config(Path.cwd())
+    enforce_bake_declaration(Path.cwd(), config)
+
+    if not ENV_FILE.exists():
+        raise RuntimeError("No .env file found")
+
+    if not LOCK_FILE.exists():
+        raise RuntimeError("No .env.lock found; run 'uv run build --lock-only' in the repo that authors the images")
+
+    if gpu == "auto":
+        gpu = detect_gpu()
+
+    resolve_auth_mode(ENV_FILE)
+
+    if BAKE_FILE.exists():
+        os.environ.update(compute_service_shas(Path.cwd(), parse_bake(BAKE_FILE)))
+
+    # Always layer the dev overlay so containers from a prior --dev bring-up get torn down too
+    compose_files = expand_compose_files(config, gpu, include_dev=True)
+    if not compose_files:
+        raise RuntimeError("No compose files declared; a builds-only repo has no stack to tear down")
+
+    files_args = " ".join(f"-f {compose_file}" for compose_file in compose_files)
+    command = (
+        "docker compose "
+        f"{files_args} "
+        "--profile keycloak "  # Always include so any keycloak containers from a previous AUTH_MODE=keycloak run get torn down
+        "--env-file .env "
+        f"--env-file {LOCK_FILE} "
+        "down --remove-orphans"
+    )
+
+    if volumes:
+        command += " -v"
+
+    bash_handoff(command)
+
+
+def main():
+    app()
+
+
+if __name__ == "__main__":
+    main()