docker-devkit 0.3.0.dev116__tar.gz → 0.4.0__tar.gz

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/AGENTS.md

@@ -22,19 +22,19 @@ Entry points (`[project.scripts]`): `up` → `up.py:app`, `down` → `down.py:ap
 
 | Module | Role |
 |---|---|
-| `up.py` | `docker compose up`. Flags: `--attached`/`-a`, `--quiet-pull`/`-q`, `--build`, `--gpu auto\|cuda\|rocm\|none`, `--dev`, `--allow-upstream-fallback`. |
+| `up.py` | `docker compose up`. Flags: `--attached`/`-a`, `--quiet-pull`/`-q`, `--build`, `--gpu auto\|cuda\|rocm\|none`, `--dev`. |
 | `down.py` | `docker compose down`. Flags: `--volumes`/`-v` (also removes named volumes), `--gpu`. |
-| `build_docker.py` | `run_build()` + the `build` command — cross-builds all images per the resolved image manifest, writing the paired lock (one `NAME=path:tag@sha256:…` entry per `x-base-images` declaration; `@digest`-pinned declarations pass through unresolved; mirror-prefixed declarations resolve their digest against the upstream reference). `--push` is the CI shape (push images + registry caches + digest-manifest shelving) and hard-requires `x-registry-cache`; the flag order mirrors the binding's run line (`--push`, `--targets`, `--gpu`), and `--targets` accepts both repeated flags and one space-joined value. Push mode reads its registry facts from the neutral `REGISTRY`/`REGISTRY_TOKEN`/`REGISTRY_USERNAME` environment — no `GITHUB_*` reads. |
+| `build_docker.py` | `run_build()` + the `build` command — cross-builds all images per the resolved image manifest, writing the paired lock (one `NAME={prefix}/{ref}@sha256:…` entry per upstream-native `x-base-images` declaration, the mirror-prefixed form derived from `mirror.prefix`; `@digest`-pinned declarations pass through unresolved; digests resolve against the upstream reference). `--push` is the CI shape (push images + registry caches + digest-manifest shelving) and hard-requires `x-registry-cache`; the flag order mirrors the binding's run line (`--push`, `--targets`, `--gpu`), and `--targets` accepts both repeated flags and one space-joined value. Push mode reads its registry facts from the neutral `REGISTRY`/`REGISTRY_TOKEN`/`REGISTRY_USERNAME` environment — no `GITHUB_*` reads. |
 | `score_generate.py` | `generate-score` — deterministic Score-manifest generation from a consumer's `stack/score/` sources: `${ALL_CAPS}` placeholder expansion into tempdir-rendered workload copies, idempotent `score-compose`/`score-k8s` init (state preserved across runs), and a determinism suite (document sort, state-path normalisation, 0644 modes, LF newlines). Consumer policy — workloads, provisioners, patch templates, output paths, storage classes, publishes, tool versions — is declared in the consumer's root `docker-devkit.yaml` under `generate-score:` (`ScoreConfig`, model in `config.py`); `load_score_config()` reads it, `ensure_score_tools()` fetches the pinned score-compose/score-k8s release binaries (for CI staleness gates). Output and state paths resolve against the repo root: the compose half runs in `score_dir` (its `.score-compose` state stays there, gitignored), the k8s half runs in the configured `k8s_state` parent so the committed `.score-k8s` state lands in the generated tier. `--local` writes the k8s artifact to the configured gitignored local path with the local storage class. |
 | `config.py` | `DockerDevkitConfig` — the root `docker-devkit.yaml` schema: a required `requires` PEP 440 specifier plus optional `lifecycle` / `mirror` / `generate-score` sections, `extra="forbid"` on every model. `load_config(root)` is the single strictyaml entry point: parses the file (returns `None` when absent — the no-config → single-file-default path for `up`/`down`), then self-checks the installed docker-devkit version against `requires` (refuses loudly outside range; the `0.0.0.dev0` dev sentinel bypasses the check, not the field). The section models (`LifecycleConfig`, `MirrorConfig`, `ScoreConfig`) live here; `load_lifecycle_config` / `load_mirror_config` / `load_score_config` in their modules are thin wrappers over it. |
-| `lifecycle.py` | `LifecycleConfig` (model in `config.py`) — the `lifecycle:` section declaration (`files`: compose files in layering order with `{gpu}` templating; `dev_file`: optional dev overlay); `load_lifecycle_config()` (reads the `lifecycle:` section via `config.load_config`, `None` when the section or file is absent), `resolve_manifest()` / `require_manifest()` (the raising form) / `resolve_lock()` (the image manifest and its lock are a named pair — `workloads/images.yml` + `workloads/images.lock`, with the legacy root `compose.bake.yml` + `.env.lock` pair still resolved so repos can upgrade before moving files), `enforce_bake_declaration()` (a manifest under either name requires a declaration), `expand_compose_files()` (assembles the `-f` list; a `{gpu}` entry is dropped when the GPU resolves to `none`; `dev_file` appended when a dev overlay applies). |
+| `lifecycle.py` | `LifecycleConfig` (model in `config.py`) — the `lifecycle:` section declaration (`files`: compose files in layering order with `{gpu}` templating; `dev_file`: optional dev overlay); `load_lifecycle_config()` (reads the `lifecycle:` section via `config.load_config`, `None` when the section or file is absent), `resolve_manifest()` / `require_manifest()` (the raising form) / `resolve_lock()` (the image manifest and its lock are a named pair — `workloads/images.yml` + `workloads/images.lock`), `enforce_bake_declaration()` (a manifest requires a declaration), `expand_compose_files()` (assembles the `-f` list; a `{gpu}` entry is dropped when the GPU resolves to `none`; `dev_file` appended when a dev overlay applies). |
 | `detect_gpu.py` | `detect_gpu()` → `cuda`/`rocm`/`none` from host devices; the `Gpu` literal. |
 | `matrix.py` | `docker-build-matrix` — emit the CI build leg list as a bare JSON array on stdout (the binding's getter pipes it through its `matrix` verb, which composes the envelope and guards the empty case): one entry per variant × cohort cell, `targets` = that cell's explicitly-buildable services with dependents grouped with the base their `additional_contexts: target:` references pull in (one leg builds a whole dependency-closure group in a single bake invocation). The leg schema — `targets` (space-joined), `variant` (`common`/`cuda`/`rocm`, from the service-name suffix) — is named contract with the binding; renaming a key breaks its fanout YAML. The cross-compile cohort (services in `x-cross-compile-targets` or declaring `platforms:`) gets its own legs — per-arch builds are the QEMU-slow ones and parallelize away from native builds; a group spanning cohort membership is a loud error. Tagless services never become explicit targets (bake rejects a tagless target under `--push`) — they ride as in-invocation dependencies; a group spanning gpu variants is a loud error. |
 | `modes.py` | `resolve_auth_mode()` — validates `PUBLIC_URL` + `AUTH_MODE` from `.env`, rejecting `keycloak` over cleartext `http://`. |
 | `context_sha.py` | `compute_service_shas(repo_root, bake: BakeDocument)` — per-service `tree-<hash>` image tags over the `.dockerignore`-allowlisted git tree. The workload-tree root is derived as the single top-level directory containing every tagged service's Dockerfile (a bake graph spanning several top-level directories is a loud error); files under it that belong to no tagged service are excluded from every SHA rather than shared into all of them. |
 | `documents.py` | `BakeDocument` + `parse_bake(path)` — the typed bake-file model every consumer of bake data goes through (build, up, down, generate-score, the reference collectors). A field validator enforces at parse time that every `x-base-images` value is `path:tag` or `path@digest` (a `${…}` hole or a bare path is a validation error, killing the silent-unpinned `FROM` → `:latest` failure mode). `digest_pinned(reference)` is the shared tail test. |
-| `image_refs.py` | Image-reference collection, split by source. `declared_references(root)` enumerates bake `x-base-images` declarations from `workloads/images.yml` and the legacy `compose*.bake.yml` names (parsed and validated via `BakeDocument`) — the single source consumer mirroring consumes. `stray_references(root, …globs)` scans Dockerfile `FROM`, Dockerfile `COPY --from=` (slash-bearing captures only, so build-stage names are skipped), and yaml `image:` lines — hand-written literal refs that bypass the declaration system — across the `workloads/` and `docker/` Dockerfile trees and the `stack/score/` + `score/` source trees; `unpinned_references(root)` polices exactly those strays (every slash-bearing reference must carry a tag, a digest, or a trailing `${build-arg}` suffix — consumer CIs fail on its result). `bake_base_image_refs(bake: BakeDocument)` adapts declarations to `ImageReference` pairs; `version_coupling_violations(root, couplings)` (each `VersionCoupling` names a pyproject-declared version and the `VersionSite` regexes that must restate it — bake `x-base-images` values or raw file text — so non-derivable couplings like uv `required-version` ↔ uv-base-image tag can't drift silently; the mechanism lives here, the registry in consumer repos' preflight) is unchanged. Remote digest resolution goes through `imagetools inspect`. Compose files are parsed by nothing in this package. |
-| `mirror.py` | The mirror namespace's config, inversion, and verbs. `MirrorConfig` (model in `config.py`, the `mirror:` section, key `prefix`), `load_mirror_config()` (reads the `mirror:` section via `config.load_config`, `None` when the section or file is absent), `is_mirrored`/`upstream_ref` (prefix inversion preserving the `:tag@sha256:…` tail), `reference_exists` (imagetools existence probe honoring docker config creds), `mirror_targets` (declared refs filtered by prefix, sorted), the `mirror` verb (crane install + per-target `crane copy`), and `up`'s fallback helpers (`upstream_fallback`, `write_fallback_overlay`). Constraints in "Mirror namespace and upstream fallback". |
+| `image_refs.py` | Image-reference collection, split by source. `declared_references(root)` enumerates bake `x-base-images` declarations from `workloads/images.yml` (parsed and validated via `BakeDocument`) — the single source consumer mirroring consumes. `stray_references(root, …globs)` scans Dockerfile `FROM`, Dockerfile `COPY --from=` (slash-bearing captures only, so build-stage names are skipped), and yaml `image:` lines — hand-written literal refs that bypass the declaration system — across the `workloads/` and `docker/` Dockerfile trees and the `stack/score/` + `score/` source trees; `unpinned_references(root)` polices exactly those strays (every slash-bearing reference must carry a tag, a digest, or a trailing `${build-arg}` suffix — consumer CIs fail on its result). `bake_base_image_refs(bake: BakeDocument)` adapts declarations to `ImageReference` pairs; `version_coupling_violations(root, couplings)` (each `VersionCoupling` names a pyproject-declared version and the `VersionSite` regexes that must restate it — bake `x-base-images` values or raw file text — so non-derivable couplings like uv `required-version` ↔ uv-base-image tag can't drift silently; the mechanism lives here, the registry in consumer repos' preflight) is unchanged. Remote digest resolution goes through `imagetools inspect`. Compose files are parsed by nothing in this package. |
+| `mirror.py` | The mirror namespace's config, injection, and verbs. `MirrorConfig` (model in `config.py`, the `mirror:` section, key `prefix`), `load_mirror_config()` (reads the `mirror:` section via `config.load_config`, `None` when the section or file is absent), `mirrored_ref` (prefix injection with the double-prefix guard), `reference_exists` (imagetools existence probe honoring docker config creds), `enforce_mirrored` (`up`'s fail-closed probe over lock values), and the `mirror` verb (crane install + one `crane copy {ref} {prefix}/{ref}` per declared reference). Constraints in "Mirror namespace". |
 
 ## Lifecycle declaration contract
 
@@ -47,23 +47,22 @@ placeframe's shape lives in placeframe's `docker-devkit.yaml`, not in this packa
 ## File and env conventions the commands assume
 
 - `.env` (required) — carries `PUBLIC_URL` and `AUTH_MODE`; `modes.py` reads it and rejects `keycloak` over cleartext `http://`.
-- `workloads/images.lock` (required in every repo, checked in) — third-party and base-image pins, written by `build`: one `NAME=path:tag@sha256:…` entry per manifest declaration. `up`/`down` refuse without it; consumers satisfy it with a committed lock filling the `${…_IMAGE:?err}` holes. The legacy name `.env.lock` resolves while the manifest still carries its legacy name; the lock always sits beside its manifest as the resolved pair.
-- `workloads/images.yml` (where a local build graph exists, required) — the image manifest and buildx-bake description; its top-level `x-cross-compile-targets` list (optional) is the set of services excluded from the default target list because they need per-arch treatment, opt-in via `--targets`. The legacy root name `compose.bake.yml` resolves identically. `build` reads exactly one manifest: the first of `workloads/images.yml`, `compose.bake.yml` that exists. `buildx bake` resolves the manifest's relative paths against its invocation cwd, so `build` always invokes from the repo root.
+- `workloads/images.lock` (required in every repo, checked in) — third-party and base-image pins, written by `build`: one `NAME={prefix}/{ref}@sha256:…` entry per manifest declaration. `up`/`down` refuse without it; consumers satisfy it with a committed lock filling the `${…_IMAGE:?err}` holes. The lock sits beside its manifest as the resolved pair.
+- `workloads/images.yml` (where a local build graph exists, required) — the image manifest and buildx-bake description; its top-level `x-cross-compile-targets` list (optional) is the set of services excluded from the default target list because they need per-arch treatment, opt-in via `--targets`. `build` reads exactly one manifest: `workloads/images.yml`. `buildx bake` resolves the manifest's relative paths against its invocation cwd, so `build` always invokes from the repo root.
 - the compose layers declared in the `lifecycle:` section of `docker-devkit.yaml`; the GPU one is picked by `detect_gpu()` (or the `--gpu` override).
 - `.dockerignore` (where images are built locally) — allowlist that `context_sha.py` hashes over to compute per-service `tree-<hash>` tags. See "Docker build context" in `AGENTS-SHARED.md` for the allowlist rationale.
 
 `build` injects the `tree-<hash>` tags of locally-built images into the environment rather than writing them anywhere; `up`/`down` do the same via `compute_service_shas`. The lock stays a pure third-party pin file — a CI invariant forbids built-image digests in the committed, reviewed lock.
 
-## Mirror namespace and upstream fallback
+## Mirror namespace
 
 Third-party images are consumed through an org-owned mirror namespace (`ghcr.io/outernet-foundation/mirror/<upstream-host>/<path>…`) instead of pulling upstream registries directly: manifests at upstream get GC'd, and an upstream yank once broke external cloners — digest pins alone don't save you when the manifest is gone. Durability for strangers beats upstream egress: strangers, CI, and prod always pull the mirror. Populating it is CI's job, single writer — engineers never push to the org namespace, and error text must never teach them to (no "run this populate command" hints).
 
-The mirror is declared, not inferred: the `mirror:` section with `prefix` in the consumer's root `docker-devkit.yaml`. Absent section → all mirror behavior off. Under the section:
+Declarations are upstream-native and the mirror-prefixed form is derived: `x-base-images` values spell the upstream reference, and the `mirror:` section's `prefix` in the consumer's root `docker-devkit.yaml` is the single place the namespace is named. `mirrored_ref(reference, prefix)` is the only join — it injects the prefix and rejects an already-prefixed reference (the double-prefix guard: drift toward the namespace is a loud error, never a silent skip). Declaring `x-base-images` with no `mirror:` section is a hard `build` error; the `mirror` verb requires the section outright. Under the section:
 
-- `build` resolves lock digests for mirror-prefixed declarations against the upstream reference (prefix stripped, tag/digest tail preserved) and records it under the mirror name. Valid because `crane copy` produces byte-identical manifests — upstream and mirror digests are the same digest; a manifest re-wrap (`docker buildx imagetools`-shaped tooling) would change the top-level digest and break lock equality. This makes declare → re-lock walkable before CI has mirrored anything; lock lines are never hand-authored.
-- `up` fails closed: after lock resolution it probes each mirror-prefixed lock entry; a missing mirror image is a hard `RuntimeError`, not a warning — people ignore warnings, and warnings are errors someone was too lazy to fix. The only escape is the per-run `--allow-upstream-fallback` flag. Under the flag, missing entries become digest-preserving address substitutions (same bytes from the upstream host), printed as plain statements — the operator asked for fallback, the run reports what it did. The rewrite is authoritative on both consumption paths: an appended `--env-file` overlay (compose's later env files win) plus `os.environ` before `build`'s bake (shell env wins over env files; `run_build` re-applies the overlay after its own lock update so bake is never clobbered). CI never passes the flag — the mirror job runs before anything needs the images.
-- The `mirror` verb (crane install from a pinned release, per-target `crane copy`, count footer) uses bashrun + plain prints — it needs nothing from any runner floor.
-- Non-mirror lock entries (e.g. `placeframe/*` publishes) are not probed and keep failing naturally at pull. There is no config-level ban on the flag: an air-gap fails closed anyway (no route to upstream — the pull dies with or without it).
+- `build` resolves lock digests against the upstream reference and records each entry under the derived mirror name (`NAME={prefix}/{ref}@sha256:…`). Valid because `crane copy` produces byte-identical manifests — upstream and mirror digests are the same digest; a manifest re-wrap (`docker buildx imagetools`-shaped tooling) would change the top-level digest and break lock equality. This makes declare → re-lock walkable before CI has mirrored anything; lock lines are never hand-authored.
+- `up` fails closed: it probes every lock entry (each is mirror-prefixed by construction); a missing mirror image is a hard `RuntimeError`, not a warning — people ignore warnings, and warnings are errors someone was too lazy to fix. There is no escape flag: a missing entry means the declaring change has not been through CI's mirror job yet, and the fix is to let it run. CI ordering never wants the escape — the mirror job runs before anything needs the images.
+- The `mirror` verb (crane install from a pinned release, one `crane copy {ref} {prefix}/{ref}` per declared reference, count footer) uses bashrun + plain prints — it needs nothing from any runner floor. Every declared reference is a target; nothing is filtered by prefix, so a drifted declaration cannot silently mirror zero.
 
 The build-artifact-registry edge exists for the builds shelf: a `build --push` run distills bake's `--metadata-file` into a digest manifest (`target → {ref, digest, tags[]}` JSON) and pushes it through build-artifact-registry's `push_build` — one **shard** per matrix leg, tagged `sha-{full}-{key}` where the key is the sha256-8 of the leg's sorted targets (legs run in parallel, so a shared tag would be last-writer-wins; the pull side enumerates a SHA's shards and merges, and consumption only ever runs after every leg finished). The shelf root comes from the neutral `REGISTRY` environment and the SHA from the checkout HEAD (`git rev-parse HEAD`), stamping the certified PR head the release channels query back at; the keying law is stated in github-actions-toolkit's `AGENTS.md`. `workloads/images.lock` stays in source control (the stapler that consumed the shelf copy is gone). ORAS CLI knowledge lives only in build-artifact-registry (org law); local builds (no `--push`) never touch the shelf.
 

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: docker-devkit
-Version: 0.3.0.dev116
+Version: 0.4.0
 Summary: Docker-stack lifecycle commands (up/down/build) driven by a pyproject-declared compose assembly
 License-File: LICENSE
 License-File: NOTICE

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "docker-devkit"
-version = "0.3.0.dev116"
+version = "0.4.0"
 description = "Docker-stack lifecycle commands (up/down/build) driven by a pyproject-declared compose assembly"
 requires-python = ">=3.13"
 dependencies = [

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/release-devkit.yaml

@@ -1,6 +1,6 @@
 packages:
   docker-devkit:
     path: .
-    major_minor: "0.3"
+    major_minor: "0.4"
     registry: pypi
     identity: docker-devkit

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/src/docker_devkit/build_docker.py

@@ -14,6 +14,7 @@ import typer
 from bashrun.bash import bash, bash_output
 from build_artifact_registry.builds import push_build
 from build_artifact_registry.setup_oras import install_oras
+from .config import MirrorConfig
 from .detect_gpu import GPU_TYPES, Gpu, detect_gpu
 from .documents import BakeDocument, digest_pinned, parse_bake
 from pydantic_settings import BaseSettings
@@ -21,7 +22,7 @@ from .context_sha import compute_service_shas
 
 from .image_refs import bake_base_image_refs, resolve_remote_digest
 from .lifecycle import require_manifest, resolve_lock
-from .mirror import is_mirrored, load_mirror_config, upstream_ref
+from .mirror import load_mirror_config, mirrored_ref
 from .modes import parse_env_file
 
 
@@ -85,11 +86,10 @@ def run_build(
     lock_only: bool = False,
     gpu_only: bool = False,
     no_cache: bool = False,
-    env_overlay: dict[str, str] | None = None,
 ) -> None:
     root = Path.cwd()
     manifest = require_manifest(root)
-    lock_file = resolve_lock(root, manifest)
+    lock_file = resolve_lock(root)
     mirror_config = load_mirror_config(root)
 
     bake = parse_bake(manifest)
@@ -117,24 +117,7 @@ def run_build(
     if gpu == "auto" and not lock_only:
         gpu = detect_gpu()
 
-    # One lock entry per declared third-party image: NAME=path:tag@sha256:…
-    # Mirror-prefixed declarations resolve their digest against upstream (crane copies
-    # are byte-identical, so the digest is valid under both names) and record it under
-    # the mirror name — declaring and re-locking works before CI has mirrored anything.
-    for occurrence in bake_base_image_refs(bake):
-        if digest_pinned(occurrence.reference):
-            lock_data[occurrence.name] = occurrence.reference
-            continue
-        if (
-            not upgrade
-            and occurrence.name in lock_data
-            and lock_data[occurrence.name].startswith(occurrence.reference + "@")
-        ):
-            continue
-        inspect_target = occurrence.reference
-        if mirror_config is not None and is_mirrored(occurrence.reference, mirror_config.prefix):
-            inspect_target = upstream_ref(occurrence.reference, mirror_config.prefix)
-        lock_data[occurrence.name] = f"{occurrence.reference}@{resolve_remote_digest(inspect_target)}"
+    update_lock_entries(bake, mirror_config, lock_data, upgrade=upgrade)
 
     # Update main lock file
     lock_file.write_text(
@@ -147,8 +130,6 @@ def run_build(
 
     # Update environment with external dependency image digests
     os.environ.update(lock_data)
-    if env_overlay:
-        os.environ.update(env_overlay)
 
     # Build command arguments
     command_arguments: list[str] = []
@@ -226,6 +207,31 @@ def run_build(
         push_image_digests(shelf, sha, baked_images, targets)
 
 
+# One lock entry per declared third-party image: NAME={prefix}/{ref}@sha256:… — the
+# mirror-prefixed form is derived from the mirror prefix, the digest resolves against
+# the upstream reference (crane copies are byte-identical, so the digest is valid
+# under both names) and records under the mirror name.
+def update_lock_entries(
+    bake: BakeDocument,
+    mirror_config: MirrorConfig | None,
+    lock_data: dict[str, str],
+    *,
+    upgrade: bool,
+) -> None:
+    if not bake.base_images:
+        return
+    if mirror_config is None:
+        raise RuntimeError("x-base-images declared without a mirror section in docker-devkit.yaml")
+    for occurrence in bake_base_image_refs(bake):
+        lock_base = mirrored_ref(occurrence.reference, mirror_config.prefix)
+        if digest_pinned(occurrence.reference):
+            lock_data[occurrence.name] = lock_base
+            continue
+        if not upgrade and occurrence.name in lock_data and lock_data[occurrence.name].startswith(lock_base + "@"):
+            continue
+        lock_data[occurrence.name] = f"{lock_base}@{resolve_remote_digest(occurrence.reference)}"
+
+
 def leg_key(targets: list[str]) -> str:
     return hashlib.sha256(",".join(sorted(targets)).encode("utf-8")).hexdigest()[:8]
 

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/src/docker_devkit/down.py

@@ -31,7 +31,7 @@ def down(
     config = load_lifecycle_config(root)
     enforce_bake_declaration(root, config)
     manifest = resolve_manifest(root)
-    lock_file = resolve_lock(root, manifest)
+    lock_file = resolve_lock(root)
 
     if not ENV_FILE.exists():
         raise RuntimeError("No .env file found")

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/src/docker_devkit/image_refs.py

@@ -12,6 +12,7 @@ from bashrun.bash import bash_output
 from pydantic import RootModel
 
 from .documents import BakeDocument, parse_bake
+from .lifecycle import MANIFEST_PATH
 
 BUILD_ARG_PATTERN = re.compile(r"\$\{[A-Za-z0-9_]+\}")
 FROM_PATTERN = re.compile(r"^FROM\s+(?:--platform=\S+\s+)?(\S+)", re.MULTILINE)
@@ -60,15 +61,11 @@ def unpinned_references(root: Path) -> list[str]:
     ]
 
 
-def declared_references(
-    root: Path, bake_globs: tuple[str, ...] = ("workloads/images.yml", "compose*.bake.yml")
-) -> list[ImageReference]:
-    return [
-        reference
-        for bake_glob in bake_globs
-        for path in sorted(root.glob(bake_glob))
-        for reference in bake_base_image_refs(parse_bake(path))
-    ]
+def declared_references(root: Path) -> list[ImageReference]:
+    manifest = root / MANIFEST_PATH
+    if not manifest.exists():
+        return []
+    return bake_base_image_refs(parse_bake(manifest))
 
 
 def stray_references(

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/src/docker_devkit/lifecycle.py

@@ -8,11 +8,9 @@ from .detect_gpu import Gpu
 DEFAULT_COMPOSE_FILE = Path("compose.yml")
 GPU_PLACEHOLDER = "{gpu}"
 
-# The image manifest and its lock are a named pair living in the same directory. The
-# legacy names keep resolving so a repo can upgrade the devkit before moving its files.
-MANIFEST_CANDIDATES = (Path("workloads/images.yml"), Path("compose.bake.yml"))
-MANIFEST_LOCK_PAIR = {"images.yml": "images.lock", "compose.bake.yml": ".env.lock"}
-LOCK_CANDIDATES = (Path("workloads/images.lock"), Path(".env.lock"))
+# The image manifest and its lock are a named pair living in the same directory.
+MANIFEST_PATH = Path("workloads/images.yml")
+LOCK_PATH = Path("workloads/images.lock")
 
 
 def load_lifecycle_config(root: Path) -> LifecycleConfig | None:
@@ -21,27 +19,25 @@ def load_lifecycle_config(root: Path) -> LifecycleConfig | None:
 
 
 def resolve_manifest(root: Path) -> Path | None:
-    return next((root / candidate for candidate in MANIFEST_CANDIDATES if (root / candidate).exists()), None)
+    manifest = root / MANIFEST_PATH
+    return manifest if manifest.exists() else None
 
 
 def require_manifest(root: Path) -> Path:
     manifest = resolve_manifest(root)
     if manifest is None:
-        raise RuntimeError(f"No image manifest found; {MANIFEST_CANDIDATES[0]} is the canonical name")
+        raise RuntimeError(f"No image manifest at {MANIFEST_PATH}")
     return manifest
 
 
-def resolve_lock(root: Path, manifest: Path | None) -> Path:
-    if manifest is not None:
-        return manifest.parent / MANIFEST_LOCK_PAIR[manifest.name]
-    existing = next((root / candidate for candidate in LOCK_CANDIDATES if (root / candidate).exists()), None)
-    return existing if existing is not None else root / LOCK_CANDIDATES[0]
+def resolve_lock(root: Path) -> Path:
+    return root / LOCK_PATH
 
 
 def enforce_bake_declaration(root: Path, config: LifecycleConfig | None) -> None:
-    if config is None and any((root / candidate).exists() for candidate in MANIFEST_CANDIDATES):
+    if config is None and (root / MANIFEST_PATH).exists():
         raise RuntimeError(
-            "image manifest present without a lifecycle section in docker-devkit.yaml; declare the section or rename the file"
+            "image manifest present without a lifecycle section in docker-devkit.yaml; declare the section"
         )
 
 

docker_devkit-0.4.0/src/docker_devkit/mirror.py

@@ -0,0 +1,72 @@
+from __future__ import annotations
+
+import os
+import shlex
+from pathlib import Path
+
+import typer
+from bashrun.bash import bash, bash_check
+
+from .config import MirrorConfig, load_config
+from .image_refs import declared_references
+
+app = typer.Typer(add_completion=False, pretty_exceptions_show_locals=False)
+
+CRANE_VERSION = "v0.22.1"
+
+
+@app.command()
+def mirror() -> None:
+    root = Path.cwd()
+    config = load_mirror_config(root)
+    if config is None:
+        raise RuntimeError("a mirror section with 'prefix' is required in docker-devkit.yaml to mirror")
+
+    targets = sorted(declared_references(root), key=lambda occurrence: occurrence.reference)
+    print(f"Install crane {CRANE_VERSION}")
+    sudo = "sudo " if os.geteuid() != 0 else ""
+    archive = "go-containerregistry_Linux_x86_64.tar.gz"
+    bash(f"curl -fsSLO https://github.com/google/go-containerregistry/releases/download/{CRANE_VERSION}/{archive}")
+    bash(f"{sudo}tar -xzf {archive} -C /usr/local/bin crane")
+    Path(archive).unlink()
+    for target in targets:
+        destination = mirrored_ref(target.reference, config.prefix)
+        print(f"Mirror {target.reference} -> {destination}")
+        bash(f"crane copy {target.reference} {destination}")
+    print(f"Mirrored images: {len(targets)}")
+
+
+def load_mirror_config(root: Path) -> MirrorConfig | None:
+    config = load_config(root)
+    return config.mirror if config is not None else None
+
+
+def mirrored_ref(reference: str, prefix: str) -> str:
+    if reference.startswith(f"{prefix}/"):
+        raise RuntimeError(
+            f"x-base-images values are upstream-native; {reference} already carries the mirror prefix {prefix}"
+        )
+    return f"{prefix}/{reference}"
+
+
+def reference_exists(reference: str) -> bool:
+    return bash_check(f"docker buildx imagetools inspect {shlex.quote(reference)}")
+
+
+def enforce_mirrored(lock_data: dict[str, str]) -> None:
+    for value in sorted(lock_data.values()):
+        if reference_exists(value):
+            continue
+        reference, _, digest = value.rpartition("@")
+        raise RuntimeError(
+            f"Image {reference or value} (digest {digest or 'unknown'}) is not mirrored yet; "
+            "the CI mirror job populates it when the declaring change runs"
+        )
+
+
+def main() -> None:
+    app()
+
+
+if __name__ == "__main__":
+    main()

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/src/docker_devkit/up.py

@@ -16,7 +16,7 @@ from .lifecycle import (
     resolve_lock,
     resolve_manifest,
 )
-from .mirror import load_mirror_config, upstream_fallback, write_fallback_overlay
+from .mirror import enforce_mirrored, load_mirror_config
 from .modes import parse_env_file, resolve_auth_mode
 
 ENV_FILE = Path(".env")
@@ -42,19 +42,12 @@ def up(
         "--dev",
         help="Layer the declared dev overlay (compose.dev.yml shape) over the production stack for bind-mount/debug bring-up.",
     ),
-    allow_upstream_fallback: Annotated[
-        bool,
-        typer.Option(
-            "--allow-upstream-fallback",
-            help="Pull unmirrored lock entries from upstream at the same digest instead of failing closed.",
-        ),
-    ] = False,
 ) -> None:
     root = Path.cwd()
     config = load_lifecycle_config(root)
     enforce_bake_declaration(root, config)
     manifest = resolve_manifest(root)
-    lock_file = resolve_lock(root, manifest)
+    lock_file = resolve_lock(root)
 
     if dev and (config is None or config.dev_file is None):
         raise RuntimeError("--dev requires a dev_file in the lifecycle section of docker-devkit.yaml")
@@ -75,21 +68,12 @@ def up(
 
     auth_mode = resolve_auth_mode(ENV_FILE)
 
-    substitutions: dict[str, str] = {}
-    fallback_overlay: Path | None = None
     mirror_config = load_mirror_config(root)
     if mirror_config is not None:
-        substitutions = upstream_fallback(
-            parse_env_file(lock_file), mirror_config.prefix, allow_upstream_fallback=allow_upstream_fallback
-        )
-        if substitutions:
-            for name, upstream in sorted(substitutions.items()):
-                print(f"Falling back to upstream for {name}: {upstream}", flush=True)
-            os.environ.update(substitutions)
-            fallback_overlay = write_fallback_overlay(substitutions)
+        enforce_mirrored(parse_env_file(lock_file))
 
     if build:
-        run_build(gpu=gpu, env_overlay=substitutions)
+        run_build(gpu=gpu)
 
     if manifest is not None:
         os.environ.update(compute_service_shas(root, parse_bake(manifest)))
@@ -101,8 +85,6 @@ def up(
     files_args = " ".join(f"-f {compose_file}" for compose_file in compose_files)
     profile_flag = "--profile keycloak " if auth_mode == "keycloak" else ""
     compose_args = f"{files_args} {profile_flag}--project-directory {root} --env-file .env --env-file {lock_file}"
-    if fallback_overlay is not None:
-        compose_args += f" --env-file {fallback_overlay}"
 
     up_command = f"docker compose {compose_args} up"
     if not build:

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/tests/test_image_refs.py

@@ -24,14 +24,14 @@ VERSION_COUPLINGS = [
     VersionCoupling(
         name="uv",
         pyproject_key="tool.uv.required-version",
-        sites=(VersionSite("uv base tag", "compose*.bake.yml", r"uv:([^@]+?)-", "UV_BASE_IMAGE"),),
+        sites=(VersionSite("uv base tag", "workloads/images.yml", r"uv:([^@]+?)-", "UV_BASE_IMAGE"),),
     ),
     VersionCoupling(
         name="python",
         pyproject_key="project.requires-python",
         sites=(
-            VersionSite("uv base python component", "compose*.bake.yml", r"python([0-9][0-9.]*)", "UV_BASE_IMAGE"),
-            VersionSite("python base tag", "compose.zed.bake.yml", r"python:([0-9][0-9.]*)", "PYTHON_BASE_IMAGE"),
+            VersionSite("uv base python component", "workloads/images.yml", r"python([0-9][0-9.]*)", "UV_BASE_IMAGE"),
+            VersionSite("python base tag", "workloads/images.yml", r"python:([0-9][0-9.]*)", "PYTHON_BASE_IMAGE"),
             VersionSite("uv python install", "docker/zed-capture/Dockerfile", r"uv python install ([0-9][0-9.]*)"),
         ),
     ),
@@ -66,10 +66,6 @@ class TestDeclaredReferences:
         )
         assert declared_references(tmp_path) == [ImageReference("ALPINE_IMAGE", "alpine:3.20")]
 
-    def test_should_yield_legacy_bake_declarations(self, tmp_path: Path):
-        (tmp_path / "compose.bake.yml").write_text("x-base-images:\n  ALPINE_IMAGE: alpine:3.20\n", encoding="utf-8")
-        assert declared_references(tmp_path) == [ImageReference("ALPINE_IMAGE", "alpine:3.20")]
-
     def test_should_return_empty_for_empty_tree(self, tmp_path: Path):
         assert declared_references(tmp_path) == []
 
@@ -83,7 +79,8 @@ class TestDeclaredReferences:
         assert declared_references(tmp_path) == []
 
     def test_should_reject_unpinned_declaration(self, tmp_path: Path):
-        (tmp_path / "compose.bake.yml").write_text("x-base-images:\n  ALPINE_IMAGE: alpine\n", encoding="utf-8")
+        (tmp_path / "workloads").mkdir()
+        (tmp_path / "workloads" / "images.yml").write_text("x-base-images:\n  ALPINE_IMAGE: alpine\n", encoding="utf-8")
         with pytest.raises(ValidationError):
             declared_references(tmp_path)
 
@@ -179,14 +176,11 @@ class TestVersionCouplingViolations:
         (tmp_path / "pyproject.toml").write_text(
             '[project]\nrequires-python = ">=3.13"\n\n[tool.uv]\nrequired-version = "==0.12.15"\n', encoding="utf-8"
         )
-        (tmp_path / "compose.bake.yml").write_text(
-            f"x-base-images:\n  UV_BASE_IMAGE: ghcr.io/outernet-foundation/mirror/ghcr.io/astral-sh/uv:{uv_tag}\n",
-            encoding="utf-8",
-        )
-        (tmp_path / "compose.zed.bake.yml").write_text(
+        (tmp_path / "workloads").mkdir()
+        (tmp_path / "workloads" / "images.yml").write_text(
             "x-base-images:\n"
-            f"  UV_BASE_IMAGE: ghcr.io/outernet-foundation/mirror/ghcr.io/astral-sh/uv:{uv_tag}\n"
-            f"  PYTHON_BASE_IMAGE: ghcr.io/outernet-foundation/mirror/docker.io/library/python:{python_tag}\n",
+            f"  UV_BASE_IMAGE: ghcr.io/astral-sh/uv:{uv_tag}\n"
+            f"  PYTHON_BASE_IMAGE: docker.io/library/python:{python_tag}\n",
             encoding="utf-8",
         )
         docker_directory = tmp_path / "docker" / "zed-capture"
@@ -200,40 +194,36 @@ class TestVersionCouplingViolations:
     def test_should_flag_drifted_uv_base_tag(self, tmp_path: Path):
         self._write_version_tree(tmp_path, uv_tag="0.12.14-python3.13-trixie-slim")
         assert version_coupling_violations(tmp_path, VERSION_COUPLINGS) == [
-            f"uv: uv base tag at {tmp_path / 'compose.bake.yml'}: expected 0.12.15, found 0.12.14",
-            f"uv: uv base tag at {tmp_path / 'compose.zed.bake.yml'}: expected 0.12.15, found 0.12.14",
+            f"uv: uv base tag at {tmp_path / 'workloads' / 'images.yml'}: expected 0.12.15, found 0.12.14",
         ]
 
     def test_should_flag_unversioned_uv_base_tag(self, tmp_path: Path):
         self._write_version_tree(tmp_path, uv_tag="python3.13-trixie-slim")
         assert version_coupling_violations(tmp_path, VERSION_COUPLINGS) == [
-            f"uv: uv base tag at {tmp_path / 'compose.bake.yml'}: expected 0.12.15, found python3.13",
-            f"uv: uv base tag at {tmp_path / 'compose.zed.bake.yml'}: expected 0.12.15, found python3.13",
+            f"uv: uv base tag at {tmp_path / 'workloads' / 'images.yml'}: expected 0.12.15, found python3.13",
         ]
 
     def test_should_flag_uv_base_without_python_component(self, tmp_path: Path):
         self._write_version_tree(tmp_path, uv_tag="0.12.15-trixie-slim")
         assert version_coupling_violations(tmp_path, VERSION_COUPLINGS) == [
-            f"python: uv base python component at {tmp_path / 'compose.bake.yml'}: expected 3.13, found <missing>",
-            f"python: uv base python component at {tmp_path / 'compose.zed.bake.yml'}: expected 3.13, found <missing>",
+            f"python: uv base python component at {tmp_path / 'workloads' / 'images.yml'}: expected 3.13, found <missing>",
         ]
 
     def test_should_flag_python_component_drift(self, tmp_path: Path):
         self._write_version_tree(tmp_path, python_tag="3.14-slim@sha256:" + "c" * 64, uv_python_install="3.14")
         assert version_coupling_violations(tmp_path, VERSION_COUPLINGS) == [
-            f"python: python base tag at {tmp_path / 'compose.zed.bake.yml'}: expected 3.13, found 3.14",
+            f"python: python base tag at {tmp_path / 'workloads' / 'images.yml'}: expected 3.13, found 3.14",
             f"python: uv python install at {tmp_path / 'docker' / 'zed-capture' / 'Dockerfile'}: expected 3.13, found 3.14",
         ]
 
     def test_should_flag_missing_base_image_key(self, tmp_path: Path):
         self._write_version_tree(tmp_path)
-        (tmp_path / "compose.zed.bake.yml").write_text(
-            "x-base-images:\n"
-            "  UV_BASE_IMAGE: ghcr.io/outernet-foundation/mirror/ghcr.io/astral-sh/uv:0.12.15-python3.13-trixie-slim\n",
+        (tmp_path / "workloads" / "images.yml").write_text(
+            "x-base-images:\n  UV_BASE_IMAGE: ghcr.io/astral-sh/uv:0.12.15-python3.13-trixie-slim\n",
             encoding="utf-8",
         )
         assert version_coupling_violations(tmp_path, VERSION_COUPLINGS) == [
-            f"python: python base tag at {tmp_path / 'compose.zed.bake.yml'}: expected 3.13, found <missing>"
+            f"python: python base tag at {tmp_path / 'workloads' / 'images.yml'}: expected 3.13, found <missing>"
         ]
 
     def test_should_flag_site_whose_files_are_absent(self, tmp_path: Path):

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/tests/test_lifecycle.py

@@ -114,40 +114,19 @@ class TestExpandComposeFiles:
 
 
 class TestResolveManifest:
-    def test_prefers_images_manifest(self, tmp_path: Path):
+    def test_resolves_images_manifest(self, tmp_path: Path):
         (tmp_path / "workloads").mkdir()
         (tmp_path / "workloads" / "images.yml").write_text("", encoding="utf-8")
-        (tmp_path / "compose.bake.yml").write_text("", encoding="utf-8")
 
         assert resolve_manifest(tmp_path) == tmp_path / "workloads" / "images.yml"
 
-    def test_falls_back_to_legacy_bake_file(self, tmp_path: Path):
-        (tmp_path / "compose.bake.yml").write_text("", encoding="utf-8")
-
-        assert resolve_manifest(tmp_path) == tmp_path / "compose.bake.yml"
-
     def test_returns_none_without_manifest(self, tmp_path: Path):
         assert resolve_manifest(tmp_path) is None
 
 
 class TestResolveLock:
-    def test_images_manifest_pairs_with_images_lock(self, tmp_path: Path):
-        manifest = tmp_path / "workloads" / "images.yml"
-
-        assert resolve_lock(tmp_path, manifest) == tmp_path / "workloads" / "images.lock"
-
-    def test_legacy_manifest_pairs_with_legacy_lock(self, tmp_path: Path):
-        manifest = tmp_path / "compose.bake.yml"
-
-        assert resolve_lock(tmp_path, manifest) == tmp_path / ".env.lock"
-
-    def test_without_manifest_prefers_existing_lock(self, tmp_path: Path):
-        (tmp_path / ".env.lock").write_text("", encoding="utf-8")
-
-        assert resolve_lock(tmp_path, None) == tmp_path / ".env.lock"
-
-    def test_without_manifest_defaults_to_images_lock(self, tmp_path: Path):
-        assert resolve_lock(tmp_path, None) == tmp_path / "workloads" / "images.lock"
+    def test_pairs_with_the_manifest(self, tmp_path: Path):
+        assert resolve_lock(tmp_path) == tmp_path / "workloads" / "images.lock"
 
 
 class TestEnforceBakeDeclaration:
@@ -157,11 +136,6 @@ class TestEnforceBakeDeclaration:
 
         enforce_bake_declaration(tmp_path, LifecycleConfig())
 
-    def test_accepts_bake_with_declaration(self, tmp_path: Path):
-        (tmp_path / "compose.bake.yml").write_text("", encoding="utf-8")
-
-        enforce_bake_declaration(tmp_path, LifecycleConfig())
-
     def test_accepts_missing_manifest_without_declaration(self, tmp_path: Path):
         enforce_bake_declaration(tmp_path, None)
 
@@ -171,9 +145,3 @@ class TestEnforceBakeDeclaration:
 
         with pytest.raises(RuntimeError, match="lifecycle"):
             enforce_bake_declaration(tmp_path, None)
-
-    def test_rejects_bake_without_declaration(self, tmp_path: Path):
-        (tmp_path / "compose.bake.yml").write_text("", encoding="utf-8")
-
-        with pytest.raises(RuntimeError, match="lifecycle"):
-            enforce_bake_declaration(tmp_path, None)

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/tests/test_mirror.py

@@ -1,5 +1,4 @@
 import os
-import tempfile
 from pathlib import Path
 
 import pytest
@@ -9,22 +8,14 @@ from typer.testing import CliRunner
 
 from docker_devkit import build_docker, mirror
 from docker_devkit.documents import BakeDocument
-from docker_devkit.image_refs import ImageReference
 from docker_devkit.mirror import MirrorConfig
 
 MIRROR_PREFIX = "ghcr.io/outernet-foundation/mirror"
 FAKE_DIGEST = "sha256:" + "a" * 64
 
-MIRROR_REFERENCES = [
-    f"{MIRROR_PREFIX}/docker.io/library/python:3.13-slim@{FAKE_DIGEST}",
-    f"{MIRROR_PREFIX}/ghcr.io/astral-sh/uv:0.12.15-python3.13-trixie-slim",
-    f"{MIRROR_PREFIX}/quay.io/keycloak/keycloak:26.3.5@{FAKE_DIGEST}",
-    f"{MIRROR_PREFIX}/docker.io/livekit/livekit-server:v1.12.0@{FAKE_DIGEST}",
-]
-
 MANIFEST = (
     "x-base-images:\n"
-    f"  UV_IMAGE: {MIRROR_PREFIX}/ghcr.io/astral-sh/uv:0.12.15\n"
+    "  UV_IMAGE: ghcr.io/astral-sh/uv:0.12.15\n"
     "  API_IMAGE: ghcr.io/outernet-foundation/placeframe/api:1.2.3\n"
     "services:\n"
     "  api:\n"
@@ -35,9 +26,8 @@ MANIFEST = (
 )
 
 LOCKED_ERROR = (
-    f"Image {MIRROR_PREFIX}/docker.io/library/alpine:3.20 (digest {FAKE_DIGEST}) is not mirrored yet. "
-    "If this is a new upstream dependency that has not been through CI, pass --allow-upstream-fallback "
-    "to pull the same digest from upstream for this run."
+    f"Image {MIRROR_PREFIX}/docker.io/library/alpine:3.20 (digest {FAKE_DIGEST}) is not mirrored yet; "
+    "the CI mirror job populates it when the declaring change runs"
 )
 
 runner = CliRunner()
@@ -108,10 +98,6 @@ def _bash_check_false(command: str) -> bool:
     return False
 
 
-def _must_not_be_probed(reference: str) -> bool:
-    raise AssertionError(f"unexpected existence probe: {reference}")
-
-
 def _no_service_shas(root: Path, bake: BakeDocument) -> dict[str, str]:
     return {}
 
@@ -124,29 +110,6 @@ def _fixed_digest(reference: str) -> str:
     return FAKE_DIGEST
 
 
-class TestInversion:
-    @pytest.mark.parametrize("reference", MIRROR_REFERENCES)
-    def test_round_trips_through_the_prefix(self, reference: str):
-        assert mirror.is_mirrored(reference, MIRROR_PREFIX)
-        assert f"{MIRROR_PREFIX}/{mirror.upstream_ref(reference, MIRROR_PREFIX)}" == reference
-
-    @pytest.mark.parametrize("reference", MIRROR_REFERENCES)
-    def test_preserves_tag_and_digest_tail(self, reference: str):
-        assert mirror.upstream_ref(reference, MIRROR_PREFIX).rpartition("/")[2] == reference.rpartition("/")[2]
-
-    def test_dockerio_path_maps_to_fully_qualified_upstream(self):
-        reference = f"{MIRROR_PREFIX}/docker.io/livekit/livekit-server:v1.12.0@{FAKE_DIGEST}"
-        assert (
-            mirror.upstream_ref(reference, MIRROR_PREFIX) == f"docker.io/livekit/livekit-server:v1.12.0@{FAKE_DIGEST}"
-        )
-
-    def test_prefix_boundary_requires_slash(self):
-        assert not mirror.is_mirrored(f"{MIRROR_PREFIX}-evil/docker.io/library/alpine:3.20", MIRROR_PREFIX)
-
-    def test_publish_entries_are_not_mirrored(self):
-        assert not mirror.is_mirrored("ghcr.io/outernet-foundation/placeframe/api:1.2.3", MIRROR_PREFIX)
-
-
 class TestLoadMirrorConfig:
     def test_loads_declared_prefix(self, tmp_path: Path):
         _write_config(tmp_path, {"mirror": {"prefix": MIRROR_PREFIX}})
@@ -174,14 +137,15 @@ class TestLoadMirrorConfig:
             mirror.load_mirror_config(tmp_path)
 
 
-class TestMirrorTargets:
-    def test_filters_declared_references_by_prefix_and_sorts(self, tmp_path: Path):
-        (tmp_path / "workloads").mkdir()
-        (tmp_path / "workloads" / "images.yml").write_text(MANIFEST, encoding="utf-8")
+class TestMirroredRef:
+    def test_prefixes_upstream_native_references(self):
+        assert mirror.mirrored_ref("ghcr.io/astral-sh/uv:0.12.15", MIRROR_PREFIX) == (
+            f"{MIRROR_PREFIX}/ghcr.io/astral-sh/uv:0.12.15"
+        )
 
-        assert mirror.mirror_targets(tmp_path, MIRROR_PREFIX) == [
-            ImageReference("UV_IMAGE", f"{MIRROR_PREFIX}/ghcr.io/astral-sh/uv:0.12.15")
-        ]
+    def test_rejects_already_prefixed_references(self):
+        with pytest.raises(RuntimeError, match="upstream-native"):
+            mirror.mirrored_ref(f"{MIRROR_PREFIX}/ghcr.io/astral-sh/uv:0.12.15", MIRROR_PREFIX)
 
 
 class TestReferenceExists:
@@ -198,59 +162,33 @@ class TestReferenceExists:
         assert not mirror.reference_exists("docker.io/library/alpine:3.20")
 
 
-class TestUpstreamFallback:
+class TestEnforceMirrored:
     def test_fails_closed_with_the_locked_message(self, monkeypatch: pytest.MonkeyPatch):
         monkeypatch.setattr(mirror, "reference_exists", _bash_check_false)
         lock_data = {"ALPINE_IMAGE": f"{MIRROR_PREFIX}/docker.io/library/alpine:3.20@{FAKE_DIGEST}"}
 
         with pytest.raises(RuntimeError) as excinfo:
-            mirror.upstream_fallback(lock_data, MIRROR_PREFIX, allow_upstream_fallback=False)
+            mirror.enforce_mirrored(lock_data)
 
         assert str(excinfo.value) == LOCKED_ERROR
 
-    def test_flag_substitutes_exactly_the_missing_entries(self, monkeypatch: pytest.MonkeyPatch):
-        present = f"{MIRROR_PREFIX}/ghcr.io/astral-sh/uv:0.12.15"
-        absent = f"{MIRROR_PREFIX}/docker.io/library/alpine:3.20@{FAKE_DIGEST}"
-        existence = SelectiveExistence(present)
-        monkeypatch.setattr(mirror, "reference_exists", existence.exists)
-        lock_data = {
-            "UV_IMAGE": present,
-            "ALPINE_IMAGE": absent,
-            "API_IMAGE": "ghcr.io/outernet-foundation/placeframe/api:1.2.3",
-        }
-
-        substitutions = mirror.upstream_fallback(lock_data, MIRROR_PREFIX, allow_upstream_fallback=True)
-
-        assert substitutions == {"ALPINE_IMAGE": f"docker.io/library/alpine:3.20@{FAKE_DIGEST}"}
-        assert existence.probed == [present, absent]
-
-    def test_returns_empty_when_everything_is_mirrored(self, monkeypatch: pytest.MonkeyPatch):
+    def test_passes_when_every_entry_exists(self, monkeypatch: pytest.MonkeyPatch):
         monkeypatch.setattr(mirror, "reference_exists", _always_true)
         lock_data = {"ALPINE_IMAGE": f"{MIRROR_PREFIX}/docker.io/library/alpine:3.20@{FAKE_DIGEST}"}
 
-        assert mirror.upstream_fallback(lock_data, MIRROR_PREFIX, allow_upstream_fallback=False) == {}
+        mirror.enforce_mirrored(lock_data)
 
-    def test_non_mirror_entries_are_never_probed(self, monkeypatch: pytest.MonkeyPatch):
-        monkeypatch.setattr(mirror, "reference_exists", _must_not_be_probed)
-        lock_data = {"API_IMAGE": "ghcr.io/outernet-foundation/placeframe/api:1.2.3"}
-
-        assert mirror.upstream_fallback(lock_data, MIRROR_PREFIX, allow_upstream_fallback=True) == {}
-
-
-class TestWriteFallbackOverlay:
-    def test_writes_sorted_entries_under_a_pid_named_tempfile(self):
-        substitutions = {
-            "UV_IMAGE": "ghcr.io/astral-sh/uv:0.12.15",
-            "ALPINE_IMAGE": f"docker.io/library/alpine:3.20@{FAKE_DIGEST}",
-        }
+    def test_reports_the_first_missing_entry_deterministically(self, monkeypatch: pytest.MonkeyPatch):
+        present = f"{MIRROR_PREFIX}/ghcr.io/astral-sh/uv:0.12.15@{FAKE_DIGEST}"
+        absent = f"{MIRROR_PREFIX}/docker.io/library/alpine:3.20@{FAKE_DIGEST}"
+        existence = SelectiveExistence(present)
+        monkeypatch.setattr(mirror, "reference_exists", existence.exists)
+        lock_data = {"UV_IMAGE": present, "ALPINE_IMAGE": absent}
 
-        overlay = mirror.write_fallback_overlay(substitutions)
+        with pytest.raises(RuntimeError, match="alpine"):
+            mirror.enforce_mirrored(lock_data)
 
-        assert overlay.parent == Path(tempfile.gettempdir())
-        assert overlay.name == f"docker-devkit-upstream-fallback-{os.getpid()}.env"
-        assert overlay.read_text(encoding="utf-8") == (
-            f"ALPINE_IMAGE=docker.io/library/alpine:3.20@{FAKE_DIGEST}\nUV_IMAGE=ghcr.io/astral-sh/uv:0.12.15\n"
-        )
+        assert existence.probed == [absent]
 
 
 class TestMirrorVerb:
@@ -259,7 +197,7 @@ class TestMirrorVerb:
         (tmp_path / "workloads").mkdir()
         (tmp_path / "workloads" / "images.yml").write_text(MANIFEST, encoding="utf-8")
 
-    def test_copies_each_mirror_target_from_upstream(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
+    def test_copies_every_declared_reference(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
         recorder = CommandRecorder()
         monkeypatch.setattr(mirror, "bash", recorder.run)
         monkeypatch.chdir(tmp_path)
@@ -268,10 +206,13 @@ class TestMirrorVerb:
         result = runner.invoke(mirror.app, [])
 
         assert result.exit_code == 0
+        upstream_uv = "ghcr.io/astral-sh/uv:0.12.15"
+        upstream_api = "ghcr.io/outernet-foundation/placeframe/api:1.2.3"
         assert [command for command in recorder.commands if command.startswith("crane copy")] == [
-            f"crane copy ghcr.io/astral-sh/uv:0.12.15 {MIRROR_PREFIX}/ghcr.io/astral-sh/uv:0.12.15"
+            f"crane copy {upstream_uv} {MIRROR_PREFIX}/{upstream_uv}",
+            f"crane copy {upstream_api} {MIRROR_PREFIX}/{upstream_api}",
         ]
-        assert "Mirrored images: 1" in result.stdout
+        assert "Mirrored images: 2" in result.stdout
 
     def test_errors_without_mirror_table(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
         monkeypatch.chdir(tmp_path)
@@ -283,13 +224,13 @@ class TestMirrorVerb:
 
 
 class TestUpstreamAwareLockResolution:
-    def _write_tree(self, tmp_path: Path, *, mirror: bool) -> None:
+    def _write_tree(self, tmp_path: Path, *, mirror: bool, manifest: str = MANIFEST) -> None:
         if mirror:
             _write_config(tmp_path, {"mirror": {"prefix": MIRROR_PREFIX}})
         (tmp_path / "workloads").mkdir()
-        (tmp_path / "workloads" / "images.yml").write_text(MANIFEST, encoding="utf-8")
+        (tmp_path / "workloads" / "images.yml").write_text(manifest, encoding="utf-8")
 
-    def test_resolves_mirror_declarations_against_upstream(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
+    def test_records_declarations_under_the_mirror_name(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
         self._write_tree(tmp_path, mirror=True)
         monkeypatch.chdir(tmp_path)
         recorder = DigestRecorder()
@@ -303,25 +244,49 @@ class TestUpstreamAwareLockResolution:
             "ghcr.io/outernet-foundation/placeframe/api:1.2.3",
         ]
         lock = (tmp_path / "workloads" / "images.lock").read_text(encoding="utf-8")
-        assert f"API_IMAGE=ghcr.io/outernet-foundation/placeframe/api:1.2.3@{FAKE_DIGEST}" in lock
+        assert f"API_IMAGE={MIRROR_PREFIX}/ghcr.io/outernet-foundation/placeframe/api:1.2.3@{FAKE_DIGEST}" in lock
         assert f"UV_IMAGE={MIRROR_PREFIX}/ghcr.io/astral-sh/uv:0.12.15@{FAKE_DIGEST}" in lock
 
-    def test_without_table_inspects_the_declared_reference(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
+    def test_requires_a_mirror_section_for_base_images(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
         self._write_tree(tmp_path, mirror=False)
         monkeypatch.chdir(tmp_path)
+        monkeypatch.setattr(build_docker, "compute_service_shas", _no_service_shas)
+
+        with pytest.raises(RuntimeError, match="mirror section"):
+            build_docker.run_build(lock_only=True)
+
+    def test_digest_pinned_declarations_pass_through_prefixed(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
+        manifest = f"x-base-images:\n  ALPINE_IMAGE: docker.io/library/alpine@{FAKE_DIGEST}\n"
+        self._write_tree(tmp_path, mirror=True, manifest=manifest)
+        monkeypatch.chdir(tmp_path)
         recorder = DigestRecorder()
         monkeypatch.setattr(build_docker, "resolve_remote_digest", recorder.resolve)
         monkeypatch.setattr(build_docker, "compute_service_shas", _no_service_shas)
 
         build_docker.run_build(lock_only=True)
 
-        assert recorder.inspected == [
-            f"{MIRROR_PREFIX}/ghcr.io/astral-sh/uv:0.12.15",
-            "ghcr.io/outernet-foundation/placeframe/api:1.2.3",
-        ]
+        assert recorder.inspected == []
+        lock = (tmp_path / "workloads" / "images.lock").read_text(encoding="utf-8")
+        assert f"ALPINE_IMAGE={MIRROR_PREFIX}/docker.io/library/alpine@{FAKE_DIGEST}" in lock
+
+    def test_cached_entries_resolve_again_only_on_upgrade(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
+        self._write_tree(tmp_path, mirror=True)
+        (tmp_path / "workloads" / "images.lock").write_text(
+            f"API_IMAGE={MIRROR_PREFIX}/ghcr.io/outernet-foundation/placeframe/api:1.2.3@{FAKE_DIGEST}\n"
+            f"UV_IMAGE={MIRROR_PREFIX}/ghcr.io/astral-sh/uv:0.12.15@{FAKE_DIGEST}\n",
+            encoding="utf-8",
+        )
+        monkeypatch.chdir(tmp_path)
+        recorder = DigestRecorder()
+        monkeypatch.setattr(build_docker, "resolve_remote_digest", recorder.resolve)
+        monkeypatch.setattr(build_docker, "compute_service_shas", _no_service_shas)
+
+        build_docker.run_build(lock_only=True)
+
+        assert recorder.inspected == []
 
 
-class TestBakeConsumesEnvOverlay:
+class TestBakeSeesLockValues:
     def _write_tree(self, tmp_path: Path) -> None:
         _write_config(tmp_path, {"mirror": {"prefix": MIRROR_PREFIX}})
         (tmp_path / "workloads").mkdir()
@@ -335,17 +300,7 @@ class TestBakeConsumesEnvOverlay:
         monkeypatch.setattr(build_docker, "compute_service_shas", _no_service_shas)
         return probe
 
-    def test_overlay_wins_over_lock_values(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
-        self._write_tree(tmp_path)
-        monkeypatch.chdir(tmp_path)
-        probe = self._fake_bake(monkeypatch)
-        upstream = f"ghcr.io/astral-sh/uv:0.12.15@{FAKE_DIGEST}"
-
-        build_docker.run_build(gpu="none", env_overlay={"UV_IMAGE": upstream})
-
-        assert probe.uv_image == upstream
-
-    def test_without_overlay_bake_sees_mirror_values(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
+    def test_bake_sees_mirror_prefixed_lock_values(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
         self._write_tree(tmp_path)
         monkeypatch.chdir(tmp_path)
         probe = self._fake_bake(monkeypatch)

{docker_devkit-0.3.0.dev116 → docker_devkit-0.4.0}/tests/test_score_generate.py

@@ -35,20 +35,20 @@ def make_config(**overrides: str | list[str]) -> ScoreConfig:
 
 def test_expand_substitutes_uppercase_placeholders(monkeypatch: pytest.MonkeyPatch) -> None:
     monkeypatch.setenv("API_SHA", "tree-abc")
-    expanded = expand_placeholders("image: registry/app:${API_SHA}", "api.yaml", Path("compose.bake.yml"))
+    expanded = expand_placeholders("image: registry/app:${API_SHA}", "api.yaml", Path("workloads/images.yml"))
     assert expanded == "image: registry/app:tree-abc"
 
 
 def test_expand_leaves_score_resource_references(monkeypatch: pytest.MonkeyPatch) -> None:
     monkeypatch.setenv("API_SHA", "tree-abc")
     text = "host: ${resources.db.host}"
-    assert expand_placeholders(text, "api.yaml", Path("compose.bake.yml")) == text
+    assert expand_placeholders(text, "api.yaml", Path("workloads/images.yml")) == text
 
 
 def test_expand_exits_on_missing_placeholder(monkeypatch: pytest.MonkeyPatch) -> None:
     monkeypatch.delenv("API_SHA", raising=False)
     with pytest.raises(SystemExit, match="API_SHA"):
-        expand_placeholders("image: registry/app:${API_SHA}", "api.yaml", Path("compose.bake.yml"))
+        expand_placeholders("image: registry/app:${API_SHA}", "api.yaml", Path("workloads/images.yml"))
 
 
 def test_sort_documents_orders_by_kind_then_namespace(tmp_path: Path) -> None:

docker_devkit-0.3.0.dev116/src/docker_devkit/mirror.py

@@ -1,95 +0,0 @@
-from __future__ import annotations
-
-import os
-import shlex
-import tempfile
-from pathlib import Path
-
-import typer
-from bashrun.bash import bash, bash_check
-
-from .config import MirrorConfig, load_config
-from .image_refs import ImageReference, declared_references
-
-app = typer.Typer(add_completion=False, pretty_exceptions_show_locals=False)
-
-CRANE_VERSION = "v0.22.1"
-
-
-@app.command()
-def mirror() -> None:
-    root = Path.cwd()
-    config = load_mirror_config(root)
-    if config is None:
-        raise RuntimeError("a mirror section with 'prefix' is required in docker-devkit.yaml to mirror")
-
-    targets = mirror_targets(root, config.prefix)
-    print(f"Install crane {CRANE_VERSION}")
-    sudo = "sudo " if os.geteuid() != 0 else ""
-    archive = "go-containerregistry_Linux_x86_64.tar.gz"
-    bash(f"curl -fsSLO https://github.com/google/go-containerregistry/releases/download/{CRANE_VERSION}/{archive}")
-    bash(f"{sudo}tar -xzf {archive} -C /usr/local/bin crane")
-    Path(archive).unlink()
-    for target in targets:
-        upstream = upstream_ref(target.reference, config.prefix)
-        print(f"Mirror {upstream} -> {target.reference}")
-        bash(f"crane copy {upstream} {target.reference}")
-    print(f"Mirrored images: {len(targets)}")
-
-
-def load_mirror_config(root: Path) -> MirrorConfig | None:
-    config = load_config(root)
-    return config.mirror if config is not None else None
-
-
-def is_mirrored(reference: str, prefix: str) -> bool:
-    return reference.startswith(f"{prefix}/")
-
-
-def upstream_ref(reference: str, prefix: str) -> str:
-    return reference[len(prefix) + 1 :]
-
-
-def reference_exists(reference: str) -> bool:
-    return bash_check(f"docker buildx imagetools inspect {shlex.quote(reference)}")
-
-
-def mirror_targets(root: Path, prefix: str) -> list[ImageReference]:
-    return sorted(
-        (occurrence for occurrence in declared_references(root) if is_mirrored(occurrence.reference, prefix)),
-        key=lambda occurrence: occurrence.reference,
-    )
-
-
-def upstream_fallback(lock_data: dict[str, str], prefix: str, *, allow_upstream_fallback: bool) -> dict[str, str]:
-    missing = {
-        name: value for name, value in lock_data.items() if is_mirrored(value, prefix) and not reference_exists(value)
-    }
-    if not missing:
-        return {}
-    if not allow_upstream_fallback:
-        value = next(iter(missing.values()))
-        reference, _, digest = value.rpartition("@")
-        raise RuntimeError(
-            f"Image {reference or value} (digest {digest or 'unknown'}) is not mirrored yet. "
-            "If this is a new upstream dependency that has not been through CI, pass --allow-upstream-fallback "
-            "to pull the same digest from upstream for this run."
-        )
-    return {name: upstream_ref(value, prefix) for name, value in missing.items()}
-
-
-def write_fallback_overlay(substitutions: dict[str, str]) -> Path:
-    overlay = Path(tempfile.gettempdir()) / f"docker-devkit-upstream-fallback-{os.getpid()}.env"
-    overlay.write_text(
-        "\n".join(f"{name}={value}" for name, value in sorted(substitutions.items())) + "\n",
-        encoding="utf-8",
-    )
-    return overlay
-
-
-def main() -> None:
-    app()
-
-
-if __name__ == "__main__":
-    main()