docker-devkit 0.1.16__tar.gz → 0.1.18__tar.gz

docker_devkit-0.1.18/AGENTS.md

@@ -0,0 +1,74 @@
+# docker-devkit
+
+## What this is
+
+`docker-devkit` owns the Docker-stack lifecycle commands — `up`, `down`, `build`, `mirror` — plus the helpers they share (`detect_gpu`, `modes`, `context_sha`). It is a small, generic package that any repo shipping a compose graph can depend on to get the same bring-up/tear-down / cross-build flow, with the compose assembly declared per consumer in `[tool.docker-devkit.lifecycle]` so a repo that authors its own images and a repo that only OCI-includes an upstream artifact both work through the same commands.
+
+The package is `docker_devkit` (src-layout under `src/docker_devkit/`); all dependencies resolve from PyPI (`bashrun>=0.1.0`; git-source pins only in scratch branches testing unreleased changes). The repo and package renamed twice on the way here — `stack-lifecycle` → `stack-toolkit` (2026-09-20, before any publish) and `stack-toolkit` → `docker-devkit` (2026-09-21, member of the `-devkit` family). The PyPI identity is fresh: `docker-devkit` starts its own tag ledger at `0.1.0`; the terminal `stack-toolkit` distributions (≤0.1.1) are deprecation signposts pointing here, not this package's history.
+
+## Release flow
+
+Publishing rides `release.yml`, triggered by a successful CI run on a `main` push: the machinery — release-devkit's `publish-stable` (an inlined, version-pinned `uvx` step in the publish job, keeping OIDC identity local), never a project dependency (docker-devkit sits inside its own dependency graph; a project-level release-devkit edge is a resolver cycle) — computes the plan from the tag ledger and path-diff, patches the version ephemerally, and publishes to PyPI under OIDC trusted publishing (publisher bound to `release.yml`, no environment). The committed `pyproject.toml` version is permanently the `0.0.0.dev0` sentinel; the `docker-devkit-v*` tags are the version ledger (declared `major_minor` line in `publish-config.json`, patch-auto within the line). While pre-1.0, breaking changes ride the normal patch flow without a bump; from 1.0 on, API-breaking changes ship with a manually bumped `major_minor` — patch-auto assumes additive changes.
+
+## Shape
+
+Entry points (`[project.scripts]`): `up` → `up.py:app`, `down` → `down.py:app`, `build` → `build_docker.py:app`, `mirror` → `mirror.py:app`, `generate-score` → `score_generate.py:app`. All accept `--help`.
+
+| Module | Role |
+|---|---|
+| `up.py` | `docker compose up`. Flags: `--attached`/`-a`, `--quiet-pull`/`-q`, `--build`, `--gpu auto\|cuda\|rocm\|none`, `--dev`, `--allow-upstream-fallback`. |
+| `down.py` | `docker compose down`. Flags: `--volumes`/`-v` (also removes named volumes), `--gpu`. |
+| `build_docker.py` | `run_build()` + the `build` command — cross-builds all images per the resolved image manifest, writing the paired lock (one `NAME=path:tag@sha256:…` entry per `x-base-images` declaration; `@digest`-pinned declarations pass through unresolved; mirror-prefixed declarations resolve their digest against the upstream reference). `--mode ci` hard-requires `x-registry-cache` in the manifest. |
+| `score_generate.py` | `generate-score` — deterministic Score-manifest generation from a consumer's `stack/score/` sources: `${ALL_CAPS}` placeholder expansion into tempdir-rendered workload copies, idempotent `score-compose`/`score-k8s` init (state preserved across runs), and a determinism suite (document sort, state-path normalisation, 0644 modes, LF newlines). Consumer policy — workloads, provisioners, patch templates, output paths, storage classes, publishes, tool versions — is declared in the consumer's root `pyproject.toml` under `[tool.docker-devkit.generate-score]` (`ScoreConfig`); `load_score_config()` reads it, `ensure_score_tools()` fetches the pinned score-compose/score-k8s release binaries (for CI staleness gates). Output and state paths resolve against the repo root: the compose half runs in `score_dir` (its `.score-compose` state stays there, gitignored), the k8s half runs in the configured `k8s_state` parent so the committed `.score-k8s` state lands in the generated tier. `--local` writes the k8s artifact to the configured gitignored local path with the local storage class. |
+| `lifecycle.py` | `LifecycleConfig` — the `[tool.docker-devkit.lifecycle]` declaration (`files`: compose files in layering order with `{gpu}` templating; `dev_file`: optional dev overlay); `load_lifecycle_config()` (walks `tool → docker-devkit → lifecycle`, `None` when absent), `resolve_manifest()` / `require_manifest()` (the raising form) / `resolve_lock()` (the image manifest and its lock are a named pair — `workloads/images.yml` + `workloads/images.lock`, with the legacy root `compose.bake.yml` + `.env.lock` pair still resolved so repos can upgrade before moving files), `enforce_bake_declaration()` (a manifest under either name requires a declaration), `expand_compose_files()` (assembles the `-f` list; a `{gpu}` entry is dropped when the GPU resolves to `none`; `dev_file` appended when a dev overlay applies). |
+| `detect_gpu.py` | `detect_gpu()` → `cuda`/`rocm`/`none` from host devices; the `Gpu` literal. |
+| `modes.py` | `resolve_auth_mode()` — validates `PUBLIC_URL` + `AUTH_MODE` from `.env`, rejecting `keycloak` over cleartext `http://`. |
+| `context_sha.py` | `compute_service_shas(repo_root, bake: BakeDocument)` — per-service `tree-<hash>` image tags over the `.dockerignore`-allowlisted git tree. The workload-tree root is derived as the single top-level directory containing every tagged service's Dockerfile (a bake graph spanning several top-level directories is a loud error); files under it that belong to no tagged service are excluded from every SHA rather than shared into all of them. |
+| `documents.py` | `BakeDocument` + `parse_bake(path)` — the typed bake-file model every consumer of bake data goes through (build, up, down, generate-score, the reference collectors). A field validator enforces at parse time that every `x-base-images` value is `path:tag` or `path@digest` (a `${…}` hole or a bare path is a validation error, killing the silent-unpinned `FROM` → `:latest` failure mode). `digest_pinned(reference)` is the shared tail test. |
+| `image_refs.py` | Image-reference collection, split by source. `declared_references(root)` enumerates bake `x-base-images` declarations from `workloads/images.yml` and the legacy `compose*.bake.yml` names (parsed and validated via `BakeDocument`) — the single source consumer mirroring consumes. `stray_references(root, …globs)` scans Dockerfile `FROM`, Dockerfile `COPY --from=` (slash-bearing captures only, so build-stage names are skipped), and yaml `image:` lines — hand-written literal refs that bypass the declaration system — across the `workloads/` and `docker/` Dockerfile trees and the `stack/score/` + `score/` source trees; `unpinned_references(root)` polices exactly those strays (every slash-bearing reference must carry a tag, a digest, or a trailing `${build-arg}` suffix — consumer CIs fail on its result). `bake_base_image_refs(bake: BakeDocument)` adapts declarations to `ImageReference` pairs; `version_coupling_violations(root, couplings)` (each `VersionCoupling` names a pyproject-declared version and the `VersionSite` regexes that must restate it — bake `x-base-images` values or raw file text — so non-derivable couplings like uv `required-version` ↔ uv-base-image tag can't drift silently; the mechanism lives here, the registry in consumer repos' preflight) is unchanged. Remote digest resolution goes through `imagetools inspect`. Compose files are parsed by nothing in this package. |
+| `mirror.py` | The mirror namespace's config, inversion, and verbs. `MirrorConfig` (`[tool.docker-devkit.mirror]`, key `prefix`), `load_mirror_config()` (walks `tool → docker-devkit → mirror`, `None` when absent), `is_mirrored`/`upstream_ref` (prefix inversion preserving the `:tag@sha256:…` tail), `reference_exists` (imagetools existence probe honoring docker config creds), `mirror_targets` (declared refs filtered by prefix, sorted), the `mirror` verb (crane install + per-target `crane copy`), and `up`'s fallback helpers (`upstream_fallback`, `write_fallback_overlay`). Constraints in "Mirror namespace and upstream fallback". |
+
+## Constraints
+
+### Lifecycle declaration contract
+
+**Context.** A repo that authors its own stack builds its own images and assembles a multi-file compose graph (e.g. `compose.yml` + `compose.postgres.yml` + a GPU layer + `compose.dev.yml`), injecting per-service `${*_SHA}` image tags and an image lock of third-party digests. A consumer repo has none of that: it ships a single self-contained compose file that OCI-includes the upstream artifact already baked (image digests and internal vars frozen as literals) or includes a sibling checkout (which supplies its own lock through the include's `env_file`).
+
+**Constraint.** `up`/`down` assemble the graph from `[tool.docker-devkit.lifecycle]` in the consumer's root pyproject — one code path, declared data. Keys: `files` (compose files in layering order, `{gpu}` templating, a `{gpu}` entry dropped when the GPU resolves to `none`; defaults to empty), `dev_file` (dev overlay, appended by `up --dev` and always by `down` so teardown removes containers from prior dev bring-ups). No table → single-file default (`-f compose.yml`). `--compose-file` does not exist; raw `docker compose` is the pressure valve. `--dev` is opt-in — production-shaped bring-up is the default. The image lock is required unconditionally, no knob; consumers pin stock images via `${…_IMAGE:?err}` holes filled by a committed lock. An image manifest present under either resolved name without a lifecycle declaration is a hard error (declare the table, or the file doesn't belong under that name) — guards are budgeted for silent failures only, and a missing lock already fails loudly. The manifest remains the build/up contract: SHA injection and `--build` acceptance key on the manifest existing, and `build` always reads the resolved manifest. `up`/`down` pass `--project-directory` (the invocation root) explicitly, so compose files living in nested trees (`stack/generated/compose/`, overlays) anchor their relative paths at the repo root instead of the first `-f` file's directory. An empty assembly (`files` empty/omitted, no dev overlay) is a loud `up`/`down` error — the honest state for a builds-only repo.
+
+**Consequences.** placeframe's shape lives in placeframe's pyproject, not in this package: `files = ["compose.yml", "compose.postgres.yml", "compose.{gpu}.yml"]`, `dev_file = "compose.dev.yml"`. A consumer repo declares a bare table (builds images, no repo-local runtime graph) and runs `uv run up` against its own `.env` + committed lock.
+
+### File and env conventions the commands assume
+
+- `.env` (required) — carries `PUBLIC_URL` and `AUTH_MODE`; `modes.py` reads it and rejects `keycloak` over cleartext `http://`.
+- `workloads/images.lock` (required in every repo, checked in) — third-party and base-image pins, written by `build`: one `NAME=path:tag@sha256:…` entry per manifest declaration. `up`/`down` refuse without it; consumers satisfy it with a committed lock filling the `${…_IMAGE:?err}` holes. The legacy name `.env.lock` resolves while the manifest still carries its legacy name; the lock always sits beside its manifest as the resolved pair.
+- `workloads/images.yml` (where a local build graph exists, required) — the image manifest and buildx-bake description; its top-level `x-cross-compile-targets` list (optional) is the set of services excluded from the default target list because they need per-arch treatment, opt-in via `--targets`. The legacy root name `compose.bake.yml` resolves identically. `build` reads exactly one manifest: the first of `workloads/images.yml`, `compose.bake.yml` that exists. `buildx bake` resolves the manifest's relative paths against its invocation cwd, so `build` always invokes from the repo root.
+- the compose layers declared in `[tool.docker-devkit.lifecycle]` — e.g. placeframe's `compose.yml` / `compose.postgres.yml` / `compose.<gpu>.yml` / `compose.dev.yml`; the GPU one is picked by `detect_gpu()` (or the `--gpu` override).
+- `.dockerignore` (where images are built locally) — allowlist that `context_sha.py` hashes over to compute per-service `tree-<hash>` tags. See "Docker build context" in `AGENTS-SHARED.md` for the allowlist rationale.
+
+`build` injects the `tree-<hash>` tags of locally-built images into the environment rather than writing them anywhere; `up`/`down` do the same via `compute_service_shas`. The lock stays a pure third-party pin file — a CI invariant forbids built-image digests in the committed, reviewed lock.
+
+### Mirror namespace and upstream fallback
+
+**Context.** Third-party images are consumed through an org-owned mirror namespace (`ghcr.io/outernet-foundation/mirror/<upstream-host>/<path>…`) instead of pulling upstream registries directly: manifests at upstream get GC'd, and an upstream yank once broke external cloners — digest pins alone don't save you when the manifest is gone. Durability for strangers beats upstream egress: strangers, CI, and prod always pull the mirror. Populating it is CI's job, single writer — engineers never push to the org namespace, and error text must never teach them to (no "run this populate command" hints).
+
+**Constraint.** The mirror is declared, not inferred: `[tool.docker-devkit.mirror]` with `prefix` in the consumer's root pyproject. Absent table → all mirror behavior off; `up`/`build` behave exactly as they did before it existed. Under the table:
+
+- `build` resolves lock digests for mirror-prefixed declarations against the upstream reference (prefix stripped, tag/digest tail preserved) and records them under the mirror name. Valid because `crane copy` produces byte-identical manifests — upstream and mirror digests are the same digest; a manifest re-wrap (`docker buildx imagetools`-shaped tooling) would change the top-level digest and break lock equality. This makes declare → re-lock walkable before CI has mirrored anything; lock lines are never hand-authored.
+- `up` fails closed: after lock resolution it probes each mirror-prefixed lock entry; a missing mirror image is a hard `RuntimeError`, not a warning — people ignore warnings, and warnings are errors someone was too lazy to fix. The only escape is the per-run `--allow-upstream-fallback` flag. Under the flag, missing entries become digest-preserving address substitutions (same bytes from the upstream host), printed as plain statements — the operator asked for fallback, the run reports what it did. The rewrite is authoritative on both consumption paths: an appended `--env-file` overlay (compose's later env files win) plus `os.environ` before `build`'s bake (shell env wins over env files; `run_build` re-applies the overlay after its own lock update so bake is never clobbered). CI never passes the flag — the mirror job runs before anything needs the images.
+- The `mirror` verb (crane install from a pinned release, per-target `crane copy`, count footer) uses bashrun + plain prints, deliberately not ci-devkit's `ci_step`: docker-devkit and ci-devkit are siblings in the dependency DAG and the mirror verb is the only thing that would ever draw that edge. Zero new dependencies is a hard rule.
+
+**Consequences.** Non-mirror lock entries (e.g. `placeframe/*` publishes) are not probed and keep failing naturally at pull. There is no config-level ban on the flag: an air-gap fails closed anyway (no route to upstream — the pull dies with or without it).
+
+### Package split rationale
+
+Lifecycle (`up`/`down`/`build`) and codegen (client generation, datamodel generation, workspace locking) have disjoint dependency surfaces. Bundling them would force a consumer that only brings a stack up to install `sqlacodegen`, `psycopg`, `datamodel-code-generator`. Keeping the lifecycle in its own small package lets consumers git-reference just this without dragging the codegen surface behind it.
+
+### `--quiet-pull` logging and GPU auto-detection
+
+`--quiet-pull` suppresses compose's per-layer progress (CUDA images carry hundreds of layers) while still surfacing pull totals. `--gpu auto` resolves to `cuda`/`rocm`/`none` from host devices and selects the matching `compose.<gpu>.yml`; override only to reproduce a CI environment or test the CPU-only path.
+
+## See also
+
+- `README.md` — human-facing setup and usage.
+- [`bashrun`](https://github.com/outernet-foundation/bashrun) — the guardrailed subprocess wrapper the lifecycle commands shell out through.

{docker_devkit-0.1.16 → docker_devkit-0.1.18}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: docker-devkit
-Version: 0.1.16
+Version: 0.1.18
 Summary: Docker-stack lifecycle commands (up/down/build) driven by a pyproject-declared compose assembly
 License-File: LICENSE
 License-File: NOTICE

{docker_devkit-0.1.16 → docker_devkit-0.1.18}/README.md

@@ -1,8 +1,8 @@
 # docker-devkit
 
-Docker-stack lifecycle commands — `up`, `down`, `build` — with the compose assembly declared per repo in `[tool.docker-devkit.lifecycle]`. A repo that authors its own stack (Dockerfiles + `compose.bake.yml`) declares its file layers and gets multi-file assembly, per-service SHA injection, and `.env.lock` resolution; a consumer repo that OCI-includes an already-baked upstream stack declares a bare table and runs the same commands as a thin wrapper over its own `.env` + committed `.env.lock`.
+Docker-stack lifecycle commands — `up`, `down`, `build` — with the compose assembly declared per repo in `[tool.docker-devkit.lifecycle]`. A repo that authors its own stack (Dockerfiles + `workloads/images.yml`) declares its file layers and gets multi-file assembly, per-service SHA injection, and `workloads/images.lock` resolution; a consumer repo that OCI-includes an already-baked upstream stack declares a bare table and runs the same commands as a thin wrapper over its own `.env` + committed lock.
 
-See [`AGENTS.md`](./AGENTS.md) for the file/layout conventions the commands assume (`compose.bake.yml`, `.env.lock`, `.env.shas`, `PUBLIC_URL` / `AUTH_MODE` in `.env`) and the declaration contract.
+See [`AGENTS.md`](./AGENTS.md) for the file/layout conventions the commands assume (`workloads/images.yml` + `workloads/images.lock` — with the legacy root `compose.bake.yml` + `.env.lock` pair still resolved — and `PUBLIC_URL` / `AUTH_MODE` in `.env`) and the declaration contract.
 
 ## Setup
 
@@ -19,16 +19,20 @@ From a stack repo's root:
 ```bash
 uv run up                      # docker compose up (auto-detects GPU; assembles the declared lifecycle files)
 uv run up --dev                # layer the declared dev overlay (bind-mount/debug bring-up)
-uv run up --build              # build images locally first (requires compose.bake.yml)
+uv run up --build              # build images locally first (requires workloads/images.yml)
 uv run up --gpu none           # override GPU auto-detection
+uv run up --allow-upstream-fallback  # per-run escape: pull unmirrored lock entries from upstream at the same digest
 uv run down                    # docker compose down
 uv run down -v                 # also remove named volumes
-uv run build                   # cross-build all images per compose.bake.yml
-uv run build --lock-only       # refresh .env.lock without building
+uv run build                   # cross-build all images per workloads/images.yml
+uv run build --lock-only       # refresh the image lock without building
+uv run mirror                  # populate the org mirror namespace from mirror-prefixed declarations (CI)
 uv run generate-score          # regenerate Score compose/k8s manifests per [tool.docker-devkit.generate-score]
 uv run generate-score --local  # local-cluster variant (local storage class, gitignored output)
 ```
 
+Repos that consume third-party images through the org mirror namespace declare `[tool.docker-devkit.mirror]` (`prefix = "ghcr.io/outernet-foundation/mirror"`); `up` then fails closed on any lock entry not mirrored yet, and `build` resolves lock digests against upstream. See the "Mirror namespace and upstream fallback" section in [`AGENTS.md`](./AGENTS.md).
+
 ## Consuming from another repo
 
 Install from PyPI and use its entry points:

{docker_devkit-0.1.16 → docker_devkit-0.1.18}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "docker-devkit"
-version = "0.1.16"
+version = "0.1.18"
 description = "Docker-stack lifecycle commands (up/down/build) driven by a pyproject-declared compose assembly"
 requires-python = ">=3.13"
 dependencies = [
@@ -28,6 +28,7 @@ build-backend = "hatchling.build"
 up = "docker_devkit.up:app"
 down = "docker_devkit.down:app"
 build = "docker_devkit.build_docker:app"
+mirror = "docker_devkit.mirror:app"
 generate-score = "docker_devkit.score_generate:app"
 
 [tool.hatch.build.targets.wheel]

{docker_devkit-0.1.16 → docker_devkit-0.1.18}/src/docker_devkit/build_docker.py

@@ -13,9 +13,11 @@ from bashrun.bash import bash, bash_output
 from .detect_gpu import GPU_TYPES, Gpu, detect_gpu
 from .documents import BakeDocument, digest_pinned, parse_bake
 from pydantic_settings import BaseSettings
-
 from .context_sha import compute_service_shas
+
 from .image_refs import bake_base_image_refs, resolve_remote_digest
+from .lifecycle import require_manifest, resolve_lock
+from .mirror import is_mirrored, load_mirror_config, upstream_ref
 from .modes import parse_env_file
 
 
@@ -25,9 +27,6 @@ class Settings(BaseSettings):
 
 settings = Settings.model_validate({})
 
-LOCK_FILE = Path(".env.lock")
-ENV_SHAS_FILE = Path(".env.shas")
-DEFAULT_BAKE_FILE = Path("compose.bake.yml")
 METADATA_PATH = Path("metadata.json")
 
 Mode = Literal["local", "ci"]
@@ -50,7 +49,7 @@ def build(
     no_cache: bool = typer.Option(False, "--no-cache", help="Force rebuild by disabling cache usage."),
     targets_opt: Annotated[
         list[str] | None,
-        typer.Option("--targets", "-t", help="Build only these services (from compose.bake.yml)."),
+        typer.Option("--targets", "-t", help="Build only these services (from the image manifest)."),
     ] = None,
 ) -> None:
     run_build(
@@ -73,19 +72,20 @@ def run_build(
     gpu_only: bool = False,
     no_cache: bool = False,
     targets_opt: list[str] | None = None,
+    env_overlay: dict[str, str] | None = None,
 ) -> None:
-    bake = parse_bake(DEFAULT_BAKE_FILE)
+    root = Path.cwd()
+    manifest = require_manifest(root)
+    lock_file = resolve_lock(root, manifest)
+    mirror_config = load_mirror_config(root)
 
-    service_shas = compute_service_shas(Path.cwd(), bake)
-    os.environ.update(service_shas)
+    bake = parse_bake(manifest)
 
-    # Write .env.shas with the locally-built image tags
-    ENV_SHAS_FILE.write_text(
-        "".join(f"{key}={value}\n" for key, value in sorted(service_shas.items())), encoding="utf-8"
-    )
+    service_shas = compute_service_shas(root, bake)
+    os.environ.update(service_shas)
 
     # Load any existing lock
-    lock_data = parse_env_file(LOCK_FILE) if LOCK_FILE.exists() else {}
+    lock_data = parse_env_file(lock_file) if lock_file.exists() else {}
 
     # TOOD: Create separate commands for ci and local modes so typer can do this validation instead of us
     if mode == "ci" and gpu == "auto":
@@ -106,6 +106,9 @@ def run_build(
         gpu = detect_gpu()
 
     # One lock entry per declared third-party image: NAME=path:tag@sha256:…
+    # Mirror-prefixed declarations resolve their digest against upstream (crane copies
+    # are byte-identical, so the digest is valid under both names) and record it under
+    # the mirror name — declaring and re-locking works before CI has mirrored anything.
     for occurrence in bake_base_image_refs(bake):
         if digest_pinned(occurrence.reference):
             lock_data[occurrence.name] = occurrence.reference
@@ -116,10 +119,13 @@ def run_build(
             and lock_data[occurrence.name].startswith(occurrence.reference + "@")
         ):
             continue
-        lock_data[occurrence.name] = f"{occurrence.reference}@{resolve_remote_digest(occurrence.reference)}"
+        inspect_target = occurrence.reference
+        if mirror_config is not None and is_mirrored(occurrence.reference, mirror_config.prefix):
+            inspect_target = upstream_ref(occurrence.reference, mirror_config.prefix)
+        lock_data[occurrence.name] = f"{occurrence.reference}@{resolve_remote_digest(inspect_target)}"
 
     # Update main lock file
-    LOCK_FILE.write_text(
+    lock_file.write_text(
         "# Generated by lock.py\n" + "\n".join(f"{key}={value}" for key, value in sorted(lock_data.items())) + "\n",
         encoding="utf-8",
     )
@@ -129,6 +135,8 @@ def run_build(
 
     # Update environment with external dependency image digests
     os.environ.update(lock_data)
+    if env_overlay:
+        os.environ.update(env_overlay)
 
     # Build command arguments
     command_arguments: list[str] = []
@@ -183,7 +191,7 @@ def run_build(
     # Bake images
     command = [
         "docker buildx bake",
-        f"-f {DEFAULT_BAKE_FILE}",
+        f"-f {manifest}",
         f"--metadata-file {METADATA_PATH}",
         "--progress auto",
         "--provenance=false",

{docker_devkit-0.1.16 → docker_devkit-0.1.18}/src/docker_devkit/context_sha.py

@@ -18,6 +18,8 @@ def compute_service_shas(repo_root: Path, bake: BakeDocument) -> dict[str, str]:
             continue
         service_dirs.add(str(PurePosixPath(config.build.dockerfile).parent))
 
+    workload_root = _workload_root(service_dirs)
+
     tree_entries = bash_output("git ls-tree -r HEAD", cwd=repo_root).splitlines()
 
     dockerignore = repo_root / ".dockerignore"
@@ -31,12 +33,12 @@ def compute_service_shas(repo_root: Path, bake: BakeDocument) -> dict[str, str]:
         mode, _type, obj_hash = meta.split()
         allowed_entries.append((mode, obj_hash, path))
 
-    docker_prefix = "docker/"
+    workload_prefix = workload_root.as_posix() + "/" if workload_root is not None else None
     shared_entries: list[tuple[str, str, str]] = []
     per_service: dict[str, list[tuple[str, str, str]]] = {d: [] for d in service_dirs}
 
     for mode, obj_hash, path in allowed_entries:
-        if path.startswith(docker_prefix):
+        if workload_prefix is not None and path.startswith(workload_prefix):
             for service_dir in service_dirs:
                 if path.startswith(service_dir + "/"):
                     per_service[service_dir].append((mode, obj_hash, path))
@@ -57,3 +59,15 @@ def compute_service_shas(repo_root: Path, bake: BakeDocument) -> dict[str, str]:
         result[PurePosixPath(service_dir).name.upper().replace("-", "_") + "_SHA"] = f"tree-{tree_hash}"
 
     return result
+
+
+def _workload_root(service_dirs: set[str]) -> PurePosixPath | None:
+    # The workload tree root is the single top-level directory containing every tagged
+    # service. Files under it that belong to no tagged service are excluded from every
+    # SHA: they are workload-tree content, not shared build context.
+    tops = {PurePosixPath(service_dir).parts[0] for service_dir in service_dirs if PurePosixPath(service_dir).parts}
+    if not tops:
+        return None
+    if len(tops) > 1:
+        raise RuntimeError(f"Bake services span multiple top-level directories: {sorted(tops)}")
+    return PurePosixPath(next(iter(tops)))

{docker_devkit-0.1.16 → docker_devkit-0.1.18}/src/docker_devkit/down.py

@@ -8,11 +8,16 @@ from .detect_gpu import Gpu, detect_gpu
 
 from .context_sha import compute_service_shas
 from .documents import parse_bake
-from .lifecycle import BAKE_FILE, enforce_bake_declaration, expand_compose_files, load_lifecycle_config
+from .lifecycle import (
+    enforce_bake_declaration,
+    expand_compose_files,
+    load_lifecycle_config,
+    resolve_lock,
+    resolve_manifest,
+)
 from .modes import resolve_auth_mode
 
 ENV_FILE = Path(".env")
-LOCK_FILE = Path(".env.lock")
 
 app = typer.Typer(add_completion=False)
 
@@ -22,22 +27,27 @@ def down(
     volumes: bool = typer.Option(False, "--volumes", "-v", help="Remove named volumes."),
     gpu: Annotated[Gpu, typer.Option("--gpu", help="auto|cuda|rocm|none")] = "auto",
 ) -> None:
-    config = load_lifecycle_config(Path.cwd())
-    enforce_bake_declaration(Path.cwd(), config)
+    root = Path.cwd()
+    config = load_lifecycle_config(root)
+    enforce_bake_declaration(root, config)
+    manifest = resolve_manifest(root)
+    lock_file = resolve_lock(root, manifest)
 
     if not ENV_FILE.exists():
         raise RuntimeError("No .env file found")
 
-    if not LOCK_FILE.exists():
-        raise RuntimeError("No .env.lock found; run 'uv run build --lock-only' in the repo that authors the images")
+    if not lock_file.exists():
+        raise RuntimeError(
+            f"No image lock at {lock_file}; run 'uv run build --lock-only' in the repo that authors the images"
+        )
 
     if gpu == "auto":
         gpu = detect_gpu()
 
     resolve_auth_mode(ENV_FILE)
 
-    if BAKE_FILE.exists():
-        os.environ.update(compute_service_shas(Path.cwd(), parse_bake(BAKE_FILE)))
+    if manifest is not None:
+        os.environ.update(compute_service_shas(root, parse_bake(manifest)))
 
     # Always layer the dev overlay so containers from a prior --dev bring-up get torn down too
     compose_files = expand_compose_files(config, gpu, include_dev=True)
@@ -48,9 +58,10 @@ def down(
     command = (
         "docker compose "
         f"{files_args} "
+        f"--project-directory {root} "
         "--profile keycloak "  # Always include so any keycloak containers from a previous AUTH_MODE=keycloak run get torn down
         "--env-file .env "
-        f"--env-file {LOCK_FILE} "
+        f"--env-file {lock_file} "
         "down --remove-orphans"
     )
 

{docker_devkit-0.1.16 → docker_devkit-0.1.18}/src/docker_devkit/image_refs.py

@@ -70,30 +70,39 @@ def unpinned_references(root: Path) -> list[str]:
     ]
 
 
-def declared_references(root: Path, bake_glob: str = "compose*.bake.yml") -> list[ImageReference]:
-    return [reference for path in sorted(root.glob(bake_glob)) for reference in bake_base_image_refs(parse_bake(path))]
+def declared_references(
+    root: Path, bake_globs: tuple[str, ...] = ("workloads/images.yml", "compose*.bake.yml")
+) -> list[ImageReference]:
+    return [
+        reference
+        for bake_glob in bake_globs
+        for path in sorted(root.glob(bake_glob))
+        for reference in bake_base_image_refs(parse_bake(path))
+    ]
 
 
 def stray_references(
     root: Path,
-    dockerfile_glob: str = "docker/*/Dockerfile*",
-    image_glob: str = "score/*.yaml",
+    dockerfile_globs: tuple[str, ...] = ("workloads/*/Dockerfile*", "docker/*/Dockerfile*"),
+    image_globs: tuple[str, ...] = ("stack/score/*.yaml", "score/*.yaml"),
 ) -> list[ImageReference]:
+    dockerfile_paths = [path for dockerfile_glob in dockerfile_globs for path in sorted(root.glob(dockerfile_glob))]
+    image_paths = [path for image_glob in image_globs for path in sorted(root.glob(image_glob))]
     return (
         [
             ImageReference("", match.group(1))
-            for path in sorted(root.glob(dockerfile_glob))
+            for path in dockerfile_paths
             for match in FROM_PATTERN.finditer(path.read_text(encoding="utf-8"))
         ]
         + [
             ImageReference("", match.group(1))
-            for path in sorted(root.glob(dockerfile_glob))
+            for path in dockerfile_paths
             for match in COPY_FROM_PATTERN.finditer(path.read_text(encoding="utf-8"))
             if "/" in match.group(1)
         ]
         + [
             ImageReference("", match.group(1))
-            for path in sorted(root.glob(image_glob))
+            for path in image_paths
             for match in IMAGE_LINE_PATTERN.finditer(path.read_text(encoding="utf-8"))
         ]
     )

docker_devkit-0.1.18/src/docker_devkit/lifecycle.py

@@ -0,0 +1,69 @@
+from __future__ import annotations
+
+import tomllib
+from pathlib import Path
+
+from pydantic import BaseModel, ConfigDict, Field
+
+from .detect_gpu import Gpu
+
+DEFAULT_COMPOSE_FILE = Path("compose.yml")
+GPU_PLACEHOLDER = "{gpu}"
+
+# The image manifest and its lock are a named pair living in the same directory. The
+# legacy names keep resolving so a repo can upgrade the devkit before moving its files.
+MANIFEST_CANDIDATES = (Path("workloads/images.yml"), Path("compose.bake.yml"))
+MANIFEST_LOCK_PAIR = {"images.yml": "images.lock", "compose.bake.yml": ".env.lock"}
+LOCK_CANDIDATES = (Path("workloads/images.lock"), Path(".env.lock"))
+
+
+class LifecycleConfig(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    files: list[str] = Field(default_factory=list)
+    dev_file: str | None = None
+
+
+def load_lifecycle_config(root: Path) -> LifecycleConfig | None:
+    pyproject_path = root / "pyproject.toml"
+    if not pyproject_path.exists():
+        return None
+    data = tomllib.loads(pyproject_path.read_text(encoding="utf-8"))
+    section = data.get("tool", {}).get("docker-devkit", {}).get("lifecycle")
+    if section is None:
+        return None
+    return LifecycleConfig.model_validate(section)
+
+
+def resolve_manifest(root: Path) -> Path | None:
+    return next((root / candidate for candidate in MANIFEST_CANDIDATES if (root / candidate).exists()), None)
+
+
+def require_manifest(root: Path) -> Path:
+    manifest = resolve_manifest(root)
+    if manifest is None:
+        raise RuntimeError(f"No image manifest found; {MANIFEST_CANDIDATES[0]} is the canonical name")
+    return manifest
+
+
+def resolve_lock(root: Path, manifest: Path | None) -> Path:
+    if manifest is not None:
+        return manifest.parent / MANIFEST_LOCK_PAIR[manifest.name]
+    existing = next((root / candidate for candidate in LOCK_CANDIDATES if (root / candidate).exists()), None)
+    return existing if existing is not None else root / LOCK_CANDIDATES[0]
+
+
+def enforce_bake_declaration(root: Path, config: LifecycleConfig | None) -> None:
+    if config is None and any((root / candidate).exists() for candidate in MANIFEST_CANDIDATES):
+        raise RuntimeError(
+            "image manifest present without [tool.docker-devkit.lifecycle]; declare the table or rename the file"
+        )
+
+
+def expand_compose_files(config: LifecycleConfig | None, gpu: Gpu, *, include_dev: bool = False) -> list[Path]:
+    if config is None:
+        return [DEFAULT_COMPOSE_FILE]
+    files = [Path(entry.format(gpu=gpu)) for entry in config.files if gpu != "none" or GPU_PLACEHOLDER not in entry]
+    if include_dev and config.dev_file is not None:
+        files.append(Path(config.dev_file))
+    return files

docker_devkit-0.1.18/src/docker_devkit/mirror.py

@@ -0,0 +1,115 @@
+from __future__ import annotations
+
+import os
+import shlex
+import tempfile
+import tomllib
+from pathlib import Path
+
+import typer
+from bashrun.bash import bash, bash_check
+from pydantic import BaseModel, ConfigDict, field_validator
+
+from .image_refs import ImageReference, declared_references
+
+app = typer.Typer(add_completion=False, pretty_exceptions_show_locals=False)
+
+CRANE_VERSION = "v0.22.1"
+
+
+class MirrorConfig(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    prefix: str
+
+    @field_validator("prefix")
+    @classmethod
+    def _require_bare_prefix(cls, prefix: str) -> str:
+        if not prefix or prefix.endswith("/"):
+            raise ValueError("mirror prefix must be non-empty and carry no trailing slash")
+        return prefix
+
+
+@app.command()
+def mirror() -> None:
+    root = Path.cwd()
+    config = load_mirror_config(root)
+    if config is None:
+        raise RuntimeError("[tool.docker-devkit.mirror] with 'prefix' is required to mirror")
+
+    targets = mirror_targets(root, config.prefix)
+    print(f"Install crane {CRANE_VERSION}")
+    sudo = "sudo " if os.geteuid() != 0 else ""
+    archive = "go-containerregistry_Linux_x86_64.tar.gz"
+    bash(f"curl -fsSLO https://github.com/google/go-containerregistry/releases/download/{CRANE_VERSION}/{archive}")
+    bash(f"{sudo}tar -xzf {archive} -C /usr/local/bin crane")
+    Path(archive).unlink()
+    for target in targets:
+        upstream = upstream_ref(target.reference, config.prefix)
+        print(f"Mirror {upstream} -> {target.reference}")
+        bash(f"crane copy {upstream} {target.reference}")
+    print(f"Mirrored images: {len(targets)}")
+
+
+def load_mirror_config(root: Path) -> MirrorConfig | None:
+    pyproject_path = root / "pyproject.toml"
+    if not pyproject_path.exists():
+        return None
+    data = tomllib.loads(pyproject_path.read_text(encoding="utf-8"))
+    section = data.get("tool", {}).get("docker-devkit", {}).get("mirror")
+    if section is None:
+        return None
+    return MirrorConfig.model_validate(section)
+
+
+def is_mirrored(reference: str, prefix: str) -> bool:
+    return reference.startswith(f"{prefix}/")
+
+
+def upstream_ref(reference: str, prefix: str) -> str:
+    return reference[len(prefix) + 1 :]
+
+
+def reference_exists(reference: str) -> bool:
+    return bash_check(f"docker buildx imagetools inspect {shlex.quote(reference)}")
+
+
+def mirror_targets(root: Path, prefix: str) -> list[ImageReference]:
+    return sorted(
+        (occurrence for occurrence in declared_references(root) if is_mirrored(occurrence.reference, prefix)),
+        key=lambda occurrence: occurrence.reference,
+    )
+
+
+def upstream_fallback(lock_data: dict[str, str], prefix: str, *, allow_upstream_fallback: bool) -> dict[str, str]:
+    missing = {
+        name: value for name, value in lock_data.items() if is_mirrored(value, prefix) and not reference_exists(value)
+    }
+    if not missing:
+        return {}
+    if not allow_upstream_fallback:
+        value = next(iter(missing.values()))
+        reference, _, digest = value.rpartition("@")
+        raise RuntimeError(
+            f"Image {reference or value} (digest {digest or 'unknown'}) is not mirrored yet. "
+            "If this is a new upstream dependency that has not been through CI, pass --allow-upstream-fallback "
+            "to pull the same digest from upstream for this run."
+        )
+    return {name: upstream_ref(value, prefix) for name, value in missing.items()}
+
+
+def write_fallback_overlay(substitutions: dict[str, str]) -> Path:
+    overlay = Path(tempfile.gettempdir()) / f"docker-devkit-upstream-fallback-{os.getpid()}.env"
+    overlay.write_text(
+        "\n".join(f"{name}={value}" for name, value in sorted(substitutions.items())) + "\n",
+        encoding="utf-8",
+    )
+    return overlay
+
+
+def main() -> None:
+    app()
+
+
+if __name__ == "__main__":
+    main()