container-source-policy-windows-arm64 0.1.3 → 0.3.0

package/README.md

@@ -1,43 +1,50 @@
 # container-source-policy
 
-Generate a Docker BuildKit **source policy** file (`docker buildx build --source-policy-file …`) by parsing Dockerfiles and pinning `FROM` images to immutable digests.
+Generate a Docker BuildKit **source policy** file by parsing Dockerfiles and pinning `FROM` images to immutable digests.
 
 This helps make `docker buildx build` inputs reproducible without rewriting your Dockerfile.
 
+See the [BuildKit documentation on build reproducibility](https://github.com/moby/buildkit/blob/master/docs/build-repro.md) for more details on source policies.
+
 ## Quick start
 
 ```bash
 container-source-policy pin --stdout Dockerfile > source-policy.json
-docker buildx build --source-policy-file source-policy.json -t my-image:dev .
+EXPERIMENTAL_BUILDKIT_SOURCE_POLICY=source-policy.json docker buildx build -t my-image:dev .
 ```
 
+> **Note:** [`EXPERIMENTAL_BUILDKIT_SOURCE_POLICY`](https://docs.docker.com/build/building/variables/#experimental_buildkit_source_policy) is the environment variable used by Docker Buildx to specify a source policy file.
+
 ## Install
 
-### Go (build from source)
+Run directly without installing (recommended):
 
 ```bash
-go install github.com/tinovyatkin/container-source-policy@latest
-```
+# npm/bun
+npx container-source-policy --help
+bunx container-source-policy --help
 
-### npm (prebuilt binary)
+# Python
+uvx container-source-policy --help
 
-```bash
-npm i -g container-source-policy
-container-source-policy --help
+# Ruby (requires RubyGems 3.3+)
+gem exec container-source-policy --help
 ```
 
-### PyPI (prebuilt binary)
+Or install globally:
 
 ```bash
-pipx install container-source-policy
-container-source-policy --help
-```
+# Go (build from source)
+go install github.com/tinovyatkin/container-source-policy@latest
 
-### RubyGems (prebuilt binary)
+# npm
+npm i -g container-source-policy
 
-```bash
+# Python
+pipx install container-source-policy
+
+# Ruby
 gem install container-source-policy
-container-source-policy --help
 ```
 
 ## Usage
@@ -60,10 +67,16 @@ Write directly to a file:
 container-source-policy pin --output source-policy.json Dockerfile
 ```
 
-Then pass the policy to BuildKit / Buildx:
+Then pass the policy to BuildKit / Buildx via the environment variable:
+
+```bash
+EXPERIMENTAL_BUILDKIT_SOURCE_POLICY=source-policy.json docker buildx build .
+```
+
+Or use `buildctl` directly with the `--source-policy-file` flag:
 
 ```bash
-docker buildx build --source-policy-file source-policy.json .
+buildctl build --frontend dockerfile.v0 --local dockerfile=. --local context=. --source-policy-file source-policy.json
 ```
 
 Shell completion scripts are available via Cobra:
@@ -74,15 +87,49 @@ container-source-policy completion zsh
 
 ## What gets pinned
 
-- Looks at `FROM …` instructions across all provided Dockerfiles.
+### Container images (`FROM`, `COPY --from`, `ONBUILD`)
+
+- Looks at `FROM …`, `COPY --from=<image>`, and their `ONBUILD` variants across all provided Dockerfiles.
 - Skips:
   - `FROM scratch`
-  - `FROM <stage>` references to a previous named build stage
-  - `FROM ${VAR}` / `FROM $VAR` (unexpanded ARG/ENV variables)
+  - `FROM <stage>` / `COPY --from=<stage>` references to a previous named build stage
+  - `COPY --from=0` numeric stage indices
+  - `FROM ${VAR}` / `COPY --from=${VAR}` (unexpanded ARG/ENV variables)
   - images already written as `name@sha256:…`
 - Resolves the image manifest digest from the registry and emits BuildKit `CONVERT` rules of the form:
   - `docker-image://<as-written-in-Dockerfile>` → `docker-image://<normalized>@sha256:…`
 
+### HTTP sources (`ADD`, `ONBUILD ADD`)
+
+- Looks at `ADD <url> …` and `ONBUILD ADD <url> …` instructions with HTTP/HTTPS URLs.
+- Skips:
+  - `ADD --checksum=… <url>` (already pinned)
+  - URLs containing unexpanded variables (`${VAR}`, `$VAR`)
+  - Git URLs (handled separately, see below)
+  - Volatile content (emits warning): URLs returning `Cache-Control: no-store`, `no-cache`, `max-age=0`, or expired `Expires` headers
+- Fetches the checksum and emits `CONVERT` rules with `http.checksum` attribute.
+- **Respects `Vary` header**: captures request headers that affect response content (e.g., `User-Agent`, `Accept-Encoding`) and includes them in the policy as `http.header.*` attributes to ensure reproducible builds.
+
+**Optimized checksum fetching** — avoids downloading large files when possible:
+- `raw.githubusercontent.com`: extracts SHA256 from ETag header
+- GitHub releases: uses the API `digest` field (set `GITHUB_TOKEN` for higher rate limits)
+- S3: uses `x-amz-checksum-sha256` response header (by sending `x-amz-checksum-mode: ENABLED`)
+- Fallback: downloads and computes SHA256
+
+### Git sources (`ADD`, `ONBUILD ADD`)
+
+- Looks at `ADD <git-url> …` and `ONBUILD ADD <git-url> …` instructions with Git repository URLs.
+- Supports various Git URL formats:
+  - `https://github.com/owner/repo.git#ref`
+  - `git://host/path#ref`
+  - `git@github.com:owner/repo#ref`
+  - `ssh://git@host/path#ref`
+- Skips URLs containing unexpanded variables (`${VAR}`, `$VAR`)
+- Uses `git ls-remote` to resolve the ref (branch, tag, or commit) to a commit SHA
+- Emits `CONVERT` rules with `git.checksum` attribute (full 40-character commit SHA)
+
+Example: `ADD https://github.com/cli/cli.git#v2.40.0 /dest` pins to commit `54d56cab...`
+
 ## Development
 
 ```bash
@@ -100,11 +147,13 @@ UPDATE_SNAPS=true go test ./internal/integration/...
 ## Repository layout
 
 - `cmd/container-source-policy/cmd/`: Cobra CLI commands
-- `internal/dockerfile`: Dockerfile parsing (`FROM` extraction)
-- `internal/registry`: registry client (digest resolution)
+- `internal/dockerfile`: Dockerfile parsing (`FROM` and `ADD` extraction)
+- `internal/registry`: registry client (image digest resolution)
+- `internal/http`: HTTP client (URL checksum fetching with optimizations)
+- `internal/git`: Git client (commit SHA resolution via git ls-remote)
 - `internal/policy`: BuildKit source policy types and JSON output
 - `internal/pin`: orchestration logic for `pin`
-- `internal/integration`: end-to-end tests with a mock registry and snapshots
+- `internal/integration`: end-to-end tests with mock registry/HTTP server and snapshots
 - `packaging/`: wrappers for publishing prebuilt binaries to npm / PyPI / RubyGems
 
 ## Packaging

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "container-source-policy-windows-arm64",
-  "version": "0.1.3",
+  "version": "0.3.0",
   "description": "The windows arm64 binary for container-source-policy",
   "repository": {
     "type": "git",