@izykitten/ce 0.0.3-dev.1 → 0.0.3-dev.2

package/README.md

@@ -64,13 +64,11 @@ A branch, PR and default-branch build:
     -t '{image}:{ref},{?default}latest,{?default}caddy-{env.CADDY_VERSION}' .
 ```
 
-With `CACHE_PREFIX='{ref}'`, each branch and pull request gets a gha cache of its own, so a new one starts cold. To also read the default branch's cache, add it as a `--cache-from`. ce only prefixes a `--cache-from` whose scope matches a `--cache-to`, so this one is passed on unchanged and only read:
+With `CACHE_PREFIX='{ref}'`, each branch and pull request gets a gha cache of its own, so a new one starts cold. With `CACHE_FROM_DEFAULT=true`, or `--cache-from-default` before the engine, each gha `--cache-to` is also read from its scope on the default branch: ce expands the cache prefix as it would be there, so PR 3 reads `pr-3` and `main` and writes only `pr-3`, and with `scope=js` it reads `main-js`. The default branch comes from the event file or `DEFAULT_BRANCH`, as for `{default}`. A prefix with `{sha}` or `{revision}` cannot be worked out for it, and is an error. This only picks what is read: whether it hits depends on what the default branch last wrote.
 
-```sh
---cache type=gha,mode=max --cache-from type=gha,scope=main
-```
+The same can be written by hand, since ce only prefixes a `--cache-from` whose scope matches a `--cache-to` and passes any other on unchanged: `--cache type=gha,mode=max --cache-from type=gha,scope=main`.
 
-PR 3 then reads `pr-3` and `main` and writes only `pr-3`; main reads its own cache a second time. With a scope of its own, such as `--cache type=gha,mode=max,scope=js`, the cache to read is `main-js`. This only picks what is read: whether it hits depends on what main last wrote.
+The `--cache-from` that `--cache` makes leaves out a gha `mode=`, which only applies to export, and a `--cache-from` given twice, as the default branch's own scope is on the default branch, is passed once.
 
 A release on a SemVer tag, with the version aliases only for stable releases:
 
@@ -80,6 +78,15 @@ A release on a SemVer tag, with the version aliases only for stable releases:
 
 `{stable}` does not mean newest: rebuilding an older release moves its aliases back to it.
 
+### Checking a template
+
+`--dry-run` before the engine, or `CE_DRY_RUN=true`, prints the build summary and the engine command ce would run, and exits 0 without running it or waiting for the engine. A template that fails still exits 2, so it also works as a check before pushing a workflow:
+
+```sh
+GITHUB_REF=refs/tags/v1.2.3 GITHUB_SERVER_URL=https://code.example GITHUB_REPOSITORY=org/app \
+  ce --dry-run docker buildx build -t '{image}:{ref},{?stable}{major}.{minor}' .
+```
+
 ### OCI labels
 
 With `OCI_LABELS=true`, or `--oci-labels` before the engine (`ce --oci-labels docker buildx build …`, like the prefix options), builds get `org.opencontainers.image.source`, `revision`, `version`, `title` and `description` labels from the run. Labels the run cannot tell are left out. They override a Dockerfile `LABEL` with the same key, and a `--label` you pass wins over them. `created` is not set, so the labels do not change the image on every rebuild.
@@ -92,7 +99,7 @@ The package installs `docker-credential-ce-forgejo` next to `ce`. `ce` puts that
 
 ## Compatibility
 
-These stay the same from release to release, and [test/contract.test.ts](test/contract.test.ts) checks them by running the built package under Node and Bun:
+These behave the same under Node and Bun, and [test/contract.test.ts](test/contract.test.ts) checks them by running the built package under both. A release that changes one says so here:
 
 - commands, flags and the environment variables `login` reads;
 - the arguments the engine receives after tag and cache rewriting, and its exit code, signals and PID (`ce` replaces itself with the engine);
@@ -102,6 +109,8 @@ These stay the same from release to release, and [test/contract.test.ts](test/co
 
 The wording of messages is not part of that and may change.
 
+In 0.0.3, the `--cache-from` that `--cache` makes no longer has a gha `mode=`, which only applies to export, and a `--cache-from` given twice is passed once.
+
 Compared with the Python version:
 
 - The state file is locked with `ce-forgejo-credentials.json.pidlock`, holding the holder's PID and hostname, rather than `flock`, which Node does not have. A lock whose holder died is broken right away if the PID is gone on the same host, otherwise after two minutes. It does not exclude a Python `ce` using the same Docker config at the same moment.

package/dist/ce.js

@@ -298,7 +298,7 @@ class CiMetadata {
         const ref = this.#parsedRef();
         if (ref.kind !== "branch")
           return;
-        return ref.name === this.#defaultBranch() ? ref.name : undefined;
+        return ref.name === this.defaultBranch() ? ref.name : undefined;
       }
       case "revision":
         return this.#require("SHA", `{${name}}`);
@@ -359,7 +359,7 @@ class CiMetadata {
     const ref = this.#parsedRef();
     return ref.kind === "tag" ? SEMVER.exec(ref.name) : null;
   }
-  #defaultBranch() {
+  defaultBranch() {
     const repository = this.#ciEnv("EVENT_PATH") ? this.#payload().repository : undefined;
     const fromEvent = isObject(repository) ? repository.default_branch : undefined;
     const branch = typeof fromEvent === "string" && fromEvent || nonEmpty(this.#env.DEFAULT_BRANCH);
@@ -384,6 +384,21 @@ class CiMetadata {
     return payload;
   }
 }
+var ON_DEFAULT_BRANCH = new Set(["ref", "branch", "default"]);
+var UNKNOWN_ON_DEFAULT_BRANCH = new Set(["revision", "sha"]);
+function onDefaultBranch(vars, branch) {
+  return {
+    known: (name) => vars.known(name),
+    get(name) {
+      if (ON_DEFAULT_BRANCH.has(name))
+        return branch;
+      if (UNKNOWN_ON_DEFAULT_BRANCH.has(name)) {
+        throw new CeError(`ce: the default branch's cache scope cannot use {${name}}: this run does not know its commit`);
+      }
+      return name === "image" || name.startsWith("env.") ? vars.get(name) : undefined;
+    }
+  };
+}
 
 // src/system.ts
 import { spawnSync } from "node:child_process";
@@ -664,6 +679,7 @@ function ghaFields(value) {
   const type = fields?.findLast((field) => fieldKey(field) === "type");
   return type?.slice("type=".length) === "gha" ? fields : null;
 }
+var isGhaCache = (value) => ghaFields(value) !== null;
 var scopeIndex = (fields) => fields.findLastIndex((field) => fieldKey(field) === "scope");
 function cacheScopeName(value) {
   const fields = ghaFields(value);
@@ -697,6 +713,12 @@ function normalizeCacheScope(value, cachePrefix) {
     fields[i] = `scope=${scope}`;
   return csvJoin(fields);
 }
+function withoutCacheMode(value) {
+  const fields = ghaFields(value);
+  if (!fields?.some((field) => fieldKey(field) === "mode"))
+    return value;
+  return csvJoin(fields.filter((field) => fieldKey(field) !== "mode"));
+}
 
 // src/build-args.ts
 function consumeValue(argv, flag) {
@@ -716,7 +738,7 @@ var SUMMARIZED = {
   "--build-arg": "buildArgs",
   "--label": "labels"
 };
-function parseBuildArgs(argv, tagPrefix, cachePrefix, { vars = new CiMetadata, labels = {} } = {}) {
+function parseBuildArgs(argv, tagPrefix, cachePrefix, { vars = new CiMetadata, labels = {}, defaultBranch } = {}) {
   const args = [];
   const rawCacheFrom = [];
   const rawCacheTo = [];
@@ -770,7 +792,7 @@ function parseBuildArgs(argv, tagPrefix, cachePrefix, { vars = new CiMetadata, l
       rawCacheTo.push(attached ?? next());
     else if (name === "--cache") {
       const v = attached ?? next();
-      rawCacheFrom.push(v);
+      rawCacheFrom.push(withoutCacheMode(v));
       rawCacheTo.push(v);
     } else if (Object.hasOwn(SUMMARIZED, name)) {
       const flag = name;
@@ -798,11 +820,17 @@ function parseBuildArgs(argv, tagPrefix, cachePrefix, { vars = new CiMetadata, l
   const added = Object.entries(labels).filter(([key]) => !ownKeys.has(key)).map(([key, value]) => `${key}=${value}`);
   args.unshift(...added.map((label) => `--label=${label}`));
   summary.labels.unshift(...added);
+  const prefixTemplate = cachePrefix;
   if (rawCacheFrom.length || rawCacheTo.length)
     cachePrefix = expandPrefix(cachePrefix, vars, "cache prefix");
   const toNames = new Set(rawCacheTo.map(cacheScopeName));
-  for (const c of rawCacheFrom) {
-    const rewritten = toNames.has(cacheScopeName(c)) ? normalizeCacheScope(c, cachePrefix) : c;
+  const cacheFrom = rawCacheFrom.map((c) => toNames.has(cacheScopeName(c)) ? normalizeCacheScope(c, cachePrefix) : c);
+  const ghaTo = rawCacheTo.filter(isGhaCache);
+  if (defaultBranch && ghaTo.length) {
+    const prefix = expandPrefix(prefixTemplate, onDefaultBranch(vars, defaultBranch()), "cache prefix");
+    cacheFrom.push(...ghaTo.map((c) => normalizeCacheScope(withoutCacheMode(c), prefix)));
+  }
+  for (const rewritten of new Set(cacheFrom)) {
     args.push(`--cache-from=${rewritten}`);
     summary.cacheFrom.push(rewritten);
   }
@@ -1169,17 +1197,46 @@ function envFlag(name) {
     return true;
   throw new CeError(`ce: ${name} must be true or false, not ${JSON.stringify(process.env[name])}`);
 }
-async function runEngine(engine, argv, tagPrefix, cachePrefix, ociLabels) {
+function shellQuote(arg) {
+  return /^[A-Za-z0-9_@%+:,./-][A-Za-z0-9_@%+=:,./-]*$/u.test(arg) ? arg : `'${arg.replaceAll("'", `'\\''`)}'`;
+}
+function engineCommand(engine, args) {
+  if (engine === "docker" || !system.which("gosu"))
+    return [engine, ...args];
+  const user = process.env.USER ?? "";
+  if (!user) {
+    logError("ce: gosu is available but USER is not set; cannot drop privileges. Set USER or remove gosu from PATH.");
+    return null;
+  }
+  return ["gosu", user, engine, ...args];
+}
+async function runEngine(engine, argv, tagPrefix, cachePrefix, options = {}) {
+  const dryRun = options.dryRun ?? envFlag("CE_DRY_RUN");
   const command = findBuildCommand(engine, argv);
   let args = argv;
   let summary;
   if (command) {
     const vars = new CiMetadata;
-    const labels = !command.tagsOnly && (ociLabels ?? envFlag("OCI_LABELS")) ? vars.labels() : {};
-    const parsed = parseBuildArgs(argv.slice(command.start), tagPrefix, cachePrefix, { vars, labels });
+    const labels = !command.tagsOnly && (options.ociLabels ?? envFlag("OCI_LABELS")) ? vars.labels() : {};
+    const fromDefault = !command.tagsOnly && (options.cacheFromDefault ?? envFlag("CACHE_FROM_DEFAULT"));
+    const parsed = parseBuildArgs(argv.slice(command.start), tagPrefix, cachePrefix, {
+      vars,
+      labels,
+      defaultBranch: fromDefault ? () => vars.defaultBranch() : undefined
+    });
     args = [...argv.slice(0, command.start), ...parsed.args];
     summary = parsed.summary;
   }
+  if (dryRun) {
+    const run = engineCommand(engine, args);
+    if (!run)
+      return 1;
+    if (summary)
+      printBuildSummary(engine, args, summary);
+    io.stdout(`${run.map(shellQuote).join(" ")}
+`);
+    return 0;
+  }
   ensureHelperOnPath();
   if (!system.which(engine)) {
     logError(`Engine not found: ${engine}`);
@@ -1189,17 +1246,8 @@ async function runEngine(engine, argv, tagPrefix, cachePrefix, ociLabels) {
     return 1;
   if (summary)
     printBuildSummary(engine, args, summary);
-  if (engine === "docker")
-    return system.exec("docker", ["docker", ...args]);
-  if (system.which("gosu")) {
-    const user = process.env.USER ?? "";
-    if (!user) {
-      logError("ce: gosu is available but USER is not set; cannot drop privileges. Set USER or remove gosu from PATH.");
-      return 1;
-    }
-    return system.exec("gosu", ["gosu", user, engine, ...args]);
-  }
-  return system.exec(engine, [engine, ...args]);
+  const run = engineCommand(engine, args);
+  return run ? system.exec(run[0], run) : 1;
 }
 
 // src/login.ts
@@ -1352,7 +1400,8 @@ async function login() {
 
 // src/cli.ts
 function helpText() {
-  return `Usage: ${io.argv0()} [--tag-prefix=<prefix>] [--cache-prefix=<prefix>] [--oci-labels] [command|engine] [args]
+  return `Usage: ${io.argv0()} [--tag-prefix=<prefix>] [--cache-prefix=<prefix>] [--oci-labels]
+          [--cache-from-default] [--dry-run] [command|engine] [args]
 
 Options:
     --tag-prefix=<prefix>
@@ -1368,10 +1417,21 @@ Options:
                       Also available via CACHE_PREFIX environment variable.
     --cache=<spec>    Shorthand for passing <spec> to both --cache-from and
                       --cache-to. Supports the same forms as --cache-from/--cache-to.
+                      A gha cache-from made this way leaves out mode=, which
+                      only applies to export. The same cache-from given twice
+                      is passed once.
+    --cache-from-default
+                      Also read each gha --cache-to from its scope on the
+                      default branch, from the cache prefix as it is there,
+                      so a new branch or PR starts from its cache.
+                      Also available via CACHE_FROM_DEFAULT=true.
     --oci-labels      Add org.opencontainers.image.* labels (source, revision,
                       version, title, description) from the Forgejo Actions
                       run to builds. A --label with the same key wins.
                       Also available via OCI_LABELS=true.
+    --dry-run         Print the build summary and the engine command, and
+                      exit without running it. Template errors still exit 2.
+                      Also available via CE_DRY_RUN=true.
 
 Placeholders, in build tags and in the tag and cache prefixes:
     {name}            The value of name; an error if it has none in this run.
@@ -1428,7 +1488,7 @@ Environment variables (used by login):
 `;
 }
 async function main(argv) {
-  let ociLabels;
+  const options = {};
   let tagPrefix = process.env.TAG_PREFIX ?? "";
   let cachePrefix = process.env.CACHE_PREFIX ?? process.env.TAG_PREFIX ?? "";
   for (;; ) {
@@ -1442,7 +1502,11 @@ async function main(argv) {
     else if (a?.startsWith("--cache-prefix="))
       [cachePrefix, argv] = [a.slice("--cache-prefix=".length), argv.slice(1)];
     else if (a === "--oci-labels")
-      [ociLabels, argv] = [true, argv.slice(1)];
+      [options.ociLabels, argv] = [true, argv.slice(1)];
+    else if (a === "--cache-from-default")
+      [options.cacheFromDefault, argv] = [true, argv.slice(1)];
+    else if (a === "--dry-run")
+      [options.dryRun, argv] = [true, argv.slice(1)];
     else
       break;
   }
@@ -1456,7 +1520,7 @@ async function main(argv) {
   if (command === "credential-helper")
     return credentialHelper(rest);
   if (isEngine(command))
-    return runEngine(command, rest, tagPrefix, cachePrefix, ociLabels);
+    return runEngine(command, rest, tagPrefix, cachePrefix, options);
   io.stderr(helpText());
   return 1;
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@izykitten/ce",
-  "version": "0.0.3-dev.1",
+  "version": "0.0.3-dev.2",
   "description": "Container engine wrapper: tag/cache rewriting for docker, podman and buildah, registry login, and a Forgejo OIDC Docker credential helper",
   "type": "module",
   "license": "BSD-3-Clause",