@izykitten/ce 0.0.3-dev.2 → 0.0.3-dev.3

package/README.md

@@ -43,12 +43,23 @@ Tags and the `TAG_PREFIX`/`CACHE_PREFIX` values can use placeholders, filled fro
 | `{version}`, `{major}`, `{minor}`, `{patch}` | from a SemVer tag such as `v1.2.3` or `1.2.3-rc.1` |
 | `{stable}` | set only for a SemVer tag without a pre-release |
 | `{env.NAME}` | the environment variable `NAME`, e.g. a value from an earlier step |
+| `{arg.NAME}` | the build arg `NAME` from this command's `--build-arg`, the value the build gets |
 
 `{name}` must have a value (an empty variable has none): if it has none in this run, ce fails with exit 2 before anything runs. To leave a tag out instead, guard it with `{?name}`, which expands to nothing and keeps the tag only when `name` has a value. Every guard in a tag has to pass, and a tag's other values are only needed when it is kept. Names are checked in every tag, so a typo fails even in one a guard leaves out. If the guards leave out every tag, ce fails rather than build an untagged image.
 
-Values are made safe for a tag: any ASCII character a tag cannot have becomes `-`, so branch `ci/modernize-build` gives `ci-modernize-build`; a non-ASCII character is an error. A value never adds a tag or changes the image name. Prefixes can use `{name}` but not guards.
+Values are made safe for a tag: any ASCII character a tag cannot have becomes `-`, so branch `ci/modernize-build` gives `ci-modernize-build`; a non-ASCII character is an error. A value never adds a tag or changes the image name.
 
-Guards decide from the run's metadata, so they need it too: outside a Forgejo Actions run, `{?default}` fails like `{default}` would. Only `{env.NAME}` works everywhere. To tell whether a branch is the default one, ce reads `repository.default_branch` from the event file, or `DEFAULT_BRANCH` when the event has none, as on a scheduled run; set `DEFAULT_BRANCH: main` in such a workflow. Pull requests and tags are never the default branch, so they need neither.
+Prefixes can use guards too. A guard that does not pass leaves the prefix empty, since a prefix cannot be left out, and the prefix's other values are then not needed. `CACHE_PREFIX='{?pr}pr-{pr}'` gives each pull request a cache of its own (`pr-3`) and leaves every other build sharing the unprefixed one; with `CACHE_FROM_DEFAULT`, a PR also reads that shared one, as the prefix is empty on the default branch. The image name cannot use guards.
+
+`{arg.NAME}` takes the value from `--build-arg NAME=value`, or from the environment for `--build-arg NAME`, as the engine does, so a tag can name a value worked out on the command line:
+
+```sh
+ce docker build --build-arg VERSION="$(cat VERSION)" -t '{image}:{arg.VERSION}' .
+```
+
+The last `--build-arg NAME` wins. One whose variable is not set is passed over by Docker, which keeps an earlier value, while podman and buildah drop the value for the Containerfile's default. Where the build would use the Containerfile's default, which ce does not read, `{arg.NAME}` is an error, guarded or not, unless another guard beside it does not pass. An empty value (`--build-arg NAME=`) is no value. With podman's `--build-arg-file`, which ce does not read either, `{arg.NAME}` needs a `--build-arg NAME` of its own.
+
+Guards decide from the run's metadata, so they need it too: outside a Forgejo Actions run, `{?default}` fails like `{default}` would. Only `{env.NAME}` and `{arg.NAME}` work everywhere. A guard that does not pass decides on its own, though: a tag with one is left out, and a prefix with one is empty, even if another guard in it could not be worked out, so `{?default}{?env.X}` outside a run with `X` empty is simply left out. To tell whether a branch is the default one, ce reads `repository.default_branch` from the event file, or `DEFAULT_BRANCH` when the event has none, as on a scheduled run; set `DEFAULT_BRANCH: main` in such a workflow. Pull requests and tags are never the default branch, so they need neither.
 
 Put templates in single quotes: `?` is a glob character in zsh.
 
@@ -64,7 +75,7 @@ A branch, PR and default-branch build:
     -t '{image}:{ref},{?default}latest,{?default}caddy-{env.CADDY_VERSION}' .
 ```
 
-With `CACHE_PREFIX='{ref}'`, each branch and pull request gets a gha cache of its own, so a new one starts cold. With `CACHE_FROM_DEFAULT=true`, or `--cache-from-default` before the engine, each gha `--cache-to` is also read from its scope on the default branch: ce expands the cache prefix as it would be there, so PR 3 reads `pr-3` and `main` and writes only `pr-3`, and with `scope=js` it reads `main-js`. The default branch comes from the event file or `DEFAULT_BRANCH`, as for `{default}`. A prefix with `{sha}` or `{revision}` cannot be worked out for it, and is an error. This only picks what is read: whether it hits depends on what the default branch last wrote.
+With `CACHE_PREFIX='{ref}'`, each branch and pull request gets a gha cache of its own, so a new one starts cold. With `CACHE_FROM_DEFAULT=true`, or `--cache-from-default` before the engine, each gha `--cache-to` is also read from its scope on the default branch: ce expands the cache prefix as it would be there, so PR 3 reads `pr-3` and `main` and writes only `pr-3`, and with `scope=js` it reads `main-js`. The default branch comes from the event file or `DEFAULT_BRANCH`, as for `{default}`. A prefix that needs `{sha}` or `{revision}` there cannot be worked out for it, and is an error; one whose guards leave it empty there, like `{?pr}{sha}`, does not need them. This only picks what is read: whether it hits depends on what the default branch last wrote.
 
 The same can be written by hand, since ce only prefixes a `--cache-from` whose scope matches a `--cache-to` and passes any other on unchanged: `--cache type=gha,mode=max --cache-from type=gha,scope=main`.
 
@@ -80,7 +91,7 @@ A release on a SemVer tag, with the version aliases only for stable releases:
 
 ### Checking a template
 
-`--dry-run` before the engine, or `CE_DRY_RUN=true`, prints the build summary and the engine command ce would run, and exits 0 without running it or waiting for the engine. A template that fails still exits 2, so it also works as a check before pushing a workflow:
+`--dry-run` before the engine, or `CE_DRY_RUN=true`, prints the build summary and the engine command ce would run, and exits 0 without running it or waiting for the engine. A template that fails still exits 2, so it also works as a check before pushing a workflow. It checks the templates, not the engine: if the engine is not on `PATH`, it warns and still exits 0, so it works on a machine without one:
 
 ```sh
 GITHUB_REF=refs/tags/v1.2.3 GITHUB_SERVER_URL=https://code.example GITHUB_REPOSITORY=org/app \

package/dist/ce.js

@@ -244,7 +244,7 @@ class CiMetadata {
   get(name) {
     const envName = ENV_VARIABLE.exec(name)?.[1];
     if (envName)
-      return nonEmpty(this.#env[envName]);
+      return Object.hasOwn(this.#env, envName) ? nonEmpty(this.#env[envName]) : undefined;
     if (!this.#cache.has(name))
       this.#cache.set(name, this.#resolve(name));
     return this.#cache.get(name);
@@ -395,7 +395,7 @@ function onDefaultBranch(vars, branch) {
       if (UNKNOWN_ON_DEFAULT_BRANCH.has(name)) {
         throw new CeError(`ce: the default branch's cache scope cannot use {${name}}: this run does not know its commit`);
       }
-      return name === "image" || name.startsWith("env.") ? vars.get(name) : undefined;
+      return name === "image" || name.startsWith("env.") || name.startsWith("arg.") ? vars.get(name) : undefined;
     }
   };
 }
@@ -494,12 +494,26 @@ function parse(template, vars) {
   text(template.slice(last));
   return parts;
 }
-function expand(template, vars, encode) {
-  const parts = parse(template, vars);
+function guardsPass(parts, vars) {
+  let unknown;
   for (const part of parts) {
-    if ("name" in part && part.guard && vars.get(part.name) === undefined)
-      return null;
+    if (!("name" in part) || !part.guard)
+      continue;
+    try {
+      if (vars.get(part.name) === undefined)
+        return false;
+    } catch (exc) {
+      unknown ??= exc;
+    }
   }
+  if (unknown !== undefined)
+    throw unknown;
+  return true;
+}
+function expand(template, vars, encode) {
+  const parts = parse(template, vars);
+  if (!guardsPass(parts, vars))
+    return null;
   let out = "";
   for (const part of parts) {
     if ("text" in part)
@@ -514,16 +528,21 @@ function expand(template, vars, encode) {
   }
   return out;
 }
-function checkValue(template, vars, what) {
-  if (parse(template, vars).some((part) => ("guard" in part) && part.guard)) {
-    throw new CeError(`ce: ${what} ${JSON.stringify(template)} cannot use {?guards}; they only apply to tags`);
+function checkValue(template, vars, what, guards = false) {
+  const parts = parse(template, vars);
+  if (!guards && parts.some((part) => ("guard" in part) && part.guard)) {
+    throw new CeError(`ce: ${what} ${JSON.stringify(template)} cannot use {?guards}; they only apply to tags and prefixes`);
   }
 }
-function expandValue(template, vars, encode, what) {
-  checkValue(template, vars, what);
-  for (const part of parse(template, vars)) {
-    if ("name" in part && vars.get(part.name) === undefined) {
-      throw new CeError(`ce: {${part.name}} has no value here, in the ${what} ${JSON.stringify(template)}`);
+function expandValue(template, vars, encode, what, guards = false) {
+  checkValue(template, vars, what, guards);
+  const parts = parse(template, vars);
+  if (!guardsPass(parts, vars))
+    return "";
+  for (const part of parts) {
+    if ("name" in part && !part.guard && vars.get(part.name) === undefined) {
+      const hint = guards ? ` To leave the ${what} empty instead, guard it with {?${part.name}}.` : "";
+      throw new CeError(`ce: {${part.name}} has no value here, in the ${what} ${JSON.stringify(template)}.${hint}`);
     }
   }
   return expand(template, vars, encode);
@@ -585,7 +604,7 @@ function expandImage(template, vars) {
   return image;
 }
 function expandPrefix(prefix, vars, what) {
-  return hasPlaceholders(prefix) ? expandValue(prefix, vars, encodeTagValue, what) : prefix;
+  return hasPlaceholders(prefix) ? expandValue(prefix, vars, encodeTagValue, what, true) : prefix;
 }
 function literalRef(repo, tag) {
   if (tag.includes("/")) {
@@ -608,7 +627,7 @@ function rewriteTagValue(value, tagPrefix, vars = new CiMetadata) {
   if (hasPlaceholders(repoTemplate))
     checkValue(repoTemplate, vars, "image name");
   if (hasPlaceholders(tagPrefix))
-    checkValue(tagPrefix, vars, "tag prefix");
+    checkValue(tagPrefix, vars, "tag prefix", true);
   let repo;
   const expandRepo = () => repo ??= hasPlaceholders(repoTemplate) ? expandImage(repoTemplate, vars) : repoTemplate;
   if (tagList === undefined)
@@ -738,8 +757,27 @@ var SUMMARIZED = {
   "--build-arg": "buildArgs",
   "--label": "labels"
 };
-function parseBuildArgs(argv, tagPrefix, cachePrefix, { vars = new CiMetadata, labels = {}, defaultBranch } = {}) {
-  const args = [];
+var ARG_VARIABLE = /^arg\.([A-Za-z_][A-Za-z0-9_]*)$/u;
+function withBuildArgs(vars, values, argFile) {
+  return {
+    known: (name) => ARG_VARIABLE.test(name) || vars.known(name),
+    get(name) {
+      const arg = ARG_VARIABLE.exec(name)?.[1];
+      if (arg === undefined)
+        return vars.get(name);
+      const value = values.get(arg);
+      if (value === undefined && argFile) {
+        throw new CeError(`ce: {${name}} needs --build-arg ${arg} on this command, as ce does not read --build-arg-file`);
+      }
+      if (value == null) {
+        throw new CeError(`ce: {${name}} needs --build-arg ${arg}=<value> on this command, or --build-arg ${arg} with ${arg} set; ` + "ce does not read defaults from the Containerfile");
+      }
+      return value || undefined;
+    }
+  };
+}
+function parseBuildArgs(argv, tagPrefix, cachePrefix, { vars = new CiMetadata, labels = {}, defaultBranch, env = process.env, engine = "docker" } = {}) {
+  const parts = [];
   const rawCacheFrom = [];
   const rawCacheTo = [];
   const summary = {
@@ -754,16 +792,23 @@ function parseBuildArgs(argv, tagPrefix, cachePrefix, { vars = new CiMetadata, l
     context: ""
   };
   let templated = false;
+  const buildArgs = new Map;
+  let argFile = false;
   let rest = [];
   const addTags = (value, flag) => {
     templated ||= hasPlaceholders(value);
-    const refs = rewriteTagValue(value, tagPrefix, vars);
-    for (const ref of refs)
-      args.push(`${flag}${ref}`);
-    summary.tags.push(...refs);
+    parts.push({ tags: value, flag });
   };
   const summarize = (flag, value) => {
     const field = SUMMARIZED[flag];
+    if (field === "buildArgs") {
+      if (value.includes("="))
+        buildArgs.set(...splitAt(value, "="));
+      else if (Object.hasOwn(env, value))
+        buildArgs.set(value, env[value]);
+      else if (engine !== "docker")
+        buildArgs.set(value, null);
+    }
     if (field === "buildArgs" || field === "labels")
       summary[field].push(value);
     else
@@ -798,21 +843,34 @@ function parseBuildArgs(argv, tagPrefix, cachePrefix, { vars = new CiMetadata, l
       const flag = name;
       if (attached === undefined) {
         const v = next();
-        args.push(arg, v);
+        parts.push(arg, v);
         summarize(flag, v);
       } else {
-        args.push(arg);
+        parts.push(arg);
         summarize(flag, attached);
       }
+    } else if (name === "--build-arg-file") {
+      argFile = true;
+      parts.push(arg);
+      if (attached === undefined)
+        parts.push(next());
     } else if (arg.startsWith("-f") && arg.length > 2) {
-      args.push(arg);
+      parts.push(arg);
       summary.dockerfile = arg.slice(2);
     } else {
-      args.push(arg);
+      parts.push(arg);
       if (!arg.startsWith("-"))
         summary.context = arg;
     }
   }
+  vars = withBuildArgs(vars, buildArgs, argFile);
+  const args = parts.flatMap((part) => {
+    if (typeof part === "string")
+      return [part];
+    const refs = rewriteTagValue(part.tags, tagPrefix, vars);
+    summary.tags.push(...refs);
+    return refs.map((ref) => `${part.flag}${ref}`);
+  });
   if (templated && !summary.tags.length) {
     throw new CeError("ce: the guards in the -t templates left out every tag, so the image would have none");
   }
@@ -1212,6 +1270,7 @@ function engineCommand(engine, args) {
 }
 async function runEngine(engine, argv, tagPrefix, cachePrefix, options = {}) {
   const dryRun = options.dryRun ?? envFlag("CE_DRY_RUN");
+  ensureHelperOnPath();
   const command = findBuildCommand(engine, argv);
   let args = argv;
   let summary;
@@ -1220,6 +1279,7 @@ async function runEngine(engine, argv, tagPrefix, cachePrefix, options = {}) {
     const labels = !command.tagsOnly && (options.ociLabels ?? envFlag("OCI_LABELS")) ? vars.labels() : {};
     const fromDefault = !command.tagsOnly && (options.cacheFromDefault ?? envFlag("CACHE_FROM_DEFAULT"));
     const parsed = parseBuildArgs(argv.slice(command.start), tagPrefix, cachePrefix, {
+      engine,
       vars,
       labels,
       defaultBranch: fromDefault ? () => vars.defaultBranch() : undefined
@@ -1233,11 +1293,12 @@ async function runEngine(engine, argv, tagPrefix, cachePrefix, options = {}) {
       return 1;
     if (summary)
       printBuildSummary(engine, args, summary);
+    if (!system.which(engine))
+      logError(`ce: warning: ${engine} is not on PATH, so running this here would fail`);
     io.stdout(`${run.map(shellQuote).join(" ")}
 `);
     return 0;
   }
-  ensureHelperOnPath();
   if (!system.which(engine)) {
     logError(`Engine not found: ${engine}`);
     return 1;
@@ -1430,13 +1491,14 @@ Options:
                       run to builds. A --label with the same key wins.
                       Also available via OCI_LABELS=true.
     --dry-run         Print the build summary and the engine command, and
-                      exit without running it. Template errors still exit 2.
+                      exit without running it. Template errors still exit 2;
+                      a missing engine is only a warning.
                       Also available via CE_DRY_RUN=true.
 
 Placeholders, in build tags and in the tag and cache prefixes:
     {name}            The value of name; an error if it has none in this run.
     {?name}           Expands to nothing; leaves the tag out unless name has
-                      a value. Not allowed in prefixes.
+                      a value. In a prefix, leaves the prefix empty instead.
     {image}           Registry host and repository, lowercased
     {ref}             Branch name, pr-<number>, or tag name
     {branch} {pr} {tag}
@@ -1448,6 +1510,11 @@ Placeholders, in build tags and in the tag and cache prefixes:
                       From a SemVer tag such as v1.2.3 (v optional)
     {stable}          Set only for a SemVer tag without a pre-release
     {env.NAME}        The environment variable NAME
+    {arg.NAME}        The build arg NAME from this command's --build-arg;
+                      an error without one, even guarded, as ce does not
+                      read Containerfiles
+    A guard that does not pass leaves its tag out or its prefix empty even if
+    another guard there cannot be worked out.
     Values are made safe for a tag ("ci/x" becomes "ci-x"). Quote templates
     in single quotes. Example:
         -t '{image}:{ref},{?default}latest'

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@izykitten/ce",
-  "version": "0.0.3-dev.2",
+  "version": "0.0.3-dev.3",
   "description": "Container engine wrapper: tag/cache rewriting for docker, podman and buildah, registry login, and a Forgejo OIDC Docker credential helper",
   "type": "module",
   "license": "BSD-3-Clause",