@arc-e-tect/api-only-publisher 0.17.0 → 0.18.1

package/README.adoc

@@ -6,7 +6,7 @@
 :source-highlighter: rouge
 // The released version of this component: its snippets use it. The release
 // workflow updates it; do not change it by hand.
-:api-only-publisher-version: 0.17.0
+:api-only-publisher-version: 0.18.1
 
 [NOTE]
 .About this documentation
@@ -691,7 +691,8 @@ Configured portfolio: paths=none, location=portfolios
 |`file`
 |A local directory, laid out as `<directory>/<target>/<version>/`.
 Not a distribution mechanism between repositories: it makes the pipeline testable end to end with no infrastructure, lets a library share one build with the projects that implement it, and is a local-iteration escape hatch.
-Like a registry, it never replaces a version it holds with other documents (<<idempotent,Publishing the same version again §>>); with `clean: true` only the version being published is kept, for a directory that is a build output rather than a record of releases.
+Like a registry, it never replaces a version it holds with other documents (<<idempotent,Publishing the same version again §>>).
+With `clean: true` the directory is a build output rather than a record of releases: only the version being published is kept, and it holds the last build, so a version built again with other documents replaces it.
 
 |`maven`
 |A path publishes into a repository layout on disk; an `http(s)` URL deploys to a remote repository with an HTTP `PUT` per file, and keeps its `maven-metadata.xml` (<<maven-remote,Publishing to a remote Maven repository §>>).
@@ -776,7 +777,8 @@ What happens when the destination already has the version:
 |The same documents: nothing is written or sent, and the log says `already published, identical`, or `with the same documents` when the copy came from another commit.
 Other documents: the version is not published to that channel, and the run warns, naming the version -- a published version is never replaced, so publish the change as a new version (<<channel-problems,When a channel cannot take a version §>>).
 A copy that cannot be read, so there is nothing to compare, is treated as one with other documents, saying why it `cannot be compared`.
-`file` compares the `manifest.json` in the version's directory, and with `clean: true` removes the other versions, never this one.
+`file` compares the `manifest.json` in the version's directory.
+With `clean: true`, a build output, it replaces a version with other documents instead, logging `Replaced`, and removes the other versions; what guards a release there is the check before `publish` (<<version-guard,A changed target needs a new version §>>), and a subscriber's lockfile after it.
 `maven` and `gitlab-package` read back every file before uploading any, so a run interrupted half-way is completed by the next one; a GitLab registry that allows duplicate packages, as it does by default, would otherwise replace a version.
 A version completed beside an archive from another commit gets that archive's `manifest.json`, so the two agree on where the version came from; a POM carries no documents, and one already there is left as it is.
 `github-release` reads the manifest in the asset the release has, and never overwrites one.
@@ -2110,8 +2112,9 @@ package:
 [#docker-github-actions]
 ==== Reference pipeline: GitHub Actions
 
-Two jobs: `package` runs the image and hands the files on, `publish` checks them against the hand-off and publishes them to npm, Maven and NuGet on GitHub Packages, and to a GitHub Release, with the job's `GITHUB_TOKEN`.
-Copy it into `.github/workflows/`, then set the digest, the targets and the trigger.
+Two jobs on a tag: `package` runs the image and hands the files on, `publish` checks them against the hand-off, publishes them to npm, Maven and NuGet on GitHub Packages, and to a GitHub Release, with the job's `GITHUB_TOKEN`, and tags each target's release, `<target>-v<version>`.
+On a pull request, a third job, `check`, runs the image's `check` with no network, against the merge-base with the pull request's base branch (<<version-guard,A changed target needs a new version §>>).
+Copy it into `.github/workflows/`, then set the digest, the targets and the trigger, and `TAG_FORMAT` if `versioning.tagFormat` names the tags otherwise.
 The container-job variant above replaces its `package` job.
 
 It is exercised end to end, both variants, by a workflow in this repository that publishes to scratch packages and resolves each one with its ecosystem's own client and with the API-Only Subscriber.
@@ -2123,8 +2126,11 @@ include::pipelines/github-actions.yml[]
 
 What the two steps lose, and how the pipeline makes up for it:
 
-* *Idempotence.* A channel's remote mode leaves a version the registry has alone when it holds the same documents, and refuses it when it does not; `npm publish` or `dotnet nuget push` of an existing version fails instead.
-Each publishing step checks for the version first, compares the bytes, and publishes only what is missing, so a re-run is safe and a real conflict still fails.
+* *Idempotence.* A channel's remote mode leaves a version the registry has alone when it holds the same documents, and does not replace it when it holds others; `npm publish` or `dotnet nuget push` of an existing version fails instead.
+Each publishing step checks for the version first, compares the documents its `manifest.json` lists -- not the bytes, which differ by commit -- and publishes only what is missing, so a re-run is safe.
+A version held with other documents, or a registry that cannot be reached, is a `::warning` on the run, never a failed job, as for the Publisher's own channels (<<channel-problems,When a channel cannot take a version §>>).
+* *History.* Every job that runs the Publisher checks out the full history and the tags, `fetch-depth: 0`, which `check` and `publish` compare each target's version against.
+* *Release tags.* The last step creates each target's release tag at the commit it published, unless it exists; a tag it cannot create is a warning.
 * *Maven checksums and metadata.* The local Maven layout holds the artifact, its POM and its manifest, and no `.md5`, `.sha1` or `maven-metadata.xml`.
 `mvn deploy:deploy-file` writes those when it uploads, so the repository ends up as a remote `maven` channel would leave it, and Gradle resolves it.
 
@@ -2132,6 +2138,8 @@ Each publishing step checks for the version first, compares the bytes, and publi
 ==== Reference pipeline: GitLab CI
 
 One job, on a `v*` tag, as the GitHub Actions pipeline: the image builds, lints and publishes the targets to the project's Generic Package Registry with `gitlab-package`, and, where `apionly.yaml` configures it, to the project's Maven registry, both with the job's `CI_JOB_TOKEN`.
+In a merge request, a `check` job runs the image's `check` against the merge-base with the target branch; both jobs fetch the full history, `GIT_DEPTH: 0`, and the branch they compare with.
+`gitlab-package` with `release: true` creates each target's release, and with it the tag `<target>-v<version>` that `check` reads; without it, nothing creates that tag, and `check` compares with the merge-base alone.
 `apionly.yaml` needs no GitLab settings for it; the pipeline's header says what to add for the Maven registry.
 Copy it to `.gitlab-ci.yml`, then set the digest and the targets.
 The tag pattern keeps out the tag a GitLab release puts on the commit, `<target>-v<version>`, which would otherwise start the pipeline a second time; a library that tags its releases otherwise changes the rule, and keeps that one out.
@@ -2147,6 +2155,8 @@ include::pipelines/gitlab-ci.yml[]
 link:../docs/guides/publishing-to-gitlab/overview.adoc[Publishing to GitLab →] is the guide: tokens, a self-managed GitLab, and what consumers need.
 
 *The two-step variant*, link:pipelines/gitlab-ci-two-step.yml[`pipelines/gitlab-ci-two-step.yml` →], keeps the image sealed: `package` runs in the image, and one `publish:*` job per registry publishes to the project's package registry with that registry's own client and `CI_JOB_TOKEN`: npm, Maven and NuGet, and the archives to the generic package registry.
+As in the GitHub Actions pipeline, each compares documents and warns rather than fails, a merge request runs `check`, and a last job, `release:tags`, creates a GitLab release, and with it the tag, for each target's release; a pre-release gets none, as GitLab would make it the latest release.
+It runs on a `v*` tag, so the tags it creates do not start it again.
 It is validated against GitLab's published CI schema only; no GitLab pipeline has run it yet.
 
 [#docker-mirror-in-pipelines]

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@arc-e-tect/api-only-publisher",
-  "version": "0.17.0",
+  "version": "0.18.1",
   "description": "Builds, packs and publishes API description documents from a library of reusable fragments.",
   "license": "MIT",
   "author": "Arc-E-Tect",

package/src/channels.js

@@ -34,7 +34,10 @@ function format(template, manifest) {
  * Not a real distribution mechanism. It exists so the pipeline is testable end
  * to end before any remote exists, and as a local-iteration escape hatch
  * afterwards. Like the registries, it never replaces a version: one it holds with
- * the same documents is left alone, and one with other documents is refused.
+ * the same documents is left alone, and one with other documents is refused --
+ * unless `clean: true` declares the directory a build output, not a record of
+ * releases, where the last build is what it holds. What guards a release is then
+ * the git check before `publish`, and a subscriber's lockfile after it.
  */
 function publishFile(archive, manifest, options, log) {
     // Relative to the library root, not to wherever the CLI happened to be run.
@@ -44,21 +47,29 @@ function publishFile(archive, manifest, options, log) {
     const manifestDest = path.join(targetDir, "manifest.json");
     const files = [archiveDest, manifestDest];
 
+    const buildOutput = options.clean === true;
     const held = fs.existsSync(manifestDest) ? judgeHeld(fs.readFileSync(manifestDest), "json", manifest) : null;
-    if (held && held.verdict !== "same") {
+    const replaced = held && held.verdict !== "same";
+    if (replaced && !buildOutput) {
         throw new ChannelConflict(`${dir} already has ${manifest.target} ${manifest.version}, and it ` +
             `${held.verdict === "different" ? "is not this one" : `cannot be compared: ${held.reason}`}; ` +
             "a published version is never replaced, so publish the change as a new version");
     }
-    // `clean: true` keeps only the version being published. Used where the
-    // directory is a build output rather than a record of releases: an older
-    // version is published again from the commit that declared it.
-    if (options.clean === true && fs.existsSync(path.join(dir, manifest.target))) {
+    // `clean: true` keeps only the version being published, and the last build of it.
+    // Used where the directory is a build output rather than a record of releases: an
+    // older version is published again from the commit that declared it.
+    if (buildOutput && fs.existsSync(path.join(dir, manifest.target))) {
         for (const version of fs.readdirSync(path.join(dir, manifest.target))) {
-            if (version !== manifest.version) fs.rmSync(path.join(dir, manifest.target, version), { recursive: true, force: true });
+            if (version !== manifest.version || replaced) {
+                fs.rmSync(path.join(dir, manifest.target, version), { recursive: true, force: true });
+            }
         }
     }
-    if (held && fs.existsSync(archiveDest)) {
+    if (replaced) {
+        log(`-- Replaced ${manifest.target} ${manifest.version} in ${dir}, which clean: true makes a build output: it held ` +
+            (held.verdict === "different" ? "other documents" : `a copy that cannot be compared (${held.reason})`));
+    }
+    if (held && !replaced && fs.existsSync(archiveDest)) {
         const identical = fs.readFileSync(archiveDest).equals(fs.readFileSync(archive));
         log(`-- ${manifest.target} ${manifest.version} is already published, ` +
             `${identical ? "identical" : "with the same documents"}, in ${dir}; nothing to do`);
@@ -68,7 +79,7 @@ function publishFile(archive, manifest, options, log) {
     fs.copyFileSync(archive, archiveDest);
     fs.writeFileSync(manifestDest, JSON.stringify(manifest, null, 2) + "\n");
 
-    log(`-- Published ${manifest.target} ${manifest.version} to ${archiveDest}`);
+    if (!replaced) log(`-- Published ${manifest.target} ${manifest.version} to ${archiveDest}`);
     return { location: archiveDest, files };
 }