@arc-e-tect/api-only-publisher 0.9.0 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.adoc CHANGED
@@ -6,12 +6,12 @@
6
6
  :source-highlighter: rouge
7
7
  // The released version of this component: its snippets use it. The release
8
8
  // workflow updates it; do not change it by hand.
9
- :api-only-publisher-version: 0.9.0
9
+ :api-only-publisher-version: 0.11.0
10
10
 
11
11
  image:https://github.com/Arc-E-Tect/SoftwareEngineeringDoneRight-API/actions/workflows/nvd-cache-refresh.yml/badge.svg[Vulnerability Scan,link=https://github.com/Arc-E-Tect/SoftwareEngineeringDoneRight-API/actions/workflows/nvd-cache-refresh.yml]
12
12
  image:https://img.shields.io/npm/v/@arc-e-tect/api-only-publisher[npm,link=https://www.npmjs.com/package/@arc-e-tect/api-only-publisher]
13
13
  image:https://img.shields.io/github/v/release/Arc-E-Tect/SoftwareEngineeringDoneRight-API?filter=api-only-publisher-v*[GitHub release,link=https://github.com/Arc-E-Tect/SoftwareEngineeringDoneRight-API/releases?q=api-only-publisher]
14
- image:https://img.shields.io/badge/Node-22%2B-339933?logo=node.js[Node 22+,link=https://nodejs.org]
14
+ image:https://img.shields.io/badge/Node-22.14%2B-339933?logo=node.js[Node 22.14+,link=https://nodejs.org]
15
15
  image:https://img.shields.io/badge/License-MIT-blue[MIT,link=LICENSE]
16
16
 
17
17
  [NOTE]
@@ -169,7 +169,7 @@ A target whose closure is untouched needs no new version, however much else in t
169
169
  |`publish [--pre-release <ids>] [--target <t>]... [--channel <c>]... [--out <dir>]`
170
170
  |Pack once, then ship those same bytes to every configured channel, or to each `--channel`.
171
171
 
172
- |`split --out <dir> [--by kind\|target]`
172
+ |`split [--out <dir>] [--by kind\|target]`
173
173
  |Make the library safe to break into separate repositories.
174
174
  |===
175
175
 
@@ -344,6 +344,9 @@ A `publish: false` target keeps the version its source declares.
344
344
  . **Lint** the finished document, so a broken fragment surfaces against the target it actually affects.
345
345
  `build` prints the linter's output only when the lint fails, and writes no report; to see warnings, and to write each document's report under `reports.lint`, run `lint`.
346
346
 
347
+ . **Report operations with no example**, after the lint and apart from it: an AsyncAPI message without `examples`, or an OpenAPI request body or response without one, is logged as a finding, and `lint.examples.<kind>` decides whether that fails the build (<<examples-check,An operation with no example is reported apart from lint §>>).
348
+ Only `build` does this; `lint` does not, and no report file is written.
349
+
347
350
  [#versions]
348
351
  == Versions
349
352
 
@@ -606,7 +609,7 @@ Configured portfolio: paths=none, location=portfolios
606
609
  |`file`
607
610
  |A local directory, laid out as `<directory>/<target>/<version>/`.
608
611
  Not a distribution mechanism between repositories: it makes the pipeline testable end to end with no infrastructure, lets a library share one build with the projects that implement it, and is a local-iteration escape hatch.
609
- Publishing a version again replaces that version and keeps the others; with `clean: true` only the version being published is kept, for a directory that is a build output rather than a record of releases.
612
+ Publishing a version again writes it again and keeps the others (<<idempotent,Publishing the same version again §>>); with `clean: true` only the version being published is kept, for a directory that is a build output rather than a record of releases.
610
613
 
611
614
  |`maven`
612
615
  |A path publishes into a repository layout on disk; an `http(s)` URL deploys to a remote repository with an HTTP `PUT` per file and a bearer token.
@@ -618,20 +621,120 @@ Native for Gradle consumers: version resolution, caching and conflict handling c
618
621
  |`github-release`
619
622
  |The language-neutral floor: immutable per tag, works for private repositories with a token, readable by a consumer with no JVM and no Node.js.
620
623
  No dependency-resolution semantics and no update notification, which is why it is the fallback rather than the default.
624
+
625
+ |`nuget`
626
+ |A path writes a `.nupkg` into a local feed; an `http(s)` URL -- the feed's service index -- pushes to a remote one.
627
+ Native for .NET consumers: a `PackageReference` at a version, restored from a feed. See <<nuget,The `nuget` channel §>>.
628
+ |===
629
+
630
+ [#idempotent]
631
+ === Publishing the same version again
632
+
633
+ `publish` is idempotent: publishing a version again with the same content changes nothing, on every channel.
634
+
635
+ That rests on `pack` being reproducible.
636
+ The same documents always pack to the same archive, byte for byte: the archive is written with a fixed mode, no owner and a fixed timestamp, and `manifest.json`'s `producedAt` is not the time `pack` ran but the time of the commit being packed.
637
+ Set `SOURCE_DATE_EPOCH` (seconds since 1970, the https://reproducible-builds.org/specs/source-date-epoch/[reproducible-builds convention ↗]) to choose another; outside a git repository, where there is no commit, it is the only way to make two packs identical, and without it `producedAt` is the current time.
638
+
639
+ What happens when the destination already has the version:
640
+
641
+ [cols="1,3", options="header"]
621
642
  |===
643
+ |Destination |Already has the version
644
+
645
+ |`file`; `maven` and `nuget` to a directory; `npm` with `publish: false`
646
+ |Written again. With the same content the bytes are the same; with different content the version is replaced, and every subscriber whose lockfile recorded the old bytes fails its build until the change is published as a new version.
647
+
648
+ |`maven` and `nuget` to a URL, `npm`, `github-release`
649
+ |The same bytes: nothing is sent, and the log says `already published, identical`. Different bytes: `publish` fails, naming the version -- a published version is never replaced, so publish the change as a new version.
650
+ `github-release` compares the asset it would upload with the one the release has, and never overwrites one.
651
+ A NuGet feed that cannot return the package it has, so there is nothing to compare, fails as a different one would.
652
+ |===
653
+
654
+ [#channel-text]
655
+ === What a package says about itself
656
+
657
+ The Maven POM, the npm `package.json`, the NuGet `.nuspec` and a GitHub release all describe the contract.
658
+ By default each says `API description documents for <target>.`, and a GitHub release is titled `<target> <version>`.
659
+ `channels.<channel>.description` replaces the description, and `channels.github-release.title` and `notes` the release's title and notes (notes default to the description); `{target}` and `{version}` are filled in, as in `tagFormat`.
660
+
661
+ npm publishes a release under the dist-tag `latest` and a pre-release under `next`.
662
+ `channels.npm.releaseTag` and `prereleaseTag` name others: publishing a patch to an older line under `releaseTag: v1` keeps `latest` on the newest line, which publishing it under `latest` would move back.
622
663
 
623
664
  Credentials are read from the environment, never from configuration.
624
- `channels.maven.tokenEnv` names the variable, `MAVEN_TOKEN` by default; a configuration file gets committed, and a credential in a committed file is a credential that has leaked.
665
+ `channels.maven.tokenEnv` names the variable, `MAVEN_TOKEN` by default, and `channels.nuget.tokenEnv` the NuGet API key's, `NUGET_API_KEY` by default; a configuration file gets committed, and a credential in a committed file is a credential that has leaked.
625
666
 
626
667
  link:../docs/reference/how-publisher-and-subscriber-work-together.adoc#channels[How the Publisher and the Subscriber work together →] lists each channel's layout and options.
627
668
 
669
+ [#nuget]
670
+ === The `nuget` channel
671
+
672
+ It is published without the .NET SDK, as the Maven channel is published without Maven: a `.nupkg` is a ZIP archive the Publisher writes itself, and a push is one HTTP request.
673
+
674
+ [source,yaml]
675
+ ----
676
+ channels:
677
+ nuget:
678
+ idPrefix: Contoso.Contracts # required
679
+ # packageId: Contoso.Users # replaces the whole id, for every target
680
+ repository: build/nuget-feed # a directory, or a feed's service index URL
681
+ # tokenEnv: NUGET_API_KEY # the variable holding the API key, for a URL
682
+ # authors: Contoso API team # default: idPrefix
683
+ # license: MIT # an SPDX expression; default: UNLICENSED
684
+ # description: "{target} contract" # default: API description documents for {target}.
685
+ ----
686
+
687
+ *The package id* is `<idPrefix>.<target in PascalCase>`: target `user-account` with `idPrefix: Contoso.Contracts` is `Contoso.Contracts.UserAccount`, the dotted PascalCase .NET uses for namespaces, from a kebab-case target.
688
+ `packageId` replaces it.
689
+ An id NuGet would refuse -- anything but letters, digits and underscores in parts separated by single dots or dashes, or longer than 100 characters -- is refused before anything is written.
690
+
691
+ *The version* is the contract's, and a pre-release is a NuGet pre-release: `2.1.0-rc.1` is not picked by a floating range such as `2.1.*`, only when asked for.
692
+ A version with build metadata, such as `2.1.0+build.7`, is refused before anything is written: NuGet ignores `+metadata` when it identifies a package, so it would be the same package as every other build of `2.1.0`, and a feed refuses the second.
693
+
694
+ *A local feed* is a directory in the layout `nuget add` writes, which the NuGet documentation recommends over a flat folder:
695
+
696
+ ----
697
+ <repository>/<id, lower case>/<version>/
698
+ <id>.<version>.nupkg
699
+ <id>.<version>.nupkg.sha512
700
+ <id>.nuspec
701
+ manifest.json -- beside the package, so provenance is readable without unpacking it
702
+ ----
703
+
704
+ A .NET build uses it with `dotnet nuget add source <repository>`, or a `nuget.config` entry.
705
+
706
+ *A remote feed* is configured by its service index, the URL a feed's settings show and that ends in `index.json`, such as `https://api.nuget.org/v3/index.json`.
707
+ The Publisher reads the service index for its `PackagePublish/2.0.0` endpoint, then pushes the package there with one `PUT`, as `multipart/form-data`, with the key from the environment variable `tokenEnv` names in the `X-NuGet-ApiKey` header.
708
+ A feed that already has the version answers 409; the Publisher then reads the feed's copy back (its `PackageBaseAddress/3.0.0` resource) and succeeds when it is the same package, and fails when it is not, or cannot be read (<<idempotent,Publishing the same version again §>>).
709
+ Without the variable set, the push fails before anything is sent, naming it.
710
+ That is how nuget.org, GitHub Packages and self-hosted feeds such as BaGet take a push.
711
+ A feed that authenticates some other way -- Azure Artifacts, which expects a credential provider or a token in an `Authorization` header -- is not supported yet; only the local-HTTP-stub tests exercise the push, not a real feed.
712
+
713
+ *The package holds* the `.nuspec`, the two Open Packaging Conventions parts (`[Content_Types].xml` and `_rels/.rels`), and exactly what `pack` produced -- the documents and `manifest.json` -- extracted, never rebuilt, under `contracts/`:
714
+
715
+ ----
716
+ Contoso.Contracts.UserAccount.nuspec
717
+ [Content_Types].xml
718
+ _rels/.rels
719
+ contracts/asyncapi.yaml
720
+ contracts/manifest.json
721
+ contracts/openapi.yaml
722
+ ----
723
+
724
+ `contracts/` is the contract with the consumer, and changes only with a major version of the Publisher.
725
+ NuGet gives the folder no behaviour: nothing is copied into a project, nothing is compiled, and a restore reports no warning for a package with no assemblies.
726
+ Until a `.targets` file places and verifies the documents for a .NET build, a consumer reads them where restore put them, `~/.nuget/packages/<id, lower case>/<version>/contracts/` (or `$(NuGetPackageRoot)` in MSBuild), and verifies them against `contracts/manifest.json`.
727
+
728
+ The same packed archive always produces a byte-identical package: entries in a fixed order, a fixed timestamp.
729
+ Since `pack` is reproducible, publishing the same commit again produces the same package too, so a republish to a local feed can be checked as unchanged.
730
+
628
731
  [#why-registries]
629
- === Why npm and Maven rather than release assets alone
732
+ === Why npm, Maven and NuGet rather than release assets alone
630
733
 
631
734
  Publishing is passive.
632
735
  The producer releases `2.1.0` and nothing happens in an implementation repository until somebody looks.
633
736
 
634
- Renovate and Dependabot understand npm and Maven natively, so a contract bump arrives downstream as a pull request without any bespoke machinery.
737
+ Renovate and Dependabot understand npm, Maven and NuGet natively, so a contract bump arrives downstream as a pull request without any bespoke machinery.
635
738
  That is the practical argument for preferring them, and the reason to design the coordinates so that standard update bots can read them.
636
739
 
637
740
  [#pre-releases]
@@ -683,6 +786,8 @@ defaults:
683
786
  build:
684
787
  staging: build/staging
685
788
  dist: build/dist
789
+ # packages: build/packages # where pack and publish write archives; --out overrides
790
+ # split: build/split # where split writes its parts; --out overrides
686
791
  reports:
687
792
  lint: build/reports/lint
688
793
  lint:
@@ -699,6 +804,14 @@ channels:
699
804
  file:
700
805
  directory: build/publish
701
806
  clean: true # keep only the version being published
807
+ # Every channel that describes a package or a release takes `description`, with
808
+ # {target} and {version} filled in; default: API description documents for {target}.
809
+ # npm:
810
+ # releaseTag: latest # the dist-tag of a release; another keeps latest where it is
811
+ # prereleaseTag: next # the dist-tag of a pre-release
812
+ # github-release:
813
+ # title: "{target} {version}"
814
+ # notes: "..." # default: description
702
815
 
703
816
  targets:
704
817
  orders:
@@ -808,6 +921,8 @@ OpenAPI operation '<operationId>': <request body, or response <status>> (<fragme
808
921
  A body or response with no `content` at all, such as a `204`, has nothing to exemplify and is never reported.
809
922
 
810
923
  `lint.examples.asyncapi` and `lint.examples.openapi` each default to `warn`; `error` fails the build, `off` does not look.
924
+ The findings appear in the build log only: `lint` does not run this check, and there is no report file, so a job that only runs `lint` never sees them -- run `build`, and read its log.
925
+ With `error`, the build fails with `<n> operation(s) in <file> have no example (lint.examples.<kind> is error):`, followed by each finding.
811
926
  A library that has adopted the Microcks-based conformance emitter will usually want `lint.examples.asyncapi: error`: publishing a bundle its own toolchain cannot use is a real defect there, not merely a gap (link:../docs/reference/authoring-a-specification-library.adoc#give-every-operation-an-example[Give every operation an example →]).
812
927
 
813
928
  [#version-stamping]
@@ -1034,7 +1149,7 @@ Anything else is a bug; please report it.
1034
1149
  npm test
1035
1150
  ----
1036
1151
 
1037
- Covers every module and every command: placeholder substitution, version stamping, version files, configuration parsing, closures, packing, channels, splitting, and the scaffold.
1152
+ Covers every module and every command: placeholder substitution, version stamping, version files, configuration parsing, closures, packing, channels, splitting, aggregates, the example check, and the scaffold.
1038
1153
  `build.e2e.test.js` runs the real bundlers.
1039
1154
  The run fails below 90% of lines and functions, or 80% of branches.
1040
1155
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@arc-e-tect/api-only-publisher",
3
- "version": "0.9.0",
3
+ "version": "0.11.0",
4
4
  "description": "Builds, packs and publishes API description documents from a library of reusable fragments.",
5
5
  "license": "MIT",
6
6
  "author": "Arc-E-Tect",
@@ -0,0 +1,7 @@
1
+ "use strict";
2
+
3
+ // The error every distribution channel throws, in a module of its own so that a channel
4
+ // implemented outside channels.js can throw it without requiring channels.js back.
5
+ class ChannelError extends Error {}
6
+
7
+ module.exports = { ChannelError };
package/src/channels.js CHANGED
@@ -13,8 +13,17 @@ const path = require("path");
13
13
  const { execFileSync } = require("child_process");
14
14
 
15
15
  const { isPrerelease, npmDistTag } = require("./version-policy");
16
+ const { ChannelError } = require("./channel-error");
17
+ const { publishNuget } = require("./nuget");
18
+
19
+ /** What every package and release says it is, unless `description` says otherwise. */
20
+ const DEFAULT_DESCRIPTION = "API description documents for {target}.";
21
+
22
+ /** `template` with {target} and {version} filled in, as tagFormat has always been. */
23
+ function format(template, manifest) {
24
+ return String(template).replace(/\{target\}/g, manifest.target).replace(/\{version\}/g, manifest.version);
25
+ }
16
26
 
17
- class ChannelError extends Error {}
18
27
 
19
28
  /**
20
29
  * `file` -- publish to a local directory.
@@ -43,7 +52,7 @@ function publishFile(archive, manifest, options, log) {
43
52
  return { location: archiveDest };
44
53
  }
45
54
 
46
- function pom(groupId, artifactId, version, packaging) {
55
+ function pom(groupId, artifactId, version, packaging, description = `API description documents for ${artifactId}.`) {
47
56
  return `<?xml version="1.0" encoding="UTF-8"?>
48
57
  <project xmlns="http://maven.apache.org/POM/4.0.0"
49
58
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
@@ -53,11 +62,16 @@ function pom(groupId, artifactId, version, packaging) {
53
62
  <artifactId>${artifactId}</artifactId>
54
63
  <version>${version}</version>
55
64
  <packaging>${packaging}</packaging>
56
- <description>API description documents for ${artifactId}.</description>
65
+ <description>${description.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;")}</description>
57
66
  </project>
58
67
  `;
59
68
  }
60
69
 
70
+ /** The POM's description: configured, or the POM's own default. */
71
+ function pomDescription(options, manifest) {
72
+ return options.description ? format(options.description, manifest) : undefined;
73
+ }
74
+
61
75
  /**
62
76
  * `maven` -- publish into a Maven repository layout.
63
77
  *
@@ -90,7 +104,7 @@ function publishMaven(archive, manifest, options, log) {
90
104
  const base = `${artifactId}-${version}`;
91
105
  const artifactDest = path.join(dir, `${base}.${extension}`);
92
106
  fs.copyFileSync(archive, artifactDest);
93
- fs.writeFileSync(path.join(dir, `${base}.pom`), pom(groupId, artifactId, version, extension));
107
+ fs.writeFileSync(path.join(dir, `${base}.pom`), pom(groupId, artifactId, version, extension, pomDescription(options, manifest)));
94
108
 
95
109
  // The manifest travels beside the artifact as well as inside it, so a
96
110
  // consumer can read provenance without unpacking anything.
@@ -114,7 +128,7 @@ function publishNpm(archive, manifest, options, log) {
114
128
  const scope = options.scope;
115
129
  const name = scope ? `${scope}/${manifest.target}` : (options.namePrefix || "") + manifest.target;
116
130
  const version = manifest.version;
117
- const distTag = npmDistTag(version);
131
+ const distTag = npmDistTag(version, options);
118
132
 
119
133
  const workDir = fs.mkdtempSync(path.join(os.tmpdir(), "api-only-npm-"));
120
134
  try {
@@ -125,7 +139,7 @@ function publishNpm(archive, manifest, options, log) {
125
139
  fs.writeFileSync(path.join(workDir, "package.json"), JSON.stringify({
126
140
  name,
127
141
  version,
128
- description: `API description documents for ${manifest.target}.`,
142
+ description: format(options.description || DEFAULT_DESCRIPTION, manifest),
129
143
  license: options.license || "UNLICENSED",
130
144
  files: fs.readdirSync(workDir).filter((f) => f !== "package.json").sort(),
131
145
  // npm defaults a scoped package to restricted. A contract exists to be
@@ -150,8 +164,23 @@ function publishNpm(archive, manifest, options, log) {
150
164
  return { location, distTag, name };
151
165
  }
152
166
 
153
- const args = ["publish", "--tag", distTag];
154
- if (options.registry) args.push("--registry", options.registry);
167
+ // Idempotent: a version the registry already has is left alone when it is the
168
+ // same package, and refused when it is not -- npm never replaces a version.
169
+ const registry = options.registry ? ["--registry", options.registry] : [];
170
+ const local = JSON.parse(execFileSync("npm", ["pack", "--dry-run", "--json"],
171
+ { cwd: workDir, encoding: "utf8", stdio: "pipe" }))[0].integrity;
172
+ const published = npmPublishedIntegrity(name, version, registry);
173
+ if (published === local) {
174
+ log(`-- npm ${name}@${version} is already published, identical; nothing to do`);
175
+ return { location: name + "@" + version, distTag, name, alreadyPublished: true };
176
+ }
177
+ if (published) {
178
+ throw new ChannelError(
179
+ `npm already has ${name}@${version}, and it is not this package (${published}, not ${local}). ` +
180
+ "A published version is never replaced; publish the change as a new version.");
181
+ }
182
+
183
+ const args = ["publish", "--tag", distTag, ...registry];
155
184
  // A signed attestation tying this tarball to the commit and workflow run
156
185
  // that produced it. A tool whose whole purpose is making provenance
157
186
  // auditable should not ask to be taken on trust itself. Requires a public
@@ -166,6 +195,19 @@ function publishNpm(archive, manifest, options, log) {
166
195
  }
167
196
  }
168
197
 
198
+ /** The integrity npm records for `name@version`, or null when it has no such version. */
199
+ function npmPublishedIntegrity(name, version, registry) {
200
+ try {
201
+ const out = execFileSync("npm", ["view", `${name}@${version}`, "dist.integrity", ...registry],
202
+ { encoding: "utf8", stdio: "pipe" }).trim();
203
+ return out || null;
204
+ } catch (error) {
205
+ const output = `${error.stdout || ""}${error.stderr || ""}`;
206
+ if (/E404/.test(output)) return null;
207
+ throw new ChannelError(`could not ask npm whether ${name}@${version} is published: ${output.trim() || error.message}`);
208
+ }
209
+ }
210
+
169
211
  /**
170
212
  * `github-release` -- attach the archive to a GitHub release.
171
213
  *
@@ -181,9 +223,7 @@ function publishGithubRelease(archive, manifest, options, log) {
181
223
  const repository = options.repository;
182
224
  if (!repository) throw new ChannelError("the github-release channel requires a repository");
183
225
 
184
- const tag = (options.tagFormat || "{target}-v{version}")
185
- .replace(/\{target\}/g, manifest.target)
186
- .replace(/\{version\}/g, manifest.version);
226
+ const tag = format(options.tagFormat || "{target}-v{version}", manifest);
187
227
 
188
228
  const gh = (args) => execFileSync("gh", args, { encoding: "utf8", stdio: "pipe" });
189
229
 
@@ -196,15 +236,37 @@ function publishGithubRelease(archive, manifest, options, log) {
196
236
 
197
237
  if (!exists) {
198
238
  const args = ["release", "create", tag, "--repo", repository,
199
- "--title", `${manifest.target} ${manifest.version}`,
200
- "--notes", `API description documents for ${manifest.target}.`];
239
+ "--title", format(options.title || "{target} {version}", manifest),
240
+ "--notes", format(options.notes || options.description || DEFAULT_DESCRIPTION, manifest)];
201
241
  // A pre-release is marked as one, so that "latest release" never resolves
202
242
  // to a contract that is not finished.
203
243
  if (isPrerelease(manifest.version)) args.push("--prerelease");
204
244
  gh(args);
205
245
  }
206
246
 
207
- gh(["release", "upload", tag, archive, "--repo", repository, "--clobber"]);
247
+ // Idempotent, as the registries are: an asset the release already has is left alone
248
+ // when it is the same bytes, and refused when it is not. Never --clobber: that would
249
+ // replace a published contract under the same version.
250
+ const asset = path.basename(archive);
251
+ const assets = exists
252
+ ? gh(["release", "view", tag, "--repo", repository, "--json", "assets", "--jq", ".assets[].name"]).split("\n").map((a) => a.trim())
253
+ : [];
254
+ if (assets.includes(asset)) {
255
+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), "api-only-gh-"));
256
+ try {
257
+ gh(["release", "download", tag, "--repo", repository, "--pattern", asset, "--dir", dir]);
258
+ if (fs.readFileSync(path.join(dir, asset)).equals(fs.readFileSync(archive))) {
259
+ log(`-- ${repository} release ${tag} already has ${asset}, identical; nothing to do`);
260
+ return { location: `${repository}@${tag}`, tag, alreadyPublished: true };
261
+ }
262
+ } finally {
263
+ fs.rmSync(dir, { recursive: true, force: true });
264
+ }
265
+ throw new ChannelError(
266
+ `${repository} release ${tag} already has ${asset}, and it is not this archive. ` +
267
+ "A published version is never replaced; publish the change as a new version.");
268
+ }
269
+ gh(["release", "upload", tag, archive, "--repo", repository]);
208
270
  log(`-- Published ${manifest.target} ${manifest.version} to ${repository} release ${tag}`);
209
271
  return { location: `${repository}@${tag}`, tag };
210
272
  }
@@ -241,6 +303,18 @@ async function putFile(url, body, token, contentType) {
241
303
  }
242
304
  }
243
305
 
306
+ /** What a Maven repository holds at `url`, or null when it holds nothing there. */
307
+ async function getFile(url, token) {
308
+ const response = await fetch(url, { headers: { "Authorization": `Bearer ${token}` } });
309
+ if (response.status === 404) return null;
310
+ if (!response.ok) {
311
+ throw new ChannelError(`GET ${url} failed: ${response.status} ${response.statusText}. ` +
312
+ (response.status === 401 || response.status === 403
313
+ ? "Check the token in the environment variable named by channels.maven.tokenEnv." : ""));
314
+ }
315
+ return Buffer.from(await response.arrayBuffer());
316
+ }
317
+
244
318
  function publishMavenRemote(archive, manifest, options, log) {
245
319
  const tokenEnv = options.tokenEnv || "MAVEN_TOKEN";
246
320
  const token = process.env[tokenEnv];
@@ -258,24 +332,42 @@ function publishMavenRemote(archive, manifest, options, log) {
258
332
  `${options.groupId.split(".").join("/")}/${artifactId}/${version}/${artifactId}-${version}`;
259
333
 
260
334
  const work = (async () => {
261
- await putFile(`${base}.${extension}`, fs.readFileSync(archive), token, "application/octet-stream");
335
+ // Idempotent: an artifact the repository already has is left alone when it is
336
+ // the same bytes, and refused when it is not.
337
+ const bytes = fs.readFileSync(archive);
338
+ const existing = await getFile(`${base}.${extension}`, token);
339
+ if (existing) {
340
+ if (existing.equals(bytes)) return true;
341
+ throw new ChannelError(
342
+ `${options.repository} already has ${options.groupId}:${artifactId}:${version}, and it is not this archive. ` +
343
+ "A published version is never replaced; publish the change as a new version.");
344
+ }
345
+ await putFile(`${base}.${extension}`, bytes, token, "application/octet-stream");
262
346
  await putFile(`${base}.pom`,
263
- Buffer.from(pom(options.groupId, artifactId, version, extension), "utf8"),
347
+ Buffer.from(pom(options.groupId, artifactId, version, extension, pomDescription(options, manifest)), "utf8"),
264
348
  token, "application/xml");
265
349
  await putFile(`${base}-manifest.json`,
266
350
  Buffer.from(JSON.stringify(manifest, null, 2) + "\n", "utf8"),
267
351
  token, "application/json");
352
+ return false;
268
353
  })();
269
354
 
270
355
  // The CLI is synchronous throughout; surfacing the promise here would make
271
356
  // every caller async for one channel's benefit.
272
- return work.then(() => {
273
- log(`-- Deployed ${options.groupId}:${artifactId}:${version} to ${options.repository}`);
274
- return { location: `${base}.${extension}`, coordinates: `${options.groupId}:${artifactId}:${version}@${extension}` };
357
+ return work.then((alreadyPublished) => {
358
+ log(alreadyPublished
359
+ ? `-- ${options.groupId}:${artifactId}:${version} is already in ${options.repository}, identical; nothing to do`
360
+ : `-- Deployed ${options.groupId}:${artifactId}:${version} to ${options.repository}`);
361
+ return {
362
+ location: `${base}.${extension}`, coordinates: `${options.groupId}:${artifactId}:${version}@${extension}`,
363
+ ...(alreadyPublished ? { alreadyPublished } : {}),
364
+ };
275
365
  });
276
366
  }
277
367
 
278
- const CHANNELS = { file: publishFile, maven: publishMaven, npm: publishNpm, "github-release": publishGithubRelease };
368
+ const CHANNELS = {
369
+ file: publishFile, maven: publishMaven, npm: publishNpm, "github-release": publishGithubRelease, nuget: publishNuget,
370
+ };
279
371
 
280
372
  function publish(archive, manifest, channel, options, log = () => {}) {
281
373
  const handler = CHANNELS[channel];
@@ -287,4 +379,4 @@ function publish(archive, manifest, channel, options, log = () => {}) {
287
379
  return handler(archive, manifest, options || {}, log);
288
380
  }
289
381
 
290
- module.exports = { publish, CHANNELS, ChannelError, pom };
382
+ module.exports = { publish, CHANNELS, ChannelError, pom, format, DEFAULT_DESCRIPTION };
package/src/cli.js CHANGED
@@ -38,7 +38,7 @@ Usage:
38
38
  api-only-publisher changed --since <ref>
39
39
  api-only-publisher pack [--pre-release <ids>] [--target <name>]... [--out <dir>]
40
40
  api-only-publisher publish [--pre-release <ids>] [--target <name>]... [--channel <name>]... [--out <dir>]
41
- api-only-publisher split --out <dir> [--by kind|target]
41
+ api-only-publisher split [--out <dir>] [--by kind|target]
42
42
 
43
43
  Options:
44
44
  --target <name> Restrict to one target; repeat for several. Default: all.
@@ -60,7 +60,8 @@ Options:
60
60
  --portfolio-paths <target-prefix|none>, --portfolio-location <dir>
61
61
  config only: the value of that question; a flag always wins.
62
62
  --since <ref> changed only: the git ref to compare the working tree against.
63
- --out <dir> pack/publish/split: where to write.
63
+ --out <dir> pack/publish/split: where to write, instead of build.packages
64
+ or build.split in apionly.yaml.
64
65
  --channel <name> publish only: repeat for several. Default: every configured channel.
65
66
  --by <kind|target> split only: the boundary to split along. Default: kind.
66
67
  -C <dir> Run as if started in <dir>.
@@ -407,7 +408,7 @@ async function main(argv, io = {}) {
407
408
  return 0;
408
409
  }
409
410
  case "pack": {
410
- const outDir = path.resolve(options.dir, options.out || "build/packages");
411
+ const outDir = options.out ? path.resolve(options.dir, options.out) : config.packagesDir();
411
412
  const shipped = shippedTargets(config, targets);
412
413
  const versions = versionsOf(config, shipped, options.preRelease);
413
414
  const closures = forTargets(config, undefined, shipped);
@@ -430,7 +431,7 @@ async function main(argv, io = {}) {
430
431
  return 0;
431
432
  }
432
433
  case "publish": {
433
- const outDir = path.resolve(options.dir, options.out || "build/packages");
434
+ const outDir = options.out ? path.resolve(options.dir, options.out) : config.packagesDir();
434
435
  const configured = config.channels || {};
435
436
  const names = options.channels || Object.keys(configured);
436
437
  if (names.length === 0) {
@@ -460,7 +461,7 @@ async function main(argv, io = {}) {
460
461
  return 0;
461
462
  }
462
463
  case "split": {
463
- const outDir = path.resolve(options.dir, options.out || "build/split");
464
+ const outDir = options.out ? path.resolve(options.dir, options.out) : config.splitDir();
464
465
  prepare(config);
465
466
  const parts = split(config, { by: options.by, outDir, log });
466
467
  const duplicated = parts.reduce((n, p) => n + p.imported.length, 0);
package/src/config.js CHANGED
@@ -25,7 +25,7 @@ const SOURCES_KEYS = ["root", "openapi", "asyncapi"];
25
25
  const DEFAULTS_KEYS = ["openapi", "asyncapi", "placeholders"];
26
26
  const DEFAULTS_KIND_KEYS = { openapi: ["lint", "outputName", "fragmentPaths"], asyncapi: ["lint", "outputName", "fragmentPaths"] };
27
27
  const PLACEHOLDERS_KEYS = ["strict"];
28
- const BUILD_KEYS = ["staging", "dist"];
28
+ const BUILD_KEYS = ["staging", "dist", "packages", "split"];
29
29
  const REPORTS_KEYS = ["lint"];
30
30
  const LINT_KEYS = ["unreferenced", "examples"];
31
31
  const LINT_EXAMPLES_KEYS = ["openapi", "asyncapi"];
@@ -255,6 +255,14 @@ function load(configPath) {
255
255
  stagingDir(kind) {
256
256
  return path.join(this.stagingRoot(kind), requireString(this.sources[kind], `sources.${kind}`));
257
257
  },
258
+ // Where pack and publish write archives, and split its parts, unless --out says
259
+ // otherwise; relative to this configuration, as every other path here is.
260
+ packagesDir() {
261
+ return path.resolve(this.root, this.build.packages || "build/packages");
262
+ },
263
+ splitDir() {
264
+ return path.resolve(this.root, this.build.split || "build/split");
265
+ },
258
266
  distDir(target) {
259
267
  const dist = this.build.dist || "dist";
260
268
  return path.resolve(this.root, dist, target);
package/src/nuget.js ADDED
@@ -0,0 +1,276 @@
1
+ "use strict";
2
+
3
+ // `nuget` -- publish as a NuGet package, for a .NET build.
4
+ //
5
+ // Written directly rather than with nuget.exe or `dotnet pack`, as the Maven channel is
6
+ // written without mvn: a specification repository needs no .NET SDK to reach a .NET
7
+ // audience. A .nupkg is a ZIP archive of a .nuspec, two Open Packaging Conventions parts
8
+ // and the content, and a push is one multipart HTTP PUT.
9
+ //
10
+ // The package carries what `pack` produced, extracted, never rebuilt, under `contracts/`:
11
+ // a folder NuGet gives no behaviour, so the path is the whole contract with a consumer
12
+ // that reads it from its global packages folder today, and the later .targets that
13
+ // places and verifies it is free to decide how it is delivered. That path does not change
14
+ // without a major version of the Publisher.
15
+ //
16
+ // Renovate and Dependabot understand NuGet, so a contract bump reaches a .NET consumer as
17
+ // a pull request, as it does an npm or Maven one.
18
+
19
+ const crypto = require("crypto");
20
+ const fs = require("fs");
21
+ const os = require("os");
22
+ const path = require("path");
23
+ const { execFileSync } = require("child_process");
24
+
25
+ const { parse, isPrerelease } = require("./version-policy");
26
+ const { zip } = require("./zip");
27
+ const { ChannelError } = require("./channel-error");
28
+
29
+ /** Where the documents and manifest.json sit inside the package. */
30
+ const CONTENT_FOLDER = "contracts";
31
+
32
+ // NuGet's own rule (NuGet.Client, PackageIdValidator), and nuget.org's length limit.
33
+ const ID = /^\w+([.-]\w+)*$/;
34
+ const ID_MAX_LENGTH = 100;
35
+
36
+ const PUBLISH_RESOURCE = "PackagePublish/2.0.0";
37
+ const CONTENT_RESOURCE = "PackageBaseAddress/3.0.0";
38
+
39
+ function pascalCase(target) {
40
+ return target.split(/[-_.]/).filter(Boolean)
41
+ .map((part) => part[0].toUpperCase() + part.slice(1)).join("");
42
+ }
43
+
44
+ /**
45
+ * The package id for a target: `packageId` when configured, otherwise
46
+ * `<idPrefix>.<target in PascalCase>`, as the Maven channel takes `groupId` and an
47
+ * optional `artifactId`. Refused, before anything is written, when NuGet would refuse it.
48
+ */
49
+ function packageId(target, options) {
50
+ let id = options.packageId;
51
+ if (!id) {
52
+ if (!options.idPrefix) throw new ChannelError("the nuget channel requires an idPrefix, such as Contoso.Contracts");
53
+ id = `${options.idPrefix}.${pascalCase(target)}`;
54
+ }
55
+ if (!ID.test(id) || id.length > ID_MAX_LENGTH) {
56
+ throw new ChannelError(
57
+ `'${id}' is not a valid NuGet package id: letters, digits and underscores, in parts ` +
58
+ `separated by single dots or dashes, at most ${ID_MAX_LENGTH} characters. ` +
59
+ "Set channels.nuget.idPrefix, or channels.nuget.packageId, to one that is.");
60
+ }
61
+ return id;
62
+ }
63
+
64
+ /** Refuses a version NuGet would not tell apart from another. */
65
+ function checkVersion(version) {
66
+ if (parse(version).build) {
67
+ throw new ChannelError(
68
+ `version '${version}' carries build metadata, which NuGet ignores when it identifies a ` +
69
+ "package: it would be the same package as every other build of " +
70
+ `${version.split("+")[0]}, and a feed refuses the second. Publish it without the +metadata.`);
71
+ }
72
+ }
73
+
74
+ function escape(text) {
75
+ return String(text).replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;").replace(/"/g, "&quot;");
76
+ }
77
+
78
+ /** The .nuspec: what the npm channel writes into package.json, in NuGet's terms. */
79
+ function nuspec(id, manifest, options) {
80
+ const source = manifest.source || {};
81
+ const repository = source.repository
82
+ ? ` <repository type="git" url="${escape(source.repository)}"` +
83
+ (source.commit ? ` commit="${escape(source.commit)}"` : "") + " />\n"
84
+ : "";
85
+ return '<?xml version="1.0" encoding="utf-8"?>\n' +
86
+ '<package xmlns="http://schemas.microsoft.com/packaging/2013/05/nuspec.xsd">\n' +
87
+ " <metadata>\n" +
88
+ ` <id>${escape(id)}</id>\n` +
89
+ ` <version>${escape(manifest.version)}</version>\n` +
90
+ ` <authors>${escape(options.authors || options.idPrefix || id)}</authors>\n` +
91
+ ` <description>${escape((options.description || "API description documents for {target}.")
92
+ .replace(/\{target\}/g, manifest.target).replace(/\{version\}/g, manifest.version))}</description>\n` +
93
+ ` <license type="expression">${escape(options.license || "UNLICENSED")}</license>\n` +
94
+ repository +
95
+ " </metadata>\n" +
96
+ "</package>\n";
97
+ }
98
+
99
+ function contentTypes(names) {
100
+ // Not path.extname: it reads "_rels/.rels" as a dotfile with no extension, and OPC needs "rels".
101
+ const extension = (name) => {
102
+ const base = path.posix.basename(name);
103
+ return base.includes(".") ? base.slice(base.lastIndexOf(".") + 1) : "";
104
+ };
105
+ const extensions = [...new Set(names.map(extension).filter(Boolean))].sort();
106
+ const types = extensions.map((extension) =>
107
+ `<Default Extension="${escape(extension)}" ContentType="${extension === "rels"
108
+ ? "application/vnd.openxmlformats-package.relationships+xml" : "application/octet"}" />`);
109
+ return '<?xml version="1.0" encoding="utf-8"?>' +
110
+ `<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">${types.join("")}</Types>`;
111
+ }
112
+
113
+ function relationships(nuspecName) {
114
+ return '<?xml version="1.0" encoding="utf-8"?>' +
115
+ '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">' +
116
+ `<Relationship Type="http://schemas.microsoft.com/packaging/2010/07/manifest" Target="/${escape(nuspecName)}" Id="Rnuspec" />` +
117
+ "</Relationships>";
118
+ }
119
+
120
+ /**
121
+ * The .nupkg bytes: the .nuspec, the packaging parts, then everything the packed archive
122
+ * holds, extracted and sorted. The same archive always yields the same bytes.
123
+ */
124
+ function nupkg(archive, id, manifest, options) {
125
+ const work = fs.mkdtempSync(path.join(os.tmpdir(), "api-only-nuget-"));
126
+ try {
127
+ execFileSync("tar", ["-xzf", archive, "-C", work], { encoding: "utf8" });
128
+ const content = fs.readdirSync(work).sort().map((name) => ({
129
+ name: `${CONTENT_FOLDER}/${name}`, data: fs.readFileSync(path.join(work, name)),
130
+ }));
131
+ const nuspecName = `${id}.nuspec`;
132
+ const rels = "_rels/.rels";
133
+ const names = [nuspecName, rels, ...content.map((entry) => entry.name)];
134
+ return {
135
+ bytes: zip([
136
+ { name: nuspecName, data: Buffer.from(nuspec(id, manifest, options), "utf8") },
137
+ { name: "[Content_Types].xml", data: Buffer.from(contentTypes(names), "utf8") },
138
+ { name: rels, data: Buffer.from(relationships(nuspecName), "utf8") },
139
+ ...content,
140
+ ]),
141
+ nuspec: nuspec(id, manifest, options),
142
+ };
143
+ } finally {
144
+ fs.rmSync(work, { recursive: true, force: true });
145
+ }
146
+ }
147
+
148
+ /**
149
+ * Write the package into a local feed, in the hierarchical layout `nuget add` writes --
150
+ * `<id>/<version>/<id>.<version>.nupkg`, lower-cased, with the .sha512 and .nuspec beside
151
+ * it -- plus manifest.json, so provenance is readable without unpacking, as in Maven.
152
+ */
153
+ function publishLocal(archive, manifest, options, id, log) {
154
+ const { bytes, nuspec: spec } = nupkg(archive, id, manifest, options);
155
+ const lowerId = id.toLowerCase();
156
+ const lowerVersion = String(manifest.version).toLowerCase();
157
+ const feed = path.resolve(options.baseDir || ".", options.repository);
158
+ const dir = path.join(feed, lowerId, lowerVersion);
159
+ fs.mkdirSync(dir, { recursive: true });
160
+
161
+ const file = path.join(dir, `${lowerId}.${lowerVersion}.nupkg`);
162
+ fs.writeFileSync(file, bytes);
163
+ fs.writeFileSync(`${file}.sha512`, crypto.createHash("sha512").update(bytes).digest("base64"));
164
+ fs.writeFileSync(path.join(dir, `${lowerId}.nuspec`), spec);
165
+ fs.writeFileSync(path.join(dir, "manifest.json"), JSON.stringify(manifest, null, 2) + "\n");
166
+
167
+ log(`-- Published NuGet ${id} ${manifest.version} to ${feed}`);
168
+ return { location: file, id, version: manifest.version, prerelease: isPrerelease(manifest.version) };
169
+ }
170
+
171
+ /** The resources a feed's service index lists. */
172
+ async function feedResources(index) {
173
+ const response = await fetch(index);
174
+ if (!response.ok) {
175
+ throw new ChannelError(`could not read the NuGet service index at ${index}: ${response.status} ${response.statusText}`);
176
+ }
177
+ const resources = (await response.json()).resources || [];
178
+ const find = (type) => (resources.find((r) => r["@type"] === type) || {})["@id"] || null;
179
+ const publish = find(PUBLISH_RESOURCE);
180
+ if (!publish) {
181
+ throw new ChannelError(`the NuGet feed at ${index} has no ${PUBLISH_RESOURCE} resource, so it cannot be pushed to`);
182
+ }
183
+ return { publish, content: find(CONTENT_RESOURCE) };
184
+ }
185
+
186
+ /**
187
+ * Whether the feed's copy of this version is these bytes: true or false, or null when the
188
+ * feed offers no way to read it back.
189
+ */
190
+ async function sameAsPublished(content, id, version, bytes, key) {
191
+ if (!content) return null;
192
+ const lowerId = id.toLowerCase();
193
+ const lowerVersion = String(version).toLowerCase();
194
+ const url = `${content.replace(/\/?$/, "/")}${lowerId}/${lowerVersion}/${lowerId}.${lowerVersion}.nupkg`;
195
+ const response = await fetch(url, { headers: { "X-NuGet-ApiKey": key } });
196
+ if (!response.ok) return null;
197
+ return Buffer.from(await response.arrayBuffer()).equals(bytes);
198
+ }
199
+
200
+ /**
201
+ * Push the package to a remote feed: one PUT to the endpoint the service index names,
202
+ * the package as the first part of a multipart body, the key in X-NuGet-ApiKey.
203
+ *
204
+ * The key is read from the environment, never from configuration, as the Maven channel
205
+ * reads its token.
206
+ */
207
+ function publishRemote(archive, manifest, options, id, log) {
208
+ const tokenEnv = options.tokenEnv || "NUGET_API_KEY";
209
+ const key = process.env[tokenEnv];
210
+ if (!key) {
211
+ throw new ChannelError(
212
+ `no API key in $${tokenEnv}, which channels.nuget.tokenEnv names. ` +
213
+ "Remote publication needs one; set it in the release job's environment.");
214
+ }
215
+ const { bytes } = nupkg(archive, id, manifest, options);
216
+
217
+ return (async () => {
218
+ const { publish: endpoint, content } = await feedResources(options.repository);
219
+ const boundary = `api-only-${crypto.createHash("sha256").update(bytes).digest("hex").slice(0, 32)}`;
220
+ const body = Buffer.concat([
221
+ Buffer.from(`--${boundary}\r\nContent-Disposition: form-data; name="package"; filename="package.nupkg"\r\n` +
222
+ "Content-Type: application/octet-stream\r\n\r\n", "utf8"),
223
+ bytes,
224
+ Buffer.from(`\r\n--${boundary}--\r\n`, "utf8"),
225
+ ]);
226
+ const response = await fetch(endpoint, {
227
+ method: "PUT",
228
+ headers: {
229
+ "X-NuGet-ApiKey": key,
230
+ "Content-Type": `multipart/form-data; boundary=${boundary}`,
231
+ "Content-Length": String(body.length),
232
+ },
233
+ body,
234
+ });
235
+ if (response.status === 409) {
236
+ // Idempotent: the same package again is success, a different one is not.
237
+ const same = await sameAsPublished(content, id, manifest.version, bytes, key);
238
+ if (same) {
239
+ log(`-- NuGet ${id} ${manifest.version} is already published, identical; nothing to do`);
240
+ return {
241
+ location: endpoint, id, version: manifest.version, prerelease: isPrerelease(manifest.version),
242
+ alreadyPublished: true,
243
+ };
244
+ }
245
+ throw new ChannelError(`the NuGet feed already has ${id} ${manifest.version}` +
246
+ (same === false ? ", and it is not this package" : ", and it cannot be read back to compare") +
247
+ "; a published version is never replaced, so publish the change as a new version");
248
+ }
249
+ if (!response.ok) {
250
+ throw new ChannelError(`the NuGet feed refused the package: ${response.status} ${response.statusText}. ` +
251
+ (response.status === 401 || response.status === 403
252
+ ? "Check the key in the environment variable named by channels.nuget.tokenEnv." : ""));
253
+ }
254
+ log(`-- Pushed NuGet ${id} ${manifest.version} to ${endpoint}`);
255
+ return { location: endpoint, id, version: manifest.version, prerelease: isPrerelease(manifest.version) };
256
+ })();
257
+ }
258
+
259
+ /**
260
+ * The channel: a path writes into a local feed, an http(s) URL -- the feed's service
261
+ * index -- pushes to a remote one. Everything that can be refused is refused before
262
+ * anything is written or sent.
263
+ */
264
+ function publishNuget(archive, manifest, options, log) {
265
+ const id = packageId(manifest.target, options);
266
+ checkVersion(manifest.version);
267
+ if (!options.repository) {
268
+ throw new ChannelError("the nuget channel requires a repository: a directory for a local feed, or a feed's service index URL");
269
+ }
270
+ if (/^https?:\/\//.test(options.repository)) {
271
+ return publishRemote(archive, manifest, options, id, log);
272
+ }
273
+ return publishLocal(archive, manifest, options, id, log);
274
+ }
275
+
276
+ module.exports = { publishNuget, packageId, nuspec, checkVersion, CONTENT_FOLDER };
package/src/pack.js CHANGED
@@ -11,6 +11,8 @@ const crypto = require("crypto");
11
11
  const { execFileSync } = require("child_process");
12
12
  const YAML = require("yaml");
13
13
 
14
+ const { tgz } = require("./tar");
15
+
14
16
  const MANIFEST_NAME = "manifest.json";
15
17
  const MANIFEST_SCHEMA_VERSION = 1;
16
18
 
@@ -42,7 +44,31 @@ function gitInfo(cwd) {
42
44
  * one version to differ by a line ending, which surfaces months later as an
43
45
  * unexplainable verify failure.
44
46
  */
45
- function manifest(config, target, { version, files, closureSha256, producedAt = new Date() }) {
47
+ /**
48
+ * When the documents were produced: SOURCE_DATE_EPOCH when it is set -- the
49
+ * reproducible-builds convention, and the way to choose it -- otherwise the time of the
50
+ * commit being packed, and the current time only outside a git repository. Never simply
51
+ * "now" inside one: packing the same commit twice has to give the same manifest, and so
52
+ * the same archive, or no channel could publish the same version twice idempotently.
53
+ */
54
+ function producedAtFor(cwd) {
55
+ const epoch = process.env.SOURCE_DATE_EPOCH;
56
+ if (epoch !== undefined && epoch !== "") {
57
+ if (!/^\d+$/.test(epoch)) {
58
+ throw new PackError(`SOURCE_DATE_EPOCH must be a whole number of seconds since 1970, not '${epoch}'`);
59
+ }
60
+ return new Date(Number(epoch) * 1000);
61
+ }
62
+ try {
63
+ const seconds = execFileSync("git", ["log", "-1", "--format=%ct"], { cwd, encoding: "utf8", stdio: ["ignore", "pipe", "ignore"] }).trim();
64
+ if (/^\d+$/.test(seconds)) return new Date(Number(seconds) * 1000);
65
+ } catch {
66
+ // Not a git repository, or one with no commits yet.
67
+ }
68
+ return new Date();
69
+ }
70
+
71
+ function manifest(config, target, { version, files, closureSha256, producedAt = producedAtFor(config.root) }) {
46
72
  if (!version) throw new PackError(`target '${target}': pack requires a version`);
47
73
 
48
74
  const source = gitInfo(config.root);
@@ -111,13 +137,9 @@ function pack(config, target, { version, closureSha256, outDir, log = () => {} }
111
137
 
112
138
  // Deterministic archive: sorted entry names, and no mtime/owner noise, so the
113
139
  // same inputs produce the same bytes on any machine.
114
- const entries = documents.map((d) => path.basename(d)).concat([MANIFEST_NAME]).sort();
115
- execFileSync("tar", [
116
- "-czf", archive,
117
- "-C", distDir,
118
- "--numeric-owner",
119
- ...entries,
120
- ], { encoding: "utf8" });
140
+ const entries = documents.map((d) => path.basename(d)).concat([MANIFEST_NAME]).sort()
141
+ .map((name) => ({ name, data: fs.readFileSync(path.join(distDir, name)) }));
142
+ fs.writeFileSync(archive, tgz(entries, Math.floor(Date.parse(data.producedAt) / 1000)));
121
143
 
122
144
  log(`-- Packed ${path.basename(archive)} (${data.files.length} document(s))`);
123
145
  return { archive, manifest: data, manifestPath };
package/src/tar.js ADDED
@@ -0,0 +1,66 @@
1
+ "use strict";
2
+
3
+ // A reproducible .tgz writer, for the archive `pack` produces.
4
+ //
5
+ // The system tar records each file's modification time, owner and permissions, and gzip
6
+ // records when it ran, so packing the same documents twice gave two archives. GNU and BSD
7
+ // tar do not agree on the flags that would pin those, so the archive is written here: a
8
+ // ustar header per file, with a fixed mode, no owner, and a modification time the caller
9
+ // chooses, then gzip with no timestamp and a fixed operating-system byte.
10
+
11
+ const zlib = require("zlib");
12
+
13
+ class TarError extends Error {}
14
+
15
+ const BLOCK = 512;
16
+
17
+ function octal(value, length) {
18
+ return value.toString(8).padStart(length - 1, "0") + "\0";
19
+ }
20
+
21
+ function header(name, size, mtime) {
22
+ const nameBytes = Buffer.from(name, "utf8");
23
+ if (nameBytes.length > 100 || name.includes("/") || name === "" || name === "." || name === "..") {
24
+ throw new TarError(`${JSON.stringify(name)} cannot be an entry of the archive: a plain file name of at most 100 bytes`);
25
+ }
26
+ const block = Buffer.alloc(BLOCK, 0);
27
+ nameBytes.copy(block, 0);
28
+ block.write(octal(0o644, 8), 100, "ascii");
29
+ block.write(octal(0, 8), 108, "ascii");
30
+ block.write(octal(0, 8), 116, "ascii");
31
+ block.write(octal(size, 12), 124, "ascii");
32
+ block.write(octal(mtime, 12), 136, "ascii");
33
+ block.write(" ", 148, "ascii");
34
+ block.write("0", 156, "ascii");
35
+ block.write("ustar\0", 257, "ascii");
36
+ block.write("00", 263, "ascii");
37
+ let sum = 0;
38
+ for (const byte of block) sum += byte;
39
+ block.write(octal(sum, 7) + " ", 148, "ascii");
40
+ return block;
41
+ }
42
+
43
+ /**
44
+ * A gzipped ustar archive of `entries`, in the order given, every entry with mode 0644,
45
+ * owner 0 and modification time `mtime`. The same arguments always give the same bytes.
46
+ *
47
+ * @param {Array<{name: string, data: Buffer}>} entries plain file names and their bytes
48
+ * @param {number} mtime seconds since the epoch
49
+ * @returns {Buffer}
50
+ */
51
+ function tgz(entries, mtime) {
52
+ const parts = [];
53
+ for (const { name, data } of entries) {
54
+ parts.push(header(name, data.length, mtime), data);
55
+ const padding = (BLOCK - (data.length % BLOCK)) % BLOCK;
56
+ if (padding) parts.push(Buffer.alloc(padding, 0));
57
+ }
58
+ parts.push(Buffer.alloc(BLOCK * 2, 0));
59
+ const gzipped = zlib.gzipSync(Buffer.concat(parts), { level: 9 });
60
+ // zlib writes the operating system it was built for into byte 9; 255 is "unknown",
61
+ // so an archive packed on Windows is the same as one packed on Linux.
62
+ gzipped[9] = 0xff;
63
+ return gzipped;
64
+ }
65
+
66
+ module.exports = { tgz, TarError };
@@ -44,10 +44,11 @@ function isPrerelease(version) {
44
44
  * The npm dist-tag a version should be published under.
45
45
  *
46
46
  * A pre-release goes to `next`, never `latest`, so that `npm install <pkg>`
47
- * cannot pick one up by accident.
47
+ * cannot pick one up by accident. Both can be named otherwise -- `releaseTag` for a
48
+ * patch to an older line that must not become `latest`, say.
48
49
  */
49
- function npmDistTag(version) {
50
- return isPrerelease(version) ? "next" : "latest";
50
+ function npmDistTag(version, { releaseTag = "latest", prereleaseTag = "next" } = {}) {
51
+ return isPrerelease(version) ? prereleaseTag : releaseTag;
51
52
  }
52
53
 
53
54
  /**
package/src/zip.js ADDED
@@ -0,0 +1,103 @@
1
+ "use strict";
2
+
3
+ // A deterministic ZIP writer, for the .nupkg the nuget channel builds.
4
+ //
5
+ // Node has no ZIP writer, and the Publisher carries one runtime dependency; zip, unlike
6
+ // tar, is not reliably installed on Windows or in slim CI images. The subset needed is
7
+ // small: a local header and deflated data per entry, a central directory, and an end
8
+ // record. No ZIP64, no encryption, no extra fields.
9
+ //
10
+ // Deterministic by construction, so that the same packed archive always yields the same
11
+ // package: entries are written in the order given, every entry carries the same fixed
12
+ // timestamp, and deflate runs at a fixed level.
13
+
14
+ const zlib = require("zlib");
15
+
16
+ class ZipError extends Error {}
17
+
18
+ // 1980-01-01 00:00:00, the earliest time a DOS date can hold.
19
+ const DOS_DATE = (0 << 9) | (1 << 5) | 1;
20
+ const DOS_TIME = 0;
21
+ const UTF8_NAMES = 0x0800;
22
+
23
+ const CRC_TABLE = Array.from({ length: 256 }, (_, n) => {
24
+ let c = n;
25
+ for (let k = 0; k < 8; k++) c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1;
26
+ return c >>> 0;
27
+ });
28
+
29
+ function crc32(buffer) {
30
+ let crc = 0xffffffff;
31
+ for (const byte of buffer) crc = CRC_TABLE[(crc ^ byte) & 0xff] ^ (crc >>> 8);
32
+ return (crc ^ 0xffffffff) >>> 0;
33
+ }
34
+
35
+ /** Refuses a name that is empty, absolute, climbs out, uses backslashes, or repeats. */
36
+ function checkNames(entries) {
37
+ const seen = new Set();
38
+ for (const { name } of entries) {
39
+ if (!name || name.startsWith("/") || name.includes("\\") || name.split("/").includes("..")) {
40
+ throw new ZipError(`${JSON.stringify(name)} is not a valid entry name`);
41
+ }
42
+ if (seen.has(name)) throw new ZipError(`${JSON.stringify(name)} appears twice`);
43
+ seen.add(name);
44
+ }
45
+ }
46
+
47
+ /**
48
+ * A ZIP archive of `entries`, in the order given.
49
+ *
50
+ * @param {Array<{name: string, data: Buffer}>} entries forward-slash paths and their bytes
51
+ * @returns {Buffer}
52
+ */
53
+ function zip(entries) {
54
+ checkNames(entries);
55
+ const locals = [];
56
+ const centrals = [];
57
+ let offset = 0;
58
+ for (const { name, data } of entries) {
59
+ const nameBytes = Buffer.from(name, "utf8");
60
+ const deflated = zlib.deflateRawSync(data, { level: 9 });
61
+ const crc = crc32(data);
62
+
63
+ const local = Buffer.alloc(30);
64
+ local.writeUInt32LE(0x04034b50, 0);
65
+ local.writeUInt16LE(20, 4);
66
+ local.writeUInt16LE(UTF8_NAMES, 6);
67
+ local.writeUInt16LE(8, 8);
68
+ local.writeUInt16LE(DOS_TIME, 10);
69
+ local.writeUInt16LE(DOS_DATE, 12);
70
+ local.writeUInt32LE(crc, 14);
71
+ local.writeUInt32LE(deflated.length, 18);
72
+ local.writeUInt32LE(data.length, 22);
73
+ local.writeUInt16LE(nameBytes.length, 26);
74
+ locals.push(local, nameBytes, deflated);
75
+
76
+ const central = Buffer.alloc(46);
77
+ central.writeUInt32LE(0x02014b50, 0);
78
+ central.writeUInt16LE(20, 4);
79
+ central.writeUInt16LE(20, 6);
80
+ central.writeUInt16LE(UTF8_NAMES, 8);
81
+ central.writeUInt16LE(8, 10);
82
+ central.writeUInt16LE(DOS_TIME, 12);
83
+ central.writeUInt16LE(DOS_DATE, 14);
84
+ central.writeUInt32LE(crc, 16);
85
+ central.writeUInt32LE(deflated.length, 20);
86
+ central.writeUInt32LE(data.length, 24);
87
+ central.writeUInt16LE(nameBytes.length, 28);
88
+ central.writeUInt32LE(offset, 42);
89
+ centrals.push(central, nameBytes);
90
+
91
+ offset += local.length + nameBytes.length + deflated.length;
92
+ }
93
+ const directory = Buffer.concat(centrals);
94
+ const end = Buffer.alloc(22);
95
+ end.writeUInt32LE(0x06054b50, 0);
96
+ end.writeUInt16LE(entries.length, 8);
97
+ end.writeUInt16LE(entries.length, 10);
98
+ end.writeUInt32LE(directory.length, 12);
99
+ end.writeUInt32LE(offset, 16);
100
+ return Buffer.concat([...locals, directory, end]);
101
+ }
102
+
103
+ module.exports = { zip, crc32, ZipError };