@arc-e-tect/api-only-publisher 0.14.3 → 0.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.adoc +187 -22
- package/package.json +1 -1
- package/src/channels.js +73 -9
- package/src/cli.js +3 -0
- package/src/gitlab.js +187 -0
- package/src/pack.js +17 -2
- package/src/remote.js +60 -0
- package/src/version-policy.js +27 -1
package/README.adoc
CHANGED
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
:source-highlighter: rouge
|
|
7
7
|
// The released version of this component: its snippets use it. The release
|
|
8
8
|
// workflow updates it; do not change it by hand.
|
|
9
|
-
:api-only-publisher-version: 0.
|
|
9
|
+
:api-only-publisher-version: 0.15.0
|
|
10
10
|
|
|
11
11
|
[NOTE]
|
|
12
12
|
.About this documentation
|
|
@@ -641,7 +641,7 @@ Not a distribution mechanism between repositories: it makes the pipeline testabl
|
|
|
641
641
|
Publishing a version again writes it again and keeps the others (<<idempotent,Publishing the same version again §>>); with `clean: true` only the version being published is kept, for a directory that is a build output rather than a record of releases.
|
|
642
642
|
|
|
643
643
|
|`maven`
|
|
644
|
-
|A path publishes into a repository layout on disk; an `http(s)` URL deploys to a remote repository with an HTTP `PUT` per file and a
|
|
644
|
+
|A path publishes into a repository layout on disk; an `http(s)` URL deploys to a remote repository with an HTTP `PUT` per file, and keeps its `maven-metadata.xml` (<<maven-remote,Publishing to a remote Maven repository §>>).
|
|
645
645
|
Native for Gradle consumers: version resolution, caching and conflict handling come free, which is what lets the Subscriber carry no HTTP client of its own.
|
|
646
646
|
|
|
647
647
|
|`npm`
|
|
@@ -651,6 +651,10 @@ Native for Gradle consumers: version resolution, caching and conflict handling c
|
|
|
651
651
|
|The language-neutral floor: immutable per tag, works for private repositories with a token, readable by a consumer with no JVM and no Node.js.
|
|
652
652
|
No dependency-resolution semantics and no update notification, which is why it is the fallback rather than the default.
|
|
653
653
|
|
|
654
|
+
|`gitlab-package`
|
|
655
|
+
|A GitLab project's Generic Package Registry: the archive and `manifest.json` as a generic package per target, on gitlab.com or a self-managed GitLab.
|
|
656
|
+
In a GitLab CI job it needs no settings at all. See <<gitlab-package,The `gitlab-package` channel §>>.
|
|
657
|
+
|
|
654
658
|
|`nuget`
|
|
655
659
|
|A path writes a `.nupkg` into a local feed; an `http(s)` URL -- the feed's service index -- pushes to a remote one.
|
|
656
660
|
Native for .NET consumers: a `PackageReference` at a version, restored from a feed. See <<nuget,The `nuget` channel §>>.
|
|
@@ -674,13 +678,46 @@ What happens when the destination already has the version:
|
|
|
674
678
|
|`file`; `maven` and `nuget` to a directory; `npm` with `publish: false`
|
|
675
679
|
|Written again. With the same content the bytes are the same; with different content the version is replaced, and every subscriber whose lockfile recorded the old bytes fails its build until the change is published as a new version.
|
|
676
680
|
|
|
677
|
-
|`maven` and `nuget` to a URL, `npm`, `github-release`
|
|
681
|
+
|`maven` and `nuget` to a URL, `npm`, `github-release`, `gitlab-package`
|
|
678
682
|
|The same bytes: nothing is sent, and the log says `already published, identical`. Different bytes: `publish` fails, naming the version -- a published version is never replaced, so publish the change as a new version.
|
|
683
|
+
`maven` and `gitlab-package` read back every file before uploading any, so a run interrupted half-way is completed by the next one; a GitLab registry that allows duplicate packages, as it does by default, would otherwise replace a version.
|
|
679
684
|
`github-release` compares the asset it would upload with the one the release has, and never overwrites one.
|
|
680
685
|
`npm` compares the integrity hash the registry records, or the tarball's SHA-1 where it records only that, as GitLab's does; its files are packed with fixed modes, so the package is the same bytes whatever the umask of the machine that publishes it.
|
|
681
686
|
A NuGet feed that cannot return the package it has, so there is nothing to compare, fails as a different one would.
|
|
682
687
|
|===
|
|
683
688
|
|
|
689
|
+
[#remote-switch]
|
|
690
|
+
=== Refusing remote publishing
|
|
691
|
+
|
|
692
|
+
A channel's *local mode* writes files: `file`, `maven` and `nuget` to a directory, `npm` with `publish: false`.
|
|
693
|
+
Its *remote mode* sends them somewhere: `maven` and `nuget` to a URL, `npm` with publishing, `github-release`, `gitlab-package`.
|
|
694
|
+
|
|
695
|
+
`API_ONLY_PUBLISHER_REMOTE` decides whether `publish` may use a remote mode:
|
|
696
|
+
|
|
697
|
+
[cols="1,3", options="header"]
|
|
698
|
+
|===
|
|
699
|
+
| Value | Effect
|
|
700
|
+
|
|
701
|
+
| `allow`, or unset
|
|
702
|
+
| Every channel works. The default outside the Docker image.
|
|
703
|
+
|
|
704
|
+
| `never`
|
|
705
|
+
| `publish` refuses a run that would use any remote channel, naming each of them, before it reads a version or packs anything; the local modes work as always.
|
|
706
|
+
The Publisher then contacts nothing. The default in the Docker image (<<docker-network,Nothing phones home §>>).
|
|
707
|
+
|===
|
|
708
|
+
|
|
709
|
+
[source,text]
|
|
710
|
+
----
|
|
711
|
+
Error: API_ONLY_PUBLISHER_REMOTE=never refuses the remote channels maven, github-release, so nothing was built, packed or sent. Publish with their local modes, or set API_ONLY_PUBLISHER_REMOTE=allow where this run may contact them.
|
|
712
|
+
----
|
|
713
|
+
|
|
714
|
+
Any other value fails `publish`, whatever its channels: a typo in a switch that keeps the Publisher sealed is not ignored.
|
|
715
|
+
`--channel` narrows what is checked: `publish --channel file` succeeds under `never`, whatever else `apionly.yaml` configures.
|
|
716
|
+
The Publisher used as a library keeps the same rule.
|
|
717
|
+
|
|
718
|
+
The switch is a promise the Publisher keeps, not an enforcement: whoever runs it can change the variable.
|
|
719
|
+
Where contacting nothing must be guaranteed, isolate the job's network on the platform as well -- `docker run --network none`, or a network policy for the runner.
|
|
720
|
+
|
|
684
721
|
[#handoff]
|
|
685
722
|
=== The hand-off: what a `publish` run wrote
|
|
686
723
|
|
|
@@ -706,15 +743,15 @@ The hand-off is for a later step that publishes those files itself -- the second
|
|
|
706
743
|
[#channel-text]
|
|
707
744
|
=== What a package says about itself
|
|
708
745
|
|
|
709
|
-
The Maven POM, the npm `package.json`, the NuGet `.nuspec
|
|
746
|
+
The Maven POM, the npm `package.json`, the NuGet `.nuspec`, a GitHub release and a GitLab release all describe the contract.
|
|
710
747
|
By default each says `API description documents for <target>.`, and a GitHub release is titled `<target> <version>`.
|
|
711
|
-
`channels.<channel>.description` replaces the description, and `channels.github-release
|
|
748
|
+
`channels.<channel>.description` replaces the description, and `title` and `notes` on `channels.github-release` and `channels.gitlab-package` the release's title and notes (notes default to the description); `{target}` and `{version}` are filled in, as in `tagFormat`.
|
|
712
749
|
|
|
713
750
|
npm publishes a release under the dist-tag `latest` and a pre-release under `next`.
|
|
714
751
|
`channels.npm.releaseTag` and `prereleaseTag` name others: publishing a patch to an older line under `releaseTag: v1` keeps `latest` on the newest line, which publishing it under `latest` would move back.
|
|
715
752
|
|
|
716
753
|
Credentials are read from the environment, never from configuration.
|
|
717
|
-
`channels.maven.tokenEnv` names the variable, `MAVEN_TOKEN` by default,
|
|
754
|
+
`channels.maven.tokenEnv` names the variable, `MAVEN_TOKEN` by default, `channels.nuget.tokenEnv` the NuGet API key's, `NUGET_API_KEY` by default, and `channels.gitlab-package.tokenEnv` the GitLab token's, `GITLAB_TOKEN` by default, with `CI_JOB_TOKEN` when that is unset; a configuration file gets committed, and a credential in a committed file is a credential that has leaked.
|
|
718
755
|
|
|
719
756
|
link:../docs/reference/how-publisher-and-subscriber-work-together.adoc#channels[How the Publisher and the Subscriber work together →] lists each channel's layout and options.
|
|
720
757
|
|
|
@@ -784,6 +821,88 @@ Until a `.targets` file places and verifies the documents for a .NET build, a co
|
|
|
784
821
|
The same packed archive always produces a byte-identical package: entries in a fixed order, a fixed timestamp.
|
|
785
822
|
Since `pack` is reproducible, publishing the same commit again produces the same package too, so a republish to a local feed can be checked as unchanged.
|
|
786
823
|
|
|
824
|
+
[#gitlab-package]
|
|
825
|
+
=== The `gitlab-package` channel
|
|
826
|
+
|
|
827
|
+
It publishes each target to a GitLab project's Generic Package Registry, on gitlab.com or a self-managed GitLab, without any GitLab tooling: a package is a few HTTP requests the Publisher sends itself.
|
|
828
|
+
|
|
829
|
+
[source,yaml]
|
|
830
|
+
----
|
|
831
|
+
channels:
|
|
832
|
+
gitlab-package: # in a GitLab CI job, nothing below is needed
|
|
833
|
+
# url: https://gitlab.example.com/api/v4 # default: $CI_API_V4_URL, else https://gitlab.com/api/v4
|
|
834
|
+
# project: group/subgroup/api-library # an ID or a path; default: $CI_PROJECT_ID
|
|
835
|
+
# tokenEnv: GITLAB_TOKEN # default; when unset, $CI_JOB_TOKEN is used
|
|
836
|
+
# tokenType: private # private (default), deploy or job
|
|
837
|
+
# packageName: "{target}" # default
|
|
838
|
+
# release: false # true: also the tag's GitLab release
|
|
839
|
+
# tagFormat: "{target}-v{version}" # for release: true
|
|
840
|
+
# title, notes, description # as for github-release
|
|
841
|
+
----
|
|
842
|
+
|
|
843
|
+
*The package* is named after the target, at the contract's version, and holds two files: `<target>-<version>.tgz` and `manifest.json`, the ones the `file` channel writes.
|
|
844
|
+
A consumer downloads them from `<url>/projects/<project>/packages/generic/<target>/<version>/<file>`, or from the project's *Deploy › Package registry* page.
|
|
845
|
+
|
|
846
|
+
*In a GitLab CI job* the channel needs no settings: the API URL, the project and the job's token come from GitLab's predefined variables `CI_API_V4_URL`, `CI_PROJECT_ID` and `CI_JOB_TOKEN`.
|
|
847
|
+
link:pipelines/gitlab-ci.yml[The reference pipeline →] publishes this way, from the Docker image, in one job (<<docker-gitlab,Reference pipeline: GitLab CI §>>).
|
|
848
|
+
|
|
849
|
+
*The credential* is the first of these the environment holds:
|
|
850
|
+
|
|
851
|
+
[cols="2,2,3", options="header"]
|
|
852
|
+
|===
|
|
853
|
+
| Variable | Sent as | For
|
|
854
|
+
|
|
855
|
+
| The one `tokenEnv` names, `GITLAB_TOKEN` by default
|
|
856
|
+
| `PRIVATE-TOKEN`; with `tokenType: deploy`, `DEPLOY-TOKEN`; with `tokenType: job`, `JOB-TOKEN`
|
|
857
|
+
| A personal or project access token with the `api` scope, or a deploy token with `write_package_registry` (and `read_package_registry`, for the read-back).
|
|
858
|
+
|
|
859
|
+
| `CI_JOB_TOKEN`
|
|
860
|
+
| `JOB-TOKEN`
|
|
861
|
+
| A GitLab CI job publishing to its own project.
|
|
862
|
+
|===
|
|
863
|
+
|
|
864
|
+
Without either, `publish` fails before anything is sent, naming both.
|
|
865
|
+
A deploy token is the least privilege for packages alone: it reaches one project's package registry and nothing else.
|
|
866
|
+
|
|
867
|
+
*The same version again.*
|
|
868
|
+
GitLab allows duplicate packages by default: it accepts a different file under a version it already has, and serves that one from then on.
|
|
869
|
+
So the channel reads every file back before uploading any: an identical file is left alone, a missing one is uploaded, and a different one fails `publish` (<<idempotent,Publishing the same version again §>>).
|
|
870
|
+
Turning *Allow duplicates* off for generic packages, in the group's *Settings › Packages and registries*, makes GitLab refuse a replaced file too, from every client; the Publisher keeps a version immutable either way.
|
|
871
|
+
|
|
872
|
+
*`release: true`* also creates the tag's GitLab release, tagging `$CI_COMMIT_SHA` or, outside CI, the checkout's `HEAD`, and links both files as packages.
|
|
873
|
+
An existing release keeps everything it has, and only gets the links it lacks.
|
|
874
|
+
A pre-release publishes the package and no release: GitLab has no pre-release flag, so the release would become the project's latest.
|
|
875
|
+
A deploy token cannot create a release -- GitLab gives it the package registry only -- so `release: true` with `tokenType: deploy` is refused before anything is sent.
|
|
876
|
+
|
|
877
|
+
link:../docs/guides/publishing-to-gitlab/overview.adoc[Publishing to GitLab →] covers the rest: job-token permissions, self-managed instances and their certificates, and how consumers read the packages.
|
|
878
|
+
|
|
879
|
+
[#maven-remote]
|
|
880
|
+
=== Publishing to a remote Maven repository
|
|
881
|
+
|
|
882
|
+
An `http(s)` `repository` deploys with an HTTP `PUT` per file: the artifact, its POM, and `-manifest.json`.
|
|
883
|
+
|
|
884
|
+
[source,yaml]
|
|
885
|
+
----
|
|
886
|
+
channels:
|
|
887
|
+
maven:
|
|
888
|
+
groupId: com.example.api
|
|
889
|
+
repository: https://gitlab.example.com/api/v4/projects/1234/packages/maven
|
|
890
|
+
tokenEnv: CI_JOB_TOKEN # default: MAVEN_TOKEN
|
|
891
|
+
tokenHeader: Job-Token # default: Authorization, as a bearer token
|
|
892
|
+
# metadata: false # leave maven-metadata.xml to a repository that generates it
|
|
893
|
+
----
|
|
894
|
+
|
|
895
|
+
*The token* goes as a bearer token, `Authorization: Bearer <token>`, unless `tokenHeader` names another header, which then carries it as it is.
|
|
896
|
+
GitLab's Maven registry takes a job token as `Job-Token`, a personal or project access token as `Private-Token`, and a deploy token as `Deploy-Token`; a GitLab Maven repository URL needs the project's numeric ID, since Gradle reads `%2F` in a path as `/`.
|
|
897
|
+
|
|
898
|
+
*Every file is read back* before any is uploaded: identical is left alone, missing is uploaded, different fails `publish`, naming the file.
|
|
899
|
+
|
|
900
|
+
*`maven-metadata.xml`* lists an artifact's versions, which is how Gradle resolves a version range and how Renovate and Dependabot see a new version.
|
|
901
|
+
Nexus, Artifactory and GitHub Packages keep it themselves; GitLab does not for files uploaded this way.
|
|
902
|
+
So after a version's files, the channel adds the version to it, as `mvn deploy` does: `latest` is the highest version, `release` the highest that is not a pre-release, and `lastUpdated` the time of the commit, so the same commit writes the same file.
|
|
903
|
+
It is written only when it lacks the version, or its `latest` or `release` would change.
|
|
904
|
+
A repository that refuses it fails `publish`, naming `metadata: false`; the version's files are published by then, and a re-run retries only the metadata.
|
|
905
|
+
|
|
787
906
|
[#why-registries]
|
|
788
907
|
=== Why npm, Maven and NuGet rather than release assets alone
|
|
789
908
|
|
|
@@ -999,7 +1118,7 @@ Two questions decide how a library is configured:
|
|
|
999
1118
|
* *How many targets?*
|
|
1000
1119
|
A target is one contract with a version of its own and consumers of its own: what gets shipped.
|
|
1001
1120
|
* *How many channels?*
|
|
1002
|
-
A channel is one way of shipping it -- `file`, `maven`, `npm`, `nuget` or `
|
|
1121
|
+
A channel is one way of shipping it -- `file`, `maven`, `npm`, `nuget`, `github-release` or `gitlab-package` -- chosen to suit the technology of whoever consumes it: how it reaches them.
|
|
1003
1122
|
|
|
1004
1123
|
Each recipe below answers both, for a situation in which the answer is typical.
|
|
1005
1124
|
They are starting points, not boundaries: the situations describe where each shape usually turns up, and nothing stops a shape being used elsewhere.
|
|
@@ -1093,7 +1212,8 @@ This publishes `com.example.contracts:orders:<version>@tgz`, which a build fetch
|
|
|
1093
1212
|
*Variations.*
|
|
1094
1213
|
|
|
1095
1214
|
* A library that lives in the same repository as its implementation needs no repository at all: the `file` channel, with `directory: build/publish`, is what `init` writes, and the Subscriber reads it from there.
|
|
1096
|
-
* A consumer that is not a JVM build takes a different channel in place of `maven`: `npm` for a Node.js or TypeScript build, `nuget` for .NET, `github-release` for anything else.
|
|
1215
|
+
* A consumer that is not a JVM build takes a different channel in place of `maven`: `npm` for a Node.js or TypeScript build, `nuget` for .NET, `github-release` or `gitlab-package` for anything else.
|
|
1216
|
+
* A library hosted on GitLab publishes to its own project's registries: `gitlab-package`, which needs no settings in a GitLab CI job, and `maven` to the project's Maven registry with `tokenHeader: Job-Token` (<<gitlab-package,The `gitlab-package` channel §>>).
|
|
1097
1217
|
|
|
1098
1218
|
[#recipe-microservices]
|
|
1099
1219
|
=== Microservice architecture: several targets, one channel
|
|
@@ -1189,7 +1309,8 @@ One run publishes `com.example.partner:partner-api`, `@example-partner/partner-a
|
|
|
1189
1309
|
* `--channel` publishes to one of them on its own, for instance when one partner's feed was unreachable during the release: `npm run apionly -- publish --channel nuget`.
|
|
1190
1310
|
The other channels already have the version, and would leave it alone anyway.
|
|
1191
1311
|
* A pre-release, `publish --pre-release rc.1`, is how a partner gets an unfinished contract to start building against, without any partner's production build picking it up by accident (<<pre-releases,Pre-releases §>>).
|
|
1192
|
-
* Each channel takes its credential from the environment, never from `apionly.yaml`: `MAVEN_TOKEN` and `
|
|
1312
|
+
* Each channel takes its credential from the environment, never from `apionly.yaml`: `MAVEN_TOKEN`, `NUGET_API_KEY` and `GITLAB_TOKEN` by default, npm's own configuration for `npm`, and the GitHub CLI's for `github-release` (<<channel-text,What a package says about itself §>>).
|
|
1313
|
+
* A partner without a package manager gets a download: `github-release` where the library is on GitHub, `gitlab-package` where it is on GitLab.
|
|
1193
1314
|
* Partners that each need a repository of *the same type* -- two Maven repositories, one per partner -- cannot be two `maven` channels.
|
|
1194
1315
|
Publish to a local destination instead, `repository` or `directory` set to a directory, and let the pipeline copy the files it lists in `packages.sha256` to each partner's repository, as <<handoff,The hand-off §>> describes.
|
|
1195
1316
|
|
|
@@ -1539,13 +1660,45 @@ The Publisher is also a Docker image, for CI pipelines that may run only what ha
|
|
|
1539
1660
|
|===
|
|
1540
1661
|
|
|
1541
1662
|
Vetting the npm package does not vet what it runs: it fetches Redocly CLI and AsyncAPI CLI with `npx` when it runs.
|
|
1542
|
-
The image is one artefact, with one digest, holding one Publisher version *and* its toolchain, and it is *sealed*: it downloads nothing, and contacts nothing
|
|
1663
|
+
The image is one artefact, with one digest, holding one Publisher version *and* its toolchain, and it is *sealed*: it downloads nothing when it runs, and contacts nothing unless a job allows it to publish.
|
|
1543
1664
|
It can be pulled by digest, its signature verified, its SBOM and provenance read, scanned, and mirrored, and then a pipeline knows exactly what it runs.
|
|
1544
1665
|
|
|
1545
1666
|
Both registries serve the same image, with the same digest: Docker Hub gets a copy of what GHCR was given, never a rebuild.
|
|
1546
1667
|
|
|
1547
1668
|
link:../docs/guides/publishing-from-ci/overview.adoc[Publishing from CI with the image →] is the guide, with prompts that set up the pipeline in your own repository.
|
|
1548
1669
|
|
|
1670
|
+
Publishing from the image takes one job or two:
|
|
1671
|
+
|
|
1672
|
+
[cols="1,2,2", options="header"]
|
|
1673
|
+
|===
|
|
1674
|
+
| | One job | Two steps
|
|
1675
|
+
|
|
1676
|
+
| What runs
|
|
1677
|
+
| The image, and nothing else.
|
|
1678
|
+
| The image packages; a job per registry publishes with that registry's own client, on an image of its own.
|
|
1679
|
+
|
|
1680
|
+
| Credentials and network
|
|
1681
|
+
| The publishing job allows the image to contact the registry (`API_ONLY_PUBLISHER_REMOTE: allow`) and hands it the token.
|
|
1682
|
+
| The image never holds a credential, and runs with no network.
|
|
1683
|
+
|
|
1684
|
+
| Channels
|
|
1685
|
+
| `gitlab-package`, and `maven` to a URL: the remote modes the image is tested with.
|
|
1686
|
+
| Every channel, in its local mode.
|
|
1687
|
+
|
|
1688
|
+
| Reference pipeline
|
|
1689
|
+
| <<docker-gitlab,GitLab CI §>>
|
|
1690
|
+
| <<docker-github-actions,GitHub Actions §>>, and the <<docker-gitlab,GitLab CI §>> variant
|
|
1691
|
+
|===
|
|
1692
|
+
|
|
1693
|
+
[#docker-one-job]
|
|
1694
|
+
=== One job: the image publishes
|
|
1695
|
+
|
|
1696
|
+
The job runs `build`, `lint` and `publish` in the image, with `API_ONLY_PUBLISHER_REMOTE: allow` and the registry's token in its environment.
|
|
1697
|
+
Nothing else runs, so the image is the only artefact to vet, mirror and pin; the publishing clients and their images that two steps need are not.
|
|
1698
|
+
The image stays sealed in every other job, and the job contacts only the destinations `apionly.yaml` configures.
|
|
1699
|
+
|
|
1700
|
+
The remote modes the image is tested with are `gitlab-package` and `maven` to a URL: its smoke test publishes with both, as a non-root user on a read-only root, to a GitLab stand-in on a network with no route out (<<docker-verifying,Running the smoke test §>>).
|
|
1701
|
+
|
|
1549
1702
|
[#docker-two-steps]
|
|
1550
1703
|
=== Two steps: package in the image, publish with the platform's tools
|
|
1551
1704
|
|
|
@@ -1556,14 +1709,15 @@ It needs no network and no credential.
|
|
|
1556
1709
|
Every `publish` run writes a hand-off, <<handoff,The hand-off §>>, listing each file it wrote with its SHA-256.
|
|
1557
1710
|
The publishing job checks the files against it, and publishes exactly those bytes: the bytes in the registry are the bytes the vetted tool produced.
|
|
1558
1711
|
|
|
1559
|
-
|
|
1712
|
+
In two steps, credentials never enter the image.
|
|
1560
1713
|
|
|
1561
1714
|
[IMPORTANT]
|
|
1562
|
-
.What the image does not
|
|
1715
|
+
.What the image does not publish itself yet
|
|
1563
1716
|
====
|
|
1564
|
-
The image
|
|
1565
|
-
|
|
1566
|
-
|
|
1717
|
+
The image refuses every remote mode unless a job sets `API_ONLY_PUBLISHER_REMOTE: allow` (<<remote-switch,Refusing remote publishing §>>).
|
|
1718
|
+
Allowed, it supports `gitlab-package` and `maven` to a URL.
|
|
1719
|
+
`nuget` to a URL, `npm` with publishing and `github-release` may work from inside it too, but they are neither tested nor supported there yet -- `github-release` needs the GitHub CLI, which the image does not hold.
|
|
1720
|
+
Publish those in a second step, as <<docker-pipelines>> shows.
|
|
1567
1721
|
====
|
|
1568
1722
|
|
|
1569
1723
|
[#docker-tags]
|
|
@@ -1803,6 +1957,10 @@ The image makes no outbound connection: it sets, in its environment, what switch
|
|
|
1803
1957
|
|
|
1804
1958
|
This is tested, not assumed: the whole smoke test was run on a network whose every packet was captured, and the image sent none.
|
|
1805
1959
|
|
|
1960
|
+
Nor does `publish` contact anything: the image sets `API_ONLY_PUBLISHER_REMOTE=never`, so a run that would use a remote channel fails before it packs anything, naming the channels (<<remote-switch,Refusing remote publishing §>>).
|
|
1961
|
+
A pipeline that publishes from the image sets `API_ONLY_PUBLISHER_REMOTE: allow` on that job: a visible line, reviewed like any other.
|
|
1962
|
+
The variable is the Publisher's promise; a pipeline that must be sure the job contacts nothing also runs it without a network, `docker run --network none`.
|
|
1963
|
+
|
|
1806
1964
|
[#docker-ci-job]
|
|
1807
1965
|
==== As a CI job image
|
|
1808
1966
|
|
|
@@ -1850,19 +2008,24 @@ Each publishing step checks for the version first, compares the bytes, and publi
|
|
|
1850
2008
|
[#docker-gitlab]
|
|
1851
2009
|
==== Reference pipeline: GitLab CI
|
|
1852
2010
|
|
|
1853
|
-
|
|
1854
|
-
|
|
1855
|
-
|
|
1856
|
-
|
|
1857
|
-
====
|
|
2011
|
+
One job, on a `v*` tag, as the GitHub Actions pipeline: the image builds, lints and publishes the targets to the project's Generic Package Registry with `gitlab-package`, and, where `apionly.yaml` configures it, to the project's Maven registry, both with the job's `CI_JOB_TOKEN`.
|
|
2012
|
+
`apionly.yaml` needs no GitLab settings for it; the pipeline's header says what to add for the Maven registry.
|
|
2013
|
+
Copy it to `.gitlab-ci.yml`, then set the digest and the targets.
|
|
2014
|
+
The tag pattern keeps out the tag a GitLab release puts on the commit, `<target>-v<version>`, which would otherwise start the pipeline a second time; a library that tags its releases otherwise changes the rule, and keeps that one out.
|
|
1858
2015
|
|
|
1859
|
-
|
|
2016
|
+
It has run on gitlab.com: with the job token, it published to the project's generic and Maven registries, created the release, and, run again, found everything published and changed nothing.
|
|
2017
|
+
It is validated against GitLab's published CI schema, the image's smoke test publishes as this job does to a GitLab stand-in, and the link:../.github/workflows/api-only-publisher-image-e2e.yml[end-to-end workflow →] publishes the same way to a gitlab.com test project, with a deploy token.
|
|
1860
2018
|
|
|
1861
2019
|
[source,yaml]
|
|
1862
2020
|
----
|
|
1863
2021
|
include::pipelines/gitlab-ci.yml[]
|
|
1864
2022
|
----
|
|
1865
2023
|
|
|
2024
|
+
link:../docs/guides/publishing-to-gitlab/overview.adoc[Publishing to GitLab →] is the guide: tokens, a self-managed GitLab, and what consumers need.
|
|
2025
|
+
|
|
2026
|
+
*The two-step variant*, link:pipelines/gitlab-ci-two-step.yml[`pipelines/gitlab-ci-two-step.yml` →], keeps the image sealed: `package` runs in the image, and one `publish:*` job per registry publishes to the project's package registry with that registry's own client and `CI_JOB_TOKEN`: npm, Maven and NuGet, and the archives to the generic package registry.
|
|
2027
|
+
It is validated against GitLab's published CI schema only; no GitLab pipeline has run it yet.
|
|
2028
|
+
|
|
1866
2029
|
[#docker-mirror-in-pipelines]
|
|
1867
2030
|
==== Using a mirrored image
|
|
1868
2031
|
|
|
@@ -1956,12 +2119,14 @@ docker build -t api-only-publisher:local --build-arg VERSION=0.0.0-local .
|
|
|
1956
2119
|
docker/smoke-test.sh api-only-publisher:local 0.0.0-local
|
|
1957
2120
|
----
|
|
1958
2121
|
|
|
1959
|
-
It runs every step with
|
|
2122
|
+
It runs every step with a read-only root and your own user, and all but the last with no network: the version, the help, the toolchain labels against what is installed and against `init`'s pins, a library with two targets built, linted and packaged to every local channel in one run, ownership, byte-identical packaging twice, the hand-off, a foreign toolchain pin failing at once, a remote channel refused while sealed, the shell mode, and a build as an arbitrary UID.
|
|
2123
|
+
The last step publishes with `gitlab-package` and `maven` to a GitLab stand-in, run from the same image, on a Docker network it creates with no route out (`docker network create --internal`) and removes afterwards; it checks every request went to the project's package registries with the job token, and that a re-run uploads nothing.
|
|
1960
2124
|
|
|
1961
2125
|
link:../.github/workflows/api-only-publisher-image-e2e.yml[`api-only-publisher-image-e2e.yml` →] runs the GitHub Actions reference pipeline for real, by hand only.
|
|
1962
2126
|
link:test/docker.test.js[`test/docker.test.js` →] holds it to what the link:pipelines/[reference pipelines →] do, the same actions, commands, inputs and environment, but not to the versions they pin.
|
|
1963
2127
|
The reference pipelines are examples, maintained on their own, and the end-to-end workflow is maintained with this repository's other workflows, so a new version of an action in one never fails the Publisher's build while the other has not moved.
|
|
1964
2128
|
Give it an image by digest; it publishes version `0.<run>.<attempt>` of two scratch targets, `apionly-e2e-alpha` and `apionly-e2e-beta`, to npm, Maven and NuGet on GitHub Packages and to a scratch release, publishes again to prove a re-run is safe, resolves each package with its own client and with the Subscriber, and removes what it published.
|
|
2129
|
+
Its `gitlab` job publishes the same targets from the image, in one job, to a gitlab.com test project's Generic Package Registry and Maven registry with a deploy token (secret `GITLAB_E2E_DEPLOY_TOKEN`), publishes again, checks GitLab serves the image's bytes and lists the version in `maven-metadata.xml`, and resolves a contract with the Subscriber; a deploy token cannot delete packages, so the run's summary lists the scratch versions to delete by hand.
|
|
1965
2130
|
The image workflow, dispatched by hand, is the release's dry run: its defaults are the scratch names `ghcr.io/arc-e-tect/api-only-publisher-scratch` and `arcetect/api-only-publisher-scratch`, and it builds from the branch's source.
|
|
1966
2131
|
|
|
1967
2132
|
[#adapting]
|
package/package.json
CHANGED
package/src/channels.js
CHANGED
|
@@ -12,9 +12,11 @@ const os = require("os");
|
|
|
12
12
|
const path = require("path");
|
|
13
13
|
const { execFileSync } = require("child_process");
|
|
14
14
|
|
|
15
|
-
const { isPrerelease, npmDistTag } = require("./version-policy");
|
|
15
|
+
const { parse, compare, isPrerelease, npmDistTag } = require("./version-policy");
|
|
16
16
|
const { ChannelError } = require("./channel-error");
|
|
17
17
|
const { publishNuget } = require("./nuget");
|
|
18
|
+
const { publishGitlabPackage } = require("./gitlab");
|
|
19
|
+
const { refuseRemote } = require("./remote");
|
|
18
20
|
|
|
19
21
|
/** What every package and release says it is, unless `description` says otherwise. */
|
|
20
22
|
const DEFAULT_DESCRIPTION = "API description documents for {target}.";
|
|
@@ -301,11 +303,11 @@ function publishGithubRelease(archive, manifest, options, log) {
|
|
|
301
303
|
* configuration file gets committed, and a credential in a committed file is a
|
|
302
304
|
* credential that has leaked.
|
|
303
305
|
*/
|
|
304
|
-
async function putFile(url, body,
|
|
306
|
+
async function putFile(url, body, auth, contentType) {
|
|
305
307
|
const response = await fetch(url, {
|
|
306
308
|
method: "PUT",
|
|
307
309
|
headers: {
|
|
308
|
-
|
|
310
|
+
...auth,
|
|
309
311
|
"Content-Type": contentType,
|
|
310
312
|
"Content-Length": String(body.length),
|
|
311
313
|
},
|
|
@@ -322,8 +324,8 @@ async function putFile(url, body, token, contentType) {
|
|
|
322
324
|
}
|
|
323
325
|
|
|
324
326
|
/** What a Maven repository holds at `url`, or null when it holds nothing there. */
|
|
325
|
-
async function getFile(url,
|
|
326
|
-
const response = await fetch(url, { headers:
|
|
327
|
+
async function getFile(url, auth) {
|
|
328
|
+
const response = await fetch(url, { headers: auth });
|
|
327
329
|
if (response.status === 404) return null;
|
|
328
330
|
if (!response.ok) {
|
|
329
331
|
throw new ChannelError(`GET ${url} failed: ${response.status} ${response.statusText}. ` +
|
|
@@ -333,6 +335,54 @@ async function getFile(url, token) {
|
|
|
333
335
|
return Buffer.from(await response.arrayBuffer());
|
|
334
336
|
}
|
|
335
337
|
|
|
338
|
+
/** An HTTP header name: RFC 9110's token. */
|
|
339
|
+
const HEADER_NAME = /^[!#$%&'*+.^_`|~0-9A-Za-z-]+$/;
|
|
340
|
+
|
|
341
|
+
/** The highest of `versions` that parse as semver and pass `keep`, or null. */
|
|
342
|
+
function highest(versions, keep = () => true) {
|
|
343
|
+
const valid = versions.filter((v) => { try { parse(v); return keep(v); } catch { return false; } });
|
|
344
|
+
return valid.length ? valid.reduce((a, b) => (compare(a, b) >= 0 ? a : b)) : null;
|
|
345
|
+
}
|
|
346
|
+
|
|
347
|
+
/** The artifact's maven-metadata.xml, as `mvn deploy` keeps it. */
|
|
348
|
+
function metadataXml(groupId, artifactId, versions, lastUpdated) {
|
|
349
|
+
const latest = highest(versions);
|
|
350
|
+
const release = highest(versions, (v) => !isPrerelease(v));
|
|
351
|
+
return '<?xml version="1.0" encoding="UTF-8"?>\n<metadata>\n' +
|
|
352
|
+
` <groupId>${groupId}</groupId>\n <artifactId>${artifactId}</artifactId>\n <versioning>\n` +
|
|
353
|
+
(latest ? ` <latest>${latest}</latest>\n` : "") + (release ? ` <release>${release}</release>\n` : "") +
|
|
354
|
+
` <versions>\n${versions.map((v) => ` <version>${v}</version>\n`).join("")} </versions>\n` +
|
|
355
|
+
` <lastUpdated>${lastUpdated}</lastUpdated>\n </versioning>\n</metadata>\n`;
|
|
356
|
+
}
|
|
357
|
+
|
|
358
|
+
/**
|
|
359
|
+
* Adds the version to the artifact's maven-metadata.xml, which a repository that does not
|
|
360
|
+
* generate it -- GitLab's -- otherwise never has, so that update bots never see a new
|
|
361
|
+
* version. Written only when it lacks the version, or its latest or release would change.
|
|
362
|
+
*/
|
|
363
|
+
async function updateMetadata(url, groupId, artifactId, manifest, auth, log) {
|
|
364
|
+
const held = await getFile(url, auth);
|
|
365
|
+
const xml = held ? held.toString("utf8") : "";
|
|
366
|
+
const block = /<versions>([\s\S]*?)<\/versions>/.exec(xml);
|
|
367
|
+
const versions = block ? [...block[1].matchAll(/<version>\s*([^<\s]+)\s*<\/version>/g)].map((m) => m[1]) : [];
|
|
368
|
+
const tag = (name) => ((new RegExp(`<${name}>\\s*([^<\\s]+)\\s*</${name}>`).exec(xml)) || [])[1] || null;
|
|
369
|
+
const listed = versions.includes(manifest.version) ? versions : [...versions, manifest.version];
|
|
370
|
+
if (listed === versions && tag("latest") === highest(listed)
|
|
371
|
+
&& tag("release") === highest(listed, (v) => !isPrerelease(v))) {
|
|
372
|
+
return;
|
|
373
|
+
}
|
|
374
|
+
// The commit's time, as the archive's producedAt, so the same commit writes the same file.
|
|
375
|
+
const lastUpdated = String(manifest.producedAt || new Date().toISOString()).replace(/\.\d+Z$/, "Z").replace(/[-:TZ]/g, "");
|
|
376
|
+
try {
|
|
377
|
+
await putFile(url, Buffer.from(metadataXml(groupId, artifactId, listed, lastUpdated), "utf8"), auth, "application/xml");
|
|
378
|
+
} catch (error) {
|
|
379
|
+
throw new ChannelError(`the repository refused maven-metadata.xml for ${groupId}:${artifactId}: ${error.message.trim()} ` +
|
|
380
|
+
"The version's files are published; a re-run retries only the metadata. For a repository that generates " +
|
|
381
|
+
"its own maven-metadata.xml, set channels.maven.metadata: false.");
|
|
382
|
+
}
|
|
383
|
+
log(`-- Listed ${manifest.version} in ${groupId}:${artifactId}'s maven-metadata.xml`);
|
|
384
|
+
}
|
|
385
|
+
|
|
336
386
|
function publishMavenRemote(archive, manifest, options, log) {
|
|
337
387
|
const tokenEnv = options.tokenEnv || "MAVEN_TOKEN";
|
|
338
388
|
const token = process.env[tokenEnv];
|
|
@@ -342,12 +392,20 @@ function publishMavenRemote(archive, manifest, options, log) {
|
|
|
342
392
|
`Remote publication needs one; set it in the release job's environment.`
|
|
343
393
|
);
|
|
344
394
|
}
|
|
395
|
+
// A bearer token unless tokenHeader names another header, which then carries the
|
|
396
|
+
// token as it is: GitLab takes a job, personal or deploy token as Job-Token,
|
|
397
|
+
// Private-Token or Deploy-Token, and Bearer only for OAuth.
|
|
398
|
+
if (options.tokenHeader !== undefined && !HEADER_NAME.test(options.tokenHeader)) {
|
|
399
|
+
throw new ChannelError(`channels.maven.tokenHeader must be an HTTP header name, such as Job-Token, ` +
|
|
400
|
+
`not ${JSON.stringify(options.tokenHeader)}`);
|
|
401
|
+
}
|
|
402
|
+
const auth = options.tokenHeader === undefined ? { "Authorization": `Bearer ${token}` } : { [options.tokenHeader]: token };
|
|
345
403
|
|
|
346
404
|
const artifactId = options.artifactId || manifest.target;
|
|
347
405
|
const version = manifest.version;
|
|
348
406
|
const extension = options.extension || "tgz";
|
|
349
|
-
const
|
|
350
|
-
|
|
407
|
+
const artifactDir = `${options.repository.replace(/\/+$/, "")}/${options.groupId.split(".").join("/")}/${artifactId}`;
|
|
408
|
+
const base = `${artifactDir}/${version}/${artifactId}-${version}`;
|
|
351
409
|
|
|
352
410
|
const work = (async () => {
|
|
353
411
|
// Idempotent, file by file: each file the repository already has is left alone when
|
|
@@ -363,7 +421,7 @@ function publishMavenRemote(archive, manifest, options, log) {
|
|
|
363
421
|
];
|
|
364
422
|
const missing = [];
|
|
365
423
|
for (const file of files) {
|
|
366
|
-
const existing = await getFile(file.url,
|
|
424
|
+
const existing = await getFile(file.url, auth);
|
|
367
425
|
if (!existing) {
|
|
368
426
|
missing.push(file);
|
|
369
427
|
} else if (!existing.equals(file.bytes)) {
|
|
@@ -373,7 +431,10 @@ function publishMavenRemote(archive, manifest, options, log) {
|
|
|
373
431
|
"A published version is never replaced; publish the change as a new version.");
|
|
374
432
|
}
|
|
375
433
|
}
|
|
376
|
-
for (const file of missing) await putFile(file.url, file.bytes,
|
|
434
|
+
for (const file of missing) await putFile(file.url, file.bytes, auth, file.type);
|
|
435
|
+
if (options.metadata !== false) {
|
|
436
|
+
await updateMetadata(`${artifactDir}/maven-metadata.xml`, options.groupId, artifactId, manifest, auth, log);
|
|
437
|
+
}
|
|
377
438
|
return missing.length === 0;
|
|
378
439
|
})();
|
|
379
440
|
|
|
@@ -392,6 +453,7 @@ function publishMavenRemote(archive, manifest, options, log) {
|
|
|
392
453
|
|
|
393
454
|
const CHANNELS = {
|
|
394
455
|
file: publishFile, maven: publishMaven, npm: publishNpm, "github-release": publishGithubRelease, nuget: publishNuget,
|
|
456
|
+
"gitlab-package": publishGitlabPackage,
|
|
395
457
|
};
|
|
396
458
|
|
|
397
459
|
function publish(archive, manifest, channel, options, log = () => {}) {
|
|
@@ -401,6 +463,8 @@ function publish(archive, manifest, channel, options, log = () => {}) {
|
|
|
401
463
|
`unknown channel '${channel}'; available channels are ${Object.keys(CHANNELS).join(", ")}`
|
|
402
464
|
);
|
|
403
465
|
}
|
|
466
|
+
// Here as well as in the CLI, so the Publisher used as a library keeps it too.
|
|
467
|
+
refuseRemote([channel], { [channel]: options || {} });
|
|
404
468
|
return handler(archive, manifest, options || {}, log);
|
|
405
469
|
}
|
|
406
470
|
|
package/src/cli.js
CHANGED
|
@@ -17,6 +17,7 @@ const { pack, PackError } = require("./pack");
|
|
|
17
17
|
const { changedSince, ChangedError } = require("./changed");
|
|
18
18
|
const { split, SplitError } = require("./split");
|
|
19
19
|
const { publish, ChannelError } = require("./channels");
|
|
20
|
+
const { refuseRemote } = require("./remote");
|
|
20
21
|
const { VersionError: PolicyError, describe } = require("./version-policy");
|
|
21
22
|
const { versionOf, BundleVersionError } = require("./bundle-version");
|
|
22
23
|
const { unreferenced } = require("./unreferenced");
|
|
@@ -451,6 +452,8 @@ async function main(argv, io = {}) {
|
|
|
451
452
|
if (names.length === 0) {
|
|
452
453
|
throw new ConfigError("no channels configured; add a `channels:` block or pass --channel");
|
|
453
454
|
}
|
|
455
|
+
// Every channel this run uses, before anything is read, packed or sent.
|
|
456
|
+
refuseRemote(names, configured);
|
|
454
457
|
const shipped = shippedTargets(config, targets);
|
|
455
458
|
const versions = versionsOf(config, shipped, options.preRelease);
|
|
456
459
|
const closures = forTargets(config, undefined, shipped);
|
package/src/gitlab.js
ADDED
|
@@ -0,0 +1,187 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
|
|
3
|
+
// `gitlab-package` -- publish to a GitLab project's Generic Package Registry.
|
|
4
|
+
//
|
|
5
|
+
// One generic package per target, at the contract's version, holding the archive and
|
|
6
|
+
// manifest.json: the same two files the `file` channel writes. In a GitLab CI job it
|
|
7
|
+
// needs no settings at all: the API, the project and the job token come from GitLab's
|
|
8
|
+
// own CI variables. `release: true` also creates the tag's GitLab release, linking both
|
|
9
|
+
// files as packages.
|
|
10
|
+
//
|
|
11
|
+
// GitLab allows duplicate versions by default: a different file under a version it has
|
|
12
|
+
// is accepted, and served from then on. So every file is read back before it is
|
|
13
|
+
// uploaded, and the read-back, not GitLab, keeps a published version immutable.
|
|
14
|
+
|
|
15
|
+
const path = require("path");
|
|
16
|
+
const { execFileSync } = require("child_process");
|
|
17
|
+
|
|
18
|
+
const { isPrerelease } = require("./version-policy");
|
|
19
|
+
const { ChannelError } = require("./channel-error");
|
|
20
|
+
|
|
21
|
+
const GITLAB_COM = "https://gitlab.com/api/v4";
|
|
22
|
+
const TOKEN_HEADERS = { private: "PRIVATE-TOKEN", deploy: "DEPLOY-TOKEN", job: "JOB-TOKEN" };
|
|
23
|
+
|
|
24
|
+
/** The API, the project and the credential, from configuration first and GitLab CI variables second. */
|
|
25
|
+
function channelSettings(options, env = process.env) {
|
|
26
|
+
const tokenEnv = options.tokenEnv || "GITLAB_TOKEN";
|
|
27
|
+
let token = env[tokenEnv];
|
|
28
|
+
let tokenType = options.tokenType || "private";
|
|
29
|
+
if (!TOKEN_HEADERS[tokenType]) {
|
|
30
|
+
throw new ChannelError(`channels.gitlab-package.tokenType must be private, deploy or job, not ${JSON.stringify(tokenType)}`);
|
|
31
|
+
}
|
|
32
|
+
if (!token && env.CI_JOB_TOKEN) {
|
|
33
|
+
token = env.CI_JOB_TOKEN;
|
|
34
|
+
tokenType = "job";
|
|
35
|
+
}
|
|
36
|
+
if (!token) {
|
|
37
|
+
throw new ChannelError(
|
|
38
|
+
`no token in $${tokenEnv}, which channels.gitlab-package.tokenEnv names, and no $CI_JOB_TOKEN. ` +
|
|
39
|
+
"Set one in the release job's environment, or run in a GitLab CI job.");
|
|
40
|
+
}
|
|
41
|
+
const project = options.project === undefined ? env.CI_PROJECT_ID : String(options.project);
|
|
42
|
+
if (!project) {
|
|
43
|
+
throw new ChannelError("the gitlab-package channel needs a project: set channels.gitlab-package.project, " +
|
|
44
|
+
"or run in a GitLab CI job ($CI_PROJECT_ID)");
|
|
45
|
+
}
|
|
46
|
+
const url = (options.url || env.CI_API_V4_URL || GITLAB_COM).replace(/\/+$/, "");
|
|
47
|
+
return { url, project, tokenType, headers: { [TOKEN_HEADERS[tokenType]]: token } };
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
/** The commit a new release's tag points at: the CI job's, or the checkout's HEAD. */
|
|
51
|
+
function releaseRef(options, env = process.env) {
|
|
52
|
+
if (env.CI_COMMIT_SHA) return env.CI_COMMIT_SHA;
|
|
53
|
+
try {
|
|
54
|
+
return execFileSync("git", ["-C", path.resolve(options.baseDir || "."), "rev-parse", "HEAD"],
|
|
55
|
+
{ encoding: "utf8", stdio: "pipe" }).trim();
|
|
56
|
+
} catch {
|
|
57
|
+
throw new ChannelError("release: true needs the commit to tag: $CI_COMMIT_SHA, or a git checkout to read HEAD from");
|
|
58
|
+
}
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
/** The status and GitLab's message, without the status twice: GitLab's messages often start with it. */
|
|
62
|
+
async function answered(response) {
|
|
63
|
+
const text = await response.text();
|
|
64
|
+
let message = text;
|
|
65
|
+
try {
|
|
66
|
+
const body = JSON.parse(text);
|
|
67
|
+
message = body.message || body.error || text;
|
|
68
|
+
} catch {
|
|
69
|
+
// Not JSON: a proxy's error page, say. Reported as it came.
|
|
70
|
+
}
|
|
71
|
+
return message.startsWith(String(response.status)) ? message : `${response.status} ${message}`;
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
const TOKEN_HINT = " Check the token, and that it may read and write this project's package registry.";
|
|
75
|
+
|
|
76
|
+
/** GitLab's copy of a package file: its bytes, or null when it has none. */
|
|
77
|
+
async function readBack(fileUrl, name, headers) {
|
|
78
|
+
const response = await fetch(fileUrl, { headers });
|
|
79
|
+
if (response.status === 404) return null;
|
|
80
|
+
if (!response.ok) {
|
|
81
|
+
throw new ChannelError(`could not read ${name} back from ${fileUrl}: ${await answered(response)}.` +
|
|
82
|
+
(response.status === 401 || response.status === 403 ? TOKEN_HINT : ""));
|
|
83
|
+
}
|
|
84
|
+
return Buffer.from(await response.arrayBuffer());
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
function notThisOne(name, manifest) {
|
|
88
|
+
return new ChannelError(`GitLab already has ${name} in ${manifest.target} ${manifest.version}, and it is not this one; ` +
|
|
89
|
+
"a published version is never replaced, so publish the change as a new version");
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
/** Uploads one file unless GitLab has it: true when it was already there, identical. */
|
|
93
|
+
async function upload(fileUrl, name, bytes, manifest, headers) {
|
|
94
|
+
const held = await readBack(fileUrl, name, headers);
|
|
95
|
+
if (held) {
|
|
96
|
+
if (held.equals(bytes)) return true;
|
|
97
|
+
throw notThisOne(name, manifest);
|
|
98
|
+
}
|
|
99
|
+
const response = await fetch(fileUrl, { method: "PUT", headers, body: bytes });
|
|
100
|
+
if (response.ok) return false;
|
|
101
|
+
const refused = `${await answered(response)}`;
|
|
102
|
+
// Refused, perhaps as a duplicate another run published meanwhile: compare that one.
|
|
103
|
+
const now = await readBack(fileUrl, name, headers).catch(() => null);
|
|
104
|
+
if (now && now.equals(bytes)) return true;
|
|
105
|
+
if (now) throw notThisOne(name, manifest);
|
|
106
|
+
throw new ChannelError(`GitLab refused ${name}: ${refused}.` +
|
|
107
|
+
(response.status === 401 || response.status === 403 ? TOKEN_HINT : ""));
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
async function api(url, headers, method = "GET", body) {
|
|
111
|
+
return fetch(url, {
|
|
112
|
+
method,
|
|
113
|
+
headers: body === undefined ? headers : { ...headers, "Content-Type": "application/json" },
|
|
114
|
+
body: body === undefined ? undefined : JSON.stringify(body),
|
|
115
|
+
});
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
/** Creates the tag's release, or adds the package links an existing one lacks; never changes anything else. */
|
|
119
|
+
async function release(settings, manifest, options, links, ref, log) {
|
|
120
|
+
const { format, DEFAULT_DESCRIPTION } = require("./channels");
|
|
121
|
+
const tag = format(options.tagFormat || "{target}-v{version}", manifest);
|
|
122
|
+
const releases = `${settings.url}/projects/${encodeURIComponent(settings.project)}/releases`;
|
|
123
|
+
const existing = await api(`${releases}/${encodeURIComponent(tag)}`, settings.headers);
|
|
124
|
+
if (existing.status === 404) {
|
|
125
|
+
const created = await api(releases, settings.headers, "POST", {
|
|
126
|
+
tag_name: tag,
|
|
127
|
+
ref,
|
|
128
|
+
name: format(options.title || "{target} {version}", manifest),
|
|
129
|
+
description: format(options.notes || options.description || DEFAULT_DESCRIPTION, manifest),
|
|
130
|
+
assets: { links },
|
|
131
|
+
});
|
|
132
|
+
if (!created.ok) throw new ChannelError(`GitLab refused the release ${tag}: ${await answered(created)}`);
|
|
133
|
+
log(`-- Created the GitLab release ${tag}`);
|
|
134
|
+
return;
|
|
135
|
+
}
|
|
136
|
+
if (!existing.ok) {
|
|
137
|
+
throw new ChannelError(`could not read the GitLab release ${tag}: ${await answered(existing)}`);
|
|
138
|
+
}
|
|
139
|
+
// A release's link names are unique: one already named, or already pointing here, is left as it is.
|
|
140
|
+
const held = ((await existing.json()).assets || {}).links || [];
|
|
141
|
+
for (const link of links.filter((l) => !held.some((h) => h.url === l.url || h.name === l.name))) {
|
|
142
|
+
const added = await api(`${releases}/${encodeURIComponent(tag)}/assets/links`, settings.headers, "POST", link);
|
|
143
|
+
if (!added.ok) throw new ChannelError(`GitLab refused a link on the release ${tag}: ${await answered(added)}`);
|
|
144
|
+
log(`-- Linked ${link.name} from the GitLab release ${tag}`);
|
|
145
|
+
}
|
|
146
|
+
}
|
|
147
|
+
|
|
148
|
+
function publishGitlabPackage(archive, manifest, options, log) {
|
|
149
|
+
const { format } = require("./channels");
|
|
150
|
+
const settings = channelSettings(options);
|
|
151
|
+
const wantsRelease = options.release === true && !isPrerelease(manifest.version);
|
|
152
|
+
if (wantsRelease && settings.tokenType === "deploy") {
|
|
153
|
+
throw new ChannelError("a deploy token cannot create a GitLab release: GitLab gives it the package registry only. " +
|
|
154
|
+
"Publish with $CI_JOB_TOKEN, or a personal or project access token with the api scope, or set release: false.");
|
|
155
|
+
}
|
|
156
|
+
const ref = wantsRelease ? releaseRef(options) : null;
|
|
157
|
+
|
|
158
|
+
const packageName = format(options.packageName || "{target}", manifest);
|
|
159
|
+
const base = `${settings.url}/projects/${encodeURIComponent(settings.project)}/packages/generic/` +
|
|
160
|
+
`${encodeURIComponent(packageName)}/${encodeURIComponent(manifest.version)}`;
|
|
161
|
+
const files = [
|
|
162
|
+
{ name: path.basename(archive), bytes: require("fs").readFileSync(archive) },
|
|
163
|
+
{ name: "manifest.json", bytes: Buffer.from(JSON.stringify(manifest, null, 2) + "\n") },
|
|
164
|
+
].map((file) => ({ ...file, url: `${base}/${encodeURIComponent(file.name)}` }));
|
|
165
|
+
|
|
166
|
+
return (async () => {
|
|
167
|
+
let identical = 0;
|
|
168
|
+
for (const file of files) {
|
|
169
|
+
if (await upload(file.url, file.name, file.bytes, manifest, settings.headers)) identical += 1;
|
|
170
|
+
}
|
|
171
|
+
const alreadyPublished = identical === files.length;
|
|
172
|
+
log(alreadyPublished
|
|
173
|
+
? `-- ${packageName} ${manifest.version} is already in GitLab project ${settings.project}, identical; nothing to do`
|
|
174
|
+
: `-- Published ${packageName} ${manifest.version} to GitLab project ${settings.project}`);
|
|
175
|
+
|
|
176
|
+
if (options.release === true && !wantsRelease) {
|
|
177
|
+
log(`-- no GitLab release for a pre-release: ${manifest.version} would become the latest release`);
|
|
178
|
+
}
|
|
179
|
+
if (wantsRelease) {
|
|
180
|
+
await release(settings, manifest, options,
|
|
181
|
+
files.map((file) => ({ name: file.name, url: file.url, link_type: "package" })), ref, log);
|
|
182
|
+
}
|
|
183
|
+
return { location: files[0].url, ...(alreadyPublished ? { alreadyPublished } : {}) };
|
|
184
|
+
})();
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
module.exports = { publishGitlabPackage, channelSettings };
|
package/src/pack.js
CHANGED
|
@@ -22,6 +22,21 @@ function sha256(file) {
|
|
|
22
22
|
return crypto.createHash("sha256").update(fs.readFileSync(file)).digest("hex");
|
|
23
23
|
}
|
|
24
24
|
|
|
25
|
+
/**
|
|
26
|
+
* A repository URL fit to publish: without the credentials a remote URL can carry --
|
|
27
|
+
* GitLab CI checks out with `https://gitlab-ci-token:<job token>@host/...`, and a
|
|
28
|
+
* clone made with a token in its URL keeps it there -- and without a query string or
|
|
29
|
+
* fragment, where some hosts carry one too. An scp-like `git@host:path` and a local
|
|
30
|
+
* path are left alone: neither holds a secret.
|
|
31
|
+
*/
|
|
32
|
+
function publicUrl(url) {
|
|
33
|
+
if (!url) return url;
|
|
34
|
+
const match = /^([a-z][a-z0-9+.-]*:\/\/)([^/?#]*)([^?#]*)/i.exec(url);
|
|
35
|
+
if (!match) return url;
|
|
36
|
+
const [, scheme, authority, rest] = match;
|
|
37
|
+
return scheme + authority.slice(authority.lastIndexOf("@") + 1) + rest;
|
|
38
|
+
}
|
|
39
|
+
|
|
25
40
|
function gitInfo(cwd) {
|
|
26
41
|
const git = (args) => {
|
|
27
42
|
try {
|
|
@@ -31,7 +46,7 @@ function gitInfo(cwd) {
|
|
|
31
46
|
}
|
|
32
47
|
};
|
|
33
48
|
return {
|
|
34
|
-
repository: git(["config", "--get", "remote.origin.url"]),
|
|
49
|
+
repository: publicUrl(git(["config", "--get", "remote.origin.url"])),
|
|
35
50
|
commit: git(["rev-parse", "HEAD"]),
|
|
36
51
|
};
|
|
37
52
|
}
|
|
@@ -145,4 +160,4 @@ function pack(config, target, { version, closureSha256, outDir, log = () => {} }
|
|
|
145
160
|
return { archive, manifest: data, manifestPath };
|
|
146
161
|
}
|
|
147
162
|
|
|
148
|
-
module.exports = { pack, manifest, sha256, MANIFEST_NAME, MANIFEST_SCHEMA_VERSION, PackError };
|
|
163
|
+
module.exports = { pack, manifest, sha256, MANIFEST_NAME, MANIFEST_SCHEMA_VERSION, PackError, publicUrl };
|
package/src/remote.js
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
|
|
3
|
+
// Whether `publish` may contact anything.
|
|
4
|
+
//
|
|
5
|
+
// A channel's local mode writes files; its remote mode sends them somewhere: a
|
|
6
|
+
// repository or feed at a URL, a registry, a release. API_ONLY_PUBLISHER_REMOTE=never
|
|
7
|
+
// refuses every remote mode, so the Publisher contacts nothing. The Docker image sets
|
|
8
|
+
// it, and is sealed unless a pipeline sets allow; outside the image it is unset, and
|
|
9
|
+
// the Publisher behaves as it always has.
|
|
10
|
+
//
|
|
11
|
+
// It is a promise the Publisher keeps, not an enforcement: whoever sets the variable
|
|
12
|
+
// can change it. A guarantee needs the platform to isolate the job's network.
|
|
13
|
+
|
|
14
|
+
const { ChannelError } = require("./channel-error");
|
|
15
|
+
|
|
16
|
+
const REMOTE_ENV = "API_ONLY_PUBLISHER_REMOTE";
|
|
17
|
+
|
|
18
|
+
/** `allow`, the default when unset or empty, or `never`; anything else is refused. */
|
|
19
|
+
function remoteMode(env = process.env) {
|
|
20
|
+
const value = env[REMOTE_ENV];
|
|
21
|
+
if (value === undefined || value === "") return "allow";
|
|
22
|
+
if (value !== "allow" && value !== "never") {
|
|
23
|
+
throw new ChannelError(`${REMOTE_ENV} must be allow or never, not ${JSON.stringify(value)}`);
|
|
24
|
+
}
|
|
25
|
+
return value;
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
const URL = /^https?:\/\//;
|
|
29
|
+
|
|
30
|
+
/** Whether `channel`, with these settings, would contact anything. */
|
|
31
|
+
function isRemote(channel, settings = {}) {
|
|
32
|
+
switch (channel) {
|
|
33
|
+
case "maven":
|
|
34
|
+
case "nuget":
|
|
35
|
+
return URL.test(settings.repository || "");
|
|
36
|
+
case "npm":
|
|
37
|
+
return settings.publish !== false;
|
|
38
|
+
case "github-release":
|
|
39
|
+
case "gitlab-package":
|
|
40
|
+
return true;
|
|
41
|
+
default:
|
|
42
|
+
return false;
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
/**
|
|
47
|
+
* Refuses, under `never`, a run that would use any remote channel, naming each of them,
|
|
48
|
+
* before anything is built, packed or sent. Also refuses an invalid value, whatever the
|
|
49
|
+
* channels: a typo in a switch that keeps the Publisher sealed is not ignored.
|
|
50
|
+
*/
|
|
51
|
+
function refuseRemote(names, configured = {}, env = process.env) {
|
|
52
|
+
if (remoteMode(env) === "allow") return;
|
|
53
|
+
const remote = names.filter((name) => isRemote(name, configured[name] || {}));
|
|
54
|
+
if (remote.length === 0) return;
|
|
55
|
+
throw new ChannelError(
|
|
56
|
+
`${REMOTE_ENV}=never refuses the remote channels ${remote.join(", ")}, so nothing was built, packed or sent. ` +
|
|
57
|
+
`Publish with their local modes, or set ${REMOTE_ENV}=allow where this run may contact them.`);
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
module.exports = { REMOTE_ENV, remoteMode, isRemote, refuseRemote };
|
package/src/version-policy.js
CHANGED
|
@@ -33,6 +33,32 @@ function parse(version) {
|
|
|
33
33
|
};
|
|
34
34
|
}
|
|
35
35
|
|
|
36
|
+
/**
|
|
37
|
+
* Semver precedence: negative when `a` comes before `b`, positive after, 0 when equal.
|
|
38
|
+
* Build metadata is ignored; a release comes after its pre-releases; pre-release
|
|
39
|
+
* identifiers compare numerically when both are numbers, a number before a word.
|
|
40
|
+
*/
|
|
41
|
+
function compare(a, b) {
|
|
42
|
+
const x = parse(a);
|
|
43
|
+
const y = parse(b);
|
|
44
|
+
for (const part of ["major", "minor", "patch"]) {
|
|
45
|
+
if (x[part] !== y[part]) return x[part] - y[part];
|
|
46
|
+
}
|
|
47
|
+
if (x.prerelease === y.prerelease) return 0;
|
|
48
|
+
if (x.prerelease === null) return 1;
|
|
49
|
+
if (y.prerelease === null) return -1;
|
|
50
|
+
const xs = x.prerelease.split(".");
|
|
51
|
+
const ys = y.prerelease.split(".");
|
|
52
|
+
for (let i = 0; i < Math.min(xs.length, ys.length); i++) {
|
|
53
|
+
if (xs[i] === ys[i]) continue;
|
|
54
|
+
const [xn, yn] = [/^\d+$/.test(xs[i]), /^\d+$/.test(ys[i])];
|
|
55
|
+
if (xn && yn) return Number(xs[i]) - Number(ys[i]);
|
|
56
|
+
if (xn !== yn) return xn ? -1 : 1;
|
|
57
|
+
return xs[i] < ys[i] ? -1 : 1;
|
|
58
|
+
}
|
|
59
|
+
return xs.length - ys.length;
|
|
60
|
+
}
|
|
61
|
+
|
|
36
62
|
function isPrerelease(version) {
|
|
37
63
|
// -SNAPSHOT is not semver-legal as Maven spells it, but it is what Maven
|
|
38
64
|
// consumers expect, so it is recognised rather than rejected.
|
|
@@ -65,4 +91,4 @@ function describe(version) {
|
|
|
65
91
|
return { prerelease: true, kind: "other" };
|
|
66
92
|
}
|
|
67
93
|
|
|
68
|
-
module.exports = { parse, isPrerelease, npmDistTag, describe, VersionError, SEMVER };
|
|
94
|
+
module.exports = { parse, compare, isPrerelease, npmDistTag, describe, VersionError, SEMVER };
|