@pnpm/pnpr 0.0.0-26070301 → 0.1.0-alpha.11

package/CHANGELOG.md

@@ -0,0 +1,182 @@
+# @pnpm/pnpr
+
+## 0.1.0-alpha.11
+
+pnpr now serves Cargo, Python, and container registries alongside npm, publishes across all of them in one transaction, and signs users in through OIDC.
+
+### Minor Changes
+
+#### Cargo and Python registries
+
+- pnpr now serves Cargo and Python registries alongside npm from one instance. Hosted Cargo registries support `cargo publish`, `cargo yank`, and crate downloads. Hosted Python registries support `pip install --index-url` and `twine upload`. Upstream registries can proxy crates.io and PyPI with checksum verified downloads, and a router can combine sources from all three ecosystems.
+
+  A registry that serves more than one ecosystem addresses them through `/npm/`, `/cargo/`, and `/pypi/`. A registry that serves only one ecosystem keeps serving packages at the root, so existing npm URLs still work. A `/~<name>/` registry prefix must arrive with an unencoded `~`.
+
+  Registry names can be reused across ecosystems by grouping configuration under `registries.npm`, `registries.cargo`, `registries.pypi`, or `registries.oci`. Router sources and defaults resolve within their own ecosystem.
+
+- `pnpm install` now resolves Cargo and Python dependencies through the server configured in `pnprServer`. The client no longer fetches one sparse-index file per crate in the dependency graph, and no longer downloads a wheel to find out what it requires. If the server does not serve that resolution, pnpm resolves those dependencies locally.
+
+- `pnpm install` can install Cargo dependencies from the sparse registry configured by `cargo.indexUrl`. The generated `Cargo.lock` records that registry as the source of every crate.
+
+- `cargo search` works against a pnpr registry. It matches on crate name over the crates the registry hosts. Each result reports the crate's newest release that is not yanked, and a crate whose releases are all yanked reports its newest release.
+
+#### Container images
+
+- pnpr now serves container images [#14630](https://github.com/pnpm/pnpm/issues/14630). Declare a registry with `ecosystem: oci`, then push to it with `docker`, `podman`, or `skopeo`. The distribution API answers at `/v2/` on the host root, and an image keeps its own name with no registry key in the path. Sign in with `docker login`, using a pnpr token as the password, or set `oci.bearerAuth: true` to hand clients short lived, repository scoped credentials instead.
+
+  Ranged blob downloads, cross repository blob mounts, the referrers API, and paginated tag and repository listings are supported. Authorized users can delete unreferenced blobs, and a blob that a retained manifest still reaches stays protected while another publish is running. `oci.maxBlobBytes` and `oci.maxManifestBytes` cap what a client may upload.
+
+- pnpr can cache image pulls from Docker Hub and GHCR [#14630](https://github.com/pnpm/pnpm/issues/14630). Upstream authentication supports repository scoped bearer tokens, and pnpr verifies every cached image by digest. A proxied download whose contents fail verification is aborted and stays out of the cache.
+
+- pnpr can resume OCI uploads across replicas when it keeps blobs in S3. Abandoned shared upload sessions expire at startup after 24 hours of inactivity. `pnpr oci-gc --registry <name>` collects old, unreferenced image blobs while registry writers are stopped, and `--dry-run` previews the cleanup.
+
+#### Publishing and authentication
+
+- pnpr can now publish packages of more than one ecosystem in a single transaction. `PUT /-/pnpr/v0/publish` takes a batch whose entries each name their `ecosystem`, so a workspace that ships an npm package, a crate, and a Python distribution releases them together. OCI manifests can ride along in the same batch once their layers are uploaded [#14630](https://github.com/pnpm/pnpm/issues/14630). A batch that fails a check publishes none of it, and a server that stops midway finishes the release on the next startup. An entry without an `ecosystem` is an npm publish document, the same one `PUT /-/pnpm/v1/publish` takes.
+
+- pnpr now supports OIDC browser sign in. Administrators can map provider subjects to registry users, and GitHub Actions can publish npm packages without a persistent registry token. Workload publishing is restricted to the packages you configure.
+
+#### Builds and pipelines
+
+- pnpr can share Cargo compilation caches between CI and developers through sccache. Configure `artifacts.compilerCaches` to grant read and publication access separately. sccache can combine its local disk cache with pnpr's remote cache.
+
+- pnpr can store `pnpm pipeline` run reports. Turn it on with `pipeline.enabled` and configure each workspace's `access` and `publish` permissions under `pipeline.workspaces`. `pnpm pipeline --report` and `--report-to` submit run summaries and events, and pnpr answers with authenticated listing and detail endpoints and a web viewer. Records are kept with the hosted packages, so a run submitted through one replica is listed and served by every other.
+
+#### Discovery and administration
+
+- pnpr can now serve browser registry UIs through an origin allowlist. Search supports pagination and maintainer filters, organization package listings are available, and authenticated publishers are recorded for discovery. Upstream discovery can be enabled per registry. Passing `browse=true` to the search APIs pages through the npm packages and Cargo crates a registry hosts, respecting its routing and package access rules.
+
+  A registry directory endpoint lists the named registries, ecosystem endpoints, and routing order that the current user can see.
+
+- pnpr now reads every command line option from an environment variable when the flag is omitted. The variable is named after the flag with a `PNPR_` prefix, so `--public-url` becomes `PNPR_PUBLIC_URL` and `--disable-resolver` becomes `PNPR_DISABLE_RESOLVER`. A flag given on the command line wins over its environment variable. Boolean flags accept `true`, `1`, `yes`, `on`, `false`, `0`, `no`, and `off`.
+
+### Patch Changes
+
+- pnpr now rejects a package name that carries `?`, `#`, `%`, whitespace, or a control character, and holds artifact filenames to the same rule. A percent encoded delimiter let a request read an upstream package under a name the access rules had not checked.
+
+- pnpr now resolves JSR packages without configuration. npm.jsr.io is a built-in public route, like the npm registry. The built-in routes are reachable over HTTPS only.
+
+- A publish that loses a write race now reports `document_write_conflict` on every registry surface, and the message names the package document [#14599](https://github.com/pnpm/pnpm/issues/14599). A publish whose file another writer had already stored under the same name now answers `409` instead of `201`, which it answered while leaving the version out of the registry document. Both cases arise only where several pnpr instances share one object store.
+
+- A staged publish is now approved once, whichever replica of a shared registry the approval reaches [#12199](https://github.com/pnpm/pnpm/issues/12199). A second approval of the same stage answers `409`. Rejecting a stage stops an approval that has not published it yet.
+
+- Registration no longer fails when several users sign up at the same time against a libsql backend with a configured user limit.
+
+- pnpr now records a URL's query exactly as it was written when it redacts the credentials in an error message. It re-encoded the query, rewriting `?options=a%20b` as `?options=a+b`.
+
+## 0.1.0-alpha.10
+
+### Patch Changes
+
+- A shared build artifact publication that cannot unregister itself no longer stops the registry reclaiming space or refusing further publications. A publication says at intervals that it is still working, and a registration that has gone quiet for an hour is written off, so a publication whose bookkeeping write failed stops holding back the collector that reclaims unreferenced blobs and returns the compatibility scopes a failed publication claimed, and stops counting toward the limit on publications in flight.
+
+## 0.1.0-alpha.9
+
+### Minor Changes
+
+- Made the signed shared-artifact cache horizontally scalable with S3-compatible storage and an independent top-level `artifacts` feature toggle.
+
+- Generalized the experimental shared-artifact protocol so candidates and signed payloads identify a discriminated subject. Dependency side effects use package and source-integrity subjects, while workspace tasks use project and task subjects.
+
+  This changes shared-artifact request bodies and signed payloads. A pnpr server and its clients have to be on matching versions.
+
+### Patch Changes
+
+- A published build artifact is now immutable: one input key and one set of compatibility constraints admit one artifact, so publishing a different one over it answers `409 Conflict`, the same as a re-published `name@version`. Republishing the identical artifact still succeeds. Artifacts already stored by an earlier version keep their slot, so upgrading a populated registry does not leave them replaceable.
+
+- A published build artifact is now immutable for every consumer it can reach, not only for the exact compatibility constraints it declares. Publishing an artifact that reaches a machine an existing one already reaches answers `409 Conflict`, so a later universal, broader, or higher-floor build can no longer take precedence over the artifact a consumer already receives. Publishing across a platform matrix is unaffected: artifacts for different operating systems, architectures, or Node majors reach no machine in common and coexist as before.
+
+- pnpr reclaims unreferenced shared artifact blobs after ambiguous object-storage write failures once active publications drain.
+
+- Recognize `pnpm install --fix-lockfile`, including filtered installs, and regenerate broken lockfile metadata while preserving compatible locked versions [pnpm/pnpm#14250](https://github.com/pnpm/pnpm/issues/14250).
+
+- pnpr retains shared artifact quota after object storage reports an ambiguous write failure.
+
+## 0.1.0-alpha.8
+
+### Minor Changes
+
+- Added an opt-in proof of concept that lets installs reuse a dependency's build output across machines, by publishing and restoring signed, organization-scoped artifacts through pnpr instead of running the lifecycle scripts locally.
+
+  Configure it with the new `remoteSideEffectsCache` setting. A workspace names the eligible `organization` and `packages`; everything describing the act of signing — `publish`, `keyId`, `builderId`, `trustedKeys`, `privateKey` and the provenance fields — is refused in `pnpm-workspace.yaml` and read from the global config file or the environment instead.
+
+- Hosted pnpr registries now serve newly published original artifacts from registry-scoped SHA-512 digest URLs.
+
+- Allowed `pnpm update --patches` to refresh registry revisions through a configured pnpr server while retaining locked package versions.
+
+- Added explicit registry revision selection with `<version>+rN` and `pnpm update --patches` for refreshing revision artifacts without changing package versions. Registry-backed lockfile policy checks recognize historical revisions, and pnpr now preserves safe revision histories from upstream registries.
+
+- Added support for registry replacement tarballs using standard integrity values, explicit revision fields, registry routing from the `registries` setting, non-redirecting integrity-addressed URLs, canonical safe-integer revision numbers, and pnpr proxying for immutable upstream revision artifacts.
+
+### Patch Changes
+
+- Forward `patchedDependencies` hashes and `packageExtensions` to pnpr so server-side resolution preserves patches and package extensions in the lockfile and installed packages.
+
+## 0.1.0-alpha.7
+
+### Minor Changes
+
+- Packument responses now carry a `Last-Modified` header derived from the document's `time.modified`, so a client's release-age check can learn the package-level last-publish bound from a cheap `HEAD` probe instead of downloading the metadata body.
+
+- A pnpr resolve request now carries the client's registries the way the `registries` setting declares them — keyed by URL, with the scopes routed to each, the bare-specifier prefix each answers to, and each one's `serverType` — in place of the prefix map it used to send.
+
+  The server routes them through the same inversion the config reader runs, so a pnpr-served install resolves a scoped dependency from the registry that scope is routed to, which it previously could not: only the default registry and the prefix-addressed ones reached the server. A declared `serverType` reaches it too, so the tarball URLs pnpr omits from the lockfile match the ones the client reconstructs.
+
+  Built-in scope routes the project has not pointed elsewhere are not declared, so a pnpr server's allowlist is not asked about `npm.jsr.io` on requests that resolve no JSR package.
+
+  A registry a request only declares is no longer refused up front for being off the server's allowlist — a client describes its whole configuration, including scopes a given resolve never reaches, so a stray `@scope:registry` in a developer's `~/.npmrc` no longer fails every install against a pnpr server that does not serve it. The boundary moves to the fetch itself: an origin the resolve does reach is refused before the request leaves the server, with the same message.
+
+  This changes the resolve and verify-lockfile request bodies. A pnpr server and its clients have to be on matching versions; the protocol is still experimental and unversioned.
+
+- A resolve request now carries the client's `resolutionMode`, so an install delegated to a pnpr server picks versions the way the client would. `time-based` and `lowest-direct` reached the server as nothing at all, leaving it on its `highest` default: the returned lockfile pinned the highest satisfying version of every dependency, and the setting appeared to be ignored.
+
+  This adds a field to the resolve request body. A server older than its client ignores it and keeps resolving `highest`; the protocol is still experimental and unversioned.
+
+### Patch Changes
+
+- Fixed `pnpm` installs using pnpr to honor the client's `autoInstallPeers`, `dedupePeers`, and `excludeLinksFromLockfile` settings [pnpm/pnpm#13389](https://github.com/pnpm/pnpm/issues/13389).
+
+## 0.1.0-alpha.6
+
+### Patch Changes
+
+- Validated forwarded catalog targets and kept catalog-specific resolutions isolated in the server cache.
+
+## 0.1.0-alpha.5
+
+### Patch Changes
+
+- Fixed `catalog:` references in dependencies and overrides failing to resolve when installing through a pnpr server, which errored with "No catalog entry '<name>' was found for catalog 'default'." even though the catalog entry existed. Also fixed a crash on Windows when installing a nested workspace member (e.g. `packages/foo`) through a pnpr server [#13232](https://github.com/pnpm/pnpm/issues/13232).
+
+## 0.1.0-alpha.4
+
+### Patch Changes
+
+- Fixed Pacquet workspace commands to honor project filters, preserve complete lockfile state, and materialize only the selected dependency closure, including pnpr-backed installs.
+
+- Fixed pnpr workspace resolution to preserve project names and versions for `workspace:` dependencies.
+
+## 0.1.0-alpha.3
+
+### Patch Changes
+
+- The `--help` text now reads as user-facing help rather than developer documentation. Command and flag descriptions say what each option does for you, and the leftover markdown that was printing verbatim in the terminal — intra-doc links, an inline link, and an HTML-like path placeholder — has been cleaned out.
+
+## 0.1.0-alpha.2
+
+### Patch Changes
+
+- Bump version.
+
+## 0.1.0-alpha.1
+
+### Patch Changes
+
+- On publish, the README of the `latest` version is now hoisted to the packument's top-level `readme` (and `readmeFilename`) field, matching npm and verdaccio. Publish clients only send the readme inside the version manifest, so without this a package published to pnpr exposed no top-level readme for full-packument consumers and registry UIs to render.
+
+## 0.1.0-alpha.0
+
+### Minor Changes
+
+- `@pnpm/pnpr` is now versioned with changesets and released from the unified release workflow. Versions switch from the `0.0.0-<datestamp>` scheme to semver, starting the 0.1.0 prerelease line at 0.1.0-alpha.0.
+- The install script exits without error in the pnpm monorepo checkout, where the per-platform binary packages are not generated.

package/README.md

@@ -35,6 +35,32 @@ by default. Point a client at it with:
 pnpm config set registry http://127.0.0.1:7677/
 ```
 
+### Install accelerator
+
+pnpr also resolves dependencies on a client's behalf. Point pnpm at it
+with `pnprServer` and pnpm sends its manifests instead of resolving
+locally:
+
+```sh
+pnpm config set pnprServer http://127.0.0.1:7677/
+```
+
+npm, Cargo and Python dependencies all resolve this way.
+
+For Cargo, pnpr walks the crates.io sparse index and answers with the
+`Cargo.lock`, so a workspace resolves without the client fetching one
+index file per crate.
+
+For Python, pnpr reads the index and answers with the `pylock.toml`.
+Reading what a distribution requires means reading a wheel's `METADATA`,
+so a client resolving alone downloads whole wheels for versions it then
+rejects; pnpr reads the metadata file the index publishes beside each
+wheel instead. When an index publishes no such file, pnpr downloads the
+wheel itself and keeps the metadata it holds, so only the first client
+pays for that download.
+
+Everything pnpr reads is cached for every client that follows.
+
 ## CLI flags
 
 | Flag | Description |
@@ -45,6 +71,19 @@ pnpm config set registry http://127.0.0.1:7677/
 | `--cache <path>` | Override the disposable proxy-cache directory (the mirror of upstream registries plus the resolver cache). Defaults to a `.pnpr-cache` subdirectory of `--storage`. |
 | `--public-url <url>` | URL clients should use to reach the server, used when rewriting `dist.tarball` in served packuments. Defaults to `http://<listen>`. |
 | `--packument-ttl-secs <n>` | Seconds before a cached packument is considered stale and refetched. |
+| `--osv` | Enable local OSV npm vulnerability checks. Requires a local OSV npm database at `--osv-db` or `<cache>/osv/npm/all.zip`. |
+| `--osv-db <path>` | Path to the local OSV npm database zip or extracted JSON directory. |
+| `--disable-registry` | Disable the npm-registry surface (packument and tarball reads, publish, unpublish, dist-tag, search). |
+| `--disable-resolver` | Disable the install-accelerator surface (`/-/pnpr/v0/resolve` for npm, Cargo and Python, `/-/pnpr/v0/verify-lockfile`). |
+| `--disable-artifacts` | Disable the signed shared-artifact surface. |
+
+Every flag can also be set through an environment variable named after
+it: `PNPR_` followed by the flag name in upper case with dashes replaced
+by underscores. `--public-url` becomes `PNPR_PUBLIC_URL` and
+`--packument-ttl-secs` becomes `PNPR_PACKUMENT_TTL_SECS`. A flag given on
+the command line wins over its environment variable. Boolean flags such
+as `--disable-registry` accept `true`, `1`, `yes`, `on`, `false`, `0`,
+`no`, and `off`.
 
 Log level is controlled via the standard `RUST_LOG` environment
 variable (e.g. `RUST_LOG=debug pnpr`).
@@ -272,6 +311,40 @@ When the `backend:` block is absent, auth stays on local disk and the
 `auth.htpasswd` / `auth.tokens` settings apply as before. The
 `auth.htpasswd.max_users` registration cap is honored either way.
 
+### Running several replicas
+
+Several `pnpr` processes can serve one registry behind a load balancer once
+they share their state. Give every replica the same `s3:` bucket for hosted
+packages and the same `backend:` database for users and tokens, and give each
+its own `storage` path for what stays local.
+
+Requests need no affinity. Every write into the shared bucket is conditional:
+a publish, a `dist-tag` change, a partial unpublish, and the approval of a
+staged publish each rewrite the package document under the `ETag` they read it
+at, and a replica that loses the race re-reads and merges on top of what the
+other one wrote. A published tarball is written only if its key is still free,
+so two replicas publishing one version can never overwrite each other's bytes:
+the one whose bytes did not land is told so with a `409` instead of
+advertising an integrity the store no longer serves. A staged publish is
+approved once, whichever replica the approval reaches.
+
+This needs an object store that honors the `If-Match` and `If-None-Match`
+preconditions. AWS S3, Cloudflare R2 and MinIO do.
+
+What stays local to each replica:
+
+- the proxy cache of upstream registries, and the resolver cache
+- publish staging scratch and the commit journal that rolls an interrupted
+  publish forward, so keep each replica's `storage` path on a volume that
+  outlives its restarts
+- blob upload sessions of the image registry when hosted packages are on
+  local disk, which is why `docker push` needs sticky sessions there; with
+  `s3:` an upload continues on any replica
+- accounts and tokens, unless a `backend:` database is configured
+
+Pipeline run records are shared: a run submitted through one replica is listed
+and served by every other, and a run id stays append-only across all of them.
+
 ## License
 
 Source-available under the [PolyForm Shield License 1.0.0](https://github.com/pnpm/pnpm/blob/main/pnpr/LICENSE.md) — **not** open source. You may run, modify, and self-host pnpr for any purpose except providing a product that competes with it. Commercial / non-compete licenses are available from Zoltan Kochan (<https://kochan.io>).

package/install.js

@@ -44,6 +44,13 @@ const PLATFORMS = {
 setup()
 
 function setup () {
+  // The committed manifest has no `optionalDependencies`; generate-packages.mjs
+  // adds them at release time. Without them this is the monorepo checkout, where
+  // the wrapper is a workspace package and there is no native binary to link.
+  if (readOwnManifest().optionalDependencies == null) {
+    return
+  }
+
   const candidates = getBinCandidates()
   if (candidates.length === 0) {
     fail(`@pnpm/pnpr does not ship a prebuilt binary for ${platform}-${arch}.`)
@@ -142,6 +149,23 @@ function fail (message) {
   process.exit(1)
 }
 
+// A successful read with no optionalDependencies is the dev checkout (setup
+// no-ops there); a read/parse failure is a corrupt published package and must
+// not be silently swallowed into that same no-op path.
+function readOwnManifest () {
+  const manifestPath = path.join(ownDir, 'package.json')
+  let manifest
+  try {
+    manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8'))
+  } catch (err) {
+    throw new Error(`Failed to read ${manifestPath}: ${err.message}`)
+  }
+  if (typeof manifest !== 'object' || manifest == null || Array.isArray(manifest)) {
+    throw new Error(`Expected ${manifestPath} to contain a JSON object`)
+  }
+  return manifest
+}
+
 /**
  * Native binary specifiers to try, most-preferred first; empty when the host is
  * unsupported. The linux glibc/musl pair is ordered by detected libc, which

package/package.json

@@ -1,5 +1,6 @@
 {
   "name": "@pnpm/pnpr",
+  "version": "0.1.0-alpha.11",
   "description": "pnpm-compatible npm registry server, written in Rust",
   "keywords": [
     "pnpm",
@@ -14,6 +15,9 @@
     "url": "https://github.com/pnpm/pnpm",
     "directory": "pnpr/npm/pnpr"
   },
+  "bin": {
+    "pnpr": "bin/pnpr"
+  },
   "type": "module",
   "engines": {
     "node": ">=18.*"
@@ -23,21 +27,17 @@
     "install.js",
     "LICENSE.md"
   ],
-  "version": "0.0.0-26070301",
   "optionalDependencies": {
-    "@pnpm/pnpr.win32-x64": "0.0.0-26070301",
-    "@pnpm/pnpr.win32-arm64": "0.0.0-26070301",
-    "@pnpm/pnpr.darwin-x64": "0.0.0-26070301",
-    "@pnpm/pnpr.darwin-arm64": "0.0.0-26070301",
-    "@pnpm/pnpr.linux-x64": "0.0.0-26070301",
-    "@pnpm/pnpr.linux-arm64": "0.0.0-26070301",
-    "@pnpm/pnpr.linux-x64-musl": "0.0.0-26070301",
-    "@pnpm/pnpr.linux-arm64-musl": "0.0.0-26070301"
+    "@pnpm/pnpr.win32-x64": "0.1.0-alpha.11",
+    "@pnpm/pnpr.win32-arm64": "0.1.0-alpha.11",
+    "@pnpm/pnpr.darwin-x64": "0.1.0-alpha.11",
+    "@pnpm/pnpr.darwin-arm64": "0.1.0-alpha.11",
+    "@pnpm/pnpr.linux-x64": "0.1.0-alpha.11",
+    "@pnpm/pnpr.linux-arm64": "0.1.0-alpha.11",
+    "@pnpm/pnpr.linux-x64-musl": "0.1.0-alpha.11",
+    "@pnpm/pnpr.linux-arm64-musl": "0.1.0-alpha.11"
   },
   "scripts": {
     "preinstall": "node install.js"
-  },
-  "bin": {
-    "pnpr": "bin/pnpr"
   }
 }