@vltpkg/registry-client 0.0.0-3 → 0.0.0-30

package/README.md

@@ -2,76 +2,73 @@
 
 # @vltpkg/registry-client
 
-This is a very light wrapper around undici, optimized for interfacing with an npm registry.
+This is a very light wrapper around undici, optimized for interfacing
+with an npm registry.
 
-**[Cache Unzipped](#cache-unzipped)**
-·
-**[Integrity Options](#integrity-options)**
-·
-**[Usage](#usage)**
+**[Cache Unzipped](#cache-unzipped)** ·
+**[Integrity Options](#integrity-options)** · **[Usage](#usage)**
 
 ## Overview
 
-Any response with `immutable` in the `cache-control` header, or with a `content-type` of `application/octet-stream` or a path ending in `.tgz`, will be cached forever and never requested again as long as the cache survives.
+Any response with `immutable` in the `cache-control` header, or with a
+`content-type` of `application/octet-stream` or a path ending in
+`.tgz`, will be cached forever and never requested again as long as
+the cache survives.
 
 If the request has a cached response:
 
-- Cached responses with `immutable` in the `cache-control`
-  header will be returned from cache without a network request,
-  no matter what.
-- Cached responses with a `content-type` of
-  `application/octet-stream` will be returned from cache without
-  a network request, no matter what, because tarballs are
-  immutable.
+- Cached responses with `immutable` in the `cache-control` header will
+  be returned from cache without a network request, no matter what.
+- Cached responses with a `content-type` of `application/octet-stream`
+  will be returned from cache without a network request, no matter
+  what, because tarballs are immutable.
 - Cached responses with `max-age=<n>` or `s-max-age=<n>` will be
-  served from cache without a network request if it's less than
-  `<n>` seconds old.
+  served from cache without a network request if it's less than `<n>`
+  seconds old.
 - Otherwise, a network request to the registry will be made
-  - if an `etag` is present in the cached response, it will be
-    used as the `if-none-match` header.
-  - If a `last-modified` header is in the response, that will
-    be used as the `if-modified-since` request header.
-  - If there is no `last-modified` header, then use the `mtime`
-    of the cache file as the `if-modified-since` header.
+  - if an `etag` is present in the cached response, it will be used as
+    the `if-none-match` header.
+  - If a `last-modified` header is in the response, that will be used
+    as the `if-modified-since` request header.
+  - If there is no `last-modified` header, then use the `mtime` of the
+    cache file as the `if-modified-since` header.
 
 This is the extent of the cache control logic. It is not a
-full-featured spec-compliant caching HTTP client, because that is
-not needed for this use case. Every response will be cached, even
-if the registry headers don't technically allow it.
+full-featured spec-compliant caching HTTP client, because that is not
+needed for this use case. Every response will be cached, even if the
+registry headers don't technically allow it.
 
 ## Cache Unzipped
 
-Client always sends `accept-encoding: gzip;q=1.0, *;q=0.5`
-header when making requests, to save time on the wire.
+Client always sends `accept-encoding: gzip;q=1.0, *;q=0.5` header when
+making requests, to save time on the wire.
 
-If response has `content-encoding: gzip`, then we swap out the
-body for the unzipped response body in the cache, as if it was
-not gzipped in the first place. This _must_ be done before
-returning the response, because you can't `JSON.parse()` a
-gzipped response anyway.
+If response has `content-encoding: gzip`, then we swap out the body
+for the unzipped response body in the cache, as if it was not gzipped
+in the first place. This _must_ be done before returning the response,
+because you can't `JSON.parse()` a gzipped response anyway.
 
-If the response is `content-type: application/octet-stream` and
-starts with the gzip header, then we return the raw body as we
-received it, but as a best-effort background job, unzip it and
-update the cache entry to be an unzipped response body. This is
-done in the `@vltpkg/cache-unzip` worker.
+If the response is `content-type: application/octet-stream` and starts
+with the gzip header, then we return the raw body as we received it,
+but as a best-effort background job, unzip it and update the cache
+entry to be an unzipped response body. This is done in the
+`@vltpkg/cache-unzip` child process.
 
 So,
 
-- json responses will always be un-zipped, in the response and in
-  the cache.
-- artifact responses _may_ be gzipped (and thus, have to be
-  unzipped by the unpack operation), but will eventually be
-  cached as unzipped tarballs.
+- json responses will always be un-zipped, in the response and in the
+  cache.
+- artifact responses _may_ be gzipped (and thus, have to be unzipped
+  by the unpack operation), but will eventually be cached as unzipped
+  tarballs.
 
-Thus, the `content-length` response header will _usually_ not
-match the actual byte length of the response body.
+Thus, the `content-length` response header will _usually_ not match
+the actual byte length of the response body.
 
 ## Integrity Options
 
-An `integrity` option may be specified in a
-`fetchOptions.context` object, or in the options provided to
-`cache.set()`. For example:
+An `integrity` option may be specified in a `fetchOptions.context`
+object, or in the options provided to `cache.set()`. For example:
 
 ```js
 const integrity = `sha512-${base64hash}`
@@ -86,16 +83,16 @@ cache.set(key, value, { integrity })
 const value = await cache.fetch(key, { context: { integrity } })
 ```
 
-If the integrity provided is obviously not a valid sha512
-`Integrity` string, then it is ignored.
+If the integrity provided is obviously not a valid sha512 `Integrity`
+string, then it is ignored.
 
-Integrity values are not calculated or verified. The caller must do this
-check, if desired.
+Integrity values are not calculated or verified. The caller must do
+this check, if desired.
 
 Note that the integrity provided to `cache.fetch()` or `cache.set()`
 does _not_ typically match the calculated integrity of the object
-being cached. Typically, the integrity is related to the body of
-the response that a `@vltpkg/registry-client.CacheEntry` object
+being cached. Typically, the integrity is related to the body of the
+response that a `@vltpkg/registry-client.CacheEntry` object
 represents.
 
 ## Usage

package/dist/esm/cache-entry.d.ts

@@ -1,44 +1,113 @@
+import type { ErrorCauseOptions } from '@vltpkg/error-cause';
 import type { Integrity, JSONField } from '@vltpkg/types';
-import type { InspectOptions } from 'util';
+import ccp from 'cache-control-parser';
+import type { InspectOptions } from 'node:util';
 export type JSONObj = Record<string, JSONField>;
 declare const kCustomInspect: unique symbol;
+export type CacheEntryOptions = {
+    /**
+     * An optional body to use.
+     *
+     * This is used when decoding a cache entry from a buffer, and the body
+     * is already in a ArrayBuffer we can use. When this option is
+     * provided the `addBody` method should not be used.
+     */
+    body?: Uint8Array;
+    /**
+     * An optional content length of the body to use, if undefined the
+     * content-length header will be used.
+     */
+    contentLength?: number;
+    /**
+     * The expected integrity value for this response body
+     */
+    integrity?: Integrity;
+    /**
+     * Whether to trust the integrity, or calculate the actual value.
+     *
+     * This indicates that we just accept whatever the integrity is as the actual
+     * integrity for saving back to the cache, because it's coming directly from
+     * the registry that we fetched a packument from, and is an initial gzipped
+     * artifact request.
+     */
+    trustIntegrity?: boolean;
+    /**
+     * If the server does not serve a `stale-while-revalidate` value in the
+     * `cache-control` header, then this multiplier is applied to the `max-age`
+     * or `s-maxage` values.
+     *
+     * By default, this is `60`, so for example a response that is cacheable for
+     * 5 minutes will allow a stale response while revalidating for up to 5
+     * hours.
+     *
+     * If the server *does* provide a `stale-while-revalidate` value, then that
+     * is always used.
+     *
+     * Set to 0 to prevent any `stale-while-revalidate` behavior unless
+     * explicitly allowed by the server's `cache-control` header.
+     */
+    'stale-while-revalidate-factor'?: number;
+};
 export declare class CacheEntry {
     #private;
-    constructor(statusCode: number, headers: Buffer[], integrity?: Integrity);
+    constructor(statusCode: number, headers: Uint8Array[], { body, integrity, trustIntegrity, 'stale-while-revalidate-factor': staleWhileRevalidateFactor, contentLength, }?: CacheEntryOptions);
+    toJSON(): {
+        [k: string]: string | number | boolean | [string, string][] | Date | ccp.CacheControl | undefined;
+    };
     [kCustomInspect](depth: number, options: InspectOptions): string;
+    get date(): Date | undefined;
+    get maxAge(): number;
+    get cacheControl(): ccp.CacheControl;
+    get staleWhileRevalidate(): boolean;
+    get contentType(): string;
+    get valid(): boolean;
     /**
-     * `true` if the entry represents a cached response that is still
-     * valid to use.
+     * Add contents to the entry body.
      */
-    get valid(): boolean;
-    addBody(b: Buffer): void;
+    addBody(b: Uint8Array): void;
     get statusCode(): number;
-    get headers(): Buffer<ArrayBufferLike>[];
+    get headers(): Uint8Array[];
     /**
-     * check that the sri integrity string that was provided to the ctor
+     * Returns the body as a single Uint8Array, concatenating parts if needed.
+     */
+    get _body(): Uint8Array;
+    /**
+     * Check that the sri integrity string that was provided to the ctor
      * matches the body that we actually received. This should only be called
      * AFTER the entire body has been completely downloaded.
      *
+     * This method **will throw** if the integrity values do not match.
+     *
      * Note that this will *usually* not be true if the value is coming out of
      * the cache, because the cache entries are un-gzipped in place. It should
      * _only_ be called for artifacts that come from an actual http response.
+     *
+     * Returns true if anything was actually verified.
      */
-    checkIntegrity(): boolean;
+    checkIntegrity(context?: ErrorCauseOptions): this is CacheEntry & {
+        integrity: Integrity;
+    };
     get integrityActual(): Integrity;
-    get integrity(): `sha512-${string}` | undefined;
+    set integrityActual(i: Integrity);
+    set integrity(i: Integrity | undefined);
+    get integrity(): Integrity | undefined;
     /**
      * Give it a key, and it'll return the buffer of that header value
      */
-    getHeader(h: string): Buffer<ArrayBufferLike> | undefined;
+    getHeader(h: string): Uint8Array | undefined;
+    /**
+     * Give it a key, and it'll return the decoded string of that header value
+     */
+    getHeaderString(h: string): string | undefined;
     /**
      * Set a header to a specific value
      */
-    setHeader(h: string, value: Buffer | string): void;
+    setHeader(h: string, value: Uint8Array | string): void;
     /**
      * Return the body of the entry as a Buffer
      */
     buffer(): Buffer;
-    get body(): Buffer | Record<string, any>;
+    get body(): Uint8Array | Record<string, any>;
     get isJSON(): boolean;
     get isGzip(): boolean;
     /**
@@ -61,7 +130,8 @@ export declare class CacheEntry {
      * and this static method will decode it into a CacheEntry representing
      * the cached response.
      */
-    static decode(buffer: Buffer): CacheEntry;
+    static decode(buffer: Uint8Array): CacheEntry;
+    static isGzipEntry(buffer: Uint8Array): boolean;
     /**
      * Encode the entry as a single Buffer for writing to the cache
      */

package/dist/esm/cache-entry.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"cache-entry.d.ts","sourceRoot":"","sources":["../../src/cache-entry.ts"],"names":[],"mappings":"AAsBA,OAAO,KAAK,EAAE,SAAS,EAAE,SAAS,EAAE,MAAM,eAAe,CAAA;AAIzD,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,MAAM,CAAA;AAI1C,MAAM,MAAM,OAAO,GAAG,MAAM,CAAC,MAAM,EAAE,SAAS,CAAC,CAAA;AAyB/C,QAAA,MAAM,cAAc,eAA2C,CAAA;AAE/D,qBAAa,UAAU;;gBAUnB,UAAU,EAAE,MAAM,EAClB,OAAO,EAAE,MAAM,EAAE,EACjB,SAAS,CAAC,EAAE,SAAS;IAiBvB,CAAC,cAAc,CAAC,CAAC,KAAK,EAAE,MAAM,EAAE,OAAO,EAAE,cAAc,GAAG,MAAM;IAehE;;;OAGG;IACH,IAAI,KAAK,IAAI,OAAO,CAsBnB;IAED,OAAO,CAAC,CAAC,EAAE,MAAM;IAKjB,IAAI,UAAU,WAEb;IACD,IAAI,OAAO,8BAEV;IAED;;;;;;;;OAQG;IACH,cAAc,IAAI,OAAO;IAIzB,IAAI,eAAe,IAAI,SAAS,CAM/B;IACD,IAAI,SAAS,mCAEZ;IAED;;OAEG;IACH,SAAS,CAAC,CAAC,EAAE,MAAM;IAInB;;OAEG;IACH,SAAS,CAAC,CAAC,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,MAAM;IAI3C;;OAEG;IACH,MAAM,IAAI,MAAM;IAWhB,IAAI,IAAI,IAAI,MAAM,GAAG,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,CAEvC;IAGD,IAAI,MAAM,IAAI,OAAO,CAYpB;IAGD,IAAI,MAAM,IAAI,OAAO,CAcpB;IAED;;;;OAIG;IACH,KAAK;IAmBL;;;OAGG;IACH,IAAI;IAKJ;;OAEG;IACH,IAAI,IAAI,OAAO;IAOf;;;;OAIG;IACH,MAAM,CAAC,MAAM,CAAC,MAAM,EAAE,MAAM,GAAG,UAAU;IAiCzC;;OAEG;IAGH,MAAM,IAAI,MAAM;CAoCjB"}
+{"version":3,"file":"cache-entry.d.ts","sourceRoot":"","sources":["../../src/cache-entry.ts"],"names":[],"mappings":"AAqBA,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,qBAAqB,CAAA;AAE5D,OAAO,KAAK,EAAE,SAAS,EAAE,SAAS,EAAE,MAAM,eAAe,CAAA;AACzD,OAAO,GAAG,MAAM,sBAAsB,CAAA;AAEtC,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,WAAW,CAAA;AAS/C,MAAM,MAAM,OAAO,GAAG,MAAM,CAAC,MAAM,EAAE,SAAS,CAAC,CAAA;AAyB/C,QAAA,MAAM,cAAc,eAA2C,CAAA;AAE/D,MAAM,MAAM,iBAAiB,GAAG;IAC9B;;;;;;OAMG;IACH,IAAI,CAAC,EAAE,UAAU,CAAA;IAEjB;;;OAGG;IACH,aAAa,CAAC,EAAE,MAAM,CAAA;IAEtB;;OAEG;IACH,SAAS,CAAC,EAAE,SAAS,CAAA;IACrB;;;;;;;OAOG;IACH,cAAc,CAAC,EAAE,OAAO,CAAA;IAExB;;;;;;;;;;;;;;OAcG;IACH,+BAA+B,CAAC,EAAE,MAAM,CAAA;CACzC,CAAA;AAED,qBAAa,UAAU;;gBAsBnB,UAAU,EAAE,MAAM,EAClB,OAAO,EAAE,UAAU,EAAE,EACrB,EACE,IAAI,EACJ,SAAS,EACT,cAAsB,EACtB,+BAA+B,EAC7B,0BAA+B,EACjC,aAAa,GACd,GAAE,iBAAsB;IAwC3B,MAAM;;;IAwCN,CAAC,cAAc,CAAC,CAAC,KAAK,EAAE,MAAM,EAAE,OAAO,EAAE,cAAc,GAAG,MAAM;IAShE,IAAI,IAAI,IAAI,IAAI,GAAG,SAAS,CAK3B;IAGD,IAAI,MAAM,IAAI,MAAM,CAQnB;IAGD,IAAI,YAAY,IAAI,GAAG,CAAC,YAAY,CAKnC;IAGD,IAAI,oBAAoB,IAAI,OAAO,CAWlC;IAGD,IAAI,WAAW,WAId;IAOD,IAAI,KAAK,IAAI,OAAO,CAmBnB;IAED;;OAEG;IACH,OAAO,CAAC,CAAC,EAAE,UAAU;IAcrB,IAAI,UAAU,WAEb;IACD,IAAI,OAAO,IAAI,UAAU,EAAE,CAE1B;IAED;;OAEG;IACH,IAAI,KAAK,IAAI,UAAU,CAYtB;IAED;;;;;;;;;;;;OAYG;IACH,cAAc,CACZ,OAAO,GAAE,iBAAsB,GAC9B,IAAI,IAAI,UAAU,GAAG;QAAE,SAAS,EAAE,SAAS,CAAA;KAAE;IAchD,IAAI,eAAe,IAAI,SAAS,CAO/B;IAED,IAAI,eAAe,CAAC,CAAC,EAAE,SAAS,EAG/B;IAED,IAAI,SAAS,CAAC,CAAC,EAAE,SAAS,GAAG,SAAS,EAKrC;IACD,IAAI,SAAS,IANI,SAAS,GAAG,SAAS,CAQrC;IAED;;OAEG;IACH,SAAS,CAAC,CAAC,EAAE,MAAM,GAAG,UAAU,GAAG,SAAS;IAI5C;;OAEG;IACH,eAAe,CAAC,CAAC,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS;IAO9C;;OAEG;IACH,SAAS,CAAC,CAAC,EAAE,MAAM,EAAE,KAAK,EAAE,UAAU,GAAG,MAAM;IAI/C;;OAEG;IACH,MAAM,IAAI,MAAM;IAUhB,IAAI,IAAI,IAAI,UAAU,GAAG,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,CAE3C;IAGD,IAAI,MAAM,IAAI,OAAO,CAYpB;IAGD,IAAI,MAAM,IAAI,OAAO,CAcpB;IAED;;;;OAIG;IACH,KAAK;IAqBL;;;OAGG;IACH,IAAI;IAKJ;;OAEG;IACH,IAAI,IAAI,OAAO;IAQf;;;;OAIG;IACH,MAAM,CAAC,MAAM,CAAC,MAAM,EAAE,UAAU,GAAG,UAAU;IAuD7C,MAAM,CAAC,WAAW,CAAC,MAAM,EAAE,UAAU,GAAG,OAAO;IAO/C;;OAEG;IACH,MAAM,IAAI,MAAM;CAkDjB"}