chtypes 2.0.0.dev6__tar.gz → 2.0.0.dev8__tar.gz

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/CHANGELOG.md

@@ -8,6 +8,9 @@ The four bindings in this repository are released together and give one answer,
 
 ### Added
 
+- A fetch-only call (#492): `Registry.fetch(request) -> Resolved` resolves, fetches, verifies and installs the build a request names into the cache, as `chtypes fetch` does, and opens nothing: no library is loaded and the process setup is untouched, so it needs no `setup`. It honors the registry's fetch options (offline, the cache, and under the production channel the bases, the trust and the lock), and a concurrent `fetch` and open of one request share one fetch. The `Resolved` names the installed version, build, digests and library path; errors are the fetch codes' classes.
+- `chtypes resolve <spelling> [--json]` (#493): what a line or version resolves to on every platform the registry offers, one `resolved <version> <platform> <build> <manifest>` line each, or with `--json` one JSON array of `{platform, version, build, manifest}` objects, each signed statement verified exactly as a fetch verifies it and no layer downloaded; `--offline` answers from the cache. The four CLIs print the same bytes, so one parser reads them (a CI cache key, for example).
+- `chtypes prune [--line <line>] [--keep <n>] [--dry-run]` (#494): removes the installed builds of each line and platform that at least n newer installed builds supersede (n defaults to 1), each with its record, its blobs and its `index.json` entry; `--dry-run` prints exactly what would go. It never removes a build a running process holds: a registry now holds every build it opens or fetches, with a shared `flock` on the entry's `verified.json` until the process exits, and prune removes an entry only after taking the exclusive lock without waiting, reporting it `in-use` otherwise. It never touches a system directory, nor, on the dev channel, another fingerprint's builds. The rules are `docs/guides/fetch-v1.md` sections 1 and 3, and the `resolve-build-*` and `prune-*` conformance cases hold all four bindings to them.
 - Every settings layer's filter-declined settings, in one list (ABI v2; moves the dev fingerprint; #588): `SchemaDescription.filter_declined_settings`, a tuple of `DeclinedSetting`, decoded from `schema_description`'s new required top-level `filter_declined_settings`, over every layer known at schema compile (`setup`'s defaults as the compile captured them, the server profile, and the schema's own settings). `DeclinedSetting` gains `layer`, the highest layer that sets the name, from the new `DeclinedLayer` vocabulary (`DEFAULTS`, `SERVER`, `SCHEMA`, and its unknown(n)). One entry per name, ordered by layer and then by name bytes, as the library writes it; the binding sorts nothing.
 - `Schema.describe` raises the library's `CHS_REJECTED` with ch_code 306 (TOO_DEEP_RECURSION), a `SchemaError`, for a deep input the server's stack check refuses (ABI v2: `chs_schema_describe` now lists `CHS_REJECTED`).
 - The server's filter-declined settings (ABI v2; moves the dev fingerprint): `SchemaServer.filter_declined_settings`, a tuple of `DeclinedSetting` (`name`, `tier`), decoded from `schema_description`'s new required `server.filter_declined_settings`, with the new `DeclinedTier` vocabulary (`PREDICATE`, `PREDICATE_UNFLIPPED`, `RESULT_CONTENT`, and its unknown(n)). `compile_filter` on a schema whose server lists any is the library's `CHS_DECLINED`, an `UnsupportedError` carrying its message; a filter's own settings other than `session_timezone` are no longer declined at compile, and a declined one is named at evaluation instead.
@@ -16,11 +19,18 @@ The four bindings in this repository are released together and give one answer,
 - The v2 spec publishes the filter decline-reasons catalog (`spec/abi-v2/declines/`).
 - The v2 spec's decline catalog has 49 entries, and it publishes the settings values a server refuses outright (`spec/abi-v2/where-settings/documented-pathological.tsv`).
 - The v2 spec's decline catalog follows the producer's batch 11: its `server-profile-setting` entry now covers every settings layer the schema compiles under (the id is unchanged), the evaluation-time setting entries name only the filter's own settings, and its `stack-depth` entry lists `chs_schema_describe`.
+- The v2 spec publishes the documented platform differences (`spec/abi-v2/documented-platform/`), and its decline catalog adds `setting-format-pair`, `string-reading` and `equals-fold` (52 reasons).
 - A `410 Gone` is a retired repository, and permanent (#571). Any request the fetch layer makes (a tag, an alias, a digest, a blob, `tags/list`, the referrers API or its fallback tag, after a redirect too) that answers `410` is now the new code `CHTYPES_SOURCE_RETIRED` (exit status 10), never retried and never sent to the next base, nor from the dev channel's alias to its tag. It was `CHTYPES_SOURCE_UNREACHABLE` (exit 3) with the registry's message dropped, and Go, Python and Rust moved on to the next base, so a mirror's 404 could hide the retirement. The error names the URL that answered and carries the registry's own message, with control and bidirectional-formatting characters removed and cut to 256 code points: Go `CodeSourceRetired` (`errors.Is(err, ErrSourceRetired)`), Python and TypeScript `SourceRetiredError`, Rust `Error::SourceRetired`. The rule is `docs/guides/fetch-v1.md` section 2, "A retired repository", and the `retired-410-*` and `list-tags-retired-410` conformance cases and one shared message table (`tests/fixtures/retired-message/cases.json`) hold all four bindings to it.
 - A 2.0.0-dev SDK prefers the newest build of its own fingerprint (#511). A version request resolves `<tag>--fp-<fingerprint>` first, the alias the staging dev registry serves beside every dev tag, with the binding's own generated ABI fingerprint, and the tag itself only when every base answers the alias 404; any other failure of the alias request is that failure, with no fallback, and an alias whose index lacks the platform is `CHTYPES_ARTIFACT_UNPUBLISHED`. The dev channel's cache lookups (the installed-build lookup a registry open makes first, the offline answer, and the monotonic rule of a fetch) now see only records whose signed `abi_fingerprint` is the SDK's own, so a newer build of another fingerprint, installed by another dev SDK in the shared cache, is never returned or kept over this one's own, and is never removed or rewritten. So a dev build of a newer fingerprint no longer strands an older dev SDK until its consumers upgrade. Trust is unchanged (the signed predicate and the load checks decide), a listing never shows an alias, nothing records one, `list` and `where` are unchanged, and there is no option: both steps are the dev channel's alone, and the v1 contract and every production channel never take them. The rule is `docs/guides/fetch-v1.md` section 3, and the `dev-alias-*`, `resolve-installed-dev-alias-*` and `list-tags-dev-alias` conformance cases hold all four bindings to it.
 
 ### Changed
 
+- `chtypes list` and the tag listing name the published lines only (two-part, numeric order), as Go's always did; `chtypes resolve <line>` prints the exact version a line resolves to (#613).
+- **Breaking: the public surface the parity gate carried as exceptions is resolved, before the v2 lock (#499, #500).** A convenience exists in every binding or in none (`docs/reference/bindings-v1.md` §1, principle 2). Migration, old → new:
+  - `chtypes.errors.internal` and `chtypes.errors.misuse` → private (#499): they were the binding's own construction helpers, never documented. Catch `InternalError` and `UsageError`, as before.
+  - `SourceUnreachableError.retryable` and `.retry_after`, and the constructor's `retryable=` and `retry_after=` → removed: `retry_after` was never set, and the fetch layer has spent its retry table (`docs/guides/fetch-v1.md` §7) before it raises; a `Retry-After` it refused is named in the message.
+  - `FetchOptions(trusted_keys=(TrustedKey(keyid, public_key), …))` → `FetchOptions(trusted_keys=("<64 hex digits>", …))`: raw ed25519 public keys as hex, the `CHTYPES_TRUSTED_KEYS` spelling and Go's and Rust's, with each key id derived (`sha256-first16hex`). `chtypes.TrustedKey` is no longer public. A key that is not 64 hex digits is a `UsageError`; a channel that honors no trust override (this 2.0.0-dev SDK) ignores the option, with its one warning, as before.
+  - The `CODE_*` constants are now generated from `spec/fetch-v1/constants.json` (the private `chtypes._codes`); their names and values are unchanged.
 - `compile_filter` is the library's `CHS_DECLINED`, `UnsupportedError`, on a schema whose `filter_declined_settings` is not empty from any layer, and its message names each setting and its layer (ABI v2, #588). A declined setting from the defaults or the schema's own settings now fails the compile, where it was named at evaluation before; a filter's own settings and an evaluation's are still named at evaluation.
 - **Enforcement gate lifted** for `26.3` on `linux-amd64`.
 - **Enforcement gate lifted** for `26.7` on `linux-amd64`.
@@ -44,6 +54,17 @@ The four bindings in this repository are released together and give one answer,
 - **Enforcement gate restored** for `26.7` on `darwin-arm64`.
 - **Enforcement gate restored** for `26.8` on `darwin-arm64`.
 - **Enforcement gate restored** for `26.9` on `darwin-arm64`.
+- **Enforcement gate lifted** for `26.3` on `linux-amd64`.
+- **Enforcement gate lifted** for `26.7` on `linux-amd64`.
+- **Enforcement gate lifted** for `26.8` on `linux-amd64`.
+- **Enforcement gate lifted** for `26.9` on `linux-amd64`.
+- **Enforcement gate lifted** for `26.3` on `linux-arm64`.
+- **Enforcement gate lifted** for `26.7` on `linux-arm64`.
+- **Enforcement gate lifted** for `26.8` on `linux-arm64`.
+- **Enforcement gate lifted** for `26.9` on `linux-arm64`.
+- **Enforcement gate lifted** for `26.7` on `darwin-arm64`.
+- **Enforcement gate lifted** for `26.8` on `darwin-arm64`.
+- **Enforcement gate lifted** for `26.9` on `darwin-arm64`.
 
 ### Removed
 
@@ -51,6 +72,9 @@ The four bindings in this repository are released together and give one answer,
 
 ### Fixed
 
+- A declined export's payload is absent, as documented: it was an empty payload when the library returned a zero-length buffer for a declined export. `BatchResult.payload` and `spans` are `None` for a declined export. Whether an export was declined is decided from the document's `export_declined`, never from the buffer's length.
+- The layout's `index.json` never gains a second entry for one manifest (#487): concurrent installs of one build each appended an entry (7 or 8 entries for the same digest after 8 concurrent installs, measured on 1.0.2). The merge now deduplicates by digest, as Go and TypeScript do.
+- `--frozen` with a warm, verified cache makes zero requests, and `--frozen --offline` verifies the installed build against the lock with zero network (#414). An installed build whose `verified.json` is for the platform and records exactly the lock's pinned manifest, layer and bundle digests is used as it is. `--frozen --offline` is `CHTYPES_ARTIFACT_MISSING` when the pinned build is not installed, and `CHTYPES_ARTIFACT_PINNED` when there is no lock entry or the installed record does not match the pin; `--offline` without `--frozen` is unchanged. The `frozen-warm-cache-zero-requests`, `frozen-offline-lock-match`, `frozen-offline-lock-mismatch` and `frozen-offline-lock-not-installed` conformance cases hold all four bindings to it, under both corpora. The Python frozen path also compares the record's layer and bundle digests to the pin, and looks in the system directories too.
 - A 2.0.0-dev SDK whose own-fingerprint alias has expired, with a cached build of that fingerprint for the request, now says so in the `CHTYPES_ARTIFACT_UNPUBLISHED` message of `fetch` (#581): `; a cached build for this fingerprint (build <id>) is still usable by open; upgrade the SDK to get newer builds` is appended. The code, the exit status (4) and the cache are unchanged, and all four bindings give the same message (`fp-ahead-cached-own-fp`).
 - A 2.0.0-dev SDK whose fingerprint no published build carries yet now fails closed with a precise message (#578). When every base answers its own alias 404 and the tag names a build signed for another fingerprint, `fetch` is `CHTYPES_ARTIFACT_UNPUBLISHED` (exit status 4): `no published build for this SDK's fingerprint <own fp>; newest published on this channel is <fp> (build <id>)`. It used to install that build, or report it already installed, and the lookup at open then answered `CHTYPES_ARTIFACT_MISSING` with advice to run `fetch`: a loop with no pointer to the cause. Nothing is downloaded or installed and the cache is unchanged. An alias that answers, a tag at the SDK's own fingerprint and the v1 contract are unchanged. All four bindings run the same fetch conformance cases, `fp-ahead-*`, and the rule is `docs/guides/fetch-v1.md` section 3.
 - Opening a line no longer waits for another line's fetch: the registry never holds its lock across network I/O, and concurrent opens of one request share one fetch (#491).

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: chtypes
-Version: 2.0.0.dev6
+Version: 2.0.0.dev8
 Summary: Python binding for the chtypes artifacts: ClickHouse's own type system, per version, behind the frozen chs_* C ABI
 Project-URL: Homepage, https://github.com/wave-rf/chtypes
 Project-URL: Repository, https://github.com/wave-rf/chtypes
@@ -88,7 +88,7 @@ A bad **row** is a verdict, not an exception: `outcome` becomes `Outcome.REJECTE
 
 ## Fetching from the command line
 
-`chtypes fetch 26.8` (or `python -m chtypes fetch 26.8`) resolves, verifies and installs a build into the per-user cache; `chtypes verify`, `chtypes list` and `chtypes where` complete the four commands. `--lock FILE` records what was installed and `--frozen` fetches only what the lock pins. See the [Python API reference](https://github.com/wave-rf/chtypes/blob/main/docs/reference/python.md#the-command).
+`chtypes fetch 26.8` (or `python -m chtypes fetch 26.8`) resolves, verifies and installs a build into the per-user cache; `chtypes verify`, `chtypes list`, `chtypes where`, `chtypes resolve` and `chtypes prune` complete the six commands. `--lock FILE` records what was installed and `--frozen` fetches only what the lock pins. See the [Python API reference](https://github.com/wave-rf/chtypes/blob/main/docs/reference/python.md#the-command).
 
 ## Things specific to this binding
 

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/README.md

@@ -69,7 +69,7 @@ A bad **row** is a verdict, not an exception: `outcome` becomes `Outcome.REJECTE
 
 ## Fetching from the command line
 
-`chtypes fetch 26.8` (or `python -m chtypes fetch 26.8`) resolves, verifies and installs a build into the per-user cache; `chtypes verify`, `chtypes list` and `chtypes where` complete the four commands. `--lock FILE` records what was installed and `--frozen` fetches only what the lock pins. See the [Python API reference](https://github.com/wave-rf/chtypes/blob/main/docs/reference/python.md#the-command).
+`chtypes fetch 26.8` (or `python -m chtypes fetch 26.8`) resolves, verifies and installs a build into the per-user cache; `chtypes verify`, `chtypes list`, `chtypes where`, `chtypes resolve` and `chtypes prune` complete the six commands. `--lock FILE` records what was installed and `--frozen` fetches only what the lock pins. See the [Python API reference](https://github.com/wave-rf/chtypes/blob/main/docs/reference/python.md#the-command).
 
 ## Things specific to this binding
 

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "chtypes"
-version = "2.0.0.dev6"
+version = "2.0.0.dev8"
 description = "Python binding for the chtypes artifacts: ClickHouse's own type system, per version, behind the frozen chs_* C ABI"
 readme = "README.md"
 license = "Apache-2.0"

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/src/chtypes/__init__.py

@@ -98,7 +98,7 @@ from .errors import (
     UsageError,
 )
 from .library import Block, Filter, Library, Schema, Server, open_unverified
-from .registry import FetchOptions, Registry, Resolved, TrustedKey, cache_root, search_dirs
+from .registry import FetchOptions, Registry, Resolved, cache_root, search_dirs
 from .results import (
     AtMergeEntry,
     BatchResult,
@@ -201,7 +201,6 @@ __all__ = [
     "Span",
     "Status",
     "Transform",
-    "TrustedKey",
     "UnsupportedError",
     "UsageError",
     "Value",

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/src/chtypes/__main__.py

@@ -6,6 +6,10 @@
     chtypes verify [--cache <dir>] [--strict]   re-verify every installed build
     chtypes list   [--cache <dir>] [--offline] [--strict]
     chtypes where  [--cache <dir>] [--strict] [--all]   the cache root; --all: every search dir
+    chtypes resolve <spelling> [--json] [--cache <dir>] [--offline] [--strict]
+                   the build and manifest each platform resolves to, verified; installs nothing
+    chtypes prune  [--line <line>] [--keep <n>] [--dry-run] [--cache <dir>] [--strict]
+                   remove the builds of each line newer ones supersede, keeping the newest n
 
 A spelling is `26.8`, `26.8.15` or `26.8.15.10`. Exit statuses come from the
 `errors` table of spec/fetch-v1/constants.json (generated into the fetch layer
@@ -33,6 +37,7 @@ from collections.abc import Sequence
 
 from . import __version__ as chtypes_version
 from . import _ocifetch as fetch_layer
+from ._codes import CODE_ARTIFACT_CORRUPT
 from ._ocifetch import _channel
 from ._ocifetch import _constants as C
 from ._ocifetch._ensure import _translate_transport_error, detect_host_platform, missing_notes
@@ -41,6 +46,7 @@ from ._ocifetch._faults import probe_roots
 from ._ocifetch._http import FetchPolicy, RetryPolicy, TransportError, fetch_from_bases
 from ._ocifetch._layout import resolve_cache_root, search_roots
 from ._ocifetch._lock import load_lock
+from ._ocifetch._prune import is_line
 from .errors import ChtypesError
 from .registry import FetchOptions
 
@@ -50,7 +56,7 @@ EXIT_OK = 0
 EXIT_USAGE = 2
 # A failure that carries no code from the table (a lock file that cannot be
 # read, an interrupted run): the status the table gives its verification codes.
-_EXIT_FALLBACK = C.ERROR_EXIT_CODES["CHTYPES_ARTIFACT_CORRUPT"]
+_EXIT_FALLBACK = C.ERROR_EXIT_CODES[CODE_ARTIFACT_CORRUPT]
 _EXIT_INTERRUPTED = 130
 
 
@@ -153,7 +159,7 @@ def build_parser() -> argparse.ArgumentParser:
     listing = sub.add_parser(
         "list",
         help="what is installed, and what the registry publishes",
-        description="What is installed, and the version tags the registry publishes.",
+        description="What is installed, and the lines the registry publishes.",
     )
     cache_option(listing)
     listing.add_argument(
@@ -173,6 +179,55 @@ def build_parser() -> argparse.ArgumentParser:
         action="store_true",
         help="every directory searched, in order, the cache root first (one per line)",
     )
+
+    resolving = sub.add_parser(
+        "resolve",
+        help="the build and manifest each platform resolves to, verified; installs nothing",
+        description=(
+            "What a line or version resolves to on every platform the registry offers, "
+            "each signature verified, with no layer downloaded (docs/guides/fetch-v1.md section 3)."
+        ),
+    )
+    resolving.add_argument(
+        "spelling",
+        metavar="<spelling>",
+        help="26.8, 26.8.15 or 26.8.15.10 (no v prefix, no channel suffix)",
+    )
+    resolving.add_argument(
+        "--json",
+        action="store_true",
+        help="print one JSON array, one object per platform, instead of one line per platform",
+    )
+    cache_option(resolving)
+    resolving.add_argument(
+        "--offline",
+        action="store_true",
+        help="answer from the cache; no network (or set CHTYPES_OFFLINE=1)",
+    )
+
+    pruning = sub.add_parser(
+        "prune",
+        help="remove the builds of each line newer ones supersede, keeping the newest n",
+        description=(
+            "Remove the installed builds of each line that newer installed builds of the same line "
+            "and platform supersede, keeping the newest n, and never one a running process holds "
+            "(docs/guides/fetch-v1.md section 1)."
+        ),
+    )
+    pruning.add_argument(
+        "--line", metavar="<line>", help="prune only this line, such as 26.8 (default: every line)"
+    )
+    pruning.add_argument(
+        "--keep",
+        metavar="<n>",
+        type=int,
+        default=1,
+        help="how many of each line's newest builds to keep, per platform (at least 1; default 1)",
+    )
+    pruning.add_argument(
+        "--dry-run", action="store_true", help="print what would be removed, and remove nothing"
+    )
+    cache_option(pruning)
     return parser
 
 
@@ -198,13 +253,15 @@ def _options(args: argparse.Namespace, *, platform: str | None = None):
     return fetch._to_options(platform)
 
 
-_LINE_SPELLING = re.compile(r"^[0-9]+\.[0-9]+$")
+# The published lines: exactly two parts, no leading zeros (Go's `lineTag`).
+_LINE_SPELLING = re.compile(r"^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$")
 
 
 def _published_tags(options) -> list[str]:
-    """The registry's `tags/list`, kept to the tags that are v1 version spellings
-    (`spelling.regex`): the referrers fallback tags (`sha256-<hex>`) and anything
-    else a repository carries are not versions."""
+    """The registry's `tags/list`, kept to the published lines (two-part tags),
+    in numeric order: three- and four-part versions, aliases, the referrers
+    fallback tags (`sha256-<hex>`) and anything else a repository carries are
+    dropped (docs/guides/fetch-v1.md section 3)."""
     try:
         resp = fetch_from_bases(
             options.resolved_bases(),
@@ -221,7 +278,7 @@ def _published_tags(options) -> list[str]:
         tags = json.loads(resp.body.decode("utf-8")).get("tags") or []
     except (ValueError, AttributeError) as exc:
         raise ArtifactCorruptError(f"chtypes: tags/list was not a tag list ({exc})") from exc
-    keep = [t for t in tags if isinstance(t, str) and re.fullmatch(C.SPELLING_REGEX, t)]
+    keep = [t for t in tags if isinstance(t, str) and _LINE_SPELLING.fullmatch(t)]
     return sorted(set(keep), key=lambda t: tuple(int(p) for p in t.split(".")))
 
 
@@ -250,7 +307,7 @@ def _cmd_fetch(args: argparse.Namespace) -> int:
             lock = load_lock(options.lock_path)
             spellings = list(lock.requests)
         else:
-            spellings = [t for t in _published_tags(options) if _LINE_SPELLING.match(t)]
+            spellings = _published_tags(options)
             if not spellings:
                 _say("chtypes: the registry publishes no lines")
     else:
@@ -297,7 +354,7 @@ def _cmd_verify(args: argparse.Namespace) -> int:
             _say(f"chtypes: FAILED {r.version} {r.platform} {r.dir}: {r.detail}")
     if failed:
         _say(f"chtypes: {failed} of {len(results)} installed build(s) FAILED verification")
-        return C.ERROR_EXIT_CODES["CHTYPES_ARTIFACT_CORRUPT"]
+        return C.ERROR_EXIT_CODES[CODE_ARTIFACT_CORRUPT]
     return EXIT_OK
 
 
@@ -328,7 +385,55 @@ def _cmd_where(args: argparse.Namespace) -> int:
     return EXIT_OK
 
 
-_COMMANDS = {"fetch": _cmd_fetch, "verify": _cmd_verify, "list": _cmd_list, "where": _cmd_where}
+def _cmd_resolve(args: argparse.Namespace) -> int:
+    options = _options(args)
+    request = fetch_layer.Request(args.spelling)  # a spelling error: usage, before any I/O
+    rows, warnings = fetch_layer.resolve(request, options)
+    for warning in warnings:
+        _say(f"chtypes: warning: {warning}")
+    if args.json:
+        doc = [
+            {"platform": r.platform, "version": r.version, "build": r.build, "manifest": r.manifest}
+            for r in rows
+        ]
+        sys.stdout.write(json.dumps(doc, separators=(",", ":")) + "\n")
+    else:
+        for r in rows:
+            sys.stdout.write(f"resolved {r.version} {r.platform} {r.build} {r.manifest}\n")
+    sys.stdout.flush()
+    return EXIT_OK
+
+
+def _cmd_prune(args: argparse.Namespace) -> int:
+    if args.keep < 1:
+        raise ValueError(f"chtypes: --keep is at least 1, not {args.keep}")
+    if args.line is not None and not is_line(args.line):
+        raise ValueError(f"chtypes: --line takes a two-part line such as 26.8, not {args.line!r}")
+    options = _options(args)
+    results = fetch_layer.prune(options, line=args.line, keep=args.keep, dry_run=args.dry_run)
+    removed = 0
+    for r in results:
+        if r.in_use:
+            word = "in-use"
+        else:
+            removed += 1
+            word = "would-prune" if args.dry_run else "pruned"
+        sys.stdout.write(f"{word} {r.version} {r.platform} {r.dir}\n")
+    sys.stdout.flush()
+    _say_notes(options)
+    root = resolve_cache_root(args.cache)
+    _say(f"chtypes: {'would prune' if args.dry_run else 'pruned'} {removed} build(s) under {root}")
+    return EXIT_OK
+
+
+_COMMANDS = {
+    "fetch": _cmd_fetch,
+    "verify": _cmd_verify,
+    "list": _cmd_list,
+    "where": _cmd_where,
+    "resolve": _cmd_resolve,
+    "prune": _cmd_prune,
+}
 
 
 def main(argv: Sequence[str] | None = None) -> int:

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/src/chtypes/_abi2/_errors.py

@@ -15,8 +15,8 @@ from chtypes.errors import (
     SchemaError,
     UnsupportedError,
     UsageError,
-    misuse,
 )
+from chtypes.errors import _misuse as misuse
 
 __all__ = [
     "ArtifactCorruptError",

chtypes-2.0.0.dev8/src/chtypes/_codes.py

@@ -0,0 +1,23 @@
+# Code generated by scripts/fetch-v1/gen-constants.py from
+# spec/fetch-v1/constants.json. DO NOT EDIT.
+# Source sha256: 82afa8d6cba96c10b5945c6f7c1c8360a504cc8ac6231e74017cc6c20b908df6
+"""The shared error codes (docs/guides/fetch-v1.md §8), one per entry of the
+fetch constants' `errors`, in their order. `chtypes.errors` re-exports them
+(docs/reference/bindings-v1.md §4, the codes)."""
+
+from __future__ import annotations
+
+from typing import Final
+
+CODE_ARTIFACT_MISSING: Final[str] = "CHTYPES_ARTIFACT_MISSING"
+CODE_ARTIFACT_UNTRUSTED: Final[str] = "CHTYPES_ARTIFACT_UNTRUSTED"
+CODE_ARTIFACT_CORRUPT: Final[str] = "CHTYPES_ARTIFACT_CORRUPT"
+CODE_ARTIFACT_PINNED: Final[str] = "CHTYPES_ARTIFACT_PINNED"
+CODE_ARTIFACT_UNPUBLISHED: Final[str] = "CHTYPES_ARTIFACT_UNPUBLISHED"
+CODE_SOURCE_UNREACHABLE: Final[str] = "CHTYPES_SOURCE_UNREACHABLE"
+CODE_SOURCE_UNAUTHORIZED: Final[str] = "CHTYPES_SOURCE_UNAUTHORIZED"
+CODE_SOURCE_FORBIDDEN: Final[str] = "CHTYPES_SOURCE_FORBIDDEN"
+CODE_SOURCE_INCOMPATIBLE: Final[str] = "CHTYPES_SOURCE_INCOMPATIBLE"
+CODE_ARTIFACT_INCOMPATIBLE: Final[str] = "CHTYPES_ARTIFACT_INCOMPATIBLE"
+CODE_CACHE_UNUSABLE: Final[str] = "CHTYPES_CACHE_UNUSABLE"
+CODE_SOURCE_RETIRED: Final[str] = "CHTYPES_SOURCE_RETIRED"

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/src/chtypes/_decode.py

@@ -39,7 +39,8 @@ from ._abi2._vocab import (
     Source,
     Verdict,
 )
-from .errors import ArtifactCorruptError, internal
+from .errors import ArtifactCorruptError
+from .errors import _internal as internal
 from .results import (
     AtMergeEntry,
     BatchResult,
@@ -427,6 +428,12 @@ def decode_batch(raw: bytes, export: bytes | None) -> BatchResult:
     framing = obj.get("framing")
     engine_rows = None if obj.get("engine_rows") is None else _engine_rows(obj, where)
     spans = None if obj.get("row_spans") is None else _spans(obj, "row_spans", where)
+    export_declined = _bytes(obj, "export_declined", where)
+    if export_declined:
+        # A declined export has no payload and no spans, whatever buffer the call
+        # returned: the library may hand back a zero-length one, so the document's
+        # export_declined decides, never the buffer's length.
+        export, spans = None, None
     return BatchResult(
         outcome=Outcome.of(_text(obj, "outcome", where)),
         err_code=_int(obj, "code", where),
@@ -438,7 +445,7 @@ def decode_batch(raw: bytes, export: bytes | None) -> BatchResult:
         engine_rows=engine_rows,
         payload=export,
         spans=spans,
-        export_declined=_bytes(obj, "export_declined", where),
+        export_declined=export_declined,
         rows_passed=_u64(obj, "rows_passed", where),
         rows_cut=_u64(obj, "rows_cut", where),
         partition_count=_opt_u64(obj, "partition_count", where),

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/src/chtypes/_guard.py

@@ -14,7 +14,7 @@ import threading
 from collections.abc import Callable
 from types import TracebackType
 
-from .errors import misuse
+from .errors import _misuse as misuse
 
 __all__ = ["CloseGuard"]
 

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/src/chtypes/_input.py

@@ -14,7 +14,7 @@ import json
 from collections.abc import Mapping, Sequence
 from dataclasses import dataclass
 
-from .errors import misuse
+from .errors import _misuse as misuse
 
 __all__ = ["BytesIn", "ServerProfile", "Settings"]
 

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/src/chtypes/_ocifetch/__init__.py

@@ -6,7 +6,8 @@ logic lives in sibling modules. The seam (docs/guides/fetch-v1.md "The seam")
 is the only surface anything outside this package may use: `ensure`,
 `resolve_installed`, `list_installed`, `verify_installed` and
 `fetch_signed`, plus the `Request`/`Options`/`Resolved`/
-`VerifyResult` shapes they take and return. Every other module here
+`VerifyResult` shapes they take and return; and beside it the CLI's `resolve`
+and `prune` and the registry's in-use `hold` (§1, "In use"). Every other module here
 (`_http`, `_oci`, `_referrers`, `_dsse`, `_unpack`, `_layout`,
 `_lock`, `_errors`) is a private implementation detail of `_ensure`.
 
@@ -30,15 +31,25 @@ from chtypes._ocifetch._ensure import (
     resolve_installed,
     verify_installed,
 )
+from chtypes._ocifetch._hold import hold
+from chtypes._ocifetch._prune import Superseded, prune
+from chtypes._ocifetch._resolve import PlatformOutcome, Resolution, resolve, resolve_each
 
 __all__ = [
     "Options",
+    "PlatformOutcome",
     "Request",
+    "Resolution",
     "Resolved",
+    "Superseded",
     "VerifyResult",
     "ensure",
     "fetch_signed",
+    "hold",
     "list_installed",
+    "prune",
+    "resolve",
+    "resolve_each",
     "resolve_installed",
     "verify_installed",
 ]

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/src/chtypes/_ocifetch/_dsse.py

@@ -21,6 +21,7 @@ import base64
 import hashlib
 import json
 import os
+from collections.abc import Sequence
 from dataclasses import dataclass
 
 from chtypes import _ed25519
@@ -86,12 +87,21 @@ def env_trusted_keys() -> tuple[TrustedKey, ...] | None:
     items = [part.strip() for part in raw.split(",") if part.strip()]
     if not items:
         return None
+    return trusted_keys_from_hex(items, C.ENV_TRUSTED_KEYS_NAME)
+
+
+def trusted_keys_from_hex(items: Sequence[str], where: str) -> tuple[TrustedKey, ...]:
+    """Raw 32-byte ed25519 public keys, each as 64 hex digits (the
+    `CHTYPES_TRUSTED_KEYS` spelling, and the `trusted_keys` option's, as in
+    every binding), as trust-list records whose key id is the generated
+    constants' algorithm, sha256-first16hex of the raw key. A bad key is a
+    ValueError naming `where`."""
     keys = []
     for item in items:
         try:
-            public = _hex_to_bytes(item)
+            public = _hex_to_bytes(item.strip())
         except ValueError as exc:
-            raise ValueError(f"chtypes: {C.ENV_TRUSTED_KEYS_NAME} holds a bad key: {exc}") from None
+            raise ValueError(f"chtypes: {where} holds a bad key: {exc}") from None
         keys.append(TrustedKey(keyid=hashlib.sha256(public).hexdigest()[:16], public_key=public))
     return tuple(keys)
 

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/src/chtypes/_ocifetch/_ensure.py

@@ -867,7 +867,13 @@ def _ensure_floating(
     assert record is not None
 
     def _add_entry(doc: dict) -> dict:
-        entries = [*(doc.get("manifests") or []), {"digest": platform_desc.digest}]
+        entries = list(doc.get("manifests") or [])
+        # One entry per manifest digest: a re-install, or a concurrent one that
+        # landed first, never writes a second (public issue #487).
+        if not any(
+            isinstance(e, dict) and e.get("digest") == platform_desc.digest for e in entries
+        ):
+            entries.append({"digest": platform_desc.digest})
         return {**doc, "manifests": entries}
 
     try:
@@ -899,6 +905,82 @@ def _ensure_floating(
     return resolved
 
 
+def _installed_by_pin(
+    request: Request, platform_key: str, options: Options, pin: LockPin
+) -> Resolved | None:
+    return _lookup_by_pin(request, platform_key, options, pin)[0]
+
+
+def _lookup_by_pin(
+    request: Request, platform_key: str, options: Options, pin: LockPin
+) -> tuple[Resolved | None, bool]:
+    """The installed build a lock pin names, with no network: an unpacked
+    directory named by the pinned manifest digest, in the cache or a system
+    directory, whose `verified.json` is for this platform and records exactly
+    the pinned manifest, layer and bundle digests (public issues #414, #487).
+    `None` when no installed build matches; the flag is true when a record
+    exists under the pinned manifest but disagrees with the pin."""
+    roots = search_roots(options.cache_dir, options.system_dirs)
+    manifest_hex = parse_digest(pin.manifest)
+    mismatch = False
+    for root in roots:
+        udir = unpacked_dir_for(root, manifest_hex)
+        record = read_verified_record(udir)
+        if record is None:
+            continue
+        if (
+            record.platform == platform_key
+            and record.manifest == pin.manifest
+            and record.layer == pin.layer
+            and record.bundle == pin.bundle
+        ):
+            return (
+                _record_to_resolved(
+                    udir,
+                    record,
+                    request_spelling=request.spelling,
+                    already_installed=True,
+                    source=_source_for_dir(udir, roots),
+                ),
+                False,
+            )
+        mismatch = True
+    return None, mismatch
+
+
+def _resolve_frozen_offline(
+    request: Request, platform_key: str, options: Options, lock: Lock | None
+) -> Resolved:
+    """`--frozen --offline`: verify the installed build against the lock with
+    zero network. No lock entry for the request, or an installed record that
+    does not match the pin, is `CHTYPES_ARTIFACT_PINNED`; a pinned build that
+    is not installed is `CHTYPES_ARTIFACT_MISSING`."""
+    if lock is None:
+        raise ArtifactPinnedError("chtypes: --frozen requires a lock file; none was given")
+    pin = lock.pin_for(request.spelling, platform_key)
+    if pin is None:
+        raise ArtifactPinnedError(
+            f"chtypes: --frozen: no pin for {request.spelling!r} ({platform_key}) in the lock"
+        )
+    roots = search_roots(options.cache_dir, options.system_dirs)
+    warnings = probe_roots(roots, options.resolved_strict())
+    installed, mismatch = _lookup_by_pin(request, platform_key, options, pin)
+    if installed is None:
+        if mismatch:
+            raise ArtifactPinnedError(
+                f"chtypes: --frozen --offline: the installed build for {request.spelling} "
+                f"({platform_key}) does not match the lock's pin {pin.manifest}"
+            )
+        raise ArtifactMissingError(
+            with_notes(
+                f"chtypes: --frozen --offline: the build the lock pins ({pin.manifest}) "
+                f"is not installed for {request.spelling} ({platform_key})",
+                missing_notes(options),
+            )
+        )
+    return replace(installed, warnings=(*warnings, *installed.warnings))
+
+
 def _ensure_frozen(
     request: Request, platform_key: str, options: Options, lock: Lock | None
 ) -> Resolved:
@@ -916,17 +998,9 @@ def _ensure_frozen(
     cache_root_path = roots[0]
     scratch_root = _scratch_root(cache_root_path)
 
-    manifest_hex = parse_digest(pin.manifest)
-    existing_dir = unpacked_dir_for(cache_root_path, manifest_hex)
-    existing_record = read_verified_record(existing_dir)
-    if existing_record is not None and existing_record.platform == platform_key:
-        return _record_to_resolved(
-            existing_dir,
-            existing_record,
-            request_spelling=request.spelling,
-            already_installed=True,
-            source="cache",
-        )
+    installed = _installed_by_pin(request, platform_key, options, pin)
+    if installed is not None:
+        return installed
 
     manifest_doc, _raw = fetch_manifest_by_digest(bases, pin.manifest, policy=policy, retry=retry)
     layer_desc = manifest_layer_descriptor(manifest_doc)
@@ -1049,6 +1123,9 @@ def _ensure(
     request: Request, platform_key: str, options: Options, lock: Lock | None, roots: Sequence[Path]
 ) -> Resolved:
     try:
+        if options.resolved_offline() and options.frozen:
+            return _resolve_frozen_offline(request, platform_key, options, lock)
+
         if options.resolved_offline():
             resolved = resolve_installed(request, platform_key, options)
             if resolved is None:

{chtypes-2.0.0.dev6 → chtypes-2.0.0.dev8}/src/chtypes/_ocifetch/_errors.py

@@ -47,17 +47,19 @@ __all__ = [
     "SourceUnreachableError",
 ]
 
-CODE_ARTIFACT_MISSING = "CHTYPES_ARTIFACT_MISSING"
-CODE_ARTIFACT_UNTRUSTED = "CHTYPES_ARTIFACT_UNTRUSTED"
-CODE_ARTIFACT_CORRUPT = "CHTYPES_ARTIFACT_CORRUPT"
-CODE_ARTIFACT_PINNED = "CHTYPES_ARTIFACT_PINNED"
-CODE_ARTIFACT_UNPUBLISHED = "CHTYPES_ARTIFACT_UNPUBLISHED"
-CODE_SOURCE_UNREACHABLE = "CHTYPES_SOURCE_UNREACHABLE"
-CODE_SOURCE_UNAUTHORIZED = "CHTYPES_SOURCE_UNAUTHORIZED"
-CODE_SOURCE_FORBIDDEN = "CHTYPES_SOURCE_FORBIDDEN"
-CODE_SOURCE_INCOMPATIBLE = "CHTYPES_SOURCE_INCOMPATIBLE"
-CODE_CACHE_UNUSABLE = "CHTYPES_CACHE_UNUSABLE"
-CODE_SOURCE_RETIRED = "CHTYPES_SOURCE_RETIRED"
+from chtypes._codes import (
+    CODE_ARTIFACT_CORRUPT,
+    CODE_ARTIFACT_MISSING,
+    CODE_ARTIFACT_PINNED,
+    CODE_ARTIFACT_UNPUBLISHED,
+    CODE_ARTIFACT_UNTRUSTED,
+    CODE_CACHE_UNUSABLE,
+    CODE_SOURCE_FORBIDDEN,
+    CODE_SOURCE_INCOMPATIBLE,
+    CODE_SOURCE_RETIRED,
+    CODE_SOURCE_UNAUTHORIZED,
+    CODE_SOURCE_UNREACHABLE,
+)
 
 _CODES_USED_HERE = (
     CODE_ARTIFACT_MISSING,