chtypes 2.0.0.dev9__tar.gz → 2.0.0.dev10__tar.gz

{chtypes-2.0.0.dev9 → chtypes-2.0.0.dev10}/CHANGELOG.md

@@ -20,6 +20,8 @@ The four bindings in this repository are released together and give one answer,
 - The v2 spec's decline catalog has 49 entries, and it publishes the settings values a server refuses outright (`spec/abi-v2/where-settings/documented-pathological.tsv`).
 - The v2 spec's decline catalog follows the producer's batch 11: its `server-profile-setting` entry now covers every settings layer the schema compiles under (the id is unchanged), the evaluation-time setting entries name only the filter's own settings, and its `stack-depth` entry lists `chs_schema_describe`.
 - The v2 spec publishes the documented platform differences (`spec/abi-v2/documented-platform/`), and its decline catalog adds `setting-format-pair`, `string-reading` and `equals-fold` (52 reasons).
+- The v2 spec's decline catalog adds `server-dependent-function` (a column's DEFAULT, MATERIALIZED, ALIAS or EPHEMERAL expression, or a CHECK, that calls a function whose resolver needs a server is declined at `chs_schema_create`), `function-name-as-lambda` (a bare function name written where a higher-order function takes a lambda, such as `arrayExists(isNull, arr)`, is declined at `chs_filter_create` on 26.7, 26.8 and 26.9) and `skipped-key-in-squashed-block` (a batch whose answer depends on how the INSERT's squashing step groups its chunks is unsupported), and `call-setting-session-timezone` names only a per-call `session_timezone` the call cannot attach (55 reasons).
+- Build-level revocation for generation 2 (#625; `docs/guides/fetch-v1.md`, "Build-level revocation"). A build its signed revocation refuses is the new code `CHTYPES_ARTIFACT_REVOKED`, exit status 11 (`CODE_ARTIFACT_REVOKED`, `ArtifactRevokedError`), whose message names the advisory and whose `.revocation` carries it for programs (`Revocation`, `Advisory`; an advisory's `class` is `class_`). An install reads a revocation from the trust listing it already makes and refuses before the layer; `--frozen` makes one listing, warm or cold, so a warm frozen fetch makes 1 request instead of 0 (`--frozen --offline` stays at 0); a registry open, `preload` and a fetch that finds the build installed check it at most once per build per process, one attempt per base, no backoff and one 5 s deadline (`check_installed`); `--lock` refuses a revoked build on every platform it pins. A verified revocation is stored in the build's cache entry (`revocation.json`) and refuses it from then on with no request, offline too; `chtypes verify` reports it and exits 11. A revocation that does not verify is ignored with one warning; a status that cannot be read is one warning, or with `FetchOptions(require_revocation_check=True)` (`--require-revocation-check`) the transport's error. `FetchOptions(allow_revoked=True)` (`--allow-revoked`) uses a revoked build for one call, with a warning, never for a lock or an update. A `410` whose body's `errors[0].detail.revoked` is an object is `CHTYPES_ARTIFACT_REVOKED`; every other `410` stays `CHTYPES_SOURCE_RETIRED`. The v1 contract is unchanged.
 - A `410 Gone` is a retired repository, and permanent (#571). Any request the fetch layer makes (a tag, an alias, a digest, a blob, `tags/list`, the referrers API or its fallback tag, after a redirect too) that answers `410` is now the new code `CHTYPES_SOURCE_RETIRED` (exit status 10), never retried and never sent to the next base, nor from the dev channel's alias to its tag. It was `CHTYPES_SOURCE_UNREACHABLE` (exit 3) with the registry's message dropped, and Go, Python and Rust moved on to the next base, so a mirror's 404 could hide the retirement. The error names the URL that answered and carries the registry's own message, with control and bidirectional-formatting characters removed and cut to 256 code points: Go `CodeSourceRetired` (`errors.Is(err, ErrSourceRetired)`), Python and TypeScript `SourceRetiredError`, Rust `Error::SourceRetired`. The rule is `docs/guides/fetch-v1.md` section 2, "A retired repository", and the `retired-410-*` and `list-tags-retired-410` conformance cases and one shared message table (`tests/fixtures/retired-message/cases.json`) hold all four bindings to it.
 - A 2.0.0-dev SDK prefers the newest build of its own fingerprint (#511). A version request resolves `<tag>--fp-<fingerprint>` first, the alias the staging dev registry serves beside every dev tag, with the binding's own generated ABI fingerprint, and the tag itself only when every base answers the alias 404; any other failure of the alias request is that failure, with no fallback, and an alias whose index lacks the platform is `CHTYPES_ARTIFACT_UNPUBLISHED`. The dev channel's cache lookups (the installed-build lookup a registry open makes first, the offline answer, and the monotonic rule of a fetch) now see only records whose signed `abi_fingerprint` is the SDK's own, so a newer build of another fingerprint, installed by another dev SDK in the shared cache, is never returned or kept over this one's own, and is never removed or rewritten. So a dev build of a newer fingerprint no longer strands an older dev SDK until its consumers upgrade. Trust is unchanged (the signed predicate and the load checks decide), a listing never shows an alias, nothing records one, `list` and `where` are unchanged, and there is no option: both steps are the dev channel's alone, and the v1 contract and every production channel never take them. The rule is `docs/guides/fetch-v1.md` section 3, and the `dev-alias-*`, `resolve-installed-dev-alias-*` and `list-tags-dev-alias` conformance cases hold all four bindings to it.
 
@@ -65,6 +67,7 @@ The four bindings in this repository are released together and give one answer,
 - **Enforcement gate lifted** for `26.7` on `darwin-arm64`.
 - **Enforcement gate lifted** for `26.8` on `darwin-arm64`.
 - **Enforcement gate lifted** for `26.9` on `darwin-arm64`.
+- `docs/limitations.md` is re-checked against artifact build `20261012.142240` (v2-dev.13), the library half measured on that build: functions that need a server, in a column expression, and a body the server may read serially or in parallel under a non-zero error budget are listed under "Declined by design"; the known divergence for a column expression that calls a function needing a server is retired, since that build declines those schemas (on 26.8 and 26.9 the `AI*` and `ai*` functions compile, as the page says); and the `lossy` raw-NUL TSV gap is marked fixed on that build and still present in the 1.0 libraries. `docs/guides/filters.md` gains a section on subcolumn filters and a JSON value typed by its batch.
 
 ### Removed
 
@@ -72,6 +75,8 @@ The four bindings in this repository are released together and give one answer,
 
 ### Fixed
 
+- A response body cut off part way (shorter than its Content-Length, a reset, a TLS close) is retried like a 5xx and, once the retry table is spent, is `CHTYPES_SOURCE_UNREACHABLE`; only an oversize body is `CHTYPES_ARTIFACT_CORRUPT`, and it is never retried (#641). Python took a body that ended before its Content-Length as the whole body and failed its digest check as corrupt without a retry; now a short body, or `http.client.IncompleteRead`, is retried and ends as unreachable. The rule is `docs/guides/fetch-v1.md` section 7, and the new `body-cut-off-then-ok` and `body-cut-off-exhausted` conformance cases hold every binding to it.
+- A chunked response body cut off before its terminating zero-length chunk is held to the same rule (#644): the new `body-cut-off-chunked-then-ok` and `body-cut-off-chunked-exhausted` conformance cases cut one on the layer blob GET, and this binding already retried it and ended as `CHTYPES_SOURCE_UNREACHABLE` once the retry table was spent. No behavior change; the case now holds it.
 - A settings or query-parameter key or value that is not valid UTF-8 is refused as a usage error; it was silently replaced with U+FFFD, so the library saw a different value. A string holding a lone surrogate (what `surrogateescape` makes of undecodable bytes) was sent as a `\udXXX` escape; the same inputs are refused now, for the per-call zone, the setup defaults and the server profile too. Valid UTF-8 serializes as before.
 - A declined export's payload is absent, as documented: it was an empty payload when the library returned a zero-length buffer for a declined export. `BatchResult.payload` and `spans` are `None` for a declined export. Whether an export was declined is decided from the document's `export_declined`, never from the buffer's length.
 - The layout's `index.json` never gains a second entry for one manifest (#487): concurrent installs of one build each appended an entry (7 or 8 entries for the same digest after 8 concurrent installs, measured on 1.0.2). The merge now deduplicates by digest, as Go and TypeScript do.

{chtypes-2.0.0.dev9 → chtypes-2.0.0.dev10}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: chtypes
-Version: 2.0.0.dev9
+Version: 2.0.0.dev10
 Summary: Python binding for the chtypes artifacts: ClickHouse's own type system, per version, behind the frozen chs_* C ABI
 Project-URL: Homepage, https://github.com/wave-rf/chtypes
 Project-URL: Repository, https://github.com/wave-rf/chtypes

{chtypes-2.0.0.dev9 → chtypes-2.0.0.dev10}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "chtypes"
-version = "2.0.0.dev9"
+version = "2.0.0.dev10"
 description = "Python binding for the chtypes artifacts: ClickHouse's own type system, per version, behind the frozen chs_* C ABI"
 readme = "README.md"
 license = "Apache-2.0"

{chtypes-2.0.0.dev9 → chtypes-2.0.0.dev10}/src/chtypes/__init__.py

@@ -69,6 +69,7 @@ from .errors import (
     CODE_ARTIFACT_INCOMPATIBLE,
     CODE_ARTIFACT_MISSING,
     CODE_ARTIFACT_PINNED,
+    CODE_ARTIFACT_REVOKED,
     CODE_ARTIFACT_UNPUBLISHED,
     CODE_ARTIFACT_UNTRUSTED,
     CODE_CACHE_UNUSABLE,
@@ -77,17 +78,20 @@ from .errors import (
     CODE_SOURCE_RETIRED,
     CODE_SOURCE_UNAUTHORIZED,
     CODE_SOURCE_UNREACHABLE,
+    Advisory,
     ArtifactCorruptError,
     ArtifactError,
     ArtifactIncompatibleError,
     ArtifactMissingError,
     ArtifactPinnedError,
+    ArtifactRevokedError,
     ArtifactUnpublishedError,
     ArtifactUntrustedError,
     CacheUnusableError,
     CallError,
     ChtypesError,
     InternalError,
+    Revocation,
     SchemaError,
     SourceForbiddenError,
     SourceIncompatibleError,
@@ -130,6 +134,7 @@ __all__ = [
     "CODE_ARTIFACT_INCOMPATIBLE",
     "CODE_ARTIFACT_MISSING",
     "CODE_ARTIFACT_PINNED",
+    "CODE_ARTIFACT_REVOKED",
     "CODE_ARTIFACT_UNPUBLISHED",
     "CODE_ARTIFACT_UNTRUSTED",
     "CODE_CACHE_UNUSABLE",
@@ -139,11 +144,13 @@ __all__ = [
     "CODE_SOURCE_UNAUTHORIZED",
     "CODE_SOURCE_UNREACHABLE",
     "EXPORT_NONE",
+    "Advisory",
     "ArtifactCorruptError",
     "ArtifactError",
     "ArtifactIncompatibleError",
     "ArtifactMissingError",
     "ArtifactPinnedError",
+    "ArtifactRevokedError",
     "ArtifactUnpublishedError",
     "ArtifactUntrustedError",
     "AtMergeEntry",
@@ -183,6 +190,7 @@ __all__ = [
     "Reason",
     "Registry",
     "Resolved",
+    "Revocation",
     "RowResult",
     "Schema",
     "SchemaDescription",

{chtypes-2.0.0.dev9 → chtypes-2.0.0.dev10}/src/chtypes/__main__.py

@@ -3,6 +3,7 @@
 
     chtypes fetch <spelling>... | --all  [--platform <os-arch>] [--cache <dir>] [--lock <file>]
                                          [--frozen] [--offline] [--update] [--strict]
+                                         [--allow-revoked] [--require-revocation-check]
     chtypes verify [--cache <dir>] [--strict]   re-verify every installed build
     chtypes list   [--cache <dir>] [--offline] [--strict]
     chtypes where  [--cache <dir>] [--strict] [--all]   the cache root; --all: every search dir
@@ -37,7 +38,7 @@ from collections.abc import Sequence
 
 from . import __version__ as chtypes_version
 from . import _ocifetch as fetch_layer
-from ._codes import CODE_ARTIFACT_CORRUPT
+from ._codes import CODE_ARTIFACT_CORRUPT, CODE_ARTIFACT_REVOKED
 from ._ocifetch import _channel
 from ._ocifetch import _constants as C
 from ._ocifetch._ensure import _translate_transport_error, detect_host_platform, missing_notes
@@ -148,6 +149,19 @@ def build_parser() -> argparse.ArgumentParser:
             "(or set CHTYPES_OFFLINE=1)"
         ),
     )
+    fetch.add_argument(
+        "--allow-revoked",
+        action="store_true",
+        help=(
+            "use a build a verified revocation refuses, with one warning "
+            "(this call only; never --lock or --update)"
+        ),
+    )
+    fetch.add_argument(
+        "--require-revocation-check",
+        action="store_true",
+        help="a revocation status that cannot be read is the transport's error, not a warning",
+    )
 
     verify = sub.add_parser(
         "verify",
@@ -249,6 +263,8 @@ def _options(args: argparse.Namespace, *, platform: str | None = None):
         lock_path=lock_path,
         lock_write=lock_write,
         update=getattr(args, "update", False),
+        allow_revoked=getattr(args, "allow_revoked", False),
+        require_revocation_check=getattr(args, "require_revocation_check", False),
     )
     return fetch._to_options(platform)
 
@@ -348,10 +364,23 @@ def _cmd_verify(args: argparse.Namespace) -> int:
         return EXIT_OK
     _say_notes(options)
     failed = 0
+    revoked = 0
     for r in results:
-        if not r.ok:
+        if r.revocation is not None:
+            # A build a stored revocation refuses (docs/guides/fetch-v1.md,
+            # "Build-level revocation"); it is reported whatever its bytes say.
+            revoked += 1
+            _say(f"REVOKED {r.version} {r.platform} {r.dir}: {r.detail}")
+        elif not r.ok:
             failed += 1
             _say(f"chtypes: FAILED {r.version} {r.platform} {r.dir}: {r.detail}")
+    if revoked:
+        # Exit 11 wins over a mismatch's 1: a revocation is the more specific finding.
+        _say(
+            f"chtypes: {revoked} of {len(results)} installed build(s) are revoked "
+            f"[{CODE_ARTIFACT_REVOKED}]"
+        )
+        return C.ERROR_EXIT_CODES[CODE_ARTIFACT_REVOKED]
     if failed:
         _say(f"chtypes: {failed} of {len(results)} installed build(s) FAILED verification")
         return C.ERROR_EXIT_CODES[CODE_ARTIFACT_CORRUPT]

chtypes-2.0.0.dev10/src/chtypes/_advisory.py

@@ -0,0 +1,38 @@
+"""What a `CHTYPES_ARTIFACT_REVOKED` error carries for programs
+(docs/guides/fetch-v1.md, "Build-level revocation"; public issue #625).
+
+A leaf module, so the fetch layer's errors and the public ones share the two
+types without importing each other."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+__all__ = ["Advisory", "Revocation"]
+
+
+@dataclass(frozen=True)
+class Advisory:
+    """One advisory a revocation names, every string made safe to print. A
+    signed revocation's carries its `revoked_at`; a 410's carries the `build` it
+    names, newest first. A field the source did not send is "". `class_` is the
+    advisory's `class` (a keyword in Python)."""
+
+    id: str = ""
+    url: str = ""
+    class_: str = ""
+    summary: str = ""
+    build: str = ""
+    superseded_by: str = ""
+    revoked_at: str = ""
+
+
+@dataclass(frozen=True)
+class Revocation:
+    """`signed` is true for a verified, signed revocation of `manifest`, whose
+    one advisory it is, and false for a 410's body, which is unsigned, names no
+    manifest (`manifest` is "") and lists one advisory per revoked build."""
+
+    signed: bool = False
+    manifest: str = ""
+    advisories: tuple[Advisory, ...] = ()

{chtypes-2.0.0.dev9 → chtypes-2.0.0.dev10}/src/chtypes/_codes.py

@@ -1,6 +1,6 @@
 # Code generated by scripts/fetch-v1/gen-constants.py from
 # spec/fetch-v1/constants.json. DO NOT EDIT.
-# Source sha256: 82afa8d6cba96c10b5945c6f7c1c8360a504cc8ac6231e74017cc6c20b908df6
+# Source sha256: 5aa3d14a5d8b60399040a8242f003135fa5f6c2c402de7d4e2db4c768a3366ae
 """The shared error codes (docs/guides/fetch-v1.md §8), one per entry of the
 fetch constants' `errors`, in their order. `chtypes.errors` re-exports them
 (docs/reference/bindings-v1.md §4, the codes)."""
@@ -21,3 +21,4 @@ CODE_SOURCE_INCOMPATIBLE: Final[str] = "CHTYPES_SOURCE_INCOMPATIBLE"
 CODE_ARTIFACT_INCOMPATIBLE: Final[str] = "CHTYPES_ARTIFACT_INCOMPATIBLE"
 CODE_CACHE_UNUSABLE: Final[str] = "CHTYPES_CACHE_UNUSABLE"
 CODE_SOURCE_RETIRED: Final[str] = "CHTYPES_SOURCE_RETIRED"
+CODE_ARTIFACT_REVOKED: Final[str] = "CHTYPES_ARTIFACT_REVOKED"

{chtypes-2.0.0.dev9 → chtypes-2.0.0.dev10}/src/chtypes/_ocifetch/__init__.py

@@ -4,7 +4,7 @@ Not part of this package's public API: every name here is subject to change
 without notice until the v1 switch. `_constants` is generated; the fetch
 logic lives in sibling modules. The seam (docs/guides/fetch-v1.md "The seam")
 is the only surface anything outside this package may use: `ensure`,
-`resolve_installed`, `list_installed`, `verify_installed` and
+`resolve_installed`, `check_installed`, `list_installed`, `verify_installed` and
 `fetch_signed`, plus the `Request`/`Options`/`Resolved`/
 `VerifyResult` shapes they take and return; and beside it the CLI's `resolve`
 and `prune` and the registry's in-use `hold` (§1, "In use"). Every other module here
@@ -25,9 +25,11 @@ from chtypes._ocifetch._ensure import (
     Request,
     Resolved,
     VerifyResult,
+    check_installed,
     ensure,
     fetch_signed,
     list_installed,
+    reset_revocation_memo_for_tests,
     resolve_installed,
     verify_installed,
 )
@@ -43,11 +45,13 @@ __all__ = [
     "Resolved",
     "Superseded",
     "VerifyResult",
+    "check_installed",
     "ensure",
     "fetch_signed",
     "hold",
     "list_installed",
     "prune",
+    "reset_revocation_memo_for_tests",
     "resolve",
     "resolve_each",
     "resolve_installed",

{chtypes-2.0.0.dev9 → chtypes-2.0.0.dev10}/src/chtypes/_ocifetch/_channel.py

@@ -133,6 +133,10 @@ class Channel:
     # lookups see only records signed with it. "" does neither (the v1 contract
     # and every production channel).
     own_fingerprint: str = ""
+    # Build-level revocation (_revocation.py; docs/guides/fetch-v1.md, "Build-level
+    # revocation"): generation 2, the dev channel and the production one. The v1
+    # contract has none of it.
+    revocation: bool = False
 
 
 # What a 2.0.0-dev SDK speaks, and what every process speaks unless a test
@@ -149,6 +153,7 @@ DEV_CHANNEL: Final = Channel(
     overridable=False,
     pinnable=False,
     own_fingerprint=CHS_ABI_FINGERPRINT.removeprefix("sha256:"),
+    revocation=True,
 )
 
 # The v1 contract the fetch-v1 conformance cases specify, from the generated
@@ -183,6 +188,7 @@ PROD_V2_CHANNEL: Final = Channel(
     keys=C.RELEASE_KEYS,
     overridable=True,
     pinnable=True,
+    revocation=True,
 )
 
 _lock = threading.Lock()

{chtypes-2.0.0.dev9 → chtypes-2.0.0.dev10}/src/chtypes/_ocifetch/_constants.py

@@ -1,6 +1,6 @@
 # Code generated by scripts/fetch-v1/gen-constants.py from
 # spec/fetch-v1/constants.json. DO NOT EDIT.
-# Source sha256: 82afa8d6cba96c10b5945c6f7c1c8360a504cc8ac6231e74017cc6c20b908df6
+# Source sha256: 5aa3d14a5d8b60399040a8242f003135fa5f6c2c402de7d4e2db4c768a3366ae
 """The v1 fetch constants, generated: see docs/guides/fetch-v1.md."""
 
 from __future__ import annotations
@@ -46,6 +46,7 @@ ARTIFACT_TYPE: Final[str] = "application/vnd.wavehouse.chtypes.artifact.v1"
 GOLDENS_ARTIFACT_TYPE: Final[str] = "application/vnd.wavehouse.chtypes.goldens.v1"
 FIXTURES_ARTIFACT_TYPE: Final[str] = "application/vnd.wavehouse.chtypes.sdk-fetch-fixtures.v1"
 CHANNEL_ARTIFACT_TYPE: Final[str] = "application/vnd.wavehouse.chtypes.channel.v1"
+REVOCATION_ARTIFACT_TYPE: Final[str] = "application/vnd.wavehouse.chtypes.revocation.v1"
 
 DSSE_PAYLOAD_TYPE: Final[str] = "application/vnd.in-toto+json"
 DSSE_MAX_SIGNATURES: Final[int] = 4
@@ -56,6 +57,7 @@ PREDICATE_TYPE_ARTIFACT: Final[str] = "https://artifacts.wavehouse.dev/spec/arti
 PREDICATE_TYPE_GOLDENS: Final[str] = "https://artifacts.wavehouse.dev/spec/goldens/v1"
 PREDICATE_TYPE_FIXTURES: Final[str] = "https://artifacts.wavehouse.dev/spec/sdk-fetch-fixtures/v1"
 PREDICATE_TYPE_CHANNEL: Final[str] = "https://artifacts.wavehouse.dev/spec/channel/v1"
+PREDICATE_TYPE_REVOCATION: Final[str] = "https://artifacts.wavehouse.dev/spec/revocation/v1"
 
 KEYID_ALGORITHM: Final[str] = "sha256-first16hex"
 RELEASE_KEYS: Final[tuple[dict[str, str], ...]] = (
@@ -87,6 +89,7 @@ RETIRED_STATUSES: Final[tuple[int, ...]] = (410,)
 
 CONNECT_TIMEOUT_S: Final[float] = 30.0
 IDLE_READ_TIMEOUT_S: Final[float] = 60.0
+REVOCATION_CHECK_S: Final[float] = 5.0
 
 MANIFEST_MAX_BYTES: Final[int] = 4194304
 BUNDLE_MAX_BYTES: Final[int] = 1048576
@@ -109,6 +112,7 @@ PROD_V2_BASES: Final[tuple[str, ...]] = ("https://registry.wavehouse.dev/chtypes
 PROD_V2_SYSTEM_DIRS: Final[tuple[str, ...]] = ("/usr/local/share/chtypes/v2", "/opt/chtypes/v2")
 CACHE_UNPACKED_DIR: Final[str] = "unpacked/sha256"
 CACHE_VERIFIED_RECORD: Final[str] = "verified.json"
+CACHE_REVOCATION_RECORD: Final[str] = "revocation.json"
 CACHE_ANNOTATION_PREFIX: Final[str] = "dev.wavehouse.chtypes."
 
 LOCK_SCHEMA: Final[int] = 3
@@ -132,4 +136,5 @@ ERROR_EXIT_CODES: Final[dict[str, int]] = {
     "CHTYPES_ARTIFACT_INCOMPATIBLE": 8,
     "CHTYPES_CACHE_UNUSABLE": 9,
     "CHTYPES_SOURCE_RETIRED": 10,
+    "CHTYPES_ARTIFACT_REVOKED": 11,
 }