chtypes 2.0.0.dev10__tar.gz → 2.0.0.dev11__tar.gz

{chtypes-2.0.0.dev10 → chtypes-2.0.0.dev11}/CHANGELOG.md

@@ -8,6 +8,13 @@ The four bindings in this repository are released together and give one answer,
 
 ### Added
 
+- `column_defaults` is covered by unit tests (ABI v2): the exact `insert_options` JSON for a UTF-8 value, a non-UTF-8 value (`value_b64`), an empty value (sent as `""`) and two entries in order, and the decode of a `default_override` cell (`Source.DEFAULT_OVERRIDE`, stored). The batches guide gains a usage section. No behavior changed.
+- `CODE_DECLINED` (ABI v2, -2): the code a declined row, outcome or verdict carries, generated from the description and exported from `chtypes`, as `bindings-v1.md` section 3 promises.
+- `CallError.reason` (ABI v2): the error's catalog id, read through `chs_error_reason` and passed through verbatim (ASCII, empty if none), on all four call classes.
+- Spec batch 2, the options documents (ABI v2; moves the dev fingerprint): keyword-only `insertable_columns`, `column_defaults` (a sequence of `ColumnDefault`), `verdict_basis` (`VerdictBasis`: `INPUT`, `EXPORT`) and `defer_defaults` (`DeferDefaults`: `NONE`, `SERVER`) build `insert_options` for `Schema.row`, `Schema.rows`, `Schema.parse_block` and `Filter.rows`; `param_encoding` (`ParamEncoding`: `ESCAPED`, `RAW`) builds `filter_options` for `Schema.compile_filter`; `posture` (`Posture`: `CREATE`, `LOAD`) builds `schema_options` for `Library.compile_table`. `insert_options_extra`, `filter_options_extra` and `schema_options_extra` pass a member the binding has no name for through as JSON text, so the library, which refuses an unknown key (rule r1), is the only judge. A document with no member set is not sent. `Filter.rows` takes `columns` (`chs_filter_eval_body` takes the column list).
+- Spec batch 2, a value that is not UTF-8 (ABI v2): a settings value, a query-parameter value, a `setup` default and a server profile's settings value is sent as `{"b64": ...}` (the content `json_object_byte_values`), where the binding used to refuse it, and `SchemaServer.settings` decodes the same form back (as `surrogateescape` text, the spelling the input side takes).
+- Spec batch 2, the decline and refusal members (ABI v2): `RowResult`, `BatchResult` and `FilterResult` carry `reason` (the catalog id, present where the code is -2, `CHS_CODE_DECLINED`), `refused_at` (a `RefusalSite`: `CALL`, `BODY_READ`, `ROW`, `BODY_STEP`) and `refused_row`; `RowResult.verdict_reason`; `FilterRowError.reason`; `BatchResult.export_columns` and `BatchResult.export_declined_reason`.
+- Spec batch 2, the schema description (ABI v2): `SchemaDescription.posture`, and `Column.default_support` (a `DefaultSupport`), `Column.default_support_reason` and `Column.reads`, reserved members a build reports when it implements them. `Source.DEFAULT_DEFERRED`, `Source.DEFAULT_OVERRIDE` (stored) and `DeclinedTier.SERVER_REFUSES` join the vocabularies.
 - A fetch-only call (#492): `Registry.fetch(request) -> Resolved` resolves, fetches, verifies and installs the build a request names into the cache, as `chtypes fetch` does, and opens nothing: no library is loaded and the process setup is untouched, so it needs no `setup`. It honors the registry's fetch options (offline, the cache, and under the production channel the bases, the trust and the lock), and a concurrent `fetch` and open of one request share one fetch. The `Resolved` names the installed version, build, digests and library path; errors are the fetch codes' classes.
 - `chtypes resolve <spelling> [--json]` (#493): what a line or version resolves to on every platform the registry offers, one `resolved <version> <platform> <build> <manifest>` line each, or with `--json` one JSON array of `{platform, version, build, manifest}` objects, each signed statement verified exactly as a fetch verifies it and no layer downloaded; `--offline` answers from the cache. The four CLIs print the same bytes, so one parser reads them (a CI cache key, for example).
 - `chtypes prune [--line <line>] [--keep <n>] [--dry-run]` (#494): removes the installed builds of each line and platform that at least n newer installed builds supersede (n defaults to 1), each with its record, its blobs and its `index.json` entry; `--dry-run` prints exactly what would go. It never removes a build a running process holds: a registry now holds every build it opens or fetches, with a shared `flock` on the entry's `verified.json` until the process exits, and prune removes an entry only after taking the exclusive lock without waiting, reporting it `in-use` otherwise. It never touches a system directory, nor, on the dev channel, another fingerprint's builds. The rules are `docs/guides/fetch-v1.md` sections 1 and 3, and the `resolve-build-*` and `prune-*` conformance cases hold all four bindings to them.
@@ -27,6 +34,10 @@ The four bindings in this repository are released together and give one answer,
 
 ### Changed
 
+- **Breaking (ABI v2 dev): `BatchResult.spans` is a tuple of `Span | None`**, one entry per `rows` entry and index-aligned with it: `row_spans` carries `null` for a row the export did not carry (a skipped row, or a row a filter cut), which reads as `None`. It was the exported rows' spans only.
+- **Breaking (ABI v2 dev): `AtMergeEntry.row` is `AtMergeEntry.part_row`**, the document's `part_row`.
+- A document's `code` can be -2 (`CHS_CODE_DECLINED`) for a row, batch, filter result or verdict this build declined, and every decline names its `reason`.
+- The refusal of a settings, query-parameter, default or server-settings value that is not valid UTF-8 is gone (it is the `{"b64": ...}` form now); a key that is not valid UTF-8, a macro and a server timezone are still refused, as a `UsageError` naming the input.
 - `chtypes list` and the tag listing name the published lines only (two-part, numeric order), as Go's always did; `chtypes resolve <line>` prints the exact version a line resolves to (#613).
 - **Breaking: the public surface the parity gate carried as exceptions is resolved, before the v2 lock (#499, #500).** A convenience exists in every binding or in none (`docs/reference/bindings-v1.md` §1, principle 2). Migration, old → new:
   - `chtypes.errors.internal` and `chtypes.errors.misuse` → private (#499): they were the binding's own construction helpers, never documented. Catch `InternalError` and `UsageError`, as before.
@@ -71,6 +82,7 @@ The four bindings in this repository are released together and give one answer,
 
 ### Removed
 
+- **Breaking (ABI v2 dev): `RowResult.unsupported_settings`**: the row document no longer carries `unsupported_settings`; the batch's and the filter result's lists stay.
 - `SchemaServer.filter_declined_settings` (a breaking change inside v2 dev, #588): the server member no longer carries the list. Read `SchemaDescription.filter_declined_settings`, whose `server`-layer entries are the profile's.
 
 ### Fixed

{chtypes-2.0.0.dev10 → chtypes-2.0.0.dev11}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: chtypes
-Version: 2.0.0.dev10
+Version: 2.0.0.dev11
 Summary: Python binding for the chtypes artifacts: ClickHouse's own type system, per version, behind the frozen chs_* C ABI
 Project-URL: Homepage, https://github.com/wave-rf/chtypes
 Project-URL: Repository, https://github.com/wave-rf/chtypes
@@ -96,7 +96,7 @@ A bad **row** is a verdict, not an exception: `outcome` becomes `Outcome.REJECTE
 
 **`ctypes` releases the GIL for the whole of a foreign call**, so calls on compiled handles run in parallel across threads. The package takes no lock around a call; it keeps a close guard per handle and one setup guard.
 
-**The INSERT column list is a keyword-only `columns`** on `schema.row`, `schema.rows` and `schema.parse_block`: `schema.row(Format.JSON_EACH_ROW, body, columns=["id", "e"])`. `None` or an empty sequence is the no-list shape; a list makes the data supply exactly those columns, with a listed `EPHEMERAL` value read and in scope for the DEFAULTs that reference it but never stored.
+**The INSERT column list is a keyword-only `columns`** on `schema.row`, `schema.rows`, `schema.parse_block` and `filter.rows`: `schema.row(Format.JSON_EACH_ROW, body, columns=["id", "e"])`. `None` or an empty sequence is the no-list shape; a list makes the data supply exactly those columns, with a listed `EPHEMERAL` value read and in scope for the DEFAULTs that reference it but never stored.
 
 **Known gaps in 1.0** are listed, with workarounds, in [`docs/limitations.md`](https://github.com/wave-rf/chtypes/blob/main/docs/limitations.md#known-gaps-in-10).
 

{chtypes-2.0.0.dev10 → chtypes-2.0.0.dev11}/README.md

@@ -77,7 +77,7 @@ A bad **row** is a verdict, not an exception: `outcome` becomes `Outcome.REJECTE
 
 **`ctypes` releases the GIL for the whole of a foreign call**, so calls on compiled handles run in parallel across threads. The package takes no lock around a call; it keeps a close guard per handle and one setup guard.
 
-**The INSERT column list is a keyword-only `columns`** on `schema.row`, `schema.rows` and `schema.parse_block`: `schema.row(Format.JSON_EACH_ROW, body, columns=["id", "e"])`. `None` or an empty sequence is the no-list shape; a list makes the data supply exactly those columns, with a listed `EPHEMERAL` value read and in scope for the DEFAULTs that reference it but never stored.
+**The INSERT column list is a keyword-only `columns`** on `schema.row`, `schema.rows`, `schema.parse_block` and `filter.rows`: `schema.row(Format.JSON_EACH_ROW, body, columns=["id", "e"])`. `None` or an empty sequence is the no-list shape; a list makes the data supply exactly those columns, with a listed `EPHEMERAL` value read and in scope for the DEFAULTs that reference it but never stored.
 
 **Known gaps in 1.0** are listed, with workarounds, in [`docs/limitations.md`](https://github.com/wave-rf/chtypes/blob/main/docs/limitations.md#known-gaps-in-10).
 

{chtypes-2.0.0.dev10 → chtypes-2.0.0.dev11}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "chtypes"
-version = "2.0.0.dev10"
+version = "2.0.0.dev11"
 description = "Python binding for the chtypes artifacts: ClickHouse's own type system, per version, behind the frozen chs_* C ABI"
 readme = "README.md"
 license = "Apache-2.0"

{chtypes-2.0.0.dev10 → chtypes-2.0.0.dev11}/src/chtypes/__init__.py

@@ -47,22 +47,34 @@ from importlib.metadata import PackageNotFoundError as _PackageNotFound
 from importlib.metadata import version as _package_version
 
 from ._abi2._vocab import (
+    CODE_DECLINED,
     EXPORT_NONE,
     DeclinedLayer,
     DeclinedTier,
     DefaultKind,
+    DefaultSupport,
     DiscoverQueryParam,
     DocFlags,
     FilterOutcome,
     Format,
     MergeReason,
     Outcome,
+    Posture,
     Reason,
+    RefusalSite,
     Source,
     Status,
     Verdict,
 )
-from ._input import BytesIn, ServerProfile, Settings
+from ._input import (
+    BytesIn,
+    ColumnDefault,
+    DeferDefaults,
+    ParamEncoding,
+    ServerProfile,
+    Settings,
+    VerdictBasis,
+)
 from ._setup import setup
 from .errors import (
     CODE_ARTIFACT_CORRUPT,
@@ -143,6 +155,7 @@ __all__ = [
     "CODE_SOURCE_RETIRED",
     "CODE_SOURCE_UNAUTHORIZED",
     "CODE_SOURCE_UNREACHABLE",
+    "CODE_DECLINED",
     "EXPORT_NONE",
     "Advisory",
     "ArtifactCorruptError",
@@ -163,11 +176,14 @@ __all__ = [
     "Capabilities",
     "ChtypesError",
     "Column",
+    "ColumnDefault",
     "Computed",
     "DeclinedLayer",
     "DeclinedSetting",
     "DeclinedTier",
+    "DeferDefaults",
     "DefaultKind",
+    "DefaultSupport",
     "DiscoverQueryParam",
     "DiscoveredColumn",
     "Discovery",
@@ -187,8 +203,11 @@ __all__ = [
     "Library",
     "MergeReason",
     "Outcome",
+    "ParamEncoding",
+    "Posture",
     "Reason",
     "Registry",
+    "RefusalSite",
     "Resolved",
     "Revocation",
     "RowResult",
@@ -213,6 +232,7 @@ __all__ = [
     "UsageError",
     "Value",
     "Verdict",
+    "VerdictBasis",
     "cache_root",
     "open_unverified",
     "search_dirs",

{chtypes-2.0.0.dev10 → chtypes-2.0.0.dev11}/src/chtypes/_abi2/_decls.py

@@ -1,4 +1,4 @@
-# GENERATED by scripts/abi-v1/gen.py --major 2 from spec/abi-v2/abi.json (CHS_ABI_FINGERPRINT sha256:d60a681ea0e249ab9e2c83b50640ad9cef040261ddf61805106c93e2c68774e4) — DO NOT EDIT  # noqa: E501
+# GENERATED by scripts/abi-v1/gen.py --major 2 from spec/abi-v2/abi.json (CHS_ABI_FINGERPRINT sha256:d53239933966e15a3c363e631a94fbba1231d84c4300337115d3ae5377bb41c2) — DO NOT EDIT  # noqa: E501
 """The Python binding's generated ABI v2 layer: ctypes signatures, the handle
 classes, the two-phase resolve functions and invoke_by_name(), the test-only
 dispatcher the conformance suite drives from tests/fixtures/abi-v2/cases.json.
@@ -19,7 +19,7 @@ from dataclasses import dataclass, field
 from . import _errmap
 
 CHS_ABI_VERSION = 2
-CHS_ABI_FINGERPRINT = "sha256:d60a681ea0e249ab9e2c83b50640ad9cef040261ddf61805106c93e2c68774e4"
+CHS_ABI_FINGERPRINT = "sha256:d53239933966e15a3c363e631a94fbba1231d84c4300337115d3ae5377bb41c2"
 
 # The description's stability: "unstable" while this generation is being
 # designed, so CHS_ABI_FINGERPRINT moves with every change, and "locked" after
@@ -87,6 +87,7 @@ ALL_SYMBOLS: tuple[str, ...] = (
     "chs_error_column",
     "chs_error_free",
     "chs_error_message",
+    "chs_error_reason",
     "chs_error_status",
     "chs_filter_create",
     "chs_filter_eval_block",
@@ -248,6 +249,7 @@ _add("chs_error_ch_code", "scalar", "int32", _P("error", "handle", "chs_error"))
 _add("chs_error_ch_name", "handle", "chs_buf", _P("error", "handle", "chs_error"))
 _add("chs_error_message", "handle", "chs_buf", _P("error", "handle", "chs_error"))
 _add("chs_error_column", "handle", "chs_buf", _P("error", "handle", "chs_error"))
+_add("chs_error_reason", "handle", "chs_buf", _P("error", "handle", "chs_error"))
 _add("chs_error_free", "void", None, _P("error", "handle", "chs_error", nullable=True))
 _add(
     "chs_live_handles",
@@ -371,6 +373,7 @@ _add(
     _P("body", "bytes_in", None),
     _P("settings", "bytes_in", None),
     _P("columns", "bytes_in", None),
+    _P("options", "bytes_in", None),
     _P("out", "out_handle", "chs_buf"),
     _P("err", "out_error", "chs_error", nullable=True),
 )
@@ -386,6 +389,7 @@ _add(
     _P("filter", "handle", "chs_filter", nullable=True),
     _P("export_format", "scalar", "int32"),
     _P("doc_flags", "scalar", "uint32"),
+    _P("options", "bytes_in", None),
     _P("out", "out_handle", "chs_buf"),
     _P("out_export", "out_handle", "chs_buf", nullable=True),
     _P("err", "out_error", "chs_error", nullable=True),
@@ -398,6 +402,7 @@ _add(
     _P("expr", "bytes_in", None),
     _P("query_params", "bytes_in", None),
     _P("settings", "bytes_in", None),
+    _P("options", "bytes_in", None),
     _P("out", "out_handle", "chs_filter"),
     _P("err", "out_error", "chs_error", nullable=True),
 )
@@ -410,6 +415,8 @@ _add(
     _P("format", "enum", "chs_format"),
     _P("body", "bytes_in", None),
     _P("settings", "bytes_in", None),
+    _P("columns", "bytes_in", None),
+    _P("options", "bytes_in", None),
     _P("out", "out_handle", "chs_buf"),
     _P("err", "out_error", "chs_error", nullable=True),
 )
@@ -422,6 +429,7 @@ _add(
     _P("body", "bytes_in", None),
     _P("settings", "bytes_in", None),
     _P("columns", "bytes_in", None),
+    _P("options", "bytes_in", None),
     _P("out", "out_handle", "chs_block"),
     _P("err", "out_error", "chs_error", nullable=True),
 )
@@ -563,6 +571,7 @@ class Api:
             info.ch_name.decode("ascii", "replace") if info else "",
             message,
             info.column if info else b"",
+            reason=info.reason.decode("ascii", "replace") if info else "",
         )
 
     def live_handles(self) -> bytes:
@@ -801,6 +810,7 @@ class Api:
         body: bytes | None,
         settings: bytes | None,
         columns: bytes | None,
+        options: bytes | None,
     ) -> bytes:
         """The generated call wrapper for chs_preview_row (thread class shared).
 
@@ -813,6 +823,7 @@ class Api:
             *_bytes_in(body),
             *_bytes_in(settings),
             *_bytes_in(columns),
+            *_bytes_in(options),
             ctypes.byref(out),
             ctypes.byref(err),
         )
@@ -829,6 +840,7 @@ class Api:
         filter: FilterHandle | None,
         export_format: int,
         doc_flags: int,
+        options: bytes | None,
     ) -> tuple[bytes, bytes | None]:
         """The generated call wrapper for chs_preview_batch (thread class shared).
 
@@ -845,6 +857,7 @@ class Api:
             None if filter is None else filter.value,
             int(export_format),
             int(doc_flags),
+            *_bytes_in(options),
             ctypes.byref(out),
             ctypes.byref(out_export),
             ctypes.byref(err),
@@ -861,6 +874,7 @@ class Api:
         expr: bytes | None,
         query_params: bytes | None,
         settings: bytes | None,
+        options: bytes | None,
     ) -> FilterHandle:
         """The generated call wrapper for chs_filter_create (thread class shared).
 
@@ -872,6 +886,7 @@ class Api:
             *_bytes_in(expr),
             *_bytes_in(query_params),
             *_bytes_in(settings),
+            *_bytes_in(options),
             ctypes.byref(out),
             ctypes.byref(err),
         )
@@ -884,6 +899,8 @@ class Api:
         format: int,
         body: bytes | None,
         settings: bytes | None,
+        columns: bytes | None,
+        options: bytes | None,
     ) -> bytes:
         """The generated call wrapper for chs_filter_eval_body (thread class shared).
 
@@ -895,6 +912,8 @@ class Api:
             int(format),
             *_bytes_in(body),
             *_bytes_in(settings),
+            *_bytes_in(columns),
+            *_bytes_in(options),
             ctypes.byref(out),
             ctypes.byref(err),
         )
@@ -908,6 +927,7 @@ class Api:
         body: bytes | None,
         settings: bytes | None,
         columns: bytes | None,
+        options: bytes | None,
     ) -> BlockHandle:
         """The generated call wrapper for chs_block_create (thread class shared).
 
@@ -920,6 +940,7 @@ class Api:
             *_bytes_in(body),
             *_bytes_in(settings),
             *_bytes_in(columns),
+            *_bytes_in(options),
             ctypes.byref(out),
             ctypes.byref(err),
         )
@@ -1007,6 +1028,7 @@ class ErrorInfo:
     ch_name: bytes
     message: bytes
     column: bytes
+    reason: bytes = b""
 
 
 @dataclass
@@ -1068,6 +1090,7 @@ def _decode_error(api: Api, err_ptr: int) -> ErrorInfo:
     ch_name = _read_buf(api, api._raw["chs_error_ch_name"](err_ptr))
     message = _read_buf(api, api._raw["chs_error_message"](err_ptr))
     column = _read_buf(api, api._raw["chs_error_column"](err_ptr))
+    reason = _read_buf(api, api._raw["chs_error_reason"](err_ptr))
     api._raw["chs_error_free"](err_ptr)
     return ErrorInfo(
         status=STATUS_BY_VALUE.get(status_val, f"UNKNOWN:{status_val}"),
@@ -1075,6 +1098,7 @@ def _decode_error(api: Api, err_ptr: int) -> ErrorInfo:
         ch_name=ch_name,
         message=message,
         column=column,
+        reason=reason,
     )
 
 

{chtypes-2.0.0.dev10 → chtypes-2.0.0.dev11}/src/chtypes/_abi2/_errmap.py

@@ -1,4 +1,4 @@
-# GENERATED by scripts/abi-v1/gen.py --major 2 from spec/abi-v2/abi.json (CHS_ABI_FINGERPRINT sha256:d60a681ea0e249ab9e2c83b50640ad9cef040261ddf61805106c93e2c68774e4) — DO NOT EDIT  # noqa: E501
+# GENERATED by scripts/abi-v1/gen.py --major 2 from spec/abi-v2/abi.json (CHS_ABI_FINGERPRINT sha256:d53239933966e15a3c363e631a94fbba1231d84c4300337115d3ae5377bb41c2) — DO NOT EDIT  # noqa: E501
 """spec/abi-v2/sdk.json's error-mapping tables, as references to the
 public classes (python/src/chtypes/errors.py, re-exported by
 python/src/chtypes/_abi2/_errors.py). Only the MAPPING is generated: the

{chtypes-2.0.0.dev10 → chtypes-2.0.0.dev11}/src/chtypes/_abi2/_vocab.py

@@ -1,4 +1,4 @@
-# GENERATED by scripts/abi-v1/gen.py --major 2 from spec/abi-v2/abi.json (CHS_ABI_FINGERPRINT sha256:d60a681ea0e249ab9e2c83b50640ad9cef040261ddf61805106c93e2c68774e4) — DO NOT EDIT  # noqa: E501
+# GENERATED by scripts/abi-v1/gen.py --major 2 from spec/abi-v2/abi.json (CHS_ABI_FINGERPRINT sha256:d53239933966e15a3c363e631a94fbba1231d84c4300337115d3ae5377bb41c2) — DO NOT EDIT  # noqa: E501
 """The Python binding's generated vocabularies, from spec/abi-v2/abi.json: the C
 enums (`Format`, `Status`), the document vocabularies (`Outcome`, `FilterOutcome`,
 `Verdict`, `DefaultKind`, `MergeReason`, `DeclinedTier`, `DeclinedLayer`, `Reason`,
@@ -256,6 +256,7 @@ class DeclinedTier(StrEnum):
     PREDICATE = "predicate"
     PREDICATE_UNFLIPPED = "predicate-unflipped"
     RESULT_CONTENT = "result-content"
+    SERVER_REFUSES = "server-refuses"
 
     @classmethod
     def _missing_(cls, value: object) -> DeclinedTier | None:
@@ -299,6 +300,83 @@ class DeclinedLayer(StrEnum):
         return cls(text)
 
 
+class Posture(StrEnum):
+    """How a schema's statement was compiled: `create`, as a server's CREATE TABLE,
+    or `load`, as a server loads the stored definition of a table it holds."""
+
+    CREATE = "create"
+    LOAD = "load"
+
+    @classmethod
+    def _missing_(cls, value: object) -> Posture | None:
+        return _unknown_str(cls, value)
+
+    @property
+    def known(self) -> bool:
+        """Whether the description lists this value: False for exactly the unknown(n)
+        member an unlisted value reads as (rule r3)."""
+        return type(self)._value2member_map_.get(self._value_) is self
+
+    @classmethod
+    def of(cls, text: str) -> Posture:
+        """Map a document's spelling. One the description does not list is kept as
+        this vocabulary's unknown(n) member, carrying it (rule r3)."""
+        return cls(text)
+
+
+class RefusalSite(StrEnum):
+    """Where a batch or filter document's refusal sits: `call` (the call, its settings or its
+    column list), `body_read` (the body outside any row: a header, the framing), `row` (one
+    row, `refused_row`) or `body_step` (a whole-body step)."""
+
+    CALL = "call"
+    BODY_READ = "body_read"
+    ROW = "row"
+    BODY_STEP = "body_step"
+
+    @classmethod
+    def _missing_(cls, value: object) -> RefusalSite | None:
+        return _unknown_str(cls, value)
+
+    @property
+    def known(self) -> bool:
+        """Whether the description lists this value: False for exactly the unknown(n)
+        member an unlisted value reads as (rule r3)."""
+        return type(self)._value2member_map_.get(self._value_) is self
+
+    @classmethod
+    def of(cls, text: str) -> RefusalSite:
+        """Map a document's spelling. One the description does not list is kept as
+        this vocabulary's unknown(n) member, carrying it (rule r3)."""
+        return cls(text)
+
+
+class DefaultSupport(StrEnum):
+    """What this build does with a column's DEFAULT, as `schema_description` reports it per
+    column: `computed`, `generated`, `deferred` or `declined` (with a reason)."""
+
+    COMPUTED = "computed"
+    GENERATED = "generated"
+    DEFERRED = "deferred"
+    DECLINED = "declined"
+
+    @classmethod
+    def _missing_(cls, value: object) -> DefaultSupport | None:
+        return _unknown_str(cls, value)
+
+    @property
+    def known(self) -> bool:
+        """Whether the description lists this value: False for exactly the unknown(n)
+        member an unlisted value reads as (rule r3)."""
+        return type(self)._value2member_map_.get(self._value_) is self
+
+    @classmethod
+    def of(cls, text: str) -> DefaultSupport:
+        """Map a document's spelling. One the description does not list is kept as
+        this vocabulary's unknown(n) member, carrying it (rule r3)."""
+        return cls(text)
+
+
 class DiscoverQueryParam(StrEnum):
     """A query parameter of the SQL `discover_query` returns, which the caller binds
     when it runs the query."""
@@ -409,6 +487,8 @@ class Source:
     DEFAULT_EXPR_UNSUPPORTED: Final = "default_expr_unsupported"
     DEFAULT_VOLATILE_UNRESOLVED: Final = "default_volatile_unresolved"
     DEFAULT_PENDING: Final = "default_pending"
+    DEFAULT_DEFERRED: Final = "default_deferred"
+    DEFAULT_OVERRIDE: Final = "default_override"
 
     @staticmethod
     def known(source: str) -> bool:
@@ -417,9 +497,8 @@ class Source:
 
     @staticmethod
     def is_stored(source: str) -> bool:
-        """The description's `is_stored` fact for `source`. The description does not yet
-        say what an unknown(n) source reports (rule r3, "Facts without a fallback");
-        until it does, an unlisted source is not stored."""
+        """The description's `is_stored` fact for `source`; an unknown(n) source
+        reports its fallback's (`skipped`), as rule r3 says."""
         return _SOURCE_IS_STORED.get(source, False)
 
 
@@ -435,6 +514,8 @@ _SOURCE_IS_STORED: Final[dict[str, bool]] = {
     "default_expr_unsupported": False,
     "default_volatile_unresolved": False,
     "default_pending": False,
+    "default_deferred": False,
+    "default_override": True,
 }
 
 
@@ -450,6 +531,10 @@ class DocFlags(IntFlag):
 # `chs_preview_batch`'s `export_format` for no export (`CHS_EXPORT_NONE`).
 EXPORT_NONE: Final = -1
 
+# The code a declined row, outcome or verdict carries (`CHS_CODE_DECLINED`), outside
+# ClickHouse's code table.
+CODE_DECLINED: Final = -2
+
 # Every enum the description defines, by its own name, and the Python type that
 # spells it: rule r3's tests reach every one from here, so a new enum is covered
 # without a hand-kept list.
@@ -463,10 +548,13 @@ DESCRIBED_VOCABULARIES: Final[dict[str, type]] = {
     "batch_outcome": Outcome,
     "filter_outcome": FilterOutcome,
     "filter_verdict": Verdict,
+    "refusal_site": RefusalSite,
     "discover_query_param": DiscoverQueryParam,
     "default_kind": DefaultKind,
+    "default_support": DefaultSupport,
     "declined_tier": DeclinedTier,
     "declined_layer": DeclinedLayer,
+    "posture": Posture,
 }
 
 

{chtypes-2.0.0.dev10 → chtypes-2.0.0.dev11}/src/chtypes/_decode.py

@@ -32,10 +32,13 @@ from ._abi2._vocab import (
     DeclinedLayer,
     DeclinedTier,
     DefaultKind,
+    DefaultSupport,
     FilterOutcome,
     MergeReason,
     Outcome,
+    Posture,
     Reason,
+    RefusalSite,
     Source,
     Verdict,
 )
@@ -291,6 +294,20 @@ def _spans(obj: Mapping[str, Any], key: str, where: str) -> tuple[Span, ...]:
     return _objects(obj, key, where, _span)
 
 
+def _row_spans(obj: Mapping[str, Any], key: str, where: str) -> tuple[Span | None, ...]:
+    """`row_spans`: one entry per row, a span or null for a row not exported."""
+    out: list[Span | None] = []
+    for i, item in enumerate(_list(obj, key, where)):
+        out.append(None if item is None else _span(_obj(item, where, key), f"{where}.{key}[{i}]"))
+    return tuple(out)
+
+
+def _refused_at(obj: Mapping[str, Any], where: str) -> RefusalSite | None:
+    # An unlisted site is its unknown(n): kept, never a failure (rule r3).
+    value = _opt_text(obj, "refused_at", where)
+    return None if value is None else RefusalSite.of(value)
+
+
 def _value(obj: dict[str, Any], where: str) -> Value:
     # An unlisted src is its unknown(n): kept, never a failure (rule r3), and
     # not stored until the description says what one reports. An ABSENT src is
@@ -353,7 +370,7 @@ def _int_list(obj: Mapping[str, Any], key: str, where: str) -> tuple[int, ...] |
 def _at_merge(obj: dict[str, Any], where: str) -> AtMergeEntry:
     # An unlisted reason is its unknown(n): kept, never a failure (rule r3).
     return AtMergeEntry(
-        row=_int(obj, "row", where),
+        part_row=_int(obj, "part_row", where),
         reason=MergeReason.of(_text(obj, "reason", where)),
         column=_bytes_or_none(obj, "column", where),
         stored=_bytes_or_none(obj, "stored", where),
@@ -386,14 +403,17 @@ def decode_row(obj: Mapping[str, Any], where: str = "row") -> RowResult:
         outcome=Outcome.of(_text(obj, "outcome", where)),
         err_code=_int(obj, "code", where),
         err_msg=_bytes(obj, "err", where),
+        reason=_opt_text(obj, "reason", where),
+        refused_at=_refused_at(obj, where),
+        refused_row=_opt_u64(obj, "refused_row", where),
         columns=columns,
         values=tuple(v for v in columns if v.is_stored),
         transformed=_objects(obj, "transformed", where, _transform),
         unknown_fields=_name_list(obj, "unknown_fields", where),
-        unsupported_settings=_name_list(obj, "unsupported_settings", where),
         computed=_objects(obj, "computed", where, _computed),
         verdict=None if verdict is None else Verdict.of(verdict),
         verdict_code=_int(obj, "verdict_code", where),
+        verdict_reason=_opt_text(obj, "verdict_reason", where),
         verdict_err=_bytes(obj, "verdict_err", where),
         partition_id=partition,
         input_span=_opt_span(obj, "input_span", where),
@@ -427,7 +447,10 @@ def decode_batch(raw: bytes, export: bytes | None) -> BatchResult:
     obj = _obj(strict_loads(raw, where), where)
     framing = obj.get("framing")
     engine_rows = None if obj.get("engine_rows") is None else _engine_rows(obj, where)
-    spans = None if obj.get("row_spans") is None else _spans(obj, "row_spans", where)
+    spans = None if obj.get("row_spans") is None else _row_spans(obj, "row_spans", where)
+    export_columns = (
+        None if obj.get("export_columns") is None else _name_list(obj, "export_columns", where)
+    )
     export_declined = _bytes(obj, "export_declined", where)
     if export_declined:
         # A declined export has no payload and no spans, whatever buffer the call
@@ -439,6 +462,9 @@ def decode_batch(raw: bytes, export: bytes | None) -> BatchResult:
         outcome=Outcome.of(_text(obj, "outcome", where)),
         err_code=_int(obj, "code", where),
         err_msg=_bytes(obj, "err", where),
+        reason=_opt_text(obj, "reason", where),
+        refused_at=_refused_at(obj, where),
+        refused_row=_opt_u64(obj, "refused_row", where),
         rows=_objects(obj, "rows", where, decode_row),
         rows_read=_u64(obj, "rows_read", where),
         rows_skipped=_u64(obj, "rows_skipped", where),
@@ -446,7 +472,9 @@ def decode_batch(raw: bytes, export: bytes | None) -> BatchResult:
         engine_rows=engine_rows,
         payload=export,
         spans=spans,
+        export_columns=export_columns,
         export_declined=export_declined,
+        export_declined_reason=_opt_text(obj, "export_declined_reason", where),
         rows_passed=_u64(obj, "rows_passed", where),
         rows_cut=_u64(obj, "rows_cut", where),
         partition_count=_opt_u64(obj, "partition_count", where),
@@ -467,6 +495,7 @@ def _filter_row_error(obj: dict[str, Any], where: str) -> FilterRowError:
     return FilterRowError(
         row=_int(obj, "row", where),
         code=_int(obj, "code", where),
+        reason=_opt_text(obj, "reason", where),
         msg=_bytes(obj, "err", where),
     )
 
@@ -479,6 +508,9 @@ def decode_filter_result(raw: bytes) -> FilterResult:
         outcome=FilterOutcome.of(_text(obj, "outcome", where)),
         err_code=_int(obj, "code", where),
         err_msg=_bytes(obj, "err", where),
+        reason=_opt_text(obj, "reason", where),
+        refused_at=_refused_at(obj, where),
+        refused_row=_opt_u64(obj, "refused_row", where),
         rows_read=_u64(obj, "rows_read", where),
         unsupported_settings=_name_list(obj, "unsupported_settings", where),
         verdicts=tuple(Verdict.of(c) for c in verdicts),
@@ -487,12 +519,17 @@ def decode_filter_result(raw: bytes) -> FilterResult:
 
 
 def _column(obj: dict[str, Any], where: str) -> Column:
-    # An unlisted default_kind is its unknown(n): kept, never a failure (rule r3).
+    # An unlisted default_kind or default_support is its unknown(n): kept, never a
+    # failure (rule r3).
+    support = _opt_text(obj, "default_support", where)
     return Column(
         name=_name(obj, "name", where),
         type=_bytes(obj, "type", where),
         default_kind=DefaultKind.of(_text(obj, "default_kind", where)),
         default_expr=_bytes(obj, "default_expression", where),
+        default_support=None if support is None else DefaultSupport.of(support),
+        default_support_reason=_opt_text(obj, "default_support_reason", where),
+        reads=None if obj.get("reads") is None else _name_list(obj, "reads", where),
     )
 
 
@@ -506,6 +543,14 @@ def _string_map(obj: Mapping[str, Any], key: str, where: str) -> dict[str, str]
     inner = _obj(value, where, key)
     out: dict[str, str] = {}
     for name, item in inner.items():
+        if isinstance(item, dict):
+            # The object form of a value that was not UTF-8: its bytes, spelled
+            # with surrogateescape (the spelling the input side takes back).
+            if set(item) != {"b64"}:
+                raise _wrong(f"{where}.{key}", name, "a string or a b64 object", item)
+            item = _decode_b64(item["b64"], f"{where}.{key}", name).decode(
+                "utf-8", "surrogateescape"
+            )
         if not isinstance(item, str):
             raise _wrong(f"{where}.{key}", name, "a string", item)
         out[name] = item
@@ -543,6 +588,7 @@ def decode_schema_description(raw: bytes) -> SchemaDescription:
     # decodes exactly as it did before servers existed.
     server = obj.get("server")
     replicated = obj.get("replicated")
+    posture = _opt_text(obj, "posture", where)
     return SchemaDescription(
         columns=_objects(obj, "columns", where, _column),
         server=None
@@ -557,6 +603,7 @@ def decode_schema_description(raw: bytes) -> SchemaDescription:
         filter_declined_settings=_objects(
             obj, "filter_declined_settings", where, _declined_setting
         ),
+        posture=None if posture is None else Posture.of(posture),
     )