chtypes 0.2.0__tar.gz → 0.2.2__tar.gz

{chtypes-0.2.0 → chtypes-0.2.2}/CHANGELOG.md

@@ -4,6 +4,26 @@ All notable changes to the python binding. The format is [Keep a Changelog](http
 
 The four bindings in this repository are released together and give one answer, so an entry here has a counterpart in the other three.
 
+## [0.2.2] — 2026-09-17
+
+### Changed
+
+- **`Registry(..., verify_hashes=True)` now refuses an artifact whose manifest carries no `library_sha256`, instead of loading it silently.** This is a behavior change in a security posture, not a silent fix: previously, asking for verification against a manifest with the field absent returned as if the bytes had been checked, when they never were — the caller believed the load was verified and it was not. `verify_library` now raises `RegistryError` naming the path and saying the manifest carries no `library_sha256`, the same shape its hash-mismatch path has always raised. `verify_hashes` off is unaffected: the field stays optional for a caller who did not ask. Go, TypeScript and Rust already refused this case; this was the one gap issue #13's item A2 left (#48).
+
+### Notes
+
+- Speaks **ABI revision 4**, unchanged since 0.1.0, so no artifact needs relinking. The golden set this release was tested against is the one core serves, generated on the ClickHouse lines 24.8, 25.3, 25.8, 25.10, 26.2, 26.3, 26.4, 26.5, 26.6, 26.7 and 26.8.
+- Seven facts about the compiled library's behavior that a consumer previously had to discover by experiment are now written down: `CHECK`-constraint batch rejection, binding filter parameters as `String`, single-line compact JSON array loss under `allow_errors`, the JSONCompactEachRow export field separator, the value-injection pattern and its compile-time wrap trap, `Columns` as canonical and in declaration order, and the frozen-signatures promise reworded ahead of the next ABI revision (#56). That promise is now stated once, in `docs/support.md`, and linked from everywhere else rather than restated in nine places (#69).
+
+## [0.2.1] — 2026-09-15
+
+Released in step with the Rust crate, which gains `Registry::open` and `Error::LibraryRead`; this binding's public API is unchanged.
+
+### Notes
+
+- Speaks **ABI revision 4**, unchanged since 0.1.0, so no artifact needs relinking. The golden set this release was tested against is the one core serves, generated on the ClickHouse lines 24.8, 25.3, 25.8, 25.10, 26.2, 26.3, 26.4, 26.5, 26.6, 26.7 and 26.8.
+- **The ABI-revision refusal is now run, not assumed.** This binding's suite points a registry at a generated artifact answering the wrong revision and asserts the load is refused naming both numbers, beside a matching-revision control that must load; CI fails the build if either case did not run (#36).
+
 ## [0.2.0] — 2026-09-15
 
 ### Changed — BREAKING

{chtypes-0.2.0 → chtypes-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: chtypes
-Version: 0.2.0
+Version: 0.2.2
 Summary: Python binding for the chtypes artifacts: ClickHouse's own type system, per version, behind the frozen chs_* C ABI
 Project-URL: Homepage, https://github.com/wave-rf/chtypes
 Project-URL: Repository, https://github.com/wave-rf/chtypes

{chtypes-0.2.0 → chtypes-0.2.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "chtypes"
-version = "0.2.0"
+version = "0.2.2"
 description = "Python binding for the chtypes artifacts: ClickHouse's own type system, per version, behind the frozen chs_* C ABI"
 readme = "README.md"
 license = "Apache-2.0"

{chtypes-0.2.0 → chtypes-0.2.2}/src/chtypes/_manifest.py

@@ -106,7 +106,9 @@ def verify_library(version_dir: str | os.PathLike[str]) -> None:
             f"chtypes: {path} is {size} bytes, manifest says {manifest.library_bytes}"
         )
     if not manifest.library_sha256:
-        return
+        raise RegistryError(
+            f"chtypes: {path} cannot be verified: manifest carries no library_sha256"
+        )
     digest = hashlib.sha256()
     with path.open("rb") as handle:
         for chunk in iter(lambda: handle.read(1 << 20), b""):

chtypes-0.2.2/tests/test_abi_revision.py

@@ -0,0 +1,139 @@
+"""The ABI-revision handshake, run against a wrong-revision fixture (#36).
+
+`docs/reference/artifact.md` step 5 is normative: an artifact whose
+`chs_abi_revision()` answers a value that is neither 0 nor the revision this
+binding was written against MUST be refused, **naming both numbers**
+(`chtypes/_native.py`, `NativeLibrary.__init__`: "refusing to call through
+mismatched declarations"). This suite is the first thing in this repository
+that actually RUNS that rule instead of assuming it.
+
+The fixture is a pair of stub shared libraries generated from `include/chtypes.h`
+itself, by the served ABI-revision fixture generator (`abi-revision/gen.py`):
+one registry whose artifact answers `abi_revision + 1` (`wrong-revision/`), one
+answering `abi_revision`
+exactly (`at-revision/`), both on ClickHouse minor line "0.0". **Both halves
+matter** — without the `at-revision` control, a binding whose own
+`ABI_REVISION` had drifted would refuse everything and pass the refusal case
+for entirely the wrong reason.
+
+Every expected number comes from `fixture.json` — never from
+`chtypes.ABI_REVISION` — because a case that read the constant under test
+could not catch that constant being wrong.
+
+`isolated_search_path` (`tests/conftest.py`) is used to keep the per-user
+artifact cache and any ambient `$CHTYPES_REGISTRY`/system roots out of the
+search path: `Registry(root)` puts `root` first regardless, but this removes
+any chance that the fixture's odd "0.0" minor line is ever resolved from
+somewhere other than the fixture itself.
+
+Reopening a library after `close()` in the same process is measured to
+segfault (`docs/reference/bindings.md` §Teardown), so nothing here closes a
+loaded library: the refusal case never gets far enough to open one, the
+control case leaves its `Library` for process teardown (`chs_shutdown` runs
+via `atexit`), and neither path is ever loaded a second time.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import re
+from pathlib import Path
+from typing import Final
+
+import pytest
+
+import chtypes
+
+#: Where the fixture root lives. Built by the served abi-revision generator
+#: (`abi-revision/gen.py build`, core repo); this suite never builds one itself.
+ENV_ABI_FIXTURES: Final = "CHTYPES_ABI_FIXTURES"
+
+#: The one ClickHouse minor line every fixture artifact answers to.
+FIXTURE_MINOR: Final = "0.0"
+
+
+@pytest.fixture(scope="session")
+def abi_fixture() -> dict:
+    """Parse and validate `fixture.json`. Missing `$CHTYPES_ABI_FIXTURES` is a
+    SKIP (no fallback build); a fixture that exists but is malformed is a
+    FAILURE naming the path — a bad fixture must never read as "nothing to
+    test here"."""
+    env = os.environ.get(ENV_ABI_FIXTURES)
+    if not env:
+        pytest.skip(
+            f"no ABI revision fixture: ${ENV_ABI_FIXTURES} is unset (build one with "
+            "abi-revision/gen.py build --out DIR --header include/chtypes.h)"
+        )
+    root = Path(env)
+    fixture_path = root / "fixture.json"
+    try:
+        raw = fixture_path.read_text()
+    except OSError as exc:
+        pytest.fail(f"{fixture_path}: cannot read the ABI revision fixture: {exc}")
+    try:
+        doc = json.loads(raw)
+    except json.JSONDecodeError as exc:
+        pytest.fail(f"{fixture_path}: not valid JSON: {exc}")
+    abi_revision = doc.get("abi_revision")
+    valid = isinstance(abi_revision, int) and not isinstance(abi_revision, bool)
+    if not valid or abi_revision <= 0:
+        pytest.fail(
+            f"{fixture_path}: abi_revision must be a positive integer, got {abi_revision!r}"
+        )
+    mismatch_revision = doc.get("mismatch_revision")
+    if mismatch_revision != abi_revision + 1:
+        pytest.fail(
+            f"{fixture_path}: mismatch_revision is {mismatch_revision!r}, expected "
+            f"abi_revision ({abi_revision}) + 1"
+        )
+    doc["root"] = str(root)
+    return doc
+
+
+def _names_number(message: str, n: int) -> bool:
+    """ "The message says N", not "the message contains the digits of N": a
+    refusal naming revision 4 must not be satisfied by the 4 in a path like
+    `.../24.8/`, and one naming 5 must not be satisfied by 15."""
+    return re.search(rf"(^|[^0-9]){n}([^0-9]|$)", message) is not None
+
+
+def _without_fixture_path(message: str, root: Path) -> str:
+    """Strip every occurrence of the fixture root from a message before
+    `_names_number` checks it. The refusal message embeds the artifact's
+    PATH, and a path segment can itself hold a standalone digit (a scratchpad
+    session id such as `4d9fd784-...`), which would let the path alone
+    satisfy "the message names 4" for reasons that have nothing to do with
+    the refusal text itself. Both the given root and its resolved form are
+    stripped (macOS's `/tmp` vs `/private/tmp` differ), longest first so one
+    is never a partial match of the other.
+    """
+    candidates = sorted({str(root), os.path.realpath(root)}, key=len, reverse=True)
+    for candidate in candidates:
+        message = message.replace(candidate, "<fixture>")
+    return message
+
+
+def test_wrong_revision_is_refused(
+    abi_fixture: dict,
+    isolated_search_path: Path,
+) -> None:
+    root = Path(abi_fixture["root"], "wrong-revision")
+    registry = chtypes.Registry(root)
+    with pytest.raises(chtypes.RegistryError) as excinfo:
+        registry.for_version(FIXTURE_MINOR)
+    message = str(excinfo.value)
+    stripped = _without_fixture_path(message, root)
+    for n in (abi_fixture["mismatch_revision"], abi_fixture["abi_revision"]):
+        assert _names_number(stripped, n), f"the refusal does not name {n}: {message!r}"
+
+
+def test_matching_revision_loads(
+    abi_fixture: dict,
+    isolated_search_path: Path,
+) -> None:
+    root = Path(abi_fixture["root"], "at-revision")
+    registry = chtypes.Registry(root)
+    library = registry.for_version(FIXTURE_MINOR)
+    assert library.abi_revision == abi_fixture["abi_revision"]
+    assert library.version == abi_fixture["clickhouse_version"]

{chtypes-0.2.0 → chtypes-0.2.2}/tests/test_registry.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+import dataclasses
 import hashlib
 import json
 import os
@@ -221,6 +222,48 @@ def test_verify_library_checks_the_bytes(tmp_path: Path) -> None:
         chtypes.verify_library(tmp_path)
 
 
+def test_verify_library_refuses_a_manifest_with_no_hash(tmp_path: Path) -> None:
+    """Verification asked for and not possible is not verification (#48): Go,
+    TypeScript and Rust all refuse an artifact whose manifest carries no
+    `library_sha256` when checksums are asked for. `verify_library` runs only
+    when a caller asked for it (`Registry(verify_hashes=True)`), so a manifest
+    missing the field must raise here too, not return quietly.
+    """
+    payload = b"not really a library"
+    (tmp_path / "fake.so").write_bytes(payload)
+    (tmp_path / "manifest.json").write_text(
+        json.dumps({"library": "fake.so", "library_bytes": len(payload)})
+    )
+    with pytest.raises(chtypes.RegistryError, match="library_sha256"):
+        chtypes.verify_library(tmp_path)
+
+
+def test_registry_verify_hashes_refuses_a_manifest_with_no_hash(
+    registry: chtypes.Registry, tmp_path: Path
+) -> None:
+    """`Registry(..., verify_hashes=True)` must not load bytes it never
+    checked (#48): a manifest with the field simply absent is exactly the
+    shape a truncated or hand-assembled artifact directory has. With
+    `verify_hashes` off the field stays optional for a caller who did not
+    ask, and the same directory loads normally.
+    """
+    smallest = min(registry.libraries(), key=lambda lib: lib.manifest.library_bytes or 1 << 62)
+    fields = dataclasses.asdict(smallest.manifest)
+    del fields["library_sha256"]
+    line = tmp_path / smallest.minor
+    line.mkdir()
+    (line / smallest.manifest.library).symlink_to(Path(smallest.path).resolve())
+    (line / "manifest.json").write_text(json.dumps(fields))
+
+    strict = chtypes.Registry(tmp_path, verify_hashes=True)
+    with pytest.raises(chtypes.RegistryError, match="library_sha256"):
+        strict.libraries()
+
+    lenient = chtypes.Registry(tmp_path, verify_hashes=False)
+    loaded = lenient.for_version(smallest.minor)  # must NOT raise
+    assert loaded.version == smallest.version
+
+
 def test_verify_a_real_artifact(registry: chtypes.Registry) -> None:
     # The one integrity check that means anything, run against the real bytes: a
     # move that reported success and truncated a 232 MB library looks identical

{chtypes-0.2.0 → chtypes-0.2.2}/uv.lock

@@ -4,7 +4,7 @@ requires-python = ">=3.11"
 
 [[package]]
 name = "chtypes"
-version = "0.2.0"
+version = "0.2.2"
 source = { editable = "." }
 
 [package.dev-dependencies]