chtypes 2.0.0.dev11__tar.gz → 2.0.0.dev13__tar.gz

{chtypes-2.0.0.dev11 → chtypes-2.0.0.dev13}/CHANGELOG.md

@@ -8,7 +8,9 @@ The four bindings in this repository are released together and give one answer,
 
 ### Added
 
+- Tests: the shared revocation-statement table (`tests/fixtures/revocation-statement/cases.json`) runs against `verify_revocation` and `check_installed`: `superseded_by` must be a string (`""` = no fix), and the first verifying revocation in a listing wins. No behavior changed.
 - `column_defaults` is covered by unit tests (ABI v2): the exact `insert_options` JSON for a UTF-8 value, a non-UTF-8 value (`value_b64`), an empty value (sent as `""`) and two entries in order, and the decode of a `default_override` cell (`Source.DEFAULT_OVERRIDE`, stored). The batches guide gains a usage section. No behavior changed.
+- Docs: `docs/guides/batches.md` shows how a `column_defaults` value is read, in the body format's own field encoding (CSV quoting, TSV escapes, a JSON string or number), with a value that doesn't read refused at the call before any body byte.
 - `CODE_DECLINED` (ABI v2, -2): the code a declined row, outcome or verdict carries, generated from the description and exported from `chtypes`, as `bindings-v1.md` section 3 promises.
 - `CallError.reason` (ABI v2): the error's catalog id, read through `chs_error_reason` and passed through verbatim (ASCII, empty if none), on all four call classes.
 - Spec batch 2, the options documents (ABI v2; moves the dev fingerprint): keyword-only `insertable_columns`, `column_defaults` (a sequence of `ColumnDefault`), `verdict_basis` (`VerdictBasis`: `INPUT`, `EXPORT`) and `defer_defaults` (`DeferDefaults`: `NONE`, `SERVER`) build `insert_options` for `Schema.row`, `Schema.rows`, `Schema.parse_block` and `Filter.rows`; `param_encoding` (`ParamEncoding`: `ESCAPED`, `RAW`) builds `filter_options` for `Schema.compile_filter`; `posture` (`Posture`: `CREATE`, `LOAD`) builds `schema_options` for `Library.compile_table`. `insert_options_extra`, `filter_options_extra` and `schema_options_extra` pass a member the binding has no name for through as JSON text, so the library, which refuses an unknown key (rule r1), is the only judge. A document with no member set is not sent. `Filter.rows` takes `columns` (`chs_filter_eval_body` takes the column list).
@@ -34,7 +36,8 @@ The four bindings in this repository are released together and give one answer,
 
 ### Changed
 
-- **Breaking (ABI v2 dev): `BatchResult.spans` is a tuple of `Span | None`**, one entry per `rows` entry and index-aligned with it: `row_spans` carries `null` for a row the export did not carry (a skipped row, or a row a filter cut), which reads as `None`. It was the exported rows' spans only.
+- **Breaking (vs the 2.0.0-dev releases):** the batch preview's default `doc_flags` is now 0 (the lean result): `Schema.rows` without `doc_flags=` returns no per-value documents. Pass `doc_flags=DocFlags.ALL` explicitly to get every column's per-value documents.
+- **Breaking (ABI v2 dev): `BatchResult.spans` is a tuple of `Span | None`**, one entry per `rows` entry and index-aligned with it: `row_spans` carries `null` for a row the export did not carry (a skipped row, or a row a filter cut), which reads as `None`. It was the exported rows' spans only. Code that reads `span.off` from every entry raises `AttributeError` on a `None` entry: check each entry before reading it.
 - **Breaking (ABI v2 dev): `AtMergeEntry.row` is `AtMergeEntry.part_row`**, the document's `part_row`.
 - A document's `code` can be -2 (`CHS_CODE_DECLINED`) for a row, batch, filter result or verdict this build declined, and every decline names its `reason`.
 - The refusal of a settings, query-parameter, default or server-settings value that is not valid UTF-8 is gone (it is the `{"b64": ...}` form now); a key that is not valid UTF-8, a macro and a server timezone are still refused, as a `UsageError` naming the input.
@@ -79,6 +82,7 @@ The four bindings in this repository are released together and give one answer,
 - **Enforcement gate lifted** for `26.8` on `darwin-arm64`.
 - **Enforcement gate lifted** for `26.9` on `darwin-arm64`.
 - `docs/limitations.md` is re-checked against artifact build `20261012.142240` (v2-dev.13), the library half measured on that build: functions that need a server, in a column expression, and a body the server may read serially or in parallel under a non-zero error budget are listed under "Declined by design"; the known divergence for a column expression that calls a function needing a server is retired, since that build declines those schemas (on 26.8 and 26.9 the `AI*` and `ai*` functions compile, as the page says); and the `lossy` raw-NUL TSV gap is marked fixed on that build and still present in the 1.0 libraries. `docs/guides/filters.md` gains a section on subcolumn filters and a JSON value typed by its batch.
+- The ABI v2 spec now states the INSERT-time settings layering the library already uses, since artifact build `20261021.025722` (#632): at the INSERT, the schema's own settings (the compile profile) reach only the parse, minus a type gate, and every expression the INSERT evaluates and its limits (`max_partitions_per_insert_block` among them) run under the call's own settings, the server profile and the setup defaults only. A setting your INSERT session carries for those goes in the call's settings or the server profile (`docs/guides/settings.md`). No API change.
 
 ### Removed
 
@@ -87,6 +91,9 @@ The four bindings in this repository are released together and give one answer,
 
 ### Fixed
 
+- Installing a pre-seeded cache entry from local blobs now checks the layer blob against the manifest's layer descriptor (size and sha256) before it decompresses it, and the manifest blob against its digest and its `index.json` descriptor's size before it parses it, as a downloaded blob already was and as the other three bindings already did. The install had checked only that the signed statement names the layer's digest, then unpacked the file stored under that name without hashing it, so a file replaced under a signed layer's name, holding the same library bytes plus other members, was installed with those members. Such an entry is now not installed, and the lookup is a miss. The installed library itself was always re-hashed against the signed predicate.
+- An offline lookup no longer reads every cached blob once per pre-seeded entry (#690). For each `index.json` entry with no record, the lookup read every blob of the layout, layers included, to find its signature, so entries it could not install (a signature not present, or one no trusted key verifies) cost entries × cache bytes on every lookup: about 4.9 s for 27 such entries over 1.27 GB (`measured`, darwin-arm64). The signature referrers are now read once per lookup, and never a layer, because a blob larger than `limits.manifest_bytes` cannot be a manifest. Installed entries were already skipped. The build that is installed is verified exactly as before: its manifest against its descriptor, its whole layer against its digest, and its signature, platform, request and fingerprint. A unit test counts the blob reads of one lookup that installs nothing and holds them to at most two per blob and five per entry, with no layer read, at 2 and at 20 entries alike.
+- Docs: `docs/guides/settings.md` names the ABI v2 admission time budget `chtypes_default_eval_cpu_nanos`, the evaluating thread's CPU time. Its examples used the 1.x name `chtypes_default_eval_wall_nanos`, which v2 refuses with `CHS_INVALID_ARGUMENT`.
 - A response body cut off part way (shorter than its Content-Length, a reset, a TLS close) is retried like a 5xx and, once the retry table is spent, is `CHTYPES_SOURCE_UNREACHABLE`; only an oversize body is `CHTYPES_ARTIFACT_CORRUPT`, and it is never retried (#641). Python took a body that ended before its Content-Length as the whole body and failed its digest check as corrupt without a retry; now a short body, or `http.client.IncompleteRead`, is retried and ends as unreachable. The rule is `docs/guides/fetch-v1.md` section 7, and the new `body-cut-off-then-ok` and `body-cut-off-exhausted` conformance cases hold every binding to it.
 - A chunked response body cut off before its terminating zero-length chunk is held to the same rule (#644): the new `body-cut-off-chunked-then-ok` and `body-cut-off-chunked-exhausted` conformance cases cut one on the layer blob GET, and this binding already retried it and ended as `CHTYPES_SOURCE_UNREACHABLE` once the retry table was spent. No behavior change; the case now holds it.
 - A settings or query-parameter key or value that is not valid UTF-8 is refused as a usage error; it was silently replaced with U+FFFD, so the library saw a different value. A string holding a lone surrogate (what `surrogateescape` makes of undecodable bytes) was sent as a `\udXXX` escape; the same inputs are refused now, for the per-call zone, the setup defaults and the server profile too. Valid UTF-8 serializes as before.

{chtypes-2.0.0.dev11 → chtypes-2.0.0.dev13}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: chtypes
-Version: 2.0.0.dev11
+Version: 2.0.0.dev13
 Summary: Python binding for the chtypes artifacts: ClickHouse's own type system, per version, behind the frozen chs_* C ABI
 Project-URL: Homepage, https://github.com/wave-rf/chtypes
 Project-URL: Repository, https://github.com/wave-rf/chtypes
@@ -43,7 +43,7 @@ uv add "chtypes>=2.0.0.dev0"            # or: pip install "chtypes>=2.0.0.dev0"
 
 ```python
 import chtypes
-from chtypes import Format, Registry
+from chtypes import DocFlags, Format, Registry
 
 chtypes.setup(timezone="UTC")  # once, before the first open (optional)
 registry = Registry(autofetch=True)
@@ -51,7 +51,7 @@ library = registry.for_version("26.8")  # a release line or an exact version
 
 ddl = b"CREATE TABLE t (x UInt8, ts DateTime DEFAULT now()) ENGINE = Memory"
 with library.compile_table(ddl) as schema:
-    batch = schema.rows(Format.JSON_EACH_ROW, b'{"x":256}\n')
+    batch = schema.rows(Format.JSON_EACH_ROW, b'{"x":256}\n', doc_flags=DocFlags.ALL)
 
 row = batch.rows[0]
 print(batch.outcome)  # accepted

{chtypes-2.0.0.dev11 → chtypes-2.0.0.dev13}/README.md

@@ -24,7 +24,7 @@ uv add "chtypes>=2.0.0.dev0"            # or: pip install "chtypes>=2.0.0.dev0"
 
 ```python
 import chtypes
-from chtypes import Format, Registry
+from chtypes import DocFlags, Format, Registry
 
 chtypes.setup(timezone="UTC")  # once, before the first open (optional)
 registry = Registry(autofetch=True)
@@ -32,7 +32,7 @@ library = registry.for_version("26.8")  # a release line or an exact version
 
 ddl = b"CREATE TABLE t (x UInt8, ts DateTime DEFAULT now()) ENGINE = Memory"
 with library.compile_table(ddl) as schema:
-    batch = schema.rows(Format.JSON_EACH_ROW, b'{"x":256}\n')
+    batch = schema.rows(Format.JSON_EACH_ROW, b'{"x":256}\n', doc_flags=DocFlags.ALL)
 
 row = batch.rows[0]
 print(batch.outcome)  # accepted

{chtypes-2.0.0.dev11 → chtypes-2.0.0.dev13}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "chtypes"
-version = "2.0.0.dev11"
+version = "2.0.0.dev13"
 description = "Python binding for the chtypes artifacts: ClickHouse's own type system, per version, behind the frozen chs_* C ABI"
 readme = "README.md"
 license = "Apache-2.0"

{chtypes-2.0.0.dev11 → chtypes-2.0.0.dev13}/src/chtypes/__init__.py

@@ -13,14 +13,15 @@ C ABI. Nothing here reimplements a coercion rule, which is why the answers are
 exact by construction rather than approximately right.
 
     import chtypes
-    from chtypes import Format, Registry
+    from chtypes import DocFlags, Format, Registry
 
     chtypes.setup(timezone="UTC")                # once, before the first open
     registry = Registry(autofetch=True)
     library = registry.for_version("26.8")       # a release line or an exact version
     ddl = b"CREATE TABLE t (ts DateTime, seq UInt8) ENGINE = Memory"
     with library.compile_table(ddl) as schema:
-        batch = schema.rows(Format.JSON_EACH_ROW, b'{"ts":"2026-01-15 10:30:00","seq":256}\\n')
+        body = b'{"ts":"2026-01-15 10:30:00","seq":256}\\n'
+        batch = schema.rows(Format.JSON_EACH_ROW, body, doc_flags=DocFlags.ALL)
 
     batch.outcome        # Outcome.ACCEPTED: the insert would succeed
     batch.transformed    # the changes ClickHouse makes and reports as success

{chtypes-2.0.0.dev11 → chtypes-2.0.0.dev13}/src/chtypes/_ocifetch/_ensure.py

@@ -75,6 +75,7 @@ from chtypes._ocifetch._layout import (
 from chtypes._ocifetch._lock import Lock, LockPin, load_lock, new_lock, save_lock
 from chtypes._ocifetch._oci import (
     Descriptor,
+    copy_blob_verified,
     fetch_blob_bytes,
     fetch_blob_to_path,
     fetch_manifest_by_digest,
@@ -86,6 +87,8 @@ from chtypes._ocifetch._oci import (
     resolve_platform_manifest,
     spelling_components,
     validate_spelling,
+    verify_body_matches_digest,
+    verify_json_descriptor,
     version_within_request,
 )
 from chtypes._ocifetch._referrers import ReferrerListing, discover_referrers, read_listing
@@ -1296,6 +1299,48 @@ def _ensure(
         raise _translate_transport_error(e) from e
 
 
+def _local_bundle_referrers(
+    blobs_dir: Path, skip: Collection[str], known: dict[str, bytes] | None = None
+) -> dict[str, list[tuple[str, dict]]]:
+    """Each subject manifest digest's signature referrer manifests among
+    `blobs_dir`'s blobs, as (blob name, parsed manifest), in listing order.
+    A referrer need not be listed in `index.json`, so every blob that could
+    be one is read, once per lookup rather than once per entry. A blob named
+    in `skip` (the caller's candidate layers) and a blob larger than
+    `limits.manifest_bytes`, which cannot be a manifest, are never read, so
+    no layer is read here, and a blob in `known` (bytes the caller already
+    read, by name) is not read again (public issue #690)."""
+    out: dict[str, list[tuple[str, dict]]] = {}
+    try:
+        blob_paths = list(blobs_dir.iterdir())
+    except OSError:
+        return out
+    for blob_path in blob_paths:
+        if blob_path.name in skip:
+            continue
+        try:
+            body = known.get(blob_path.name) if known is not None else None
+            if body is None:
+                if blob_path.stat().st_size > C.MANIFEST_MAX_BYTES:
+                    continue
+                body = blob_path.read_bytes()
+            doc = json.loads(body)
+        except (ValueError, OSError, UnicodeDecodeError):
+            continue
+        if not isinstance(doc, dict) or doc.get("mediaType") != C.MEDIA_TYPE_MANIFEST:
+            continue
+        subject = doc.get("subject") or {}
+        subject_digest = subject.get("digest") if isinstance(subject, dict) else None
+        if not isinstance(subject_digest, str):
+            continue
+        # A signature referrer only: a revocation of the manifest carries the same
+        # bundle media type, and is not the signature of the artifact.
+        if doc.get("artifactType") != C.MEDIA_TYPE_BUNDLE:
+            continue
+        out.setdefault(subject_digest, []).append((blob_path.name, doc))
+    return out
+
+
 def _verify_and_install_from_local_blobs(
     *,
     source_root: Path,
@@ -1303,6 +1348,8 @@ def _verify_and_install_from_local_blobs(
     cache_write_root: Path,
     trusted_keys: tuple[TrustedKey, ...],
     visible_only: bool = False,
+    referrers: dict[str, list[tuple[str, dict]]] | None = None,
+    manifest_size: int | None = None,
 ) -> VerifiedRecord | None:
     """docs/guides/fetch-v1.md §1/§6: "A pre-seeded layout has entries in
     `index.json` with no corresponding `unpacked/` directory yet; the first
@@ -1314,7 +1361,14 @@ def _verify_and_install_from_local_blobs(
     `source_root/blobs/sha256/*`, never the network. Returns `None` — never
     raises — for a missing, unreadable or untrusted entry, the same
     tolerant shape `_find_verified_signature` uses for a network referrer:
-    a bad pre-seed is "nothing usable here," not a crash."""
+    a bad pre-seed is "nothing usable here," not a crash.
+
+    The manifest blob is checked against `manifest_digest` (and against
+    `manifest_size`, the index descriptor's size, when the caller has one)
+    before it is parsed, and the layer blob against the manifest's layer
+    descriptor (size and sha256) before it is decompressed: the same
+    checks a download gets (§7.3), so no byte the signature does not cover
+    reaches the install."""
     blobs_dir = source_root / "blobs" / "sha256"
     if not blobs_dir.is_dir():
         return None
@@ -1323,28 +1377,25 @@ def _verify_and_install_from_local_blobs(
     if not manifest_path.is_file():
         return None
     try:
-        manifest_doc = json.loads(manifest_path.read_bytes())
+        manifest_bytes = manifest_path.read_bytes()
+        verify_body_matches_digest(manifest_bytes, manifest_digest, what="pre-seeded manifest")
+        if manifest_size is not None:
+            verify_json_descriptor(
+                manifest_bytes,
+                Descriptor(media_type="", digest=manifest_digest, size=manifest_size),
+                what="pre-seeded manifest",
+            )
+        manifest_doc = json.loads(manifest_bytes)
         layer_desc = manifest_layer_descriptor(manifest_doc)
     except (ValueError, OSError, FetchError):
         return None
 
+    if referrers is None:
+        referrers = _local_bundle_referrers(blobs_dir, {parse_digest(layer_desc.digest)})
     verified = None
     bundle_digest = None
     bundle_manifest_digest = None
-    for blob_path in blobs_dir.iterdir():
-        try:
-            doc = json.loads(blob_path.read_bytes())
-        except (ValueError, OSError, UnicodeDecodeError):
-            continue
-        if not isinstance(doc, dict) or doc.get("mediaType") != C.MEDIA_TYPE_MANIFEST:
-            continue
-        subject = doc.get("subject") or {}
-        if subject.get("digest") != manifest_digest:
-            continue
-        # A signature referrer only: a revocation of the manifest carries the same
-        # bundle media type, and is not the signature of the artifact.
-        if doc.get("artifactType") != C.MEDIA_TYPE_BUNDLE:
-            continue
+    for referrer_name, doc in referrers.get(manifest_digest, ()):
         try:
             referrer_layer = manifest_single_layer(doc, expected_media_type=C.MEDIA_TYPE_BUNDLE)
             bundle_bytes = (blobs_dir / parse_digest(referrer_layer.digest)).read_bytes()
@@ -1354,7 +1405,7 @@ def _verify_and_install_from_local_blobs(
         if candidate is not None:
             verified = candidate
             bundle_digest = referrer_layer.digest
-            bundle_manifest_digest = "sha256:" + blob_path.name
+            bundle_manifest_digest = "sha256:" + referrer_name
             break
     if verified is None:
         return None
@@ -1383,9 +1434,18 @@ def _verify_and_install_from_local_blobs(
     scratch = _scratch_root(cache_write_root)
     tmp_dir = tempfile.mkdtemp(dir=str(scratch))
     try:
+        # Size and sha256 against the layer descriptor BEFORE decompressing
+        # (§7.3), on a copy in scratch, which is then the file unpacked.
+        verified_layer = os.path.join(tmp_dir, "layer")
+        copy_blob_verified(
+            str(layer_path),
+            layer_desc,
+            verified_layer,
+            what=f"pre-seeded layer {layer_desc.digest}",
+        )
         unpacked_dir = os.path.join(tmp_dir, "unpacked")
         unpack_tar_zst(
-            str(layer_path),
+            verified_layer,
             unpacked_dir,
             library_name=predicate["library"],
             library_sha256=predicate["library_sha256"],
@@ -1422,7 +1482,9 @@ def _verify_preseeded_entries(roots: Sequence[Path], trusted_keys: tuple[Trusted
     anywhere yet, and verifies+unpacks it from that root's OWN local blobs,
     writing the result into `roots[0]` (the writable cache) regardless of
     which root — including a read-only system directory — the blobs came
-    from."""
+    from. An entry that already has a record is passed over without reading
+    a blob, and each root's signature referrers are read once, never by
+    reading a candidate's layer (public issue #690)."""
     cache_write_root = roots[0]
     already_verified = {record.manifest for _dir, record in list_verified_records(roots)}
     for root in roots:
@@ -1431,16 +1493,41 @@ def _verify_preseeded_entries(roots: Sequence[Path], trusted_keys: tuple[Trusted
             index_doc = json.loads(index_path.read_text(encoding="utf-8"))
         except (OSError, ValueError):
             continue
+        blobs_dir = root / "blobs" / "sha256"
+        candidates: list[str] = []
+        candidate_layers: set[str] = set()
+        manifests_read: dict[str, bytes] = {}
+        sizes: dict[str, int] = {}
         for m in index_doc.get("manifests") or []:
             digest = m.get("digest")
             if not digest or digest in already_verified:
                 continue
+            candidates.append(digest)
+            size = m.get("size")
+            if isinstance(size, int) and not isinstance(size, bool):
+                sizes[digest] = size
+            try:
+                manifest_hex = parse_digest(digest)
+                manifest_bytes = (blobs_dir / manifest_hex).read_bytes()
+                manifests_read[manifest_hex] = manifest_bytes
+                layer = manifest_layer_descriptor(json.loads(manifest_bytes))
+                candidate_layers.add(parse_digest(layer.digest))
+            except (ValueError, OSError, FetchError):
+                continue
+        if not candidates:
+            continue
+        referrers = _local_bundle_referrers(blobs_dir, candidate_layers, manifests_read)
+        for digest in candidates:
+            if digest in already_verified:
+                continue
             record = _verify_and_install_from_local_blobs(
                 source_root=root,
                 manifest_digest=digest,
                 cache_write_root=cache_write_root,
                 trusted_keys=trusted_keys,
                 visible_only=True,
+                referrers=referrers,
+                manifest_size=sizes.get(digest),
             )
             if record is not None:
                 already_verified.add(digest)

{chtypes-2.0.0.dev11 → chtypes-2.0.0.dev13}/src/chtypes/_ocifetch/_oci.py

@@ -26,6 +26,7 @@ from chtypes._ocifetch._http import (
 
 __all__ = [
     "Descriptor",
+    "copy_blob_verified",
     "fetch_blob_bytes",
     "fetch_blob_to_path",
     "fetch_manifest_by_digest",
@@ -143,6 +144,46 @@ def verify_body_matches_digest(body: bytes, digest: str, *, what: str) -> None:
         )
 
 
+def copy_blob_verified(src_path: str, descriptor: Descriptor, dest_path: str, *, what: str) -> None:
+    """Copy a LOCAL blob file (a pre-seeded layout's `blobs/sha256/<hex>`)
+    to `dest_path`, streaming, and check the copy's size and sha256 against
+    `descriptor` BEFORE the caller does anything else with it — the local
+    counterpart of `fetch_blob_to_path`, with the same §7.3 rule ("check
+    size and sha256 before decompressing"). The caller unpacks the copy it
+    verified, never the source file, so the bytes checked are the bytes
+    used. Reads at most `descriptor.size + 1` bytes; removes `dest_path` on
+    any failure."""
+    expected = parse_digest(descriptor.digest)
+    hasher = hashlib.sha256()
+    size = 0
+    try:
+        with open(src_path, "rb") as src, open(dest_path, "wb") as dst:
+            while size <= descriptor.size:
+                chunk = src.read(min(1 << 20, descriptor.size + 1 - size))
+                if not chunk:
+                    break
+                hasher.update(chunk)
+                dst.write(chunk)
+                size += len(chunk)
+        if size > descriptor.size:
+            raise ArtifactCorruptError(
+                f"{what}: larger than the descriptor's size {descriptor.size}"
+            )
+        if size != descriptor.size:
+            raise ArtifactCorruptError(
+                f"{what}: size {size} disagreed with the descriptor's {descriptor.size}"
+            )
+        actual = hasher.hexdigest()
+        if actual != expected:
+            raise ArtifactCorruptError(f"{what}: sha256 {actual} disagreed with {expected}")
+    except BaseException:
+        try:
+            os.unlink(dest_path)
+        except OSError:
+            pass
+        raise
+
+
 def resolve_platform_manifest(index_doc: dict, platform_key: str) -> Descriptor:
     """Pick the one manifest descriptor in an OCI image index matching
     `platform_key` (e.g. "linux-arm64"), refusing a duplicate or absent

{chtypes-2.0.0.dev11 → chtypes-2.0.0.dev13}/src/chtypes/library.py

@@ -54,6 +54,8 @@ __all__ = ["Block", "Filter", "Library", "Schema", "Server", "open_unverified"]
 # One image per file, process-wide: keyed on the realpath's device and inode, so
 # two registries, two spellings or a hardlink of one artifact share one image
 # and one Library. Guarded by the setup lock, which an open holds throughout.
+# The batch preview's default `doc_flags`: 0, the lean result (no per-value documents).
+_LEAN_DOC_FLAGS = DocFlags(0)
 _IMAGES: dict[tuple[int, int], Library] = {}
 
 
@@ -212,7 +214,8 @@ class Filter(_Handle):
         insert_options_extra: Mapping[str, str] | None = None,
     ) -> FilterResult:
         """Evaluate over a body. `settings` and `session_timezone` are the body's
-        PARSE settings only: the filter's own zone was fixed when it compiled.
+        INSERT settings: they govern its parse and the expressions the INSERT
+        evaluates, never the WHERE, whose zone was fixed when the filter compiled.
         `columns` is the INSERT column list and the keywords after it the
         `insert_options` members, as for `Schema.rows`."""
         payload = body_bytes(body)
@@ -304,11 +307,14 @@ class Schema(_Handle):
         insert_options_extra: Mapping[str, str] | None = None,
         row_filter: Filter | None = None,
         export: Format | int | None = None,
-        doc_flags: DocFlags = DocFlags.ALL,
+        doc_flags: DocFlags = _LEAN_DOC_FLAGS,
     ) -> BatchResult:
         """A whole body. `export` asks for the accepted rows serialized in that
         format (`payload`, and `spans`); to insert a body whose rows carry a
-        `default_generated` column, insert `payload`, never the original input."""
+        `default_generated` column, insert `payload`, never the original input.
+
+        `doc_flags` chooses the document groups; absent means 0, the lean result
+        (no per-value documents). Pass `DocFlags.ALL` for every group."""
         payload = body_bytes(body)
         settings_bytes = settings_json(settings, session_timezone)
         columns_bytes = columns_json(columns)

{chtypes-2.0.0.dev11 → chtypes-2.0.0.dev13}/tests/abi2/test_public_library.py

@@ -158,10 +158,27 @@ def test_rows_marshals_columns_filter_export_and_doc_flags(lib, monkeypatch) ->
     assert fmt == 0 and body == b'{"a":1}\n' and settings is None and options is None
     assert json.loads(columns) == [{"name": "a"}, {"name_b64": "/w=="}]
     assert handle is not None and export == int(Format.CSV) and flags == 3
-    assert plain[5] is None and plain[6] == -1 and plain[7] == 7  # no export, all groups
+    assert plain[5] is None and plain[6] == -1 and plain[7] == 0  # no export, lean (0)
     assert plain[2] == b"1\n"  # a bytes-like body is copied to bytes
 
 
+def test_rows_doc_flags_default_is_lean_and_explicit_all_reaches_the_call(lib, monkeypatch) -> None:
+    calls: list[tuple] = []
+    real = lib._api.preview_batch
+
+    def spy(*args):
+        calls.append(args)
+        return real(*args)
+
+    monkeypatch.setattr(lib._api, "preview_batch", spy)
+    with lib.compile_table(CREATE) as schema:
+        schema.rows(Format.CSV, b"1\n")
+        schema.rows(Format.CSV, b"1\n", export=Format.CSV)
+        schema.rows(Format.CSV, b"1\n", doc_flags=DocFlags.ALL)
+    # Argument 7 of chs_preview_batch is doc_flags, as the real stub call received it.
+    assert [c[7] for c in calls] == [0, 0, int(DocFlags.ALL)]
+
+
 def test_no_export_means_no_payload(lib) -> None:
     with lib.compile_table(CREATE) as schema:
         assert schema.rows(Format.CSV, b"1\n").payload is None