dataowl 0.1.0__tar.gz

dataowl-0.1.0/.gitignore

@@ -0,0 +1,10 @@
+__pycache__/
+*.py[cod]
+.venv/
+build/
+dist/
+*.egg-info/
+.pytest_cache/
+.ruff_cache/
+.mcp/
+build_plan.md

dataowl-0.1.0/CHANGELOG.md

@@ -0,0 +1,32 @@
+# Changelog
+
+All notable changes to this project are documented in this file.
+
+## [Unreleased]
+
+## [0.1.0] - 2026-10-08
+
+First release: overview facts for a table (`inspect`).
+
+### Added
+- `dataowl.inspect(table, *, spark=None, count_views=False)` returning an `Overview` with
+  `to_dict()` and `show()`.
+- Facts from `information_schema.tables`: object type (with the raw `table_type`), format,
+  owner, comment and created time.
+- Facts from `DESCRIBE DETAIL`: size, number of files, average file size, last data
+  modification, partition and clustering columns. Not run for views.
+- Schema from `information_schema.columns`, and the field count including nested fields
+  from the Spark schema.
+- Exact row count with `COUNT(*)`. Skipped for views, materialized views, foreign tables and
+  objects whose type cannot be read, unless `count_views=True`.
+- Change Data Feed, log retention and deleted file retention from `SHOW TBLPROPERTIES`. Not
+  run for views.
+- Every fact carries its source (`metadata`, `exact`, `derived`). Facts that cannot be
+  collected are marked unavailable with a reason; only a missing table raises
+  (`TableNotFoundError`).
+- Terminal rendering of the overview.
+
+### Known deviations
+- `information_schema.columns.ordinal_position` is observed 0-based in Databricks, although
+  the documentation says it is numbered from 1. `ColumnInfo.position` keeps the raw value;
+  the `#` column in the output is a running number from 1.

dataowl-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Alexandro Dronnen
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

dataowl-0.1.0/PKG-INFO

@@ -0,0 +1,158 @@
+Metadata-Version: 2.5
+Name: dataowl
+Version: 0.1.0
+Summary: Facts about Databricks data products for building dbt staging models.
+Project-URL: Homepage, https://github.com/alexD1990/dataowl
+Project-URL: Repository, https://github.com/alexD1990/dataowl
+Project-URL: Changelog, https://github.com/alexD1990/dataowl/blob/master/CHANGELOG.md
+Project-URL: Issues, https://github.com/alexD1990/dataowl/issues
+Author: Alexandro Dronnen
+License-Expression: MIT
+License-File: LICENSE
+Keywords: data-engineering,databricks,dbt,delta-lake,unity-catalog
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Database
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Provides-Extra: dev
+Requires-Dist: mypy==2.3.1; extra == 'dev'
+Requires-Dist: pyspark>=3.4; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: ruff==0.16.9; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# dataowl
+
+dataowl reports facts about a table in Databricks (Unity Catalog / Delta Lake): size, files,
+rows, schema and table properties. It is meant for data engineers who build a dbt staging
+model and want the facts without writing exploratory SQL. It reports what exists and never
+gives recommendations.
+
+## Requirements
+
+- Unity Catalog.
+- Databricks Runtime 14.3 LTS or above, or serverless compute.
+  Verified on serverless (Spark Connect).
+- PySpark is provided by Databricks. dataowl has no other runtime dependencies.
+
+## Installation
+
+In a Databricks notebook:
+
+```python
+%pip install dataowl==0.1.0
+```
+
+## Usage
+
+```python
+import dataowl
+
+dataowl.inspect("catalog.schema.table").show()
+```
+
+As a JSON-serializable dict, where every fact has a value, a source and a reason when it
+is unavailable:
+
+```python
+overview = dataowl.inspect("catalog.schema.table")
+overview.to_dict()
+```
+
+Row counts are skipped for views, materialized views and foreign tables unless you ask for
+them:
+
+```python
+dataowl.inspect("catalog.schema.some_view", count_views=True).show()
+```
+
+Names with characters other than letters, digits and underscore are quoted with backticks:
+`` dataowl.inspect("`my-catalog`.schema.table") ``.
+
+## Example output
+
+Output for Databricks' public sample data (`samples.nyctaxi.trips`):
+
+```text
+samples.nyctaxi.trips   (MANAGED, DELTA)
+
+Size:                    354.2 KB
+Files:                   1  (avg 354.2 KB)
+Rows:                    21 932
+Columns:                 6  (6 incl. nested)
+Partitioned by:          –
+Clustered by:            –
+Change Data Feed:        true
+Log retention:           not set (default)
+Deleted file retention:  not set (default)
+Created:                 2025-09-30 11:28
+Last modified:           2026-09-14 15:07
+Owner:                   System user
+Comment:                 –
+
+SCHEMA
+  #  name                   type       nullable  comment
+  1  tpep_pickup_datetime   timestamp  yes
+  2  tpep_dropoff_datetime  timestamp  yes
+  3  trip_distance          double     yes
+  4  fare_amount            double     yes
+  5  pickup_zip             int        yes
+  6  dropoff_zip            int        yes
+```
+
+## Principles
+
+- **Read-only.** Only `SELECT`, `DESCRIBE` and `SHOW`. Nothing is written, optimized or
+  analyzed.
+- **Facts only.** No recommendations or assessments.
+- **No row values leave Spark.** Only aggregates and metadata are collected.
+- **Unavailable facts are reported with a reason.** Missing permissions or an unsupported
+  object type make the affected facts `n/a (<reason>)`; the rest of the report is still
+  produced. Only a table that does not exist or is not accessible raises
+  `TableNotFoundError`.
+
+## Facts per object type
+
+| Facts | Source | MANAGED, EXTERNAL | STREAMING_TABLE, MATERIALIZED_VIEW | FOREIGN | VIEW | UNKNOWN or n/a type |
+|---|---|---|---|---|---|---|
+| Object type, format, owner, comment, created | `information_schema.tables` | yes | yes | yes | yes | yes (n/a if the query fails) |
+| Size, files, avg file size, last modified, partitioning, clustering | `DESCRIBE DETAIL` | yes | attempted¹ | attempted¹ | no | attempted¹ |
+| Columns and schema | `information_schema.columns` | yes | yes | yes | yes | yes |
+| Field count incl. nested | Spark schema | yes | yes | yes | yes | yes |
+| Change Data Feed, log and deleted file retention | `SHOW TBLPROPERTIES` | yes | attempted¹ | attempted¹ | no | attempted¹ |
+| Rows | `COUNT(*)` | yes | MATERIALIZED_VIEW: with `count_views=True`; STREAMING_TABLE: yes | with `count_views=True` | with `count_views=True` | UNKNOWN: yes; n/a type: with `count_views=True` |
+
+¹ The query runs. If Databricks does not support it for the object, the facts are shown as
+`n/a (<reason>)`.
+
+"UNKNOWN" is a `table_type` dataowl does not recognize, such as `MANAGED_SHALLOW_CLONE`. The
+original value is shown in the header. "n/a type" means the object type could not be read.
+
+## Fields
+
+- **Last modified** is the last data change, from `DESCRIBE DETAIL`. **Created** is from
+  `information_schema.tables`.
+- **Rows** is an exact `COUNT(*)`. On large tables this costs compute.
+- **Files (avg ...)** is the size divided by the number of files.
+- **Columns (... incl. nested)** counts every field, including fields inside structs, array
+  elements and map keys and values.
+- **not set (default)** means the table property is not set, so Databricks uses its default.
+  dataowl does not report what the default is.
+- **#** in the schema table is a running number from 1 in column order.
+
+## Known deviations
+
+- `information_schema.columns.ordinal_position` is documented as numbered from 1, but has
+  been observed 0-based in Databricks (serverless, October 2026). dataowl keeps the raw
+  value in `ColumnInfo.position` and shows a running number in the `#` column.
+
+## License
+
+MIT. Copyright (c) 2026 Alexandro Dronnen. See [LICENSE](LICENSE).

dataowl-0.1.0/README.md

@@ -0,0 +1,128 @@
+# dataowl
+
+dataowl reports facts about a table in Databricks (Unity Catalog / Delta Lake): size, files,
+rows, schema and table properties. It is meant for data engineers who build a dbt staging
+model and want the facts without writing exploratory SQL. It reports what exists and never
+gives recommendations.
+
+## Requirements
+
+- Unity Catalog.
+- Databricks Runtime 14.3 LTS or above, or serverless compute.
+  Verified on serverless (Spark Connect).
+- PySpark is provided by Databricks. dataowl has no other runtime dependencies.
+
+## Installation
+
+In a Databricks notebook:
+
+```python
+%pip install dataowl==0.1.0
+```
+
+## Usage
+
+```python
+import dataowl
+
+dataowl.inspect("catalog.schema.table").show()
+```
+
+As a JSON-serializable dict, where every fact has a value, a source and a reason when it
+is unavailable:
+
+```python
+overview = dataowl.inspect("catalog.schema.table")
+overview.to_dict()
+```
+
+Row counts are skipped for views, materialized views and foreign tables unless you ask for
+them:
+
+```python
+dataowl.inspect("catalog.schema.some_view", count_views=True).show()
+```
+
+Names with characters other than letters, digits and underscore are quoted with backticks:
+`` dataowl.inspect("`my-catalog`.schema.table") ``.
+
+## Example output
+
+Output for Databricks' public sample data (`samples.nyctaxi.trips`):
+
+```text
+samples.nyctaxi.trips   (MANAGED, DELTA)
+
+Size:                    354.2 KB
+Files:                   1  (avg 354.2 KB)
+Rows:                    21 932
+Columns:                 6  (6 incl. nested)
+Partitioned by:          –
+Clustered by:            –
+Change Data Feed:        true
+Log retention:           not set (default)
+Deleted file retention:  not set (default)
+Created:                 2025-09-30 11:28
+Last modified:           2026-09-14 15:07
+Owner:                   System user
+Comment:                 –
+
+SCHEMA
+  #  name                   type       nullable  comment
+  1  tpep_pickup_datetime   timestamp  yes
+  2  tpep_dropoff_datetime  timestamp  yes
+  3  trip_distance          double     yes
+  4  fare_amount            double     yes
+  5  pickup_zip             int        yes
+  6  dropoff_zip            int        yes
+```
+
+## Principles
+
+- **Read-only.** Only `SELECT`, `DESCRIBE` and `SHOW`. Nothing is written, optimized or
+  analyzed.
+- **Facts only.** No recommendations or assessments.
+- **No row values leave Spark.** Only aggregates and metadata are collected.
+- **Unavailable facts are reported with a reason.** Missing permissions or an unsupported
+  object type make the affected facts `n/a (<reason>)`; the rest of the report is still
+  produced. Only a table that does not exist or is not accessible raises
+  `TableNotFoundError`.
+
+## Facts per object type
+
+| Facts | Source | MANAGED, EXTERNAL | STREAMING_TABLE, MATERIALIZED_VIEW | FOREIGN | VIEW | UNKNOWN or n/a type |
+|---|---|---|---|---|---|---|
+| Object type, format, owner, comment, created | `information_schema.tables` | yes | yes | yes | yes | yes (n/a if the query fails) |
+| Size, files, avg file size, last modified, partitioning, clustering | `DESCRIBE DETAIL` | yes | attempted¹ | attempted¹ | no | attempted¹ |
+| Columns and schema | `information_schema.columns` | yes | yes | yes | yes | yes |
+| Field count incl. nested | Spark schema | yes | yes | yes | yes | yes |
+| Change Data Feed, log and deleted file retention | `SHOW TBLPROPERTIES` | yes | attempted¹ | attempted¹ | no | attempted¹ |
+| Rows | `COUNT(*)` | yes | MATERIALIZED_VIEW: with `count_views=True`; STREAMING_TABLE: yes | with `count_views=True` | with `count_views=True` | UNKNOWN: yes; n/a type: with `count_views=True` |
+
+¹ The query runs. If Databricks does not support it for the object, the facts are shown as
+`n/a (<reason>)`.
+
+"UNKNOWN" is a `table_type` dataowl does not recognize, such as `MANAGED_SHALLOW_CLONE`. The
+original value is shown in the header. "n/a type" means the object type could not be read.
+
+## Fields
+
+- **Last modified** is the last data change, from `DESCRIBE DETAIL`. **Created** is from
+  `information_schema.tables`.
+- **Rows** is an exact `COUNT(*)`. On large tables this costs compute.
+- **Files (avg ...)** is the size divided by the number of files.
+- **Columns (... incl. nested)** counts every field, including fields inside structs, array
+  elements and map keys and values.
+- **not set (default)** means the table property is not set, so Databricks uses its default.
+  dataowl does not report what the default is.
+- **#** in the schema table is a running number from 1 in column order.
+
+## Known deviations
+
+- `information_schema.columns.ordinal_position` is documented as numbered from 1, but has
+  been observed 0-based in Databricks (serverless, October 2026). dataowl keeps the raw
+  value in `ColumnInfo.position` and shows a running number in the `#` column.
+
+## License
+
+MIT. Copyright (c) 2026 Alexandro Dronnen. See [LICENSE](LICENSE).

dataowl-0.1.0/pyproject.toml

@@ -0,0 +1,70 @@
+[build-system]
+requires = ["hatchling>=1.27"]
+build-backend = "hatchling.build"
+
+[project]
+name = "dataowl"
+version = "0.1.0"
+description = "Facts about Databricks data products for building dbt staging models."
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Alexandro Dronnen" }]
+keywords = ["databricks", "unity-catalog", "delta-lake", "dbt", "data-engineering"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3 :: Only",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Database",
+    "Typing :: Typed",
+]
+dependencies = []
+
+[project.urls]
+Homepage = "https://github.com/alexD1990/dataowl"
+Repository = "https://github.com/alexD1990/dataowl"
+Changelog = "https://github.com/alexD1990/dataowl/blob/master/CHANGELOG.md"
+Issues = "https://github.com/alexD1990/dataowl/issues"
+
+[project.optional-dependencies]
+dev = ["pytest", "ruff==0.16.9", "pyspark>=3.4", "mypy==2.3.1"]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/dataowl"]
+
+[tool.hatch.build.targets.sdist]
+include = ["src", "tests", "README.md", "LICENSE", "CHANGELOG.md", "pyproject.toml"]
+exclude = ["build_plan.md", ".github"]
+
+[tool.ruff]
+line-length = 100
+target-version = "py310"
+extend-exclude = ["build_plan.md"]
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B", "SIM"]
+
+[tool.ruff.lint.isort]
+required-imports = ["from __future__ import annotations"]
+
+[tool.mypy]
+files = ["src"]
+python_version = "3.10"
+disallow_untyped_defs = true
+disallow_incomplete_defs = true
+check_untyped_defs = true
+warn_return_any = true
+warn_unused_ignores = true
+warn_redundant_casts = true
+warn_unreachable = true
+strict_equality = true
+
+[tool.pytest.ini_options]
+testpaths = ["tests/unit"]
+norecursedirs = ["tests/integration"]

dataowl-0.1.0/src/dataowl/__init__.py

@@ -0,0 +1,62 @@
+"""dataowl: facts about Databricks data products for building dbt staging models."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from dataowl.collect.columns import collect_columns
+from dataowl.collect.counts import collect_row_count
+from dataowl.collect.detail import collect_detail
+from dataowl.collect.properties import collect_properties
+from dataowl.collect.tables import collect_table_info
+from dataowl.errors import TableNotFoundError
+from dataowl.identifiers import parse_table
+from dataowl.model.overview import Overview
+from dataowl.runner import get_runner
+
+if TYPE_CHECKING:
+    from pyspark.sql import SparkSession
+
+__all__ = ["Overview", "TableNotFoundError", "inspect"]
+
+
+def inspect(
+    table: str, *, spark: SparkSession | None = None, count_views: bool = False
+) -> Overview:
+    """Collect overview facts for a table.
+
+    Raises ValueError for an invalid table name, RuntimeError when no SparkSession is
+    available, and TableNotFoundError when the table does not exist or is not accessible.
+    Every other failure makes the affected facts unavailable.
+    """
+    ref = parse_table(table)
+    runner = get_runner(spark)
+
+    info = collect_table_info(runner, ref)
+    detail = collect_detail(runner, ref, info.object_type)
+    columns = collect_columns(runner, ref)
+    properties = collect_properties(runner, ref, info.object_type)
+    num_rows = collect_row_count(runner, ref, info.object_type, count_views=count_views)
+
+    return Overview(
+        table=ref,
+        object_type=info.object_type,
+        object_type_raw=info.table_type_raw,
+        format=info.format,
+        owner=info.owner,
+        comment=info.comment,
+        created=info.created,
+        last_modified=detail.last_modified,
+        size_bytes=detail.size_bytes,
+        num_files=detail.num_files,
+        avg_file_size_bytes=detail.avg_file_size_bytes,
+        num_rows=num_rows,
+        num_columns=columns.num_columns,
+        num_fields_nested=columns.num_fields_nested,
+        partition_columns=detail.partition_columns,
+        clustering_columns=detail.clustering_columns,
+        change_data_feed=properties.change_data_feed,
+        log_retention=properties.log_retention,
+        deleted_file_retention=properties.deleted_file_retention,
+        columns=columns.columns,
+    )

dataowl-0.1.0/src/dataowl/collect/__init__.py

@@ -0,0 +1,40 @@
+"""Collection layer: queries to raw data to model objects. The only layer that knows SQL."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from dataowl.model.facts import Fact
+from dataowl.model.overview import ObjectType
+from dataowl.runner import SqlRunner
+
+_MAX_REASON_LENGTH = 200
+
+NOT_FOR_VIEWS = "Not available for views"
+
+
+def is_view(object_type: Fact[ObjectType]) -> bool:
+    """True only when the object type is known to be VIEW."""
+    return object_type.available and object_type.value is ObjectType.VIEW
+
+
+def safe_query(
+    runner: SqlRunner, sql: str, params: dict[str, Any] | None = None
+) -> list[dict[str, Any]] | Exception:
+    """Run a query and return the rows, or the exception instead of raising it."""
+    try:
+        return runner.query(sql, params)
+    except Exception as exc:
+        return exc
+
+
+def short_reason(exc: Exception) -> str:
+    """First line of the error message, at most 200 characters.
+
+    Falls back to the exception type name when the message is empty.
+    """
+    lines = str(exc).strip().splitlines()
+    reason = lines[0].strip() if lines else type(exc).__name__
+    if len(reason) > _MAX_REASON_LENGTH:
+        reason = reason[: _MAX_REASON_LENGTH - 1] + "…"
+    return reason

dataowl-0.1.0/src/dataowl/collect/columns.py

@@ -0,0 +1,135 @@
+"""Schema facts from information_schema.columns and the Spark schema."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any, TypeVar
+
+from dataowl.collect import safe_query, short_reason
+from dataowl.identifiers import TableRef
+from dataowl.model.facts import Fact
+from dataowl.model.overview import ColumnInfo, ColumnsInfo
+from dataowl.runner import SqlRunner
+
+if TYPE_CHECKING:
+    from pyspark.sql.types import DataType, StructType
+
+T = TypeVar("T")
+
+NO_COLUMNS = "information_schema.columns returned no rows"
+
+_QUERY = """
+SELECT
+  column_name AS column_name,
+  ordinal_position AS ordinal_position,
+  full_data_type AS full_data_type,
+  data_type AS data_type,
+  is_nullable AS is_nullable,
+  comment AS comment
+FROM {catalog}.information_schema.columns
+WHERE table_schema = :schema
+  AND table_name = :table
+ORDER BY ordinal_position
+"""
+
+
+def collect_columns(runner: SqlRunner, ref: TableRef) -> ColumnsInfo:
+    """Read top-level columns from information_schema and count nested fields from the schema.
+
+    The two sources fail independently.
+    """
+    columns, num_columns = _top_level_columns(runner, ref)
+    return ColumnsInfo(
+        columns=columns,
+        num_columns=num_columns,
+        num_fields_nested=_nested_field_count(runner, ref),
+    )
+
+
+def count_fields(schema: StructType) -> int:
+    """Count all fields in a schema, including nested fields.
+
+    Every StructField counts 1. Fields inside a struct, inside the element type of an
+    array and inside the key and value types of a map are added recursively.
+
+    Example: ``a INT, b STRUCT<x INT, y ARRAY<STRUCT<z INT>>>, m MAP<STRING, STRUCT<v INT>>``
+    counts a, b, x, y, z, m and v, which gives 7.
+    """
+    from pyspark.sql.types import ArrayType, MapType, StructType
+
+    def nested(data_type: DataType) -> int:
+        if isinstance(data_type, StructType):
+            return sum(1 + nested(field.dataType) for field in data_type.fields)
+        if isinstance(data_type, ArrayType):
+            return nested(data_type.elementType)
+        if isinstance(data_type, MapType):
+            return nested(data_type.keyType) + nested(data_type.valueType)
+        return 0
+
+    return nested(schema)
+
+
+def _top_level_columns(
+    runner: SqlRunner, ref: TableRef
+) -> tuple[Fact[tuple[ColumnInfo, ...]], Fact[int]]:
+    result = safe_query(
+        runner,
+        _QUERY.format(catalog=ref.quoted_catalog()),
+        # Unity Catalog stores names in lower case. Lowering the parameters instead of
+        # the columns keeps the WHERE clause free of functions on information_schema columns.
+        {"schema": ref.schema.lower(), "table": ref.table.lower()},
+    )
+    if isinstance(result, Exception):
+        reason = short_reason(result)
+        return Fact.unavailable("metadata", reason), Fact.unavailable("metadata", reason)
+    if not result:
+        return Fact.unavailable("metadata", NO_COLUMNS), Fact.unavailable("metadata", NO_COLUMNS)
+
+    try:
+        columns = tuple(_column(row) for row in result)
+    except ValueError as exc:
+        reason = f"Unexpected row format in information_schema.columns: {exc}"
+        return Fact.unavailable("metadata", reason), Fact.unavailable("metadata", reason)
+    return Fact(columns, source="metadata"), Fact(len(columns), source="metadata")
+
+
+def _column(row: dict[str, Any]) -> ColumnInfo:
+    """Convert one row. Raises ValueError if a key is missing or has an unexpected type."""
+    name = _get(row, "column_name", str)
+    position = _get(row, "ordinal_position", int)
+    full_data_type = row.get("full_data_type")
+    if full_data_type is None:
+        data_type = _get(row, "data_type", str)
+    elif isinstance(full_data_type, str):
+        data_type = full_data_type
+    else:
+        raise ValueError(f"full_data_type has type {type(full_data_type).__name__}")
+    is_nullable = _get(row, "is_nullable", str)
+    if is_nullable not in ("YES", "NO"):
+        raise ValueError(f"is_nullable has value {is_nullable!r}")
+    comment = row.get("comment")
+    if comment is not None and not isinstance(comment, str):
+        raise ValueError(f"comment has type {type(comment).__name__}")
+    return ColumnInfo(
+        name=name,
+        position=position,
+        data_type=data_type,
+        nullable=is_nullable == "YES",
+        comment=comment,
+    )
+
+
+def _get(row: dict[str, Any], key: str, expected: type[T]) -> T:
+    if key not in row:
+        raise ValueError(f"missing {key}")
+    value = row[key]
+    if not isinstance(value, expected) or isinstance(value, bool):
+        raise ValueError(f"{key} has type {type(value).__name__}")
+    return value
+
+
+def _nested_field_count(runner: SqlRunner, ref: TableRef) -> Fact[int]:
+    try:
+        schema = runner.schema(ref)
+    except Exception as exc:
+        return Fact.unavailable("metadata", short_reason(exc))
+    return Fact(count_fields(schema), source="metadata")