delta-engine 0.2.0__tar.gz → 0.3.0__tar.gz

{delta_engine-0.2.0 → delta_engine-0.3.0}/.github/workflows/ci.yaml

@@ -37,9 +37,10 @@ jobs:
               schema="silver",
               name="customers",
               columns=[
-                  Column("id", Integer(), nullable=False, primary_key=True),
+                  Column("id", Integer(), nullable=False),
                   Column("name", String(), nullable=True),
               ],
+              primary_key=["id"],
           )
 
           desired = table.to_desired_table()

{delta_engine-0.2.0 → delta_engine-0.3.0}/CHANGELOG.md

@@ -1,3 +1,41 @@
+## v0.3.0 (2026-07-10)
+
+### BREAKING CHANGE
+
+- DeltaTable(metadata_only=True) is replaced by
+DeltaTable(scope="metadata"). The metadata_only parameter is removed.
+
+### Feat
+
+- **api**: declare sync scope with a scope parameter
+- declare foreign key pairings as an explicit mapping
+- declare primary keys at table level
+- canonicalize foreign key pair order in the domain
+- derive clustering from Column.cluster_key in DeltaTable
+- read observed clustering from DESCRIBE DETAIL clusteringColumns
+- diff clustering by set and emit AlterClustering
+- render clustering changes with an OPTIMIZE FULL hint
+- compile CLUSTER BY for create and in-place clustering
+- add AlterClustering action and SET_CLUSTERING phase
+- carry clustered_by on table snapshots with invariants
+- add CLUSTERING aspect and Column.cluster_key flag
+
+### Fix
+
+- reject a bare string primary_key at declaration
+- **notebooks**: migrate walkthrough to the scope parameter
+- update runtime-import CI smoke test to table-level primary_key
+- drop OPTIMIZE FULL hint from clustering removal; note clustering in metadata-only docs
+- reconcile clustering before dropping columns
+- **test**: make DESCRIBE DETAIL reader fakes support asDict()
+
+### Refactor
+
+- **api**: clean up scope resolution and structure-gated validation
+- wrap ForeignKey.columns in a read-only view; drop stale inference wording
+- drop redundant clustering guard and trim docstrings
+- declare clustering with table-level clustered_by, not a per-column flag
+
 ## v0.2.0 (2026-07-09)
 
 ### Feat

{delta_engine-0.2.0 → delta_engine-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: delta-engine
-Version: 0.2.0
+Version: 0.3.0
 Summary: Framework for defining, validating, and evolving tables in a consistent, safe, and controlled way.
 Project-URL: Homepage, https://github.com/Tomoscorbin/delta-engine
 Project-URL: Repository, https://github.com/Tomoscorbin/delta-engine
@@ -109,7 +109,7 @@ for the model, or jump to what you need:
 - [Safe-change rules](https://github.com/Tomoscorbin/delta-engine/blob/main/docs/reference-safe-change-rules.md) — changes the engine blocks at validation
 - [API reference](https://github.com/Tomoscorbin/delta-engine/blob/main/docs/reference-api.md)
 
-**Architecture** (for contributors)
+**Architecture**
 
 - [Architecture](https://github.com/Tomoscorbin/delta-engine/blob/main/docs/explanation-architecture.md) — layers, ports and adapters, design decisions
 - [Implement a custom adapter](https://github.com/Tomoscorbin/delta-engine/blob/main/docs/how-to-implement-adapter.md) — the `CatalogStateReader` and `PlanExecutor` ports

{delta_engine-0.2.0 → delta_engine-0.3.0}/README.md

@@ -83,7 +83,7 @@ for the model, or jump to what you need:
 - [Safe-change rules](https://github.com/Tomoscorbin/delta-engine/blob/main/docs/reference-safe-change-rules.md) — changes the engine blocks at validation
 - [API reference](https://github.com/Tomoscorbin/delta-engine/blob/main/docs/reference-api.md)
 
-**Architecture** (for contributors)
+**Architecture**
 
 - [Architecture](https://github.com/Tomoscorbin/delta-engine/blob/main/docs/explanation-architecture.md) — layers, ports and adapters, design decisions
 - [Implement a custom adapter](https://github.com/Tomoscorbin/delta-engine/blob/main/docs/how-to-implement-adapter.md) — the `CatalogStateReader` and `PlanExecutor` ports

delta_engine-0.3.0/body.md

@@ -0,0 +1,38 @@
+## v0.3.0 (2026-07-10)
+
+### BREAKING CHANGE
+
+- DeltaTable(metadata_only=True) is replaced by
+DeltaTable(scope="metadata"). The metadata_only parameter is removed.
+
+### Feat
+
+- **api**: declare sync scope with a scope parameter
+- declare foreign key pairings as an explicit mapping
+- declare primary keys at table level
+- canonicalize foreign key pair order in the domain
+- derive clustering from Column.cluster_key in DeltaTable
+- read observed clustering from DESCRIBE DETAIL clusteringColumns
+- diff clustering by set and emit AlterClustering
+- render clustering changes with an OPTIMIZE FULL hint
+- compile CLUSTER BY for create and in-place clustering
+- add AlterClustering action and SET_CLUSTERING phase
+- carry clustered_by on table snapshots with invariants
+- add CLUSTERING aspect and Column.cluster_key flag
+
+### Fix
+
+- reject a bare string primary_key at declaration
+- **notebooks**: migrate walkthrough to the scope parameter
+- update runtime-import CI smoke test to table-level primary_key
+- drop OPTIMIZE FULL hint from clustering removal; note clustering in metadata-only docs
+- reconcile clustering before dropping columns
+- **test**: make DESCRIBE DETAIL reader fakes support asDict()
+
+### Refactor
+
+- **api**: clean up scope resolution and structure-gated validation
+- wrap ForeignKey.columns in a read-only view; drop stale inference wording
+- drop redundant clustering guard and trim docstrings
+- declare clustering with table-level clustered_by, not a per-column flag
+

{delta_engine-0.2.0 → delta_engine-0.3.0}/docs/explanation-architecture.md

@@ -65,7 +65,7 @@ through a sync.
 | `CatalogState`     | The result of reading one table: `TablePresent`, `TableAbsent`, or `ReadFailed`.                                                                             |
 | `TableDiff`        | Typed facts describing how desired and observed state differ. It is either `TableMissing` or `TableDrift`.                                                   |
 | `Change`           | One atomic difference inside a `TableDrift` (`ColumnAdded`, `TableTagUnset`, …). Each names the `TableAspect` it belongs to and can lower itself to actions. |
-| `TableAspect`      | One managed aspect of a table: columns, table comment, properties, tags, partitioning, primary key, or foreign keys. Internal enum.                          |
+| `TableAspect`      | One managed aspect of a table: columns, table comment, properties, tags, partitioning, clustering, primary key, or foreign keys. Internal enum.              |
 | `ValidationResult` | The application policy verdict for a diff. It says whether a drift is safe to plan in this run.                                                              |
 | `ActionPlan`       | The ordered, table-local actions that should be executed if the table is allowed to run.                                                                     |
 | `ResolveResult`    | The foreign-key dependency order plus any FK-specific failures.                                                                                              |
@@ -364,9 +364,14 @@ every change in the drift belongs to a managed aspect, so
 `TableDrift.plan()` naturally produces only the managed actions, with no
 filtering logic needed.
 
-The public API exposes named modes only: `DeltaTable(metadata_only=True)` maps
-to the metadata aspects (comments, tags, key constraints). The `TableAspect`
-enum stays internal.
+The public API exposes named scopes only: `DeltaTable`'s `scope` parameter
+maps `"metadata"` to the metadata aspects (comments, tags, key constraints)
+and `"tags"` to table and column tags only. The `TableAspect` enum stays
+internal.
+
+`CLUSTERING` is not one of the metadata aspects: liquid clustering keys
+change how data files are laid out on storage, so a `metadata_only` sync
+never reconciles them, the same as `COLUMN_STRUCTURE` and `PARTITIONING`.
 
 `diff_table(desired, observed)` produces a `TableDiff`:
 
@@ -382,6 +387,7 @@ aspects:
 - table properties
 - table tags
 - partitioning
+- clustering
 - primary key
 - foreign keys
 
@@ -429,6 +435,11 @@ The phase ordering exists because backend DDL has dependencies:
 - Column nullability changes run before primary keys are set, because primary
   key columns must be non-nullable.
 - Foreign keys are set last, after the referenced primary key exists.
+- Clustering keys are altered after columns are added (a new clustering key may
+  name a column this same sync is still adding) but before columns are dropped,
+  so a table is reclustered off a column before that column is removed — a sync
+  that both drops the live clustering-key column and reclusters elsewhere must
+  not drop it while it is still the active key.
 
 The domain plan describes intent. The adapter compiler decides how each action
 is rendered for its backend.
@@ -477,9 +488,9 @@ dependency's report, but it is not retroactively converted into
 `DeltaTable` is the public declaration object, but the engine plans with
 `DesiredTable`. The lowering boundary does several important things up front:
 
-- rejects property keys the engine does not manage (valued or `None`),
-  and rejects `metadata_only=True` combined with `properties`
-- generates a primary-key constraint from columns marked `primary_key=True`
+- rejects property keys the engine does not manage (valued or `None`) and
+  rejects invalid declared property values
+- generates a primary-key constraint from the table-level `primary_key` argument
 - lowers public `ForeignKey` declarations into domain `ForeignKeyConstraint`
   values
 - validates structural invariants such as non-empty columns, unique column
@@ -495,6 +506,7 @@ customers = DeltaTable(
     schema="silver",
     name="customers",
     columns=[...],
+    primary_key=["id"],
 )
 
 orders = DeltaTable(
@@ -503,17 +515,16 @@ orders = DeltaTable(
     name="orders",
     columns=[...],
     foreign_keys=[
-        ForeignKey(local_columns=["customer_id"], references=customers),
+        ForeignKey(columns={"customer_id": "id"}, references=customers),
     ],
 )
 ```
 
-This object reference lets the API infer the referenced columns from the
-referenced table's primary key. The declaration does not repeat those columns,
-so it cannot drift from the target table's primary-key declaration. The tradeoff
-is that the referenced table must be declared in Python scope. Within one module
-that usually means defining the parent before the child; across modules it means
-importing the referenced table.
+This object reference lets the API validate the mapping against the
+referenced table's actual primary key, and keeps the reference valid if the
+target is renamed. The tradeoff is that the referenced table must be declared
+in Python scope. Within one module that usually means defining the parent
+before the child; across modules it means importing the referenced table.
 
 This source-code order does not determine execution order. The engine sorts
 prepared desired tables by qualified name for deterministic setup, then the
@@ -521,20 +532,31 @@ resolver topologically orders them by FK dependency before execution.
 
 References by dotted name are intentionally not supported in this iteration. If
 that becomes necessary, the API can be widened to accept a `QualifiedName` as an
-additional branch. That would be backward-compatible, but it would also need
-explicit referenced columns because a bare name carries no primary key object to
-inspect.
-
-Partitioning is shaped by a related decision. Primary-key membership is a
-per-column flag (`primary_key=True`), and a composite key simply takes the
-flagged columns in declaration order. Partitioning cannot reuse that shape,
-because partition order is significant _and_ independent of column order: the
-order of names in `partitioned_by` sets the physical directory nesting Delta
-writes, and that nesting can be different from the order columns appear in the
-table. A per-column flag has nowhere to express an ordering distinct from
-column declaration order, so partition columns are named in one ordered,
-table-level list instead. The differ compares that list positionally, which is
-why reordering it is drift, not a no-op.
+additional branch. That would be backward-compatible: the referenced columns are
+already explicit in the `columns` mapping, so a bare name would only lose the
+primary-key object the engine validates the mapping against.
+
+Partitioning is shaped by a related decision. `primary_key` and
+`partitioned_by` are both table-level lists of column names, but "order"
+means something different for each. A primary key's declaration order only
+controls how the constraint is rendered — identity and drift compare it as a
+set, so `(a, b)` and `(b, a)` are the same key. Partition order is
+significant instead: the order of names in `partitioned_by` sets the physical
+directory nesting Delta writes, and that nesting can be different from the
+order columns appear in the table. The differ compares that list positionally,
+which is why reordering it is drift, not a no-op.
+
+Clustering is the other physical layout, and it is declared the same way —
+`clustered_by` is a table-level list on `DeltaTable`, the sibling of
+`partitioned_by` (the two are mutually exclusive; a table has one layout
+strategy). What differs is not the declaration shape but the _comparison_:
+liquid clustering has no physical directory nesting, so key order carries no
+meaning — Delta clusters by the key _set_. So the differ compares
+`partitioned_by` positionally (reordering it is drift) but compares
+`clustered_by` as a set (reordering the keys is a no-op). The general rule: a
+physical layout is a table-level list (`partitioned_by`, `clustered_by`), and
+whether order is significant is a property of the differ, not of the
+declaration shape.
 
 ## Constraint names
 
@@ -544,7 +566,8 @@ For desired tables, the API layer generates names when a `DeltaTable` is lowered
 to a `DesiredTable`:
 
 - primary key: `{table}_pk`
-- foreign key: `{table}_{local_columns}_fk`
+- foreign key: `{table}_{local_columns}_fk`, joining the local columns in
+  sorted order, so the name is independent of declaration order
 
 For observed tables, the reader adapter reads constraint names from the catalog.
 After that, names live on `PrimaryKeyConstraint` and `ForeignKeyConstraint`
@@ -604,22 +627,29 @@ dependency cost.
 
 ## Where to make changes
 
-| Change                         | Main location                                                              | Notes                                                                                                                                                                                                                                                                 |
-| ------------------------------ | -------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Add a new backend              | `delta_engine.adapters`                                                    | Implement `CatalogStateReader` and `PlanExecutor`; keep backend exceptions inside the adapter.                                                                                                                                                                        |
+| Change                         | Main location                                                              | Notes                                                                                                                                                                                                                                                                               |
+| ------------------------------ | -------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Add a new backend              | `delta_engine.adapters`                                                    | Implement `CatalogStateReader` and `PlanExecutor`; keep backend exceptions inside the adapter.                                                                                                                                                                                      |
 | Add a new change type          | `delta_engine.domain.plan.changes`                                         | Add a frozen dataclass with an `aspect` `ClassVar[TableAspect]` and an `actions()` method; add it to the `Change` union, then emit it from the relevant `_diff_*` helper in `delta_engine.domain.plan.diff`. If the change is currently unsupported, add a rule to `validation.py`. |
-| Add a new action type          | `delta_engine.domain.plan` and adapter compiler                            | Define the action and phase in `actions.py`, emit it from the relevant change's `actions()` method, then compile it in the backend adapter.                                                                                                                           |
-| Add a safety rule              | `delta_engine.application.validation`                                      | Rules inspect the `TableDrift` changes and return `ValidationFailure` values.                                                                                                                                                                                         |
-| Add a data type                | `delta_engine.domain.model.data_type` and adapter type mapping             | The domain type is backend-free; SQL names and Spark parsing live in the Databricks adapter.                                                                                                                                                                          |
-| Change public declarations     | `delta_engine.api`, surfaced only through `delta_engine.schema`            | Keep public ergonomics in `delta_engine.schema` and lower choices into domain snapshots before the engine phases begin.                                                                                                                                               |
-| Change FK ordering or blocking | `delta_engine.application.dependency_resolution`                           | Cross-table dependency policy lives in the application layer, not in the domain plan or SQL compiler.                                                                                                                                                                 |
-| Change report output           | `delta_engine.application.report` and `delta_engine.application.rendering` | Keep display formatting out of domain objects.                                                                                                                                                                                                                        |
-| Change Databricks SQL          | `delta_engine.adapters.databricks.sql`                                     | Compile domain actions to backend statements at the adapter boundary.                                                                                                                                                                                                 |
+| Add a new action type          | `delta_engine.domain.plan` and adapter compiler                            | Define the action and phase in `actions.py`, emit it from the relevant change's `actions()` method, then compile it in the backend adapter.                                                                                                                                         |
+| Add a safety rule              | `delta_engine.application.validation`                                      | Rules inspect the `TableDrift` changes and return `ValidationFailure` values.                                                                                                                                                                                                       |
+| Add a data type                | `delta_engine.domain.model.data_type` and adapter type mapping             | The domain type is backend-free; SQL names and Spark parsing live in the Databricks adapter.                                                                                                                                                                                        |
+| Change public declarations     | `delta_engine.api`, surfaced only through `delta_engine.schema`            | Keep public ergonomics in `delta_engine.schema` and lower choices into domain snapshots before the engine phases begin.                                                                                                                                                             |
+| Change FK ordering or blocking | `delta_engine.application.dependency_resolution`                           | Cross-table dependency policy lives in the application layer, not in the domain plan or SQL compiler.                                                                                                                                                                               |
+| Change report output           | `delta_engine.application.report` and `delta_engine.application.rendering` | Keep display formatting out of domain objects.                                                                                                                                                                                                                                      |
+| Change Databricks SQL          | `delta_engine.adapters.databricks.sql`                                     | Compile domain actions to backend statements at the adapter boundary.                                                                                                                                                                                                               |
 
 ## Architectural rules
 
 - Keep PySpark and Databricks imports inside `delta_engine.adapters`.
 - Keep the domain backend-free, immutable, and deterministic.
+- Make immutability real, not conventional: frozen dataclasses copy their
+  collection fields in `__post_init__` — sequences into tuples, mappings into
+  read-only views (`MappingProxyType`), aspect sets into `frozenset` — so an
+  object cannot change after construction through a collection the caller
+  still holds. This applies at the public boundary too: `ForeignKey.columns`
+  and `Column.tags` copy what the user passed, so mutating the original
+  mapping later does not alter the declaration.
 - Put orchestration, safety policy, dependency resolution, and failure
   propagation in the application layer.
 - Put backend normalization at adapter boundaries, such as lowercasing catalog

{delta_engine-0.2.0 → delta_engine-0.3.0}/docs/explanation-safety-model.md

@@ -52,15 +52,17 @@ unmanaged aspect has drifted. It never silently reconciles something a
 declaration didn't claim responsibility for, and it never silently ignores
 drift it isn't allowed to fix — the sync fails and tells you.
 
-There are two scopes:
+There are three public scopes, selected by `DeltaTable`'s `scope` parameter:
 
-| Scope                | Manages                                                                                 | Use for                                                                                 |
-| -------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
-| Full (the default)   | Everything: columns, comments, properties, tags, partitioning, primary and foreign keys | Tables this declaration owns end to end                                                 |
-| `metadata_only=True` | Comments, tags, and key constraints only                                                | Rolling out governance metadata with a hard guarantee that no schema change can slip in |
+| Scope                  | Manages                                                                                 | Use for                                                                                 |
+| ---------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
+| `"full"` (the default) | Everything: columns, comments, properties, tags, partitioning, primary and foreign keys | Tables this declaration owns end to end                                                 |
+| `"metadata"`           | Comments, tags, and key constraints only                                                | Rolling out governance metadata with a hard guarantee that no schema change can slip in |
+| `"tags"`               | Table and column tags only                                                              | Tag governance for tables owned elsewhere, e.g. by a streaming pipeline                 |
 
 See [how to deploy metadata only](how-to-deploy-metadata-only.md) for the
-restricted scope in practice.
+metadata scope in practice, and [tags](how-to-configure-table.md#manage-tags-only)
+for tag-only declarations.
 
 ## Cross-table dependency blocking
 

{delta_engine-0.2.0 → delta_engine-0.3.0}/docs/how-to-configure-table.md

@@ -18,6 +18,7 @@ partitioned. This page is the practical reference for configuring each aspect.
 | Primary keys      | The table's primary key                               | [Primary keys](#primary-keys), below                                                  |
 | Foreign keys      | Cross-table references and sync ordering              | [Foreign keys](#foreign-keys), below                                                  |
 | Partitioning      | Partition columns, fixed at creation                  | [Partitioning](#partitioning), below                                                  |
+| Clustering        | Liquid clustering keys, reconciled in place           | [Clustering](#clustering), below                                                      |
 
 ## Properties
 
@@ -100,7 +101,8 @@ orders = DeltaTable(
     catalog="prod",
     schema="sales",
     name="orders",
-    columns=[Column("id", Integer(), nullable=False, primary_key=True)],
+    columns=[Column("id", Integer(), nullable=False)],
+    primary_key=["id"],
     properties={
         Property.CHANGE_DATA_FEED: "true",          # ensure it is set
         Property.LOG_RETENTION_DURATION: None,       # ensure it is absent
@@ -223,6 +225,36 @@ the next sync unless it is also declared.
 
 As with table tags, keys are **case-sensitive** (`PII` and `pii` are distinct).
 
+### Manage tags only
+
+Use `scope="tags"` when the table is owned elsewhere — for example by a
+streaming pipeline — but you still want delta-engine to reconcile Unity
+Catalog tags. The declaration carries the same table shape as any other, but
+only table tags and column tags are managed: columns, comments, properties,
+partitioning, primary keys, and foreign keys are never changed.
+
+```python
+from delta_engine.schema import Column, DeltaTable, String
+
+events = DeltaTable(
+    catalog="dev",
+    schema="silver",
+    name="streaming_events",
+    columns=[
+        Column("id", String()),
+        Column("email", String(), tags={"pii": "true"}),
+    ],
+    tags={"domain": "events"},
+    scope="tags",
+)
+```
+
+The live table must already exist. If a non-tag aspect drifts from the
+declaration, validation fails before any tag SQL runs; update the declaration
+to match the live table or use the full scope. Properties are the exception:
+a restricted scope never compares them, so live table properties cannot fail
+the sync.
+
 ### Requirements and limits
 
 Column tags require Unity Catalog on Databricks Runtime 13.3 LTS or later and the
@@ -270,8 +302,9 @@ drift that the sync overwrites.
 
 ## Primary keys
 
-Declare a primary key by setting `primary_key=True` on one or more columns.
-Every primary key column must be non-nullable.
+Declare a primary key by passing `primary_key` to `DeltaTable` — a list of
+column names, in the order the constraint is rendered. Every primary key
+column must be non-nullable.
 
 ```python
 from delta_engine.schema import Column, DeltaTable, Integer, String
@@ -281,10 +314,11 @@ orders = DeltaTable(
     schema="silver",
     name="orders",
     columns=[
-        Column("order_id", Integer(), nullable=False, primary_key=True),
+        Column("order_id", Integer(), nullable=False),
         Column("customer_id", Integer(), nullable=False),
         Column("status", String()),
     ],
+    primary_key=["order_id"],
 )
 ```
 
@@ -295,8 +329,8 @@ exposed on the table object.
 
 ### Composite primary keys
 
-Set `primary_key=True` on several columns. The constraint covers them in
-declaration order.
+List several column names in `primary_key`. The constraint covers them in
+that order.
 
 ```python
 order_items = DeltaTable(
@@ -304,10 +338,11 @@ order_items = DeltaTable(
     schema="silver",
     name="order_items",
     columns=[
-        Column("order_id", Integer(), nullable=False, primary_key=True),
-        Column("line_number", Integer(), nullable=False, primary_key=True),
+        Column("order_id", Integer(), nullable=False),
+        Column("line_number", Integer(), nullable=False),
         Column("product_id", Integer(), nullable=False),
     ],
+    primary_key=["order_id", "line_number"],
 )
 ```
 
@@ -315,8 +350,8 @@ order_items = DeltaTable(
 
 A primary key identifies a row, so a nullable key column is not a well-formed
 table definition — and Databricks rejects a nullable primary key at execution
-time regardless. The engine enforces this early: declaring `primary_key=True`
-on a nullable column raises `ValueError` when the `DeltaTable` is constructed,
+time regardless. The engine enforces this early: naming a nullable column in
+`primary_key` raises `ValueError` when the `DeltaTable` is constructed,
 before any sync runs.
 
 ### Constraints are informational
@@ -354,9 +389,8 @@ table with the original error. See
 
 ## Foreign keys
 
-Pass `foreign_keys` with one `ForeignKey` per constraint. Each names the local
-columns and the table they reference; the referenced columns are inferred from
-that table's primary key.
+Pass `foreign_keys` with one `ForeignKey` per constraint. `columns` maps each
+local column to the referenced-table primary-key column it references.
 
 ```python
 from delta_engine.schema import Column, DeltaTable, ForeignKey, Long, String
@@ -366,9 +400,10 @@ customers = DeltaTable(
     schema="silver",
     name="customers",
     columns=[
-        Column("id", Long(), nullable=False, primary_key=True),
+        Column("id", Long(), nullable=False),
         Column("name", String()),
     ],
+    primary_key=["id"],
 )
 
 orders = DeltaTable(
@@ -376,22 +411,23 @@ orders = DeltaTable(
     schema="silver",
     name="orders",
     columns=[
-        Column("order_id", Long(), nullable=False, primary_key=True),
+        Column("order_id", Long(), nullable=False),
         Column("customer_id", Long(), nullable=False),
         Column("status", String()),
     ],
+    primary_key=["order_id"],
     foreign_keys=[
-        ForeignKey(local_columns=["customer_id"], references=customers),
+        ForeignKey(columns={"customer_id": "id"}, references=customers),
     ],
 )
 ```
 
 Referencing the target `DeltaTable` object — rather than a dotted table name —
-is what lets the engine infer the referenced columns from that table's primary
+is what lets the engine validate the mapping against that table's actual primary
 key, and keeps the reference valid if the target is renamed. The constraint
-name is generated at lowering as `{table}_{local_columns}_fk`
-(`orders_customer_id_fk` above). The name cannot be chosen, and drift matching
-never depends on it — a foreign key created outside the engine under a
+name is generated at lowering as `{table}_{local_columns}_fk`, joining the
+local columns in sorted order (`orders_customer_id_fk` above). The name cannot
+be chosen, and drift matching never depends on it — a foreign key created outside the engine under a
 different name still matches by content.
 
 Generated names join local columns with underscores, so two foreign keys over
@@ -418,20 +454,20 @@ employees = DeltaTable(
     schema="silver",
     name="employees",
     columns=[
-        Column("id", Long(), nullable=False, primary_key=True),
+        Column("id", Long(), nullable=False),
         Column("manager_id", Long()),
     ],
+    primary_key=["id"],
     foreign_keys=[
-        ForeignKey(local_columns=["manager_id"], references=Self),
+        ForeignKey(columns={"manager_id": "id"}, references=Self),
     ],
 )
 ```
 
 ### Composite foreign keys
 
-For a composite primary key, list `local_columns` in the referenced table's
-primary-key declaration order. The referenced columns are inferred one-to-one
-in that same order.
+For a composite primary key, map each local column to the primary-key column
+it references.
 
 ```python
 customer_accounts = DeltaTable(
@@ -439,9 +475,10 @@ customer_accounts = DeltaTable(
     schema="silver",
     name="customer_accounts",
     columns=[
-        Column("tenant_id", Long(), nullable=False, primary_key=True),
-        Column("id", Long(), nullable=False, primary_key=True),
+        Column("tenant_id", Long(), nullable=False),
+        Column("id", Long(), nullable=False),
     ],
+    primary_key=["tenant_id", "id"],
 )
 
 order_lines = DeltaTable(
@@ -449,20 +486,24 @@ order_lines = DeltaTable(
     schema="silver",
     name="order_lines",
     columns=[
-        Column("order_line_id", Long(), nullable=False, primary_key=True),
+        Column("order_line_id", Long(), nullable=False),
         Column("tenant_id", Long(), nullable=False),
         Column("customer_id", Long(), nullable=False),
     ],
+    primary_key=["order_line_id"],
     foreign_keys=[
         ForeignKey(
-            # aligns with customer_accounts PK (tenant_id, id)
-            local_columns=["tenant_id", "customer_id"],
+            columns={"tenant_id": "tenant_id", "customer_id": "id"},
             references=customer_accounts,
         ),
     ],
 )
 ```
 
+The mapping states which primary-key column each local column references;
+its order never matters, and a mapping that does not cover the referenced
+table's primary key exactly fails when the `DeltaTable` is constructed.
+
 ### Dependency ordering
 
 A foreign key can only be added once its referenced table exists with the
@@ -573,3 +614,57 @@ Partition columns also cannot be of complex type (`Array`, `Map`, `Struct`,
 rejected when the `DeltaTable` is constructed. See
 [safe-change rules](reference-safe-change-rules.md) for the full set of changes
 the engine rejects.
+
+## Clustering
+
+Declare Delta liquid clustering keys with the `clustered_by` argument — a
+table-level list of column names, the same shape as `partitioned_by`:
+
+```python
+from delta_engine.schema import Column, DeltaTable, String
+
+events = DeltaTable(
+    catalog="dev",
+    schema="silver",
+    name="events",
+    columns=[
+        Column("region", String()),
+        Column("event_type", String()),
+    ],
+    clustered_by=["region"],
+)
+```
+
+`DeltaTable.clustered_by` exposes the declared tuple of clustering column
+names, in declaration order. Key order does not matter — Delta clusters by the
+key set — so reordering the keys is never treated as drift.
+
+A table cannot declare both `partitioned_by` and `clustered_by` — Delta
+supports one physical layout strategy per table — and a declaration is
+limited to four clustering keys. Both are rejected when the `DeltaTable` is
+constructed. See
+[limitations](reference-limitations.md) for the unsupported key types.
+
+### Clustering is reconciled in place
+
+Unlike partitioning, clustering keys are not fixed at creation. The engine
+changes the key set with `ALTER TABLE ... CLUSTER BY (...)` (or `CLUSTER BY
+NONE` to remove clustering) whenever the declaration changes — no table
+recreation required.
+
+The `ALTER` is a metadata change: it sets the _target_ clustering keys but
+does not rewrite existing data, so it stays cheap regardless of table size.
+Liquid clustering still lays data out physically — it co-locates rows within
+files rather than in partition directories — but existing files keep their
+old clustering until a later `OPTIMIZE` (or `OPTIMIZE FULL` to recluster the
+whole table) rewrites them. This is why partitioning is blocked while
+clustering is not: changing partition columns would mean physically rewriting
+every data file up front. See
+[safe-change rules](reference-safe-change-rules.md) for the full contrast.
+
+### Drift
+
+The engine compares clustering keys by _set_, not by order: declaring the
+same keys in a different order is not drift and plans nothing. This mirrors
+primary keys, and is unlike partitioning, where order is a physical layout
+decision.