delta-engine 0.2.0__tar.gz → 0.4.0__tar.gz

{delta_engine-0.2.0 → delta_engine-0.4.0}/.claude/CLAUDE.md

@@ -10,16 +10,17 @@ This file contains project-level instructions that should be available in every
 
 Main source layout:
 
-* `src/delta_engine/domain`: backend-free domain model, table snapshots, diffs, actions, and deterministic plans.
-* `src/delta_engine/application`: orchestration, ports, validation, dependency resolution, reporting, and errors.
-* `src/delta_engine/adapters`: backend integration. The current adapter is Databricks/Spark.
-* `src/delta_engine/api`: implementation of public declaration objects.
-* `src/delta_engine/schema.py`: public schema declaration import surface.
-* `src/delta_engine/databricks.py`: public Databricks helper import surface with lazy adapter imports.
-* `tests`: unit, integration, adapter, and end-to-end tests.
-* `docs`: Sphinx/MyST documentation.
-
-The library should remain importable for schema declaration and planning without requiring PySpark unless the Databricks adapter is actually used.
+- `src/delta_engine/domain`: backend-free domain model, table snapshots, diffs, actions, and deterministic plans.
+- `src/delta_engine/application`: orchestration, ports, validation, dependency resolution, reporting, and errors.
+- `src/delta_engine/adapters`: backend integration. The Databricks adapter has two backends: Spark and SQL warehouse.
+- `src/delta_engine/api`: implementation of public declaration objects.
+- `src/delta_engine/schema.py`: public schema declaration import surface.
+- `src/delta_engine/databricks.py`: public Databricks helper import surface with lazy adapter imports.
+- `src/delta_engine/cli`: read-only `plan` command over the warehouse backend; requires the `cli` extra.
+- `tests`: unit, integration, adapter, and end-to-end tests.
+- `docs`: Sphinx/MyST documentation.
+
+The library should remain importable for schema declaration and planning without requiring PySpark unless the Spark backend is used; the SQL warehouse backend runs without PySpark entirely.
 
 ## Commands
 
@@ -35,7 +36,7 @@ Run tests:
 uv run pytest
 uv run pytest tests/domain/plan/test_diff.py
 uv run pytest tests/domain/plan/test_diff.py::test_name
-uv run pytest -m "not local_e2e"
+uv run pytest -m "not local_e2e and not databricks_e2e"
 ```
 
 The first test run starts a local Spark session through the `spark` fixture in `tests/conftest.py` and can take 30-60 seconds.
@@ -90,7 +91,7 @@ uv run lint-imports
 uv run --group docs sphinx-build -b html docs docs/_build/html -W
 ```
 
-Do not run credentialed Databricks tests unless explicitly asked.
+This repo carries no tests that need Databricks credentials; the full suite runs locally. Live verification against a real workspace happens in a separate project outside this repo.
 
 ## Architecture boundaries
 
@@ -100,20 +101,23 @@ The import architecture is enforced by `import-linter` in `pyproject.toml`. Do n
 
 Rules:
 
-* `domain` must stay backend-free, immutable, and deterministic.
-* `application` owns orchestration, ports, safety policy, dependency resolution, reports, and failure propagation.
-* `adapters` own backend integration, SQL compilation, Spark/Databricks parsing, identifier quoting, and backend exception translation.
-* `api` owns public declaration implementation and lowers declarations into domain snapshots.
-* `schema.py` and `databricks.py` are user-facing facades; keep them thin.
-* PySpark, Delta, Py4J, Spark SQL details, and Databricks-specific assumptions must not leak into `domain` or `application`.
+- `domain` must stay backend-free, immutable, and deterministic.
+- `application` owns orchestration, ports, safety policy, dependency resolution, reports, and failure propagation.
+- `adapters` own backend integration, SQL compilation, Spark/Databricks parsing, identifier quoting, and backend exception translation.
+- `api` owns public declaration implementation and lowers declarations into domain snapshots.
+- `schema.py` and `databricks.py` are user-facing facades; keep them thin.
+- `cli.connection` may import the Databricks SDK and SQL connector only to
+  resolve unified authentication and own a connection; backend reads and
+  execution stay in adapters.
+- PySpark, Delta, Py4J, Spark SQL details, and Databricks-specific assumptions must not leak into `domain` or `application`.
 
 Expected dependency direction:
 
 ```text
-databricks | schema | adapters | api -> application -> domain
+cli -> databricks | schema | adapters | api -> application -> domain
 ```
 
-A separate import-linter contract forbids `delta` and `pyspark` imports from `schema`, `api`, `application`, and `domain`.
+Two more import-linter contracts forbid `delta` and `pyspark` imports: one covering `schema`, `api`, `application`, and `domain`; another covering `cli`, with a carve-out for the one legitimate edge (the lazy Spark facade import). A further contract confines `typer`/`click`/`rich` imports to `cli`.
 
 ## Sync lifecycle
 
@@ -134,8 +138,8 @@ A table that fails an early phase keeps that failure in its report and is skippe
 
 Both application ports are total from the engine's perspective:
 
-* `CatalogStateReader.fetch_state(...)` returns `TablePresent`, `TableAbsent`, or `ReadFailed`.
-* `PlanExecutor.execute(...)` returns `ExecutionSummary`.
+- `CatalogStateReader.fetch_state(...)` returns `TablePresent`, `TableAbsent`, or `ReadFailed`.
+- `PlanExecutor.execute(...)` returns `ExecutionSummary`.
 
 Adapters should catch backend exceptions and convert them into typed failures rather than raising backend-specific exceptions through the port.
 
@@ -153,8 +157,8 @@ Action ordering is defined by `ActionPhase` and then by action `subject`.
 
 Properties and tags intentionally use different semantics:
 
-* Properties use exact-declaration semantics. A declared `None` asserts absence and plans an unset when present.
-* Tags use full-state semantics. An observed-only tag is drift and should be unset when tags are managed.
+- Properties use exact-declaration semantics. A declared `None` asserts absence and plans an unset when present.
+- Tags use full-state semantics. An observed-only tag is drift and should be unset when tags are managed.
 
 `managed_aspects` is part of the safety model. Drift outside the aspects managed by a declaration should fail validation rather than be silently reconciled.
 
@@ -178,16 +182,17 @@ Foreign-key declarations reference the target `DeltaTable` object, or `Self`, ra
 | Report/output formatting             | `src/delta_engine/application/report.py` and `src/delta_engine/application/rendering.py` |
 | Databricks SQL generation            | `src/delta_engine/adapters/databricks/sql`                                               |
 | Public Databricks helper             | `src/delta_engine/databricks.py`                                                         |
+| CLI commands, output, exit codes     | `src/delta_engine/cli`                                                                   |
 | Documentation                        | `docs`                                                                                   |
 
 Before changing architecture, action planning, validation, adapters, or public behaviour, read the relevant docs instead of guessing:
 
-* `docs/explanation-architecture.md`
-* `docs/how-to-implement-adapter.md`
-* `docs/how-to-add-action-type.md`
-* `docs/reference-safe-change-rules.md`
-* `docs/how-to-handle-sync-failures.md`
-* `docs/how-to-deploy-metadata-only.md`
+- `docs/explanation-architecture.md`
+- `docs/how-to-implement-adapter.md`
+- `docs/how-to-add-action-type.md`
+- `docs/reference-safe-change-rules.md`
+- `docs/how-to-handle-sync-failures.md`
+- `docs/how-to-deploy-metadata-only.md`
 
 ## Documentation
 
@@ -209,6 +214,6 @@ When a change touches domain, application, adapter, and docs, keep the conceptua
 
 Before finishing, state:
 
-* what changed
-* what checks were run
-* what risks or follow-ups remain
+- what changed
+- what checks were run
+- what risks or follow-ups remain

{delta_engine-0.2.0 → delta_engine-0.4.0}/.claude/rules/documentation.md

@@ -79,7 +79,7 @@ Use public import paths in user-facing examples:
 
 ```python
 from delta_engine.schema import DeltaTable
-from delta_engine.databricks import build_engine
+from delta_engine.databricks import build_spark_engine
 ```
 
 Avoid examples that import from internal modules unless the doc is explicitly for contributors.

{delta_engine-0.2.0 → delta_engine-0.4.0}/.claude/rules/testing.md

@@ -1,9 +1,6 @@
 ---
-
 paths:
-
-- "tests/**/*.py"
-
+  - "tests/**/*.py"
 ---
 
 # Testing rules
@@ -54,7 +51,9 @@ For Databricks executor changes, assert execution summaries, successful action r
 
 Do not require real Databricks for ordinary unit tests.
 
-Credentialed Databricks tests must remain opt-in through the `databricks_e2e` marker.
+This repo carries no tests that need Databricks credentials; live verification against Unity Catalog happens in a separate project outside this repo.
+
+If a credentialed Databricks test is ever added, it must be opt-in through the `databricks_e2e` marker and must not run in default CI.
 
 Local Spark/Delta end-to-end tests should stay marked with `local_e2e`.
 
@@ -78,7 +77,7 @@ uv run pytest
 To skip local Spark/Delta end-to-end tests:
 
 ```bash
-uv run pytest -m "not local_e2e"
+uv run pytest -m "not local_e2e and not databricks_e2e"
 ```
 
-Do not run credentialed Databricks tests unless explicitly asked.
+Every test in this repo runs locally without Databricks credentials.

{delta_engine-0.2.0 → delta_engine-0.4.0}/.claude/skills/add-tests/SKILL.md

@@ -302,7 +302,7 @@ def test_metadata_only_declaration_carries_properties_without_deploying_them():
 
 ### Outgoing Interaction Test
 
-Use a fake only when the behaviour is the outgoing interaction — here, the executor handing compiled SQL to Spark. This project's own adapter tests use exactly this pattern (see `tests/adapters/databricks/test_executor.py`).
+Use a fake only when the behaviour is the outgoing interaction — here, the executor handing compiled SQL to Spark. This project's own adapter tests use exactly this pattern (see `tests/adapters/databricks/spark/test_executor.py`).
 
 ```python
 class _FakeSpark:
@@ -318,11 +318,12 @@ class _FakeSpark:
 def test_executor_runs_the_compiled_statement_against_spark():
     # given
     spark = _FakeSpark()
-    executor = DatabricksExecutor(spark)
+    executor = SparkExecutor(spark)
     plan = ActionPlan((SetTableComment("new comment"),))
 
     # when
-    executor.execute(QualifiedName("catalog", "schema", "table"), plan)
+    statements = executor.compile(QualifiedName("catalog", "schema", "table"), plan)
+    executor.execute(statements)
 
     # then
     assert len(spark.executed) == 1

{delta_engine-0.2.0 → delta_engine-0.4.0}/.github/workflows/ci.yaml

@@ -37,9 +37,10 @@ jobs:
               schema="silver",
               name="customers",
               columns=[
-                  Column("id", Integer(), nullable=False, primary_key=True),
+                  Column("id", Integer(), nullable=False),
                   Column("name", String(), nullable=True),
               ],
+              primary_key=["id"],
           )
 
           desired = table.to_desired_table()

delta_engine-0.4.0/.github/workflows/docs.yaml

@@ -0,0 +1,52 @@
+name: Docs
+
+on:
+  push:
+    branches: [main]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: pages
+  cancel-in-progress: false
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v8.2.0
+        with:
+          enable-cache: true
+          cache-dependency-glob: uv.lock
+
+      - name: Set up Python
+        run: uv python install 3.12
+
+      - name: Install dependencies
+        run: uv sync --locked --group docs
+
+      - name: Build docs
+        run: uv run sphinx-build -W -b html docs docs/_build/html
+
+      - name: Upload Pages artifact
+        uses: actions/upload-pages-artifact@v4
+        with:
+          path: docs/_build/html
+
+  deploy:
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

delta_engine-0.4.0/.github/workflows/live.yaml

@@ -0,0 +1,92 @@
+# Credentialed live suite against a real Databricks SQL warehouse.
+#
+# Required repository variables:
+#   DATABRICKS_HOST            workspace URL
+#   DATABRICKS_HTTP_PATH       SQL warehouse endpoint path
+#   DELTA_ENGINE_E2E_CATALOG   catalog the suite may create tables in
+#   DELTA_ENGINE_E2E_SCHEMA    schema the suite may create tables in
+# Required repository secret (either one):
+#   DATABRICKS_TOKEN           personal access token, or
+#   DATABRICKS_CLIENT_ID       service principal for OIDC workload identity
+#                              federation (needs a Databricks federation
+#                              policy for this repository)
+name: Live Databricks Tests
+
+on:
+  workflow_dispatch:
+  schedule:
+    # Weekly drift check (Mondays 07:00 UTC): catches Databricks-side
+    # behaviour changes the hermetic suite cannot see. Non-blocking — it
+    # never gates a PR.
+    - cron: "0 7 * * 1"
+
+# One live run at a time: the suite shares a schema and a warehouse.
+concurrency:
+  group: ${{ github.workflow }}
+  cancel-in-progress: false
+
+permissions:
+  contents: read
+  id-token: write
+
+jobs:
+  live:
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    env:
+      DATABRICKS_HOST: ${{ vars.DATABRICKS_HOST }}
+      DATABRICKS_HTTP_PATH: ${{ vars.DATABRICKS_HTTP_PATH }}
+      DELTA_ENGINE_E2E_CATALOG: ${{ vars.DELTA_ENGINE_E2E_CATALOG }}
+      DELTA_ENGINE_E2E_SCHEMA: ${{ vars.DELTA_ENGINE_E2E_SCHEMA }}
+    steps:
+      - uses: actions/checkout@v5
+
+      - name: Require live configuration
+        # Missing configuration must fail loudly here: without these the
+        # suite's fixtures skip every test and the run would pass green.
+        run: |
+          missing=0
+          for name in DATABRICKS_HOST DATABRICKS_HTTP_PATH DELTA_ENGINE_E2E_CATALOG DELTA_ENGINE_E2E_SCHEMA; do
+            if [ -z "${!name}" ]; then
+              echo "::error::repository variable ${name} is not set"
+              missing=1
+            fi
+          done
+          exit "${missing}"
+
+      - name: Select Databricks authentication
+        env:
+          TOKEN: ${{ secrets.DATABRICKS_TOKEN }}
+          CLIENT_ID: ${{ secrets.DATABRICKS_CLIENT_ID }}
+        run: |
+          if [ -n "${TOKEN}" ]; then
+            echo "DATABRICKS_TOKEN=${TOKEN}" >> "$GITHUB_ENV"
+            echo "authenticating with a personal access token"
+          elif [ -n "${CLIENT_ID}" ]; then
+            echo "DATABRICKS_CLIENT_ID=${CLIENT_ID}" >> "$GITHUB_ENV"
+            echo "DATABRICKS_AUTH_TYPE=github-oidc" >> "$GITHUB_ENV"
+            echo "authenticating with OIDC workload identity federation"
+          else
+            echo "::error::set the DATABRICKS_TOKEN or DATABRICKS_CLIENT_ID repository secret"
+            exit 1
+          fi
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v8.2.0
+        with:
+          enable-cache: true
+          cache-dependency-glob: uv.lock
+
+      - name: Set up Python
+        run: uv python install 3.12
+
+      - name: Install dependencies
+        run: uv sync --locked
+
+      - name: Run live suite
+        # -n 8: the suite is I/O-bound on warehouse round-trips, not runner
+        # CPU, so running more workers than cores still speeds it up. Tests are
+        # isolated (uuid-named tables, per-test cleanup), so they parallelise
+        # safely; excess concurrency degrades to warehouse queueing, not
+        # failure. Tune to the warehouse's concurrency headroom.
+        run: uv run pytest tests/live -m databricks_e2e --no-cov -rA -n 8

{delta_engine-0.2.0 → delta_engine-0.4.0}/.gitignore

@@ -46,3 +46,4 @@ htmlcov/
 .superpowers/
 docs/superpowers/
 docs/_build/
+docs/generated/

{delta_engine-0.2.0 → delta_engine-0.4.0}/CHANGELOG.md

@@ -1,3 +1,157 @@
+## v0.4.0 (2026-07-14)
+
+### BREAKING CHANGE
+
+- public result/failure types lose statement_preview in
+favour of statement; failure text renders 'SQL:' instead of 'SQL preview:'.
+- install with delta-engine[spark] instead of
+delta-engine[databricks]; import build_spark_engine instead of build_engine
+from delta_engine.databricks.
+- ExecutionFailure.statement_index replaces
+action_index; the run report's execution.total now counts planned
+statements.
+- SyncReport.any_failures is now SyncReport.has_failures,
+mirroring TableRunReport.has_failures.
+
+### Feat
+
+- render column renames in sync reports
+- compile RenameColumn to ALTER TABLE ... RENAME COLUMN
+- reject ambiguous declared renames; point guard at renamed_from
+- relabel observed columns through declared renames in diff_table
+- add ColumnRenamed change and RenameColumn action
+- require column mapping at construction for rename hints
+- enforce rename-hint coherence on DesiredTable
+- add renamed_from declaration hint to Column
+- adopt unified auth and explicit declaration selection in the CLI
+- add DuplicateTableDefinitionError and extract prepare_desired_tables
+- wire the delta-engine console script with graceful degradation
+- add delta-engine apply command
+- add delta-engine plan command
+- add render_planned_sql for planned-statement previews
+- add CLI warehouse connection resolution
+- add CLI declaration loading for module[:attr] specs
+- expose build_sql_engine for PySpark-free warehouse syncs
+- add warehouse plan executor
+- add warehouse catalog state reader
+- add information_schema table-row and columns query builders
+- add PySpark-free DDL type-string parser to the shared sql package
+- export TableRunReport and concrete failure types
+- add to_dict projection on sync and table run reports
+- record compiled SQL statements on every table run report
+- add PlanExecutor.compile returning the plan's SQL statements
+- add has_changes and rename SyncReport.any_failures to has_failures
+- adopt the full Delta type-widening matrix
+- permit safe type widenings gated on delta.enableTypeWidening
+- lower column type changes to AlterColumnType
+- add AlterColumnType action with SQL compilation and diff rendering
+- register delta.enableTypeWidening as a managed property
+
+### Fix
+
+- Update verification table header in conftest.py
+- enforce Databricks 256-character tag key and value limits
+- harden correctness edge cases
+- compile empty column comments as COMMENT '' instead of UNSET COMMENT
+- register the parent in the FK-rename live sync
+- harden declarative column rename planning
+- filter lazy connector warning
+- reject path-like declaration references and honour module __getattr__
+- suppress irrelevant PyArrow warning
+- close a false-positive gap in the CLI PySpark-free contract
+- compile empty column comments as COMMENT '' instead of UNSET COMMENT
+- make WarehouseExecutor.execute total across cursor lifecycle
+- treat pathological nesting depth as unmappable in the type parser
+- lint
+
+### Refactor
+
+- plan explicit constraint drops around column renames
+- rename domain Column to DesiredColumn
+- drop compiler-facing alias properties from actions
+- split table drift into actions and findings
+- remove primary-key replacement correlation
+- clarify planning domain boundaries
+- collapse changes into canonical actions
+- single-source the duplicate-table rule in engine preparation
+- delegate CLI authentication to Databricks SDK
+- narrow CLI to read-only OIDC plans
+- fail fast on a missing HTTP path and pin cwd import precedence
+- remove TableSnapshot in favour of standalone table types
+- readers build ObservedColumn for catalog state
+- split observed catalog columns into ObservedColumn
+- fold desired-table preparation into the engine, move DesiredTableSource to ports
+- route reader imports through the sql facade, widen rows.py framing
+- acquire the warehouse cursor lazily inside the execution loop
+- share exception naming via duck-typed py4j probe, delete the injection seam
+- read Spark catalog types through the shared DDL parser, delete spark/types.py
+- move the shared per-column read policy behind column_from_catalog
+- fuse DESCRIBE DETAIL properties mapping into one boundary function
+- share quoted-token reading between backtick names and string literals
+- narrow the exception seam to type naming, delete errors.py
+- share DESCRIBE DETAIL row mapping, tolerant of both shapes
+- rename sql_type_for_data_type to render_data_type
+- record exact statements and exception messages on failures
+- align execution and error-translation vocabulary
+- move information_schema row mappers into the shared sql package
+- extract shared statement-execution loop from the spark executor
+- extract shared exception-summarising core from the spark backend
+- rename the databricks extra to spark and build_engine to build_spark_engine
+- enforce the PySpark-free shared sql core with import-linter
+- rename Databricks reader and executor to Spark names
+- split Spark type parsing out of the shared sql package
+- move Spark-coupled adapter code into a spark subpackage
+- denominate execution results in statements
+- polish the CI report contract before it ships
+- execute compiled statements instead of recompiling the plan
+- fold compile back into the plan phase
+- extract compile into its own sync phase
+- compile plans unconditionally, dropping the empty-plan guard
+- extract action interpretation into diff_entries module
+- reintroduce _SelfReference class in delta_table.py
+- state the FK name-collision fact without API name-generation advice
+- dedupe partition/clustering key-list validation in TableSnapshot
+- name the resolved referenced side of a foreign key
+- merge partition and clustering validators into _validate_layout
+
+## v0.3.0 (2026-07-10)
+
+### BREAKING CHANGE
+
+- DeltaTable(metadata_only=True) is replaced by
+DeltaTable(scope="metadata"). The metadata_only parameter is removed.
+
+### Feat
+
+- **api**: declare sync scope with a scope parameter
+- declare foreign key pairings as an explicit mapping
+- declare primary keys at table level
+- canonicalize foreign key pair order in the domain
+- derive clustering from Column.cluster_key in DeltaTable
+- read observed clustering from DESCRIBE DETAIL clusteringColumns
+- diff clustering by set and emit AlterClustering
+- render clustering changes with an OPTIMIZE FULL hint
+- compile CLUSTER BY for create and in-place clustering
+- add AlterClustering action and SET_CLUSTERING phase
+- carry clustered_by on table snapshots with invariants
+- add CLUSTERING aspect and Column.cluster_key flag
+
+### Fix
+
+- reject a bare string primary_key at declaration
+- **notebooks**: migrate walkthrough to the scope parameter
+- update runtime-import CI smoke test to table-level primary_key
+- drop OPTIMIZE FULL hint from clustering removal; note clustering in metadata-only docs
+- reconcile clustering before dropping columns
+- **test**: make DESCRIBE DETAIL reader fakes support asDict()
+
+### Refactor
+
+- **api**: clean up scope resolution and structure-gated validation
+- wrap ForeignKey.columns in a read-only view; drop stale inference wording
+- drop redundant clustering guard and trim docstrings
+- declare clustering with table-level clustered_by, not a per-column flag
+
 ## v0.2.0 (2026-07-09)
 
 ### Feat

delta_engine-0.4.0/PKG-INFO

@@ -0,0 +1,145 @@
+Metadata-Version: 2.4
+Name: delta-engine
+Version: 0.4.0
+Summary: Framework for defining, validating, and evolving tables in a consistent, safe, and controlled way.
+Project-URL: Homepage, https://github.com/Tomoscorbin/delta-engine
+Project-URL: Repository, https://github.com/Tomoscorbin/delta-engine
+Project-URL: Documentation, https://tomoscorbin.github.io/delta-engine/
+Project-URL: Issues, https://github.com/Tomoscorbin/delta-engine/issues
+Author-email: Tomos Corbin <tomoscorbin@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: data-engineering,databricks,ddl,delta-lake,migrations,schema
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Database
+Classifier: Typing :: Typed
+Requires-Python: >=3.12
+Provides-Extra: cli
+Requires-Dist: databricks-sdk>=0.70.0; extra == 'cli'
+Requires-Dist: databricks-sql-connector>=4.0.0; extra == 'cli'
+Requires-Dist: typer>=0.12; extra == 'cli'
+Provides-Extra: spark
+Requires-Dist: delta-spark>=4.0.0; extra == 'spark'
+Requires-Dist: pyspark>=4.0.0; extra == 'spark'
+Provides-Extra: sql
+Requires-Dist: databricks-sql-connector>=4.0.0; extra == 'sql'
+Description-Content-Type: text/markdown
+
+# delta-engine
+
+[![Live Databricks Tests](https://github.com/Tomoscorbin/delta-engine/actions/workflows/live.yaml/badge.svg?branch=main)](https://github.com/Tomoscorbin/delta-engine/actions/workflows/live.yaml)
+
+Declarative, safety-first schema and metadata management for Delta Lake tables on Databricks.
+
+Define the state your tables should have in Python. Delta Engine reads their current Unity Catalog state, calculates the difference, validates whether each change is safe, and executes only the DDL needed to reconcile them.
+
+There are no migration scripts to maintain and no DDL statements to hand-order. Your declarations remain the source of truth across the lifetime of the table.
+
+## What Delta Engine gives you
+
+- **Desired-state reconciliation:** declare the complete table state rather than a sequence of migrations.
+- **Safe in-place evolution:** unsafe changes, such as type narrowing, repartitioning or destructive schema changes, are rejected before any SQL runs.
+- **Reviewable plans:** preview semantic changes and compiled DDL with a dry run before applying them.
+- **Drift detection:** compare version-controlled declarations against the live Unity Catalog state in local workflows or CI.
+- **Scoped ownership:** manage governance metadata without taking ownership of a table’s schema or data lifecycle.
+- **Dependency-aware execution:** synchronise groups of tables in the correct order when primary-key and foreign-key relationships exist.
+
+Delta Engine does not need to own how a table’s data is produced. It can manage the catalog state around tables populated by PySpark jobs, declarative pipelines, dbt models or other systems.
+
+## Install
+
+```bash
+pip install delta-engine
+```
+
+The base package is pure Python with no runtime dependencies: declaring and
+planning schemas needs no PySpark. Running a sync needs either a Spark
+session or a Databricks SQL warehouse connection. Inside a Databricks
+notebook, Spark is already provided; elsewhere, install the `[spark]` extra
+for local development against a Spark session, or the `[sql]` extra to sync
+through a SQL warehouse instead — schema sync from CI without a Spark session
+or cluster. See the
+[installation guide](https://tomoscorbin.github.io/delta-engine/installation.html)
+for both extras and their requirements.
+
+Install `delta-engine[cli]` to run
+`delta-engine plan MODULE:ATTRIBUTE` through any standard Databricks
+unified-auth configuration. The CLI always shows the semantic diff, report,
+and planned SQL and never applies the generated plan. See the
+[CLI reference](https://tomoscorbin.github.io/delta-engine/reference-cli.html).
+
+## Quickstart
+
+```python
+from delta_engine.databricks import build_spark_engine
+from delta_engine.schema import Column, DeltaTable, Integer, String
+
+customers = DeltaTable(
+    catalog="dev",
+    schema="silver",
+    name="customers",
+    columns=[
+        Column("id", Integer(), nullable=False),
+        Column("name", String()),
+    ],
+)
+
+engine = build_spark_engine(spark)  # `spark` is provided by your Databricks notebook
+engine.sync(customers)         # creates the table, or no-ops if it already matches
+```
+
+Every sync follows the same process:
+
+```
+read live state
+    → compare with the declaration
+    → validate the differences
+    → build a deterministic plan
+    → resolve table dependencies
+    → execute
+```
+
+Validation happens before execution. When a table contains an unsafe change, Delta Engine does not execute a partially valid plan for that table.
+
+## Documentation
+
+Start with [how a sync works](https://tomoscorbin.github.io/delta-engine/explanation-sync-lifecycle.html)
+for the model, or jump to what you need:
+
+**Getting started**
+
+- [Installation](https://tomoscorbin.github.io/delta-engine/installation.html)
+- [Getting started tutorial](https://tomoscorbin.github.io/delta-engine/tutorial-getting-started.html) — define a table and run your first sync
+
+**Concepts**
+
+- [How a sync works](https://tomoscorbin.github.io/delta-engine/explanation-sync-lifecycle.html) — the phases between calling `sync` and getting a report
+- [The safety model](https://tomoscorbin.github.io/delta-engine/explanation-safety-model.html) — what the engine blocks, and why
+
+**How-to guides**
+
+- [Configure a table](https://tomoscorbin.github.io/delta-engine/how-to-configure-table.html) — properties, tags, comments, keys, and partitioning
+- [Deploy metadata only](https://tomoscorbin.github.io/delta-engine/how-to-deploy-metadata-only.html) — roll out governance metadata with no schema change
+- [Preview changes with a dry run](https://tomoscorbin.github.io/delta-engine/how-to-preview-changes.html)
+- [Gate schema changes in CI](https://tomoscorbin.github.io/delta-engine/how-to-gate-changes-in-ci.html) — report planned changes and fail unreadable or unsafe plans
+- [Handle sync failures](https://tomoscorbin.github.io/delta-engine/how-to-handle-sync-failures.html) — inspect `SyncReport` and act on each status
+
+**Reference**
+
+- [CLI](https://tomoscorbin.github.io/delta-engine/reference-cli.html) — the read-only plan command, connection contract, output, and exit codes
+- [Capabilities and limitations](https://tomoscorbin.github.io/delta-engine/reference-limitations.html) — what the engine can and cannot manage
+- [Data types](https://tomoscorbin.github.io/delta-engine/reference-data-types.html) — supported types and Spark SQL equivalents
+- [Safe-change rules](https://tomoscorbin.github.io/delta-engine/reference-safe-change-rules.html) — changes the engine blocks at validation
+- [Run report schema](https://tomoscorbin.github.io/delta-engine/reference-run-report.html) — the `to_dict()` payload, field by field
+- [API reference](https://tomoscorbin.github.io/delta-engine/reference-api.html)
+
+**Architecture**
+
+- [Architecture](https://tomoscorbin.github.io/delta-engine/explanation-architecture.html) — layers, ports and adapters, design decisions
+- [Implement a custom adapter](https://tomoscorbin.github.io/delta-engine/how-to-implement-adapter.html) — the `CatalogStateReader` and `PlanExecutor` ports
+- [Add a new action type](https://tomoscorbin.github.io/delta-engine/how-to-add-action-type.html) — extend `Action`, `ActionPhase`, and the compiler