deltaplan 0.1.0a1__tar.gz

deltaplan-0.1.0a1/.gitignore

@@ -0,0 +1,17 @@
+.venv/
+__pycache__/
+*.pyc
+dist/
+build/
+*.egg-info/
+.DS_Store
+
+# tool caches
+.pytest_cache/
+.ruff_cache/
+
+# mkdocs build output
+site/
+
+# local plan artefacts
+plan.json

deltaplan-0.1.0a1/CHANGELOG.md

@@ -0,0 +1,270 @@
+# Changelog
+
+All notable changes to this project are documented here. The format is based on
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
+adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0a1] - 2026-09-18
+
+The first public release: an alpha. Everything in the design is built and tested
+offline, against a fake warehouse that interprets deltaplan's own SQL. The live
+suite has only just started running against a real workspace — its first run
+found a wrong assumption about `information_schema`, fixed here — so expect more
+of those before 0.1.0.
+
+### Added
+
+- **Table renames**: `renamed_from:` on a table plans `ALTER TABLE … RENAME TO`
+  as its first step, and the table's other changes follow under the new name.
+  The old name is never treated as an orphan, so a strict schema renames rather
+  than drops. `apply` checks for staleness under the name the table was read by.
+- **Asset Bundles.** `bundle: databricks.yml` in `deltaplan.yml` takes the
+  targets from the bundle: names, the default, each target's workspace, and
+  its variables (defaults, overrides, `BUNDLE_VAR_*`, `${var.…}` and
+  `${bundle.target}` references, `include:` files). A `warehouse_id` lookup is
+  resolved by name once connected. Variables that only a workspace could
+  resolve are reported, with the reason, when a spec uses one.
+- Specs may write a variable as `${var.name}`, as bundles do.
+- A target can be marked `default: true`; `-t` is then optional.
+- **SQL functions** (`function:` specs): parameters, return type, body,
+  comment and `EXECUTE` grants. Created before the tables and views that call
+  them, replaced when their definition changes (grants put back), never
+  dropped. Introspected from `information_schema.routines` and imported.
+- **Foreign keys** (`foreign_key:` constraints), introspected, diffed, planned
+  and imported. They are planned after every table, so the table they reference
+  exists first, and matched by what they mean rather than only by name.
+- Plan files now read back a table's hooks; they were written but dropped on
+  the way in.
+- **Identity, generated and default columns** (`identity:`, `generated:`,
+  `default:`). `CREATE TABLE` has all three. Defaults can be set, changed and
+  dropped later, with the `allowColumnDefaults` feature enabled first as its own
+  step; identity and generated columns exist only from creation, so adding or
+  changing one on an existing table is a step deltaplan won't run, with the
+  reason. Rewrites carry defaults; tables with identity or generated columns
+  are never rewritten. Introspected and imported, so an imported table
+  re-creates faithfully.
+- `plan` and `drift` note a `renamed_from` hint that has done its job and can
+  be deleted. (The design puts this in `validate`, which can't see the live
+  table.) A table with only notes still reads "No changes".
+- **Backfills.** `using:` on a column being added fills the existing rows
+  (`UPDATE … WHERE col IS NULL`) before `SET NOT NULL` — so a NOT NULL column
+  can be added to a table with data. Without it, the plan warns and says what to
+  add.
+- **Table hooks** (`hooks: {before, after}`), the design's simple pre/post SQL
+  hooks: run as written around a table's changes, only when it has some.
+- **Schemas are created when a spec needs them**, once each, just before the
+  first table or view in them — so a fresh target plans from nothing. Catalogs
+  are never created, and schemas are never dropped. The history schema is
+  created the same way.
+
+- **A target can name its workspace**: `profile:` on a target picks a
+  `~/.databrickscfg` profile, and `--profile` overrides it on every command that
+  connects — dev and prod are usually different workspaces.
+- A runbook for the live test suite in the testing guide.
+- The terminal plan ends with the same warnings the pull-request comment
+  raises: that it destroys something, or has steps `apply` will refuse.
+- **Milestone 5 (governance) is complete.**
+- **Views.** A spec with `view:` and a `query:` describes a view. The query is
+  what is compared — whitespace aside — and a change replaces the view, with
+  its tags and grants put back as they were, and the old definition as undo.
+  Views are planned after tables and after the views they read; a cycle is an
+  error. Views can be claimed, dropped in a strict schema, and `import`ed.
+- A table is never turned into a view or the reverse; planning stops instead.
+- Materialized views and streaming tables are recognised and skipped — they
+  report their storage as Delta, and would otherwise have been treated as
+  tables.
+- **Column masks and row filters**, handled as security controls: set or
+  replaced when the spec declares them, never removed because a spec is silent,
+  inline in `CREATE TABLE` so a new table is never unprotected, refused up front
+  if the function is missing, and never rewritten — the staging copy would hold
+  possibly unmasked data.
+- A step's precheck now carries its own `refusal`, so a refused step says
+  exactly why ("the masking function … does not exist", "ssn still has NULLs").
+- **Grants** (`grants:` on a table). A principal the spec names gets exactly
+  the privileges listed — granted or revoked to match, each revoke with a
+  warning and its undo; principals it doesn't name are left alone. Privileges
+  are checked against a known list, because as keywords they can't be quoted.
+  A rewrite puts back grants to principals the spec doesn't name.
+- **Column tags** (`tags:` on a column), additive like table tags. A rewrite
+  puts back the table and column tags the spec doesn't declare, so rebuilding a
+  table never diffs away what deltaplan doesn't manage.
+
+### Fixed
+
+- A new view was headed `~ update` and counted as a change rather than an add.
+- `plan -o plan.txt` also printed the whole plan to stdout.
+- **A rewrite could drop a column without `--allow-destructive`.** A rewrite
+  copies only the columns the spec lists, so a column the spec also removed went
+  with it — inside a step classed `rewrite`, which the flag doesn't gate. The
+  step that replaces the table is now `destructive` whenever the rewrite drops a
+  column or field, and names it.
+- **A lossy conversion could NULL values silently.** The staging step now
+  checks that every row arrived and no converted column gained NULLs, and stops
+  the run — before the original table is touched — if either did.
+- **A rewrite dropped properties and constraints nobody declared** — a
+  retention setting such as `delta.logRetentionDuration`, a CHECK or primary key
+  someone else added. The replacement now carries them across, as it already
+  did tags and grants; a view replace carries its properties the same way.
+- **Partitioning, identity and generated columns, and column defaults went
+  unnoticed** — and a rewrite would have dropped them (an identity column coming
+  back as a plain BIGINT). They are now read from the catalog, reported as not
+  modelled, and a table that has any is never rewritten.
+- **`import` wrote properties Delta maintains itself** — among them
+  `delta.columnMapping.maxColumnId`, which every later plan would then have set
+  back to a stale value as columns were added. Import now writes intent only,
+  `validate` refuses a spec that declares a Delta-maintained property, and
+  `delta.feature.*` flags are no longer reported as unmanaged.
+- A failed postcheck reports what it means, not a generic message.
+- The live suite skipped entirely unless `DATABRICKS_HOST` was set, so anyone
+  authenticating with a profile would never have run it. It now accepts any
+  source the Databricks SDK does, and says why when it skips.
+- Failing to connect to a workspace is a message naming the profile, not a
+  traceback.
+
+- **A differently-cased name could drop a live table.** A spec naming
+  `main.sales.Orders` against the live `orders`, in a strict schema, planned a
+  no-op create and a DROP of the real table. Names are now compared the way Unity
+  Catalog compares them: object names in lower case, column and field names
+  ignoring case.
+- **The apply lock could expire under a long run.** It is now renewed before
+  every step, and a run that finds it has lost the lock stops rather than carry
+  on beside a second one.
+- A precheck whose own query fails is recorded as a failed step, instead of
+  escaping as a traceback; warehouse errors during `apply` and `force-unlock` are
+  reported as messages.
+
+### Before the first release
+
+deltaplan was built in milestones before anything was published. Their
+numbers were internal and never released; what each added is kept here.
+
+#### Milestone 4
+
+##### Added
+
+- **Milestone 4 (CI) is complete.**
+- `--format md`: the plan as a pull-request comment — summary, GitHub alerts for
+  anything destructive, expensive or impossible, a `diff` block per table so
+  additions and removals are coloured, the numbered steps with their risk, and
+  the SQL folded away. Falls back to leaving out the SQL, then to a summary
+  table, when a plan is too long for a comment.
+- `deltaplan show plan.json`: render a saved plan in any format without a
+  warehouse — exactly what `apply` of that file would run.
+- `deltaplan drift`: exits 0 in sync, 2 on drift, 1 on error. Drift is anything
+  `apply` would do; unmanaged objects are not drift.
+- A GitHub Action (`uses: misja-pronk/deltaplan@v0`) that runs `plan` or
+  `drift`, writes the job summary, and comments on the pull request — updating
+  its own comment rather than adding one per push. Inputs reach its script
+  through the environment, never by interpolation, and a test holds it to that.
+- CI lints the workflows with actionlint; releases move the major-version tag
+  the Action is used by.
+
+#### Ownership and strict schemas
+
+##### Added
+
+- **Ownership is claimed.** A spec for a table deltaplan didn't create plans a
+  visible `CLAIM ownership` step that marks it managed — which is how an
+  `import`ed table is handed over on its first apply.
+- **Strict schemas.** A managed table whose spec was deleted is dropped in a
+  strict schema (destructive, so `--allow-destructive` applies, with `UNDROP`
+  as the way back) and kept — but listed — in an additive one. Tables deltaplan
+  didn't create are never touched in either mode.
+- `schemas:` in `deltaplan.yml` sets the mode per schema, as the design
+  specifies; the target's `mode` is the default.
+- `history_schema` and `schemas:` keys may use target variables
+  (`${catalog}.deltaplan`), so one project file serves every catalog.
+- `deltaplan plan --clone` adds a `SHALLOW CLONE` of each table before the first
+  step that could lose its data.
+- `planning.py`: the specs-to-plan pipeline, out of the CLI, so `plan`, `drift`
+  and the GitHub Action share it.
+
+##### Changed
+
+- The plan summary counts destroyed tables; it was hard-coded to zero.
+
+#### Milestone 3
+
+##### Added
+
+- **Milestone 3 (rewrites) is complete**: a table that can't be patched is
+  rebuilt, and `apply` runs it.
+- A rewrite stages the converted data beside the table, **replaces** the table
+  from that staging table (keeping its identity and Delta history, so the
+  recorded restore point means something, and with no window where the table is
+  empty), puts back what a query result can't carry — `NOT NULL`, comments, tags,
+  constraints — with ordinary `ALTER`s, and drops the staging table.
+- deltaplan writes the conversion where it honestly can: a cast between scalars,
+  `named_struct` matched **by name** rather than by position, `transform` over an
+  array of structs, and `CAST(NULL AS …)` for a column that didn't exist.
+- `using:` on a column — a SQL expression over the live table — for conversions
+  deltaplan won't invent: a struct becoming an array, a map whose shape moved, or
+  any change that needs a decision rather than a cast.
+- The plan file now carries both sides of each diff, so it records what was
+  compared and a rewrite knows what it is rebuilding into.
+
+##### Changed
+
+- `apply` no longer refuses plans containing rewrites. It still refuses any plan
+  with a step deltaplan couldn't generate, naming the step and what it needs.
+
+##### Fixed
+
+- Table-level changes (properties, tags) were rendered one level too deep, as
+  though they were nested inside a column.
+
+#### Milestone 2
+
+##### Added
+
+- **Milestone 2 (apply) is complete**: `deltaplan apply plan.json` and
+  `deltaplan force-unlock`.
+- Executor with the design's four promises: a fresh run refuses a stale plan
+  (recomputed state fingerprint), steps are skipped when the change they
+  implement is already true of the live table, a failed run resumes from the
+  history table instead of starting over, and a lock row per target keeps two
+  applies apart. A restore point is recorded before every destructive step.
+- Run history in Delta tables (`runs`, `steps`, `lock`) in the schema named by
+  `history_schema`, created on first use.
+- The plan file is now read as well as written, so `apply` consumes exactly what
+  `plan` produced — asserted by a round-trip test.
+- A fake warehouse (`tests/fake_warehouse.py`) that interprets deltaplan's own
+  SQL against in-memory models, so `plan → apply → re-plan is empty` is asserted
+  offline for every kind of change. See [docs/testing.md](docs/testing.md).
+
+##### Fixed
+
+- The table features deltaplan enables itself as prerequisites
+  (`delta.columnMapping.mode`, `delta.enableTypeWidening`) are no longer reported
+  back as unmanaged properties after an apply.
+
+#### Milestone 1
+
+##### Added
+
+- **Milestone 1 (read-only) is complete**: `validate`, `import` and `plan`.
+- Type tree and parser for Databricks type strings, including nested
+  struct/array/map, decimals, backticked field names, and `not null` / `comment`
+  inside structs.
+- YAML loader with `${var}` substitution per target, both type notations, and
+  errors that carry file, line and column — including for unknown keys.
+- A `deltaplan.yml` project file: where specs live, and what each target
+  substitutes.
+- Pure differ: recursive diff at Databricks' nested paths, declared renames via
+  `renamed_from`, kind-change detection, and opt-in column-order diffing.
+- Pure planner: changes become ordered steps classified `meta` / `feature` /
+  `rewrite` / `destructive`, with column mapping and type widening inserted as
+  their own prerequisite steps, and a conservative widening matrix.
+- Renderers: the terminal layout from the design document, and JSON for
+  `-o plan.json`.
+- Introspection of live Unity Catalog state through `information_schema` and
+  `DESCRIBE DETAIL`, plus a live integration suite that asserts the Databricks
+  behaviour the planner relies on.
+- Project scaffold: uv + hatchling packaging (src layout, Apache-2.0), mise tasks,
+  ruff + ty configuration, pytest with a `unit` / `integration` split, and CI for
+  lint, types, tests, docs and the built wheel.
+- `docs/DESIGN.md` as the source of truth, plus a mkdocs-material site published to
+  GitHub Pages.
+- A `deltaplan version` command, so the packaging is testable end to end.

deltaplan-0.1.0a1/CLAUDE.md

@@ -0,0 +1,134 @@
+# CLAUDE.md
+
+`deltaplan`: declarative plan/apply for Databricks SQL tables. Read `docs/DESIGN.md` first; it is the source of truth. If code and design disagree, flag it instead of silently picking one.
+
+## Rules
+
+- Differ and planner are pure: no I/O, no SDK imports, no clock, no env.
+- Domain model: frozen, slotted stdlib dataclasses with tuples. Pydantic/msgspec only in the loader.
+- Never generate SQL by string-concatenating unquoted identifiers. One `quote_ident()` helper, used everywhere.
+- Anything not modelled on a live table is reported as unmanaged, never diffed away.
+- No destructive step without the `destructive` risk class.
+- Every Databricks behaviour assumption gets a test and a link to the docs in the test docstring. If unsure about a behaviour, say so and add a `TODO(verify)` — do not guess.
+- Small PR-sized commits, conventional commit messages.
+
+## Layout
+
+```
+src/deltaplan/
+  model/        types.py, table.py, change.py, plan.py
+  typeparser.py
+  loader.py
+  introspect.py
+  differ.py
+  planner.py
+  render/       rich.py, markdown.py, json.py
+  executor.py   (milestone 2)
+  cli.py
+tests/
+  unit/
+  integration/
+  snapshots/
+docs/           DESIGN.md (source of truth) + the mkdocs site
+examples/
+```
+
+## Commands
+
+Tooling is mise + the Astral stack (uv, ruff, ty) — same as `isolinear`. Never use
+pip/virtualenv, black/flake8/isort, or mypy.
+
+```
+mise install                   # pinned Python + uv
+uv sync
+uv run pytest tests/unit
+uv run pytest -m integration   # needs DATABRICKS_HOST / token / warehouse id
+uv run ruff check . && uv run ruff format --check .
+uv run ty check
+```
+
+`mise run check` is the full gate (lint + format check + types + unit tests);
+`mise tasks` lists the rest. Docs: `mise run docs` (mkdocs-material, published to
+Pages). Releases are version-driven from `pyproject.toml` — see CONTRIBUTING.md.
+
+## Milestone 1 (read-only) — done
+
+All eight items are built, lint/type/test clean, with golden plans in
+`tests/snapshots/` and a live suite in `tests/integration/`. Deliberate
+departures from this document, each explained in the commit that made it:
+`ty` instead of pyright; a hand-written validator in the loader rather than
+Pydantic/msgspec, so errors carry file:line:column; `deltaplan.yml` invented for
+the project/target config; nested YAML types extended from struct to array and
+map; `sql.py` added for `quote_ident()`; rewrites classified but not generated (milestone 3); the Markdown
+renderer deferred to milestone 4.
+
+**Milestone 2 (apply) is done too**: `executor.py`, `history.py` (the design's
+runs/steps/lock tables), `apply` and `force-unlock`. Two more departures worth
+knowing: the design's per-step precheck/postcheck queries are replaced by
+`differ.is_applied()`, which asks the *model* whether a change is already true
+of the live table — it reuses tested code instead of inventing SQL we cannot
+verify (`precheck` survives as a precondition guard, e.g. SET NOT NULL); and
+`history.py` is a new module the layout above doesn't list, because putting the
+history tables inside `executor.py` would have made one file do two jobs.
+
+Testing without a workspace: `tests/fake_warehouse.py` is an in-memory catalog
+that interprets deltaplan's own SQL, so plan → apply → re-plan can be asserted
+offline for every change kind. It proves our SQL matches our intent; only
+`tests/integration/` proves Databricks agrees. Read `docs/testing.md` before
+adding a test, and keep the fake's `FakeSqlError` loud — a statement shape it
+doesn't know must fail, not pass.
+
+**Milestone 3 (rewrites) is done.** A table with a rewrite-class change is
+rebuilt whole rather than patched: stage the converted data, REPLACE the table
+from the staging table (identity and history kept, no empty window), put back
+what a query result can't carry with ordinary ALTERs, drop the staging table.
+Two departures worth knowing: the design's single `CREATE OR REPLACE TABLE …
+AS SELECT` is staged in two statements, because a self-referencing RTAS is
+unverified (TODO(verify) in `replace_table_sql`) and staging makes the expensive
+step repeatable; and `using:` is a new spec hint for conversions deltaplan won't
+invent. A rewrite cannot set NOT NULL on a nested field — it refuses rather than
+dropping the constraint.
+
+**The rest of the design's safety model is in too**: ownership claims (a spec
+for someone else's table plans a visible `claim_table`), strict schemas (managed
+tables whose spec is gone become `drop_table`, destructive), and
+`plan --clone` for a SHALLOW CLONE before risky steps. The specs-to-plan
+pipeline lives in `planning.py` (a module the layout above doesn't list),
+because `plan`, `drift` and the Action all need it. One departure: the design
+says the mode is per schema; `deltaplan.yml` has a per-schema `schemas:` map
+*and* keeps the target's `mode` as the default for unlisted schemas.
+
+**Milestone 4 (CI) is done**: `render/markdown.py`, `show`, `drift` (exit 0/2/1),
+and a composite GitHub Action — `action.yml` at the repo root, with its comment
+script in `action/upsert_comment.py` (stdlib only, tested against a fake API).
+Change labels are shared by both renderers in `render/labels.py`. The Action
+must never interpolate `${{ }}` into a `run:` script — `test_action.py`
+enforces it.
+
+**Milestone 5 (governance) is done**: column tags, grants (per principal),
+column masks and row filters (additive, never removed, never rewritten), and
+views (`model/view.py`; `Relation = Table | View`; tables and views share
+`Securable`). Every milestone in DESIGN.md is built.
+
+Since then, beyond the design: schema creation, hooks and backfills, identity /
+generated / default columns, foreign keys, SQL functions (`model/function.py`;
+`Relation = Table | View | Function`) and Asset Bundle targets (`bundle.py`, a
+new module the layout doesn't list: it reads someone else's YAML leniently,
+which `loader.py`'s strict validator shouldn't). External tables are out of
+scope for now — managed tables only.
+
+What remains is verification, not construction: nothing has run against a real
+workspace. Every `TODO(verify)` in `src/` names an assumption the live suite in
+`tests/integration/` is written to settle — run it (`uv run pytest -m
+integration`) before trusting any of this with production tables.
+
+1. ~~Scaffold: `pyproject.toml` (uv, src layout, Apache-2.0), ruff, ty, pytest, GitHub Actions for lint + unit tests, README stub, move `DESIGN.md` to `docs/`.~~ **Done.**
+2. `model/types.py` + `typeparser.py`: type tree and parser for Databricks type strings incl. nested struct/array/map, decimal, backticked field names, `NOT NULL` and comments inside structs. Round-trip tests.
+3. `model/table.py` + `loader.py`: YAML → model, both type notations, `${var}` substitution, strict unknown-key errors with file/line paths.
+4. `differ.py`: recursive diff with paths, `renamed_from` handling, kind-change detection. Snapshot tests.
+5. `planner.py`: changes → steps with risk classes and prerequisite steps (column mapping, type widening). SQL generation. Snapshot tests.
+6. `render/rich.py`: the plan layout from the design doc, nested changes as a tree.
+7. `introspect.py`: live state via `databricks-sdk` Statement Execution API. Integration tests with ephemeral schema.
+8. `cli.py`: `validate`, `import`, `plan`.
+
+Stop after each numbered item and summarise what was built and what was assumed.

deltaplan-0.1.0a1/CONTRIBUTING.md

@@ -0,0 +1,130 @@
+# Contributing to deltaplan
+
+Thanks for your interest! Issues and pull requests are very welcome.
+
+`deltaplan` is pre-alpha: the read-only milestone (`validate`, `import`, `plan`) is
+still being built. [`docs/DESIGN.md`](docs/DESIGN.md) is the source of truth — if the
+code and the design disagree, that is a bug in one of them, so please say which.
+
+## Toolchain
+
+deltaplan uses [`mise`](https://mise.jdx.dev) to pin tools and the all-Astral
+stack — [`uv`](https://docs.astral.sh/uv/) (env / deps / run),
+[`ruff`](https://docs.astral.sh/ruff/) (lint + format), and
+[`ty`](https://docs.astral.sh/ty/) (type check).
+
+```sh
+mise install   # installs the pinned Python + uv (optional but recommended)
+uv sync        # creates .venv and installs deps + dev tools
+```
+
+## Day-to-day
+
+```sh
+uv run deltaplan                  # run the CLI
+uv run pytest tests/unit          # fast tests, no workspace needed
+uv run ruff check . && uv run ruff format .   # lint + format
+uv run ty check                   # type check
+```
+
+`mise run check` runs the whole gate (lint, format check, types, unit tests) in one
+go; `mise tasks` lists the rest.
+
+All of these run in CI on every push/PR — please make sure they're green before
+opening a PR. New behaviour should come with a test.
+
+## Architecture
+
+```
+spec (YAML) ─┐
+             ├─> differ ─> changes ─> planner ─> plan (JSON) ─> renderer
+live (UC) ───┘                                        │
+                                                      └─> executor ─> history
+```
+
+The dependency rule is simple: **the middle of the pipeline does no I/O.**
+
+- **`model/`** — frozen, slotted stdlib dataclasses holding tuples, so everything is
+  hashable. No Pydantic, no SDK.
+- **`loader.py` / `introspect.py`** — the only places that read YAML or talk to a
+  workspace. Validation happens at this edge and nowhere else.
+- **`differ.py` / `planner.py`** — pure functions: no I/O, no SDK imports, no clock,
+  no environment. They must be unit-testable without a workspace.
+- **`render/`** — rich / markdown / json views of the same `Plan` object, sharing
+  their wording through `render/labels.py`.
+- **`planning.py`** — specs and a warehouse in, a plan out: the pipeline `plan`,
+  `drift` and the GitHub Action share.
+- **`executor.py`** / **`history.py`** — the only places that run SQL that changes
+  anything.
+- **`action.yml`** + **`action/`** — the GitHub Action, at the repo root so
+  `uses: misja-pronk/deltaplan@v0` finds it.
+
+House rules worth repeating:
+
+- Never build SQL by concatenating unquoted identifiers — there is one
+  `quote_ident()` helper, and it is used everywhere.
+- Anything not modelled on a live table is reported as **unmanaged** and never
+  diffed away. Only tables deltaplan created can be drop candidates.
+- No destructive step without the `destructive` risk class.
+- Every Databricks behaviour assumption gets a test and a link to the docs in the
+  test docstring. If a behaviour is unclear, add a `TODO(verify)` and say so — don't
+  guess.
+
+## Tests
+
+Three layers, each honest about what it proves:
+
+1. **Unit tests** (`tests/unit/`) — the pure middle of the pipeline, with golden plans
+   in `tests/snapshots/`.
+2. **Convergence against a fake warehouse** (`tests/fake_warehouse.py`) — an in-memory
+   catalog that interprets the statements the planner generates, so `plan → apply →
+   re-plan is empty` can be asserted offline, for every kind of change. It proves our
+   SQL means what our changes mean; it cannot prove Databricks accepts it.
+3. **Integration tests** (`tests/integration/`) — marked `@pytest.mark.integration`,
+   skipped without credentials, run nightly against a real workspace in an ephemeral
+   schema. The only source of truth about Databricks.
+
+Every discovered Databricks limitation becomes a test in layer 3, with a link to the
+documentation in its docstring. There is a fuller description in
+[docs/testing.md](docs/testing.md).
+
+## Commits & PRs
+
+- Small, PR-sized commits with [conventional commit](https://www.conventionalcommits.org/)
+  messages (`feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`).
+- Describe the *why*, not just the *what*.
+- By contributing you agree your work is licensed under the project's
+  [Apache-2.0 License](LICENSE).
+
+## Releasing
+
+Releases are **version-driven**: the `version` in `pyproject.toml` is the single
+source of truth, and merging a bump to `main` ships it. No manual tagging.
+
+1. On a branch, bump the version:
+
+   ```sh
+   uv version --bump patch   # or: minor / major — edits pyproject.toml + uv.lock
+   ```
+
+2. In `CHANGELOG.md`, rename the `## [Unreleased]` heading to `## [X.Y.Z]` (the
+   new version) and start a fresh, empty `## [Unreleased]` above it. Those notes
+   become the GitHub release body.
+3. Open a PR. When it merges to `main`, the [`release`](.github/workflows/release.yml)
+   workflow builds the wheel + sdist, publishes to **PyPI** (via Trusted Publishing —
+   no API token), and creates the **`vX.Y.Z`** tag and GitHub release.
+
+A merge that doesn't change the version is a no-op, and a version that's already
+tagged or already on PyPI is skipped — so the workflow is safe to re-run.
+
+> **One-time setup.** Releasing is deliberately dormant until deltaplan is public.
+> Two things switch it on: a [PyPI Trusted Publisher](https://docs.pypi.org/trusted-publishers/)
+> for repository `misja-pronk/deltaplan`, workflow `release.yml`, environment `pypi`
+> (at <https://pypi.org/manage/account/publishing/>), and the repository variable
+> `RELEASE_ENABLED=true`.
+
+## Previewing the docs
+
+```sh
+uv run --group docs mkdocs serve
+```