dbly 0.1.0__tar.gz → 0.3.0__tar.gz

dbly-0.3.0/.github/FUNDING.yml

@@ -0,0 +1 @@
+ko_fi: angrydata

dbly-0.3.0/CHANGELOG.md

@@ -0,0 +1,92 @@
+# Changelog
+
+All notable changes to `dbly` are documented here.
+
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and this project
+adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.3.0] — 2026-07-21
+
+### Added
+
+- **Plan against the working tree.** `dbly plan --worktree` (alias `--dirty`) and
+  `dbly check --worktree` diff the *working directory* — including uncommitted edits and
+  untracked new object files — instead of a git ref, for the fast edit→plan loop. Preview
+  only; `apply` still requires a real committed ref for the ledger.
+- **Git-style ref decoration.** `dbly status` and the plan header now show the tag/branch
+  names pointing at a SHA next to it — e.g. `deployed ref: v0.1, main (a712b631)`. The
+  ledger still stores only the SHA (for stable diffs); the names are resolved at display time.
+
+### Fixed
+
+- **Column type changes are now detected.** A changed column type (e.g. Oracle `NUMBER` →
+  `NUMBER(10)`) previously produced "nothing to do" — the diff matched columns by name only.
+  It now emits an `ALTER TABLE … MODIFY`/`ALTER COLUMN` step (flagged destructive, never
+  auto-applied) with a data-compatibility warning. Comparison is precision/scale-aware and
+  normalizes introspection noise (e.g. SQL Server `COLLATE`) to avoid false positives.
+- **`check` no longer reports live Oracle objects as "missing".** The Oracle adapter now
+  resolves its `default_schema` to the connected user and carries the owner on introspected
+  objects, so drift keys align with the desired side (previously the owner was dropped on the
+  live side, so every object looked missing even right after a successful `apply`).
+
+### Changed
+
+- The plan output now surfaces each step's **note inline** (e.g. *"NOT NULL without default
+  on existing table — unsafe"*), so *why* a step is flagged is visible without cross-checking
+  the warnings block.
+
+## [0.2.0] — 2026-06-28
+
+### Added
+
+- **Explicit, run-once migrations.** Drop ordered SQL scripts in `migrations/` (`0001_…sql`)
+  for changes the additive diff cannot do safely — renaming a column, type changes with data
+  transformation, backfills. Each is tracked by id in the `dbly_state` ledger and runs
+  exactly once.
+- On **upgrade**, pending migrations run *before* the object reconciliation, so they reshape
+  the schema first. On a **fresh database** they are *baselined* (recorded, not run) since the
+  canonical object files already describe the end state.
+
+### Changed
+
+- A table touched by a pending migration **defers its additive diff** for that deploy, so an
+  explicit rename no longer collides with an auto-generated `ADD COLUMN`.
+- `migrations/` files are excluded from object and drift discovery (they are imperative
+  scripts, not declarative objects).
+
+## [0.1.0] — 2026-06-27
+
+### Added
+
+- **Indexes and sequences as first-class managed objects.** Correct identity (index name +
+  indexed-table schema), dependency-safe ordering (sequences → tables → indexes), and
+  create-if-missing handling (no more erroneous re-apply of non-idempotent
+  `CREATE INDEX`/`CREATE SEQUENCE`).
+- **Live inventory introspection** across tables, views, functions, procedures, triggers,
+  indexes and sequences for all four engines, with a canonical source hash for procedural
+  objects.
+- **Real drift detection in `dbly check`** — reports missing (in repo, not in DB), orphaned
+  (`--orphans`), table column drift, and advisory definition drift; schema-normalized so an
+  unqualified repo object matches the engine's implicit schema.
+
+## [0.0.1] — 2026-06-27
+
+### Added
+
+- Initial release. **State-based, cross-engine database deployment** (PostgreSQL, SQL Server,
+  Oracle, SQLite), git-driven and parser-assisted (sqlglot).
+- `plan` / `apply` workflow with additive table diffs generated from
+  `CREATE TABLE IF NOT EXISTS` (destructive changes flagged, never auto-applied).
+- `plan --sql` — exports a self-contained SQL script for a hand/offline deploy.
+- `dbly init` — privileged greenfield groundwork (`CREATE DATABASE`/roles/extensions).
+- Pre-/post-deploy hooks accepting `.sql` and `.py` (configurable interpreter, e.g. ArcPy).
+- Connection profiles (DBFit-compatible `connection.properties`) with `${ENV}` placeholders
+  for CI/CD.
+- CI and PyPI publish workflows (trusted publishing).
+
+[Unreleased]: https://github.com/angrydat/dbly/compare/v0.2.0...HEAD
+[0.2.0]: https://github.com/angrydat/dbly/compare/v0.1.0...v0.2.0
+[0.1.0]: https://github.com/angrydat/dbly/compare/v0.0.1...v0.1.0
+[0.0.1]: https://github.com/angrydat/dbly/releases/tag/v0.0.1

{dbly-0.1.0 → dbly-0.3.0}/PKG-INFO

@@ -1,10 +1,11 @@
 Metadata-Version: 2.4
 Name: dbly
-Version: 0.1.0
+Version: 0.3.0
 Summary: State-based, cross-engine database deployment — git-driven, parser-assisted, SQL-first.
 Project-URL: Homepage, https://angrydata.info/dbly
 Project-URL: Repository, https://github.com/angrydat/dbly
 Project-URL: Issues, https://github.com/angrydat/dbly/issues
+Project-URL: Changelog, https://github.com/angrydat/dbly/blob/main/CHANGELOG.md
 Author-email: Jürgen Zornig <info@angrydata.info>
 License: MIT
 Keywords: ci-cd,database,deployment,git,migration,oracle,postgres,schema,sql,sqlite,sqlserver,state-based
@@ -81,10 +82,17 @@ db/
     v_open_orders.vw
     grants.sql
   init/                 # optional: privileged greenfield groundwork (CREATE DATABASE/ROLE…)
+  migrations/           # optional: ordered, run-once ALTERs for renames / data backfills
 hooks/pre/  hooks/post/  # optional: .sql or .py hooks (e.g. ArcGIS/ArcPy steps)
 .dbignore               # files in the repo that should not be deployed
 ```
 
+Most changes are declarative — edit the object file, dbly figures out the additive diff. For
+changes the diff can't do safely (renaming a column, moving data), drop an ordered script in
+`migrations/` (`0001_…sql`); it runs exactly once, is recorded in the ledger, and the table
+it touches defers to it for that deploy. On a fresh database, migrations are *baselined*
+(recorded, not run) since the object files already describe the end state.
+
 ## Connect
 
 A connection profile (reuses the familiar `connection.properties` format):

{dbly-0.1.0 → dbly-0.3.0}/README.md

@@ -43,10 +43,17 @@ db/
     v_open_orders.vw
     grants.sql
   init/                 # optional: privileged greenfield groundwork (CREATE DATABASE/ROLE…)
+  migrations/           # optional: ordered, run-once ALTERs for renames / data backfills
 hooks/pre/  hooks/post/  # optional: .sql or .py hooks (e.g. ArcGIS/ArcPy steps)
 .dbignore               # files in the repo that should not be deployed
 ```
 
+Most changes are declarative — edit the object file, dbly figures out the additive diff. For
+changes the diff can't do safely (renaming a column, moving data), drop an ordered script in
+`migrations/` (`0001_…sql`); it runs exactly once, is recorded in the ledger, and the table
+it touches defers to it for that deploy. On a fresh database, migrations are *baselined*
+(recorded, not run) since the object files already describe the end state.
+
 ## Connect
 
 A connection profile (reuses the familiar `connection.properties` format):

dbly-0.3.0/TODO.md

@@ -0,0 +1,70 @@
+# dbly — TODO / Ideen-Backlog
+
+Kurzliste möglicher Verbesserungen, entstanden aus dem Vergleich mit **Atlas**
+(atlasgo.io). Das sind Funktionen, die Atlas (teils nur in der kommerziellen
+Pro-Edition) bietet und die dbly aktuell **nicht** hat — als Anregung, nicht als
+Roadmap. dbly-Stärken (frei, MIT, Oracle/MSSQL nativ, `.py`-Hooks, `plan --sql`)
+bleiben der Ausgangspunkt.
+
+## Migrations- & Sicherheits-Analyse
+- **Migration-Linting / Analyzer**: destruktive/riskante Schritte erkennen und
+  warnen bzw. blocken (Atlas `migrate lint`). dbly kennt bisher nur additiv vs.
+  destruktiv + `--allow-destructive`.
+- **Custom Schema Rules / Policies** (z. B. „jede Tabelle braucht PK", Namens-
+  konventionen, Boolean = NUMBER(1) CHECK (0,-1) erzwingen).
+- **Pre-Migration-Checks** (Vorbedingungen prüfen, z. B. Spalte leer vor NOT NULL).
+- **PII-/Security-as-Code**: Spalten als sensibel markieren; RLS deklarativ.
+
+## Sichtbarkeit
+- **ERD / Schema-Visualisierung** + Auto-Doku aus dem Objektmodell.
+- **Schema-Registry** / zentrale „source of truth" mit Versionsvergleich über
+  Umgebungen hinweg.
+- **Kontinuierliches Drift-Monitoring**: dbly hat `check` on-demand; es fehlt ein
+  laufendes/alarmierendes Monitoring.
+- ~~**Ref-Dekoration in `status` / Plan-Header** (Komfort)~~ ✅ **erledigt (0.3.0)**:
+  `status` und der Plan-Header zeigen jetzt Tag-/Branch-Namen neben dem SHA
+  (`deployed ref: v0.1, main (9ff5e440)`), aufgelöst via `git tag/branch --points-at`.
+  Der Ledger speichert weiterhin nur den SHA.
+
+## Autoren-Erlebnis
+- Zusätzliche **Schema-Sprache (HCL o. ä.)** und/oder **ORM-Loader** — DB-agnostische
+  Definition neben plain SQL.
+- **Testing-Framework** für Schema/Objekte (Atlas `atlas test`).
+- **Interaktive Migrations / Checkpoints** für lange Migrationsketten.
+- ~~**Plan gegen den Working-Tree** (uncommittete Änderungen)~~ ✅ **erledigt (0.3.0)**:
+  `dbly plan --worktree` (Alias `--dirty`) und `dbly check --worktree` planen/prüfen
+  den Arbeitsstand inkl. ungetrackter neuer Objektdateien. Preview-only — `apply`
+  braucht weiterhin einen committeten Ref für den Ledger.
+
+## Integration
+- Breitere fertige **CI-Rezepte** (GitLab, Azure DevOps, CircleCI) — aktuell nur
+  GH-Actions/Bitbucket-Beispiele.
+- **Terraform-Provider / K8s-Operator** (GitOps).
+- **Rollout-/Approval-Flow** (Freigabe vor Apply in Prod).
+
+## Bekannte Design-Frage (aus dem MA31-PoC)
+- Im Planner werden **Tabellen in Changeset-Reihenfolge** emittiert;
+  `topological_order` wird nur auf *replaceable* (Views/Functions) angewandt. Bei
+  Inline-FKs zwischen Tabellen ist auf einem frischen Ziel die Anlege-Reihenfolge
+  dann nicht FK-sicher — prüfen, ob Tabellen ebenfalls topologisch sortiert werden
+  sollten (oder ob der Adapter das per retry-until-stable abfängt).
+
+## Bugs (aus MA31-PoC, Oracle)
+
+- ~~**Spalten-Typänderung wird nicht erkannt.**~~ ✅ **erledigt (0.3.0)**: Der
+  Spalten-Diff verglich nur den Namen; `NUMBER` → `NUMBER(10)` blieb unbemerkt. Jetzt
+  wird bei geändertem Typ ein `ALTER TABLE … MODIFY`/`ALTER COLUMN`-Schritt erzeugt
+  (als `destructive` markiert, nie auto-appliziert, mit Datenkompatibilitäts-Warnung).
+  Der Vergleich ist Precision/Scale-bewusst (`NUMBER(10)` ≡ `NUMBER(10,0)`) und
+  normalisiert Introspektions-Rauschen (z. B. SQL-Server `COLLATE`) gegen False Positives.
+- ~~**ADD COLUMN fälschlich als „destructive" klassifiziert.**~~ ✅ **geklärt (0.3.0)**:
+  `TESTFELD` war `NOT NULL` **ohne** Default — die `destructive`-Einstufung ist damit
+  korrekt (kann auf einer befüllten Tabelle nicht sicher hinzugefügt werden). Verbessert:
+  der Grund (*„NOT NULL without default … — unsafe"*) wird jetzt als Hinweis direkt am
+  Plan-Schritt angezeigt, nicht nur im Warnungsblock.
+- ~~**`check` meldet vorhandene Objekte als „missing".**~~ ✅ **erledigt (0.3.0)**:
+  Ursache war eine asymmetrische Key-Normalisierung — der Oracle-Adapter setzte kein
+  `default_schema` und ließ den Owner in `inventory()` weg, sodass die Live-Keys
+  (`table:<name>`) nicht zu den Desired-Keys (`table:dbb.<name>`) passten. Der Adapter
+  löst `default_schema` jetzt zum verbundenen `USER` auf und führt den Owner mit; beide
+  Seiten normalisieren identisch.

{dbly-0.1.0 → dbly-0.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "dbly"
-version = "0.1.0"
+version = "0.3.0"
 description = "State-based, cross-engine database deployment — git-driven, parser-assisted, SQL-first."
 readme = "README.md"
 license = { text = "MIT" }
@@ -54,6 +54,7 @@ all = ["oracledb>=2.0", "pymssql>=2.3"]
 Homepage = "https://angrydata.info/dbly"
 Repository = "https://github.com/angrydat/dbly"
 Issues = "https://github.com/angrydat/dbly/issues"
+Changelog = "https://github.com/angrydat/dbly/blob/main/CHANGELOG.md"
 
 [project.scripts]
 dbly = "dbly.cli:main"

{dbly-0.1.0 → dbly-0.3.0}/src/dbly/__init__.py

@@ -4,4 +4,4 @@ git (what changed) + sqlglot (what it is) + live introspection (what's really th
 See CONCEPT.md for the design rationale.
 """
 
-__version__ = "0.1.0"
+__version__ = "0.3.0"

{dbly-0.1.0 → dbly-0.3.0}/src/dbly/adapters/base.py

@@ -13,7 +13,7 @@ from __future__ import annotations
 
 import abc
 
-from sqlalchemy import Engine
+from sqlalchemy import Engine, text
 
 from dbly.config import ConnectionConfig
 from dbly.engine import make_engine
@@ -71,6 +71,15 @@ class Adapter(abc.ABC):
         dialect-agnostic and delegates the rendering here.
         """
 
+    @abc.abstractmethod
+    def modify_column_sql(self, table: ObjectId, col: Column) -> str:
+        """Generate the ``ALTER TABLE … <MODIFY|ALTER COLUMN>`` that changes a column's type.
+
+        Oracle uses ``MODIFY``; Postgres uses ``ALTER COLUMN … TYPE``; T-SQL uses
+        ``ALTER COLUMN``. SQLite cannot change a column's type in place and returns a
+        commented no-op (a type change there needs a manual table rebuild).
+        """
+
     # --- execution ---------------------------------------------------------------------
     @abc.abstractmethod
     def apply(self, statements: list[str]) -> None:
@@ -95,6 +104,20 @@ class Adapter(abc.ABC):
     @abc.abstractmethod
     def record_deploy(self, ref: str, migration_ids: list[str]) -> None: ...
 
+    # --- explicit migrations (run-once, ledger-tracked) --------------------------------
+    def applied_migrations(self) -> set[str]:
+        """Migration ids already recorded in the ledger (standard SQL, all engines)."""
+        self.ensure_state_table()
+        with self.engine.connect() as conn:
+            rows = conn.execute(
+                text("SELECT DISTINCT migration_id FROM dbly_state WHERE migration_id IS NOT NULL")
+            )
+            return {r[0] for r in rows}
+
+    def record_migration(self, ref: str, migration_id: str) -> None:
+        """Mark a migration applied (reuses the per-engine record_deploy insert)."""
+        self.record_deploy(ref, [migration_id])
+
     # --- pure SQL builders (no connection — used by `plan --sql` export) ----------------
     @abc.abstractmethod
     def state_table_ddl(self) -> str:

{dbly-0.1.0 → dbly-0.3.0}/src/dbly/adapters/mssql.py

@@ -125,6 +125,10 @@ class MssqlAdapter(Adapter):
             parts.append(f"DEFAULT {col.default}")
         return " ".join(parts) + ";"
 
+    def modify_column_sql(self, table: ObjectId, col: Column) -> str:
+        # T-SQL ALTER COLUMN preserves nullability unless restated; type only here.
+        return f"ALTER TABLE {table} ALTER COLUMN {col.name} {col.type};"
+
     def apply(self, statements: list[str]) -> None:
         with self.engine.begin() as conn:
             for stmt in statements:

{dbly-0.1.0 → dbly-0.3.0}/src/dbly/adapters/oracle.py

@@ -89,6 +89,22 @@ class OracleAdapter(Adapter):
         # unquoted Oracle identifiers are upper-cased on storage
         return ident.upper() if ident else ident
 
+    @property
+    def default_schema(self) -> str | None:  # type: ignore[override]
+        """The connected user — Oracle's implicit schema for unqualified objects.
+
+        Resolved from the live session (``SELECT USER``) and cached. Drift matching uses this
+        so a repo object under folder ``dbb`` aligns with the live ``DBB``-owned object;
+        without it every object read back by ``inventory()`` looked "missing" (owner dropped
+        on one side of the key). Same source as ``inventory()``'s ``owner = USER`` filter.
+        """
+        cached = getattr(self, "_ds_cache", None)
+        if cached is None:
+            with self.engine.connect() as conn:
+                cached = conn.exec_driver_sql("SELECT USER FROM dual").scalar()
+            self._ds_cache = cached
+        return cached
+
     def table_exists(self, schema: str | None, name: str) -> bool:
         return inspect(self.engine).has_table(self._norm(name), schema=self._norm(schema))
 
@@ -139,6 +155,7 @@ class OracleAdapter(Adapter):
             "('FUNCTION','PROCEDURE','TRIGGER','PACKAGE','PACKAGE BODY','TYPE') "
             "ORDER BY name, type, line"
         )
+        owner = self.default_schema  # objects are filtered by owner = USER — carry it on the id
         found: dict[str, LiveObject] = {}
         src_text: dict[str, list[str]] = {}
         with self.engine.connect() as conn:
@@ -146,13 +163,13 @@ class OracleAdapter(Adapter):
                 kind = self._TYPEMAP.get(otype)
                 if kind is None:
                     continue
-                obj = LiveObject(kind, ObjectId(None, name))
+                obj = LiveObject(kind, ObjectId(owner, name))
                 found[obj.key()] = obj
             for name, otype, line in conn.execute(sources):
                 kind = self._TYPEMAP.get(otype)
                 if kind is None:
                     continue
-                key = LiveObject(kind, ObjectId(None, name)).key()
+                key = LiveObject(kind, ObjectId(owner, name)).key()
                 src_text.setdefault(key, []).append(line or "")
         for key, lines in src_text.items():
             if key in found:
@@ -167,6 +184,9 @@ class OracleAdapter(Adapter):
             parts.append("NOT NULL")
         return " ".join(parts) + ";"
 
+    def modify_column_sql(self, table: ObjectId, col: Column) -> str:
+        return f"ALTER TABLE {table} MODIFY {col.name} {col.type};"
+
     def apply(self, statements: list[str]) -> None:
         # No transactional rollback (DDL auto-commits). Sequential; commit covers any DML.
         with self.engine.connect() as conn:

{dbly-0.1.0 → dbly-0.3.0}/src/dbly/adapters/postgres.py

@@ -111,6 +111,9 @@ class PostgresAdapter(Adapter):
             parts.append(f"DEFAULT {col.default}")
         return " ".join(parts) + ";"
 
+    def modify_column_sql(self, table: ObjectId, col: Column) -> str:
+        return f"ALTER TABLE {table} ALTER COLUMN {col.name} TYPE {col.type};"
+
     def apply(self, statements: list[str]) -> None:
         # transactional DDL → one atomic transaction
         with self.engine.begin() as conn:

{dbly-0.1.0 → dbly-0.3.0}/src/dbly/adapters/sqlite.py

@@ -80,6 +80,14 @@ class SqliteAdapter(Adapter):
             parts.append(f"DEFAULT {col.default}")
         return " ".join(parts) + ";"
 
+    def modify_column_sql(self, table: ObjectId, col: Column) -> str:
+        # SQLite cannot change a column's type in place — a rebuild is required. Emit a
+        # commented no-op so the (destructive, never auto-applied) step documents the limit.
+        return (
+            f"-- SQLite cannot ALTER COLUMN {table.name}.{col.name} to {col.type} "
+            "in place; rebuild the table manually."
+        )
+
     def apply(self, statements: list[str]) -> None:
         with self.engine.begin() as conn:
             for stmt in statements:

{dbly-0.1.0 → dbly-0.3.0}/src/dbly/cli.py

@@ -21,7 +21,7 @@ from dbly.engine import detect_dialect
 from dbly.parsing import sqlglot_dialect
 from dbly.planner import build_plan
 from dbly.model import Plan, Severity
-from dbly.repo import Repo
+from dbly.repo import WORKTREE, Repo
 
 app = typer.Typer(
     name="dbly",
@@ -48,13 +48,32 @@ def _main(
     pass
 
 
-def _make_plan(repo_path: Path, target: str, from_ref: Optional[str], to_ref: str) -> Plan:
+def _decorations(repo_path: Path, plan: Plan) -> dict[str, str]:
+    """Map each real ref in the plan to its git tag/branch names (for the plan header)."""
+    try:
+        repo = Repo(repo_path)
+    except ValueError:
+        return {}
+    out: dict[str, str] = {}
+    for ref in {plan.from_ref, plan.to_ref}:
+        if not ref or ref == WORKTREE:
+            continue
+        names = repo.ref_names(ref)
+        if names:
+            out[ref] = ", ".join(names)
+    return out
+
+
+def _make_plan(
+    repo_path: Path, target: str, from_ref: Optional[str], to_ref: str,
+    *, worktree: bool = False,
+) -> Plan:
     repo = Repo(repo_path)
     cfg = resolve_target(target)
     dialect = sqlglot_dialect(detect_dialect(cfg))
     adapter = get_adapter(cfg)
     try:
-        resolved_to = repo.resolve_ref(to_ref)
+        resolved_to = repo.resolve_ref(WORKTREE if worktree else to_ref)
         if from_ref is not None:
             resolved_from = repo.resolve_ref(from_ref)
         else:
@@ -80,10 +99,15 @@ def plan(
     sql: Optional[Path] = typer.Option(
         None, "--sql", help="write an executable SQL script for a hand/offline deploy."
     ),
+    worktree: bool = typer.Option(
+        False, "--worktree", "--dirty",
+        help="plan against the working tree (uncommitted + untracked object files), "
+             "not a git ref — for the fast edit→plan loop.",
+    ),
 ) -> None:
     """Compute and show the deployment plan."""
-    plan_obj = _make_plan(repo_path, target, from_ref, to)
-    report.render_plan(plan_obj, console)
+    plan_obj = _make_plan(repo_path, target, from_ref, to, worktree=worktree)
+    report.render_plan(plan_obj, console, ref_names=_decorations(repo_path, plan_obj))
     if out:
         out.write_text(report.plan_to_yaml(plan_obj), encoding="utf-8")
         console.print(f"\n[dim]plan (YAML) written to {out}[/dim]")
@@ -123,7 +147,7 @@ def apply(
     else:
         plan_obj = _make_plan(repo_path, target, from_ref, to)
 
-    report.render_plan(plan_obj, console)
+    report.render_plan(plan_obj, console, ref_names=_decorations(repo_path, plan_obj))
 
     destructive = [s for s in plan_obj.steps if s.severity is Severity.DESTRUCTIVE]
     if destructive and not allow_destructive:
@@ -137,7 +161,7 @@ def apply(
         s.sql for s in plan_obj.steps
         if allow_destructive or s.severity is not Severity.DESTRUCTIVE
     ]
-    if not statements:
+    if not statements and not plan_obj.migrations and not plan_obj.baselined:
         console.print("[green]nothing to apply.[/green]")
         return
 
@@ -147,13 +171,24 @@ def apply(
     try:
         _run_hooks(repo, "pre", py_interpreter)
         adapter.ensure_state_table()
-        adapter.apply(statements)
+        # baseline (bootstrap): record migrations as applied without running them
+        for mid in plan_obj.baselined:
+            adapter.record_migration(plan_obj.to_ref, mid)
+        # explicit migrations run first — they reshape the schema before reconciliation
+        for m in plan_obj.migrations:
+            adapter.run_init_script(m.sql)  # engine-aware multi-statement / PL-SQL runner
+            adapter.record_migration(plan_obj.to_ref, m.id)
+            console.print(f"[magenta]migration[/magenta] applied {m.id}")
+        if statements:
+            adapter.apply(statements)
         adapter.record_deploy(plan_obj.to_ref, migration_ids=[])
         _run_hooks(repo, "post", py_interpreter)
     finally:
         adapter.dispose()
-    console.print(f"[green]applied[/green] {len(statements)} step(s); "
-                  f"deployed ref → {plan_obj.to_ref}")
+    console.print(
+        f"[green]applied[/green] {len(statements)} step(s), "
+        f"{len(plan_obj.migrations)} migration(s); deployed ref → {plan_obj.to_ref}"
+    )
 
 
 @app.command()
@@ -192,12 +227,15 @@ def bootstrap(
 ) -> None:
     """Install into an empty database (no baseline — full apply)."""
     plan_obj = _make_plan(repo_path, target, None, to)
-    report.render_plan(plan_obj, console)
+    report.render_plan(plan_obj, console, ref_names=_decorations(repo_path, plan_obj))
     console.print("\n[dim]review, then run `dbly apply` to execute.[/dim]")
 
 
 @app.command()
-def status(target: str = typer.Option(..., "--target")) -> None:
+def status(
+    target: str = typer.Option(..., "--target"),
+    repo_path: Path = typer.Option(Path("."), "--repo", help="repository root (for ref names)."),
+) -> None:
     """Show the deployed ref recorded on the target."""
     cfg = resolve_target(target)
     adapter = get_adapter(cfg)
@@ -205,10 +243,17 @@ def status(target: str = typer.Option(..., "--target")) -> None:
         ref = adapter.get_deployed_ref()
     finally:
         adapter.dispose()
-    if ref:
-        console.print(f"deployed ref: [cyan]{ref}[/cyan]")
-    else:
+    if not ref:
         console.print("[yellow]no deploy recorded — database is unmanaged or empty.[/yellow]")
+        return
+    try:
+        names = Repo(repo_path).ref_names(ref)
+    except ValueError:
+        names = []  # not a git repo — SHA only
+    if names:
+        console.print(f"deployed ref: [cyan]{', '.join(names)}[/cyan] [dim]({ref[:8]})[/dim]")
+    else:
+        console.print(f"deployed ref: [cyan]{ref}[/cyan]")
 
 
 @app.command()
@@ -219,6 +264,10 @@ def check(
     orphans: bool = typer.Option(
         False, "--orphans", help="also report objects in the DB but not in the repo."
     ),
+    worktree: bool = typer.Option(
+        False, "--worktree", "--dirty",
+        help="compare the working tree (uncommitted + untracked) against the DB, not a git ref.",
+    ),
 ) -> None:
     """Detect drift: compare the desired state at <to> against the live database."""
     repo = Repo(repo_path)
@@ -228,7 +277,8 @@ def check(
     try:
         rep = drift.compute_drift(
             repo, adapter,
-            to_ref=repo.resolve_ref(to), dialect=dialect, include_orphans=orphans,
+            to_ref=repo.resolve_ref(WORKTREE if worktree else to),
+            dialect=dialect, include_orphans=orphans,
         )
     finally:
         adapter.dispose()

{dbly-0.1.0 → dbly-0.3.0}/src/dbly/model.py

@@ -128,6 +128,19 @@ class Step:
     note: str | None = None
 
 
+@dataclass(slots=True)
+class Migration:
+    """An explicit, run-once, hand-written migration script (CONCEPT.md §5).
+
+    For changes the additive diff cannot do safely — renames, type changes with data
+    transformation, backfills. Tracked by ``id`` in the ledger so it runs exactly once.
+    """
+
+    id: str
+    sql: str
+    source_file: Path
+
+
 @dataclass(slots=True)
 class Plan:
     """An ordered, reviewable set of steps — the artifact of ``dbly plan``."""
@@ -135,6 +148,8 @@ class Plan:
     target: str
     from_ref: str | None
     to_ref: str
+    migrations: list[Migration] = field(default_factory=list)   # pending — run before steps
+    baselined: list[str] = field(default_factory=list)          # recorded, not run (bootstrap)
     steps: list[Step] = field(default_factory=list)
     warnings: list[str] = field(default_factory=list)
 

{dbly-0.1.0 → dbly-0.3.0}/src/dbly/parsing.py

@@ -11,6 +11,7 @@ applied verbatim per dialect by the adapter.
 from __future__ import annotations
 
 import hashlib
+import re
 from pathlib import Path
 
 import sqlglot
@@ -121,6 +122,24 @@ def parse_file(
     return objects
 
 
+def referenced_tables(sql: str, *, dialect: str | None = None) -> set[str]:
+    """Lower-cased table names referenced anywhere in a SQL script (best effort).
+
+    Used to detect which tables a pending migration touches, so the additive diff defers to
+    the migration for those tables. Returns an empty set if sqlglot cannot parse the script.
+    """
+    names: set[str] = set()
+    try:
+        for stmt in sqlglot.parse(sql, read=dialect):
+            if stmt is None:
+                continue
+            for tbl in stmt.find_all(exp.Table):
+                names.add(tbl.name.lower())
+    except Exception:  # noqa: BLE001 — procedural/odd SQL; suppression is best-effort
+        pass
+    return names
+
+
 def canonical_hash(sql: str | None, *, dialect: str | None = None) -> str | None:
     """A formatting-insensitive hash of a definition, for advisory drift detection.
 
@@ -175,6 +194,41 @@ def desired_columns(sql: str, *, dialect: str | None = None) -> list[Column]:
     return columns
 
 
+_NUMERIC_TYPES = {"NUMBER", "NUMERIC", "DECIMAL", "DEC"}
+
+
+def canonical_type(type_str: str) -> str:
+    """Normalize a column type for cross-boundary comparison (desired vs. introspected).
+
+    The desired side is rendered by sqlglot (e.g. ``NUMBER(10)``); the actual side comes from
+    SQLAlchemy introspection (e.g. ``NUMBER(10, 0)``). Whitespace/case and an *omitted numeric
+    scale* are normalized so semantically-equal types compare equal — while a genuine
+    precision/scale change (``NUMBER`` → ``NUMBER(10)``) is preserved.
+
+    Honest scope: this is a string-level normalizer, not a type system. Exotic spellings that
+    differ across the sqlglot↔SQLAlchemy boundary may still compare unequal; a resulting
+    MODIFY step is flagged destructive (never auto-applied) and carries a warning, so a false
+    positive surfaces as a reviewable note rather than silent data loss.
+    """
+    s = " ".join(type_str.strip().upper().split())
+    # Drop introspection decorations that carry no shape info (e.g. SQL Server appends
+    # `COLLATE "SQL_Latin1_General_CP1_CI_AS"` to reflected string types).
+    s = re.split(r"\s+COLLATE\s+", s, maxsplit=1)[0].strip()
+    m = re.match(r"^([A-Z0-9_ ]+?)\s*\(([^)]*)\)\s*$", s)
+    if not m:
+        return s.replace(" ", "")
+    base = m.group(1).strip().replace(" ", "")
+    params = [p.strip() for p in m.group(2).split(",") if p.strip()]
+    if base in _NUMERIC_TYPES and len(params) == 1:
+        params.append("0")  # NUMBER(p) ≡ NUMBER(p,0) — scale defaults to 0
+    return f"{base}({','.join(params)})"
+
+
+def types_differ(desired: str, actual: str) -> bool:
+    """Whether a declared (desired) column type differs from the live (actual) one."""
+    return canonical_type(desired) != canonical_type(actual)
+
+
 def topological_order(objects: list[ParsedObject]) -> list[ParsedObject]:
     """Order replaceable objects so dependencies come first (CONCEPT.md §8).