sustained 2.2.0__tar.gz → 2.4.0__tar.gz

{sustained-2.2.0 → sustained-2.4.0}/CHANGELOG.md

@@ -1,5 +1,28 @@
 # Changelog
 
+## 2.4.0 (2026-08-14)
+
+### Added
+
+- Constraint-aware introspection: primary keys, unique constraints, foreign keys, column defaults, and indexes are read from SQLite PRAGMA tables or information_schema, with graceful degradation and system schemas filtered.
+- Type and nullability changes now generate migrations: in-place reversible ALTER COLUMN on Postgres (with `type_casts` USING hints), MSSQL, and DuckDB; automatic table rebuild with row copy on SQLite.
+- Rename hints: `renames={'table.old': 'new'}` and `table_renames` produce reversible RENAME statements (sp_rename on MSSQL) instead of destructive drop-plus-add.
+- Declared indexes on models via `Index`; created with the table and diffed for additions, definition changes, and opt-in drops, all reversible.
+- `backfill` on ColumnDef: NOT NULL adds and tightenings emit add-nullable, UPDATE, SET NOT NULL, or fold into the SQLite rebuild.
+- Length and precision changes detected when both sides report them.
+- Constraint notes: PK, FK, unique, and default drift reported in the diff, never auto-migrated.
+- Offline scripts: `migration_sql()` and `Migrator.script()` render the SQL a run would execute for DBA review.
+- `AsyncMigrator`: the migration runner on an AsyncAdapter with transactional application and awaited callable steps.
+
+## 2.3.0 (2026-08-14)
+
+### Added
+
+- Schema autogeneration: `diff_schema()` introspects the live database (SQLite PRAGMA on the default dialect, `information_schema.columns` elsewhere) and reports missing tables, new columns, extra objects, and changed columns with a readable `summary()`. Type comparison round-trips through each dialect's own type mapping, so tables created from models diff clean.
+- `autogenerate()`: builds a `Migration` from the diff. Additive steps are reversible (CREATE/DROP TABLE, ADD/DROP COLUMN pairs). Drops require `allow_drops=True` and carry no down step; changed column types block generation unless explicitly ignored; NOT NULL adds without defaults and primary key or autoincrement adds are rejected.
+- `Migrator.sync(models)`: diff, generate, register, and apply in one call, idempotent when the schema is current. `Migrator.down_to(id)` reverts newest-first until the target is the most recent applied migration.
+- Compilers render `ADD COLUMN` and `DROP COLUMN` statements, with the T-SQL `ADD` spelling on MSSQL.
+
 ## 2.2.0 (2026-08-14)
 
 ### Added

{sustained-2.2.0 → sustained-2.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: sustained
-Version: 2.2.0
+Version: 2.4.0
 Summary: A Python query builder inspired by Objection.js
 Project-URL: Homepage, https://github.com/wetherc/sustained
 Project-URL: Issues, https://github.com/wetherc/sustained/issues
@@ -16,7 +16,7 @@ Description-Content-Type: text/markdown
 
 A Python query builder and lightweight ORM inspired by [Objection.js](https://vincit.github.io/objection.js/).
 
-Sustained builds parameterized SQL for the default (ANSI), Postgres, MSSQL, Presto, and DuckDB dialects. It executes queries against any DB-API 2.0 connection or connection pool with transactions, hydrates rows into model instances or DataFrames, writes data with `insert()`, `update()`, `delete()`, upserts, and `INSERT ... SELECT`, and eager loads relations. Filters compose as typed predicates: `User.query().where((User.c.age > 21) & User.c.name.like('A%'))`. Models can declare typed columns and generate their DDL, an explicit migration runner manages schema changes, and async services run the same queries through driver adapters with `await query.arun()`.
+Sustained builds parameterized SQL for the default (ANSI), Postgres, MSSQL, Presto, and DuckDB dialects. It executes queries against any DB-API 2.0 connection or connection pool with transactions, hydrates rows into model instances or DataFrames, writes data with `insert()`, `update()`, `delete()`, upserts, and `INSERT ... SELECT`, and eager loads relations. Filters compose as typed predicates: `User.query().where((User.c.age > 21) & User.c.name.like('A%'))`. Models declare typed columns and manage their own schema: `Migrator.sync(models)` diffs the live database, generates the migration, applies it, and `down()` rolls it back, with destructive changes gated behind explicit opt-ins. Async services run the same queries through driver adapters with `await query.arun()`.
 
 ## Installation
 

{sustained-2.2.0 → sustained-2.4.0}/README.md

@@ -2,7 +2,7 @@
 
 A Python query builder and lightweight ORM inspired by [Objection.js](https://vincit.github.io/objection.js/).
 
-Sustained builds parameterized SQL for the default (ANSI), Postgres, MSSQL, Presto, and DuckDB dialects. It executes queries against any DB-API 2.0 connection or connection pool with transactions, hydrates rows into model instances or DataFrames, writes data with `insert()`, `update()`, `delete()`, upserts, and `INSERT ... SELECT`, and eager loads relations. Filters compose as typed predicates: `User.query().where((User.c.age > 21) & User.c.name.like('A%'))`. Models can declare typed columns and generate their DDL, an explicit migration runner manages schema changes, and async services run the same queries through driver adapters with `await query.arun()`.
+Sustained builds parameterized SQL for the default (ANSI), Postgres, MSSQL, Presto, and DuckDB dialects. It executes queries against any DB-API 2.0 connection or connection pool with transactions, hydrates rows into model instances or DataFrames, writes data with `insert()`, `update()`, `delete()`, upserts, and `INSERT ... SELECT`, and eager loads relations. Filters compose as typed predicates: `User.query().where((User.c.age > 21) & User.c.name.like('A%'))`. Models declare typed columns and manage their own schema: `Migrator.sync(models)` diffs the live database, generates the migration, applies it, and `down()` rolls it back, with destructive changes gated behind explicit opt-ins. Async services run the same queries through driver adapters with `await query.arun()`.
 
 ## Installation
 

{sustained-2.2.0 → sustained-2.4.0}/docs/index.md

@@ -18,7 +18,7 @@ If you are new to Sustained, it's recommended to read the guides in the followin
 5.  **[Filtering](./filtering):** Dive into the various `where` methods for filtering your results.
 6.  **[Relations and Joins](./relations):** Learn how to define relationships between models and join them in your queries.
 7.  **[Executing Queries](./executing):** Run queries against a database, hydrate results into models, write data, and eager load relations.
-8.  **[Schema and Migrations](./schema):** Declare typed columns on models, generate DDL, and run ordered migrations.
+8.  **[Schema and Migrations](./schema):** Declare typed columns on models, then let the migrator diff the live database, generate migrations, apply them, and roll them back.
 
 ## API Reference
 

sustained-2.4.0/docs/schema.md

@@ -0,0 +1,140 @@
+---
+layout: default
+title: Schema and Migrations
+---
+
+Sustained manages your database schema from your models. Declare typed columns once, then let the migrator create tables, detect drift, generate migrations, apply them, and roll them back.
+
+## Automated Migration in One Call
+
+`Migrator.sync()` diffs the live database against your models, generates the migration, records it, and applies it. Run it again after changing a model and only the difference is applied. Nothing to hand-write for additive changes.
+
+```python
+from sustained import Model
+from sustained.migrations import Migrator
+from sustained.schema import Integer, String, Text
+
+class User(Model):
+    tableName = 'users'
+    tableColumns = {
+        'id': Integer(primary_key=True, autoincrement=True),
+        'email': String(120, unique=True, nullable=False),
+    }
+
+migrator = Migrator(conn, [])
+migrator.sync([User])          # creates the users table
+
+User.tableColumns['bio'] = Text()
+migrator.sync([User])          # adds only the bio column
+```
+
+## Automated Rollback
+
+Generated migrations are reversible: a created table carries its DROP, an added column carries its DROP COLUMN. Roll back one step, several steps, or down to a known-good migration.
+
+```python
+migrator.down()                # revert the newest applied migration
+migrator.down(steps=2)         # revert the two newest
+migrator.down_to('auto_20260814...')  # revert until this id is newest
+```
+
+Every applied migration is recorded in a tracking table, each runs inside a transaction, and a failing step rolls itself back and leaves earlier migrations applied.
+
+## Inspecting Drift Before Applying
+
+`diff_schema()` reports every difference without touching anything. `autogenerate()` builds the migration so you can review its statements before it runs.
+
+```python
+from sustained.autogenerate import autogenerate, diff_schema
+
+print(diff_schema(conn, [User]).summary())
+# add column users.bio
+# drop column users.legacy (destructive)
+
+migration = autogenerate(conn, [User], id='add_bio')
+print(migration.up)    # ['ALTER TABLE users ADD COLUMN bio TEXT']
+print(migration.down)  # ['ALTER TABLE users DROP COLUMN bio']
+```
+
+## Safety Rules
+
+Autogeneration refuses to guess about anything that loses data or fails on populated tables:
+
+- **Drops are opt-in.** Extra tables and columns raise unless `allow_drops=True`. A migration containing drops has no down step, because the dropped data cannot come back.
+- **Type and nullability changes migrate per dialect.** Postgres, MSSQL, and DuckDB alter in place with reversible down steps; Postgres casts take a hint through `type_casts={'table.col': 'col::integer'}`. SQLite rebuilds the table (create new, copy rows, replace), which is not reversible. Pass `ignore_changed_columns=True` to skip them entirely.
+- **NOT NULL needs a value for existing rows.** Adding or tightening to NOT NULL requires a `default` or a `backfill` value on the ColumnDef; generation emits add-nullable, UPDATE backfill, SET NOT NULL, or folds the backfill into a SQLite rebuild. New primary key or autoincrement columns cannot be added with ALTER TABLE.
+- The migration tracking table is excluded from diffing, and `exclude_tables` protects any other tables Sustained does not manage.
+
+Renames cannot be detected from the catalog, so pass hints: `sync(models, renames={'users.name': 'full_name'}, table_renames={'old': 'new'})` emits reversible RENAME statements instead of a destructive drop-plus-add.
+
+Primary key, foreign key, column-level unique, and default differences are reported as constraint notes in the diff but never auto-migrated.
+
+## Typed Columns
+
+`tableColumns` maps column names to typed definitions: `Integer`, `BigInteger`, `String(length)`, `Text`, `Boolean`, `Float`, `Numeric(precision, scale)`, `Date`, `Timestamp`, `Json`.
+
+```python
+from sustained.schema import Boolean, Integer, String, Timestamp
+from sustained.types import Expression
+
+class User(Model):
+    tableName = 'users'
+    tableColumns = {
+        'id': Integer(primary_key=True, autoincrement=True),
+        'email': String(120, unique=True, nullable=False),
+        'active': Boolean(default=True),
+        'created_at': Timestamp(default=Expression('CURRENT_TIMESTAMP')),
+    }
+```
+
+Definitions support composite primary keys (mark several columns `primary_key=True`), `unique`, literal or raw `Expression` defaults, foreign keys through `references='table.column'`, and `backfill` values for NOT NULL migrations. Models also declare named indexes, which `create_table()` and generated migrations create and keep in sync:
+
+```python
+from sustained.schema import Index
+
+class User(Model):
+    tableName = 'users'
+    tableColumns = {...}
+    indexes = [Index('ix_users_email', 'email', unique=True)]
+```
+
+`autoincrement` requires a single integer primary key; DuckDB and Presto raise because they have no identity columns. A model with `tableColumns` also gets strict column-name access automatically: a typo'd column raises `AttributeError`.
+
+## Generating and Running DDL Directly
+
+```python
+User.create_table_sql()   # dialect-specific CREATE TABLE
+User.create_table(conn)   # execute it
+User.drop_table(conn)     # DROP TABLE IF EXISTS
+```
+
+Types map per dialect: `BIT`, `NVARCHAR`, and `DATETIME2` on MSSQL; `JSONB` and identity columns on Postgres; plain `INTEGER PRIMARY KEY` rowid behavior on the default dialect. Schema introspection reads SQLite's PRAGMA tables on the default dialect and `information_schema.columns` elsewhere.
+
+## Hand-Written Migrations
+
+Anything autogeneration will not express, write explicitly. A `Migration` pairs an id with an up step and an optional down step; steps are a SQL string, a list of statements, or a callable receiving the connection. Hand-written and generated migrations share one ordered list and one tracking table.
+
+```python
+from sustained.migrations import Migration, Migrator, create_table_migration
+
+migrations = [
+    create_table_migration(User),
+    Migration(
+        'split_name_column',
+        up=[
+            'ALTER TABLE users ADD COLUMN first_name TEXT',
+            "UPDATE users SET first_name = substr(name, 1, instr(name, ' ') - 1) WHERE 1 = 1",
+        ],
+        down='ALTER TABLE users DROP COLUMN first_name',
+    ),
+]
+
+migrator = Migrator(conn, migrations)
+migrator.up()                        # apply all pending
+migrator.up(target='create_users')   # stop after a target
+migrator.status()                    # [(id, applied), ...]
+```
+
+## Offline Review and Async
+
+`migrator.script('up')` renders every statement a run would execute, including tracking bookkeeping, without touching the database, for review or DBA handoff; `script('down')` renders the rollback. For async services, `AsyncMigrator` in `sustained.aio_migrations` runs the same `Migration` objects on an `AsyncAdapter` with the same `up`, `down`, `down_to`, and `status` surface; callable steps receive the adapter and are awaited.

{sustained-2.2.0 → sustained-2.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "sustained"
-version = "2.2.0"
+version = "2.4.0"
 description = "A Python query builder inspired by Objection.js"
 readme = "README.md"
 requires-python = ">=3.9"

sustained-2.4.0/src/sustained/aio_migrations.py

@@ -0,0 +1,154 @@
+"""
+Async migration runner.
+
+AsyncMigrator mirrors Migrator on an AsyncAdapter: same Migration objects,
+same tracking table, same ordering rules. String and list steps execute
+through the adapter; callable steps receive the adapter and are awaited
+when they return a coroutine. Each migration runs inside an
+async_transaction() block.
+"""
+
+from __future__ import annotations
+
+import inspect
+from datetime import datetime, timezone
+from typing import Any, List, Optional, Tuple
+
+from sustained.aio import AsyncAdapter, async_transaction
+from sustained.dialects import Dialects
+from sustained.migrations import Migration, MigrationStep
+
+
+class AsyncMigrator:
+    """Applies and reverts an ordered list of migrations on an adapter."""
+
+    def __init__(
+        self,
+        adapter: AsyncAdapter,
+        migrations: List[Migration],
+        table: str = "sustained_migrations",
+        dialect: Dialects = Dialects.DEFAULT,
+    ) -> None:
+        ids = [m.id for m in migrations]
+        duplicates = {i for i in ids if ids.count(i) > 1}
+        if duplicates:
+            raise ValueError(f"Duplicate migration ids: {sorted(duplicates)}.")
+        self._adapter = adapter
+        self._migrations = list(migrations)
+        self._table = table
+        self._dialect = dialect
+        self._compiler = Dialects.get_compiler(dialect)
+
+    def _table_sql(self) -> str:
+        return self._compiler.quote_identifier(self._table)
+
+    async def _run_step(self, step: MigrationStep) -> None:
+        if callable(step):
+            result = step(self._adapter)
+            if inspect.isawaitable(result):
+                await result
+            return
+        statements = [step] if isinstance(step, str) else list(step)
+        for statement in statements:
+            await self._adapter.execute(statement, ())
+
+    async def _ensure_tracking_table(self) -> None:
+        from sustained.schema import String, Text, build_create_table_sql
+
+        sql = build_create_table_sql(
+            self._compiler,
+            self._table_sql(),
+            {
+                "id": String(255, primary_key=True),
+                "applied_at": Text(nullable=False),
+            },
+            if_not_exists=True,
+        )
+        await self._adapter.execute(sql, ())
+        await self._adapter.commit()
+
+    async def applied(self) -> List[str]:
+        """Returns the applied migration ids in application order."""
+        await self._ensure_tracking_table()
+        _, rows = await self._adapter.fetch(
+            f"SELECT id FROM {self._table_sql()} ORDER BY applied_at, id", ()
+        )
+        return [row[0] for row in rows]
+
+    async def pending(self) -> List[Migration]:
+        """Returns the registered migrations that have not been applied."""
+        applied = set(await self.applied())
+        return [m for m in self._migrations if m.id not in applied]
+
+    async def status(self) -> List[Tuple[str, bool]]:
+        """Returns (id, applied) pairs for every registered migration."""
+        applied = set(await self.applied())
+        return [(m.id, m.id in applied) for m in self._migrations]
+
+    async def up(self, target: Optional[str] = None) -> List[str]:
+        """
+        Applies pending migrations in order, stopping after the target id
+        when one is given. Returns the ids that were applied.
+        """
+        migrations = self._migrations
+        if target is not None:
+            ids = [m.id for m in migrations]
+            if target not in ids:
+                raise ValueError(f"Unknown migration target: {target!r}.")
+            migrations = migrations[: ids.index(target) + 1]
+
+        already_applied = set(await self.applied())
+        placeholder = self._compiler.placeholder()
+        applied_now: List[str] = []
+        for migration in migrations:
+            if migration.id in already_applied:
+                continue
+            async with async_transaction(self._adapter):
+                await self._run_step(migration.up)
+                timestamp = datetime.now(timezone.utc).isoformat()
+                await self._adapter.execute(
+                    f"INSERT INTO {self._table_sql()} (id, applied_at) "
+                    f"VALUES ({placeholder}, {placeholder})",
+                    (migration.id, timestamp),
+                )
+            applied_now.append(migration.id)
+        return applied_now
+
+    async def down(self, steps: int = 1) -> List[str]:
+        """
+        Reverts the most recently applied migrations, newest first. Every
+        reverted migration must define a down step and be registered with
+        this migrator. Returns the ids that were reverted.
+        """
+        applied = await self.applied()
+        by_id = {m.id: m for m in self._migrations}
+        placeholder = self._compiler.placeholder()
+        reverted: List[str] = []
+        for migration_id in reversed(applied[-steps:] if steps else []):
+            migration = by_id.get(migration_id)
+            if migration is None:
+                raise ValueError(
+                    f"Applied migration '{migration_id}' is not registered "
+                    "with this migrator; cannot revert."
+                )
+            if migration.down is None:
+                raise ValueError(f"Migration '{migration_id}' has no down step.")
+            async with async_transaction(self._adapter):
+                await self._run_step(migration.down)
+                await self._adapter.execute(
+                    f"DELETE FROM {self._table_sql()} WHERE id = {placeholder}",
+                    (migration_id,),
+                )
+            reverted.append(migration_id)
+        return reverted
+
+    async def down_to(self, target: str) -> List[str]:
+        """
+        Reverts applied migrations newest-first until the target is the
+        most recent applied migration. The target itself stays applied.
+        """
+        applied = await self.applied()
+        if target not in applied:
+            raise ValueError(f"Migration '{target}' is not applied.")
+        steps = len(applied) - applied.index(target) - 1
+        return await self.down(steps) if steps else []