sustained 2.2.0__tar.gz → 2.3.0__tar.gz

{sustained-2.2.0 → sustained-2.3.0}/CHANGELOG.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## 2.3.0 (2026-08-14)
+
+### Added
+
+- Schema autogeneration: `diff_schema()` introspects the live database (SQLite PRAGMA on the default dialect, `information_schema.columns` elsewhere) and reports missing tables, new columns, extra objects, and changed columns with a readable `summary()`. Type comparison round-trips through each dialect's own type mapping, so tables created from models diff clean.
+- `autogenerate()`: builds a `Migration` from the diff. Additive steps are reversible (CREATE/DROP TABLE, ADD/DROP COLUMN pairs). Drops require `allow_drops=True` and carry no down step; changed column types block generation unless explicitly ignored; NOT NULL adds without defaults and primary key or autoincrement adds are rejected.
+- `Migrator.sync(models)`: diff, generate, register, and apply in one call, idempotent when the schema is current. `Migrator.down_to(id)` reverts newest-first until the target is the most recent applied migration.
+- Compilers render `ADD COLUMN` and `DROP COLUMN` statements, with the T-SQL `ADD` spelling on MSSQL.
+
 ## 2.2.0 (2026-08-14)
 
 ### Added

{sustained-2.2.0 → sustained-2.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: sustained
-Version: 2.2.0
+Version: 2.3.0
 Summary: A Python query builder inspired by Objection.js
 Project-URL: Homepage, https://github.com/wetherc/sustained
 Project-URL: Issues, https://github.com/wetherc/sustained/issues
@@ -16,7 +16,7 @@ Description-Content-Type: text/markdown
 
 A Python query builder and lightweight ORM inspired by [Objection.js](https://vincit.github.io/objection.js/).
 
-Sustained builds parameterized SQL for the default (ANSI), Postgres, MSSQL, Presto, and DuckDB dialects. It executes queries against any DB-API 2.0 connection or connection pool with transactions, hydrates rows into model instances or DataFrames, writes data with `insert()`, `update()`, `delete()`, upserts, and `INSERT ... SELECT`, and eager loads relations. Filters compose as typed predicates: `User.query().where((User.c.age > 21) & User.c.name.like('A%'))`. Models can declare typed columns and generate their DDL, an explicit migration runner manages schema changes, and async services run the same queries through driver adapters with `await query.arun()`.
+Sustained builds parameterized SQL for the default (ANSI), Postgres, MSSQL, Presto, and DuckDB dialects. It executes queries against any DB-API 2.0 connection or connection pool with transactions, hydrates rows into model instances or DataFrames, writes data with `insert()`, `update()`, `delete()`, upserts, and `INSERT ... SELECT`, and eager loads relations. Filters compose as typed predicates: `User.query().where((User.c.age > 21) & User.c.name.like('A%'))`. Models declare typed columns and manage their own schema: `Migrator.sync(models)` diffs the live database, generates the migration, applies it, and `down()` rolls it back, with destructive changes gated behind explicit opt-ins. Async services run the same queries through driver adapters with `await query.arun()`.
 
 ## Installation
 

{sustained-2.2.0 → sustained-2.3.0}/README.md

@@ -2,7 +2,7 @@
 
 A Python query builder and lightweight ORM inspired by [Objection.js](https://vincit.github.io/objection.js/).
 
-Sustained builds parameterized SQL for the default (ANSI), Postgres, MSSQL, Presto, and DuckDB dialects. It executes queries against any DB-API 2.0 connection or connection pool with transactions, hydrates rows into model instances or DataFrames, writes data with `insert()`, `update()`, `delete()`, upserts, and `INSERT ... SELECT`, and eager loads relations. Filters compose as typed predicates: `User.query().where((User.c.age > 21) & User.c.name.like('A%'))`. Models can declare typed columns and generate their DDL, an explicit migration runner manages schema changes, and async services run the same queries through driver adapters with `await query.arun()`.
+Sustained builds parameterized SQL for the default (ANSI), Postgres, MSSQL, Presto, and DuckDB dialects. It executes queries against any DB-API 2.0 connection or connection pool with transactions, hydrates rows into model instances or DataFrames, writes data with `insert()`, `update()`, `delete()`, upserts, and `INSERT ... SELECT`, and eager loads relations. Filters compose as typed predicates: `User.query().where((User.c.age > 21) & User.c.name.like('A%'))`. Models declare typed columns and manage their own schema: `Migrator.sync(models)` diffs the live database, generates the migration, applies it, and `down()` rolls it back, with destructive changes gated behind explicit opt-ins. Async services run the same queries through driver adapters with `await query.arun()`.
 
 ## Installation
 

{sustained-2.2.0 → sustained-2.3.0}/docs/index.md

@@ -18,7 +18,7 @@ If you are new to Sustained, it's recommended to read the guides in the followin
 5.  **[Filtering](./filtering):** Dive into the various `where` methods for filtering your results.
 6.  **[Relations and Joins](./relations):** Learn how to define relationships between models and join them in your queries.
 7.  **[Executing Queries](./executing):** Run queries against a database, hydrate results into models, write data, and eager load relations.
-8.  **[Schema and Migrations](./schema):** Declare typed columns on models, generate DDL, and run ordered migrations.
+8.  **[Schema and Migrations](./schema):** Declare typed columns on models, then let the migrator diff the live database, generate migrations, apply them, and roll them back.
 
 ## API Reference
 

sustained-2.3.0/docs/schema.md

@@ -0,0 +1,123 @@
+---
+layout: default
+title: Schema and Migrations
+---
+
+Sustained manages your database schema from your models. Declare typed columns once, then let the migrator create tables, detect drift, generate migrations, apply them, and roll them back.
+
+## Automated Migration in One Call
+
+`Migrator.sync()` diffs the live database against your models, generates the migration, records it, and applies it. Run it again after changing a model and only the difference is applied. Nothing to hand-write for additive changes.
+
+```python
+from sustained import Model
+from sustained.migrations import Migrator
+from sustained.schema import Integer, String, Text
+
+class User(Model):
+    tableName = 'users'
+    tableColumns = {
+        'id': Integer(primary_key=True, autoincrement=True),
+        'email': String(120, unique=True, nullable=False),
+    }
+
+migrator = Migrator(conn, [])
+migrator.sync([User])          # creates the users table
+
+User.tableColumns['bio'] = Text()
+migrator.sync([User])          # adds only the bio column
+```
+
+## Automated Rollback
+
+Generated migrations are reversible: a created table carries its DROP, an added column carries its DROP COLUMN. Roll back one step, several steps, or down to a known-good migration.
+
+```python
+migrator.down()                # revert the newest applied migration
+migrator.down(steps=2)         # revert the two newest
+migrator.down_to('auto_20260814...')  # revert until this id is newest
+```
+
+Every applied migration is recorded in a tracking table, each runs inside a transaction, and a failing step rolls itself back and leaves earlier migrations applied.
+
+## Inspecting Drift Before Applying
+
+`diff_schema()` reports every difference without touching anything. `autogenerate()` builds the migration so you can review its statements before it runs.
+
+```python
+from sustained.autogenerate import autogenerate, diff_schema
+
+print(diff_schema(conn, [User]).summary())
+# add column users.bio
+# drop column users.legacy (destructive)
+
+migration = autogenerate(conn, [User], id='add_bio')
+print(migration.up)    # ['ALTER TABLE users ADD COLUMN bio TEXT']
+print(migration.down)  # ['ALTER TABLE users DROP COLUMN bio']
+```
+
+## Safety Rules
+
+Autogeneration refuses to guess about anything that loses data or fails on populated tables:
+
+- **Drops are opt-in.** Extra tables and columns raise unless `allow_drops=True`. A migration containing drops has no down step, because the dropped data cannot come back.
+- **Type changes are never auto-migrated.** A changed type or nullability is reported in the diff and blocks generation until you write that migration by hand or pass `ignore_changed_columns=True`. SQLite cannot alter a column type in place, and a silent rewrite is exactly the change a human should review.
+- **Unsafe adds are rejected.** A new NOT NULL column needs a default; new primary key or autoincrement columns cannot be added with ALTER TABLE.
+- The migration tracking table is excluded from diffing, and `exclude_tables` protects any other tables Sustained does not manage.
+
+Diffing compares column presence, type, and nullability. Constraint changes (primary keys, unique indexes, foreign keys) are out of scope.
+
+## Typed Columns
+
+`tableColumns` maps column names to typed definitions: `Integer`, `BigInteger`, `String(length)`, `Text`, `Boolean`, `Float`, `Numeric(precision, scale)`, `Date`, `Timestamp`, `Json`.
+
+```python
+from sustained.schema import Boolean, Integer, String, Timestamp
+from sustained.types import Expression
+
+class User(Model):
+    tableName = 'users'
+    tableColumns = {
+        'id': Integer(primary_key=True, autoincrement=True),
+        'email': String(120, unique=True, nullable=False),
+        'active': Boolean(default=True),
+        'created_at': Timestamp(default=Expression('CURRENT_TIMESTAMP')),
+    }
+```
+
+Definitions support composite primary keys (mark several columns `primary_key=True`), `unique`, literal or raw `Expression` defaults, and foreign keys through `references='table.column'`. `autoincrement` requires a single integer primary key; DuckDB and Presto raise because they have no identity columns. A model with `tableColumns` also gets strict column-name access automatically: a typo'd column raises `AttributeError`.
+
+## Generating and Running DDL Directly
+
+```python
+User.create_table_sql()   # dialect-specific CREATE TABLE
+User.create_table(conn)   # execute it
+User.drop_table(conn)     # DROP TABLE IF EXISTS
+```
+
+Types map per dialect: `BIT`, `NVARCHAR`, and `DATETIME2` on MSSQL; `JSONB` and identity columns on Postgres; plain `INTEGER PRIMARY KEY` rowid behavior on the default dialect. Schema introspection reads SQLite's PRAGMA tables on the default dialect and `information_schema.columns` elsewhere.
+
+## Hand-Written Migrations
+
+Anything autogeneration will not express, write explicitly. A `Migration` pairs an id with an up step and an optional down step; steps are a SQL string, a list of statements, or a callable receiving the connection. Hand-written and generated migrations share one ordered list and one tracking table.
+
+```python
+from sustained.migrations import Migration, Migrator, create_table_migration
+
+migrations = [
+    create_table_migration(User),
+    Migration(
+        'split_name_column',
+        up=[
+            'ALTER TABLE users ADD COLUMN first_name TEXT',
+            "UPDATE users SET first_name = substr(name, 1, instr(name, ' ') - 1) WHERE 1 = 1",
+        ],
+        down='ALTER TABLE users DROP COLUMN first_name',
+    ),
+]
+
+migrator = Migrator(conn, migrations)
+migrator.up()                        # apply all pending
+migrator.up(target='create_users')   # stop after a target
+migrator.status()                    # [(id, applied), ...]
+```

{sustained-2.2.0 → sustained-2.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "sustained"
-version = "2.2.0"
+version = "2.3.0"
 description = "A Python query builder inspired by Objection.js"
 readme = "README.md"
 requires-python = ">=3.9"

sustained-2.3.0/src/sustained/autogenerate.py

@@ -0,0 +1,321 @@
+"""
+Schema autogeneration: diff the live database against model tableColumns
+declarations and produce a Migration.
+
+diff_schema() introspects the database and reports missing tables, new
+columns, extra tables and columns, and changed columns. autogenerate()
+turns the additive part of that diff into a Migration with matching down
+steps, so the change is fully reversible: CREATE TABLE reverses with DROP
+TABLE, ADD COLUMN reverses with DROP COLUMN.
+
+Destructive changes never happen silently. Dropping extra tables and
+columns requires allow_drops=True, and those steps carry no down step
+because the dropped data cannot be restored. Changed column types and
+nullability are reported but never migrated automatically: SQLite cannot
+alter a column's type in place, and a type rewrite is exactly the change a
+human should review. autogenerate() raises when the diff contains changes
+it will not express, unless told to ignore them.
+
+Only column presence, type, and nullability are compared. Constraint
+changes such as primary keys, unique indexes, and foreign keys are out of
+scope for diffing.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import TYPE_CHECKING, Any, Dict, List, NamedTuple, Optional, Tuple, Type
+
+from sustained.dialects import Dialects
+from sustained.migrations import Migration
+from sustained.schema import build_create_table_sql, render_column_sql
+
+if TYPE_CHECKING:
+    from sustained.model import Model
+    from sustained.schema import ColumnDef
+
+
+class IntrospectedColumn(NamedTuple):
+    """One column as reported by the database."""
+
+    raw_type: str
+    nullable: bool
+    primary_key: bool
+
+
+# Engine type spellings mapped to Sustained's logical types. Both sides of
+# a comparison pass through this table, so a model column compared against
+# the table its own DDL created always matches.
+_TYPE_SYNONYMS = {
+    "INT": "INTEGER",
+    "INT4": "INTEGER",
+    "INTEGER": "INTEGER",
+    "BIGINT": "BIGINT",
+    "INT8": "BIGINT",
+    "VARCHAR": "VARCHAR",
+    "CHARACTER VARYING": "VARCHAR",
+    "NVARCHAR": "VARCHAR",
+    "TEXT": "TEXT",
+    "BOOLEAN": "BOOLEAN",
+    "BOOL": "BOOLEAN",
+    "BIT": "BOOLEAN",
+    "FLOAT": "FLOAT",
+    "FLOAT8": "FLOAT",
+    "DOUBLE": "FLOAT",
+    "DOUBLE PRECISION": "FLOAT",
+    "REAL": "FLOAT",
+    "NUMERIC": "NUMERIC",
+    "DECIMAL": "NUMERIC",
+    "DATE": "DATE",
+    "TIMESTAMP": "TIMESTAMP",
+    "TIMESTAMP WITHOUT TIME ZONE": "TIMESTAMP",
+    "TIMESTAMP WITH TIME ZONE": "TIMESTAMP",
+    "DATETIME": "TIMESTAMP",
+    "DATETIME2": "TIMESTAMP",
+    "JSON": "JSON",
+    "JSONB": "JSON",
+}
+
+_TYPE_PARAMS_RE = re.compile(r"\s*\(.*\)\s*$")
+
+
+def normalize_type(raw: str) -> str:
+    """
+    Reduces an engine type spelling to a logical type name, dropping length
+    and precision parameters. Unknown spellings return uppercased as-is.
+    """
+    base = _TYPE_PARAMS_RE.sub("", raw).strip().upper()
+    return _TYPE_SYNONYMS.get(base, base)
+
+
+class SchemaDiff:
+    """The differences between declared models and the live database."""
+
+    def __init__(self) -> None:
+        self.missing_tables: List[Type["Model"]] = []
+        self.new_columns: List[Tuple[Type["Model"], str, "ColumnDef"]] = []
+        self.extra_tables: List[str] = []
+        self.extra_columns: List[Tuple[str, str]] = []
+        self.changed_columns: List[Tuple[str, str, str, str]] = []
+
+    def is_empty(self) -> bool:
+        return not (
+            self.missing_tables
+            or self.new_columns
+            or self.extra_tables
+            or self.extra_columns
+            or self.changed_columns
+        )
+
+    def summary(self) -> str:
+        """A human-readable description of every difference."""
+        lines: List[str] = []
+        for model in self.missing_tables:
+            lines.append(f"create table {model.tableName}")
+        for model, name, _ in self.new_columns:
+            lines.append(f"add column {model.tableName}.{name}")
+        for table in self.extra_tables:
+            lines.append(f"drop table {table} (destructive)")
+        for table, name in self.extra_columns:
+            lines.append(f"drop column {table}.{name} (destructive)")
+        for table, name, actual, expected in self.changed_columns:
+            lines.append(
+                f"change column {table}.{name}: database has {actual}, "
+                f"model declares {expected} (not auto-migrated)"
+            )
+        return "\n".join(lines) if lines else "schema up to date"
+
+
+def introspect_schema(
+    connection: Any, dialect: Dialects = Dialects.DEFAULT
+) -> Dict[str, Dict[str, IntrospectedColumn]]:
+    """
+    Reads tables and columns from the database. The default dialect reads
+    SQLite's PRAGMA tables; every other dialect reads
+    information_schema.columns. Table and column names are keyed lowercase.
+    """
+    cursor = connection.cursor()
+    schema: Dict[str, Dict[str, IntrospectedColumn]] = {}
+
+    if dialect == Dialects.DEFAULT:
+        cursor.execute(
+            "SELECT name FROM sqlite_master WHERE type = 'table' "
+            "AND name NOT LIKE 'sqlite_%'"
+        )
+        tables = [row[0] for row in cursor.fetchall()]
+        for table in tables:
+            cursor.execute(f"PRAGMA table_info({table})")
+            columns = {}
+            for _, name, raw_type, notnull, _, pk in cursor.fetchall():
+                columns[name.lower()] = IntrospectedColumn(
+                    raw_type=raw_type or "",
+                    nullable=not notnull,
+                    primary_key=bool(pk),
+                )
+            schema[table.lower()] = columns
+        return schema
+
+    cursor.execute(
+        "SELECT table_name, column_name, data_type, is_nullable "
+        "FROM information_schema.columns ORDER BY table_name, ordinal_position"
+    )
+    for table, name, data_type, is_nullable in cursor.fetchall():
+        table_columns = schema.setdefault(table.lower(), {})
+        table_columns[name.lower()] = IntrospectedColumn(
+            raw_type=data_type or "",
+            nullable=str(is_nullable).upper() == "YES",
+            primary_key=False,
+        )
+    return schema
+
+
+def diff_schema(
+    connection: Any,
+    models: List[Type["Model"]],
+    dialect: Dialects = Dialects.DEFAULT,
+    exclude_tables: Tuple[str, ...] = ("sustained_migrations",),
+) -> SchemaDiff:
+    """
+    Compares the models' tableColumns declarations against the live
+    database and returns the differences.
+    """
+    compiler = Dialects.get_compiler(dialect)
+    diff = SchemaDiff()
+
+    declared: Dict[str, Type["Model"]] = {}
+    for model in models:
+        if not model.tableName or not model.tableColumns:
+            raise ValueError(
+                f"Model '{model.__name__}' needs tableName and tableColumns "
+                "to participate in schema diffing."
+            )
+        key = model.tableName.lower()
+        if key in declared:
+            raise ValueError(f"Two models declare the table '{model.tableName}'.")
+        declared[key] = model
+
+    excluded = {t.lower() for t in exclude_tables}
+    actual = introspect_schema(connection, dialect)
+
+    for table_key, model in declared.items():
+        assert model.tableColumns is not None
+        actual_columns = actual.get(table_key)
+        if actual_columns is None:
+            diff.missing_tables.append(model)
+            continue
+        for name, coldef in model.tableColumns.items():
+            actual_col = actual_columns.get(name.lower())
+            if actual_col is None:
+                diff.new_columns.append((model, name, coldef))
+                continue
+            expected_type = normalize_type(compiler.compile_column_type(coldef))
+            actual_type = normalize_type(actual_col.raw_type)
+            type_changed = expected_type != actual_type
+            # SQLite reports INTEGER PRIMARY KEY as nullable, so nullability
+            # is only compared on non-key columns.
+            null_changed = (
+                not coldef.primary_key
+                and not actual_col.primary_key
+                and actual_col.nullable != coldef.nullable
+            )
+            if type_changed or null_changed:
+                expected_desc = expected_type + ("" if coldef.nullable else " NOT NULL")
+                actual_desc = actual_type + ("" if actual_col.nullable else " NOT NULL")
+                diff.changed_columns.append(
+                    (model.tableName or "", name, actual_desc, expected_desc)
+                )
+        declared_names = {c.lower() for c in model.tableColumns}
+        for name in actual_columns:
+            if name not in declared_names:
+                diff.extra_columns.append((model.tableName or "", name))
+
+    for table_key in actual:
+        if table_key not in declared and table_key not in excluded:
+            diff.extra_tables.append(table_key)
+
+    return diff
+
+
+def autogenerate(
+    connection: Any,
+    models: List[Type["Model"]],
+    id: str,
+    dialect: Dialects = Dialects.DEFAULT,
+    allow_drops: bool = False,
+    ignore_changed_columns: bool = False,
+    exclude_tables: Tuple[str, ...] = ("sustained_migrations",),
+) -> Optional[Migration]:
+    """
+    Diffs the database against the models and builds a Migration for the
+    differences. Returns None when the schema is up to date.
+
+    Missing tables and new columns generate reversible steps. Extra tables
+    and columns generate drops only with allow_drops=True, and the
+    migration then has no down step, since dropped data cannot come back.
+    Changed columns raise unless ignore_changed_columns=True, because type
+    and nullability rewrites need a hand-written migration.
+    """
+    compiler = Dialects.get_compiler(dialect)
+    diff = diff_schema(connection, models, dialect, exclude_tables)
+
+    if diff.changed_columns and not ignore_changed_columns:
+        details = "; ".join(
+            f"{table}.{name}: {actual} -> {expected}"
+            for table, name, actual, expected in diff.changed_columns
+        )
+        raise ValueError(
+            "Changed columns need a hand-written migration: "
+            f"{details}. Pass ignore_changed_columns=True to skip them."
+        )
+    if (diff.extra_tables or diff.extra_columns) and not allow_drops:
+        dropped = [t for t in diff.extra_tables] + [
+            f"{t}.{c}" for t, c in diff.extra_columns
+        ]
+        raise ValueError(
+            "The database has objects the models do not declare: "
+            f"{', '.join(dropped)}. Pass allow_drops=True to generate the "
+            "drops, or add them to exclude_tables."
+        )
+
+    up_steps: List[str] = []
+    down_steps: List[str] = []
+    reversible = True
+
+    for model in diff.missing_tables:
+        assert model.tableColumns is not None
+        table_sql = model._qualified_table_sql()
+        up_steps.append(build_create_table_sql(compiler, table_sql, model.tableColumns))
+        down_steps.insert(0, model.drop_table_sql())
+
+    for model, name, coldef in diff.new_columns:
+        if coldef.primary_key or coldef.autoincrement:
+            raise ValueError(
+                f"Cannot add '{model.tableName}.{name}' with ALTER TABLE: "
+                "primary key and autoincrement columns need a hand-written "
+                "migration."
+            )
+        if not coldef.nullable and coldef.default is None:
+            raise ValueError(
+                f"Cannot add NOT NULL column '{model.tableName}.{name}' "
+                "without a default; existing rows would have no value."
+            )
+        table_sql = model._qualified_table_sql()
+        column_sql = render_column_sql(compiler, name, coldef, inline_pk=False)
+        up_steps.append(compiler.compile_add_column(table_sql, column_sql))
+        down_steps.insert(0, compiler.compile_drop_column(table_sql, name))
+
+    if allow_drops:
+        for table, name in diff.extra_columns:
+            table_sql = compiler.quote_fully_qualified_identifier(table)
+            up_steps.append(compiler.compile_drop_column(table_sql, name))
+            reversible = False
+        for table in diff.extra_tables:
+            table_sql = compiler.quote_fully_qualified_identifier(table)
+            up_steps.append(f"DROP TABLE {table_sql}")
+            reversible = False
+
+    if not up_steps:
+        return None
+    return Migration(
+        id=id, up=up_steps, down=down_steps if reversible and down_steps else None
+    )

{sustained-2.2.0 → sustained-2.3.0}/src/sustained/compilers/base.py

@@ -224,6 +224,15 @@ class Compiler:
         """
         return ""
 
+    def compile_add_column(self, table_sql: str, column_sql: str) -> str:
+        """Renders an ALTER TABLE statement that adds one column."""
+        return f"ALTER TABLE {table_sql} ADD COLUMN {column_sql}"
+
+    def compile_drop_column(self, table_sql: str, column_name: str) -> str:
+        """Renders an ALTER TABLE statement that drops one column."""
+        quoted = self.quote_identifier(column_name)
+        return f"ALTER TABLE {table_sql} DROP COLUMN {quoted}"
+
     def compile_returning(self, columns_sql: str) -> str:
         """
         Renders a RETURNING clause for DML statements. Dialects without

{sustained-2.2.0 → sustained-2.3.0}/src/sustained/compilers/mssql.py

@@ -74,6 +74,10 @@ class MssqlCompiler(Compiler):
     def compile_identity(self) -> str:
         return "IDENTITY(1,1)"
 
+    def compile_add_column(self, table_sql: str, column_sql: str) -> str:
+        # T-SQL spells it ADD without the COLUMN keyword.
+        return f"ALTER TABLE {table_sql} ADD {column_sql}"
+
     def compile_with_keyword(self, recursive: bool) -> str:
         # T-SQL uses plain WITH for recursive CTEs.
         return "WITH"

{sustained-2.2.0 → sustained-2.3.0}/src/sustained/migrations.py

@@ -85,6 +85,7 @@ class Migrator:
         self._connection = connection
         self._migrations = list(migrations)
         self._table = table
+        self._dialect = dialect
         self._compiler = Dialects.get_compiler(dialect)
 
     def _table_sql(self) -> str:
@@ -152,6 +153,51 @@ class Migrator:
             applied_now.append(migration.id)
         return applied_now
 
+    def sync(
+        self,
+        models: List[Type["Model"]],
+        allow_drops: bool = False,
+        ignore_changed_columns: bool = False,
+        migration_id: Optional[str] = None,
+    ) -> List[str]:
+        """
+        Diffs the database against the models, registers the generated
+        migration, and applies everything pending. Returns the applied ids;
+        an empty list means the schema was already up to date.
+
+        Additive changes generate reversible steps, so down() rolls the
+        sync back. Drops require allow_drops=True and are not reversible.
+        """
+        from sustained.autogenerate import autogenerate
+
+        self._ensure_tracking_table()
+        generated_id = migration_id or datetime.now(timezone.utc).strftime(
+            "auto_%Y%m%d%H%M%S_%f"
+        )
+        migration = autogenerate(
+            self._connection,
+            models,
+            id=generated_id,
+            dialect=self._dialect,
+            allow_drops=allow_drops,
+            ignore_changed_columns=ignore_changed_columns,
+            exclude_tables=(self._table,),
+        )
+        if migration is not None:
+            self._migrations.append(migration)
+        return self.up()
+
+    def down_to(self, target: str) -> List[str]:
+        """
+        Reverts applied migrations newest-first until the target is the
+        most recent applied migration. The target itself stays applied.
+        """
+        applied = self.applied()
+        if target not in applied:
+            raise ValueError(f"Migration '{target}' is not applied.")
+        steps = len(applied) - applied.index(target) - 1
+        return self.down(steps) if steps else []
+
     def down(self, steps: int = 1) -> List[str]:
         """
         Reverts the most recently applied migrations, newest first. Every

{sustained-2.2.0 → sustained-2.3.0}/src/sustained/schema.py

@@ -119,6 +119,34 @@ def Json(**kwargs: Any) -> ColumnDef:
     return ColumnDef("JSON", **kwargs)
 
 
+def render_column_sql(
+    compiler: "Compiler",
+    name: str,
+    col: ColumnDef,
+    inline_pk: bool,
+) -> str:
+    """Renders one column definition for CREATE TABLE or ADD COLUMN."""
+    parts = [compiler.quote_identifier(name), compiler.compile_column_type(col)]
+    if col.autoincrement:
+        identity = compiler.compile_identity()
+        if identity:
+            parts.append(identity)
+    if col.primary_key and inline_pk:
+        parts.append("PRIMARY KEY")
+    elif not col.nullable:
+        parts.append("NOT NULL")
+    if col.unique and not col.primary_key:
+        parts.append("UNIQUE")
+    if col.default is not None:
+        parts.append(f"DEFAULT {compiler.format_value(col.default)}")
+    if col.references is not None:
+        ref_table, ref_column = col.references.rsplit(".", 1)
+        quoted_table = compiler.quote_fully_qualified_identifier(ref_table)
+        quoted_column = compiler.quote_identifier(ref_column)
+        parts.append(f"REFERENCES {quoted_table} ({quoted_column})")
+    return " ".join(parts)
+
+
 def build_create_table_sql(
     compiler: "Compiler",
     table_sql: str,
@@ -144,25 +172,7 @@ def build_create_table_sql(
     inline_pk = len(primary_keys) == 1
 
     for name, col in columns.items():
-        parts = [compiler.quote_identifier(name), compiler.compile_column_type(col)]
-        if col.autoincrement:
-            identity = compiler.compile_identity()
-            if identity:
-                parts.append(identity)
-        if col.primary_key and inline_pk:
-            parts.append("PRIMARY KEY")
-        elif not col.nullable:
-            parts.append("NOT NULL")
-        if col.unique and not col.primary_key:
-            parts.append("UNIQUE")
-        if col.default is not None:
-            parts.append(f"DEFAULT {compiler.format_value(col.default)}")
-        if col.references is not None:
-            ref_table, ref_column = col.references.rsplit(".", 1)
-            quoted_table = compiler.quote_fully_qualified_identifier(ref_table)
-            quoted_column = compiler.quote_identifier(ref_column)
-            parts.append(f"REFERENCES {quoted_table} ({quoted_column})")
-        column_parts.append(" ".join(parts))
+        column_parts.append(render_column_sql(compiler, name, col, inline_pk))
 
     if len(primary_keys) > 1:
         pk_sql = ", ".join(compiler.quote_identifier(c) for c in primary_keys)

sustained-2.3.0/tests/test_autogenerate.py

@@ -0,0 +1,204 @@
+"""
+Tests for schema diffing, migration autogeneration, and rollback of
+generated migrations, against in-memory SQLite.
+"""
+
+import sqlite3
+import unittest
+
+from sustained import Model, create_model
+from sustained.autogenerate import (
+    autogenerate,
+    diff_schema,
+    introspect_schema,
+    normalize_type,
+)
+from sustained.dialects import Dialects
+from sustained.migrations import Migration, Migrator
+from sustained.schema import Boolean, Integer, String, Text
+
+
+def make_model(name, table, columns):
+    model = create_model(name, table)
+    model.tableColumns = columns
+    model.columns = tuple(columns)
+    return model
+
+
+class AutogenTestCase(unittest.TestCase):
+    def setUp(self):
+        self.conn = sqlite3.connect(":memory:")
+        self.User = make_model(
+            f"AgU_{self.id().rsplit('.', 1)[-1]}",
+            "ag_users",
+            {
+                "id": Integer(primary_key=True),
+                "email": String(120, nullable=False),
+            },
+        )
+
+    def tearDown(self):
+        self.conn.close()
+
+
+class TestNormalizeType(unittest.TestCase):
+    def test_synonyms_and_parameters(self):
+        self.assertEqual(normalize_type("VARCHAR(120)"), "VARCHAR")
+        self.assertEqual(normalize_type("character varying"), "VARCHAR")
+        self.assertEqual(normalize_type("NVARCHAR(MAX)"), "VARCHAR")
+        self.assertEqual(normalize_type("DOUBLE PRECISION"), "FLOAT")
+        self.assertEqual(normalize_type("jsonb"), "JSON")
+        self.assertEqual(normalize_type("DATETIME2"), "TIMESTAMP")
+        self.assertEqual(normalize_type("BIT"), "BOOLEAN")
+
+    def test_unknown_passthrough(self):
+        self.assertEqual(normalize_type("GEOMETRY"), "GEOMETRY")
+
+
+class TestDiffSchema(AutogenTestCase):
+    def test_missing_table_detected(self):
+        diff = diff_schema(self.conn, [self.User])
+        self.assertEqual(diff.missing_tables, [self.User])
+        self.assertIn("create table ag_users", diff.summary())
+
+    def test_round_trip_is_empty(self):
+        self.User.create_table(self.conn)
+        diff = diff_schema(self.conn, [self.User])
+        self.assertTrue(diff.is_empty())
+        self.assertEqual(diff.summary(), "schema up to date")
+
+    def test_new_column_detected(self):
+        self.User.create_table(self.conn)
+        self.User.tableColumns["bio"] = Text()
+        diff = diff_schema(self.conn, [self.User])
+        self.assertEqual(len(diff.new_columns), 1)
+        self.assertEqual(diff.new_columns[0][1], "bio")
+
+    def test_extra_objects_detected(self):
+        self.User.create_table(self.conn)
+        self.conn.execute("ALTER TABLE ag_users ADD COLUMN legacy TEXT")
+        self.conn.execute("CREATE TABLE orphan (id INTEGER)")
+        diff = diff_schema(self.conn, [self.User])
+        self.assertEqual(diff.extra_columns, [("ag_users", "legacy")])
+        self.assertEqual(diff.extra_tables, ["orphan"])
+
+    def test_changed_column_detected(self):
+        self.User.create_table(self.conn)
+        changed = make_model(
+            "AgChanged",
+            "ag_users",
+            {"id": Integer(primary_key=True), "email": Boolean()},
+        )
+        diff = diff_schema(self.conn, [changed])
+        self.assertEqual(len(diff.changed_columns), 1)
+        self.assertIn("not auto-migrated", diff.summary())
+
+    def test_tracking_table_excluded(self):
+        self.conn.execute("CREATE TABLE sustained_migrations (id TEXT)")
+        self.User.create_table(self.conn)
+        diff = diff_schema(self.conn, [self.User])
+        self.assertEqual(diff.extra_tables, [])
+
+    def test_duplicate_table_declarations_rejected(self):
+        other = make_model("AgDup", "ag_users", {"id": Integer(primary_key=True)})
+        with self.assertRaises(ValueError):
+            diff_schema(self.conn, [self.User, other])
+
+    def test_model_without_table_columns_rejected(self):
+        bare = create_model("AgBare", "bare_tbl")
+        with self.assertRaises(ValueError):
+            diff_schema(self.conn, [bare])
+
+
+class TestAutogenerate(AutogenTestCase):
+    def test_up_to_date_returns_none(self):
+        self.User.create_table(self.conn)
+        self.assertIsNone(autogenerate(self.conn, [self.User], id="noop"))
+
+    def test_create_table_migration_is_reversible(self):
+        migration = autogenerate(self.conn, [self.User], id="m1")
+        self.assertEqual(migration.down, ["DROP TABLE IF EXISTS ag_users"])
+        self.assertIn("CREATE TABLE", migration.up[0])
+
+    def test_add_column_migration_is_reversible(self):
+        self.User.create_table(self.conn)
+        self.User.tableColumns["bio"] = Text()
+        migration = autogenerate(self.conn, [self.User], id="m2")
+        self.assertEqual(migration.up, ["ALTER TABLE ag_users ADD COLUMN bio TEXT"])
+        self.assertEqual(migration.down, ["ALTER TABLE ag_users DROP COLUMN bio"])
+
+    def test_drops_require_opt_in(self):
+        self.User.create_table(self.conn)
+        self.conn.execute("ALTER TABLE ag_users ADD COLUMN legacy TEXT")
+        with self.assertRaises(ValueError):
+            autogenerate(self.conn, [self.User], id="m3")
+        migration = autogenerate(self.conn, [self.User], id="m3", allow_drops=True)
+        self.assertEqual(migration.up, ["ALTER TABLE ag_users DROP COLUMN legacy"])
+        self.assertIsNone(migration.down)
+
+    def test_changed_columns_block_generation(self):
+        self.User.create_table(self.conn)
+        changed = make_model(
+            "AgBlock",
+            "ag_users",
+            {"id": Integer(primary_key=True), "email": Boolean()},
+        )
+        with self.assertRaises(ValueError):
+            autogenerate(self.conn, [changed], id="m4")
+        self.assertIsNone(
+            autogenerate(self.conn, [changed], id="m4", ignore_changed_columns=True)
+        )
+
+    def test_not_null_add_without_default_rejected(self):
+        self.User.create_table(self.conn)
+        self.User.tableColumns["req"] = String(10, nullable=False)
+        with self.assertRaises(ValueError):
+            autogenerate(self.conn, [self.User], id="m5")
+
+    def test_primary_key_add_rejected(self):
+        self.User.create_table(self.conn)
+        self.User.tableColumns["id2"] = Integer(primary_key=True)
+        with self.assertRaises(ValueError):
+            autogenerate(self.conn, [self.User], id="m6")
+
+
+class TestMigratorSync(AutogenTestCase):
+    def test_sync_creates_and_is_idempotent(self):
+        migrator = Migrator(self.conn, [])
+        applied = migrator.sync([self.User])
+        self.assertEqual(len(applied), 1)
+        self.assertTrue(diff_schema(self.conn, [self.User]).is_empty())
+        self.assertEqual(migrator.sync([self.User]), [])
+
+    def test_sync_then_down_rolls_back(self):
+        migrator = Migrator(self.conn, [])
+        migrator.sync([self.User])
+        self.User.tableColumns["bio"] = Text()
+        migrator.sync([self.User])
+        migrator.down()
+        columns = introspect_schema(self.conn)["ag_users"]
+        self.assertNotIn("bio", columns)
+
+    def test_down_to_target(self):
+        migrator = Migrator(
+            self.conn,
+            [
+                Migration("a", up="CREATE TABLE ta (id INTEGER)", down="DROP TABLE ta"),
+                Migration("b", up="CREATE TABLE tb (id INTEGER)", down="DROP TABLE tb"),
+                Migration("c", up="CREATE TABLE tc (id INTEGER)", down="DROP TABLE tc"),
+            ],
+        )
+        migrator.up()
+        reverted = migrator.down_to("a")
+        self.assertEqual(reverted, ["c", "b"])
+        self.assertEqual(migrator.applied(), ["a"])
+        self.assertEqual(migrator.down_to("a"), [])
+
+    def test_down_to_unapplied_target_raises(self):
+        migrator = Migrator(self.conn, [])
+        with self.assertRaises(ValueError):
+            migrator.down_to("nope")
+
+
+if __name__ == "__main__":
+    unittest.main()

sustained-2.2.0/docs/schema.md

@@ -1,64 +0,0 @@
----
-layout: default
-title: Schema and Migrations
----
-
-Models can declare their physical shape and generate DDL from it, and the migration runner applies ordered schema changes.
-
-## Typed Columns
-
-Declare `tableColumns` as a dict of column name to a typed definition. The factories are `Integer`, `BigInteger`, `String(length)`, `Text`, `Boolean`, `Float`, `Numeric(precision, scale)`, `Date`, `Timestamp`, and `Json`.
-
-```python
-from sustained import Model
-from sustained.schema import Boolean, Integer, String, Timestamp
-from sustained.types import Expression
-
-class User(Model):
-    tableName = 'users'
-    tableColumns = {
-        'id': Integer(primary_key=True, autoincrement=True),
-        'email': String(120, unique=True, nullable=False),
-        'active': Boolean(default=True),
-        'created_at': Timestamp(default=Expression('CURRENT_TIMESTAMP')),
-    }
-```
-
-Definitions support composite primary keys (mark several columns `primary_key=True`), `unique`, literal defaults or raw `Expression` defaults, and foreign keys through `references='table.column'`. `autoincrement` requires a single integer primary key; DuckDB and Presto raise because they have no identity columns.
-
-A model with `tableColumns` also gets strict column-name access automatically: a typo'd column raises `AttributeError`.
-
-## Generating and Running DDL
-
-```python
-User.create_table_sql()               # dialect-specific CREATE TABLE
-User.create_table(conn)               # execute it
-User.drop_table(conn)                 # DROP TABLE IF EXISTS
-```
-
-Types map per dialect: `BIT`, `NVARCHAR`, and `DATETIME2` on MSSQL; `JSONB` and identity columns on Postgres; plain `INTEGER PRIMARY KEY` rowid behavior on the default dialect.
-
-## Migrations
-
-Migrations are explicit and ordered. Each pairs an id with an up step and an optional down step; steps are a SQL string, a list of statements, or a callable receiving the connection.
-
-```python
-from sustained.migrations import Migration, Migrator, create_table_migration
-
-migrations = [
-    create_table_migration(User),
-    Migration(
-        'add_last_login',
-        up='ALTER TABLE users ADD COLUMN last_login TEXT',
-        down='ALTER TABLE users DROP COLUMN last_login',
-    ),
-]
-
-migrator = Migrator(conn, migrations)
-migrator.up()          # apply all pending
-migrator.up(target='create_users')  # stop after a target
-migrator.status()      # [(id, applied), ...]
-migrator.down()        # revert the newest applied migration
-```
-
-Applied ids live in a tracking table that the migrator creates on first use. Each migration runs inside a transaction, so a failing step leaves the schema at the previous migration. There is no automatic diffing against the database catalog; write migrations explicitly or derive create/drop pairs from models with `create_table_migration()`.