plain.postgres 0.107.1__tar.gz → 0.108.1__tar.gz

{plain_postgres-0.107.1 → plain_postgres-0.108.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: plain.postgres
-Version: 0.107.1
+Version: 0.108.1
 Summary: Model your data and store it in a database.
 Author-email: Dave Gaeddert <dave.gaeddert@dropseed.dev>
 License-Expression: BSD-3-Clause

{plain_postgres-0.107.1 → plain_postgres-0.108.1}/plain/postgres/CHANGELOG.md

@@ -1,5 +1,28 @@
 # plain-postgres changelog
 
+## [0.108.1](https://github.com/dropseed/plain/releases/plain-postgres@0.108.1) (2026-06-09)
+
+### What's changed
+
+- `ensure_connection()` now detects a pooled connection the server has closed while it was held (Postgres restart, failover, idle timeout) and discards it before establishing a fresh one — logging a "Discarding dead database connection" warning — instead of repeatedly failing on the dead connection. Inside an atomic block the dead connection is intentionally left in place so the transaction's normal error handling runs, rather than silently continuing the rest of the block outside its transaction. ([9e82cb4454](https://github.com/dropseed/plain/commit/9e82cb4454))
+- Query span instrumentation no longer pays for expensive attributes when the span is sampled out: the per-query Python stack walk (for `code.*` attributes) and the `DEBUG`-mode stringification of query parameters now only happen when the span is actually recording. The cheap connection attributes are still passed at span creation so attribute-aware samplers can see them in `should_sample()`. ([af736043b0](https://github.com/dropseed/plain/commit/af736043b0))
+
+### Upgrade instructions
+
+- No changes required.
+
+## [0.108.0](https://github.com/dropseed/plain/releases/plain-postgres@0.108.0) (2026-06-09)
+
+### What's changed
+
+- A partial index of exactly `WHERE <fk> IS NOT NULL` on the FK column now counts as FK index coverage — every FK lookup and referencing-side sweep is a `WHERE fk = ?`, which implies the predicate, so Postgres can always use it. Both the `postgres.missing_fk_indexes` preflight check (an `Index`/`UniqueConstraint` with `condition=Q(fk__isnull=False)`) and the doctor's missing-FK check (matching the live index predicate) accept it; any other partial predicate still doesn't count. ([4836069767](https://github.com/dropseed/plain/commit/4836069767))
+- The missing-FK-index warning now points out when an existing composite index or constraint already contains the FK at a non-leading position, suggesting a reorder to lead with the FK instead of adding a new index (safe when every query using the composite filters all of its columns with equality). ([084b6abba5](https://github.com/dropseed/plain/commit/084b6abba5))
+- Internal: the preflight checks moved from a single `preflight.py` into a `preflight/` package (`models.py`, `database.py`, `indexes.py`). Check names and behavior are unchanged. ([edc792d358](https://github.com/dropseed/plain/commit/edc792d358))
+
+### Upgrade instructions
+
+- No changes required.
+
 ## [0.107.1](https://github.com/dropseed/plain/releases/plain-postgres@0.107.1) (2026-06-08)
 
 ### What's changed

{plain_postgres-0.107.1 → plain_postgres-0.108.1}/plain/postgres/connection.py

@@ -170,7 +170,19 @@ class DatabaseConnection:
         self.set_autocommit(True)
 
     def ensure_connection(self) -> None:
-        """Guarantee that a connection to the database is established."""
+        """Guarantee that a live connection to the database is established."""
+        if (
+            self.connection is not None
+            and self.connection.closed
+            and not self.in_atomic_block
+        ):
+            # The server closed this connection while we held it (restart,
+            # failover, idle timeout) — psycopg marks it closed once an
+            # operation fails on it. Mid-atomic, swapping it would silently
+            # run the rest of the block outside its transaction — leave the
+            # dead connection for Atomic.__exit__'s error recovery instead.
+            logger.warning("Discarding dead database connection")
+            self.close()
         if self.connection is None:
             self.connect()
 

{plain_postgres-0.107.1 → plain_postgres-0.108.1}/plain/postgres/introspection/health/checks_structural.py

@@ -192,10 +192,11 @@ def check_missing_fk_indexes(
     Partial indexes (``WHERE`` clause set on ``pg_index.indpred``) don't
     count: Postgres only uses them for queries that imply the partial
     predicate, so FK lookups and cascade deletes outside that predicate
-    still sequential-scan. The narrow ``WHERE fk IS NOT NULL`` case —
-    which Postgres can match to ``WHERE fk = ?`` — is conservatively
-    treated as not covering; users wanting guaranteed FK coverage should
-    add a regular non-partial index. Match the preflight's coverage rule.
+    still sequential-scan. The one exception is a predicate of exactly
+    ``<fk> IS NOT NULL`` on the FK column itself — every FK lookup and
+    referencing-side sweep is a ``WHERE fk = ?``, which implies
+    ``fk IS NOT NULL``, so Postgres can always use that partial. Match
+    the preflight's coverage rule.
     """
     cursor.execute("""
         SELECT
@@ -216,7 +217,11 @@ def check_missing_fk_indexes(
               FROM pg_catalog.pg_index i
               WHERE i.indrelid = c.conrelid
                 AND i.indkey[0] = c.conkey[1]
-                AND i.indpred IS NULL
+                AND (
+                    i.indpred IS NULL
+                    OR pg_get_expr(i.indpred, i.indrelid) =
+                        format('(%s IS NOT NULL)', quote_ident(a.attname))
+                )
           )
         ORDER BY ct.relname, a.attname
     """)

{plain_postgres-0.107.1 → plain_postgres-0.108.1}/plain/postgres/migrations/state.py

@@ -550,7 +550,7 @@ class StateModelsRegistry(ModelsRegistry):
         self.ready = True
 
         # There shouldn't be any operations pending at this point.
-        from plain.postgres.preflight import _check_lazy_references
+        from plain.postgres.preflight.models import _check_lazy_references
 
         if errors := _check_lazy_references(self, packages_registry):
             raise ValueError("\n".join(error.fix for error in errors))

{plain_postgres-0.107.1 → plain_postgres-0.108.1}/plain/postgres/otel.py

@@ -305,7 +305,10 @@ def db_span(
     # Single settings_dict read — the property delegates to source.config.
     cfg = db.settings_dict
 
-    # Build attribute set following semantic conventions
+    # Cheap attributes are passed at span creation so attribute-aware
+    # samplers can see them in should_sample(). Expensive ones (the
+    # per-query stack walk, DEBUG params) are added after, only when the
+    # span actually records.
     attrs: dict[str, Any] = {
         DB_SYSTEM_NAME: DB_SYSTEM,
         DB_NAMESPACE: cfg.get("DATABASE"),
@@ -314,8 +317,6 @@ def db_span(
         DB_OPERATION_NAME: operation,
     }
 
-    attrs.update(_get_code_attributes())
-
     # Add collection name if detected
     if collection_name:
         attrs[DB_COLLECTION_NAME] = collection_name
@@ -335,31 +336,41 @@ def db_span(
             attrs[SERVER_PORT] = port_int
             attrs[NETWORK_PEER_PORT] = port_int
 
-    # Add query parameters as attributes when DEBUG is True
-    if settings.DEBUG and params is not None:
-        # Convert params to appropriate format based on type
-        if isinstance(params, dict):
-            # Dictionary params (e.g., for named placeholders)
-            for key, value in params.items():
-                attrs[f"{DB_QUERY_PARAMETER_TEMPLATE}.{key}"] = str(value)
-        elif isinstance(params, list | tuple):
-            # Sequential params (e.g., for %s or ? placeholders)
-            for i, value in enumerate(params):
-                attrs[f"{DB_QUERY_PARAMETER_TEMPLATE}.{i + 1}"] = str(value)
-        else:
-            # Single param (rare but possible)
-            attrs[f"{DB_QUERY_PARAMETER_TEMPLATE}.1"] = str(params)
-
     with tracer.start_as_current_span(
         span_name, kind=SpanKind.CLIENT, attributes=attrs
     ) as span:
+        if span.is_recording():
+            expensive_attrs = _get_code_attributes()
+
+            # Add query parameters as attributes when DEBUG is True
+            if settings.DEBUG and params is not None:
+                # Convert params to appropriate format based on type
+                if isinstance(params, dict):
+                    # Dictionary params (e.g., for named placeholders)
+                    for key, value in params.items():
+                        expensive_attrs[f"{DB_QUERY_PARAMETER_TEMPLATE}.{key}"] = str(
+                            value
+                        )
+                elif isinstance(params, list | tuple):
+                    # Sequential params (e.g., for %s or ? placeholders)
+                    for i, value in enumerate(params):
+                        expensive_attrs[f"{DB_QUERY_PARAMETER_TEMPLATE}.{i + 1}"] = str(
+                            value
+                        )
+                else:
+                    # Single param (rare but possible)
+                    expensive_attrs[f"{DB_QUERY_PARAMETER_TEMPLATE}.1"] = str(params)
+
+            span.set_attributes(expensive_attrs)
+
         start = time.perf_counter()
         try:
             yield span
         except Exception as exc:
             # record_exception + set_status(ERROR) handled by
             # start_as_current_span when the exception propagates out.
-            span.set_attribute(ERROR_TYPE, format_exception_type(exc))
+            if span.is_recording():
+                span.set_attribute(ERROR_TYPE, format_exception_type(exc))
             raise
         duration_s = time.perf_counter() - start
 

plain_postgres-0.108.1/plain/postgres/preflight/__init__.py

@@ -0,0 +1,2 @@
+# Import these to force registration of checks
+from . import database, indexes, models  # NOQA

plain_postgres-0.108.1/plain/postgres/preflight/database.py

@@ -0,0 +1,158 @@
+"""Preflight checks on the database connection and migration state."""
+
+from __future__ import annotations
+
+from plain.postgres.db import get_connection
+from plain.preflight import PreflightCheck, PreflightResult, register_check
+from plain.runtime import settings
+
+
+@register_check("postgres.middleware_installed")
+class CheckMiddlewareInstalled(PreflightCheck):
+    """Errors if `DatabaseConnectionMiddleware` isn't in `MIDDLEWARE`.
+
+    Without it, pooled connections are only released by GC at the end of
+    each request — relying on refcount timing under load is a recipe for
+    pool exhaustion under cyclic refs or delayed finalization.
+    """
+
+    REQUIRED = "plain.postgres.DatabaseConnectionMiddleware"
+
+    def run(self) -> list[PreflightResult]:
+        if self.REQUIRED in settings.MIDDLEWARE:
+            return []
+        return [
+            PreflightResult(
+                fix=(
+                    f"Add '{self.REQUIRED}' to MIDDLEWARE so pooled "
+                    "database connections are returned at the end of each "
+                    "request. Place it first so its after_response runs "
+                    "after any middleware that queries the database."
+                ),
+                id="postgres.middleware_not_installed",
+            )
+        ]
+
+
+@register_check("postgres.postgres_version")
+class CheckPostgresVersion(PreflightCheck):
+    """Checks that the PostgreSQL server meets the minimum version requirement."""
+
+    MINIMUM_VERSION = 16
+
+    def run(self) -> list[PreflightResult]:
+        conn = get_connection()
+        conn.ensure_connection()
+        assert conn.connection is not None
+        major, minor = divmod(conn.connection.info.server_version, 10000)
+        if major < self.MINIMUM_VERSION:
+            return [
+                PreflightResult(
+                    fix=f"PostgreSQL {self.MINIMUM_VERSION} or later is required (found {major}.{minor}).",
+                    id="postgres.postgres_version_too_old",
+                )
+            ]
+        return []
+
+
+@register_check("postgres.database_tables")
+class CheckDatabaseTables(PreflightCheck):
+    """Checks for unknown tables in the database when plain.postgres is available."""
+
+    def run(self) -> list[PreflightResult]:
+        from plain.postgres.introspection import get_unknown_tables
+
+        unknown_tables = get_unknown_tables()
+
+        if not unknown_tables:
+            return []
+
+        table_names = ", ".join(unknown_tables)
+        return [
+            PreflightResult(
+                fix=f"Unknown tables in default database: {table_names}. "
+                "Tables may be from packages/models that have been uninstalled. "
+                "Make sure you have a backup, then run `plain postgres drop-unknown-tables` to remove them.",
+                id="postgres.unknown_database_tables",
+                warning=True,
+            )
+        ]
+
+
+@register_check("postgres.prunable_migrations")
+class CheckPrunableMigrations(PreflightCheck):
+    """Warns about stale migration records in the database."""
+
+    def run(self) -> list[PreflightResult]:
+        # Import here to avoid circular import issues
+        from plain.postgres.migrations.loader import MigrationLoader
+        from plain.postgres.migrations.recorder import MigrationRecorder
+
+        errors = []
+
+        # Load migrations from disk and database
+        conn = get_connection()
+        loader = MigrationLoader(conn, ignore_no_migrations=True)
+        recorder = MigrationRecorder(conn)
+        recorded_migrations = recorder.applied_migrations()
+
+        # disk_migrations should not be None after MigrationLoader initialization,
+        # but check to satisfy type checker
+        if loader.disk_migrations is None:
+            return errors
+
+        # Find all prunable migrations (recorded but not on disk)
+        all_prunable = [
+            migration
+            for migration in recorded_migrations
+            if migration not in loader.disk_migrations
+        ]
+
+        if not all_prunable:
+            return errors
+
+        # Separate into existing packages vs orphaned packages
+        existing_packages = set(loader.migrated_packages)
+        prunable_existing: list[tuple[str, str]] = []
+        prunable_orphaned: list[tuple[str, str]] = []
+
+        for migration in all_prunable:
+            package, name = migration
+            if package in existing_packages:
+                prunable_existing.append(migration)
+            else:
+                prunable_orphaned.append(migration)
+
+        # Build the warning message
+        total_count = len(all_prunable)
+        message_parts = [
+            f"Found {total_count} stale migration record{'s' if total_count != 1 else ''} in the database."
+        ]
+
+        if prunable_existing:
+            existing_list = ", ".join(
+                f"{pkg}.{name}" for pkg, name in prunable_existing[:3]
+            )
+            if len(prunable_existing) > 3:
+                existing_list += f" (and {len(prunable_existing) - 3} more)"
+            message_parts.append(f"From existing packages: {existing_list}.")
+
+        if prunable_orphaned:
+            orphaned_list = ", ".join(
+                f"{pkg}.{name}" for pkg, name in prunable_orphaned[:3]
+            )
+            if len(prunable_orphaned) > 3:
+                orphaned_list += f" (and {len(prunable_orphaned) - 3} more)"
+            message_parts.append(f"From removed packages: {orphaned_list}.")
+
+        message_parts.append("Run 'plain migrations prune' to review and remove them.")
+
+        errors.append(
+            PreflightResult(
+                fix=" ".join(message_parts),
+                id="postgres.prunable_migrations",
+                warning=True,
+            )
+        )
+
+        return errors

plain_postgres-0.108.1/plain/postgres/preflight/indexes.py

@@ -0,0 +1,250 @@
+"""Preflight checks for index coverage on app models."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from plain.packages import packages_registry
+from plain.postgres.constraints import UniqueConstraint
+from plain.postgres.expressions import F, OrderBy
+from plain.postgres.fields.related import ForeignKeyField
+from plain.postgres.query_utils import Q
+from plain.postgres.registry import models_registry
+from plain.preflight import PreflightCheck, PreflightResult, register_check
+
+
+def _get_app_models() -> list[Any]:
+    """Return models from the user's app packages only (not framework/third-party)."""
+    app_models = []
+    for package_config in packages_registry.get_package_configs():
+        if package_config.name.startswith("app."):
+            app_models.extend(
+                models_registry.get_models(package_label=package_config.package_label)
+            )
+    return app_models
+
+
+def _collect_model_indexes(model: Any) -> list[tuple[str, list[str], bool]]:
+    """Collect (name, fields, is_unique) for non-partial indexes/constraints.
+
+    Partials are skipped: they only apply to rows matching their predicate,
+    so they're never interchangeable with a full index for duplicate
+    detection or reorder suggestions.
+    """
+    all_indexes: list[tuple[str, list[str], bool]] = []
+
+    for index in model.model_options.indexes:
+        if index.fields and not index.is_partial:
+            fields = [f.lstrip("-") for f in index.fields]
+            all_indexes.append((index.name, fields, False))
+
+    for constraint in model.model_options.constraints:
+        if (
+            isinstance(constraint, UniqueConstraint)
+            and constraint.fields
+            and not constraint.is_partial
+        ):
+            all_indexes.append((constraint.name, list(constraint.fields), True))
+
+    return all_indexes
+
+
+def _bare_column_name(expr: Any) -> str | None:
+    """Return the column name if `expr` resolves to a bare column, else `None`.
+
+    Postgres can range-scan the leading column of an index for `WHERE col = ?`
+    only when that column is a real attribute, not an expression — so a
+    compound leading expression like `Lower("email")` returns `None` here.
+    Sort direction (`F("col").desc()` / `OrderBy(F)`) doesn't affect equality
+    lookups, so we unwrap one layer of `OrderBy` around a bare `F`.
+    """
+    if isinstance(expr, OrderBy):
+        expr = expr.expression
+    if isinstance(expr, F):
+        return expr.name
+    return None
+
+
+def _leading_field_name(
+    fields: tuple[str, ...] | list[str], expressions: tuple
+) -> str | None:
+    """The leading column's field name, or `None` if it's an expression."""
+    if fields:
+        return fields[0].lstrip("-")
+    if expressions:
+        return _bare_column_name(expressions[0])
+    return None
+
+
+def _condition_is_not_null_on(condition: Q, field_name: str) -> bool:
+    """True if `condition` is exactly ``Q(<field_name>__isnull=False)``."""
+    return (
+        not condition.negated
+        and len(condition.children) == 1
+        and condition.children[0] == (f"{field_name}__isnull", False)
+    )
+
+
+def _fk_covered_field_names(model: Any) -> set[str]:
+    """Field names that appear as the leading column of an index or unique
+    constraint — covering arbitrary FK lookups via the index's leading
+    column. Includes expression-based indexes/constraints whose leading
+    expression is a bare ``F(field_name)``.
+
+    Partial indexes/constraints (declared with ``condition=Q(...)``) are
+    excluded: Postgres can only use them for queries whose predicate
+    implies the partial-index predicate, so an FK lookup or cascade
+    delete that doesn't filter by that condition still does a sequential
+    scan. The one exception is a predicate of exactly
+    ``Q(<fk>__isnull=False)`` on the leading FK itself — every FK lookup
+    and referencing-side sweep is a ``WHERE fk = ?``, which implies
+    ``fk IS NOT NULL``, so Postgres can always use that partial. Match
+    the doctor's coverage rule in
+    ``introspection/health/checks_structural.py``.
+    """
+    covered: set[str] = set()
+
+    def _record(index_or_constraint: Any) -> None:
+        leading = _leading_field_name(
+            index_or_constraint.fields, index_or_constraint.expressions
+        )
+        if leading is None:
+            return
+        if not index_or_constraint.is_partial or _condition_is_not_null_on(
+            index_or_constraint.condition, leading
+        ):
+            covered.add(leading)
+
+    for index in model.model_options.indexes:
+        _record(index)
+
+    for constraint in model.model_options.constraints:
+        if isinstance(constraint, UniqueConstraint):
+            _record(constraint)
+
+    return covered
+
+
+def _composite_containing(model: Any, field_name: str) -> tuple[str, list[str]] | None:
+    """First non-partial index/constraint with `field_name` at a non-leading position.
+
+    A non-leading column doesn't cover the FK, but reordering the composite to
+    lead with it often can — worth suggesting in the fix message.
+    """
+    for name, fields, _is_unique in _collect_model_indexes(model):
+        if field_name in fields[1:]:
+            return name, fields
+    return None
+
+
+@register_check("postgres.missing_fk_indexes")
+class CheckMissingFKIndexes(PreflightCheck):
+    """Warns about foreign key fields without index coverage."""
+
+    def run(self) -> list[PreflightResult]:
+        results = []
+
+        for model in _get_app_models():
+            covered_fields = _fk_covered_field_names(model)
+
+            for field in model._model_meta.local_fields:
+                if (
+                    isinstance(field, ForeignKeyField)
+                    and not field.primary_key
+                    and field.name not in covered_fields
+                ):
+                    fix = (
+                        f"Foreign key '{field.name}' has no index coverage. "
+                        f"Add an Index on [\"{field.name}\"] or a constraint with '{field.name}' as the first field."
+                    )
+
+                    if composite := _composite_containing(model, field.name):
+                        composite_name, composite_fields = composite
+                        fix += (
+                            f" Alternatively, '{composite_name}' on [{', '.join(composite_fields)}] "
+                            f"already includes '{field.name}' — reordering it to put '{field.name}' first "
+                            f"covers this FK without a new index (safe when every query using it "
+                            f"filters all of its columns with equality)."
+                        )
+
+                    results.append(
+                        PreflightResult(
+                            fix=fix,
+                            obj=f"{model.model_options.label}.{field.name}",
+                            id="postgres.missing_fk_index",
+                            warning=True,
+                        )
+                    )
+
+        return results
+
+
+@register_check("postgres.duplicate_indexes")
+class CheckDuplicateIndexes(PreflightCheck):
+    """Warns about indexes redundant with other indexes or constraints.
+
+    Catches both prefix-redundancy (a 1-column index shadowed by a wider
+    composite) and exact-column duplicates (an `Index(fields=["x"])` that
+    duplicates a same-column `UniqueConstraint`).
+    """
+
+    def run(self) -> list[PreflightResult]:
+        results = []
+
+        for model in _get_app_models():
+            all_indexes = _collect_model_indexes(model)
+
+            flagged: set[str] = set()
+            for i, idx_a in enumerate(all_indexes):
+                for idx_b in all_indexes[i + 1 :]:
+                    for shorter, longer in [(idx_a, idx_b), (idx_b, idx_a)]:
+                        s_name, s_fields, s_unique = shorter
+                        l_name, l_fields, l_unique = longer
+
+                        if s_name in flagged:
+                            continue
+
+                        is_prefix_dup = (
+                            not s_unique
+                            and len(s_fields) < len(l_fields)
+                            and l_fields[: len(s_fields)] == s_fields
+                        )
+                        is_exact_dup = (
+                            s_fields == l_fields
+                            and not s_unique
+                            and (l_unique or s_name > l_name)
+                        )
+
+                        if not (is_prefix_dup or is_exact_dup):
+                            continue
+
+                        if is_prefix_dup:
+                            fix = (
+                                f"Index '{s_name}' on [{', '.join(s_fields)}] "
+                                f"is redundant with '{l_name}' on [{', '.join(l_fields)}]. "
+                                f"The longer index covers the same queries."
+                            )
+                        elif l_unique:
+                            fix = (
+                                f"Index '{s_name}' on [{', '.join(s_fields)}] "
+                                f"is redundant with '{l_name}' on the same columns. "
+                                f"The unique-backed index already covers these queries."
+                            )
+                        else:
+                            fix = (
+                                f"Index '{s_name}' on [{', '.join(s_fields)}] "
+                                f"is an exact duplicate of '{l_name}'. "
+                                f"Drop one of them."
+                            )
+
+                        results.append(
+                            PreflightResult(
+                                fix=fix,
+                                obj=model.model_options.label,
+                                id="postgres.duplicate_index",
+                                warning=True,
+                            )
+                        )
+                        flagged.add(s_name)
+
+        return results