loom-kernel 2.11.1__tar.gz → 2.13.0__tar.gz

{loom_kernel-2.11.1 → loom_kernel-2.13.0}/CHANGELOG.md

@@ -4,6 +4,61 @@
 
 ### core
 
+- **core:** range-partitioned row-scoped tables.
+  `__partition_by__ = ("RANGE", "<column>")` on a `RowScoped` model compiles to
+  `postgresql_partition_by`; the column must be a `DateTime` column of the
+  primary key and every unique key must contain it, LIST and HASH are refused,
+  all at compile time. Partitions are created ahead of time by the migrator
+  through guard revision 2 (`0002.sql`, new functions `create_range_partition`
+  and `detach_partition`, `EXECUTE` for the owner only), which creates,
+  registers and protects each partition with its parent's scopes in the
+  caller's transaction: `op.ensure_range_partitions(table, start, end,
+  interval=...)` in a revision, or
+  `loom.core.repository.sqlalchemy.rls.ensure_range_partitions(url_or_connection,
+  application, model, start, end, interval=...)` at runtime. Partitions are
+  named `<table>_p<YYYYMM>` (`_p<YYYY>`, `_p<YYYYMMDD>` for year and day) with
+  midnight-UTC bounds. Retention uses `op.detach_range_partitions(...,
+  drop=...)` or `detach_range_partitions(...)`, which unprotects, detaches and
+  optionally drops. Dropping a partitioned table in an autogenerated revision
+  drops its partitions first (`op.drop_partitions`). `check` and `verify` count
+  registered partitions with their parent; `verify` reports
+  `partition.registration` for a partition registered unlike its parent.
+  Re-apply the bootstrap to install revision 2; partition calls on an older
+  guard raise `guard revision pending`. `ScopedTable` gains `partition_by`, and
+  `require_guard_revision(_sync)` take an optional `minimum`.
+- **core:** a model declares named checks and partial unique indexes.
+  `__checks__ = {rule: sql_expression}` compiles one `CHECK` constraint per
+  rule, named by the rule; `__partial_unique__ = {rule: (columns, where)}`
+  compiles a unique index `uq_<table>_<rule>` restricted to the rows matching
+  `where`, and on a row-scoped table its columns must contain the boundary
+  (C5, refused at compile time as the guard refuses it with `LG002`). The
+  expression and the predicate are product code with the trust of a
+  hand-written Alembic revision and must be static literals with no runtime
+  input; loom checks only that each is a non-empty string. The static-SQL lint
+  exempts exactly the two calls that compile them and now also watches
+  `CheckConstraint`. Alembic's autogenerate does not compare checks or index
+  predicates, so adding, changing or removing a check on an existing table,
+  or changing a predicate, is a hand-written revision.
+- **core:** `database.schema.naming_convention` hands a SQLAlchemy
+  `naming_convention` (`pk`, `fk`, `uq`, `ck`, `ix`) to the application
+  metadata of the migration path and to the metadata the REST runtime and the
+  Celery worker compile, so all of them name constraints alike; absent,
+  nothing changes. Use
+  `fk_%(table_name)s_%(column_0_N_name)s`: every composite FK of a scoped
+  table starts with the boundary column. Compilation now refuses two
+  constraints or indexes of one table resolving to the same name, and a
+  check, `__indexes__` or `__partial_unique__` name over Postgres's 63 bytes,
+  instead of failing or truncating at DDL time. `reset_registry` takes the
+  convention as `naming_convention=`; called without it, it leaves the
+  convention as it is, and `None` restores SQLAlchemy's default.
+
+- **core:** two column types. `Bytes` (`loom.core.model.Bytes`, also inferred
+  from a `bytes` annotation, which used to fall back to `JSON`) compiles to
+  `LargeBinary`, `bytea` on Postgres; `Postgres.INET` compiles to Postgres
+  `inet`. The SQLAlchemy repository binds `bytes` as they are instead of
+  base64 text, and a network address the driver returns for `inet` reaches a
+  `str` field as its text form (`10.0.0.1`, `2001:db8::1/64`).
+
 - **core:** a memory cache alias can be bounded. `cache: max_size` (entries) and
   the new `cache: max_bytes` (stored payload bytes) serve every
   `aiocache.SimpleMemoryCache` alias through `loom.core.cache.memory.BoundedMemoryCache`,
@@ -91,6 +146,12 @@
   (`_safe_pydantic_schema`) through the same `LoomType` the cache and MCP
   boundaries already use; `LoomRestAdapter`'s Struct projection goes
   through it too.
+- **rest:** `create_app` accepts `modules: Sequence[Callable[[LoomContainer], None]]`,
+  mirroring the Celery `create_app`. Each module is called with the container,
+  in order, before repositories are registered and before use cases are
+  verified, so a use case can inject a port no repository provides (a clock,
+  an identity verifier, a bridge to another bounded context). Without the
+  module, startup still fails with the same `ResolutionError`.
 
 ### prefect
 
@@ -231,6 +292,15 @@
 
 ### core
 
+- **core:** the RLS guard refuses, in the same statement, a drop outside the
+  hatch that breaks its invariants. `DROP POLICY` or `DROP TRIGGER
+  loom_deny_owner_dml` on a scoped table used to commit, because drops are not
+  reported to `ddl_command_end`; the schema then failed the assertion (`LG002`)
+  only at the next DDL. The `sql_drop` trigger now runs the assertion when the
+  drop removes an object of the application schema and the hatch is closed;
+  dropping a plain index still succeeds. Shipped in guard revision 2
+  (`0002.sql`, `CREATE OR REPLACE FUNCTION on_sql_drop`); re-apply the
+  bootstrap to install it.
 - **core:** a model with a relation or projection field imports again on
   Python 3.12.4 and later when its module uses
   `from __future__ import annotations`. It failed with

{loom_kernel-2.11.1 → loom_kernel-2.13.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: loom-kernel
-Version: 2.11.1
+Version: 2.13.0
 Summary: Loom Python project
 License-File: LICENSE
 Classifier: Programming Language :: Python :: 3
@@ -42,6 +42,9 @@ Provides-Extra: celery
 Requires-Dist: celery<6.0,>=5.3; extra == 'celery'
 Requires-Dist: kombu<6.0,>=5.3; extra == 'celery'
 Requires-Dist: redis<6.0,>=5.0; extra == 'celery'
+Provides-Extra: cli
+Requires-Dist: pyyaml>=6; extra == 'cli'
+Requires-Dist: typer<1,>=0.12; extra == 'cli'
 Provides-Extra: clickhouse
 Requires-Dist: clickhouse-connect<1.0,>=0.9.2; extra == 'clickhouse'
 Provides-Extra: config
@@ -103,6 +106,7 @@ Requires-Dist: pydantic<3.0,>=2.0; extra == 'rest'
 Requires-Dist: uvicorn<1.0,>=0.30; extra == 'rest'
 Requires-Dist: uvloop<1.0,>=0.21; (sys_platform != 'win32') and extra == 'rest'
 Provides-Extra: sqlalchemy
+Requires-Dist: alembic<2,>=1.13; extra == 'sqlalchemy'
 Requires-Dist: greenlet>=3.0; extra == 'sqlalchemy'
 Requires-Dist: sqlalchemy<3.0,>=2.0; extra == 'sqlalchemy'
 Provides-Extra: streaming

{loom_kernel-2.11.1 → loom_kernel-2.13.0}/pyproject.toml

@@ -23,6 +23,9 @@ dependencies = [
     "opentelemetry-api>=1.27,<2.0",
 ]
 
+[project.scripts]
+loom = "loom.cli:main"
+
 [project.optional-dependencies]
 # The testing helpers under loom.*.testing define pytest fixtures, so importing
 # them needs pytest at runtime like any other declared dependency.
@@ -40,6 +43,7 @@ rest = [
 sqlalchemy = [
     "sqlalchemy>=2.0,<3.0",
     "greenlet>=3.0",
+    "alembic>=1.13,<2",
 ]
 clickhouse = [
     "clickhouse-connect>=0.9.2,<1.0",
@@ -163,6 +167,10 @@ celery = [
     "kombu>=5.3,<6.0",
     "redis>=5.0,<6.0",
 ]
+cli = [
+    "pyyaml>=6",
+    "typer>=0.12,<1",
+]
 
 [project.entry-points."loom.ai.engines"]
 pydantic-ai = "loom.ai.engines.pydantic_ai.provider:PydanticAIEngineProvider"
@@ -225,6 +233,7 @@ dev = [
     "aiocache>=0.12,<1.0",
     "aiosqlite>=0.20.0",
     "asyncpg>=0.30",
+    "alembic>=1.13,<2",
     # Declared, not merely transitive: one cache test pins what the key renderer
     # cannot separate inside an object msgspec expands itself, and an attrs class
     # is the only such object the stdlib can build.
@@ -274,6 +283,7 @@ dev = [
     "fasta2a>=0.6,<1",
     "a2a-sdk>=1.1,<2",
     "pydantic-ai-harness[skills]>=0.28,<1",
+    "typer>=0.12,<1",
 ]
 
 [tool.hatch.build.targets.wheel]
@@ -287,6 +297,7 @@ packages = [
 include = [
     "src/loom/py.typed",
     "src/loom/ai/declarative/schemas/*.json",
+    "src/loom/core/repository/sqlalchemy/rls/guard/*.sql",
 ]
 
 [tool.hatch.build.targets.sdist]
@@ -408,6 +419,12 @@ module = [
 ]
 ignore_missing_imports = true
 
+[[tool.mypy.overrides]]
+module = [
+    "loom.core.backend.scoped_ddl",
+]
+disallow_untyped_calls = false
+
 [tool.pytest.ini_options]
 minversion = "8.0"
 testpaths = [

{loom_kernel-2.11.1 → loom_kernel-2.13.0}/src/loom/celery/bootstrap.py

@@ -118,6 +118,16 @@ _T = TypeVar("_T")
 # ---------------------------------------------------------------------------
 
 
+class _WorkerSchemaConfig(msgspec.Struct, kw_only=True):
+    """The part of ``database.schema`` the worker compiles models with.
+
+    Attributes:
+        naming_convention: SQLAlchemy naming convention keyed by constraint kind.
+    """
+
+    naming_convention: dict[str, str] | None = None
+
+
 class _WorkerDbConfig(msgspec.Struct, kw_only=True):
     """Database settings for the worker process.
 
@@ -125,11 +135,13 @@ class _WorkerDbConfig(msgspec.Struct, kw_only=True):
         url: Async SQLAlchemy connection URL.
         echo: Log all SQL statements when ``True``.
         pool_pre_ping: Test connections before checkout.
+        schema: Schema settings the models are compiled with.
     """
 
     url: str
     echo: bool = False
     pool_pre_ping: bool = True
+    schema: _WorkerSchemaConfig = msgspec.field(default_factory=_WorkerSchemaConfig)
 
 
 class _DiscoveryModules(msgspec.Struct, kw_only=True):
@@ -621,16 +633,28 @@ def _register_repositories(
     return build_sqlalchemy_repository_registration_module(session_manager, models)
 
 
-def _compile_models(models: Sequence[type[BaseModel]]) -> tuple[type[BaseModel], ...]:
+def _worker_naming_convention(ctx: ConfigContext) -> Mapping[str, str] | None:
+    """The ``database.schema.naming_convention`` the REST runtime and migrations also read."""
+    db_cfg = ctx.section(ConfigKey.DATABASE, _WorkerDbConfig)
+    return db_cfg.schema.naming_convention
+
+
+def _compile_models(
+    models: Sequence[type[BaseModel]], *, naming_convention: Mapping[str, str] | None
+) -> tuple[type[BaseModel], ...]:
     """Compile and normalize discovered models for SQLAlchemy repositories."""
     if not models:
         return ()
     try:
-        from loom.core.backend.sqlalchemy import compile_all, reset_registry
+        from loom.core.backend.sqlalchemy import (
+            compile_all,
+            configured_naming_convention,
+            reset_registry,
+        )
     except ImportError as exc:
         raise _sqlalchemy_extra_error() from exc
     ordered = tuple(dict.fromkeys(models))
-    reset_registry()
+    reset_registry(naming_convention=configured_naming_convention(naming_convention))
     compile_all(*ordered)
     return ordered
 
@@ -656,6 +680,8 @@ def _compile_db_layer(
     session_manager: SessionManager | None,
     models: Sequence[type[BaseModel]],
     user_modules: Sequence[Callable[[LoomContainer], None]],
+    *,
+    naming_convention: Mapping[str, str] | None,
 ) -> tuple[tuple[type[BaseModel], ...], tuple[Callable[[LoomContainer], None], ...]]:
     """Compile models and register repositories; no-op when DB is not configured.
 
@@ -663,6 +689,7 @@ def _compile_db_layer(
         session_manager: Active session manager, or ``None`` for pure jobs.
         models: Model classes discovered or declared for this worker.
         user_modules: Container modules provided by the caller.
+        naming_convention: The configured ``database.schema.naming_convention``.
 
     Returns:
         Tuple of (normalized_models, runtime_modules).  When no DB is
@@ -671,7 +698,7 @@ def _compile_db_layer(
     base = tuple(user_modules)
     if session_manager is None:
         return tuple(models), base
-    normalized = _compile_models(models)
+    normalized = _compile_models(models, naming_convention=naming_convention)
     if not normalized:
         return tuple(models), base
     return normalized, (*base, _register_repositories(session_manager, normalized))
@@ -904,7 +931,10 @@ def bootstrap_worker(
         _apply_job_config_if_present(ctx, job_type)
 
     uow_factory, session_manager = _resolve_uow_factory(ctx)
-    final_models, runtime_modules = _compile_db_layer(session_manager, resolved.models, modules)
+    convention = _worker_naming_convention(ctx) if session_manager is not None else None
+    final_models, runtime_modules = _compile_db_layer(
+        session_manager, resolved.models, modules, naming_convention=convention
+    )
     runtime_modules = (*runtime_modules, cache_module_for(ctx))
 
     resolved = _WorkerResolved(

loom_kernel-2.13.0/src/loom/cli/__init__.py

@@ -0,0 +1,20 @@
+"""The ``loom`` command line, installed with the ``cli`` extra."""
+
+from __future__ import annotations
+
+import sys
+
+CLI_DEPENDENCIES = frozenset({"typer", "yaml"})
+INSTALL_HINT = 'the loom command needs the cli extra: pip install "loom-kernel[cli]"'
+
+
+def main() -> None:
+    """Entry point of the ``loom`` script; without the ``cli`` extra it prints how to install it."""
+    try:
+        from loom.cli.app import app
+    except ModuleNotFoundError as exc:
+        if exc.name not in CLI_DEPENDENCIES:
+            raise
+        print(INSTALL_HINT, file=sys.stderr)
+        raise SystemExit(1) from exc
+    app()

loom_kernel-2.13.0/src/loom/cli/app.py

@@ -0,0 +1,10 @@
+"""The ``loom`` command tree."""
+
+from __future__ import annotations
+
+import typer
+
+from loom.cli.schema import schema_app
+
+app = typer.Typer(no_args_is_help=True, help="Operate the database side of a loom application.")
+app.add_typer(schema_app, name="schema")

loom_kernel-2.13.0/src/loom/cli/schema.py

@@ -0,0 +1,92 @@
+"""``loom schema``: the commands that prepare a row-scoped schema."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Annotated, Any
+
+import typer
+import yaml
+
+from loom.core.schema_names import SchemaNames
+
+schema_app = typer.Typer(no_args_is_help=True, help="Prepare a row-scoped schema.")
+
+
+@schema_app.command("init")
+def init(
+    name: Annotated[str, typer.Argument(help="The application schema.")],
+    config: Annotated[
+        Path | None,
+        typer.Option("--config", help="Merge the names into this configuration file."),
+    ] = None,
+) -> None:
+    """Derive the names around a schema once and write them into the configuration.
+
+    Without ``--config`` the block is printed. With it the names are merged
+    into ``database.schema``; a name the product already changed is never
+    overwritten.
+    """
+    try:
+        block = _block(name, SchemaNames.derived(name))
+        if config is None:
+            typer.echo(yaml.safe_dump({"database": {"schema": block}}, sort_keys=False), nl=False)
+            return
+        _write(config, block)
+    except (ValueError, yaml.YAMLError) as exc:
+        typer.echo(str(exc), err=True)
+        raise typer.Exit(1) from exc
+
+
+def _write(config: Path, block: dict[str, Any]) -> None:
+    document = _load(config)
+    database = _section(document, "database", "database")
+    schema = _section(database, "schema", "database.schema")
+    conflicts = _merge(schema, block, "database.schema")
+    if conflicts:
+        raise ValueError(
+            "these names differ from the derived ones and were kept: " + ", ".join(conflicts)
+        )
+    config.write_text(yaml.safe_dump(document, sort_keys=False))
+
+
+def _load(config: Path) -> dict[str, Any]:
+    if not config.exists():
+        return {}
+    loaded = yaml.safe_load(config.read_text())
+    if loaded is None:
+        return {}
+    if not isinstance(loaded, dict):
+        raise ValueError(f"{config} must hold a mapping at its top level")
+    return loaded
+
+
+def _section(parent: dict[str, Any], key: str, path: str) -> dict[str, Any]:
+    current = parent.get(key)
+    if current is None:
+        current = parent[key] = {}
+    if not isinstance(current, dict):
+        raise ValueError(f"{path} must be a mapping")
+    return current
+
+
+def _block(name: str, names: SchemaNames) -> dict[str, Any]:
+    return {
+        "name": name,
+        "guard": names.guard,
+        "groups": {"readers": names.readers, "writers": names.writers},
+        "version_tables": {"structure": names.version_table, "data": names.data_version_table},
+    }
+
+
+def _merge(target: dict[str, Any], values: dict[str, Any], path: str) -> list[str]:
+    conflicts: list[str] = []
+    for key, value in values.items():
+        current = target.get(key)
+        if current is None:
+            target[key] = value
+        elif isinstance(value, dict) and isinstance(current, dict):
+            conflicts += _merge(current, value, f"{path}.{key}")
+        elif current != value:
+            conflicts.append(f"{path}.{key}")
+    return conflicts

{loom_kernel-2.11.1 → loom_kernel-2.13.0}/src/loom/core/authz/__init__.py

@@ -16,15 +16,20 @@ from loom.core.authz._decide import (
 )
 from loom.core.authz._grants import Grant, GrantSource, InMemoryGrantSource
 from loom.core.authz._roles import (
+    CompositionRules,
     Permission,
     Role,
     RoleCatalog,
+    RoleSpec,
     UnknownPermission,
     UnknownRole,
 )
 from loom.core.authz._scope import Scope
+from loom.core.authz.product import AuthzProduct, load_authz_product
 
 __all__ = [
+    "AuthzProduct",
+    "CompositionRules",
     "Decision",
     "Grant",
     "GrantCheck",
@@ -33,11 +38,13 @@ __all__ = [
     "Permission",
     "Role",
     "RoleCatalog",
+    "RoleSpec",
     "Scope",
     "UnknownPermission",
     "UnknownRole",
     "can_grant",
     "can_revoke",
     "evaluate",
+    "load_authz_product",
     "scopes_with",
 ]