plain.postgres 0.118.0__tar.gz → 0.119.0__tar.gz

{plain_postgres-0.118.0 → plain_postgres-0.119.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: plain.postgres
-Version: 0.118.0
+Version: 0.119.0
 Summary: Model your data and store it in a database.
 Author-email: Dave Gaeddert <dave.gaeddert@dropseed.dev>
 License-Expression: BSD-3-Clause
@@ -20,6 +20,7 @@ Description-Content-Type: text/markdown
     - [Middleware](#middleware)
     - [Bypassing a connection pooler for management operations](#bypassing-a-connection-pooler-for-management-operations)
 - [Querying](#querying)
+- [Returning affected rows](#returning-affected-rows)
 - [Schema management](#schema-management)
     - [Syncing](#syncing)
     - [Structural migrations](#structural-migrations)
@@ -43,21 +44,41 @@ Description-Content-Type: text/markdown
 from datetime import datetime
 
 from plain import postgres
-from plain.postgres import types
+from plain.postgres import Field, types
 from plain.passwords.models import PasswordField
 
 
 @postgres.register_model
 class User(postgres.Model):
-    email: str = types.EmailField()
-    password = PasswordField()
-    is_admin: bool = types.BooleanField(default=False)
-    created_at: datetime = types.DateTimeField(create_now=True)
+    email: Field[str] = types.EmailField()
+    password: Field[str] = PasswordField()
+    is_admin: Field[bool] = types.BooleanField(default=False)
+    created_at: Field[datetime] = types.DateTimeField(create_now=True)
 
     def __str__(self) -> str:
         return self.email
 ```
 
+Annotate each field with `Field[T]` (the value type) — that's what gives the
+model a type-checked constructor: `User(email="a@b.com")` flags wrong value
+types, unknown field names, and missing required fields. A field is optional in
+that constructor only when its definition passes `default=` (this is general,
+not nullable-specific — a `required=False` field with no `default=` is still a
+required constructor arg), so nullable fields use `Field[T | None]` with
+`default=None`. DB-owned fields (`id`, `create_now`, generated values) are
+auto-excluded from the constructor.
+
+Field types declared outside `plain.postgres.types` — `PasswordField`, or one of
+your own — are the exception: their stub still types the value, but they are
+always _optional_ in the constructor, even when the column is `NOT NULL`. PEP
+681 recognizes a field declaration by matching the constructor against a fixed
+list, and that list is baked into `plain.postgres` where a third-party field
+type can't join it, so the checker reads the assignment as a plain default
+value. Wrong value types and unknown field names are still caught; only
+requiredness is lost, and omitting one surfaces as a `NOT NULL` error on insert
+instead. [Sharing fields across models](#sharing-fields-across-models) shows how
+a package that ships a field type can keep it required.
+
 Every model automatically includes an `id` field which serves as the primary
 key. The name `id` is reserved and can't be used for other fields.
 
@@ -191,13 +212,143 @@ first_10_users = User.query.all()[:10]
 
 For more advanced querying options, see the [`QuerySet`](./query.py#QuerySet) class.
 
+### Typed conditions with where()
+
+`where()` is a typed alternative to `filter()`. Instead of string keyword lookups, you build each condition from a field, so a type checker catches a misspelled field or a wrong value type at the call site:
+
+```python
+from plain import postgres
+from plain.postgres import Field, types
+
+
+@postgres.register_model
+class User(postgres.Model):
+    email: Field[str] = types.EmailField()
+    role: Field[str] = types.TextField(max_length=20)
+    age: Field[int | None] = types.IntegerField(allow_null=True, default=None)
+
+
+# Each argument is a condition; multiple arguments are ANDed together.
+admins = User.query.where(
+    User.role.equals("admin"),
+    User.age.gte(18),
+)
+```
+
+Every field exposes `equals`, `not_equal`, `gt`, `gte`, `lt`, `lte`, `is_null`, and `is_in`. Text fields add `contains`, `icontains`, `startswith`, and `endswith`. Each returns a `Q`, so you can combine them with `|` and `&` or negate with `~`:
+
+```python
+# Membership, negation, and OR
+User.query.where(User.role.is_in(["admin", "staff"]))
+User.query.where(~User.role.equals("guest"))
+User.query.where(User.email.endswith("@example.com") | User.role.equals("admin"))
+```
+
+Conditions traverse foreign keys — accessing a field through a relation builds the joined lookup:
+
+```python
+# Q(author__email="a@example.com")
+Post.query.where(Post.author.email.equals("a@example.com"))
+```
+
+A relation is a path to traverse, not a field, so it carries no conditions of its own. To match on the relation itself, traverse to the key it points at — that's the typed spelling of `filter(author=author)`, and it compiles to the same SQL:
+
+```python
+Post.query.where(Post.author.id.equals(author.id))
+Post.query.where(Post.author.id.is_in([a.id for a in authors]))
+Post.query.where(Post.author.id.is_null())  # nullable relation
+```
+
+`Post.author.equals(author)` raises `AttributeError` naming this spelling (an `AttributeError`, so `hasattr` and `getattr(..., default)` keep behaving). It isn't an oversight: to the type checker `Post.author` is `type[Author]`, which is what makes `Post.author.email.equals(...)` type-check, and a condition method there would be a runtime method the checker rejects.
+
+Traversal starts from a **forward foreign key**. Once inside one, every relation you pass through is another hop, many-to-many included — `WidgetTag.widget.tags.name.equals("metal")` builds `Q(widget__tags__name="metal")` — and the same rule applies to the relation itself: `WidgetTag.widget.tags.equals(tag)` points you at `WidgetTag.widget.tags.id.equals(tag.id)`.
+
+A class-level many-to-many (`Widget.tags`) is _not_ an entry point: it has no traversal wiring, and it is typed `ManyToManyManager[Tag]`, so it could not be typed as one either. Use the string path there — `Widget.query.filter(tags__name="metal")`. Reverse relations aren't traversable for the same reason (a reverse accessor is a `ClassVar`, so there is nothing for the related model to offer the checker), and the error says so.
+
+A traversed field _is_ the related field, carrying the relation path as its name — so it offers exactly the conditions that field offers, including an encrypted field's refusals.
+
+**A condition belongs to the model whose field built it.** `Order.query.where(User.email.equals("x"))` raises `TypeError` naming both models. A type checker can't catch this — `Field[str]` is `Field[str]` whichever model declared it — and without the check the lookup name `"email"` just resolves against `Order`, which is silently the wrong column when both models have one. A traversed condition belongs to the model the traversal _started_ from, so `Order.query.where(Order.user.email.equals("x"))` is `Order`'s, not `User`'s. A hand-written `Q(email="x")` names no model and isn't checked — it's `filter()`'s untyped spelling and behaves like it.
+
+[Encrypted fields](#encrypted-fields) reject value comparisons because their ciphertext is non-deterministic — only `is_null()` is available, and any other condition method (`equals`, `is_in`, …) raises `TypeError`.
+
+### Selecting columns with select()
+
+`select()` pulls back specific columns as typed rows instead of model instances. You pass typed field references, and a type checker knows the exact shape of each row:
+
+```python
+from plain.postgres import Field, types
+
+
+@postgres.register_model
+class User(postgres.Model):
+    email: Field[str] = types.EmailField()
+    age: Field[int | None] = types.IntegerField(allow_null=True, default=None)
+
+
+# list-like of tuple[str, int | None], precisely typed
+rows = User.query.where(User.age.gte(18)).select(User.email, User.age)
+for email, age in rows:
+    ...
+```
+
+There are three modes:
+
+- **Tuples** (default) — one tuple per row, typed per column: `select(User.email, User.age)` yields `tuple[str, int | None]`.
+- **Flat scalars** — a single column unwrapped, with `flat=True`: `select(User.email, flat=True)` yields `str`. `flat=True` accepts exactly one column.
+- **Dataclasses** — map each column onto a dataclass with `result_type=`: `select(User.email, User.age, result_type=UserStats)` yields `UserStats`. Columns map to dataclass fields **positionally**, so the selection order must match the dataclass field order, and each selected field's name must match the dataclass field at the same position.
+
+```python
+from dataclasses import dataclass
+
+
+@dataclass
+class UserStats:
+    email: str
+    age: int | None
+
+
+stats = User.query.select(User.email, User.age, result_type=UserStats)
+```
+
+The return value is a [`RowQuerySet`](./query.py#RowQuerySet) — that is the name to reach for when you need to annotate one:
+
+```python
+from plain.postgres import RowQuerySet
+
+
+def adults() -> RowQuerySet[tuple[str, int | None]]:
+    return User.query.where(User.age.gte(18)).select(User.email, User.age)
+```
+
+You can select expression columns too — `select(User.id, Sum("amount"))`, or an `F()` — but an expression column types as `Any` (its output type isn't tracked yet). The fields around it stay precise, so `select(User.id, Sum("amount"))` types as `tuple[int, Any]`.
+
+Per-column typing runs to **ten columns**. An eleventh is still selected and still returns rows, but the row type degrades to `tuple[Any, ...]` — reach for `result_type=` when a row is that wide.
+
+`select()` goes last in a chain: `annotate()` must come before it, because an annotation appends a column and would change the row shape out from under the type `select()` declared. `annotate()` after `select()` raises `TypeError` saying so. `prefetch_related()` is refused in both orders — a prefetch attaches related objects to a model instance's attributes, and a row has nowhere to put them; select the columns you need from the related model instead.
+
+Re-selecting replaces the **column list**, not the joins: an expression that reached through a relation (`select(Upper("tags__name"))`) leaves its join in place, so a later `select(Widget.name)` still returns one row per joined row — and `count()`/`exists()` count those. This is `annotate(...)` followed by `values_list(...)` behaving as it always has; trimming joins no queryset needs any more is out of scope here.
+
+`distinct()` with an `order_by()` on a column you didn't select returns duplicates: the ordering column has to go into the `SELECT` list for Postgres to sort by it, so `SELECT DISTINCT` deduplicates on that column too. Order by something you selected, or drop the ordering. This is `values_list()`'s behavior as well, not new to `select()`.
+
+Columns annotated `Field[Any]` are rejected by `select()`, because `Any` satisfies the model-valued `__get__` overload and class access resolves as `type[Any]` rather than a field. That's the same reason the field-annotation guidance says never to annotate a field `Field[Any]` — use the concrete type, or `Field[object]` when the column really does hold arbitrary JSON, which `select()` types as `object`.
+
+**`select()` returns rows, not partial model instances.** This is deliberate: a model instance with only some columns loaded is a type-level lie — the type checker thinks every field is present, so touching an unselected column looks fine but fails or fires a hidden query at runtime. Honest tuples/dataclasses keep the types truthful. As a result, iteration, `first()`, `get()`, `iterator()`, and slicing all return rows, and anything that would read or write model rows, or change the selected columns — `update()`, `delete()`, `get_or_create()`, `values()`, `values_list()`, `annotate()`, `prefetch_related()` — raises `TypeError`. `update()` and `delete()` refuse a queryset in row mode however it got there, `values()` and `values_list()` included.
+
+`select()` takes typed references only — a bare string like `select("email")` raises `TypeError` (use `User.email`).
+
+**A column belongs to the model whose field built it**, the same as [a condition does](#querying-with-typed-conditions): `Order.query.select(User.email)` raises `TypeError` naming both models. A type checker can't catch it — `Field[str]` is `Field[str]` whichever model declared it — and without the check the name `"email"` just resolves against `Order`, silently the wrong column when both models have one. Expressions are unaffected: `F("email")` and `Upper("email")` take a string resolved against whatever query they land in, like `filter()`'s kwargs.
+
+**Relations are not selectable yet.** `select(Post.author)` (the relation) and `select(Post.author.city)` (a column through it) both raise `TypeError`, and so does `select(Post.author.id)` — the foreign key column itself. The reason is nullability: a column reached through a relation arrives over a join, so a nullable relation yields `None` where the traversed field's type says it can't. Until `select()` can express that, `values_list("author__id", flat=True)` is the spelling, and the error message names it.
+
+**`select()` hands back a plain `RowQuerySet`, not your custom QuerySet subclass.** Chain your own methods before `select()`, not after — `User.query.active().select(...)` works, `User.query.select(...).active()` raises `AttributeError`.
+
 ### Custom QuerySets
 
-You can customize [`QuerySet`](./query.py#QuerySet) classes to provide specialized query methods. Define a custom QuerySet and assign it to your model's `query` attribute:
+You can customize [`QuerySet`](./query.py#QuerySet) classes to provide specialized query methods. Define a custom QuerySet and assign it to your model's `query` attribute as a `ClassVar` (so it isn't treated as a constructor field):
 
 ```python
-from typing import Self
-from plain.postgres import types
+from typing import ClassVar, Self
+from plain.postgres import Field, types
 
 
 class PublishedQuerySet(postgres.QuerySet["Article"]):
@@ -210,10 +361,10 @@ class PublishedQuerySet(postgres.QuerySet["Article"]):
 
 @postgres.register_model
 class Article(postgres.Model):
-    title: str = types.TextField(max_length=200)
-    status: str = types.TextField(max_length=20)
+    title: Field[str] = types.TextField(max_length=200)
+    status: Field[str] = types.TextField(max_length=20)
 
-    query = PublishedQuerySet()
+    query: ClassVar[PublishedQuerySet] = PublishedQuerySet()
 
 
 # Usage - all methods available on Article.query
@@ -233,24 +384,11 @@ special_qs = SpecialQuerySet.from_model(Article)
 
 ### Typing QuerySets
 
-For better type checking of query results, you can explicitly type the `query` attribute:
-
-```python
-from __future__ import annotations
-
-from plain import postgres
-from plain.postgres import types
-
-
-@postgres.register_model
-class User(postgres.Model):
-    email: str = types.EmailField()
-    is_admin: bool = types.BooleanField(default=False)
-
-    query: postgres.QuerySet[User] = postgres.QuerySet()
-```
-
-With this annotation, type checkers will know that `User.query.get()` returns a `User` instance and `User.query.filter()` returns `QuerySet[User]`. This is optional but improves IDE autocomplete and type checking.
+`Model.query` is typed automatically — `User.query.get()` returns a `User` and
+`User.query.filter()` returns `QuerySet[User]` with no extra annotation. Don't
+redeclare `query` just to type it; the base provides `QuerySet[Self]`. Declare
+`query` only when attaching a **custom** QuerySet, and then as a `ClassVar`
+(see [Custom QuerySets](#custom-querysets) above).
 
 ### Raw SQL
 
@@ -410,6 +548,176 @@ for name in names:
 Tag.query.bulk_create([Tag(name=name) for name in names])
 ```
 
+`bulk_create` is insert-only. To insert new rows and update the ones that
+already exist in a single statement, use `bulk_upsert` (below).
+
+#### Use `bulk_upsert` to insert-or-update in one statement
+
+`bulk_upsert(objs, *, update_fields, unique_fields, batch_size=None)` issues one
+`INSERT ... ON CONFLICT (unique_fields) DO UPDATE SET ... RETURNING` per batch.
+Rows that don't exist yet are inserted; rows that collide on `unique_fields` have
+their `update_fields` overwritten. You get back the objects you passed in, in the
+order you passed them (a new list — `objs` itself is never reordered), each with
+its DB-generated fields (primary key, DB defaults) populated.
+
+```python
+# Insert new items, refresh `value`/`expires_at` on any existing key.
+CachedItem.query.bulk_upsert(
+    [CachedItem(key=k, value=v, expires_at=exp) for k, v in items],
+    update_fields=[CachedItem.value, CachedItem.expires_at],
+    unique_fields=[CachedItem.key],
+)
+```
+
+- `update_fields` and `unique_fields` take field references (`Model.field`), not
+  strings. A foreign key is named by the relation itself — `Model.tenant`, which
+  resolves to the `tenant_id` column. (This is the one write API that takes
+  `Model.fk`. `returning()` refuses it, because there it would be ambiguous with
+  asking for the whole related object; here a column list can only mean the
+  column.)
+- `unique_fields` must name the **primary key** or a `UniqueConstraint` declared
+  on the model (no condition, no expressions) — this is the conflict target. A
+  unique `Index` is not enough; declare a `UniqueConstraint`.
+- `update_fields` must be concrete, non-primary-key, must not name the same
+  column twice (Postgres assigns each column once per statement), and must not
+  overlap `unique_fields`. A column the database fills in (`create_now`,
+  `generate=True`, `RandomStringField`) can't be named either — the update would
+  overwrite the stored value with a freshly evaluated default.
+- Every object must have a non-null value for every unique field. `NULL` never
+  conflicts in Postgres, so it can't be upserted. A database-generated column
+  (`create_now`, `generate=True`, `RandomStringField`) can't be a unique field
+  either — your objects never hold its value, so it could never conflict. Nor
+  can an `update_now=True` column, which is stamped again on every write.
+- **Two objects with the same unique key in one batch raise `ValueError`.**
+  Postgres won't touch a row twice in one statement. Split across batches it's
+  allowed — the first inserts, the second updates, and the later write wins.
+- **`update_now=True` columns are refreshed on a conflict automatically.** You
+  don't name them in `update_fields`; a row that gets updated gets a fresh
+  stamp, and the object handed back carries the same one.
+- **An `id` you set is kept on the insert path; on a conflict the stored row
+  wins.** A new row is written with the `id` you gave it. A conflicting one
+  already has an `id`, and that is the one hydrated back onto your object — the
+  row in the table is the truth. An `id` that collides with a _different_ row
+  raises `psycopg.errors.UniqueViolation`, like any set-based write.
+- Every object is sorted by its conflict key before anything is sent, so
+  concurrent `bulk_upsert` calls over overlapping keys lock rows in the same
+  order and can't deadlock each other. Returned rows are mapped onto the objects
+  by position, exactly as `bulk_create` does.
+- Like `bulk_create`, the write is against the table: a filter on the queryset
+  you call it from doesn't narrow or exclude anything.
+
+For a single row, reach for `upsert` (below) instead — it returns the object and
+a `created` flag rather than a list.
+
+#### Use `upsert` for a single insert-or-update
+
+`upsert(*, unique_fields, defaults=None, create_defaults=None, conflict_defaults=None, **kwargs)`
+is the single-row counterpart to `bulk_upsert`. It runs one
+`INSERT ... ON CONFLICT (unique_fields) DO UPDATE SET ... RETURNING` statement and
+returns `(obj, created)` — `created` is `True` when a new row was inserted,
+`False` when the conflicting row was updated. The object is hydrated from the
+post-write row, so there's no second query.
+
+```python
+# Insert the flag, or refresh used_at on the existing one.
+flag, created = Flag.query.upsert(
+    name="beta-dashboard",
+    defaults={"used_at": timezone.now()},
+    unique_fields=[Flag.name],
+)
+```
+
+`unique_fields` takes field references (`Model.field`), like `bulk_upsert`. The
+value sources below stay string-keyed — they follow the `kwargs` idiom.
+
+Value sources, lowest precedence first — where the same key appears in two of
+them, `create_defaults` loses to `defaults`, which loses to `**kwargs`:
+
+- `create_defaults` is applied on **insert only** — extras that must not change
+  when the row already exists.
+- `defaults` and `**kwargs` are applied on **both** insert and conflict-update.
+  `kwargs` carries the identifying values (including the unique fields).
+- `conflict_defaults` applies to the `DO UPDATE SET` **only** — it never changes
+  the inserted row. A value can be a plain value or an expression, so
+  `{"count": F("count") + 1}` is an atomic counter that reads the existing row:
+
+```python
+view, created = PageView.query.upsert(
+    path="/home",
+    conflict_defaults={"count": F("count") + 1},
+    unique_fields=[PageView.path],
+)
+```
+
+Two expressions reach two different rows inside a conflict update. `F("count")`
+reads the row already **stored**; `Excluded("count")` reads the row the INSERT
+**proposed**, compiling to `EXCLUDED."count"`. Combine them to accumulate the
+incoming value instead of overwriting it — the whole statement is one atomic
+`UPDATE`, so concurrent callers each add their own delta:
+
+```python
+from plain.postgres import Excluded, F
+
+# Add this batch's 7 views to whatever is already stored.
+view, created = PageView.query.upsert(
+    path="/home",
+    count=7,
+    conflict_defaults={"count": F("count") + Excluded("count")},
+    unique_fields=[PageView.path],
+)
+```
+
+`Excluded()` is only meaningful while the conflict update's assignments are
+being built. Anywhere else raises a `FieldError` — including as an inserted
+value in `kwargs`/`defaults`/`create_defaults` of the very same call, where it
+would be naming the row being written, and in `filter()`, `update()` or
+`annotate()`.
+
+On conflict the `SET` clause covers:
+
+- every non-unique, non-PK column from `kwargs`/`defaults`, each taking the value
+  the INSERT proposed;
+- every `DateTimeField(update_now=True)` column, whose fresh `pre_save()`
+  timestamp rides along in the INSERT and would otherwise go stale (a
+  `create_now`-only column is _not_ in the `SET`, so it keeps its original
+  value);
+- every column `conflict_defaults` names — replacing the proposed value when the
+  column is already in the `SET`, adding it when it isn't.
+
+Columns nobody wrote are left alone, and `create_defaults` never take part in the
+update. Naming a **database-owned** column (`create_now`, `generate=True`,
+`RandomStringField`) in `kwargs`/`defaults` is an error rather than a silent
+reset: `EXCLUDED` carries a freshly evaluated default, so the conflict update
+would overwrite the stored creation timestamp every time. A column that's also
+`update_now` is exempt — refreshing it is the point. Unlike `bulk_upsert`, `upsert` derives its `SET` columns rather than
+taking them, so a `conflict_defaults` key may not name a unique field — that's
+the conflict target.
+
+Every value source resolves callables, and every key must name a **column** — a
+settable property is refused, since the `SET` clause is derived from columns and
+a property could only ever be written on the insert half.
+
+`unique_fields` must name a `UniqueConstraint` declared on the model (no
+condition, no expressions) and every unique field must be non-null. It can't be
+the primary key: Postgres generates the identity value, so a caller has nothing
+to conflict on.
+
+Three things to keep in mind:
+
+- **`kwargs` are values, not filters.** A keyword that isn't part of the conflict
+  key doesn't narrow which row is matched — `unique_fields` alone decides that —
+  it's just another column written to whichever row conflicts.
+- **The queryset's filters don't scope it either.** `qs.filter(...).upsert(...)`
+  writes the conflicting row whether or not it matches the filter — the conflict
+  constraint decides which row is touched. It isn't refused because the
+  related-manager wrappers call through a filtered queryset. To scope an upsert,
+  fold the scoping column into `unique_fields` (and into the constraint), or do a
+  locked read and write instead.
+- **The merged row isn't validated**, and a constraint violation surfaces as a raw
+  `psycopg.IntegrityError`, not a `ValidationError`. `upsert` looks single-row
+  like `create()`, but it's a set-based write like `bulk_upsert` (see
+  [Validation](#validation)).
+
 #### Use queryset `.update()` / `.delete()` for mass operations
 
 ```python
@@ -450,6 +758,44 @@ for row in HugeTable.query.iterator(chunk_size=2000):
     process(row)
 ```
 
+## Returning affected rows
+
+`QuerySet.update()` and `QuerySet.delete()` return an `int` rowcount. Chain `returning()` before the write to get the affected rows back instead — Postgres' `RETURNING` clause fetches them in the same statement, so there's no second query.
+
+```python
+# No arguments: rows come back as model instances.
+running = Job.query.filter(status="pending").returning().update(status="running")
+for job in running:
+    print(job.id, job.status)  # reflects the post-update values
+
+# Field references: rows come back as dicts of just those columns.
+deleted = (
+    Event.query.filter(created_at__lt=cutoff)
+    .returning(Event.id, Event.payload)
+    .delete()
+)
+for row in deleted:
+    print(row["id"], row["payload"])  # the rows as they were deleted
+```
+
+- **`returning()`** returns full model instances. For `update()` they hold the new values and stay live. For `delete()` they are **read-only snapshots**: every value is there to read, the id included, but the row is gone, so `create()`, `update()` and `delete()` on them raise.
+    - That is the opposite of `Model.delete()`, which clears the instance's id and leaves it re-creatable — that instance is a row you still hold and may want to put back, while a snapshot is a record of one that was removed, and its id is the point.
+- **`returning(Model.field, ...)`** returns a list of dicts with only those columns. Pass field references (`Model.field`), not strings; a many-to-many field or one from another model raises an error at the `returning()` call.
+- **A foreign key can't be named here.** At class level `Model.fk` is the relation — that is what lets `where()` traverse it, as in `Child.parent.name.equals(...)` — not its column, so `returning(Child.parent)` raises `FieldError`. Foreign key columns come back through no-argument `returning()`, which hands you whole instances.
+- Without `returning()`, `update()`/`delete()` return an `int` as before.
+- `returning()` only applies to `update()` and `delete()`. Any other write on the same queryset — `create()`, `bulk_create()`, `bulk_upsert()`, `bulk_update()`, `get_or_create()`, `upsert()` — raises `TypeError` rather than quietly dropping it.
+- `returning()` keeps the queryset's own class, so a custom `QuerySet` and its methods survive it. Chain your own methods before `returning()` — a type checker sees the returning shape after it, not your subclass.
+
+A row lock belongs on the read side of the write, and it composes in either order. The write then needs an open `transaction.atomic()`, and is emitted as a locking sub-select so the lock has somewhere to live — see [Locking a set-based write](#locking-a-set-based-write).
+
+`returning()` is inert for reads. It describes what the next `update()` or `delete()` hands back, so iterating, `count()`, `first()` and `values()` on the same queryset behave exactly as they would without it — which is what lets you inspect a chain before writing it.
+
+The values you get back are whatever the statement wrote, exactly as Postgres holds them. A set-based `update()` doesn't run Python-side field hooks, so an `update_now=True` timestamp comes back unchanged unless the `update()` set it.
+
+`RETURNING` only reports rows of the statement's own target table. Rows removed by a cascading `ON DELETE` are never included — a `delete()` with `returning()` gives you the parent rows you deleted, not the children Postgres cascaded.
+
+Every affected row is fetched and built into memory at once, so `returning()` belongs on writes you've already bounded by a filter. For a write that spans a whole table, take the rowcount and page through the rows separately.
+
 ## Transactions
 
 By default, each query runs in its own implicit transaction and is committed immediately (autocommit mode). When you need multiple queries to succeed or fail together — like creating a user and their profile — wrap them in an explicit transaction.
@@ -509,6 +855,81 @@ with read_only():
     User.query.count()  # still works — outer txn is healthy
 ```
 
+### Row-level locking
+
+Lock the rows a query selects so concurrent transactions can't change them until yours commits. Postgres offers four lock strengths, from strongest to weakest, each with its own QuerySet method:
+
+```python
+with transaction.atomic():
+    account = Account.query.for_update().get(id=1)  # FOR UPDATE
+    account.balance -= 100
+    account.update(fields=["balance"])
+```
+
+| Method                | SQL clause          | Use it when                                                                |
+| --------------------- | ------------------- | -------------------------------------------------------------------------- |
+| `for_update()`        | `FOR UPDATE`        | You intend to update or delete the row.                                    |
+| `for_no_key_update()` | `FOR NO KEY UPDATE` | Same, but you won't touch the primary key — lets key-share locks proceed.  |
+| `for_share()`         | `FOR SHARE`         | You need the row to stay put while you read it, but others may also share. |
+| `for_key_share()`     | `FOR KEY SHARE`     | Weakest — only blocks changes to the row's key.                            |
+
+Locking requires an open transaction. The method itself just builds the queryset — evaluating a locked queryset outside `transaction.atomic()` is what raises `TransactionManagementError`.
+
+All four accept the same options:
+
+- `nowait=True` — raise instead of waiting if a row is already locked.
+- `skip_locked=True` — skip already-locked rows instead of waiting (can't be combined with `nowait`).
+- `of=("self", "related")` — lock only the named tables in a join rather than every selected row.
+
+```python
+# Claim the next available job without blocking on rows another worker holds
+job = Job.query.for_update(skip_locked=True).filter(status="pending").first()
+```
+
+Chaining more than one lock method keeps only the last one, options included.
+
+Postgres can only lock rows that map one-to-one onto table rows, so a lock can't be combined with `distinct()`, an aggregate annotation, or a window annotation. Either order raises `psycopg.NotSupportedError` when the queryset is built, naming the lock method:
+
+```python
+Widget.query.distinct().for_update()  # NotSupportedError
+Widget.query.for_update().distinct()  # same error, either way round
+```
+
+`count()` and `aggregate()` are the exception — they compile to an aggregate query of their own, so they drop the lock rather than reject it.
+
+#### Locking a set-based write
+
+A lock also applies to `update()` and `delete()` on the same queryset. Neither statement takes a locking clause of its own, so the write is emitted as a locking sub-select:
+
+```python
+# Claim *all* pending rows matching the filter, in one statement
+with transaction.atomic():
+    claimed = (
+        Job.query.filter(status="pending")
+        .for_update(skip_locked=True)
+        .returning()
+        .update(status="running")
+    )
+```
+
+```sql
+UPDATE "jobs" SET "status" = 'running'
+WHERE "id" IN (
+    SELECT U0."id" FROM "jobs" U0 WHERE U0."status" = 'pending' FOR UPDATE OF U0 SKIP LOCKED
+)
+RETURNING ...
+```
+
+It is still one statement. A second worker running the same write skips the rows the first one holds instead of blocking on them, so each row is claimed once.
+
+Note what this is and isn't: it takes **every** row the filter matches, so it suits draining a batch, not handing one unit of work to one worker. A bounded claim would need a sliced write (`[:1]`), and `update()`/`delete()` reject a sliced queryset — so for a per-worker claim, take one row with the locked read above (`for_update(skip_locked=True)` + `first()`) and write it separately.
+
+The `transaction.atomic()` is required, same as for a locked read: a locked write outside a transaction raises `TransactionManagementError`. Nothing else honors the lock — without a transaction there is nothing for it to be held until.
+
+**A locked write locks only the target table's rows.** When the filter spans a relation, the sub-select joins the other tables to look values up, and a bare `FOR UPDATE` would lock a row in each of them — so a write whose filter reads a parent would wait on (or, with `skip_locked=True`, silently skip) rows it never touches. The clause is emitted as `FOR UPDATE OF <target>` so that can't happen. To lock related rows as well, take them with a separate locked read.
+
+That is also why `of=` can only name `"self"` on a write: the sub-select reads one column — this table's id — so a related name has nothing to point at, and `update()`/`delete()` raise `TypeError` rather than let it fail deeper down. Dropping `of=` is the same thing; the write locks its own rows either way.
+
 ## Schema management
 
 Schema changes fall into three categories, each with a different author and apply model:
@@ -743,9 +1164,9 @@ Convergence compares the indexes, constraints, foreign keys, nullability, and [s
 ```python
 @postgres.register_model
 class User(postgres.Model):
-    email: str = types.EmailField()
-    username: str = types.TextField(max_length=150)
-    age: int = types.IntegerField()
+    email: Field[str] = types.EmailField()
+    username: Field[str] = types.TextField(max_length=150)
+    age: Field[int] = types.IntegerField()
 
     model_options = postgres.Options(
         indexes=[
@@ -854,24 +1275,24 @@ from decimal import Decimal
 from datetime import datetime
 
 from plain import postgres
-from plain.postgres import types
+from plain.postgres import Field, types
 
 
 class Product(postgres.Model):
     # Text fields
-    name: str = types.TextField(max_length=200)
-    description: str = types.TextField()
+    name: Field[str] = types.TextField(max_length=200)
+    description: Field[str] = types.TextField()
 
     # Numeric fields
-    price: Decimal = types.DecimalField(max_digits=10, decimal_places=2)
-    quantity: int = types.IntegerField(default=0)
+    price: Field[Decimal] = types.DecimalField(max_digits=10, decimal_places=2)
+    quantity: Field[int] = types.IntegerField(default=0)
 
     # Boolean fields
-    is_active: bool = types.BooleanField(default=True)
+    is_active: Field[bool] = types.BooleanField(default=True)
 
     # Date and time fields
-    created_at: datetime = types.DateTimeField(create_now=True)
-    updated_at: datetime = types.DateTimeField(update_now=True)
+    created_at: Field[datetime] = types.DateTimeField(create_now=True)
+    updated_at: Field[datetime] = types.DateTimeField(update_now=True)
 ```
 
 **Text fields:**
@@ -914,43 +1335,81 @@ See [Encrypted fields](#encrypted-fields) for details.
 
 For relationship fields, see [Relationships](#relationships).
 
-For nullable fields, use `| None` in the annotation:
+For nullable fields, use `| None` in the annotation and `default=None` to make
+the field optional in the constructor:
 
 ```python
-published_at: datetime | None = types.DateTimeField(allow_null=True, required=False)
+published_at: Field[datetime | None] = types.DateTimeField(
+    allow_null=True, required=False, default=None
+)
 ```
 
 ### Sharing fields across models
 
-To share common fields across multiple models, use Python classes as mixins. The final, registered model must inherit directly from `postgres.Model` and the mixins should not.
+To share common fields across multiple models, use Python classes as mixins. A mixin that declares fields must inherit `postgres.ModelMixin`, and the final, registered model must inherit directly from `postgres.Model` (the mixins must not).
 
 ```python
 from datetime import datetime
 
 from plain import postgres
-from plain.postgres import types
+from plain.postgres import Field, ModelMixin, types
 
 
 # Regular Python class for shared fields
-class TimestampedMixin:
-    created_at: datetime = types.DateTimeField(create_now=True)
-    updated_at: datetime = types.DateTimeField(update_now=True)
+class TimestampedMixin(ModelMixin):
+    created_at: Field[datetime] = types.DateTimeField(create_now=True)
+    updated_at: Field[datetime] = types.DateTimeField(update_now=True)
+    source: Field[str] = types.TextField(max_length=50, required=False, default="")
 
 
 # Models inherit from the mixin AND postgres.Model
 @postgres.register_model
 class User(TimestampedMixin, postgres.Model):
-    email: str = types.EmailField()
-    password = PasswordField()
-    is_admin: bool = types.BooleanField(default=False)
+    email: Field[str] = types.EmailField()
+    password: Field[str] = PasswordField()
+    is_admin: Field[bool] = types.BooleanField(default=False)
 
 
 @postgres.register_model
 class Note(TimestampedMixin, postgres.Model):
-    content: str = types.TextField(max_length=1024)
-    liked: bool = types.BooleanField(default=False)
+    content: Field[str] = types.TextField(max_length=1024)
+    liked: Field[bool] = types.BooleanField(default=False)
+```
+
+`ModelMixin` is what puts the mixin's fields into each model's typed
+constructor. The runtime collects fields off the whole MRO either way, but a
+mixin inheriting nothing isn't visible to PEP 681, so the checker would reject
+`Note(source="import")` on code that runs fine. `ModelMixin` carries no runtime
+behavior — it's the same transform models get, and mixins are still declarations
+you never instantiate directly.
+
+That is also the one way to keep a **custom field type** required in the
+constructor. A package that ships its own field type can declare the field on a
+mixin carrying a transform that lists its constructor, and models mixing it in
+get the field with its requiredness intact:
+
+```python
+from typing import dataclass_transform
+
+from plain.postgres import Field
+from plain.passwords.types import PasswordField
+
+
+@dataclass_transform(kw_only_default=True, field_specifiers=(PasswordField,))
+class _PasswordFieldSpec: ...
+
+
+class PasswordMixin(_PasswordFieldSpec):
+    password: Field[str] = PasswordField()
 ```
 
+A model mixing in `PasswordMixin` now gets `password` as a required constructor
+argument, so `User(email="a@b.com")` is a type error again. The specifier list
+has to name every constructor the mixin's own body uses, core ones included.
+Declared in the model's own body instead, `password` stays optional to the
+checker — a model body is governed by the specifier list `plain.postgres`
+declares, which a package outside it can't extend.
+
 ### Encrypted fields
 
 Encrypted fields transparently encrypt values before writing to the database and decrypt on read. Use them for third-party credentials, API keys, OAuth tokens, and other secrets your application needs back in plaintext.
@@ -959,28 +1418,46 @@ This is **not** for passwords or tokens you issue — those should be hashed (on
 
 ```python
 from plain import postgres
-from plain.postgres import types
+from plain.postgres import EncryptedField, Field, types
 
 
 @postgres.register_model
 class Integration(postgres.Model):
-    name: str = types.TextField(max_length=100)
-    api_key: str = types.EncryptedTextField(max_length=200)
-    credentials: dict = types.EncryptedJSONField(required=False, allow_null=True)
+    name: Field[str] = types.TextField(max_length=100)
+    api_key: EncryptedField[str] = types.EncryptedTextField(max_length=200)
+    credentials: EncryptedField[dict | None] = types.EncryptedJSONField(
+        required=False, allow_null=True, default=None
+    )
 ```
 
+Annotate encrypted fields `EncryptedField[T]`, not `Field[T]`. The annotation is
+what the type checker reads, and `EncryptedField[T]` is the `Field[T]` subclass
+that declares the blocked conditions — with a plain `Field[T]`,
+`Integration.api_key.equals("x")` type-checks its way to a runtime `TypeError`
+instead of being rejected at the call site. It types the constructor exactly as
+`Field[T]` does.
+
 Values are encrypted using Fernet (AES-128-CBC + HMAC-SHA256) with a key derived from `SECRET_KEY`. The `cryptography` package is required — install it with `pip install cryptography`.
 
 **Available fields:**
 
+- `EncryptedField[T]` — the annotation type; also the shared base the two fields below derive from.
 - `EncryptedTextField` — encrypts text, stored as `text` in the database regardless of `max_length` (ciphertext is longer than plaintext). `max_length` is enforced on the plaintext value during validation.
 - `EncryptedJSONField` — serializes to JSON, encrypts, and stores as `text`. Supports custom `encoder` and `decoder` parameters (same as `JSONField`).
 
 **Limitations:**
 
-- **No lookups** — encrypted values are non-deterministic (same plaintext produces different ciphertext each time), so filtering on encrypted fields doesn't work. Only `isnull` lookups are supported.
+- **No lookups** — encrypted values are non-deterministic (same plaintext produces different ciphertext each time), so filtering on encrypted fields doesn't work. Only `isnull` lookups are supported. Comparing against a value raises `TypeError` rather than silently matching nothing — both `filter(api_key="x")` and the typed [condition methods](#typed-conditions-with-where) (`equals`, `contains`, …), which are also rejected at the call site when the field is annotated `EncryptedField[T]`. `filter(api_key=None)` still rewrites to `IS NULL`.
+- **`get_or_create()` must not look up an encrypted field.** `get_or_create(api_key="k")` raises, and the error says to move the value into `defaults=`. This is a deliberate break: it previously "worked" by creating a new row on every call, because the lookup could never match existing ciphertext. An encrypted value can be written, just not looked up:
+
+    ```python
+    Integration.query.get_or_create(name="acme", defaults={"api_key": "k"})
+    ```
+
+    The same applies to `upsert()`, and to an expression right-hand side like `filter(api_key=F("name"))` — the column is still ciphertext.
+
 - **No indexes or constraints** — encrypted fields cannot be used in indexes or unique constraints. Preflight checks will catch this.
-- **Only `default=""`** — on `EncryptedTextField` (paired with `required=False`), the empty string is stored as plaintext `''`, so it's the one value expressible as a column `DEFAULT` (declare it to add the field to a populated table). Any other default would need ciphertext, which is non-deterministic. `EncryptedJSONField` accepts no default at all — even `{}` serializes to text that would need ciphertext; use `allow_null=True`.
+- **Only `default=""`** — on `EncryptedTextField` (paired with `required=False`), the empty string is stored as plaintext `''`, so it's the one value expressible as a column `DEFAULT` (declare it to add the field to a populated table). Any other default would need ciphertext, which is non-deterministic. `EncryptedJSONField` has no persistent default at all — even `{}` serializes to text that would need ciphertext — so pair `allow_null=True` with `default=None`, which stores nothing and just marks the field optional in the constructor.
 
 **Key rotation:**
 
@@ -996,12 +1473,12 @@ Use [`ForeignKeyField`](./fields/related.py#ForeignKeyField) for many-to-one and
 
 ```python
 from plain import postgres
-from plain.postgres import types
+from plain.postgres import Field, types
 
 
 @postgres.register_model
 class Book(postgres.Model):
-    title: str = types.TextField(max_length=200)
+    title: Field[str] = types.TextField(max_length=200)
     author: Author = types.ForeignKeyField("Author", on_delete=postgres.CASCADE)
     tags = types.ManyToManyField("Tag")
 ```
@@ -1019,6 +1496,8 @@ book.author.name  # one query — loads the rest of the row
 
 The first access to any non-key field loads the whole row in a single query. There is no separate `author_id` attribute — `book.author.id` is the foreign key value, and it is type-checked because `book.author` is an `Author`. In loops, use `select_related()` to load related rows up front and avoid a query per row.
 
+A foreign key with no value raises `RelatedObjectDoesNotExist` on access. That attribute still lives on the descriptor at runtime, but class-level access is now typed as the related model (that is what makes `Book.author.name.equals(...)` work), so `Book.author.RelatedObjectDoesNotExist` is a type error. Catch it as `Author.DoesNotExist` — the exception subclasses both that and `AttributeError` — or as `AttributeError`.
+
 The partial-instance shortcut is safe because Plain always creates a database foreign-key constraint, so the referenced row is guaranteed to exist.
 
 ### Constraints are checked immediately
@@ -1034,21 +1513,26 @@ A migration can add a column, backfill it in `RunPython`, and drop or alter colu
 When you define a `ForeignKey` or `ManyToManyField`, Plain automatically creates a reverse accessor on the related model (like `author.book_set`). You can explicitly declare these reverse relationships using [`ReverseForeignKey`](./fields/reverse_descriptors.py#ReverseForeignKey) and [`ReverseManyToMany`](./fields/reverse_descriptors.py#ReverseManyToMany):
 
 ```python
+from typing import ClassVar
+
 from plain import postgres
-from plain.postgres import types
+from plain.postgres import Field, types
 
 
 @postgres.register_model
 class Author(postgres.Model):
-    name: str = types.TextField(max_length=200)
-    # Explicit reverse accessor for all books by this author
-    books = types.ReverseForeignKey(to="Book", field="author")
+    name: Field[str] = types.TextField(max_length=200)
+    # Explicit reverse accessor for all books by this author.
+    # ClassVar keeps it out of the typed constructor (it's an accessor, not a field).
+    books: ClassVar[types.ReverseForeignKey[Book]] = types.ReverseForeignKey(
+        to="Book", field="author"
+    )
 
 
 @postgres.register_model
 class Book(postgres.Model):
-    title: str = types.TextField(max_length=200)
-    author: Author = types.ForeignKeyField(Author, on_delete=postgres.CASCADE)
+    title: Field[str] = types.TextField(max_length=200)
+    author: Field[Author] = types.ForeignKeyField(Author, on_delete=postgres.CASCADE)
 
 
 # Usage
@@ -1065,14 +1549,16 @@ For many-to-many relationships:
 ```python
 @postgres.register_model
 class Feature(postgres.Model):
-    name: str = types.TextField(max_length=100)
+    name: Field[str] = types.TextField(max_length=100)
     # Explicit reverse accessor for all cars with this feature
-    cars = types.ReverseManyToMany(to="Car", field="features")
+    cars: ClassVar[types.ReverseManyToMany[Car]] = types.ReverseManyToMany(
+        to="Car", field="features"
+    )
 
 
 @postgres.register_model
 class Car(postgres.Model):
-    model: str = types.TextField(max_length=100)
+    model: Field[str] = types.TextField(max_length=100)
     features = types.ManyToManyField(Feature)
 
 
@@ -1091,16 +1577,16 @@ for car in feature.cars.all():
 
 Reverse relations are optional — if you don't declare them, the automatic `{model}_set` accessor still works.
 
-To get type checking for custom QuerySet methods on reverse relations, specify the QuerySet type as a second parameter:
+Annotate reverse relations with `ClassVar` — they're class-level accessors, not constructor fields, so `ClassVar` keeps them out of the typed `Model(...)` constructor (same as `query`). To get type checking for custom QuerySet methods, specify the QuerySet type as a second parameter:
 
 ```python
 # Basic usage
-books: types.ReverseForeignKey[Book] = types.ReverseForeignKey(
+books: ClassVar[types.ReverseForeignKey[Book]] = types.ReverseForeignKey(
     to="Book", field="author"
 )
 
 # With custom QuerySet for proper method recognition
-books: types.ReverseForeignKey[Book, BookQuerySet] = types.ReverseForeignKey(
+books: ClassVar[types.ReverseForeignKey[Book, BookQuerySet]] = types.ReverseForeignKey(
     to="Book", field="author"
 )
 
@@ -1117,8 +1603,8 @@ author.books.query.published()
 ```python
 @postgres.register_model
 class User(postgres.Model):
-    email: str = types.EmailField()
-    age: int = types.IntegerField()
+    email: Field[str] = types.EmailField()
+    age: Field[int] = types.IntegerField()
 
     model_options = postgres.Options(
         constraints=[
@@ -1156,7 +1642,7 @@ except (psycopg.IntegrityError, ValidationError):
     ...  # lost a race — reload and retry, or report it
 ```
 
-For a plain insert-or-update with no per-row logic, `bulk_create(..., update_conflicts=True, unique_fields=[...])` is an atomic upsert with no race to catch.
+For a plain insert-or-update with no per-row logic there's no race to catch in the first place: `upsert(**values, unique_fields=[...])` for one row and `bulk_upsert(objs, update_fields=[...], unique_fields=[...])` for many are each a single atomic statement.
 
 ### Indexes and constraints
 
@@ -1164,9 +1650,9 @@ You can optimize queries and ensure data integrity with indexes and constraints.
 
 ```python
 class User(postgres.Model):
-    email: str = types.EmailField()
-    username: str = types.TextField(max_length=150)
-    age: int = types.IntegerField()
+    email: Field[str] = types.EmailField()
+    username: Field[str] = types.TextField(max_length=150)
+    age: Field[int] = types.IntegerField()
 
     model_options = postgres.Options(
         indexes=[
@@ -1241,14 +1727,14 @@ Add indexes for columns that appear in `.filter()`, `.order_by()`, or `.exclude(
 ```python
 # Bad — full table scan on every filtered query
 class Order(postgres.Model):
-    status: str = types.TextField(max_length=20)
-    created_at: datetime = types.DateTimeField()
+    status: Field[str] = types.TextField(max_length=20)
+    created_at: Field[datetime] = types.DateTimeField()
 
 
 # Good — indexed for common queries
 class Order(postgres.Model):
-    status: str = types.TextField(max_length=20)
-    created_at: datetime = types.DateTimeField()
+    status: Field[str] = types.TextField(max_length=20)
+    created_at: Field[datetime] = types.DateTimeField()
 
     model_options = postgres.Options(
         indexes=[postgres.Index(fields=["status", "-created_at"])],
@@ -1293,10 +1779,10 @@ Use `default=""` instead of `allow_null=True` to avoid two representations of "e
 
 ```python
 # Bad — NULL and "" both mean "empty"
-nickname: str = types.TextField(max_length=50, allow_null=True)
+nickname: Field[str] = types.TextField(max_length=50, allow_null=True)
 
 # Good — single empty representation
-nickname: str = types.TextField(max_length=50, default="")
+nickname: Field[str] = types.TextField(max_length=50, default="")
 ```
 
 ## Forms