TypeDAL 4.0.2__tar.gz → 4.1.0__tar.gz

{typedal-4.0.2 → typedal-4.1.0}/CHANGELOG.md

@@ -2,6 +2,23 @@
 
 <!--next-version-placeholder-->
 
+## v4.1.0 (2025-11-26)
+
+### Feature
+
+* **relationships:** Improved join features such as lazy=LazyPolicy, explicit=bool ([`f612d43`](https://github.com/trialandsuccess/TypeDAL/commit/f612d43b3a8f474a39bbcc68c1ce9b688939b13a))
+
+### Fix
+
+* Move `LazyPolicy` to config so TypeDALConfig works on < 3.14 again ([`7ad2b19`](https://github.com/trialandsuccess/TypeDAL/commit/7ad2b1934e129307f7aca9d20055e10df5636c92))
+
+### Documentation
+
+* Implemented outstanding todo's into documenation ([`6acb6d8`](https://github.com/trialandsuccess/TypeDAL/commit/6acb6d8ec0ae0ce5b92533e607e4d9a1b65600bc))
+* Continued working on docs todo list ([`022c95d`](https://github.com/trialandsuccess/TypeDAL/commit/022c95dc276ca356cf7e2c884e4d6d6b179aa108))
+* Started on the todo list of doc changes ([`2ad04c4`](https://github.com/trialandsuccess/TypeDAL/commit/2ad04c4de619af9ec1335c645d3e0ad11b25f9a0))
+* Added todo for what to update in docs ([`00eba3e`](https://github.com/trialandsuccess/TypeDAL/commit/00eba3e5d0b347a05d13bfa9dee36054cb261648))
+
 ## v4.0.2 (2025-10-14)
 
 ### Fix

{typedal-4.0.2 → typedal-4.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: TypeDAL
-Version: 4.0.2
+Version: 4.1.0
 Summary: Typing support for PyDAL
 Project-URL: Documentation, https://typedal.readthedocs.io/
 Project-URL: Issues, https://github.com/trialandsuccess/TypeDAL/issues

typedal-4.1.0/docs/1_getting_started.md

@@ -0,0 +1,91 @@
+# 1. Getting Started
+
+TypeDAL is built on top of pyDAL, which has excellent documentation. It is recommended to be familiar with at least the
+basics of this abstraction, before using this library. TypeDAL uses many of the same concepts, but with slightly
+different wording and syntax in some cases.
+
+[web2py - Chapter 6: The database abstraction layer](http://www.web2py.com/books/default/chapter/29/06/the-database-abstraction-layer)
+
+---
+
+### Installation
+
+```shell
+pip install typedal
+# or, if you're using py4web:
+pip install typedal[py4web]
+```
+
+### First Steps
+
+```python
+from typedal import TypeDAL
+# or, if in py4web:
+from typedal.for_py4web import TypeDAL
+
+db = TypeDAL("sqlite:memory")
+```
+
+TypeDAL accepts the same connection string format and other arguments as `pydal.DAL`.
+Again, see
+[their documentation](http://www.web2py.com/books/default/chapter/29/06/the-database-abstraction-layer#The-DAL-A-quick-tour)
+for more info about this. For additional configuration options specific to TypeDAL, see the [7. Advanced Configuration](./7_configuration.md) page.
+
+When using py4web, it is recommended to import the py4web-specific TypeDAL, which is a Fixture that handles database
+connections on request (just like the py4web specific DAL class does). More information about this can be found on [5. py4web](./5_py4web.md)
+
+### Simple Queries
+
+For direct SQL access, use `executesql()`:
+
+```python
+rows = db.executesql("SELECT * FROM some_table")
+```
+
+#### Safely Injecting Variables
+
+Use t-strings (Python 3.14+) for automatic SQL escaping:
+
+```python
+name = "Robert'); DROP TABLE Students;--"
+rows = db.executesql(t"SELECT * FROM some_table WHERE name = {name}")
+```
+
+Or use the `placeholders` argument with positional or named parameters:
+
+```python
+# Positional
+rows = db.executesql(
+    "SELECT * FROM some_table WHERE name = %s AND age > %s",
+    placeholders=[name, 18]
+)
+
+# Named
+rows = db.executesql(
+    "SELECT * FROM some_table WHERE name = %(name)s AND age > %(age)s",
+    placeholders={"name": name, "age": 18}
+)
+```
+
+#### Result Formatting
+
+By default, `executesql()` returns rows as tuples. To map results to specific fields, use `fields` (takes
+Field/TypedField objects) or `colnames` (takes column name strings):
+
+```python
+rows = db.executesql(
+    "SELECT id, name FROM some_table",
+    colnames=["id", "name"]
+)
+
+rows = db.executesql(
+    "SELECT id, name FROM some_table",
+    fields=[some_table.id, some_table.name]  # Requires table definition
+)
+```
+
+You can also use `as_dict` or `as_ordered_dict` to return dictionaries instead of tuples.
+
+Most of the time, you probably don't want to write raw queries. For that, you'll need to define some tables!
+Head to [2. Defining Tables](./2_defining_tables.md) to learn how.
+

{typedal-4.0.2 → typedal-4.1.0}/docs/2_defining_tables.md

@@ -102,5 +102,6 @@ def my_after_delete(query: Set):
 
 row.delete_record() # to trigger
 MyTable.where(...).delete() # to trigger
-
 ```
+
+"Now that we have some tables, it's time to actually query them! Let's go to [3. Building Queries](./3_building_queries.md) to learn how.

{typedal-4.0.2 → typedal-4.1.0}/docs/3_building_queries.md

@@ -30,27 +30,33 @@ The query builder uses the builder pattern, so you can keep adding to it (in any
 data:
 
 ```python
-Person
+builder = Person
 .where(Person.id > 0)
-.where(Person.id < 99, Person.id == 101)
+.where(Person.id < 99, Person.id == 101)  # id < 99 OR id == 101
 .select(Reference.id, Reference.title)
 .join('reference')
 .select(Person.ALL)
-.paginate(limit=5, page=2)  # final step: actually runs the query
+
+# final step: actually runs the query:
+builder.paginate(limit=5, page=2)
+
+# to get the SQL (for debugging or subselects), you can use:
+builder.to_sql()
 ```
 
 ```sql
-SELECT "person".*,
-       "reference"."id",
-       "reference"."title"
-FROM "person",
-     "reference"
-WHERE (("person"."id" IN (SELECT "person"."id"
-                          FROM "person"
-                          WHERE ((("person"."id" > 0)) AND
-                                 (("person"."id" < 99) OR ("person"."id" = 101)))
-                          ORDER BY "person"."id" LIMIT 1 OFFSET 0))
-  AND ("person"."reference" = "reference"."id") );
+SELECT "person".*
+     , "reference"."id"
+     , "reference"."title"
+    FROM "person"
+       , "reference"
+    WHERE (("person"."id" IN (SELECT "person"."id"
+                                  FROM "person"
+                                  WHERE ((("person"."id" > 0)) AND
+                                         (("person"."id" < 99) OR ("person"."id" = 101)))
+                                  ORDER BY "person"."id"
+                                  LIMIT 1 OFFSET 0))
+        AND ("person"."reference" = "reference"."id"));
 ```
 
 ### where
@@ -58,27 +64,81 @@ WHERE (("person"."id" IN (SELECT "person"."id"
 In pydal, this is the part that would be in `db(...)`.
 Can be used in multiple ways:
 
-- `.where(Query)` -> with a direct query such as `Table.id == 5`
+- `.where(query)` -> with a direct query such as `query = (Table.id == 5)`
 - `.where(lambda table: table.id == 5)` -> with a query via a lambda
 - `.where(id=5)` -> via keyword arguments
+- `.where({"id": 5})` -> via a dictionary (equivalent to keyword args)
+- `.where(Table.field)` -> with a Field directly, checks if field is not null
+
+When using multiple `.where()` calls, they will be ANDed together:
+`.where(lambda table: table.id == 5).where(active=True)` equals `(table.id == 5) & (table.active == True)`
 
-When using multiple where's, they will be ANDed:  
-`.where(lambda table: table.id == 5).where(lambda table: table.id == 6)` equals `(table.id == 5) & (table.id=6)`  
-When passing multiple queries to a single .where, they will be ORed:  
-`.where(lambda table: table.id == 5, lambda table: table.id == 6)` equals `(table.id == 5) | (table.id=6)`
+When passing multiple arguments to a single `.where()`, they will be ORed:
+`.where({"id": 5}, {"id": 6})` equals `(table.id == 5) | (table.id == 6)`
 
 ### select
 
-Here you can enter any number of fields as arguments: database columns by name ('id'), by field reference (table.id) or
-other (e.g. table.ALL).
+Here you can enter any number of fields as arguments: database columns by name ('id'), by field reference (Table.id),
+other (e.g. Table.ALL), or Expression objects.
 
 ```python
 Person.select('id', Person.name, Person.ALL)  # defaults to Person.ALL if select is omitted.
 ```
 
-You can also specify extra options such as `orderby` here. For supported keyword arguments, see
+You can also specify extra options as keyword arguments. Supported options are: `orderby`, `groupby`, `limitby`,
+`distinct`, `having`, `orderby_on_limitby`, `join`, `left`, `cache`, see also
 the [web2py docs](http://www.web2py.com/books/default/chapter/29/06/the-database-abstraction-layer#orderby-groupby-limitby-distinct-having-orderby_on_limitby-join-left-cache).
 
+```python
+Person.select(Person.name, distinct=True)
+```
+
+If you only want a list of name strings (in this example) instead of Person instances, you could use the column() method
+instead:
+
+```python
+Person.column(Person.name, distinct=True)
+```
+
+You can use `.orderby(*fields)` as an alternative to `select(orderby=...)`:
+
+```python
+Person.where(...).orderby(~Person.name, "age")
+```
+
+`.orderby()` accepts field references (`Table.field` or `"field_name""`), reverse ordering (`~Table.field`), or the
+literal `"<random>"`. Multiple field references can be passed (except when using `<random>`).
+
+#### Raw SQL Expressions
+
+For complex SQL that can't be expressed with field references, use `sql_expression()`:
+
+```python
+# Simple arithmetic
+expr = db.sql_expression("age * 2")
+Person.select(expr)
+
+# Safe parameter injection with t-strings (Python 3.14+)
+min_age = 21
+expr = db.sql_expression(t"age >= {min_age}", output_type="boolean")
+Person.where(expr).select()
+
+# Positional arguments
+expr = db.sql_expression("age > %s AND status = %s", 18, "active", output_type="boolean")
+Person.where(expr).select()
+
+# Named arguments
+expr = db.sql_expression(
+    "EXTRACT(year FROM %(date_col)s) = %(year)s",
+    date_col="created_at",
+    year=2023,
+    output_type="boolean"
+)
+Person.where(expr).select()
+```
+
+Expressions can be used in `where()`, `select()`, `orderby()`, and other query methods.
+
 ### join
 
 Include relationship fields in the result.
@@ -93,6 +153,8 @@ This can be overwritten with the `method` keyword argument (left or inner)
 Person.join('articles', method='inner')  # will only yield persons that have related articles
 ```
 
+For more details about relationships and joins, see [4. Relationships](./4_relationships.md).
+
 ### cache
 
 ```python
@@ -120,7 +182,7 @@ time.
 class ...
 ```
 
-In order to enable this functionality, TypeDAL adds a `before update` and `before delete` hook to your tables, 
+In order to enable this functionality, TypeDAL adds a `before update` and `before delete` hook to your tables,
 which manages the dependencies. You can disable this behavior by passing `cache_dependency=False` to `db.define`.
 Be aware doing this might break some caching functionality!
 
@@ -136,12 +198,15 @@ The Query Builder has a few operations that don't return a new builder instance:
 - paginate: this works similarly to `collect`, but returns a PaginatedRows instead, which has a `.next()`
   and `.previous()` method to easily load more pages.
 - collect_or_fail: where `collect` may return an empty result, this variant will raise an error if there are no results.
+- execute: get the raw rows matching your query as returned by pydal, without entity mapping or relationship loading.
+  Useful for subqueries or when you need lower-level control.
 - first: get the first entity matching your query, possibly with relationships loaded (if .join was used)
 - first_or_fail: where `first` may return an empty result, this variant will raise an error if there are no results.
+- to_sql: get the SQL string that would run, useful for debugging, subqueries and other advanced SQL operations.
 - update: instead of selecting rows, update those matching the current query (see [Delete](#delete))
 - delete: instead of selecting rows, delete those matching the current query (see [Update](#update))
 
-Additionally, you can directly call `.all()`, `.collect()`, `.count()`, `.first()` on a model.
+Additionally, you can directly call `.all()`, `.collect()`, `.count()`, `.first()` on a model (e.g. `User.all()`).
 
 ## Update
 

{typedal-4.0.2 → typedal-4.1.0}/docs/4_relationships.md

@@ -20,25 +20,37 @@ class Post(TypedTable):
     author: Author
 
 
-authors_with_roles = Author.join('role').collect()
+authors_with_roles = Author.join('roles').collect()
 posts_with_author = Post.join().collect()  # join can be called without arguments to join all relationships (in this case only 'author')
+post_deep = Post.join("author.roles").collect()  # nested relationship, accessible via post.author.roles
 ```
 
 In this example, the `Post` table contains a reference to the `Author` table. In that case, the `Author` `id` is stored
-in the
-`Post`'s `author` column.
+in the `Post`'s `author` column.
 Furthermore, the `Author` table contains a `list:reference` to `list[Role]`. This means multiple `id`s from the `Role`
-table
-can be stored in the `roles` column of `Author`.
+table can be stored in the `roles` column of `Author`.
 
 For these two cases, a Relationship is set-up automatically, which means `.join()` can work with those.
 
+### Alternative Join Syntax
+
+You can pass the relationship object directly instead of its name as a string:
+
+```python
+posts_with_author = Post.join(Post.author).collect()
+```
+
+This works, but note that `Post.author` is typed as `Relationship[Author]` at the class level, while `row.author` is
+typed as `Author` at the instance level. Some editors may complain about type mismatches when using this syntax (e.g.,
+reporting that `list[Tag]` isn't a `Relationship`). If you encounter type checking issues, use the string syntax
+instead.
+
 ## Other Relationships
 
 To get the reverse relationship, you'll have to tell TypeDAL how the two tables relate to each other (since guessing is
 complex and unreliable).
 
-For example, to set up the reverse relationshp from author to posts:
+For example, to set up the reverse relationship from author to posts:
 
 ```python
 @db.define()
@@ -46,7 +58,6 @@ class Author(TypedTable):
     name: str
 
     posts = relationship(list["Post"], condition=lambda author, post: author.id == post.author, join="left")
-
 ```
 
 Note that `"Post"` is in quotes. This is because the `Post` class is defined later, so a reference to it is not
@@ -62,7 +73,7 @@ class Role(TypedTable):
     authors = relationship(list["Author"], condition=lambda role, author: author.roles.contains(role.id), join="left")
 ```
 
-Here, contains is used since `Author.roles` is a `list:reference`.
+Here, `contains` is used since `Author.roles` is a `list:reference`.
 See [the web2py docs](http://www.web2py.com/books/default/chapter/29/06/the-database-abstraction-layer#list-type-and-contains)
 for more details.
 
@@ -82,8 +93,8 @@ class Sidekick(TypedTable):
     superhero: SuperHero
 ```
 
-In this example, `Relationship["Sidekick"]` is added as an extra type hint, since the reference to the table
-in `relationship("Sidekick", ...)` is a string. This has to be passed as a string, since the Sidekick class is defined
+In this example, `Relationship["Sidekick"]` is added as an extra type hint, since the reference to the table in
+`relationship("Sidekick", ...)` is a string. This has to be passed as a string, since the Sidekick class is defined
 after the superhero class.
 Adding the `Relationship["Sidekick"]` hint is optional, but recommended to improve editor support.
 
@@ -107,6 +118,7 @@ class Post(TypedTable):
         tag.on(tag.id == tagged.tag),
     ])
 
+
 # without unique alias:
 
 @db.define()
@@ -126,7 +138,65 @@ class Tagged(TypedTable):
 ```
 
 Instead of a `condition`, it is recommended to define an `on`. Using a condition is possible, but could lead to pydal
-generation a `CROSS JOIN` instead of a `LEFT JOIN`, which is bad for performance.
+generating a `CROSS JOIN` instead of a `LEFT JOIN`, which is bad for performance.
 In this example, `Tag` is connected to `Post` and vice versa via the `Tagged` table.
 It is recommended to use the tables received as arguments from the lambda (e.g. `tag.on` instead of `Tag.on` directly),
 since these use aliases under the hood, which prevents conflicts when joining the same table multiple times.
+
+## Lazy Loading and Explicit Relationships
+
+### Lazy Policy
+
+The `lazy` parameter on a relationship controls what happens when you access relationship data without explicitly
+joining it first:
+
+```python
+@db.define()
+class User(TypedTable):
+    name: str
+    posts = relationship(list["Post"], condition=lambda user, post: user.id == post.author, lazy="forbid")
+```
+
+Available policies:
+
+- **`"forbid"`**: Raises an error. Prevents N+1 query problems by making them fail fast.
+- **`"warn"`**: Returns an empty value (empty list or `None`) with a console warning.
+- **`"ignore"`**: Returns an empty value silently.
+- **`"tolerate"`**: Fetches the data but logs a warning about potential performance issues.
+- **`"allow"`**: Fetches the data silently.
+
+If `lazy=None` (the default), the relationship uses the database's default lazy policy. You can set this globally via
+`TypeDAL`'s `lazy_policy` option (see [7. Advanced Configuration](./7_configuration.md) for configuration details),
+which defaults to `"tolerate"`.
+
+### Explicit Relationships
+
+Use `explicit=True` for relationships that are expensive to join or rarely needed:
+
+```python
+@db.define()
+class User(TypedTable):
+    name: str
+    audit_logs = relationship(list["AuditLog"], condition=lambda user, log: user.id == log.user, explicit=True)
+```
+
+When you call `.join()` without arguments, explicit relationships are skipped:
+
+```python
+user = User.join().first()  # user.audit_logs follows the lazy policy (empty/error/warning depending on setting)
+```
+
+To include an explicit relationship, reference it by name:
+
+```python
+user = User.join("audit_logs").first()  # now user.audit_logs is populated
+```
+
+## What's Next?
+
+Depending on your setup:
+
+- **Using py4web or web2py?** → [5. py4web & web2py](./5_py4web.md)
+- **Ready to manage your database?** → [6. Migrations](./6_migrations.md)
+- **Dive deeper into functionality?** → [8.: Mixins](./8_mixins.md)
+

typedal-4.1.0/docs/6_migrations.md

@@ -0,0 +1,90 @@
+# 6. Migrations
+
+By default, pydal manages migrations in your database schema
+automatically ([See also: the web2py docs](http://www.web2py.com/books/default/chapter/29/06/the-database-abstraction-layer#Migrations)).
+This can be problematic in a production environment where you want to disable automatic table changes and manage these
+by hand.
+
+TypeDAL integrates with [`edwh-migrate`](https://pypi.org/project/edwh-migrate/) to make this easier. With this tool,
+you write migrations (`CREATE`, `ALTER`, `DROP` statements) in SQL and it keeps track of which actions have already been
+executed on your database.
+
+To make this process easier, TypeDAL also integrates with [`pydal2sql`](https://pypi.org/project/pydal2sql/), which can
+convert your pydal/TypeDAL table definitions into `CREATE` statements for new tables, or `ALTER` statements for existing
+tables.
+
+## Installation
+
+To enable the migrations functionality within TypeDAL, install it with the migrations extra:
+
+```bash
+pip install typedal[migrations] # also included in typedal[all]
+```
+
+## Minimal Configuration
+
+To use migrations, you need to configure TypeDAL in your `pyproject.toml`. At minimum, you must set:
+
+- `database`: Your database URI
+- `dialect`: The database type (e.g., `sqlite`, `postgres`)
+- `migrate`: Set to `false` to disable pydal's automatic migrations
+- `flag_location`: Where edwh-migrate stores its migration tracking
+- `input`: Path to your table definitions (e.g., `data_model.py`)
+- `output`: Where generated migrations are written (e.g., `migrations/`)
+
+Optionally:
+
+- `database_to_restore`: Path to a SQL file to restore before running migrations on a fresh database.
+
+Here's a minimal example:
+
+```toml
+[tool.typedal]
+database = "sqlite://"
+dialect = "sqlite"
+migrate = false
+flag_location = "migrations/.flags"
+input = "path/to/data_model.py"
+output = "path/to/migrations.py"
+```
+
+For dynamic properties or secrets (like a database with credentials), 
+add them to your `.env` file or set them as environment variables (optionally prefixed with `TYPEDAL_`):
+
+```env
+TYPEDAL_DATABASE = "psql://user:password@host:5432/database"
+```
+
+> **Full configuration reference**: For all available options, multiple connections, environment overrides, and other
+> settings, see [7. Configuration](./7_configuration.md).
+
+You can generate a config interactively with `typedal setup`, or view your current config with `typedal --show-config`.
+
+## Generate Migrations (pydal2sql)
+
+With your config in place, generate migrations from your table definitions:
+
+```bash
+typedal migrations.generate
+```
+
+You can override config values with CLI flags. See all options:
+
+```bash
+typedal migrations.generate --help
+```
+
+## Run Migrations (edwh-migrate)
+
+Apply your migrations to the database:
+
+```bash
+typedal migrations.run
+```
+
+With a correctly configured setup, this should function without extra arguments.
+You can however overwrite the behavior as defined in the config. See all options:
+
+```bash
+typedal migrations.run --help
+```