xtr-orm 1.5.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- xtr_orm-1.5.0/LICENSE +21 -0
- xtr_orm-1.5.0/PKG-INFO +487 -0
- xtr_orm-1.5.0/README.md +448 -0
- xtr_orm-1.5.0/pyproject.toml +200 -0
- xtr_orm-1.5.0/pyproject.toml.orig +210 -0
- xtr_orm-1.5.0/src/xtr_orm/__init__.py +51 -0
- xtr_orm-1.5.0/src/xtr_orm/bundle/__init__.py +11 -0
- xtr_orm-1.5.0/src/xtr_orm/bundle/connection_config.py +250 -0
- xtr_orm-1.5.0/src/xtr_orm/bundle/orm_bundle.py +346 -0
- xtr_orm-1.5.0/src/xtr_orm/bundle/orm_config.py +101 -0
- xtr_orm-1.5.0/src/xtr_orm/command/__init__.py +56 -0
- xtr_orm-1.5.0/src/xtr_orm/command/connection_command.py +134 -0
- xtr_orm-1.5.0/src/xtr_orm/command/connections.py +37 -0
- xtr_orm-1.5.0/src/xtr_orm/command/database_create_command.py +53 -0
- xtr_orm-1.5.0/src/xtr_orm/command/database_drop_command.py +65 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_current_command.py +44 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_diff_command.py +60 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_dump_schema_command.py +56 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_execute_command.py +86 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_generate_command.py +47 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_latest_command.py +44 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_list_command.py +71 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_migrate_command.py +108 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_rollup_command.py +43 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_status_command.py +66 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_up_to_date_command.py +83 -0
- xtr_orm-1.5.0/src/xtr_orm/command/migrations_version_command.py +89 -0
- xtr_orm-1.5.0/src/xtr_orm/command/run_sql_command.py +77 -0
- xtr_orm-1.5.0/src/xtr_orm/connection_registry.py +192 -0
- xtr_orm-1.5.0/src/xtr_orm/database/__init__.py +7 -0
- xtr_orm-1.5.0/src/xtr_orm/database/database_manager.py +196 -0
- xtr_orm-1.5.0/src/xtr_orm/exception/__init__.py +19 -0
- xtr_orm-1.5.0/src/xtr_orm/exception/invalid_argument_error.py +28 -0
- xtr_orm-1.5.0/src/xtr_orm/exception/migration_error.py +28 -0
- xtr_orm-1.5.0/src/xtr_orm/exception/orm_error.py +14 -0
- xtr_orm-1.5.0/src/xtr_orm/exception/session_unavailable_error.py +29 -0
- xtr_orm-1.5.0/src/xtr_orm/exception/unknown_connection_error.py +28 -0
- xtr_orm-1.5.0/src/xtr_orm/exception/unsupported_database_error.py +29 -0
- xtr_orm-1.5.0/src/xtr_orm/messenger/__init__.py +23 -0
- xtr_orm-1.5.0/src/xtr_orm/messenger/close_connection_middleware.py +75 -0
- xtr_orm-1.5.0/src/xtr_orm/messenger/open_transaction_logger_middleware.py +90 -0
- xtr_orm-1.5.0/src/xtr_orm/messenger/transaction_middleware.py +104 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/__init__.py +29 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/_environment.py +188 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/_history.py +178 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/_revisions.py +198 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/_schema.py +66 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/available_migration.py +27 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/direction.py +14 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/executed_migration.py +31 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/execution_result.py +26 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/migration_plan.py +26 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/migration_status.py +48 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/migrations_config.py +162 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/migrator.py +680 -0
- xtr_orm-1.5.0/src/xtr_orm/migrations/template/script.py.mako +26 -0
- xtr_orm-1.5.0/src/xtr_orm/py.typed +0 -0
xtr_orm-1.5.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 xterr
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
xtr_orm-1.5.0/PKG-INFO
ADDED
|
@@ -0,0 +1,487 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: xtr-orm
|
|
3
|
+
Version: 1.5.0
|
|
4
|
+
Summary: Databases for async applications: engines, sessions and versioned schema migrations, configured once per connection.
|
|
5
|
+
Keywords: orm,database,migrations,sqlalchemy,alembic,asyncio
|
|
6
|
+
Author: Xterr
|
|
7
|
+
Author-email: Xterr <me@xterr.dev>
|
|
8
|
+
License-Expression: MIT
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Classifier: Development Status :: 3 - Alpha
|
|
11
|
+
Classifier: Framework :: AsyncIO
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
17
|
+
Classifier: Topic :: Database
|
|
18
|
+
Classifier: Typing :: Typed
|
|
19
|
+
Requires-Dist: advanced-alchemy>=1.11
|
|
20
|
+
Requires-Dist: alembic>=1.20
|
|
21
|
+
Requires-Dist: sqlalchemy[asyncio]>=2.0.41
|
|
22
|
+
Requires-Dist: typing-extensions>=4.4
|
|
23
|
+
Requires-Dist: xtr-logging-contracts>=1.0,<2
|
|
24
|
+
Requires-Dist: xtr-console>=1.0,<2 ; extra == 'console'
|
|
25
|
+
Requires-Dist: xtr-dependency-injection>=1.0,<2 ; extra == 'di'
|
|
26
|
+
Requires-Dist: xtr-service-contracts>=1.0,<2 ; extra == 'di'
|
|
27
|
+
Requires-Dist: xtr-messenger>=1.0,<2 ; extra == 'messenger'
|
|
28
|
+
Requires-Dist: asyncmy>=0.2.9 ; extra == 'mysql'
|
|
29
|
+
Requires-Dist: asyncpg>=0.29 ; extra == 'postgres'
|
|
30
|
+
Requires-Dist: aiosqlite>=0.20 ; extra == 'sqlite'
|
|
31
|
+
Requires-Python: >=3.11
|
|
32
|
+
Provides-Extra: console
|
|
33
|
+
Provides-Extra: di
|
|
34
|
+
Provides-Extra: messenger
|
|
35
|
+
Provides-Extra: mysql
|
|
36
|
+
Provides-Extra: postgres
|
|
37
|
+
Provides-Extra: sqlite
|
|
38
|
+
Description-Content-Type: text/markdown
|
|
39
|
+
|
|
40
|
+
<div align="center">
|
|
41
|
+
|
|
42
|
+
# xtr-orm
|
|
43
|
+
|
|
44
|
+
**Databases for async applications: engines, sessions and versioned schema migrations, configured once per connection.**
|
|
45
|
+
|
|
46
|
+
<img alt="python 3.11+" src="https://img.shields.io/badge/python-%E2%89%A5%203.11-3776AB?logo=python&logoColor=white">
|
|
47
|
+
<img alt="typed" src="https://img.shields.io/badge/typed-ty%20%2B%20basedpyright-1f6feb">
|
|
48
|
+
<img alt="license MIT" src="https://img.shields.io/badge/license-MIT-blue">
|
|
49
|
+
|
|
50
|
+
</div>
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## Why?
|
|
55
|
+
|
|
56
|
+
An application with a database needs the same things every time: an engine per database, a
|
|
57
|
+
session per request or message, a way to change the schema one reviewed step at a time, and
|
|
58
|
+
commands to run those steps from a deploy script. Each is a few dozen lines of glue, written
|
|
59
|
+
again in every project.
|
|
60
|
+
|
|
61
|
+
This package is that glue, done once. You describe each connection — a URL, its replicas,
|
|
62
|
+
its engine and session options, where its revisions live — and get:
|
|
63
|
+
|
|
64
|
+
- 🔌 **Engines and sessions** built from one configuration, on
|
|
65
|
+
[SQLAlchemy](https://www.sqlalchemy.org/)'s async API and
|
|
66
|
+
[advanced-alchemy](https://advanced-alchemy.litestar.dev/)'s models, repositories and
|
|
67
|
+
services.
|
|
68
|
+
- 📖 **Read replicas** — reads go to a replica, writes and everything after them to the
|
|
69
|
+
primary.
|
|
70
|
+
- 🧬 **Versioned migrations** on [Alembic](https://alembic.sqlalchemy.org/): diffed from your
|
|
71
|
+
models, run up or down to any revision, one at a time or all at once — with no `env.py` and
|
|
72
|
+
no `alembic.ini` to maintain.
|
|
73
|
+
- 📜 **A full history**: every revision applied, when, and how long it took.
|
|
74
|
+
- 🧾 **One transaction per message** — with [xtr-messenger](../xtr-messenger), every handler of
|
|
75
|
+
a message shares one session, committed once they all succeeded and rolled back otherwise.
|
|
76
|
+
- 🛠️ **Fifteen console commands**, from `orm:database:create` to `orm:migrations:up-to-date`.
|
|
77
|
+
|
|
78
|
+
## Install
|
|
79
|
+
|
|
80
|
+
```sh
|
|
81
|
+
uv add "xtr-orm[postgres]" # asyncpg
|
|
82
|
+
uv add "xtr-orm[di,console,messenger,sqlite]" # everything, on SQLite
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
| Extra | Brings | For |
|
|
86
|
+
|---|---|---|
|
|
87
|
+
| `postgres` / `mysql` / `sqlite` | `asyncpg` / `asyncmy` / `aiosqlite` | The async driver of your database |
|
|
88
|
+
| `di` | `xtr-dependency-injection` | The `OrmBundle`: engines, sessions, migrators from one configuration |
|
|
89
|
+
| `console` | `xtr-console` | The `orm:*` commands |
|
|
90
|
+
| `messenger` | `xtr-messenger` | The [message bus middleware](#message-bus-middleware) |
|
|
91
|
+
|
|
92
|
+
## Quick start
|
|
93
|
+
|
|
94
|
+
Without a container, a `Migrator` runs one database's revisions:
|
|
95
|
+
|
|
96
|
+
```python
|
|
97
|
+
from sqlalchemy.ext.asyncio import create_async_engine
|
|
98
|
+
from xtr_orm import Migrator, MigrationsConfig
|
|
99
|
+
|
|
100
|
+
from app.models import Base
|
|
101
|
+
|
|
102
|
+
engine = create_async_engine("postgresql+asyncpg://app:secret@db/shop")
|
|
103
|
+
migrator = Migrator(engine, MigrationsConfig(directory="migrations"), Base.metadata)
|
|
104
|
+
|
|
105
|
+
await migrator.diff("add the catalogue") # writes migrations/<date>_<rev>_add_the_catalogue.py
|
|
106
|
+
await migrator.migrate() # runs every revision not applied yet
|
|
107
|
+
status = await migrator.status()
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
## Migrations
|
|
111
|
+
|
|
112
|
+
Revisions are ordinary migration files, one per change, in the connection's migrations
|
|
113
|
+
directory. The package writes them from its own template and runs them itself, handing the
|
|
114
|
+
migration runner the live connection and your table definitions directly — there is no
|
|
115
|
+
start-up script to keep in step with your models.
|
|
116
|
+
|
|
117
|
+
The database keeps two tables, in the same schema:
|
|
118
|
+
|
|
119
|
+
- the **version table** (`alembic_version`) holds the revisions the database is *at*. It
|
|
120
|
+
decides what a migration runs.
|
|
121
|
+
- the **history table** (`alembic_version_history`) holds a row per revision ever applied —
|
|
122
|
+
`version`, `executed_at`, `execution_time` in milliseconds. Each row is written by the step
|
|
123
|
+
that applies its revision, in the same transaction, so the two agree. It is what lets a
|
|
124
|
+
report name every applied revision, including one whose file is gone.
|
|
125
|
+
|
|
126
|
+
A **target** — for `migrate` and `plan` — is a revision (a full or partial id), a relative
|
|
127
|
+
one (`+1`, `-2`, `ae10+1`), or `latest`, `first` (before every revision), `next`, `prev` or
|
|
128
|
+
`current`. A revision already applied is migrated *down* to; one that is not, *up* to.
|
|
129
|
+
|
|
130
|
+
| `Migrator` method | Does |
|
|
131
|
+
|---|---|
|
|
132
|
+
| `status()` | Where the database stands: current, latest, previous, next, available, executed, new, and executed but without a file |
|
|
133
|
+
| `plan(target)` / `migrate(target)` / `migrate_sql(target)` | What reaching a target runs / run it / the SQL it would run, recording included |
|
|
134
|
+
| `plan_for_versions(versions, direction)` / `execute(...)` / `execute_sql(...)` | What running exactly those revisions, each up or down, runs — every id resolved and checked / run them, nothing before or after / the SQL it would run |
|
|
135
|
+
| `generate(message)` | Write an empty revision after the latest |
|
|
136
|
+
| `diff(message, allow_empty=, from_empty_schema=)` | Write what differs between the database and the table definitions |
|
|
137
|
+
| `dump_schema(message, table_filters=)` | Write a first revision creating every table an existing database has |
|
|
138
|
+
| `add_versions(versions)` / `add_all_versions()` | Record revisions as applied, running nothing |
|
|
139
|
+
| `delete_versions(versions)` / `delete_all_versions()` | Record revisions as not applied, running nothing |
|
|
140
|
+
| `rollup()` | Forget every applied revision, then record the single remaining one |
|
|
141
|
+
|
|
142
|
+
**Running one revision alone.** The version table holds only where the database *is*, so a
|
|
143
|
+
revision can be run — or recorded — on its own only where that stays true: going up, it must
|
|
144
|
+
not be applied and must follow only applied revisions; going down, it must be applied and
|
|
145
|
+
followed by none that is. Several versions are checked in turn, so a revision and the one
|
|
146
|
+
after it can go together. Anything else is refused before the database is touched.
|
|
147
|
+
|
|
148
|
+
**Up to date** means every revision with a file is applied — the database's versions against
|
|
149
|
+
the files, not its schema against your models; `diff` compares those.
|
|
150
|
+
|
|
151
|
+
`MigrationsConfig`:
|
|
152
|
+
|
|
153
|
+
| Field | Default | Meaning |
|
|
154
|
+
|---|---|---|
|
|
155
|
+
| `directory` | `"migrations"` | Where revision files are read and written |
|
|
156
|
+
| `version_table` / `history_table` | `"alembic_version"` / `"alembic_version_history"` | The two tables |
|
|
157
|
+
| `version_table_schema` | `None` | Their schema; the connection's default when `None` |
|
|
158
|
+
| `file_template` | `"{year}{month}{day}{hour}{minute}{second}_{rev}_{slug}"` | How a file is named; also `{epoch}` |
|
|
159
|
+
| `truncate_slug_length` / `timezone` | `40` / `None` | The longest message part of a name / the zone of a revision's date |
|
|
160
|
+
| `template_directory` | `None` | A directory with a `script.py.mako` to render new revisions from |
|
|
161
|
+
| `recursive_version_locations` | `False` | Read subdirectories of `directory` too |
|
|
162
|
+
| `post_write_hooks` | `()` | Programs run on each file written, such as a formatter |
|
|
163
|
+
| `transaction_per_migration` / `transactional_ddl` | `False` / `None` | Commit after each revision / run schema changes in the transaction |
|
|
164
|
+
| `compare_type` / `compare_server_default` | `False` / `False` | What a diff compares besides tables, columns, indexes and constraints |
|
|
165
|
+
| `render_as_batch` | `False` | Write table changes as batches, which SQLite needs |
|
|
166
|
+
| `include_schemas` / `include_name` / `include_object` | `False` / `None` / `None` | What a diff reads |
|
|
167
|
+
| `render_item` / `process_revision_directives` / `user_module_prefix` | `None` | How a diff is written |
|
|
168
|
+
| `context_options` | `{}` | Any other migration context option, passed as it is |
|
|
169
|
+
|
|
170
|
+
A custom column type is written under its own module's name, and the revision imports that
|
|
171
|
+
module — `advanced_alchemy.types.guid.GUID(length=16)` with `import advanced_alchemy.types.guid`;
|
|
172
|
+
set `user_module_prefix` or `render_item` to write it otherwise. `compare_type` is off by
|
|
173
|
+
default because a type the database stores as another — a UUID kept as bytes on SQLite — would
|
|
174
|
+
otherwise differ in every diff.
|
|
175
|
+
|
|
176
|
+
### Your own revision template
|
|
177
|
+
|
|
178
|
+
New revisions are rendered from `script.py.mako` — the package's own, or the one in
|
|
179
|
+
`template_directory`. It receives what Alembic's own templates do: `message`, `up_revision`,
|
|
180
|
+
`down_revision`, `branch_labels`, `depends_on`, `create_date`, `imports`, `upgrades`,
|
|
181
|
+
`downgrades`, and the `comma` filter (`${down_revision | comma,n}`). A template written for an
|
|
182
|
+
Alembic project works as it is; nothing but that file is read from the directory.
|
|
183
|
+
|
|
184
|
+
```python
|
|
185
|
+
MigrationsConfig(
|
|
186
|
+
template_directory="%kernel.project_dir%/resources/migrations",
|
|
187
|
+
user_module_prefix="sa.", # a template aliasing sa.GUID = GUID, as advanced-alchemy's does
|
|
188
|
+
)
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
A template calling `op.get_context().autocommit_block()` commits a revision's schema changes
|
|
192
|
+
before the version and history tables are written, and gives up the transaction a failing
|
|
193
|
+
revision would roll back; set `transaction_per_migration=True` with it, and keep it for the
|
|
194
|
+
revisions that need it (`CREATE INDEX CONCURRENTLY`).
|
|
195
|
+
|
|
196
|
+
## Use in an application
|
|
197
|
+
|
|
198
|
+
Everything adding this package to an application on
|
|
199
|
+
[xtr-dependency-injection](../xtr-dependency-injection) takes — and, read backwards, what
|
|
200
|
+
removing it undoes.
|
|
201
|
+
|
|
202
|
+
- **Install** — `uv add "xtr-orm[di,console,postgres]"`; `mysql` or `sqlite` instead of
|
|
203
|
+
`postgres` for those databases; drop `console` to go without the commands; add `messenger`
|
|
204
|
+
for the [message bus middleware](#message-bus-middleware).
|
|
205
|
+
- **Activate** — `OrmBundle: {"all": True}` in `BUNDLES` in `<app>/bundles.py`, imported from
|
|
206
|
+
`xtr_orm.bundle`.
|
|
207
|
+
- **Brings along** — the logging and console bundles, when those packages are installed.
|
|
208
|
+
With the messenger bundle active too, the [middleware](#message-bus-middleware) are
|
|
209
|
+
registered for a bus to list.
|
|
210
|
+
- **Configure** — optional: with no configuration there is one connection, `default`, reading
|
|
211
|
+
`DATABASE_URL`, with its revisions in `migrations` in the project directory and diffs
|
|
212
|
+
against the models advanced-alchemy registers. Connections, replicas and table definitions go
|
|
213
|
+
in `<app>/config/orm.py`, a `@configure` function returning `OrmConfig` — see
|
|
214
|
+
[Kernel / bundle](#kernel--bundle).
|
|
215
|
+
- **Environment** — `DATABASE_URL`, read when a connection is first used — engine, session,
|
|
216
|
+
migrator or command — not when the application boots. Its query may carry options; see
|
|
217
|
+
[Options in the URL](#options-in-the-url).
|
|
218
|
+
- **Ignore** — nothing: commit `migrations/`.
|
|
219
|
+
- **Remove** — take the `orm_*` entries out of every bus's `middleware`, drop the `BUNDLES`
|
|
220
|
+
entry, delete `<app>/config/orm.py`, then `uv remove xtr-orm`.
|
|
221
|
+
- **Check** — `debug:bundles` shows `orm` as `listed` and `active`, and
|
|
222
|
+
`orm:migrations:status` reaches the database.
|
|
223
|
+
|
|
224
|
+
## Kernel / bundle
|
|
225
|
+
|
|
226
|
+
```python
|
|
227
|
+
# app/bundles.py
|
|
228
|
+
from xtr_orm.bundle import OrmBundle
|
|
229
|
+
|
|
230
|
+
BUNDLES = {OrmBundle: {"all": True}}
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
```python
|
|
234
|
+
# app/config/orm.py
|
|
235
|
+
from xtr_dependency_injection import configure, env
|
|
236
|
+
from xtr_orm.bundle import ConnectionConfig, OrmConfig
|
|
237
|
+
|
|
238
|
+
from app.models import Base
|
|
239
|
+
|
|
240
|
+
|
|
241
|
+
@configure
|
|
242
|
+
def orm() -> OrmConfig:
|
|
243
|
+
return OrmConfig(
|
|
244
|
+
connections={
|
|
245
|
+
"default": ConnectionConfig(
|
|
246
|
+
url=env("DATABASE_URL"),
|
|
247
|
+
replicas={"replica1": env("DATABASE_REPLICA_URL")},
|
|
248
|
+
engine_options={"pool_size": 10, "pool_pre_ping": True},
|
|
249
|
+
session_options={"expire_on_commit": False},
|
|
250
|
+
metadata=Base.metadata,
|
|
251
|
+
),
|
|
252
|
+
"reports": ConnectionConfig(url=env("REPORTS_URL"), bind_key="reports"),
|
|
253
|
+
},
|
|
254
|
+
)
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
For every connection, qualified by its name — and the default connection's without a
|
|
258
|
+
qualifier too:
|
|
259
|
+
|
|
260
|
+
| Service | What it is |
|
|
261
|
+
|---|---|
|
|
262
|
+
| `SQLAlchemyAsyncConfig` | advanced-alchemy's configuration, for its repositories and services |
|
|
263
|
+
| `AsyncEngine` | The engine on the primary database |
|
|
264
|
+
| `async_sessionmaker[AsyncSession]` | The session maker — `RoutingAsyncSessionMaker` instead for a connection with replicas |
|
|
265
|
+
| `AsyncSession` | One per unit of work — a request, a message, a command — closed, and rolled back if not committed, when the unit ends |
|
|
266
|
+
| `Migrator` / `DatabaseManager` | The connection's migrations / its database, both on the primary |
|
|
267
|
+
| `ConnectionRegistry` | Every connection by name, unqualified — what the commands use |
|
|
268
|
+
|
|
269
|
+
```python
|
|
270
|
+
from typing import Annotated
|
|
271
|
+
|
|
272
|
+
from sqlalchemy.ext.asyncio import AsyncSession
|
|
273
|
+
from xtr_dependency_injection import Injected, Target, as_service
|
|
274
|
+
|
|
275
|
+
|
|
276
|
+
@as_service
|
|
277
|
+
class InvoiceRepository:
|
|
278
|
+
def __init__(self, session: Injected[AsyncSession]) -> None: ...
|
|
279
|
+
|
|
280
|
+
|
|
281
|
+
async def report(session: Annotated[AsyncSession, Target("reports")]) -> None: ...
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
Every engine is disposed when the container closes. With the logging bundle active, an `orm`
|
|
285
|
+
channel is added and every migrator logs each revision it runs there. With the messenger
|
|
286
|
+
bundle active, the [message bus middleware](#message-bus-middleware) are registered by name, and
|
|
287
|
+
every message is a unit of work: its handlers share one `AsyncSession` per connection.
|
|
288
|
+
|
|
289
|
+
| `OrmConfig` field | Default | Meaning |
|
|
290
|
+
|---|---|---|
|
|
291
|
+
| `default_connection` | `"default"` | The connection provided without a qualifier, and used by every command not given `--connection` |
|
|
292
|
+
| `connections` | `{"default": ConnectionConfig()}` | Every connection, by name |
|
|
293
|
+
|
|
294
|
+
| `ConnectionConfig` field | Default | Meaning |
|
|
295
|
+
|---|---|---|
|
|
296
|
+
| `url` | `"%env(DATABASE_URL)%"` | The database URL, with an async driver; its query may carry options — see below |
|
|
297
|
+
| `replicas` | `{}` | Read replicas, each a URL by name |
|
|
298
|
+
| `keep_replica` | `False` | Go back to the replicas once a session commits |
|
|
299
|
+
| `engine_options` | `{}` | `echo`, `pool_size`, `max_overflow`, `pool_pre_ping`, `pool_recycle`, `isolation_level`, `connect_args`, … |
|
|
300
|
+
| `session_options` | `{}` | `expire_on_commit`, `autoflush`, `info`, … |
|
|
301
|
+
| `alchemy_options` | `{}` | Any other option of advanced-alchemy's configuration, such as `enable_touch_updated_timestamp_listener` |
|
|
302
|
+
| `bind_key` | `None` | Which models belong to this database, when they declare several |
|
|
303
|
+
| `metadata` | `None` | The table definitions a diff compares with; advanced-alchemy's for `bind_key` when `None` |
|
|
304
|
+
| `migrations` | `None` | `MigrationsConfig`; `migrations` in the project directory for the default connection, `migrations/<name>` for any other, when `None` |
|
|
305
|
+
|
|
306
|
+
### Options in the URL
|
|
307
|
+
|
|
308
|
+
One environment variable can configure a whole connection. An engine or session option, or
|
|
309
|
+
`keep_replica`, is taken out of the URL's query and merged over the one configured — the URL
|
|
310
|
+
wins; every other option stays on the URL for the driver:
|
|
311
|
+
|
|
312
|
+
```sh
|
|
313
|
+
DATABASE_URL="postgresql+asyncpg://app:secret@db/shop?pool_size=20&expire_on_commit=false&ssl=require"
|
|
314
|
+
```
|
|
315
|
+
|
|
316
|
+
| Read from the URL | Options |
|
|
317
|
+
|---|---|
|
|
318
|
+
| Engine | `echo`, `echo_pool`, `enable_from_linting`, `hide_parameters`, `insertmanyvalues_page_size`, `isolation_level`, `label_length`, `logging_name`, `max_identifier_length`, `max_overflow`, `paramstyle`, `pool_logging_name`, `pool_pre_ping`, `pool_recycle`, `pool_size`, `pool_timeout`, `pool_use_lifo`, `query_cache_size`, `use_insertmanyvalues` |
|
|
319
|
+
| Session | `autobegin`, `autoflush`, `expire_on_commit`, `join_transaction_mode`, `twophase` |
|
|
320
|
+
| Connection | `keep_replica` |
|
|
321
|
+
|
|
322
|
+
Flags read `1`/`true`/`yes`/`on` and `0`/`false`/`no`/`off`; numbers are numbers; `echo` also
|
|
323
|
+
takes `debug`. A value that does not read fails the service that needed the URL, naming the
|
|
324
|
+
option. Only the connection's own URL is read this way, not its replicas'.
|
|
325
|
+
|
|
326
|
+
### Read replicas
|
|
327
|
+
|
|
328
|
+
With `replicas`, a session reads from a replica picked at random, and goes to the primary —
|
|
329
|
+
`url` — for a write, for a transaction's statements, and for every read after them, so it
|
|
330
|
+
sees what it wrote. It stays there for the rest of its unit of work, unless `keep_replica`
|
|
331
|
+
sends it back to the replicas once it commits. Each unit of work starts on the replicas,
|
|
332
|
+
whatever the one before wrote. Migrations and the database commands always use the primary.
|
|
333
|
+
|
|
334
|
+
A block can be sent elsewhere with advanced-alchemy's `primary_context()` and
|
|
335
|
+
`replica_context()`, from `advanced_alchemy.routing`.
|
|
336
|
+
|
|
337
|
+
## Console commands
|
|
338
|
+
|
|
339
|
+
Every command takes `--connection NAME`; the default connection's is used without it. The
|
|
340
|
+
commands that change the database ask first; `-n` answers yes.
|
|
341
|
+
|
|
342
|
+
| Command | Does |
|
|
343
|
+
|---|---|
|
|
344
|
+
| `orm:database:create [--if-not-exists]` | Creates the database, through the server's maintenance database |
|
|
345
|
+
| `orm:database:drop --force [--if-exists]` | Drops it; refused without `--force` |
|
|
346
|
+
| `orm:run-sql SQL [--force-fetch]` | Runs a statement, committed; shows its rows or how many it changed — when the driver can tell |
|
|
347
|
+
| `orm:migrations:current` | The revisions the database is at — `base` for none |
|
|
348
|
+
| `orm:migrations:latest` | The revisions no other follows |
|
|
349
|
+
| `orm:migrations:list` | Every revision with its status, when it ran and how long it took |
|
|
350
|
+
| `orm:migrations:status` | The tables, the database, previous / current / next / latest, and the counts |
|
|
351
|
+
| `orm:migrations:up-to-date [-u] [-l]` | Exits 1 with revisions not applied; 2 with `--fail-on-unregistered` and applied revisions without a file |
|
|
352
|
+
| `orm:migrations:migrate [TARGET] [--dry-run] [--write-sql PATH] [--allow-no-migration]` | Runs every revision to `TARGET`, `latest` by default |
|
|
353
|
+
| `orm:migrations:execute VERSION... [--up \| --down] [--dry-run] [--write-sql PATH]` | Runs exactly those revisions |
|
|
354
|
+
| `orm:migrations:generate [--message M]` | Writes an empty revision |
|
|
355
|
+
| `orm:migrations:diff [--message M] [--allow-empty-diff] [--from-empty-schema]` | Writes what the models add; the database must be up to date |
|
|
356
|
+
| `orm:migrations:dump-schema [--message M] [--filter-tables RE]...` | Writes a first revision from an existing database; `rollup` then records it |
|
|
357
|
+
| `orm:migrations:version [VERSION] (--add \| --delete) [--all]` | Records revisions as applied or not, running nothing |
|
|
358
|
+
| `orm:migrations:rollup` | Records the single remaining revision as all there is |
|
|
359
|
+
|
|
360
|
+
`--write-sql` writes to the file named, or to a dated file in the directory named.
|
|
361
|
+
|
|
362
|
+
Without a container, give the commands their connections once:
|
|
363
|
+
|
|
364
|
+
```python
|
|
365
|
+
from xtr_orm import ConnectionRegistry, DatabaseManager
|
|
366
|
+
from xtr_orm.command import use_connections
|
|
367
|
+
|
|
368
|
+
registry = ConnectionRegistry()
|
|
369
|
+
registry.register("default", engine=engine, migrator=migrator, database=DatabaseManager(url))
|
|
370
|
+
use_connections(registry)
|
|
371
|
+
```
|
|
372
|
+
|
|
373
|
+
## Message bus middleware
|
|
374
|
+
|
|
375
|
+
With the `messenger` extra and a kernel running both bundles, a message handler asks for a
|
|
376
|
+
session the way any service does, and three middleware — registered by name — decide what
|
|
377
|
+
happens to it. Nothing is added to a bus by itself: the bus lists the ones it wants.
|
|
378
|
+
|
|
379
|
+
```python
|
|
380
|
+
# app/billing/handlers.py
|
|
381
|
+
@as_message_handler(IssueInvoice)
|
|
382
|
+
async def issue(message: IssueInvoice, session: Injected[AsyncSession]) -> None:
|
|
383
|
+
session.add(Invoice(order_id=message.order_id)) # no commit: orm_transaction does it
|
|
384
|
+
|
|
385
|
+
|
|
386
|
+
@as_message_handler(IssueInvoice)
|
|
387
|
+
class NotifyAccounting:
|
|
388
|
+
# A scoped repository on the same session: one transaction for both handlers.
|
|
389
|
+
async def __call__(
|
|
390
|
+
self, message: IssueInvoice, invoices: Injected[InvoiceRepository]
|
|
391
|
+
) -> None: ...
|
|
392
|
+
```
|
|
393
|
+
|
|
394
|
+
```python
|
|
395
|
+
# app/config/messenger.py
|
|
396
|
+
MessageBusConfig(
|
|
397
|
+
middleware=[
|
|
398
|
+
"orm_close_connection",
|
|
399
|
+
"orm_transaction",
|
|
400
|
+
{"orm_transaction": {"connection_name": "reports"}}, # another connection too
|
|
401
|
+
"orm_open_transaction_logger",
|
|
402
|
+
],
|
|
403
|
+
)
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
| Name | Class | Does | Arguments |
|
|
407
|
+
|---|---|---|---|
|
|
408
|
+
| `orm_transaction` | `TransactionMiddleware` | Runs every handler of a message in one transaction, committed once they all succeeded and rolled back otherwise; a failure drops the other handlers' handled stamps, so a retry runs them all again | `connection_name`, the default connection when left out |
|
|
409
|
+
| `orm_close_connection` | `CloseConnectionMiddleware` | Closes the connections once a worker handled a message it consumed, so an idle worker holds none — once the last, when it handles several at once | `connection_names`, every connection when left out |
|
|
410
|
+
| `orm_open_transaction_logger` | `OpenTransactionLoggerMiddleware` | Logs an error — to the application's logger when the logging bundle is active — when a handler began a transaction — `begin()`, `begin_nested()` — and left it open | `connection_names`, every connection when left out |
|
|
411
|
+
|
|
412
|
+
- **One session per message.** The messenger bundle makes every message a unit of work, and a
|
|
413
|
+
scoped service — the session, a repository built on it — is the unit's: every handler of
|
|
414
|
+
the message gets the same one, released once the message is done with. A message
|
|
415
|
+
dispatched while another is handled joins its unit, and its transaction — in a savepoint of
|
|
416
|
+
its own, so when it fails only its handlers' work is rolled back, and the handler that
|
|
417
|
+
dispatched it decides whether the rest goes on.
|
|
418
|
+
- **Handlers do not commit** under `orm_transaction`; a handler that does ends the
|
|
419
|
+
transaction early, and one using `async with session.begin()` is refused, the transaction
|
|
420
|
+
being open already.
|
|
421
|
+
- **Order matters.** `orm_close_connection` outermost, so it closes after the transaction
|
|
422
|
+
ended; `orm_open_transaction_logger` inside `orm_transaction`, so it looks before the
|
|
423
|
+
commit closes everything.
|
|
424
|
+
- **Only messages a worker consumed are closed after** — one carrying a `ReceivedStamp` on
|
|
425
|
+
arrival. A message handled during its dispatch, `sync://` included, closes nothing. A worker
|
|
426
|
+
handling several at once closes once the last of them is done, keeping its pool meanwhile.
|
|
427
|
+
- **Follow-ups wait for the commit when stamped.** A message a handler dispatches to a
|
|
428
|
+
transport is sent at once — before the commit, and whether or not it comes. Dispatch it
|
|
429
|
+
with `DispatchAfterCurrentBusStamp` and it goes out only once the message being handled
|
|
430
|
+
was committed, and never if it was rolled back:
|
|
431
|
+
|
|
432
|
+
```python
|
|
433
|
+
await bus.dispatch(SendReceipt(order.id), DispatchAfterCurrentBusStamp())
|
|
434
|
+
```
|
|
435
|
+
- **A dropped connection** is found by the engine option `pool_pre_ping` as a connection is
|
|
436
|
+
taken from the pool — `?pool_pre_ping=true` on a worker's URL — so there is no middleware
|
|
437
|
+
for that.
|
|
438
|
+
|
|
439
|
+
Without a container, build them with a `ConnectionRegistry` whose connections are registered
|
|
440
|
+
with a `session` — what returns the session the handlers use — and put the instances in the
|
|
441
|
+
configuration.
|
|
442
|
+
|
|
443
|
+
## Creating and dropping databases
|
|
444
|
+
|
|
445
|
+
`DatabaseManager(url)` creates and drops the database a URL names. A database cannot be
|
|
446
|
+
created from a connection to itself, so the server is reached through its maintenance
|
|
447
|
+
database — `postgres` on PostgreSQL, `master` on SQL Server, none on MySQL and MariaDB — with
|
|
448
|
+
every statement committed as it runs, and with every connection parameter of the URL but its
|
|
449
|
+
database: the credentials, the host and the driver's query options, such as `ssl`. A SQLite
|
|
450
|
+
database is its file. Any other server is refused with `UnsupportedDatabaseError`.
|
|
451
|
+
|
|
452
|
+
## Errors
|
|
453
|
+
|
|
454
|
+
Every error derives from `OrmError`.
|
|
455
|
+
|
|
456
|
+
| Error | Raised when |
|
|
457
|
+
|---|---|
|
|
458
|
+
| `InvalidArgumentError` | A configuration cannot be used — an empty name, an unknown option, a URL option that does not read (also a `ValueError`) |
|
|
459
|
+
| `MigrationError` | A migration is refused or its revisions cannot be read — an unknown or ambiguous version, a revision out of order, a diff of a database behind its revisions |
|
|
460
|
+
| `UnknownConnectionError` | A connection is asked for by a name nothing registered (also a `LookupError`) |
|
|
461
|
+
| `UnsupportedDatabaseError` | A database is created or dropped on a server whose statements are not known |
|
|
462
|
+
| `SessionUnavailableError` | A connection's session is asked for outside a unit of work, or of a connection registered without one |
|
|
463
|
+
|
|
464
|
+
## Known limitations
|
|
465
|
+
|
|
466
|
+
- **advanced-alchemy's model registry is process-wide.** Its `metadata_registry` maps a
|
|
467
|
+
`bind_key` to table definitions for the whole process, so two kernels in one process share
|
|
468
|
+
it. Give a connection its `metadata` explicitly to keep a kernel's diff to its own models.
|
|
469
|
+
- **The version table holds only the current revisions**, so a revision is run or recorded
|
|
470
|
+
alone only where the database stays consistent — see [Migrations](#migrations). The history
|
|
471
|
+
table records every one.
|
|
472
|
+
- **Async only**: engines and sessions are async; there is no sync counterpart.
|
|
473
|
+
|
|
474
|
+
## Development
|
|
475
|
+
|
|
476
|
+
Developed in the [python-xtr](https://github.com/xterr/python-xtr) monorepo, under
|
|
477
|
+
`packages/xtr-orm`; run the commands below from there. Every test runs on SQLite files in the
|
|
478
|
+
test's own directory; no server is needed.
|
|
479
|
+
|
|
480
|
+
```sh
|
|
481
|
+
uv sync
|
|
482
|
+
uv run ruff check && uv run ruff format --check && uv run basedpyright && uv run ty check && uv run pytest
|
|
483
|
+
```
|
|
484
|
+
|
|
485
|
+
## License
|
|
486
|
+
|
|
487
|
+
MIT — see [LICENSE](LICENSE).
|