asyncpg-typed 0.1.1__tar.gz → 0.1.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- asyncpg_typed-0.1.3/PKG-INFO +293 -0
- asyncpg_typed-0.1.3/README.md +255 -0
- asyncpg_typed-0.1.3/asyncpg_typed/__init__.py +964 -0
- asyncpg_typed-0.1.3/asyncpg_typed.egg-info/PKG-INFO +293 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/asyncpg_typed.egg-info/SOURCES.txt +1 -0
- asyncpg_typed-0.1.3/tests/test_code.py +279 -0
- asyncpg_typed-0.1.3/tests/test_data.py +717 -0
- asyncpg_typed-0.1.3/tests/test_sql.py +229 -0
- asyncpg_typed-0.1.1/PKG-INFO +0 -190
- asyncpg_typed-0.1.1/README.md +0 -152
- asyncpg_typed-0.1.1/asyncpg_typed/__init__.py +0 -877
- asyncpg_typed-0.1.1/asyncpg_typed.egg-info/PKG-INFO +0 -190
- asyncpg_typed-0.1.1/tests/test_code.py +0 -186
- asyncpg_typed-0.1.1/tests/test_data.py +0 -427
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/LICENSE +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/MANIFEST.in +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/asyncpg_typed/py.typed +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/asyncpg_typed.egg-info/dependency_links.txt +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/asyncpg_typed.egg-info/requires.txt +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/asyncpg_typed.egg-info/top_level.txt +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/asyncpg_typed.egg-info/zip-safe +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/pyproject.toml +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/setup.cfg +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/tests/__init__.py +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/tests/connection.py +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/tests/test_template.py +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/tests/test_type.py +0 -0
- {asyncpg_typed-0.1.1 → asyncpg_typed-0.1.3}/tests/test_vector.py +0 -0
|
@@ -0,0 +1,293 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: asyncpg_typed
|
|
3
|
+
Version: 0.1.3
|
|
4
|
+
Summary: Type-safe queries for asyncpg
|
|
5
|
+
Author-email: Levente Hunyadi <hunyadi@gmail.com>
|
|
6
|
+
Maintainer-email: Levente Hunyadi <hunyadi@gmail.com>
|
|
7
|
+
License-Expression: MIT
|
|
8
|
+
Project-URL: Homepage, https://github.com/hunyadi/asyncpg_typed
|
|
9
|
+
Project-URL: Source, https://github.com/hunyadi/asyncpg_typed
|
|
10
|
+
Keywords: asyncpg,typed,database-client,postgres
|
|
11
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Operating System :: OS Independent
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
20
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
21
|
+
Classifier: Programming Language :: SQL
|
|
22
|
+
Classifier: Topic :: Database
|
|
23
|
+
Classifier: Topic :: Utilities
|
|
24
|
+
Classifier: Typing :: Typed
|
|
25
|
+
Requires-Python: >=3.10
|
|
26
|
+
Description-Content-Type: text/markdown
|
|
27
|
+
License-File: LICENSE
|
|
28
|
+
Requires-Dist: asyncpg>=0.31
|
|
29
|
+
Requires-Dist: typing-extensions>=4.15; python_version < "3.11"
|
|
30
|
+
Provides-Extra: vector
|
|
31
|
+
Requires-Dist: asyncpg-vector>=0.1; extra == "vector"
|
|
32
|
+
Provides-Extra: dev
|
|
33
|
+
Requires-Dist: asyncpg-stubs>=0.31; extra == "dev"
|
|
34
|
+
Requires-Dist: build>=1.3; extra == "dev"
|
|
35
|
+
Requires-Dist: mypy>=1.19; extra == "dev"
|
|
36
|
+
Requires-Dist: ruff>=0.14; extra == "dev"
|
|
37
|
+
Dynamic: license-file
|
|
38
|
+
|
|
39
|
+
# Type-safe queries for asyncpg
|
|
40
|
+
|
|
41
|
+
[asyncpg](https://magicstack.github.io/asyncpg/current/) is a high-performance database client to connect to a PostgreSQL server, and execute SQL statements using the async/await paradigm in Python. The library exposes a `Connection` object, which has methods like `execute` and `fetch` that run SQL queries against the database. Unfortunately, these methods take the query as a plain `str`, arguments as `object`, and the resultset is exposed as a `Record`, which is a `tuple`/`dict` hybrid whose `get` and indexer have a return type of `Any`. There is no mechanism to check compatibility of input or output arguments, even if their types are preliminarily known.
|
|
42
|
+
|
|
43
|
+
This Python library provides "compile-time" validation for SQL queries that linters and type checkers can enforce. By creating a generic `SQL` object and associating input and output type information with the query, the signatures of `execute` and `fetch` reveal the exact expected and returned types.
|
|
44
|
+
|
|
45
|
+
|
|
46
|
+
## Motivating example
|
|
47
|
+
|
|
48
|
+
```python
|
|
49
|
+
# create a typed object, setting expected and returned types
|
|
50
|
+
select_where_sql = sql(
|
|
51
|
+
"""--sql
|
|
52
|
+
SELECT boolean_value, integer_value, string_value
|
|
53
|
+
FROM sample_data
|
|
54
|
+
WHERE boolean_value = $1 AND integer_value > $2
|
|
55
|
+
ORDER BY integer_value;
|
|
56
|
+
""",
|
|
57
|
+
args=tuple[bool, int],
|
|
58
|
+
resultset=tuple[bool, int, str | None],
|
|
59
|
+
)
|
|
60
|
+
|
|
61
|
+
conn = await asyncpg.connect(host="localhost", port=5432, user="postgres", password="postgres")
|
|
62
|
+
try:
|
|
63
|
+
# ✅ Valid signature
|
|
64
|
+
rows = await select_where_sql.fetch(conn, False, 2)
|
|
65
|
+
|
|
66
|
+
# ✅ Type of "rows" is "list[tuple[bool, int, str | None]]"
|
|
67
|
+
reveal_type(rows)
|
|
68
|
+
|
|
69
|
+
# ⚠️ Argument missing for parameter "arg2"
|
|
70
|
+
rows = await select_where_sql.fetch(conn, False)
|
|
71
|
+
|
|
72
|
+
# ⚠️ Argument of type "float" cannot be assigned to parameter "arg2" of type "int" in function "fetch"; "float" is not assignable to "int"
|
|
73
|
+
rows = await select_where_sql.fetch(conn, False, 3.14)
|
|
74
|
+
|
|
75
|
+
finally:
|
|
76
|
+
await conn.close()
|
|
77
|
+
|
|
78
|
+
# create a list of data-class instances from a list of typed tuples
|
|
79
|
+
@dataclass
|
|
80
|
+
class DataObject:
|
|
81
|
+
boolean_value: bool
|
|
82
|
+
integer_value: int
|
|
83
|
+
string_value: str | None
|
|
84
|
+
|
|
85
|
+
# ✅ Valid initializer call
|
|
86
|
+
items = [DataObject(*row) for row in rows]
|
|
87
|
+
|
|
88
|
+
@dataclass
|
|
89
|
+
class MismatchedObject:
|
|
90
|
+
boolean_value: bool
|
|
91
|
+
integer_value: int
|
|
92
|
+
string_value: str
|
|
93
|
+
|
|
94
|
+
# ⚠️ Argument of type "int | None" cannot be assigned to parameter "integer_value" of type "int" in function "__init__"; "None" is not assignable to "int"
|
|
95
|
+
items = [MismatchedObject(*row) for row in rows]
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
|
|
99
|
+
## Syntax
|
|
100
|
+
|
|
101
|
+
### Creating a SQL object
|
|
102
|
+
|
|
103
|
+
Instantiate a SQL object with the `sql` function:
|
|
104
|
+
|
|
105
|
+
```python
|
|
106
|
+
def sql(
|
|
107
|
+
stmt: LiteralString | string.templatelib.Template,
|
|
108
|
+
*,
|
|
109
|
+
args: None | type[tuple[P1, P2]] | type[tuple[P1, P2, P3]] | ... = None,
|
|
110
|
+
resultset: None | type[tuple[R1, R2]] | type[tuple[R1, R2, R3]] | ... = None,
|
|
111
|
+
arg: None | type[P] = None,
|
|
112
|
+
result: None | type[R] = None,
|
|
113
|
+
) -> _SQL: ...
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
#### Parameters to factory function
|
|
117
|
+
|
|
118
|
+
The parameter `stmt` represents a SQL expression, either as a literal string or a template (i.e. a *t-string*).
|
|
119
|
+
|
|
120
|
+
If the expression is a string, it can have PostgreSQL parameter placeholders such as `$1`, `$2` or `$3`:
|
|
121
|
+
|
|
122
|
+
```python
|
|
123
|
+
"INSERT INTO table_name (col_1, col_2, col_3) VALUES ($1, $2, $3);"
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
If the expression is a *t-string*, it can have replacement fields that evaluate to integers:
|
|
127
|
+
|
|
128
|
+
```python
|
|
129
|
+
t"INSERT INTO table_name (col_1, col_2, col_3) VALUES ({1}, {2}, {3});"
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
The parameters `args` and `resultset` take a `tuple` of several types `Px` or `Rx`.
|
|
133
|
+
|
|
134
|
+
The parameters `arg` and `result` take a single type `P` or `R`. Passing a simple type (e.g. `type[T]`) directly via `arg` and `result` is for convenience, and is equivalent to passing a one-element tuple of the same simple type (i.e. `type[tuple[T]]`) via `args` and `resultset`.
|
|
135
|
+
|
|
136
|
+
The number of types in `args` must correspond to the number of query parameters. (This is validated on calling `sql(...)` for the *t-string* syntax.) The number of types in `resultset` must correspond to the number of columns returned by the query.
|
|
137
|
+
|
|
138
|
+
#### Argument and resultset types
|
|
139
|
+
|
|
140
|
+
When passing Python types via the parameters `args` and `resultset`, each type may be any of the following:
|
|
141
|
+
|
|
142
|
+
* (required) simple type
|
|
143
|
+
* optional simple type (`T | None`)
|
|
144
|
+
* special union type
|
|
145
|
+
|
|
146
|
+
Simple types include:
|
|
147
|
+
|
|
148
|
+
* `bool`
|
|
149
|
+
* numeric types:
|
|
150
|
+
* `int`
|
|
151
|
+
* `float`
|
|
152
|
+
* `decimal.Decimal`
|
|
153
|
+
* date and time types:
|
|
154
|
+
* `datetime.date`
|
|
155
|
+
* `datetime.time`
|
|
156
|
+
* `datetime.datetime`
|
|
157
|
+
* `datetime.timedelta`
|
|
158
|
+
* `str`
|
|
159
|
+
* `bytes`
|
|
160
|
+
* `uuid.UUID`
|
|
161
|
+
* types defined in the module [ipaddress](https://docs.python.org/3/library/ipaddress.html):
|
|
162
|
+
* `ipaddress.IPv4Address`
|
|
163
|
+
* `ipaddress.IPv6Address`
|
|
164
|
+
* `ipaddress.IPv4Network`
|
|
165
|
+
* `ipaddress.IPv6Network`
|
|
166
|
+
* [asyncpg representations](https://magicstack.github.io/asyncpg/current/api/index.html#module-asyncpg.types) of PostgreSQL geometric types:
|
|
167
|
+
* `asyncpg.Point`
|
|
168
|
+
* `asyncpg.Line`
|
|
169
|
+
* `asyncpg.LineSegment`
|
|
170
|
+
* `asyncpg.Box`
|
|
171
|
+
* `asyncpg.Path`
|
|
172
|
+
* `asyncpg.Polygon`
|
|
173
|
+
* `asyncpg.Circle`
|
|
174
|
+
* concrete types of [asyncpg.Range](https://magicstack.github.io/asyncpg/current/api/index.html#asyncpg.types.Range):
|
|
175
|
+
* `asyncpg.Range[int]`
|
|
176
|
+
* `asyncpg.Range[Decimal]`
|
|
177
|
+
* `asyncpg.Range[date]`
|
|
178
|
+
* `asyncpg.Range[datetime]`
|
|
179
|
+
* a user-defined enumeration class that derives from `StrEnum`
|
|
180
|
+
|
|
181
|
+
Custom Python types corresponding to PostgreSQL scalar or [composite types](https://www.postgresql.org/docs/current/rowtypes.html) are permitted. These types need to be pre-registered with [set_type_codec](https://magicstack.github.io/asyncpg/current/api/index.html#asyncpg.connection.Connection.set_type_codec) passing an encoder, a decoder and typically `format="tuple"`.
|
|
182
|
+
|
|
183
|
+
In general, union types are not allowed. However, there are notable exceptions. Special union types are as follows:
|
|
184
|
+
|
|
185
|
+
* `JsonType` to represent an object reconstructed from a JSON string
|
|
186
|
+
* `IPv4Address | IPv6Address` to denote either an IPv4 or IPv6 address
|
|
187
|
+
* `IPv4Network | IPv6Network` to denote either an IPv4 or IPv6 network definition
|
|
188
|
+
|
|
189
|
+
Types are grouped together with `tuple`:
|
|
190
|
+
|
|
191
|
+
```python
|
|
192
|
+
tuple[bool, int, str | None]
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Both `args` and `resultset` types must be compatible with their corresponding PostgreSQL query parameter types and resultset column types, respectively. The following table shows the mapping between PostgreSQL and Python types. When there are multiple options separated by a slash, either of the types can be specified as a source or target type.
|
|
196
|
+
|
|
197
|
+
| PostgreSQL type | Python type |
|
|
198
|
+
| ---------------------------- | ---------------------------------- |
|
|
199
|
+
| `bool` | `bool` |
|
|
200
|
+
| `smallint` | `int` |
|
|
201
|
+
| `integer` | `int` |
|
|
202
|
+
| `bigint` | `int` |
|
|
203
|
+
| `real`/`float4` | `float` |
|
|
204
|
+
| `double`/`float8` | `float` |
|
|
205
|
+
| `decimal`/`numeric` | `Decimal` |
|
|
206
|
+
| `date` | `date` |
|
|
207
|
+
| `time` | `time` (naive) |
|
|
208
|
+
| `timetz` | `time` (tz) |
|
|
209
|
+
| `timestamp` | `datetime` (naive) |
|
|
210
|
+
| `timestamptz` | `datetime` (tz) |
|
|
211
|
+
| `interval` | `timedelta` |
|
|
212
|
+
| `char(N)` | `str` |
|
|
213
|
+
| `varchar(N)` | `str` |
|
|
214
|
+
| `text` | `str` |
|
|
215
|
+
| `bytea` | `bytes` |
|
|
216
|
+
| `uuid` | `UUID` |
|
|
217
|
+
| `cidr` | `IPvXNetwork` |
|
|
218
|
+
| `inet` | `IPvXNetwork`/`IPvXAddress` |
|
|
219
|
+
| `macaddr` | `str` |
|
|
220
|
+
| `macaddr8` | `str` |
|
|
221
|
+
| `json` | `str`/`JsonType` |
|
|
222
|
+
| `jsonb` | `str`/`JsonType` |
|
|
223
|
+
| `xml` | `str` |
|
|
224
|
+
| any enumeration type | `E: StrEnum` |
|
|
225
|
+
| `point` | `asyncpg.Point` |
|
|
226
|
+
| `line` | `asyncpg.Line` |
|
|
227
|
+
| `lseg` | `asyncpg.LineSegment` |
|
|
228
|
+
| `box` | `asyncpg.Box` |
|
|
229
|
+
| `path` | `asyncpg.Path` |
|
|
230
|
+
| `polygon` | `asyncpg.Polygon` |
|
|
231
|
+
| `circle` | `asyncpg.Circle` |
|
|
232
|
+
| `int4range` | `asyncpg.Range[int]` |
|
|
233
|
+
| `int8range` | `asyncpg.Range[int]` |
|
|
234
|
+
| `numrange` | `asyncpg.Range[Decimal]` |
|
|
235
|
+
| `tsrange` | `asyncpg.Range[datetime]` (naive) |
|
|
236
|
+
| `tstzrange` | `asyncpg.Range[datetime]` (tz) |
|
|
237
|
+
| `daterange` | `asyncpg.Range[date]` |
|
|
238
|
+
|
|
239
|
+
|
|
240
|
+
PostgreSQL types `json` and `jsonb` are [returned by asyncpg](https://magicstack.github.io/asyncpg/current/usage.html#type-conversion) as Python type `str`. However, if we specify the union type `JsonType` in `args` or `resultset`, the JSON string is parsed as if by calling `json.loads()`. If the library `orjson` is present, its faster routines are invoked instead of the slower standard library implementation in the module `json`.
|
|
241
|
+
|
|
242
|
+
`JsonType` is defined in the module `asyncpg_typed` as follows:
|
|
243
|
+
|
|
244
|
+
```python
|
|
245
|
+
JsonType = None | bool | int | float | str | dict[str, "JsonType"] | list["JsonType"]
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
`IPvXNetwork` is a shorthand for either of the following:
|
|
249
|
+
|
|
250
|
+
* `IPv4Network`
|
|
251
|
+
* `IPv6Network`
|
|
252
|
+
* their union type `IPv4Network | IPv6Network`
|
|
253
|
+
|
|
254
|
+
`IPvXAddress` stands for either of the following:
|
|
255
|
+
|
|
256
|
+
* `IPv4Address`
|
|
257
|
+
* `IPv6Address`
|
|
258
|
+
* their union type `IPv4Address | IPv6Address`
|
|
259
|
+
|
|
260
|
+
#### SQL statement as an f-string
|
|
261
|
+
|
|
262
|
+
In addition to the `sql` function, SQL objects can be created with the functionally identical `unsafe_sql` function. As opposed to its safer alternative, the first parameter of `unsafe_sql` has the plain type `str`, allowing us to pass an f-string. This can prove useful if we want to inject the value of a Python variable at location where binding parameters are not permitted by PostgreSQL syntax, e.g. substitute the name of a database table to dynamically create a SQL statement.
|
|
263
|
+
|
|
264
|
+
### Using a SQL object
|
|
265
|
+
|
|
266
|
+
The function `sql` returns an object that derives from the base class `_SQL` and is specific to the number and types of parameters passed in `args` and `resultset`.
|
|
267
|
+
|
|
268
|
+
The following functions are available on SQL objects:
|
|
269
|
+
|
|
270
|
+
```python
|
|
271
|
+
async def execute(self, connection: Connection, *args: *P) -> None: ...
|
|
272
|
+
async def executemany(self, connection: Connection, args: Iterable[tuple[*P]]) -> None: ...
|
|
273
|
+
async def fetch(self, connection: Connection, *args: *P) -> list[tuple[*R]]: ...
|
|
274
|
+
async def fetchmany(self, connection: Connection, args: Iterable[tuple[*P]]) -> list[tuple[*R]]: ...
|
|
275
|
+
async def fetchrow(self, connection: Connection, *args: *P) -> tuple[*R] | None: ...
|
|
276
|
+
async def fetchval(self, connection: Connection, *args: *P) -> R1: ...
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
`Connection` may be an `asyncpg.Connection` or an `asyncpg.pool.PoolConnectionProxy` acquired from a connection pool.
|
|
280
|
+
|
|
281
|
+
`*P` and `*R` denote several types (a type pack) corresponding to those listed in `args` and `resultset`, respectively.
|
|
282
|
+
|
|
283
|
+
Only those functions are prompted on code completion that make sense in the context of the given number of input and output arguments. Specifically, `fetchval` is available only for a single type passed to `resultset`, and `executemany` and `fetchmany` are available only if the query takes (one or more) parameters.
|
|
284
|
+
|
|
285
|
+
#### Run-time behavior
|
|
286
|
+
|
|
287
|
+
When a call such as `sql.executemany(conn, records)` or `sql.fetch(conn, param1, param2)` is made on a `SQL` object at run time, the library invokes `connection.prepare(sql)` to create a `PreparedStatement` and compares the actual statement signature against the expected Python types. If the expected and actual signatures don't match, an exception `TypeMismatchError` (subclass of `TypeError`) is raised.
|
|
288
|
+
|
|
289
|
+
The set of values for an enumeration type is validated when a prepared statement is created. The string values declared in a Python `StrEnum` are compared against the values listed in PostgreSQL `CREATE TYPE ... AS ENUM` by querying the system table `pg_enum`. If there are missing or extra values on either side, an exception `EnumMismatchError` (subclass of `TypeError`) is raised.
|
|
290
|
+
|
|
291
|
+
Unfortunately, PostgreSQL doesn't propagate nullability via prepared statements: resultset types that are declared as required (e.g. `T` as opposed to `T | None`) are validated at run time. When a `None` value is encountered for a required type, an exception `NoneTypeError` (subclass of `TypeError`) is raised.
|
|
292
|
+
|
|
293
|
+
PostgreSQL doesn't differentiate between IPv4 and IPv6 network definitions, or IPv4 and IPv6 addresses in the types `cidr` and `inet`. This means that semantically a union type is returned. If you specify a more restrictive type, the resultset data is validated dynamically at run time.
|
|
@@ -0,0 +1,255 @@
|
|
|
1
|
+
# Type-safe queries for asyncpg
|
|
2
|
+
|
|
3
|
+
[asyncpg](https://magicstack.github.io/asyncpg/current/) is a high-performance database client to connect to a PostgreSQL server, and execute SQL statements using the async/await paradigm in Python. The library exposes a `Connection` object, which has methods like `execute` and `fetch` that run SQL queries against the database. Unfortunately, these methods take the query as a plain `str`, arguments as `object`, and the resultset is exposed as a `Record`, which is a `tuple`/`dict` hybrid whose `get` and indexer have a return type of `Any`. There is no mechanism to check compatibility of input or output arguments, even if their types are preliminarily known.
|
|
4
|
+
|
|
5
|
+
This Python library provides "compile-time" validation for SQL queries that linters and type checkers can enforce. By creating a generic `SQL` object and associating input and output type information with the query, the signatures of `execute` and `fetch` reveal the exact expected and returned types.
|
|
6
|
+
|
|
7
|
+
|
|
8
|
+
## Motivating example
|
|
9
|
+
|
|
10
|
+
```python
|
|
11
|
+
# create a typed object, setting expected and returned types
|
|
12
|
+
select_where_sql = sql(
|
|
13
|
+
"""--sql
|
|
14
|
+
SELECT boolean_value, integer_value, string_value
|
|
15
|
+
FROM sample_data
|
|
16
|
+
WHERE boolean_value = $1 AND integer_value > $2
|
|
17
|
+
ORDER BY integer_value;
|
|
18
|
+
""",
|
|
19
|
+
args=tuple[bool, int],
|
|
20
|
+
resultset=tuple[bool, int, str | None],
|
|
21
|
+
)
|
|
22
|
+
|
|
23
|
+
conn = await asyncpg.connect(host="localhost", port=5432, user="postgres", password="postgres")
|
|
24
|
+
try:
|
|
25
|
+
# ✅ Valid signature
|
|
26
|
+
rows = await select_where_sql.fetch(conn, False, 2)
|
|
27
|
+
|
|
28
|
+
# ✅ Type of "rows" is "list[tuple[bool, int, str | None]]"
|
|
29
|
+
reveal_type(rows)
|
|
30
|
+
|
|
31
|
+
# ⚠️ Argument missing for parameter "arg2"
|
|
32
|
+
rows = await select_where_sql.fetch(conn, False)
|
|
33
|
+
|
|
34
|
+
# ⚠️ Argument of type "float" cannot be assigned to parameter "arg2" of type "int" in function "fetch"; "float" is not assignable to "int"
|
|
35
|
+
rows = await select_where_sql.fetch(conn, False, 3.14)
|
|
36
|
+
|
|
37
|
+
finally:
|
|
38
|
+
await conn.close()
|
|
39
|
+
|
|
40
|
+
# create a list of data-class instances from a list of typed tuples
|
|
41
|
+
@dataclass
|
|
42
|
+
class DataObject:
|
|
43
|
+
boolean_value: bool
|
|
44
|
+
integer_value: int
|
|
45
|
+
string_value: str | None
|
|
46
|
+
|
|
47
|
+
# ✅ Valid initializer call
|
|
48
|
+
items = [DataObject(*row) for row in rows]
|
|
49
|
+
|
|
50
|
+
@dataclass
|
|
51
|
+
class MismatchedObject:
|
|
52
|
+
boolean_value: bool
|
|
53
|
+
integer_value: int
|
|
54
|
+
string_value: str
|
|
55
|
+
|
|
56
|
+
# ⚠️ Argument of type "int | None" cannot be assigned to parameter "integer_value" of type "int" in function "__init__"; "None" is not assignable to "int"
|
|
57
|
+
items = [MismatchedObject(*row) for row in rows]
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
## Syntax
|
|
62
|
+
|
|
63
|
+
### Creating a SQL object
|
|
64
|
+
|
|
65
|
+
Instantiate a SQL object with the `sql` function:
|
|
66
|
+
|
|
67
|
+
```python
|
|
68
|
+
def sql(
|
|
69
|
+
stmt: LiteralString | string.templatelib.Template,
|
|
70
|
+
*,
|
|
71
|
+
args: None | type[tuple[P1, P2]] | type[tuple[P1, P2, P3]] | ... = None,
|
|
72
|
+
resultset: None | type[tuple[R1, R2]] | type[tuple[R1, R2, R3]] | ... = None,
|
|
73
|
+
arg: None | type[P] = None,
|
|
74
|
+
result: None | type[R] = None,
|
|
75
|
+
) -> _SQL: ...
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
#### Parameters to factory function
|
|
79
|
+
|
|
80
|
+
The parameter `stmt` represents a SQL expression, either as a literal string or a template (i.e. a *t-string*).
|
|
81
|
+
|
|
82
|
+
If the expression is a string, it can have PostgreSQL parameter placeholders such as `$1`, `$2` or `$3`:
|
|
83
|
+
|
|
84
|
+
```python
|
|
85
|
+
"INSERT INTO table_name (col_1, col_2, col_3) VALUES ($1, $2, $3);"
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
If the expression is a *t-string*, it can have replacement fields that evaluate to integers:
|
|
89
|
+
|
|
90
|
+
```python
|
|
91
|
+
t"INSERT INTO table_name (col_1, col_2, col_3) VALUES ({1}, {2}, {3});"
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
The parameters `args` and `resultset` take a `tuple` of several types `Px` or `Rx`.
|
|
95
|
+
|
|
96
|
+
The parameters `arg` and `result` take a single type `P` or `R`. Passing a simple type (e.g. `type[T]`) directly via `arg` and `result` is for convenience, and is equivalent to passing a one-element tuple of the same simple type (i.e. `type[tuple[T]]`) via `args` and `resultset`.
|
|
97
|
+
|
|
98
|
+
The number of types in `args` must correspond to the number of query parameters. (This is validated on calling `sql(...)` for the *t-string* syntax.) The number of types in `resultset` must correspond to the number of columns returned by the query.
|
|
99
|
+
|
|
100
|
+
#### Argument and resultset types
|
|
101
|
+
|
|
102
|
+
When passing Python types via the parameters `args` and `resultset`, each type may be any of the following:
|
|
103
|
+
|
|
104
|
+
* (required) simple type
|
|
105
|
+
* optional simple type (`T | None`)
|
|
106
|
+
* special union type
|
|
107
|
+
|
|
108
|
+
Simple types include:
|
|
109
|
+
|
|
110
|
+
* `bool`
|
|
111
|
+
* numeric types:
|
|
112
|
+
* `int`
|
|
113
|
+
* `float`
|
|
114
|
+
* `decimal.Decimal`
|
|
115
|
+
* date and time types:
|
|
116
|
+
* `datetime.date`
|
|
117
|
+
* `datetime.time`
|
|
118
|
+
* `datetime.datetime`
|
|
119
|
+
* `datetime.timedelta`
|
|
120
|
+
* `str`
|
|
121
|
+
* `bytes`
|
|
122
|
+
* `uuid.UUID`
|
|
123
|
+
* types defined in the module [ipaddress](https://docs.python.org/3/library/ipaddress.html):
|
|
124
|
+
* `ipaddress.IPv4Address`
|
|
125
|
+
* `ipaddress.IPv6Address`
|
|
126
|
+
* `ipaddress.IPv4Network`
|
|
127
|
+
* `ipaddress.IPv6Network`
|
|
128
|
+
* [asyncpg representations](https://magicstack.github.io/asyncpg/current/api/index.html#module-asyncpg.types) of PostgreSQL geometric types:
|
|
129
|
+
* `asyncpg.Point`
|
|
130
|
+
* `asyncpg.Line`
|
|
131
|
+
* `asyncpg.LineSegment`
|
|
132
|
+
* `asyncpg.Box`
|
|
133
|
+
* `asyncpg.Path`
|
|
134
|
+
* `asyncpg.Polygon`
|
|
135
|
+
* `asyncpg.Circle`
|
|
136
|
+
* concrete types of [asyncpg.Range](https://magicstack.github.io/asyncpg/current/api/index.html#asyncpg.types.Range):
|
|
137
|
+
* `asyncpg.Range[int]`
|
|
138
|
+
* `asyncpg.Range[Decimal]`
|
|
139
|
+
* `asyncpg.Range[date]`
|
|
140
|
+
* `asyncpg.Range[datetime]`
|
|
141
|
+
* a user-defined enumeration class that derives from `StrEnum`
|
|
142
|
+
|
|
143
|
+
Custom Python types corresponding to PostgreSQL scalar or [composite types](https://www.postgresql.org/docs/current/rowtypes.html) are permitted. These types need to be pre-registered with [set_type_codec](https://magicstack.github.io/asyncpg/current/api/index.html#asyncpg.connection.Connection.set_type_codec) passing an encoder, a decoder and typically `format="tuple"`.
|
|
144
|
+
|
|
145
|
+
In general, union types are not allowed. However, there are notable exceptions. Special union types are as follows:
|
|
146
|
+
|
|
147
|
+
* `JsonType` to represent an object reconstructed from a JSON string
|
|
148
|
+
* `IPv4Address | IPv6Address` to denote either an IPv4 or IPv6 address
|
|
149
|
+
* `IPv4Network | IPv6Network` to denote either an IPv4 or IPv6 network definition
|
|
150
|
+
|
|
151
|
+
Types are grouped together with `tuple`:
|
|
152
|
+
|
|
153
|
+
```python
|
|
154
|
+
tuple[bool, int, str | None]
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
Both `args` and `resultset` types must be compatible with their corresponding PostgreSQL query parameter types and resultset column types, respectively. The following table shows the mapping between PostgreSQL and Python types. When there are multiple options separated by a slash, either of the types can be specified as a source or target type.
|
|
158
|
+
|
|
159
|
+
| PostgreSQL type | Python type |
|
|
160
|
+
| ---------------------------- | ---------------------------------- |
|
|
161
|
+
| `bool` | `bool` |
|
|
162
|
+
| `smallint` | `int` |
|
|
163
|
+
| `integer` | `int` |
|
|
164
|
+
| `bigint` | `int` |
|
|
165
|
+
| `real`/`float4` | `float` |
|
|
166
|
+
| `double`/`float8` | `float` |
|
|
167
|
+
| `decimal`/`numeric` | `Decimal` |
|
|
168
|
+
| `date` | `date` |
|
|
169
|
+
| `time` | `time` (naive) |
|
|
170
|
+
| `timetz` | `time` (tz) |
|
|
171
|
+
| `timestamp` | `datetime` (naive) |
|
|
172
|
+
| `timestamptz` | `datetime` (tz) |
|
|
173
|
+
| `interval` | `timedelta` |
|
|
174
|
+
| `char(N)` | `str` |
|
|
175
|
+
| `varchar(N)` | `str` |
|
|
176
|
+
| `text` | `str` |
|
|
177
|
+
| `bytea` | `bytes` |
|
|
178
|
+
| `uuid` | `UUID` |
|
|
179
|
+
| `cidr` | `IPvXNetwork` |
|
|
180
|
+
| `inet` | `IPvXNetwork`/`IPvXAddress` |
|
|
181
|
+
| `macaddr` | `str` |
|
|
182
|
+
| `macaddr8` | `str` |
|
|
183
|
+
| `json` | `str`/`JsonType` |
|
|
184
|
+
| `jsonb` | `str`/`JsonType` |
|
|
185
|
+
| `xml` | `str` |
|
|
186
|
+
| any enumeration type | `E: StrEnum` |
|
|
187
|
+
| `point` | `asyncpg.Point` |
|
|
188
|
+
| `line` | `asyncpg.Line` |
|
|
189
|
+
| `lseg` | `asyncpg.LineSegment` |
|
|
190
|
+
| `box` | `asyncpg.Box` |
|
|
191
|
+
| `path` | `asyncpg.Path` |
|
|
192
|
+
| `polygon` | `asyncpg.Polygon` |
|
|
193
|
+
| `circle` | `asyncpg.Circle` |
|
|
194
|
+
| `int4range` | `asyncpg.Range[int]` |
|
|
195
|
+
| `int8range` | `asyncpg.Range[int]` |
|
|
196
|
+
| `numrange` | `asyncpg.Range[Decimal]` |
|
|
197
|
+
| `tsrange` | `asyncpg.Range[datetime]` (naive) |
|
|
198
|
+
| `tstzrange` | `asyncpg.Range[datetime]` (tz) |
|
|
199
|
+
| `daterange` | `asyncpg.Range[date]` |
|
|
200
|
+
|
|
201
|
+
|
|
202
|
+
PostgreSQL types `json` and `jsonb` are [returned by asyncpg](https://magicstack.github.io/asyncpg/current/usage.html#type-conversion) as Python type `str`. However, if we specify the union type `JsonType` in `args` or `resultset`, the JSON string is parsed as if by calling `json.loads()`. If the library `orjson` is present, its faster routines are invoked instead of the slower standard library implementation in the module `json`.
|
|
203
|
+
|
|
204
|
+
`JsonType` is defined in the module `asyncpg_typed` as follows:
|
|
205
|
+
|
|
206
|
+
```python
|
|
207
|
+
JsonType = None | bool | int | float | str | dict[str, "JsonType"] | list["JsonType"]
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
`IPvXNetwork` is a shorthand for either of the following:
|
|
211
|
+
|
|
212
|
+
* `IPv4Network`
|
|
213
|
+
* `IPv6Network`
|
|
214
|
+
* their union type `IPv4Network | IPv6Network`
|
|
215
|
+
|
|
216
|
+
`IPvXAddress` stands for either of the following:
|
|
217
|
+
|
|
218
|
+
* `IPv4Address`
|
|
219
|
+
* `IPv6Address`
|
|
220
|
+
* their union type `IPv4Address | IPv6Address`
|
|
221
|
+
|
|
222
|
+
#### SQL statement as an f-string
|
|
223
|
+
|
|
224
|
+
In addition to the `sql` function, SQL objects can be created with the functionally identical `unsafe_sql` function. As opposed to its safer alternative, the first parameter of `unsafe_sql` has the plain type `str`, allowing us to pass an f-string. This can prove useful if we want to inject the value of a Python variable at location where binding parameters are not permitted by PostgreSQL syntax, e.g. substitute the name of a database table to dynamically create a SQL statement.
|
|
225
|
+
|
|
226
|
+
### Using a SQL object
|
|
227
|
+
|
|
228
|
+
The function `sql` returns an object that derives from the base class `_SQL` and is specific to the number and types of parameters passed in `args` and `resultset`.
|
|
229
|
+
|
|
230
|
+
The following functions are available on SQL objects:
|
|
231
|
+
|
|
232
|
+
```python
|
|
233
|
+
async def execute(self, connection: Connection, *args: *P) -> None: ...
|
|
234
|
+
async def executemany(self, connection: Connection, args: Iterable[tuple[*P]]) -> None: ...
|
|
235
|
+
async def fetch(self, connection: Connection, *args: *P) -> list[tuple[*R]]: ...
|
|
236
|
+
async def fetchmany(self, connection: Connection, args: Iterable[tuple[*P]]) -> list[tuple[*R]]: ...
|
|
237
|
+
async def fetchrow(self, connection: Connection, *args: *P) -> tuple[*R] | None: ...
|
|
238
|
+
async def fetchval(self, connection: Connection, *args: *P) -> R1: ...
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
`Connection` may be an `asyncpg.Connection` or an `asyncpg.pool.PoolConnectionProxy` acquired from a connection pool.
|
|
242
|
+
|
|
243
|
+
`*P` and `*R` denote several types (a type pack) corresponding to those listed in `args` and `resultset`, respectively.
|
|
244
|
+
|
|
245
|
+
Only those functions are prompted on code completion that make sense in the context of the given number of input and output arguments. Specifically, `fetchval` is available only for a single type passed to `resultset`, and `executemany` and `fetchmany` are available only if the query takes (one or more) parameters.
|
|
246
|
+
|
|
247
|
+
#### Run-time behavior
|
|
248
|
+
|
|
249
|
+
When a call such as `sql.executemany(conn, records)` or `sql.fetch(conn, param1, param2)` is made on a `SQL` object at run time, the library invokes `connection.prepare(sql)` to create a `PreparedStatement` and compares the actual statement signature against the expected Python types. If the expected and actual signatures don't match, an exception `TypeMismatchError` (subclass of `TypeError`) is raised.
|
|
250
|
+
|
|
251
|
+
The set of values for an enumeration type is validated when a prepared statement is created. The string values declared in a Python `StrEnum` are compared against the values listed in PostgreSQL `CREATE TYPE ... AS ENUM` by querying the system table `pg_enum`. If there are missing or extra values on either side, an exception `EnumMismatchError` (subclass of `TypeError`) is raised.
|
|
252
|
+
|
|
253
|
+
Unfortunately, PostgreSQL doesn't propagate nullability via prepared statements: resultset types that are declared as required (e.g. `T` as opposed to `T | None`) are validated at run time. When a `None` value is encountered for a required type, an exception `NoneTypeError` (subclass of `TypeError`) is raised.
|
|
254
|
+
|
|
255
|
+
PostgreSQL doesn't differentiate between IPv4 and IPv6 network definitions, or IPv4 and IPv6 addresses in the types `cidr` and `inet`. This means that semantically a union type is returned. If you specify a more restrictive type, the resultset data is validated dynamically at run time.
|