yggio-prisma 0.0.0-stage → 0.0.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +233 -2
- package/dist/index.cjs +1349 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.cts +625 -0
- package/dist/index.d.ts +625 -0
- package/dist/index.js +1287 -0
- package/dist/index.js.map +1 -0
- package/package.json +72 -4
- package/src/arrow.ts +344 -0
- package/src/document.ts +228 -0
- package/src/extension.ts +212 -0
- package/src/index.ts +12 -0
- package/src/landing.ts +436 -0
- package/src/mysql.ts +436 -0
- package/src/pushdown.ts +444 -0
package/README.md
CHANGED
|
@@ -1,3 +1,234 @@
|
|
|
1
|
-
#
|
|
1
|
+
# yggio-prisma
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
A Prisma Client extension whose query engine is the yggio plan. A plan's text
|
|
4
|
+
is read by the core's parser; what the database can answer exactly is pushed
|
|
5
|
+
into `findMany` arguments, and the residual runs natively over the rows they
|
|
6
|
+
return, landed once under the model's typed struct field - a decimal at its
|
|
7
|
+
column's scale, a UUID, an instant at its unit. The documentation page is
|
|
8
|
+
[Prisma](https://platob.github.io/yggio/expression/prisma/).
|
|
9
|
+
|
|
10
|
+
| | |
|
|
11
|
+
| --- | --- |
|
|
12
|
+
| Install | `npm install yggio-prisma yggio` - `yggio` is a peer at the exact version, `@prisma/client` (`>=5`) an optional peer: nothing of it is imported, the extension is a plain object |
|
|
13
|
+
| Prisma | `$extends` from Prisma 5; `$use` in Prisma 5 and 6; tested against 7.10.1 |
|
|
14
|
+
| Doors | `yggioExtension(options)` for `prisma.$extends`: `model.plan(text, params?)` on every model, `yggioWhere(text, params?)` under any operation's `where`; `yggioMiddleware(options)`, the marker filter for `$use` |
|
|
15
|
+
| By hand | `compilePlan(plan, model, { yggio, params?, provider?, insensitive?, field? })` -> `{ args, residual, complete, pushed, columns }`; `modelField(model, yggio, columns?, provider?)`; `landRows(model, rows, yggio, columns?)`; `runPlan(landedRows, residual, field, yggio)`; `filterRows(rows, filter, model, columns, yggio)` |
|
|
16
|
+
| Arrow over MySQL | `modelArrowReader(model, rows, yggio, { field?, batchSize? })` - a model's rows as Arrow batches cast onto the crate's field by the core; `writeModel(delegate, source, model, yggio, { mode?, batchSize?, key?, client? })` - an Arrow source through `createMany` or `upsert` in batches; `mysqlDatatype`, `mysqlColumn`, `mysqlModel`, `fieldModel`, `prismaSchema` - the MySQL type map, both ways ([MySQL](#mysql)) |
|
|
17
|
+
| Refusals | `PlanRefused`, an `Error` naming what it refuses |
|
|
18
|
+
|
|
19
|
+
## Install
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
npm install yggio-prisma yggio @prisma/client
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
`yggio` is a peer at the exact version and is never imported: every door
|
|
26
|
+
takes it as an argument, so the application loads the addon once and hands
|
|
27
|
+
it over. `@prisma/client` is an optional peer, needed only for a generated
|
|
28
|
+
client; a MySQL application adds its driver adapter
|
|
29
|
+
(`@prisma/adapter-mariadb`). The package carries `dist/` (ESM and CommonJS,
|
|
30
|
+
declarations) and its sources; `npm pack` builds it first (`prepack`).
|
|
31
|
+
`node/market/examples/market-app` writes its schema with
|
|
32
|
+
`prismaSchema(fieldModels(yggio, kinds), { output })`, generates the client,
|
|
33
|
+
and reads and writes the walked messages through `modelArrowReader` and
|
|
34
|
+
`writeModel` ([Applications](https://platob.github.io/yggio/market/applications/)).
|
|
35
|
+
|
|
36
|
+
## The extension
|
|
37
|
+
|
|
38
|
+
```ts
|
|
39
|
+
import * as yggio from 'yggio'
|
|
40
|
+
import { yggioExtension, yggioWhere } from 'yggio-prisma'
|
|
41
|
+
|
|
42
|
+
const db = prisma.$extends(yggioExtension({ yggio, models: [TRADE], provider: 'postgresql' }))
|
|
43
|
+
|
|
44
|
+
// findMany with what pushes, then the residual natively.
|
|
45
|
+
const totals = await db.trade.plan(
|
|
46
|
+
'select venue, sum(quantity) as total from trade where price > :floor group by venue order by total desc',
|
|
47
|
+
{ floor: 10 },
|
|
48
|
+
)
|
|
49
|
+
|
|
50
|
+
// The marker filter: compiled where the operation runs, its keys beside any Prisma filter of the call.
|
|
51
|
+
const buys = await db.trade.findMany({ where: { ...yggioWhere("side = 'BUY' and venue like 'X%'"), tradable: true } })
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
| Option | Is |
|
|
55
|
+
| --- | --- |
|
|
56
|
+
| `yggio` | the `yggio` module as the caller loaded it |
|
|
57
|
+
| `models` | a list of DMMF models, or `Prisma.dmmf` itself (`{ datamodel: { models, enums } }`), whose `enums` give each enum column its `values`; a model is found by the delegate's `$name`, case aside |
|
|
58
|
+
| `provider` | the datasource provider word - `sqlite`, `postgresql`, `cockroachdb`, `mongodb`, `mysql`, `sqlserver`; text equality pushes exactly under the first four alone; another word is refused |
|
|
59
|
+
| `insensitive` | `true` pushes `ilike` as `mode: 'insensitive'`, only for a database that takes that mode |
|
|
60
|
+
| `marker` | the `where` key `yggioWhere` writes, `$yggio` by default; pass the same name as `yggioWhere`'s third argument |
|
|
61
|
+
|
|
62
|
+
Prisma 7's own `Prisma.dmmf` keeps no `isRequired`, `isList` or `nativeType`,
|
|
63
|
+
and no enum's members: hand each model with them, as the schema states them,
|
|
64
|
+
and each enum column its `values` (`Object.values(Side)`). A model without
|
|
65
|
+
`isRequired` lands every column nullable, and no null test and no order push;
|
|
66
|
+
an enum column without `values` pushes no equality and no `in`.
|
|
67
|
+
|
|
68
|
+
`plan` answers the rows as Prisma answered them where nothing remains or the
|
|
69
|
+
residual is a filter alone, else one object per native output row keyed by the
|
|
70
|
+
plan's output columns. The pushed `where` sits beside the caller's at the top
|
|
71
|
+
level, a key both state joined by `AND`, so a unique key stays where
|
|
72
|
+
`findUnique`, `update`, `delete` and `upsert` read it. A marker filter Prisma
|
|
73
|
+
answers whole runs on any operation; a residual filter runs over `findMany`
|
|
74
|
+
rows alone.
|
|
75
|
+
|
|
76
|
+
## The middleware
|
|
77
|
+
|
|
78
|
+
```ts
|
|
79
|
+
prisma.$use(yggioMiddleware({ yggio, models: Prisma.dmmf }))
|
|
80
|
+
const buys = await prisma.trade.findMany({ where: yggioWhere("side = 'BUY' and venue like 'X%'") })
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
The same marker filter over `{ model, action, args }` and `next`; middleware
|
|
84
|
+
adds no model method, so it has no `plan`.
|
|
85
|
+
|
|
86
|
+
## Pushdown
|
|
87
|
+
|
|
88
|
+
| Plan | Prisma | Condition |
|
|
89
|
+
| --- | --- | --- |
|
|
90
|
+
| `col = v`, `<>`, `<`, `<=`, `>`, `>=` (either side) | `{ col: { equals \| not \| lt \| lte \| gt \| gte: v } }` | a literal or parameter, not null; equality on every type but `Json`, an enum's where it states its `values`; ranges on `Int`, `BigInt`, `Float`, `Decimal`, `DateTime` alone |
|
|
91
|
+
| `col in (...)`, `not col in (...)` | `{ in }`, `{ notIn }` | every item a non-null literal or parameter |
|
|
92
|
+
| `col between a and b` | `{ gte: a, lte: b }` | a range type, each bound as its comparison pushes |
|
|
93
|
+
| text `col = v`, `col in (...)` | `equals`, `in`, checked again | under `mysql`, `sqlserver` or no `provider`, whose default collations fold case and accents (MySQL's pads trailing spaces); `<>`, `not` and `not in` over text stay whole. Exact under `sqlite`, `postgresql`, `cockroachdb`, `mongodb` |
|
|
94
|
+
| `col = v`, `col in (...)` no row can equal | `{ in: [] }`, the item dropped | an enum value that is no member, a fraction on `Int` or `BigInt`; the equality checked again, `<>` left to the residual |
|
|
95
|
+
| `col > 1.5`, `>=`, `<`, `<=` on `Int`, `BigInt` | `{ gte: 2 }`, `{ lte: 1 }` | the whole bound inside the fraction |
|
|
96
|
+
| a `DateTime` bound between two milliseconds | the millisecond outside it - floored for `>`, `>=`, raised for `<`, `<=`; `=` the range around it - checked again | a native type of more than three fractional digits; `<>` and `in` stay |
|
|
97
|
+
| `col is null`, `col is not null` | `{ equals: null }`, `{ not: null }` | the model states the column optional |
|
|
98
|
+
| `col like 'X%'`, `'%X'`, `'%X%'`, `'X'` | `startsWith`, `endsWith`, `contains`, `equals` | a `String` column that is not `@db.Uuid`, no `_`, no `escape`, `%` at the ends alone; pushed to prune and checked again natively, since a database's `LIKE` may fold case |
|
|
99
|
+
| `col ilike ...` | the same, `mode: 'insensitive'` | `insensitive: true`; checked again |
|
|
100
|
+
| `a or b` | `OR: [...]` | every branch pushes |
|
|
101
|
+
| `not a` | `NOT`, `notIn` | `a` pushes exactly |
|
|
102
|
+
| conjuncts on one column | one object | their keys differ and none states `mode`; else `AND: [...]` |
|
|
103
|
+
| `order by col [desc] [nulls first]` | `orderBy: [{ col: 'asc' \| 'desc' }]`, `{ col: { sort, nulls } }` on an optional column, `nulls: 'last'` unless stated | every key a bare column of a range type or `Boolean` whose nullability the model states, no alias of that name, no grouping; one key that does not push keeps the whole order |
|
|
104
|
+
| `limit`, `offset` | `take`, `skip` | the filter pushed whole with nothing to check again, the order pushed or absent, no `group by`, `having` or aggregate |
|
|
105
|
+
| `select a, b` | `select: { a: true, b: true, ...what the residual reads }` | no star; bare columns with no alias push whole, computed projections and aliases stay; a plan reading no column selects the id |
|
|
106
|
+
| `group by`, `having`, aggregates, `exclude` | - | the residual, behind the pushed `where` |
|
|
107
|
+
| `from` | - | absent or the model's name |
|
|
108
|
+
|
|
109
|
+
A text column pushes equality, `in` and `like` alone, a UUID column equality
|
|
110
|
+
and `in`: a database collation orders text by locale, where the native orders
|
|
111
|
+
by code units. Every literal or parameter a pushed filter compares is
|
|
112
|
+
converted by the column, in the core: a `Decimal` at its scale as text, a
|
|
113
|
+
`DateTime` a `Date` from a `Date`, an ISO text or an epoch count, a `Float` a
|
|
114
|
+
float, a `BigInt` a `bigint`, a UUID canonical.
|
|
115
|
+
|
|
116
|
+
## The residual
|
|
117
|
+
|
|
118
|
+
The residual is the plan less what pushed, its parameters bound and every
|
|
119
|
+
literal or parameter its `where` compares typed by the datatype of the term it
|
|
120
|
+
meets (`price > 2.5` on a `decimal128` is a decimal comparison). A residual
|
|
121
|
+
that is a filter alone keeps the rows Prisma answered, untouched; any other
|
|
122
|
+
answers the native output. A plan Prisma answers whole answers its rows as
|
|
123
|
+
Prisma typed them - a `Decimal` a `Prisma.Decimal`.
|
|
124
|
+
|
|
125
|
+
## Types
|
|
126
|
+
|
|
127
|
+
| Prisma type | yggio datatype | Back from a native run |
|
|
128
|
+
| --- | --- | --- |
|
|
129
|
+
| `String` | `utf8`; `uuid` under `@db.Uuid` | `string`, a UUID lower case and hyphenated |
|
|
130
|
+
| `Int` | `int32`; `uint32` under `@db.UnsignedInt`, `@db.UnsignedMediumInt`, `@db.Oid` | `number` |
|
|
131
|
+
| `BigInt` | `int64`; `uint64` under `@db.UnsignedBigInt` | `bigint` |
|
|
132
|
+
| `Float` | `float64` | `number` |
|
|
133
|
+
| `Decimal` | `decimal(p,s)` under `@db.Decimal(p,s)`, the core's narrowest width for the digits, else `decimal128(38,18)` | `string`, the canonical text (`'12.5'`) |
|
|
134
|
+
| `Boolean` | `boolean` | `boolean` |
|
|
135
|
+
| `DateTime` | `datetime64(ms, UTC)`; `datetime64(us, UTC)` past three fractional digits | `Date`, floored to the millisecond |
|
|
136
|
+
| `Json` | `utf8`, the JSON text | `string`, the JSON text |
|
|
137
|
+
| `Bytes` | `binary` | `Buffer` |
|
|
138
|
+
| an enum | `utf8` | `string` |
|
|
139
|
+
| a list | `serie<T>` | an array |
|
|
140
|
+
|
|
141
|
+
A `Decimal` no native type states the precision and scale of lands as
|
|
142
|
+
`decimal128(38,18)`: a value past 18 fractional digits or 20 integral ones
|
|
143
|
+
fails the residual run (`decimal has more fractional digits than the field
|
|
144
|
+
allows`, `decimal coefficient exceeds 38 digits`).
|
|
145
|
+
|
|
146
|
+
An aggregate answers by its own datatype: `count` a `bigint`, a `sum` over
|
|
147
|
+
integers or decimals a decimal's text, `avg` a `number`. A column is nullable
|
|
148
|
+
unless the model states it required; relations are left out.
|
|
149
|
+
|
|
150
|
+
## Refusals
|
|
151
|
+
|
|
152
|
+
- A section that is not a read: `create`, a write, a join.
|
|
153
|
+
- `from` naming anything but the model, a location, or a plan.
|
|
154
|
+
- A parameter the call does not give; a literal a column cannot hold
|
|
155
|
+
(`id = 'abc'`, `u = -1` on an unsigned column) - a fraction meeting an
|
|
156
|
+
integer column is none: it bounds the range or matches nothing.
|
|
157
|
+
- A Prisma type with no yggio datatype; an unknown model; a `provider` that is
|
|
158
|
+
no datasource provider.
|
|
159
|
+
- `yggioWhere` text holding anything but a `where` clause, or no text.
|
|
160
|
+
- A residual marker filter under any operation but `findMany`, or beside
|
|
161
|
+
`take`, `skip`, `cursor` or `distinct`, which would cut rows before it runs.
|
|
162
|
+
|
|
163
|
+
## MySQL
|
|
164
|
+
|
|
165
|
+
The crate's rows - `fix.MarketMessage.field()`, `fix.underlyingField()`,
|
|
166
|
+
`fix.micEntryField()`, `fix.legalEntityField()`, `fix.pluginField()`,
|
|
167
|
+
`fix.fxRateField()` - live in MySQL under one type map, `src/mysql.ts`, read
|
|
168
|
+
in both directions. Every instant is UTC: a `DATETIME` is the wall clock
|
|
169
|
+
Prisma writes and reads in UTC, a `TIMESTAMP` is UTC by MySQL's definition.
|
|
170
|
+
|
|
171
|
+
| MySQL column | Lands as | A column of the crate | Written as |
|
|
172
|
+
| --- | --- | --- | --- |
|
|
173
|
+
| `TINYINT(1)`, `BIT(1)` | `boolean` | `boolean` | `Boolean` |
|
|
174
|
+
| `TINYINT`, `SMALLINT`, `MEDIUMINT`, `INT`, `BIGINT`, each `UNSIGNED` | `int8` .. `uint64` | every integer width | `Int @db.TinyInt` .. `BigInt @db.UnsignedBigInt` |
|
|
175
|
+
| `FLOAT`, `DOUBLE` | `float32`, `float64` | `float32`, `float64` | `Float @db.Float`, `Float` |
|
|
176
|
+
| `DECIMAL(p,s)`, the default `DECIMAL(65,30)` | `decimal(p,s)`, the core's narrowest width for the digits | a decimal at its precision and scale (`decimal` 38 and 18) to 65 digits and scale 30; `bigdecimal` refused | `Decimal @db.Decimal(p,s)` |
|
|
177
|
+
| `DATETIME(n)`, `TIMESTAMP(n)` | `datetime64(s)` at 0 digits, `ms` to 3, `us` to 6, UTC | `datetime64(unit, zone)` | `DateTime @db.DateTime(0 \| 3 \| 6)`; a nanosecond column at six, sent as a `Date` floored to the millisecond - no cast cuts it |
|
|
178
|
+
| `DATE`, `TIME(n)` | `date32`, `time(unit)` (`time32`/`time64`, the core's width) | `date32`, `date64`, `time32`, `time64` | `DateTime @db.Date`, `@db.Time(n)` |
|
|
179
|
+
| `VARCHAR(n)`, `CHAR(n)`, `TEXT` | `utf8` | text; a code (`mic`, `ccy`, `lei`, ...) and `timezone` | `String @db.Text`, the key `@db.VarChar(191)`; `@db.VarChar(n)`, `@db.Char(n)`; a code `@db.VarChar(64)` |
|
|
180
|
+
| `BINARY(16)`; `BINARY(n)`, `VARBINARY(n)`, `BLOB` | `uuid`; `fixed_binary(n)`, `sized_binary(n)`, `binary` | `uuid`; `fixed_binary`, `sized_binary`, `binary` | `Bytes @db.Binary(16)`; `@db.Binary(n)`, `@db.VarBinary(n)`, `Bytes` |
|
|
181
|
+
| `JSON` | `utf8`, the text; the map, struct or serie the generated model states | `map`, `struct`, `serie` | `Json` |
|
|
182
|
+
| `ENUM` | the kind a generated model states; else the kind its name spells (`Side` is `side`) where every member is the kind's, as the core reads them; else `utf8` | `side`, `state`, `marketdatakind`, `marketdatatype`, `timeinforce` | `enum Side { UKNW BUY ... }` from the kinds' member tables |
|
|
183
|
+
|
|
184
|
+
Under every other provider, or none, a MySQL native spelling is read as the
|
|
185
|
+
Prisma type's own default (`@db.TinyInt` is `int32`, `@db.Date` an instant).
|
|
186
|
+
`mysql/schema.prisma` is generated by `npm run mysql:schema` - `fieldModels`
|
|
187
|
+
over the crate's fields as the addon states them (`fix.MarketMessage.field()`,
|
|
188
|
+
`fix.underlyingField()`, ...) and `yggio-market/meta/kinds.json` for the
|
|
189
|
+
enums - never edited by hand; `scripts/mysql-schema.mjs --check` and the
|
|
190
|
+
`arrow` suite hold it current, and the suite holds the committed meta tables
|
|
191
|
+
to the same fields. Each column is read off its `DataType` - family,
|
|
192
|
+
precision, scale, unit, widths - never off its text. Each model is keyed by
|
|
193
|
+
the one column its field's `SORT:by` names (`rowKey`: `underlying` by `code`,
|
|
194
|
+
`mic` by `mic`, `legalentity` by `lei`), else by `STATED_KEYS` with their
|
|
195
|
+
reasons (`fixmsg` and `plugin` state no order, keyed by `uuid` and
|
|
196
|
+
`pluginid`; `fxrate`'s `crosscode, transunix` is an order and no key, keyed by
|
|
197
|
+
`uuid`): the key a `merge` upserts by. `mysql/prisma.config.ts` reads the
|
|
198
|
+
server from `MYSQL_URL`.
|
|
199
|
+
|
|
200
|
+
```js
|
|
201
|
+
// The rows a model answered, as Arrow under the crate's field: landed under the model, cast by the core.
|
|
202
|
+
const model = fieldModel(yggio.fix.MarketMessage.field(), { kinds })
|
|
203
|
+
const reader = modelArrowReader(model, await prisma.fixmsg.findMany(), yggio, { field: yggio.fix.MarketMessage.field() })
|
|
204
|
+
// An Arrow source into the model: 500 rows a createMany, or one upsert per row by the key under merge.
|
|
205
|
+
await writeModel(prisma.fixmsg, messages.intoArrowReader(), model, yggio, { mode: 'merge', client: prisma })
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
A refusal names the model, the rows of the batch and, in the core's words,
|
|
209
|
+
the column and the row (`fixmsg rows 0..500: ...price...`; a stream's
|
|
210
|
+
`fixmsg rows from 500: ...`). Through Prisma an instant crosses as a `Date`,
|
|
211
|
+
to the millisecond, both ways - the `Date` floors a nanosecond source, which
|
|
212
|
+
the generated model lands as it is; `writeModel` sends `skipDuplicates:
|
|
213
|
+
false`, so `skippedRows` is 0 against Prisma and counts only a delegate
|
|
214
|
+
answering a short `count`; a `DATE` or `TIME` as a `Date` too, landed as
|
|
215
|
+
the day or the clock. `overwrite` empties the model at its first `createMany`,
|
|
216
|
+
once that batch is cast and spelled, in one `$transaction` with it where a
|
|
217
|
+
`client` is given. `scripts/check_mysql_interop.py` at the repository root
|
|
218
|
+
runs the exchange against a real MySQL in both directions (the Node half
|
|
219
|
+
`mysql/roundtrip.test.ts`, `npm run mysql:roundtrip`), refused by name where
|
|
220
|
+
none is reachable. Against `MYSQL_URL` it runs `prisma db push --force-reset`,
|
|
221
|
+
which drops every table of that database, so it refuses a database not named
|
|
222
|
+
`yggio` or ending in `_yggio_interop`; the container it starts takes a plain
|
|
223
|
+
`db push`.
|
|
224
|
+
|
|
225
|
+
## Tests
|
|
226
|
+
|
|
227
|
+
`npm test` generates the integration client (`prisma generate` into
|
|
228
|
+
`tests/generated/`), runs every suite against the addon beside this package -
|
|
229
|
+
the integration one over an embedded database file through Prisma's driver adapter - then
|
|
230
|
+
`tsc --noEmit`. `npm run mysql:schema` regenerates `mysql/schema.prisma`,
|
|
231
|
+
`npm run mysql:generate` the MySQL client the exchange drives (`pretest`
|
|
232
|
+
generates it too, so the round trip type-checks with no server), and
|
|
233
|
+
`npm run mysql:roundtrip` is the exchange's Node half, which `MYSQL_URL`
|
|
234
|
+
must name a server for.
|