yggio-prisma 0.0.0-stage → 0.0.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,3 +1,234 @@
1
- # Temporary Holding Version
1
+ # yggio-prisma
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ A Prisma Client extension whose query engine is the yggio plan. A plan's text
4
+ is read by the core's parser; what the database can answer exactly is pushed
5
+ into `findMany` arguments, and the residual runs natively over the rows they
6
+ return, landed once under the model's typed struct field - a decimal at its
7
+ column's scale, a UUID, an instant at its unit. The documentation page is
8
+ [Prisma](https://platob.github.io/yggio/expression/prisma/).
9
+
10
+ | | |
11
+ | --- | --- |
12
+ | Install | `npm install yggio-prisma yggio` - `yggio` is a peer at the exact version, `@prisma/client` (`>=5`) an optional peer: nothing of it is imported, the extension is a plain object |
13
+ | Prisma | `$extends` from Prisma 5; `$use` in Prisma 5 and 6; tested against 7.10.1 |
14
+ | Doors | `yggioExtension(options)` for `prisma.$extends`: `model.plan(text, params?)` on every model, `yggioWhere(text, params?)` under any operation's `where`; `yggioMiddleware(options)`, the marker filter for `$use` |
15
+ | By hand | `compilePlan(plan, model, { yggio, params?, provider?, insensitive?, field? })` -> `{ args, residual, complete, pushed, columns }`; `modelField(model, yggio, columns?, provider?)`; `landRows(model, rows, yggio, columns?)`; `runPlan(landedRows, residual, field, yggio)`; `filterRows(rows, filter, model, columns, yggio)` |
16
+ | Arrow over MySQL | `modelArrowReader(model, rows, yggio, { field?, batchSize? })` - a model's rows as Arrow batches cast onto the crate's field by the core; `writeModel(delegate, source, model, yggio, { mode?, batchSize?, key?, client? })` - an Arrow source through `createMany` or `upsert` in batches; `mysqlDatatype`, `mysqlColumn`, `mysqlModel`, `fieldModel`, `prismaSchema` - the MySQL type map, both ways ([MySQL](#mysql)) |
17
+ | Refusals | `PlanRefused`, an `Error` naming what it refuses |
18
+
19
+ ## Install
20
+
21
+ ```bash
22
+ npm install yggio-prisma yggio @prisma/client
23
+ ```
24
+
25
+ `yggio` is a peer at the exact version and is never imported: every door
26
+ takes it as an argument, so the application loads the addon once and hands
27
+ it over. `@prisma/client` is an optional peer, needed only for a generated
28
+ client; a MySQL application adds its driver adapter
29
+ (`@prisma/adapter-mariadb`). The package carries `dist/` (ESM and CommonJS,
30
+ declarations) and its sources; `npm pack` builds it first (`prepack`).
31
+ `node/market/examples/market-app` writes its schema with
32
+ `prismaSchema(fieldModels(yggio, kinds), { output })`, generates the client,
33
+ and reads and writes the walked messages through `modelArrowReader` and
34
+ `writeModel` ([Applications](https://platob.github.io/yggio/market/applications/)).
35
+
36
+ ## The extension
37
+
38
+ ```ts
39
+ import * as yggio from 'yggio'
40
+ import { yggioExtension, yggioWhere } from 'yggio-prisma'
41
+
42
+ const db = prisma.$extends(yggioExtension({ yggio, models: [TRADE], provider: 'postgresql' }))
43
+
44
+ // findMany with what pushes, then the residual natively.
45
+ const totals = await db.trade.plan(
46
+ 'select venue, sum(quantity) as total from trade where price > :floor group by venue order by total desc',
47
+ { floor: 10 },
48
+ )
49
+
50
+ // The marker filter: compiled where the operation runs, its keys beside any Prisma filter of the call.
51
+ const buys = await db.trade.findMany({ where: { ...yggioWhere("side = 'BUY' and venue like 'X%'"), tradable: true } })
52
+ ```
53
+
54
+ | Option | Is |
55
+ | --- | --- |
56
+ | `yggio` | the `yggio` module as the caller loaded it |
57
+ | `models` | a list of DMMF models, or `Prisma.dmmf` itself (`{ datamodel: { models, enums } }`), whose `enums` give each enum column its `values`; a model is found by the delegate's `$name`, case aside |
58
+ | `provider` | the datasource provider word - `sqlite`, `postgresql`, `cockroachdb`, `mongodb`, `mysql`, `sqlserver`; text equality pushes exactly under the first four alone; another word is refused |
59
+ | `insensitive` | `true` pushes `ilike` as `mode: 'insensitive'`, only for a database that takes that mode |
60
+ | `marker` | the `where` key `yggioWhere` writes, `$yggio` by default; pass the same name as `yggioWhere`'s third argument |
61
+
62
+ Prisma 7's own `Prisma.dmmf` keeps no `isRequired`, `isList` or `nativeType`,
63
+ and no enum's members: hand each model with them, as the schema states them,
64
+ and each enum column its `values` (`Object.values(Side)`). A model without
65
+ `isRequired` lands every column nullable, and no null test and no order push;
66
+ an enum column without `values` pushes no equality and no `in`.
67
+
68
+ `plan` answers the rows as Prisma answered them where nothing remains or the
69
+ residual is a filter alone, else one object per native output row keyed by the
70
+ plan's output columns. The pushed `where` sits beside the caller's at the top
71
+ level, a key both state joined by `AND`, so a unique key stays where
72
+ `findUnique`, `update`, `delete` and `upsert` read it. A marker filter Prisma
73
+ answers whole runs on any operation; a residual filter runs over `findMany`
74
+ rows alone.
75
+
76
+ ## The middleware
77
+
78
+ ```ts
79
+ prisma.$use(yggioMiddleware({ yggio, models: Prisma.dmmf }))
80
+ const buys = await prisma.trade.findMany({ where: yggioWhere("side = 'BUY' and venue like 'X%'") })
81
+ ```
82
+
83
+ The same marker filter over `{ model, action, args }` and `next`; middleware
84
+ adds no model method, so it has no `plan`.
85
+
86
+ ## Pushdown
87
+
88
+ | Plan | Prisma | Condition |
89
+ | --- | --- | --- |
90
+ | `col = v`, `<>`, `<`, `<=`, `>`, `>=` (either side) | `{ col: { equals \| not \| lt \| lte \| gt \| gte: v } }` | a literal or parameter, not null; equality on every type but `Json`, an enum's where it states its `values`; ranges on `Int`, `BigInt`, `Float`, `Decimal`, `DateTime` alone |
91
+ | `col in (...)`, `not col in (...)` | `{ in }`, `{ notIn }` | every item a non-null literal or parameter |
92
+ | `col between a and b` | `{ gte: a, lte: b }` | a range type, each bound as its comparison pushes |
93
+ | text `col = v`, `col in (...)` | `equals`, `in`, checked again | under `mysql`, `sqlserver` or no `provider`, whose default collations fold case and accents (MySQL's pads trailing spaces); `<>`, `not` and `not in` over text stay whole. Exact under `sqlite`, `postgresql`, `cockroachdb`, `mongodb` |
94
+ | `col = v`, `col in (...)` no row can equal | `{ in: [] }`, the item dropped | an enum value that is no member, a fraction on `Int` or `BigInt`; the equality checked again, `<>` left to the residual |
95
+ | `col > 1.5`, `>=`, `<`, `<=` on `Int`, `BigInt` | `{ gte: 2 }`, `{ lte: 1 }` | the whole bound inside the fraction |
96
+ | a `DateTime` bound between two milliseconds | the millisecond outside it - floored for `>`, `>=`, raised for `<`, `<=`; `=` the range around it - checked again | a native type of more than three fractional digits; `<>` and `in` stay |
97
+ | `col is null`, `col is not null` | `{ equals: null }`, `{ not: null }` | the model states the column optional |
98
+ | `col like 'X%'`, `'%X'`, `'%X%'`, `'X'` | `startsWith`, `endsWith`, `contains`, `equals` | a `String` column that is not `@db.Uuid`, no `_`, no `escape`, `%` at the ends alone; pushed to prune and checked again natively, since a database's `LIKE` may fold case |
99
+ | `col ilike ...` | the same, `mode: 'insensitive'` | `insensitive: true`; checked again |
100
+ | `a or b` | `OR: [...]` | every branch pushes |
101
+ | `not a` | `NOT`, `notIn` | `a` pushes exactly |
102
+ | conjuncts on one column | one object | their keys differ and none states `mode`; else `AND: [...]` |
103
+ | `order by col [desc] [nulls first]` | `orderBy: [{ col: 'asc' \| 'desc' }]`, `{ col: { sort, nulls } }` on an optional column, `nulls: 'last'` unless stated | every key a bare column of a range type or `Boolean` whose nullability the model states, no alias of that name, no grouping; one key that does not push keeps the whole order |
104
+ | `limit`, `offset` | `take`, `skip` | the filter pushed whole with nothing to check again, the order pushed or absent, no `group by`, `having` or aggregate |
105
+ | `select a, b` | `select: { a: true, b: true, ...what the residual reads }` | no star; bare columns with no alias push whole, computed projections and aliases stay; a plan reading no column selects the id |
106
+ | `group by`, `having`, aggregates, `exclude` | - | the residual, behind the pushed `where` |
107
+ | `from` | - | absent or the model's name |
108
+
109
+ A text column pushes equality, `in` and `like` alone, a UUID column equality
110
+ and `in`: a database collation orders text by locale, where the native orders
111
+ by code units. Every literal or parameter a pushed filter compares is
112
+ converted by the column, in the core: a `Decimal` at its scale as text, a
113
+ `DateTime` a `Date` from a `Date`, an ISO text or an epoch count, a `Float` a
114
+ float, a `BigInt` a `bigint`, a UUID canonical.
115
+
116
+ ## The residual
117
+
118
+ The residual is the plan less what pushed, its parameters bound and every
119
+ literal or parameter its `where` compares typed by the datatype of the term it
120
+ meets (`price > 2.5` on a `decimal128` is a decimal comparison). A residual
121
+ that is a filter alone keeps the rows Prisma answered, untouched; any other
122
+ answers the native output. A plan Prisma answers whole answers its rows as
123
+ Prisma typed them - a `Decimal` a `Prisma.Decimal`.
124
+
125
+ ## Types
126
+
127
+ | Prisma type | yggio datatype | Back from a native run |
128
+ | --- | --- | --- |
129
+ | `String` | `utf8`; `uuid` under `@db.Uuid` | `string`, a UUID lower case and hyphenated |
130
+ | `Int` | `int32`; `uint32` under `@db.UnsignedInt`, `@db.UnsignedMediumInt`, `@db.Oid` | `number` |
131
+ | `BigInt` | `int64`; `uint64` under `@db.UnsignedBigInt` | `bigint` |
132
+ | `Float` | `float64` | `number` |
133
+ | `Decimal` | `decimal(p,s)` under `@db.Decimal(p,s)`, the core's narrowest width for the digits, else `decimal128(38,18)` | `string`, the canonical text (`'12.5'`) |
134
+ | `Boolean` | `boolean` | `boolean` |
135
+ | `DateTime` | `datetime64(ms, UTC)`; `datetime64(us, UTC)` past three fractional digits | `Date`, floored to the millisecond |
136
+ | `Json` | `utf8`, the JSON text | `string`, the JSON text |
137
+ | `Bytes` | `binary` | `Buffer` |
138
+ | an enum | `utf8` | `string` |
139
+ | a list | `serie<T>` | an array |
140
+
141
+ A `Decimal` no native type states the precision and scale of lands as
142
+ `decimal128(38,18)`: a value past 18 fractional digits or 20 integral ones
143
+ fails the residual run (`decimal has more fractional digits than the field
144
+ allows`, `decimal coefficient exceeds 38 digits`).
145
+
146
+ An aggregate answers by its own datatype: `count` a `bigint`, a `sum` over
147
+ integers or decimals a decimal's text, `avg` a `number`. A column is nullable
148
+ unless the model states it required; relations are left out.
149
+
150
+ ## Refusals
151
+
152
+ - A section that is not a read: `create`, a write, a join.
153
+ - `from` naming anything but the model, a location, or a plan.
154
+ - A parameter the call does not give; a literal a column cannot hold
155
+ (`id = 'abc'`, `u = -1` on an unsigned column) - a fraction meeting an
156
+ integer column is none: it bounds the range or matches nothing.
157
+ - A Prisma type with no yggio datatype; an unknown model; a `provider` that is
158
+ no datasource provider.
159
+ - `yggioWhere` text holding anything but a `where` clause, or no text.
160
+ - A residual marker filter under any operation but `findMany`, or beside
161
+ `take`, `skip`, `cursor` or `distinct`, which would cut rows before it runs.
162
+
163
+ ## MySQL
164
+
165
+ The crate's rows - `fix.MarketMessage.field()`, `fix.underlyingField()`,
166
+ `fix.micEntryField()`, `fix.legalEntityField()`, `fix.pluginField()`,
167
+ `fix.fxRateField()` - live in MySQL under one type map, `src/mysql.ts`, read
168
+ in both directions. Every instant is UTC: a `DATETIME` is the wall clock
169
+ Prisma writes and reads in UTC, a `TIMESTAMP` is UTC by MySQL's definition.
170
+
171
+ | MySQL column | Lands as | A column of the crate | Written as |
172
+ | --- | --- | --- | --- |
173
+ | `TINYINT(1)`, `BIT(1)` | `boolean` | `boolean` | `Boolean` |
174
+ | `TINYINT`, `SMALLINT`, `MEDIUMINT`, `INT`, `BIGINT`, each `UNSIGNED` | `int8` .. `uint64` | every integer width | `Int @db.TinyInt` .. `BigInt @db.UnsignedBigInt` |
175
+ | `FLOAT`, `DOUBLE` | `float32`, `float64` | `float32`, `float64` | `Float @db.Float`, `Float` |
176
+ | `DECIMAL(p,s)`, the default `DECIMAL(65,30)` | `decimal(p,s)`, the core's narrowest width for the digits | a decimal at its precision and scale (`decimal` 38 and 18) to 65 digits and scale 30; `bigdecimal` refused | `Decimal @db.Decimal(p,s)` |
177
+ | `DATETIME(n)`, `TIMESTAMP(n)` | `datetime64(s)` at 0 digits, `ms` to 3, `us` to 6, UTC | `datetime64(unit, zone)` | `DateTime @db.DateTime(0 \| 3 \| 6)`; a nanosecond column at six, sent as a `Date` floored to the millisecond - no cast cuts it |
178
+ | `DATE`, `TIME(n)` | `date32`, `time(unit)` (`time32`/`time64`, the core's width) | `date32`, `date64`, `time32`, `time64` | `DateTime @db.Date`, `@db.Time(n)` |
179
+ | `VARCHAR(n)`, `CHAR(n)`, `TEXT` | `utf8` | text; a code (`mic`, `ccy`, `lei`, ...) and `timezone` | `String @db.Text`, the key `@db.VarChar(191)`; `@db.VarChar(n)`, `@db.Char(n)`; a code `@db.VarChar(64)` |
180
+ | `BINARY(16)`; `BINARY(n)`, `VARBINARY(n)`, `BLOB` | `uuid`; `fixed_binary(n)`, `sized_binary(n)`, `binary` | `uuid`; `fixed_binary`, `sized_binary`, `binary` | `Bytes @db.Binary(16)`; `@db.Binary(n)`, `@db.VarBinary(n)`, `Bytes` |
181
+ | `JSON` | `utf8`, the text; the map, struct or serie the generated model states | `map`, `struct`, `serie` | `Json` |
182
+ | `ENUM` | the kind a generated model states; else the kind its name spells (`Side` is `side`) where every member is the kind's, as the core reads them; else `utf8` | `side`, `state`, `marketdatakind`, `marketdatatype`, `timeinforce` | `enum Side { UKNW BUY ... }` from the kinds' member tables |
183
+
184
+ Under every other provider, or none, a MySQL native spelling is read as the
185
+ Prisma type's own default (`@db.TinyInt` is `int32`, `@db.Date` an instant).
186
+ `mysql/schema.prisma` is generated by `npm run mysql:schema` - `fieldModels`
187
+ over the crate's fields as the addon states them (`fix.MarketMessage.field()`,
188
+ `fix.underlyingField()`, ...) and `yggio-market/meta/kinds.json` for the
189
+ enums - never edited by hand; `scripts/mysql-schema.mjs --check` and the
190
+ `arrow` suite hold it current, and the suite holds the committed meta tables
191
+ to the same fields. Each column is read off its `DataType` - family,
192
+ precision, scale, unit, widths - never off its text. Each model is keyed by
193
+ the one column its field's `SORT:by` names (`rowKey`: `underlying` by `code`,
194
+ `mic` by `mic`, `legalentity` by `lei`), else by `STATED_KEYS` with their
195
+ reasons (`fixmsg` and `plugin` state no order, keyed by `uuid` and
196
+ `pluginid`; `fxrate`'s `crosscode, transunix` is an order and no key, keyed by
197
+ `uuid`): the key a `merge` upserts by. `mysql/prisma.config.ts` reads the
198
+ server from `MYSQL_URL`.
199
+
200
+ ```js
201
+ // The rows a model answered, as Arrow under the crate's field: landed under the model, cast by the core.
202
+ const model = fieldModel(yggio.fix.MarketMessage.field(), { kinds })
203
+ const reader = modelArrowReader(model, await prisma.fixmsg.findMany(), yggio, { field: yggio.fix.MarketMessage.field() })
204
+ // An Arrow source into the model: 500 rows a createMany, or one upsert per row by the key under merge.
205
+ await writeModel(prisma.fixmsg, messages.intoArrowReader(), model, yggio, { mode: 'merge', client: prisma })
206
+ ```
207
+
208
+ A refusal names the model, the rows of the batch and, in the core's words,
209
+ the column and the row (`fixmsg rows 0..500: ...price...`; a stream's
210
+ `fixmsg rows from 500: ...`). Through Prisma an instant crosses as a `Date`,
211
+ to the millisecond, both ways - the `Date` floors a nanosecond source, which
212
+ the generated model lands as it is; `writeModel` sends `skipDuplicates:
213
+ false`, so `skippedRows` is 0 against Prisma and counts only a delegate
214
+ answering a short `count`; a `DATE` or `TIME` as a `Date` too, landed as
215
+ the day or the clock. `overwrite` empties the model at its first `createMany`,
216
+ once that batch is cast and spelled, in one `$transaction` with it where a
217
+ `client` is given. `scripts/check_mysql_interop.py` at the repository root
218
+ runs the exchange against a real MySQL in both directions (the Node half
219
+ `mysql/roundtrip.test.ts`, `npm run mysql:roundtrip`), refused by name where
220
+ none is reachable. Against `MYSQL_URL` it runs `prisma db push --force-reset`,
221
+ which drops every table of that database, so it refuses a database not named
222
+ `yggio` or ending in `_yggio_interop`; the container it starts takes a plain
223
+ `db push`.
224
+
225
+ ## Tests
226
+
227
+ `npm test` generates the integration client (`prisma generate` into
228
+ `tests/generated/`), runs every suite against the addon beside this package -
229
+ the integration one over an embedded database file through Prisma's driver adapter - then
230
+ `tsc --noEmit`. `npm run mysql:schema` regenerates `mysql/schema.prisma`,
231
+ `npm run mysql:generate` the MySQL client the exchange drives (`pretest`
232
+ generates it too, so the round trip type-checks with no server), and
233
+ `npm run mysql:roundtrip` is the exchange's Node half, which `MYSQL_URL`
234
+ must name a server for.