yggio-prisma 0.0.6 → 0.0.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +87 -3
- package/dist/index.cjs +577 -23
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +369 -39
- package/dist/index.d.ts +369 -39
- package/dist/index.js +552 -24
- package/dist/index.js.map +1 -1
- package/package.json +9 -4
- package/src/arrow.ts +344 -0
- package/src/extension.ts +3 -2
- package/src/index.ts +2 -0
- package/src/landing.ts +110 -22
- package/src/mysql.ts +436 -0
- package/src/pushdown.ts +85 -0
package/README.md
CHANGED
|
@@ -12,9 +12,27 @@ column's scale, a UUID, an instant at its unit. The documentation page is
|
|
|
12
12
|
| Install | `npm install yggio-prisma yggio` - `yggio` is a peer at the exact version, `@prisma/client` (`>=5`) an optional peer: nothing of it is imported, the extension is a plain object |
|
|
13
13
|
| Prisma | `$extends` from Prisma 5; `$use` in Prisma 5 and 6; tested against 7.10.1 |
|
|
14
14
|
| Doors | `yggioExtension(options)` for `prisma.$extends`: `model.plan(text, params?)` on every model, `yggioWhere(text, params?)` under any operation's `where`; `yggioMiddleware(options)`, the marker filter for `$use` |
|
|
15
|
-
| By hand | `compilePlan(plan, model, { yggio, params?, provider?, insensitive?, field? })` -> `{ args, residual, complete, pushed, columns }`; `modelField(model, yggio, columns?)`; `landRows(model, rows, yggio, columns?)`; `runPlan(landedRows, residual, field, yggio)`; `filterRows(rows, filter, model, columns, yggio)` |
|
|
15
|
+
| By hand | `compilePlan(plan, model, { yggio, params?, provider?, insensitive?, field? })` -> `{ args, residual, complete, pushed, columns }`; `modelField(model, yggio, columns?, provider?)`; `landRows(model, rows, yggio, columns?)`; `runPlan(landedRows, residual, field, yggio)`; `filterRows(rows, filter, model, columns, yggio)` |
|
|
16
|
+
| Arrow over MySQL | `modelArrowReader(model, rows, yggio, { field?, batchSize? })` - a model's rows as Arrow batches cast onto the crate's field by the core; `writeModel(delegate, source, model, yggio, { mode?, batchSize?, key?, client? })` - an Arrow source through `createMany` or `upsert` in batches; `mysqlDatatype`, `mysqlColumn`, `mysqlModel`, `fieldModel`, `prismaSchema` - the MySQL type map, both ways ([MySQL](#mysql)) |
|
|
16
17
|
| Refusals | `PlanRefused`, an `Error` naming what it refuses |
|
|
17
18
|
|
|
19
|
+
## Install
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
npm install yggio-prisma yggio @prisma/client
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
`yggio` is a peer at the exact version and is never imported: every door
|
|
26
|
+
takes it as an argument, so the application loads the addon once and hands
|
|
27
|
+
it over. `@prisma/client` is an optional peer, needed only for a generated
|
|
28
|
+
client; a MySQL application adds its driver adapter
|
|
29
|
+
(`@prisma/adapter-mariadb`). The package carries `dist/` (ESM and CommonJS,
|
|
30
|
+
declarations) and its sources; `npm pack` builds it first (`prepack`).
|
|
31
|
+
`node/market/examples/market-app` writes its schema with
|
|
32
|
+
`prismaSchema(fieldModels(yggio, kinds), { output })`, generates the client,
|
|
33
|
+
and reads and writes the walked messages through `modelArrowReader` and
|
|
34
|
+
`writeModel` ([Applications](https://platob.github.io/yggio/market/applications/)).
|
|
35
|
+
|
|
18
36
|
## The extension
|
|
19
37
|
|
|
20
38
|
```ts
|
|
@@ -112,7 +130,7 @@ Prisma typed them - a `Decimal` a `Prisma.Decimal`.
|
|
|
112
130
|
| `Int` | `int32`; `uint32` under `@db.UnsignedInt`, `@db.UnsignedMediumInt`, `@db.Oid` | `number` |
|
|
113
131
|
| `BigInt` | `int64`; `uint64` under `@db.UnsignedBigInt` | `bigint` |
|
|
114
132
|
| `Float` | `float64` | `number` |
|
|
115
|
-
| `Decimal` | `
|
|
133
|
+
| `Decimal` | `decimal(p,s)` under `@db.Decimal(p,s)`, the core's narrowest width for the digits, else `decimal128(38,18)` | `string`, the canonical text (`'12.5'`) |
|
|
116
134
|
| `Boolean` | `boolean` | `boolean` |
|
|
117
135
|
| `DateTime` | `datetime64(ms, UTC)`; `datetime64(us, UTC)` past three fractional digits | `Date`, floored to the millisecond |
|
|
118
136
|
| `Json` | `utf8`, the JSON text | `string`, the JSON text |
|
|
@@ -142,9 +160,75 @@ unless the model states it required; relations are left out.
|
|
|
142
160
|
- A residual marker filter under any operation but `findMany`, or beside
|
|
143
161
|
`take`, `skip`, `cursor` or `distinct`, which would cut rows before it runs.
|
|
144
162
|
|
|
163
|
+
## MySQL
|
|
164
|
+
|
|
165
|
+
The crate's rows - `fix.MarketMessage.field()`, `fix.underlyingField()`,
|
|
166
|
+
`fix.micEntryField()`, `fix.legalEntityField()`, `fix.pluginField()`,
|
|
167
|
+
`fix.fxRateField()` - live in MySQL under one type map, `src/mysql.ts`, read
|
|
168
|
+
in both directions. Every instant is UTC: a `DATETIME` is the wall clock
|
|
169
|
+
Prisma writes and reads in UTC, a `TIMESTAMP` is UTC by MySQL's definition.
|
|
170
|
+
|
|
171
|
+
| MySQL column | Lands as | A column of the crate | Written as |
|
|
172
|
+
| --- | --- | --- | --- |
|
|
173
|
+
| `TINYINT(1)`, `BIT(1)` | `boolean` | `boolean` | `Boolean` |
|
|
174
|
+
| `TINYINT`, `SMALLINT`, `MEDIUMINT`, `INT`, `BIGINT`, each `UNSIGNED` | `int8` .. `uint64` | every integer width | `Int @db.TinyInt` .. `BigInt @db.UnsignedBigInt` |
|
|
175
|
+
| `FLOAT`, `DOUBLE` | `float32`, `float64` | `float32`, `float64` | `Float @db.Float`, `Float` |
|
|
176
|
+
| `DECIMAL(p,s)`, the default `DECIMAL(65,30)` | `decimal(p,s)`, the core's narrowest width for the digits | a decimal at its precision and scale (`decimal` 38 and 18) to 65 digits and scale 30; `bigdecimal` refused | `Decimal @db.Decimal(p,s)` |
|
|
177
|
+
| `DATETIME(n)`, `TIMESTAMP(n)` | `datetime64(s)` at 0 digits, `ms` to 3, `us` to 6, UTC | `datetime64(unit, zone)` | `DateTime @db.DateTime(0 \| 3 \| 6)`; a nanosecond column at six, sent as a `Date` floored to the millisecond - no cast cuts it |
|
|
178
|
+
| `DATE`, `TIME(n)` | `date32`, `time(unit)` (`time32`/`time64`, the core's width) | `date32`, `date64`, `time32`, `time64` | `DateTime @db.Date`, `@db.Time(n)` |
|
|
179
|
+
| `VARCHAR(n)`, `CHAR(n)`, `TEXT` | `utf8` | text; a code (`mic`, `ccy`, `lei`, ...) and `timezone` | `String @db.Text`, the key `@db.VarChar(191)`; `@db.VarChar(n)`, `@db.Char(n)`; a code `@db.VarChar(64)` |
|
|
180
|
+
| `BINARY(16)`; `BINARY(n)`, `VARBINARY(n)`, `BLOB` | `uuid`; `fixed_binary(n)`, `sized_binary(n)`, `binary` | `uuid`; `fixed_binary`, `sized_binary`, `binary` | `Bytes @db.Binary(16)`; `@db.Binary(n)`, `@db.VarBinary(n)`, `Bytes` |
|
|
181
|
+
| `JSON` | `utf8`, the text; the map, struct or serie the generated model states | `map`, `struct`, `serie` | `Json` |
|
|
182
|
+
| `ENUM` | the kind a generated model states; else the kind its name spells (`Side` is `side`) where every member is the kind's, as the core reads them; else `utf8` | `side`, `state`, `marketdatakind`, `marketdatatype`, `timeinforce` | `enum Side { UKNW BUY ... }` from the kinds' member tables |
|
|
183
|
+
|
|
184
|
+
Under every other provider, or none, a MySQL native spelling is read as the
|
|
185
|
+
Prisma type's own default (`@db.TinyInt` is `int32`, `@db.Date` an instant).
|
|
186
|
+
`mysql/schema.prisma` is generated by `npm run mysql:schema` - `fieldModels`
|
|
187
|
+
over the crate's fields as the addon states them (`fix.MarketMessage.field()`,
|
|
188
|
+
`fix.underlyingField()`, ...) and `yggio-market/meta/kinds.json` for the
|
|
189
|
+
enums - never edited by hand; `scripts/mysql-schema.mjs --check` and the
|
|
190
|
+
`arrow` suite hold it current, and the suite holds the committed meta tables
|
|
191
|
+
to the same fields. Each column is read off its `DataType` - family,
|
|
192
|
+
precision, scale, unit, widths - never off its text. Each model is keyed by
|
|
193
|
+
the one column its field's `SORT:by` names (`rowKey`: `underlying` by `code`,
|
|
194
|
+
`mic` by `mic`, `legalentity` by `lei`), else by `STATED_KEYS` with their
|
|
195
|
+
reasons (`fixmsg` and `plugin` state no order, keyed by `uuid` and
|
|
196
|
+
`pluginid`; `fxrate`'s `crosscode, transunix` is an order and no key, keyed by
|
|
197
|
+
`uuid`): the key a `merge` upserts by. `mysql/prisma.config.ts` reads the
|
|
198
|
+
server from `MYSQL_URL`.
|
|
199
|
+
|
|
200
|
+
```js
|
|
201
|
+
// The rows a model answered, as Arrow under the crate's field: landed under the model, cast by the core.
|
|
202
|
+
const model = fieldModel(yggio.fix.MarketMessage.field(), { kinds })
|
|
203
|
+
const reader = modelArrowReader(model, await prisma.fixmsg.findMany(), yggio, { field: yggio.fix.MarketMessage.field() })
|
|
204
|
+
// An Arrow source into the model: 500 rows a createMany, or one upsert per row by the key under merge.
|
|
205
|
+
await writeModel(prisma.fixmsg, messages.intoArrowReader(), model, yggio, { mode: 'merge', client: prisma })
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
A refusal names the model, the rows of the batch and, in the core's words,
|
|
209
|
+
the column and the row (`fixmsg rows 0..500: ...price...`; a stream's
|
|
210
|
+
`fixmsg rows from 500: ...`). Through Prisma an instant crosses as a `Date`,
|
|
211
|
+
to the millisecond, both ways - the `Date` floors a nanosecond source, which
|
|
212
|
+
the generated model lands as it is; `writeModel` sends `skipDuplicates:
|
|
213
|
+
false`, so `skippedRows` is 0 against Prisma and counts only a delegate
|
|
214
|
+
answering a short `count`; a `DATE` or `TIME` as a `Date` too, landed as
|
|
215
|
+
the day or the clock. `overwrite` empties the model at its first `createMany`,
|
|
216
|
+
once that batch is cast and spelled, in one `$transaction` with it where a
|
|
217
|
+
`client` is given. `scripts/check_mysql_interop.py` at the repository root
|
|
218
|
+
runs the exchange against a real MySQL in both directions (the Node half
|
|
219
|
+
`mysql/roundtrip.test.ts`, `npm run mysql:roundtrip`), refused by name where
|
|
220
|
+
none is reachable. Against `MYSQL_URL` it runs `prisma db push --force-reset`,
|
|
221
|
+
which drops every table of that database, so it refuses a database not named
|
|
222
|
+
`yggio` or ending in `_yggio_interop`; the container it starts takes a plain
|
|
223
|
+
`db push`.
|
|
224
|
+
|
|
145
225
|
## Tests
|
|
146
226
|
|
|
147
227
|
`npm test` generates the integration client (`prisma generate` into
|
|
148
228
|
`tests/generated/`), runs every suite against the addon beside this package -
|
|
149
229
|
the integration one over an embedded database file through Prisma's driver adapter - then
|
|
150
|
-
`tsc --noEmit`.
|
|
230
|
+
`tsc --noEmit`. `npm run mysql:schema` regenerates `mysql/schema.prisma`,
|
|
231
|
+
`npm run mysql:generate` the MySQL client the exchange drives (`pretest`
|
|
232
|
+
generates it too, so the round trip type-checks with no server), and
|
|
233
|
+
`npm run mysql:roundtrip` is the exchange's Node half, which `MYSQL_URL`
|
|
234
|
+
must name a server for.
|