yggio-prisma 0.0.6 → 0.0.15

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -12,9 +12,27 @@ column's scale, a UUID, an instant at its unit. The documentation page is
12
12
  | Install | `npm install yggio-prisma yggio` - `yggio` is a peer at the exact version, `@prisma/client` (`>=5`) an optional peer: nothing of it is imported, the extension is a plain object |
13
13
  | Prisma | `$extends` from Prisma 5; `$use` in Prisma 5 and 6; tested against 7.10.1 |
14
14
  | Doors | `yggioExtension(options)` for `prisma.$extends`: `model.plan(text, params?)` on every model, `yggioWhere(text, params?)` under any operation's `where`; `yggioMiddleware(options)`, the marker filter for `$use` |
15
- | By hand | `compilePlan(plan, model, { yggio, params?, provider?, insensitive?, field? })` -> `{ args, residual, complete, pushed, columns }`; `modelField(model, yggio, columns?)`; `landRows(model, rows, yggio, columns?)`; `runPlan(landedRows, residual, field, yggio)`; `filterRows(rows, filter, model, columns, yggio)` |
15
+ | By hand | `compilePlan(plan, model, { yggio, params?, provider?, insensitive?, field? })` -> `{ args, residual, complete, pushed, columns }`; `modelField(model, yggio, columns?, provider?)`; `landRows(model, rows, yggio, columns?)`; `runPlan(landedRows, residual, field, yggio)`; `filterRows(rows, filter, model, columns, yggio)` |
16
+ | Arrow over MySQL | `modelArrowReader(model, rows, yggio, { field?, batchSize? })` - a model's rows as Arrow batches cast onto the crate's field by the core; `writeModel(delegate, source, model, yggio, { mode?, batchSize?, key?, client? })` - an Arrow source through `createMany` or `upsert` in batches; `mysqlDatatype`, `mysqlColumn`, `mysqlModel`, `fieldModel`, `prismaSchema` - the MySQL type map, both ways ([MySQL](#mysql)) |
16
17
  | Refusals | `PlanRefused`, an `Error` naming what it refuses |
17
18
 
19
+ ## Install
20
+
21
+ ```bash
22
+ npm install yggio-prisma yggio @prisma/client
23
+ ```
24
+
25
+ `yggio` is a peer at the exact version and is never imported: every door
26
+ takes it as an argument, so the application loads the addon once and hands
27
+ it over. `@prisma/client` is an optional peer, needed only for a generated
28
+ client; a MySQL application adds its driver adapter
29
+ (`@prisma/adapter-mariadb`). The package carries `dist/` (ESM and CommonJS,
30
+ declarations) and its sources; `npm pack` builds it first (`prepack`).
31
+ `node/market/examples/market-app` writes its schema with
32
+ `prismaSchema(fieldModels(yggio, kinds), { output })`, generates the client,
33
+ and reads and writes the walked messages through `modelArrowReader` and
34
+ `writeModel` ([Applications](https://platob.github.io/yggio/market/applications/)).
35
+
18
36
  ## The extension
19
37
 
20
38
  ```ts
@@ -112,7 +130,7 @@ Prisma typed them - a `Decimal` a `Prisma.Decimal`.
112
130
  | `Int` | `int32`; `uint32` under `@db.UnsignedInt`, `@db.UnsignedMediumInt`, `@db.Oid` | `number` |
113
131
  | `BigInt` | `int64`; `uint64` under `@db.UnsignedBigInt` | `bigint` |
114
132
  | `Float` | `float64` | `number` |
115
- | `Decimal` | `decimal128(p,s)` under `@db.Decimal(p,s)` (`decimal256` past 38 digits), else `decimal128(38,18)` | `string`, the canonical text (`'12.5'`) |
133
+ | `Decimal` | `decimal(p,s)` under `@db.Decimal(p,s)`, the core's narrowest width for the digits, else `decimal128(38,18)` | `string`, the canonical text (`'12.5'`) |
116
134
  | `Boolean` | `boolean` | `boolean` |
117
135
  | `DateTime` | `datetime64(ms, UTC)`; `datetime64(us, UTC)` past three fractional digits | `Date`, floored to the millisecond |
118
136
  | `Json` | `utf8`, the JSON text | `string`, the JSON text |
@@ -142,9 +160,109 @@ unless the model states it required; relations are left out.
142
160
  - A residual marker filter under any operation but `findMany`, or beside
143
161
  `take`, `skip`, `cursor` or `distinct`, which would cut rows before it runs.
144
162
 
163
+ ## MySQL
164
+
165
+ The crate's rows - `fix.MarketMessage.field()`, `fix.underlyingField()`,
166
+ `fix.micEntryField()`, `fix.legalEntityField()`, `fix.pluginField()`,
167
+ `fix.fxRateField()` - live in MySQL under one type map, `src/mysql.ts`, read
168
+ in both directions. Every instant is UTC: a `DATETIME` is the wall clock
169
+ Prisma writes and reads in UTC, a `TIMESTAMP` is UTC by MySQL's definition.
170
+
171
+ | MySQL column | Lands as | A column of the crate | Written as |
172
+ | --- | --- | --- | --- |
173
+ | `TINYINT(1)`, `BIT(1)` | `boolean` | `boolean` | `Boolean` |
174
+ | `TINYINT`, `SMALLINT`, `MEDIUMINT`, `INT`, `BIGINT`, each `UNSIGNED` | `int8` .. `uint64` | every integer width | `Int @db.TinyInt` .. `BigInt @db.UnsignedBigInt` |
175
+ | `FLOAT`, `DOUBLE` | `float32`, `float64` | `float32`, `float64` | `Float @db.Float`, `Float` |
176
+ | `DECIMAL(p,s)`, the default `DECIMAL(65,30)` | `decimal(p,s)`, the core's narrowest width for the digits | a decimal at its precision and scale (`decimal` 38 and 18) to 65 digits and scale 30; `bigdecimal` refused | `Decimal @db.Decimal(p,s)` |
177
+ | `DATETIME(n)`, `TIMESTAMP(n)` | `datetime64(s)` at 0 digits, `ms` to 3, `us` to 6, UTC | `datetime64(unit, zone)` | `DateTime @db.DateTime(0 \| 3 \| 6)`; a nanosecond column at six, sent as a `Date` floored to the millisecond - no cast cuts it |
178
+ | `DATE`, `TIME(n)` | `date32`, `time(unit)` (`time32`/`time64`, the core's width) | `date32`, `date64`, `time32`, `time64` | `DateTime @db.Date`, `@db.Time(n)` |
179
+ | `VARCHAR(n)`, `CHAR(n)`, `TEXT` | `utf8` | text; a code (`mic`, `ccy`, `lei`, ...) and `timezone` | `String @db.Text`, the key and an indexed column `@db.VarChar(191)`; `@db.VarChar(n)`, `@db.Char(n)`; a code `@db.VarChar(64)` |
180
+ | `BINARY(16)`; `BINARY(n)`, `VARBINARY(n)`, `BLOB` | `uuid`; `fixed_binary(n)`, `sized_binary(n)`, `binary` | `uuid`; `fixed_binary`, `sized_binary`, `binary` | `Bytes @db.Binary(16)`; `@db.Binary(n)`, `@db.VarBinary(n)`, `Bytes` |
181
+ | `JSON` | `utf8`, the text; the map, struct or serie the generated model states | `map`, `struct`, `serie` | `Json` |
182
+ | `ENUM` | the kind a generated model states; else the kind its name spells (`Side` is `side`) where every member is the kind's, as the core reads them; else `utf8` | `side`, `state`, `marketdatakind`, `marketdatatype`, `timeinforce` | `enum Side { UKNW BUY ... }` from the kinds' member tables |
183
+
184
+ Under every other provider, or none, a MySQL native spelling is read as the
185
+ Prisma type's own default (`@db.TinyInt` is `int32`, `@db.Date` an instant).
186
+ `mysql/schema.prisma` is generated by `npm run mysql:schema` - `fieldModels`
187
+ over the crate's fields as the addon states them (`fix.MarketMessage.field()`,
188
+ `fix.underlyingField()`, ...) and `yggio-market/meta/kinds.json` for the
189
+ enums - never edited by hand; `scripts/mysql-schema.mjs --check` and the
190
+ `arrow` suite hold it current, and the suite holds the committed meta tables
191
+ to the same fields. Each column is read off its `DataType` - family,
192
+ precision, scale, unit, widths - never off its text. Each model is keyed by
193
+ the one column its field's `SORT:by` names (`rowKey`: `underlying` by `code`,
194
+ `mic` by `mic`, `legalentity` by `lei`), else by `STATED_KEYS` with their
195
+ reasons (`fixmsg` and `plugin` state no order, keyed by `uuid` and
196
+ `pluginid`; `fxrate`'s `crosscode, markettimestamp` is an order and no key, keyed by
197
+ `uuid`): the key a `merge` upserts by. Each model is indexed by what its
198
+ field declares (`rowIndexes`): one `@@index` over the columns its
199
+ `PARTITION:by` marks - an identity entry, never a transform such as
200
+ `months(markettimestamp)` - then the keys its `SORT:by` names - `fxrate` by
201
+ `crosscode, markettimestamp` - none where that is the key itself (`underlying`
202
+ by `code`, `fixdefinition` by `category, tag, name`); a message declares no layout, so
203
+ `STATED_INDEXES` gives `fixmsg` the three the message search reads,
204
+ `(underlyingcategory, markettimestamp)`, `(underlyingcode, markettimestamp)` and `(crosscode, markettimestamp)`.
205
+ MySQL indexes no `TEXT`, so an indexed text column is `VARCHAR(191)`, as a
206
+ key is; an index over a `JSON` or `BLOB` column is refused by name.
207
+ `mysql/prisma.config.ts` reads the server from `MYSQL_URL`.
208
+
209
+ ### Search
210
+
211
+ `searchWhere(text, underlyings, model, options)` is the core's one message
212
+ search - `fix.Underlyings.messageSearch(text, modelField(model), { registry,
213
+ limit, since, until })` - compiled by `compilePlan`, which takes the core's
214
+ `Plan` as it takes a text. A text the `uuid` column reads is that message
215
+ alone; else the branches OR-ed, each carrying its own pruning: the
216
+ underlying rows the text names, `underlyingcode in [...]` within their categories
217
+ `underlyingcategory in [...]`; the plugins `registry` names, `plugincode` or
218
+ `lastplugincode in [...]`; for a text with no blank, `crosscode` and
219
+ `underlyingcode` equalities, and `mic` where the text is a listed market; the
220
+ `like` prune over `crosscode`, `underlyingcode` and `text` only where neither an
221
+ underlying nor a plugin resolved. The window `markettimestamp >= since` and
222
+ `markettimestamp < until` (nanoseconds) bounds every branch, and the rows come
223
+ newest first (`orderBy: [{ markettimestamp: 'desc' }]`), so the three indexes
224
+ above answer each branch. MySQL's text equality folds case and pads, so the
225
+ text clauses only prune and the residual rechecks the rows natively.
226
+
227
+ ```js
228
+ const seed = yggio.fix.Underlyings.seeded()
229
+ const { args, residual } = searchWhere('AAPL', seed, model, { yggio, provider: 'mysql', since: 1_790_812_800_000_000_000n })
230
+ const pruned = await prisma.fixmsg.findMany(args)
231
+ const rows = residual?.filter === undefined ? pruned : filterRows(pruned, residual.filter, model, null, yggio)
232
+ ```
233
+
234
+ ```js
235
+ // The rows a model answered, as Arrow under the crate's field: landed under the model, cast by the core.
236
+ const model = fieldModel(yggio.fix.MarketMessage.field(), { kinds })
237
+ const reader = modelArrowReader(model, await prisma.fixmsg.findMany(), yggio, { field: yggio.fix.MarketMessage.field() })
238
+ // An Arrow source into the model: 500 rows a createMany, or one upsert per row by the key under merge.
239
+ await writeModel(prisma.fixmsg, messages.intoArrowReader(), model, yggio, { mode: 'merge', client: prisma })
240
+ ```
241
+
242
+ A refusal names the model, the rows of the batch and, in the core's words,
243
+ the column and the row (`fixmsg rows 0..500: ...price...`; a stream's
244
+ `fixmsg rows from 500: ...`). Through Prisma an instant crosses as a `Date`,
245
+ to the millisecond, both ways - the `Date` floors a nanosecond source, which
246
+ the generated model lands as it is; `writeModel` sends `skipDuplicates:
247
+ false`, so `skippedRows` is 0 against Prisma and counts only a delegate
248
+ answering a short `count`; a `DATE` or `TIME` as a `Date` too, landed as
249
+ the day or the clock. `overwrite` empties the model at its first `createMany`,
250
+ once that batch is cast and spelled, in one `$transaction` with it where a
251
+ `client` is given. `scripts/check_mysql_interop.py` at the repository root
252
+ runs the exchange against a real MySQL in both directions (the Node half
253
+ `mysql/roundtrip.test.ts`, `npm run mysql:roundtrip`), refused by name where
254
+ none is reachable. Against `MYSQL_URL` it runs `prisma db push --force-reset`,
255
+ which drops every table of that database, so it refuses a database not named
256
+ `yggio` or ending in `_yggio_interop`; the container it starts takes a plain
257
+ `db push`.
258
+
145
259
  ## Tests
146
260
 
147
261
  `npm test` generates the integration client (`prisma generate` into
148
262
  `tests/generated/`), runs every suite against the addon beside this package -
149
263
  the integration one over an embedded database file through Prisma's driver adapter - then
150
- `tsc --noEmit`.
264
+ `tsc --noEmit`. `npm run mysql:schema` regenerates `mysql/schema.prisma`,
265
+ `npm run mysql:generate` the MySQL client the exchange drives (`pretest`
266
+ generates it too, so the round trip type-checks with no server), and
267
+ `npm run mysql:roundtrip` is the exchange's Node half, which `MYSQL_URL`
268
+ must name a server for.