measurr 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +244 -53
- package/dist/analytics.d.ts +51 -14
- package/dist/dataset.d.ts +18 -0
- package/dist/handler.d.ts +9 -4
- package/dist/handler.js +54 -8
- package/dist/{index-g0ah4ehn.js → index-8nhqje4s.js} +260 -66
- package/dist/{index-rgqsrd72.js → index-vjak6aa0.js} +16 -2
- package/dist/index.d.ts +4 -4
- package/dist/index.js +2 -2
- package/dist/metrics.d.ts +152 -35
- package/dist/metrics.js +316 -83
- package/dist/plan.d.ts +3 -0
- package/dist/query.d.ts +169 -10
- package/dist/shapes.d.ts +81 -11
- package/dist/shapes.js +121 -13
- package/dist/testing.d.ts +19 -1
- package/dist/testing.js +42 -3
- package/package.json +3 -3
- package/src/analytics.ts +354 -73
- package/src/dataset.ts +49 -2
- package/src/handler.ts +38 -18
- package/src/index.ts +9 -2
- package/src/metrics.ts +646 -144
- package/src/plan.ts +64 -23
- package/src/query.ts +133 -15
- package/src/shapes.ts +293 -29
- package/src/testing.ts +64 -1
- package/dist/index-psw9s799.js +0 -46
package/README.md
CHANGED
|
@@ -159,8 +159,8 @@ With them:
|
|
|
159
159
|
the field.
|
|
160
160
|
|
|
161
161
|
`AnalyticsResultSchema` is any dataset's result without its keys: rows of `{ key, label }`
|
|
162
|
-
per grouped dimension and a number or null per measure, with totals, `previous?`, `notes
|
|
163
|
-
`truncated`. `analytics.resultSchema(key)` is the same envelope with the dataset's fields, so a
|
|
162
|
+
per grouped dimension and a number or null per measure, with totals, `previous?`, `notes`,
|
|
163
|
+
`truncated` and `fields`. `analytics.resultSchema(key)` is the same envelope with the dataset's fields, so a
|
|
164
164
|
result that passes one passes the other. Use it for a tool's output contract.
|
|
165
165
|
|
|
166
166
|
`analyticsToolGuidance` is a few markdown bullets for a model's instructions: use the tool for
|
|
@@ -176,14 +176,47 @@ ${analyticsToolGuidance}`
|
|
|
176
176
|
```
|
|
177
177
|
|
|
178
178
|
`onQuery(event, ctx)` hears every `query` once it succeeds or fails:
|
|
179
|
-
`{ dataset, durationMs, rows, outcome: 'success' | 'error', cache?, error? }`. `dataset`
|
|
180
|
-
when the input named no known dataset, so free text from a model never becomes a metric
|
|
179
|
+
`{ dataset, metric?, durationMs, rows, outcome: 'success' | 'error', cache?, error? }`. `dataset`
|
|
180
|
+
is null when the input named no known dataset, so free text from a model never becomes a metric
|
|
181
|
+
label. `metric` is the metric the registry ran the query for.
|
|
181
182
|
`cache` is `'hit'` or `'miss'` when a cache is configured. What the hook throws or rejects with is
|
|
182
183
|
swallowed; it cannot fail a query.
|
|
183
184
|
|
|
184
185
|
Each measure in `listDatasets` has a `valueType`: `count` (a whole number, never null),
|
|
185
186
|
`ratio` (0-1, from `share` and `ratio`) or `number` (sums, averages, medians), so a UI knows
|
|
186
|
-
to format a share as a percentage.
|
|
187
|
+
to format a share as a percentage. A resolved measure has the `result` it declares.
|
|
188
|
+
|
|
189
|
+
## Query options
|
|
190
|
+
|
|
191
|
+
```ts
|
|
192
|
+
const result = await analytics.query(
|
|
193
|
+
{ dataset: 'orders', measures: ['orders'], groupBy: ['day', 'status'], limit: 400, totals: false },
|
|
194
|
+
ctx,
|
|
195
|
+
{ signal: request.signal }
|
|
196
|
+
)
|
|
197
|
+
result.fields
|
|
198
|
+
// { groupBy: [{ key: 'day', label: 'Day', kind: 'timeBucket', unit: 'day' },
|
|
199
|
+
// { key: 'status', label: 'Status', kind: 'enum', values: [{ key: 'OPEN', label: 'Open' }, ...] }],
|
|
200
|
+
// measures: [{ key: 'orders', label: 'Orders', valueType: 'count' }] }
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
- `fields` describes every result: its grouped dimensions (kind, a time bucket's `unit`, a
|
|
204
|
+
closed dimension's `values`) and its measures (label, `valueType`), so a chart drawn from a
|
|
205
|
+
model's tool output needs no guessing.
|
|
206
|
+
- `totals: false` skips a grouped query's totals statement, and the result has no `totals`
|
|
207
|
+
(its type says so). The presenters do not need them. It is for app code: `querySchema` does
|
|
208
|
+
not offer it to a model, which would then add up overlapping rows.
|
|
209
|
+
- `createAnalytics({ maxLimit })` raises the highest `limit` `query` accepts, for app code such
|
|
210
|
+
as a long time series. The schema `querySchema` builds keeps 100.
|
|
211
|
+
- `createAnalytics({ concurrency })` caps how many statements run at once on each source, so a
|
|
212
|
+
large dashboard cannot take every pooled connection.
|
|
213
|
+
- `signal` aborts a query: it rejects with the signal's reason, statements waiting for a slot
|
|
214
|
+
never start, and `execute` gets `{ signal }` as its second argument to cancel a running one.
|
|
215
|
+
- `analytics.period(input, ctx)` resolves a period the way `query` does (its local days, start
|
|
216
|
+
and end instants and length, or null for all time), and `analytics.scope(ctx)` returns the
|
|
217
|
+
tenant scope `query` would use, refusing a missing one. Use them when code reads the same rows
|
|
218
|
+
another way (Prisma, another database).
|
|
219
|
+
|
|
187
220
|
|
|
188
221
|
## Caching
|
|
189
222
|
|
|
@@ -209,9 +242,17 @@ createAnalytics({
|
|
|
209
242
|
```
|
|
210
243
|
|
|
211
244
|
A stored value that is not a result of its dataset counts as a miss, and a cache that throws or
|
|
212
|
-
rejects is a miss or a skipped write, never a failed query.
|
|
213
|
-
|
|
214
|
-
|
|
245
|
+
rejects is a miss or a skipped write, never a failed query. The key also hashes the dataset's
|
|
246
|
+
definition (its compiled SQL, labels and values), so a deploy that changes a measure misses the
|
|
247
|
+
cache. Resolver code is not part of it: prefix keys with a build id (as above) if a changed
|
|
248
|
+
resolver's old numbers must not outlive a deploy. A dataset that uses `minutesAgo` is as stale as
|
|
249
|
+
the time to live.
|
|
250
|
+
|
|
251
|
+
With a cache, identical queries that run at the same time (two metrics of one dashboard, two
|
|
252
|
+
people opening it) share one run. The entry names the `metric` a registry ran the query for, so
|
|
253
|
+
`set` can pick a time to live per metric. The key holds the tenant scope and nothing else from
|
|
254
|
+
`ctx`, so a cached or shared result must not depend on anything else in it: a resolver that
|
|
255
|
+
reads the user from `ctx` belongs in an uncached instance.
|
|
215
256
|
|
|
216
257
|
## Chart shapes
|
|
217
258
|
|
|
@@ -234,20 +275,26 @@ toLeaderboard(result, { by: 'region', measure: 'revenue' }) // ranked rows with
|
|
|
234
275
|
```
|
|
235
276
|
|
|
236
277
|
- `toTimeseries` gives every bucket of the period a point; buckets without rows get `fill` (0 by
|
|
237
|
-
default, pass `null` for averages). An
|
|
238
|
-
previous period's points are on its own
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
278
|
+
default, pass `null` for averages). An ungrouped series exists even for a period without rows.
|
|
279
|
+
An all-time result has only the buckets it found. The previous period's points are on its own
|
|
280
|
+
buckets; compare them by position.
|
|
281
|
+
- `seriesKeys` (and `segmentKeys` for distributions) lists keys to draw even when no row has
|
|
282
|
+
them, a zero lane per channel, first and in that order. `'all'` takes a closed dimension's
|
|
283
|
+
values from the result's `fields`.
|
|
284
|
+
- A grouped query returns at most `limit` rows (20 by default, 100 at most, or `maxLimit`), and
|
|
285
|
+
the shape keeps `truncated`. Set a `limit` that covers every bucket and series: 30 days split
|
|
286
|
+
into three series fit in 100, 90 days into eleven do not.
|
|
242
287
|
- `toDistribution`'s `total` is the measure over every row, so many-to-many shares can add up to
|
|
243
|
-
more than 1. `toPivot` sorts numeric keys (weekdays,
|
|
288
|
+
more than 1; it is null for a result without totals. `toPivot` sorts numeric keys (weekdays,
|
|
289
|
+
hours).
|
|
244
290
|
|
|
245
291
|
### Presentation
|
|
246
292
|
|
|
247
|
-
`presentKpi`, `presentTimeseries`, `presentDistribution
|
|
248
|
-
into the wire outputs a dashboard draws (`KpiOutput`,
|
|
249
|
-
`
|
|
250
|
-
value is 0, keys are strings, and the caller picks labels, units and chart types
|
|
293
|
+
`presentKpi`, `presentTimeseries`, `presentDistribution`, `presentLeaderboard`, `presentPivot`
|
|
294
|
+
and `presentTable` turn a result into the wire outputs a dashboard draws (`KpiOutput`,
|
|
295
|
+
`TimeSeriesOutput`, `DistributionOutput`, `LeaderboardOutput`, `PivotOutput`, `TableOutput`). A
|
|
296
|
+
missing value is 0, keys are strings, and the caller picks labels, units and chart types. An
|
|
297
|
+
output whose query was cut at its `limit` has `truncated: true`:
|
|
251
298
|
|
|
252
299
|
```ts
|
|
253
300
|
presentKpi(result, 'paidShare', { label: 'Paid', percent: true })
|
|
@@ -255,32 +302,44 @@ presentKpi(result, 'paidShare', { label: 'Paid', percent: true })
|
|
|
255
302
|
|
|
256
303
|
presentTimeseries(result, {
|
|
257
304
|
bucket: 'day', measure: 'orders', series: 'status',
|
|
305
|
+
seriesKeys: 'all', // a lane for every status, rows or not
|
|
258
306
|
keys: { PAID: 'paid' }, // wire keys, by String(key); 'null' names the empty group
|
|
259
307
|
labels: { paid: 'Settled' }, // labels by wire key
|
|
260
308
|
chartType: 'line'
|
|
261
309
|
})
|
|
262
|
-
// points are { ts: Date, value }: ts is the bucket's local start date as
|
|
310
|
+
// points are { ts: Date, bucket: '2026-10-01', value }: ts is the bucket's local start date as
|
|
311
|
+
// UTC midnight, so format `bucket`, not `ts` in the viewer's timezone
|
|
263
312
|
|
|
264
313
|
presentDistribution(result, { by: 'tag', measure: 'orders', chartType: 'donut', order: ['t2'] })
|
|
265
314
|
// total is what the segments add up to, percent is 0-100; keys order omits go last
|
|
266
315
|
|
|
267
316
|
presentLeaderboard(result, { by: 'region', measure: 'orders', meta: (values) => ({ revenue: values.revenue }) })
|
|
317
|
+
|
|
318
|
+
presentPivot(result, { rows: 'weekday', columns: 'hour', measure: 'orders', totals: true })
|
|
319
|
+
// rows and columns by label; totals are sums of the cells, so leave them off for an average
|
|
320
|
+
|
|
321
|
+
presentTable(result, { columns: ['region', 'orders'], labels: { orders: 'Count' } })
|
|
322
|
+
// a column per grouped dimension (its label; a bucket's date) and per measure, from `fields`
|
|
268
323
|
```
|
|
269
324
|
|
|
270
325
|
`isKpi`, `isTimeSeries`, `isDistribution`, `isTable`, `isLeaderboard` and `isPivot` narrow a
|
|
271
|
-
`MetricOutput`.
|
|
326
|
+
`MetricOutput`. `OUTPUT_KINDS` lists every kind, and `pickRenderer(map, output, fallback)` picks
|
|
327
|
+
a renderer from a map that must name every kind (a new kind fails typecheck) and falls back for a
|
|
328
|
+
kind the client does not know yet. The module is pure, so a browser bundle can import the types
|
|
329
|
+
and guards.
|
|
272
330
|
|
|
273
331
|
## Dashboard metrics
|
|
274
332
|
|
|
275
333
|
`measurr/metrics` names the numbers a dashboard shows: one query and the output it is drawn
|
|
276
|
-
in. The registry runs a batch for a
|
|
334
|
+
in, or a `resolve` function for a number a dataset cannot give. The registry runs a batch for a
|
|
335
|
+
caller and streams one chunk per metric.
|
|
277
336
|
|
|
278
337
|
```ts
|
|
279
338
|
import { createMetricRegistry, metricDefiner } from 'measurr/metrics'
|
|
280
339
|
import { presentKpi, presentTimeseries } from 'measurr/shapes'
|
|
281
340
|
|
|
282
341
|
// Types only, so metric modules need not import the analytics instance.
|
|
283
|
-
export const defineMetric = metricDefiner<(typeof datasets)[number], Role>()
|
|
342
|
+
export const defineMetric = metricDefiner<(typeof datasets)[number], Role, {}, Ctx>()
|
|
284
343
|
|
|
285
344
|
const orderTrend = defineMetric({
|
|
286
345
|
key: 'orders.trend',
|
|
@@ -288,48 +347,123 @@ const orderTrend = defineMetric({
|
|
|
288
347
|
description: 'Orders over time',
|
|
289
348
|
allowedRoles: ['owner', 'member'],
|
|
290
349
|
supportsTimeRange: true, // false for a snapshot: all time, no previous period
|
|
291
|
-
filterSchema: z.object({ status: z.enum(['OPEN', 'PAID']).optional() }),
|
|
350
|
+
filterSchema: z.object({ status: z.enum(['OPEN', 'PAID']).optional() }), // optional
|
|
292
351
|
query: ({ bucket, filters }) => ({
|
|
293
352
|
dataset: 'orders',
|
|
294
353
|
measures: ['orders'],
|
|
295
354
|
groupBy: [bucket],
|
|
296
355
|
filters: filters.status ? [{ dimension: 'status', op: 'in', values: [filters.status] }] : [],
|
|
297
|
-
limit: 100
|
|
356
|
+
limit: 100,
|
|
357
|
+
totals: false // no presenter reads them; saves a statement per period
|
|
298
358
|
}),
|
|
299
359
|
shape: (result, { bucket }) =>
|
|
300
360
|
presentTimeseries(result, { bucket, measure: 'orders', chartType: 'bar' })
|
|
301
361
|
})
|
|
302
362
|
|
|
363
|
+
const myOpenItems = defineMetric({
|
|
364
|
+
key: 'workItems.mine',
|
|
365
|
+
kind: 'kpi',
|
|
366
|
+
description: 'Open items assigned to me',
|
|
367
|
+
supportsTimeRange: false,
|
|
368
|
+
// `ctx` is the caller; the tenant scope applies anyway.
|
|
369
|
+
query: ({ ctx }) => ({
|
|
370
|
+
dataset: 'workItems',
|
|
371
|
+
measures: ['items'],
|
|
372
|
+
filters: [{ dimension: 'assignee', op: 'in', values: [ctx.userId] }]
|
|
373
|
+
}),
|
|
374
|
+
shape: (result) => presentKpi(result, 'items')
|
|
375
|
+
})
|
|
376
|
+
|
|
377
|
+
// Prisma, another database: the registry still decides the period, the previous period and
|
|
378
|
+
// the tenant scope. Filter every read by `tenant`.
|
|
379
|
+
const seatsUsed = defineMetric({
|
|
380
|
+
key: 'billing.seats',
|
|
381
|
+
kind: 'kpi',
|
|
382
|
+
description: 'Seats in use',
|
|
383
|
+
supportsTimeRange: true,
|
|
384
|
+
resolve: async ({ period, tenant, signal }) => ({
|
|
385
|
+
kind: 'kpi',
|
|
386
|
+
value: await countSeats({ organizationIds: tenant, from: period?.start, to: period?.end, signal })
|
|
387
|
+
})
|
|
388
|
+
})
|
|
389
|
+
|
|
303
390
|
export const metrics = createMetricRegistry({
|
|
304
391
|
analytics,
|
|
305
|
-
metrics: [orderTrend,
|
|
306
|
-
canRun: (metric, ctx) =>
|
|
307
|
-
|
|
392
|
+
metrics: [orderTrend, myOpenItems, seatsUsed] as const,
|
|
393
|
+
canRun: (metric, ctx) => (metric.allowedRoles ?? ALL_ROLES).includes(ctx.role),
|
|
394
|
+
onError: (error, { key, code }) => logger.error('metric.failed', { key, code, error })
|
|
308
395
|
})
|
|
309
396
|
|
|
310
397
|
const input = metrics.requestSchema.extend({ refresh: z.boolean().default(false) }).parse(body)
|
|
311
|
-
for await (const chunk of metrics.run(input, ctx)) send(chunk)
|
|
312
|
-
// { key, requestKey, result: { current, previous?, supportsTimeRange }, done }
|
|
313
|
-
// or { key, requestKey, error, done }
|
|
398
|
+
for await (const chunk of metrics.run(input, ctx, { signal: request.signal })) send(chunk)
|
|
399
|
+
// { key, requestKey, result: { current, previous?, supportsTimeRange, period }, done }
|
|
400
|
+
// or { key, requestKey, error, code, done }
|
|
401
|
+
|
|
402
|
+
const { results, errors } = await metrics.collect(input, ctx) // by request key, typed per key
|
|
403
|
+
const seats = await metrics.one('billing.seats', { period: { last: { days: 7 } } }, ctx)
|
|
314
404
|
```
|
|
315
405
|
|
|
316
|
-
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
406
|
+
- A request names its period like a query, `period: { last: { days: 7 } }` or any other
|
|
407
|
+
`PeriodInput`, counted in `analytics.timezone(ctx)`, so a client that does not know the
|
|
408
|
+
tenant's timezone can still ask for "the last 7 days". Or it sends `from` and `to` as
|
|
409
|
+
instants, counted as the local days they fall on; `from` alone counts up to today. `to`
|
|
410
|
+
without `from`, or `period` with either, rejects with `invalid_period`. Without any, a metric
|
|
411
|
+
counts all time; a snapshot always does.
|
|
412
|
+
- Every result carries the `period` it counted: `{ from, to, timezone }` or `{ all: true,
|
|
413
|
+
timezone }`, so a chart can mark today without a second request.
|
|
414
|
+
- `query` returns the query without `period` or `compareToPrevious`. `compareToPrevious` adds
|
|
415
|
+
the previous period, whose series points are put on the current buckets so a chart overlays
|
|
416
|
+
them. With week or month buckets the two periods can differ by a bucket: points pair from the
|
|
417
|
+
end, and a previous point before the current period's first bucket is dropped.
|
|
418
|
+
- `granularity` (hour to year, inferred from the period's days when omitted: day up to 31
|
|
419
|
+
days, week up to 90, else month) picks `run.bucket`: `day`, `week` or `month`; `hour` draws
|
|
420
|
+
days and `quarter` and `year` draw months, and the output's `granularity` says which. Name
|
|
421
|
+
time-bucket dimensions after them so `groupBy: [bucket]` works.
|
|
422
|
+
- `query` and `shape` get `run.ctx`, and `metricDefiner`'s fourth parameter types it. A filter
|
|
423
|
+
on the caller's own id is part of the query, so the cache key already separates users.
|
|
424
|
+
- A `resolve` metric gets the filters, bucket and granularity, the `period` (local days with
|
|
425
|
+
`start` and `end` instants, or null for all time), the `tenant` from `analytics.scope(ctx)`
|
|
426
|
+
(`allTenants` included), `ctx` and a `signal`. It is called once for the period and, with
|
|
427
|
+
`compareToPrevious`, once for the previous one. Nothing scopes its reads but the code: filter
|
|
428
|
+
every read by `tenant`.
|
|
429
|
+
- A metric fails alone, with a `code` next to its message: `unknown_metric`, `forbidden`
|
|
430
|
+
(`canRun` refused it), `invalid_filters`, `truncated`, `shape_failed`, `internal` (a driver
|
|
431
|
+
error or a throwing `resolve`; the message does not say what), or the engine's codes such as
|
|
432
|
+
`missing_tenant`. `onError(error, { key, requestKey, code, metric }, ctx)` hears every
|
|
433
|
+
failure with what was thrown; `onShapeError` still works and is deprecated.
|
|
434
|
+
- A time series whose query was cut at its `limit` fails with `truncated` instead of drawing
|
|
435
|
+
zeros for the newest buckets. Raise its `limit` (and `maxLimit`) or ask for a shorter period.
|
|
436
|
+
- `filterSchema` is optional; without it a metric takes no filters.
|
|
437
|
+
- `metricDefiner<Datasets, Role, Extra, Ctx>()` takes the app's own definition fields as
|
|
438
|
+
`Extra` (catalog metadata, UI hints for filters); the registry carries them to `canRun` and
|
|
328
439
|
`available` without reading them.
|
|
329
|
-
- Keys and outputs stay literal: a chunk narrows to its metric by `key`, and
|
|
330
|
-
`MetricOutputOf<typeof metrics, 'orders.trend'>`,
|
|
331
|
-
`registry.get(key)` are typed per metric.
|
|
332
|
-
allows, for a catalog.
|
|
440
|
+
- Keys and outputs stay literal: a chunk narrows to its metric by `key`, `collect`'s results and
|
|
441
|
+
`one` are typed per key, and `MetricOutputOf<typeof metrics, 'orders.trend'>`,
|
|
442
|
+
`MetricFiltersOf`, `MetricKeysOfKind` and `registry.get(key)` are typed per metric.
|
|
443
|
+
`registry.available(ctx)` lists what `canRun` allows, for a catalog.
|
|
444
|
+
- A consumer that stops reading `run` (a closed stream) aborts the statements still running;
|
|
445
|
+
`signal` does the same from outside.
|
|
446
|
+
- `metricOutputSchemas` (one per kind), `MetricOutputSchema` and `MetricResultSchema` are Zod
|
|
447
|
+
schemas for an endpoint's output contract or a cached output; `ts` is coerced, so JSON parses
|
|
448
|
+
back.
|
|
449
|
+
|
|
450
|
+
**Access belongs to `ctx`.** `canRun` decides which metrics a caller may run, and `tenant(ctx)`
|
|
451
|
+
decides what any of them can count. Keep both in one discriminated context and never add a
|
|
452
|
+
bypass flag:
|
|
453
|
+
|
|
454
|
+
```ts
|
|
455
|
+
type Ctx =
|
|
456
|
+
| { kind: 'member'; organizationId: OrganizationId; role: Role }
|
|
457
|
+
| { kind: 'staff'; organizationIds: readonly OrganizationId[] | typeof allTenants; role: 'staff' }
|
|
458
|
+
|
|
459
|
+
canRun: (metric, ctx) => (metric.allowedRoles ?? ALL_ROLES).includes(ctx.role),
|
|
460
|
+
tenant: (ctx) => (ctx.kind === 'member' ? ctx.organizationId : ctx.organizationIds)
|
|
461
|
+
```
|
|
462
|
+
|
|
463
|
+
A job that reports for one organization (a monthly PDF, an email) is a caller too: build a
|
|
464
|
+
`ctx` scoped to that organization, the same one a member of it would get, rather than a staff
|
|
465
|
+
context with every tenant. A metric that forgets a scope then counts too little, never another
|
|
466
|
+
customer's rows.
|
|
333
467
|
|
|
334
468
|
## Serving queries
|
|
335
469
|
|
|
@@ -340,7 +474,9 @@ for await (const chunk of metrics.run(input, ctx)) send(chunk)
|
|
|
340
474
|
others; an error other than `AnalyticsError` is reported as `{ code: 'internal' }` without
|
|
341
475
|
its message.
|
|
342
476
|
- `streamBatch(analytics, queries, ctx)` yields `{ key, result }` or `{ key, error }` as each
|
|
343
|
-
query finishes, for a streaming endpoint.
|
|
477
|
+
query finishes, for a streaming endpoint. A consumer that stops reading aborts the statements
|
|
478
|
+
still running.
|
|
479
|
+
- Both take `{ signal }` as a fourth argument; aborting it rejects with the signal's reason.
|
|
344
480
|
- `toHttpError(error)` maps an `AnalyticsError` to `{ status, code, message, data? }`:
|
|
345
481
|
`forbidden` is 403 `FORBIDDEN`, `missing_tenant` 401 `UNAUTHORIZED`, the rest 400
|
|
346
482
|
`BAD_REQUEST` with the code and details. The codes are the names oRPC and tRPC use:
|
|
@@ -472,8 +608,8 @@ before any statement runs; a narrower scope is never ignored. `listDatasets(ctx)
|
|
|
472
608
|
| Dataset | One fact table: tenant column (or `null`), optional scope, time fields, default period, measures, dimensions, source. |
|
|
473
609
|
| Measure | `count`, `countWhere`, `countDistinct`, `sum`, `avg`, `median`, `share` (one condition over another), `ratio` (one measure over another, after aggregation) or `resolved` (computed by the app after the query runs). Optionally restricted with `access`. |
|
|
474
610
|
| Dimension | `enum`, `computed` (closed value sets), `relation` (to-one), `manyToMany`, `number`, `timeBucket` (day, ISO week, month), `timePart` (ISO weekday, hour). Optionally restricted with `access`. |
|
|
475
|
-
| Query | `dataset`, 1-4 `measures`, 0-2 `groupBy`, `filters`, `period`, `time`, `compareToPrevious`, `sort` (by an asked measure or grouped dimension), `limit` (1-100
|
|
476
|
-
| Result | `{ period, rows, totals, previous?, notes, truncated }`; `period` is `{ from, to, timezone }` or `{ all: true, timezone }`. |
|
|
611
|
+
| Query | `dataset`, 1-4 `measures`, 0-2 `groupBy`, `filters`, `period`, `time`, `compareToPrevious`, `sort` (by an asked measure or grouped dimension), `limit` (1-100 or `maxLimit`, default 20), and for app code `totals`. |
|
|
612
|
+
| Result | `{ period, rows, totals, previous?, notes, truncated, fields }`; `period` is `{ from, to, timezone }` or `{ all: true, timezone }`. `previous` is typed as present when the query says `compareToPrevious: true`. |
|
|
477
613
|
|
|
478
614
|
**Operators per dimension kind.** Closed and open dimensions take `in`, `notIn`, `isEmpty`
|
|
479
615
|
and `isNotEmpty`; numeric and time dimensions take `between`, `isEmpty` and `isNotEmpty`. The
|
|
@@ -485,6 +621,18 @@ keys or names (case-insensitive); the engine resolves them before planning. An u
|
|
|
485
621
|
fails with `AnalyticsError('unknown_value')` listing the valid values, so a model can retry
|
|
486
622
|
once.
|
|
487
623
|
|
|
624
|
+
The lookup reads the fact table of the tenant by default, every row it ever had. Give the
|
|
625
|
+
dimension a `lookup` to read the names from their own table instead; the key and the name must
|
|
626
|
+
be columns of it, and `tenantColumn` scopes it (null for a table every tenant shares). The
|
|
627
|
+
valid values are then every entry of that table, not only those with rows.
|
|
628
|
+
|
|
629
|
+
```ts
|
|
630
|
+
team: dimension.relation({
|
|
631
|
+
label: 'Team', key: team.col('id'), name: team.col('name'),
|
|
632
|
+
lookup: { tenantColumn: team.col('organizationId') }
|
|
633
|
+
})
|
|
634
|
+
```
|
|
635
|
+
|
|
488
636
|
**Errors carry typed details.** `isAnalyticsError(error, code?)` narrows by code:
|
|
489
637
|
`unknown_value` details are `{ dimension, unknown, valid }` and `invalid_query` details are the
|
|
490
638
|
schema's issues; the other codes have none.
|
|
@@ -521,7 +669,15 @@ every unit. The generated schema states each dataset's default, so a model knows
|
|
|
521
669
|
|
|
522
670
|
**Totals are a separate, ungrouped statement**, so a many-to-many grouping (an order with
|
|
523
671
|
two tags appears in two rows) never inflates them. Filters on many-to-many dimensions compile
|
|
524
|
-
to `EXISTS`, so they never multiply rows either.
|
|
672
|
+
to `EXISTS`, so they never multiply rows either. A grouped query is therefore two statements
|
|
673
|
+
per period, four with `compareToPrevious`, plus one per open-dimension label lookup; `totals:
|
|
674
|
+
false` drops the totals statements.
|
|
675
|
+
|
|
676
|
+
**Daily rollups cannot see part of a day.** A dataset over a table with one row per day (usage
|
|
677
|
+
per unit per day) counts whole local days. It cannot start a unit at the hour it joined or end a
|
|
678
|
+
period mid-day, so a total read from raw events with exact instants can differ on a unit's first
|
|
679
|
+
day and at the edges of the period. Filter such rows by day (`day >= joinedDay`) and say so in
|
|
680
|
+
the dataset's description, or compute the exact number in a `resolve` metric.
|
|
525
681
|
|
|
526
682
|
**Result types are honest.** `count` measures are `number`; `sum`, `avg` and `median` are
|
|
527
683
|
`number | null`; `share` and `ratio` are a branded `Ratio | null` (0-1). A row has a field only
|
|
@@ -592,6 +748,27 @@ const main: AnalyticsSource = {
|
|
|
592
748
|
}
|
|
593
749
|
```
|
|
594
750
|
|
|
751
|
+
Each statement there is an interactive transaction: five round trips (begin, two `SET`s, the
|
|
752
|
+
query, commit). One round trip does the same when the connection itself is read-only with a
|
|
753
|
+
timeout, for example a role made for analytics:
|
|
754
|
+
|
|
755
|
+
```sql
|
|
756
|
+
CREATE ROLE analytics_reader LOGIN PASSWORD '...';
|
|
757
|
+
GRANT SELECT ON ALL TABLES IN SCHEMA public TO analytics_reader;
|
|
758
|
+
ALTER ROLE analytics_reader SET default_transaction_read_only = on;
|
|
759
|
+
ALTER ROLE analytics_reader SET statement_timeout = '5s';
|
|
760
|
+
```
|
|
761
|
+
|
|
762
|
+
```ts
|
|
763
|
+
const analyticsDb = new PrismaClient({ datasourceUrl: process.env.ANALYTICS_DATABASE_URL })
|
|
764
|
+
const main: AnalyticsSource = {
|
|
765
|
+
dialect: postgresDialect({ timestamps: 'utc' }),
|
|
766
|
+
execute: ({ text, values }) => analyticsDb.$queryRawUnsafe(text, ...values)
|
|
767
|
+
}
|
|
768
|
+
```
|
|
769
|
+
|
|
770
|
+
`concurrency` on `createAnalytics` keeps a dashboard from taking every connection of the pool.
|
|
771
|
+
|
|
595
772
|
**`measurr/clickhouse`** `clickhouseDialect()`: typed `{pN:Type}` parameters, backquoted
|
|
596
773
|
identifiers, `-If` and `-OrNull` aggregates (an empty group is null, as in Postgres), an exact
|
|
597
774
|
interpolated median, `uniqExact` for distinct counts, and buckets with `toTimeZone`. Every
|
|
@@ -652,6 +829,17 @@ list are refused with `forbidden`. Every other check runs as usual.
|
|
|
652
829
|
await assertDatasetContract({ dataset: maintenance, source, tenant: allTenants })
|
|
653
830
|
```
|
|
654
831
|
|
|
832
|
+
`assertMetricContract({ registry, ctx })` runs every metric of a registry once over the last 30
|
|
833
|
+
days (or `period`), comparing with the previous period, and fails when one errors, draws
|
|
834
|
+
another kind than it declares, or a metric over a time range has no previous period. Give it a
|
|
835
|
+
`ctx` that `canRun` allows everything for, scoped to a seeded tenant.
|
|
836
|
+
|
|
837
|
+
```ts
|
|
838
|
+
import { assertMetricContract } from 'measurr/testing'
|
|
839
|
+
|
|
840
|
+
await assertMetricContract({ registry: metrics, ctx: seededOwner })
|
|
841
|
+
```
|
|
842
|
+
|
|
655
843
|
## Extending
|
|
656
844
|
|
|
657
845
|
- **A measure or dimension** is one entry in a dataset. Dimensions are groupable and
|
|
@@ -677,13 +865,16 @@ Kept stable for publishing: `createAnalytics` and its options (with `authorize`,
|
|
|
677
865
|
`Dialect`, `SqlRenderer` and `AnalyticsSource` contracts; `measure.resolved` with the
|
|
678
866
|
`resolvers` option (`MeasureResolver`, `ResolverInput`, `ResolverOutput`); `postgresDialect`,
|
|
679
867
|
`clickhouseDialect` and `toQueryParams`,
|
|
680
|
-
`checkDatasetContract` / `assertDatasetContract
|
|
868
|
+
`checkDatasetContract` / `assertDatasetContract`, `analytics.period` and `analytics.scope`,
|
|
869
|
+
the query options (`totals`, `signal`) and the `maxLimit` and `concurrency` options, result
|
|
870
|
+
`fields`, dimension `lookup`; `measurr/shapes` (`toKpi`, `toTimeseries`,
|
|
681
871
|
`toDistribution`, `toPivot`, `toLeaderboard`) and `measurr/handler` (`queryBatch`,
|
|
682
872
|
`streamBatch`, `toHttpError`); and the types `AnalyticsQuery`, `AnalyticsResult`,
|
|
683
873
|
`ResultPeriod`, `PeriodInput`, `Tenant`, `TenantId`, `AccessKey`, `AnalyticsCache`, `FieldKey`,
|
|
684
874
|
`Ratio`, `AnalyticsError` codes and their details (`isAnalyticsError`). `ExprNode` and
|
|
685
875
|
`SelectStatement` are exported for dialect authors and may grow new node kinds. The
|
|
686
|
-
presenters and wire outputs in `measurr/shapes`, `measurr/metrics`
|
|
876
|
+
presenters and wire outputs in `measurr/shapes`, `measurr/metrics` (with `resolve` metrics and
|
|
877
|
+
the output schemas), `checkMetricContract` / `assertMetricContract` and `analytics.timezone`
|
|
687
878
|
are new and may still change before they join this list.
|
|
688
879
|
|
|
689
880
|
## Install
|
|
@@ -692,7 +883,7 @@ are new and may still change before they join this list.
|
|
|
692
883
|
npm install measurr zod
|
|
693
884
|
```
|
|
694
885
|
|
|
695
|
-
`zod` 4 is a peer dependency. The types need TypeScript
|
|
886
|
+
`zod` 4 is a peer dependency. The types need TypeScript 5.9 or newer.
|
|
696
887
|
|
|
697
888
|
## Development
|
|
698
889
|
|
package/dist/analytics.d.ts
CHANGED
|
@@ -2,13 +2,26 @@ import { z } from 'zod';
|
|
|
2
2
|
import type { CompiledStatement, Dialect } from './compile.js';
|
|
3
3
|
import type { AnyDataset, AnyDimension, Dataset, DimensionValue, MeasureValue, ResolvedMeasure } from './dataset.js';
|
|
4
4
|
import { type NormalizedFilter, type Tenant } from './plan.js';
|
|
5
|
-
import { type
|
|
5
|
+
import { type PeriodInput, type ResolvedPeriod } from './period.js';
|
|
6
|
+
import { type DatasetQuery, type QueryResult, type ResultOf, type ResultPeriod, type ResultWithoutTotals } from './query.js';
|
|
6
7
|
export type Row = Record<string, unknown>;
|
|
7
8
|
/** A database: a dialect to compile with and a function that runs the compiled SQL. */
|
|
8
9
|
export type AnalyticsSource = {
|
|
9
10
|
readonly dialect: Dialect;
|
|
10
|
-
/**
|
|
11
|
-
|
|
11
|
+
/**
|
|
12
|
+
* Runs one read-only statement and returns its rows as plain objects. `signal` aborts when
|
|
13
|
+
* the caller gives up (a closed stream), so a driver that supports it can cancel.
|
|
14
|
+
*/
|
|
15
|
+
execute(statement: CompiledStatement, options?: {
|
|
16
|
+
signal?: AbortSignal | undefined;
|
|
17
|
+
}): Promise<readonly Row[]>;
|
|
18
|
+
};
|
|
19
|
+
/** Per call: a signal that cancels the query, and the metric it runs for. */
|
|
20
|
+
export type QueryOptions = {
|
|
21
|
+
/** Aborting rejects the query with the signal's reason and stops its statements. */
|
|
22
|
+
signal?: AbortSignal | undefined;
|
|
23
|
+
/** The metric the query runs for; `onQuery` and the cache hear it. */
|
|
24
|
+
metric?: string | undefined;
|
|
12
25
|
};
|
|
13
26
|
export type AnalyticsErrorCode = 'invalid_query' | 'forbidden' | 'missing_tenant' | 'invalid_timezone' | 'invalid_period' | 'unknown_value';
|
|
14
27
|
/** What `details` holds for each error code. */
|
|
@@ -46,6 +59,8 @@ export declare function isAnalyticsError<C extends AnalyticsErrorCode = Analytic
|
|
|
46
59
|
export type AnalyticsQueryEvent = {
|
|
47
60
|
/** The dataset asked for; null when the input named no known dataset. */
|
|
48
61
|
dataset: string | null;
|
|
62
|
+
/** The metric the query ran for, when a metric registry ran it. */
|
|
63
|
+
metric?: string;
|
|
49
64
|
durationMs: number;
|
|
50
65
|
/** Rows returned; 0 on error. */
|
|
51
66
|
rows: number;
|
|
@@ -132,6 +147,16 @@ export type AnalyticsOptions<DS extends readonly AnyDataset[], Ctx> = {
|
|
|
132
147
|
timezone?: ((ctx: Ctx) => string | null | undefined | Promise<string | null | undefined>) | undefined;
|
|
133
148
|
/** The clock periods and `minutesAgo` resolve against. */
|
|
134
149
|
now?: (() => Date) | undefined;
|
|
150
|
+
/**
|
|
151
|
+
* The highest `limit` `query` accepts, for app code such as a long time series. The schemas
|
|
152
|
+
* `querySchema` gives a model keep 100. Defaults to 100.
|
|
153
|
+
*/
|
|
154
|
+
maxLimit?: number | undefined;
|
|
155
|
+
/**
|
|
156
|
+
* How many statements run at once on each source, so a large dashboard cannot take every
|
|
157
|
+
* pooled connection. Unlimited when omitted.
|
|
158
|
+
*/
|
|
159
|
+
concurrency?: number | undefined;
|
|
135
160
|
/**
|
|
136
161
|
* Called once per `query`, after it succeeds or fails: for metrics and logs. What it
|
|
137
162
|
* throws or rejects with is swallowed, so it can never fail a query.
|
|
@@ -139,7 +164,8 @@ export type AnalyticsOptions<DS extends readonly AnyDataset[], Ctx> = {
|
|
|
139
164
|
onQuery?: ((event: AnalyticsQueryEvent, ctx: Ctx) => unknown) | undefined;
|
|
140
165
|
/**
|
|
141
166
|
* Where finished results are kept and looked up. What it throws or rejects with is
|
|
142
|
-
* swallowed: a failing cache is a miss, never a failed query.
|
|
167
|
+
* swallowed: a failing cache is a miss, never a failed query. With a cache, identical
|
|
168
|
+
* queries that run at the same time also share one run.
|
|
143
169
|
*/
|
|
144
170
|
cache?: AnalyticsCache<Ctx, DS[number]['key']> | undefined;
|
|
145
171
|
/**
|
|
@@ -162,15 +188,16 @@ export type AnalyticsCache<Ctx, Key extends string = string> = {
|
|
|
162
188
|
* caller (a refresh button); its fresh result is still stored. A stored value that is not a
|
|
163
189
|
* result of this dataset counts as a miss, so a changed definition does not break queries.
|
|
164
190
|
*/
|
|
165
|
-
get(key: string, entry:
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
191
|
+
get(key: string, entry: CacheEntry<Ctx, Key>): unknown;
|
|
192
|
+
/** Stores a result; pick the time to live per `dataset` (or per `metric`). */
|
|
193
|
+
set(key: string, result: QueryResult<Row, Row> | ResultWithoutTotals<Row>, entry: CacheEntry<Ctx, Key>): unknown;
|
|
194
|
+
};
|
|
195
|
+
/** What the cache is told about a result besides its key. */
|
|
196
|
+
export type CacheEntry<Ctx, Key extends string = string> = {
|
|
197
|
+
dataset: Key;
|
|
198
|
+
ctx: Ctx;
|
|
199
|
+
/** The metric the query runs for, when a metric registry runs it. */
|
|
200
|
+
metric?: string;
|
|
174
201
|
};
|
|
175
202
|
export type DatasetCatalogEntry<Key extends string = string> = {
|
|
176
203
|
key: Key;
|
|
@@ -204,7 +231,7 @@ export type DatasetCatalogEntry<Key extends string = string> = {
|
|
|
204
231
|
export type Analytics<DS extends readonly AnyDataset[], Ctx> = {
|
|
205
232
|
readonly datasets: DS;
|
|
206
233
|
/** Parses, authorizes, plans, compiles, runs and shapes one query. */
|
|
207
|
-
query<const Q extends DatasetQuery<DS[number]>>(input: Q, ctx: Ctx): Promise<ResultOf<DS[number], Q>>;
|
|
234
|
+
query<const Q extends DatasetQuery<DS[number]>>(input: Q, ctx: Ctx, options?: QueryOptions): Promise<ResultOf<DS[number], Q>>;
|
|
208
235
|
/** The compiled statements a query would run, without running them or resolving labels. */
|
|
209
236
|
explain(input: DatasetQuery<DS[number]>, ctx: Ctx): Promise<CompiledStatement[]>;
|
|
210
237
|
/**
|
|
@@ -231,6 +258,16 @@ export type Analytics<DS extends readonly AnyDataset[], Ctx> = {
|
|
|
231
258
|
* when it has none. Rejects with `invalid_timezone` for a name the runtime does not know.
|
|
232
259
|
*/
|
|
233
260
|
timezone(ctx: Ctx): Promise<string>;
|
|
261
|
+
/**
|
|
262
|
+
* A period in `ctx`'s timezone: its local days and the instants they start and end at, or
|
|
263
|
+
* null for all time. Rejects with `invalid_period` as `query` would.
|
|
264
|
+
*/
|
|
265
|
+
period(input: PeriodInput, ctx: Ctx): Promise<ResolvedPeriod | null>;
|
|
266
|
+
/**
|
|
267
|
+
* The tenant scope every query for `ctx` gets, for code that reads the same rows another
|
|
268
|
+
* way (Prisma, another database). Rejects with `missing_tenant` as `query` would.
|
|
269
|
+
*/
|
|
270
|
+
scope(ctx: Ctx): Tenant;
|
|
234
271
|
/** The datasets `ctx` may query, with the measures and dimensions it may use. */
|
|
235
272
|
listDatasets(ctx: Ctx): Promise<DatasetCatalogEntry<DS[number]['key']>[]>;
|
|
236
273
|
};
|
package/dist/dataset.d.ts
CHANGED
|
@@ -162,11 +162,18 @@ export type EnumDimension<K extends string = string, Empty extends null = null>
|
|
|
162
162
|
readonly values: Readonly<Record<K, string>>;
|
|
163
163
|
readonly empty: string | undefined;
|
|
164
164
|
};
|
|
165
|
+
/** Where an open dimension's values are looked up: their own table, scoped by its column. */
|
|
166
|
+
export type DimensionLookup = {
|
|
167
|
+
readonly table: TableRef;
|
|
168
|
+
/** Null for a table every tenant shares. */
|
|
169
|
+
readonly tenantColumn: ExprNode | null;
|
|
170
|
+
};
|
|
165
171
|
export type RelationDimension<K = unknown> = DimensionBase<'open', K> & {
|
|
166
172
|
readonly kind: 'relation';
|
|
167
173
|
readonly key: ExprNode;
|
|
168
174
|
readonly name: ExprNode;
|
|
169
175
|
readonly empty: string | undefined;
|
|
176
|
+
readonly lookup: DimensionLookup | undefined;
|
|
170
177
|
};
|
|
171
178
|
export type ManyToManyDimension = DimensionBase<'open', string> & {
|
|
172
179
|
readonly kind: 'manyToMany';
|
|
@@ -176,6 +183,7 @@ export type ManyToManyDimension = DimensionBase<'open', string> & {
|
|
|
176
183
|
readonly name: ExprNode;
|
|
177
184
|
readonly empty: string | undefined;
|
|
178
185
|
readonly note: string | undefined;
|
|
186
|
+
readonly lookup: DimensionLookup | undefined;
|
|
179
187
|
};
|
|
180
188
|
export type NumberDimension = DimensionBase<'numeric', number> & {
|
|
181
189
|
readonly kind: 'number';
|
|
@@ -226,6 +234,14 @@ type EmptyOption = {
|
|
|
226
234
|
/** The label of rows whose value is null. "None" when omitted. */
|
|
227
235
|
empty?: string | undefined;
|
|
228
236
|
};
|
|
237
|
+
/**
|
|
238
|
+
* Reads a filter's names from the dimension's own table instead of from every fact row of
|
|
239
|
+
* the tenant. `tenantColumn` scopes that table; null when every tenant shares it. The key and
|
|
240
|
+
* the name must be columns of that table.
|
|
241
|
+
*/
|
|
242
|
+
type LookupOption = {
|
|
243
|
+
tenantColumn: Expr<SqlType> | null;
|
|
244
|
+
} | undefined;
|
|
229
245
|
export declare const dimension: {
|
|
230
246
|
/**
|
|
231
247
|
* A column holding one value of a closed set, such as a Prisma enum. `labels` must name
|
|
@@ -254,6 +270,7 @@ export declare const dimension: {
|
|
|
254
270
|
key: Expr<Key extends number ? 'number' : 'string', Key | null>;
|
|
255
271
|
name: Expr<'string'>;
|
|
256
272
|
empty?: string | undefined;
|
|
273
|
+
lookup?: LookupOption;
|
|
257
274
|
}): RelationDimension<Key> & DimensionAccess<G, F, NoInfer<Access>>;
|
|
258
275
|
/**
|
|
259
276
|
* A to-many relation through a link table, such as tags. Grouping counts a row once per
|
|
@@ -273,6 +290,7 @@ export declare const dimension: {
|
|
|
273
290
|
/** Explains the overlap to the reader of a grouped result. */
|
|
274
291
|
note?: string | undefined;
|
|
275
292
|
empty?: string | undefined;
|
|
293
|
+
lookup?: LookupOption;
|
|
276
294
|
}): ManyToManyDimension & DimensionAccess<G, F, NoInfer<Access>>;
|
|
277
295
|
/** A numeric column, grouped by its exact values and filtered with `between`. */
|
|
278
296
|
number<G extends boolean = boolean, F extends boolean = boolean, Access extends string = never>(expr: Expr<'number'>, options: DimensionOptions<G, F, Access> & {
|