measurr 0.4.0 → 0.5.0

package/README.md

@@ -176,9 +176,95 @@ ${analyticsToolGuidance}`
 ```
 
 `onQuery(event, ctx)` hears every `query` once it succeeds or fails:
-`{ dataset, durationMs, rows, outcome: 'success' | 'error', error? }`. `dataset` is null when
-the input named no known dataset, so free text from a model never becomes a metric label.
-What the hook throws or rejects with is swallowed; it cannot fail a query.
+`{ dataset, durationMs, rows, outcome: 'success' | 'error', cache?, error? }`. `dataset` is null
+when the input named no known dataset, so free text from a model never becomes a metric label.
+`cache` is `'hit'` or `'miss'` when a cache is configured. What the hook throws or rejects with is
+swallowed; it cannot fail a query.
+
+Each measure in `listDatasets` has a `valueType`: `count` (a whole number, never null),
+`ratio` (0-1, from `share` and `ratio`) or `number` (sums, averages, medians), so a UI knows
+to format a share as a percentage.
+
+## Caching
+
+`cache: { get, set }` keeps finished results in a store of your choice, such as Redis. A query
+is parsed and authorized first, so a cached result never reaches a caller who may not run the
+query. The key is `measurr:<dataset>:<sha256>`, hashed from the query with its defaults, the
+tenant scope, the timezone and the resolved days (so `last 30 days` moves on at local midnight).
+Results are plain JSON.
+
+```ts
+const TTL_SECONDS: Record<AnalyticsDatasetKey, number> = { orders: 300, units: 60 }
+
+createAnalytics({
+	// ...
+	cache: {
+		// A miss bypasses the cache; the fresh result is still stored.
+		get: async (key, { ctx }) =>
+			ctx.refresh ? null : JSON.parse((await redis.get(`${BUILD_ID}:${key}`)) ?? 'null'),
+		set: (key, result, { dataset }) =>
+			redis.setex(`${BUILD_ID}:${key}`, TTL_SECONDS[dataset], JSON.stringify(result))
+	}
+})
+```
+
+A stored value that is not a result of its dataset counts as a miss, and a cache that throws or
+rejects is a miss or a skipped write, never a failed query. A changed measure keeps its key, so
+prefix keys with a build id (as above) if old numbers must not outlive a deploy. A dataset that
+uses `minutesAgo` is as stale as the time to live.
+
+## Chart shapes
+
+`measurr/shapes` turns a result into what a chart or a KPI card draws. The functions are pure
+and add no numbers the result does not hold; with `compareToPrevious` they carry the previous
+period too.
+
+```ts
+import { toDistribution, toKpi, toPivot, toTimeseries, toLeaderboard } from 'measurr/shapes'
+
+toKpi(result, 'orders')
+// { kind: 'kpi', value: 12, previous: 8, change: 0.5 }
+
+toTimeseries(result, { bucket: 'day', unit: 'day', measure: 'orders', series: 'status' })
+// { kind: 'timeseries', unit, series: [{ key, label, points: [{ bucket: '2026-10-01', value }], previous? }] }
+
+toDistribution(result, { by: 'status', measure: 'orders' }) // segments with share of the total
+toPivot(result, { rows: 'weekday', columns: 'hour', measure: 'orders' }) // values[row][column]
+toLeaderboard(result, { by: 'region', measure: 'revenue' }) // ranked rows with every measure
+```
+
+- `toTimeseries` gives every bucket of the period a point; buckets without rows get `fill` (0 by
+  default, pass `null` for averages). An all-time result has only the buckets it found. The
+  previous period's points are on its own buckets; compare them by position.
+- A grouped query returns at most `limit` rows (20 by default, 100 at most), and the shape keeps
+  `truncated`. Set `limit: 100` for a series; 30 days split into three series fit, 90 days into
+  eleven do not.
+- `toDistribution`'s `total` is the measure over every row, so many-to-many shares can add up to
+  more than 1. `toPivot` sorts numeric keys (weekdays, hours).
+
+## Serving queries
+
+`measurr/handler` serves queries from any server framework:
+
+- `queryBatch(analytics, { total: q1, byStatus: q2 }, ctx)` runs named queries in parallel and
+  resolves to `{ results, errors }`, typed per name. One failing query does not fail the
+  others; an error other than `AnalyticsError` is reported as `{ code: 'internal' }` without
+  its message.
+- `streamBatch(analytics, queries, ctx)` yields `{ key, result }` or `{ key, error }` as each
+  query finishes, for a streaming endpoint.
+- `toHttpError(error)` maps an `AnalyticsError` to `{ status, code, message, data? }`:
+  `forbidden` is 403 `FORBIDDEN`, `missing_tenant` 401 `UNAUTHORIZED`, the rest 400
+  `BAD_REQUEST` with the code and details. The codes are the names oRPC and tRPC use:
+
+```ts
+try {
+	return await analytics.query(input, ctx)
+} catch (error) {
+	if (!(error instanceof AnalyticsError)) throw error
+	const { code, message, data } = toHttpError(error)
+	throw new ORPCError(code, { message, data })
+}
+```
 
 ## Tenants and platform admins
 
@@ -420,16 +506,18 @@ await assertDatasetContract({ dataset: maintenance, source, tenant: allTenants }
 
 ## Stable API
 
-Kept stable for publishing: `createAnalytics` and its options (with `authorize` and
-`authorizeField`), `allTenants`, `analytics.query`, `explain`, `querySchema`, `resultSchema`,
-`listDatasets`; `AnalyticsResultSchema`, `analyticsToolGuidance`, `AnalyticsQueryEvent`;
-`defineDataset` (with `defaultPeriod` and `tenantColumn: null`), `measure.*` and `dimension.*`
-(with `access`), `table` / `Table`, the expression builders; the `Dialect`,
-`SqlRenderer` and `AnalyticsSource` contracts; `postgresDialect`, `checkDatasetContract` /
-`assertDatasetContract`; and the types `AnalyticsQuery`, `AnalyticsResult`, `ResultPeriod`,
-`PeriodInput`, `Tenant`, `TenantId`, `AccessKey`, `FieldKey`, `Ratio`, `AnalyticsError` codes
-and their details (`isAnalyticsError`). `ExprNode` and `SelectStatement` are exported for
-dialect authors and may grow new node kinds.
+Kept stable for publishing: `createAnalytics` and its options (with `authorize`,
+`authorizeField` and `cache`), `allTenants`, `analytics.query`, `explain`, `querySchema`,
+`resultSchema`, `listDatasets`; `AnalyticsResultSchema`, `analyticsToolGuidance`,
+`AnalyticsQueryEvent`; `defineDataset` (with `defaultPeriod` and `tenantColumn: null`),
+`measure.*` and `dimension.*` (with `access`), `table` / `Table`, the expression builders; the
+`Dialect`, `SqlRenderer` and `AnalyticsSource` contracts; `postgresDialect`,
+`checkDatasetContract` / `assertDatasetContract`; `measurr/shapes` (`toKpi`, `toTimeseries`,
+`toDistribution`, `toPivot`, `toLeaderboard`) and `measurr/handler` (`queryBatch`,
+`streamBatch`, `toHttpError`); and the types `AnalyticsQuery`, `AnalyticsResult`,
+`ResultPeriod`, `PeriodInput`, `Tenant`, `TenantId`, `AccessKey`, `AnalyticsCache`, `FieldKey`,
+`Ratio`, `AnalyticsError` codes and their details (`isAnalyticsError`). `ExprNode` and
+`SelectStatement` are exported for dialect authors and may grow new node kinds.
 
 ## Install
 

package/dist/analytics.d.ts

@@ -50,6 +50,8 @@ export type AnalyticsQueryEvent = {
     /** Rows returned; 0 on error. */
     rows: number;
     outcome: 'success' | 'error';
+    /** Whether `cache` answered the query; absent without a cache or when it never got that far. */
+    cache?: 'hit' | 'miss';
     /** What the query threw, on error. */
     error?: unknown;
 };
@@ -90,6 +92,33 @@ export type AnalyticsOptions<DS extends readonly AnyDataset[], Ctx> = {
      * throws or rejects with is swallowed, so it can never fail a query.
      */
     onQuery?: ((event: AnalyticsQueryEvent, ctx: Ctx) => unknown) | undefined;
+    /**
+     * Where finished results are kept and looked up. What it throws or rejects with is
+     * swallowed: a failing cache is a miss, never a failed query.
+     */
+    cache?: AnalyticsCache<Ctx, DS[number]['key']> | undefined;
+};
+/**
+ * A store for finished results, such as Redis. Keys name the dataset and hash everything that
+ * decides the result: the query with its defaults, the tenant scope, the timezone and the
+ * resolved days. A query is parsed and authorized before the cache is asked, so a cached result
+ * never reaches a caller who may not run the query. Results are plain JSON.
+ */
+export type AnalyticsCache<Ctx, Key extends string = string> = {
+    /**
+     * The stored result, or null/undefined for a miss. Return a miss to bypass the cache for a
+     * caller (a refresh button); its fresh result is still stored. A stored value that is not a
+     * result of this dataset counts as a miss, so a changed definition does not break queries.
+     */
+    get(key: string, entry: {
+        dataset: Key;
+        ctx: Ctx;
+    }): unknown;
+    /** Stores a result; pick the time to live per `dataset`. */
+    set(key: string, result: QueryResult<Row, Row>, entry: {
+        dataset: Key;
+        ctx: Ctx;
+    }): unknown;
 };
 export type DatasetCatalogEntry<Key extends string = string> = {
     key: Key;
@@ -103,6 +132,8 @@ export type DatasetCatalogEntry<Key extends string = string> = {
         key: string;
         label: string;
         description: string | undefined;
+        /** `count`: a whole number, never null. `ratio`: 0-1 (`share`, `ratio`). `number`: the rest. */
+        valueType: 'count' | 'number' | 'ratio';
     }[];
     dimensions: {
         key: string;

package/dist/handler.d.ts

@@ -0,0 +1,56 @@
+/**
+ * Serving queries from any server framework: several queries in one request (all at once or
+ * as each finishes), and engine errors as HTTP errors. Framework-neutral; an oRPC or tRPC
+ * procedure wraps these in a few lines.
+ */
+import { type Analytics, AnalyticsError, type AnalyticsErrorCode, type AnalyticsErrorDetails } from './analytics.js';
+import type { AnyDataset } from './dataset.js';
+import type { DatasetQuery, ResultOf } from './query.js';
+/** Why one query of a batch failed. Errors other than `AnalyticsError` are not described. */
+export type BatchError = {
+    code: AnalyticsErrorCode | 'internal';
+    message: string;
+};
+export type BatchResult<DS extends readonly AnyDataset[], Q> = {
+    results: {
+        [K in keyof Q]?: ResultOf<DS[number], Q[K]>;
+    };
+    errors: {
+        [K in keyof Q]?: BatchError;
+    };
+};
+export type BatchChunk<DS extends readonly AnyDataset[], Q> = {
+    [K in keyof Q & string]: {
+        key: K;
+        result: ResultOf<DS[number], Q[K]>;
+    } | {
+        key: K;
+        error: BatchError;
+    };
+}[keyof Q & string];
+type Queries<DS extends readonly AnyDataset[]> = Record<string, DatasetQuery<DS[number]>>;
+/**
+ * Runs named queries in parallel. One query failing does not fail the others: its error is
+ * under its name. Each query is authorized, cached and reported like a single `query`.
+ */
+export declare function queryBatch<DS extends readonly AnyDataset[], Ctx, const Q extends Queries<DS>>(analytics: Analytics<DS, Ctx>, queries: Q, ctx: Ctx): Promise<BatchResult<DS, Q>>;
+/** Runs named queries in parallel and yields each as it finishes, for a streaming endpoint. */
+export declare function streamBatch<DS extends readonly AnyDataset[], Ctx, const Q extends Queries<DS>>(analytics: Analytics<DS, Ctx>, queries: Q, ctx: Ctx): AsyncGenerator<BatchChunk<DS, Q>, void, undefined>;
+export type HttpError = {
+    status: 400 | 401 | 403;
+    /** The status name oRPC and tRPC use for it. */
+    code: 'BAD_REQUEST' | 'UNAUTHORIZED' | 'FORBIDDEN';
+    message: string;
+    /** For a bad request: the engine's code and details, so a model can fix its query. */
+    data?: {
+        code: AnalyticsErrorCode;
+        details: AnalyticsErrorDetails[AnalyticsErrorCode];
+    };
+};
+/**
+ * An engine error as an HTTP error: `forbidden` is 403, `missing_tenant` 401, the rest are
+ * 400 with the code and details. Messages are safe to show; for `unknown_value` they list the
+ * valid values.
+ */
+export declare function toHttpError(error: AnalyticsError): HttpError;
+export {};

package/dist/handler.js

@@ -0,0 +1,49 @@
+import {
+  AnalyticsError2
+} from "./index-zgb1e89d.js";
+
+// src/handler.ts
+async function queryBatch(analytics, queries, ctx) {
+  const results = {};
+  const errors = {};
+  for await (const chunk of streamBatch(analytics, queries, ctx)) {
+    if ("error" in chunk)
+      errors[chunk.key] = chunk.error;
+    else
+      results[chunk.key] = chunk.result;
+  }
+  return { results, errors };
+}
+async function* streamBatch(analytics, queries, ctx) {
+  const pending = new Map(Object.entries(queries).map(([key, query]) => {
+    const settled = analytics.query(query, ctx).then((result) => ({ key, result }), (error) => ({ key, error: batchError(error) }));
+    return [key, settled];
+  }));
+  while (pending.size > 0) {
+    const chunk = await Promise.race(pending.values());
+    pending.delete(chunk.key);
+    yield chunk;
+  }
+}
+function batchError(error) {
+  return error instanceof AnalyticsError2 ? { code: error.code, message: error.message } : { code: "internal", message: "The query failed" };
+}
+function toHttpError(error) {
+  if (error.code === "forbidden") {
+    return { status: 403, code: "FORBIDDEN", message: error.message };
+  }
+  if (error.code === "missing_tenant") {
+    return { status: 401, code: "UNAUTHORIZED", message: error.message };
+  }
+  return {
+    status: 400,
+    code: "BAD_REQUEST",
+    message: error.message,
+    data: { code: error.code, details: error.details }
+  };
+}
+export {
+  queryBatch,
+  streamBatch,
+  toHttpError
+};

package/dist/index-g255jb1s.js

@@ -0,0 +1,149 @@
+// src/period.ts
+var MAX_PERIOD_DAYS = 731;
+var DEFAULT_PERIOD = {
+  last: { days: 30 }
+};
+var DAY_MS = 86400000;
+var formatterCache = new Map;
+function formatter(timezone) {
+  let cached = formatterCache.get(timezone);
+  if (!cached) {
+    cached = new Intl.DateTimeFormat("en-US", {
+      timeZone: timezone,
+      hourCycle: "h23",
+      year: "numeric",
+      month: "2-digit",
+      day: "2-digit",
+      hour: "2-digit",
+      minute: "2-digit",
+      second: "2-digit"
+    });
+    formatterCache.set(timezone, cached);
+  }
+  return cached;
+}
+function isValidTimezone(timezone) {
+  if (!/^[A-Za-z][A-Za-z0-9_+\-/]*$/.test(timezone))
+    return false;
+  try {
+    formatter(timezone);
+    return true;
+  } catch {
+    return false;
+  }
+}
+function localParts(instant, timezone) {
+  const parts = {};
+  for (const part of formatter(timezone).formatToParts(instant)) {
+    if (part.type !== "literal")
+      parts[part.type] = Number(part.value);
+  }
+  return {
+    year: parts.year ?? 0,
+    month: parts.month ?? 1,
+    day: parts.day ?? 1,
+    hour: parts.hour ?? 0,
+    minute: parts.minute ?? 0,
+    second: parts.second ?? 0
+  };
+}
+function offsetAt(instant, timezone) {
+  const p = localParts(instant, timezone);
+  const asUtc = Date.UTC(p.year, p.month - 1, p.day, p.hour, p.minute, p.second);
+  return asUtc - (instant.getTime() - instant.getUTCMilliseconds());
+}
+function localDate(instant, timezone) {
+  const p = localParts(instant, timezone);
+  return formatDate(Date.UTC(p.year, p.month - 1, p.day));
+}
+function startOfLocalDay(date, timezone) {
+  const wallClock = parseDate(date);
+  const firstGuess = wallClock - offsetAt(new Date(wallClock), timezone);
+  const corrected = wallClock - offsetAt(new Date(firstGuess), timezone);
+  return new Date(corrected);
+}
+function parseDate(date) {
+  const [year, month, day] = date.split("-").map(Number);
+  return Date.UTC(year ?? 0, (month ?? 1) - 1, day ?? 1);
+}
+function formatDate(utcMs) {
+  return new Date(utcMs).toISOString().slice(0, 10);
+}
+function addDays(date, days) {
+  return formatDate(parseDate(date) + days * DAY_MS);
+}
+function daysBetween(from, to) {
+  return Math.round((parseDate(to) - parseDate(from)) / DAY_MS);
+}
+function isoWeekday(date) {
+  const day = new Date(parseDate(date)).getUTCDay();
+  return day === 0 ? 7 : day;
+}
+
+class PeriodError extends Error {
+  name = "PeriodError";
+}
+function resolvePeriod(input, timezone, now) {
+  const today = localDate(now, timezone);
+  const [from, to] = periodDays(input, today);
+  if (daysBetween(from, to) < 0) {
+    throw new PeriodError(`The period starts (${from}) after it ends (${to}).`);
+  }
+  return periodFromDays(from, to, timezone);
+}
+function periodDays(input, today) {
+  if ("last" in input)
+    return [addDays(today, -(input.last.days - 1)), today];
+  if ("from" in input)
+    return [input.from, input.to];
+  const monthStart = `${today.slice(0, 8)}01`;
+  const weekStart = addDays(today, -(isoWeekday(today) - 1));
+  switch (input.preset) {
+    case "today":
+      return [today, today];
+    case "yesterday":
+      return [addDays(today, -1), addDays(today, -1)];
+    case "thisWeek":
+      return [weekStart, today];
+    case "lastWeek":
+      return [addDays(weekStart, -7), addDays(weekStart, -1)];
+    case "thisMonth":
+      return [monthStart, today];
+    case "lastMonth": {
+      const lastMonthEnd = addDays(monthStart, -1);
+      return [`${lastMonthEnd.slice(0, 8)}01`, lastMonthEnd];
+    }
+    case "thisYear":
+      return [`${today.slice(0, 4)}-01-01`, today];
+  }
+}
+function periodFromDays(from, to, timezone) {
+  const days = daysBetween(from, to) + 1;
+  if (days > MAX_PERIOD_DAYS) {
+    throw new PeriodError(`A period may cover at most ${MAX_PERIOD_DAYS} days (two years); ${from} to ${to} is ${days} days.`);
+  }
+  return {
+    from,
+    to,
+    timezone,
+    start: startOfLocalDay(from, timezone),
+    end: startOfLocalDay(addDays(to, 1), timezone),
+    days
+  };
+}
+function previousPeriod(period) {
+  const to = addDays(period.from, -1);
+  const from = addDays(to, -(period.days - 1));
+  return periodFromDays(from, to, period.timezone);
+}
+function bucketStarts(from, to, unit) {
+  let start = unit === "day" ? from : unit === "week" ? addDays(from, -(isoWeekday(from) - 1)) : `${from.slice(0, 8)}01`;
+  const starts = [];
+  while (start <= to) {
+    starts.push(start);
+    start = unit === "day" ? addDays(start, 1) : unit === "week" ? addDays(start, 7) : formatDate(Date.UTC(Number(start.slice(0, 4)), Number(start.slice(5, 7)), 1));
+  }
+  return starts;
+}
+
+export { MAX_PERIOD_DAYS, DEFAULT_PERIOD, isValidTimezone, PeriodError, resolvePeriod, previousPeriod, bucketStarts };