inibase 3.0.0 → 3.1.0

package/README.md

@@ -858,6 +858,97 @@ await db.get("user", undefined, { sort: {age: -1, username: "asc"} });
 </blockquote>
 </details>
 
+<details>
+<summary>Computed Fields</summary>
+<blockquote>
+
+A `computed` column is derived from other columns of the same row whenever the
+row is written (`post` / `put`, including backfill on `updateTable`). The
+result is stored in its own column file like any other value, so reads and
+criteria queries work unchanged.
+
+```ts
+const db = new Inibase("/databaseName");
+
+await db.createTable("product", [
+	{ key: "name", type: "string" },
+	{ key: "price", type: "number" },
+]);
+
+// ids are assigned per table in schema order: customer=1, status=2, items=3,
+// product=4, quantity=5, unitPriceCents=6, totalCents=7, totalCentsLive=8
+await db.createTable("orders", [
+	{ key: "customer", type: "string" },
+	{ key: "status", type: "number" },
+	{
+		key: "items",
+		type: "array",
+		children: [
+			{ key: "product", type: "table", table: "product" },
+			{ key: "quantity", type: "number" },
+			{ key: "unitPriceCents", type: "number" },
+		],
+	},
+	{ key: "totalCents", type: "number", computed: "sum(5, 6)" }, // quantity x unitPriceCents
+	{ key: "totalCentsLive", type: "number", computed: "sum(5, 4.2)" }, // quantity x product.price
+]);
+
+const posted = await db.post(
+	"orders",
+	{
+		customer: "acme",
+		status: 1,
+		items: [
+			{ product: "id-of-widget", quantity: 2, unitPriceCents: 250 },
+			{ product: "id-of-gadget", quantity: 1, unitPriceCents: 100 },
+		],
+	},
+	undefined,
+	true,
+);
+// posted.totalCents === 600      (2*250 + 1*100)
+// posted.totalCentsLive === 847  (2*price(widget) + 1*price(gadget))
+```
+
+**Expression language (v1, integer-only).**
+
+- Operators: `+` `-` `,` (multiply) `/` `%`; `( )` for grouping. Multiplication
+  binds tighter than addition (`1 , 2 + 3` = `(1*2) + 3`).
+- Helpers: `sum` `count` `avg` `min` `max` iterate an array-of-objects found by
+  the ids inside the parentheses (`sum(5, 6)` = quantity x unit-price per item).
+- Paths: `id ("." id)*` where `.` hops through a `table` link
+  (`4.2` = price of the linked product row).
+- Integers only: there are no decimal literals — `.` is the path separator, so
+  `3.14` is a path (`field 3`, hop `field 14`), never 3.14. Fractional results
+  are written as division: `314 / 100` → 3.14.
+- A bare integer that matches a field id in the table's schema is that field
+  (ids are locative); a bare integer that matches no field is a literal.
+
+**Rules & limits (v1).**
+
+- Computed fields cannot be `required`, `unique` or `regex` (a
+  `COMPUTED_FIELD_CONFLICT` error).
+- Computed values are never user-settable (`post`/`put` with a computed key
+  throws `COMPUTED_FIELD_SETTABLE`).
+- Values are evaluated at write time and persisted; compiled expressions are
+  stored as `{ expr, ast }` in the schema. The AST carries only field ids, so
+  renaming a column (or its linked table's columns) never retargets an
+  expression.
+- `updateTable` backfills existing rows when a computed field is added or its
+  expression changes; a backfill that fails (missing dependency, arithmetic
+  error, dangling link) aborts the migration and leaves the schema untouched.
+- Aggregates: `sum` over no values is `0`, `count` is the element count, and
+  `avg`/`min`/`max` over no values throw `COMPUTED_FIELD_ARITHMETIC`.
+- Non-numeric operands, division/modulo by zero, and arithmetic over missing
+  (null) values throw `COMPUTED_FIELD_ARITHMETIC`; a link to a missing row
+  throws `COMPUTED_FIELD_DANGLING_LINK`.
+- Only top-level schema fields can be computed in v1; array contents are
+  reachable through helpers. Missing link values make a single bare path
+  evaluate to null (stored empty).
+
+</blockquote>
+</details>
+
 </blockquote>
 </details>
 
@@ -889,6 +980,28 @@ await db.get("user", undefined, { sort: {age: -1, username: "asc"} });
 > > [!WARNING]
 > > The numbers above were measured **before** always-on fsync + write-ahead journaling landed (they no longer reflect current hot-path write costs). Run `pnpm benchmark:durability` for the crash-atomic numbers.
 
+### Computed fields
+
+Write-time evaluation cost (helpers `sum(3 , 4)`, `avg(4)`, `min(4)`, `max(4)`, `count(3)` over an `items` array with 3 lines, plus an arithmetic field `itemTotal × (314 / 100)`), compared against the identical table without computed fields:
+
+| rows | POST bulk (plain / computed) | POST single (plain / computed) | PUT recompute (plain / computed) |
+|------|------------------------------|--------------------------------|----------------------------------|
+| 10   | 21.90 / 22.69 ms             | 25.24 / 26.80 ms               | 23.10 / 37.07 ms                 |
+| 100  | 22.00 / 25.23 ms             | 25.24 / 26.80 ms               | 22.16 / 40.00 ms                 |
+| 1000 | 28.79 / 35.15 ms             | 25.24 / 26.80 ms               | 21.01 / 45.05 ms                 |
+
+> Min of 3 rounds (fsync + journal on); the plain-vs-computed delta is the pure expression-evaluation cost (~6 ms/1000 rows of helpers on POST, more on `PUT` because every matched row is recomputed). GET all (1110 rows): 11.44 / 15.90 ms — computed values are stored in real column files, so reads never evaluate; the gap is the larger column count to scan. Run `pnpm benchmark:computed` to reproduce.
+
+**Link-heavy: shared product catalog with batched link-hop reads.** Every order's line items reference a shared 20-row catalog; `totalCentsLive = sum(quantity, product.price)` follows one link hop per line item. The engine batches the hops across the whole post: each batch resolves at most one read per **distinct** `(table, column, id)` triple (here: catalogSize=20 product rows) instead of one full row-level read per line item (2000 reads for 1000 orders × 2 items):
+
+| rows | POST bulk (lk link hop) |
+|------|-------------------------|
+| 10   | 42.29 ms                |
+| 100  | 47.76 ms                |
+| 1000 | 59.46 ms                |
+
+> Shared 20-row catalog (10/100/1000 orders × 2 line items). Without batching every line item re-reads its product row; with it each batch resolves ≤ catalogSize distinct linked rows once (deduplicated across every line item and every order of the batch) and re-evaluates against the warm in-memory cache. A dangling link anywhere in the batch still rejects the whole post (`COMPUTED_FIELD_DANGLING_LINK`). HTTP-request bound, not disk-bound: the cost scales with the number of *distinct* linked rows the catalog actually has, not with line-item count.
+
 ## Roadmap
 
 - [x] Actions:
@@ -930,6 +1043,7 @@ await db.get("user", undefined, { sort: {age: -1, username: "asc"} });
   - [ ] Encryption
   - [x] Data Compression
   - [x] Caching System
+  - [x] Computed fields (v1 id-only expression language)
   - [ ] Suggest [new feature +](https://github.com/inicontent/inibase/discussions/new?category=ideas)
 
 ## License

package/dist/expression.d.ts

@@ -0,0 +1,159 @@
+/**
+ * Computed-fields expression language (v1 — id-only).
+ *
+ * Grammar
+ * -------
+ * ```
+ * expression := term (("+" | "-") term)*
+ * term       := factor (("," | "/" | "%") factor)*   // "," = multiply
+ * factor     := integer-literal | path | function-call | "(" expression ")"
+ * path       := id ( "." id )*                       // "." = link/binding hop
+ * function   := "sum" | "count" | "avg" | "min" | "max" "(" expression ")"
+ * ```
+ *
+ * Every symbol is a numeric field `id` (per-table dense counter, nested
+ * children included). There are no decimal literals: `.` is reserved as the
+ * path separator, so `3.4` is a link hop (`field 3`, then `field 4` in the
+ * table `field 3` links to), never the decimal 3.4. Fractional results are
+ * reachable via division (`314 / 100` → 3.14).
+ *
+ * A bare integer that **matches a field id in the table's schema** is that
+ * field's value (ids are locative); a bare integer that matches no field id
+ * is an integer literal. This is what makes both `sum(4, 3.4)` (id 4 =
+ * quantity) and `314 / 100` (no such ids) usable.
+ *
+ * Compiled expressions are persisted with the schema as
+ * `{ expr: string, ast: CompiledExpressionNode }` so key/table renames never
+ * retarget an expression: the AST carries only field ids (+ the id of the
+ * array ancestor a helper iterates), and every key path is re-derived from
+ * the *current* schema at write time.
+ */
+import type { ErrorLang, Field, Schema } from "./index.js";
+/** Cap on the raw `computed` string length (bound work at validation). */
+export declare const COMPUTED_EXPR_MAX_LENGTH = 512;
+/** Cap on the parsed AST depth (protects the parser/evaluator recursion). */
+export declare const COMPUTED_EXPR_MAX_DEPTH = 64;
+export type ComputedFunctionName = "sum" | "count" | "avg" | "min" | "max";
+export type BinaryOp = "add" | "sub" | "mul" | "div" | "mod";
+/** Raw AST produced by {@link parseExpression} (field ids unresolved). */
+export type RawExpressionNode = {
+    kind: "num";
+    value: number;
+} | {
+    kind: "path";
+    ids: number[];
+} | {
+    kind: "bin";
+    op: BinaryOp;
+    left: RawExpressionNode;
+    right: RawExpressionNode;
+} | {
+    kind: "fn";
+    name: ComputedFunctionName;
+    arg: RawExpressionNode;
+};
+/**
+ * Compiled AST stored in the schema. Path nodes carry only ids (+ the id of
+ * the array ancestor a helper iterates); all key paths are resolved against
+ * the current schema at evaluation time, so renames never break expressions.
+ */
+export type CompiledExpressionNode = {
+    kind: "num";
+    value: number;
+} | {
+    kind: "path";
+    ids: number[];
+    arrayFieldId: number | null;
+} | {
+    kind: "bin";
+    op: BinaryOp;
+    left: CompiledExpressionNode;
+    right: CompiledExpressionNode;
+} | {
+    kind: "fn";
+    name: ComputedFunctionName;
+    /** id of the array ancestor every path in `arg` lives in. */
+    arrayFieldId: number;
+    arg: CompiledExpressionNode;
+};
+/** Persisted form of a `computed` schema property. (A type alias so it stays
+ *  assignable to Inison's recursive `Data` type.) */
+export type ComputedFieldSpec = {
+    expr: string;
+    ast: CompiledExpressionNode;
+};
+/**
+ * Schema index used for id resolution: maps a field id to its dotted key path,
+ * the resolved field, and the nearest array-of-objects ancestor (if any).
+ */
+export interface FieldRef {
+    key: string;
+    field: Field;
+    /** Nearest `array`-typed ancestor with object children, or null. */
+    arrayAncestor: {
+        id: number;
+        key: string;
+    } | null;
+    /** True when `arrayAncestor` itself sits inside another array of objects. */
+    nestedInArrayOfArrays: boolean;
+}
+/**
+ * Parse a `computed` expression into a raw AST. Throws
+ * `COMPUTED_FIELD_SYNTAX` on invalid syntax, oversized input or excessive
+ * nesting.
+ */
+export declare function parseExpression(source: string, language?: ErrorLang, fieldKey?: string): RawExpressionNode;
+/**
+ * Build the id → {@link FieldRef} index of a schema (nested children included,
+ * using dotted key paths). Container fields (array/object with object
+ * children) are indexed too so unknown ids are detected, but they can't be
+ * referenced by an expression.
+ */
+export declare function buildFieldIndex(schema: Schema): Map<number, FieldRef>;
+export interface ResolveContext {
+    language: ErrorLang;
+    /** Key of the computed field being compiled (error context). */
+    ownKey: string;
+    /** Id index of the table the computed field belongs to. */
+    index: Map<number, FieldRef>;
+    /** Fetch (and cache) the id index of another table, or undefined. */
+    getTableIndex: (tableName: string) => Promise<Map<number, FieldRef> | undefined>;
+}
+/**
+ * Resolve a raw expression against the schema (and linked tables), returning
+ * the compiled node plus the set of field ids it reads from the current
+ * table (used for dependency ordering / cycle detection between computed
+ * fields).
+ */
+export declare function resolveExpression(expression: RawExpressionNode, ctx: ResolveContext): Promise<{
+    ast: CompiledExpressionNode;
+    deps: Set<number>;
+}>;
+/** Every field id a compiled expression reads from the *current* table. */
+export declare function collectFieldDeps(node: CompiledExpressionNode): Set<number>;
+export interface ComputedFieldMeta {
+    id: number;
+    key: string;
+    deps: Set<number>;
+}
+/**
+ * Order computed fields so every dependency is evaluated before its dependents.
+ * Throws `COMPUTED_FIELD_CYCLE` when a field (transitively) depends on itself.
+ */
+export declare function topoSortComputedFields(fields: ComputedFieldMeta[], language: ErrorLang): ComputedFieldMeta[];
+/**
+ * Flatten a formatted row into a dot-notation record. Objects are flattened
+ * (`meta.age`), arrays of objects are combined per child key
+ * (`items.quantity` -> array of per-element values) and everything else is
+ * stored under its dotted key.
+ */
+export declare function flattenRecord(obj: Record<string, any>, prefix?: string): Record<string, any>;
+/**
+ * Resolve a dotted key path against a structured frame in place — the direct
+ * read-path counterpart of `flattenRecord`. Walks `obj` segment by segment
+ * and returns `undefined` when any intermediate is null, undefined, a
+ * non-object, or an array (arrays resolve numerically only, so they never
+ * match a dotted segment), mirroring the leaves `flattenRecord` would have
+ * produced without materialising the flattened record.
+ */
+export declare function resolveFramePath(obj: Record<string, any>, dottedKey: string): any;