basaltdb 0.1.0 → 0.2.0

package/ORM.md

@@ -0,0 +1,87 @@
+# `basaltdb` models — capabilities & gaps
+
+`basaltdb` ships a **lightweight ORM** (`db.model(name, schema)` → `Model`). This
+doc is honest about what it does, and how it compares to the connectors/ORMs
+people know: **mongoose** (MongoDB), **mysql2 + Sequelize/Prisma** (MySQL).
+
+Guiding principle: the model layer only emits SQL the Basalt engine actually
+supports. We'd rather throw a clear error (or omit a feature) than generate SQL
+that fails at runtime. Everything marked 🔒 is blocked on an **engine** feature,
+not the client (see the [roadmap](https://github.com/basalt-db/basalt/blob/main/ROADMAP.md)).
+
+---
+
+## What the model layer does today ✅
+
+- **Define a model** with fields as SQL types or friendly aliases
+  (`string`→VARCHAR, `int`→INT, `bool`→BOOLEAN, `date`/`datetime`→DATE/TIMESTAMP,
+  `decimal`→DECIMAL, …), a `primaryKey`, and per-field `index`.
+- **`sync()`** — create the table (and secondary indexes) if it doesn't exist;
+  `sync({force:true})` drops and recreates.
+- **CRUD**: `insert` (one or many), `find` / `findOne` / `findByPk`, `count`,
+  `update`, `delete`.
+- **Where-spec** object → SQL: equality, arrays → `IN` (expanded to an `OR`
+  chain the engine supports), and operators `= != <> > >= < <= between` (plus
+  `eq/ne/gt/gte/lt/lte/in` aliases). Multiple keys AND-ed.
+- **`find` options**: `select`, `orderBy`, `limit`.
+- **`toCreateSQL()`** to inspect the generated DDL.
+- Values are escaped; identifiers never come from user input.
+- Same API embedded (WASM) or over HTTP.
+
+## Feature comparison
+
+| Capability | mongoose (Mongo) | Sequelize/Prisma (MySQL) | basaltdb Model |
+|---|:--:|:--:|:--:|
+| Define schema / model | ✅ | ✅ | ✅ |
+| Create table/collection from model | ✅ (implicit) | ✅ `sync`/migrate | ✅ `sync()` |
+| Insert one / many | ✅ | ✅ | ✅ |
+| Find with filters | ✅ rich | ✅ rich | ✅ eq / IN / range / between |
+| Update / delete by filter | ✅ | ✅ | ✅ |
+| Count | ✅ | ✅ | ✅ |
+| Order / limit | ✅ | ✅ | ✅ |
+| **Pagination (offset/cursor)** | ✅ | ✅ | ❌ 🔒 no `OFFSET` in engine |
+| **`LIKE` / text search** | ✅ | ✅ | ❌ 🔒 engine has no `LIKE` (throws a clear error) |
+| **Aggregations (group/having)** | ✅ | ✅ | 🟡 via raw `db.query`, not the model API yet |
+| **Relations / joins / populate / include** | ✅ | ✅ | ❌ use raw `JOIN` via `db.query` |
+| **Validation / defaults / hooks** | ✅ | ✅ | ❌ not yet (client-side, could add) |
+| **Migrations (alter schema)** | n/a | ✅ | ❌ 🔒 no `ALTER TABLE` |
+| **Transactions** | ✅ | ✅ | ❌ 🔒 engine has no transactions |
+| **Prepared statements / bind params** | ✅ | ✅ | ❌ 🔒 engine re-parses each call |
+| **Unique / FK / check constraints** | 🟡 | ✅ | ❌ 🔒 only PRIMARY KEY today |
+| **Upsert (`ON CONFLICT`/`save`)** | ✅ | ✅ | ❌ 🔒 no upsert in engine |
+| **Auto-increment / default id** | ✅ | ✅ | ❌ supply the PK yourself |
+| TypeScript types | 🟡/✅ | ✅ | ✅ (`.d.ts`; not schema-inferred generics yet) |
+
+## Gaps we *could* close in the client (no engine change needed)
+
+These are pure-client and are the natural next PRs to `basalt-js`:
+
+1. **Model-level aggregations** — `User.groupBy(['tier'], { count:true, sum:'amount' })`
+   emitting `GROUP BY` (the engine supports it).
+2. **Client-side pagination** — `find(..., { limit, page })` by fetching and
+   slicing, until the engine gains `OFFSET`.
+3. **Validation / defaults / lifecycle hooks** (`beforeInsert`, `afterFind`).
+4. **`raw()` join helper** — a typed wrapper over `db.query` for `JOIN`s so
+   relations feel first-class even without `populate`.
+5. **Schema-inferred TS generics** — infer row types from the model schema so
+   `find()` returns a typed object, like Prisma.
+6. **`upsert()` emulation** — try-insert-then-update (non-atomic; documented).
+7. **`describe()` / diff** — compare a model to the live schema and warn on drift.
+
+## Gaps that need the engine first 🔒
+
+Tracked in the [roadmap](https://github.com/basalt-db/basalt/blob/main/ROADMAP.md):
+`LIKE`/text search & `OFFSET` (Tier 2.1), **transactions** (Tier 1.3),
+**`ALTER TABLE`/migrations** and **constraints/FKs/upsert** (Tier 2.1), and
+**prepared statements** (Tier 2.1). The highest-leverage single item is the
+**PostgreSQL wire protocol** (Tier 4): with it, mature ORMs (Prisma, Sequelize,
+TypeORM, Drizzle) could target Basalt directly and this bespoke layer becomes
+optional.
+
+## When to use what
+
+- **Model API** — everyday CRUD on a known schema. Cleanest for app code.
+- **Query builder** (`db.table(...)`) — ad-hoc selects with fluent chaining.
+- **Raw `db.query(sql)`** — anything the above don't cover yet (JOINs, GROUP BY,
+  window-less analytics). Always available; the model/builder are conveniences on
+  top of it.

package/README.md

@@ -52,6 +52,16 @@ await db.schema();                           // tables + columns
 
 In Node < 18 (no global `fetch`) pass one: `new BasaltClient(url, { fetch })`.
 
+> **Need a server?** The client/server mode talks to a running Basalt server.
+> Set up and run the database engine from the main repo —
+> **[basalt-db/basalt → Quickstart](https://github.com/basalt-db/basalt#quickstart)**:
+> ```bash
+> git clone https://github.com/basalt-db/basalt.git && cd basalt
+> make basalt-server
+> ./basalt-server demo_root 8090     # HTTP API + Workbench on http://127.0.0.1:8090
+> ```
+> (The **embedded** mode above needs no server — the engine runs in-process.)
+
 ## Query builder (ORM-lite)
 
 A small fluent builder that generates SQL. It works the same on both `Basalt`
@@ -69,6 +79,46 @@ db.table('users').where('id', 6).delete();
 Values are escaped (`lit()`); **identifiers are never taken from user input**.
 Call `.toSQL()` on any builder to see the generated statement.
 
+## Models (lightweight ORM)
+
+Define a table as a model and do typed CRUD without hand-writing SQL. All model
+methods return Promises and work the same embedded or over HTTP.
+
+```js
+import { Basalt } from 'basaltdb';
+const db = await Basalt.create();
+
+const User = db.model('users', {
+  id:     { type: 'BIGINT', primaryKey: true },
+  name:   'string',                       // string → VARCHAR
+  tier:   { type: 'int', index: true },   // creates a secondary index on sync()
+  active: 'boolean',
+  joined: 'date',
+});
+
+await User.sync();                                  // CREATE TABLE (+ index) if absent
+await User.insert([{ id: 1, name: 'alice', tier: 1, active: true, joined: '2024-01-02' },
+                   { id: 2, name: 'bob',   tier: 2, active: false }]);
+
+await User.find({ tier: 2 }, { orderBy: ['id', 'DESC'], limit: 10 });
+await User.find({ tier: [1, 2] });                  // array → IN (expanded to OR)
+await User.find({ id: { '>=': 1, '<': 100 } });     // operators
+await User.findByPk(1);
+await User.count({ active: true });
+await User.update({ tier: 3 }, { id: 2 });
+await User.delete({ id: 2 });
+User.toCreateSQL();                                 // inspect the DDL
+```
+
+**Where-spec:** `{ col: value }` (equality), `{ col: [a, b] }` (IN), or
+`{ col: { op: value } }` where `op` ∈ `= != <> > >= < <= eq ne gt gte lt lte in between`.
+Multiple keys are AND-ed. Types accept SQL names (`BIGINT`, `VARCHAR`, `DECIMAL`,
+`TIMESTAMP`, …) or friendly aliases (`string`, `int`, `bool`, `date`, `datetime`).
+
+See **[ORM.md](ORM.md)** for what the model layer does and does **not** do
+compared to mongoose / Sequelize / Prisma, and why (it's honest about the gaps
+that come from the young engine).
+
 ## What Basalt supports (and doesn't, yet)
 
 This is a young, learning-oriented engine — the builder deliberately stays close

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "basaltdb",
-  "version": "0.1.0",
-  "description": "Official JavaScript/TypeScript client for Basalt — a from-scratch HTAP database engine. Run it embedded in-process via WebAssembly (Node & browser), or talk to a Basalt server over HTTP.",
+  "version": "0.2.0",
+  "description": "Official JavaScript/TypeScript client & lightweight ORM for Basalt — a from-scratch HTAP database engine. Run it embedded in-process via WebAssembly (Node & browser), or talk to a Basalt server over HTTP. Define models and do typed CRUD.",
   "type": "module",
   "main": "./src/index.js",
   "module": "./src/index.js",
@@ -16,23 +16,40 @@
     "src/index.js",
     "src/index.d.ts",
     "src/engine.js",
-    "README.md"
+    "README.md",
+    "ORM.md"
   ],
   "scripts": {
-    "test": "node test/smoke.mjs",
+    "test": "node test/smoke.mjs && node test/orm.mjs",
     "prepublishOnly": "npm test"
   },
   "keywords": [
-    "database", "sql", "htap", "olap", "oltp", "columnar", "wasm",
-    "webassembly", "embedded-database", "basalt", "in-memory", "analytics"
+    "database",
+    "orm",
+    "models",
+    "sql",
+    "htap",
+    "olap",
+    "oltp",
+    "columnar",
+    "wasm",
+    "webassembly",
+    "embedded-database",
+    "basalt",
+    "in-memory",
+    "analytics"
   ],
   "homepage": "https://github.com/basalt-db/basalt-js",
   "repository": {
     "type": "git",
     "url": "git+https://github.com/basalt-db/basalt-js.git"
   },
-  "bugs": { "url": "https://github.com/basalt-db/basalt-js/issues" },
+  "bugs": {
+    "url": "https://github.com/basalt-db/basalt-js/issues"
+  },
   "license": "MIT",
-  "engines": { "node": ">=18" },
+  "engines": {
+    "node": ">=18"
+  },
   "sideEffects": false
 }

package/src/index.d.ts

@@ -36,6 +36,36 @@ export class QueryBuilder {
   delete(): Promise<Result> | Result;
 }
 
+export type FieldType =
+  | 'BIGINT' | 'INT' | 'SMALLINT' | 'DOUBLE' | 'VARCHAR' | 'CHAR' | 'TEXT'
+  | 'BOOLEAN' | 'DATE' | 'TIMESTAMP' | 'DECIMAL'
+  | 'string' | 'number' | 'int' | 'integer' | 'bigint' | 'float' | 'double'
+  | 'bool' | 'boolean' | 'date' | 'datetime' | 'timestamp' | 'decimal' | (string & {});
+export interface FieldDef { type: FieldType; primaryKey?: boolean; pk?: boolean; index?: boolean; }
+export type ModelSchema = Record<string, FieldType | FieldDef>;
+
+/** Operators usable in a where-spec: { age: { '>=': 18 }, id: { in: [1,2] } }. */
+export type WhereSpec = Record<string,
+  Value | Value[] | Partial<Record<'='|'!='|'<>'|'>'|'>='|'<'|'<='|'eq'|'ne'|'gt'|'gte'|'lt'|'lte'|'in'|'between'|'like', any>>>;
+export interface FindOptions { select?: string[]; orderBy?: string | [string, 'ASC'|'DESC'|'asc'|'desc']; limit?: number; }
+
+/** A table mapped as a model — typed CRUD without hand-writing SQL. */
+export class Model {
+  name: string;
+  pk: string | null;
+  toCreateSQL(): string;
+  sync(opts?: { force?: boolean }): Promise<Model>;
+  insert(rows: Record<string, Value> | Record<string, Value>[]): Promise<Result>;
+  find(where?: WhereSpec, opts?: FindOptions): Promise<Record<string, any>[]>;
+  findOne(where?: WhereSpec, opts?: FindOptions): Promise<Record<string, any> | null>;
+  findByPk(value: Value): Promise<Record<string, any> | null>;
+  count(where?: WhereSpec): Promise<number>;
+  update(patch: Record<string, Value>, where?: WhereSpec): Promise<Result>;
+  delete(where?: WhereSpec): Promise<Result>;
+}
+export function defineModel(runner: Basalt | BasaltClient, name: string, schema: ModelSchema): Model;
+export function buildWhere(where?: WhereSpec): string;
+
 /** Embedded, in-process (WebAssembly) database. */
 export class Basalt {
   static create(): Promise<Basalt>;
@@ -43,6 +73,7 @@ export class Basalt {
   query(sql: string): Record<string, any>[];
   schema(): TableInfo[];
   table(name: string): QueryBuilder;
+  model(name: string, schema: ModelSchema): Model;
 }
 
 export interface BasaltClientOptions {
@@ -62,6 +93,7 @@ export class BasaltClient {
   dropDatabase(name: string): Promise<any>;
   use(db: string): this;
   table(name: string): QueryBuilder;
+  model(name: string, schema: ModelSchema): Model;
 }
 
 export function lit(v: Value): string;

package/src/index.js

@@ -99,6 +99,109 @@ class QueryBuilder {
   }
 }
 
+// ---- models / lightweight ORM --------------------------------------------
+// Define a table as a model, then do typed CRUD without hand-writing SQL. Works
+// on both the embedded engine and the HTTP client (every method returns a
+// Promise). See ORM.md for what it does and does not do vs mongoose/Sequelize.
+const TYPE_ALIASES = {
+  string:'VARCHAR', text:'VARCHAR', varchar:'VARCHAR', char:'CHAR',
+  int:'INT', integer:'INT', smallint:'SMALLINT', number:'DOUBLE',
+  bigint:'BIGINT', long:'BIGINT', float:'DOUBLE', double:'DOUBLE', real:'DOUBLE',
+  bool:'BOOLEAN', boolean:'BOOLEAN', date:'DATE', datetime:'TIMESTAMP',
+  timestamp:'TIMESTAMP', decimal:'DECIMAL', numeric:'DECIMAL',
+};
+function normField(name, def){
+  if (typeof def === 'string') def = { type: def };
+  const raw = String(def.type || 'VARCHAR');
+  const type = TYPE_ALIASES[raw.toLowerCase()] || raw.toUpperCase();
+  return { name, type, primaryKey: !!(def.primaryKey || def.pk), index: !!def.index };
+}
+// Build a WHERE clause from a spec object:
+//   { id: 1 }                      → WHERE id = 1
+//   { tier: [1,2] }                → WHERE tier IN (1, 2)
+//   { age: { '>=': 18, '<': 65 } } → WHERE age >= 18 AND age < 65
+//   { name: { like: 'a%' } } / { ts: { between: [lo, hi] } } / { id: { in:[…] } }
+const OPS = { eq:'=', ne:'!=', gt:'>', gte:'>=', lt:'<', lte:'<=',
+  '=':'=', '!=':'!=', '<>':'<>', '>':'>', '>=':'>=', '<':'<', '<=':'<=' };
+// IN isn't in the engine's grammar yet, so we expand it to an OR-chain the
+// engine does support: col IN (a,b) → (col = a OR col = b).
+function inClause(col, list){
+  if (!Array.isArray(list) || !list.length) throw new BasaltError(`empty IN list for "${col}"`);
+  return '(' + list.map(x => `${col} = ${lit(x)}`).join(' OR ') + ')';
+}
+export function buildWhere(where){
+  if (!where) return '';
+  const keys = Object.keys(where); if (!keys.length) return '';
+  const parts = keys.map(col => {
+    const v = where[col];
+    if (Array.isArray(v)) return inClause(col, v);
+    if (v !== null && typeof v === 'object' && !(v instanceof Date)) {
+      return Object.entries(v).map(([op, val]) => {
+        const o = op.toLowerCase();
+        if (o === 'in') return inClause(col, val);
+        if (o === 'between') return `${col} BETWEEN ${lit(val[0])} AND ${lit(val[1])}`;
+        if (o === 'like') throw new BasaltError('LIKE is not supported by the Basalt engine yet — see ROADMAP.md');
+        const sql = OPS[o]; if (!sql) throw new BasaltError('unknown operator: ' + op);
+        return `${col} ${sql} ${lit(val)}`;
+      }).join(' AND ');
+    }
+    return `${col} = ${lit(v)}`;
+  });
+  return ' WHERE ' + parts.join(' AND ');
+}
+
+export class Model {
+  constructor(runner, name, schema){
+    this.r = runner; this.name = name; this.schema = schema;
+    this.fields = Object.entries(schema).map(([k, v]) => normField(k, v));
+    this.pk = (this.fields.find(f => f.primaryKey) || {}).name || null;
+  }
+  /** The `CREATE TABLE` statement this model maps to. */
+  toCreateSQL(){
+    const cols = this.fields.map(f => '  ' + f.name + ' ' + f.type + (f.primaryKey ? ' PRIMARY KEY' : ''));
+    return `CREATE TABLE ${this.name} (\n${cols.join(',\n')}\n)`;
+  }
+  indexDDL(){ return this.fields.filter(f => f.index && !f.primaryKey)
+    .map(f => `CREATE INDEX ${this.name}_${f.name} ON ${this.name} (${f.name})`); }
+  _run(sql){ return Promise.resolve(this.r._run(sql)); }
+  /** Create the table (+ indexes) if it doesn't exist. `{force:true}` drops first. */
+  async sync(opts = {}){
+    const tables = await Promise.resolve(this.r.schema());
+    const exists = (tables || []).some(t => t.name === this.name);
+    if (exists && opts.force) { await this._run('DROP TABLE IF EXISTS ' + this.name); }
+    if (!exists || opts.force) {
+      await this._run(this.toCreateSQL());
+      for (const d of this.indexDDL()) await this._run(d);
+    }
+    return this;
+  }
+  /** Insert one object or an array of objects (values follow the model's column order). */
+  async insert(rows){
+    const list = Array.isArray(rows) ? rows : [rows];
+    if (!list.length) return { message: 'INSERT 0' };
+    const cols = this.fields.map(f => f.name);
+    const values = list.map(o => `(${cols.map(c => lit(o[c] === undefined ? null : o[c])).join(', ')})`).join(', ');
+    return this._run(`INSERT INTO ${this.name} (${cols.join(', ')}) VALUES ${values}`);
+  }
+  /** Find rows. opts: { select:[...], orderBy:'col'|['col','DESC'], limit:n }. */
+  async find(where, opts = {}){
+    let sql = 'SELECT ' + (opts.select ? opts.select.join(', ') : '*') + ' FROM ' + this.name + buildWhere(where);
+    if (opts.orderBy){ const [c, d] = Array.isArray(opts.orderBy) ? opts.orderBy : [opts.orderBy, 'ASC'];
+      sql += ` ORDER BY ${c} ${String(d).toUpperCase()}`; }
+    if (opts.limit != null) sql += ' LIMIT ' + (opts.limit | 0);
+    return (await this._run(sql)).toObjects();
+  }
+  async findOne(where, opts = {}){ return (await this.find(where, { ...opts, limit: 1 }))[0] ?? null; }
+  async findByPk(v){ if (!this.pk) throw new BasaltError(`model ${this.name} has no primary key`);
+    return this.findOne({ [this.pk]: v }); }
+  async count(where){ return Number((await this._run('SELECT COUNT(*) AS n FROM ' + this.name + buildWhere(where))).toObjects()[0].n); }
+  async update(patch, where){ const set = Object.keys(patch).map(c => `${c} = ${lit(patch[c])}`).join(', ');
+    return this._run(`UPDATE ${this.name} SET ${set}${buildWhere(where)}`); }
+  async delete(where){ return this._run(`DELETE FROM ${this.name}${buildWhere(where)}`); }
+}
+/** Bind a model to any runner (Basalt or BasaltClient). */
+export function defineModel(runner, name, schema){ return new Model(runner, name, schema); }
+
 // ---- embedded engine (WASM, in-process) ----------------------------------
 export class Basalt {
   constructor(mod) {
@@ -121,6 +224,8 @@ export class Basalt {
   schema() { return JSON.parse(this._schemaFn()).tables; }
   /** Start a query builder on a table. */
   table(name) { return new QueryBuilder(this, name); }
+  /** Define a model (see Model) bound to this database. */
+  model(name, schema) { return new Model(this, name, schema); }
 }
 
 // ---- HTTP client (talks to a running `basalt` server) --------------------
@@ -155,6 +260,8 @@ export class BasaltClient {
   use(db) { this.db = db; return this; }
   /** Start a query builder on a table. */
   table(name) { return new QueryBuilder(this, name); }
+  /** Define a model (see Model) bound to this client. */
+  model(name, schema) { return new Model(this, name, schema); }
 }
 
 export default Basalt;