sqlstack 1.0.17 → 1.0.19

package/readme.md

@@ -2,19 +2,31 @@
 
 **SQL-first data access for Node.js and TypeScript.**
 
-sqlstack lets you write real SQL, keep it organized next to your code, and use small, composable decorators to add behavior—all without ORM complexity. Think of it as a thin, decorator-powered layer on top of your database driver.
+Write real SQL next to your code, and use small, composable decorators to bind, execute, and shape results—without ORM complexity. If you already like dropping down to “raw queries”, this is for you.
+
+## Table of Contents
+
+- [Install](#install)
+- [Quick Start (5 minutes)](#quick-start-5-minutes)
+- [Writing and Extending SQL](#writing-and-extending-sql)
+- [Configuration](#configuration)
+- [SQL Parameters](#sql-parameters)
+- [How It Works](#how-it-works)
+- [Result Types](#result-types)
+- [Error Handling](#error-handling)
+- [Inline SQL](#inline-sql)
+- [Advanced: Direct Database Access](#advanced-direct-database-access)
+- [Database Support](#database-support)
+- [Philosophy](#philosophy)
+- [Contributing](#contributing)
+- [License](#license)
 
-## Key Features
+## Install
 
-- **Real SQL in `.sql` files**: No query builders. SQL is the source of truth.
-- **TypeScript-first**: Declare your own argument and return types; optional JSON Schema validation.
-- **Composable decorators**: `@Single`, `@Only`, `@Exist`, `@Defaults`, `@Page`, `@MissingAsNull`, `@ValidateResult`, `@Transform` (order-independent).
-- **Minimal runtime**: A thin executor layer on top of your DB driver (no ORM overhead).
-- **Multi-database support**: Postgres, MySQL/MariaDB, SQLite (more adapters planned).
-- **Structured errors**: Catch specific errors like `ExactlyOneRowError` and `ValidationError`.
-- **Simple configuration**: Register databases once at startup; set per-class and per-method overrides.
+### Prerequisites
 
-## Install
+- Node.js 18+ (or a modern LTS runtime).
+- Works with both TypeScript and JavaScript; examples use TypeScript syntax.
 
 ```bash
 npm install sqlstack
@@ -34,6 +46,14 @@ npm install better-sqlite3 # SQLite
 
 ## Quick Start (5 minutes)
 
+At a high level, you:
+
+- Register one or more database connections with `SqlStackDB`.
+- Write normal classes with methods, decorate them with `@QueryBinder` and `@Query`.
+- Put SQL in `.sql` files next to those classes.
+
+When you call a decorated method, sqlstack loads the SQL, binds parameters from your method arguments, executes it via the selected adapter, then optionally shapes and validates the results via decorators.
+
 ### 1. Register Your Database
 
 Create a `boot.ts` (or similar) file at your app's entry point:
@@ -53,13 +73,15 @@ Load this file once during app startup.
 ### 2. Create a Repository Class
 
 ```ts
-// users/index.ts
+// src/users/index.ts
 import { QueryBinder, Query, SqlStackError } from "sqlstack";
 
 @QueryBinder() // looks for .sql files in the same folder by default
-export class UsersRepo {
+export class UsersRepository {
   @Query()
-  async findByEmail(_a: { email: string }): Promise<User[]> {
+  async findByEmail(params: { email: string }): Promise<User[]> {
+    // Implementation bodies are never called; this method exists
+    // only to define the TypeScript signature for @Query.
     throw new SqlStackError("replaced by @Query");
   }
 }
@@ -85,36 +107,177 @@ WHERE email = :email;
 ### 4. Use Your Repository
 
 ```ts
-const repo = new UsersRepo();
+const repo = new UsersRepository();
 const users = await repo.findByEmail({ email: "alice@example.com" });
 console.log(users); // User[]
 ```
 
 That's it! The `@Query` decorator intercepts the method, loads and executes the SQL, and returns results.
 
----
+You can then add more methods on the same repository that use sqlstack’s SQL helpers:
 
-## How It Works
+```ts
+@QueryBinder()
+export class UsersRepository {
+  @Query()
+  async findByEmail(params: { email: string }): Promise<User[]> {
+    throw new SqlStackError("replaced by @Query");
+  }
 
-### Execution Pipeline
+  @Query()
+  async updateProfile(params: { id: string; name?: string; status?: string }): Promise<WriteResult> {
+    throw new SqlStackError("replaced by @Query");
+  }
+
+  @Query()
+  async createUser(params: { id: string; email: string; name?: string }): Promise<WriteResult> {
+    throw new SqlStackError("replaced by @Query");
+  }
+
+  @Query()
+  async search(filters: { name?: string; status?: string }): Promise<User[]> {
+    throw new SqlStackError("replaced by @Query");
+  }
+}
+```
 
-sqlstack executes decorators in a **fixed, internal order** regardless of how you stack them:
+**users/updateProfile.sql** — partial update with `:update`:
 
+```sql
+UPDATE users
+  :update(name, status, updated_at = now())
+WHERE id = :id;
 ```
-1. @Defaults        (fill in missing parameter values)
-   ↓
-2. @MissingAsNull   (bind missing named parameters as NULL)
-   ↓
-3. @Query           (execute the SQL)
-   ↓
-4. @Single/@Only/@Exist/@None  (shape the results)
-   ↓
-5. @ValidateResult  (validate against JSON Schema)
-   ↓
-6. @Transform       (map flat rows to nested objects)
+
+**users/createUser.sql** — partial insert with `:insert`:
+
+```sql
+INSERT INTO users :insert(id, email, name, status = 'active');
+```
+
+**users/search.sql** — dynamic filters with `:filter`:
+
+```sql
+SELECT id, email, name, status
+FROM users
+WHERE :filter(name, status);
+```
+
+As your needs grow, you evolve the SQL in these files (and use helpers like `:update`, `:insert`, and `:filter`) while keeping the TypeScript method interfaces—and all call sites—unchanged.
+
+### 5. Go Further with SQL Helpers
+
+Once you’re comfortable with basic `.sql` files, you can start using:
+
+- `:insert(...)` to build column lists and `VALUES` from your named parameters.
+- `:update(...)` to generate `SET` clauses for partial updates.
+- `:filter(...)` to build dynamic `WHERE` / `HAVING` conditions from whichever filters are defined.
+
+These helpers expand into plain SQL; the full behavior and examples are covered in the **Writing and Extending SQL** section below.
+
+## Writing and Extending SQL
+
+sqlstack is **SQL-first**: you write normal SQL, and sqlstack gives you a few helpers to make it easier to evolve queries over time without changing your method interface.
+
+### Plain SQL in Files
+
+Start with regular SQL in `.sql` files:
+
+```sql
+-- users/findByEmail.sql
+SELECT id, email, name, status
+FROM users
+WHERE email = :email;
+```
+
+Later, you can refactor this query (add joins, filters, or switch to a CTE) without changing the TypeScript method that calls it, as long as the returned columns still match your declared return type.
+
+### Extending SQL with Helpers
+
+sqlstack adds a small set of macros that expand into standard SQL, so you keep full control. They are designed to feel like “obvious SQL you would have written by hand.”
+
+#### `:insert(...)`
+
+```sql
+-- invoices/createInvoice.sql
+INSERT INTO invoices :insert(case_id, invoice_number, invoice_date, due_date, status = 'draft');
+```
+
+```ts
+await repo.createInvoice({
+  case_id: 1,
+  invoice_number: "INV-001",
+  invoice_date: "2024-01-01",
+  // due_date: undefined
+});
+```
+
+```sql
+-- expands to
+INSERT INTO invoices (case_id, invoice_number, invoice_date, status)
+VALUES (?, ?, ?, 'draft');
+```
+
+#### `:update(...)`
+
+```sql
+-- invoices/updateInvoice.sql
+UPDATE invoices
+  :update(updated_at = now(), invoice_date, due_date)
+WHERE id = :id;
 ```
 
-You can list decorators in any order; they always execute in this pipeline. See [docs/decorators.md](docs/decorators.md) for details.
+```ts
+await repo.updateInvoice({
+  id: 1,
+  invoice_date: "2024-01-01",
+  // due_date: undefined
+});
+```
+
+```sql
+-- expands to
+UPDATE invoices
+SET updated_at = now(), invoice_date = ?
+WHERE id = ?;
+```
+
+#### `:filter(...)`
+
+```sql
+-- users/search.sql
+SELECT id, email, name, status
+FROM users
+WHERE :filter(
+  name,              -- name = ?
+  age >= :min_age,   -- age >= ?
+  status = 'active', -- literal
+  name LIKE :search, -- name LIKE ?
+  id IN (:ids)       -- id IN (?, ?, ?)
+);
+```
+
+```ts
+await repo.search({
+  name: "Alice",
+  min_age: 18,
+  search: "%ali%",
+  ids: [1, 2, 3],
+});
+```
+
+```sql
+-- expands to
+SELECT id, email, name, status
+FROM users
+WHERE name = ?
+  AND age >= ?
+  AND status = 'active'
+  AND name LIKE ?
+  AND id IN (?, ?, ?);
+```
+
+You can gradually adopt these helpers as your queries become more dynamic; they never hide the SQL that is ultimately sent to the database, they just generate the repetitive parts.
 
 ### Validation Controls
 
@@ -126,6 +289,35 @@ Validation (via `@ValidateResult`) is **enabled by default**. Control it with en
 
 ---
 
+## How It Works
+
+### Execution Pipeline
+
+When you invoke a decorated method, sqlstack:
+
+- Merges any default values into your arguments.
+- Loads SQL (from a file or inline), binds placeholders, and executes it via the chosen adapter.
+- If the adapter returned rows (a SELECT/WITH), it shapes and optionally validates those rows.
+- If the adapter returned a write result (INSERT/UPDATE/DELETE/DDL), it returns that result object as-is.
+
+Decorators run in a **fixed, internal order** regardless of how you stack them:
+
+```
+1. @Defaults        (fill in missing parameter values)
+   ↓
+2. @Query           (execute the SQL)
+   ↓
+3. @Single/@Only/@Exist/@None  (shape the results)
+   ↓
+4. @ValidateResult  (validate against JSON Schema)
+   ↓
+5. @Transform       (map flat rows to nested objects)
+```
+
+You can list decorators in any order; they always execute in this pipeline. See docs/decorators.md for details.
+
+---
+
 ## Configuration
 
 ### Register Databases
@@ -262,57 +454,6 @@ await repo.listByIds({ ids });
 - `null` → SQL `NULL`
 - `undefined` → throws an error; always provide a value or omit the parameter
 
-### Automatic NULL Binding with `@MissingAsNull`
-
-By default, missing or undefined named parameters throw an error. Use `@MissingAsNull` to automatically bind missing parameters as `NULL` instead, enabling partial updates with `COALESCE`/`IFNULL`:
-
-```ts
-import { QueryBinder, MissingAsNull, Query, SqlStackError } from "sqlstack";
-
-@QueryBinder()
-class UsersRepo {
-  @MissingAsNull()
-  @Query({
-    sql: `
-      UPDATE users
-      SET name = COALESCE(:name, name),
-          email = COALESCE(:email, email),
-          status = COALESCE(:status, status)
-      WHERE id = :id
-    `
-  })
-  async updateUserPartial(_a: { id: string; name?: string; email?: string; status?: string }): Promise<any> {
-    throw new SqlStackError("replaced");
-  }
-}
-```
-
-Call with partial arguments:
-```ts
-await repo.updateUserPartial({ id: "u1", name: "Alice Updated" });
-// Only updates name; email and status keep their old values via COALESCE
-```
-
-**Optional: Require Specific Parameters**
-
-Use `@MissingAsNull({ require: ['id'] })` to mark certain parameters as required:
-
-```ts
-@MissingAsNull({ require: ['id'] })
-@Query({
-  sql: `UPDATE users SET name = COALESCE(:name, name) WHERE id = :id`
-})
-async updateUser(_a: { id: string; name?: string }): Promise<any> {
-  throw new SqlStackError("replaced");
-}
-```
-
-If `id` is missing or undefined, an error is thrown. Other parameters default to `NULL`.
-
-**Note:** `@MissingAsNull` only applies to named parameters. Positional parameters (`:arg1`, `:arg2`, etc.) ignore the decorator.
-
----
-
 ## SQL File Resolution
 
 The `@Query()` decorator locates `.sql` files using a simple, predictable rule. See [docs/file_resolution.md](docs/file_resolution.md) for complete details.
@@ -401,7 +542,7 @@ import { QueryBinder, Page, Query, SqlStackError } from "sqlstack";
 class UsersRepo {
   @Page(10) // limit = 10, pages 1-based
   @Query()
-  async listUsers(_a: { page?: number; status?: string }): Promise<User[]> {
+  async listUsers(params: { status?: string; page?: number }): Promise<User[]> {
     throw new SqlStackError();
   }
 }
@@ -424,10 +565,10 @@ await repo.listUsers({ status: "active", page: 2 }); // page 2 of 10 per page
 
 **Zero-based pages:** Use `@Page(10, { zeroBase: true })`.
 
-**Trailing options:** Pass `{ page: N }` as the last argument for both named and positional styles:
+**Trailing options (multiple parameters):** When your method has more than one parameter, pass `{ page: N }` as the last argument:
 ```ts
-await repo.list({ user_id: 1 }, { page: 2 });        // named
-await repo.list(userId, { page: 2 });                // positional
+await repo.list({ user_id: 1 }, { page: 2 }); // named-style params + trailing options
+await repo.list(userId, { page: 2 });         // positional id + trailing options
 ```
 
 ### Result Shape Decorators
@@ -509,11 +650,9 @@ Example schema:
 }
 ```
 
-Validation is **enabled by default**; control with `SQLSTACK_ENABLE_VALIDATION` and `SQLSTACK_DISABLE_VALIDATION` environment variables.
-
 ### `@Transform`
 
-Map flat SQL rows to nested objects using a nesting spec:
+Use `DTO()` with `@Transform` to turn flat joined rows into nested objects, automatically deduplicating parents and children.
 
 ```ts
 import { DTO, Transform, Query, QueryBinder } from 'sqlstack';
@@ -553,20 +692,60 @@ class Repo {
 }
 ```
 
-**Features:**
-- Arbitrary depth: nest collections by adding objects under new keys.
-- Deduplication: nodes are de-duplicated by their `id` at each level.
-- Null-safe: branches with `NULL` child ids are skipped.
-- Order: sibling order follows your SQL `ORDER BY`.
-- Direct mapping: use `'target.path': 'source_column'` to populate nested properties from flat columns.
+Given flat rows like:
+
+```ts
+[
+  { group_id: 'g1', group_name: 'Chest', muscle_id: 'm1', muscle_name: 'Pecs',    exercise_id: 'e1', exercise_name: 'Bench'   },
+  { group_id: 'g1', group_name: 'Chest', muscle_id: 'm1', muscle_name: 'Pecs',    exercise_id: 'e2', exercise_name: 'Incline' },
+  { group_id: 'g1', group_name: 'Chest', muscle_id: 'm2', muscle_name: 'Delts',   exercise_id: 'e3', exercise_name: 'Raise'   },
+]
+```
+
+the transform returns:
+
+```ts
+[
+  {
+    id: 'g1',
+    name: 'Chest',
+    muscles: [
+      {
+        id: 'm1',
+        name: 'Pecs',
+        exercises: [
+          { id: 'e1', name: 'Bench' },
+          { id: 'e2', name: 'Incline' },
+        ],
+      },
+      {
+        id: 'm2',
+        name: 'Delts',
+        exercises: [
+          { id: 'e3', name: 'Raise' },
+        ],
+      },
+    ],
+  },
+];
+```
+
+**Direct mapping with dot paths**
 
-Example with direct mapping:
 ```ts
 // Row: { id: '1', muscle: 'biceps', group: 'arms' }
 const toSimple = DTO({ id: 'id', name: 'muscle', 'group.name': 'group' } as any, { single: true });
 // Result: { id: '1', name: 'biceps', group: { name: 'arms' } }
 ```
 
+**More behaviors**
+
+- If you omit `id` at a level, DTO uses all mapped fields there as a composite identity to dedupe children.
+- Child branches where all identifying columns are `NULL` are skipped (common with LEFT JOINs).
+- Ordering of children follows your SQL `ORDER BY`; sort in SQL, not in the DTO.
+
+See `docs/DTO.md` for a full guide with more patterns and edge cases.
+
 ---
 
 ## Result Types
@@ -629,6 +808,16 @@ type PostgresWriteResult = WriteResult & {
 };
 ```
 
+### Evolving Queries Without Breaking Callers
+
+Your method signature is the contract; the SQL is an implementation detail.
+
+- Call sites depend only on the TypeScript interface (`args` and return type), not on the shape of the underlying SQL file.
+- You can freely refactor a `.sql` file to add joins, filters, or pagination, or switch from a simple query to a CTE, without touching call sites—as long as the returned columns still match your declared return type (and any validation schema).
+- Decorators like `@Transform` make it easy to map new or reorganized SQL results back into the same nested DTO shape, so you can take advantage of more advanced SQL while keeping your public interface stable.
+
+This lets you start with very simple queries and incrementally adopt more complex SQL over time, with TypeScript and optional JSON Schema validation guarding the boundary.
+
 ---
 
 ## Error Handling
@@ -729,7 +918,7 @@ The placeholder style is inferred from the selected database. Override per-metho
 
 **SQL-first:** SQL is the source of truth. No hidden queries.
 
-**Decorators as metadata:** Decorators (`@Defaults`, `@MissingAsNull`, `@Single`, `@Only`, `@Exist`, `@ValidateResult`, `@Transform`) are metadata labels. They always execute in a fixed internal order, so decorator order doesn't matter on your class.
+**Decorators as metadata:** Decorators (`@Defaults`, `@Single`, `@Only`, `@Exist`, `@ValidateResult`, `@Transform`) are metadata labels. They always execute in a fixed internal order, so decorator order doesn't matter on your class.
 
 **Escape hatches:** Use inline SQL for simple queries; keep complex SQL in `.sql` files. You stay in control.