sqlite-zod-orm 3.0.0 → 3.2.1

package/README.md

@@ -1,314 +1,436 @@
-# SatiDB 🗄️
+# sqlite-zod-orm
 
-A modern, type-safe SQLite wrapper with **reactive editing**, **automatic relationships**, and **fluent APIs**. Built for TypeScript/JavaScript with Zod schema validation.
+Type-safe SQLite ORM for Bun. Define schemas with Zod, get a fully-typed database with automatic relationships, lazy navigation, and zero SQL.
 
-## ✨ Key Features
-
-- **🔄 Reactive Editing**: Change properties directly, auto-persists to database
-- **📊 Type-Safe Schemas**: Powered by Zod for runtime validation
-- **🔗 Automatic Relationships**: Belongs-to, one-to-many, many-to-many support
-- **📺 Event Subscriptions**: Listen to insert/update/delete events
-- **🏗️ Explicit Junction Tables**: Full control over many-to-many relationships
-- **🚀 Fluent API**: Intuitive, chainable operations
-- **⚡ Zero-Config**: Works with Bun SQLite out of the box
+```bash
+bun add sqlite-zod-orm
+```
 
-## 🚀 Quick Start
+## Quick Start
 
 ```typescript
-import { MyDatabase, z } from './satidb';
-
-// Define schemas with relationships
-const AuthorSchema = z.object({
-  name: z.string(),
-  posts: z.lazy(() => z.array(PostSchema)).optional(),
+import { Database, z } from 'sqlite-zod-orm';
+
+const db = new Database(':memory:', {
+  users: z.object({
+    name: z.string(),
+    email: z.string().email(),
+    role: z.string().default('member'),
+  }),
 });
 
-const PostSchema = z.object({
-  title: z.string(),
-  content: z.string(),
-  author: z.lazy(() => AuthorSchema).optional(),
-});
+const alice = db.users.insert({ name: 'Alice', email: 'alice@example.com', role: 'admin' });
+const admin = db.users.select().where({ role: 'admin' }).get();  // single row
+const all   = db.users.select().all();                           // all rows
+```
+
+---
+
+## Defining Relationships
+
+FK columns go in your schema. The `relations` config declares which FK points to which table:
 
-// Create database
-const db = new MyDatabase(':memory:', {
+```typescript
+const AuthorSchema = z.object({ name: z.string(), country: z.string() });
+const BookSchema   = z.object({ title: z.string(), year: z.number(), author_id: z.number().optional() });
+
+const db = new Database(':memory:', {
   authors: AuthorSchema,
-  posts: PostSchema,
+  books:   BookSchema,
+}, {
+  relations: {
+    books: { author_id: 'authors' },
+  },
 });
+```
 
-// Use it!
-const author = db.authors.insert({ name: 'Jane Doe' });
-const post = author.posts.push({ title: 'Hello World', content: '...' });
+`books: { author_id: 'authors' }` tells the ORM that `books.author_id` is a foreign key referencing `authors.id`. The ORM automatically:
 
-// Reactive editing - just change properties!
-post.title = 'Hello TypeScript'; // Automatically saved to DB
-```
+- Adds `FOREIGN KEY (author_id) REFERENCES authors(id)` constraint
+- Infers the inverse one-to-many `authors → books`
+- Enables lazy navigation: `book.author()` and `author.books()`
+- Enables fluent joins: `db.books.select().join(db.authors).all()`
+
+The nav method name is derived by stripping `_id` from the FK column: `author_id` → `author()`.
 
-## 📖 Core Concepts
+---
 
-### Schemas & Relationships
+## Querying — `select()` is the only path
 
-Define entities using Zod schemas with lazy relationships:
+All queries go through `select()`:
 
 ```typescript
-// One-to-many: Author has many Posts
-const AuthorSchema = z.object({
-  name: z.string(),
-  posts: z.lazy(() => z.array(PostSchema)).optional(), // one-to-many
-});
+// Single row
+const user = db.users.select().where({ id: 1 }).get();
 
-// Belongs-to: Post belongs to Author  
-const PostSchema = z.object({
-  title: z.string(),
-  author: z.lazy(() => AuthorSchema).optional(), // belongs-to
-});
+// All matching rows
+const admins = db.users.select().where({ role: 'admin' }).all();
+
+// All rows
+const everyone = db.users.select().all();
+
+// Count
+const count = db.users.select().count();
 ```
 
-### Explicit Junction Tables
+### Operators
 
-For many-to-many relationships, define explicit junction entities with additional fields:
+`$gt` `$gte` `$lt` `$lte` `$ne` `$in`
 
 ```typescript
-// Many-to-many with rich junction table
-const PostTagSchema = z.object({
-  post: z.lazy(() => PostSchema).optional(),
-  tag: z.lazy(() => TagSchema).optional(),
-  appliedAt: z.date().default(() => new Date()),
-  appliedBy: z.string(),
-  priority: z.number().default(0),
-});
+const topScorers = db.users.select()
+  .where({ score: { $gt: 50 } })
+  .orderBy('score', 'desc')
+  .limit(10)
+  .all();
+```
 
-// Usage
-const postTag = post.postTags.push({ 
-  tagId: tag.id, 
-  appliedBy: 'editor',
-  priority: 5 
+### `$or`
+
+```typescript
+const results = db.users.select()
+  .where({ $or: [{ role: 'admin' }, { score: { $gt: 50 } }] })
+  .all();
+```
+
+### Fluent Join
+
+Auto-infers foreign keys from relationships:
+
+```typescript
+const rows = db.books.select('title', 'year')
+  .join(db.authors, ['name', 'country'])
+  .where({ year: { $gt: 1800 } })
+  .orderBy('year', 'asc')
+  .all();
+// → [{ title: 'War and Peace', year: 1869, authors_name: 'Leo Tolstoy', ... }]
+```
+
+### `db.query()` — Proxy Query (SQL-like)
+
+Full SQL-like control with destructured table aliases:
+
+```typescript
+const rows = db.query(c => {
+  const { authors: a, books: b } = c;
+  return {
+    select: { author: a.name, book: b.title, year: b.year },
+    join: [[b.author_id, a.id]],
+    where: { [a.country]: 'Russia' },
+    orderBy: { [b.year]: 'asc' },
+  };
 });
 ```
 
-**Why explicit junction tables?**
-- Full control over additional fields (timestamps, metadata, etc.)
-- Junction tables are queryable entities themselves
-- More flexible than auto-generated tables
-- Clearer data modeling
+---
 
-## 🎯 API Reference
+## Lazy Navigation
 
-### Creating & Querying
+Relationship fields become callable methods on entities. The method name is the FK column with `_id` stripped:
 
 ```typescript
-// Insert new entities
-const user = db.users.insert({ name: 'Alice', email: 'alice@example.com' });
+// belongs-to: book.author_id → book.author()
+const book = db.books.select().where({ title: 'War and Peace' }).get()!;
+const author = book.author();       // → { name: 'Leo Tolstoy', ... }
 
-// Find multiple with conditions
-const activeUsers = db.users.find({ active: true, $limit: 10 });
+// one-to-many: author → books
+const books = tolstoy.books();      // → [{ title: 'War and Peace' }, ...]
 
-// Get single entity
-const user = db.users.get({ email: 'alice@example.com' });
-const userById = db.users.get('user-id-123');
+// Chain
+const allByAuthor = book.author().books();
+```
 
-// Update
-const updated = db.users.update('user-id', { name: 'Alice Smith' });
+---
+
+## CRUD
+
+```typescript
+// Insert (defaults fill in automatically)
+const user = db.users.insert({ name: 'Alice', role: 'admin' });
+
+// Insert with FK
+const book = db.books.insert({ title: 'War and Peace', year: 1869, author_id: tolstoy.id });
+
+// Read
+const one   = db.users.select().where({ id: 1 }).get();
+const some  = db.users.select().where({ role: 'admin' }).all();
+const all   = db.users.select().all();
+const count = db.users.select().count();
+
+// Entity-level update
+user.update({ role: 'superadmin' });
+
+// Update by ID
+db.users.update(1, { role: 'superadmin' });
+
+// Fluent update with WHERE
+db.users.update({ role: 'member' }).where({ role: 'guest' }).exec();
 
 // Upsert
-const user = db.users.upsert({ email: 'alice@example.com', name: 'Alice' });
+db.users.upsert({ name: 'Alice' }, { name: 'Alice', role: 'admin' });
 
 // Delete
-db.users.delete('user-id');
+db.users.delete(1);
 ```
 
-### Relationships
+### Auto-Persist Proxy
+
+Setting a property on an entity auto-updates the DB:
 
 ```typescript
-// One-to-many: Add related entities
-const post = author.posts.push({ title: 'New Post', content: '...' });
+const alice = db.users.select().where({ id: 1 }).get()!;
+alice.score = 200;    // → UPDATE users SET score = 200 WHERE id = 1
+```
 
-// Query related entities
-const authorPosts = author.posts(); // All posts
-const recentPosts = author.posts({ $limit: 5, $sortBy: 'createdAt:desc' });
+---
 
-// Belongs-to: Navigate relationships
-const postAuthor = post.author();
+## Schema Validation
 
-// Get specific related entity
-const specificPost = author.post('post-id-123');
+Zod validates every insert and update at runtime:
+
+```typescript
+db.users.insert({ name: '', email: 'bad', age: -1 });  // throws ZodError
 ```
 
-### Reactive Editing
+---
+
+## Indexes
 
 ```typescript
-const user = db.users.get('user-id');
+const db = new Database(':memory:', schemas, {
+  indexes: {
+    users: ['email', ['name', 'role']],
+    books: ['author_id', 'year'],
+  },
+});
+```
 
-// Just change properties - automatically persists!
-user.name = 'New Name';
-user.email = 'new@email.com';
-user.active = false;
+---
 
-// No need to call .save() or .update()
-// Changes are immediately persisted to database
-```
+## Reactivity — Three Ways to React to Changes
+
+sqlite-zod-orm provides three reactivity mechanisms for different use cases:
 
-### Event Subscriptions
+| System | Detects | Scope | Overhead | Best for |
+|---|---|---|---|---|
+| **CRUD Events** | insert, update, delete | In-process, per table | Zero (synchronous) | Side effects, caching, logs |
+| **Smart Polling** | insert, delete, update* | Any query result | Lightweight fingerprint check | Live UI, dashboards |
+| **Change Tracking** | insert, update, delete | Per table or global | Trigger-based WAL | Cross-process sync, audit |
+
+\* Smart polling detects UPDATEs automatically when `changeTracking` is enabled. Without it, only inserts and deletes are detected.
+
+---
+
+### 1. CRUD Events — `db.table.subscribe(event, callback)`
+
+Synchronous callbacks fired immediately after each CRUD operation. Zero overhead; the callback runs inline.
 
 ```typescript
-// Listen to database events
+// Listen for new users
 db.users.subscribe('insert', (user) => {
-  console.log(`New user: ${user.name}`);
+  console.log('New user:', user.name);   // fires on every db.users.insert(...)
 });
 
-db.posts.subscribe('update', (post) => {
-  console.log(`Post updated: ${post.title}`);
+// Listen for updates
+db.users.subscribe('update', (user) => {
+  console.log('Updated:', user.name, '→', user.role);
 });
 
+// Listen for deletes
 db.users.subscribe('delete', (user) => {
-  console.log(`User deleted: ${user.name}`);
+  console.log('Deleted:', user.name);
 });
+
+// Stop listening
+db.users.unsubscribe('update', myCallback);
 ```
 
-### Transactions
+**Use cases:**
+- Invalidating a cache after writes
+- Logging / audit trail
+- Sending notifications
+- Keeping derived data in sync (e.g., a counter table)
+
+The database also extends Node's `EventEmitter`, so you can use `db.on()`:
 
 ```typescript
-const result = db.transaction(() => {
-  const author = db.authors.insert({ name: 'Jane' });
-  const post1 = author.posts.push({ title: 'Post 1' });
-  const post2 = author.posts.push({ title: 'Post 2' });
-  return { author, posts: [post1, post2] };
+db.on('insert', (tableName, entity) => {
+  console.log(`New row in ${tableName}:`, entity.id);
 });
 ```
 
-## 🔗 Relationship Types
+---
 
-### One-to-Many
+### 2. Smart Polling — `select().subscribe(callback, options)`
 
-```typescript
-const PersonalitySchema = z.object({
-  name: z.string(),
-  chats: z.lazy(() => z.array(ChatSchema)).optional(),
-});
-
-const ChatSchema = z.object({
-  title: z.string(),
-  personality: z.lazy(() => PersonalitySchema).optional(),
-});
+Query-level polling that watches *any query result* for changes. Instead of re-fetching all rows every tick, it runs a **lightweight fingerprint query** (`SELECT COUNT(*), MAX(id)`) with the same WHERE clause. The full query only re-executes when the fingerprint changes.
 
-// Usage
-const personality = db.personalities.insert({ name: 'Assistant' });
-const chat = personality.chats.push({ title: 'Help Session' });
-const chatPersonality = chat.personality(); // Navigate back
+```typescript
+// Watch for admin list changes, poll every second
+const unsub = db.users.select()
+  .where({ role: 'admin' })
+  .orderBy('name', 'asc')
+  .subscribe((admins) => {
+    console.log('Admin list:', admins.map(a => a.name));
+  }, { interval: 1000 });
+
+// Stop watching
+unsub();
 ```
 
-### Many-to-Many (Explicit Junction)
+**Options:**
 
-```typescript
-// Define all three entities
-const UserSchema = z.object({
-  name: z.string(),
-  likes: z.lazy(() => z.array(LikeSchema)).optional(),
-});
+| Option | Default | Description |
+|---|---|---|
+| `interval` | `500` | Polling interval in milliseconds |
+| `immediate` | `true` | Whether to fire the callback immediately with the current result |
 
-const ProductSchema = z.object({
-  name: z.string(),
-  likes: z.lazy(() => z.array(LikeSchema)).optional(),
-});
+**How the fingerprint works:**
 
-// Junction table with additional fields
-const LikeSchema = z.object({
-  user: z.lazy(() => UserSchema).optional(),
-  product: z.lazy(() => ProductSchema).optional(),
-  rating: z.number().min(1).max(5),
-  likedAt: z.date().default(() => new Date()),
-});
+```
+┌─────────────────────────────────────┐
+│  Every {interval}ms:                │
+│                                     │
+│  1. Run: SELECT COUNT(*), MAX(id)   │
+│     FROM users WHERE role = 'admin' │
+│                                     │  ← fast, no data transfer
+│  2. Compare fingerprint to last     │
+│                                     │
+│  3. If changed → re-run full query  │  ← only when needed
+│     and call your callback          │
+└─────────────────────────────────────┘
+```
 
-// Usage
-const user = db.users.insert({ name: 'Alice' });
-const product = db.products.insert({ name: 'Laptop' });
+**What it detects:**
 
-// Create relationship with metadata
-const like = user.likes.push({ 
-  productId: product.id, 
-  rating: 5 
-});
+| Operation | Without `changeTracking` | With `changeTracking` |
+|---|---|---|
+| INSERT | ✅ (MAX(id) increases) | ✅ |
+| DELETE | ✅ (COUNT changes) | ✅ |
+| UPDATE | ❌ (fingerprint unchanged) | ✅ (change sequence bumps) |
 
-// Query from both sides
-const userLikes = user.likes().map(l => l.product()); // Products Alice likes
-const productLikers = product.likes().map(l => l.user()); // Users who like laptop
+> **Tip:** Enable `changeTracking: true` if you need `.subscribe()` to react to UPDATEs.
+> The overhead is minimal — one trigger per table that appends to a `_changes` log.
 
-// Junction table is a full entity
-like.rating = 4; // Reactive update on junction table!
-```
+**Use cases:**
+- Live dashboards (poll every 1-5s)
+- Real-time chat message lists
+- Auto-refreshing data tables
+- Watching filtered subsets of data
 
-## 📝 Advanced Usage
+---
 
-### Custom ID Generation
+### 3. Change Tracking — `changeTracking: true`
 
-IDs are automatically generated based on entity data hash. For custom IDs:
+A trigger-based WAL (write-ahead log) that records every INSERT, UPDATE, and DELETE to a `_changes` table. This is the foundation for cross-process sync and audit trails.
 
 ```typescript
-const entity = db.entities.insert({ 
-  id: 'custom-id-123', // Explicit ID
-  name: 'Custom Entity' 
+const db = new Database(':memory:', schemas, {
+  changeTracking: true,
 });
 ```
 
-### Query Options
+When enabled, the ORM creates:
+- A `_changes` table: `(id, table_name, row_id, action, changed_at)`
+- An index on `(table_name, id)` for fast lookups
+- Triggers on each table for INSERT, UPDATE, and DELETE
+
+**Reading changes:**
 
 ```typescript
-const results = db.posts.find({
-  published: true,
-  $limit: 10,
-  $offset: 20,
-  $sortBy: 'createdAt:desc'
-});
-```
+// Get the current sequence number (latest change ID)
+const seq = db.getChangeSeq();            // global
+const seq = db.getChangeSeq('users');     // per table
 
-### Schema Validation
+// Get all changes since a sequence number
+const changes = db.getChangesSince(0);    // all changes ever
+const changes = db.getChangesSince(seq);  // new changes since seq
 
-All data is validated against Zod schemas:
+// Each change looks like:
+// { id: 42, table_name: 'users', row_id: 7, action: 'UPDATE', changed_at: '2024-...' }
+```
 
-```typescript
-const UserSchema = z.object({
-  name: z.string().min(2),
-  email: z.string().email(),
-  age: z.number().optional(),
-});
+**Polling for changes (external sync pattern):**
 
-// Throws validation error if invalid
-const user = db.users.insert({ 
-  name: 'A', // Too short!
-  email: 'invalid-email' // Invalid format!
-});
+```typescript
+let lastSeq = 0;
+
+setInterval(() => {
+  const changes = db.getChangesSince(lastSeq);
+  if (changes.length > 0) {
+    lastSeq = changes[changes.length - 1].id;
+    for (const change of changes) {
+      console.log(`${change.action} on ${change.table_name} row ${change.row_id}`);
+    }
+  }
+}, 1000);
 ```
 
-## 🎯 Best Practices
+**Use cases:**
+- Syncing between processes (e.g., worker → main thread)
+- Building an event-sourced system
+- Replication to another database
+- Audit logging with timestamps
+- Powering smart polling UPDATE detection
 
-1. **Use Explicit Junction Tables**: Define junction entities with additional fields rather than relying on auto-generated tables
-2. **Leverage Reactive Editing**: Change properties directly instead of calling update methods
-3. **Subscribe to Events**: Use subscriptions for real-time updates and side effects
-4. **Validate with Zod**: Define comprehensive schemas with proper validation rules
-5. **Use Transactions**: Wrap related operations in transactions for data consistency
+---
 
-## 🔄 Migration from Traditional ORMs
+### Choosing the Right System
 
-```typescript
-// Traditional ORM
-const user = await User.findById(id);
-user.name = 'New Name';
-await user.save(); // Explicit save
-
-// SatiDB
-const user = db.users.get(id);
-user.name = 'New Name'; // Automatically saved!
 ```
+Do you need to react to your own writes?
+  → CRUD Events (db.table.subscribe)
 
-## 🛠️ Requirements
+Do you need to watch a query result set?
+  → Smart Polling (select().subscribe)
+  → Enable changeTracking if you need UPDATE detection
 
-- Bun runtime with SQLite support
-- TypeScript (recommended) or JavaScript
-- Zod for schema validation
+Do you need cross-process sync or audit?
+  → Change Tracking (changeTracking: true + getChangesSince)
+```
+
+All three systems can be used together. `changeTracking` enhances smart polling automatically — no code changes needed.
 
-## 📄 License
+---
 
-MIT License - feel free to use in your projects!
+## Examples & Tests
+
+```bash
+bun examples/example.ts    # comprehensive demo
+bun test                    # 91 tests
+```
 
 ---
 
-**SatiDB** - Reactive, type-safe database operations made simple. 🚀
+## API Reference
+
+| Method | Description |
+|---|---|
+| `new Database(path, schemas, options?)` | Create database with Zod schemas |
+| **Querying** | |
+| `db.table.select(...cols?).where(filter).get()` | Single row |
+| `db.table.select(...cols?).where(filter).all()` | Array of rows |
+| `db.table.select().count()` | Count rows |
+| `db.table.select().join(db.other, cols?).all()` | Fluent join (auto FK) |
+| `db.query(c => { ... })` | Proxy callback (SQL-like JOINs) |
+| **Writing** | |
+| `db.table.insert(data)` | Insert with validation |
+| `db.table.update(id, data)` | Update by ID |
+| `db.table.update(data).where(filter).exec()` | Fluent update |
+| `db.table.upsert(match, data)` | Insert or update |
+| `db.table.delete(id)` | Delete by ID |
+| **Navigation** | |
+| `entity.navMethod()` | Lazy navigation (FK name minus `_id`) |
+| `entity.update(data)` | Update entity in-place |
+| `entity.delete()` | Delete entity |
+| **Reactivity** | |
+| `db.table.subscribe(event, cb)` | CRUD events: `'insert'`, `'update'`, `'delete'` |
+| `db.table.unsubscribe(event, cb)` | Remove CRUD event listener |
+| `db.on(event, cb)` | EventEmitter: listen across all tables |
+| `select().subscribe(cb, opts?)` | Smart polling (fingerprint-based) |
+| `db.getChangeSeq(table?)` | Current change sequence number |
+| `db.getChangesSince(seq, table?)` | Changes since sequence (change tracking) |
+
+## License
+
+MIT