dzql 0.4.3 → 0.4.4

package/README.md

@@ -7,7 +7,7 @@ PostgreSQL-powered framework with automatic CRUD operations, live query subscrip
 - **[Documentation Hub](docs/)** - Complete documentation index
 - **[Getting Started Tutorial](docs/getting-started/tutorial.md)** - Complete tutorial with working todo app
 - **[API Reference](docs/reference/api.md)** - Complete API documentation
-- **[Live Query Subscriptions](docs/getting-started/subscriptions-quick-start.md)** - Real-time denormalized documents (NEW in v0.2.0)
+- **[Live Query Subscriptions](docs/getting-started/subscriptions-quick-start.md)** - Real-time denormalized documents
 - **[Compiler Documentation](docs/compiler/)** - Entity compilation guide and coding standards
 - **[Claude Guide](docs/for-ai/claude-guide.md)** - Development guide for AI assistants
 - **[Venues Example](../venues/)** - Full working application
@@ -32,7 +32,7 @@ await ws.connect();
 const user = await ws.api.save.users({ name: 'Alice' });
 const results = await ws.api.search.users({ filters: { name: 'alice' } });
 
-// NEW in v0.2.0: Live query subscriptions
+// Live query subscriptions
 const { data, unsubscribe } = await ws.api.subscribe_venue_detail(
   { venue_id: 123 },
   (updated) => console.log('Venue changed!', updated)
@@ -59,17 +59,20 @@ See **[Compiler Documentation](docs/compiler/)** for complete usage guide, codin
 ## Testing
 
 ```bash
-# Start test database
-cd tests/test-utils && docker compose up -d
+# From repository root - start test database
+docker compose up -d
+
+# Initialize test database
+bun run test:init
 
 # Run tests
 bun test
 
 # Stop database
-cd tests/test-utils && docker compose down
+docker compose down
 ```
 
-All tests use `bun:test` framework with automatic database setup/teardown. See **[tests/test-utils/README.md](tests/test-utils/README.md)** for details.
+All tests use `bun:test` framework. See **[tests/README.md](../../tests/README.md)** for details.
 
 ## License
 

package/docs/README.md

@@ -7,6 +7,7 @@ Complete documentation for the DZQL PostgreSQL-powered framework.
 New to DZQL? Start here:
 
 - **[Tutorial](getting-started/tutorial.md)** - Complete step-by-step guide with a working todo app
+- **[Interpreter vs Compiler](guides/interpreter-vs-compiler.md)** - Understand the two execution modes
 - **[Subscriptions Quick Start](getting-started/subscriptions-quick-start.md)** - Get real-time subscriptions working in 5 minutes
 
 ## 📖 Guides
@@ -14,6 +15,9 @@ New to DZQL? Start here:
 Feature-specific guides and how-tos:
 
 - **[Live Query Subscriptions](guides/subscriptions.md)** - Real-time denormalized documents
+- **[Many-to-Many Relationships](guides/many-to-many.md)** - Junction table management
+- **[Field Defaults](guides/field-defaults.md)** - Auto-populate fields on create
+- **[Custom Functions](guides/custom-functions.md)** - Extend with PostgreSQL or Bun functions
 - **[Client Stores](guides/client-stores.md)** - Pinia store patterns for Vue.js
 
 ## 📘 Reference
@@ -29,7 +33,7 @@ Complete API documentation:
 - [Quickstart](compiler/QUICKSTART.md) - Get started with the DZQL compiler
 - [Advanced Filters](compiler/ADVANCED_FILTERS.md) - Complex search operators
 - [Coding Standards](compiler/CODING_STANDARDS.md) - Best practices for DZQL code
-- [Comparison](compiler/COMPARISON.md) - DZQL vs other approaches
+- [Comparison](compiler/COMPARISON.md) - Runtime vs compiled side-by-side
 
 ## 🤖 For AI Assistants
 
@@ -40,28 +44,6 @@ Complete API documentation:
 - [npm Package](https://www.npmjs.com/package/dzql)
 - [GitHub Repository](https://github.com/blueshed/dzql)
 - [Issue Tracker](https://github.com/blueshed/dzql/issues)
-- [Changelog](../../../CHANGELOG.md)
-- [Contributing](../../../CONTRIBUTING.md)
-
-## 🏗️ Architecture
-
-Looking for architecture and design docs? See the [repository docs](../../../docs/):
-
-- [Permissions System](../../../docs/architecture/PERMISSIONS.md)
-- [Project Roadmap](../../../docs/architecture/ROADMAP.md)
-- [Subscription Architecture](../../../docs/architecture/SUBSCRIPTIONS_STRATEGY.md)
-
-## 🧪 Development
-
-Contributing to DZQL? See development documentation:
-
-- [TDD Workflow](../../../docs/development/TDD_WORKFLOW.md)
-- [WebSocket Testing](../../../docs/development/WEBSOCKET_TESTING.md)
-- [Claude Web Setup](../../../docs/development/CLAUDE-WEB.md)
-
-## 📦 Package Contents
-
-This documentation is published with the npm package. For repository-wide documentation (contributors, development workflow, architecture), see [`/docs/`](../../../docs/) in the repository root.
 
 ## Need Help?
 

package/docs/for-ai/claude-guide.md

@@ -2,6 +2,38 @@
 
 This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
 
+## Quick Reference Card
+
+```
+DZQL QUICK REFERENCE
+====================
+
+5 Operations:     get, save, delete, lookup, search
+2 Modes:          Interpreter (runtime) | Compiler (static SQL)
+Client API:       ws.api.{operation}.{entity}(params)
+Server API:       db.api.{operation}.{entity}(params, userId)
+
+Entity Registration:
+  dzql.register_entity(
+    table_name,           -- 'todos'
+    label_field,          -- 'title' (for lookups)
+    searchable_fields,    -- ARRAY['title', 'description']
+    fk_includes,          -- '{"org": "organisations"}'
+    soft_delete,          -- false
+    temporal_fields,      -- '{}'
+    notification_paths,   -- '{"ownership": ["@org_id->acts_for..."]}'
+    permission_paths,     -- '{"view": [], "create": [...]}'
+    graph_rules,          -- '{"on_create": {...}, "many_to_many": {...}}'
+    field_defaults        -- '{"owner_id": "@user_id"}'
+  )
+
+M2M id_field naming:  tag_ids (singular + _ids), NOT tags_ids
+Permission [] = public, omitted = denied
+Path syntax: @field->table[filter]{temporal}.target_field
+
+Compile: dzql compile entities.sql -o compiled/
+```
+
 ## Project Overview
 
 DZQL is a PostgreSQL-powered framework that eliminates CRUD boilerplate by providing automatic database operations, real-time WebSocket synchronization, and graph-based relationship management. The core concept: register an entity in PostgreSQL and instantly get 5 standard operations (get, save, delete, lookup, search) plus real-time notifications with zero code.

package/docs/guides/interpreter-vs-compiler.md

@@ -0,0 +1,237 @@
+# Interpreter vs Compiler Mode
+
+DZQL offers two execution modes for your entities. Understanding when to use each is fundamental to getting the best out of the framework.
+
+## Quick Summary
+
+| Aspect | Interpreter | Compiler |
+|--------|-------------|----------|
+| **Setup** | Register entity, use immediately | Register entity, compile, deploy SQL |
+| **Performance** | ~8-12ms per operation | ~2-4ms per operation |
+| **Debugging** | Opaque (dynamic SQL) | Transparent (static SQL) |
+| **Best For** | Development, prototyping | Production, performance-critical |
+
+## How It Works
+
+### Interpreter Mode (Runtime)
+
+Entity configuration is stored as JSON in `dzql.entities` table and parsed at runtime:
+
+```
+Client Request → generic_exec() → Parse JSON config → Build SQL → Execute
+```
+
+**Characteristics:**
+- Zero build step - changes take effect immediately
+- JSON config parsed on every request
+- Dynamic SQL generated at runtime
+- Generic query plans (harder to optimize)
+
+**Usage:**
+```sql
+-- Register entity
+SELECT dzql.register_entity('todos', 'title', ARRAY['title'], ...);
+
+-- Use immediately via generic executor
+SELECT dzql.generic_exec('save', 'todos', '{"title": "Buy milk"}'::jsonb, 1);
+```
+
+### Compiler Mode (Static)
+
+Entity configuration is compiled into dedicated PostgreSQL functions:
+
+```
+Entity Definition → dzql compile → Static SQL Functions → Deploy → Execute
+```
+
+**Characteristics:**
+- Build step required
+- No JSON parsing at runtime
+- Static SQL with specific query plans
+- PostgreSQL can optimize and cache plans
+
+**Usage:**
+```bash
+# Compile entities to SQL
+dzql compile entities.sql -o compiled/
+
+# Deploy to database
+psql < compiled/entities.sql
+```
+
+```sql
+-- Use compiled functions directly
+SELECT save_todos('{"title": "Buy milk"}'::jsonb, 1);
+```
+
+## The Server Automatically Chooses
+
+The DZQL server (`db.js`) automatically tries compiled functions first:
+
+```javascript
+// In callDZQLOperation()
+try {
+  // Try compiled function: save_todos()
+  const result = await sql.unsafe(`SELECT save_todos($1, $2)`, [data, userId]);
+  return result[0].result;
+} catch (error) {
+  // If compiled function doesn't exist, fall back to interpreter
+  if (error.message.includes('save_todos') && error.code === '42883') {
+    return await sql`SELECT dzql.generic_exec('save', 'todos', ${data}, ${userId})`;
+  }
+  throw error;
+}
+```
+
+This means you can:
+1. Start with interpreter mode during development
+2. Compile and deploy when ready for production
+3. Mix and match - some entities compiled, others interpreted
+
+## Performance Comparison
+
+### Interpreter (Runtime Parsing)
+
+```sql
+SELECT dzql.generic_exec('save', 'venues', '{"name": "MSG"}'::jsonb, 42);
+```
+
+**Execution steps:**
+1. Fetch entity config from `dzql.entities` (table lookup)
+2. Parse `permission_paths` JSONB
+3. Build permission query dynamically
+4. Parse `graph_rules` JSONB
+5. Execute rules via dynamic SQL
+6. Parse `notification_paths` JSONB
+7. Resolve paths dynamically
+8. Execute the actual save
+
+**Cost:** ~8-12ms, 3-5 JSONB parses, unpredictable query plans
+
+### Compiler (Pre-built Functions)
+
+```sql
+SELECT save_venues('{"name": "MSG"}'::jsonb, 42);
+```
+
+**Execution steps:**
+1. Call `can_update_venues()` - pre-compiled permission check
+2. Execute INSERT/UPDATE - direct SQL
+3. Call `graph_venues_on_create()` - pre-compiled graph rules
+4. Call `resolve_notification_paths_venues()` - pre-compiled
+5. Done
+
+**Cost:** ~2-4ms, 0 JSONB parses, optimized query plans
+
+## When to Use Each
+
+### Use Interpreter When:
+- Rapid prototyping and development
+- Schema changes frequently
+- Learning DZQL concepts
+- Small applications with low traffic
+- Need maximum flexibility
+
+### Use Compiler When:
+- Production deployments
+- Performance is critical
+- Need predictable query performance
+- Want reviewable/auditable SQL
+- Large teams (generated SQL is easy to review)
+- Complex permission or graph rules
+
+### Recommended Workflow:
+1. **Development:** Use interpreter for fast iteration
+2. **Staging:** Compile and test performance
+3. **Production:** Deploy compiled functions
+
+## Compiling Entities
+
+### Via CLI
+
+```bash
+# Single file
+dzql compile database/entities.sql -o compiled/
+
+# Multiple files
+dzql compile database/*.sql -o compiled/
+```
+
+### Programmatically
+
+```javascript
+import { DZQLCompiler } from 'dzql/compiler';
+
+const compiler = new DZQLCompiler();
+const result = compiler.compileFromSQL(sqlContent);
+
+console.log(result.sql);  // Generated PostgreSQL functions
+```
+
+### What Gets Generated
+
+For each entity, the compiler generates:
+
+| Function | Purpose |
+|----------|---------|
+| `get_{entity}(id, user_id)` | Retrieve single record |
+| `save_{entity}(data, user_id)` | Create or update |
+| `delete_{entity}(id, user_id)` | Delete record |
+| `lookup_{entity}(term, user_id)` | Autocomplete search |
+| `search_{entity}(filters, user_id)` | Paginated search |
+| `can_view_{entity}(user_id, record)` | Permission check |
+| `can_create_{entity}(user_id, record)` | Permission check |
+| `can_update_{entity}(user_id, record)` | Permission check |
+| `can_delete_{entity}(user_id, record)` | Permission check |
+
+## Debugging
+
+### Interpreter Mode
+
+Debugging is harder because SQL is generated dynamically:
+
+```sql
+-- You see this
+EXPLAIN ANALYZE SELECT dzql.generic_exec('save', 'venues', '...');
+
+-- But the actual query is hidden inside
+```
+
+### Compiler Mode
+
+Standard PostgreSQL tools work:
+
+```sql
+-- See the actual function
+\sf save_venues
+
+-- Analyze performance
+EXPLAIN ANALYZE SELECT save_venues('{"name": "MSG"}'::jsonb, 42);
+
+-- Check slow queries
+SELECT * FROM pg_stat_statements WHERE query LIKE '%save_venues%';
+```
+
+## Feature Parity
+
+Both modes support the same features:
+
+| Feature | Interpreter | Compiler |
+|---------|-------------|----------|
+| CRUD operations | ✅ | ✅ |
+| Permission paths | ✅ | ✅ |
+| Graph rules | ✅ | ✅ |
+| Notification paths | ✅ | ✅ |
+| FK includes | ✅ | ✅ |
+| Many-to-many | ✅ | ✅ |
+| Field defaults | ✅ | ✅ |
+| Soft delete | ✅ | ✅ |
+| Temporal fields | ✅ | ✅ |
+
+The difference is purely in execution speed and debuggability, not functionality.
+
+## See Also
+
+- [Compiler Quickstart](../compiler/QUICKSTART.md) - Get started with compilation
+- [Compiler Comparison](../compiler/COMPARISON.md) - Detailed side-by-side analysis
+- [API Reference](../reference/api.md) - The 5 operations

package/docs/guides/many-to-many.md

@@ -161,6 +161,23 @@ SELECT dzql.register_entity(
 | `id_field` | Yes | Field name for ID array in API | `"tag_ids"` |
 | `expand` | No | Include full objects (default: false) | `false` or `true` |
 
+### Naming Convention for `id_field`
+
+**Important:** The `id_field` should use the **singular** form of the target entity, not plural:
+
+| Target Entity | Correct `id_field` | Wrong |
+|---------------|-------------------|-------|
+| `tags` | `tag_ids` | `tags_ids` |
+| `roles` | `role_ids` | `roles_ids` |
+| `categories` | `category_ids` | `categories_ids` |
+| `users` | `user_ids` | `users_ids` |
+
+This convention matches common ORM patterns and is more readable:
+- `tag_ids: [1, 2, 3]` reads as "tag IDs"
+- `tags_ids: [1, 2, 3]` reads awkwardly as "tags IDs"
+
+Using the wrong naming will cause the M2M sync to fail because the generated code looks for a field name that doesn't match what clients send.
+
 ### The `expand` Flag
 
 Controls whether full related objects are included in responses:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dzql",
-  "version": "0.4.3",
+  "version": "0.4.4",
   "description": "PostgreSQL-powered framework with zero boilerplate CRUD operations and real-time WebSocket synchronization",
   "type": "module",
   "main": "src/server/index.js",
@@ -24,7 +24,7 @@
   ],
   "scripts": {
     "test": "bun test ../../tests/core/*.test.js",
-    "prepublishOnly": "echo '✅ Publishing DZQL v0.3.5...'"
+    "prepublishOnly": "echo '✅ Publishing DZQL...'  "
   },
   "dependencies": {
     "jose": "^6.1.0",