dzql 0.2.3 → 0.3.0

package/docs/guides/custom-functions.md

@@ -0,0 +1,362 @@
+# Custom Functions
+
+Add custom business logic to your entities with automatic compilation support.
+
+## Overview
+
+DZQL's compiler now automatically includes custom SQL functions defined after entity registration. This eliminates the need to manually maintain functions in multiple locations.
+
+## Benefits
+
+- **Single Source of Truth** - Define functions once in entity files
+- **No Manual Syncing** - Compiler handles everything
+- **Functions Stay With Entities** - Clear organization
+- **Automatic Registration** - Registry entries included
+
+## How It Works
+
+### Before (Manual Duplication)
+
+You had to maintain functions in two places:
+
+```sql
+-- entities/calendar.sql (source)
+SELECT dzql.register_entity('tags', ...);
+
+CREATE FUNCTION toggle_resource_tag(...) RETURNS JSONB AS $$ ... $$;
+INSERT INTO dzql.registry (fn_regproc) VALUES ('toggle_resource_tag'::regproc);
+```
+
+Then manually copy to:
+
+```sql
+-- init_db/002_schema.sql (deployment)
+CREATE FUNCTION toggle_resource_tag(...) RETURNS JSONB AS $$ ... $$;
+INSERT INTO dzql.registry (fn_regproc) VALUES ('toggle_resource_tag'::regproc);
+```
+
+**Problems:**
+- Functions exist in two places
+- Easy to forget to sync
+- Unclear which is source of truth
+
+### After (Automatic Pass-through)
+
+Just define functions once after entity registration:
+
+```sql
+-- entities/calendar.sql
+SELECT dzql.register_entity('tags', 'name', ...);
+
+-- Custom function - automatically passed through by compiler!
+CREATE OR REPLACE FUNCTION toggle_resource_tag(
+  p_user_id INT,
+  p_resource_id INT,
+  p_tag_id INT
+) RETURNS JSONB AS $$
+DECLARE
+  v_exists BOOLEAN;
+BEGIN
+  SELECT EXISTS(
+    SELECT 1 FROM resource_tags
+    WHERE resource_id = p_resource_id AND tag_id = p_tag_id
+  ) INTO v_exists;
+
+  IF v_exists THEN
+    DELETE FROM resource_tags
+    WHERE resource_id = p_resource_id AND tag_id = p_tag_id;
+  ELSE
+    INSERT INTO resource_tags (resource_id, tag_id)
+    VALUES (p_resource_id, p_tag_id);
+  END IF;
+
+  RETURN jsonb_build_object('success', true, 'exists', NOT v_exists);
+END;
+$$ LANGUAGE plpgsql SECURITY DEFINER;
+
+-- Register for RPC access
+INSERT INTO dzql.registry (fn_regproc)
+VALUES ('toggle_resource_tag'::regproc);
+```
+
+After `dzql compile`, the custom function automatically appears in:
+- `init_db/tags.sql` under "Custom Functions" section
+
+## What Gets Passed Through
+
+The compiler extracts:
+
+1. **CREATE FUNCTION statements**
+   ```sql
+   CREATE FUNCTION my_function(...) RETURNS ... AS $$ ... $$;
+   CREATE OR REPLACE FUNCTION my_function(...) RETURNS ... AS $$ ... $$;
+   ```
+
+2. **Registry registrations**
+   ```sql
+   INSERT INTO dzql.registry (fn_regproc) VALUES ('my_function'::regproc);
+   ```
+
+3. **Alternative registration syntax** (if you use it)
+   ```sql
+   SELECT dzql.register_function('my_function');
+   ```
+
+## Scope
+
+Custom functions are extracted from **after** the entity registration until:
+- The next `dzql.register_entity()` call, OR
+- End of file
+
+```sql
+-- Entity 1
+SELECT dzql.register_entity('users', ...);
+CREATE FUNCTION user_helper() ...;  -- Included with users entity
+
+-- Entity 2
+SELECT dzql.register_entity('posts', ...);
+CREATE FUNCTION post_helper() ...;  -- Included with posts entity
+```
+
+Each entity gets its own custom functions isolated in the compiled output.
+
+## Compiled Output Format
+
+```sql
+-- ============================================================================
+-- DZQL Compiled Functions for: tags
+-- Generated: 2025-11-20T15:00:00.000Z
+-- ============================================================================
+
+-- [Generated CRUD functions: get_tags, save_tags, delete_tags, etc.]
+
+-- ============================================================================
+-- Custom Functions for: tags
+-- Pass-through from entity definition
+-- ============================================================================
+
+CREATE OR REPLACE FUNCTION toggle_resource_tag(...) ...;
+
+INSERT INTO dzql.registry (fn_regproc) VALUES ('toggle_resource_tag'::regproc);
+```
+
+## Use Cases
+
+### Toggle Relationships
+
+```sql
+CREATE OR REPLACE FUNCTION toggle_favorite(
+  p_user_id INT,
+  p_item_id INT
+) RETURNS JSONB AS $$
+  -- Toggle logic here
+$$ LANGUAGE plpgsql SECURITY DEFINER;
+```
+
+### Computed Fields
+
+```sql
+CREATE OR REPLACE FUNCTION calculate_item_score(
+  p_user_id INT,
+  p_item_id INT
+) RETURNS JSONB AS $$
+  -- Scoring logic here
+$$ LANGUAGE plpgsql;
+```
+
+### Bulk Operations
+
+```sql
+CREATE OR REPLACE FUNCTION bulk_assign_tags(
+  p_user_id INT,
+  p_resource_ids INT[],
+  p_tag_id INT
+) RETURNS JSONB AS $$
+  -- Bulk assignment here
+$$ LANGUAGE plpgsql SECURITY DEFINER;
+```
+
+### Complex Business Logic
+
+```sql
+CREATE OR REPLACE FUNCTION approve_workflow(
+  p_user_id INT,
+  p_document_id INT
+) RETURNS JSONB AS $$
+  -- Multi-step approval logic
+$$ LANGUAGE plpgsql SECURITY DEFINER;
+```
+
+## Best Practices
+
+### 1. Keep Functions Close to Entities
+
+Define functions right after the related entity registration:
+
+```sql
+SELECT dzql.register_entity('documents', ...);
+
+-- Related custom functions
+CREATE FUNCTION approve_document(...) ...;
+CREATE FUNCTION reject_document(...) ...;
+```
+
+### 2. Always Register Functions
+
+Don't forget to register for RPC access:
+
+```sql
+CREATE FUNCTION my_function(...) ...;
+
+-- Required for client access!
+INSERT INTO dzql.registry (fn_regproc) VALUES ('my_function'::regproc);
+```
+
+### 3. Use SECURITY DEFINER Carefully
+
+Only use `SECURITY DEFINER` when the function needs elevated privileges:
+
+```sql
+-- Needs elevated privileges (good use of SECURITY DEFINER)
+CREATE FUNCTION admin_delete_user(...)
+RETURNS void AS $$ ... $$
+LANGUAGE plpgsql SECURITY DEFINER;
+
+-- User operates on own data (use SECURITY INVOKER or default)
+CREATE FUNCTION update_profile(...)
+RETURNS jsonb AS $$ ... $$
+LANGUAGE plpgsql;  -- SECURITY INVOKER is default
+```
+
+### 4. Include Permission Checks
+
+Custom functions should validate permissions:
+
+```sql
+CREATE OR REPLACE FUNCTION custom_operation(
+  p_user_id INT,
+  p_item_id INT
+) RETURNS JSONB AS $$
+BEGIN
+  -- Always check permission first!
+  IF NOT dzql.check_permission(p_user_id, 'update', 'items',
+    (SELECT to_jsonb(items.*) FROM items WHERE id = p_item_id)
+  ) THEN
+    RAISE EXCEPTION 'Permission denied';
+  END IF;
+
+  -- Business logic here
+  ...
+END;
+$$ LANGUAGE plpgsql SECURITY DEFINER;
+```
+
+## Limitations
+
+### Functions Not Related to Entities
+
+If you have utility functions not tied to a specific entity, you have two options:
+
+**Option 1:** Create a dedicated entity file
+```sql
+-- entities/utils.sql
+-- No entity registration, just functions
+
+CREATE FUNCTION general_utility(...) ...;
+INSERT INTO dzql.registry (fn_regproc) VALUES ('general_utility'::regproc);
+```
+
+**Option 2:** Keep in manual migration file
+```sql
+-- init_db/002_utilities.sql
+CREATE FUNCTION general_utility(...) ...;
+```
+
+### Detection Logic
+
+The compiler extracts SQL between entity registrations. Only these patterns are recognized:
+
+- `CREATE FUNCTION ...` or `CREATE OR REPLACE FUNCTION ...`
+- `INSERT INTO dzql.registry ...`
+- `SELECT dzql.register_function(...)`
+
+Other SQL (like `CREATE TYPE`, `CREATE INDEX`) is **not** automatically passed through.
+
+## Migration
+
+### For Existing Projects
+
+If you already have custom functions duplicated:
+
+1. **Keep functions in entity files** (e.g., `entities/calendar.sql`)
+2. **Remove from manual migration files** (e.g., `init_db/002_schema.sql`)
+3. **Recompile:** `dzql compile entities/*.sql`
+4. **Deploy:** Updated compiled functions
+
+Your entity files become the single source of truth!
+
+## Example: Complete Entity with Custom Functions
+
+```sql
+-- entities/resources.sql
+
+-- Create table
+CREATE TABLE resources (
+  id serial PRIMARY KEY,
+  org_id integer REFERENCES organisations(id),
+  title text NOT NULL,
+  description text,
+  owner_id integer REFERENCES users(id)
+);
+
+-- Register entity
+SELECT dzql.register_entity(
+  'resources',
+  'title',
+  ARRAY['title', 'description'],
+  '{"org": "organisations"}',
+  false,
+  '{}',
+  '{}',
+  '{"view": [], "create": [], "update": ["@owner_id"], "delete": ["@owner_id"]}',
+  '{}',
+  '{"owner_id": "@user_id"}'
+);
+
+-- Custom function (automatically passed through)
+CREATE OR REPLACE FUNCTION assign_resource_to_user(
+  p_user_id INT,
+  p_resource_id INT,
+  p_target_user_id INT
+) RETURNS JSONB
+LANGUAGE plpgsql SECURITY DEFINER AS $$
+BEGIN
+  -- Check permission
+  IF NOT dzql.check_permission(p_user_id, 'update', 'resources',
+    (SELECT to_jsonb(resources.*) FROM resources WHERE id = p_resource_id)
+  ) THEN
+    RAISE EXCEPTION 'Permission denied';
+  END IF;
+
+  -- Update owner
+  UPDATE resources
+  SET owner_id = p_target_user_id
+  WHERE id = p_resource_id;
+
+  -- Return updated resource
+  RETURN (SELECT to_jsonb(resources.*) FROM resources WHERE id = p_resource_id);
+END;
+$$;
+
+-- Register for RPC access
+INSERT INTO dzql.registry (fn_regproc)
+VALUES ('assign_resource_to_user'::regproc);
+```
+
+After compilation, everything is in `init_db/resources.sql` - no manual copying needed!
+
+## See Also
+
+- [Many-to-Many Support](./many-to-many.md) - M2M relationships eliminate many toggle functions
+- [Field Defaults](./field-defaults.md) - Auto-populate fields
+- [Graph Rules](../reference/api.md#graph-rules) - Automatic relationship management

package/docs/guides/field-defaults.md

@@ -0,0 +1,240 @@
+# Field Defaults
+
+Auto-populate fields with default values during entity creation.
+
+## Overview
+
+Field defaults allow you to automatically set field values when creating new records, eliminating the need for clients to manually send fields like `owner_id`, `created_at`, or `status` on every save operation.
+
+## Benefits
+
+- **Less Client Code** - No need to send the same fields repeatedly
+- **Prevents Errors** - Can't forget required fields
+- **Enforces Security** - Server controls defaults (e.g., current user as owner)
+- **Cleaner API** - Focus on actual data, not boilerplate
+
+## Configuration
+
+Field defaults are configured in the 10th parameter of `dzql.register_entity()`:
+
+```sql
+SELECT dzql.register_entity(
+  'resources',
+  'title',
+  ARRAY['title'],
+  '{}',  -- fk_includes
+  false, -- soft_delete
+  '{}',  -- temporal_fields
+  '{}',  -- notification_paths
+  '{}',  -- permission_paths
+  '{}',  -- graph_rules
+  '{
+    "owner_id": "@user_id",
+    "created_by": "@user_id",
+    "created_at": "@now",
+    "status": "draft"
+  }'   -- field_defaults (10th parameter)
+);
+```
+
+## Available Variables
+
+Field defaults support special variables that are resolved at runtime:
+
+| Variable | Value | Example Use Case |
+|----------|-------|------------------|
+| `@user_id` | Current user ID from `p_user_id` | Ownership, audit trails |
+| `@now` | Current timestamp | `created_at`, `updated_at` |
+| `@today` | Current date | `valid_from`, `date_created` |
+| Literal values | Any JSON value | `"draft"`, `0`, `true` |
+
+## Behavior
+
+### INSERT Operations
+
+Field defaults are **only applied during INSERT** (creating new records):
+
+```javascript
+// Client doesn't send owner_id
+await api.save_resources({
+  data: { title: "Conference Room A" }
+})
+
+// Server auto-populates:
+// - owner_id = current user ID
+// - created_at = current timestamp
+// - status = "draft"
+```
+
+### UPDATE Operations
+
+Field defaults are **NOT applied during UPDATE** (modifying existing records):
+
+```javascript
+// Updating existing record
+await api.save_resources({
+  data: {
+    id: 1,
+    title: "Updated Title"
+  }
+})
+
+// created_at is NOT changed
+// owner_id is NOT changed
+```
+
+### Explicit Values Override Defaults
+
+If the client explicitly provides a value, it takes precedence:
+
+```javascript
+await api.save_resources({
+  data: {
+    title: "Room A",
+    status: "published"  // Overrides "draft" default
+  }
+})
+```
+
+## Common Use Cases
+
+### Ownership Tracking
+
+```sql
+SELECT dzql.register_entity(
+  'documents',
+  'title',
+  ARRAY['title'],
+  '{}', false, '{}', '{}', '{}', '{}',
+  '{
+    "owner_id": "@user_id",
+    "created_by": "@user_id"
+  }'
+);
+```
+
+### Timestamps
+
+```sql
+SELECT dzql.register_entity(
+  'posts',
+  'title',
+  ARRAY['title'],
+  '{}', false, '{}', '{}', '{}', '{}',
+  '{
+    "created_at": "@now",
+    "published_at": "@now"
+  }'
+);
+```
+
+### Status/Workflow
+
+```sql
+SELECT dzql.register_entity(
+  'orders',
+  'order_number',
+  ARRAY['order_number'],
+  '{}', false, '{}', '{}', '{}', '{}',
+  '{
+    "status": "pending",
+    "priority": "normal",
+    "auto_process": "true"
+  }'
+);
+```
+
+### Multi-Tenant
+
+```sql
+SELECT dzql.register_entity(
+  'items',
+  'name',
+  ARRAY['name'],
+  '{}', false, '{}', '{}', '{}', '{}',
+  '{
+    "tenant_id": "@user_id",
+    "created_at": "@now",
+    "is_active": "true"
+  }'
+);
+```
+
+## Security Considerations
+
+Field defaults improve security by:
+
+1. **Preventing client-side tampering** - Server controls sensitive defaults
+2. **Enforcing ownership** - Can't set wrong `owner_id`
+3. **Audit trail integrity** - Timestamps set server-side
+4. **Consistent initialization** - Every record starts in known state
+
+## Example: Before vs After
+
+### Before (Manual)
+
+```javascript
+// Client must remember to send owner_id every time
+await api.save_tags({
+  data: {
+    name: "Important",
+    owner_id: user.id,        // ← Easy to forget
+    created_at: new Date(),   // ← Manual
+    status: "active"          // ← Repetitive
+  }
+})
+```
+
+### After (Automatic)
+
+```javascript
+// Client sends only actual data
+await api.save_tags({
+  data: {
+    name: "Important"
+    // owner_id, created_at, status auto-populated!
+  }
+})
+```
+
+## Backwards Compatibility
+
+Field defaults are **completely optional**:
+
+- Entities without field defaults work exactly as before
+- No migration needed for existing entities
+- Can be added incrementally
+
+## Implementation Details
+
+### Storage
+
+Field defaults are stored in the `dzql.entities` table:
+
+```sql
+SELECT field_defaults FROM dzql.entities WHERE table_name = 'resources';
+```
+
+Result:
+```json
+{
+  "owner_id": "@user_id",
+  "created_at": "@now",
+  "status": "draft"
+}
+```
+
+### Resolution
+
+Variables are resolved in `generic_save()` using the existing `dzql.resolve_graph_variable()` function:
+
+1. Check if field is missing in `p_data`
+2. Get default value from entity config
+3. If starts with `@`, resolve the variable
+4. Add to data being inserted
+
+## See Also
+
+- [Entity Registration](../reference/api.md#register_entity) - Full registration API
+- [Many-to-Many Support](./many-to-many.md) - Relationship defaults
+- [Custom Functions](./custom-functions.md) - Extending entities