dzql 0.5.23 → 0.5.25

package/docs/README.md

@@ -16,6 +16,7 @@ Feature-specific guides and how-tos:
 
 - **[Live Query Subscriptions](guides/subscriptions.md)** - Real-time denormalized documents
 - **[Many-to-Many Relationships](guides/many-to-many.md)** - Junction table management
+- **[Composite Primary Keys](guides/composite-primary-keys.md)** - Tables with compound keys
 - **[Field Defaults](guides/field-defaults.md)** - Auto-populate fields on create
 - **[Custom Functions](guides/custom-functions.md)** - Extend with PostgreSQL or Bun functions
 - **[Client Stores](guides/client-stores.md)** - Pinia store patterns for Vue.js

package/docs/for-ai/claude-guide.md

@@ -23,10 +23,16 @@ Entity Registration:
     temporal_fields,      -- '{}'
     notification_paths,   -- '{"ownership": ["@org_id->acts_for..."]}'
     permission_paths,     -- '{"view": [], "create": [...]}'
-    graph_rules,          -- '{"on_create": {...}, "many_to_many": {...}}'
+    graph_rules,          -- '{"on_create": {...}, "many_to_many": {...}, "primary_key": [...]}'
     field_defaults        -- '{"owner_id": "@user_id"}'
   )
 
+Composite Primary Keys:
+  graph_rules: '{"primary_key": ["entity_type", "entity_id"]}'
+  - GET/DELETE accept JSONB: get_table(user_id, '{"col1": "val", "col2": 123}')
+  - SAVE detects insert/update by checking if all PK fields exist
+  - Columns ending with _id are cast to ::int, others stay text
+
 M2M id_field naming:  tag_ids (singular + _ids), NOT tags_ids
 Permission [] = public, omitted = denied
 Path syntax: @field->table[filter]{temporal}.target_field

package/docs/guides/composite-primary-keys.md

@@ -0,0 +1,158 @@
+# Composite Primary Keys
+
+DZQL supports tables with composite (compound) primary keys. This guide explains how to register entities with composite keys and how the generated CRUD functions work.
+
+## When to Use Composite Keys
+
+Composite primary keys are useful for:
+
+- **Junction tables** with additional data (beyond simple M2M)
+- **Position/state tables** keyed by entity type and ID
+- **Multi-tenant tables** keyed by tenant + entity
+- **Versioned records** keyed by ID + version
+
+## Registering an Entity with a Composite Key
+
+To register an entity with a composite primary key, add `primary_key` to the `graph_rules` parameter:
+
+```sql
+-- Create a table with composite primary key
+CREATE TABLE canvas_positions (
+  entity_type VARCHAR(50) NOT NULL,
+  entity_id INTEGER NOT NULL,
+  x INTEGER NOT NULL,
+  y INTEGER NOT NULL,
+  width INTEGER DEFAULT 100,
+  height INTEGER DEFAULT 100,
+  PRIMARY KEY (entity_type, entity_id)
+);
+
+-- Register with DZQL using composite key
+SELECT dzql.register_entity(
+  'canvas_positions',                          -- table_name
+  'entity_type',                               -- label_field
+  array['x', 'y', 'width', 'height'],          -- searchable_fields
+  '{}',                                        -- fk_includes
+  false,                                       -- soft_delete
+  '{}',                                        -- temporal_fields
+  '{}',                                        -- notification_paths
+  jsonb_build_object(                          -- permission_paths
+    'view', array[]::text[],
+    'create', array[]::text[],
+    'update', array[]::text[],
+    'delete', array[]::text[]
+  ),
+  jsonb_build_object(                          -- graph_rules
+    'primary_key', array['entity_type', 'entity_id']
+  )
+);
+```
+
+The `primary_key` array specifies the columns that form the composite key, in order.
+
+## Generated Function Signatures
+
+When you compile an entity with a composite primary key, the generated functions have different signatures:
+
+### GET Function
+
+```sql
+-- Accepts JSONB with all PK fields
+SELECT get_canvas_positions(
+  1,                                           -- user_id
+  '{"entity_type": "node", "entity_id": 42}'   -- composite PK as JSONB
+);
+```
+
+### SAVE Function
+
+```sql
+-- Insert: provide all PK fields plus data
+SELECT save_canvas_positions(
+  1,                                           -- user_id
+  '{"entity_type": "node", "entity_id": 42, "x": 100, "y": 200}'
+);
+
+-- Update: same signature, existing record detected by PK
+SELECT save_canvas_positions(
+  1,
+  '{"entity_type": "node", "entity_id": 42, "x": 150, "y": 250}'
+);
+```
+
+The save function determines insert vs update by checking if a record with the composite key exists.
+
+### DELETE Function
+
+```sql
+-- Accepts JSONB with all PK fields
+SELECT delete_canvas_positions(
+  1,                                           -- user_id
+  '{"entity_type": "node", "entity_id": 42}'   -- composite PK as JSONB
+);
+```
+
+### SEARCH Function
+
+Search works the same as simple PK entities - it returns paginated results with all fields.
+
+## Type Casting
+
+DZQL automatically determines type casting for PK columns:
+
+- Columns named `id` or ending with `_id` are cast to `::int`
+- Other columns (like `entity_type`) are left as text
+
+This means for a key like `(entity_type, entity_id)`:
+- `entity_type` is compared as text
+- `entity_id` is cast to integer
+
+## Events and Notifications
+
+Events for composite PK entities include the full composite key in the `pk` field:
+
+```json
+{
+  "table_name": "canvas_positions",
+  "op": "insert",
+  "pk": {"entity_type": "node", "entity_id": 42},
+  "data": {"entity_type": "node", "entity_id": 42, "x": 100, "y": 200}
+}
+```
+
+## Limitations
+
+- **M2M relationships**: Tables with composite PKs can have M2M relationships, but this is an advanced use case. The M2M sync uses the first PK column for junction table lookups.
+- **Auto-increment**: Composite keys don't support auto-increment. All PK values must be provided on insert.
+
+## Example: Template Dependencies
+
+A practical example - tracking dependencies between templates:
+
+```sql
+CREATE TABLE template_dependencies (
+  template_id INTEGER NOT NULL REFERENCES templates(id),
+  depends_on_template_id INTEGER NOT NULL REFERENCES templates(id),
+  dependency_type VARCHAR(20) DEFAULT 'requires',
+  PRIMARY KEY (template_id, depends_on_template_id)
+);
+
+SELECT dzql.register_entity(
+  'template_dependencies',
+  'dependency_type',
+  array['dependency_type'],
+  '{"template": "templates", "depends_on": "templates"}',
+  false,
+  '{}',
+  '{}',
+  jsonb_build_object(
+    'view', array[]::text[],
+    'create', array[]::text[],
+    'update', array[]::text[],
+    'delete', array[]::text[]
+  ),
+  jsonb_build_object(
+    'primary_key', array['template_id', 'depends_on_template_id']
+  )
+);
+```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dzql",
-  "version": "0.5.23",
+  "version": "0.5.25",
   "description": "PostgreSQL-powered framework with zero boilerplate CRUD operations and real-time WebSocket synchronization",
   "type": "module",
   "main": "src/server/index.js",

package/src/compiler/codegen/operation-codegen.js

@@ -11,6 +11,18 @@ export class OperationCodegen {
     this.isCompositePK = this.primaryKey.length > 1 || this.primaryKey[0] !== 'id';
   }
 
+  /**
+   * Determine appropriate type cast for a column name
+   * Only casts to int if column is 'id' or ends with '_id'
+   * @private
+   */
+  _getTypeCast(columnName) {
+    if (columnName === 'id' || columnName.endsWith('_id')) {
+      return '::int';
+    }
+    return '';
+  }
+
   /**
    * Generate all operation functions
    * @returns {string} SQL for all operations
@@ -36,7 +48,7 @@ export class OperationCodegen {
     // For composite PKs, accept JSONB containing all PK fields
     // For simple PKs, accept INT for backwards compatibility
     if (this.isCompositePK) {
-      const whereClause = this.primaryKey.map(col => `${col} = (p_pk->>'${col}')::int`).join(' AND ');
+      const whereClause = this.primaryKey.map(col => `${col} = (p_pk->>'${col}')${this._getTypeCast(col)}`).join(' AND ');
       const pkDescription = this.primaryKey.join(', ');
 
       return `-- GET operation for ${this.tableName} (composite primary key: ${pkDescription})
@@ -125,6 +137,21 @@ $$ LANGUAGE plpgsql SECURITY DEFINER;`;
     const m2mExpansion = this._generateM2MExpansion();
     const fieldDefaults = this._generateFieldDefaults();
 
+    // For composite PKs, generate a different function signature and logic
+    if (this.isCompositePK) {
+      return this._generateCompositePKSaveFunction({
+        graphRulesCall,
+        notificationSQL,
+        filterSensitiveFields,
+        m2mVariables,
+        m2mExtraction,
+        m2mSync,
+        m2mExpansion,
+        fieldDefaults
+      });
+    }
+
+    // Simple PK (id column) - original implementation for backwards compatibility
     return `-- SAVE operation for ${this.tableName}
 CREATE OR REPLACE FUNCTION save_${this.tableName}(
   p_user_id INT,
@@ -207,6 +234,141 @@ ${m2mExpansion}
 ${graphRulesCall}
 ${notificationSQL}
 
+${filterSensitiveFields}
+
+  RETURN v_output;
+END;
+$$ LANGUAGE plpgsql SECURITY DEFINER;`;
+  }
+
+  /**
+   * Generate SAVE function for composite primary keys
+   * @private
+   */
+  _generateCompositePKSaveFunction(helpers) {
+    const {
+      graphRulesCall,
+      notificationSQL,
+      filterSensitiveFields,
+      m2mVariables,
+      m2mExtraction,
+      m2mSync,
+      m2mExpansion,
+      fieldDefaults
+    } = helpers;
+
+    const pkDescription = this.primaryKey.join(', ');
+
+    // Generate WHERE clause for composite PK lookup
+    // e.g., "entity_type = (p_data->>'entity_type') AND entity_id = (p_data->>'entity_id')::int"
+    const whereClause = this.primaryKey.map(col => {
+      return `${col} = (p_data->>'${col}')${this._getTypeCast(col)}`;
+    }).join(' AND ');
+
+    // For use inside format() string literal with %L placeholders
+    // e.g., "entity_type = %L AND entity_id = %L"
+    const whereClauseForFormat = this.primaryKey.map(col => `${col} = %L`).join(' AND ');
+
+    // Format arguments for the WHERE clause (to be passed to format())
+    // e.g., "(p_data->>'entity_type'), (p_data->>'entity_id')::int"
+    const whereClauseFormatArgs = this.primaryKey.map(col => {
+      return `(p_data->>'${col}')${this._getTypeCast(col)}`;
+    }).join(', ');
+
+    // Check if all PK fields are present in p_data
+    const pkNullCheck = this.primaryKey.map(col => `p_data->>'${col}' IS NULL`).join(' OR ');
+
+    // Build the list of PK field names for exclusion from SET clause
+    const pkFieldsExclusion = this.primaryKey.map(col => `jsonb_object_keys != '${col}'`).join(' AND ');
+
+    // For M2M sync and expansion, we need to reference the PK fields from v_result
+    const m2mSyncCompositePK = this._generateM2MSyncCompositePK();
+    const m2mExpansionCompositePK = this._generateM2MExpansionCompositePK();
+    const m2mExpansionForBeforeCompositePK = this._generateM2MExpansionForBeforeCompositePK();
+
+    return `-- SAVE operation for ${this.tableName} (composite primary key: ${pkDescription})
+CREATE OR REPLACE FUNCTION save_${this.tableName}(
+  p_user_id INT,
+  p_data JSONB
+) RETURNS JSONB AS $$
+DECLARE
+  v_result ${this.tableName}%ROWTYPE;
+  v_existing ${this.tableName}%ROWTYPE;
+  v_output JSONB;
+  v_before JSONB;
+  v_is_insert BOOLEAN := false;
+  v_notify_users INT[];
+${m2mVariables}
+BEGIN
+${m2mExtraction}
+  -- Determine if this is insert or update (composite PK: ${pkDescription})
+  IF ${pkNullCheck} THEN
+    -- Missing one or more PK fields - this is an insert
+    v_is_insert := true;
+  ELSE
+    -- Try to fetch existing record by composite PK
+    SELECT * INTO v_existing
+    FROM ${this.tableName}
+    WHERE ${whereClause};
+
+    v_is_insert := NOT FOUND;
+  END IF;
+
+  -- Check permissions
+  IF v_is_insert THEN
+    IF NOT can_create_${this.tableName}(p_user_id, p_data) THEN
+      RAISE EXCEPTION 'Permission denied: create on ${this.tableName}';
+    END IF;
+  ELSE
+    IF NOT can_update_${this.tableName}(p_user_id, to_jsonb(v_existing)) THEN
+      RAISE EXCEPTION 'Permission denied: update on ${this.tableName}';
+    END IF;
+  END IF;
+
+  -- Expand M2M for existing record (for UPDATE events "before" field)
+  IF NOT v_is_insert THEN
+    v_before := to_jsonb(v_existing);
+${m2mExpansionForBeforeCompositePK}
+  END IF;
+${fieldDefaults}
+  -- Perform UPSERT
+  IF v_is_insert THEN
+    -- Dynamic INSERT from JSONB
+    EXECUTE (
+      SELECT format(
+        'INSERT INTO ${this.tableName} (%s) VALUES (%s) RETURNING *',
+        string_agg(quote_ident(key), ', '),
+        string_agg(quote_nullable(value), ', ')
+      )
+      FROM jsonb_each_text(p_data) kv(key, value)
+    ) INTO v_result;
+  ELSE
+    -- Dynamic UPDATE from JSONB (only if there are non-PK fields to update)
+    IF (SELECT COUNT(*) FROM jsonb_object_keys(p_data) WHERE ${pkFieldsExclusion}) > 0 THEN
+      EXECUTE (
+        SELECT format(
+          'UPDATE ${this.tableName} SET %s WHERE ${whereClauseForFormat} RETURNING *',
+          string_agg(quote_ident(key) || ' = ' || quote_nullable(value), ', '),
+          ${whereClauseFormatArgs}
+        )
+        FROM jsonb_each_text(p_data) kv(key, value)
+        WHERE ${this.primaryKey.map(col => `key != '${col}'`).join(' AND ')}
+      ) INTO v_result;
+    ELSE
+      -- No fields to update (only M2M fields were provided), just fetch existing
+      v_result := v_existing;
+    END IF;
+  END IF;
+
+${m2mSyncCompositePK}
+
+  -- Prepare output with M2M fields (BEFORE event creation for real-time notifications!)
+  v_output := to_jsonb(v_result);
+${m2mExpansionCompositePK}
+
+${graphRulesCall}
+${notificationSQL}
+
 ${filterSensitiveFields}
 
   RETURN v_output;
@@ -224,7 +386,7 @@ $$ LANGUAGE plpgsql SECURITY DEFINER;`;
 
     // For composite PKs, accept JSONB containing all PK fields
     if (this.isCompositePK) {
-      const whereClause = this.primaryKey.map(col => `${col} = (p_pk->>'${col}')::int`).join(' AND ');
+      const whereClause = this.primaryKey.map(col => `${col} = (p_pk->>'${col}')${this._getTypeCast(col)}`).join(' AND ');
       const pkDescription = this.primaryKey.join(', ');
 
       const deleteSQL = this.entity.softDelete
@@ -740,6 +902,129 @@ $$ LANGUAGE plpgsql SECURITY DEFINER;`;
     return expansions.join('');
   }
 
+  /**
+   * Generate M2M sync for composite primary keys
+   * For tables with composite PKs, M2M relationships are rare but possible
+   * @private
+   */
+  _generateM2MSyncCompositePK() {
+    const manyToMany = this.entity.manyToMany || {};
+    if (Object.keys(manyToMany).length === 0) return '';
+
+    // For composite PKs, we need to build a composite key reference
+    // This is a rare case - most tables with composite PKs are junction tables themselves
+    const syncs = [];
+
+    for (const [relationKey, config] of Object.entries(manyToMany)) {
+      const idField = config.id_field;
+      const junctionTable = config.junction_table;
+      const localKey = config.local_key;
+      const foreignKey = config.foreign_key;
+
+      // For composite PK tables with M2M, we'd need a different junction table structure
+      // This is an edge case - log a warning comment in the generated SQL
+      syncs.push(`
+  -- ============================================================================
+  -- M2M Sync: ${relationKey} (junction: ${junctionTable}) - COMPOSITE PK
+  -- Note: M2M on composite PK tables requires junction table to reference all PK columns
+  -- ============================================================================
+  IF v_${idField} IS NOT NULL THEN
+    -- Delete relationships not in new list (using first PK column as reference)
+    DELETE FROM ${junctionTable}
+    WHERE ${localKey} = v_result.${this.primaryKey[0]}
+      AND (${foreignKey} <> ALL(v_${idField}) OR v_${idField} = '{}');
+
+    -- Insert new relationships (idempotent)
+    IF array_length(v_${idField}, 1) > 0 THEN
+      INSERT INTO ${junctionTable} (${localKey}, ${foreignKey})
+      SELECT v_result.${this.primaryKey[0]}, unnest(v_${idField})
+      ON CONFLICT (${localKey}, ${foreignKey}) DO NOTHING;
+    END IF;
+  END IF;`);
+    }
+
+    return syncs.join('');
+  }
+
+  /**
+   * Generate M2M expansion for composite primary keys (for SAVE output)
+   * @private
+   */
+  _generateM2MExpansionCompositePK() {
+    const manyToMany = this.entity.manyToMany || {};
+    if (Object.keys(manyToMany).length === 0) return '';
+
+    const expansions = [];
+
+    for (const [relationKey, config] of Object.entries(manyToMany)) {
+      const idField = config.id_field;
+      const junctionTable = config.junction_table;
+      const localKey = config.local_key;
+      const foreignKey = config.foreign_key;
+      const targetEntity = config.target_entity;
+      const expand = config.expand || false;
+
+      // Use first PK column for M2M lookups
+      expansions.push(`
+  -- Add M2M IDs: ${idField} (composite PK table)
+  v_output := v_output || jsonb_build_object('${idField}',
+    (SELECT COALESCE(jsonb_agg(${foreignKey} ORDER BY ${foreignKey}), '[]'::jsonb)
+     FROM ${junctionTable} WHERE ${localKey} = v_result.${this.primaryKey[0]})
+  );`);
+
+      if (expand) {
+        expansions.push(`
+  -- Expand M2M objects: ${relationKey} (expand=true, composite PK table)
+  v_output := v_output || jsonb_build_object('${relationKey}',
+    (SELECT COALESCE(jsonb_agg(to_jsonb(t.*) ORDER BY t.id), '[]'::jsonb)
+     FROM ${junctionTable} jt
+     JOIN ${targetEntity} t ON t.id = jt.${foreignKey}
+     WHERE jt.${localKey} = v_result.${this.primaryKey[0]})
+  );`);
+      }
+    }
+
+    return expansions.join('');
+  }
+
+  /**
+   * Generate M2M expansion for "before" record with composite primary keys
+   * @private
+   */
+  _generateM2MExpansionForBeforeCompositePK() {
+    const manyToMany = this.entity.manyToMany || {};
+    if (Object.keys(manyToMany).length === 0) return '';
+
+    const expansions = [];
+
+    for (const [relationKey, config] of Object.entries(manyToMany)) {
+      const idField = config.id_field;
+      const junctionTable = config.junction_table;
+      const localKey = config.local_key;
+      const foreignKey = config.foreign_key;
+      const targetEntity = config.target_entity;
+      const expand = config.expand || false;
+
+      expansions.push(`
+    v_before := v_before || jsonb_build_object('${idField}',
+      (SELECT COALESCE(jsonb_agg(${foreignKey} ORDER BY ${foreignKey}), '[]'::jsonb)
+       FROM ${junctionTable} WHERE ${localKey} = v_existing.${this.primaryKey[0]})
+    );`);
+
+      if (expand) {
+        expansions.push(`
+    v_before := v_before || jsonb_build_object('${relationKey}',
+      (SELECT COALESCE(jsonb_agg(to_jsonb(t.*) ORDER BY t.id), '[]'::jsonb)
+       FROM ${junctionTable} jt
+       JOIN ${targetEntity} t ON t.id = jt.${foreignKey}
+       WHERE jt.${localKey} = v_existing.${this.primaryKey[0]})
+    );`);
+      }
+    }
+
+    return expansions.join('');
+  }
+
   /**
    * Generate FK expansions for GET
    * @private

package/src/server/namespace.js

@@ -10,6 +10,7 @@ const DEFAULT_USER_ID = 1;
 
 /**
  * Discover available entities from dzql.entities table or compiled functions
+ * @returns {Promise<Object>} Map of entity name to {label, searchable, description}
  */
 async function discoverEntities() {
   // First try dzql.entities table (runtime mode)
@@ -58,6 +59,9 @@ async function discoverEntities() {
 /**
  * DZQL operations namespace - provides CLI-style access to DZQL operations
  *
+ * Each method outputs JSON to console and calls sql.end() before returning,
+ * making instances single-use. On error, methods call process.exit(1).
+ *
  * Usage in tasks.js:
  * ```js
  * import { DzqlNamespace } from 'dzql/namespace';
@@ -70,11 +74,17 @@ async function discoverEntities() {
  * ```
  */
 export class DzqlNamespace {
+  /**
+   * @param {number} [userId=1] - User ID for permission checks
+   */
   constructor(userId = DEFAULT_USER_ID) {
     this.userId = userId;
   }
 
-  /** List all available entities */
+  /**
+   * List all available entities
+   * @returns {Promise<void>} Outputs JSON to console
+   */
   async entities(c) {
     try {
       const entities = await discoverEntities();
@@ -89,7 +99,12 @@ export class DzqlNamespace {
     }
   }
 
-  /** Search an entity */
+  /**
+   * Search an entity
+   * @example invj dzql:search venues '{"query": "test"}'
+   * @param {string} entity - Entity/table name to search
+   * @param {string} [argsJson] - JSON string with search args (query, limit, offset, filters)
+   */
   async search(c, entity, argsJson = "{}") {
     if (!entity) {
       console.error("Error: entity name required");
@@ -123,7 +138,12 @@ export class DzqlNamespace {
     }
   }
 
-  /** Get entity by ID */
+  /**
+   * Get entity by ID
+   * @example invj dzql:get venues '{"id": 1}'
+   * @param {string} entity - Entity/table name
+   * @param {string} [argsJson] - JSON string with {id: number}
+   */
   async get(c, entity, argsJson = "{}") {
     if (!entity) {
       console.error("Error: entity name required");
@@ -155,7 +175,12 @@ export class DzqlNamespace {
     }
   }
 
-  /** Save (create or update) entity */
+  /**
+   * Save (create or update) entity
+   * @example invj dzql:save venues '{"name": "New Venue", "org_id": 1}'
+   * @param {string} entity - Entity/table name
+   * @param {string} [argsJson] - JSON string with entity data (include id to update, omit to create)
+   */
   async save(c, entity, argsJson = "{}") {
     if (!entity) {
       console.error("Error: entity name required");
@@ -189,7 +214,12 @@ export class DzqlNamespace {
     }
   }
 
-  /** Delete entity by ID */
+  /**
+   * Delete entity by ID
+   * @example invj dzql:delete venues '{"id": 1}'
+   * @param {string} entity - Entity/table name
+   * @param {string} [argsJson] - JSON string with {id: number}
+   */
   async delete(c, entity, argsJson = "{}") {
     if (!entity) {
       console.error("Error: entity name required");
@@ -221,7 +251,12 @@ export class DzqlNamespace {
     }
   }
 
-  /** Lookup entity (for dropdowns/autocomplete) */
+  /**
+   * Lookup entity (for dropdowns/autocomplete)
+   * @example invj dzql:lookup organisations '{"query": "acme"}'
+   * @param {string} entity - Entity/table name
+   * @param {string} [argsJson] - JSON string with {query: string, limit?: number}
+   */
   async lookup(c, entity, argsJson = "{}") {
     if (!entity) {
       console.error("Error: entity name required");