dzql 0.6.13 → 0.6.15

package/docs/README.md

@@ -278,32 +278,75 @@ console.log(store.documents);
 
 ### How Realtime Works
 
-The WebSocket is a pure transport layer - it does not manage or cache data. Stores own their data.
+DZQL uses a simple, unified broadcast pattern for all stores:
 
-When data changes in the database:
-1. PostgreSQL triggers emit atomic events to the `dzql_v2.events` table
-2. The runtime calls `*_affected_keys()` to find which subscriptions are affected
-3. Events are broadcast to subscribed clients via WebSocket
-4. The **store's** `subscription_event` function receives the event
-5. The store's `applyPatch` function updates its local document in-place
-6. Vue reactivity updates the UI automatically
+1. **Database events:** PostgreSQL triggers emit events to `dzql_v2.events`
+2. **Server broadcasts:** Runtime sends `{table}:{op}` messages (e.g., `venues:update`) to clients based on:
+   - **Subscriptions:** Connections with matching `affected_keys`
+   - **Notifications:** Users in `notify_users` (from entity notification paths)
+3. **Auto-dispatch:** The WebSocket client routes broadcasts to registered store handlers
+4. **Store updates:** Each store's `table_changed` method applies updates to local data
+5. **Vue reactivity:** UI updates automatically
 
-**No refetching required** - changes are applied incrementally via patching.
+**No refetching required** - changes are applied incrementally.
 
-### Patch Events
+### The `table_changed` Pattern
 
-The store receives events with this structure:
+Every generated store implements `table_changed` and self-registers with the WebSocket client:
+
+```typescript
+// Generated store (simplified)
+export const useVenuesStore = defineStore('venues-store', () => {
+  const records = ref([]);
+
+  function table_changed(table: string, op: string, pk: Record<string, unknown>, data: unknown) {
+    if (table !== 'venues') return;
+    // Update records based on op (insert/update/delete)
+  }
+
+  // Self-register - no manual setup needed!
+  ws.registerStore(table_changed);
+
+  return { records, get, save, search, table_changed };
+});
+```
+
+**User code - just works:**
+```typescript
+const venuesStore = useVenuesStore();  // Auto-registers for broadcasts
+await venuesStore.search({ org_id: 1 });
+// records update automatically when broadcasts arrive - no setup needed!
+```
+
+### Broadcast Message Format
 
 ```typescript
 {
-  table: 'sites',           // Which table changed
-  op: 'insert' | 'update' | 'delete',
-  pk: { id: 123 },          // Primary key of affected row
-  data: { ... }             // Full row data
+  "jsonrpc": "2.0",
+  "method": "venues:update",  // {table}:{op}
+  "params": {
+    "pk": { "id": 123 },
+    "data": { "id": 123, "name": "Updated Venue", ... }
+  }
 }
 ```
 
-The generated `applyPatch` function routes events to the correct location in the document graph based on the subscribable's `includes` structure.
+### Entity Notifications
+
+Entities can define `notifications` paths to specify who receives broadcasts:
+
+```typescript
+export const entities = {
+  venues: {
+    schema: { id: 'serial PRIMARY KEY', org_id: 'int', name: 'text' },
+    notifications: {
+      members: ['@org_id->acts_for[org_id=$]{active}.user_id']
+    }
+  }
+};
+```
+
+When a venue is created/updated/deleted, all active members of that org receive the broadcast.
 
 ## Why "Compile-Only"?
 

package/docs/for_ai.md

@@ -262,9 +262,59 @@ for (const [key, docState] of Object.entries(store.documents)) {
 - Same params = same cached subscription (deduplication by JSON key)
 - The `ready` Promise is stored for repeat callers to await
 - **Stores own their data** - the WebSocket is just transport
-- Realtime patches are applied by the store's `applyPatch()` function
 - Data is reactive - changes trigger Vue reactivity automatically
-- The store routes patch events by table name to the correct location in the document graph
+
+### How Realtime Works
+
+DZQL uses a unified `table_changed` pattern for all stores:
+
+1. **Database events:** PostgreSQL triggers emit events to `dzql_v2.events`
+2. **Server broadcasts:** Runtime sends `{table}:{op}` messages (e.g., `venues:update`) to clients
+3. **Auto-dispatch:** WebSocket client routes broadcasts to registered store handlers
+4. **Store updates:** Each store's `table_changed` method applies updates to local data
+
+**Stores self-register - no manual setup needed:**
+
+```typescript
+// Generated store (simplified)
+export const useVenuesStore = defineStore('venues-store', () => {
+  const records = ref([]);
+
+  function table_changed(table: string, op: string, pk: Record<string, unknown>, data: unknown) {
+    if (table !== 'venues') return;
+    // Update records based on op (insert/update/delete)
+  }
+
+  // Self-register with WebSocket
+  ws.registerStore(table_changed);
+
+  return { records, get, save, search, table_changed };
+});
+```
+
+**User code - just works:**
+```typescript
+const venuesStore = useVenuesStore();  // Auto-registers for broadcasts
+await venuesStore.search({ org_id: 1 });
+// records update automatically when broadcasts arrive!
+```
+
+### Entity Notifications
+
+Entities can define `notifications` paths to specify who receives broadcasts:
+
+```javascript
+export const entities = {
+  venues: {
+    schema: { id: 'serial PRIMARY KEY', org_id: 'int', name: 'text' },
+    notifications: {
+      members: ['@org_id->acts_for[org_id=$]{active}.user_id']
+    }
+  }
+};
+```
+
+When a venue is created/updated/deleted, all active members of that org receive the broadcast
 
 ### Common Patterns
 

package/docs/project-setup.md

@@ -387,6 +387,8 @@ async function deleteSite(id: number) {
 </script>
 ```
 
+**How realtime works:** Stores self-register with the WebSocket client on creation. When the server broadcasts `{table}:{op}` messages (e.g., `sites:insert`), each store's `table_changed` handler automatically applies the update to local data. No manual dispatcher setup required - just use the store and it works.
+
 ## 10. CLI Database Access with invj
 
 Create `tasks.js` in the project root to enable CLI database operations:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dzql",
-  "version": "0.6.13",
+  "version": "0.6.15",
   "description": "Database-first real-time framework with TypeScript support",
   "repository": {
     "type": "git",
@@ -24,7 +24,9 @@
     "README.md"
   ],
   "scripts": {
-    "test": "bun test"
+    "test": "bun run test:setup && bun test; bun run test:teardown",
+    "test:setup": "docker compose down -v && docker compose up -d --wait",
+    "test:teardown": "docker compose down -v"
   },
   "keywords": [
     "postgresql",

package/src/cli/codegen/notification.ts

@@ -0,0 +1,219 @@
+/**
+ * Notification Path Code Generator
+ * Generates PostgreSQL functions to resolve notification paths to user IDs
+ */
+
+import type { EntityIR } from "../../shared/ir.js";
+import { compilePermission } from "../compiler/permissions.js";
+
+/**
+ * Generate notification resolution function for an entity
+ * Returns a function that resolves notification paths to an array of user IDs
+ */
+export function generateNotificationFunction(name: string, entityIR: EntityIR): string {
+  const notifications = entityIR.notifications || {};
+  const pathNames = Object.keys(notifications);
+
+  if (pathNames.length === 0) {
+    // No notifications - return empty function
+    return `
+-- Notification resolution for ${name} (no paths configured)
+CREATE OR REPLACE FUNCTION dzql_v2.${name}_notify_users(
+  p_user_id INT,
+  p_data JSONB
+) RETURNS INT[]
+LANGUAGE plpgsql
+STABLE
+SECURITY DEFINER
+SET search_path = dzql_v2, public
+AS $$
+BEGIN
+  RETURN ARRAY[]::INT[];
+END;
+$$;`;
+  }
+
+  // Generate SQL for each notification path
+  const pathQueries: string[] = [];
+
+  for (const [pathName, paths] of Object.entries(notifications)) {
+    if (!paths || !Array.isArray(paths)) continue;
+
+    for (const path of paths) {
+      // Use the same permission compiler - it generates EXISTS subqueries
+      // but we need SELECT user_id instead
+      const userQuery = compileNotificationPath(name, path);
+      if (userQuery) {
+        pathQueries.push(`  -- ${pathName}: ${path}
+  v_users := v_users || ARRAY(${userQuery});`);
+      }
+    }
+  }
+
+  const pathSQL = pathQueries.length > 0
+    ? pathQueries.join('\n\n')
+    : '  -- No valid notification paths';
+
+  return `
+-- Notification resolution for ${name}
+CREATE OR REPLACE FUNCTION dzql_v2.${name}_notify_users(
+  p_user_id INT,
+  p_data JSONB
+) RETURNS INT[]
+LANGUAGE plpgsql
+STABLE
+SECURITY DEFINER
+SET search_path = dzql_v2, public
+AS $$
+DECLARE
+  v_users INT[] := ARRAY[]::INT[];
+BEGIN
+${pathSQL}
+
+  -- Return unique user IDs (excluding the acting user to avoid self-notification)
+  RETURN ARRAY(SELECT DISTINCT unnest(v_users) WHERE unnest != p_user_id);
+END;
+$$;`;
+}
+
+/**
+ * Compile a notification path to a SELECT query returning user IDs
+ *
+ * Path formats:
+ * - @field_id                          -> Direct field reference (must be user_id)
+ * - @field->table[filter]{temporal}.user_id -> Traverse to get user_id
+ */
+function compileNotificationPath(entityName: string, path: string): string | null {
+  path = path.trim();
+
+  // Direct field reference: @author_id (field must contain user_id)
+  if (path.match(/^@\w+$/) && !path.includes('->')) {
+    const field = path.slice(1);
+    return `SELECT (p_data->>'${field}')::int WHERE (p_data->>'${field}') IS NOT NULL`;
+  }
+
+  // Traversal path: @org_id->acts_for[org_id=$]{active}.user_id
+  const traversalMatch = path.match(/^@(\w+)->(.+)\.(\w+)$/);
+  if (traversalMatch) {
+    const [, startField, middle, endField] = traversalMatch;
+    return compileTraversalPath(startField, middle, endField);
+  }
+
+  // Multi-hop: @venue_id->venues.org_id->acts_for[org_id=$]{active}.user_id
+  const multiHopMatch = path.match(/^@(\w+)->(.+)\.(\w+)->(.+)\.(\w+)$/);
+  if (multiHopMatch) {
+    const [, field1, table1, field2, rest, endField] = multiHopMatch;
+    return compileMultiHopPath(field1, table1, field2, rest, endField);
+  }
+
+  // Table-first: table[filter]{temporal}.field->...
+  if (!path.startsWith('@')) {
+    return compileTableFirstPath(path);
+  }
+
+  console.warn(`[Notification] Unsupported path format: ${path}`);
+  return null;
+}
+
+/**
+ * Compile single-hop traversal: @org_id->acts_for[org_id=$]{active}.user_id
+ */
+function compileTraversalPath(startField: string, middle: string, endField: string): string {
+  // Parse: acts_for[org_id=$]{active}
+  const tableMatch = middle.match(/^(\w+)(?:\[([^\]]+)\])?(?:\{([^}]+)\})?$/);
+  if (!tableMatch) {
+    console.warn(`[Notification] Cannot parse traversal: ${middle}`);
+    return `SELECT NULL::int WHERE FALSE`;
+  }
+
+  const [, table, filterStr, temporalField] = tableMatch;
+  const conditions: string[] = [];
+
+  // Parse filters: org_id=$
+  if (filterStr) {
+    const filters = filterStr.split(',').map(f => f.trim());
+    for (const filter of filters) {
+      const [field, value] = filter.split('=').map(s => s.trim());
+      if (value === '$') {
+        // $ means "use the start field value"
+        conditions.push(`${table}.${field} = (p_data->>'${startField}')::int`);
+      } else if (value.startsWith('@')) {
+        conditions.push(`${table}.${field} = (p_data->>'${value.slice(1)}')::int`);
+      } else {
+        conditions.push(`${table}.${field} = ${value}`);
+      }
+    }
+  } else {
+    // Default: join on start field
+    conditions.push(`${table}.id = (p_data->>'${startField}')::int`);
+  }
+
+  // Temporal filter: {active} means active = true AND valid_to IS NULL
+  if (temporalField) {
+    conditions.push(`${table}.${temporalField} = true`);
+    conditions.push(`${table}.valid_to IS NULL`);
+  }
+
+  const whereClause = conditions.join(' AND ');
+
+  return `SELECT ${table}.${endField} FROM ${table} WHERE ${whereClause}`;
+}
+
+/**
+ * Compile multi-hop path: @venue_id->venues.org_id->acts_for[org_id=$]{active}.user_id
+ */
+function compileMultiHopPath(
+  field1: string,
+  table1: string,
+  field2: string,
+  rest: string,
+  endField: string
+): string {
+  // First hop: get field2 from table1
+  const subquery1 = `(SELECT ${field2} FROM ${table1} WHERE id = (p_data->>'${field1}')::int)`;
+
+  // Parse second hop: acts_for[org_id=$]{active}
+  const tableMatch = rest.match(/^(\w+)(?:\[([^\]]+)\])?(?:\{([^}]+)\})?$/);
+  if (!tableMatch) {
+    console.warn(`[Notification] Cannot parse second hop: ${rest}`);
+    return `SELECT NULL::int WHERE FALSE`;
+  }
+
+  const [, table2, filterStr, temporalField] = tableMatch;
+  const conditions: string[] = [];
+
+  // Parse filters
+  if (filterStr) {
+    const filters = filterStr.split(',').map(f => f.trim());
+    for (const filter of filters) {
+      const [field, value] = filter.split('=').map(s => s.trim());
+      if (value === '$') {
+        // $ in second hop means "use the result of first hop"
+        conditions.push(`${table2}.${field} = ${subquery1}`);
+      } else if (value.startsWith('@')) {
+        conditions.push(`${table2}.${field} = (p_data->>'${value.slice(1)}')::int`);
+      } else {
+        conditions.push(`${table2}.${field} = ${value}`);
+      }
+    }
+  }
+
+  // Temporal filter
+  if (temporalField) {
+    conditions.push(`${table2}.${temporalField} = true`);
+    conditions.push(`${table2}.valid_to IS NULL`);
+  }
+
+  const whereClause = conditions.join(' AND ');
+
+  return `SELECT ${table2}.${endField} FROM ${table2} WHERE ${whereClause}`;
+}
+
+/**
+ * Compile table-first path: contractor_rights[package_id=@package_id]{active}.contractor_org_id->...
+ */
+function compileTableFirstPath(path: string): string {
+  // This is complex - for now return null and log warning
+  console.warn(`[Notification] Table-first paths not yet supported: ${path}`);
+  return `SELECT NULL::int WHERE FALSE`;
+}

package/src/cli/codegen/pinia.ts

@@ -60,14 +60,6 @@ export interface ${pascalName} {
 ${columnTypes}
 }
 
-/** Event payload for table changes */
-export interface TableChangedPayload {
-  table: string;
-  operation: 'insert' | 'update' | 'delete';
-  pk: { ${pkField}: ${pkType} };
-  data: ${pascalName} | null;
-}
-
 export const use${pascalName}Store = defineStore('${entityName}-store', () => {
   const records: Ref<${pascalName}[]> = ref([]);
   const loading: Ref<boolean> = ref(false);
@@ -132,33 +124,37 @@ export const use${pascalName}Store = defineStore('${entityName}-store', () => {
     }
   }
 
-  function table_changed(payload: TableChangedPayload): void {
-    if (payload.table === '${entityName}') {
-      const pkValue = payload.pk?.${pkField};
-      const existingIndex = records.value.findIndex((r: ${pascalName}) => r.${pkField} === pkValue);
-
-      switch (payload.operation) {
-        case 'insert':
-          if (payload.data && existingIndex === -1) {
-            records.value.push(payload.data);
-          }
-          break;
-        case 'update':
-          if (payload.data && existingIndex !== -1) {
-            Object.assign(records.value[existingIndex], payload.data);
-          } else if (payload.data) {
-            records.value.push(payload.data);
-          }
-          break;
-        case 'delete':
-          if (existingIndex !== -1) {
-            records.value.splice(existingIndex, 1);
-          }
-          break;
-      }
+  // Broadcast handler - automatically registered with ws
+  function table_changed(table: string, op: string, pk: Record<string, unknown>, data: ${pascalName} | null): void {
+    if (table !== '${entityName}') return;
+
+    const pkValue = pk?.${pkField} as ${pkType};
+    const existingIndex = records.value.findIndex((r) => r.${pkField} === pkValue);
+
+    switch (op) {
+      case 'insert':
+        if (data && existingIndex === -1) {
+          records.value.push(data);
+        }
+        break;
+      case 'update':
+        if (data && existingIndex !== -1) {
+          Object.assign(records.value[existingIndex], data);
+        } else if (data) {
+          records.value.push(data);
+        }
+        break;
+      case 'delete':
+        if (existingIndex !== -1) {
+          records.value.splice(existingIndex, 1);
+        }
+        break;
     }
   }
 
+  // Self-register with WebSocket for broadcasts
+  ws.registerStore(table_changed);
+
   return {
     records,
     loading,

package/src/cli/codegen/sql.ts

@@ -50,12 +50,28 @@ CREATE TABLE IF NOT EXISTS dzql_v2.events (
   data jsonb,
   old_data jsonb,
   user_id int,
+  affected_keys text[] DEFAULT ARRAY[]::text[],
+  notify_users int[] DEFAULT ARRAY[]::int[],
   created_at timestamptz DEFAULT now()
 );
 
 -- Commit Sequence
 CREATE SEQUENCE IF NOT EXISTS dzql_v2.commit_seq;
 
+-- Default compute_affected_keys (returns empty array, overwritten when subscribables exist)
+CREATE OR REPLACE FUNCTION dzql_v2.compute_affected_keys(
+  p_table TEXT,
+  p_op TEXT,
+  p_data JSONB
+) RETURNS TEXT[]
+LANGUAGE plpgsql
+IMMUTABLE
+AS $$
+BEGIN
+  RETURN ARRAY[]::text[];
+END;
+$$;
+
 -- === AUTH FUNCTIONS ===
 
 -- Register User
@@ -309,6 +325,7 @@ DECLARE
   v_old_data jsonb;
   v_commit_id bigint;
   v_op text;
+  v_notify_users int[];
 ${m2mVarDeclarations}
 BEGIN
   v_commit_id := nextval('dzql_v2.commit_seq');
@@ -349,8 +366,11 @@ ${m2mExtraction}
 ${m2mSync}
 ${m2mExpansion}
 
-  -- Emit Event
-  INSERT INTO dzql_v2.events (commit_id, table_name, op, pk, data, old_data, user_id)
+  -- Resolve notification recipients
+  v_notify_users := dzql_v2.${name}_notify_users(p_user_id, v_result);
+
+  -- Emit Event with pre-computed affected keys and notify users
+  INSERT INTO dzql_v2.events (commit_id, table_name, op, pk, data, old_data, user_id, affected_keys, notify_users)
   VALUES (
     v_commit_id,
     '${name}',
@@ -358,7 +378,9 @@ ${m2mExpansion}
     ${pkJsonbExpr},
     v_result,
     v_old_data, -- NULL for insert
-    p_user_id
+    p_user_id,
+    dzql_v2.compute_affected_keys('${name}', v_op, v_result),
+    v_notify_users
   );
 
   -- Notify Runtime
@@ -419,6 +441,7 @@ AS $$
 DECLARE
   v_old_data jsonb;
   v_commit_id bigint;
+  v_notify_users int[];
 BEGIN
   v_commit_id := nextval('dzql_v2.commit_seq');
 
@@ -440,8 +463,11 @@ BEGIN
   -- Perform ${softDelete ? 'Soft ' : ''}Delete
   ${deleteOperation};
 
-  -- Emit Event (always 'delete' operation for client-side removal)
-  INSERT INTO dzql_v2.events (commit_id, table_name, op, pk, data, old_data, user_id)
+  -- Resolve notification recipients
+  v_notify_users := dzql_v2.${name}_notify_users(p_user_id, v_old_data);
+
+  -- Emit Event with pre-computed affected keys and notify users (always 'delete' operation for client-side removal)
+  INSERT INTO dzql_v2.events (commit_id, table_name, op, pk, data, old_data, user_id, affected_keys, notify_users)
   VALUES (
     v_commit_id,
     '${name}',
@@ -449,7 +475,9 @@ BEGIN
     ${pkJsonbExpr},
     v_old_data,  -- Include full data for subscription resolution
     v_old_data,
-    p_user_id
+    p_user_id,
+    dzql_v2.compute_affected_keys('${name}', 'delete', v_old_data),
+    v_notify_users
   );
 
   -- Notify Runtime
@@ -689,7 +717,11 @@ $$;
 
 // === AGGREGATE GENERATOR ===
 export function generateEntitySQL(name: string, entityIR: EntityIR): string {
+  // Import here to avoid circular dependency
+  const { generateNotificationFunction } = require('./notification.js');
+
   return [
+    generateNotificationFunction(name, entityIR),
     generateSaveFunction(name, entityIR),
     generateDeleteFunction(name, entityIR),
     generateGetFunction(name, entityIR),