dzql 0.2.1 → 0.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -4,11 +4,12 @@ PostgreSQL-powered framework with automatic CRUD operations, live query subscrip
4
4
 
5
5
  ## Documentation
6
6
 
7
- - **[Getting Started Guide](docs/GETTING_STARTED.md)** - Complete tutorial with working todo app
8
- - **[API Reference](docs/REFERENCE.md)** - Complete API documentation
9
- - **[Live Query Subscriptions](../../docs/SUBSCRIPTIONS_QUICK_START.md)** - Real-time denormalized documents (NEW in v0.2.0)
7
+ - **[Documentation Hub](docs/)** - Complete documentation index
8
+ - **[Getting Started Tutorial](docs/getting-started/tutorial.md)** - Complete tutorial with working todo app
9
+ - **[API Reference](docs/reference/api.md)** - Complete API documentation
10
+ - **[Live Query Subscriptions](docs/getting-started/subscriptions-quick-start.md)** - Real-time denormalized documents (NEW in v0.2.0)
10
11
  - **[Compiler Documentation](docs/compiler/)** - Entity compilation guide and coding standards
11
- - **[Claude Guide](docs/CLAUDE.md)** - Development guide for AI assistants
12
+ - **[Claude Guide](docs/for-ai/claude-guide.md)** - Development guide for AI assistants
12
13
  - **[Venues Example](../venues/)** - Full working application
13
14
 
14
15
  ## Quick Install
package/docs/README.md ADDED
@@ -0,0 +1,71 @@
1
+ # DZQL Documentation
2
+
3
+ Complete documentation for the DZQL PostgreSQL-powered framework.
4
+
5
+ ## 📚 Getting Started
6
+
7
+ New to DZQL? Start here:
8
+
9
+ - **[Tutorial](getting-started/tutorial.md)** - Complete step-by-step guide with a working todo app
10
+ - **[Subscriptions Quick Start](getting-started/subscriptions-quick-start.md)** - Get real-time subscriptions working in 5 minutes
11
+
12
+ ## 📖 Guides
13
+
14
+ Feature-specific guides and how-tos:
15
+
16
+ - **[Live Query Subscriptions](guides/subscriptions.md)** - Real-time denormalized documents
17
+ - **[Client Stores](guides/client-stores.md)** - Pinia store patterns for Vue.js
18
+
19
+ ## 📘 Reference
20
+
21
+ Complete API documentation:
22
+
23
+ - **[API Reference](reference/api.md)** - The 5 operations, entities, permissions, graph rules
24
+ - **[Client API](reference/client.md)** - WebSocket client and connection management
25
+ - **[Compiler](compiler/)** - Entity compilation, code generation, and coding standards
26
+
27
+ ### Compiler Documentation
28
+
29
+ - [Quickstart](compiler/QUICKSTART.md) - Get started with the DZQL compiler
30
+ - [Advanced Filters](compiler/ADVANCED_FILTERS.md) - Complex search operators
31
+ - [Coding Standards](compiler/CODING_STANDARDS.md) - Best practices for DZQL code
32
+ - [Comparison](compiler/COMPARISON.md) - DZQL vs other approaches
33
+
34
+ ## 🤖 For AI Assistants
35
+
36
+ - **[Claude Guide](for-ai/claude-guide.md)** - Complete guide for AI-assisted DZQL development
37
+
38
+ ## 🔗 Quick Links
39
+
40
+ - [npm Package](https://www.npmjs.com/package/dzql)
41
+ - [GitHub Repository](https://github.com/blueshed/dzql)
42
+ - [Issue Tracker](https://github.com/blueshed/dzql/issues)
43
+ - [Changelog](../../../CHANGELOG.md)
44
+ - [Contributing](../../../CONTRIBUTING.md)
45
+
46
+ ## 🏗️ Architecture
47
+
48
+ Looking for architecture and design docs? See the [repository docs](../../../docs/):
49
+
50
+ - [Permissions System](../../../docs/architecture/PERMISSIONS.md)
51
+ - [Project Roadmap](../../../docs/architecture/ROADMAP.md)
52
+ - [Subscription Architecture](../../../docs/architecture/SUBSCRIPTIONS_STRATEGY.md)
53
+
54
+ ## 🧪 Development
55
+
56
+ Contributing to DZQL? See development documentation:
57
+
58
+ - [TDD Workflow](../../../docs/development/TDD_WORKFLOW.md)
59
+ - [WebSocket Testing](../../../docs/development/WEBSOCKET_TESTING.md)
60
+ - [Claude Web Setup](../../../docs/development/CLAUDE-WEB.md)
61
+
62
+ ## 📦 Package Contents
63
+
64
+ This documentation is published with the npm package. For repository-wide documentation (contributors, development workflow, architecture), see [`/docs/`](../../../docs/) in the repository root.
65
+
66
+ ## Need Help?
67
+
68
+ - 📖 Check the guides above
69
+ - 🐛 [Report an issue](https://github.com/blueshed/dzql/issues)
70
+ - 💬 [Start a discussion](https://github.com/blueshed/dzql/discussions)
71
+ - 🤖 Ask your AI assistant (they have access to this documentation!)
@@ -0,0 +1,84 @@
1
+ # DZQL Compiler Documentation
2
+
3
+ The DZQL Compiler transforms declarative entity definitions into optimized PostgreSQL stored procedures.
4
+
5
+ ## Quick Start
6
+
7
+ - **[Quickstart Guide](QUICKSTART.md)** - Get started with the compiler in 5 minutes
8
+
9
+ ## Guides
10
+
11
+ - **[Advanced Filters](ADVANCED_FILTERS.md)** - Complex search operators and patterns
12
+ - **[Coding Standards](CODING_STANDARDS.md)** - Best practices for DZQL code
13
+
14
+ ## Reference
15
+
16
+ - **[Comparison](COMPARISON.md)** - How DZQL compares to other approaches
17
+ - **[Session Summary](SESSION_SUMMARY.md)** - Development session documentation
18
+ - **[Summary](SUMMARY.md)** - Compiler overview and architecture
19
+ - **[Overnight Build](OVERNIGHT_BUILD.md)** - Batch compilation process
20
+
21
+ ## Using the Compiler
22
+
23
+ ### Via CLI
24
+
25
+ ```bash
26
+ dzql compile database/domain.sql -o compiled/
27
+ ```
28
+
29
+ ### Programmatically
30
+
31
+ ```javascript
32
+ import { DZQLCompiler } from 'dzql/compiler';
33
+
34
+ const compiler = new DZQLCompiler();
35
+ const result = compiler.compileFromSQL(sqlContent);
36
+
37
+ console.log(result.sql); // Generated PostgreSQL
38
+ ```
39
+
40
+ ### Registering Entities
41
+
42
+ ```sql
43
+ SELECT dzql.register_entity(
44
+ 'todos', -- Table name
45
+ 'title', -- Label field
46
+ array['title', 'description'], -- Searchable fields
47
+ '{}'::jsonb, -- FK includes
48
+ false, -- Soft delete
49
+ '{}'::jsonb, -- Graph rules
50
+ jsonb_build_object( -- Notification paths
51
+ 'owner', array['@user_id']
52
+ ),
53
+ jsonb_build_object( -- Permission paths
54
+ 'view', array['@user_id'],
55
+ 'create', array['@user_id'],
56
+ 'update', array['@user_id'],
57
+ 'delete', array['@user_id']
58
+ ),
59
+ '{}'::jsonb -- Temporal config
60
+ );
61
+ ```
62
+
63
+ This generates 5 PostgreSQL functions:
64
+ - `get_todos(params, user_id)` - Retrieve single record
65
+ - `save_todos(params, user_id)` - Create or update
66
+ - `delete_todos(params, user_id)` - Delete record
67
+ - `lookup_todos(params, user_id)` - Autocomplete
68
+ - `search_todos(params, user_id)` - Search with filters
69
+
70
+ ## Architecture
71
+
72
+ The compiler uses a three-phase approach:
73
+
74
+ 1. **Parse** - Extract entity definitions from SQL
75
+ 2. **Generate** - Create optimized PostgreSQL functions
76
+ 3. **Deploy** - Execute generated SQL
77
+
78
+ All business logic runs in PostgreSQL, not application code.
79
+
80
+ ## See Also
81
+
82
+ - [Main Documentation](../) - Full DZQL documentation
83
+ - [API Reference](../reference/api.md) - The 5 operations
84
+ - [For AI](../for-ai/claude-guide.md) - AI-assisted development
@@ -1163,7 +1163,7 @@ array['name', 'address', 'city', 'description', 'notes', 'tags', 'metadata']
1163
1163
 
1164
1164
  ## Additional Resources
1165
1165
 
1166
- - **API Reference**: See [packages/dzql/REFERENCE.md](packages/dzql/REFERENCE.md) for complete API documentation
1167
- - **Tutorial**: See [packages/dzql/GETTING_STARTED.md](packages/dzql/GETTING_STARTED.md) for hands-on guide
1166
+ - **API Reference**: See [API Reference](../reference/api.md) for complete API documentation
1167
+ - **Tutorial**: See [Getting Started Tutorial](../getting-started/tutorial.md) for hands-on guide
1168
1168
  - **Examples**: See `packages/venues/` for complete working application
1169
1169
  - **Tests**: See `packages/venues/tests/` for comprehensive test patterns
@@ -61,9 +61,9 @@ All change detection happens in PostgreSQL - zero configuration needed on the se
61
61
 
62
62
  ## Next Steps
63
63
 
64
- - [Full Documentation](./LIVE_QUERY_SUBSCRIPTIONS.md)
65
- - [Permission Paths Guide](./PATH_DSL.md)
66
- - [Example Subscribables](../packages/dzql/examples/subscribables/)
64
+ - [Full Documentation](../guides/subscriptions.md)
65
+ - [Permission Paths Guide](../../../../docs/architecture/PERMISSIONS.md)
66
+ - [API Reference](../reference/api.md)
67
67
 
68
68
  ## Common Patterns
69
69
 
@@ -2,7 +2,7 @@
2
2
 
3
3
  DZQL is a PostgreSQL framework that gives you **atomic real-time updates** via WebSocket. Every database change broadcasts instantly to all connected clients. Zero boilerplate.
4
4
 
5
- > **See also:** [REFERENCE.md](REFERENCE.md) for complete API documentation | [CLAUDE.md](../../docs/CLAUDE.md) for AI development guide
5
+ > **See also:** [API Reference](../reference/api.md) for complete API documentation | [Claude Guide](../for-ai/claude-guide.md) for AI development guide
6
6
 
7
7
  ## The Core Pattern
8
8
 
@@ -529,7 +529,7 @@ ws.api.subscribe_venue_detail(
529
529
 
530
530
  ## See Also
531
531
 
532
- - [Vision Document](../vision.md) - Architecture overview and patterns
533
- - [Path DSL](./PATH_DSL.md) - Permission path syntax
534
- - [WebSocket API](./WEBSOCKET_API.md) - Full WebSocket protocol reference
535
- - [Compiler Reference](./COMPILER.md) - Code generation details
532
+ - [Vision Document](../../../../vision.md) - Architecture overview and patterns
533
+ - [Permission Paths](../../../../docs/architecture/PERMISSIONS.md) - Permission path DSL syntax
534
+ - [Subscription Architecture](../../../../docs/architecture/SUBSCRIPTIONS_STRATEGY.md) - Design decisions
535
+ - [Compiler Documentation](../compiler/) - Code generation and compilation guide
@@ -1,6 +1,6 @@
1
1
  # DZQL API Reference
2
2
 
3
- Complete API documentation for DZQL framework. For tutorials, see [GETTING_STARTED.md](GETTING_STARTED.md). For AI development guide, see [CLAUDE.md](../../docs/CLAUDE.md).
3
+ Complete API documentation for DZQL framework. For tutorials, see [Getting Started Tutorial](../getting-started/tutorial.md). For AI development guide, see [Claude Guide](../for-ai/claude-guide.md).
4
4
 
5
5
  ## Table of Contents
6
6
 
@@ -869,7 +869,7 @@ ws.onBroadcast((method, params) => {
869
869
 
870
870
  Subscribe to denormalized documents and receive automatic updates when underlying data changes. Subscriptions use a PostgreSQL-first architecture where all change detection happens in the database.
871
871
 
872
- For complete documentation, see **[Live Query Subscriptions Guide](../../../docs/LIVE_QUERY_SUBSCRIPTIONS.md)** and **[Quick Start](../../../docs/SUBSCRIPTIONS_QUICK_START.md)**.
872
+ For complete documentation, see **[Live Query Subscriptions Guide](../guides/subscriptions.md)** and **[Quick Start](../getting-started/subscriptions-quick-start.md)**.
873
873
 
874
874
  ### Quick Example
875
875
 
@@ -997,8 +997,8 @@ SELECT dzql.register_subscribable(
997
997
 
998
998
  ### See Also
999
999
 
1000
- - **[Live Query Subscriptions Guide](../../../docs/LIVE_QUERY_SUBSCRIPTIONS.md)** - Complete reference
1001
- - **[Quick Start Guide](../../../docs/SUBSCRIPTIONS_QUICK_START.md)** - 5-minute tutorial
1000
+ - **[Live Query Subscriptions Guide](../guides/subscriptions.md)** - Complete reference
1001
+ - **[Quick Start Guide](../getting-started/subscriptions-quick-start.md)** - 5-minute tutorial
1002
1002
  - **[Permission Paths](#permission--notification-paths)** - Path DSL syntax
1003
1003
 
1004
1004
  ---
@@ -1093,7 +1093,7 @@ const result = await db.api.myCustomFunction({param: 'value'}, userId);
1093
1093
 
1094
1094
  ## See Also
1095
1095
 
1096
- - [GETTING_STARTED.md](GETTING_STARTED.md) - Hands-on tutorial
1097
- - [CLAUDE.md](../../docs/CLAUDE.md) - AI development guide
1098
- - [README.md](../../README.md) - Project overview
1099
- - [Venues Example](../venues/) - Complete working application
1096
+ - [Getting Started Tutorial](../getting-started/tutorial.md) - Hands-on tutorial
1097
+ - [Claude Guide](../for-ai/claude-guide.md) - AI development guide
1098
+ - [Project README](../../../../README.md) - Project overview
1099
+ - [Venues Example](../../../venues/) - Complete working application
@@ -154,7 +154,7 @@ You now have:
154
154
 
155
155
  ## Next Steps
156
156
 
157
- - Read [CLIENT-STORES.md](./CLIENT-STORES.md) for complete API reference
157
+ - Read [Client Stores Guide](../guides/client-stores.md) for complete API reference
158
158
  - Customize the App.vue template
159
159
  - Add your own components
160
160
  - Style with Tailwind/DaisyUI
@@ -179,5 +179,5 @@ Check `packages/client` for a complete working example.
179
179
  ## Help
180
180
 
181
181
  For more help, see:
182
- - [CLIENT-STORES.md](./CLIENT-STORES.md) - Complete documentation
182
+ - [Client Stores Guide](../guides/client-stores.md) - Complete documentation
183
183
  - [GitHub Issues](https://github.com/blueshed/dzql/issues)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dzql",
3
- "version": "0.2.1",
3
+ "version": "0.2.2",
4
4
  "description": "PostgreSQL-powered framework with zero boilerplate CRUD operations and real-time WebSocket synchronization",
5
5
  "type": "module",
6
6
  "main": "src/server/index.js",
@@ -22,7 +22,7 @@
22
22
  ],
23
23
  "scripts": {
24
24
  "test": "bun test",
25
- "prepublishOnly": "echo '✅ Publishing DZQL v0.2.1...'"
25
+ "prepublishOnly": "echo '✅ Publishing DZQL v0.2.2...'"
26
26
  },
27
27
  "dependencies": {
28
28
  "jose": "^6.1.0",
@@ -30,6 +30,13 @@ begin
30
30
  'temporal_fields', e.temporal_fields,
31
31
  'notification_paths', e.notification_paths,
32
32
  'permission_paths', e.permission_paths,
33
+ 'primary_key', (
34
+ -- Get primary key columns
35
+ select jsonb_agg(a.attname order by a.attnum)
36
+ from pg_index i
37
+ join pg_attribute a on a.attrelid = i.indrelid and a.attnum = any(i.indkey)
38
+ where i.indrelid = e.table_name::regclass and i.indisprimary
39
+ ),
33
40
  'schema', (
34
41
  -- Get column schema from information_schema
35
42
  select jsonb_agg(
package/src/server/db.js CHANGED
@@ -10,11 +10,17 @@ const DB_MAX_CONNECTIONS = parseInt(process.env.DB_MAX_CONNECTIONS || "10", 10);
10
10
  const DB_IDLE_TIMEOUT = parseInt(process.env.DB_IDLE_TIMEOUT || "20", 10);
11
11
  const DB_CONNECT_TIMEOUT = parseInt(process.env.DB_CONNECT_TIMEOUT || "10", 10);
12
12
 
13
+ // SSL configuration for Heroku and other hosted databases
14
+ const sslConfig = process.env.DATABASE_SSL === 'true'
15
+ ? { rejectUnauthorized: false } // Heroku uses self-signed certs
16
+ : undefined;
17
+
13
18
  // Main PostgreSQL connection for queries
14
19
  export const sql = postgres(DATABASE_URL, {
15
20
  max: DB_MAX_CONNECTIONS,
16
21
  idle_timeout: DB_IDLE_TIMEOUT,
17
22
  connect_timeout: DB_CONNECT_TIMEOUT,
23
+ ssl: sslConfig,
18
24
  // Suppress NOTICE messages in test environment
19
25
  onnotice: process.env.NODE_ENV === 'test' ? () => {} : undefined,
20
26
  });
@@ -24,6 +30,7 @@ export const listen_sql = postgres(DATABASE_URL, {
24
30
  max: 1,
25
31
  idle_timeout: 0,
26
32
  connect_timeout: DB_CONNECT_TIMEOUT,
33
+ ssl: sslConfig,
27
34
  // Suppress NOTICE messages in test environment
28
35
  onnotice: process.env.NODE_ENV === 'test' ? () => {} : undefined,
29
36
  });
@@ -1,488 +0,0 @@
1
- # Live Query Subscriptions Strategy
2
-
3
- **Date:** 2025-11-16
4
- **Status:** Phase 1 Complete (Compiler), Phase 2-4 Pending
5
-
6
- ---
7
-
8
- ## Overview
9
-
10
- Live Query Subscriptions implement **Pattern 1** from `vision.md` - allowing clients to subscribe to denormalized documents and receive automatic updates when any related data changes.
11
-
12
- ### Architecture Principles
13
-
14
- 1. **PostgreSQL-First**: Database determines which subscriptions are affected
15
- 2. **Compiler-Driven**: All logic compiled to PostgreSQL functions (zero runtime interpretation)
16
- 3. **In-Memory Subscriptions**: Server holds active subscriptions in memory for performance
17
- 4. **Naming Convention**: `subscribe_<name>` / `unsubscribe_<name>` for pattern matching
18
-
19
- ---
20
-
21
- ## Core Concept: Subscribables
22
-
23
- **Subscribables** are separate from entities - they define denormalized documents that:
24
- - Combine data from multiple entities (root + relations)
25
- - Have their own access control (permission paths)
26
- - Define subscription parameters (subscription key)
27
- - Compile to PostgreSQL functions that determine affected documents
28
-
29
- ### Example Subscribable
30
-
31
- ```sql
32
- SELECT dzql.register_subscribable(
33
- 'venue_detail', -- Subscribable name
34
-
35
- -- Permission: who can subscribe?
36
- jsonb_build_object(
37
- 'subscribe', ARRAY['@org_id->acts_for[org_id=$]{active}.user_id']
38
- ),
39
-
40
- -- Parameters: subscription key
41
- jsonb_build_object(
42
- 'venue_id', 'int'
43
- ),
44
-
45
- -- Root entity
46
- 'venues',
47
-
48
- -- Relations to include
49
- jsonb_build_object(
50
- 'org', 'organisations',
51
- 'sites', jsonb_build_object(
52
- 'entity', 'sites',
53
- 'filter', 'venue_id=$venue_id'
54
- ),
55
- 'packages', jsonb_build_object(
56
- 'entity', 'packages',
57
- 'filter', 'venue_id=$venue_id',
58
- 'include', jsonb_build_object(
59
- 'allocations', 'allocations'
60
- )
61
- )
62
- )
63
- );
64
- ```
65
-
66
- ---
67
-
68
- ## Generated PostgreSQL Functions
69
-
70
- For each subscribable, the compiler generates 3 functions:
71
-
72
- ###1. Access Control: `<name>_can_subscribe(user_id, params)`
73
-
74
- ```sql
75
- CREATE OR REPLACE FUNCTION venue_detail_can_subscribe(
76
- p_user_id INT,
77
- p_params JSONB
78
- ) RETURNS BOOLEAN AS $$
79
- BEGIN
80
- -- Check permission path: @org_id->acts_for[org_id=$]{active}.user_id
81
- RETURN EXISTS (
82
- SELECT 1
83
- FROM venues v
84
- JOIN acts_for af ON af.org_id = v.org_id
85
- WHERE v.id = (p_params->>'venue_id')::int
86
- AND af.user_id = p_user_id
87
- AND af.valid_to IS NULL
88
- );
89
- END;
90
- $$ LANGUAGE plpgsql STABLE SECURITY DEFINER;
91
- ```
92
-
93
- ### 2. Query Function: `get_<name>(params, user_id)`
94
-
95
- ```sql
96
- CREATE OR REPLACE FUNCTION get_venue_detail(
97
- p_params JSONB,
98
- p_user_id INT
99
- ) RETURNS JSONB AS $$
100
- DECLARE
101
- v_venue_id int;
102
- v_result JSONB;
103
- BEGIN
104
- v_venue_id := (p_params->>'venue_id')::int;
105
-
106
- -- Check access control
107
- IF NOT venue_detail_can_subscribe(p_user_id, p_params) THEN
108
- RAISE EXCEPTION 'Permission denied';
109
- END IF;
110
-
111
- -- Build document with root and all relations
112
- SELECT jsonb_build_object(
113
- 'venues', row_to_json(root.*),
114
- 'org', (SELECT row_to_json(o.*) FROM organisations o WHERE o.id = root.org_id),
115
- 'sites', (SELECT jsonb_agg(s.*) FROM sites s WHERE s.venue_id = root.id),
116
- 'packages', (
117
- SELECT jsonb_agg(
118
- jsonb_build_object(
119
- 'package', row_to_json(p.*),
120
- 'allocations', (SELECT jsonb_agg(a.*) FROM allocations a WHERE a.package_id = p.id)
121
- )
122
- )
123
- FROM packages p WHERE p.venue_id = root.id
124
- )
125
- )
126
- INTO v_result
127
- FROM venues root
128
- WHERE root.id = v_venue_id;
129
-
130
- RETURN v_result;
131
- END;
132
- $$ LANGUAGE plpgsql SECURITY DEFINER;
133
- ```
134
-
135
- ### 3. Affected Documents: `<name>_affected_documents(table, op, old, new)`
136
-
137
- ```sql
138
- CREATE OR REPLACE FUNCTION venue_detail_affected_documents(
139
- p_table_name TEXT,
140
- p_op TEXT,
141
- p_old JSONB,
142
- p_new JSONB
143
- ) RETURNS JSONB[] AS $$
144
- DECLARE
145
- v_affected JSONB[];
146
- BEGIN
147
- CASE p_table_name
148
- -- Venue changed: affects subscription for that venue
149
- WHEN 'venues' THEN
150
- v_affected := ARRAY[
151
- jsonb_build_object('venue_id', COALESCE(p_new->>'id', p_old->>'id')::int)
152
- ];
153
-
154
- -- Organisation changed: affects all venues in that org
155
- WHEN 'organisations' THEN
156
- SELECT ARRAY_AGG(jsonb_build_object('venue_id', v.id))
157
- INTO v_affected
158
- FROM venues v
159
- WHERE v.org_id = COALESCE((p_new->>'id')::int, (p_old->>'id')::int);
160
-
161
- -- Site changed: affects parent venue
162
- WHEN 'sites' THEN
163
- v_affected := ARRAY[
164
- jsonb_build_object('venue_id', COALESCE(p_new->>'venue_id', p_old->>'venue_id')::int)
165
- ];
166
-
167
- -- Package changed: affects parent venue
168
- WHEN 'packages' THEN
169
- v_affected := ARRAY[
170
- jsonb_build_object('venue_id', COALESCE(p_new->>'venue_id', p_old->>'venue_id')::int)
171
- ];
172
-
173
- -- Allocation changed: affects venue via package
174
- WHEN 'allocations' THEN
175
- SELECT ARRAY_AGG(jsonb_build_object('venue_id', p.venue_id))
176
- INTO v_affected
177
- FROM packages p
178
- WHERE p.id = COALESCE((p_new->>'package_id')::int, (p_old->>'package_id')::int);
179
-
180
- ELSE
181
- v_affected := ARRAY[]::JSONB[];
182
- END CASE;
183
-
184
- RETURN v_affected;
185
- END;
186
- $$ LANGUAGE plpgsql IMMUTABLE;
187
- ```
188
-
189
- ---
190
-
191
- ## Server Implementation (In-Memory)
192
-
193
- ### Subscription Registry
194
-
195
- ```javascript
196
- // In-memory storage
197
- const subscriptions = new Map();
198
- // subscription_id -> { subscribable, user_id, connection_id, params }
199
-
200
- const connectionSubscriptions = new Map();
201
- // connection_id -> Set<subscription_id>
202
- ```
203
-
204
- ### RPC Handlers
205
-
206
- ```javascript
207
- // Pattern matching on method names
208
- if (method.startsWith('subscribe_')) {
209
- const subscribableName = method.replace('subscribe_', '');
210
- const subscriptionId = crypto.randomUUID();
211
-
212
- // Execute initial query (checks permissions)
213
- const data = await db.query(
214
- `SELECT get_${subscribableName}($1, $2) as data`,
215
- [params, userId]
216
- );
217
-
218
- // Store in memory
219
- subscriptions.set(subscriptionId, {
220
- subscribable: subscribableName,
221
- user_id: userId,
222
- connection_id: connectionId,
223
- params
224
- });
225
-
226
- return {
227
- subscription_id: subscriptionId,
228
- data: data.rows[0].data
229
- };
230
- }
231
-
232
- if (method.startsWith('unsubscribe_')) {
233
- // Remove from in-memory registry
234
- // Find and delete by params + connection
235
- }
236
- ```
237
-
238
- ### Event Listener
239
-
240
- ```javascript
241
- setupListeners(async (event) => {
242
- const { table, op, before, after } = event;
243
-
244
- // EXISTING: Pattern 2 - Need to Know notifications
245
- broadcast(...);
246
-
247
- // NEW: Pattern 1 - Live Query subscriptions
248
-
249
- // Group subscriptions by subscribable name
250
- const subsByName = new Map();
251
- for (const [subId, sub] of subscriptions.entries()) {
252
- if (!subsByName.has(sub.subscribable)) {
253
- subsByName.set(sub.subscribable, []);
254
- }
255
- subsByName.get(sub.subscribable).push({ subId, ...sub });
256
- }
257
-
258
- // For each subscribable, ask PostgreSQL which instances are affected
259
- for (const [subscribableName, subs] of subsByName.entries()) {
260
- const result = await db.query(
261
- `SELECT ${subscribableName}_affected_documents($1, $2, $3, $4) as affected`,
262
- [table, op, before, after]
263
- );
264
-
265
- const affectedParamSets = result.rows[0]?.affected || [];
266
-
267
- // Match affected params to active subscriptions (in-memory)
268
- for (const affectedParams of affectedParamSets) {
269
- for (const sub of subs) {
270
- if (paramsMatch(sub.params, affectedParams)) {
271
- // Re-execute query
272
- const updated = await db.query(
273
- `SELECT get_${subscribableName}($1, $2) as data`,
274
- [sub.params, sub.user_id]
275
- );
276
-
277
- // Broadcast to connection
278
- broadcastToConnection(sub.connection_id, {
279
- method: 'subscription:update',
280
- params: {
281
- subscription_id: sub.subId,
282
- data: updated.rows[0].data
283
- }
284
- });
285
- }
286
- }
287
- }
288
- }
289
- });
290
- ```
291
-
292
- ---
293
-
294
- ## Client API
295
-
296
- ```javascript
297
- // Subscribe
298
- const { data, unsubscribe } = await ws.api.subscribe_venue_detail(
299
- { venue_id: 1 },
300
- (updated) => {
301
- console.log('Venue updated:', updated);
302
- // Update UI
303
- }
304
- );
305
-
306
- // Initial data available immediately
307
- console.log('Initial:', data);
308
-
309
- // Unsubscribe
310
- unsubscribe();
311
-
312
- // Or call directly
313
- await ws.api.unsubscribe_venue_detail({ venue_id: 1 });
314
- ```
315
-
316
- ---
317
-
318
- ## Implementation Status
319
-
320
- ### ✅ Phase 1: Compiler (COMPLETE)
321
-
322
- **Files Created:**
323
- - `/packages/dzql/src/compiler/codegen/subscribable-codegen.js` - Code generation
324
- - `/packages/dzql/src/compiler/parser/subscribable-parser.js` - SQL parsing
325
- - `/packages/dzql/src/compiler/compiler.js` - Extended with subscribable support
326
-
327
- **Exports:**
328
- - `compileSubscribable(subscribable)` - Compile single subscribable
329
- - `compileAllSubscribables(subscribables[])` - Compile multiple
330
- - `compileSubscribablesFromSQL(sqlContent)` - Parse and compile from SQL file
331
-
332
- **Generated Functions:**
333
- - `<name>_can_subscribe(user_id, params)` - Access control
334
- - `get_<name>(params, user_id)` - Query function
335
- - `<name>_affected_documents(table, op, old, new)` - Affected params
336
-
337
- **Known Issue:**
338
- - Parser needs improvement for nested `jsonb_build_object()` calls
339
- - Test compilation failing on parameter splitting
340
-
341
- ---
342
-
343
- ### 🔨 Phase 2: Database Schema (TODO)
344
-
345
- **Tasks:**
346
- 1. Create `dzql.subscribables` table (metadata only)
347
- 2. Create `register_subscribable()` SQL function
348
- 3. Migration file: `011_subscriptions.sql`
349
-
350
- **Schema:**
351
- ```sql
352
- CREATE TABLE IF NOT EXISTS dzql.subscribables (
353
- name TEXT PRIMARY KEY,
354
- permission_paths jsonb NOT NULL,
355
- param_schema jsonb NOT NULL,
356
- root_entity text NOT NULL,
357
- relations jsonb NOT NULL,
358
- created_at timestamptz DEFAULT NOW()
359
- );
360
- ```
361
-
362
- ---
363
-
364
- ### 🔨 Phase 3: Server Integration (TODO)
365
-
366
- **Files to Modify:**
367
- 1. `/packages/dzql/src/server/ws.js`
368
- - Add in-memory subscription registry
369
- - Add `subscribe_*` / `unsubscribe_*` handlers
370
- - Add `broadcastToConnection()` function
371
-
372
- 2. `/packages/dzql/src/server/index.js`
373
- - Extend event listener to check affected subscriptions
374
- - Re-execute queries and broadcast updates
375
-
376
- **Estimated Effort:** 2-3 days
377
-
378
- ---
379
-
380
- ### 🔨 Phase 4: Client Integration (TODO)
381
-
382
- **Files to Modify:**
383
- 1. `/packages/dzql/src/client/ws.js`
384
- - Add `subscribe_*` method handling
385
- - Handle `subscription:update` messages
386
- - Return `{ data, unsubscribe }` pattern
387
-
388
- **Estimated Effort:** 1-2 days
389
-
390
- ---
391
-
392
- ## Testing Strategy
393
-
394
- ### Unit Tests (Compiler)
395
- ```javascript
396
- test('generates subscribable functions', () => {
397
- const result = compileSubscribable({
398
- name: 'venue_detail',
399
- permissionPaths: { subscribe: ['@org_id->acts_for...'] },
400
- paramSchema: { venue_id: 'int' },
401
- rootEntity: 'venues',
402
- relations: { org: 'organisations', sites: 'sites' }
403
- });
404
-
405
- expect(result.sql).toContain('venue_detail_can_subscribe');
406
- expect(result.sql).toContain('get_venue_detail');
407
- expect(result.sql).toContain('venue_detail_affected_documents');
408
- });
409
- ```
410
-
411
- ### Integration Tests (Database)
412
- ```sql
413
- -- Test affected documents function
414
- SELECT venue_detail_affected_documents(
415
- 'venues', 'update',
416
- '{"id": 1, "name": "Old"}'::jsonb,
417
- '{"id": 1, "name": "New"}'::jsonb
418
- );
419
- -- Should return: [{"venue_id": 1}]
420
-
421
- -- Test query function
422
- SELECT get_venue_detail('{"venue_id": 1}'::jsonb, 5);
423
- -- Should return denormalized document
424
- ```
425
-
426
- ### E2E Tests
427
- ```javascript
428
- test('subscription receives updates', async () => {
429
- const updates = [];
430
-
431
- const { data, unsubscribe } = await ws.api.subscribe_venue_detail(
432
- { venue_id: 1 },
433
- (updated) => updates.push(updated)
434
- );
435
-
436
- // Trigger change
437
- await ws.api.save.venues({ id: 1, name: 'Updated' });
438
-
439
- await waitFor(() => updates.length > 0);
440
- expect(updates[0].venues.name).toBe('Updated');
441
-
442
- unsubscribe();
443
- });
444
- ```
445
-
446
- ---
447
-
448
- ## Next Steps
449
-
450
- 1. **Fix Parser** - Handle nested `jsonb_build_object()` correctly
451
- 2. **Test Compilation** - Verify generated SQL is correct
452
- 3. **Create Migration** - `011_subscriptions.sql` with schema
453
- 4. **Implement Server Handlers** - In-memory subscriptions + event processing
454
- 5. **Implement Client Support** - `subscribe_*` methods
455
- 6. **Integration Testing** - End-to-end subscription flow
456
- 7. **Documentation** - API docs and examples
457
-
458
- ---
459
-
460
- ## Estimated Timeline
461
-
462
- | Phase | Tasks | Effort |
463
- |-------|-------|--------|
464
- | Phase 1: Compiler | ✅ Complete | 4 hours |
465
- | Phase 2: Database | Schema + migration | 1 day |
466
- | Phase 3: Server | Handlers + event processing | 2-3 days |
467
- | Phase 4: Client | Client API | 1-2 days |
468
- | Testing | Unit + Integration + E2E | 2-3 days |
469
- | **Total** | | **7-10 days** |
470
-
471
- ---
472
-
473
- ## Success Criteria
474
-
475
- - ✅ Compiler generates 3 PostgreSQL functions per subscribable
476
- - ⏳ PostgreSQL determines affected subscription instances (not server)
477
- - ⏳ Server holds subscriptions in-memory (fast lookup)
478
- - ⏳ Naming convention: `subscribe_<name>` / `unsubscribe_<name>`
479
- - ⏳ Client receives automatic updates on data changes
480
- - ⏳ < 100ms latency from DB change to client update
481
- - ⏳ Supports 1000+ concurrent subscriptions per server
482
- - ⏳ Zero runtime interpretation (all logic compiled)
483
-
484
- ---
485
-
486
- **Implementation by:** Claude Sonnet 4.5
487
- **Project:** DZQL Live Query Subscriptions
488
- **Version:** 0.1.4+subscriptions