dzql 0.6.5 → 0.6.7

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dzql",
-  "version": "0.6.5",
+  "version": "0.6.7",
   "description": "Database-first real-time framework with TypeScript support",
   "repository": {
     "type": "git",
@@ -18,7 +18,9 @@
   },
   "files": [
     "src",
-    "docs",
+    "docs/README.md",
+    "docs/project-setup.md",
+    "docs/for_ai.md",
     "README.md"
   ],
   "scripts": {

package/docs/feature-requests/applyPatch-bug-report.md

@@ -1,85 +0,0 @@
-# DZQL Bug Report
-
-## Generated Store applyPatch Doesn't Match Data Structure
-
-**Severity:** High (Breaking)
-
-**Status:** ✅ FIXED
-
-**Description:**
-The generated Pinia store's `applyPatch` function assumes a flat data structure, but the subscribable SQL was returning nested wrapper objects.
-
-**SQL was returning:**
-```json
-{
-  "venues": {...},
-  "sites": [
-    { "sites": { "id": 1, "name": "..." }, "allocations": [...] },
-    { "sites": { "id": 2, "name": "..." }, "allocations": [...] }
-  ]
-}
-```
-
-**Fix:**
-Updated `subscribable_sql.ts` to use JSONB concatenation (`||`) to merge entity fields with nested includes into a flat structure:
-
-```sql
-SELECT jsonb_agg(
-  row_to_json(rel.*) || jsonb_build_object(
-    'allocations', COALESCE((
-      SELECT jsonb_agg(row_to_json(nested.*))
-      FROM allocations nested
-      WHERE nested.site_id = rel.id
-    ), '[]'::jsonb))
-)
-```
-
-**SQL now returns:**
-```json
-{
-  "venues": {...},
-  "sites": [
-    { "id": 1, "name": "...", "allocations": [...] },
-    { "id": 2, "name": "...", "allocations": [...] }
-  ]
-}
-```
-
-This flat structure matches what `applyPatch` and `handleArrayPatch` expect, so realtime updates now work correctly for nested includes.
-
----
-
-## Bug 2: json/jsonb Type Mismatch in Subscribable SQL
-
-**Severity:** Critical (Blocking)
-
-**Status:** ✅ FIXED
-
-**Description:**
-The generated subscribable SQL used `row_to_json()` which returns `json` type, but concatenated it with `jsonb_build_object()` which returns `jsonb`. PostgreSQL cannot concatenate these types.
-
-**Generated SQL (before):**
-```sql
-row_to_json(rel.*) || jsonb_build_object(...)
-```
-
-**Error:**
-```
-operator does not exist: json || jsonb
-```
-
-**Fix:**
-Changed to use `to_jsonb()` instead of `row_to_json()` when concatenating with nested includes:
-
-```sql
-to_jsonb(rel.*) || jsonb_build_object(...)
-```
-
----
-
-## Environment
-
-- dzql version: local development (linked)
-- Database: PostgreSQL 17 (Docker)
-- Client: Vue 3 + Pinia + TypeScript
-- Runtime: Bun

package/docs/feature-requests/connection-ready-profile.md

@@ -1,57 +0,0 @@
-# Feature Request: Send User Profile on WebSocket Connect
-
-**Status: IMPLEMENTED**
-
-## Summary
-
-When a client connects to the DZQL WebSocket server, the server should immediately send a connection:ready message containing the authenticated user profile (or null if not authenticated).
-
-## Current Behavior
-
-1. Client connects with optional ?token=... in URL
-2. Server opens connection but sends nothing
-3. Client must call auth RPC to authenticate, then separately fetch profile
-4. Client has no immediate knowledge of auth state
-
-## Proposed Behavior
-
-1. Client connects with optional ?token=... in URL
-2. Server validates token (if present) and fetches user profile
-3. Server sends first message:
-
-{"method": "connection:ready", "params": {"user": {"id": 1, "name": "...", "email": "..."} | null}}
-
-4. Client knows auth state immediately, can render accordingly
-
-## Why This Matters
-
-- Single source of truth: WebSocket connection determines auth state, not localStorage
-- No race conditions: UI waits for connection:ready before rendering
-- Simpler client code: No need for separate auth check after connect
-- Better UX: App shows loading state until connection ready, then immediately correct view
-
-## Client Usage Pattern
-
-Template:
-  div v-if="!ready" - loading spinner
-  LoginModal v-else-if="!user"
-  RouterView v-else
-
-## Suggested Server Changes
-
-In src/runtime/ws.ts, modify handleOpen to:
-1. Parse token from URL query params
-2. If valid, verify token and fetch user profile
-3. Send connection:ready message with user (or null)
-
-## Suggested Client Changes
-
-In src/client/ws.ts:
-1. Add user and ready properties to class
-2. Handle connection:ready message in handleMessage
-3. Add onReady callback method
-
-## Migration
-
-- Existing clients that dont handle connection:ready will ignore it (no breaking change)
-- New clients can opt-in to the pattern

package/docs/feature-requests/hidden-bug-report.md

@@ -1,111 +0,0 @@
-# DZQL Bug Report
-
-Bugs discovered while building the Venues application with DZQL.
-
----
-
-## Bug 1: Hidden Fields Exposed in Subscribables
-
-**Severity:** High (Security)
-
-**Status:** ✅ FIXED
-
-**Description:**
-Fields marked as `hidden: true` in the domain schema are still exposed when using subscribables. The generated SQL uses `row_to_json(root.*)` which includes all columns regardless of the `hidden` property.
-
-**Fix:**
-Added `buildVisibleRowJson` helper in `subscribable_sql.ts` that generates `jsonb_build_object()` with only visible fields instead of `row_to_json(root.*)`. Also updated all other SQL generators (`sql.ts`) to exclude hidden fields from `get_*`, `search_*`, `save_*`, and `delete_*` functions.
-
----
-
-## Bug 2: Generated Pinia Stores Don't Await Async Subscribe
-
-**Severity:** Medium
-
-**Status:** ✅ FIXED
-
-**Description:**
-The generated Pinia stores call the async `subscribe` method but don't await it. This means `store.data` is undefined immediately after calling `store.subscribe()` even though the subscription has been initiated.
-
-**Fix:**
-Updated `subscribable_store.ts` to make the `bind` function async. It now:
-1. Creates a `ready` Promise that resolves when first data arrives
-2. Awaits the `ready` Promise before returning
-3. Stores the `ready` Promise so repeat calls can also await it
-
-**Generated Code (after fix):**
-```typescript
-async function bind(params) {
-  const key = JSON.stringify(params);
-  if (documents.value[key]) {
-    const existing = documents.value[key];
-    if (existing.loading.value) {
-      await existing.ready;
-    }
-    return existing;
-  }
-
-  const docState = ref(null);
-  const loading = ref(true);
-  let resolveReady;
-  const ready = new Promise((resolve) => { resolveReady = resolve; });
-
-  ws.api.subscribe_venue_detail(params, (initialData) => {
-    docState.value = initialData;
-    loading.value = false;
-    resolveReady();
-  });
-
-  documents.value[key] = { data: docState, loading, ready };
-  await ready;
-  return documents.value[key];
-}
-```
-
-**Usage:**
-```typescript
-const store = useMyProfileStore();
-const { data, loading } = await store.bind({ user_id: 1 });
-// data.value is now populated
-```
-
----
-
-## Bug 3: Subscribable Permission Check Uses Param Name Instead of Column Name
-
-**Severity:** High (Breaking)
-
-**Status:** ✅ FIXED
-
-**Description:**
-When generating SQL for subscribable permission checks, the compiler uses the parameter name instead of the actual database column name. This causes SQL errors when the param name differs from the column name.
-
-**Fix:**
-Updated `compileSubscribePermission` in `subscribable_sql.ts` to map param names to the root entity's `id` column when they match the rootKey. For example, `@org_id` now correctly resolves to `v_root.id` instead of `v_root.org_id`.
-
----
-
-## Summary
-
-| Bug | Severity | Status | Notes |
-|-----|----------|--------|-------|
-| Hidden fields exposed | High | ✅ Fixed | Uses `jsonb_build_object` to exclude hidden fields |
-| Stores don't await | Medium | ✅ Fixed | `bind()` is now async and awaits first data |
-| Permission param/column mismatch | High | ✅ Fixed | Maps param name to `id` column |
-
----
-
-## Environment
-
-- dzql version: (linked local development version)
-- Database: PostgreSQL 17 (via Docker)
-- Client: Vue 3 + Pinia + TypeScript
-- Runtime: Bun
-
----
-
-## Related Files
-
-Detailed bug documents created in dzql docs:
-- `/packages/tzql/docs/feature-requests/hidden-fields-in-subscribables.md`
-- `/packages/tzql/docs/feature-requests/subscribable-param-key-bug.md`

package/docs/feature-requests/hidden-fields-subscribables.md

@@ -1,34 +0,0 @@
-# Bug: Hidden fields exposed in subscribables
-
-**Status: IMPLEMENTED**
-
-## Issue
-
-Entity hidden fields are not excluded from subscribable queries.
-
-## Example
-
-domain.js:
-users: {
-  schema: { ... },
-  hidden: ["password_hash"],  // Should be excluded from all queries
-}
-
-But get_my_profile returns:
-{
-  "users": {
-    "password_hash": "$2a$06$..."  // EXPOSED!
-  }
-}
-
-## Root Cause
-
-The subscribable SQL generator uses row_to_json(root.*) which returns all columns.
-
-## Fix
-
-When generating subscribable SQL, exclude hidden fields by using explicit column list.
-
-## Impact
-
-Security vulnerability - sensitive data like password hashes exposed to clients.

package/docs/feature-requests/subscribable-param-key-bug.md

@@ -1,38 +0,0 @@
-# Bug: Subscribable permission check uses param name instead of column name
-
-## Issue
-
-When generating subscribable SQL, the permission check incorrectly uses the param name instead of the actual column name on the root entity.
-
-## Example
-
-Domain:
-```javascript
-org_dashboard: {
-  params: { org_id: "int" },
-  root: { entity: "organisations", key: "org_id" },
-  canSubscribe: ["@org_id->acts_for[org_id=$]{active}.user_id"]
-}
-```
-
-Generated SQL:
-```sql
--- WRONG: v_root.org_id doesn't exist on organisations table
-WHERE acts_for.org_id = v_root.org_id
-```
-
-Should be:
-```sql
--- CORRECT: organisations.id is the actual column
-WHERE acts_for.org_id = v_root.id
-```
-
-## Error
-
-```
-ERROR: record "v_root" has no field "org_id"
-```
-
-## Fix
-
-The compiler should resolve `key: "org_id"` to mean "param org_id maps to the root entity's primary key (id)", not "access v_root.org_id".

package/docs/feature-requests/todo.md

@@ -1,146 +0,0 @@
-# DZQL v2 Gap Analysis TODO
-
-Gap analysis comparing v1 tests to v2 functionality. Updated 2024-12-17.
-
----
-
-## High Priority - COMPLETED
-
-### Soft Delete ✅
-- [x] Add `softDelete` handling to `generateDeleteFunction()` in `src/cli/codegen/sql.ts`
-  - Sets `deleted_at = now()` instead of `DELETE FROM` when entity has `softDelete: true`
-- [x] Add `deleted_at IS NULL` filter to `generateSearchFunction()`
-- [x] GET still retrieves deleted records (for audit purposes)
-- [x] Add soft delete integration tests (3 tests)
-- Reference: `tests/integration/features.test.ts` - "Soft Delete" describe block
-
-### Field Defaults ✅
-- [x] Add `fieldDefaults` handling to `generateSaveFunction()` INSERT branch in `src/cli/codegen/sql.ts`
-  - `@user_id` → `p_user_id`
-  - `@now` → `now()`
-  - `@today` → `current_date`
-  - Literal values → insert as-is
-- [x] Only apply defaults on INSERT, not UPDATE (uses COALESCE)
-- [x] Add field defaults integration tests (4 tests)
-- Reference: `tests/integration/features.test.ts` - "Field Defaults" describe block
-
-### Composite Primary Key Support ✅
-- [x] Update `generateSaveFunction()` to handle multi-column PKs
-  - Dynamic PK null check and EXISTS query
-  - WHERE clause supports all PK fields
-- [x] Update `generateDeleteFunction()` for composite PKs
-- [x] Update `generateGetFunction()` for composite PKs
-- [x] Events store composite PK as JSONB object
-- [x] Proper type casting for date, boolean, numeric columns
-- [x] Add composite PK integration tests (4 tests)
-- Reference: `tests/integration/features.test.ts` - "Composite Primary Keys" describe block
-
----
-
-## Medium Priority - V1 Features Not Ported
-
-### Dashboard Collection (filter: TRUE)
-- [ ] Add `filter: TRUE` handling to `generateSubscribableGetFunction()` in `src/cli/codegen/subscribable_sql.ts`
-  - Fetch ALL rows regardless of FK relationship to root
-- [ ] Add `filter: TRUE` handling to `affected_keys` function
-  - Return `'{}'::jsonb` to notify ALL subscribers
-- [ ] Add dashboard collection integration test
-- Reference: `tests/integration/dashboard-collection.test.js` (v1)
-
-### Null-Root Dashboard
-- [ ] Support subscribables with no root entity parameter
-  - `root.key: '@user_id'` or similar built-in
-- [ ] Add null-root dashboard test
-- Reference: `tests/integration/null-root-dashboard.test.js` (v1)
-
-### Delete Subscription Resolution
-- [ ] Verify `affected_keys` function receives full record data on DELETE
-- [ ] Add integration test for DELETE event subscription routing
-- Reference: `tests/integration/delete-subscription-resolution.test.js` (v1)
-
-### Atomic Subscription Updates
-- [ ] Add integration test for Pinia store `applyPatch` with real WebSocket events
-- [ ] Verify schema inclusion in subscribe response
-- Reference: `tests/integration/atomic-subscription-updates.test.js` (v1)
-
----
-
-## Low Priority - COMPLETED
-
-### Custom Functions Pass-through ✅
-- [x] Parse `customFunctions` from domain definition in IR generator
-- [x] Include custom function SQL in generated migrations
-- [x] Add custom functions to manifest allowlist for runtime security
-- [x] Add custom functions integration tests (4 tests)
-- Reference: `tests/integration/features.test.ts` - "Custom Functions" describe block
-
-### JavaScript Custom Functions ✅
-- [x] Add JS function handler registry (`src/runtime/js_functions.ts`)
-- [x] Update `handleRequest` to check for JS handlers (takes precedence over SQL)
-- [x] JS functions receive context with `userId`, `params`, and `db.query()` access
-- [x] Export `registerJsFunction` API from runtime
-- [x] Add JavaScript custom function integration tests (5 tests)
-- Reference: `tests/integration/features.test.ts` - "JavaScript Custom Functions" describe block
-
-### Temporal Table Support
-- [ ] Implement `temporal.validFrom` / `temporal.validTo` filtering
-- [ ] Auto-filter to current validity period
-- Schema already defined in `venues.js:acts_for.temporal`
-
-### On-Delete Graph Rules
-- [ ] Verify `onDelete` graph rules receive correct record data
-- [ ] Test cascading deletes
-- Reference: `tests/integration/on-delete-graph-rules.test.js` (v1)
-
----
-
-## Integration Test Coverage
-
-### Missing Integration Tests (port from v1)
-- [ ] Security / SQL injection prevention (`security.test.js`)
-- [ ] M2M runtime operations (`m2m-runtime.test.js`)
-- [ ] Permissions enforcement (`permissions.test.js` - more comprehensive)
-- [ ] Event validation (`event-validation.test.js`)
-- [ ] Namespace/entity discovery (`namespace-cli.test.js`)
-- [ ] Migration runner (`migrations.test.js`)
-
-### Current V2 Integration Tests
-- `tests/integration/db.test.ts` - CRUD, permissions, events, graph rules ✅
-- `tests/integration/e2e.test.ts` - Compiler pipeline ✅
-- `tests/integration/full_stack.test.ts` - Runtime + WebSocket + Pinia ✅
-- `tests/integration/features.test.ts` - Search filters, deep paths, M2M, soft delete, field defaults, composite PK ✅
-
----
-
-## Not Applicable to V2
-
-### Interpreted Mode
-- V1 had runtime `register_entity` + generic functions
-- V2 is compile-only by design - no gap, intentional removal
-
----
-
-## Feature Comparison Summary
-
-| Feature | V1 | V2 | Status |
-|---------|----|----|--------|
-| Authentication | ✅ | ✅ | Done |
-| CRUD Operations | ✅ | ✅ | Done |
-| Permissions | ✅ | ✅ | Done |
-| Graph Rules | ✅ | ✅ | Done |
-| M2M Relationships | ✅ | ✅ | Done |
-| Events | ✅ | ✅ | Done |
-| Subscribables | ✅ | ✅ | Done |
-| Security (allowlist) | ✅ | ✅ | Done |
-| Search Operators | ✅ | ✅ | Done |
-| Deep Permission Paths | ✅ | ✅ | Done |
-| Pinia Stores | ❌ | ✅ | New in V2 |
-| .env Support | ❌ | ✅ | New in V2 |
-| Soft Delete | ✅ | ✅ | Done |
-| Field Defaults | ✅ | ✅ | Done |
-| Composite PK | ✅ | ✅ | Done |
-| Custom Functions (SQL) | ✅ | ✅ | Done |
-| Custom Functions (JS) | ✅ | ✅ | Done |
-| Dashboard Collection | ✅ | ❌ | **TODO** |
-| Null-Root Dashboard | ✅ | ❌ | **TODO** |
-| Interpreted Mode | ✅ | ❌ | Removed |