@venizia/ignis-docs 0.0.3 → 0.0.4-0

package/wiki/changelogs/2025-12-29-dynamic-binding-registration.md

@@ -0,0 +1,104 @@
+---
+title: Dynamic Binding Registration Fix
+description: Fix components registering datasources/controllers during configure() phase
+---
+
+# Changelog - 2025-12-29
+
+## Dynamic Binding Registration Fix
+
+Components can now register datasources and controllers during their `configure()` phase, and those bindings will be properly configured.
+
+## Overview
+
+- **Simple Fix**: Re-run `registerDataSources()` after `registerComponents()`
+- **Future Enhancement**: Multi-pass configuration loop for comprehensive dynamic registration
+
+## Problem
+
+When components register datasources during `configure()`, those datasources were never configured because `registerDataSources()` had already completed.
+
+**Initialization Order (Before):**
+1. `registerDataSources()` → configures initial datasources
+2. `registerComponents()` → component registers NEW datasource (too late!)
+3. `registerControllers()` → runs normally
+
+The new datasource never gets its `configure()` called.
+
+## Solution (Simple Fix)
+
+**File:** `packages/core/src/base/applications/base.ts`
+
+Call `registerDataSources()` again after `registerComponents()`:
+
+```typescript
+// Before:
+await this.registerDataSources();
+await this.registerComponents();
+await this.registerControllers();
+
+// After:
+await this.registerDataSources();
+await this.registerComponents();
+await this.registerDataSources(); // Re-run for datasources added by components
+await this.registerControllers();
+```
+
+## Files Changed
+
+### Core Package (`packages/core`)
+
+| File | Changes |
+|------|---------|
+| `src/base/applications/base.ts` | Added second `registerDataSources()` call after `registerComponents()` |
+
+## No Breaking Changes
+
+All changes are internal. Components that register datasources will now work correctly.
+
+## Future Enhancement
+
+For comprehensive dynamic registration support (components registering other components, controllers registering datasources, etc.), a multi-pass configuration loop can be implemented:
+
+### Multi-Pass Configuration Loop (Future)
+
+Instead of sequential registration, use a unified loop that continues until no new bindings are discovered:
+
+```typescript
+protected async runConfigurationLoop(): Promise<void> {
+  const configured = {
+    datasources: new Set<string>(),
+    components: new Set<string>(),
+    controllers: new Set<string>(),
+  };
+
+  let hasNewBindings = true;
+  let iteration = 0;
+  const MAX_ITERATIONS = 100;
+
+  while (hasNewBindings && iteration < MAX_ITERATIONS) {
+    iteration++;
+    hasNewBindings = false;
+
+    // Order: datasources -> components -> controllers
+    const newDs = await this.configureNewBindings('datasources', configured.datasources);
+    const newComp = await this.configureNewBindings('components', configured.components);
+    const newCtrl = await this.configureNewBindings('controllers', configured.controllers);
+
+    hasNewBindings = newDs || newComp || newCtrl;
+  }
+}
+```
+
+**Benefits of Multi-Pass:**
+- Handles arbitrary nesting depth
+- Components can register datasources, controllers, or other components
+- Datasources configured before components that depend on them
+- MAX_ITERATIONS prevents infinite loops from circular dependencies
+
+**LoopBack 4 Insights Applied:**
+- Three-phase execution pattern (configure → discover → load)
+- Observer ordering (datasources before components before controllers)
+- Stabilization detection (loop until no new bindings)
+
+See full implementation plan in architecture documentation.

package/wiki/changelogs/2025-12-29-snowflake-uid-helper.md

@@ -0,0 +1,100 @@
+---
+title: Snowflake UID Helper
+description: New Snowflake ID generator with Base62 encoding for distributed systems
+---
+
+# Changelog - 2025-12-29
+
+## Snowflake UID Helper
+
+New unique ID generator using Twitter's Snowflake algorithm with Base62 encoding, suitable for distributed systems.
+
+## Overview
+
+- **New Helper**: `SnowflakeUidHelper` for generating unique, time-sortable IDs
+- **Base62 Encoding**: Compact string output (10-12 characters)
+- **Configurable**: Optional `workerId` and `epoch` parameters
+- **No Environment Variables**: Clean constructor-based configuration
+
+## New Features
+
+### SnowflakeUidHelper
+
+**File:** `packages/helpers/src/helpers/uid/helper.ts`
+
+**Problem:** Need unique, time-sortable IDs for distributed systems without external dependencies.
+
+**Solution:** Snowflake ID generator with 70-bit structure (48-10-12):
+- 48 bits: timestamp (~8,919 years from epoch)
+- 10 bits: worker ID (1024 workers max)
+- 12 bits: sequence (4096 per ms per worker)
+
+```typescript
+import { SnowflakeUidHelper, SnowflakeConfig } from '@venizia/ignis-helpers';
+
+// Use defaults (workerId: 199, epoch: 2025-01-01 00:00:00 UTC)
+const generator = new SnowflakeUidHelper();
+
+// Or with custom values
+const customGenerator = new SnowflakeUidHelper({
+  workerId: 123,
+  epoch: BigInt(1735689600000),
+});
+
+// Generate IDs
+const id = generator.nextId();           // Base62: "9du1sJXO88"
+const snowflake = generator.nextSnowflake(); // BigInt: 130546360012247045n
+
+// Parse IDs
+const parsed = generator.parseId("9du1sJXO88");
+// => { raw, timestamp, workerId, sequence }
+```
+
+**Benefits:**
+- High throughput: 4,096,000 IDs/second/worker
+- Time-sortable: IDs are chronologically ordered
+- Compact: 10-12 character Base62 strings
+- No collisions: Worker ID + sequence ensures uniqueness
+- Long lifespan: Until ~10,944 AD
+
+### Configuration Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `workerId` | `number` | `199` | Worker ID (0-1023) |
+| `epoch` | `bigint` | `1735689600000n` | Custom epoch (2025-01-01 UTC) |
+
+### Available Methods
+
+| Method | Returns | Description |
+|--------|---------|-------------|
+| `nextId()` | `string` | Generate Base62 encoded ID |
+| `nextSnowflake()` | `bigint` | Generate raw Snowflake ID |
+| `encodeBase62(num)` | `string` | Encode bigint to Base62 |
+| `decodeBase62(str)` | `bigint` | Decode Base62 to bigint |
+| `parseId(base62Id)` | `ISnowflakeParsedId` | Parse and extract components |
+| `extractTimestamp(id)` | `Date` | Extract timestamp from ID |
+| `extractWorkerId(id)` | `number` | Extract worker ID from ID |
+| `extractSequence(id)` | `number` | Extract sequence from ID |
+| `getWorkerId()` | `number` | Get current worker ID |
+
+## Files Changed
+
+### Helpers Package (`packages/helpers`)
+
+| File | Changes |
+|------|---------|
+| `src/helpers/uid/helper.ts` | New Snowflake ID generator class |
+| `src/helpers/uid/index.ts` | Export helper |
+| `src/helpers/index.ts` | Added uid export |
+
+### Docs Package (`packages/docs`)
+
+| File | Changes |
+|------|---------|
+| `wiki/references/helpers/uid.md` | New documentation |
+| `wiki/references/helpers/index.md` | Added UID to helper list |
+
+## No Breaking Changes
+
+This is a new feature addition. No existing APIs were modified.

package/wiki/changelogs/2025-12-30-repository-enhancements.md

@@ -0,0 +1,214 @@
+---
+title: Repository Enhancements
+description: Hidden properties, filter improvements, and code quality updates
+---
+
+# Changelog - 2025-12-30
+
+## Repository Enhancements
+
+This release introduces hidden properties configuration for models, array-based field selection, JSON path ordering, and several code quality improvements.
+
+## Overview
+
+- **Hidden Properties**: Configure properties that are never returned through repository queries (SQL-level exclusion)
+- **Array Fields Format**: Simpler syntax for field selection using arrays
+- **JSON Path Ordering**: Order by nested fields within JSON/JSONB columns
+- **Code Quality**: Refactored validation logic, improved caching patterns, consistent resolver pattern
+
+
+## Hidden Properties
+
+### Configuration
+
+Configure hidden properties in the `@model` decorator. These properties are excluded at the SQL level, never leaving the database in repository operations.
+
+```typescript
+import { pgTable, text } from 'drizzle-orm/pg-core';
+import { BaseEntity, model, generateIdColumnDefs } from '@venizia/ignis';
+
+@model({
+  type: 'entity',
+  settings: {
+    hiddenProperties: ['password', 'secret'],
+  },
+})
+export class User extends BaseEntity<typeof User.schema> {
+  static override schema = pgTable('User', {
+    ...generateIdColumnDefs({ id: { dataType: 'string' } }),
+    email: text('email').notNull(),
+    password: text('password'),
+    secret: text('secret'),
+  });
+}
+```
+
+### Behavior
+
+| Operation | Behavior |
+|-----------|----------|
+| `find()`, `findOne()`, `findById()` | Hidden excluded from SELECT |
+| `create()`, `createAll()` | Hidden excluded from RETURNING |
+| `updateById()`, `updateAll()` | Hidden excluded from RETURNING |
+| `deleteById()`, `deleteAll()` | Hidden excluded from RETURNING |
+| `count()`, `existsWith()` | Can filter by hidden fields |
+| Where clause filtering | Hidden fields usable in filters |
+| Direct connector query | Hidden fields **included** (bypasses repository) |
+
+### Accessing Hidden Data
+
+When you need to access hidden properties, use the connector directly:
+
+```typescript
+// Repository - excludes hidden
+const user = await userRepo.findById({ id: '123' });
+// { id: '123', email: 'john@example.com' }
+
+// Connector - includes all fields
+const connector = userRepo.getConnector();
+const [fullUser] = await connector
+  .select()
+  .from(User.schema)
+  .where(eq(User.schema.id, '123'));
+// { id: '123', email: 'john@example.com', password: '...', secret: '...' }
+```
+
+### Relations Support
+
+Hidden properties are recursively excluded from included relations:
+
+```typescript
+const post = await postRepo.findOne({
+  filter: {
+    include: [{ relation: 'author' }]
+  }
+});
+// post.author excludes password and secret if User model has them configured
+```
+
+
+## Array Fields Format
+
+**Before:** Only object format was supported:
+```typescript
+fields: { id: true, email: true, name: true }
+```
+
+**After:** Array format is now supported (recommended):
+```typescript
+fields: ['id', 'email', 'name']
+```
+
+Both formats produce the same result. The array format is more concise and easier to read.
+
+**Type Definition:**
+```typescript
+type TFields<T> = Partial<{ [K in keyof T]: boolean }> | Array<keyof T>;
+```
+
+
+## JSON Path Ordering
+
+Order by nested fields within JSON/JSONB columns using dot notation and array indices.
+
+```typescript
+// Simple nested field
+order: ['metadata.priority DESC']
+// SQL: ORDER BY "metadata" #> '{priority}' DESC
+
+// Deeply nested field
+order: ['settings.display.theme ASC']
+// SQL: ORDER BY "settings" #> '{display,theme}' ASC
+
+// Array element
+order: ['tags[0] ASC']
+// SQL: ORDER BY "tags" #> '{0}' ASC
+
+// Complex path with array
+order: ['data.items[2].name DESC']
+// SQL: ORDER BY "data" #> '{items,2,name}' DESC
+```
+
+**JSONB Sort Order:**
+
+| Type | Sort Order |
+|------|------------|
+| `null` | First (lowest) |
+| `boolean` | `false` < `true` |
+| `number` | Numeric order |
+| `string` | Lexicographic |
+| `array` | Element-wise |
+| `object` | Key-value |
+
+**Security:** Built-in SQL injection prevention via regex validation for path components.
+
+
+## Code Quality Improvements
+
+### 1. Consistent Resolver Pattern
+
+Added `THiddenPropertiesResolver` type and renamed method to match `getRelationResolver()` pattern:
+
+```typescript
+// Consistent pattern
+relationResolver: this.getRelationResolver(),
+hiddenPropertiesResolver: this.getHiddenPropertiesResolver(),
+```
+
+### 2. Early Return Pattern
+
+Refactored `resolveConnector()` from nested conditionals to early return:
+
+```typescript
+// Before
+if (transaction) {
+  if (!transaction.isActive) { throw... }
+  return transaction.connector;
+}
+return this.dataSource.connector;
+
+// After
+if (!transaction) {
+  return this.dataSource.connector;
+}
+if (!transaction.isActive) { throw... }
+return transaction.connector;
+```
+
+### 3. Extracted Validation Helper
+
+Created shared `validateWhereCondition()` method to eliminate duplicate validation logic.
+
+### 4. Improved Caching Pattern
+
+Fixed caching by initializing `_visibleColumns` to `null` as sentinel for "not computed yet".
+
+### 5. Added Null Check
+
+Added defensive check for `connector.query` in `getQueryInterface()`.
+
+
+## Files Changed
+
+### Core Package (`packages/core`)
+
+| File | Changes |
+|------|---------|
+| `src/helpers/inversion/common/types.ts` | Added `IModelSettings` with `hiddenProperties` |
+| `src/base/repositories/common/types.ts` | Updated `TFields` to support array format |
+| `src/base/repositories/core/base.ts` | Added `getHiddenPropertiesResolver()`, caching, refactored `resolveConnector` |
+| `src/base/repositories/core/readable.ts` | Added null check in `getQueryInterface` |
+| `src/base/repositories/core/persistable.ts` | Added `validateWhereCondition`, updated CRUD methods |
+| `src/base/repositories/operators/filter.ts` | Added `THiddenPropertiesResolver`, JSON ordering, array fields |
+
+### Examples (`examples/vert`)
+
+| File | Changes |
+|------|---------|
+| `src/models/entities/user.model.ts` | Added hiddenProperties config |
+| `src/services/repository-test.service.ts` | Added 21 hidden properties test cases |
+
+
+## No Breaking Changes
+
+All changes are additive. Existing code continues to work without modification.

package/wiki/changelogs/2025-12-31-json-path-filtering-array-operators.md

@@ -0,0 +1,214 @@
+---
+title: JSON Path Filtering & Array Operators
+description: Added JSON/JSONB path filtering support and PostgreSQL array column operators
+---
+
+# Changelog - 2025-12-31
+
+## JSON Path Filtering & PostgreSQL Array Operators
+
+This release adds powerful query capabilities for JSON/JSONB columns and PostgreSQL array columns, enabling complex filtering patterns without raw SQL.
+
+## Overview
+
+- **JSON Path Filtering**: Filter by nested JSON fields using dot notation (e.g., `'jValue.metadata.score': { gt: 80 }`)
+- **Array Column Operators**: PostgreSQL-specific operators `contains`, `containedBy`, `overlaps` for array columns
+- **Safe Numeric Casting**: Automatic type-safe numeric comparison for mixed-type JSON fields
+- **NOT BETWEEN Operator**: Added `notBetween` operator for range exclusion queries
+- **Code Refactoring**: Unified JSON path validation and extraction logic
+
+## New Features
+
+### JSON Path Filtering
+
+**Files:**
+- `packages/core/src/base/repositories/operators/filter.ts`
+- `packages/core/src/base/repositories/operators/query.ts`
+
+**Problem:** Filtering by nested JSON/JSONB fields required raw SQL or manual extraction, making queries complex and error-prone.
+
+**Solution:** Detect JSON paths in filter keys (containing `.` or `[`) and automatically generate PostgreSQL `#>>` extraction expressions with proper type handling.
+
+```typescript
+// Before: Not possible with standard filters
+
+// After: Native JSON path support
+await repo.find({
+  filter: {
+    where: {
+      // Simple nested field
+      'jValue.priority': 3,
+
+      // Deep nesting
+      'jValue.metadata.level': 'high',
+
+      // Array index access
+      'jValue.tags[0]': 'important',
+
+      // With operators
+      'jValue.metadata.score': { gte: 70, lte: 90 },
+
+      // Pattern matching
+      'jValue.metadata.level': { ilike: '%igh%' }
+    }
+  }
+});
+```
+
+**Generated SQL:**
+```sql
+WHERE "jValue" #>> '{priority}' = '3'
+  AND "jValue" #>> '{metadata,level}' = 'high'
+  AND "jValue" #>> '{tags,0}' = 'important'
+  AND CASE WHEN ("jValue" #>> '{metadata,score}') ~ '^-?[0-9]+'
+      THEN ("jValue" #>> '{metadata,score}')::numeric ELSE NULL END >= 70
+  AND CASE WHEN ("jValue" #>> '{metadata,score}') ~ '^-?[0-9]+'
+      THEN ("jValue" #>> '{metadata,score}')::numeric ELSE NULL END <= 90
+  AND "jValue" #>> '{metadata,level}' ILIKE '%igh%'
+```
+
+**Benefits:**
+- Intuitive dot notation for nested fields
+- Automatic numeric casting for comparison operators
+- Safe handling of mixed-type JSON (non-numeric values become NULL, not errors)
+- SQL injection prevention via path component validation
+
+### Safe Numeric Casting for JSON
+
+**Problem:** JSON fields can contain mixed types (`{ score: 85 }` vs `{ score: "high" }`). Casting to numeric would crash on non-numeric values.
+
+**Solution:** Safe casting pattern that validates numeric format before casting:
+
+```sql
+CASE WHEN ("jValue" #>> '{score}') ~ '^-?[0-9]+(\.[0-9]+)?$'
+     THEN ("jValue" #>> '{score}')::numeric
+     ELSE NULL
+END
+```
+
+This ensures:
+- Numeric values: Compared correctly as numbers
+- String values: Treated as NULL (excluded from numeric comparisons)
+- No database errors on mixed-type JSON data
+
+### PostgreSQL Array Column Operators
+
+**File:** `packages/core/src/base/repositories/operators/query.ts`
+
+**Problem:** PostgreSQL array columns (`varchar[]`, `integer[]`, etc.) require special operators (`@>`, `<@`, `&&`) that weren't available in the filter builder.
+
+**Solution:** Added three new operators with automatic type handling:
+
+| Operator | PostgreSQL | Description |
+|----------|------------|-------------|
+| `contains` | `@>` | Array contains ALL specified elements |
+| `containedBy` | `<@` | Array is subset of specified elements |
+| `overlaps` | `&&` | Arrays share ANY common element |
+
+```typescript
+// Schema: tags varchar(100)[]
+
+// Find products with BOTH 'electronics' AND 'featured'
+await repo.find({
+  filter: {
+    where: { tags: { contains: ['electronics', 'featured'] } }
+  }
+});
+// SQL: "tags"::text[] @> ARRAY['electronics', 'featured']::text[]
+
+// Find products where ALL tags are in allowed list
+await repo.find({
+  filter: {
+    where: { tags: { containedBy: ['sale', 'featured', 'new'] } }
+  }
+});
+// SQL: "tags"::text[] <@ ARRAY['sale', 'featured', 'new']::text[]
+
+// Find products with 'sale' OR 'premium' tag
+await repo.find({
+  filter: {
+    where: { tags: { overlaps: ['sale', 'premium'] } }
+  }
+});
+// SQL: "tags"::text[] && ARRAY['sale', 'premium']::text[]
+```
+
+**Type Compatibility:**
+- String arrays (`varchar[]`, `text[]`, `char[]`): Both column and value cast to `text[]`
+- Numeric arrays (`integer[]`, `numeric[]`): No casting needed
+- Boolean arrays (`boolean[]`): No casting needed
+
+**Empty Array Behavior:**
+
+| Operator | Empty `[]` | Result |
+|----------|------------|--------|
+| `contains` | `{ contains: [] }` | `true` - everything contains empty set |
+| `containedBy` | `{ containedBy: [] }` | Only rows with empty arrays |
+| `overlaps` | `{ overlaps: [] }` | `false` - nothing overlaps with empty |
+
+### NOT BETWEEN Operator
+
+**File:** `packages/core/src/base/repositories/operators/query.ts`
+
+**Problem:** No way to filter for values outside a range.
+
+**Solution:** Added `notBetween` operator:
+
+```typescript
+await repo.find({
+  filter: {
+    where: {
+      score: { notBetween: [40, 60] }  // Scores < 40 OR > 60
+    }
+  }
+});
+// SQL: WHERE NOT (score BETWEEN 40 AND 60)
+```
+
+## Security Enhancements
+
+### JSON Path Validation
+
+JSON path components are validated against a strict pattern to prevent SQL injection:
+
+```typescript
+// Valid patterns (allowed)
+/^[a-zA-Z_][a-zA-Z0-9_-]*$|^\d+$/
+
+// Examples:
+'jValue.user_id'         // ✅ Valid identifier
+'jValue.meta-data'       // ✅ Kebab-case allowed
+'jValue.items[0]'        // ✅ Array index
+'jValue.nested[2].field' // ✅ Mixed access
+
+// Invalid (throws error)
+'jValue.field;DROP TABLE' // ❌ SQL injection attempt
+'jValue.123invalid'       // ❌ Starts with number
+```
+
+## Files Changed
+
+### Core Package (`packages/core`)
+
+| File | Changes |
+|------|---------|
+| `src/base/repositories/operators/filter.ts` | Added JSON path filtering (`buildJsonWhereCondition`, `validateJsonColumn`, `isJsonPath`, `isOperatorObject`), refactored `buildJsonOrderBy` to reuse validation |
+| `src/base/repositories/operators/query.ts` | Added array operators (`contains`, `containedBy`, `overlaps`), `notBetween`, `hasNumericComparison` helper, `buildPgArrayComparison` helper |
+
+### Examples (`examples/vert`)
+
+| File | Changes |
+|------|---------|
+| `src/models/entities/product.model.ts` | Added array column (`tags varchar(100)[]`) for testing |
+| `src/services/tests/` | New test services for JSON and array operators |
+
+## No Breaking Changes
+
+All changes are additive enhancements. Existing filter queries work unchanged.
+
+## Documentation
+
+Full documentation with examples available at:
+- [Repositories - JSON Path Filtering](/references/base/repositories#json-path-filtering)
+- [Repositories - Array Column Operators](/references/base/repositories#array-column-operators-postgresql)
+- [Repositories - Query Operators](/references/base/repositories#query-operators)