@venizia/ignis-docs 0.0.2 → 0.0.4-0

package/wiki/changelogs/2025-12-26-nested-relations-and-generics.md

@@ -0,0 +1,86 @@
+---
+title: Nested Relations & Generic Types
+description: Added support for deep nested inclusions and generic return types in repositories
+---
+
+# Changelog - 2025-12-26
+
+## Nested Relations Support & Generic Repository Types
+
+This update introduces support for deeply nested relation queries in repositories and enables stronger type safety via generic return types.
+
+## Overview
+
+- **Nested Inclusions**: `include` filters now work recursively to any depth.
+- **Generic Repository Methods**: `find<R>`, `findOne<R>`, etc. now support custom return types.
+- **FilterBuilder Decoupling**: Decoupled filter builder from MetadataRegistry for cleaner architecture.
+
+## New Features
+
+### Nested Relations Support
+
+**File:** `packages/core/src/base/repositories/operators/filter.ts`
+
+**Problem:** Previously, the `FilterBuilder` could only resolve relations for the root entity. Nested includes (e.g., `include: [{ relation: 'a', scope: { include: [{ relation: 'b' }] } }]`) failed because it didn't know the schema of relation 'a'.
+
+**Solution:** The builder now accepts a `relationResolver` function (injected from the Repository) which allows it to dynamically lookup schemas and relations for any entity during recursive traversal.
+
+```typescript
+// Now works recursively!
+const result = await repo.findOne({
+  filter: {
+    include: [{
+      relation: 'orders',
+      scope: {
+        include: [{ relation: 'items' }] // Level 2
+      }
+    }]
+  }
+});
+```
+
+### Generic Repository Methods
+
+**File:** `packages/core/src/base/repositories/common/types.ts`
+
+**Problem:** Repository methods like `findOne` were hardcoded to return the base `DataObject`. When using `include`, the return type didn't reflect the added relations, forcing users to use `as any` or unsafe casts.
+
+**Solution:** All read and write methods now accept a generic type parameter `<R>` which defaults to the entity's schema but can be overridden.
+
+```typescript
+// Define expected shape
+type UserWithPosts = User & { posts: Post[] };
+
+// Type-safe call
+const user = await userRepo.findOne<UserWithPosts>({
+  filter: { include: [{ relation: 'posts' }] }
+});
+
+// TypeScript knows 'posts' exists!
+console.log(user?.posts.length);
+```
+
+**Benefits:**
+- Stronger type safety in application code.
+- Reduces need for `as any` casting.
+- Better IDE auto-completion.
+
+## Files Changed
+
+### Core Package (`packages/core`)
+
+| File | Changes |
+|------|---------|
+| `src/base/repositories/common/types.ts` | Added generic `<R>` to all repository interface methods |
+| `src/base/repositories/core/base.ts` | Implemented `getRelationResolver` and updated abstract signatures |
+| `src/base/repositories/core/readable.ts` | Updated `find`, `findOne`, `findById` to support generics |
+| `src/base/repositories/core/persistable.ts` | Updated `create`, `update`, `delete` methods to support generics |
+| `src/base/repositories/operators/filter.ts` | Added `relationResolver` support for recursive includes |
+
+## Migration Guide
+
+### No Breaking Changes
+
+This update is fully backward compatible.
+- Existing calls to repository methods will continue to work (using the default `DataObject` return type).
+- The new generic type parameter is optional.

package/wiki/changelogs/2025-12-26-transaction-support.md

@@ -0,0 +1,57 @@
+---
+title: Transaction Support
+description: Explicit transaction support for atomic operations across repositories
+---
+
+# Changelog - 2025-12-26
+
+## Transaction Support
+
+Added explicit transaction support to the repository layer, enabling atomic operations across multiple services and repositories.
+
+## Overview
+
+- **Explicit Transactions**: `beginTransaction()` method on repositories.
+- **Isolation Levels**: Support for `READ COMMITTED`, `REPEATABLE READ`, and `SERIALIZABLE`.
+- **Pass-through Context**: Transaction objects can be passed via `options`.
+
+## New Features
+
+### Repository Transactions
+
+**File:** `packages/core/src/base/repositories/core/base.ts`
+
+**Problem:** Previously, transactions relied on Drizzle's callback API, which made it difficult to share a transaction context across different services or decoupled components.
+
+**Solution:** Introduced explicit `ITransaction` objects managed by the repository/datasource.
+
+```typescript
+const tx = await repo.beginTransaction();
+
+try {
+  await repo.create({ data: {...}, options: { transaction: tx } });
+  await otherRepo.create({ data: {...}, options: { transaction: tx } });
+  
+  await tx.commit();
+} catch (err) {
+  await tx.rollback();
+}
+```
+
+## Files Changed
+
+### Core Package (`packages/core`)
+
+| File | Changes |
+|------|---------|
+| `src/base/repositories/core/base.ts` | Added `beginTransaction` and transaction handling logic |
+| `src/base/repositories/core/readable.ts` | Updated read methods to respect transaction context |
+| `src/base/repositories/core/persistable.ts` | Updated write methods to respect transaction context |
+| `src/base/datasources/common/types.ts` | Added `ITransaction`, `ITransactionOptions`, and `IsolationLevels` |
+| `src/base/datasources/base.ts` | Implemented `beginTransaction` on `BaseDataSource` |
+
+## Migration Guide
+
+### No Breaking Changes
+
+This is an additive feature. Existing code using standard CRUD methods will continue to work without modification (using implicit auto-commit transactions).

package/wiki/changelogs/2025-12-29-dynamic-binding-registration.md

@@ -0,0 +1,104 @@
+---
+title: Dynamic Binding Registration Fix
+description: Fix components registering datasources/controllers during configure() phase
+---
+
+# Changelog - 2025-12-29
+
+## Dynamic Binding Registration Fix
+
+Components can now register datasources and controllers during their `configure()` phase, and those bindings will be properly configured.
+
+## Overview
+
+- **Simple Fix**: Re-run `registerDataSources()` after `registerComponents()`
+- **Future Enhancement**: Multi-pass configuration loop for comprehensive dynamic registration
+
+## Problem
+
+When components register datasources during `configure()`, those datasources were never configured because `registerDataSources()` had already completed.
+
+**Initialization Order (Before):**
+1. `registerDataSources()` → configures initial datasources
+2. `registerComponents()` → component registers NEW datasource (too late!)
+3. `registerControllers()` → runs normally
+
+The new datasource never gets its `configure()` called.
+
+## Solution (Simple Fix)
+
+**File:** `packages/core/src/base/applications/base.ts`
+
+Call `registerDataSources()` again after `registerComponents()`:
+
+```typescript
+// Before:
+await this.registerDataSources();
+await this.registerComponents();
+await this.registerControllers();
+
+// After:
+await this.registerDataSources();
+await this.registerComponents();
+await this.registerDataSources(); // Re-run for datasources added by components
+await this.registerControllers();
+```
+
+## Files Changed
+
+### Core Package (`packages/core`)
+
+| File | Changes |
+|------|---------|
+| `src/base/applications/base.ts` | Added second `registerDataSources()` call after `registerComponents()` |
+
+## No Breaking Changes
+
+All changes are internal. Components that register datasources will now work correctly.
+
+## Future Enhancement
+
+For comprehensive dynamic registration support (components registering other components, controllers registering datasources, etc.), a multi-pass configuration loop can be implemented:
+
+### Multi-Pass Configuration Loop (Future)
+
+Instead of sequential registration, use a unified loop that continues until no new bindings are discovered:
+
+```typescript
+protected async runConfigurationLoop(): Promise<void> {
+  const configured = {
+    datasources: new Set<string>(),
+    components: new Set<string>(),
+    controllers: new Set<string>(),
+  };
+
+  let hasNewBindings = true;
+  let iteration = 0;
+  const MAX_ITERATIONS = 100;
+
+  while (hasNewBindings && iteration < MAX_ITERATIONS) {
+    iteration++;
+    hasNewBindings = false;
+
+    // Order: datasources -> components -> controllers
+    const newDs = await this.configureNewBindings('datasources', configured.datasources);
+    const newComp = await this.configureNewBindings('components', configured.components);
+    const newCtrl = await this.configureNewBindings('controllers', configured.controllers);
+
+    hasNewBindings = newDs || newComp || newCtrl;
+  }
+}
+```
+
+**Benefits of Multi-Pass:**
+- Handles arbitrary nesting depth
+- Components can register datasources, controllers, or other components
+- Datasources configured before components that depend on them
+- MAX_ITERATIONS prevents infinite loops from circular dependencies
+
+**LoopBack 4 Insights Applied:**
+- Three-phase execution pattern (configure → discover → load)
+- Observer ordering (datasources before components before controllers)
+- Stabilization detection (loop until no new bindings)
+
+See full implementation plan in architecture documentation.

package/wiki/changelogs/2025-12-29-snowflake-uid-helper.md

@@ -0,0 +1,100 @@
+---
+title: Snowflake UID Helper
+description: New Snowflake ID generator with Base62 encoding for distributed systems
+---
+
+# Changelog - 2025-12-29
+
+## Snowflake UID Helper
+
+New unique ID generator using Twitter's Snowflake algorithm with Base62 encoding, suitable for distributed systems.
+
+## Overview
+
+- **New Helper**: `SnowflakeUidHelper` for generating unique, time-sortable IDs
+- **Base62 Encoding**: Compact string output (10-12 characters)
+- **Configurable**: Optional `workerId` and `epoch` parameters
+- **No Environment Variables**: Clean constructor-based configuration
+
+## New Features
+
+### SnowflakeUidHelper
+
+**File:** `packages/helpers/src/helpers/uid/helper.ts`
+
+**Problem:** Need unique, time-sortable IDs for distributed systems without external dependencies.
+
+**Solution:** Snowflake ID generator with 70-bit structure (48-10-12):
+- 48 bits: timestamp (~8,919 years from epoch)
+- 10 bits: worker ID (1024 workers max)
+- 12 bits: sequence (4096 per ms per worker)
+
+```typescript
+import { SnowflakeUidHelper, SnowflakeConfig } from '@venizia/ignis-helpers';
+
+// Use defaults (workerId: 199, epoch: 2025-01-01 00:00:00 UTC)
+const generator = new SnowflakeUidHelper();
+
+// Or with custom values
+const customGenerator = new SnowflakeUidHelper({
+  workerId: 123,
+  epoch: BigInt(1735689600000),
+});
+
+// Generate IDs
+const id = generator.nextId();           // Base62: "9du1sJXO88"
+const snowflake = generator.nextSnowflake(); // BigInt: 130546360012247045n
+
+// Parse IDs
+const parsed = generator.parseId("9du1sJXO88");
+// => { raw, timestamp, workerId, sequence }
+```
+
+**Benefits:**
+- High throughput: 4,096,000 IDs/second/worker
+- Time-sortable: IDs are chronologically ordered
+- Compact: 10-12 character Base62 strings
+- No collisions: Worker ID + sequence ensures uniqueness
+- Long lifespan: Until ~10,944 AD
+
+### Configuration Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `workerId` | `number` | `199` | Worker ID (0-1023) |
+| `epoch` | `bigint` | `1735689600000n` | Custom epoch (2025-01-01 UTC) |
+
+### Available Methods
+
+| Method | Returns | Description |
+|--------|---------|-------------|
+| `nextId()` | `string` | Generate Base62 encoded ID |
+| `nextSnowflake()` | `bigint` | Generate raw Snowflake ID |
+| `encodeBase62(num)` | `string` | Encode bigint to Base62 |
+| `decodeBase62(str)` | `bigint` | Decode Base62 to bigint |
+| `parseId(base62Id)` | `ISnowflakeParsedId` | Parse and extract components |
+| `extractTimestamp(id)` | `Date` | Extract timestamp from ID |
+| `extractWorkerId(id)` | `number` | Extract worker ID from ID |
+| `extractSequence(id)` | `number` | Extract sequence from ID |
+| `getWorkerId()` | `number` | Get current worker ID |
+
+## Files Changed
+
+### Helpers Package (`packages/helpers`)
+
+| File | Changes |
+|------|---------|
+| `src/helpers/uid/helper.ts` | New Snowflake ID generator class |
+| `src/helpers/uid/index.ts` | Export helper |
+| `src/helpers/index.ts` | Added uid export |
+
+### Docs Package (`packages/docs`)
+
+| File | Changes |
+|------|---------|
+| `wiki/references/helpers/uid.md` | New documentation |
+| `wiki/references/helpers/index.md` | Added UID to helper list |
+
+## No Breaking Changes
+
+This is a new feature addition. No existing APIs were modified.

package/wiki/changelogs/2025-12-30-repository-enhancements.md

@@ -0,0 +1,214 @@
+---
+title: Repository Enhancements
+description: Hidden properties, filter improvements, and code quality updates
+---
+
+# Changelog - 2025-12-30
+
+## Repository Enhancements
+
+This release introduces hidden properties configuration for models, array-based field selection, JSON path ordering, and several code quality improvements.
+
+## Overview
+
+- **Hidden Properties**: Configure properties that are never returned through repository queries (SQL-level exclusion)
+- **Array Fields Format**: Simpler syntax for field selection using arrays
+- **JSON Path Ordering**: Order by nested fields within JSON/JSONB columns
+- **Code Quality**: Refactored validation logic, improved caching patterns, consistent resolver pattern
+
+
+## Hidden Properties
+
+### Configuration
+
+Configure hidden properties in the `@model` decorator. These properties are excluded at the SQL level, never leaving the database in repository operations.
+
+```typescript
+import { pgTable, text } from 'drizzle-orm/pg-core';
+import { BaseEntity, model, generateIdColumnDefs } from '@venizia/ignis';
+
+@model({
+  type: 'entity',
+  settings: {
+    hiddenProperties: ['password', 'secret'],
+  },
+})
+export class User extends BaseEntity<typeof User.schema> {
+  static override schema = pgTable('User', {
+    ...generateIdColumnDefs({ id: { dataType: 'string' } }),
+    email: text('email').notNull(),
+    password: text('password'),
+    secret: text('secret'),
+  });
+}
+```
+
+### Behavior
+
+| Operation | Behavior |
+|-----------|----------|
+| `find()`, `findOne()`, `findById()` | Hidden excluded from SELECT |
+| `create()`, `createAll()` | Hidden excluded from RETURNING |
+| `updateById()`, `updateAll()` | Hidden excluded from RETURNING |
+| `deleteById()`, `deleteAll()` | Hidden excluded from RETURNING |
+| `count()`, `existsWith()` | Can filter by hidden fields |
+| Where clause filtering | Hidden fields usable in filters |
+| Direct connector query | Hidden fields **included** (bypasses repository) |
+
+### Accessing Hidden Data
+
+When you need to access hidden properties, use the connector directly:
+
+```typescript
+// Repository - excludes hidden
+const user = await userRepo.findById({ id: '123' });
+// { id: '123', email: 'john@example.com' }
+
+// Connector - includes all fields
+const connector = userRepo.getConnector();
+const [fullUser] = await connector
+  .select()
+  .from(User.schema)
+  .where(eq(User.schema.id, '123'));
+// { id: '123', email: 'john@example.com', password: '...', secret: '...' }
+```
+
+### Relations Support
+
+Hidden properties are recursively excluded from included relations:
+
+```typescript
+const post = await postRepo.findOne({
+  filter: {
+    include: [{ relation: 'author' }]
+  }
+});
+// post.author excludes password and secret if User model has them configured
+```
+
+
+## Array Fields Format
+
+**Before:** Only object format was supported:
+```typescript
+fields: { id: true, email: true, name: true }
+```
+
+**After:** Array format is now supported (recommended):
+```typescript
+fields: ['id', 'email', 'name']
+```
+
+Both formats produce the same result. The array format is more concise and easier to read.
+
+**Type Definition:**
+```typescript
+type TFields<T> = Partial<{ [K in keyof T]: boolean }> | Array<keyof T>;
+```
+
+
+## JSON Path Ordering
+
+Order by nested fields within JSON/JSONB columns using dot notation and array indices.
+
+```typescript
+// Simple nested field
+order: ['metadata.priority DESC']
+// SQL: ORDER BY "metadata" #> '{priority}' DESC
+
+// Deeply nested field
+order: ['settings.display.theme ASC']
+// SQL: ORDER BY "settings" #> '{display,theme}' ASC
+
+// Array element
+order: ['tags[0] ASC']
+// SQL: ORDER BY "tags" #> '{0}' ASC
+
+// Complex path with array
+order: ['data.items[2].name DESC']
+// SQL: ORDER BY "data" #> '{items,2,name}' DESC
+```
+
+**JSONB Sort Order:**
+
+| Type | Sort Order |
+|------|------------|
+| `null` | First (lowest) |
+| `boolean` | `false` < `true` |
+| `number` | Numeric order |
+| `string` | Lexicographic |
+| `array` | Element-wise |
+| `object` | Key-value |
+
+**Security:** Built-in SQL injection prevention via regex validation for path components.
+
+
+## Code Quality Improvements
+
+### 1. Consistent Resolver Pattern
+
+Added `THiddenPropertiesResolver` type and renamed method to match `getRelationResolver()` pattern:
+
+```typescript
+// Consistent pattern
+relationResolver: this.getRelationResolver(),
+hiddenPropertiesResolver: this.getHiddenPropertiesResolver(),
+```
+
+### 2. Early Return Pattern
+
+Refactored `resolveConnector()` from nested conditionals to early return:
+
+```typescript
+// Before
+if (transaction) {
+  if (!transaction.isActive) { throw... }
+  return transaction.connector;
+}
+return this.dataSource.connector;
+
+// After
+if (!transaction) {
+  return this.dataSource.connector;
+}
+if (!transaction.isActive) { throw... }
+return transaction.connector;
+```
+
+### 3. Extracted Validation Helper
+
+Created shared `validateWhereCondition()` method to eliminate duplicate validation logic.
+
+### 4. Improved Caching Pattern
+
+Fixed caching by initializing `_visibleColumns` to `null` as sentinel for "not computed yet".
+
+### 5. Added Null Check
+
+Added defensive check for `connector.query` in `getQueryInterface()`.
+
+
+## Files Changed
+
+### Core Package (`packages/core`)
+
+| File | Changes |
+|------|---------|
+| `src/helpers/inversion/common/types.ts` | Added `IModelSettings` with `hiddenProperties` |
+| `src/base/repositories/common/types.ts` | Updated `TFields` to support array format |
+| `src/base/repositories/core/base.ts` | Added `getHiddenPropertiesResolver()`, caching, refactored `resolveConnector` |
+| `src/base/repositories/core/readable.ts` | Added null check in `getQueryInterface` |
+| `src/base/repositories/core/persistable.ts` | Added `validateWhereCondition`, updated CRUD methods |
+| `src/base/repositories/operators/filter.ts` | Added `THiddenPropertiesResolver`, JSON ordering, array fields |
+
+### Examples (`examples/vert`)
+
+| File | Changes |
+|------|---------|
+| `src/models/entities/user.model.ts` | Added hiddenProperties config |
+| `src/services/repository-test.service.ts` | Added 21 hidden properties test cases |
+
+
+## No Breaking Changes
+
+All changes are additive. Existing code continues to work without modification.