langaro-api 1.2.3 → 1.2.4

package/lib/cli/documentation-templates/02-crud-layer.md

@@ -0,0 +1,504 @@
+# CRUD Layer (knex-extended-crud)
+
+The `CRUD` class from `knex-extended-crud` is the foundation of all database operations. Every model and service inherits from it.
+
+---
+
+## Constructor Parameters
+
+When the loader creates a model, it passes these to the CRUD constructor:
+
+```javascript
+{
+  knex,                        // Knex instance (injected by loader)
+  table,                       // Table name (injected by loader)
+  hide: [],                    // Fields never returned in responses
+  append: {},                  // Relationship definitions
+  fields: {},                  // Field configuration
+  schema: null,                // Yup validation schema
+  fieldsSpec: [],              // Per-field specifications
+  forceInteger: [],            // Fields stored as int but exposed as float (÷100)
+  sortable: false,             // Enable manual sorting
+  redis: null,                 // Redis instance for query caching
+  cacheEnabled: true,          // Master cache toggle
+  cacheTimeout: 300000,        // Schema cache TTL in ms (5 min)
+  singularTableName: null,     // Override singular form (for relationship detection)
+  pluralTableName: null,       // Override plural form
+  notSearchableFields: [],     // Fields excluded from search
+  modelsPath: '',              // Path to model files (for append resolution)
+}
+```
+
+---
+
+## Default Options
+
+Every CRUD instance has these defaults for query operations:
+
+```javascript
+this.options = {
+  perPage: 100,              // Records per page
+  currentPage: 1,            // Starting page
+  isLengthAware: true,       // Include total count in pagination
+  exactMatch: false,         // Search uses LIKE %term% by default
+  sortBy: 'id',             // Default sort column
+  sort: 'desc',             // Default sort direction
+}
+```
+
+---
+
+## Read Methods
+
+### `get(options, transaction)`
+
+Retrieves records with pagination, filtering, searching, and relationship loading.
+
+**Options (CRUDGetOptions):**
+
+```javascript
+{
+  // Pagination
+  perPage: 100,                    // Records per page
+  currentPage: 1,                  // Page number
+
+  // Sorting
+  sortBy: 'created_at',           // Sort column
+  sort: 'desc',                   // 'asc' or 'desc'
+
+  // Field selection
+  showOnly: ['id', 'name'],       // Return ONLY these fields
+  show: ['field1'],               // Whitelist (inverse of hide)
+
+  // Filtering
+  where: (query) => {             // Knex query builder function
+    query.where('status', 'active');
+  },
+  andWhere: [                      // Array-based conditions (see andWhere section)
+    ['status', 'active'],
+    ['amount', '>=', 100],
+  ],
+
+  // Search
+  search: 'john',                 // Search term
+  searchFields: ['name', 'email'],// Fields to search (overrides model config)
+  searchExactMatch: false,        // true = exact, false = fuzzy (%term%)
+
+  // Relationships
+  append: ['companies', 'users'], // Relations to load
+  appendOptions: {                // Per-relation options
+    companies: { showOnly: ['id', 'name'] },
+    users: { show: ['id', 'email'] },
+  },
+
+  // Caching
+  cache: true,                    // Read from Redis cache
+  cacheHours: 24,                 // Write result to cache with TTL
+
+  // Special
+  firstOnly: true,                // Return only the first result
+}
+```
+
+**Returns:**
+```javascript
+{
+  success: true,
+  data: [...],                    // Array of records (or single object if firstOnly)
+  pagination: {                   // Only if isLengthAware
+    perPage: 100,
+    currentPage: 1,
+    total: 500,
+    lastPage: 5
+  },
+  cached: true                    // Present if served from cache
+}
+```
+
+### `getWhere(prop, value, options, transaction)`
+
+Retrieves records matching a single condition. Shorthand for `get()` with a where clause.
+
+```javascript
+const { data: user } = await this.service.getWhere('id', userId, {
+  firstOnly: true,
+  append: ['users_permissions'],
+});
+```
+
+### `search(field, term, options, transaction)`
+
+Full-text search on a specific field. Uses case-insensitive `LOWER()` comparison.
+
+```javascript
+const results = await this.service.search('name', 'john', {
+  perPage: 50,
+  searchExactMatch: false,
+});
+```
+
+The `field` must exist in the table and not be in `notSearchableFields`.
+
+### `count(options, transaction)`
+
+Counts matching records.
+
+```javascript
+const { data: total } = await this.service.count({
+  andWhere: [['status', 'active']],
+});
+// total = 125
+```
+
+### `tableInfo(options, transaction)`
+
+Returns schema information for the table. Cached in memory.
+
+```javascript
+const { data: schema } = await this.service.tableInfo();
+// schema = { id: { type: 'uuid', nullable: false, ... }, name: { type: 'varchar', ... } }
+```
+
+### `requiredFields(options, transaction)`
+
+Returns list of non-nullable field names.
+
+```javascript
+const { data: required } = await this.service.requiredFields();
+// required = ['name', 'email', 'company_id']
+```
+
+---
+
+## Write Methods
+
+### `create(data, transaction)`
+
+Creates a single record.
+
+**Automatic behaviors:**
+- Generates UUID for `id` if not provided
+- Hashes `password` field with bcrypt (10 salt rounds)
+- Converts `forceInteger` fields (multiplies by 100)
+- Validates against Yup schema
+- Clears table query cache
+
+```javascript
+const { data: created } = await this.service.create({
+  name: 'John',
+  email: 'john@example.com',
+  company_id: companyId,
+});
+// created = { id: 'uuid-xxx' }
+```
+
+**Returns:** `{ success: true, data: { id: 'generated-uuid' } }`
+
+### `batchCreate(data[], transaction)`
+
+Creates multiple records in a single INSERT.
+
+```javascript
+const { data: created } = await this.service.batchCreate([
+  { name: 'John', email: 'john@example.com' },
+  { name: 'Jane', email: 'jane@example.com' },
+]);
+// created = [{ id: 'uuid-1' }, { id: 'uuid-2' }]
+```
+
+Same auto-behaviors as `create()` applied to each item.
+
+---
+
+## Update Methods
+
+### `updateWhere(prop, value, data, options, transaction)`
+
+Updates records matching a condition.
+
+**Parameters:**
+- `prop` — Column to match (e.g., `'id'`)
+- `value` — Value to match
+- `data` — Fields to update
+- `options` — Update options
+- `transaction` — Optional Knex transaction
+
+**Options (CRUDUpdateOptions):**
+```javascript
+{
+  allowForbiddenUpdates: false,    // Allow updating protected fields
+  where: (query) => {},            // Additional where clause
+  andWhere: [...],                 // Additional conditions
+  skipCacheInvalidation: false,    // Don't clear Redis cache
+  whenSuccess: (data, value) => {},// Callback after successful update
+  extraValidations: [              // Custom validation functions
+    async (existingItem, newData, trx) => {
+      // Throw to prevent update
+    }
+  ],
+}
+```
+
+**Protected fields** (cannot be updated without `allowForbiddenUpdates`):
+- `id`, `created_at`, `updated_at`
+- Fields listed in `fields.notUpdatable`
+
+**Automatic behaviors:**
+- Per-field Yup schema validation
+- Password auto-hashing
+- JSON field stringification
+- Foreign key reference validation
+- Cache invalidation
+
+```javascript
+await this.service.updateWhere('id', userId, {
+  name: 'New Name',
+  email: 'new@email.com',
+});
+```
+
+**Returns:** `{ success: true, data: { id: 'uuid-xxx' } }`
+
+### `batchUpdate(data[], options, transaction)`
+
+Updates multiple records with concurrency control.
+
+**Options (CRUDBatchUpdateOptions):**
+```javascript
+{
+  getItemBy: 'id',                // Field to identify each item (default: 'id')
+  socket: socketInstance,         // Socket.io for progress events
+  ...CRUDUpdateOptions
+}
+```
+
+**Behavior:**
+1. Pre-fetches all items in a single query
+2. Limits concurrency to 10 parallel updates (prevents connection pool exhaustion)
+3. Creates its own transaction if none provided (auto commit/rollback)
+4. Emits Socket.io progress events: `batchUpdate:started`, `batchUpdate:progress`, `batchUpdate:completed`
+
+```javascript
+const results = await this.service.batchUpdate([
+  { id: 'uuid-1', name: 'Updated 1' },
+  { id: 'uuid-2', name: 'Updated 2' },
+], { socket: io.sockets.in(userId) });
+// results = [{ id: 'uuid-1', success: true }, { id: 'uuid-2', success: true }]
+```
+
+---
+
+## Delete Methods
+
+### `deleteWhere(prop, values, options, transaction)`
+
+Deletes records matching condition(s).
+
+```javascript
+// Single delete
+await this.service.deleteWhere('id', itemId);
+
+// Batch delete
+await this.service.deleteWhere('id', [id1, id2, id3]);
+
+// Conditional delete
+await this.service.deleteWhere('id', itemId, {
+  andWhere: [['company_id', companyId]],
+});
+```
+
+**Returns:** `{ success: true, data: { id: [...] } }`
+
+---
+
+## andWhere Query Builder
+
+The `andWhere` option provides a declarative way to build complex WHERE clauses:
+
+```javascript
+andWhere: [
+  // Equality: [column, value]
+  ['status', 'active'],
+
+  // Comparison: [column, operator, value]
+  ['amount', '>=', 100],
+  ['created_at', '<', '2025-01-01'],
+
+  // IN operator: [column, 'in', array]
+  ['status', 'in', ['active', 'pending']],
+
+  // NOT IN: [column, 'not in', array]
+  ['role', 'not in', ['banned']],
+
+  // BETWEEN: [column, 'between', [start, end]]
+  ['created_at', 'between', ['2025-01-01', '2025-12-31']],
+
+  // NULL handling
+  ['deleted_at', null],           // WHERE deleted_at IS NULL
+  ['deleted_at', '<>', null],     // WHERE deleted_at IS NOT NULL
+
+  // LIKE: [column, 'like', pattern]
+  ['name', 'like', '%john%'],
+
+  // IN with NULL: [column, 'in', [values, null]]
+  ['status', 'in', ['active', null]],  // WHERE status IN ('active') OR status IS NULL
+
+  // Boolean columns (TINYINT): use 1/0, NOT true/false
+  ['is_active', 1],              // WHERE is_active = 1
+  ['is_deleted', 0],             // WHERE is_deleted = 0
+]
+```
+
+**Supported operators:** `=`, `<>`, `>`, `<`, `>=`, `<=`, `in`, `not in`, `between`, `not between`, `like`
+
+**Boolean gotcha:** MySQL stores booleans as `TINYINT(1)`. In `andWhere`, use `1`/`0` — not `true`/`false`. JavaScript `true`/`false` does not work correctly with the query builder.
+
+---
+
+## Relationship System (append)
+
+Relationships are configured in the model file and resolved at query time.
+
+### Configuration (in model file)
+
+```javascript
+module.exports = {
+  append: {
+    companies: 'one',                    // This table has company_id FK → load one company
+    posts: 'many',                       // Other table has this_table_id FK → load many posts
+    categories: 'categories_users_join', // Many-to-many via join table
+  },
+};
+```
+
+### Relationship Types
+
+**one** — This table has a foreign key pointing to the related table:
+- `users` table has `company_id` → `append: { companies: 'one' }`
+- Returns: each user gets a `companies` object
+
+**many** — The related table has a foreign key pointing to this table:
+- `posts` table has `user_id` → on users model: `append: { posts: 'many' }`
+- Returns: each user gets a `posts` array
+
+**join table** — Many-to-many via intermediate table:
+- `categories_users_join` table with `category_id` and `user_id`
+- On users model: `append: { categories: 'categories_users_join' }`
+
+### Nested Appending
+
+Up to 3 levels of nesting:
+```javascript
+append: ['companies', 'companies.subscriptions', 'companies.subscriptions.plans']
+```
+
+### appendOptions
+
+Control which fields are loaded per relation:
+```javascript
+{
+  append: ['companies', 'users'],
+  appendOptions: {
+    companies: { showOnly: ['id', 'name'] },  // Only id and name
+    users: { show: ['id', 'email', 'name'] }, // Only these fields
+  },
+}
+```
+
+---
+
+## Validation Pipeline
+
+When creating or updating records, the CRUD class runs validation in this order:
+
+### On Create (`defaultInsertValidations`):
+1. Check for forbidden fields (fields in `hide` list)
+2. Validate required fields (non-nullable fields)
+3. Run `validateAndFormatFields`:
+   - Field exists in table schema
+   - Nullable check
+   - Empty string check (if `notEmpty` spec)
+   - Max length check
+   - Allowed values check (enum)
+   - Yup schema validation (per-field)
+   - Unique constraint check
+   - Foreign key reference validation
+   - Boolean type check
+   - Integer type check
+   - JSON field handling (auto-stringify objects)
+   - Email lowercase normalization
+
+### On Update (`defaultUpdateValidations`):
+1. Verify item exists (throws NOT_FOUND)
+2. Check for protected fields (id, created_at, updated_at, notUpdatable fields)
+3. Run custom `extraValidations` if provided
+4. Check for forbidden fields (unless `allowForbiddenUpdates`)
+5. Run same `validateAndFormatFields` pipeline (per changed field only)
+
+---
+
+## Transaction Management
+
+```javascript
+const trx = await this.service.createTransaction();
+try {
+  const { data: user } = await this.service.create(userData, trx);
+  await this.services.PermissionsServices.batchCreate(permissions, trx);
+  await trx.commit();
+  return user;
+} catch (error) {
+  await trx.rollback();
+  throw error;
+}
+```
+
+**Rules:**
+- All methods accept an optional `transaction` parameter as their last argument
+- If no transaction is provided, methods use the default knex instance
+- `batchUpdate` creates its own transaction if none is provided
+- Always use try/catch/rollback — there is no auto-rollback
+
+---
+
+## Redis Query Cache
+
+The CRUD layer supports two caching tiers:
+
+### In-Memory Schema Cache
+- Stores table schema info (`tableInfo` results)
+- TTL: `cacheTimeout` (default 5 min)
+- Cleared via `clearSchemaCache()` or `clearTableCache(tableName)`
+
+### Redis Query Cache
+- Stores full query results
+- Key format: `crud-cache:{table}:{method}:{sha256hash}`
+- **Read from cache**: set `cache: true` in options
+- **Write to cache**: set `cacheHours: N` in options
+- **Invalidation**: Automatic on create/update/delete (unless `skipCacheInvalidation`)
+- **Disable**: `disableCache()` / `enableCache()`
+
+**Cache management methods:**
+```javascript
+this.service.clearQueryCache();       // Invalidate this table's cache
+this.service.clearSchemaCache();      // Clear all schema caches
+this.service.clearTableCache('users');// Clear specific table schema
+this.service.refreshCache(trx);      // Pre-load caches
+CRUD.clearAllCrudCache(redis);       // Global invalidation (static)
+```
+
+---
+
+## forceInteger Fields
+
+For fields that store monetary values as integers (cents) but should be exposed as decimals:
+
+```javascript
+// Model config
+{ forceInteger: ['price', 'amount'] }
+
+// On create: value * 100 before INSERT
+// price: 29.99 → stored as 2999
+
+// On read: value / 100 after SELECT
+// stored 2999 → returned as 29.99
+```