rez_core 6.1.0 → 6.1.1

package/src/module/workflow-schedule/README.md

@@ -1,422 +1,422 @@
-# Workflow Schedule Module
-
-A comprehensive NestJS module for scheduling and executing workflows using Bull queues and Redis.
-
-## Features
-
-- ✅ Schedule workflows with cron expressions
-- ✅ Timezone support (IANA timezone format)
-- ✅ Pause/resume schedules
-- ✅ Manual trigger execution
-- ✅ Comprehensive execution logging
-- ✅ Retry with exponential backoff
-- ✅ Batch processing (100 records per batch)
-- ✅ Multiple action types:
-  - Send Email
-  - Update Records
-  - Create Tasks
-  - Send Notifications
-
-## Installation
-
-### 1. Install Required Dependencies
-
-```bash
-npm install @nestjs/bull bull redis cron-parser
-npm install --save-dev @types/bull
-```
-
-Note: `cron-parser` includes its own TypeScript definitions, so no separate `@types` package is needed.
-
-### 2. Configure Redis
-
-Add Redis configuration to your `.env` file:
-
-```env
-REDIS_HOST=localhost
-REDIS_PORT=6379
-REDIS_PASSWORD=
-REDIS_DB=0
-```
-
-### 3. Run Migrations
-
-The module includes three migrations:
-
-1. **AddWorkflowTypeColumn** - Adds `workflow_type` to `cr_wf_master` table
-2. **CreateScheduledWorkflowsTable** - Creates `cr_wf_scheduled_workflow` table
-3. **CreateWorkflowExecutionLogsTable** - Creates `cr_wf_execution_log` table
-
-To run migrations:
-
-```bash
-# Using TypeORM CLI
-npx typeorm migration:run -d src/config/database.config.ts
-
-# Or using npm script (if configured)
-npm run migration:run
-```
-
-### 4. Start Redis Server
-
-Make sure Redis is running:
-
-```bash
-# If using Docker
-docker run -d -p 6379:6379 redis
-
-# If installed locally
-redis-server
-```
-
-## Module Structure
-
-```
-src/module/workflow-schedule/
-├── constants/
-│   └── schedule.constants.ts          # Constants and enums
-├── interfaces/
-│   └── schedule-job-data.interface.ts # TypeScript interfaces
-├── entities/
-│   ├── scheduled-workflow.entity.ts   # ScheduledWorkflow entity
-│   └── workflow-execution-log.entity.ts # WorkflowExecutionLog entity
-├── dto/
-│   ├── create-schedule.dto.ts         # Create schedule DTO
-│   ├── update-schedule.dto.ts         # Update schedule DTO
-│   └── get-execution-logs.dto.ts      # Query execution logs DTO
-├── processors/
-│   └── schedule.processor.ts          # Bull job processor
-├── service/
-│   └── workflow-schedule.service.ts   # Business logic (9 methods)
-├── controller/
-│   └── workflow-schedule.controller.ts # REST API (9 endpoints)
-└── workflow-schedule.module.ts        # Module definition
-```
-
-## API Endpoints
-
-### 1. Create Schedule
-```http
-POST /workflow-schedule/create
-Authorization: Bearer <token>
-
-{
-  "workflow_id": 1,
-  "workflow_name": "Daily Lead Assignment",
-  "name": "Assign leads every morning",
-  "description": "Automatically assign new leads to sales team",
-  "cron_expression": "0 9 * * 1-5",
-  "timezone": "Asia/Kolkata",
-  "actions": [
-    {
-      "actionType": "UPDATE_RECORDS",
-      "targetEntityType": "LEAD",
-      "actionConfig": {
-        "updateFields": {
-          "status": "ASSIGNED"
-        }
-      }
-    }
-  ]
-}
-```
-
-### 2. Update Schedule
-```http
-PUT /workflow-schedule/update
-
-{
-  "id": 1,
-  "cron_expression": "0 10 * * 1-5"
-}
-```
-
-### 3. Get Schedule by ID
-```http
-GET /workflow-schedule/:id
-```
-
-### 4. Get All Schedules
-```http
-GET /workflow-schedule/list?page=1&size=10&schedule_status=ACTIVE
-```
-
-### 5. Pause Schedule
-```http
-POST /workflow-schedule/:id/pause
-```
-
-### 6. Resume Schedule
-```http
-POST /workflow-schedule/:id/resume
-```
-
-### 7. Delete Schedule
-```http
-DELETE /workflow-schedule/:id
-```
-
-### 8. Manual Trigger
-```http
-POST /workflow-schedule/:id/trigger
-
-{
-  "metadata": {
-    "triggered_reason": "Testing"
-  }
-}
-```
-
-### 9. Get Execution Logs
-```http
-POST /workflow-schedule/execution-logs
-
-{
-  "schedule_id": 1,
-  "execution_status": "COMPLETED",
-  "page": 1,
-  "size": 10
-}
-```
-
-### 10. Get Execution Statistics
-```http
-GET /workflow-schedule/:id/stats
-```
-
-## Service Methods
-
-The `WorkflowScheduleService` provides 9 core methods:
-
-1. **createSchedule** - Create a new scheduled workflow
-2. **updateSchedule** - Update an existing schedule
-3. **getScheduleById** - Get schedule by ID
-4. **getAllSchedules** - Get all schedules with pagination
-5. **pauseSchedule** - Pause a running schedule
-6. **resumeSchedule** - Resume a paused schedule
-7. **deleteSchedule** - Delete (soft delete) a schedule
-8. **triggerManualExecution** - Manually trigger execution
-9. **getExecutionLogs** - Query execution logs with filters
-
-## Cron Expression Examples
-
-```javascript
-// Every day at 9 AM
-"0 9 * * *"
-
-// Every weekday at 9 AM
-"0 9 * * 1-5"
-
-// Every hour
-"0 * * * *"
-
-// Every 30 minutes
-"*/30 * * * *"
-
-// First day of every month at midnight
-"0 0 1 * *"
-
-// Every Monday at 8 AM
-"0 8 * * 1"
-```
-
-## Action Types
-
-### 1. Send Email
-```json
-{
-  "actionType": "SEND_EMAIL",
-  "targetEntityType": "USR",
-  "actionConfig": {
-    "subject": "Weekly Report",
-    "template": "weekly-report-template",
-    "from": "noreply@example.com"
-  },
-  "filterCriteria": {
-    "status": "ACTIVE"
-  }
-}
-```
-
-### 2. Update Records
-```json
-{
-  "actionType": "UPDATE_RECORDS",
-  "targetEntityType": "LEAD",
-  "actionConfig": {
-    "updateFields": {
-      "status": "FOLLOW_UP",
-      "last_contacted": "NOW()"
-    }
-  },
-  "filterCriteria": {
-    "status": "NEW"
-  }
-}
-```
-
-### 3. Create Task
-```json
-{
-  "actionType": "CREATE_TASK",
-  "targetEntityType": "LEAD",
-  "actionConfig": {
-    "taskName": "Follow up with lead",
-    "taskDescription": "Scheduled follow-up task"
-  },
-  "filterCriteria": {
-    "status": "QUALIFIED"
-  }
-}
-```
-
-### 4. Send Notification
-```json
-{
-  "actionType": "SEND_NOTIFICATION",
-  "targetEntityType": "USR",
-  "actionConfig": {
-    "eventType": "WORKFLOW_SCHEDULED",
-    "message": "Your weekly report is ready"
-  }
-}
-```
-
-## Retry Configuration
-
-```json
-{
-  "retry_config": {
-    "maxRetries": 3,
-    "retryDelay": 5000,
-    "backoffMultiplier": 2
-  }
-}
-```
-
-This will retry failed jobs:
-- 1st retry: after 5 seconds
-- 2nd retry: after 10 seconds (5 * 2)
-- 3rd retry: after 20 seconds (10 * 2)
-
-## Monitoring
-
-### Check Queue Status
-
-Bull provides a web UI for monitoring queues:
-
-```bash
-npm install bull-board
-```
-
-### View Execution Logs
-
-Use the execution logs API to monitor:
-- Success rate
-- Failed executions
-- Average execution time
-- Records processed
-
-```http
-GET /workflow-schedule/:id/stats
-```
-
-## Database Schema
-
-### cr_wf_scheduled_workflow
-- Stores schedule configuration
-- Includes cron expression, timezone, retry config
-- Tracks execution count and next execution time
-
-### cr_wf_execution_log
-- Logs every execution
-- Tracks success/failure status
-- Records processing metrics
-- Stores error details
-
-## Environment Variables
-
-```env
-# Redis Configuration
-REDIS_HOST=localhost
-REDIS_PORT=6379
-REDIS_PASSWORD=
-REDIS_DB=0
-
-# Database Configuration
-DB_HOST=localhost
-DB_PORT=3306
-DB_USER=root
-DB_PASS=password
-DB_NAME=core
-```
-
-## Testing
-
-### Manual Testing
-
-1. Create a schedule with a simple cron (e.g., every minute: `* * * * *`)
-2. Check execution logs after 1-2 minutes
-3. Test pause/resume functionality
-4. Test manual trigger
-5. Verify retry mechanism by creating failing actions
-
-### Unit Testing
-
-```typescript
-import { Test } from '@nestjs/testing';
-import { WorkflowScheduleService } from './workflow-schedule.service';
-
-describe('WorkflowScheduleService', () => {
-  let service: WorkflowScheduleService;
-
-  beforeEach(async () => {
-    const module = await Test.createTestingModule({
-      providers: [WorkflowScheduleService],
-    }).compile();
-
-    service = module.get<WorkflowScheduleService>(WorkflowScheduleService);
-  });
-
-  it('should create a schedule', async () => {
-    // Test implementation
-  });
-});
-```
-
-## Troubleshooting
-
-### Redis Connection Issues
-- Verify Redis is running: `redis-cli ping`
-- Check Redis connection settings in `.env`
-
-### Jobs Not Processing
-- Check Bull queue status
-- Verify cron expression is valid
-- Check schedule is enabled and active
-
-### Migration Issues
-- Ensure database user has CREATE/ALTER permissions
-- Run migrations in order (1, 2, 3)
-
-## Performance Considerations
-
-- **Batch Size**: Default is 100 records. Adjust based on your use case.
-- **Retry Strategy**: Balance between retry attempts and system load.
-- **Redis Memory**: Monitor Redis memory usage for large queues.
-- **Database Indexes**: Migrations include optimized indexes.
-
-## Security
-
-- All endpoints are protected with `JwtAuthGuard`
-- Organization-level data isolation
-- Soft delete for schedules (audit trail)
-- Execution logs retention (configurable)
-
-## Support
-
-For issues or questions:
-1. Check the logs: `src/module/workflow-schedule/`
-2. Review Bull queue status
-3. Check Redis connection
-4. Verify database migrations ran successfully
+# Workflow Schedule Module
+
+A comprehensive NestJS module for scheduling and executing workflows using Bull queues and Redis.
+
+## Features
+
+- ✅ Schedule workflows with cron expressions
+- ✅ Timezone support (IANA timezone format)
+- ✅ Pause/resume schedules
+- ✅ Manual trigger execution
+- ✅ Comprehensive execution logging
+- ✅ Retry with exponential backoff
+- ✅ Batch processing (100 records per batch)
+- ✅ Multiple action types:
+  - Send Email
+  - Update Records
+  - Create Tasks
+  - Send Notifications
+
+## Installation
+
+### 1. Install Required Dependencies
+
+```bash
+npm install @nestjs/bull bull redis cron-parser
+npm install --save-dev @types/bull
+```
+
+Note: `cron-parser` includes its own TypeScript definitions, so no separate `@types` package is needed.
+
+### 2. Configure Redis
+
+Add Redis configuration to your `.env` file:
+
+```env
+REDIS_HOST=localhost
+REDIS_PORT=6379
+REDIS_PASSWORD=
+REDIS_DB=0
+```
+
+### 3. Run Migrations
+
+The module includes three migrations:
+
+1. **AddWorkflowTypeColumn** - Adds `workflow_type` to `cr_wf_master` table
+2. **CreateScheduledWorkflowsTable** - Creates `cr_wf_scheduled_workflow` table
+3. **CreateWorkflowExecutionLogsTable** - Creates `cr_wf_execution_log` table
+
+To run migrations:
+
+```bash
+# Using TypeORM CLI
+npx typeorm migration:run -d src/config/database.config.ts
+
+# Or using npm script (if configured)
+npm run migration:run
+```
+
+### 4. Start Redis Server
+
+Make sure Redis is running:
+
+```bash
+# If using Docker
+docker run -d -p 6379:6379 redis
+
+# If installed locally
+redis-server
+```
+
+## Module Structure
+
+```
+src/module/workflow-schedule/
+├── constants/
+│   └── schedule.constants.ts          # Constants and enums
+├── interfaces/
+│   └── schedule-job-data.interface.ts # TypeScript interfaces
+├── entities/
+│   ├── scheduled-workflow.entity.ts   # ScheduledWorkflow entity
+│   └── workflow-execution-log.entity.ts # WorkflowExecutionLog entity
+├── dto/
+│   ├── create-schedule.dto.ts         # Create schedule DTO
+│   ├── update-schedule.dto.ts         # Update schedule DTO
+│   └── get-execution-logs.dto.ts      # Query execution logs DTO
+├── processors/
+│   └── schedule.processor.ts          # Bull job processor
+├── service/
+│   └── workflow-schedule.service.ts   # Business logic (9 methods)
+├── controller/
+│   └── workflow-schedule.controller.ts # REST API (9 endpoints)
+└── workflow-schedule.module.ts        # Module definition
+```
+
+## API Endpoints
+
+### 1. Create Schedule
+```http
+POST /workflow-schedule/create
+Authorization: Bearer <token>
+
+{
+  "workflow_id": 1,
+  "workflow_name": "Daily Lead Assignment",
+  "name": "Assign leads every morning",
+  "description": "Automatically assign new leads to sales team",
+  "cron_expression": "0 9 * * 1-5",
+  "timezone": "Asia/Kolkata",
+  "actions": [
+    {
+      "actionType": "UPDATE_RECORDS",
+      "targetEntityType": "LEAD",
+      "actionConfig": {
+        "updateFields": {
+          "status": "ASSIGNED"
+        }
+      }
+    }
+  ]
+}
+```
+
+### 2. Update Schedule
+```http
+PUT /workflow-schedule/update
+
+{
+  "id": 1,
+  "cron_expression": "0 10 * * 1-5"
+}
+```
+
+### 3. Get Schedule by ID
+```http
+GET /workflow-schedule/:id
+```
+
+### 4. Get All Schedules
+```http
+GET /workflow-schedule/list?page=1&size=10&schedule_status=ACTIVE
+```
+
+### 5. Pause Schedule
+```http
+POST /workflow-schedule/:id/pause
+```
+
+### 6. Resume Schedule
+```http
+POST /workflow-schedule/:id/resume
+```
+
+### 7. Delete Schedule
+```http
+DELETE /workflow-schedule/:id
+```
+
+### 8. Manual Trigger
+```http
+POST /workflow-schedule/:id/trigger
+
+{
+  "metadata": {
+    "triggered_reason": "Testing"
+  }
+}
+```
+
+### 9. Get Execution Logs
+```http
+POST /workflow-schedule/execution-logs
+
+{
+  "schedule_id": 1,
+  "execution_status": "COMPLETED",
+  "page": 1,
+  "size": 10
+}
+```
+
+### 10. Get Execution Statistics
+```http
+GET /workflow-schedule/:id/stats
+```
+
+## Service Methods
+
+The `WorkflowScheduleService` provides 9 core methods:
+
+1. **createSchedule** - Create a new scheduled workflow
+2. **updateSchedule** - Update an existing schedule
+3. **getScheduleById** - Get schedule by ID
+4. **getAllSchedules** - Get all schedules with pagination
+5. **pauseSchedule** - Pause a running schedule
+6. **resumeSchedule** - Resume a paused schedule
+7. **deleteSchedule** - Delete (soft delete) a schedule
+8. **triggerManualExecution** - Manually trigger execution
+9. **getExecutionLogs** - Query execution logs with filters
+
+## Cron Expression Examples
+
+```javascript
+// Every day at 9 AM
+"0 9 * * *"
+
+// Every weekday at 9 AM
+"0 9 * * 1-5"
+
+// Every hour
+"0 * * * *"
+
+// Every 30 minutes
+"*/30 * * * *"
+
+// First day of every month at midnight
+"0 0 1 * *"
+
+// Every Monday at 8 AM
+"0 8 * * 1"
+```
+
+## Action Types
+
+### 1. Send Email
+```json
+{
+  "actionType": "SEND_EMAIL",
+  "targetEntityType": "USR",
+  "actionConfig": {
+    "subject": "Weekly Report",
+    "template": "weekly-report-template",
+    "from": "noreply@example.com"
+  },
+  "filterCriteria": {
+    "status": "ACTIVE"
+  }
+}
+```
+
+### 2. Update Records
+```json
+{
+  "actionType": "UPDATE_RECORDS",
+  "targetEntityType": "LEAD",
+  "actionConfig": {
+    "updateFields": {
+      "status": "FOLLOW_UP",
+      "last_contacted": "NOW()"
+    }
+  },
+  "filterCriteria": {
+    "status": "NEW"
+  }
+}
+```
+
+### 3. Create Task
+```json
+{
+  "actionType": "CREATE_TASK",
+  "targetEntityType": "LEAD",
+  "actionConfig": {
+    "taskName": "Follow up with lead",
+    "taskDescription": "Scheduled follow-up task"
+  },
+  "filterCriteria": {
+    "status": "QUALIFIED"
+  }
+}
+```
+
+### 4. Send Notification
+```json
+{
+  "actionType": "SEND_NOTIFICATION",
+  "targetEntityType": "USR",
+  "actionConfig": {
+    "eventType": "WORKFLOW_SCHEDULED",
+    "message": "Your weekly report is ready"
+  }
+}
+```
+
+## Retry Configuration
+
+```json
+{
+  "retry_config": {
+    "maxRetries": 3,
+    "retryDelay": 5000,
+    "backoffMultiplier": 2
+  }
+}
+```
+
+This will retry failed jobs:
+- 1st retry: after 5 seconds
+- 2nd retry: after 10 seconds (5 * 2)
+- 3rd retry: after 20 seconds (10 * 2)
+
+## Monitoring
+
+### Check Queue Status
+
+Bull provides a web UI for monitoring queues:
+
+```bash
+npm install bull-board
+```
+
+### View Execution Logs
+
+Use the execution logs API to monitor:
+- Success rate
+- Failed executions
+- Average execution time
+- Records processed
+
+```http
+GET /workflow-schedule/:id/stats
+```
+
+## Database Schema
+
+### cr_wf_scheduled_workflow
+- Stores schedule configuration
+- Includes cron expression, timezone, retry config
+- Tracks execution count and next execution time
+
+### cr_wf_execution_log
+- Logs every execution
+- Tracks success/failure status
+- Records processing metrics
+- Stores error details
+
+## Environment Variables
+
+```env
+# Redis Configuration
+REDIS_HOST=localhost
+REDIS_PORT=6379
+REDIS_PASSWORD=
+REDIS_DB=0
+
+# Database Configuration
+DB_HOST=localhost
+DB_PORT=3306
+DB_USER=root
+DB_PASS=password
+DB_NAME=core
+```
+
+## Testing
+
+### Manual Testing
+
+1. Create a schedule with a simple cron (e.g., every minute: `* * * * *`)
+2. Check execution logs after 1-2 minutes
+3. Test pause/resume functionality
+4. Test manual trigger
+5. Verify retry mechanism by creating failing actions
+
+### Unit Testing
+
+```typescript
+import { Test } from '@nestjs/testing';
+import { WorkflowScheduleService } from './workflow-schedule.service';
+
+describe('WorkflowScheduleService', () => {
+  let service: WorkflowScheduleService;
+
+  beforeEach(async () => {
+    const module = await Test.createTestingModule({
+      providers: [WorkflowScheduleService],
+    }).compile();
+
+    service = module.get<WorkflowScheduleService>(WorkflowScheduleService);
+  });
+
+  it('should create a schedule', async () => {
+    // Test implementation
+  });
+});
+```
+
+## Troubleshooting
+
+### Redis Connection Issues
+- Verify Redis is running: `redis-cli ping`
+- Check Redis connection settings in `.env`
+
+### Jobs Not Processing
+- Check Bull queue status
+- Verify cron expression is valid
+- Check schedule is enabled and active
+
+### Migration Issues
+- Ensure database user has CREATE/ALTER permissions
+- Run migrations in order (1, 2, 3)
+
+## Performance Considerations
+
+- **Batch Size**: Default is 100 records. Adjust based on your use case.
+- **Retry Strategy**: Balance between retry attempts and system load.
+- **Redis Memory**: Monitor Redis memory usage for large queues.
+- **Database Indexes**: Migrations include optimized indexes.
+
+## Security
+
+- All endpoints are protected with `JwtAuthGuard`
+- Organization-level data isolation
+- Soft delete for schedules (audit trail)
+- Execution logs retention (configurable)
+
+## Support
+
+For issues or questions:
+1. Check the logs: `src/module/workflow-schedule/`
+2. Review Bull queue status
+3. Check Redis connection
+4. Verify database migrations ran successfully