fiberx-backend-toolkit 0.0.1

package/README.md

@@ -0,0 +1,215 @@
+# FiberX Backend Toolkit
+
+A comprehensive TypeScript backend toolkit providing shared domain logic, infrastructure helpers, and utilities for FiberX server-side applications and services.
+
+## Overview
+
+The FiberX Backend Toolkit is a centralized library designed to streamline backend development across FiberX services. It provides database abstraction layers, migration management, utilities for common operations, and TypeScript type definitions for consistency across the ecosystem.
+
+## Features
+
+- **Database Management**: Multi-connector support with Sequelize integration for ORM operations
+- **Schema Migration System**: Automated schema management, migration generation, and execution
+- **Seeding Support**: Database seeder generation and execution tools
+- **Utility Functions**: 
+  - Input validation and transformation
+  - Environment variable management
+  - SQL formatting and query building
+  - Structured logging with file persistence
+- **TypeScript Support**: Fully typed exports for schema, migration, and utility types
+- **Flexible Exports**: Modular exports for `utils`, `types`, and `database` modules
+
+## Installation
+
+```bash
+npm install fiberx-backend-toolkit
+```
+
+## Requirements
+
+- Node.js 20.x or higher
+- npm or yarn package manager
+
+## Quick Start
+
+### Import Types
+
+```typescript
+import type { SchemaInterface, MigrationInterface } from "fiberx-backend-toolkit/types";
+```
+
+### Import Utilities
+
+```typescript
+import { LoggerUtil, InputValidatorUtil, EnvManagerUtil } from "fiberx-backend-toolkit/utils";
+
+// Initialize logger
+const logger = new LoggerUtil("my-module");
+logger.info("Application started");
+
+// Use environment manager
+const apiKey = EnvManagerUtil.get("API_KEY");
+```
+
+### Database Operations
+
+```typescript
+import { SequelizeConnector, MigrationRunnerScript } from "fiberx-backend-toolkit/database";
+
+// Initialize database connector
+const connector = new SequelizeConnector(config);
+
+// Run migrations
+const migrationRunner = new MigrationRunnerScript(connector);
+await migrationRunner.run();
+```
+
+## Core Modules
+
+### Utilities (`/src/utils`)
+
+- **LoggerUtil**: Structured logging with file-based persistence
+- **InputValidatorUtil**: Input validation and sanitization
+- **InputTransformerUtil**: Data transformation utilities
+- **EnvManagerUtil**: Environment variable management and parsing
+- **SqlFormatterUtil**: SQL query formatting and beautification
+
+### Database (`/src/database`)
+
+#### Connectors
+- **BaseConnector**: Abstract base class for database connections
+- **SequelizeConnector**: Sequelize ORM connector implementation
+
+#### Schema Utilities
+- **SchemaDiffUtil**: Compare database schemas and detect changes
+- **SchemaNormalizerUtil**: Normalize schema definitions for consistency
+
+#### Scripts
+- **CreateSchemaScript**: Generate initial database schema
+- **MakeMigrationsScript**: Auto-generate migration files from schema changes
+- **MigrationRunnerScript**: Execute pending database migrations
+- **CreateSeederScript**: Generate seeder files for initial data
+- **SeederRunnerScript**: Execute database seeders
+
+### Types (`/src/types`)
+
+- **SchemaInterface**: Database schema type definitions
+- **MigrationInterface**: Migration type definitions
+- **UtilInterface**: Utility function types
+
+## Module Exports
+
+The toolkit provides structured exports for easy integration:
+
+```typescript
+// Main exports (all modules)
+import * as toolkit from "fiberx-backend-toolkit";
+
+// Utils only
+import { LoggerUtil, InputValidatorUtil } from "fiberx-backend-toolkit/utils";
+
+// Database only
+import { SequelizeConnector, MigrationRunnerScript } from "fiberx-backend-toolkit/database";
+
+// Types only
+import type { SchemaInterface } from "fiberx-backend-toolkit/types";
+```
+
+## Configuration
+
+The toolkit uses standardized directory conventions:
+
+```
+project_root/
+├── logs/                              # Log files directory
+├── environment_variables/             # Environment variable files
+├── src/database/
+│   ├── schemas/                      # Schema definitions
+│   ├── schema_snapshots/             # Schema snapshots for comparison
+│   ├── models/                       # Sequelize models
+│   ├── migrations/                   # Database migrations
+│   └── seeders/                      # Database seeders
+```
+
+## Dependencies
+
+### Core Dependencies
+- **sequelize**: ^6.37.7 - SQL ORM
+- **sequelize-typescript**: ^2.1.6 - TypeScript decorators for Sequelize
+- **axios**: ^1.11.0 - HTTP client
+- **bcrypt**: ^6.0.0 - Password hashing
+- **jsonwebtoken**: ^9.0.2 - JWT handling
+- **dayjs**: ^1.11.18 - Date/time utilities
+- **js-yaml**: ^4.1.0 - YAML parsing
+- **uuid**: ^12.0.0 - UUID generation
+
+## Build
+
+```bash
+npm run build
+```
+
+The build process uses `tsup` for optimized TypeScript bundling.
+
+## API Documentation
+
+### LoggerUtil
+
+```typescript
+const logger = new LoggerUtil("module-name");
+
+logger.info("Information message");
+logger.warn("Warning message");
+logger.error("Error message");
+logger.debug("Debug message");
+```
+
+Logs are stored locally in the configured `LOG_DIR`.
+
+### InputValidatorUtil
+
+Validates and sanitizes user inputs with built-in rules and custom validators.
+
+### EnvManagerUtil
+
+```typescript
+const dbHost = EnvManagerUtil.get("DB_HOST", "localhost");
+const dbPort = EnvManagerUtil.getAsNumber("DB_PORT", 5432);
+```
+
+### SequelizeConnector
+
+Provides abstraction over Sequelize for database operations with connection pooling and query management.
+
+## Project Structure
+
+```
+fiberx-backend-toolkit/
+├── src/
+│   ├── code_templates/         # Code generation templates
+│   ├── config/                 # Configuration constants
+│   ├── database/               # Database module
+│   ├── types/                  # TypeScript type definitions
+│   ├── utils/                  # Utility functions
+│   └── index.ts                # Main entry point
+├── package.json
+├── tsconfig.json
+├── tsup.config.ts              # Build configuration
+└── README.md
+```
+
+## License
+
+ISC
+
+## Author
+
+David Matt-Ojo
+
+## Repository
+
+[GitHub - fiberx-backend-toolkit](https://github.com/fiberx-innovations/fiberx-backend-toolkit)
+
+## Support
+
+For issues, bugs, or feature requests, please visit the [issues page](https://github.com/fiberx-innovations/fiberx-backend-toolkit/issues).

package/package.json

@@ -0,0 +1,52 @@
+{
+    "name": "fiberx-backend-toolkit",
+    "version": "0.0.1",
+    "description": "A TypeScript backend toolkit providing shared domain logic, infrastructure helpers, and utilities for FiberX server-side applications and services.",
+    "type": "module",
+    "main": "./dist/index.js",
+    "types": "./dist/index.d.ts",
+    "exports": {
+        ".": "./dist/index.js",
+        "./utils": "./dist/utils/index.js",
+        "./types": "./dist/types/index.js",
+        "./database": "./dist/database/index.js"
+    },
+    "scripts": {
+        "build": "tsup"
+    },
+    "homepage": "https://github.com/fiberx-innovations/fiberx-backend-toolkit#readme",
+    "bugs": {
+        "url": "https://github.com/fiberx-innovations/fiberx-backend-toolkit/issues"
+    },
+    "repository": {
+        "type": "git",
+        "url": "git+https://github.com/fiberx-innovations/fiberx-backend-toolkit.git"
+    },
+    "license": "ISC",
+    "author": "David Matt-Ojo",
+    "dependencies": {
+        "axios": "^1.11.0",
+        "bcrypt": "^6.0.0",
+        "dayjs": "^1.11.18",
+        "js-yaml": "^4.1.0",
+        "jsonwebtoken": "^9.0.2",
+        "sequelize": "^6.37.7",
+        "sequelize-typescript": "^2.1.6",
+        "uuid": "^12.0.0"
+    },
+    "devDependencies": {
+        "@types/axios": "^0.9.36",
+        "@types/bcrypt": "^6.0.0",
+        "@types/js-yaml": "^4.0.9",
+        "@types/jsonwebtoken": "^9.0.10",
+        "@types/uuid": "^10.0.0",
+        "ts-node": "^10.9.2",
+        "tsc-alias": "^1.8.16",
+        "tsconfig-paths": "^4.2.0",
+        "tsup": "^8.5.1",
+        "typescript": "^5.8.3"
+    },
+    "engines": {
+        "node": "20.x"
+    }
+}

package/src/code_templates/sequelize_code_template.ts

@@ -0,0 +1,362 @@
+import { 
+    IndexFieldOptionsInterface,
+    SchemaColumnInterface, 
+    SchemaDiffInterface 
+} from "@/types/schema_type";
+
+const SEQUELIZE_SCHEMA_CODE_TEMPLATE = (
+    schema_name: string, 
+    table_name: string, 
+    model_name: string, 
+    migration_priority: number
+): string => 
+{
+    return `
+import { DataTypes } from "sequelize";
+import { SchemaDefinitionInterface } from "fiberx-backend-toolkit/types/schema_type";
+
+const ${schema_name}: SchemaDefinitionInterface = {
+    database_name: "public",
+    migration_priority: ${migration_priority},
+
+    table_name: "${table_name}",
+    model_name: "${model_name}",
+    timestamps:true,
+
+    columns: {
+        id: {
+            type: DataTypes.BIGINT,
+            primaryKey: true,
+            autoIncrement: true,
+            allowNull: false,
+        },
+
+        created_at: {
+            type: DataTypes.DATE,
+            defaultValue: DataTypes.NOW,
+        },
+
+        updated_at: {
+            type: DataTypes.DATE,
+            allowNull: true,
+            defaultValue: null,
+        },
+    },
+
+    indexes: [
+        { fields: [""], name: "" },
+    ],
+};
+
+export default ${schema_name};
+`
+}
+
+const SEQUELIZE_CREATE_NEW_SCHEMA_MIGRATION_FILE_CODE_TEMPLATE = ( 
+    schema_table_name: string, 
+    schema_model_name: string,
+    schema_file_name: string 
+): string => `
+import { QueryInterface, DataTypes, Sequelize } from "sequelize";
+import { LoggerUtil } from "fiberx-backend-toolkit//utils";
+import { SchemaDefinitionInterface } from "fiberx-backend-toolkit/types/schema_type";
+import ${schema_model_name}Schema from "@/database/schemas/${schema_file_name.replace(".ts", "")}";
+
+class Create${schema_model_name}TableMigration {
+    private name: string = "create_table_${schema_table_name}_migration_file";
+    private schema: SchemaDefinitionInterface = ${schema_model_name}Schema;
+    private readonly logger: LoggerUtil = new LoggerUtil(this.name);
+
+    constructor() {}
+
+    // Migration UP
+    public async up(
+        queryInterface: QueryInterface, 
+        Sequelize: typeof import("sequelize")
+    ) {
+        try {
+            const { table_name, columns = {}, indexes = [] } = this.schema;
+
+            await queryInterface.createTable(table_name, columns);
+
+            for (const index_field_obj of indexes) {
+
+                const { fields, name, index_options } = index_field_obj;
+                const index_name = \`idx_\${table_name}_\${fields.join("_")}\`
+            
+                await queryInterface.addIndex(
+                    table_name, 
+                    fields, 
+                    { name: index_name, ...index_options }
+                );
+            }
+
+            this.logger.success(\`✅ Table "\${table_name}" for schema "\${model_name}Schema" created with "\${this.index_attributes.length}" indexed Fields.\`);
+        } 
+        catch (error: any) {
+            this.logger.error(\`🚫 Migration failed: "\${error.message}"\`, { error });
+            throw new Error(\`🚫 Migration failed: "\${error.message}"\`);
+        }
+    }
+
+    // Migration DOWN
+    public async down(
+        queryInterface: QueryInterface, 
+        Sequelize: typeof import("sequelize")
+    ) {
+        try {
+            const { table_name, database_name, model_name } = this.schema;
+            await queryInterface.dropTable(table_name, {});
+            this.logger.success(\`🗑️ Table "\${table_name}" dropped successfully.\`);
+        } catch (error: any) {
+            this.logger.error(\`🚫 Migration rollback failed: "\${error.message}"\`, { error });
+            throw new Error(\`🚫 Migration rollback failed: "\${error.message}"\`);
+        }
+    }
+}
+
+export default Create${schema_model_name}TableMigration;
+`;
+
+const SEQUELIZE_UPDATE_EXISTING_SCHEMA_MIGRATION_FILE_CODE_TEMPLATE = (
+    schema_table_name: string,
+    schema_model_name: string,
+    diff: SchemaDiffInterface,
+    schema_file_name: string
+): string => `
+import { QueryInterface, DataTypes, Sequelize } from "sequelize";
+import { LoggerUtil } from "fiberx-backend-toolkit//utils";
+import { SchemaDefinitionInterface } from "fiberx-backend-toolkit/types/schema_type";
+import ${schema_model_name}Schema from "@/database/schemas/${schema_file_name.replace(".ts", "")}";
+
+class Update${schema_model_name}TableMigration {
+    private name: string = "update_table_${schema_table_name}_migration_file";
+    private schema: SchemaDefinitionInterface = ${schema_model_name}Schema;
+    private readonly logger: LoggerUtil = new LoggerUtil(this.name);
+
+    constructor() {}
+
+    public async up(
+        queryInterface: QueryInterface, 
+        Sequelize: typeof import("sequelize")
+    ) {
+        try {
+            const { table_name, model_name, columns = {}, indexes = [] } = this.schema;
+
+        ${diff.table_renamed ? `
+            await queryInterface.renameTable(
+                "${diff.table_renamed.from}",
+                "${diff.table_renamed.to}"
+            );` 
+            : 
+            ""
+        }
+
+        // Add columns
+        ${diff.columns.added.map((column_name: string) => `
+            await queryInterface.addColumn(
+                table_name,
+                "${column_name}",
+                columns["${column_name}"]
+            );
+        `).join("")}
+
+        // Modify columns
+        ${Object.entries(diff.columns.modified).map(([col_name, col_modified]) => `
+            await queryInterface.changeColumn(
+                table_name, 
+                "${col_name}", 
+                ${JSON.stringify(col_modified?.after)}
+            );
+            `
+        ).join("")}
+
+        // Remove columns
+        ${Object.entries(diff.columns.modified).map(([col_name, col_Def]) => `
+            await queryInterface.removeColumn(
+                table_name, 
+                "${col_name}"
+            );
+            `
+        ).join("")}
+
+
+        // Add indexes
+        ${diff.indexes.added.map( (idx: IndexFieldOptionsInterface) => `
+            await queryInterface.addIndex(
+                table_name, 
+                ${JSON.stringify(idx.fields)}, 
+                ${JSON.stringify(idx)}
+            );
+            `
+        ).join("")}
+
+        // Modify indexes (remove + add)
+        ${Object.entries(diff.indexes.modified).map(([idx_name, idx_modified]) => `
+            await queryInterface.removeIndex(
+                table_name, 
+                ${JSON.stringify(idx_modified.before.name)}
+            );
+
+            await queryInterface.addIndex(
+                table_name, 
+                ${JSON.stringify(idx_modified.after.fields)}, 
+                ${JSON.stringify(idx_modified.after)}
+            );
+            `
+        ).join("")}
+
+        // Remove indexes
+        ${diff.indexes.removed.map( (idx: IndexFieldOptionsInterface) => `
+            await queryInterface.removeIndex(
+                table_name, 
+                ${JSON.stringify(idx.name)}
+            );
+            `
+        ).join("")}
+
+
+            this.logger.success(\`✅ Table "\${table_name}" for schema "\${model_name}Schema" updated successfully.\`);
+        } 
+        catch (error: any) {
+            this.logger.error(\`🚫 Update migration failed: "\${error.message}"\`, { error });
+            throw new Error(\`🚫 Update migration failed: "\${error.message}"\`);
+        }
+    }
+
+    public async down(
+        queryInterface: QueryInterface, 
+        Sequelize: typeof import("sequelize")
+    ) {
+        try {
+            const { table_name, model_name, columns = {} } = this.schema;
+
+        // Revert Added indexes
+        ${diff.indexes.added.map( (idx: IndexFieldOptionsInterface) => `
+            await queryInterface.removeIndex(
+                table_name, 
+                ${JSON.stringify(idx.name)}
+            );`
+        ).join("")}
+
+        // Revert Modified Indexes
+        ${Object.entries(diff.indexes.modified).map(([idx_name, idx_modified]) => `
+            await queryInterface.removeIndex(
+                table_name, 
+                ${JSON.stringify(idx_modified.after.name)}
+            );
+
+
+            await queryInterface.addIndex(
+                table_name, 
+                ${JSON.stringify(idx_modified.before.fields)}, 
+                ${JSON.stringify(idx_modified.before)}
+            );`
+        ).join("")}
+
+        // Revert removed indexes
+        ${diff.indexes.removed.map( (idx: IndexFieldOptionsInterface) => `
+            await queryInterface.addIndex(
+                table_name, 
+                ${JSON.stringify(idx.fields)}, 
+                ${JSON.stringify(idx)}
+            );`
+        ).join("")}
+
+        // Revert Modified columns
+        ${Object.entries(diff.columns.modified).map(([col_name, col_modified]) => `
+            await queryInterface.changeColumn(
+                table_name, 
+                "${col_name}", 
+                ${JSON.stringify(col_modified.before)}
+            );`
+        ).join("")}
+
+        // Revert added columns
+        ${diff.columns.added.map((column_name: string) => `
+            await queryInterface.removeColumn(table_name, "${column_name}");
+        `).join("")}
+
+        ${Object.entries(diff.columns.modified).map(([col_name, col_Def]) => `
+            await queryInterface.addColumn(
+                table_name, 
+                "${col_name}", 
+                ${JSON.stringify(col_Def)}
+            );
+        `).join("")}
+
+        ${diff.table_renamed ? `
+            await queryInterface.renameTable(
+                "${diff.table_renamed.to}",
+                "${diff.table_renamed.from}"
+            );` : ""
+        }
+
+            this.logger.success(\`🗑️ Rollback of table "\${table_name}" completed successfully.\`);
+        } 
+        catch (error: any) {
+            this.logger.error(\`🚫 Rollback migration failed: "\${error.message}"\`, { error });
+            throw new Error(\`🚫 Rollback migration failed: "\${error.message}"\`);
+        }
+    }
+}
+
+export default Update${schema_model_name}TableMigration;
+`;
+
+const SEQUELIZE_SEEDER_TEMPLATE = (
+    class_name: string, 
+    table_name: string
+): string => `
+'use strict';
+
+import { QueryInterface } from 'sequelize';
+
+class ${class_name}Seeder {
+	/**
+	 * Seed method
+	 */
+	public async up(queryInterface: QueryInterface): Promise<void> {
+        const seed_data = this.getSeedData();
+
+		if (!seed_data || seed_data.length === 0) {
+			throw new Error("❌ Seeder data for 'roles' is empty. Aborting bulk insert.");
+		}
+
+		await queryInterface.bulkInsert('${table_name.toLowerCase()}', seed_data);
+	}
+
+	/**
+	 * Undo seed method
+	 */
+	public async down(queryInterface: QueryInterface): Promise<void> {
+        const seed_data = this.getSeedData();
+
+		if (!seed_data || seed_data.length === 0) {
+			throw new Error("❌ Seeder data for 'roles' is empty. Nothing to delete.");
+		}
+
+		const ids = seed_data.map(row => row.id);
+
+		await queryInterface.bulkDelete('${table_name.toLowerCase()}', { id: ids });
+	}
+
+	/**
+	 * Provide seed data array
+	 */
+	private getSeedData(): Array<Record<string, any>> {
+		// TODO: Replace this with real seed data
+		return [];
+	}
+}
+
+export default  ${class_name}Seeder;
+`;
+
+export {
+    SEQUELIZE_SCHEMA_CODE_TEMPLATE,
+    SEQUELIZE_SEEDER_TEMPLATE,
+    SEQUELIZE_CREATE_NEW_SCHEMA_MIGRATION_FILE_CODE_TEMPLATE,
+    SEQUELIZE_UPDATE_EXISTING_SCHEMA_MIGRATION_FILE_CODE_TEMPLATE
+}
+

package/src/config/constants.ts

@@ -0,0 +1,14 @@
+import path from "path";
+
+// Database constants
+export const BASE_DIR                           = process.cwd();
+export const LOG_DIR                            = path.join(BASE_DIR, "logs");
+export const ENV_VAR_DIR                        = path.join(BASE_DIR, "environment_varaiables");
+export const SCHEMAS_DIR                        = path.join(BASE_DIR, "src/database/schemas");
+export const SCHEMA_SNAPSHOTS_DIR               = path.join(BASE_DIR, "src/database/schema_snapshots");
+export const MODELS_DIR                         = path.join(BASE_DIR, "src/database/models");
+export const MIGRATIONS_DIR                     = path.join(BASE_DIR, "src/database/migrations");
+export const SEEDERS_DIR                        = path.join(BASE_DIR, "src/database/seeders");
+
+export const SEQUELIZE_META_TABLE_NAME          = "sequelize_database_tables_meta";
+export const SEQUELIZE_SEEDER_META_TABLE_NAME        = "sequelize_database_table_seeder_meta";

package/src/database/connectors/base_connector.ts

@@ -0,0 +1,21 @@
+
+import { LoggerUtil, EnvManagerUtil } from "@/utils";
+
+/**
+ * Base connector abstraction for all DB / external service connectors
+ */
+abstract class BaseConnector<TConnection> {
+    public readonly env_manager = EnvManagerUtil.getInstance();
+    public readonly logger: LoggerUtil;
+
+    protected constructor(
+        protected readonly connector_name: string
+    ) {
+        this.logger = new LoggerUtil(connector_name);
+    }
+
+    // Connect and return the underlying connection instance
+    public abstract connect(): TConnection;
+}
+
+export default BaseConnector;