qtools-library-dot-d 1.0.8 → 1.0.10

package/README.md

@@ -229,6 +229,8 @@ This pattern is used extensively in the qtools ecosystem for building scalable,
 
 ## HISTORY
 
+v1.10 - there was a reason I did not return the exception list. Undone.
+v1.09 - fixed broken error handling when library file path is bad; added demonstration application
 v1.08 - updated README (several times)
 v1.05 - cleaned up toString() report, returns exception list (or err if callback) if needed, added qtools-functional-library
 v1.04 - Now tolerates (and reports) directories and files that are not modules.

package/libDotD_Working_Example/README.md

@@ -0,0 +1,234 @@
+# Dynamic dotD Libraries with qtools-library-dot-d
+
+This project demonstrates the **qtools-library-dot-d** system for creating dynamic, modular libraries in Node.js. The dotD (dot-D) pattern provides a powerful way to build extensible applications through automatic module discovery and loading.
+
+## What is qtools-library-dot-d?
+
+**qtools-library-dot-d** is a Node.js library that enables dynamic module loading from directories. It automatically discovers JavaScript files in specified directories and makes them available as methods on a library object, creating a plugin-like architecture without the complexity of traditional plugin systems.
+
+## The dotD Pattern
+
+The "dotD" naming convention (short for "dot-D") refers to directories that contain dynamically loadable modules:
+
+- `modules.d/` - Directory containing module files
+- `endpoints.d/` - Directory containing API endpoint modules  
+- `accessPoints.d/` - Directory containing data access modules
+- `mappers.d/` - Directory containing data mapping modules
+
+Each `.js` file in a dotD directory becomes a method on the library object, with the filename determining the method name.
+
+## Why Use Dynamic dotD Libraries?
+
+### 1. **Automatic Module Discovery**
+
+No need to manually import or register modules. Simply add a `.js` file to the dotD directory and it's automatically available.
+
+```javascript
+// Traditional approach - manual imports
+const module1 = require('./modules/module1');
+const module2 = require('./modules/module2');
+const module3 = require('./modules/module3');
+
+// dotD approach - automatic discovery
+const library = qtLibDotD({ libraryName: 'myLibrary' });
+library.setLibraryPath('./modules.d');
+library.loadModules();
+// All modules in modules.d/ are now available as library.moduleName()
+```
+
+### 2. **Plugin Architecture**
+
+Create extensible applications where new functionality can be added by simply dropping files into directories.
+
+```javascript
+// Add new functionality by creating a new file
+// modules.d/newFeature.js -> library.newFeature()
+```
+
+### 3. **Shared Resources**
+
+Pass common resources (configuration, database connections, utilities) to all modules through `passThroughParameters`.
+
+```javascript
+const passThroughParameters = {
+    config: getConfig(),
+    database: dbConnection,
+    logger: xLog,
+    utilities: commonUtils
+};
+library.loadModules({ passThroughParameters });
+```
+
+### 4. **Clean Separation of Concerns**
+
+Each module is self-contained and focused on a single responsibility, making code easier to maintain and test.
+
+### 5. **Runtime Flexibility**
+
+Modules can be loaded synchronously or asynchronously, and the library can be sealed to prevent modifications after loading.
+
+## Real-World Use Cases
+
+### API Endpoint Management
+
+```javascript
+// endpoints.d/userEndpoints.js
+// endpoints.d/productEndpoints.js  
+// endpoints.d/orderEndpoints.js
+// Each file defines related API endpoints
+```
+
+### Data Access Layer
+
+```javascript
+// accessPoints.d/getUserData.js
+// accessPoints.d/saveUserData.js
+// accessPoints.d/deleteUserData.js
+// Each file handles specific data operations
+```
+
+### Business Logic Modules
+
+```javascript
+// processors.d/validateOrder.js
+// processors.d/calculateTax.js
+// processors.d/sendNotification.js
+// Each file contains specific business logic
+```
+
+### CLI Command Tools
+
+```javascript
+// commands.d/deploy.js
+// commands.d/backup.js
+// commands.d/migrate.js
+// Each file implements a CLI command
+```
+
+## How It Works
+
+### 1. **Library Creation**
+
+```javascript
+const qtLibDotD = require('qtools-library-dot-d');
+const library = qtLibDotD({ libraryName: 'myLibrary' });
+```
+
+### 2. **Module Directory Setup**
+
+```javascript
+library.setLibraryPath('./modules.d');
+```
+
+### 3. **Shared Resource Definition**
+
+```javascript
+const passThroughParameters = {
+    config: appConfig,
+    database: dbConnection,
+    logger: logger
+};
+```
+
+### 4. **Module Loading**
+
+```javascript
+// Synchronous
+library.loadModules({ passThroughParameters });
+
+// Asynchronous
+library.loadModules({ passThroughParameters }, callback);
+```
+
+### 5. **Library Sealing**
+
+```javascript
+library.seal(); // Prevents further modifications
+```
+
+### 6. **Module Usage**
+
+```javascript
+// If you have modules.d/processOrder.js
+library.processOrder(orderData);
+
+// If you have modules.d/sendEmail.js  
+library.sendEmail(recipient, message);
+```
+
+## Module Structure
+
+Every dotD module follows this pattern:
+
+```javascript
+const moduleName = __filename.replace(__dirname + '/', '').replace(/.js$/, '');
+
+const moduleFunction = ({ moduleName } = {}) => ({ dotD, passThroughParameters } = {}) => {
+
+    const workingFunction = (/* your parameters */) => {
+        // Access shared resources
+        const config = passThroughParameters.config;
+        const logger = passThroughParameters.logger;
+
+        // Your module logic here
+
+        return result;
+    };
+
+    // Register the function
+    dotD.library.add(moduleName, workingFunction);
+};
+
+module.exports = moduleFunction({ moduleName });
+```
+
+## Benefits Over Traditional Approaches
+
+| Traditional           | dotD Libraries              |
+| --------------------- | --------------------------- |
+| Manual imports        | Automatic discovery         |
+| Tight coupling        | Loose coupling              |
+| Hard to extend        | Easy to extend              |
+| Configuration in code | Configuration through files |
+| Monolithic            | Modular                     |
+| Static structure      | Dynamic structure           |
+
+## Getting Started
+
+1. **Install the dependency:**
+   
+   ```bash
+   npm install qtools-library-dot-d
+   ```
+
+2. **Run the example:**
+   
+   ```bash
+   npm start
+   ```
+
+3. **Create your own modules:**
+   
+   - Add `.js` files to `demoModules_DotD/`
+   - Follow the module pattern shown in the examples
+   - Access shared resources via `passThroughParameters`
+
+## Advanced Features
+
+- **Async module loading** for modules that need initialization
+- **Error handling** during module loading
+- **Library introspection** with `toString()` and `qtdProcessLog()`
+- **Multiple libraries** in the same application
+- **Nested module structures** for complex applications
+
+## Integration with Larger Systems
+
+The dotD pattern works particularly well in:
+
+- **Web frameworks** (Express, Koa) for route organization
+- **API systems** for endpoint management
+- **CLI tools** for command organization  
+- **Microservices** for feature modularity
+- **Plugin systems** for extensible applications
+
+This pattern is used extensively in the qtools ecosystem for building scalable, maintainable Node.js applications.

package/libDotD_Working_Example/demoModules_DotD/demo1.js

@@ -0,0 +1,46 @@
+#!/usr/bin/env node
+'use strict';
+// Suppress punycode deprecation warning
+process.noDeprecation = true;
+
+const moduleName = __filename.replace(__dirname + '/', '').replace(/.js$/, ''); //this just seems to come in handy a lot
+
+//START OF moduleFunction() ============================================================
+// DOTD MODULE STRUCTURE GUIDE
+// ============================
+// Every dotD module follows this exact pattern:
+// 1. Export a function that takes { moduleName } and returns another function
+// 2. The returned function receives { dotD, passThroughParameters }
+// 3. Define your working function that implements the module's functionality
+// 4. Add the working function to the library using dotD.library.add()
+// 5. The module name (from filename) becomes the method name on the library
+
+const moduleFunction = ({ moduleName } = {}) => ({ dotD, passThroughParameters } = {}) => {
+
+	// STEP 1: Define your module's main functionality
+	// This function will be called when someone uses libraryInstance.demo1()
+	const workingFunction = () => {
+		// ACCESS SHARED DATA: Use passThroughParameters for shared resources
+		console.log(`${passThroughParameters.message} [${moduleName}]`);
+		
+		// RETURN VALUES: Your function can return any value
+		return 'hello';
+		
+		// COMMON PATTERNS:
+		// - Access shared config: passThroughParameters.config
+		// - Use shared logger: passThroughParameters.xLog('message')
+		// - Access database: passThroughParameters.database.query(...)
+		// - Call shared utilities: passThroughParameters.utilityFunction()
+	};
+	
+	// STEP 2: Register the function with the library
+	// The moduleName (from filename) becomes the method name
+	// Example: demo1.js creates libraryInstance.demo1()
+	dotD.library.add(moduleName, workingFunction);
+};
+
+//END OF moduleFunction() ============================================================
+
+// STEP 3: Export the configured module function
+// This runs immediately when the module is loaded
+module.exports = moduleFunction({ moduleName }); //runs it right now

package/libDotD_Working_Example/demoModules_DotD/demo2.js

@@ -0,0 +1,48 @@
+#!/usr/bin/env node
+'use strict';
+// Suppress punycode deprecation warning
+process.noDeprecation = true;
+
+const moduleName = __filename.replace(__dirname + '/', '').replace(/.js$/, ''); //this just seems to come in handy a lot
+
+//START OF moduleFunction() ============================================================
+// DOTD MODULE WITH PARAMETERS AND SHARED FUNCTIONS
+// =================================================
+// This example shows how to:
+// 1. Accept parameters in your module functions
+// 2. Use shared functions from passThroughParameters
+// 3. Combine local logic with shared utilities
+
+const moduleFunction = ({ moduleName } = {}) => ({ dotD, passThroughParameters } = {}) => {
+
+	// STEP 1: Define a function that accepts parameters
+	// This demonstrates how module functions can receive arguments
+	const workingFunction = (arg1, arg2) => {
+		// USING SHARED FUNCTIONS: Call functions from passThroughParameters
+		// In this example, 'add' was passed in from the main library setup
+		const result = passThroughParameters.add(arg1, arg2);
+		
+		console.log(`Pass Through Function: ${arg1}+${arg2}=${result} [${moduleName}]`);
+		
+		// COMMON PATTERNS WITH PARAMETERS:
+		// - Validation: if (!arg1 || !arg2) throw new Error('Missing arguments');
+		// - Default values: arg1 = arg1 || defaultValue;
+		// - Type checking: if (typeof arg1 !== 'number') return null;
+		// - Using shared config: const timeout = passThroughParameters.config.timeout;
+		// - Logging with shared logger: passThroughParameters.xLog(`Processing ${arg1}, ${arg2}`);
+		
+		return result; // Return meaningful data
+	};
+	
+	// STEP 2: Register the function (same as demo1)
+	dotD.library.add(moduleName, workingFunction);
+	
+	// ALTERNATIVE PATTERNS:
+	// You could also add multiple functions from one module:
+	// dotD.library.add(`${moduleName}Sync`, syncVersion);
+	// dotD.library.add(`${moduleName}Async`, asyncVersion);
+};
+
+//END OF moduleFunction() ============================================================
+
+module.exports = moduleFunction({ moduleName }); //runs it right now

package/libDotD_Working_Example/index.js

@@ -0,0 +1,133 @@
+#!/usr/bin/env node
+'use strict';
+// Suppress punycode deprecation warning
+process.noDeprecation = true;
+
+const moduleName = __filename.replace(__dirname + '/', '').replace(/.js$/, ''); //this just seems to come in handy a lot
+
+// const qt = require('qtools-functional-library'); //also exposes qtLog(); qt.help({printOutput:true, queryString:'.*', sendJson:false});
+
+const os = require('os');
+const path = require('path');
+const fs = require('fs');
+
+// DYNAMIC LIBRARY CREATION GUIDE
+// ===============================
+// This example demonstrates how to create a dynamic dotD library using qtools-library-dot-d
+// 
+// KEY CONCEPTS:
+// 1. dotD libraries automatically load modules from a specified directory
+// 2. All .js files in the dotD directory become available as methods on the library object
+// 3. Modules receive shared data through passThroughParameters
+// 4. Libraries can be loaded synchronously or asynchronously
+// 5. Always call seal() after loading to finalize the library
+
+const qtLibDotD = require('qtools-library-dot-d');
+
+
+//START OF moduleFunction() ============================================================
+const moduleFunction =
+	({ moduleName } = {}) =>
+	({ unused } = {}) => {
+	
+		// ========================================================================
+		// SYNCHRONOUS DEMO - Use this pattern for simple, immediate library creation
+
+		const synchronousDemo = () => {
+			// STEP 1: Create the library instance with a descriptive name
+			const demoDotD = qtLibDotD({
+				libraryName: 'demoDotD', // This name appears in logs and debugging output
+			});
+
+			// STEP 2: Define the path to your dotD modules directory
+			// Convention: Use 'ModuleName_DotD' or 'moduleName.d' for the directory name
+			const libraryModulesPath = path.join(__dirname, 'demoModules_DotD');
+
+			// STEP 3: Define passThroughParameters - shared data/functions for all modules
+			// These parameters are available in every module via the passThroughParameters argument
+			// Common uses: configuration, shared utilities, database connections, logging functions
+			const passThroughParameters = {
+				message: 'Hello from a dotDlib module', // Example: shared message
+				add: (a, b) => a + b, // Example: shared utility function
+				// Add your own shared resources here:
+				// config: getConfig(),
+				// logger: xLog,
+				// database: dbConnection,
+			};
+
+			// STEP 4: Set the library path and load modules
+			demoDotD.setLibraryPath(libraryModulesPath);
+			demoDotD.loadModules({ passThroughParameters }); // Synchronous loading
+
+			// STEP 5: Seal the library to prevent further modifications
+			demoDotD.seal();
+
+			// DEBUGGING: Uncomment to see detailed loading information
+			// console.log(demoDotD.qtdProcessLog());
+			
+			// INSPECTION: See what modules were loaded
+			console.log(demoDotD.toString());
+
+			// USAGE: Call your dynamically loaded modules
+			// Each .js file in the dotD directory becomes a method with the same name
+			demoDotD.demo1(); // Calls demoModules_DotD/demo1.js
+			demoDotD.demo2(10, 7); // Calls demoModules_DotD/demo2.js with arguments
+		};
+			console.log(`SYNCHRONOUS DEMO =====================================`);
+			synchronousDemo();
+	
+		// ========================================================================
+		// ASYNCHRONOUS DEMO - Use this pattern when modules need async initialization
+
+		const asynchronousDemo = (callback) => {
+			// Create a local callback to handle async completion
+			const localCallback = (err) => {
+				// IMPORTANT: Always seal after async loading completes
+				demoDotD.seal();
+				callback(err, demoDotD);
+			};
+
+			// STEP 1: Create the library instance
+			const demoDotD = qtLibDotD({
+				libraryName: 'demoDotD',
+			});
+
+			// STEP 2: Define module directory path
+			const libraryModulesPath = path.join(__dirname, 'demoModules_DotD');
+
+			// STEP 3: Define shared parameters (same as synchronous version)
+			const passThroughParameters = {
+				message: 'Hello from a dotDlib module',
+				add: (a, b) => a + b,
+				// For async loading, you might include:
+				// database: await dbConnection(),
+				// config: await loadConfig(),
+			};
+
+			// STEP 4: Set path and load modules asynchronously
+			demoDotD.setLibraryPath(libraryModulesPath);
+			demoDotD.loadModules({ passThroughParameters }, localCallback); // Async loading with callback
+		};
+
+		console.log(`\n\nASYNCHRONOUS DEMO =====================================`);
+
+		// Define callback to handle async completion
+		const callback = (err, demoDotD) => {
+			if (err) {
+				console.error('Error loading library:', err);
+				return;
+			}
+			// Library is now ready to use
+			console.log(demoDotD.toString());
+			demoDotD.demo1();
+			demoDotD.demo2(10, 7);
+		};
+		
+		// Start async loading
+		asynchronousDemo(callback);
+	};
+
+//END OF moduleFunction() ============================================================
+
+module.exports = moduleFunction({ moduleName })({});
+

package/libDotD_Working_Example/node_modules/.package-lock.json

@@ -0,0 +1,29 @@
+{
+  "name": "libdotd_example",
+  "version": "1.0.0",
+  "lockfileVersion": 3,
+  "requires": true,
+  "packages": {
+    "node_modules/qtools-asynchronous-pipe-plus": {
+      "version": "1.0.5",
+      "resolved": "https://registry.npmjs.org/qtools-asynchronous-pipe-plus/-/qtools-asynchronous-pipe-plus-1.0.5.tgz",
+      "integrity": "sha512-ejueK++dNl6AJL0OK14zTewXDg1+Wq8zFknI83THmgR9rNTB7EfngLt031/hkkc3/VpnqdaeO7mNqnu6YAzgIQ=="
+    },
+    "node_modules/qtools-functional-library": {
+      "version": "2.0.4",
+      "resolved": "https://registry.npmjs.org/qtools-functional-library/-/qtools-functional-library-2.0.4.tgz",
+      "integrity": "sha512-noztSkJI37xcOTer5HVgOnuY/eZ1FeqtSvlpzhmSq7jjT6dQCQjIzwjoe1GqHc5Dt9Os+OFawAUz3hMpO4V1ng==",
+      "license": "ISC"
+    },
+    "node_modules/qtools-library-dot-d": {
+      "version": "1.0.5",
+      "resolved": "https://registry.npmjs.org/qtools-library-dot-d/-/qtools-library-dot-d-1.0.5.tgz",
+      "integrity": "sha512-MEkrc1NCk8OleAlUPrlJ4Pl8z9v4gOEQt6PIjTc7r16x4StnfQ/ntTfaFF2kS4pi6O1zY1YBpB8vG1q0rdwTcQ==",
+      "license": "ISC",
+      "dependencies": {
+        "qtools-asynchronous-pipe-plus": "^1.0.2",
+        "qtools-functional-library": "^2.0.4"
+      }
+    }
+  }
+}