retold-data-service 2.0.8 → 2.0.10

package/.vscode/settings.json

@@ -1,17 +1,36 @@
 {
     "sqltools.connections": [
-        {
-            "mysqlOptions": {
-                "authProtocol": "default"
-            },
-            "previewLimit": 50,
-            "server": "localhost",
-            "port": 3306,
-            "driver": "MariaDB",
-            "name": "Local",
-            "database": "bookstore",
-            "username": "root",
-            "password": "123456789"
-        }
-    ]
+		{
+			"mysqlOptions": {
+				"authProtocol": "default",
+				"enableSsl": "Disabled"
+			},
+			"ssh": "Disabled",
+			"previewLimit": 50,
+			"server": "localhost",
+			"port": 3306,
+			"driver": "MariaDB",
+			"name": "Local MariaDB Harness",
+			"database": "bookstore",
+			"username": "root",
+			"password": "123456789",
+			"group": "Retold Development"
+		},
+		{
+			"mysqlOptions": {
+				"authProtocol": "default",
+				"enableSsl": "Disabled"
+			},
+			"ssh": "Disabled",
+			"previewLimit": 50,
+			"server": "localhost",
+			"port": 31306,
+			"driver": "MariaDB",
+			"name": "Containerized MariaDB Harness",
+			"group": "Retold Development",
+			"database": "bookstore",
+			"username": "root",
+			"password": "123456789"
+		}
+	]
 }

package/CONTRIBUTING.md

@@ -0,0 +1,50 @@
+# Contributing to Retold
+
+We welcome contributions to Retold and its modules. This guide covers the expectations and process for contributing.
+
+## Code of Conduct
+
+The Retold community values **empathy**, **equity**, **kindness**, and **thoughtfulness**. We expect all participants to treat each other with respect, assume good intent, and engage constructively. These values apply to all interactions: pull requests, issues, discussions, and code review.
+
+## How to Contribute
+
+### Pull Requests
+
+Pull requests are the preferred method for contributing changes. To submit one:
+
+1. Fork the module repository you want to change
+2. Create a branch for your work
+3. Make your changes, following the code style of the module you are editing
+4. Ensure your changes have test coverage (see below)
+5. Open a pull request against the module's main branch
+
+**Submitting a pull request does not guarantee it will be accepted.** Maintainers review contributions for fit, quality, and alignment with the project's direction. A PR may be declined, or you may be asked to revise it. This is normal and not a reflection on the quality of your effort.
+
+### Reporting Issues
+
+If you find a bug or have a feature suggestion, open an issue on the relevant module's repository. Include enough detail to reproduce the problem or understand the proposal.
+
+## Test Coverage
+
+Every commit must include test coverage for the changes it introduces. Retold modules use Mocha in TDD style. Before submitting:
+
+- **Write tests** for any new functionality or bug fixes
+- **Run the existing test suite** with `npm test` and confirm all tests pass
+- **Check coverage** with `npm run coverage` if the module supports it
+
+Pull requests that break existing tests or lack coverage for new code will not be merged.
+
+## Code Style
+
+Follow the conventions of the module you are working in. The general Retold style is:
+
+- **Tabs** for indentation, never spaces
+- **Plain JavaScript** only (no TypeScript)
+- **Allman-style braces** (opening brace on its own line)
+- **Variable naming:** `pVariable` for parameters, `tmpVariable` for temporaries, `libSomething` for imports
+
+When in doubt, match what the surrounding code does.
+
+## Repository Structure
+
+Each module is its own git repository. The [retold](https://github.com/stevenvelozo/retold) repository tracks module organization but does not contain module source code. Direct your pull request to the specific module repository where your change belongs.

package/README.md

@@ -1,86 +1,230 @@
 # Retold Data Service
 
-Provide a consistent back or mid-tier data service.
+> An all-in-one Fable service that turns a Stricture schema into a complete REST API
+
+Retold Data Service combines Meadow (data access), Orator (API server), and Meadow Endpoints (REST routes) into a single service provider. Point it at a compiled Stricture model and it auto-generates typed CRUD endpoints for every entity -- complete with filtering, pagination, soft deletes, and behavior injection hooks.
+
+## Features
+
+- **Zero-Boilerplate REST** - Define your schema once, get full CRUD endpoints for every entity automatically
+- **Provider-Agnostic** - Swap between MySQL, MSSQL, SQLite, or ALASQL without changing application code
+- **Schema-Driven** - Stricture DDL compiles into a model that drives endpoint generation, validation, and defaults
+- **Lifecycle Hooks** - `onBeforeInitialize`, `onInitialize`, and `onAfterInitialize` for custom startup logic
+- **Behavior Injection** - Pre- and post-operation hooks on every CRUD operation for custom business logic
+- **DAL Access** - Direct programmatic data access alongside the REST endpoints for server-side logic
+- **Fable Service Provider** - First-class service in the Fable ecosystem with logging, configuration, and DI
+- **Double-Init Protection** - Guards against accidental re-initialization
+
+## Quick Start
+
+```javascript
+const libFable = require('fable');
+const libRetoldDataService = require('retold-data-service');
+
+const _Fable = new libFable({
+	APIServerPort: 8086,
+	MySQL: {
+		Server: '127.0.0.1',
+		Port: 3306,
+		User: 'root',
+		Password: 'secret',
+		Database: 'bookstore',
+		ConnectionPoolLimit: 20
+	},
+	MeadowConnectionMySQLAutoConnect: true
+});
+
+_Fable.serviceManager.addServiceType('RetoldDataService', libRetoldDataService);
+
+_Fable.serviceManager.instantiateServiceProvider('RetoldDataService', {
+	StorageProvider: 'MySQL',
+	StorageProviderModule: 'meadow-connection-mysql',
+	FullMeadowSchemaPath: `${__dirname}/model/`,
+	FullMeadowSchemaFilename: 'MeadowModel-Extended.json'
+});
+
+_Fable.RetoldDataService.initializeService(
+	(pError) =>
+	{
+		if (pError)
+		{
+			return console.log('Error initializing data service:', pError);
+		}
+		console.log('REST API is running on port 8086');
+	});
+```
+
+## Installation
+
+```bash
+npm install retold-data-service
+```
+
+## How It Works
+
+Retold Data Service orchestrates the full Meadow stack into a single initialization sequence:
+
+```
+Fable (Core)
+  └── Retold Data Service
+        ├── Orator (API Server)
+        │     └── Restify (HTTP Engine)
+        ├── Meadow (Data Access Layer)
+        │     ├── Schema (from compiled Stricture model)
+        │     ├── FoxHound (Query DSL)
+        │     └── Provider (MySQL / MSSQL / SQLite / ALASQL)
+        └── Meadow Endpoints (REST Routes)
+              ├── Create   POST   /1.0/Entity
+              ├── Read     GET    /1.0/Entity/:ID
+              ├── Reads    GET    /1.0/Entities/:Begin/:Cap
+              ├── Update   PUT    /1.0/Entity
+              ├── Delete   DELETE /1.0/Entity/:ID
+              ├── Count    GET    /1.0/Entities/Count
+              ├── Schema   GET    /1.0/Entity/Schema
+              └── New      GET    /1.0/Entity/Schema/New
+```
+
+## Options
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `StorageProvider` | `'MySQL'` | Database provider name |
+| `StorageProviderModule` | `'meadow-connection-mysql'` | Node module for the provider |
+| `FullMeadowSchemaPath` | `process.cwd() + '/model/'` | Path to the compiled schema |
+| `FullMeadowSchemaFilename` | `'MeadowModel-Extended.json'` | Compiled model filename |
+| `AutoStartOrator` | `true` | Start the HTTP server automatically |
+| `APIServerPort` | `8080` | Port for the HTTP server |
+
+## SQLite Example
+
+For embedded or test scenarios, use the in-memory SQLite provider:
+
+```javascript
+const libMeadowConnectionSQLite = require('meadow-connection-sqlite');
+
+const _Fable = new libFable({
+	APIServerPort: 8086,
+	SQLite: { SQLiteFilePath: ':memory:' }
+});
+
+_Fable.serviceManager.addServiceType('RetoldDataService', libRetoldDataService);
+_Fable.serviceManager.addServiceType('MeadowSQLiteProvider', libMeadowConnectionSQLite);
+_Fable.serviceManager.instantiateServiceProvider('MeadowSQLiteProvider');
+
+_Fable.MeadowSQLiteProvider.connectAsync(
+	(pError) =>
+	{
+		_Fable.serviceManager.instantiateServiceProvider('RetoldDataService', {
+			StorageProvider: 'SQLite',
+			StorageProviderModule: 'meadow-connection-sqlite',
+			FullMeadowSchemaPath: `${__dirname}/model/`,
+			FullMeadowSchemaFilename: 'MeadowModel-Extended.json'
+		});
+
+		_Fable.RetoldDataService.initializeService(
+			(pError) =>
+			{
+				console.log('REST API running with in-memory SQLite');
+			});
+	});
+```
+
+## Behavior Injection
+
+Add custom logic before or after any CRUD operation:
+
+```javascript
+_Fable.MeadowEndpoints.Book.controller.BehaviorInjection.setBehavior(
+	'Create-PreOperation',
+	(pRequest, pRequestState, fRequestComplete) =>
+	{
+		if (!pRequestState.RecordToCreate.Title)
+		{
+			pRequest.CommonServices.sendCodedResponse(
+				pRequestState.response, 400, 'Title is required');
+			return fRequestComplete(true);
+		}
+		return fRequestComplete(false);
+	});
+```
 
-## Basic Usage
+## Lifecycle Hooks
 
-This library can run in any of three types of configurations:
+Customize initialization by subclassing:
 
-* Configuration Driven
-* Managed via Backplane Endpoints
-* Hybrid (configuration plus endpoint)
-
-## Configuration-Driven Mode
-
-The service looks for a `Retold` stanza in the configuration.  For instance:
-
-```
+```javascript
+class MyDataService extends libRetoldDataService
 {
-    ...
-    "Retold": {
-        "MeadowModel": "./meadow-schema-extended.json"
-    },
-    ...
+	onAfterInitialize(fCallback)
+	{
+		this.fable.log.info('Injecting custom behaviors...');
+		this.fable.MeadowEndpoints.Book.controller.BehaviorInjection
+			.setBehavior('Read-PostOperation', myCustomHook);
+		return fCallback();
+	}
 }
 ```
 
-The three auto-configured parameters are:
-
-* `MeadowModel`
-* `MeadowEntitySchema`
-* `StrictureDDL`
+## DAL Access
 
-The service tries to do the right thing with strings versus arrays -- you can pass an array of models or a single.  Likewise, you can pass an array of schemas or a single.  It also supports grabbing all files of a single folder with the `/*` suffix, and recursive with the `/**` suffix.  When used in this fashion, only `.json` files are loaded from the folder(s).
+Query data directly alongside the REST endpoints:
 
-## Supplied Backplane Endpoints
+```javascript
+let tmpQuery = _Fable.DAL.Book.query
+	.addFilter('Genre', 'Science Fiction')
+	.addSort({ Column: 'PublicationYear', Direction: 'Descending' })
+	.setCap(25)
+	.setBegin(0);
 
-### Load a Meadow Model
-
-```
-POST /BackPlane/${VERSION}/Load/MeadowModel
+_Fable.DAL.Book.doReads(tmpQuery,
+	(pError, pQuery, pRecords) =>
+	{
+		console.log(`Found ${pRecords.length} sci-fi books`);
+	});
 ```
 
-This endpoint accepts a JSON blob of an entire Meadow model.  It loads the entire model as a set of endpoints, and connects each endpoint to the default provider.
+## Backplane Endpoints
 
-### Load a Meadow Entity Schema
+For dynamic model loading at runtime, Retold Data Service provides backplane endpoints:
 
-```
-POST /BackPlane/${VERSION}/Load/MeadowSchema
-```
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/BackPlane/:Version/Load/MeadowModel` | POST | Load a full Meadow model |
+| `/BackPlane/:Version/Load/MeadowSchema` | POST | Load a Meadow entity schema |
+| `/BackPlane/:Version/Load/StrictureDDL` | POST | Load and compile a Stricture DDL |
+| `/BackPlane/:Version/Model/Primary/` | GET | Get the primary service model |
+| `/BackPlane/:Version/Model/Composite/` | GET | Get composite models |
+| `/BackPlane/:Version/Settings` | POST | Merge in provider settings |
+| `/BackPlane/:Version/SetProvider/:Name` | GET | Set the default provider |
+| `/BackPlane/:Version/SetProvider/:Name/:Entity` | GET | Set a per-entity provider |
 
-This endpoint accepts a JSON blob of an entire Meadow model.  It loads the entire model as a set of endpoints, and connects each endpoint to the default provider.
+## Testing
 
-### Load a DDL and Compile it
-```
-POST /BackPlane/${VERSION}/Load/StrictureDDL
+```bash
+npm test
 ```
 
-This endpoint loads a Stricture DDL, compiles it into the composite Entity schemas and loads them as endpoints.
+Tests use an in-memory SQLite provider and require no external database server.
 
+## Documentation
 
-### Access Primary Service Model and Composite Models
-```
-GET /BackPlane/${VERSION}/Model/Primary/
-```
-
-GET /BackPlane/${VERSION}/Model/Composite/
+Detailed documentation is available in the `docs/` folder and can be served locally:
 
-### Merge in Settings (to push default configurations for providers)
-```
-POST /BackPlane/${VERSION}/Settings
+```bash
+npx docsify-cli serve docs
 ```
 
-### Set the Default Provider
-```
-GET /BackPlane/${VERSION}/SetProvider/:ProviderName
-```
-
-### Set an Entity-Specific Provider
-```
-GET /BackPlane/${VERSION}/SetProvider/:ProviderName/:Entity
-```
+## Related Packages
 
+- [meadow](https://github.com/stevenvelozo/meadow) - Data access and ORM
+- [meadow-endpoints](https://github.com/stevenvelozo/meadow-endpoints) - Auto-generated REST endpoints
+- [orator](https://github.com/stevenvelozo/orator) - API server abstraction
+- [fable](https://github.com/stevenvelozo/fable) - Application services framework
 
+## License
 
+MIT
 
+## Contributing
 
+Pull requests are welcome. For details on our code of conduct, contribution process, and testing requirements, see the [Retold Contributing Guide](https://github.com/stevenvelozo/retold/blob/main/docs/contributing.md).

package/docs/README.md

@@ -0,0 +1,76 @@
+# Retold Data Service
+
+Retold Data Service is an all-in-one module that combines Fable, Meadow, Orator, and Meadow Endpoints into a single service.  Given a compiled Stricture schema, it automatically creates a data access layer (DAL) and RESTful API endpoints for every entity in your model.
+
+## Features
+
+- **Schema-Driven** — load a compiled Stricture model and get full CRUD endpoints for every table automatically
+- **Pluggable Storage** — configure any Meadow-compatible provider (MySQL, SQLite, MSSQL, ALASQL) via options
+- **Fable Service** — extends `fable-serviceproviderbase`, integrating with the Fable ecosystem for logging, configuration, and service management
+- **Lifecycle Hooks** — override `onBeforeInitialize`, `onInitialize`, and `onAfterInitialize` for custom startup logic
+- **Behavior Injection** — add pre- and post-operation hooks to any CRUD endpoint
+- **DAL Access** — query your data directly through `fable.DAL.<Entity>` for programmatic access alongside the REST API
+- **Auto-Start** — optionally start the Orator web server automatically on initialization
+
+## Quick Start
+
+```javascript
+const libFable = require('fable');
+const libRetoldDataService = require('retold-data-service');
+
+const _Fable = new libFable({
+    Product: 'MyApp',
+    APIServerPort: 8086,
+    MySQL: {
+        Server: '127.0.0.1',
+        Port: 3306,
+        User: 'root',
+        Password: 'secret',
+        Database: 'mydb',
+        ConnectionPoolLimit: 20
+    },
+    MeadowConnectionMySQLAutoConnect: true
+});
+
+_Fable.serviceManager.addServiceType('RetoldDataService', libRetoldDataService);
+_Fable.serviceManager.instantiateServiceProvider('RetoldDataService', {
+    FullMeadowSchemaPath: `${__dirname}/model/`,
+    FullMeadowSchemaFilename: 'MeadowModel-Extended.json',
+    StorageProvider: 'MySQL',
+    StorageProviderModule: 'meadow-connection-mysql',
+    AutoStartOrator: true
+});
+
+_Fable.RetoldDataService.initializeService(
+    (pError) =>
+    {
+        if (pError) return console.error('Startup failed:', pError);
+        console.log('Data service running on port 8086');
+    });
+```
+
+## Installation
+
+```bash
+npm install retold-data-service
+```
+
+## Related Packages
+
+| Package | Purpose |
+|---------|---------|
+| [fable](https://github.com/stevenvelozo/fable) | Core service framework |
+| [meadow](https://github.com/stevenvelozo/meadow) | Data access layer and ORM |
+| [foxhound](https://github.com/stevenvelozo/foxhound) | Query DSL for SQL generation |
+| [meadow-endpoints](https://github.com/stevenvelozo/meadow-endpoints) | REST endpoint generation |
+| [orator](https://github.com/stevenvelozo/orator) | API server abstraction |
+| [stricture](https://github.com/stevenvelozo/stricture) | Schema definition DDL compiler |
+| [retold-harness](https://github.com/stevenvelozo/retold-harness) | Application harness using Retold Data Service |
+
+## Testing
+
+```bash
+npm test
+```
+
+Tests run against an in-memory SQLite database and require no external services.

package/docs/_sidebar.md

@@ -0,0 +1,13 @@
+- Getting Started
+  - [Introduction](/)
+  - [Architecture](architecture.md)
+  - [Configuration](configuration.md)
+- Usage
+  - [Initialization](initialization.md)
+  - [Endpoints](endpoints.md)
+  - [DAL Access](dal-access.md)
+  - [Behavior Injection](behavior-injection.md)
+- Advanced
+  - [Lifecycle Hooks](lifecycle-hooks.md)
+  - [Storage Providers](storage-providers.md)
+  - [Schema Definition](schema-definition.md)

package/docs/architecture.md

@@ -0,0 +1,94 @@
+# Architecture
+
+Retold Data Service orchestrates several Retold modules into a unified service that turns a schema into a running API.
+
+## Component Stack
+
+```
+┌─────────────────────────────────────────────┐
+│            Your Application Code            │
+│   - Configure options                       │
+│   - Override lifecycle hooks                 │
+│   - Inject behaviors                        │
+└──────────────────┬──────────────────────────┘
+                   │
+┌──────────────────▼──────────────────────────┐
+│          Retold Data Service                │
+│   - Loads Stricture model                   │
+│   - Creates DAL per entity                  │
+│   - Creates MeadowEndpoints per entity      │
+│   - Manages service lifecycle               │
+└──────────────────┬──────────────────────────┘
+                   │
+┌──────────────────▼──────────────────────────┐
+│              Orator (API Server)            │
+│   - HTTP server via Restify                 │
+│   - Route registration                      │
+│   - Request handling                        │
+└──────────────────┬──────────────────────────┘
+                   │
+┌──────────────────▼──────────────────────────┐
+│         Meadow Endpoints (REST Layer)       │
+│   - CRUD route handlers                     │
+│   - Behavior injection hooks                │
+│   - Session management                      │
+│   - Authorization enforcement               │
+└──────────────────┬──────────────────────────┘
+                   │
+┌──────────────────▼──────────────────────────┐
+│            Meadow (Data Access)             │
+│   - Schema management                       │
+│   - Query building via FoxHound             │
+│   - Provider-based query execution          │
+│   - Audit stamping and soft deletes         │
+└──────────────────┬──────────────────────────┘
+                   │
+┌──────────────────▼──────────────────────────┐
+│          Storage Provider                   │
+│   MySQL │ SQLite │ MSSQL │ ALASQL          │
+└─────────────────────────────────────────────┘
+```
+
+## How It Works
+
+1. **Registration** — `RetoldDataService` is registered with Fable's service manager and instantiated with options
+2. **Server Setup** — Orator and its Restify service server are registered and configured
+3. **Initialization** — calling `initializeService()` triggers an ordered sequence:
+   - `onBeforeInitialize()` — your custom pre-initialization logic
+   - Orator web server start (if `AutoStartOrator` is true)
+   - Persistence engine initialization (loads the storage provider module)
+   - `onInitialize()` — your custom initialization logic
+   - Data endpoint initialization (loads model, creates DALs and endpoints)
+   - `onAfterInitialize()` — your custom post-initialization logic
+4. **Serving** — the Restify server listens for HTTP requests and routes them through Meadow Endpoints to the DAL
+
+## Key Objects
+
+After initialization, these objects are available on the Fable instance:
+
+| Object | Type | Description |
+|--------|------|-------------|
+| `fable.RetoldDataService` | Service | The data service instance |
+| `fable.Orator` | Service | The Orator API server |
+| `fable.OratorServiceServer` | Service | The Restify server |
+| `fable.DAL.<Entity>` | Meadow | DAL for each entity in the model |
+| `fable.MeadowEndpoints.<Entity>` | MeadowEndpoints | Endpoint controller for each entity |
+
+## Service Provider Pattern
+
+Retold Data Service follows the Fable service provider pattern:
+
+```javascript
+const libFableServiceProviderBase = require('fable-serviceproviderbase');
+
+class RetoldDataService extends libFableServiceProviderBase
+{
+    constructor(pFable, pOptions, pServiceHash)
+    {
+        super(pFable, pOptions, pServiceHash);
+        this.serviceType = 'RetoldDataService';
+    }
+}
+```
+
+This means it inherits logging, configuration access, and service lifecycle management from the base class.