retold-data-service 2.0.9 → 2.0.11

package/docs/configuration.md

@@ -0,0 +1,93 @@
+# Configuration
+
+Retold Data Service accepts options during instantiation that control model loading, storage provider selection, and server behavior.
+
+## Options Reference
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `StorageProvider` | String | `'MySQL'` | Meadow provider name (`MySQL`, `SQLite`, `MSSQL`, `ALASQL`) |
+| `StorageProviderModule` | String | `'meadow-connection-mysql'` | npm module name for the storage provider |
+| `FullMeadowSchemaPath` | String | `${process.cwd()}/model/` | Directory path where the compiled schema lives |
+| `FullMeadowSchemaFilename` | String | `'MeadowModel-Extended.json'` | Filename of the compiled Stricture schema |
+| `AutoInitializeDataService` | Boolean | `true` | Whether to auto-initialize on construction |
+| `AutoStartOrator` | Boolean | `true` | Whether to start the Orator web server during initialization |
+
+## Fable Settings
+
+The Fable instance configuration controls server port and database connection:
+
+```javascript
+{
+    Product: 'MyApp',
+    APIServerPort: 8086,
+
+    // For MySQL
+    MySQL: {
+        Server: '127.0.0.1',
+        Port: 3306,
+        User: 'root',
+        Password: 'secret',
+        Database: 'mydb',
+        ConnectionPoolLimit: 20
+    },
+    MeadowConnectionMySQLAutoConnect: true,
+
+    // For SQLite
+    SQLite: {
+        SQLiteFilePath: './data/myapp.db'
+    }
+}
+```
+
+## Example: MySQL Configuration
+
+```javascript
+_Fable.serviceManager.instantiateServiceProvider('RetoldDataService', {
+    FullMeadowSchemaPath: `${__dirname}/model/`,
+    FullMeadowSchemaFilename: 'MeadowModel-Extended.json',
+    StorageProvider: 'MySQL',
+    StorageProviderModule: 'meadow-connection-mysql',
+    AutoStartOrator: true
+});
+```
+
+## Example: SQLite Configuration
+
+```javascript
+// Register the SQLite provider before creating the data service
+const libMeadowConnectionSQLite = require('meadow-connection-sqlite');
+_Fable.serviceManager.addServiceType('MeadowSQLiteProvider', libMeadowConnectionSQLite);
+_Fable.serviceManager.instantiateServiceProvider('MeadowSQLiteProvider');
+
+_Fable.MeadowSQLiteProvider.connectAsync(
+    (pError) =>
+    {
+        _Fable.serviceManager.instantiateServiceProvider('RetoldDataService', {
+            FullMeadowSchemaPath: `${__dirname}/model/`,
+            FullMeadowSchemaFilename: 'MeadowModel-Extended.json',
+            StorageProvider: 'SQLite',
+            StorageProviderModule: 'meadow-connection-sqlite',
+            AutoStartOrator: true
+        });
+    });
+```
+
+## Example: Disabling Auto-Start
+
+```javascript
+_Fable.serviceManager.instantiateServiceProvider('RetoldDataService', {
+    FullMeadowSchemaPath: `${__dirname}/model/`,
+    FullMeadowSchemaFilename: 'MeadowModel-Extended.json',
+    AutoStartOrator: false
+});
+
+// Manually start later
+_Fable.RetoldDataService.initializeService(
+    (pError) =>
+    {
+        // Service initialized without starting the web server
+        // Start it manually when ready:
+        _Fable.Orator.startService(() => console.log('Server started'));
+    });
+```

package/docs/cover.md

@@ -0,0 +1,11 @@
+# Retold Data Service <small>2</small>
+
+> All-in-one data service for the Retold ecosystem
+
+- Schema-driven REST API generation from Stricture DDL models
+- Automatic CRUD endpoints for every entity in your data model
+- Pluggable storage providers: MySQL, SQLite, MSSQL, ALASQL
+- Built on Fable, Meadow, and Orator with lifecycle hooks for customization
+
+[GitHub](https://github.com/stevenvelozo/retold-data-service)
+[Get Started](#retold-data-service)

package/docs/dal-access.md

@@ -0,0 +1,87 @@
+# DAL Access
+
+In addition to the REST endpoints, you can query your data directly through the Meadow DAL objects.
+
+## Accessing DAL Objects
+
+After initialization, every entity in your model has a DAL object accessible via `fable.DAL`:
+
+```javascript
+// Read a single record
+let tmpQuery = _Fable.DAL.Book.query.addFilter('IDBook', 42);
+
+_Fable.DAL.Book.doRead(tmpQuery,
+    (pError, pQuery, pRecord) =>
+    {
+        console.log('Found:', pRecord.Title);
+    });
+```
+
+## Available Operations
+
+| Method | Description |
+|--------|-------------|
+| `doCreate(query, callback)` | Insert a new record |
+| `doRead(query, callback)` | Read a single record |
+| `doReads(query, callback)` | Read multiple records |
+| `doUpdate(query, callback)` | Update an existing record |
+| `doDelete(query, callback)` | Soft-delete a record |
+| `doUndelete(query, callback)` | Restore a soft-deleted record |
+| `doCount(query, callback)` | Count matching records |
+
+## Query Building
+
+Each DAL has a `query` property (a FoxHound query instance) that you can configure with filters, sorting, and pagination:
+
+```javascript
+// Read all Science Fiction books, newest first, page 1
+let tmpQuery = _Fable.DAL.Book.query
+    .addFilter('Genre', 'Science Fiction')
+    .addSort({Column: 'PublicationYear', Direction: 'Descending'})
+    .setCap(25)
+    .setBegin(0);
+
+_Fable.DAL.Book.doReads(tmpQuery,
+    (pError, pQuery, pRecords) =>
+    {
+        console.log(`Found ${pRecords.length} books`);
+    });
+```
+
+## Creating Records
+
+```javascript
+let tmpQuery = _Fable.DAL.Author.query
+    .addRecord({Name: 'Frank Herbert'});
+
+_Fable.DAL.Author.doCreate(tmpQuery,
+    (pError, pCreateQuery, pReadQuery, pRecord) =>
+    {
+        console.log('Created author with ID:', pRecord.IDAuthor);
+    });
+```
+
+## Counting Records
+
+```javascript
+let tmpQuery = _Fable.DAL.Review.query;
+
+_Fable.DAL.Review.doCount(tmpQuery,
+    (pError, pQuery, pCount) =>
+    {
+        console.log('Total reviews:', pCount);
+    });
+```
+
+## When to Use DAL vs Endpoints
+
+Use the **REST endpoints** when:
+- Building a web UI that communicates over HTTP
+- Exposing an API for external consumers
+- You need session management and authorization
+
+Use **DAL access** when:
+- Running server-side business logic
+- Performing batch operations
+- Writing behavior injection hooks that need to query related entities
+- Running background tasks or workers

package/docs/endpoints.md

@@ -0,0 +1,121 @@
+# Endpoints
+
+Once initialized, Retold Data Service automatically creates RESTful endpoints for every entity in your Stricture model.
+
+## URL Pattern
+
+All endpoints follow the Meadow Endpoints URL pattern:
+
+```
+/{Version}/{Entity}         — singular entity operations (Read, Create, Update, Delete)
+/{Version}/{Entity}s        — plural entity operations (Reads, Count)
+```
+
+The version defaults to `1.0`.
+
+## Available Endpoints
+
+For an entity named `Book`, these endpoints are created:
+
+### Read (Single Record)
+
+```
+GET /1.0/Book/:IDRecord
+```
+
+Returns a single record by primary key.  Returns a 404 error object if not found.
+
+### Read (Multiple Records)
+
+```
+GET /1.0/Books
+GET /1.0/Books/:Begin/:Cap
+GET /1.0/Books/FilteredTo/:Filter
+GET /1.0/Books/FilteredTo/:Filter/:Begin/:Cap
+```
+
+Returns an array of records.  Supports pagination via `Begin` and `Cap` parameters, and filtering via the Meadow filter stanza format.
+
+### Create
+
+```
+POST /1.0/Book
+Body: { "Title": "Dune", "Genre": "Science Fiction" }
+```
+
+Creates a new record and returns the created record (with auto-generated ID and GUID).
+
+### Update
+
+```
+PUT /1.0/Book
+Body: { "IDBook": 1, "Title": "Dune (Updated)" }
+```
+
+Updates an existing record and returns the updated record.  The request body must include the primary key.
+
+### Delete (Soft Delete)
+
+```
+DELETE /1.0/Book/:IDRecord
+```
+
+Soft-deletes a record (sets `Deleted = 1`).  Returns the number of affected records.
+
+### Count
+
+```
+GET /1.0/Books/Count
+GET /1.0/Books/Count/FilteredTo/:Filter
+GET /1.0/Books/Count/By/:ByField/:ByValue
+```
+
+Returns the count of matching records as `{ "Count": N }`.
+
+### Schema
+
+```
+GET /1.0/Book/Schema
+```
+
+Returns the JSON Schema for the entity.
+
+### New Default Record
+
+```
+GET /1.0/Book/Schema/New
+```
+
+Returns a new default record with all fields set to their default values.
+
+## Filter Stanza Format
+
+Filters use the Meadow filter stanza format in the URL:
+
+```
+FBV~Column~Operator~Value
+```
+
+| Code | Meaning |
+|------|---------|
+| `FBV` | Filter By Value |
+| `EQ` | Equals |
+| `NE` | Not Equals |
+| `GT` | Greater Than |
+| `GE` | Greater Than or Equal |
+| `LT` | Less Than |
+| `LE` | Less Than or Equal |
+| `LK` | LIKE |
+
+### Examples
+
+```
+# Filter by exact genre
+GET /1.0/Books/FilteredTo/FBV~Genre~EQ~Science Fiction
+
+# Filter by title pattern (URL-encode the % as %25)
+GET /1.0/Books/FilteredTo/FBV~Title~LK~%25Dune%25
+
+# Count filtered records
+GET /1.0/Books/Count/FilteredTo/FBV~Genre~EQ~Fantasy
+```

package/docs/index.html

@@ -0,0 +1,39 @@
+<!doctype html>
+<html lang="en">
+	<head>
+		<meta charset="utf-8">
+		<meta http-equiv="X-UA-Compatible" content="IE=edge">
+		<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
+		<meta name="description" content="Documentation powered by pict-docuserve">
+
+		<title>Documentation</title>
+
+		<!-- Application Stylesheet -->
+		<link href="https://cdn.jsdelivr.net/npm/pict-docuserve@0/dist/css/docuserve.css" rel="stylesheet">
+		<!-- KaTeX stylesheet for LaTeX equation rendering -->
+		<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.16.21/dist/katex.min.css">
+		<!-- PICT Dynamic View CSS Container -->
+		<style id="PICT-CSS"></style>
+
+		<!-- Load the PICT library from jsDelivr CDN -->
+		<script src="https://cdn.jsdelivr.net/npm/pict@1/dist/pict.min.js" type="text/javascript"></script>
+		<!-- Bootstrap the Application -->
+		<script type="text/javascript">
+//<![CDATA[
+		Pict.safeOnDocumentReady(() => { Pict.safeLoadPictApplication(PictDocuserve, 2)});
+//]]>
+		</script>
+	</head>
+	<body>
+		<!-- The root container for the Pict application -->
+		<div id="Docuserve-Application-Container"></div>
+
+		<!-- Mermaid diagram rendering -->
+		<script src="https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.min.js"></script>
+		<script>mermaid.initialize({ startOnLoad: false, theme: 'default' });</script>
+		<!-- KaTeX for LaTeX equation rendering -->
+		<script defer src="https://cdn.jsdelivr.net/npm/katex@0.16.21/dist/katex.min.js"></script>
+		<!-- Load the Docuserve PICT Application Bundle from jsDelivr CDN -->
+		<script src="https://cdn.jsdelivr.net/npm/pict-docuserve@0/dist/pict-docuserve.min.js" type="text/javascript"></script>
+	</body>
+</html>

package/docs/initialization.md

@@ -0,0 +1,77 @@
+# Initialization
+
+Retold Data Service follows an ordered initialization sequence.  Understanding this sequence is key to customizing the service behavior.
+
+## Basic Initialization
+
+```javascript
+const libFable = require('fable');
+const libRetoldDataService = require('retold-data-service');
+
+const _Fable = new libFable(settings);
+
+// 1. Register the service type
+_Fable.serviceManager.addServiceType('RetoldDataService', libRetoldDataService);
+
+// 2. Instantiate the service with options
+_Fable.serviceManager.instantiateServiceProvider('RetoldDataService', options);
+
+// 3. Initialize the service
+_Fable.RetoldDataService.initializeService(
+    (pError) =>
+    {
+        if (pError) return console.error('Init failed:', pError);
+        console.log('Ready!');
+    });
+```
+
+## Initialization Sequence
+
+When `initializeService()` is called, the following steps run in order:
+
+```
+1. onBeforeInitialize()       ← your custom hook
+2. Start Orator web server    ← if AutoStartOrator is true
+3. initializePersistenceEngine()  ← loads the storage provider module
+4. onInitialize()             ← your custom hook
+5. initializeDataEndpoints()  ← loads model, creates DAL + endpoints
+6. onAfterInitialize()        ← your custom hook
+```
+
+### Step 5: initializeDataEndpoints()
+
+This is the core step where the model is loaded and endpoints are created:
+
+1. Load the compiled Stricture schema from `FullMeadowSchemaPath + FullMeadowSchemaFilename`
+2. Extract the list of table names from `model.Tables`
+3. For each table:
+   - Create a Meadow DAL from the table's `MeadowSchema` package
+   - Set the DAL's provider to the configured `StorageProvider`
+   - Create a Meadow Endpoints controller for the DAL
+   - Connect the endpoint routes to the Orator service server
+
+## Double-Initialization Protection
+
+Calling `initializeService()` a second time returns an error:
+
+```javascript
+_Fable.RetoldDataService.initializeService(
+    (pError) =>
+    {
+        // pError.message: "Retold Data Service Application is being
+        //                   initialized but has already been initialized..."
+    });
+```
+
+## Stopping the Service
+
+```javascript
+_Fable.RetoldDataService.stopService(
+    (pError) =>
+    {
+        if (pError) return console.error('Stop failed:', pError);
+        console.log('Service stopped');
+    });
+```
+
+Calling `stopService()` on an uninitialized service returns an error.

package/docs/lifecycle-hooks.md

@@ -0,0 +1,81 @@
+# Lifecycle Hooks
+
+Retold Data Service provides three lifecycle hooks that you can override to customize initialization behavior.
+
+## Available Hooks
+
+| Hook | Fires | Use Case |
+|------|-------|----------|
+| `onBeforeInitialize(fCallback)` | Before Orator starts and before the persistence engine loads | Environment checks, early configuration |
+| `onInitialize(fCallback)` | After Orator starts and persistence engine loads, before data endpoints are created | Register additional services, run migrations |
+| `onAfterInitialize(fCallback)` | After all data endpoints are created and routes are connected | Post-startup tasks, inject behaviors, warm caches |
+
+## Overriding Hooks
+
+You can override the hooks by subclassing `RetoldDataService`:
+
+```javascript
+const libRetoldDataService = require('retold-data-service');
+
+class MyDataService extends libRetoldDataService
+{
+    onBeforeInitialize(fCallback)
+    {
+        this.fable.log.info('Custom pre-initialization...');
+        // Perform environment validation, load configs, etc.
+        return fCallback();
+    }
+
+    onInitialize(fCallback)
+    {
+        this.fable.log.info('Custom initialization...');
+        // Run database migrations, register extra services, etc.
+        return fCallback();
+    }
+
+    onAfterInitialize(fCallback)
+    {
+        this.fable.log.info('Custom post-initialization...');
+        // Inject behaviors, seed data, start background tasks, etc.
+        this.fable.MeadowEndpoints.Book.controller.BehaviorInjection
+            .setBehavior('Read-PostOperation', myCustomHook);
+        return fCallback();
+    }
+}
+```
+
+## Hook Execution Order
+
+```
+initializeService() called
+    │
+    ├── onBeforeInitialize()
+    │
+    ├── Start Orator (if AutoStartOrator)
+    │
+    ├── initializePersistenceEngine()
+    │
+    ├── onInitialize()
+    │
+    ├── initializeDataEndpoints()
+    │       ├── Load schema model
+    │       ├── Create DAL for each entity
+    │       ├── Create Endpoints for each entity
+    │       └── Connect routes to Orator
+    │
+    ├── onAfterInitialize()
+    │
+    └── callback(error)
+```
+
+## Error Handling
+
+If any hook or step passes an error to its callback, the initialization chain stops and the error is passed to the `initializeService` callback:
+
+```javascript
+onInitialize(fCallback)
+{
+    // Signal an error to stop initialization
+    return fCallback(new Error('Database migration failed'));
+}
+```

package/docs/schema-definition.md

@@ -0,0 +1,128 @@
+# Schema Definition
+
+Retold Data Service loads a compiled Stricture schema model to create its entities and endpoints.
+
+## The Stricture DDL
+
+Schemas are defined using Stricture's DDL syntax, then compiled into a JSON model that Retold Data Service understands.
+
+### DDL Example
+
+```
+!Book
+@IDBook
+%GUIDBook
+&CreateDate
+#CreatingIDUser
+&UpdateDate
+#UpdatingIDUser
+^Deleted
+&DeleteDate
+#DeletingIDUser
+$Title 200
+$Type 32
+$Genre 128
+$ISBN 64
+$Language 12
+#PublicationYear
+
+!Author
+@IDAuthor
+%GUIDAuthor
+&CreateDate
+#CreatingIDUser
+&UpdateDate
+#UpdatingIDUser
+^Deleted
+&DeleteDate
+#DeletingIDUser
+$Name 200
+```
+
+### DDL Symbols
+
+| Symbol | Meaning | Meadow Type |
+|--------|---------|-------------|
+| `!` | Table/entity name | — |
+| `@` | Auto-increment ID | `AutoIdentity` |
+| `%` | Auto GUID | `AutoGUID` |
+| `&` | DateTime column | `CreateDate`, `UpdateDate`, `DeleteDate`, or `DateTime` |
+| `#` | Numeric column | `Integer`, `CreateIDUser`, `UpdateIDUser`, `DeleteIDUser` |
+| `$` | String column (with size) | `String` |
+| `^` | Boolean column | `Deleted` or `Boolean` |
+| `.` | Decimal column (with precision) | `Decimal` |
+| `*` | Text column | `String` (large) |
+
+## Compiling the Model
+
+Use Stricture to compile the DDL into JSON:
+
+```bash
+npx stricture -i model/ddl/BookStore.ddl
+```
+
+This produces a `MeadowModel-Extended.json` file containing the full model with:
+- Column definitions per table
+- Meadow schema per table (with `AutoIdentity`, `AutoGUID`, `CreateDate`, etc.)
+- JSON Schema per table
+- Default objects per table
+- Authorization rules per table
+
+## Model JSON Structure
+
+```json
+{
+    "Tables": {
+        "Book": {
+            "TableName": "Book",
+            "Columns": [...],
+            "MeadowSchema": {
+                "Scope": "Book",
+                "DefaultIdentifier": "IDBook",
+                "Schema": [
+                    {"Column": "IDBook", "Type": "AutoIdentity"},
+                    {"Column": "GUIDBook", "Type": "AutoGUID"},
+                    {"Column": "Title", "Type": "String", "Size": "200"},
+                    ...
+                ],
+                "DefaultObject": {
+                    "IDBook": 0,
+                    "Title": "",
+                    ...
+                },
+                "Authorization": { ... }
+            }
+        },
+        "Author": { ... }
+    }
+}
+```
+
+## Build Script
+
+The test model includes a build script for compiling the DDL:
+
+```bash
+npm run build-test-model
+```
+
+This runs `cd test && npx stricture -i model/ddl/BookStore.ddl` to regenerate the compiled model from the DDL.
+
+## Schema Types Reference
+
+| Type | Description | Auto-Managed |
+|------|-------------|-------------|
+| `AutoIdentity` | Auto-increment primary key | On Create (NULL/omit) |
+| `AutoGUID` | Unique identifier | On Create (UUID generated) |
+| `CreateDate` | Creation timestamp | On Create |
+| `CreateIDUser` | Creator user ID | On Create |
+| `UpdateDate` | Last update timestamp | On Create and Update |
+| `UpdateIDUser` | Last updater user ID | On Create and Update |
+| `Deleted` | Soft-delete flag | On Delete (set to 1) |
+| `DeleteDate` | Deletion timestamp | On Delete |
+| `DeleteIDUser` | Deleter user ID | On Delete |
+| `String` | Text data | — |
+| `Integer` | Numeric data | — |
+| `Decimal` | Decimal data | — |
+| `Boolean` | Boolean data | — |
+| `DateTime` | Date/time data | — |