@vida-global/core 2.0.1 → 2.0.3

package/README.md

@@ -1,10 +1,11 @@
 # Vida/Core
-This package is intended to contain core elements used across all Vida Apps. Keep in mind that it's included in the `vida-apps-tools` package, so customers developing Vida Apps will have access to it.
+This package contains core elements used across all Vida Apps. It is included in the `vida-apps-tools` package, so customers developing Vida Apps have access to it.
 
-# Guides
 
-- [ActiveRecord](lib/active_record)
-- [HttpClient](lib/http)
-- [Logger](lib/logger)
-- [VidaServer](lib/server)
-- [JobQueue](lib/jobQueue)
+## Guides
+
+- [ActiveRecord](lib/activeRecord) — relational model layer over Sequelize with configuration, migrations, querying, virtual fields, and optional Redis caching.
+- [HttpClient](lib/http) — base class for building typed HTTP clients with shared headers, request/response handling, aborts, and error inspection.
+- [Logger](lib/logger) — scoped, child-aware logger with environment-driven levels.
+- [VidaServer](lib/server) — Express-based server with auto-loaded controllers, validation, rendering, callbacks, cookies, and SSE streaming.
+- [JobQueue](lib/jobQueue) — BullMQ-backed queue with workers, retries/backoff, progress reporting, and queue inspection helpers.

package/config/newrelic-config.js

@@ -86,6 +86,8 @@ exports.config = {
     // the platform's log pipeline.
     application_logging: { enabled: false },
 
+    logging: { enabled: false },
+
     /**
      * Ignore routine, non-business routes so they don't dominate transaction
      * lists or skew apdex. Patterns are anchored regexes matched against the

package/lib/activeRecord/README.md

@@ -1,118 +1,104 @@
-# Configuration
-To configure your database, create a `config.js` file in the `config/db/` directory of your project. That file should follow this format:
-```
+# ActiveRecord
+`ActiveRecord` is a thin wrapper around [Sequelize](https://sequelize.org/docs/v6/) that adds Vida conventions: snake_cased schema, automatic `id`/`created_at`/`updated_at` columns, optional Redis-backed record caching, virtual/override field accessors, and a migration CLI. Direct Sequelize features (associations, scopes, validators, hooks) continue to work as documented upstream.
+
+
+## Setup
+
+Create `config/db/config.js` at the root of your project. The file maps **database IDs** (the `default` ID is used unless a model overrides `static get databaseId`) to per-environment connection details.
+
+```js
 module.exports = {
     default: {
         development: {
             database: process.env.DATABASE,
-            host: process.env.DB_HOST,
+            host:     process.env.DB_HOST,
             password: process.env.DB_PASSWORD,
-            username: process.env.DB_USERNAME
-        },
-        production: {
-            ...
+            username: process.env.DB_USERNAME,
         },
-        ...
-    }
-}
-```
-
-Add additional entries for multiple databases
-```
-module.exports = {
-    default: {
-        ...
+        production: { /* ... */ },
     },
+
+    // Additional databases get their own top-level key.
     metrics: {
-        development: {
-            database: process.env.DATABASE,
-            host: process.env.DB_HOST,
-            password: process.env.DB_PASSWORD,
-            username: process.env.DB_USERNAME
-        },
-        production: {
-            ...
-        },
-        ...
+        development: { /* ... */ },
+        production:  { /* ... */ },
     },
-    ...
 }
 ```
 
+### Connection pooling
 
-### Connection Pooling
-Minimum and maximum connection pool sizes can be configured in config.js, but default to 0 and 5, respectively.
-```
+Min/max pool sizes default to `0` and `5`. Override per environment.
+
+```js
 development: {
     database: process.env.DATABASE,
-    host: process.env.DB_HOST,
+    host:     process.env.DB_HOST,
     password: process.env.DB_PASSWORD,
     username: process.env.DB_USERNAME,
-    pool: {
-        min: 1,
-        max: 10
-    }
+    pool:     { min: 1, max: 10 },
 }
 ```
 
+### Read replicas
 
-### Replication
-Read replicas can be configured by adding a `readers` key to your configuration
-```
+Add a `readers` array to route reads against replicas.
+
+```js
 development: {
     database: process.env.DATABASE,
-    host: process.env.DB_HOST,
+    host:     process.env.DB_HOST,
     password: process.env.DB_PASSWORD,
     username: process.env.DB_USERNAME,
     readers: [
-        {
-            database: ...
-            host: ...
-            password: ...
-            username: ...
-        },
-        {
-            ...
-        }
-    }
+        { database: '...', host: '...', password: '...', username: '...' },
+        { /* ... */ },
+    ],
 }
 ```
 
 
-# Migrations
-To prepare your project to run database migrations, add the following lines to the `scripts` section of your `package.json`:
-```
-"db:create_migration": "node node_modules/@vida-global/core/scripts/activeRecord/migrate.js create_migration",
-"db:migrate": "node node_modules/@vida-global/core/scripts/activeRecord/migrate.js migrate",
-"db:rollback": "node node_modules/@vida-global/core/scripts/activeRecord/migrate.js rollback"
-```
+## Migrations
 
-To generate a new migration file, run: `npm run db:create_migration createUsers`. This will automatically generate a migration file. Your migration can create tables, create indexes, and add, remove, or update columns.
+Wire up the migration CLI by adding scripts to `package.json`:
+
+```json
+"scripts": {
+    "db:create_migration": "node node_modules/@vida-global/core/scripts/activeRecord/migrate.js create_migration",
+    "db:migrate":          "node node_modules/@vida-global/core/scripts/activeRecord/migrate.js migrate",
+    "db:rollback":         "node node_modules/@vida-global/core/scripts/activeRecord/migrate.js rollback"
+}
 ```
+
+Generate a migration with `npm run db:create_migration createUsers`. Pass `-- --databaseId metrics` to scope the migration to a non-default database. Run `npm run db:migrate` to apply migrations or `npm run db:rollback` to undo the most recent one.
+
+```js
 module.exports = {
     up: async function() {
         await this.createTable('users', {
-            email: this.DataTypes.STRING,
-            team_id: this.DataTypes.INTEGER,
+            email:    this.DataTypes.STRING,
+            team_id:  this.DataTypes.INTEGER,
             is_admin: {
-                type: this.DataTypes.BOOLEAN,
+                type:         this.DataTypes.BOOLEAN,
                 defaultValue: false,
-                allowNull: false,
+                allowNull:    false,
             }
         });
 
-        await this.addIndex('users', ['team_id'], {where: {team_id: {[this.Operators.ne]: null}}});
-        await this.addIndex('users', ['email'], {unique: true});
+        await this.addIndex('users', ['team_id'], { where: { team_id: { [this.Operators.ne]: null } } });
+        await this.addIndex('users', ['email'],   { unique: true });
     },
 
-    down: async function () {
+    down: async function() {
         await this.dropTable('users');
     }
 }
 ```
-Available migrations are:
-```
-createTable(tableName, details)
+
+Available migration helpers:
+
+```js
+createTable(tableName, details, { timestamps })
 dropTable(tableName)
 addColumn(tableName, columnName, columnDetails)
 removeColumn(tableName, columnName)
@@ -121,22 +107,26 @@ removeIndex(tableName, indexNameOrAttributes, concurrently=false)
 renameColumn(tableName, oldName, newName)
 changeColumn(tableName, columnName, dataTypeOrOptions)
 ```
-To run your migrations, run `npm run db:migrate` or `npm run db:rollback` to rollback a single migration.
 
-**NOTE**: By default, `createTable` automatically adds an autoincrement, primary key `id` column, a `created_at` column, and an `updated_at` column.
+**Conventions:**
 
+- `createTable` adds an auto-increment `id` primary key and `created_at` / `updated_at` columns by default. Pass `{ timestamps: false }` to skip the timestamps.
+- Column names are automatically converted to snake_case, so `teamId` in the details object becomes `team_id` in the database.
+- Each migration runs inside a transaction.
 
-# Models
-The following code will create a basic model representing a user in our database:
-```
-const { ActiveRecord } = require('@vida-global/core')
+
+## Models
+
+Subclass `BaseRecord` and call `initialize()`. The framework reads the table schema and configures the model.
+
+```js
+const { ActiveRecord } = require('@vida-global/core');
 
 class User extends ActiveRecord.BaseRecord {
 }
-
 User.initialize();
 
-const user = new User({email: 'mark@vida.inc', team_id: 1, is_admin: false});
+const user = new User({ email: 'mark@vida.inc', team_id: 1, is_admin: false });
 await user.save();
 
 user.is_admin = true;
@@ -145,61 +135,180 @@ await user.save();
 await user.destroy();
 ```
 
+### Querying
 
-### Querying Models
-```
-const users = await User.findAll();
-const users = await User.findAll{where: {team_id: 1}});
-const users = await User.findAll{where: User.Operator
+```js
+const allUsers   = await User.findAll();
+const teamUsers  = await User.findAll({ where: { team_id: 1 } });
 
-const users = await User.findAll({where: {
-    team_id: {
-        [Recording.Operators.or]: [1,2]
-    }
-}});
+const eitherTeam = await User.findAll({ where: {
+    team_id: { [User.Operators.or]: [1, 2] }
+} });
 
-```
+const { count, rows } = await User.findAndCountAll({
+    where:  { email: { [User.Operators.like]: '%@vida.inc' } },
+    offset: 0,
+    limit:  10,
+});
 
+const totalCount = await User.count({ where: { team_id: 1 } });
 
-### Aggregators and Functions
+// Convenience for findByPk / list-by-pk
+const byId    = await User.find(1);
+const byIds   = await User.find([1, 2, 3]);
 ```
+
+More query examples: [Sequelize model querying basics](https://sequelize.org/docs/v6/core-concepts/model-querying-basics/).
+
+### Aggregators and functions
+
+Sequelize's `fn` and `col` helpers are exposed on the model class:
+
+```js
 const rows = await User.findAll({
     attributes: ['team_id', [User.fn('MAX', User.col('id')), 'max_id']],
-    group: 'team_id'
+    group:      'team_id',
 });
+```
 
-const { count, users } = await User.findAndCountAll({
-    where: {
-        email: {
-            [User.Operators.like]: '%@vida.inc',
-        },
-    },
-    offset: 0,
-    limit: 1,
+### API serialization
+
+`record.toApiResponse()` returns a plain object containing every column with overridden getters applied. The server's response renderer calls this automatically when a record is included in an action's return value.
+
+### Closing connections
+
+```js
+await User.closeConnection();      // close the connection for this model's database
+await User.closeAllConnections();  // close every connection (all databases, all models)
+```
+
+
+## Custom field accessors
+
+Define `_get${ColumnName}` / `_set${ColumnName}` on the class to override the read/write behavior of a column. The framework wires them up automatically when the model initializes.
+
+```js
+class User extends ActiveRecord.BaseRecord {
+    _getEmail() {
+        return this.getDataValue('email')?.toLowerCase();
+    }
+
+    _setEmail(value) {
+        this.setDataValue('email', value?.trim());
+    }
+}
+User.initialize();
+```
+
+
+## Virtual fields
+
+Add a derived attribute by defining `_virtualGet${PropertyName}` (and optionally `_virtualSet${PropertyName}`). The framework registers a Sequelize `VIRTUAL` column for it, so the property appears in `dataValues`, `toJSON()`, and `toApiResponse()`. Virtual accessors are inherited through the class hierarchy.
+
+```js
+class User extends ActiveRecord.BaseRecord {
+    _virtualGetFullName() {
+        return `${this.first_name} ${this.last_name}`;
+    }
+}
+User.initialize();
+
+const user = await User.find(1);
+user.fullName; // "Ada Lovelace"
+```
+
+
+## Caching
+
+Opt a model in to Redis-backed caching by overriding `isCacheable`. With caching on:
+
+- `Model.find(id)` and `Model.find([id1, id2])` consult Redis first and fall back to the database for misses, repopulating the cache for next time.
+- `afterSave` and `afterDestroy` hooks automatically invalidate the cached record.
+- Cache keys are namespaced as `AR:<ClassName>` (`<ClassName>#find:<id>`).
+
+```js
+class User extends ActiveRecord.BaseRecord {
+    static get isCacheable() { return true; }
+}
+User.initialize();
+
+const user = await User.find(1);             // cache miss; reads DB and caches
+const same = await User.find(1);             // cache hit
+const fresh = await User.find(1, { clear: true }); // forces DB and refreshes the cache
+```
+
+Use `cachedIds(key, where)` to cache the result of a list query keyed by an arbitrary string:
+
+```js
+const adminIds = await User.cachedIds('admins', { is_admin: true });
+```
+
+A record can be evicted manually:
+
+```js
+await user.clearSelfCache();
+```
+
+Override `updateCache(options)` on the model to perform additional cache invalidation alongside the automatic single-record eviction.
+
+This feature requires the Redis client wired up via `lib/redis`.
+
+
+## Hooks
+
+`BaseRecord` inherits Sequelize's full hook system. Register a hook on the model class with `addHook`:
+
+```js
+User.addHook('beforeCreate', async (user, options) => {
+    user.api_token = await generateToken();
 });
 
-const totalCount = await User.count({where: {team_id: 1}});
+User.addHook('afterSave', async (user, options) => {
+    await broadcast('users.changed', { id: user.id });
+});
 ```
-More examples can be found here, https://sequelize.org/docs/v6/core-concepts/model-querying-basics/.
 
-**Note** The `sequelize.fn`, `sequelize.col`, and `Op` properties have been added as helpers on the Model (e.g. `User.fn('MAX', User.col('id'), 'max_id')` and `User.Operators.or`)
+See [Sequelize hooks](https://sequelize.org/docs/v6/other-topics/hooks/) for the full list of hook points. The caching layer above uses `afterSave` and `afterDestroy` under the hood when `isCacheable` is true.
+
+
+## Associations
+
+Define associations using Sequelize's standard API after the model is initialized:
+
+```js
+User.initialize();
+Team.initialize();
 
+User.belongsTo(Team, { foreignKey: 'team_id' });
+Team.hasMany(User,   { foreignKey: 'team_id' });
+```
 
-### Associations
-**TODO**
+See [Sequelize associations](https://sequelize.org/docs/v6/core-concepts/assocs/) for the full reference.
 
 
-### Validators
-**TODO**
+## Validators
 
+Sequelize's column-level validators continue to work — declare them on the model's column definition or via the `validate` option:
 
-### Hooks
-**TODO**
+```js
+class User extends ActiveRecord.BaseRecord {
+    static get initializationOptions() {
+        const opts = super.initializationOptions;
+        opts.options.validate = {
+            adminMustHaveTeam() {
+                if (this.is_admin && !this.team_id) {
+                    throw new Error('admin users must belong to a team');
+                }
+            }
+        };
+        return opts;
+    }
+}
+```
 
+See [Sequelize validators](https://sequelize.org/docs/v6/core-concepts/validations-and-constraints/).
 
-### Paranoid
-**TODO**
 
+## TODO
 
-# Closing Connections
-When cleaning up your application, close all open connections with `User.closeAllConnections()`. This will close all connections for all models, not just the `User` model.
+- **Paranoid (soft delete).** The infrastructure for a `deleted_at` column is wired in `initializationOptions`, but `paranoid: true` is not set, so soft deletes are not yet usable out of the box.

package/lib/http/README.md

@@ -1,42 +1,107 @@
 # HttpClient
-The `HttpClient` is a generic client for making HTTP requests. It can be used on its own, but is best suited for creating subclasses intended for specific APIs. By overriding the `urlRoot` and `defaultHeaders` methods, an API client can be made that reduces duplicated code.
-```
+`HttpClient` is a generic client for making HTTP requests. It is best used as a base class for typed wrappers around a specific API: subclasses override `urlRoot` and `defaultHeaders` and expose one method per endpoint, which keeps third-party SDK details out of the rest of the codebase.
+
+
+## Core Concepts
+
+Every HTTP call is made through one of four verbs (`get`, `post`, `put`, `delete`) defined on `HttpClient`. Each verb takes an endpoint path and an options object; the client composes that with the subclass's `urlRoot` and `defaultHeaders`. Responses with a status under 300 resolve as `{ data, status }`. Anything else throws an `HttpError`; aborts throw an `HttpAbortError`.
+
+
+## Usage
+
+Subclass `HttpClient` and expose endpoint methods.
+
+```js
+const { HttpClient } = require('@vida-global/core');
+
 class MyApiClient extends HttpClient {
-    constructor({dev, token}) {
+    constructor({ dev, token }) {
+        super();
         this.#token = token;
         this.#dev   = dev;
     }
 
+    get urlRoot() {
+        return this.#dev ? 'https://staging.foo.com' : 'https://foo.com';
+    }
+
     get defaultHeaders() {
         const headers = super.defaultHeaders;
         headers.Authorization = `Bearer ${this.#token}`;
         return headers;
     }
 
-    get urlRoot() {
-        if (this.#dev) {
-            return "https://staging.foo.com";
-        } else {
-            return "https://foo.com";
-        }
+    async getTickets({ page, pageSize }) {
+        return await this.get('/tickets', { requestParams: { page, pageSize } });
     }
-}
 
-const client = new MyApiClient(true, '123abctoken');
-const requestParameters = {baz: 1, ban: 2};
-const response = await client.get("/foo/bar", {requestParameters});
+    async createTicket(ticket) {
+        return await this.post('/tickets', { body: ticket });
+    }
+}
+```
 
-const body = {baz: 1, ban: 2};
-const response = await client.post("/foo/bar", { body });
+```js
+const client   = new MyApiClient({ dev: true, token: '123abctoken' });
+const response = await client.getTickets({ page: 1, pageSize: 25 });
+// response = { data: <parsed JSON>, status: 200 }
 ```
 
+### Verbs and options
+
+| Method | Options |
+|---|---|
+| `get(endpoint, opts)` | `requestParams`, `headers`, `timeout`, `signal` |
+| `post(endpoint, opts)` | `body`, `headers`, `timeout`, `signal` |
+| `put(endpoint, opts)` | `body`, `headers`, `timeout`, `signal` |
+| `delete(endpoint, opts)` | `requestParams`, `headers`, `timeout`, `signal` |
+
+For `GET`/`DELETE`, `requestParams` becomes the query string. For `POST`/`PUT`, `body` becomes the request body (serialized as JSON by default).
+
+
+### Query string encoding
+
+For `GET`/`DELETE`, the way `requestParams` is appended to the URL depends on the `Content-Type` header:
+
+- `application/json` (the default) — non-primitive values are `JSON.stringify`'d into a single query-string value.
+- `application/x-www-form-urlencoded` — arrays expand to repeated keys (`?tag=a&tag=b`).
+
+Override `defaultHeaders` (or pass a one-off `headers` option) to switch encoding strategies.
+
+
 ## Aborting requests
-All request methods accept `timeout` (ms) and/or a caller-supplied `signal` (`AbortSignal`). When both are provided, they are composed via `AbortSignal.any` — whichever fires first aborts the request. If the request is aborted (by the timeout or the external signal), an `HttpAbortError` is thrown with `reason` set to either `'timed out'` or `'aborted'`.
 
-```
-await client.post("/foo/bar", { body, timeout: 5000 });
+All request methods accept `timeout` (ms) and/or a caller-supplied `signal` (`AbortSignal`). When both are provided they are composed via `AbortSignal.any` — whichever fires first aborts the request. If the request is aborted, an `HttpAbortError` is thrown with `reason` set to either `'timed out'` or `'aborted'`.
+
+```js
+await client.post('/foo/bar', { body, timeout: 5000 });
 
 const ctrl = new AbortController();
-await client.get("/foo/bar", { signal: ctrl.signal });
+await client.get('/foo/bar', { signal: ctrl.signal });
+ctrl.abort();
 ```
 
+
+## Errors
+
+Non-2xx responses throw `HttpError`. Inspect:
+
+| Property | Description |
+|---|---|
+| `err.status` | Numeric HTTP status. |
+| `err.statusText` | HTTP reason phrase. |
+| `err.url` | Full request URL. |
+| `err.responseHeaders` | Cloned response headers. |
+| `err.requestPayload` | Cloned outgoing payload (method, headers, body). |
+| `await err.data()` | Parsed response body — JSON when possible, raw text otherwise. |
+
+`HttpAbortError` exposes `err.reason` (`'timed out'` or `'aborted'`) and `err.requestPayload`.
+
+
+## Advanced
+
+Subclasses customize behavior through getters on the class:
+
+- `urlRoot` — base URL prepended to every endpoint.
+- `defaultHeaders` — merged into every request (per-call `headers` override).
+- `logger` — scoped logger used for the `API CALL: ...` debug line emitted on each request (defaults to `logger.http`).