@memberjunction/core 5.42.0 → 5.44.0

package/readme.md

@@ -852,18 +852,23 @@ export class MyEngine extends BaseEngine<MyEngine> {
         return super.getInstance<MyEngine>();
     }
 
-    public MyData: SomeEntity[] = [];
+    private _myData: SomeEntity[] = [];
 
-    protected get Config(): BaseEnginePropertyConfig[] {
-        return [
+    public get MyData(): SomeEntity[] {
+        return this.GetConfigData<SomeEntity>('_myData');
+    }
+
+    public async Config(forceRefresh?: boolean, contextUser?: UserInfo): Promise<void> {
+        const params: Partial<BaseEnginePropertyConfig>[] = [
             {
-                PropertyName: 'MyData',
+                PropertyName: '_myData',
                 EntityName: 'Some Entity',
                 Filter: 'IsActive = 1',
                 OrderBy: 'Name ASC',
                 AutoRefresh: true      // Auto-refresh on entity save/delete events
             }
         ];
+        return await this.Load(params, undefined, forceRefresh, contextUser);
     }
 }
 
@@ -879,6 +884,31 @@ Key features:
 - Local caching support via `CacheLocal` and `CacheLocalTTL` options
 - Supports both entity and dataset loading
 
+#### Permission-Constrained Loading
+
+When a user lacks read permissions on entities an engine loads, the engine enters a **permission-constrained** state instead of failing with errors or retrying endlessly. This is an all-or-nothing check — if any entity config is denied, all configs for that engine are skipped.
+
+The `GetConfigData<E>(propertyName)` method is the canonical way for engine getters to expose loaded data. It checks the data map for permission denial and throws a `PermissionConstrainedError` if the config was skipped, preventing consumers from silently operating on empty arrays.
+
+```typescript
+// Consumer that wants graceful degradation (optional feature)
+if (!AIEngineBase.Instance.IsPermissionConstrained) {
+    const models = AIEngineBase.Instance.Models;
+    // ... render AI features
+} else {
+    // ... hide AI features, show notice
+}
+
+// Consumer that requires the data (hard error if missing)
+const queries = QueryEngine.Instance.Queries; // throws PermissionConstrainedError if denied
+```
+
+| State | `Loaded` | `IsPermissionConstrained` | Behavior |
+|---|---|---|---|
+| Not loaded | `false` | `false` | `EnsureLoaded()` retries normally |
+| Loaded normally | `true` | `false` | Normal operation |
+| Permission-constrained | `true` | `true` | `GetConfigData()` throws `PermissionConstrainedError`, no retry, no entity event handling |
+
 ---
 
 ### BaseEngineRegistry — cross-engine cache reverse lookup
@@ -1383,6 +1413,39 @@ CodeNameFromString('First Name');          // 'FirstName'
 
 ---
 
+## Fire-and-Forget Entity Saves (`BaseEntitySaveQueue`)
+
+`BaseEntitySaveQueue` is the entity-aware façade over `@memberjunction/global`'s `KeyedSerialTaskQueue` for **non-blocking persistence** — writing observability/log rows (agent-run steps, action-execution logs, AI prompt runs, record-process details) without blocking the work that produced them on a DB round-trip.
+
+```typescript
+import { BaseEntitySaveQueue } from '@memberjunction/core';
+
+const queue = new BaseEntitySaveQueue();
+
+// Fire-and-forget INSERT of a freshly NewRecord()'d entity.
+queue.Insert(logEntity);
+
+// Fire-and-forget UPDATE chained after that entity's INSERT. The mutation runs INSIDE the
+// post-INSERT task, so the INSERT's finalizeSave reload can never revert it.
+queue.Update(logEntity, (e) => { e.Set('EndedAt', new Date()); e.Set('Status', 'Completed'); });
+
+// At a run/goal boundary, flush to await all pending saves + surface failure counts.
+const { failures } = await queue.Flush();
+```
+
+**Why the `Update(applyMutation)` shape matters.** A fire-and-forget INSERT serializes the entity's current fields and, on completion, `BaseEntity.finalizeSave` reloads the inserted row (`init()` + `SetMany`). Any field mutated on that same instance *while the INSERT is in flight* is reverted, and a force-persisted UPDATE then writes the stale values — the classic "stuck at Running" bug. Because the queue runs `applyMutation` **inside** the post-INSERT task, the mutation always lands after the reload, making that race **impossible by construction**.
+
+| Method | Purpose |
+|---|---|
+| `Insert(entity)` | Fire-and-forget create. The entity instance is the serialization key, so a later `Update` of the same instance waits for it. |
+| `Update(entity, applyMutation?)` | Fire-and-forget, force-persisted (`IgnoreDirtyState`) update chained after the INSERT; `applyMutation` runs post-INSERT (race-safe). |
+| `Flush()` | Await all pending saves; returns `{ failures, rejections }`. Call at a run/goal boundary. |
+| `new BaseEntitySaveQueue({ onError })` | Route failure messages to a structured logger (e.g. a category/metadata logger) instead of the default `LogError`. |
+
+Single-primary-key entities only; the queue logs (never throws) on a failed save, since these rows are observability and must not break the work that produced them.
+
+---
+
 ## Error Handling
 
 RunView and RunQuery do NOT throw exceptions on failure. Always check `Success`:
@@ -1514,6 +1577,37 @@ This library is written in TypeScript and provides full type definitions. All ge
 
 ISC License - see LICENSE file for details.
 
+## Remote Operations (the 4th Data Primitive)
+
+`BaseRemotableOperation<TInput, TOutput>` (defined in this package) is a typed, provider-routed server capability invoked from **one call site** on both the client (marshalled over GraphQL) and the server (in-process) — the missing peer of the three primitives MJCore already gives you:
+
+```mermaid
+graph LR
+    subgraph "MJ data primitives — one call site, provider-routed"
+        A["BaseEntity<br/><i>record CRUD</i>"]
+        B["RunView<br/><i>dynamic set reads</i>"]
+        C["RunQuery<br/><i>stored queries</i>"]
+        D["BaseRemotableOperation<br/><b>typed RPC</b>"]
+    end
+    style D fill:#8b5cf6,color:#fff,stroke:#6d28d9
+```
+
+`entity.Save()` · `rv.RunView()` · `rq.RunQuery()` · **`op.Execute()`** — same shape, same tier-agnostic DX.
+
+Before this primitive, exposing one non-CRUD capability ("render a template", "run a process") to the browser meant hand-writing a stack — a TypeGraphQL resolver, a typed GraphQL client (or an inline `gql` string + a provider cast), an Angular wrapper, **and** the input/output types twice (client + server), kept in sync by hand. A Remote Operation replaces all of it with one typed object:
+
+```typescript
+// typed in, typed out — identical on client and server; a wrong field is a compile error
+const result = await new TemplateRunOperation().Execute({ templateID, data });
+result.Output?.output;
+```
+
+New operations are declared as `MJ: Remote Operations` metadata rows; CodeGen emits the typed base, and the body is written by hand (**Manual**), authored by an LLM from the row's `Description` and approved (**AI**), or left as emitted boilerplate (**Default**). Transport, auth, the long-running progress channel, and approval gating are written **once** in the framework and shared by every operation.
+
+> **Visual before/after**: See the [**Remote Operations Showcase**](./docs/REMOTE_OPERATIONS_SHOWCASE.md) — a diagram-driven tour of the layers this removes, built from two real migrations. *(Best starting point for sharing with the team.)*
+>
+> **Full Guide**: See the [**Remote Operations Guide**](../../guides/REMOTE_OPERATIONS_GUIDE.md) for when to use it (vs. an Action or a bespoke resolver), the three authoring modes, calling conventions, the auth chain, and long-running progress.
+
 ## Virtual Entities
 
 Virtual entities are **read-only entities backed by SQL views** rather than physical database tables. They appear in the metadata catalog alongside regular entities but have no underlying base table — only a base view. This makes them ideal for exposing aggregated data, cross-database views, or complex computed datasets as first-class entities.
@@ -1718,6 +1812,56 @@ const params = EntityInfo.BuildOrganicKeyViewParams(record, relatedEntity, organ
 
 > **Full Guide**: See [Organic Keys Guide](./docs/organic-keys.md) for the complete schema, all 4 query patterns, normalization strategies, CodeGen configuration, Angular UI integration, and an end-to-end setup walkthrough.
 
+## Entity Field Rules
+
+`EntityFieldRules` is the **metadata-aware** layer on top of the pure field-rules engine in
+[`@memberjunction/global`](../MJGlobal/README.md#field-rules-engine). The pure engine is deliberately
+metadata-blind — it computes a per-field diff from a plain `Record<string, unknown>` and an injected
+lookup resolver, so it runs anywhere. `EntityFieldRules` adds the things that only make sense when the
+**target is a real MJ entity** and that need this package's metadata layer:
+
+| Adds | Why it needs core |
+|---|---|
+| **`Validate(entityName, ruleSet)`** — target field exists? writable (not PK/read-only/virtual)? source `field` refs valid? | `EntityInfo` / `EntityFieldInfo` |
+| **Type coercion** — a formula yielding `"42"` becomes numeric `42` for a numeric column | `EntityFieldInfo.TSType` |
+| **Built-in lookup resolver** for `lookup` rule sources | `RunView` |
+| **`ApplyToEntity(entity, ruleSet, { DryRun })`** — write the computed values + `Save()` (Record Changes captures before/after) | `BaseEntity` |
+
+**Scope:** the *target is always an MJ entity*; the *source* may be the entity's own fields plus an
+optional injected `Context` (a data context, a query result, an agent's output, related-entity lookups)
+— all data you already hold. When the *other side* is a **live external system**, that is the domain of
+[`@memberjunction/integration`](../Integration/engine/README.md#field-mapping--the-shared-transform-engine),
+which uses the same pure transform engine. `EntityFieldRules` is a writer *to* entities, not a
+bidirectional mapper.
+
+```ts
+import { EntityFieldRules } from '@memberjunction/core';
+import type { FieldRuleSet } from '@memberjunction/global';
+
+const ruleSet: FieldRuleSet = {
+    Rules: [
+        { TargetField: 'Description', Source: { Kind: 'formula', Expression: "fields.Name + ' (normalized)'" } },
+        { TargetField: 'Status', Source: { Kind: 'static', Value: 'Inactive' }, Condition: 'DaysSinceActivity > 365' },
+    ],
+};
+
+// 1) Pre-flight (synchronous, safe to run in a UX on every edit)
+const check = EntityFieldRules.Validate('Accounts', ruleSet);
+if (!check.Valid) console.warn(check.Errors);
+
+// 2) Dry-run preview (computes the diff, writes nothing)
+const rules = new EntityFieldRules(contextUser);
+const preview = await rules.ApplyToEntity(account, ruleSet, { DryRun: true });
+// preview.Changes → per-field old → new; preview.Saved === false
+
+// 3) Apply for real (writes + Save → Record Changes versioning)
+const result = await rules.ApplyToEntity(account, ruleSet);
+```
+
+> For **bulk** updates across a view / list / filtered set, the `FieldRulesProcessor` in
+> `@memberjunction/record-set-processor` runs `EntityFieldRules` per record with batching, concurrency,
+> and dry-run — that's the rules-based bulk-update tool.
+
 ## Documentation
 
 For detailed guides on specific topics, see the [docs/](./docs/) folder: