@memberjunction/record-set-processor-base 0.0.1 → 5.42.0

package/README.md

@@ -1,45 +1,53 @@
 # @memberjunction/record-set-processor-base
 
-## ⚠️ IMPORTANT NOTICE ⚠️
-
-**This package is created solely for the purpose of setting up OIDC (OpenID Connect) trusted publishing with npm.**
-
-This is **NOT** a functional package and contains **NO** code or functionality beyond the OIDC setup configuration.
-
-## Purpose
-
-This package exists to:
-1. Configure OIDC trusted publishing for the package name `@memberjunction/record-set-processor-base`
-2. Enable secure, token-less publishing from CI/CD workflows
-3. Establish provenance for packages published under this name
-
-## What is OIDC Trusted Publishing?
-
-OIDC trusted publishing allows package maintainers to publish packages directly from their CI/CD workflows without needing to manage npm access tokens. Instead, it uses OpenID Connect to establish trust between the CI/CD provider (like GitHub Actions) and npm.
-
-## Setup Instructions
-
-To properly configure OIDC trusted publishing for this package:
-
-1. Go to [npmjs.com](https://www.npmjs.com/) and navigate to your package settings
-2. Configure the trusted publisher (e.g., GitHub Actions)
-3. Specify the repository and workflow that should be allowed to publish
-4. Use the configured workflow to publish your actual package
-
-## DO NOT USE THIS PACKAGE
-
-This package is a placeholder for OIDC configuration only. It:
-- Contains no executable code
-- Provides no functionality
-- Should not be installed as a dependency
-- Exists only for administrative purposes
-
-## More Information
-
-For more details about npm's trusted publishing feature, see:
-- [npm Trusted Publishing Documentation](https://docs.npmjs.com/generating-provenance-statements)
-- [GitHub Actions OIDC Documentation](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect)
-
----
-
-**Maintained for OIDC setup purposes only**
+Client-safe foundation for the MemberJunction **Record Set Processing** substrate: the shared types,
+the three pluggable seams, and the built-in source adapters. The server engine that drives these
+lives in [`@memberjunction/record-set-processor`](../engine).
+
+## What's in here
+
+| Export | Kind | Purpose |
+|---|---|---|
+| `IRecordSetSource` | seam | yields the record set in cursor-paginated batches |
+| `IRecordProcessor` | seam | does the work for a single record |
+| `IProcessRunTracker` | seam | persists run lifecycle, per-record detail, checkpoints, pause/cancel |
+| `RecordSetProcessOptions` | type | the engine's per-run options |
+| `RecordRef`, `RecordBatch`, `ProcessCursor`, `RecordResult`, `ProcessRunResult`, … | types | the data shapes shared across the seams |
+| `ArraySource`, `ViewSource`, `ListSource`, `FilterSource`, `KeysetSource` | sources | the built-in record-set sources |
+
+## The three seams
+
+A processing job is a composition of three independent choices:
+
+- **Source** — *what* records to iterate (a User View, a List, an ad-hoc filter, an in-memory array,
+  or a keyset sweep). Sources paginate themselves and hand back an opaque `ProcessCursor` the engine
+  round-trips for resume.
+- **Processor** — *what to do* with each record (an Action, an Agent, an Infer-&-Write-Back step, or
+  a function), returning a `RecordResult` (`Succeeded` / `Failed` / `Skipped`).
+- **Tracker** — *where to persist* run + per-record audit. The default writes the generic
+  `MJ: Process Runs` / `MJ: Process Run Details` tables; domain consumers can supply their own.
+
+## Source adapters
+
+| Source | Pagination | Notes |
+|---|---|---|
+| `ArraySource` | offset (in-memory) | a fixed list of `RecordRef`; also use for `SingleRecord` scopes |
+| `ViewSource` | offset | resolves a saved User View; views may carry arbitrary order/filter so offset is used |
+| `ListSource` | offset | iterates a List's members via `MJ: List Details` |
+| `FilterSource` | keyset → offset | entity + ad-hoc WHERE; keyset when the entity has a single orderable PK |
+| `KeysetSource` | keyset (required) | like `FilterSource` but asserts keyset eligibility — for large background sweeps |
+
+## Implementing a source
+
+```typescript
+import { IRecordSetSource, RecordBatch, ProcessCursor, SourceDescriptor } from '@memberjunction/record-set-processor-base';
+
+export class MySource implements IRecordSetSource {
+    Describe(): SourceDescriptor { return { SourceType: 'Filter', SourceFilter: '...' }; }
+    async NextBatch(cursor: ProcessCursor | undefined, batchSize: number, contextUser, provider): Promise<RecordBatch> {
+        // fetch up to `batchSize` records after `cursor`, return them plus the next cursor + exhausted flag
+    }
+}
+```
+
+This package carries no server-only dependencies and is safe to import on the client.

package/dist/index.d.ts

@@ -0,0 +1,9 @@
+/**
+ * @fileoverview Public API for the Record Set Processor base package — client-safe types, the three
+ * pluggable seams (source / processor / tracker), and the built-in source adapters.
+ * @module @memberjunction/record-set-processor-base
+ */
+export * from './types.js';
+export * from './interfaces.js';
+export * from './sources/index.js';
+//# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,cAAc,SAAS,CAAC;AACxB,cAAc,cAAc,CAAC;AAC7B,cAAc,WAAW,CAAC"}

package/dist/index.js

@@ -0,0 +1,9 @@
+/**
+ * @fileoverview Public API for the Record Set Processor base package — client-safe types, the three
+ * pluggable seams (source / processor / tracker), and the built-in source adapters.
+ * @module @memberjunction/record-set-processor-base
+ */
+export * from './types.js';
+export * from './interfaces.js';
+export * from './sources/index.js';
+//# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,cAAc,SAAS,CAAC;AACxB,cAAc,cAAc,CAAC;AAC7B,cAAc,WAAW,CAAC"}

package/dist/interfaces.d.ts

@@ -0,0 +1,165 @@
+/**
+ * @fileoverview The three pluggable seams of the Record Set Processor — source, processor, and
+ * persistence tracker — plus the engine's options shape. The engine ({@link
+ * @memberjunction/record-set-processor}) routes every set-iterating job through these interfaces.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { IMetadataProvider, UserInfo } from '@memberjunction/core';
+import { ProcessCursor, ProcessRunSummary, ProgressInfo, RecordBatch, RecordRef, RecordResult, RunCounts, SourceTypeValue, TriggeredByValue } from './types.js';
+/** Lightweight, persistable description of a source — used to populate the run header. */
+export interface SourceDescriptor {
+    /** The kind of source (View / List / Filter / Array / Keyset / SingleRecord). */
+    SourceType: SourceTypeValue;
+    /** The source's defining ID (ViewID or ListID) when applicable. */
+    SourceID?: string;
+    /** The source's resolved filter snapshot, when applicable. */
+    SourceFilter?: string;
+    /** The target entity ID, when the source knows it up-front. */
+    EntityID?: string;
+}
+/**
+ * SOURCE seam — yields the record set in cursor-paginated batches. Implementations decide their own
+ * pagination strategy (offset vs. keyset); the cursor is opaque to the engine and round-tripped
+ * through the tracker for resume.
+ */
+export interface IRecordSetSource {
+    /**
+     * Returns the next batch of records after the given cursor.
+     * @param cursor - The cursor returned by the previous batch, or `undefined` to start from the beginning.
+     * @param batchSize - Maximum number of records to return.
+     * @param contextUser - The acting user (required server-side for permissions).
+     * @param provider - Optional metadata provider for entity resolution.
+     * @returns A batch with the records, the next cursor, and an exhausted flag.
+     */
+    NextBatch(cursor: ProcessCursor | undefined, batchSize: number, contextUser: UserInfo, provider?: IMetadataProvider): Promise<RecordBatch>;
+    /** Returns a persistable description of this source for the run header. */
+    Describe(): SourceDescriptor;
+}
+/** Per-record execution context handed to an {@link IRecordProcessor}. */
+export interface RecordProcessorContext {
+    /** The acting user. */
+    contextUser: UserInfo;
+    /** The owning provider — use this for data access (never `new Metadata()`), per the multi-provider rule. */
+    provider: IMetadataProvider;
+    /** ID of the current process run, when one is being tracked. */
+    processRunID?: string;
+}
+/**
+ * PROCESSOR (executor) seam — does the work for a single record and returns its outcome. Concrete
+ * processors wrap an Action, an Agent (via Execute Agent), an Infer-&-Write-Back step, or a plain
+ * function.
+ */
+export interface IRecordProcessor {
+    /**
+     * Processes one record.
+     * @param record - The record to process.
+     * @param context - The per-record execution context.
+     * @returns The record's outcome (Succeeded / Failed / Skipped) plus any payload / trace links.
+     */
+    ProcessRecord(record: RecordRef, context: RecordProcessorContext): Promise<RecordResult>;
+}
+/** Opaque handle returned by a tracker's `BeginRun`, threaded back through the other tracker calls. */
+export interface RunHandle {
+    /** ID of the persisted run, when the tracker persists one. */
+    ProcessRunID?: string;
+    /** Tracker-private state. */
+    [key: string]: unknown;
+}
+/** Metadata describing a run, supplied to a tracker's `BeginRun`. */
+export interface ProcessRunMeta {
+    /** FK to the originating Record Process definition (NULL for ad-hoc / engine-driven runs). */
+    RecordProcessID?: string;
+    /** FK to the owning `ScheduledJobRun` when launched by the scheduler (NULL otherwise). */
+    ScheduledJobRunID?: string;
+    /** FK to the target entity. */
+    EntityID?: string;
+    /** What triggered the run. */
+    TriggeredBy: TriggeredByValue;
+    /** The kind of source. */
+    SourceType: SourceTypeValue;
+    /** ViewID / ListID, when applicable. */
+    SourceID?: string;
+    /** Resolved filter snapshot, when applicable. */
+    SourceFilter?: string;
+    /** Effective batch size. */
+    BatchSize?: number;
+    /** Known total record count, or null. */
+    TotalItemCount?: number | null;
+    /** JSON-serializable snapshot of the effective configuration. */
+    Configuration?: unknown;
+}
+/**
+ * PERSISTENCE seam — records run lifecycle, per-record detail, checkpoints, and the pause/cancel
+ * handshake. The default `GenericProcessRunTracker` (in the engine package) writes `MJ: Process
+ * Runs` / `MJ: Process Run Details`; domain consumers can supply their own (e.g. the classifier's
+ * Content Process Runs), and `NoOpTracker` discards everything for fire-and-forget single-record work.
+ */
+export interface IProcessRunTracker {
+    /** Begins a run and returns a handle threaded through the remaining calls. */
+    BeginRun(meta: ProcessRunMeta, contextUser: UserInfo, provider?: IMetadataProvider): Promise<RunHandle>;
+    /** Records the outcome of a single processed record. */
+    RecordResult(handle: RunHandle, record: RecordRef, result: RecordResult, contextUser: UserInfo, provider?: IMetadataProvider): Promise<void>;
+    /**
+     * Persists the resume cursor + running counts after a batch and reports whether to continue.
+     * Implementations that support pause/cancel re-check their backing row's cancellation flag here
+     * and return `false` to request a graceful pause.
+     * @returns `true` to continue, `false` to pause/stop.
+     */
+    Checkpoint(handle: RunHandle, cursor: ProcessCursor, counts: RunCounts, contextUser: UserInfo, provider?: IMetadataProvider): Promise<boolean>;
+    /** Finalizes the run with its terminal status + summary counts. */
+    CompleteRun(handle: RunHandle, summary: ProcessRunSummary, contextUser: UserInfo, provider?: IMetadataProvider): Promise<void>;
+    /** Loads the resume cursor for a run being resumed, or `undefined` to start from the beginning. */
+    LoadResumeCursor(handle: RunHandle, contextUser: UserInfo, provider?: IMetadataProvider): Promise<ProcessCursor | undefined>;
+}
+/** Options for a single `RecordSetProcessor.Process()` invocation. */
+export interface RecordSetProcessOptions {
+    /** The record-set source to iterate. */
+    source: IRecordSetSource;
+    /** The per-record processor (executor). */
+    processor: IRecordProcessor;
+    /** Persistence tracker (defaults to `GenericProcessRunTracker` in the engine). */
+    tracker?: IProcessRunTracker;
+    /** The acting user (required). */
+    contextUser: UserInfo;
+    /** The owning provider; defaults to the active provider when omitted. */
+    provider?: IMetadataProvider;
+    /** Optional FK to the originating Record Process definition. */
+    recordProcessID?: string;
+    /** Optional FK to the owning `ScheduledJobRun` when launched by the scheduler. */
+    scheduledJobRunID?: string;
+    /** Optional target entity ID (otherwise derived from the source). */
+    entityID?: string;
+    /** What triggered the run (default `Manual`). */
+    triggeredBy?: TriggeredByValue;
+    /** Records per batch (default 100). */
+    batchSize?: number;
+    /** Maximum records processed concurrently within a batch (default 1). */
+    maxConcurrency?: number;
+    /** Error-rate circuit breaker threshold, as a percentage (default 20). */
+    errorThresholdPercent?: number;
+    /** Delay between batches in milliseconds (default 0). */
+    delayBetweenBatchesMs?: number;
+    /** Optional rate-limit applied once per batch before processing. */
+    rateLimit?: {
+        requestsPerMinute?: number;
+        tokensPerMinute?: number;
+    };
+    /** Whether to resume from a prior checkpoint when one exists (default true). */
+    resume?: boolean;
+    /** Hard cap on the number of records processed this run. */
+    maxRecords?: number;
+    /** Progress callback invoked after each batch. */
+    onProgress?: (progress: ProgressInfo) => void;
+    /**
+     * Budget-gate hook invoked after each batch. Return `{ continue: false, reason }` to pause the
+     * run (e.g. when a per-run cost/item budget is exhausted).
+     */
+    onAfterBatch?: (batch: RecordRef[], processed: number) => Promise<{
+        continue: boolean;
+        reason?: string;
+    } | null>;
+    /** JSON-serializable snapshot of configuration, persisted on the run header. */
+    configuration?: unknown;
+}
+export type { ProgressInfo, ProcessRunSummary };
+//# sourceMappingURL=interfaces.d.ts.map

package/dist/interfaces.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"interfaces.d.ts","sourceRoot":"","sources":["../src/interfaces.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAE,iBAAiB,EAAE,QAAQ,EAAE,MAAM,sBAAsB,CAAC;AACnE,OAAO,EACH,aAAa,EACb,iBAAiB,EACjB,YAAY,EACZ,WAAW,EACX,SAAS,EACT,YAAY,EACZ,SAAS,EACT,eAAe,EACf,gBAAgB,EACnB,MAAM,SAAS,CAAC;AAEjB,0FAA0F;AAC1F,MAAM,WAAW,gBAAgB;IAC7B,iFAAiF;IACjF,UAAU,EAAE,eAAe,CAAC;IAC5B,mEAAmE;IACnE,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,8DAA8D;IAC9D,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,+DAA+D;IAC/D,QAAQ,CAAC,EAAE,MAAM,CAAC;CACrB;AAED;;;;GAIG;AACH,MAAM,WAAW,gBAAgB;IAC7B;;;;;;;OAOG;IACH,SAAS,CAAC,MAAM,EAAE,aAAa,GAAG,SAAS,EAAE,SAAS,EAAE,MAAM,EAAE,WAAW,EAAE,QAAQ,EAAE,QAAQ,CAAC,EAAE,iBAAiB,GAAG,OAAO,CAAC,WAAW,CAAC,CAAC;IAE3I,2EAA2E;IAC3E,QAAQ,IAAI,gBAAgB,CAAC;CAChC;AAED,0EAA0E;AAC1E,MAAM,WAAW,sBAAsB;IACnC,uBAAuB;IACvB,WAAW,EAAE,QAAQ,CAAC;IACtB,4GAA4G;IAC5G,QAAQ,EAAE,iBAAiB,CAAC;IAC5B,gEAAgE;IAChE,YAAY,CAAC,EAAE,MAAM,CAAC;CACzB;AAED;;;;GAIG;AACH,MAAM,WAAW,gBAAgB;IAC7B;;;;;OAKG;IACH,aAAa,CAAC,MAAM,EAAE,SAAS,EAAE,OAAO,EAAE,sBAAsB,GAAG,OAAO,CAAC,YAAY,CAAC,CAAC;CAC5F;AAED,uGAAuG;AACvG,MAAM,WAAW,SAAS;IACtB,8DAA8D;IAC9D,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,6BAA6B;IAC7B,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CAC1B;AAED,qEAAqE;AACrE,MAAM,WAAW,cAAc;IAC3B,8FAA8F;IAC9F,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,0FAA0F;IAC1F,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,+BAA+B;IAC/B,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,8BAA8B;IAC9B,WAAW,EAAE,gBAAgB,CAAC;IAC9B,0BAA0B;IAC1B,UAAU,EAAE,eAAe,CAAC;IAC5B,wCAAwC;IACxC,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,iDAAiD;IACjD,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,4BAA4B;IAC5B,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,yCAAyC;IACzC,cAAc,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC/B,iEAAiE;IACjE,aAAa,CAAC,EAAE,OAAO,CAAC;CAC3B;AAED;;;;;GAKG;AACH,MAAM,WAAW,kBAAkB;IAC/B,8EAA8E;IAC9E,QAAQ,CAAC,IAAI,EAAE,cAAc,EAAE,WAAW,EAAE,QAAQ,EAAE,QAAQ,CAAC,EAAE,iBAAiB,GAAG,OAAO,CAAC,SAAS,CAAC,CAAC;IAExG,wDAAwD;IACxD,YAAY,CAAC,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,YAAY,EAAE,WAAW,EAAE,QAAQ,EAAE,QAAQ,CAAC,EAAE,iBAAiB,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IAE7I;;;;;OAKG;IACH,UAAU,CAAC,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,aAAa,EAAE,MAAM,EAAE,SAAS,EAAE,WAAW,EAAE,QAAQ,EAAE,QAAQ,CAAC,EAAE,iBAAiB,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC;IAE/I,mEAAmE;IACnE,WAAW,CAAC,MAAM,EAAE,SAAS,EAAE,OAAO,EAAE,iBAAiB,EAAE,WAAW,EAAE,QAAQ,EAAE,QAAQ,CAAC,EAAE,iBAAiB,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IAE/H,mGAAmG;IACnG,gBAAgB,CAAC,MAAM,EAAE,SAAS,EAAE,WAAW,EAAE,QAAQ,EAAE,QAAQ,CAAC,EAAE,iBAAiB,GAAG,OAAO,CAAC,aAAa,GAAG,SAAS,CAAC,CAAC;CAChI;AAED,sEAAsE;AACtE,MAAM,WAAW,uBAAuB;IACpC,wCAAwC;IACxC,MAAM,EAAE,gBAAgB,CAAC;IACzB,2CAA2C;IAC3C,SAAS,EAAE,gBAAgB,CAAC;IAC5B,kFAAkF;IAClF,OAAO,CAAC,EAAE,kBAAkB,CAAC;IAC7B,kCAAkC;IAClC,WAAW,EAAE,QAAQ,CAAC;IACtB,yEAAyE;IACzE,QAAQ,CAAC,EAAE,iBAAiB,CAAC;IAC7B,gEAAgE;IAChE,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,kFAAkF;IAClF,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,qEAAqE;IACrE,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,iDAAiD;IACjD,WAAW,CAAC,EAAE,gBAAgB,CAAC;IAC/B,uCAAuC;IACvC,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,yEAAyE;IACzE,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,0EAA0E;IAC1E,qBAAqB,CAAC,EAAE,MAAM,CAAC;IAC/B,yDAAyD;IACzD,qBAAqB,CAAC,EAAE,MAAM,CAAC;IAC/B,oEAAoE;IACpE,SAAS,CAAC,EAAE;QAAE,iBAAiB,CAAC,EAAE,MAAM,CAAC;QAAC,eAAe,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IACrE,gFAAgF;IAChF,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,4DAA4D;IAC5D,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,kDAAkD;IAClD,UAAU,CAAC,EAAE,CAAC,QAAQ,EAAE,YAAY,KAAK,IAAI,CAAC;IAC9C;;;OAGG;IACH,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,SAAS,EAAE,EAAE,SAAS,EAAE,MAAM,KAAK,OAAO,CAAC;QAAE,QAAQ,EAAE,OAAO,CAAC;QAAC,MAAM,CAAC,EAAE,MAAM,CAAA;KAAE,GAAG,IAAI,CAAC,CAAC;IACjH,gFAAgF;IAChF,aAAa,CAAC,EAAE,OAAO,CAAC;CAC3B;AAGD,YAAY,EAAE,YAAY,EAAE,iBAAiB,EAAE,CAAC"}

package/dist/interfaces.js

@@ -0,0 +1,8 @@
+/**
+ * @fileoverview The three pluggable seams of the Record Set Processor — source, processor, and
+ * persistence tracker — plus the engine's options shape. The engine ({@link
+ * @memberjunction/record-set-processor}) routes every set-iterating job through these interfaces.
+ * @module @memberjunction/record-set-processor-base
+ */
+export {};
+//# sourceMappingURL=interfaces.js.map

package/dist/interfaces.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"interfaces.js","sourceRoot":"","sources":["../src/interfaces.ts"],"names":[],"mappings":"AAAA;;;;;GAKG"}

package/dist/sources/arraySource.d.ts

@@ -0,0 +1,22 @@
+/**
+ * @fileoverview In-memory array source — yields a fixed list of record references in batches.
+ * Useful for `SingleRecord` / explicit-array scopes and for tests.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { IRecordSetSource, SourceDescriptor } from '../interfaces.js';
+import { ProcessCursor, RecordBatch, RecordRef, SourceTypeValue } from '../types.js';
+/** A source backed by an in-memory array of {@link RecordRef}. */
+export declare class ArraySource implements IRecordSetSource {
+    private readonly records;
+    private readonly entityID?;
+    private readonly sourceType;
+    /**
+     * @param records - The records to yield.
+     * @param entityID - Optional entity ID for the run header (otherwise taken from the first record).
+     * @param sourceType - Reported source type (default `Array`; pass `SingleRecord` for a one-record scope).
+     */
+    constructor(records: RecordRef[], entityID?: string, sourceType?: SourceTypeValue);
+    Describe(): SourceDescriptor;
+    NextBatch(cursor: ProcessCursor | undefined, batchSize: number): Promise<RecordBatch>;
+}
+//# sourceMappingURL=arraySource.d.ts.map

package/dist/sources/arraySource.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"arraySource.d.ts","sourceRoot":"","sources":["../../src/sources/arraySource.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,MAAM,eAAe,CAAC;AACnE,OAAO,EAAE,aAAa,EAAE,WAAW,EAAE,SAAS,EAAE,eAAe,EAAE,MAAM,UAAU,CAAC;AAElF,kEAAkE;AAClE,qBAAa,WAAY,YAAW,gBAAgB;IAO5C,OAAO,CAAC,QAAQ,CAAC,OAAO;IACxB,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC;IAC1B,OAAO,CAAC,QAAQ,CAAC,UAAU;IAR/B;;;;OAIG;gBAEkB,OAAO,EAAE,SAAS,EAAE,EACpB,QAAQ,CAAC,EAAE,MAAM,EACjB,UAAU,GAAE,eAAyB;IAGnD,QAAQ,IAAI,gBAAgB;IAItB,SAAS,CAAC,MAAM,EAAE,aAAa,GAAG,SAAS,EAAE,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,WAAW,CAAC;CAWrG"}

package/dist/sources/arraySource.js

@@ -0,0 +1,33 @@
+/**
+ * @fileoverview In-memory array source — yields a fixed list of record references in batches.
+ * Useful for `SingleRecord` / explicit-array scopes and for tests.
+ * @module @memberjunction/record-set-processor-base
+ */
+/** A source backed by an in-memory array of {@link RecordRef}. */
+export class ArraySource {
+    /**
+     * @param records - The records to yield.
+     * @param entityID - Optional entity ID for the run header (otherwise taken from the first record).
+     * @param sourceType - Reported source type (default `Array`; pass `SingleRecord` for a one-record scope).
+     */
+    constructor(records, entityID, sourceType = 'Array') {
+        this.records = records;
+        this.entityID = entityID;
+        this.sourceType = sourceType;
+    }
+    Describe() {
+        return { SourceType: this.sourceType, EntityID: this.entityID ?? this.records[0]?.EntityID };
+    }
+    async NextBatch(cursor, batchSize) {
+        const offset = cursor?.Offset ?? 0;
+        const slice = this.records.slice(offset, offset + batchSize);
+        const next = offset + slice.length;
+        return {
+            Records: slice,
+            NextCursor: { Offset: next },
+            Exhausted: next >= this.records.length,
+            TotalRowCount: this.records.length,
+        };
+    }
+}
+//# sourceMappingURL=arraySource.js.map

package/dist/sources/arraySource.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"arraySource.js","sourceRoot":"","sources":["../../src/sources/arraySource.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAKH,kEAAkE;AAClE,MAAM,OAAO,WAAW;IACpB;;;;OAIG;IACH,YACqB,OAAoB,EACpB,QAAiB,EACjB,aAA8B,OAAO;QAFrC,YAAO,GAAP,OAAO,CAAa;QACpB,aAAQ,GAAR,QAAQ,CAAS;QACjB,eAAU,GAAV,UAAU,CAA2B;IACvD,CAAC;IAEG,QAAQ;QACX,OAAO,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,EAAE,QAAQ,EAAE,IAAI,CAAC,QAAQ,IAAI,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,CAAC;IACjG,CAAC;IAEM,KAAK,CAAC,SAAS,CAAC,MAAiC,EAAE,SAAiB;QACvE,MAAM,MAAM,GAAG,MAAM,EAAE,MAAM,IAAI,CAAC,CAAC;QACnC,MAAM,KAAK,GAAG,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,MAAM,EAAE,MAAM,GAAG,SAAS,CAAC,CAAC;QAC7D,MAAM,IAAI,GAAG,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC;QACnC,OAAO;YACH,OAAO,EAAE,KAAK;YACd,UAAU,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE;YAC5B,SAAS,EAAE,IAAI,IAAI,IAAI,CAAC,OAAO,CAAC,MAAM;YACtC,aAAa,EAAE,IAAI,CAAC,OAAO,CAAC,MAAM;SACrC,CAAC;IACN,CAAC;CACJ"}

package/dist/sources/filterSource.d.ts

@@ -0,0 +1,23 @@
+/**
+ * @fileoverview Filter source — iterates an entity's rows matching an ad-hoc WHERE clause. Uses
+ * keyset (seek) pagination when the entity has a single orderable PK, otherwise offset.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { EntityInfo, IMetadataProvider, UserInfo } from '@memberjunction/core';
+import { IRecordSetSource, SourceDescriptor } from '../interfaces.js';
+import { ProcessCursor, RecordBatch } from '../types.js';
+/** A source backed by an entity + ad-hoc filter. */
+export declare class FilterSource implements IRecordSetSource {
+    private readonly entityName;
+    private readonly filter?;
+    private entity?;
+    /**
+     * @param entityName - The target entity name.
+     * @param filter - Optional WHERE clause (no leading `WHERE`).
+     */
+    constructor(entityName: string, filter?: string);
+    Describe(): SourceDescriptor;
+    protected resolveEntity(provider?: IMetadataProvider): EntityInfo;
+    NextBatch(cursor: ProcessCursor | undefined, batchSize: number, contextUser: UserInfo, provider?: IMetadataProvider): Promise<RecordBatch>;
+}
+//# sourceMappingURL=filterSource.d.ts.map

package/dist/sources/filterSource.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"filterSource.d.ts","sourceRoot":"","sources":["../../src/sources/filterSource.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAE,UAAU,EAAE,iBAAiB,EAAY,QAAQ,EAAE,MAAM,sBAAsB,CAAC;AACzF,OAAO,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,MAAM,eAAe,CAAC;AACnE,OAAO,EAAE,aAAa,EAAE,WAAW,EAAE,MAAM,UAAU,CAAC;AAGtD,oDAAoD;AACpD,qBAAa,YAAa,YAAW,gBAAgB;IAOrC,OAAO,CAAC,QAAQ,CAAC,UAAU;IAAU,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAC;IANzE,OAAO,CAAC,MAAM,CAAC,CAAa;IAE5B;;;OAGG;gBAC0B,UAAU,EAAE,MAAM,EAAmB,MAAM,CAAC,EAAE,MAAM;IAE1E,QAAQ,IAAI,gBAAgB;IAInC,SAAS,CAAC,aAAa,CAAC,QAAQ,CAAC,EAAE,iBAAiB,GAAG,UAAU;IAapD,SAAS,CAAC,MAAM,EAAE,aAAa,GAAG,SAAS,EAAE,SAAS,EAAE,MAAM,EAAE,WAAW,EAAE,QAAQ,EAAE,QAAQ,CAAC,EAAE,iBAAiB,GAAG,OAAO,CAAC,WAAW,CAAC;CAI1J"}

package/dist/sources/filterSource.js

@@ -0,0 +1,38 @@
+/**
+ * @fileoverview Filter source — iterates an entity's rows matching an ad-hoc WHERE clause. Uses
+ * keyset (seek) pagination when the entity has a single orderable PK, otherwise offset.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { Metadata } from '@memberjunction/core';
+import { pageEntityByFilter } from './sourceUtil.js';
+/** A source backed by an entity + ad-hoc filter. */
+export class FilterSource {
+    /**
+     * @param entityName - The target entity name.
+     * @param filter - Optional WHERE clause (no leading `WHERE`).
+     */
+    constructor(entityName, filter) {
+        this.entityName = entityName;
+        this.filter = filter;
+    }
+    Describe() {
+        return { SourceType: 'Filter', SourceFilter: this.filter, EntityID: this.entity?.ID };
+    }
+    resolveEntity(provider) {
+        if (this.entity) {
+            return this.entity;
+        }
+        const md = provider ?? Metadata.Provider;
+        const entity = md.EntityByName(this.entityName);
+        if (!entity) {
+            throw new Error(`FilterSource: entity '${this.entityName}' not found in metadata`);
+        }
+        this.entity = entity;
+        return entity;
+    }
+    async NextBatch(cursor, batchSize, contextUser, provider) {
+        const entity = this.resolveEntity(provider);
+        return pageEntityByFilter({ entity, filter: this.filter, cursor, batchSize, contextUser, preferKeyset: true });
+    }
+}
+//# sourceMappingURL=filterSource.js.map

package/dist/sources/filterSource.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"filterSource.js","sourceRoot":"","sources":["../../src/sources/filterSource.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAiC,QAAQ,EAAY,MAAM,sBAAsB,CAAC;AAGzF,OAAO,EAAE,kBAAkB,EAAE,MAAM,cAAc,CAAC;AAElD,oDAAoD;AACpD,MAAM,OAAO,YAAY;IAGrB;;;OAGG;IACH,YAA6B,UAAkB,EAAmB,MAAe;QAApD,eAAU,GAAV,UAAU,CAAQ;QAAmB,WAAM,GAAN,MAAM,CAAS;IAAG,CAAC;IAE9E,QAAQ;QACX,OAAO,EAAE,UAAU,EAAE,QAAQ,EAAE,YAAY,EAAE,IAAI,CAAC,MAAM,EAAE,QAAQ,EAAE,IAAI,CAAC,MAAM,EAAE,EAAE,EAAE,CAAC;IAC1F,CAAC;IAES,aAAa,CAAC,QAA4B;QAChD,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;YACd,OAAO,IAAI,CAAC,MAAM,CAAC;QACvB,CAAC;QACD,MAAM,EAAE,GAAG,QAAQ,IAAI,QAAQ,CAAC,QAAQ,CAAC;QACzC,MAAM,MAAM,GAAG,EAAE,CAAC,YAAY,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;QAChD,IAAI,CAAC,MAAM,EAAE,CAAC;YACV,MAAM,IAAI,KAAK,CAAC,yBAAyB,IAAI,CAAC,UAAU,yBAAyB,CAAC,CAAC;QACvF,CAAC;QACD,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,OAAO,MAAM,CAAC;IAClB,CAAC;IAEM,KAAK,CAAC,SAAS,CAAC,MAAiC,EAAE,SAAiB,EAAE,WAAqB,EAAE,QAA4B;QAC5H,MAAM,MAAM,GAAG,IAAI,CAAC,aAAa,CAAC,QAAQ,CAAC,CAAC;QAC5C,OAAO,kBAAkB,CAAC,EAAE,MAAM,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,MAAM,EAAE,SAAS,EAAE,WAAW,EAAE,YAAY,EAAE,IAAI,EAAE,CAAC,CAAC;IACnH,CAAC;CACJ"}

package/dist/sources/index.d.ts

@@ -0,0 +1,11 @@
+/**
+ * @fileoverview Barrel for the built-in record-set source adapters.
+ * @module @memberjunction/record-set-processor-base
+ */
+export * from './sourceUtil.js';
+export * from './arraySource.js';
+export * from './viewSource.js';
+export * from './listSource.js';
+export * from './filterSource.js';
+export * from './keysetSource.js';
+//# sourceMappingURL=index.d.ts.map

package/dist/sources/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/sources/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,cAAc,cAAc,CAAC;AAC7B,cAAc,eAAe,CAAC;AAC9B,cAAc,cAAc,CAAC;AAC7B,cAAc,cAAc,CAAC;AAC7B,cAAc,gBAAgB,CAAC;AAC/B,cAAc,gBAAgB,CAAC"}

package/dist/sources/index.js

@@ -0,0 +1,11 @@
+/**
+ * @fileoverview Barrel for the built-in record-set source adapters.
+ * @module @memberjunction/record-set-processor-base
+ */
+export * from './sourceUtil.js';
+export * from './arraySource.js';
+export * from './viewSource.js';
+export * from './listSource.js';
+export * from './filterSource.js';
+export * from './keysetSource.js';
+//# sourceMappingURL=index.js.map

package/dist/sources/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/sources/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,cAAc,cAAc,CAAC;AAC7B,cAAc,eAAe,CAAC;AAC9B,cAAc,cAAc,CAAC;AAC7B,cAAc,cAAc,CAAC;AAC7B,cAAc,gBAAgB,CAAC;AAC/B,cAAc,gBAAgB,CAAC"}

package/dist/sources/keysetSource.d.ts

@@ -0,0 +1,24 @@
+/**
+ * @fileoverview Keyset source — like {@link FilterSource} but explicitly requires keyset (seek)
+ * pagination, asserting the entity has a single orderable PK. Preferred for large background sweeps
+ * where deep offset pagination would degrade. Reports `SourceType: 'Keyset'`.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { IMetadataProvider, UserInfo } from '@memberjunction/core';
+import { IRecordSetSource, SourceDescriptor } from '../interfaces.js';
+import { ProcessCursor, RecordBatch } from '../types.js';
+/** A source backed by an entity + ad-hoc filter, paginated strictly via keyset. */
+export declare class KeysetSource implements IRecordSetSource {
+    private readonly entityName;
+    private readonly filter?;
+    private entity?;
+    /**
+     * @param entityName - The target entity name (must have a single orderable PK).
+     * @param filter - Optional WHERE clause (no leading `WHERE`).
+     */
+    constructor(entityName: string, filter?: string);
+    Describe(): SourceDescriptor;
+    private resolveEntity;
+    NextBatch(cursor: ProcessCursor | undefined, batchSize: number, contextUser: UserInfo, provider?: IMetadataProvider): Promise<RecordBatch>;
+}
+//# sourceMappingURL=keysetSource.d.ts.map

package/dist/sources/keysetSource.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"keysetSource.d.ts","sourceRoot":"","sources":["../../src/sources/keysetSource.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAc,iBAAiB,EAAY,QAAQ,EAAE,MAAM,sBAAsB,CAAC;AACzF,OAAO,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,MAAM,eAAe,CAAC;AACnE,OAAO,EAAE,aAAa,EAAE,WAAW,EAAE,MAAM,UAAU,CAAC;AAGtD,mFAAmF;AACnF,qBAAa,YAAa,YAAW,gBAAgB;IAOrC,OAAO,CAAC,QAAQ,CAAC,UAAU;IAAU,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAC;IANzE,OAAO,CAAC,MAAM,CAAC,CAAa;IAE5B;;;OAGG;gBAC0B,UAAU,EAAE,MAAM,EAAmB,MAAM,CAAC,EAAE,MAAM;IAE1E,QAAQ,IAAI,gBAAgB;IAInC,OAAO,CAAC,aAAa;IAgBR,SAAS,CAAC,MAAM,EAAE,aAAa,GAAG,SAAS,EAAE,SAAS,EAAE,MAAM,EAAE,WAAW,EAAE,QAAQ,EAAE,QAAQ,CAAC,EAAE,iBAAiB,GAAG,OAAO,CAAC,WAAW,CAAC;CAI1J"}

package/dist/sources/keysetSource.js

@@ -0,0 +1,42 @@
+/**
+ * @fileoverview Keyset source — like {@link FilterSource} but explicitly requires keyset (seek)
+ * pagination, asserting the entity has a single orderable PK. Preferred for large background sweeps
+ * where deep offset pagination would degrade. Reports `SourceType: 'Keyset'`.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { Metadata } from '@memberjunction/core';
+import { canUseKeyset, pageEntityByFilter } from './sourceUtil.js';
+/** A source backed by an entity + ad-hoc filter, paginated strictly via keyset. */
+export class KeysetSource {
+    /**
+     * @param entityName - The target entity name (must have a single orderable PK).
+     * @param filter - Optional WHERE clause (no leading `WHERE`).
+     */
+    constructor(entityName, filter) {
+        this.entityName = entityName;
+        this.filter = filter;
+    }
+    Describe() {
+        return { SourceType: 'Keyset', SourceFilter: this.filter, EntityID: this.entity?.ID };
+    }
+    resolveEntity(provider) {
+        if (this.entity) {
+            return this.entity;
+        }
+        const md = provider ?? Metadata.Provider;
+        const entity = md.EntityByName(this.entityName);
+        if (!entity) {
+            throw new Error(`KeysetSource: entity '${this.entityName}' not found in metadata`);
+        }
+        if (!canUseKeyset(entity)) {
+            throw new Error(`KeysetSource: entity '${this.entityName}' lacks a single orderable PK; use FilterSource instead`);
+        }
+        this.entity = entity;
+        return entity;
+    }
+    async NextBatch(cursor, batchSize, contextUser, provider) {
+        const entity = this.resolveEntity(provider);
+        return pageEntityByFilter({ entity, filter: this.filter, cursor, batchSize, contextUser, preferKeyset: true });
+    }
+}
+//# sourceMappingURL=keysetSource.js.map

package/dist/sources/keysetSource.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"keysetSource.js","sourceRoot":"","sources":["../../src/sources/keysetSource.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAiC,QAAQ,EAAY,MAAM,sBAAsB,CAAC;AAGzF,OAAO,EAAE,YAAY,EAAE,kBAAkB,EAAE,MAAM,cAAc,CAAC;AAEhE,mFAAmF;AACnF,MAAM,OAAO,YAAY;IAGrB;;;OAGG;IACH,YAA6B,UAAkB,EAAmB,MAAe;QAApD,eAAU,GAAV,UAAU,CAAQ;QAAmB,WAAM,GAAN,MAAM,CAAS;IAAG,CAAC;IAE9E,QAAQ;QACX,OAAO,EAAE,UAAU,EAAE,QAAQ,EAAE,YAAY,EAAE,IAAI,CAAC,MAAM,EAAE,QAAQ,EAAE,IAAI,CAAC,MAAM,EAAE,EAAE,EAAE,CAAC;IAC1F,CAAC;IAEO,aAAa,CAAC,QAA4B;QAC9C,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;YACd,OAAO,IAAI,CAAC,MAAM,CAAC;QACvB,CAAC;QACD,MAAM,EAAE,GAAG,QAAQ,IAAI,QAAQ,CAAC,QAAQ,CAAC;QACzC,MAAM,MAAM,GAAG,EAAE,CAAC,YAAY,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;QAChD,IAAI,CAAC,MAAM,EAAE,CAAC;YACV,MAAM,IAAI,KAAK,CAAC,yBAAyB,IAAI,CAAC,UAAU,yBAAyB,CAAC,CAAC;QACvF,CAAC;QACD,IAAI,CAAC,YAAY,CAAC,MAAM,CAAC,EAAE,CAAC;YACxB,MAAM,IAAI,KAAK,CAAC,yBAAyB,IAAI,CAAC,UAAU,yDAAyD,CAAC,CAAC;QACvH,CAAC;QACD,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,OAAO,MAAM,CAAC;IAClB,CAAC;IAEM,KAAK,CAAC,SAAS,CAAC,MAAiC,EAAE,SAAiB,EAAE,WAAqB,EAAE,QAA4B;QAC5H,MAAM,MAAM,GAAG,IAAI,CAAC,aAAa,CAAC,QAAQ,CAAC,CAAC;QAC5C,OAAO,kBAAkB,CAAC,EAAE,MAAM,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,MAAM,EAAE,SAAS,EAAE,WAAW,EAAE,YAAY,EAAE,IAAI,EAAE,CAAC,CAAC;IACnH,CAAC;CACJ"}

package/dist/sources/listSource.d.ts

@@ -0,0 +1,19 @@
+/**
+ * @fileoverview List source — yields the members of an `MJ: Lists` record (read from `MJ: List
+ * Details`) in offset-paginated batches. List Details already store a composite-key-safe `RecordID`.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { IMetadataProvider, UserInfo } from '@memberjunction/core';
+import { IRecordSetSource, SourceDescriptor } from '../interfaces.js';
+import { ProcessCursor, RecordBatch } from '../types.js';
+/** A source backed by a List (its members, via List Details). */
+export declare class ListSource implements IRecordSetSource {
+    private readonly listID;
+    private entityID?;
+    /** @param listID - The `MJ: Lists` ID whose members to iterate. */
+    constructor(listID: string);
+    Describe(): SourceDescriptor;
+    private resolveEntityID;
+    NextBatch(cursor: ProcessCursor | undefined, batchSize: number, contextUser: UserInfo, _provider?: IMetadataProvider): Promise<RecordBatch>;
+}
+//# sourceMappingURL=listSource.d.ts.map

package/dist/sources/listSource.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"listSource.d.ts","sourceRoot":"","sources":["../../src/sources/listSource.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAE,iBAAiB,EAAW,QAAQ,EAAE,MAAM,sBAAsB,CAAC;AAC5E,OAAO,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,MAAM,eAAe,CAAC;AACnE,OAAO,EAAE,aAAa,EAAE,WAAW,EAAa,MAAM,UAAU,CAAC;AAEjE,iEAAiE;AACjE,qBAAa,UAAW,YAAW,gBAAgB;IAInC,OAAO,CAAC,QAAQ,CAAC,MAAM;IAHnC,OAAO,CAAC,QAAQ,CAAC,CAAS;IAE1B,mEAAmE;gBACtC,MAAM,EAAE,MAAM;IAEpC,QAAQ,IAAI,gBAAgB;YAIrB,eAAe;IAuBhB,SAAS,CAAC,MAAM,EAAE,aAAa,GAAG,SAAS,EAAE,SAAS,EAAE,MAAM,EAAE,WAAW,EAAE,QAAQ,EAAE,SAAS,CAAC,EAAE,iBAAiB,GAAG,OAAO,CAAC,WAAW,CAAC;CAyB3J"}

package/dist/sources/listSource.js

@@ -0,0 +1,64 @@
+/**
+ * @fileoverview List source — yields the members of an `MJ: Lists` record (read from `MJ: List
+ * Details`) in offset-paginated batches. List Details already store a composite-key-safe `RecordID`.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { RunView } from '@memberjunction/core';
+/** A source backed by a List (its members, via List Details). */
+export class ListSource {
+    /** @param listID - The `MJ: Lists` ID whose members to iterate. */
+    constructor(listID) {
+        this.listID = listID;
+    }
+    Describe() {
+        return { SourceType: 'List', SourceID: this.listID, EntityID: this.entityID };
+    }
+    async resolveEntityID(contextUser) {
+        if (this.entityID) {
+            return this.entityID;
+        }
+        const rv = new RunView();
+        const result = await rv.RunView({
+            EntityName: 'MJ: Lists',
+            ExtraFilter: `ID='${this.listID}'`,
+            Fields: ['EntityID'],
+            ResultType: 'simple',
+            MaxRows: 1,
+        }, contextUser);
+        if (!result.Success) {
+            throw new Error(`ListSource: failed loading list '${this.listID}': ${result.ErrorMessage}`);
+        }
+        const row = (result.Results ?? [])[0];
+        if (!row?.EntityID) {
+            throw new Error(`ListSource: list '${this.listID}' not found`);
+        }
+        this.entityID = row.EntityID;
+        return this.entityID;
+    }
+    async NextBatch(cursor, batchSize, contextUser, _provider) {
+        const entityID = await this.resolveEntityID(contextUser);
+        const offset = cursor?.Offset ?? 0;
+        const rv = new RunView();
+        const result = await rv.RunView({
+            EntityName: 'MJ: List Details',
+            ExtraFilter: `ListID='${this.listID}'`,
+            Fields: ['RecordID'],
+            OrderBy: 'ID',
+            ResultType: 'simple',
+            StartRow: offset,
+            MaxRows: batchSize,
+        }, contextUser);
+        if (!result.Success) {
+            throw new Error(`ListSource: failed loading members for list '${this.listID}': ${result.ErrorMessage}`);
+        }
+        const rows = (result.Results ?? []);
+        const records = rows.map((row) => ({ EntityID: entityID, RecordID: String(row.RecordID) }));
+        return {
+            Records: records,
+            NextCursor: { Offset: offset + records.length },
+            Exhausted: records.length < batchSize,
+            TotalRowCount: result.TotalRowCount,
+        };
+    }
+}
+//# sourceMappingURL=listSource.js.map

package/dist/sources/listSource.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"listSource.js","sourceRoot":"","sources":["../../src/sources/listSource.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAqB,OAAO,EAAY,MAAM,sBAAsB,CAAC;AAI5E,iEAAiE;AACjE,MAAM,OAAO,UAAU;IAGnB,mEAAmE;IACnE,YAA6B,MAAc;QAAd,WAAM,GAAN,MAAM,CAAQ;IAAG,CAAC;IAExC,QAAQ;QACX,OAAO,EAAE,UAAU,EAAE,MAAM,EAAE,QAAQ,EAAE,IAAI,CAAC,MAAM,EAAE,QAAQ,EAAE,IAAI,CAAC,QAAQ,EAAE,CAAC;IAClF,CAAC;IAEO,KAAK,CAAC,eAAe,CAAC,WAAqB;QAC/C,IAAI,IAAI,CAAC,QAAQ,EAAE,CAAC;YAChB,OAAO,IAAI,CAAC,QAAQ,CAAC;QACzB,CAAC;QACD,MAAM,EAAE,GAAG,IAAI,OAAO,EAAE,CAAC;QACzB,MAAM,MAAM,GAAG,MAAM,EAAE,CAAC,OAAO,CAAC;YAC5B,UAAU,EAAE,WAAW;YACvB,WAAW,EAAE,OAAO,IAAI,CAAC,MAAM,GAAG;YAClC,MAAM,EAAE,CAAC,UAAU,CAAC;YACpB,UAAU,EAAE,QAAQ;YACpB,OAAO,EAAE,CAAC;SACb,EAAE,WAAW,CAAC,CAAC;QAChB,IAAI,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC;YAClB,MAAM,IAAI,KAAK,CAAC,oCAAoC,IAAI,CAAC,MAAM,MAAM,MAAM,CAAC,YAAY,EAAE,CAAC,CAAC;QAChG,CAAC;QACD,MAAM,GAAG,GAAG,CAAC,MAAM,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC,CAAC,CAAsC,CAAC;QAC3E,IAAI,CAAC,GAAG,EAAE,QAAQ,EAAE,CAAC;YACjB,MAAM,IAAI,KAAK,CAAC,qBAAqB,IAAI,CAAC,MAAM,aAAa,CAAC,CAAC;QACnE,CAAC;QACD,IAAI,CAAC,QAAQ,GAAG,GAAG,CAAC,QAAQ,CAAC;QAC7B,OAAO,IAAI,CAAC,QAAQ,CAAC;IACzB,CAAC;IAEM,KAAK,CAAC,SAAS,CAAC,MAAiC,EAAE,SAAiB,EAAE,WAAqB,EAAE,SAA6B;QAC7H,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,eAAe,CAAC,WAAW,CAAC,CAAC;QACzD,MAAM,MAAM,GAAG,MAAM,EAAE,MAAM,IAAI,CAAC,CAAC;QACnC,MAAM,EAAE,GAAG,IAAI,OAAO,EAAE,CAAC;QACzB,MAAM,MAAM,GAAG,MAAM,EAAE,CAAC,OAAO,CAAC;YAC5B,UAAU,EAAE,kBAAkB;YAC9B,WAAW,EAAE,WAAW,IAAI,CAAC,MAAM,GAAG;YACtC,MAAM,EAAE,CAAC,UAAU,CAAC;YACpB,OAAO,EAAE,IAAI;YACb,UAAU,EAAE,QAAQ;YACpB,QAAQ,EAAE,MAAM;YAChB,OAAO,EAAE,SAAS;SACrB,EAAE,WAAW,CAAC,CAAC;QAChB,IAAI,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC;YAClB,MAAM,IAAI,KAAK,CAAC,gDAAgD,IAAI,CAAC,MAAM,MAAM,MAAM,CAAC,YAAY,EAAE,CAAC,CAAC;QAC5G,CAAC;QACD,MAAM,IAAI,GAAG,CAAC,MAAM,CAAC,OAAO,IAAI,EAAE,CAA2B,CAAC;QAC9D,MAAM,OAAO,GAAgB,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC,EAAE,QAAQ,EAAE,QAAQ,EAAE,QAAQ,EAAE,MAAM,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC,CAAC;QACzG,OAAO;YACH,OAAO,EAAE,OAAO;YAChB,UAAU,EAAE,EAAE,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,EAAE;YAC/C,SAAS,EAAE,OAAO,CAAC,MAAM,GAAG,SAAS;YACrC,aAAa,EAAE,MAAM,CAAC,aAAa;SACtC,CAAC;IACN,CAAC;CACJ"}

package/dist/sources/sourceUtil.d.ts

@@ -0,0 +1,30 @@
+/**
+ * @fileoverview Shared helpers for source adapters: composite-key-safe record-ID serialization,
+ * keyset-pagination eligibility, and a generic entity-filter pager that picks keyset (seek) or
+ * offset pagination as appropriate.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { EntityInfo, UserInfo } from '@memberjunction/core';
+import { ProcessCursor, RecordBatch } from '../types.js';
+/**
+ * Serializes a record's primary key to a composite-key-safe string. Single-PK entities return the
+ * raw value; composite-PK entities use `CompositeKey.ToConcatenatedString()`.
+ */
+export declare function serializeRecordId(entity: EntityInfo, row: Record<string, unknown>): string;
+/** Returns true when the entity has a single, orderable primary key suitable for keyset pagination. */
+export declare function canUseKeyset(entity: EntityInfo): boolean;
+/**
+ * Pages an entity's rows by filter, returning one batch of {@link RecordRef}. Uses keyset (seek)
+ * pagination when `preferKeyset` is set and the entity supports it (single orderable PK), otherwise
+ * falls back to offset (StartRow) pagination. Selects only the PK columns and bypasses the cache —
+ * these single-use sweep pages should never pollute the local cache.
+ */
+export declare function pageEntityByFilter(opts: {
+    entity: EntityInfo;
+    filter?: string;
+    cursor: ProcessCursor | undefined;
+    batchSize: number;
+    contextUser: UserInfo;
+    preferKeyset: boolean;
+}): Promise<RecordBatch>;
+//# sourceMappingURL=sourceUtil.d.ts.map

package/dist/sources/sourceUtil.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"sourceUtil.d.ts","sourceRoot":"","sources":["../../src/sources/sourceUtil.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAgB,UAAU,EAAW,QAAQ,EAAE,MAAM,sBAAsB,CAAC;AACnF,OAAO,EAAE,aAAa,EAAE,WAAW,EAAa,MAAM,UAAU,CAAC;AAYjE;;;GAGG;AACH,wBAAgB,iBAAiB,CAAC,MAAM,EAAE,UAAU,EAAE,GAAG,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,MAAM,CAO1F;AAED,uGAAuG;AACvG,wBAAgB,YAAY,CAAC,MAAM,EAAE,UAAU,GAAG,OAAO,CASxD;AAED;;;;;GAKG;AACH,wBAAsB,kBAAkB,CAAC,IAAI,EAAE;IAC3C,MAAM,EAAE,UAAU,CAAC;IACnB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,MAAM,EAAE,aAAa,GAAG,SAAS,CAAC;IAClC,SAAS,EAAE,MAAM,CAAC;IAClB,WAAW,EAAE,QAAQ,CAAC;IACtB,YAAY,EAAE,OAAO,CAAC;CACzB,GAAG,OAAO,CAAC,WAAW,CAAC,CA6CvB"}

package/dist/sources/sourceUtil.js

@@ -0,0 +1,87 @@
+/**
+ * @fileoverview Shared helpers for source adapters: composite-key-safe record-ID serialization,
+ * keyset-pagination eligibility, and a generic entity-filter pager that picks keyset (seek) or
+ * offset pagination as appropriate.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { CompositeKey, RunView } from '@memberjunction/core';
+/** PK column types over which keyset (seek) pagination is safe and stable. */
+const KEYSET_ORDERABLE_PK_TYPES = new Set([
+    'uniqueidentifier', 'uuid', 'int', 'integer', 'bigint', 'smallint', 'tinyint',
+    'decimal', 'numeric', 'money', 'smallmoney', 'float', 'real', 'double precision',
+    'char', 'varchar', 'nchar', 'nvarchar', 'character', 'character varying',
+    'date', 'datetime', 'datetime2', 'datetimeoffset', 'smalldatetime', 'time',
+    'timestamp', 'timestamp with time zone', 'timestamp without time zone',
+    'bit', 'boolean', 'serial', 'bigserial',
+]);
+/**
+ * Serializes a record's primary key to a composite-key-safe string. Single-PK entities return the
+ * raw value; composite-PK entities use `CompositeKey.ToConcatenatedString()`.
+ */
+export function serializeRecordId(entity, row) {
+    if (entity.PrimaryKeys.length === 1) {
+        return String(row[entity.PrimaryKeys[0].Name]);
+    }
+    const ck = new CompositeKey();
+    ck.KeyValuePairs = entity.PrimaryKeys.map((pk) => ({ FieldName: pk.Name, Value: row[pk.Name] }));
+    return ck.ToConcatenatedString();
+}
+/** Returns true when the entity has a single, orderable primary key suitable for keyset pagination. */
+export function canUseKeyset(entity) {
+    if (!entity.FirstPrimaryKey || entity.PrimaryKeys.length !== 1) {
+        return false;
+    }
+    const normalizedType = (entity.FirstPrimaryKey.Type || '')
+        .replace(/\s*\([^)]*\)\s*$/, '') // strip parameterization like "nvarchar(255)"
+        .trim()
+        .toLowerCase();
+    return KEYSET_ORDERABLE_PK_TYPES.has(normalizedType);
+}
+/**
+ * Pages an entity's rows by filter, returning one batch of {@link RecordRef}. Uses keyset (seek)
+ * pagination when `preferKeyset` is set and the entity supports it (single orderable PK), otherwise
+ * falls back to offset (StartRow) pagination. Selects only the PK columns and bypasses the cache —
+ * these single-use sweep pages should never pollute the local cache.
+ */
+export async function pageEntityByFilter(opts) {
+    const { entity, filter, cursor, batchSize, contextUser, preferKeyset } = opts;
+    const pkName = entity.FirstPrimaryKey?.Name;
+    const useKeyset = preferKeyset && canUseKeyset(entity) && !!pkName;
+    const rv = new RunView();
+    const sharedParams = {
+        EntityName: entity.Name,
+        Fields: entity.PrimaryKeys.map((pk) => pk.Name),
+        ResultType: 'simple',
+        MaxRows: batchSize,
+        BypassCache: true,
+        ExtraFilter: filter,
+    };
+    const offset = cursor?.Offset ?? 0;
+    const result = useKeyset
+        ? await rv.RunView({
+            ...sharedParams,
+            OrderBy: pkName,
+            AfterKey: cursor?.Key != null ? CompositeKey.FromKeyValuePair(pkName, cursor.Key) : undefined,
+        }, contextUser)
+        : await rv.RunView({ ...sharedParams, StartRow: offset }, contextUser);
+    if (!result.Success) {
+        throw new Error(`Record set source failed paging '${entity.Name}': ${result.ErrorMessage}`);
+    }
+    const rows = (result.Results ?? []);
+    const records = rows.map((row) => ({
+        EntityID: entity.ID,
+        RecordID: serializeRecordId(entity, row),
+        Record: row,
+    }));
+    const exhausted = records.length < batchSize;
+    let nextCursor;
+    if (useKeyset) {
+        const lastValue = rows.length > 0 ? rows[rows.length - 1][pkName] : undefined;
+        nextCursor = { Key: lastValue != null ? String(lastValue) : cursor?.Key };
+    }
+    else {
+        nextCursor = { Offset: offset + records.length };
+    }
+    return { Records: records, NextCursor: nextCursor, Exhausted: exhausted, TotalRowCount: result.TotalRowCount };
+}
+//# sourceMappingURL=sourceUtil.js.map

package/dist/sources/sourceUtil.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"sourceUtil.js","sourceRoot":"","sources":["../../src/sources/sourceUtil.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAE,YAAY,EAAc,OAAO,EAAY,MAAM,sBAAsB,CAAC;AAGnF,8EAA8E;AAC9E,MAAM,yBAAyB,GAAG,IAAI,GAAG,CAAS;IAC9C,kBAAkB,EAAE,MAAM,EAAE,KAAK,EAAE,SAAS,EAAE,QAAQ,EAAE,UAAU,EAAE,SAAS;IAC7E,SAAS,EAAE,SAAS,EAAE,OAAO,EAAE,YAAY,EAAE,OAAO,EAAE,MAAM,EAAE,kBAAkB;IAChF,MAAM,EAAE,SAAS,EAAE,OAAO,EAAE,UAAU,EAAE,WAAW,EAAE,mBAAmB;IACxE,MAAM,EAAE,UAAU,EAAE,WAAW,EAAE,gBAAgB,EAAE,eAAe,EAAE,MAAM;IAC1E,WAAW,EAAE,0BAA0B,EAAE,6BAA6B;IACtE,KAAK,EAAE,SAAS,EAAE,QAAQ,EAAE,WAAW;CAC1C,CAAC,CAAC;AAEH;;;GAGG;AACH,MAAM,UAAU,iBAAiB,CAAC,MAAkB,EAAE,GAA4B;IAC9E,IAAI,MAAM,CAAC,WAAW,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAClC,OAAO,MAAM,CAAC,GAAG,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;IACnD,CAAC;IACD,MAAM,EAAE,GAAG,IAAI,YAAY,EAAE,CAAC;IAC9B,EAAE,CAAC,aAAa,GAAG,MAAM,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,SAAS,EAAE,EAAE,CAAC,IAAI,EAAE,KAAK,EAAE,GAAG,CAAC,EAAE,CAAC,IAAI,CAAU,EAAE,CAAC,CAAC,CAAC;IAC1G,OAAO,EAAE,CAAC,oBAAoB,EAAE,CAAC;AACrC,CAAC;AAED,uGAAuG;AACvG,MAAM,UAAU,YAAY,CAAC,MAAkB;IAC3C,IAAI,CAAC,MAAM,CAAC,eAAe,IAAI,MAAM,CAAC,WAAW,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC7D,OAAO,KAAK,CAAC;IACjB,CAAC;IACD,MAAM,cAAc,GAAG,CAAC,MAAM,CAAC,eAAe,CAAC,IAAI,IAAI,EAAE,CAAC;SACrD,OAAO,CAAC,kBAAkB,EAAE,EAAE,CAAC,CAAC,8CAA8C;SAC9E,IAAI,EAAE;SACN,WAAW,EAAE,CAAC;IACnB,OAAO,yBAAyB,CAAC,GAAG,CAAC,cAAc,CAAC,CAAC;AACzD,CAAC;AAED;;;;;GAKG;AACH,MAAM,CAAC,KAAK,UAAU,kBAAkB,CAAC,IAOxC;IACG,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,SAAS,EAAE,WAAW,EAAE,YAAY,EAAE,GAAG,IAAI,CAAC;IAC9E,MAAM,MAAM,GAAG,MAAM,CAAC,eAAe,EAAE,IAAI,CAAC;IAC5C,MAAM,SAAS,GAAG,YAAY,IAAI,YAAY,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC;IAEnE,MAAM,EAAE,GAAG,IAAI,OAAO,EAAE,CAAC;IACzB,MAAM,YAAY,GAAG;QACjB,UAAU,EAAE,MAAM,CAAC,IAAI;QACvB,MAAM,EAAE,MAAM,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,IAAI,CAAC;QAC/C,UAAU,EAAE,QAAiB;QAC7B,OAAO,EAAE,SAAS;QAClB,WAAW,EAAE,IAAI;QACjB,WAAW,EAAE,MAAM;KACtB,CAAC;IAEF,MAAM,MAAM,GAAG,MAAM,EAAE,MAAM,IAAI,CAAC,CAAC;IACnC,MAAM,MAAM,GAAG,SAAS;QACpB,CAAC,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC;YACf,GAAG,YAAY;YACf,OAAO,EAAE,MAAM;YACf,QAAQ,EAAE,MAAM,EAAE,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,YAAY,CAAC,gBAAgB,CAAC,MAAgB,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,SAAS;SAC1G,EAAE,WAAW,CAAC;QACf,CAAC,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,EAAE,GAAG,YAAY,EAAE,QAAQ,EAAE,MAAM,EAAE,EAAE,WAAW,CAAC,CAAC;IAE3E,IAAI,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC;QAClB,MAAM,IAAI,KAAK,CAAC,oCAAoC,MAAM,CAAC,IAAI,MAAM,MAAM,CAAC,YAAY,EAAE,CAAC,CAAC;IAChG,CAAC;IAED,MAAM,IAAI,GAAG,CAAC,MAAM,CAAC,OAAO,IAAI,EAAE,CAA8B,CAAC;IACjE,MAAM,OAAO,GAAgB,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;QAC5C,QAAQ,EAAE,MAAM,CAAC,EAAE;QACnB,QAAQ,EAAE,iBAAiB,CAAC,MAAM,EAAE,GAAG,CAAC;QACxC,MAAM,EAAE,GAAG;KACd,CAAC,CAAC,CAAC;IACJ,MAAM,SAAS,GAAG,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC;IAE7C,IAAI,UAAyB,CAAC;IAC9B,IAAI,SAAS,EAAE,CAAC;QACZ,MAAM,SAAS,GAAG,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,MAAgB,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QACxF,UAAU,GAAG,EAAE,GAAG,EAAE,SAAS,IAAI,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,MAAM,EAAE,GAAG,EAAE,CAAC;IAC9E,CAAC;SAAM,CAAC;QACJ,UAAU,GAAG,EAAE,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC;IACrD,CAAC;IAED,OAAO,EAAE,OAAO,EAAE,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,SAAS,EAAE,SAAS,EAAE,aAAa,EAAE,MAAM,CAAC,aAAa,EAAE,CAAC;AACnH,CAAC"}

package/dist/sources/viewSource.d.ts

@@ -0,0 +1,20 @@
+/**
+ * @fileoverview User-View source — resolves a saved User View and yields its members (PKs only) in
+ * offset-paginated batches. A view may carry an arbitrary OrderBy/filter that defeats keyset
+ * pagination, so this source uses offset (StartRow) paging.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { IMetadataProvider, UserInfo } from '@memberjunction/core';
+import { IRecordSetSource, SourceDescriptor } from '../interfaces.js';
+import { ProcessCursor, RecordBatch } from '../types.js';
+/** A source backed by a saved User View. */
+export declare class ViewSource implements IRecordSetSource {
+    private readonly viewID;
+    private entity?;
+    /** @param viewID - The `MJ: User Views` ID to iterate. */
+    constructor(viewID: string);
+    Describe(): SourceDescriptor;
+    private resolveEntity;
+    NextBatch(cursor: ProcessCursor | undefined, batchSize: number, contextUser: UserInfo, provider?: IMetadataProvider): Promise<RecordBatch>;
+}
+//# sourceMappingURL=viewSource.d.ts.map

package/dist/sources/viewSource.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"viewSource.d.ts","sourceRoot":"","sources":["../../src/sources/viewSource.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAc,iBAAiB,EAAqB,QAAQ,EAAE,MAAM,sBAAsB,CAAC;AAClG,OAAO,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,MAAM,eAAe,CAAC;AACnE,OAAO,EAAE,aAAa,EAAE,WAAW,EAAa,MAAM,UAAU,CAAC;AAGjE,4CAA4C;AAC5C,qBAAa,UAAW,YAAW,gBAAgB;IAInC,OAAO,CAAC,QAAQ,CAAC,MAAM;IAHnC,OAAO,CAAC,MAAM,CAAC,CAAa;IAE5B,0DAA0D;gBAC7B,MAAM,EAAE,MAAM;IAEpC,QAAQ,IAAI,gBAAgB;YAIrB,aAAa;IAiBd,SAAS,CAAC,MAAM,EAAE,aAAa,GAAG,SAAS,EAAE,SAAS,EAAE,MAAM,EAAE,WAAW,EAAE,QAAQ,EAAE,QAAQ,CAAC,EAAE,iBAAiB,GAAG,OAAO,CAAC,WAAW,CAAC;CA2B1J"}

package/dist/sources/viewSource.js

@@ -0,0 +1,62 @@
+/**
+ * @fileoverview User-View source — resolves a saved User View and yields its members (PKs only) in
+ * offset-paginated batches. A view may carry an arbitrary OrderBy/filter that defeats keyset
+ * pagination, so this source uses offset (StartRow) paging.
+ * @module @memberjunction/record-set-processor-base
+ */
+import { Metadata, RunView } from '@memberjunction/core';
+import { serializeRecordId } from './sourceUtil.js';
+/** A source backed by a saved User View. */
+export class ViewSource {
+    /** @param viewID - The `MJ: User Views` ID to iterate. */
+    constructor(viewID) {
+        this.viewID = viewID;
+    }
+    Describe() {
+        return { SourceType: 'View', SourceID: this.viewID, EntityID: this.entity?.ID };
+    }
+    async resolveEntity(provider) {
+        if (this.entity) {
+            return this.entity;
+        }
+        const md = provider ?? Metadata.Provider;
+        const entityName = await RunView.GetEntityNameFromRunViewParams({ ViewID: this.viewID }, provider ?? null);
+        if (!entityName) {
+            throw new Error(`ViewSource: could not determine the entity for view '${this.viewID}'`);
+        }
+        const entity = md.EntityByName(entityName);
+        if (!entity) {
+            throw new Error(`ViewSource: entity '${entityName}' not found in metadata`);
+        }
+        this.entity = entity;
+        return entity;
+    }
+    async NextBatch(cursor, batchSize, contextUser, provider) {
+        const entity = await this.resolveEntity(provider);
+        const offset = cursor?.Offset ?? 0;
+        const rv = new RunView();
+        const result = await rv.RunView({
+            ViewID: this.viewID,
+            Fields: entity.PrimaryKeys.map((pk) => pk.Name),
+            ResultType: 'simple',
+            StartRow: offset,
+            MaxRows: batchSize,
+        }, contextUser);
+        if (!result.Success) {
+            throw new Error(`ViewSource: failed running view '${this.viewID}': ${result.ErrorMessage}`);
+        }
+        const rows = (result.Results ?? []);
+        const records = rows.map((row) => ({
+            EntityID: entity.ID,
+            RecordID: serializeRecordId(entity, row),
+            Record: row,
+        }));
+        return {
+            Records: records,
+            NextCursor: { Offset: offset + records.length },
+            Exhausted: records.length < batchSize,
+            TotalRowCount: result.TotalRowCount,
+        };
+    }
+}
+//# sourceMappingURL=viewSource.js.map

package/dist/sources/viewSource.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"viewSource.js","sourceRoot":"","sources":["../../src/sources/viewSource.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAiC,QAAQ,EAAE,OAAO,EAAY,MAAM,sBAAsB,CAAC;AAGlG,OAAO,EAAE,iBAAiB,EAAE,MAAM,cAAc,CAAC;AAEjD,4CAA4C;AAC5C,MAAM,OAAO,UAAU;IAGnB,0DAA0D;IAC1D,YAA6B,MAAc;QAAd,WAAM,GAAN,MAAM,CAAQ;IAAG,CAAC;IAExC,QAAQ;QACX,OAAO,EAAE,UAAU,EAAE,MAAM,EAAE,QAAQ,EAAE,IAAI,CAAC,MAAM,EAAE,QAAQ,EAAE,IAAI,CAAC,MAAM,EAAE,EAAE,EAAE,CAAC;IACpF,CAAC;IAEO,KAAK,CAAC,aAAa,CAAC,QAA4B;QACpD,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;YACd,OAAO,IAAI,CAAC,MAAM,CAAC;QACvB,CAAC;QACD,MAAM,EAAE,GAAG,QAAQ,IAAI,QAAQ,CAAC,QAAQ,CAAC;QACzC,MAAM,UAAU,GAAG,MAAM,OAAO,CAAC,8BAA8B,CAAC,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,EAAE,QAAQ,IAAI,IAAI,CAAC,CAAC;QAC3G,IAAI,CAAC,UAAU,EAAE,CAAC;YACd,MAAM,IAAI,KAAK,CAAC,wDAAwD,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC;QAC5F,CAAC;QACD,MAAM,MAAM,GAAG,EAAE,CAAC,YAAY,CAAC,UAAU,CAAC,CAAC;QAC3C,IAAI,CAAC,MAAM,EAAE,CAAC;YACV,MAAM,IAAI,KAAK,CAAC,uBAAuB,UAAU,yBAAyB,CAAC,CAAC;QAChF,CAAC;QACD,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,OAAO,MAAM,CAAC;IAClB,CAAC;IAEM,KAAK,CAAC,SAAS,CAAC,MAAiC,EAAE,SAAiB,EAAE,WAAqB,EAAE,QAA4B;QAC5H,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,aAAa,CAAC,QAAQ,CAAC,CAAC;QAClD,MAAM,MAAM,GAAG,MAAM,EAAE,MAAM,IAAI,CAAC,CAAC;QACnC,MAAM,EAAE,GAAG,IAAI,OAAO,EAAE,CAAC;QACzB,MAAM,MAAM,GAAG,MAAM,EAAE,CAAC,OAAO,CAAC;YAC5B,MAAM,EAAE,IAAI,CAAC,MAAM;YACnB,MAAM,EAAE,MAAM,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,IAAI,CAAC;YAC/C,UAAU,EAAE,QAAQ;YACpB,QAAQ,EAAE,MAAM;YAChB,OAAO,EAAE,SAAS;SACrB,EAAE,WAAW,CAAC,CAAC;QAChB,IAAI,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC;YAClB,MAAM,IAAI,KAAK,CAAC,oCAAoC,IAAI,CAAC,MAAM,MAAM,MAAM,CAAC,YAAY,EAAE,CAAC,CAAC;QAChG,CAAC;QACD,MAAM,IAAI,GAAG,CAAC,MAAM,CAAC,OAAO,IAAI,EAAE,CAA8B,CAAC;QACjE,MAAM,OAAO,GAAgB,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;YAC5C,QAAQ,EAAE,MAAM,CAAC,EAAE;YACnB,QAAQ,EAAE,iBAAiB,CAAC,MAAM,EAAE,GAAG,CAAC;YACxC,MAAM,EAAE,GAAG;SACd,CAAC,CAAC,CAAC;QACJ,OAAO;YACH,OAAO,EAAE,OAAO;YAChB,UAAU,EAAE,EAAE,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,EAAE;YAC/C,SAAS,EAAE,OAAO,CAAC,MAAM,GAAG,SAAS;YACrC,aAAa,EAAE,MAAM,CAAC,aAAa;SACtC,CAAC;IACN,CAAC;CACJ"}

package/dist/types.d.ts

@@ -0,0 +1,100 @@
+/**
+ * @fileoverview Core data types for the Record Set Processor substrate — the shapes shared between
+ * the source/processor/tracker seams and the engine. These types are pure data (no behavior) and
+ * carry no server-only dependencies, so they are safe to import on the client.
+ * @module @memberjunction/record-set-processor-base
+ */
+/** A reference to a single record to be processed. */
+export interface RecordRef {
+    /** ID of the entity this record belongs to. */
+    EntityID: string;
+    /** The record's primary key, serialized to a composite-key-safe string. */
+    RecordID: string;
+    /** Optional already-loaded record data (a plain row or a `BaseEntity`), to save a re-fetch. */
+    Record?: unknown;
+}
+/**
+ * Opaque resume cursor round-tripped between a source and the run tracker. A source populates
+ * exactly one of these depending on how it paginates; the engine never interprets it.
+ */
+export interface ProcessCursor {
+    /** Offset (StartRow) for offset-paginated sources. */
+    Offset?: number;
+    /** Last-seen key for keyset (seek) paginated sources — the single-PK value as a string. */
+    Key?: string;
+}
+/** One page of records returned by a source, plus the cursor to fetch the next page. */
+export interface RecordBatch {
+    /** The records in this batch (empty when the source is exhausted). */
+    Records: RecordRef[];
+    /** Cursor to pass to the next `NextBatch()` call. */
+    NextCursor: ProcessCursor;
+    /** True when this is the final batch — no further records remain. */
+    Exhausted: boolean;
+    /** Total matching rows in the source, when the source knows it (e.g. from `RunView.TotalRowCount`). */
+    TotalRowCount?: number;
+}
+/** Lifecycle status of a process run. Mirrors the `MJ: Process Runs.Status` value list. */
+export type ProcessRunStatusValue = 'Pending' | 'Running' | 'Paused' | 'Completed' | 'Failed' | 'Cancelled';
+/** Per-record outcome status. Mirrors the `MJ: Process Run Details.Status` value list. */
+export type RecordResultStatusValue = 'Succeeded' | 'Failed' | 'Skipped';
+/** What triggered a run. Mirrors the `MJ: Process Runs.TriggeredBy` value list. */
+export type TriggeredByValue = 'OnChange' | 'Schedule' | 'OnDemand' | 'Manual';
+/** The kind of record-set source. Mirrors the `MJ: Process Runs.SourceType` value list. */
+export type SourceTypeValue = 'View' | 'List' | 'Filter' | 'Array' | 'Keyset' | 'SingleRecord';
+/** The outcome of processing a single record, returned by an {@link IRecordProcessor}. */
+export interface RecordResult {
+    /** Whether the record succeeded, failed, or was skipped. */
+    Status: RecordResultStatusValue;
+    /** Structured output payload produced for the record (persisted as JSON on the detail row). */
+    ResultPayload?: unknown;
+    /** Error detail when `Status` is `Failed`. */
+    ErrorMessage?: string;
+    /** Processing duration in milliseconds (the engine fills this in if the processor does not). */
+    DurationMs?: number;
+    /** Number of attempts made for this record. */
+    AttemptCount?: number;
+    /** Deep-trace link to an Action Execution Log, when the work was an Action. */
+    ActionExecutionLogID?: string;
+    /** Deep-trace link to an AI Agent Run, when the work was an Agent. */
+    AIAgentRunID?: string;
+    /** Deep-trace link to an AI Prompt Run, when the work was an Infer-and-Write-Back. */
+    AIPromptRunID?: string;
+}
+/** Running tallies for a process run. */
+export interface RunCounts {
+    /** Records handed to the processor so far. */
+    Processed: number;
+    /** Records that succeeded. */
+    Success: number;
+    /** Records that failed. */
+    Error: number;
+    /** Records that were skipped. */
+    Skipped: number;
+}
+/** Progress snapshot emitted to a run's `onProgress` callback. */
+export interface ProgressInfo extends RunCounts {
+    /** Known total record count, or null when the total is not known up-front. */
+    Total: number | null;
+    /** The record currently being processed, when available. */
+    CurrentRecordID?: string;
+}
+/** Final (or interim) summary of a process run. */
+export interface ProcessRunSummary extends RunCounts {
+    /** Terminal or current run status. */
+    Status: ProcessRunStatusValue;
+    /** Known total record count, or null. */
+    Total: number | null;
+    /** When the run started. */
+    StartTime?: Date;
+    /** When the run ended. */
+    EndTime?: Date;
+    /** Run-level error message when `Status` is `Failed`. */
+    ErrorMessage?: string;
+}
+/** Result returned by `RecordSetProcessor.Process()` — a run summary plus the persisted run ID. */
+export interface ProcessRunResult extends ProcessRunSummary {
+    /** ID of the persisted `MJ: Process Runs` row, when a persisting tracker was used. */
+    ProcessRunID?: string;
+}
+//# sourceMappingURL=types.d.ts.map

package/dist/types.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,sDAAsD;AACtD,MAAM,WAAW,SAAS;IACtB,+CAA+C;IAC/C,QAAQ,EAAE,MAAM,CAAC;IACjB,2EAA2E;IAC3E,QAAQ,EAAE,MAAM,CAAC;IACjB,+FAA+F;IAC/F,MAAM,CAAC,EAAE,OAAO,CAAC;CACpB;AAED;;;GAGG;AACH,MAAM,WAAW,aAAa;IAC1B,sDAAsD;IACtD,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,2FAA2F;IAC3F,GAAG,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,wFAAwF;AACxF,MAAM,WAAW,WAAW;IACxB,sEAAsE;IACtE,OAAO,EAAE,SAAS,EAAE,CAAC;IACrB,qDAAqD;IACrD,UAAU,EAAE,aAAa,CAAC;IAC1B,qEAAqE;IACrE,SAAS,EAAE,OAAO,CAAC;IACnB,uGAAuG;IACvG,aAAa,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,2FAA2F;AAC3F,MAAM,MAAM,qBAAqB,GAAG,SAAS,GAAG,SAAS,GAAG,QAAQ,GAAG,WAAW,GAAG,QAAQ,GAAG,WAAW,CAAC;AAE5G,0FAA0F;AAC1F,MAAM,MAAM,uBAAuB,GAAG,WAAW,GAAG,QAAQ,GAAG,SAAS,CAAC;AAEzE,mFAAmF;AACnF,MAAM,MAAM,gBAAgB,GAAG,UAAU,GAAG,UAAU,GAAG,UAAU,GAAG,QAAQ,CAAC;AAE/E,2FAA2F;AAC3F,MAAM,MAAM,eAAe,GAAG,MAAM,GAAG,MAAM,GAAG,QAAQ,GAAG,OAAO,GAAG,QAAQ,GAAG,cAAc,CAAC;AAE/F,0FAA0F;AAC1F,MAAM,WAAW,YAAY;IACzB,4DAA4D;IAC5D,MAAM,EAAE,uBAAuB,CAAC;IAChC,+FAA+F;IAC/F,aAAa,CAAC,EAAE,OAAO,CAAC;IACxB,8CAA8C;IAC9C,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,gGAAgG;IAChG,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,+CAA+C;IAC/C,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,+EAA+E;IAC/E,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B,sEAAsE;IACtE,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,sFAAsF;IACtF,aAAa,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,yCAAyC;AACzC,MAAM,WAAW,SAAS;IACtB,8CAA8C;IAC9C,SAAS,EAAE,MAAM,CAAC;IAClB,8BAA8B;IAC9B,OAAO,EAAE,MAAM,CAAC;IAChB,2BAA2B;IAC3B,KAAK,EAAE,MAAM,CAAC;IACd,iCAAiC;IACjC,OAAO,EAAE,MAAM,CAAC;CACnB;AAED,kEAAkE;AAClE,MAAM,WAAW,YAAa,SAAQ,SAAS;IAC3C,8EAA8E;IAC9E,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;IACrB,4DAA4D;IAC5D,eAAe,CAAC,EAAE,MAAM,CAAC;CAC5B;AAED,mDAAmD;AACnD,MAAM,WAAW,iBAAkB,SAAQ,SAAS;IAChD,sCAAsC;IACtC,MAAM,EAAE,qBAAqB,CAAC;IAC9B,yCAAyC;IACzC,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;IACrB,4BAA4B;IAC5B,SAAS,CAAC,EAAE,IAAI,CAAC;IACjB,0BAA0B;IAC1B,OAAO,CAAC,EAAE,IAAI,CAAC;IACf,yDAAyD;IACzD,YAAY,CAAC,EAAE,MAAM,CAAC;CACzB;AAED,mGAAmG;AACnG,MAAM,WAAW,gBAAiB,SAAQ,iBAAiB;IACvD,sFAAsF;IACtF,YAAY,CAAC,EAAE,MAAM,CAAC;CACzB"}

package/dist/types.js

@@ -0,0 +1,8 @@
+/**
+ * @fileoverview Core data types for the Record Set Processor substrate — the shapes shared between
+ * the source/processor/tracker seams and the engine. These types are pure data (no behavior) and
+ * carry no server-only dependencies, so they are safe to import on the client.
+ * @module @memberjunction/record-set-processor-base
+ */
+export {};
+//# sourceMappingURL=types.js.map

package/dist/types.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"types.js","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":"AAAA;;;;;GAKG"}

package/package.json

@@ -1,10 +1,32 @@
 {
   "name": "@memberjunction/record-set-processor-base",
-  "version": "0.0.1",
-  "description": "OIDC trusted publishing setup package for @memberjunction/record-set-processor-base",
-  "keywords": [
-    "oidc",
-    "trusted-publishing",
-    "setup"
-  ]
+  "type": "module",
+  "version": "5.42.0",
+  "description": "MemberJunction: Record Set Processor Base - interfaces, types, and source adapters for batched, resumable record-set processing. Client-safe.",
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "files": [
+    "/dist"
+  ],
+  "scripts": {
+    "build": "tsc && tsc-alias -f",
+    "watch": "tsc --watch",
+    "test": "vitest run",
+    "test:watch": "vitest"
+  },
+  "author": "MemberJunction.com",
+  "license": "ISC",
+  "dependencies": {
+    "@memberjunction/core": "5.42.0",
+    "@memberjunction/global": "5.42.0"
+  },
+  "devDependencies": {
+    "@types/node": "24.10.11",
+    "typescript": "^5.9.3",
+    "vitest": "^3.1.1"
+  },
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/MemberJunction/MJ"
+  }
 }