@exellix/exellix-matrix-persister 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +3 -0
- package/README.md +109 -0
- package/dist/collection-names.d.ts +7 -0
- package/dist/collection-names.d.ts.map +1 -0
- package/dist/collection-names.js +7 -0
- package/dist/collection-names.js.map +1 -0
- package/dist/create-exellix-matrix-data-tier.d.ts +10 -0
- package/dist/create-exellix-matrix-data-tier.d.ts.map +1 -0
- package/dist/create-exellix-matrix-data-tier.js +64 -0
- package/dist/create-exellix-matrix-data-tier.js.map +1 -0
- package/dist/index.d.ts +5 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +4 -0
- package/dist/index.js.map +1 -0
- package/dist/load-matrix-data-tier-factory.d.ts +6 -0
- package/dist/load-matrix-data-tier-factory.d.ts.map +1 -0
- package/dist/load-matrix-data-tier-factory.js +24 -0
- package/dist/load-matrix-data-tier-factory.js.map +1 -0
- package/dist/resolve-options.d.ts +11 -0
- package/dist/resolve-options.d.ts.map +1 -0
- package/dist/resolve-options.js +18 -0
- package/dist/resolve-options.js.map +1 -0
- package/dist/types.d.ts +45 -0
- package/dist/types.d.ts.map +1 -0
- package/dist/types.js +2 -0
- package/dist/types.js.map +1 -0
- package/docs/integration.md +47 -0
- package/package.json +51 -0
package/.env.example
ADDED
package/README.md
ADDED
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
# @exellix/exellix-matrix-persister
|
|
2
|
+
|
|
3
|
+
Mongo-backed **factory** for Exellix execution-matrix persistence: operational collections (rows, failures, snapshots) plus matrix and graph config collections. Host apps import **`createExellixMatrixDataTier`** instead of wiring **`mongodb`** or **`@x12i/xronox-store`** themselves.
|
|
4
|
+
|
|
5
|
+
Domain behavior (claims, materialization, filters) stays in **`@exellix/exellix-runtime`**. This package is the supported **wiring** surface for workers and read tiers.
|
|
6
|
+
|
|
7
|
+
## Install
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
npm install @exellix/exellix-matrix-persister @exellix/exellix-runtime
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
`@exellix/exellix-runtime` is a **peer** dependency (install it next to this package).
|
|
14
|
+
|
|
15
|
+
### Private registry (GitHub Packages)
|
|
16
|
+
|
|
17
|
+
If `@exellix/*` is published to GitHub npm, set a token and `.npmrc` (do **not** commit tokens):
|
|
18
|
+
|
|
19
|
+
```ini
|
|
20
|
+
@exellix:registry=https://npm.pkg.github.com
|
|
21
|
+
//npm.pkg.github.com/:_authToken=${NODE_AUTH_TOKEN}
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Export `NODE_AUTH_TOKEN` (PAT with `read:packages`) before `npm install`.
|
|
25
|
+
|
|
26
|
+
## Usage
|
|
27
|
+
|
|
28
|
+
```ts
|
|
29
|
+
import { createExellixMatrixDataTier } from '@exellix/exellix-matrix-persister';
|
|
30
|
+
|
|
31
|
+
const tier = await createExellixMatrixDataTier({
|
|
32
|
+
mongoUri: process.env.MONGO_URI,
|
|
33
|
+
executionDb: 'exellix-runtime',
|
|
34
|
+
configDb: 'exellix',
|
|
35
|
+
});
|
|
36
|
+
|
|
37
|
+
await tier.init?.();
|
|
38
|
+
|
|
39
|
+
// Pass into createMatrixWorker / openMatrixReadTier / createExecutionMatrixRuntime
|
|
40
|
+
const { rows, failures, snapshots, matrices, graphs } = tier;
|
|
41
|
+
|
|
42
|
+
await tier.close();
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
### Returned shape
|
|
46
|
+
|
|
47
|
+
| Field | Role |
|
|
48
|
+
|-------------|------|
|
|
49
|
+
| `rows` | Execution matrix rows |
|
|
50
|
+
| `failures` | Failure records |
|
|
51
|
+
| `snapshots` | Snapshot side-channel |
|
|
52
|
+
| `matrices` | Matrix configs (`exellix_matrix_configs`) |
|
|
53
|
+
| `graphs` | Graph configs (`exellix_graph_configs`) |
|
|
54
|
+
| `init?` | Optional warm-up (no-op after factory; safe to call) |
|
|
55
|
+
| `close` | Closes stores and any Mongo driver pass-through |
|
|
56
|
+
| `probe?` | `{ executionStore, configStore, executionMongoUri, configMongoUri }` for dependency health checks |
|
|
57
|
+
|
|
58
|
+
Collection names and indexes follow **`@exellix/exellix-runtime`** (`src/execution-matrix/collections.ts`).
|
|
59
|
+
|
|
60
|
+
## Environment
|
|
61
|
+
|
|
62
|
+
| Variable | Default | Used by |
|
|
63
|
+
|----------------|-------------------|---------|
|
|
64
|
+
| `MONGO_URI` | _(required)_ | Connection string |
|
|
65
|
+
| `execution_db` | `exellix-runtime` | Rows, failures, snapshots |
|
|
66
|
+
| `config_db` | `exellix` | Matrix + graph configs |
|
|
67
|
+
|
|
68
|
+
Options passed to `createExellixMatrixDataTier({ ... })` override env for that call.
|
|
69
|
+
|
|
70
|
+
## Consumers
|
|
71
|
+
|
|
72
|
+
**`@exellix/exellix-jobs`** and **`@exellix/exellix-matrix-read`** try this package first for `createExellixMatrixDataTier`, then **`@x12i/xmemory-store`** if the factory is re-exported there later. Installing **`@exellix/exellix-matrix-persister`** is enough for `openExellixMatrixPersistenceFromXmemoryStore` / `openMatrixPersistenceFromXmemoryStore` to succeed without host glue.
|
|
73
|
+
|
|
74
|
+
## API surface
|
|
75
|
+
|
|
76
|
+
- **`createExellixMatrixDataTier(options?)`** — main factory
|
|
77
|
+
- **`resolveExellixMatrixDataTierOptions(overrides?, env?)`** — merge options + `process.env` (throws if `mongoUri` missing)
|
|
78
|
+
- **`loadMatrixDataTierFactory()`** — resolves the factory from this package or `@x12i/xmemory-store` (for advanced dynamic wiring)
|
|
79
|
+
|
|
80
|
+
Types are exported from the package entry (`CreateExellixMatrixDataTierOptions`, `ExellixMatrixDataTier`, etc.).
|
|
81
|
+
|
|
82
|
+
## Development
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
npm install
|
|
86
|
+
npm run build
|
|
87
|
+
npm test
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
### Live tests (real Mongo)
|
|
91
|
+
|
|
92
|
+
1. Copy **`.env.example`** → **`.env`** and set `MONGO_URI` (and optional `execution_db` / `config_db`).
|
|
93
|
+
2. Run:
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
npm run test:live
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
The default **`npm test`** suite skips live tests when `MONGO_URI` is unset. Vitest is configured with **`MODE=debug`** so importing `@exellix/exellix-runtime` in tests does not fail on `@x12i/funcx` mode checks.
|
|
100
|
+
|
|
101
|
+
More detail: [docs/integration.md](./docs/integration.md). For a minimal env template, see [`.env.example`](./.env.example).
|
|
102
|
+
|
|
103
|
+
## Specification
|
|
104
|
+
|
|
105
|
+
Normative persistence contract: **`createExellixMatrixDataTier`** in **`@exellix/exellix-runtime`** — see [client-integration.md](https://github.com/exellix/exellix-runtime/blob/main/docs/client-integration.md).
|
|
106
|
+
|
|
107
|
+
## License
|
|
108
|
+
|
|
109
|
+
UNLICENSED (internal Exellix package unless published otherwise).
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
/** Mongo collection names — mirror `@exellix/exellix-runtime` execution-matrix constants. */
|
|
2
|
+
export declare const EXECUTION_MATRIX_ROWS_COLLECTION: "exellix_execution_matrix_rows";
|
|
3
|
+
export declare const EXECUTION_MATRIX_FAILURES_COLLECTION: "exellix_execution_matrix_failures";
|
|
4
|
+
export declare const EXECUTION_MATRIX_SNAPSHOTS_COLLECTION: "exellix_execution_matrix_snapshots";
|
|
5
|
+
export declare const EXELLIX_MATRIX_CONFIGS_COLLECTION: "exellix_matrix_configs";
|
|
6
|
+
export declare const EXELLIX_GRAPH_CONFIGS_COLLECTION: "exellix_graph_configs";
|
|
7
|
+
//# sourceMappingURL=collection-names.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"collection-names.d.ts","sourceRoot":"","sources":["../src/collection-names.ts"],"names":[],"mappings":"AAAA,6FAA6F;AAC7F,eAAO,MAAM,gCAAgC,EAAG,+BAAwC,CAAC;AACzF,eAAO,MAAM,oCAAoC,EAAG,mCAA4C,CAAC;AACjG,eAAO,MAAM,qCAAqC,EAAG,oCAA6C,CAAC;AACnG,eAAO,MAAM,iCAAiC,EAAG,wBAAiC,CAAC;AACnF,eAAO,MAAM,gCAAgC,EAAG,uBAAgC,CAAC"}
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
/** Mongo collection names — mirror `@exellix/exellix-runtime` execution-matrix constants. */
|
|
2
|
+
export const EXECUTION_MATRIX_ROWS_COLLECTION = 'exellix_execution_matrix_rows';
|
|
3
|
+
export const EXECUTION_MATRIX_FAILURES_COLLECTION = 'exellix_execution_matrix_failures';
|
|
4
|
+
export const EXECUTION_MATRIX_SNAPSHOTS_COLLECTION = 'exellix_execution_matrix_snapshots';
|
|
5
|
+
export const EXELLIX_MATRIX_CONFIGS_COLLECTION = 'exellix_matrix_configs';
|
|
6
|
+
export const EXELLIX_GRAPH_CONFIGS_COLLECTION = 'exellix_graph_configs';
|
|
7
|
+
//# sourceMappingURL=collection-names.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"collection-names.js","sourceRoot":"","sources":["../src/collection-names.ts"],"names":[],"mappings":"AAAA,6FAA6F;AAC7F,MAAM,CAAC,MAAM,gCAAgC,GAAG,+BAAwC,CAAC;AACzF,MAAM,CAAC,MAAM,oCAAoC,GAAG,mCAA4C,CAAC;AACjG,MAAM,CAAC,MAAM,qCAAqC,GAAG,oCAA6C,CAAC;AACnG,MAAM,CAAC,MAAM,iCAAiC,GAAG,wBAAiC,CAAC;AACnF,MAAM,CAAC,MAAM,gCAAgC,GAAG,uBAAgC,CAAC"}
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
import type { CreateExellixMatrixDataTierOptions, ExellixMatrixDataTier } from './types.js';
|
|
2
|
+
/**
|
|
3
|
+
* Opens Mongo-backed persistence for Exellix execution-matrix operational data and config collections.
|
|
4
|
+
* Host applications import this factory instead of `mongodb` or `@x12i/xronox-store`.
|
|
5
|
+
*
|
|
6
|
+
* Databases (defaults): `exellix-runtime` (rows, failures, snapshots), `exellix` (matrix_configs, graph_configs).
|
|
7
|
+
* Indexes follow `@exellix/exellix-runtime` `src/execution-matrix/collections.ts`.
|
|
8
|
+
*/
|
|
9
|
+
export declare function createExellixMatrixDataTier(options?: CreateExellixMatrixDataTierOptions): Promise<ExellixMatrixDataTier>;
|
|
10
|
+
//# sourceMappingURL=create-exellix-matrix-data-tier.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"create-exellix-matrix-data-tier.d.ts","sourceRoot":"","sources":["../src/create-exellix-matrix-data-tier.ts"],"names":[],"mappings":"AAYA,OAAO,KAAK,EAAE,kCAAkC,EAAE,qBAAqB,EAAE,MAAM,YAAY,CAAC;AAE5F;;;;;;GAMG;AACH,wBAAsB,2BAA2B,CAC/C,OAAO,GAAE,kCAAuC,GAC/C,OAAO,CAAC,qBAAqB,CAAC,CA0DhC"}
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
import { createExellixConfigStoreFromXronox, createExecutionMatrixRuntimeFromXronox, } from '@exellix/exellix-runtime/internal/xronox-matrix';
|
|
2
|
+
import { EXECUTION_MATRIX_FAILURES_COLLECTION, EXECUTION_MATRIX_ROWS_COLLECTION, EXECUTION_MATRIX_SNAPSHOTS_COLLECTION, EXELLIX_GRAPH_CONFIGS_COLLECTION, EXELLIX_MATRIX_CONFIGS_COLLECTION, } from './collection-names.js';
|
|
3
|
+
import { resolveExellixMatrixDataTierOptions } from './resolve-options.js';
|
|
4
|
+
/**
|
|
5
|
+
* Opens Mongo-backed persistence for Exellix execution-matrix operational data and config collections.
|
|
6
|
+
* Host applications import this factory instead of `mongodb` or `@x12i/xronox-store`.
|
|
7
|
+
*
|
|
8
|
+
* Databases (defaults): `exellix-runtime` (rows, failures, snapshots), `exellix` (matrix_configs, graph_configs).
|
|
9
|
+
* Indexes follow `@exellix/exellix-runtime` `src/execution-matrix/collections.ts`.
|
|
10
|
+
*/
|
|
11
|
+
export async function createExellixMatrixDataTier(options = {}) {
|
|
12
|
+
const resolved = resolveExellixMatrixDataTierOptions(options);
|
|
13
|
+
const passThrough = resolved.mongoDriverPassThrough !== false;
|
|
14
|
+
const exec = await createExecutionMatrixRuntimeFromXronox({
|
|
15
|
+
mongoUri: resolved.mongoUri,
|
|
16
|
+
mongoDb: resolved.executionDb,
|
|
17
|
+
mongoRole: resolved.executionMongoRole,
|
|
18
|
+
mongoDriverPassThrough: passThrough,
|
|
19
|
+
serializeClaims: resolved.serializeClaims ?? true,
|
|
20
|
+
});
|
|
21
|
+
const cfg = await createExellixConfigStoreFromXronox({
|
|
22
|
+
mongoUri: resolved.mongoUri,
|
|
23
|
+
mongoDb: resolved.configDb,
|
|
24
|
+
mongoRole: resolved.configMongoRole,
|
|
25
|
+
mongoDriverPassThrough: passThrough,
|
|
26
|
+
});
|
|
27
|
+
const rows = exec.store.collection(EXECUTION_MATRIX_ROWS_COLLECTION);
|
|
28
|
+
const failures = exec.store.collection(EXECUTION_MATRIX_FAILURES_COLLECTION);
|
|
29
|
+
const snapshots = exec.store.collection(EXECUTION_MATRIX_SNAPSHOTS_COLLECTION);
|
|
30
|
+
const matrices = cfg.store.collection(EXELLIX_MATRIX_CONFIGS_COLLECTION);
|
|
31
|
+
const graphs = cfg.store.collection(EXELLIX_GRAPH_CONFIGS_COLLECTION);
|
|
32
|
+
let closed = false;
|
|
33
|
+
const close = async () => {
|
|
34
|
+
if (closed) {
|
|
35
|
+
return;
|
|
36
|
+
}
|
|
37
|
+
closed = true;
|
|
38
|
+
await exec.store.close().catch(() => undefined);
|
|
39
|
+
await cfg.store.close().catch(() => undefined);
|
|
40
|
+
await exec.closeMongoDriverPassThrough?.().catch(() => undefined);
|
|
41
|
+
await cfg.closeMongoDriverPassThrough?.().catch(() => undefined);
|
|
42
|
+
};
|
|
43
|
+
const probe = {
|
|
44
|
+
executionStore: exec.store,
|
|
45
|
+
configStore: cfg.store,
|
|
46
|
+
executionMongoUri: resolved.mongoUri,
|
|
47
|
+
configMongoUri: resolved.mongoUri,
|
|
48
|
+
};
|
|
49
|
+
return {
|
|
50
|
+
rows,
|
|
51
|
+
failures,
|
|
52
|
+
snapshots,
|
|
53
|
+
matrices,
|
|
54
|
+
graphs,
|
|
55
|
+
init: async () => undefined,
|
|
56
|
+
close,
|
|
57
|
+
executionStore: exec.store,
|
|
58
|
+
configStore: cfg.store,
|
|
59
|
+
executionMongoUri: resolved.mongoUri,
|
|
60
|
+
configMongoUri: resolved.mongoUri,
|
|
61
|
+
probe,
|
|
62
|
+
};
|
|
63
|
+
}
|
|
64
|
+
//# sourceMappingURL=create-exellix-matrix-data-tier.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"create-exellix-matrix-data-tier.js","sourceRoot":"","sources":["../src/create-exellix-matrix-data-tier.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,kCAAkC,EAClC,sCAAsC,GACvC,MAAM,iDAAiD,CAAC;AACzD,OAAO,EACL,oCAAoC,EACpC,gCAAgC,EAChC,qCAAqC,EACrC,gCAAgC,EAChC,iCAAiC,GAClC,MAAM,uBAAuB,CAAC;AAC/B,OAAO,EAAE,mCAAmC,EAAE,MAAM,sBAAsB,CAAC;AAG3E;;;;;;GAMG;AACH,MAAM,CAAC,KAAK,UAAU,2BAA2B,CAC/C,UAA8C,EAAE;IAEhD,MAAM,QAAQ,GAAG,mCAAmC,CAAC,OAAO,CAAC,CAAC;IAC9D,MAAM,WAAW,GAAG,QAAQ,CAAC,sBAAsB,KAAK,KAAK,CAAC;IAE9D,MAAM,IAAI,GAAG,MAAM,sCAAsC,CAAC;QACxD,QAAQ,EAAE,QAAQ,CAAC,QAAQ;QAC3B,OAAO,EAAE,QAAQ,CAAC,WAAW;QAC7B,SAAS,EAAE,QAAQ,CAAC,kBAAkB;QACtC,sBAAsB,EAAE,WAAW;QACnC,eAAe,EAAE,QAAQ,CAAC,eAAe,IAAI,IAAI;KAClD,CAAC,CAAC;IAEH,MAAM,GAAG,GAAG,MAAM,kCAAkC,CAAC;QACnD,QAAQ,EAAE,QAAQ,CAAC,QAAQ;QAC3B,OAAO,EAAE,QAAQ,CAAC,QAAQ;QAC1B,SAAS,EAAE,QAAQ,CAAC,eAAe;QACnC,sBAAsB,EAAE,WAAW;KACpC,CAAC,CAAC;IAEH,MAAM,IAAI,GAAG,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,gCAAgC,CAAC,CAAC;IACrE,MAAM,QAAQ,GAAG,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,oCAAoC,CAAC,CAAC;IAC7E,MAAM,SAAS,GAAG,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,qCAAqC,CAAC,CAAC;IAC/E,MAAM,QAAQ,GAAG,GAAG,CAAC,KAAK,CAAC,UAAU,CAAC,iCAAiC,CAAC,CAAC;IACzE,MAAM,MAAM,GAAG,GAAG,CAAC,KAAK,CAAC,UAAU,CAAC,gCAAgC,CAAC,CAAC;IAEtE,IAAI,MAAM,GAAG,KAAK,CAAC;IACnB,MAAM,KAAK,GAAG,KAAK,IAAmB,EAAE;QACtC,IAAI,MAAM,EAAE,CAAC;YACX,OAAO;QACT,CAAC;QACD,MAAM,GAAG,IAAI,CAAC;QACd,MAAM,IAAI,CAAC,KAAK,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;QAChD,MAAM,GAAG,CAAC,KAAK,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;QAC/C,MAAM,IAAI,CAAC,2BAA2B,EAAE,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;QAClE,MAAM,GAAG,CAAC,2BAA2B,EAAE,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;IACnE,CAAC,CAAC;IAEF,MAAM,KAAK,GAAG;QACZ,cAAc,EAAE,IAAI,CAAC,KAAK;QAC1B,WAAW,EAAE,GAAG,CAAC,KAAK;QACtB,iBAAiB,EAAE,QAAQ,CAAC,QAAQ;QACpC,cAAc,EAAE,QAAQ,CAAC,QAAQ;KAClC,CAAC;IAEF,OAAO;QACL,IAAI;QACJ,QAAQ;QACR,SAAS;QACT,QAAQ;QACR,MAAM;QACN,IAAI,EAAE,KAAK,IAAI,EAAE,CAAC,SAAS;QAC3B,KAAK;QACL,cAAc,EAAE,IAAI,CAAC,KAAK;QAC1B,WAAW,EAAE,GAAG,CAAC,KAAK;QACtB,iBAAiB,EAAE,QAAQ,CAAC,QAAQ;QACpC,cAAc,EAAE,QAAQ,CAAC,QAAQ;QACjC,KAAK;KACN,CAAC;AACJ,CAAC"}
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
1
|
+
export { createExellixMatrixDataTier } from './create-exellix-matrix-data-tier.js';
|
|
2
|
+
export { loadMatrixDataTierFactory, type MatrixDataTierFactory } from './load-matrix-data-tier-factory.js';
|
|
3
|
+
export { resolveExellixMatrixDataTierOptions } from './resolve-options.js';
|
|
4
|
+
export type { CreateExellixMatrixDataTierOptions, ExellixMatrixDataTier, ExellixMatrixDataTierProbe, ExellixMatrixPersistenceCollection, } from './types.js';
|
|
5
|
+
//# sourceMappingURL=index.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,2BAA2B,EAAE,MAAM,sCAAsC,CAAC;AACnF,OAAO,EAAE,yBAAyB,EAAE,KAAK,qBAAqB,EAAE,MAAM,oCAAoC,CAAC;AAC3G,OAAO,EAAE,mCAAmC,EAAE,MAAM,sBAAsB,CAAC;AAC3E,YAAY,EACV,kCAAkC,EAClC,qBAAqB,EACrB,0BAA0B,EAC1B,kCAAkC,GACnC,MAAM,YAAY,CAAC"}
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
export { createExellixMatrixDataTier } from './create-exellix-matrix-data-tier.js';
|
|
2
|
+
export { loadMatrixDataTierFactory } from './load-matrix-data-tier-factory.js';
|
|
3
|
+
export { resolveExellixMatrixDataTierOptions } from './resolve-options.js';
|
|
4
|
+
//# sourceMappingURL=index.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,2BAA2B,EAAE,MAAM,sCAAsC,CAAC;AACnF,OAAO,EAAE,yBAAyB,EAA8B,MAAM,oCAAoC,CAAC;AAC3G,OAAO,EAAE,mCAAmC,EAAE,MAAM,sBAAsB,CAAC"}
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
export type MatrixDataTierFactory = (options: Record<string, unknown>) => Promise<unknown>;
|
|
2
|
+
/**
|
|
3
|
+
* Resolves `createExellixMatrixDataTier` from this package or, when upstream re-exports it, `@x12i/xmemory-store`.
|
|
4
|
+
*/
|
|
5
|
+
export declare function loadMatrixDataTierFactory(): Promise<MatrixDataTierFactory>;
|
|
6
|
+
//# sourceMappingURL=load-matrix-data-tier-factory.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"load-matrix-data-tier-factory.d.ts","sourceRoot":"","sources":["../src/load-matrix-data-tier-factory.ts"],"names":[],"mappings":"AAIA,MAAM,MAAM,qBAAqB,GAAG,CAClC,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,KAC7B,OAAO,CAAC,OAAO,CAAC,CAAC;AAEtB;;GAEG;AACH,wBAAsB,yBAAyB,IAAI,OAAO,CAAC,qBAAqB,CAAC,CAoBhF"}
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
const FACTORY = 'createExellixMatrixDataTier';
|
|
2
|
+
const FACTORY_PACKAGES = ['@exellix/exellix-matrix-persister', '@x12i/xmemory-store'];
|
|
3
|
+
/**
|
|
4
|
+
* Resolves `createExellixMatrixDataTier` from this package or, when upstream re-exports it, `@x12i/xmemory-store`.
|
|
5
|
+
*/
|
|
6
|
+
export async function loadMatrixDataTierFactory() {
|
|
7
|
+
const errors = [];
|
|
8
|
+
for (const pkg of FACTORY_PACKAGES) {
|
|
9
|
+
try {
|
|
10
|
+
const mod = (await import(pkg));
|
|
11
|
+
const factory = mod[FACTORY];
|
|
12
|
+
if (typeof factory === 'function') {
|
|
13
|
+
return factory;
|
|
14
|
+
}
|
|
15
|
+
errors.push(`${pkg}: "${FACTORY}" is not exported`);
|
|
16
|
+
}
|
|
17
|
+
catch (err) {
|
|
18
|
+
const message = err instanceof Error ? err.message : String(err);
|
|
19
|
+
errors.push(`${pkg}: ${message}`);
|
|
20
|
+
}
|
|
21
|
+
}
|
|
22
|
+
throw new Error(`createExellixMatrixDataTier is not available. Tried:\n${errors.map((e) => ` - ${e}`).join('\n')}`);
|
|
23
|
+
}
|
|
24
|
+
//# sourceMappingURL=load-matrix-data-tier-factory.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"load-matrix-data-tier-factory.js","sourceRoot":"","sources":["../src/load-matrix-data-tier-factory.ts"],"names":[],"mappings":"AAAA,MAAM,OAAO,GAAG,6BAA6B,CAAC;AAE9C,MAAM,gBAAgB,GAAG,CAAC,mCAAmC,EAAE,qBAAqB,CAAU,CAAC;AAM/F;;GAEG;AACH,MAAM,CAAC,KAAK,UAAU,yBAAyB;IAC7C,MAAM,MAAM,GAAa,EAAE,CAAC;IAE5B,KAAK,MAAM,GAAG,IAAI,gBAAgB,EAAE,CAAC;QACnC,IAAI,CAAC;YACH,MAAM,GAAG,GAAG,CAAC,MAAM,MAAM,CAAC,GAAG,CAAC,CAA4B,CAAC;YAC3D,MAAM,OAAO,GAAG,GAAG,CAAC,OAAO,CAAC,CAAC;YAC7B,IAAI,OAAO,OAAO,KAAK,UAAU,EAAE,CAAC;gBAClC,OAAO,OAAgC,CAAC;YAC1C,CAAC;YACD,MAAM,CAAC,IAAI,CAAC,GAAG,GAAG,MAAM,OAAO,mBAAmB,CAAC,CAAC;QACtD,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,OAAO,GAAG,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;YACjE,MAAM,CAAC,IAAI,CAAC,GAAG,GAAG,KAAK,OAAO,EAAE,CAAC,CAAC;QACpC,CAAC;IACH,CAAC;IAED,MAAM,IAAI,KAAK,CACb,yDAAyD,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CACpG,CAAC;AACJ,CAAC"}
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
import type { CreateExellixMatrixDataTierOptions } from './types.js';
|
|
2
|
+
export type ResolvedExellixMatrixDataTierOptions = CreateExellixMatrixDataTierOptions & {
|
|
3
|
+
mongoUri: string;
|
|
4
|
+
executionDb: string;
|
|
5
|
+
configDb: string;
|
|
6
|
+
};
|
|
7
|
+
/**
|
|
8
|
+
* Merges caller options with `MONGO_URI`, `execution_db`, and `config_db` (ERC / jobs convention).
|
|
9
|
+
*/
|
|
10
|
+
export declare function resolveExellixMatrixDataTierOptions(overrides?: CreateExellixMatrixDataTierOptions, env?: NodeJS.ProcessEnv): ResolvedExellixMatrixDataTierOptions;
|
|
11
|
+
//# sourceMappingURL=resolve-options.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"resolve-options.d.ts","sourceRoot":"","sources":["../src/resolve-options.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,kCAAkC,EAAE,MAAM,YAAY,CAAC;AAKrE,MAAM,MAAM,oCAAoC,GAAG,kCAAkC,GAAG;IACtF,QAAQ,EAAE,MAAM,CAAC;IACjB,WAAW,EAAE,MAAM,CAAC;IACpB,QAAQ,EAAE,MAAM,CAAC;CAClB,CAAC;AAEF;;GAEG;AACH,wBAAgB,mCAAmC,CACjD,SAAS,GAAE,kCAAuC,EAClD,GAAG,GAAE,MAAM,CAAC,UAAwB,GACnC,oCAAoC,CAatC"}
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
const DEFAULT_EXECUTION_DB = 'exellix-runtime';
|
|
2
|
+
const DEFAULT_CONFIG_DB = 'exellix';
|
|
3
|
+
/**
|
|
4
|
+
* Merges caller options with `MONGO_URI`, `execution_db`, and `config_db` (ERC / jobs convention).
|
|
5
|
+
*/
|
|
6
|
+
export function resolveExellixMatrixDataTierOptions(overrides = {}, env = process.env) {
|
|
7
|
+
const mongoUri = (overrides.mongoUri ?? env.MONGO_URI?.trim() ?? '').trim();
|
|
8
|
+
if (!mongoUri) {
|
|
9
|
+
throw new Error('createExellixMatrixDataTier: mongoUri missing — set MONGO_URI or pass options.mongoUri');
|
|
10
|
+
}
|
|
11
|
+
return {
|
|
12
|
+
...overrides,
|
|
13
|
+
mongoUri,
|
|
14
|
+
executionDb: overrides.executionDb ?? (env.execution_db?.trim() || DEFAULT_EXECUTION_DB),
|
|
15
|
+
configDb: overrides.configDb ?? (env.config_db?.trim() || DEFAULT_CONFIG_DB),
|
|
16
|
+
};
|
|
17
|
+
}
|
|
18
|
+
//# sourceMappingURL=resolve-options.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"resolve-options.js","sourceRoot":"","sources":["../src/resolve-options.ts"],"names":[],"mappings":"AAEA,MAAM,oBAAoB,GAAG,iBAAiB,CAAC;AAC/C,MAAM,iBAAiB,GAAG,SAAS,CAAC;AAQpC;;GAEG;AACH,MAAM,UAAU,mCAAmC,CACjD,YAAgD,EAAE,EAClD,MAAyB,OAAO,CAAC,GAAG;IAEpC,MAAM,QAAQ,GAAG,CAAC,SAAS,CAAC,QAAQ,IAAI,GAAG,CAAC,SAAS,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;IAC5E,IAAI,CAAC,QAAQ,EAAE,CAAC;QACd,MAAM,IAAI,KAAK,CACb,wFAAwF,CACzF,CAAC;IACJ,CAAC;IACD,OAAO;QACL,GAAG,SAAS;QACZ,QAAQ;QACR,WAAW,EAAE,SAAS,CAAC,WAAW,IAAI,CAAC,GAAG,CAAC,YAAY,EAAE,IAAI,EAAE,IAAI,oBAAoB,CAAC;QACxF,QAAQ,EAAE,SAAS,CAAC,QAAQ,IAAI,CAAC,GAAG,CAAC,SAAS,EAAE,IAAI,EAAE,IAAI,iBAAiB,CAAC;KAC7E,CAAC;AACJ,CAAC"}
|
package/dist/types.d.ts
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
import type { createExecutionMatrixRuntime } from '@exellix/exellix-runtime';
|
|
2
|
+
import type { createExecutionMatrixRuntimeFromXronox, createExellixConfigStoreFromXronox } from '@exellix/exellix-runtime/internal/xronox-matrix';
|
|
3
|
+
/** xronox-shaped collection handle expected by `@exellix/exellix-runtime` factories. */
|
|
4
|
+
export type ExellixMatrixPersistenceCollection = Parameters<typeof createExecutionMatrixRuntime>[0];
|
|
5
|
+
type ExecutionMatrixXronoxBundle = Awaited<ReturnType<typeof createExecutionMatrixRuntimeFromXronox>>;
|
|
6
|
+
type ConfigXronoxBundle = Awaited<ReturnType<typeof createExellixConfigStoreFromXronox>>;
|
|
7
|
+
export type CreateExellixMatrixDataTierOptions = {
|
|
8
|
+
mongoUri?: string;
|
|
9
|
+
executionDb?: string;
|
|
10
|
+
configDb?: string;
|
|
11
|
+
executionMongoRole?: string;
|
|
12
|
+
configMongoRole?: string;
|
|
13
|
+
/**
|
|
14
|
+
* When `false`, skips native `mongodb` pass-through hooks on the underlying xronox store
|
|
15
|
+
* (legacy behavior). Default `true`.
|
|
16
|
+
*/
|
|
17
|
+
mongoDriverPassThrough?: boolean;
|
|
18
|
+
/** Passed through to {@link createExecutionMatrixRuntime} (default `true` for workers). */
|
|
19
|
+
serializeClaims?: boolean;
|
|
20
|
+
[key: string]: unknown;
|
|
21
|
+
};
|
|
22
|
+
export type ExellixMatrixDataTierProbe = {
|
|
23
|
+
executionStore: ExecutionMatrixXronoxBundle['store'];
|
|
24
|
+
configStore: ConfigXronoxBundle['store'];
|
|
25
|
+
executionMongoUri: string;
|
|
26
|
+
configMongoUri: string;
|
|
27
|
+
};
|
|
28
|
+
export type ExellixMatrixDataTier = {
|
|
29
|
+
rows: ExellixMatrixPersistenceCollection;
|
|
30
|
+
failures: ExellixMatrixPersistenceCollection;
|
|
31
|
+
snapshots: ExellixMatrixPersistenceCollection;
|
|
32
|
+
matrices: ExellixMatrixPersistenceCollection;
|
|
33
|
+
graphs: ExellixMatrixPersistenceCollection;
|
|
34
|
+
/** Optional warm-up (stores are initialized during factory creation; this is a no-op by default). */
|
|
35
|
+
init?: () => Promise<void>;
|
|
36
|
+
close: () => Promise<void>;
|
|
37
|
+
/** Optional handles for dependency health probes (`/health?probe=dependencies`). */
|
|
38
|
+
executionStore?: ExecutionMatrixXronoxBundle['store'];
|
|
39
|
+
configStore?: ConfigXronoxBundle['store'];
|
|
40
|
+
executionMongoUri?: string;
|
|
41
|
+
configMongoUri?: string;
|
|
42
|
+
probe?: ExellixMatrixDataTierProbe;
|
|
43
|
+
};
|
|
44
|
+
export {};
|
|
45
|
+
//# sourceMappingURL=types.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,4BAA4B,EAAE,MAAM,0BAA0B,CAAC;AAC7E,OAAO,KAAK,EACV,sCAAsC,EACtC,kCAAkC,EACnC,MAAM,iDAAiD,CAAC;AAEzD,wFAAwF;AACxF,MAAM,MAAM,kCAAkC,GAAG,UAAU,CAAC,OAAO,4BAA4B,CAAC,CAAC,CAAC,CAAC,CAAC;AAEpG,KAAK,2BAA2B,GAAG,OAAO,CAAC,UAAU,CAAC,OAAO,sCAAsC,CAAC,CAAC,CAAC;AACtG,KAAK,kBAAkB,GAAG,OAAO,CAAC,UAAU,CAAC,OAAO,kCAAkC,CAAC,CAAC,CAAC;AAEzF,MAAM,MAAM,kCAAkC,GAAG;IAC/C,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB;;;OAGG;IACH,sBAAsB,CAAC,EAAE,OAAO,CAAC;IACjC,2FAA2F;IAC3F,eAAe,CAAC,EAAE,OAAO,CAAC;IAC1B,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB,CAAC;AAEF,MAAM,MAAM,0BAA0B,GAAG;IACvC,cAAc,EAAE,2BAA2B,CAAC,OAAO,CAAC,CAAC;IACrD,WAAW,EAAE,kBAAkB,CAAC,OAAO,CAAC,CAAC;IACzC,iBAAiB,EAAE,MAAM,CAAC;IAC1B,cAAc,EAAE,MAAM,CAAC;CACxB,CAAC;AAEF,MAAM,MAAM,qBAAqB,GAAG;IAClC,IAAI,EAAE,kCAAkC,CAAC;IACzC,QAAQ,EAAE,kCAAkC,CAAC;IAC7C,SAAS,EAAE,kCAAkC,CAAC;IAC9C,QAAQ,EAAE,kCAAkC,CAAC;IAC7C,MAAM,EAAE,kCAAkC,CAAC;IAC3C,qGAAqG;IACrG,IAAI,CAAC,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IAC3B,KAAK,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IAC3B,oFAAoF;IACpF,cAAc,CAAC,EAAE,2BAA2B,CAAC,OAAO,CAAC,CAAC;IACtD,WAAW,CAAC,EAAE,kBAAkB,CAAC,OAAO,CAAC,CAAC;IAC1C,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,KAAK,CAAC,EAAE,0BAA0B,CAAC;CACpC,CAAC"}
|
package/dist/types.js
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"types.js","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":""}
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# Integration
|
|
2
|
+
|
|
3
|
+
## Install
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
npm install @exellix/exellix-matrix-persister @exellix/exellix-runtime
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
## Factory
|
|
10
|
+
|
|
11
|
+
```ts
|
|
12
|
+
import { createExellixMatrixDataTier } from '@exellix/exellix-matrix-persister';
|
|
13
|
+
|
|
14
|
+
const tier = await createExellixMatrixDataTier({
|
|
15
|
+
mongoUri: process.env.MONGO_URI,
|
|
16
|
+
executionDb: 'exellix-runtime',
|
|
17
|
+
configDb: 'exellix',
|
|
18
|
+
});
|
|
19
|
+
|
|
20
|
+
await tier.init?.();
|
|
21
|
+
// tier.rows | failures | snapshots | matrices | graphs → createMatrixWorker / openMatrixReadTier
|
|
22
|
+
await tier.close();
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
## Environment
|
|
26
|
+
|
|
27
|
+
| Variable | Default | Purpose |
|
|
28
|
+
|----------|---------|---------|
|
|
29
|
+
| `MONGO_URI` | _(required)_ | Mongo connection string |
|
|
30
|
+
| `execution_db` | `exellix-runtime` | Rows, failures, snapshots |
|
|
31
|
+
| `config_db` | `exellix` | Matrix and graph config |
|
|
32
|
+
|
|
33
|
+
## `@exellix/exellix-jobs` / `@exellix/exellix-matrix-read`
|
|
34
|
+
|
|
35
|
+
Add a peer or dependency on `@exellix/exellix-matrix-persister`. Consumer packages resolve `createExellixMatrixDataTier` from this package first, then from `@x12i/xmemory-store` when re-exported upstream.
|
|
36
|
+
|
|
37
|
+
## Live test
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
MONGO_URI="mongodb://localhost:27017" npm run test:live
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Skips automatically when `MONGO_URI` is unset.
|
|
44
|
+
|
|
45
|
+
## Normative FR
|
|
46
|
+
|
|
47
|
+
See `@exellix/exellix-runtime` [exellix-matrix-mongo-persistence-fr.md](https://github.com/exellix/exellix-runtime/blob/main/docs/platform-requests/requests/exellix-matrix-mongo-persistence-fr.md).
|
package/package.json
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@exellix/exellix-matrix-persister",
|
|
3
|
+
"version": "1.0.1",
|
|
4
|
+
"description": "Mongo persistence factory for Exellix execution-matrix rows, failures, snapshots, and matrix/graph config (hides mongodb/xronox from hosts).",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"engines": {
|
|
7
|
+
"node": ">=20"
|
|
8
|
+
},
|
|
9
|
+
"publishConfig": {
|
|
10
|
+
"access": "public",
|
|
11
|
+
"registry": "https://registry.npmjs.org/"
|
|
12
|
+
},
|
|
13
|
+
"main": "./dist/index.js",
|
|
14
|
+
"types": "./dist/index.d.ts",
|
|
15
|
+
"exports": {
|
|
16
|
+
".": {
|
|
17
|
+
"types": "./dist/index.d.ts",
|
|
18
|
+
"import": "./dist/index.js"
|
|
19
|
+
}
|
|
20
|
+
},
|
|
21
|
+
"files": [
|
|
22
|
+
"dist",
|
|
23
|
+
"docs",
|
|
24
|
+
"README.md",
|
|
25
|
+
".env.example"
|
|
26
|
+
],
|
|
27
|
+
"scripts": {
|
|
28
|
+
"build": "tsc -p tsconfig.build.json",
|
|
29
|
+
"clean": "node -e \"import('node:fs').then(fs=>fs.promises.rm('dist',{recursive:true,force:true}))\"",
|
|
30
|
+
"prepack": "npm run build",
|
|
31
|
+
"test": "vitest run",
|
|
32
|
+
"test:live": "vitest run test/create-exellix-matrix-data-tier.live.spec.ts",
|
|
33
|
+
"test:watch": "vitest"
|
|
34
|
+
},
|
|
35
|
+
"peerDependencies": {
|
|
36
|
+
"@exellix/exellix-runtime": "^5.1.8"
|
|
37
|
+
},
|
|
38
|
+
"devDependencies": {
|
|
39
|
+
"@exellix/exellix-runtime": "^5.1.8",
|
|
40
|
+
"@types/node": "^22.10.2",
|
|
41
|
+
"typescript": "^5.7.2",
|
|
42
|
+
"vitest": "^3.0.0"
|
|
43
|
+
},
|
|
44
|
+
"keywords": [
|
|
45
|
+
"exellix",
|
|
46
|
+
"execution-matrix",
|
|
47
|
+
"mongodb",
|
|
48
|
+
"persistence"
|
|
49
|
+
],
|
|
50
|
+
"license": "exellix-license"
|
|
51
|
+
}
|