@memberjunction/materialization 0.0.0 → 6.1.0-edge.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +183 -0
- package/README.md +25 -32
- package/dist/MaterializationFreshness.d.ts +55 -0
- package/dist/MaterializationFreshness.d.ts.map +1 -0
- package/dist/MaterializationFreshness.js +124 -0
- package/dist/MaterializationFreshness.js.map +1 -0
- package/dist/MaterializationRefresher.d.ts +621 -0
- package/dist/MaterializationRefresher.d.ts.map +1 -0
- package/dist/MaterializationRefresher.js +1558 -0
- package/dist/MaterializationRefresher.js.map +1 -0
- package/dist/index.d.ts +3 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +3 -0
- package/dist/index.js.map +1 -0
- package/package.json +32 -8
package/LICENSE
ADDED
|
@@ -0,0 +1,183 @@
|
|
|
1
|
+
Business Source License 1.1
|
|
2
|
+
|
|
3
|
+
License text copyright (c) 2024 MariaDB plc, All Rights Reserved.
|
|
4
|
+
"Business Source License" is a trademark of MariaDB plc.
|
|
5
|
+
|
|
6
|
+
-----------------------------------------------------------------------------
|
|
7
|
+
|
|
8
|
+
Parameters
|
|
9
|
+
|
|
10
|
+
Licensor: Blue Cypress, Inc.
|
|
11
|
+
|
|
12
|
+
Licensed Work: MemberJunction.
|
|
13
|
+
The Licensed Work is (c) 2023-2026 Blue Cypress, Inc.
|
|
14
|
+
|
|
15
|
+
Additional Use Grant: Subject to the terms of this License, Licensor grants
|
|
16
|
+
you the following additional rights to make Production
|
|
17
|
+
Use of the Licensed Work.
|
|
18
|
+
|
|
19
|
+
1. Internal Use
|
|
20
|
+
|
|
21
|
+
You may make production use of the Licensed Work for
|
|
22
|
+
your own internal business or organizational operations.
|
|
23
|
+
|
|
24
|
+
2. Nonprofit Use
|
|
25
|
+
|
|
26
|
+
If you are a Nonprofit, you may make production use of
|
|
27
|
+
the Licensed Work for the operations and activities of
|
|
28
|
+
your Organizational Family.
|
|
29
|
+
|
|
30
|
+
3. MemberJunction Certified Program Use
|
|
31
|
+
|
|
32
|
+
If you are authorized by Licensor under the
|
|
33
|
+
MemberJunction Certified Program to provide professional
|
|
34
|
+
services using the Licensed Work, you may make
|
|
35
|
+
production use of the Licensed Work in providing such
|
|
36
|
+
professional services to a client, provided that:
|
|
37
|
+
|
|
38
|
+
(a) the Licensed Work is deployed in, and the applicable
|
|
39
|
+
production use occurs within, an environment owned,
|
|
40
|
+
leased, licensed, subscribed to, or otherwise controlled
|
|
41
|
+
by that client; and
|
|
42
|
+
|
|
43
|
+
(b) the production use is for that client's own internal
|
|
44
|
+
business or organizational operations or is otherwise
|
|
45
|
+
independently permitted to that client under this
|
|
46
|
+
Additional Use Grant.
|
|
47
|
+
|
|
48
|
+
4. Definitions Applicable to the Additional Use Grant
|
|
49
|
+
|
|
50
|
+
"Affiliate" means, with respect to a specified Person,
|
|
51
|
+
any other Person that directly or indirectly Controls,
|
|
52
|
+
is Controlled by, or is under common Control with such
|
|
53
|
+
specified Person.
|
|
54
|
+
|
|
55
|
+
"Control" (including the terms "Controls," "Controlled
|
|
56
|
+
by," and "under common Control with") means the direct
|
|
57
|
+
or indirect possession of the power to direct or cause
|
|
58
|
+
the direction of the management and policies of a
|
|
59
|
+
Person, whether through ownership of voting interests,
|
|
60
|
+
by contract, or otherwise.
|
|
61
|
+
|
|
62
|
+
"Organizational Family" means, with respect to a Person,
|
|
63
|
+
(a) such Person and its Affiliates, and (b) any
|
|
64
|
+
nonprofit organization, governmental entity, chapter,
|
|
65
|
+
division, local affiliate, regional affiliate, state
|
|
66
|
+
affiliate, national affiliate, or other entity that is
|
|
67
|
+
formally affiliated with such Person through governing
|
|
68
|
+
documents, a charter, bylaws, a membership agreement, or
|
|
69
|
+
another written organizational instrument, and is
|
|
70
|
+
recognized under such documents as part of the same
|
|
71
|
+
organizational structure.
|
|
72
|
+
|
|
73
|
+
"Nonprofit" means a Person recognized by the Internal
|
|
74
|
+
Revenue Service as exempt from federal income taxation
|
|
75
|
+
under Section 501(c)(3), 501(c)(4), 501(c)(5), or
|
|
76
|
+
501(c)(6) of the Internal Revenue Code, or a foreign
|
|
77
|
+
organization recognized under substantially equivalent
|
|
78
|
+
laws.
|
|
79
|
+
|
|
80
|
+
A Person claiming eligibility as a Nonprofit shall, upon
|
|
81
|
+
Licensor's reasonable request, provide documentation
|
|
82
|
+
reasonably sufficient to demonstrate that it qualifies
|
|
83
|
+
as a Nonprofit. If such Person materially misrepresents,
|
|
84
|
+
or is unable to demonstrate, its qualification as a
|
|
85
|
+
Nonprofit, the rights granted to such Person under
|
|
86
|
+
Section 2 of this Additional Use Grant shall terminate.
|
|
87
|
+
|
|
88
|
+
"MemberJunction Certified Program" means Licensor's
|
|
89
|
+
then-current program for certifying and authorizing a
|
|
90
|
+
Person to provide professional services using the
|
|
91
|
+
Licensed Work.
|
|
92
|
+
|
|
93
|
+
"Person" means any individual, corporation, limited
|
|
94
|
+
liability company, partnership, association, nonprofit
|
|
95
|
+
organization, governmental entity, or other legal or
|
|
96
|
+
organizational entity.
|
|
97
|
+
|
|
98
|
+
Change Date: Four (4) years from the date the Licensed Work is first
|
|
99
|
+
made available.
|
|
100
|
+
|
|
101
|
+
Change License: MIT License.
|
|
102
|
+
|
|
103
|
+
For information about alternative licensing arrangements for the Licensed
|
|
104
|
+
Work, please contact Blue Cypress, Inc.
|
|
105
|
+
|
|
106
|
+
-----------------------------------------------------------------------------
|
|
107
|
+
|
|
108
|
+
Terms
|
|
109
|
+
|
|
110
|
+
The Licensor hereby grants you the right to copy, modify, create derivative
|
|
111
|
+
works, redistribute, and make non-production use of the Licensed Work. The
|
|
112
|
+
Licensor may make an Additional Use Grant, above, permitting limited
|
|
113
|
+
production use.
|
|
114
|
+
|
|
115
|
+
Effective on the Change Date, or the fourth anniversary of the first publicly
|
|
116
|
+
available distribution of a specific version of the Licensed Work under this
|
|
117
|
+
License, whichever comes first, the Licensor hereby grants you rights under
|
|
118
|
+
the terms of the Change License, and the rights granted in the paragraph
|
|
119
|
+
above terminate.
|
|
120
|
+
|
|
121
|
+
If your use of the Licensed Work does not comply with the requirements
|
|
122
|
+
currently in effect as described in this License, you must purchase a
|
|
123
|
+
commercial license from the Licensor, its affiliated entities, or authorized
|
|
124
|
+
resellers, or you must refrain from using the Licensed Work.
|
|
125
|
+
|
|
126
|
+
All copies of the original and modified Licensed Work, and derivative works
|
|
127
|
+
of the Licensed Work, are subject to this License. This License applies
|
|
128
|
+
separately for each version of the Licensed Work and the Change Date may vary
|
|
129
|
+
for each version of the Licensed Work released by Licensor.
|
|
130
|
+
|
|
131
|
+
You must conspicuously display this License on each original or modified copy
|
|
132
|
+
of the Licensed Work. If you receive the Licensed Work in original or
|
|
133
|
+
modified form from a third party, the terms and conditions set forth in this
|
|
134
|
+
License apply to your use of that work.
|
|
135
|
+
|
|
136
|
+
Any use of the Licensed Work in violation of this License will automatically
|
|
137
|
+
terminate your rights under this License for the current and all other
|
|
138
|
+
versions of the Licensed Work.
|
|
139
|
+
|
|
140
|
+
This License does not grant you any right in any trademark or logo of
|
|
141
|
+
Licensor or its affiliates (provided that you may use a trademark or logo of
|
|
142
|
+
Licensor as expressly required by this License).
|
|
143
|
+
|
|
144
|
+
TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON
|
|
145
|
+
AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS,
|
|
146
|
+
EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF
|
|
147
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND
|
|
148
|
+
TITLE.
|
|
149
|
+
|
|
150
|
+
MariaDB hereby grants you permission to use this License's text to license
|
|
151
|
+
your works, and to refer to it using the trademark "Business Source License",
|
|
152
|
+
as long as you comply with the Covenants of Licensor below.
|
|
153
|
+
|
|
154
|
+
-----------------------------------------------------------------------------
|
|
155
|
+
|
|
156
|
+
Covenants of Licensor
|
|
157
|
+
|
|
158
|
+
In consideration of the right to use this License's text and the "Business
|
|
159
|
+
Source License" name and trademark, Licensor covenants to MariaDB, and to all
|
|
160
|
+
other recipients of the licensed work to be provided by Licensor:
|
|
161
|
+
|
|
162
|
+
1. To specify as the Change License the GPL Version 2.0 or any later version,
|
|
163
|
+
or a license that is compatible with GPL Version 2.0 or a later version,
|
|
164
|
+
where "compatible" means that software provided under the Change License
|
|
165
|
+
can be included in a program with software provided under GPL Version 2.0
|
|
166
|
+
or a later version. Licensor may specify additional Change Licenses without
|
|
167
|
+
limitation.
|
|
168
|
+
|
|
169
|
+
2. To either: (a) specify an additional grant of rights to use that does not
|
|
170
|
+
impose any additional restriction on the right granted in this License, as
|
|
171
|
+
the Additional Use Grant; or (b) insert the text "None".
|
|
172
|
+
|
|
173
|
+
3. To specify a Change Date.
|
|
174
|
+
|
|
175
|
+
4. Not to modify this License in any other way.
|
|
176
|
+
|
|
177
|
+
-----------------------------------------------------------------------------
|
|
178
|
+
|
|
179
|
+
Notice
|
|
180
|
+
|
|
181
|
+
The Business Source License (this document, or the "License") is not an Open
|
|
182
|
+
Source license. However, the Licensed Work will eventually be made available
|
|
183
|
+
under an Open Source License, as stated in this License.
|
package/README.md
CHANGED
|
@@ -1,45 +1,38 @@
|
|
|
1
1
|
# @memberjunction/materialization
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Runtime engine that refreshes **materialized** query and entity-base-view results in MemberJunction. A materialization is a physical, read-only snapshot of a stored Query's result (or an entity's base view), served through a stable wrapper view so reads can opt in with `DataSource: 'Materialized'` and fall back to `Live` on any uncertainty.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
This package owns the **refresh** side of that lifecycle. Provisioning (minting the snapshot table, wrapper view, and metadata) is a CodeGen concern; reading (the `DataSource` redirect) lives in `@memberjunction/generic-database-provider`.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
## What it does
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
- **`MaterializationRefresher`** — rebuilds a snapshot and swaps it into place atomically:
|
|
10
|
+
- **Full rebuild** — builds a fresh *shadow* table from the source (the expensive read happens outside any transaction), then swaps it into the canonical name inside a single transaction (drop → rename → repoint the wrapper view → restore the surrogate index). SQL Server uses `SET XACT_ABORT ON` + `sp_rename` + `EXEC('CREATE OR ALTER VIEW …')`; PostgreSQL keeps the `CREATE OR REPLACE VIEW` repoint *inside* the transaction. Either way a reader sees the whole old snapshot or the whole new one — never a half-swapped view.
|
|
11
|
+
- **Incremental / dirty-group merge** — for keyed/aggregation materializations, applies only the changed groups against a watermark, with a periodic forced full rebuild (`FULL_REBUILD_EVERY_N_INCREMENTAL_REFRESHES`) to reconcile balanced-delete drift.
|
|
12
|
+
- **External-source rebuild** — parameterized, bound-value inserts for materializations sourced from external data providers.
|
|
13
|
+
- Persists `Status`/`RowCount`/`LastRefreshedAt`/`NextRefreshAt` via guarded conditional updates that never clobber a concurrent `DriftHold`/`Disabled`.
|
|
14
|
+
- **`MaterializationFreshness`** — helpers for reasoning about whether a snapshot is current.
|
|
10
15
|
|
|
11
|
-
|
|
12
|
-
1. Configure OIDC trusted publishing for the package name `@memberjunction/materialization`
|
|
13
|
-
2. Enable secure, token-less publishing from CI/CD workflows
|
|
14
|
-
3. Establish provenance for packages published under this name
|
|
16
|
+
All statement builders are **pure and unit-tested**; the identifiers they interpolate are validated as plain SQL identifiers before any DDL is built (a materialization's names come from the writable `MJ: Materialized Results` row, so this fails closed against a tampered row driving arbitrary DDL). Live-DB behavior is covered by the integration tier.
|
|
15
17
|
|
|
16
|
-
##
|
|
18
|
+
## Usage
|
|
17
19
|
|
|
18
|
-
|
|
20
|
+
The engine is normally driven by the scheduled-job refresh driver (`MaterializationRefreshScheduledJobDriver` in `@memberjunction/scheduling-engine`); a manual "refresh now" path can call it directly:
|
|
19
21
|
|
|
20
|
-
|
|
22
|
+
```typescript
|
|
23
|
+
import { MaterializationRefresher } from '@memberjunction/materialization';
|
|
21
24
|
|
|
22
|
-
|
|
25
|
+
const refresher = new MaterializationRefresher();
|
|
26
|
+
const result = await refresher.RefreshOne(materializedResult, contextUser, provider);
|
|
27
|
+
if (!result.Success) {
|
|
28
|
+
// result.ErrorMessage carries the reason; the row is left in a safe state.
|
|
29
|
+
}
|
|
30
|
+
```
|
|
23
31
|
|
|
24
|
-
|
|
25
|
-
2. Configure the trusted publisher (e.g., GitHub Actions)
|
|
26
|
-
3. Specify the repository and workflow that should be allowed to publish
|
|
27
|
-
4. Use the configured workflow to publish your actual package
|
|
32
|
+
`RefreshOne` returns a structured result rather than throwing — errors are logged and reported, and a row held for review (`DriftHold`) or `Disabled` is refused rather than silently reactivated.
|
|
28
33
|
|
|
29
|
-
##
|
|
34
|
+
## Related
|
|
30
35
|
|
|
31
|
-
|
|
32
|
-
-
|
|
33
|
-
-
|
|
34
|
-
- Should not be installed as a dependency
|
|
35
|
-
- Exists only for administrative purposes
|
|
36
|
-
|
|
37
|
-
## More Information
|
|
38
|
-
|
|
39
|
-
For more details about npm's trusted publishing feature, see:
|
|
40
|
-
- [npm Trusted Publishing Documentation](https://docs.npmjs.com/generating-provenance-statements)
|
|
41
|
-
- [GitHub Actions OIDC Documentation](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect)
|
|
42
|
-
|
|
43
|
-
---
|
|
44
|
-
|
|
45
|
-
**Maintained for OIDC setup purposes only**
|
|
36
|
+
- `@memberjunction/generic-database-provider` — the `DataSource: 'Materialized'` read redirect and the materialized read-query builder.
|
|
37
|
+
- `@memberjunction/codegen-lib` — materialization provisioning (mint), parameterized-query classification, and drift detection.
|
|
38
|
+
- `@memberjunction/scheduling-engine` — the scheduled refresh driver.
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
import { IMetadataProvider, UserInfo } from '@memberjunction/core';
|
|
2
|
+
/**
|
|
3
|
+
* Phase 4 (plan §13 — mixed-freshness joins): once live and materialized (snapshot) entities coexist,
|
|
4
|
+
* a caller can read/join across them and get subtly inconsistent results (e.g. a member present live but
|
|
5
|
+
* not yet in last night's snapshot). MJ can't stop a caller from doing this, but per §13 the
|
|
6
|
+
* selection-contract metadata should let an agent NOTICE and FLAG it rather than silently return it.
|
|
7
|
+
*
|
|
8
|
+
* This module surfaces that: given the entities a caller intends to read and whether each is read live or
|
|
9
|
+
* from its materialized snapshot, it reports whether the read mixes freshness (live + snapshot, or a held/
|
|
10
|
+
* stale snapshot) so the caller/agent can decide. It does not block anything — it's an advisory signal.
|
|
11
|
+
*/
|
|
12
|
+
/** One entity's freshness posture within a planned read/join. */
|
|
13
|
+
export interface EntityFreshness {
|
|
14
|
+
entityName: string;
|
|
15
|
+
/** True when this entity is being read from its materialized snapshot (DataSource:'Materialized'). */
|
|
16
|
+
isMaterialized: boolean;
|
|
17
|
+
/** MaterializedResult.Status, when materialized (e.g. 'Active', 'Stale', 'DriftHold'). */
|
|
18
|
+
status?: string;
|
|
19
|
+
/** When the snapshot was last refreshed, when materialized. */
|
|
20
|
+
lastRefreshedAt?: Date | null;
|
|
21
|
+
}
|
|
22
|
+
/** The mixed-freshness verdict for a planned read/join across a set of entities. */
|
|
23
|
+
export interface MixedFreshnessReport {
|
|
24
|
+
/** True when the read mixes a LIVE entity with a MATERIALIZED (snapshot) one — the canonical hazard. */
|
|
25
|
+
mixed: boolean;
|
|
26
|
+
hasLive: boolean;
|
|
27
|
+
hasMaterialized: boolean;
|
|
28
|
+
/** Materialized entities that are NOT cleanly fresh (Status not 'Active') — e.g. Stale / DriftHold. */
|
|
29
|
+
unhealthySnapshots: string[];
|
|
30
|
+
/** Spread (ms) between the newest and oldest snapshot refresh across materialized entities (0 if <2). */
|
|
31
|
+
snapshotFreshnessSpreadMs: number;
|
|
32
|
+
/** Human/agent-readable advisory when there is anything to flag; undefined when the read is uniform. */
|
|
33
|
+
warning?: string;
|
|
34
|
+
entities: EntityFreshness[];
|
|
35
|
+
}
|
|
36
|
+
/**
|
|
37
|
+
* Pure analysis of a planned read's freshness posture. No DB/IO — fully unit-testable.
|
|
38
|
+
* Flags: (a) mixing live + snapshot in one read; (b) any non-Active snapshot (Stale/DriftHold); and
|
|
39
|
+
* reports the refresh-time spread across snapshots so an agent can weigh cross-snapshot skew.
|
|
40
|
+
*/
|
|
41
|
+
export declare function analyzeMixedFreshness(entities: EntityFreshness[]): MixedFreshnessReport;
|
|
42
|
+
/** A planned read of one entity: its name and whether the caller intends to read it live or materialized. */
|
|
43
|
+
export interface PlannedEntityRead {
|
|
44
|
+
entityName: string;
|
|
45
|
+
dataSource?: 'Live' | 'Materialized';
|
|
46
|
+
}
|
|
47
|
+
/**
|
|
48
|
+
* Loads freshness facts for a planned read and analyzes it. For each entity the caller intends to read
|
|
49
|
+
* from its snapshot (`dataSource:'Materialized'`), looks up the base-view MaterializedResult's Status +
|
|
50
|
+
* LastRefreshedAt; entities read live (or with no materialization) are reported as live. Advisory only.
|
|
51
|
+
*/
|
|
52
|
+
export declare class MaterializationFreshness {
|
|
53
|
+
static Inspect(reads: PlannedEntityRead[], contextUser: UserInfo, provider: IMetadataProvider): Promise<MixedFreshnessReport>;
|
|
54
|
+
}
|
|
55
|
+
//# sourceMappingURL=MaterializationFreshness.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"MaterializationFreshness.d.ts","sourceRoot":"","sources":["../src/MaterializationFreshness.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,iBAAiB,EAAW,QAAQ,EAA4B,MAAM,sBAAsB,CAAC;AAWtG;;;;;;;;;GASG;AAEH,iEAAiE;AACjE,MAAM,WAAW,eAAe;IAC5B,UAAU,EAAE,MAAM,CAAC;IACnB,sGAAsG;IACtG,cAAc,EAAE,OAAO,CAAC;IACxB,0FAA0F;IAC1F,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,+DAA+D;IAC/D,eAAe,CAAC,EAAE,IAAI,GAAG,IAAI,CAAC;CACjC;AAED,oFAAoF;AACpF,MAAM,WAAW,oBAAoB;IACjC,wGAAwG;IACxG,KAAK,EAAE,OAAO,CAAC;IACf,OAAO,EAAE,OAAO,CAAC;IACjB,eAAe,EAAE,OAAO,CAAC;IACzB,uGAAuG;IACvG,kBAAkB,EAAE,MAAM,EAAE,CAAC;IAC7B,yGAAyG;IACzG,yBAAyB,EAAE,MAAM,CAAC;IAClC,wGAAwG;IACxG,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,eAAe,EAAE,CAAC;CAC/B;AAED;;;;GAIG;AACH,wBAAgB,qBAAqB,CAAC,QAAQ,EAAE,eAAe,EAAE,GAAG,oBAAoB,CAyCvF;AAED,6GAA6G;AAC7G,MAAM,WAAW,iBAAiB;IAC9B,UAAU,EAAE,MAAM,CAAC;IACnB,UAAU,CAAC,EAAE,MAAM,GAAG,cAAc,CAAC;CACxC;AAED;;;;GAIG;AACH,qBAAa,wBAAwB;WACb,OAAO,CACvB,KAAK,EAAE,iBAAiB,EAAE,EAC1B,WAAW,EAAE,QAAQ,EACrB,QAAQ,EAAE,iBAAiB,GAC5B,OAAO,CAAC,oBAAoB,CAAC;CA2EnC"}
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
import { RunView, IsMaterializedDataSource } from '@memberjunction/core';
|
|
2
|
+
/**
|
|
3
|
+
* Pure analysis of a planned read's freshness posture. No DB/IO — fully unit-testable.
|
|
4
|
+
* Flags: (a) mixing live + snapshot in one read; (b) any non-Active snapshot (Stale/DriftHold); and
|
|
5
|
+
* reports the refresh-time spread across snapshots so an agent can weigh cross-snapshot skew.
|
|
6
|
+
*/
|
|
7
|
+
export function analyzeMixedFreshness(entities) {
|
|
8
|
+
const materialized = entities.filter((e) => e.isMaterialized);
|
|
9
|
+
const hasLive = entities.some((e) => !e.isMaterialized);
|
|
10
|
+
const hasMaterialized = materialized.length > 0;
|
|
11
|
+
const mixed = hasLive && hasMaterialized;
|
|
12
|
+
const unhealthySnapshots = materialized
|
|
13
|
+
.filter((e) => e.status && e.status !== 'Active')
|
|
14
|
+
.map((e) => `${e.entityName} (${e.status})`);
|
|
15
|
+
const refreshTimes = materialized
|
|
16
|
+
.map((e) => (e.lastRefreshedAt ? new Date(e.lastRefreshedAt).getTime() : null))
|
|
17
|
+
// Exclude NaN (an unparseable lastRefreshedAt) as well as null: NaN passes `!= null`, and a single
|
|
18
|
+
// NaN poisons Math.max/Math.min (both return NaN), silently suppressing the real cross-snapshot spread
|
|
19
|
+
// warning. Dropping the bad value lets the spread be computed from the parseable timestamps.
|
|
20
|
+
.filter((t) => t != null && !Number.isNaN(t));
|
|
21
|
+
const snapshotFreshnessSpreadMs = refreshTimes.length >= 2 ? Math.max(...refreshTimes) - Math.min(...refreshTimes) : 0;
|
|
22
|
+
const notes = [];
|
|
23
|
+
if (mixed) {
|
|
24
|
+
const liveNames = entities.filter((e) => !e.isMaterialized).map((e) => e.entityName);
|
|
25
|
+
const matNames = materialized.map((e) => e.entityName);
|
|
26
|
+
notes.push(`reads mix live [${liveNames.join(', ')}] with materialized snapshot [${matNames.join(', ')}] — results may be inconsistent (a row present live may be absent from the snapshot, or vice-versa)`);
|
|
27
|
+
}
|
|
28
|
+
if (unhealthySnapshots.length) {
|
|
29
|
+
notes.push(`snapshot(s) not fresh: [${unhealthySnapshots.join(', ')}]`);
|
|
30
|
+
}
|
|
31
|
+
if (materialized.length >= 2 && snapshotFreshnessSpreadMs > 0) {
|
|
32
|
+
notes.push(`snapshots were refreshed at different times (spread ${Math.round(snapshotFreshnessSpreadMs / 1000)}s)`);
|
|
33
|
+
}
|
|
34
|
+
return {
|
|
35
|
+
mixed,
|
|
36
|
+
hasLive,
|
|
37
|
+
hasMaterialized,
|
|
38
|
+
unhealthySnapshots,
|
|
39
|
+
snapshotFreshnessSpreadMs,
|
|
40
|
+
warning: notes.length ? notes.join('; ') : undefined,
|
|
41
|
+
entities,
|
|
42
|
+
};
|
|
43
|
+
}
|
|
44
|
+
/**
|
|
45
|
+
* Loads freshness facts for a planned read and analyzes it. For each entity the caller intends to read
|
|
46
|
+
* from its snapshot (`dataSource:'Materialized'`), looks up the base-view MaterializedResult's Status +
|
|
47
|
+
* LastRefreshedAt; entities read live (or with no materialization) are reported as live. Advisory only.
|
|
48
|
+
*/
|
|
49
|
+
export class MaterializationFreshness {
|
|
50
|
+
static async Inspect(reads, contextUser, provider) {
|
|
51
|
+
// Resolve every planned read to its entity. The two materialization types have DIFFERENT freshness
|
|
52
|
+
// semantics:
|
|
53
|
+
// • Base-view materialization reuses the SOURCE entity and exposes BOTH a live and a snapshot read,
|
|
54
|
+
// so it is a snapshot read only when the caller asked for one (dataSource:'Materialized'). Matched
|
|
55
|
+
// by SourceEntityID.
|
|
56
|
+
// • Query materialization mints a NEW virtual entity whose base view IS the snapshot (there is no
|
|
57
|
+
// live alternative), so a read of it is ALWAYS a snapshot read regardless of the dataSource flag.
|
|
58
|
+
// Matched by GeneratedEntityID.
|
|
59
|
+
// We therefore look up materializations for ALL reads (not only dataSource:'Materialized' ones).
|
|
60
|
+
const resolved = reads.map((r) => ({ read: r, entity: provider.EntityByName(r.entityName) }));
|
|
61
|
+
const wantedIds = resolved.map((x) => x.entity?.ID).filter((id) => !!id);
|
|
62
|
+
// Advisory/hot path reading only scalar fields (never mutated/saved) → ResultType:'simple' + explicit
|
|
63
|
+
// Fields, not full MJMaterializedResultEntity objects (the RunView anti-pattern the root CLAUDE.md
|
|
64
|
+
// flags). Scoped by entity ID so we don't pull the whole materialization set. (uniqueidentifier/uuid
|
|
65
|
+
// comparison is type-normalized on both engines, so raw-cased ID literals match regardless of casing.)
|
|
66
|
+
const baseViewByEntityId = new Map();
|
|
67
|
+
const queryByEntityId = new Map();
|
|
68
|
+
if (wantedIds.length > 0) {
|
|
69
|
+
const inList = wantedIds.map((id) => `'${id.replace(/'/g, "''")}'`).join(', ');
|
|
70
|
+
// Use the PASSED-IN provider (not the global default new RunView()) — entity resolution above uses
|
|
71
|
+
// `provider`, so in a multi-provider client the materialization rows must come from that SAME
|
|
72
|
+
// server/DB, or the entity IDs won't match and every read is misreported as not-materialized.
|
|
73
|
+
const rv = RunView.FromMetadataProvider(provider);
|
|
74
|
+
const res = await rv.RunView({
|
|
75
|
+
EntityName: 'MJ: Materialized Results',
|
|
76
|
+
ExtraFilter: `(SourceType='EntityBaseView' AND SourceEntityID IN (${inList})) OR (SourceType='Query' AND GeneratedEntityID IN (${inList}))`,
|
|
77
|
+
Fields: ['SourceType', 'SourceEntityID', 'GeneratedEntityID', 'Status', 'LastRefreshedAt'],
|
|
78
|
+
ResultType: 'simple',
|
|
79
|
+
}, contextUser);
|
|
80
|
+
if (res.Success) {
|
|
81
|
+
// Key each row by the ID that identifies the ENTITY a caller reads (lowercased for UUID-casing
|
|
82
|
+
// robustness): base-view → SourceEntityID, query → GeneratedEntityID.
|
|
83
|
+
for (const mr of res.Results) {
|
|
84
|
+
if (mr.SourceType === 'Query') {
|
|
85
|
+
if (mr.GeneratedEntityID)
|
|
86
|
+
queryByEntityId.set(mr.GeneratedEntityID.toLowerCase(), mr);
|
|
87
|
+
}
|
|
88
|
+
else if (mr.SourceEntityID) {
|
|
89
|
+
baseViewByEntityId.set(mr.SourceEntityID.toLowerCase(), mr);
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
const toFreshness = (name, mr) => ({
|
|
95
|
+
entityName: name,
|
|
96
|
+
isMaterialized: true,
|
|
97
|
+
status: mr.Status ?? undefined,
|
|
98
|
+
// ResultType:'simple' can deliver a datetimeoffset as a raw string; normalize to Date | null.
|
|
99
|
+
lastRefreshedAt: mr.LastRefreshedAt ? new Date(mr.LastRefreshedAt) : null,
|
|
100
|
+
});
|
|
101
|
+
const entities = resolved.map(({ read, entity }) => {
|
|
102
|
+
if (!entity)
|
|
103
|
+
return { entityName: read.entityName, isMaterialized: false };
|
|
104
|
+
const id = entity.ID.toLowerCase();
|
|
105
|
+
// Query materialization → always a snapshot read (no live form), flag regardless of dataSource.
|
|
106
|
+
const queryMat = queryByEntityId.get(id);
|
|
107
|
+
if (queryMat)
|
|
108
|
+
return toFreshness(read.entityName, queryMat);
|
|
109
|
+
// Base-view materialization → a snapshot read ONLY when the caller opted in via dataSource.
|
|
110
|
+
// NOTE: reporting isMaterialized:false here is a freshness *report* only; it does NOT mean a
|
|
111
|
+
// DataSource:'Materialized' RunView would fall back to live — GetEffectiveBaseView resolves
|
|
112
|
+
// 'Materialized' to materialized_vw<CodeName> with no existence check, so such a read on an entity
|
|
113
|
+
// with no materialization targets a missing view and errors at the DB.
|
|
114
|
+
if (IsMaterializedDataSource(read.dataSource)) {
|
|
115
|
+
const baseMat = baseViewByEntityId.get(id);
|
|
116
|
+
if (baseMat)
|
|
117
|
+
return toFreshness(read.entityName, baseMat);
|
|
118
|
+
}
|
|
119
|
+
return { entityName: read.entityName, isMaterialized: false };
|
|
120
|
+
});
|
|
121
|
+
return analyzeMixedFreshness(entities);
|
|
122
|
+
}
|
|
123
|
+
}
|
|
124
|
+
//# sourceMappingURL=MaterializationFreshness.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"MaterializationFreshness.js","sourceRoot":"","sources":["../src/MaterializationFreshness.ts"],"names":[],"mappings":"AAAA,OAAO,EAAqB,OAAO,EAAY,wBAAwB,EAAE,MAAM,sBAAsB,CAAC;AAgDtG;;;;GAIG;AACH,MAAM,UAAU,qBAAqB,CAAC,QAA2B;IAC7D,MAAM,YAAY,GAAG,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,cAAc,CAAC,CAAC;IAC9D,MAAM,OAAO,GAAG,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,cAAc,CAAC,CAAC;IACxD,MAAM,eAAe,GAAG,YAAY,CAAC,MAAM,GAAG,CAAC,CAAC;IAChD,MAAM,KAAK,GAAG,OAAO,IAAI,eAAe,CAAC;IAEzC,MAAM,kBAAkB,GAAG,YAAY;SAClC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,MAAM,IAAI,CAAC,CAAC,MAAM,KAAK,QAAQ,CAAC;SAChD,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,GAAG,CAAC,CAAC,UAAU,KAAK,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC;IAEjD,MAAM,YAAY,GAAG,YAAY;SAC5B,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;QAC/E,mGAAmG;QACnG,uGAAuG;QACvG,6FAA6F;SAC5F,MAAM,CAAC,CAAC,CAAC,EAAe,EAAE,CAAC,CAAC,IAAI,IAAI,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAC/D,MAAM,yBAAyB,GAC3B,YAAY,CAAC,MAAM,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,YAAY,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,YAAY,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAEzF,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,IAAI,KAAK,EAAE,CAAC;QACR,MAAM,SAAS,GAAG,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,cAAc,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC;QACrF,MAAM,QAAQ,GAAG,YAAY,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC;QACvD,KAAK,CAAC,IAAI,CAAC,mBAAmB,SAAS,CAAC,IAAI,CAAC,IAAI,CAAC,iCAAiC,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,qGAAqG,CAAC,CAAC;IACjN,CAAC;IACD,IAAI,kBAAkB,CAAC,MAAM,EAAE,CAAC;QAC5B,KAAK,CAAC,IAAI,CAAC,2BAA2B,kBAAkB,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IAC5E,CAAC;IACD,IAAI,YAAY,CAAC,MAAM,IAAI,CAAC,IAAI,yBAAyB,GAAG,CAAC,EAAE,CAAC;QAC5D,KAAK,CAAC,IAAI,CAAC,uDAAuD,IAAI,CAAC,KAAK,CAAC,yBAAyB,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC;IACxH,CAAC;IAED,OAAO;QACH,KAAK;QACL,OAAO;QACP,eAAe;QACf,kBAAkB;QAClB,yBAAyB;QACzB,OAAO,EAAE,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,SAAS;QACpD,QAAQ;KACX,CAAC;AACN,CAAC;AAQD;;;;GAIG;AACH,MAAM,OAAO,wBAAwB;IAC1B,MAAM,CAAC,KAAK,CAAC,OAAO,CACvB,KAA0B,EAC1B,WAAqB,EACrB,QAA2B;QAE3B,mGAAmG;QACnG,aAAa;QACb,sGAAsG;QACtG,uGAAuG;QACvG,yBAAyB;QACzB,oGAAoG;QACpG,sGAAsG;QACtG,oCAAoC;QACpC,iGAAiG;QACjG,MAAM,QAAQ,GAAG,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,MAAM,EAAE,QAAQ,CAAC,YAAY,CAAC,CAAC,CAAC,UAAU,CAAC,EAAE,CAAC,CAAC,CAAC;QAC9F,MAAM,SAAS,GAAG,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,EAAE,EAAgB,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;QAEvF,sGAAsG;QACtG,mGAAmG;QACnG,qGAAqG;QACrG,uGAAuG;QACvG,MAAM,kBAAkB,GAAG,IAAI,GAAG,EAAoC,CAAC;QACvE,MAAM,eAAe,GAAG,IAAI,GAAG,EAAoC,CAAC;QACpE,IAAI,SAAS,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YACvB,MAAM,MAAM,GAAG,SAAS,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,IAAI,EAAE,CAAC,OAAO,CAAC,IAAI,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YAC/E,mGAAmG;YACnG,8FAA8F;YAC9F,8FAA8F;YAC9F,MAAM,EAAE,GAAG,OAAO,CAAC,oBAAoB,CAAC,QAAQ,CAAC,CAAC;YAClD,MAAM,GAAG,GAAG,MAAM,EAAE,CAAC,OAAO,CACxB;gBACI,UAAU,EAAE,0BAA0B;gBACtC,WAAW,EAAE,uDAAuD,MAAM,uDAAuD,MAAM,IAAI;gBAC3I,MAAM,EAAE,CAAC,YAAY,EAAE,gBAAgB,EAAE,mBAAmB,EAAE,QAAQ,EAAE,iBAAiB,CAAC;gBAC1F,UAAU,EAAE,QAAQ;aACvB,EACD,WAAW,CACd,CAAC;YACF,IAAI,GAAG,CAAC,OAAO,EAAE,CAAC;gBACd,+FAA+F;gBAC/F,sEAAsE;gBACtE,KAAK,MAAM,EAAE,IAAI,GAAG,CAAC,OAAO,EAAE,CAAC;oBAC3B,IAAI,EAAE,CAAC,UAAU,KAAK,OAAO,EAAE,CAAC;wBAC5B,IAAI,EAAE,CAAC,iBAAiB;4BAAE,eAAe,CAAC,GAAG,CAAC,EAAE,CAAC,iBAAiB,CAAC,WAAW,EAAE,EAAE,EAAE,CAAC,CAAC;oBAC1F,CAAC;yBAAM,IAAI,EAAE,CAAC,cAAc,EAAE,CAAC;wBAC3B,kBAAkB,CAAC,GAAG,CAAC,EAAE,CAAC,cAAc,CAAC,WAAW,EAAE,EAAE,EAAE,CAAC,CAAC;oBAChE,CAAC;gBACL,CAAC;YACL,CAAC;QACL,CAAC;QAED,MAAM,WAAW,GAAG,CAAC,IAAY,EAAE,EAA4B,EAAmB,EAAE,CAAC,CAAC;YAClF,UAAU,EAAE,IAAI;YAChB,cAAc,EAAE,IAAI;YACpB,MAAM,EAAE,EAAE,CAAC,MAAM,IAAI,SAAS;YAC9B,8FAA8F;YAC9F,eAAe,EAAE,EAAE,CAAC,eAAe,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,EAAE,CAAC,eAAe,CAAC,CAAC,CAAC,CAAC,IAAI;SAC5E,CAAC,CAAC;QAEH,MAAM,QAAQ,GAAsB,QAAQ,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE;YAClE,IAAI,CAAC,MAAM;gBAAE,OAAO,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,EAAE,cAAc,EAAE,KAAK,EAAE,CAAC;YAC3E,MAAM,EAAE,GAAG,MAAM,CAAC,EAAE,CAAC,WAAW,EAAE,CAAC;YACnC,gGAAgG;YAChG,MAAM,QAAQ,GAAG,eAAe,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;YACzC,IAAI,QAAQ;gBAAE,OAAO,WAAW,CAAC,IAAI,CAAC,UAAU,EAAE,QAAQ,CAAC,CAAC;YAC5D,4FAA4F;YAC5F,6FAA6F;YAC7F,4FAA4F;YAC5F,mGAAmG;YACnG,uEAAuE;YACvE,IAAI,wBAAwB,CAAC,IAAI,CAAC,UAAU,CAAC,EAAE,CAAC;gBAC5C,MAAM,OAAO,GAAG,kBAAkB,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;gBAC3C,IAAI,OAAO;oBAAE,OAAO,WAAW,CAAC,IAAI,CAAC,UAAU,EAAE,OAAO,CAAC,CAAC;YAC9D,CAAC;YACD,OAAO,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,EAAE,cAAc,EAAE,KAAK,EAAE,CAAC;QAClE,CAAC,CAAC,CAAC;QAEH,OAAO,qBAAqB,CAAC,QAAQ,CAAC,CAAC;IAC3C,CAAC;CACJ"}
|