@x12i/memorix-format 2.2.1 → 2.2.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,9 @@
1
1
  # Changelog
2
2
 
3
+ ## 2.2.3
4
+
5
+ - Docs hygiene: drop identity-era appendix and ebook correction notes; association doc is a thin present-tense stub pointing at `meta/associated-conventions.json` + pipeline ops docs.
6
+
3
7
  ## 2.2.0
4
8
 
5
9
  - Depends on `@x12i/memorix-base-format@^1.2.0` (`concept._extIdentifier`, `_system.state`).
package/README.md CHANGED
@@ -1,38 +1,25 @@
1
1
  # @x12i/memorix-format
2
2
 
3
- Memorix **product** record format — profiles and conventions on top of
3
+ Memorix **product** record format — open-bucket conventions and migrate tooling layered on
4
4
  [`@x12i/memorix-base-format`](https://www.npmjs.com/package/@x12i/memorix-base-format).
5
5
 
6
6
  | Package | Owns |
7
7
  |---|---|
8
8
  | `@x12i/memorix-base-format` | Fixed skeleton: `recordId`, `concept`, `data`, `_…`; unnamed open buckets; ContentTypeMeta |
9
- | `@x12i/memorix-format` | Memorix open-bucket meanings, `associated*` rules, snapshot SCOPE, built-in profiles |
9
+ | `@x12i/memorix-format` | Known open-bucket meanings, `associated*` rules, snapshot SCOPE, built-in profiles, **migrate** CLI |
10
10
 
11
- Product meta-metadata is packaged under [`meta/`](./meta/) (open buckets, content types, association conventions, profiles). Managed-descriptor inventory lives in `meta/inventory/` and is not format law.
11
+ **Object types belong in Catalox descriptors, not here.** Format docs and examples use abstract names (`items`, `groups`, `locations`, `documents`) so domain packs are never confused with format law.
12
12
 
13
- ## Migrate existing databases
13
+ Product meta lives under [`meta/`](./meta/). Managed-descriptor inventory in `meta/inventory/` is not format law.
14
14
 
15
- Present-tense format only. To transform stored JSON / exports:
15
+ Normative: [docs/memorix-format-requirements.md](./docs/memorix-format-requirements.md). Narrative: [docs/memorix-format.md](./docs/memorix-format.md).
16
16
 
17
- ```bash
18
- # Dry-run validate after transform (stdout summary)
19
- npx memorix-format migrate ./export-dir/ --content-type=snapshots
20
-
21
- # Write cleaned files
22
- npx memorix-format migrate ./export-dir/ --out=./clean-dir/
23
-
24
- # Or overwrite in place after validation passes
25
- npx memorix-format migrate ./records.json --in-place
26
- ```
27
-
28
- Defaults applied by migrate:
17
+ ## Layering
29
18
 
30
- - `concept.identifier` → `concept._extIdentifier` (`identifierKey` → `key`)
31
- - root `identity` / `lifecycle` / `sourceFacts` / `index` removed (lifecycle events folded into `_system.lifecycle`)
32
- - root `_graphRuns` / `_jobTypeRuns` → under `_system`
33
- - `_system.state` set to `active` when missing (maps old `lifecycle.status` when present)
34
-
35
- Writers set `_system.state: "active"` on new records. Pipeline format enforce defaults to **hard** (`MEMORIX_FORMAT_ENFORCE=warn` to soften during cutover).
19
+ 1. Base envelope (identity / core / underscore / open).
20
+ 2. Product open buckets when present: `analytics`, `enrichment`, `insights`, `associated*`.
21
+ 3. **SCOPE:** full open-bucket surface primarily on **snapshots**; other content types keep substance in `.data` unless ContentTypeMeta opts in.
22
+ 4. Readers / human projection **omit** root `_…` and `concept._extIdentifier` by default; keep `recordId` for joins.
36
23
 
37
24
  ## Install
38
25
 
@@ -41,6 +28,27 @@ npm install @x12i/memorix-format
41
28
  # pulls @x12i/memorix-base-format
42
29
  ```
43
30
 
31
+ ## Abstract snapshot example
32
+
33
+ ```json
34
+ {
35
+ "recordId": "rec-item-1",
36
+ "concept": {
37
+ "_extIdentifier": {
38
+ "kind": "sku",
39
+ "sku": "SKU-100",
40
+ "key": "sku:SKU-100"
41
+ },
42
+ "title": "Blue widget"
43
+ },
44
+ "data": { "sku": "SKU-100", "color": "blue" },
45
+ "associatedData": [{ "locationCode": "WH-A", "qty": 12 }],
46
+ "analytics": { "linkedLocations": { "total": 1 } },
47
+ "insights": { "priority": { "level": "high", "confidence": 0.9 } },
48
+ "_system": { "state": "active" }
49
+ }
50
+ ```
51
+
44
52
  ## SDK
45
53
 
46
54
  ```ts
@@ -51,14 +59,12 @@ import {
51
59
  snapshotsContentTypeMeta,
52
60
  } from "@x12i/memorix-format";
53
61
 
54
- // Product validation (base + ASSOC/SCOPE + optional profile)
55
62
  validateMemorixRecord(record, {
56
63
  profile: "snapshots",
57
- objectType: "vulnerabilities",
64
+ objectType: "items",
58
65
  contentType: "snapshots",
59
66
  });
60
67
 
61
- // Base only (no product vocabulary)
62
68
  validateMemorixBaseRecord(record);
63
69
  classifyRootKey("associatedData"); // "open"
64
70
  ```
@@ -66,13 +72,44 @@ classifyRootKey("associatedData"); // "open"
66
72
  ## CLI
67
73
 
68
74
  ```bash
69
- npx memorix-format validate ./record.json --profile=snapshots --object-type=assets
75
+ npx memorix-format validate ./record.json --profile=snapshots --object-type=items
70
76
  npx memorix-format validate ./dir/ --content-type=events
71
77
  npx memorix-format profiles
72
78
  npx memorix-format docs memorix-format-requirements.md
73
79
  ```
74
80
 
75
- ## Migration from 1.0.0
81
+ ## Migrate (recommended transform)
82
+
83
+ Present-tense format only. Transform exports / sample JSON; **live Mongo apply is a separate ops checklist**.
84
+
85
+ ```bash
86
+ # Dry-run (stdout summary; validates after transform)
87
+ npx memorix-format migrate ./export-dir/ --content-type=snapshots
88
+
89
+ # Write cleaned files
90
+ npx memorix-format migrate ./export-dir/ --out=./clean-dir/
91
+
92
+ # Overwrite in place after validation passes
93
+ npx memorix-format migrate ./records.json --in-place
94
+ ```
95
+
96
+ Defaults applied by migrate:
97
+
98
+ - `concept.identifier` → `concept._extIdentifier` (`identifierKey` → `key`)
99
+ - root `identity` / `lifecycle` / `sourceFacts` / `index` removed (lifecycle events folded into `_system.lifecycle`)
100
+ - root `_graphRuns` / `_jobTypeRuns` → under `_system`
101
+ - `_system.state` set to `active` when missing (maps old `lifecycle.status` when present)
102
+
103
+ Writers set `_system.state: "active"` on new records. Pipeline format enforce defaults to **hard** (`MEMORIX_FORMAT_ENFORCE=warn` to soften during cutover).
104
+
105
+ ### Runbook
106
+
107
+ 1. Dry-run against an export or package examples.
108
+ 2. Write `--out` and re-validate.
109
+ 3. Only then apply to Mongo under ops change control (not part of this package’s default workflow).
110
+
111
+ ## Related
76
112
 
77
- See [CHANGELOG.md](./CHANGELOG.md). Envelope validation moved to base-format;
78
- this package no longer pretends product bucket names are part of the base law.
113
+ - Base envelope SOT: [`@x12i/memorix-base-format`](https://www.npmjs.com/package/@x12i/memorix-base-format)
114
+ - Associations: [docs/association-format.md](./docs/association-format.md)
115
+ - Changelog: [CHANGELOG.md](./CHANGELOG.md)
@@ -1,466 +1,44 @@
1
1
  # Association format
2
2
 
3
- An **association** connects one snapshot record to one or more related records. Associated content is stored as root-level properties on snapshot records using the reserved `associated` prefix.
3
+ Associated content is stored as **root-level open buckets** on snapshot records using the reserved `associated` prefix.
4
4
 
5
- Associations are materialized by `@x12i/memorix-associator` presets, orchestrated by `@x12i/memorix-pipeline` enrichment. See [Running associations](./running-associations.md).
5
+ **Normative naming / forms:** packaged [`meta/associated-conventions.json`](../meta/associated-conventions.json).
6
+ **Operational detail** (presets, apply, enrichment orchestration): [`memorix-pipeline/docs/association-format.md`](../../memorix-pipeline/docs/association-format.md) and [running-associations.md](../../memorix-pipeline/docs/running-associations.md).
6
7
 
7
- ---
8
-
9
- ## Core rule
10
-
11
- Record-to-record association is always resolved from **snapshot record to snapshot record first**:
12
-
13
- 1. The lookup starts from the current object's snapshot record.
14
- 2. The matching target is another snapshot record in a `*-snapshots` collection.
15
- 3. Use the matched target snapshot's `recordId` to fetch non-snapshot content (`discovery`, `analysis`, `decisions`, etc.).
16
- 4. Copy flat `content.data` into `associated<ObjectType><ContentType>` on the source snapshot.
17
-
18
- The snapshot resolves the link; embedded content is always fetched **after** snapshot match by `recordId` (or equivalent link field such as `concept.identifier.recordId` or `data.sourceRecordId`).
19
-
20
- **Never join a source snapshot directly to a non-snapshot content collection by business keys** (for example `vulnerabilities-snapshots.data.assetIp` → `vulnerabilities-discovery.data.assetIp`). Same-object associations still match snapshots first, then fetch content by matched snapshot `recordId`.
8
+ This package validates association **shape** (ASSOC rules in [memorix-format-requirements.md](./memorix-format-requirements.md)); it does not invent object-type names.
21
9
 
22
10
  ---
23
11
 
24
- ## Naming conventions
25
-
26
- ### Legacy (live data today)
27
-
28
- Short names without object type segment: `associatedData`, `associatedDiscovery`, `associatedAnalysis`, `associatedAssets`, `associatedSubnets`, etc.
29
-
30
- - Remain on existing records; not bulk-migrated.
31
- - Registry entries carry `"namingConvention": "legacy"`.
32
- - Legacy items may include inline `recordId` on copied payloads.
33
-
34
- ### v2: `associated<ObjectType><ContentType>`
35
-
36
- | Segment | Rule | Example |
37
- |---------|------|---------|
38
- | Prefix | `associated` (reserved) | — |
39
- | ObjectType | PascalCase canonical object type | `security-groups` → `SecurityGroups` |
40
- | ContentType | PascalCase canonical content type | `discovery` → `Discovery` |
41
-
42
- Examples:
43
-
44
- ```ts
45
- associatedSecurityGroupsAnalysis
46
- associatedVulnerabilitiesGroupsDiscovery
47
- associatedVulnerabilitiesGroupsSnapshots
48
- associatedUsersDecision
49
- ```
50
-
51
- Do **not** pluralize the content type (`Discovery`, not `Discoveries`). The property is already an array.
52
-
53
- Registry entries for new associations carry `"namingConvention": "v2"`.
54
-
55
- ### Ranked top associations: `associatedTop<ObjectType><ContentType>`
56
-
57
- Same copy forms as above, but candidates are **sorted by importance** then **capped**.
58
-
59
- | Segment | Rule | Example |
60
- |---------|------|---------|
61
- | Prefix | `associatedTop` | — |
62
- | ObjectType | PascalCase | `Vulnerabilities` |
63
- | ContentType | PascalCase | `Analysis` |
64
-
65
- Example: `associatedTopVulnerabilitiesAnalysis`
66
-
67
- **Importance** comes from the linked object type’s descriptor `importanceIndicator` (e.g. vulnerabilities → `data.severity`), or from the association rule’s `rankBy` override.
68
-
69
- **Default cap:** `5` (`DEFAULT_TOP_ASSOCIATED_ITEMS`). Override per rule with `maxItems`.
70
-
71
- **Empty results:** omit the property entirely (never write `[]`).
72
-
73
- ```json
74
- {
75
- "associatedProperty": "associatedTopVulnerabilitiesAnalysis",
76
- "maxItems": 5,
77
- "rankBy": {
78
- "path": "data.severity",
79
- "valueKind": "severity",
80
- "order": "desc"
81
- },
82
- "match": {
83
- "collection": "vulnerabilities-snapshots",
84
- "joinPaths": ["data.assetIp"]
85
- },
86
- "content": {
87
- "collection": "vulnerabilities-analysis",
88
- "linkPaths": ["data.sourceRecordId", "recordId"]
89
- },
90
- "payload": { "source": "target.data", "format": "v2" }
91
- }
92
- ```
93
-
94
- For snapshot-mediated rules, `rankBy.path` is evaluated on the **matched snapshot** (not the copied content). Severity order: `critical > high > medium > low > none`.
95
-
96
- ### Derived (not copy-form associations)
97
-
98
- Resolver outputs such as `associatedTopology` are **derived** fields computed during pipeline resolution — not associator copy forms. Document separately; do not rename to v2 copy naming.
99
-
100
- ---
101
-
102
- ## Association forms
103
-
104
- The associator supports four copy forms via `rule.payload.source` (see request schema in `@x12i/memorix-associator`).
105
-
106
- ### 1. Normal data association
107
-
108
- **Property:** `associated<ObjectType><ContentType>`
109
-
110
- **Copy:** linked record's `data` object (flat, not wrapped).
111
-
112
- ```json
113
- {
114
- "associatedSecurityGroupsAnalysis": [
115
- {
116
- "_system": {
117
- "association": {
118
- "fromProperty": "data.securityGroups",
119
- "toProperty": "identifier",
120
- "associatedAt": "2026-07-08T09:00:00.000Z",
121
- "recordId": "security-group-record-id-1"
122
- }
123
- },
124
- "summary": "The group appears to provide privileged access.",
125
- "impact": "High",
126
- "classification": "Privileged Security Group"
127
- }
128
- ]
129
- }
130
- ```
131
-
132
- Request payload:
133
-
134
- ```json
135
- {
136
- "payload": {
137
- "format": "v2",
138
- "source": "target.data",
139
- "system": { "include": true, "fromProperty": "data.securityGroups", "toProperty": "identifier" }
140
- }
141
- }
142
- ```
143
-
144
- Rules:
145
-
146
- - Business data stays flat beside `_system`.
147
- - Do **not** wrap copied data inside another `data` object.
148
- - `_system` is optional. Omit when consumed as business or LLM context.
149
-
150
- ### 2. Snapshot root property association
151
-
152
- **Property:** `associated<ObjectType>Snapshots<RootProperty>`
153
-
154
- Copies a root property from the associated **snapshot** record (not a separate content collection).
155
-
156
- Example: `associatedVulnerabilitiesGroupsSnapshotsAnalytics` — copies `analytics` from `vulnerabilities-groups-snapshots`.
157
-
158
- ```json
159
- {
160
- "payload": {
161
- "format": "v2",
162
- "source": { "snapshotRootProperty": "analytics" },
163
- "system": { "include": true }
164
- }
165
- }
166
- ```
167
-
168
- When `system.include` is true, the item is wrapped as `{ _system, value }`. Prefer the dedicated enrichment form below when copying the `enrichment` bucket (flat copy, no `value` wrapper).
169
-
170
- ### Analytics rollups (sibling — not association)
171
-
172
- **Do not** use association rules to *compute* counts into the source record’s own `analytics` bucket. Associations **copy** linked payloads into `associated*`. Numeric rollups are a separate JSON rule type:
173
-
174
- - Spec: `.operational/analytics-rollups/FORMAT.md` (`memorix.analyticsRollup.v1`)
175
- - Presets: `.operational/analytics-rollups/presets/*.json`
176
- - Runner: `scripts/memorix-2.0-migrate/patches/run-analytics-rollups.mjs`
177
-
178
- Use the **same join keys** as association presets (`assetIp`, `name`↔`vulnerabilityName`, CIDR). Writing `analytics` on the target is a rollup; copying another record’s `analytics` into an `associated*SnapshotsAnalytics` array is still an association (form 2 above).
179
-
180
- ### 2b. Enrichment association
181
-
182
- **Property:** `associated<ObjectType>Enrichment`
183
-
184
- Copies the linked **snapshot** root `enrichment` object **flat** (same shape as content associations: fields beside optional `_system`, not wrapped in `value`).
185
-
186
- `enrichment` is an envelope bucket on snapshots (CVSS, EPSS, KEV, compositeScore, …) — **not** a separate `*-enrichment` content-type collection.
187
-
188
- Example: `associatedVulnerabilitiesGroupsEnrichment` on `vulnerabilities-snapshots`.
189
-
190
- ```json
191
- {
192
- "associatedVulnerabilitiesGroupsEnrichment": [
193
- {
194
- "cvss": { "baseScore": 9.8 },
195
- "epss": { "epss": 0.94 },
196
- "kev": { "isKnownExploited": true },
197
- "compositeScore": 0.97,
198
- "riskLevel": "CRITICAL"
199
- }
200
- ]
201
- }
202
- ```
203
-
204
- ```json
205
- {
206
- "associatedProperty": "associatedVulnerabilitiesGroupsEnrichment",
207
- "payload": {
208
- "format": "v2",
209
- "source": "target.enrichment",
210
- "includeRecordId": false
211
- },
212
- "target": {
213
- "collection": "vulnerabilities-groups-snapshots",
214
- "joinPaths": ["data.name"]
215
- }
216
- }
217
- ```
218
-
219
- Rules:
220
-
221
- - Omit the property when no linked snapshot has a non-empty `enrichment` object.
222
- - Do not invent a `vulnerabilities-groups-enrichment` collection for this form.
223
- - Naming helper: `buildAssociatedPropertyName({ form: "enrichment", objectType: "vulnerabilities-groups" })`.
224
-
225
- ### 3. Concepts association
226
-
227
- **Property:** `associated<ObjectType>Concepts`
228
-
229
- Copies `concept` from associated snapshot records (not full `data`).
230
-
231
- ```json
232
- {
233
- "associatedSecurityGroupsConcepts": [
234
- {
235
- "identifier": { "kind": "group-name", "groupName": "Domain Admins" },
236
- "title": "Domain Admins"
237
- }
238
- ]
239
- }
240
- ```
241
-
242
- ```json
243
- {
244
- "payload": {
245
- "format": "v2",
246
- "source": "target.concept",
247
- "system": { "include": true }
248
- }
249
- }
250
- ```
251
-
252
- ### 4. Record IDs association
253
-
254
- **Property:** `associated<ObjectType>RecordIds` (use `RecordIds`, not `RecordsIds`)
255
-
256
- **Value:** array of strings. Must **not** include `_system`.
257
-
258
- ```json
259
- {
260
- "associatedSecurityGroupsRecordIds": [
261
- "security-group-record-id-1",
262
- "security-group-record-id-2"
263
- ]
264
- }
265
- ```
266
-
267
- ```json
268
- {
269
- "payload": {
270
- "format": "v2",
271
- "source": "target.recordId"
272
- }
273
- }
274
- ```
275
-
276
- ---
277
-
278
- ## `_system.association` metadata
279
-
280
- Reserved for technical traceability — not business data.
281
-
282
- | Field | Meaning |
283
- |-------|---------|
284
- | `fromProperty` | Path on the source snapshot that supplied the join value |
285
- | `toProperty` | Path on the linked snapshot used for matching |
286
- | `associatedAt` | ISO date-time when the association was created or refreshed |
287
- | `recordId` | Linked snapshot `recordId` (v2, when enabled) |
288
-
289
- Included when `payload.format === "v2"` and `payload.system.include === true`. All fields are optional even when `_system` is present.
290
-
291
- Strip `_system` when presenting associated content to humans, LLMs, or downstream analysis.
292
-
293
- ---
294
-
295
- ## Identifier and identity matching
296
-
297
- Prefer the linked snapshot's `identity` or `concept.identifier`:
298
-
299
- ```json
300
- { "identity": "group-name:Domain Admins" }
301
- ```
302
-
303
- ```json
304
- {
305
- "identifier": { "kind": "group-name", "groupName": "Domain Admins" }
306
- }
307
- ```
308
-
309
- Canonical location is `concept.identifier`. Root `identifier` / `identity` are accepted when present.
310
-
311
- Generate identity strings deterministically: join identifier values in order with `:` (e.g. `assetIp:10.100.18.64`).
12
+ ## Core rule
312
13
 
313
- Association rules declare join paths in `sourceJoinPaths` and snapshot match paths in `match.joinPaths` (or legacy `target.joinPaths` for snapshot-only rules). Content fetch rules add `content.linkPaths` that reference the matched snapshot `recordId`. v2 rules should target `concept.identifier` or generated `identity` where possible.
14
+ Association is always **snapshot-first**:
314
15
 
315
- Associator request shape for content associations:
16
+ 1. Start from the source object’s snapshot.
17
+ 2. Match a target **snapshot** (`*-snapshots`).
18
+ 3. Use the matched snapshot’s `recordId` to fetch non-snapshot content (`discovery`, `analysis`, `decisions`, …).
19
+ 4. Copy flat content into an `associated…` array on the source snapshot.
316
20
 
317
- ```json
318
- {
319
- "sourceJoinPaths": ["data.assetIp"],
320
- "match": {
321
- "collection": "vulnerabilities-snapshots",
322
- "joinPaths": ["data.assetIp", "concept.identifier.assetIp", "recordId"]
323
- },
324
- "content": {
325
- "collection": "vulnerabilities-discovery",
326
- "contentType": "discovery",
327
- "linkPaths": ["concept.identifier.recordId", "data.sourceRecordId", "recordId"]
328
- }
329
- }
330
- ```
21
+ Never join a source snapshot directly to a non-snapshot collection by business keys.
331
22
 
332
23
  ---
333
24
 
334
- ## Worked example: security group analysis into users
335
-
336
- Collections:
337
-
338
- ```text
339
- users-snapshots
340
- security-groups-snapshots
341
- security-groups-analysis
342
- ```
343
-
344
- User snapshot source:
345
-
346
- ```json
347
- {
348
- "data": { "securityGroups": ["Domain Admins", "VPN Users"] }
349
- }
350
- ```
351
-
352
- Security group snapshot identifier:
25
+ ## Naming
353
26
 
354
- ```json
355
- {
356
- "recordId": "security-group-record-id-1",
357
- "concept": {
358
- "identifier": { "kind": "group-name", "groupName": "Domain Admins" },
359
- "title": "Domain Admins"
360
- }
361
- }
362
- ```
27
+ | Form | Pattern | Example (abstract) |
28
+ |---|---|---|
29
+ | Legacy short names | `associatedData`, `associatedDiscovery`, … | still present on older records |
30
+ | v2 content | `associated<ObjectType><ContentType>` | `associatedItemGroupsAnalysis` |
31
+ | Ranked top | `associatedTop<ObjectType><ContentType>` | `associatedTopItemsAnalysis` |
32
+ | Enrichment copy | `associated<ObjectType>Enrichment` | `associatedItemGroupsEnrichment` |
363
33
 
364
- Process:
365
-
366
- 1. Start from `users-snapshots` record.
367
- 2. Read join values from `data.securityGroups`.
368
- 3. Match against `security-groups-snapshots` via `concept.identifier` / identity.
369
- 4. Use matched snapshot `recordId` to fetch `security-groups-analysis` records.
370
- 5. Copy each analysis record's `data` flat into `associatedSecurityGroupsAnalysis`.
371
-
372
- Result:
373
-
374
- ```json
375
- {
376
- "recordId": "user-record-id-1",
377
- "data": {
378
- "username": "ami",
379
- "securityGroups": ["Domain Admins"]
380
- },
381
- "associatedSecurityGroupsAnalysis": [
382
- {
383
- "_system": {
384
- "association": {
385
- "fromProperty": "data.securityGroups",
386
- "toProperty": "concept.identifier",
387
- "associatedAt": "2026-07-08T09:00:00.000Z"
388
- }
389
- },
390
- "summary": "The group appears to provide privileged access.",
391
- "impact": "High",
392
- "classification": "Privileged Security Group"
393
- }
394
- ]
395
- }
396
- ```
34
+ Object-type segments come from Catalox descriptors — not from this format package.
397
35
 
398
36
  ---
399
37
 
400
- ## Implementation status
401
-
402
- | Capability | Status |
403
- |------------|--------|
404
- | Legacy `associatedData` / `associatedDiscovery` / … presets | **Live** — registry `namingConvention: "legacy"` |
405
- | v2 property naming | **Target** — new registry entries use `associated<ObjectType><ContentType>` |
406
- | v2 payload forms (`target.data`, `target.concept`, `target.recordId`, snapshot root) | **Supported** in `@x12i/memorix-associator` request schema |
407
- | v2 `_system.association` metadata | **Supported** when `format: "v2"` and `system.include: true` |
408
- | Registry-driven preset generation | **Live** — `generate-presets-from-registry.mjs` |
409
- | Full enrichment orchestration | **Live** — `run-enrichment-pipeline.mjs` |
410
-
411
- New associations should be added as registry entries + presets using v2 naming and payload format. Existing legacy properties continue to refresh through their current presets without rename.
412
-
413
- ### Pilot: vulnerability discovery and decisions (v2)
414
-
415
- Snapshot-mediated flow:
416
-
417
- 1. Read source `vulnerabilities-snapshots.data.assetIp`.
418
- 2. Match target `vulnerabilities-snapshots` by snapshot paths (`data.assetIp`, `concept.identifier.assetIp`, `identity`, `recordId`).
419
- 3. Take matched target snapshot `recordId`.
420
- 4. Fetch `vulnerabilities-discovery` / `vulnerabilities-decisions` by record-id link (`concept.identifier.recordId`, `data.sourceRecordId`, or root `recordId`).
421
- 5. Copy flat `content.data` into `associatedVulnerabilitiesDiscovery` / `associatedVulnerabilitiesDecisions`.
422
-
423
- | Property | Match collection | Content collection | Join key |
424
- |----------|------------------|--------------------|----------|
425
- | `associatedVulnerabilitiesDiscovery` | `vulnerabilities-snapshots` | `vulnerabilities-discovery` | `data.assetIp` → snapshot → `recordId` |
426
- | `associatedVulnerabilitiesDecisions` | `vulnerabilities-snapshots` | `vulnerabilities-decisions` | `data.assetIp` → snapshot → `recordId` |
427
-
428
- Target source collection: `vulnerabilities-snapshots` (`MEMORIX_EVENTS_DB`).
429
-
430
- Refresh rule keys:
431
-
432
- - `vulnerabilities.vulnerabilities.discovery.by-assetIp`
433
- - `vulnerabilities.vulnerabilities.decisions.by-assetIp`
434
-
435
- ### Pilot: asset-side vulnerability analysis (v2)
436
-
437
- Snapshot-mediated flow (source is **assets**, not vulnerabilities):
438
-
439
- 1. Read source `assets-snapshots.data.assetIps` (also `data.assetIp`, `data.ip_address`).
440
- 2. Match `vulnerabilities-snapshots` by `data.assetIp` / `concept.identifier.assetIp`.
441
- 3. Take matched snapshot `recordId`.
442
- 4. Fetch `vulnerabilities-analysis` by record-id link.
443
- 5. Copy flat `content.data` into `associatedVulnerabilitiesAnalysis` on the asset snapshot.
444
-
445
- | Property | Match collection | Content collection | Join key |
446
- |----------|------------------|--------------------|----------|
447
- | `associatedVulnerabilitiesAnalysis` | `vulnerabilities-snapshots` | `vulnerabilities-analysis` | `data.assetIps` → snapshot `data.assetIp` → `recordId` |
448
-
449
- Target source collection: `assets-snapshots` (`MEMORIX_ENTITIES_DB`).
450
-
451
- Refresh rule key: `assets.vulnerabilities.analysis.by-assetIp`
452
-
453
- ---
38
+ ## Payload rules
454
39
 
455
- ## Legacy properties reference
40
+ - Associated properties are **arrays of plain objects** (linked payloads), never full envelopes.
41
+ - **Omit** the property when empty — never write `[]`.
42
+ - Prefer traceability in optional item-level `_system.association` (v2) rather than wrapping envelopes.
456
43
 
457
- | Property | Typical link |
458
- |----------|--------------|
459
- | `associatedData` | Linked snapshots / generic data |
460
- | `associatedDiscovery` | Linked discovery |
461
- | `associatedAnalysis` | Linked analysis |
462
- | `associatedAssets` | Asset snapshots by IP |
463
- | `associatedSubnets` | Subnet snapshots (explicit) |
464
- | `associatedTopology` | Derived (resolver, not associator copy) |
465
- | `associatedVulnerabilityGroup` | Vuln-group snapshots (explicit) |
466
- | `associatedZoneAnalysis` | Topology-zone analysis |
44
+ See requirements **ASSOC-*** and the pipeline association doc for matching / payload forms.