@x12i/memorix-format 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +90 -0
- package/dist/associated.d.ts +38 -0
- package/dist/associated.d.ts.map +1 -0
- package/dist/associated.js +59 -0
- package/dist/associated.js.map +1 -0
- package/dist/cli.d.ts +3 -0
- package/dist/cli.d.ts.map +1 -0
- package/dist/cli.js +199 -0
- package/dist/cli.js.map +1 -0
- package/dist/concept.d.ts +16 -0
- package/dist/concept.d.ts.map +1 -0
- package/dist/concept.js +75 -0
- package/dist/concept.js.map +1 -0
- package/dist/constants.d.ts +21 -0
- package/dist/constants.d.ts.map +1 -0
- package/dist/constants.js +49 -0
- package/dist/constants.js.map +1 -0
- package/dist/index.d.ts +9 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +7 -0
- package/dist/index.js.map +1 -0
- package/dist/inspect.d.ts +4 -0
- package/dist/inspect.d.ts.map +1 -0
- package/dist/inspect.js +51 -0
- package/dist/inspect.js.map +1 -0
- package/dist/types.d.ts +81 -0
- package/dist/types.d.ts.map +1 -0
- package/dist/types.js +2 -0
- package/dist/types.js.map +1 -0
- package/dist/utils.d.ts +6 -0
- package/dist/utils.d.ts.map +1 -0
- package/dist/utils.js +23 -0
- package/dist/utils.js.map +1 -0
- package/dist/validate.d.ts +14 -0
- package/dist/validate.d.ts.map +1 -0
- package/dist/validate.js +211 -0
- package/dist/validate.js.map +1 -0
- package/docs/association-format.md +466 -0
- package/docs/memorix-format-additions.md +956 -0
- package/docs/memorix-format-for-dummies-association-corrections.md +113 -0
- package/docs/memorix-format.md +282 -0
- package/docs/memorix-workflow-format.md +114 -0
- package/examples/invalid-record.json +11 -0
- package/examples/valid-snapshot.json +38 -0
- package/package.json +67 -0
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
# Corrections To Association Guidance In Memorix Format For Dummies
|
|
2
|
+
|
|
3
|
+
This document records required fixes to association guidance in `ebooks/memorix-format-for-dummies.pdf`. The PDF's **Chapter 5** (vulnerability discovery/decisions case file) teaches a **direct content join** model that is incorrect.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Core correction
|
|
8
|
+
|
|
9
|
+
Association is always **snapshot-first**, then **content by matched snapshot `recordId`**:
|
|
10
|
+
|
|
11
|
+
```text
|
|
12
|
+
source snapshot (join value from data / identifier / identity)
|
|
13
|
+
→ match target *-snapshots collection
|
|
14
|
+
→ take matched snapshot recordId
|
|
15
|
+
→ fetch discovery / analysis / decisions by record-id link field
|
|
16
|
+
→ copy flat content.data into associated<ObjectType><ContentType>
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
**Never** join a source snapshot directly to a non-snapshot collection by business keys such as `data.assetIp`.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Chapter-by-chapter fixes
|
|
24
|
+
|
|
25
|
+
### Cover / Chapter 0 — Vulnerability patch framing
|
|
26
|
+
|
|
27
|
+
**Incorrect:** The vulnerability patch wires `vulnerabilities-snapshots` directly to `vulnerabilities-discovery` / `vulnerabilities-decisions` by `data.assetIp`.
|
|
28
|
+
|
|
29
|
+
**Correct:** The patch enriches `vulnerabilities-snapshots` using snapshot-mediated resolution: match snapshots first (including same-object `vulnerabilities-snapshots`), then fetch discovery/decisions content linked to the matched snapshot `recordId`.
|
|
30
|
+
|
|
31
|
+
### Chapter 4 — Core association rules
|
|
32
|
+
|
|
33
|
+
**Add before naming and payload rules:**
|
|
34
|
+
|
|
35
|
+
> Record-to-record association always resolves through snapshots. Non-snapshot collections (`discovery`, `analysis`, `decisions`, …) are fetched only after a snapshot match yields a target `recordId`. Associated content is never the source of truth for the link.
|
|
36
|
+
|
|
37
|
+
### Chapter 5 — Vulnerability discovery/decisions case file (full rewrite)
|
|
38
|
+
|
|
39
|
+
Replace the direct-join walkthrough with:
|
|
40
|
+
|
|
41
|
+
1. Read source `vulnerabilities-snapshots.data.assetIp`.
|
|
42
|
+
2. Match target `vulnerabilities-snapshots` by snapshot paths such as `data.assetIp`, `concept.identifier`, or `identity`.
|
|
43
|
+
3. Take matched target snapshot `recordId`.
|
|
44
|
+
4. Fetch `vulnerabilities-discovery` / `vulnerabilities-decisions` by a record-id link field such as `data.sourceRecordId`, `concept.identifier.recordId`, or root `recordId`.
|
|
45
|
+
5. Copy flat `content.data` into `associatedVulnerabilitiesDiscovery` / `associatedVulnerabilitiesDecisions`.
|
|
46
|
+
|
|
47
|
+
Payload rules for this pilot:
|
|
48
|
+
|
|
49
|
+
- Flat copied `data` only
|
|
50
|
+
- No nested `data` wrapper
|
|
51
|
+
- No inline `recordId` on items
|
|
52
|
+
- No `_system.association` metadata
|
|
53
|
+
|
|
54
|
+
### Chapter 9 — Checklist and glossary
|
|
55
|
+
|
|
56
|
+
**Checklist additions:**
|
|
57
|
+
|
|
58
|
+
- [ ] Snapshot match collection is `*-snapshots`
|
|
59
|
+
- [ ] Content fetch uses `recordId` link paths, not business-key joins on content
|
|
60
|
+
- [ ] `_system.association.toProperty` refers to the snapshot match path
|
|
61
|
+
|
|
62
|
+
**Glossary correction:**
|
|
63
|
+
|
|
64
|
+
| Term | Incorrect PDF usage | Correct definition |
|
|
65
|
+
|------|---------------------|-------------------|
|
|
66
|
+
| Association key | Business field on content collection | Join path on source snapshot used to match target snapshots |
|
|
67
|
+
| Linked collection | Content collection joined directly | Snapshot match collection + optional content collection fetched by `recordId` |
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## Replacement example (vulnerability discovery/decisions)
|
|
72
|
+
|
|
73
|
+
```markdown
|
|
74
|
+
1. Read source `vulnerabilities-snapshots.data.assetIp`.
|
|
75
|
+
2. Match target `vulnerabilities-snapshots` by snapshot paths such as `data.assetIp`, `concept.identifier`, or `identity`.
|
|
76
|
+
3. Take matched target snapshot `recordId`.
|
|
77
|
+
4. Fetch `vulnerabilities-discovery` / `vulnerabilities-decisions` by a record-id link field such as `data.sourceRecordId`, `concept.identifier.recordId`, or root `recordId`.
|
|
78
|
+
5. Copy flat `content.data` into `associatedVulnerabilitiesDiscovery` / `associatedVulnerabilitiesDecisions`.
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
---
|
|
82
|
+
|
|
83
|
+
## Incorrect vs correct
|
|
84
|
+
|
|
85
|
+
| Aspect | Incorrect (PDF Chapter 5) | Correct |
|
|
86
|
+
|--------|---------------------------|---------|
|
|
87
|
+
| Join path | `vulnerabilities-snapshots.data.assetIp` → `vulnerabilities-discovery.data.assetIp` | `vulnerabilities-snapshots.data.assetIp` → `vulnerabilities-snapshots` (snapshot match) → `recordId` → `vulnerabilities-discovery` (by `concept.identifier.recordId`) |
|
|
88
|
+
| Phase count | One-phase direct content join | Two-phase: snapshot match, then content fetch |
|
|
89
|
+
| Same-object rule | Exception allowing direct snapshot→content | Same-object still matches snapshots first |
|
|
90
|
+
| Bridge pattern | Implicit direct join | Explicit `match` + `content` in associator rules |
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## Associator rule shape (reference)
|
|
95
|
+
|
|
96
|
+
```json
|
|
97
|
+
{
|
|
98
|
+
"sourceJoinPaths": ["data.assetIp"],
|
|
99
|
+
"match": {
|
|
100
|
+
"collection": "vulnerabilities-snapshots",
|
|
101
|
+
"joinPaths": ["data.assetIp", "concept.identifier.assetIp", "identity", "recordId"]
|
|
102
|
+
},
|
|
103
|
+
"content": {
|
|
104
|
+
"collection": "vulnerabilities-discovery",
|
|
105
|
+
"contentType": "discovery",
|
|
106
|
+
"linkPaths": ["concept.identifier.recordId", "data.sourceRecordId", "recordId"]
|
|
107
|
+
},
|
|
108
|
+
"associatedProperty": "associatedVulnerabilitiesDiscovery",
|
|
109
|
+
"payload": { "format": "v2", "source": "target.data", "includeRecordId": false }
|
|
110
|
+
}
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
See also: [memorix-pipeline/docs/association-format.md](../memorix-pipeline/docs/association-format.md)
|
|
@@ -0,0 +1,282 @@
|
|
|
1
|
+
# Memorix Record Format
|
|
2
|
+
|
|
3
|
+
> Packaged as **`@x12i/memorix-format`** (types, validator, CLI).
|
|
4
|
+
|
|
5
|
+
## Purpose
|
|
6
|
+
|
|
7
|
+
A Memorix record is a structured memory object: a stable envelope around domain payload, with clear separation between business data, derived analytics, external enrichment, AI insights, and cross-entity references.
|
|
8
|
+
|
|
9
|
+
The clean format removes the legacy 2.0 `identity` block, root-level `lifecycle`, `sourceFacts`, and redundant `concept.name`. Every field belongs in exactly one place.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Record envelope
|
|
14
|
+
|
|
15
|
+
```ts
|
|
16
|
+
type MemorixRecord = {
|
|
17
|
+
_id?: string; // MongoDB document id (storage only)
|
|
18
|
+
recordId: string; // stable logical record id
|
|
19
|
+
concept: MemorixConcept;
|
|
20
|
+
data: JsonObject; // required — see data bucket rules
|
|
21
|
+
|
|
22
|
+
analytics?: JsonObject;
|
|
23
|
+
enrichment?: JsonObject;
|
|
24
|
+
insights?: JsonObject;
|
|
25
|
+
associatedData?: JsonObject[];
|
|
26
|
+
associatedDiscovery?: JsonObject[];
|
|
27
|
+
associatedAnalysis?: JsonObject[];
|
|
28
|
+
associatedSiblings?: JsonObject[];
|
|
29
|
+
associatedChildren?: JsonObject[];
|
|
30
|
+
// associated<CustomName>?: JsonObject[];
|
|
31
|
+
|
|
32
|
+
_system?: MemorixSystemBlock;
|
|
33
|
+
};
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Associated properties use the reserved `associated` prefix. **Legacy** names (`associatedData`, `associatedDiscovery`, …) remain on existing data. **v2** names follow `associated<ObjectType><ContentType>` — see [memorix-pipeline/docs/association-format.md](../memorix-pipeline/docs/association-format.md).
|
|
37
|
+
|
|
38
|
+
The root holds only `_id`, `recordId`, `concept`, the data buckets, associated properties, and `_system`. No other top-level keys.
|
|
39
|
+
|
|
40
|
+
### Removed from 2.0
|
|
41
|
+
|
|
42
|
+
| Legacy field | Fate |
|
|
43
|
+
|---|---|
|
|
44
|
+
| `identity` (`target`, `targetId`, `objectType`, `contentType`, `version`) | Dropped; `recordId` is root-level |
|
|
45
|
+
| Root `lifecycle` (`createdAt`, `modifiedAt`, `capturedAt`, `status`) | Replaced by `_system.lifecycle` event array |
|
|
46
|
+
| `sourceFacts` | Promoted into `data` or `associatedData` |
|
|
47
|
+
| `concept.name` | Dropped; use `concept.identifier` + `concept.title` |
|
|
48
|
+
| Root `_graphRuns` / `_jobTypeRuns` | Moved to `_system` |
|
|
49
|
+
|
|
50
|
+
`objectType` is implied by collection name. `contentType` and `version` were constant across all records and are not carried forward.
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## Data And Associated Buckets
|
|
55
|
+
|
|
56
|
+
Every piece of business data lives in **exactly one** of these buckets (plus the envelope fields above):
|
|
57
|
+
|
|
58
|
+
| Bucket | Contents |
|
|
59
|
+
|---|---|
|
|
60
|
+
| **`data`** | Raw facts about the entity itself — IP, severity, session timeout, zone name |
|
|
61
|
+
| **`analytics`** | Rollups and counts computed from other records — vulnerability totals, affected-asset counts |
|
|
62
|
+
| **`enrichment`** | External, sourced values joined from feeds or prior systems — CVSS, EPSS, KEV, VPR |
|
|
63
|
+
| **`insights`** | AI-generated judgments — exploitability, potential impact, classification, risk narratives |
|
|
64
|
+
| **`associatedData`** | Linked snapshot/data content copied from associated records' `data` payloads |
|
|
65
|
+
| **`associatedDiscovery`** | Linked discovery content copied from associated records' `data` payloads |
|
|
66
|
+
| **`associatedAnalysis`** | Linked analysis content copied from associated records' `data` payloads |
|
|
67
|
+
| **`associatedSiblings`** | Self-association: other snapshot `data` payloads sharing the same join key (e.g. assets by shared `data.subnetCidr` in `data.subnetCidrs[]`); each item includes linked `recordId`. Default cap: 99 items. |
|
|
68
|
+
| **`associatedChildren`** | Parent-to-child association (subnets only today): asset `data` payloads whose `data.subnetCidrs` contains the subnet `data.cidr`; each item includes linked `recordId`. Default cap: 99 items. |
|
|
69
|
+
|
|
70
|
+
Do not duplicate a value across buckets. When reshaping legacy records, promote `sourceFacts` values into the correct bucket rather than wrapping them.
|
|
71
|
+
|
|
72
|
+
Associated properties are always arrays and are omitted when no linked data was resolved. Do not write empty arrays, and do not switch between object and array shape. Custom associated content uses the same root-level pattern: `associatedCompliance`, `associatedBusinessContext`, `associatedExposure`, and so on.
|
|
73
|
+
|
|
74
|
+
**Ranked top associations** use `associatedTop<ObjectType><ContentType>` (e.g. `associatedTopVulnerabilitiesAnalysis`). Candidates are sorted by the linked object type’s `importanceIndicator` (or rule `rankBy`) and capped (default 5). See [association-format.md](../memorix-pipeline/docs/association-format.md).
|
|
75
|
+
|
|
76
|
+
**Enrichment associations** use `associated<ObjectType>Enrichment` (e.g. `associatedVulnerabilitiesGroupsEnrichment`). They copy the linked snapshot’s root `enrichment` bucket (external feed values: CVSS, EPSS, KEV, compositeScore) as a flat object array — not a separate `*-enrichment` content collection. See [association-format.md](../memorix-pipeline/docs/association-format.md).
|
|
77
|
+
|
|
78
|
+
Associated items contain the linked record's `data` payload (or `enrichment` / `concept` / root property for the specialized forms). Legacy associations may also include inline `recordId` and `contentType`; v2 associations store traceability in optional item-level `_system.association` metadata. Do not wrap associated items with redundant envelope keys; failure and diagnostic details belong in `<objectType>-failed` collections instead.
|
|
79
|
+
|
|
80
|
+
**Associated property metadata:** Consumers should prefer Memorix associated-property metadata when available (`@x12i/memorix-associator`). Associated snapshot properties are exposed through managed descriptors plus discovery. Until metadata is available, or when reading older/custom records, fall back to the prefix rule: root-level fields on snapshots whose names start with `associated`. The well-known names `associatedData`, `associatedDiscovery`, and `associatedAnalysis` are conventions only — the complete set comes from metadata plus the `associated*` fallback rule.
|
|
81
|
+
|
|
82
|
+
When a field can naturally have more than one value, processing contracts use array shape even if the current record has only one value. For assets, `data.assetIps`, `data.subnets[]`, and `data.subnetCidrs[]` are the normalized join fields. Each `data.subnets[]` entry is a structured subnet object `{ cidr, networkAddress, prefixLength, mask }` — one resolved subnet per asset IP, chosen by longest-prefix match. Multiple entries exist only when multiple asset IPs resolve to different canonical subnets. `data.subnetCidrs[]` is the derived flat join index used by association rules.
|
|
83
|
+
|
|
84
|
+
For subnets, `data.cidr` is the canonical join key. Subnet records also carry structured `data.subnet` plus flattened `data.networkAddress`, `data.prefixLength`, and `data.mask`, all derived deterministically from `data.cidr`.
|
|
85
|
+
|
|
86
|
+
Assets also carry `data.assetIpClassifications[]` when topology resolution runs. Each item records `{ ip, scope, confidence, reason, reasoning, matchedSubnetCidr?, matchedSubnetIp? }`, where `scope` is `inside`, `outside`, or `unknown`. Convenience arrays `data.insideAssetIps`, `data.outsideAssetIps`, and `data.unknownAssetIps` may be populated from those classifications. `inside` means the IP either matched canonical topology with high confidence, or is RFC1918 private space without a matching topology CIDR with medium confidence. `outside` means public/non-RFC1918 with no topology match. `unknown` covers invalid, loopback, and link-local addresses.
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## Concept
|
|
91
|
+
|
|
92
|
+
```ts
|
|
93
|
+
type MemorixConcept = {
|
|
94
|
+
identifier: {
|
|
95
|
+
kind: string;
|
|
96
|
+
/** Primary value — property name matches `kind`. */
|
|
97
|
+
[kind: string]: string;
|
|
98
|
+
/**
|
|
99
|
+
* Deterministic join key: `kind` first, then the kind-named value,
|
|
100
|
+
* then any other scalar fields in sorted key order, joined by `:`.
|
|
101
|
+
* See `.operational/fr/2026-07-09-memorix-format-identifier-key.md`.
|
|
102
|
+
*/
|
|
103
|
+
identifierKey: string;
|
|
104
|
+
};
|
|
105
|
+
title: string;
|
|
106
|
+
};
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
`identifier` is a discriminated object: `kind` names the id type, and a same-named property holds the value. `identifierKey` is required on every identifier (root and nested concepts).
|
|
110
|
+
|
|
111
|
+
```json
|
|
112
|
+
"concept": {
|
|
113
|
+
"identifier": {
|
|
114
|
+
"kind": "ipAddress",
|
|
115
|
+
"ipAddress": "10.150.68.31",
|
|
116
|
+
"identifierKey": "ipAddress:10.150.68.31"
|
|
117
|
+
},
|
|
118
|
+
"title": "BGAK105-002"
|
|
119
|
+
}
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
`concept.name` is not used. `title` is the human-readable display name; it may differ from the identifier value (e.g. hostname vs IP).
|
|
123
|
+
|
|
124
|
+
### Identifier `kind` by object type
|
|
125
|
+
|
|
126
|
+
| Object type | `kind` | Example identifier |
|
|
127
|
+
|---|---|---|
|
|
128
|
+
| Assets | `ipAddress` | `{"kind":"ipAddress","ipAddress":"10.150.68.31","identifierKey":"ipAddress:10.150.68.31"}` |
|
|
129
|
+
| Vulnerability groups | `pluginId` (preferred) or `cveId` | `{"kind":"pluginId","pluginId":"142594","identifierKey":"pluginId:142594"}` |
|
|
130
|
+
| Vulnerability instances (host findings) | `hostVulnerability` | `{"kind":"hostVulnerability","hostVulnerability":"10.100.18.64::pluginId:160208","assetIp":"10.100.18.64","vulnerabilityGroupIdentifierKey":"pluginId:160208","identifierKey":"hostVulnerability:10.100.18.64::pluginId:160208:10.100.18.64:pluginId:160208"}` |
|
|
131
|
+
| Subnets | `cidr` (canonical) or `subnetIp` (legacy, quarantined) | `{"kind":"cidr","cidr":"1.1.1.5/32","identifierKey":"cidr:1.1.1.5/32"}` |
|
|
132
|
+
| Topology zones | `zoneName` | `{"kind":"zoneName","zoneName":"LAN","identifierKey":"zoneName:LAN"}` |
|
|
133
|
+
| Topology VRs | `vrName` | `{"kind":"vrName","vrName":"VR-UPB-MED","identifierKey":"vrName:VR-UPB-MED"}` |
|
|
134
|
+
| Users | `sessionKey` | `{"kind":"sessionKey","sessionKey":"med\\cap\|10.39.44.205\|UIA","identifierKey":"sessionKey:med\\cap\|10.39.44.205\|UIA"}` |
|
|
135
|
+
|
|
136
|
+
Vulnerability groups are **plugin-scoped**: prefer `pluginId` whenever present. Use `cveId` only when there is no plugin id — the same CVE can appear on multiple plugins, so CVE is not a unique group key. Keep `data.cveId` for display/filter. Vulnerability **instances** also denormalize `vulnerabilityGroupIdentifierKey` at the record root for queries.
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
## `_system`
|
|
141
|
+
|
|
142
|
+
System metadata — provenance, processing history, graph-run stamps — never belongs in `data` or the other business buckets.
|
|
143
|
+
|
|
144
|
+
```ts
|
|
145
|
+
type MemorixLifecycleEvent = { process: string; processedAt: string };
|
|
146
|
+
|
|
147
|
+
type MemorixSystemBlock = {
|
|
148
|
+
lifecycle?: MemorixLifecycleEvent[];
|
|
149
|
+
_graphRuns?: Record<string, GraphRunEntry>;
|
|
150
|
+
_jobTypeRuns?: Record<string, GraphRunEntry>;
|
|
151
|
+
provenance?: JsonObject;
|
|
152
|
+
// other system keys as needed (memorix, scoping, …)
|
|
153
|
+
};
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
### Lifecycle
|
|
157
|
+
|
|
158
|
+
Root-level `lifecycle` is gone. Processing events live at `_system.lifecycle`:
|
|
159
|
+
|
|
160
|
+
```json
|
|
161
|
+
"lifecycle": [
|
|
162
|
+
{ "process": "enriched", "processedAt": "2026-04-05T08:01:15.511Z" }
|
|
163
|
+
]
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
- Each entry is `{ process, processedAt }`.
|
|
167
|
+
- When `createdAt`, `modifiedAt`, and `capturedAt` are identical (the common snapshot case), emit a single `"enriched"` event rather than three duplicate timestamps.
|
|
168
|
+
- Distinct events (e.g. a future re-enrichment) append as additional array entries.
|
|
169
|
+
- `status` is not stored; it was uniformly `"active"` across all records.
|
|
170
|
+
- Omit `lifecycle` entirely when no timestamp is known (e.g. Users records with no source dates) — do not emit an empty array.
|
|
171
|
+
|
|
172
|
+
`_system.provenance.enrichedAt` (when present) is separate provenance metadata about a prior enrichment step, not a substitute for `lifecycle`.
|
|
173
|
+
|
|
174
|
+
### Graph and job-type runs
|
|
175
|
+
|
|
176
|
+
`_graphRuns` and `_jobTypeRuns` are stored under `_system`, keyed by graph id and sanitized job-type id respectively. Each entry tracks execution status, timestamps, and result metadata. These fields are written by Memorix writer during graph execution and are not part of the domain payload.
|
|
177
|
+
|
|
178
|
+
Dot-path references: `_system._graphRuns.<graphId>`, `_system._jobTypeRuns.<jobTypeId>`.
|
|
179
|
+
|
|
180
|
+
---
|
|
181
|
+
|
|
182
|
+
## Complete example
|
|
183
|
+
|
|
184
|
+
Vulnerability group record showing all envelope fields and every bucket:
|
|
185
|
+
|
|
186
|
+
```json
|
|
187
|
+
{
|
|
188
|
+
"_id": "6914a132880706529ece3da1",
|
|
189
|
+
"recordId": "69d216ed358778e93215787e",
|
|
190
|
+
"concept": {
|
|
191
|
+
"identifier": {
|
|
192
|
+
"kind": "pluginId",
|
|
193
|
+
"pluginId": "142594",
|
|
194
|
+
"identifierKey": "pluginId:142594"
|
|
195
|
+
},
|
|
196
|
+
"title": "Oracle WebLogic Server RCE (CVE-2020-14882)"
|
|
197
|
+
},
|
|
198
|
+
"data": {
|
|
199
|
+
"name": "Oracle WebLogic Server RCE (CVE-2020-14882)",
|
|
200
|
+
"vulnerability_family": "Web Servers",
|
|
201
|
+
"vulnerability_type": "cve",
|
|
202
|
+
"cveId": "CVE-2020-14882",
|
|
203
|
+
"severity": "critical",
|
|
204
|
+
"pluginId": 142594
|
|
205
|
+
},
|
|
206
|
+
"associatedData": [
|
|
207
|
+
{ "assetIp": "10.100.24.31", "hostname": "BGAK105-002" },
|
|
208
|
+
{ "assetIp": "10.100.24.41", "hostname": "BGAK105-003" }
|
|
209
|
+
],
|
|
210
|
+
"analytics": {
|
|
211
|
+
"affectedAssets": { "total": 2 }
|
|
212
|
+
},
|
|
213
|
+
"enrichment": {
|
|
214
|
+
"cvss": {
|
|
215
|
+
"baseScore": 9.8,
|
|
216
|
+
"cvssVector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H"
|
|
217
|
+
},
|
|
218
|
+
"epss": { "epss": 0.94454, "percentile": 0.99992 },
|
|
219
|
+
"kev": { "isKnownExploited": true },
|
|
220
|
+
"compositeScore": 0.974,
|
|
221
|
+
"riskLevel": "CRITICAL",
|
|
222
|
+
"vprScore": "9.2"
|
|
223
|
+
},
|
|
224
|
+
"insights": {
|
|
225
|
+
"exploitability": { "level": "HIGH", "confidence": 0.92 },
|
|
226
|
+
"potentialImpact": { "level": "CRITICAL", "confidence": 0.88 }
|
|
227
|
+
},
|
|
228
|
+
"_system": {
|
|
229
|
+
"provenance": {
|
|
230
|
+
"sourceId": "6914a132880706529ece3da1",
|
|
231
|
+
"snapshotRecordId": "69d216ed358778e93215787e",
|
|
232
|
+
"source": {
|
|
233
|
+
"database": "poc-data-mapping",
|
|
234
|
+
"collection": "vulnerabilities-groups-rawdata",
|
|
235
|
+
"server": "default"
|
|
236
|
+
}
|
|
237
|
+
},
|
|
238
|
+
"lifecycle": [
|
|
239
|
+
{ "process": "enriched", "processedAt": "2026-04-05T08:01:15.511Z" }
|
|
240
|
+
],
|
|
241
|
+
"_graphRuns": {
|
|
242
|
+
"graph-t92669z": {
|
|
243
|
+
"status": "done",
|
|
244
|
+
"startedAt": "2026-07-01T23:41:23.233Z",
|
|
245
|
+
"completedAt": "2026-07-01T23:42:05.604Z",
|
|
246
|
+
"lastJobRunId": "f7af7f7a-0c5c-4825-9282-d39bea836682",
|
|
247
|
+
"attempts": 1,
|
|
248
|
+
"result": {
|
|
249
|
+
"target": "entity",
|
|
250
|
+
"entityName": "vulnerabilities-groups",
|
|
251
|
+
"contentType": "inferences",
|
|
252
|
+
"recordId": "75de9244-8d44-43b8-8698-7e22621b1087"
|
|
253
|
+
}
|
|
254
|
+
}
|
|
255
|
+
}
|
|
256
|
+
}
|
|
257
|
+
}
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
Not every record populates every bucket. Assets typically have `data`, `analytics`, and optionally `insights` and `associatedData`; only vulnerability groups carry `enrichment` today.
|
|
261
|
+
|
|
262
|
+
---
|
|
263
|
+
|
|
264
|
+
## Migration notes
|
|
265
|
+
|
|
266
|
+
- Reshape scripts (`scripts/memorix-neo-reshape/transform-20-to-clean.mjs`) convert 2.0 envelopes to this format.
|
|
267
|
+
- Fixed POC reference data lives in `temp/data-and-format-fixes/` with per-collection change notes in `CHANGES.md`.
|
|
268
|
+
- Type definitions: `memorix-descriptors/src/record/envelope.ts`.
|
|
269
|
+
|
|
270
|
+
### Subnets / topology subnets merge (2026-07-05)
|
|
271
|
+
|
|
272
|
+
`topology-subnets` was merged into `subnets`. Canonical `subnets` records are topology
|
|
273
|
+
CIDR segments (`concept.identifier.kind: "cidr"`), spread across content types
|
|
274
|
+
`snapshots` / `core` / `inferences` / `scoped`. A subset of the old, pre-merge `subnets`
|
|
275
|
+
records (subnetIp-only, no CIDR) had zero overlap with any canonical CIDR and were
|
|
276
|
+
quarantined rather than dropped — they live in the `legacy` content type
|
|
277
|
+
(`subnets-legacy`), keyed by `concept.identifier.kind: "subnetIp"`, with an explicit
|
|
278
|
+
`_system.quarantine` block (`legacy: true`, `status`, `reason`) marking them as not
|
|
279
|
+
eligible for `assets -> subnets` joins. `assets.data.subnetCidr` (precomputed via CIDR
|
|
280
|
+
containment against canonical `subnets-snapshots`) is the join key used for that
|
|
281
|
+
relation — see `scripts/memorix-subnets-merge/` for the migration and backfill scripts,
|
|
282
|
+
and `temp/part-b-implementation-design-2026-07-05T06-20-00Z.md` for the full design.
|
|
@@ -0,0 +1,114 @@
|
|
|
1
|
+
# Memorix Workflow Format (`memorix-workflow/1`)
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Versioned JSON for **product workflow state** attached to a subject object via the
|
|
6
|
+
content type `workflow` (collection `{objectType}-workflow`).
|
|
7
|
+
|
|
8
|
+
This is **not** inventory (snapshots), **not** AI analysis/decision, and **not**
|
|
9
|
+
vendor audit telemetry. It stores human/product work state: assignee, status,
|
|
10
|
+
append-only audit of UI mutations, and close-via tasks.
|
|
11
|
+
|
|
12
|
+
Catalog `memorix-workflow-field-schemas` defines which scalar/enum fields may
|
|
13
|
+
appear under `data` (beyond reserved `audit` / `tasks`).
|
|
14
|
+
|
|
15
|
+
## Collection postfix
|
|
16
|
+
|
|
17
|
+
| Descriptor content-type key | Postfix | Collection | Cardinality |
|
|
18
|
+
|---|---|---|---|
|
|
19
|
+
| `workflow` | `workflow` | `{objectType}-workflow` | `1:1` |
|
|
20
|
+
|
|
21
|
+
Join key: same `recordId` as the subject entity’s canonical snapshot.
|
|
22
|
+
|
|
23
|
+
## Canonical document
|
|
24
|
+
|
|
25
|
+
```ts
|
|
26
|
+
type MemorixWorkflowDocument = {
|
|
27
|
+
format: "memorix-workflow/1";
|
|
28
|
+
recordId: string;
|
|
29
|
+
objectType: string;
|
|
30
|
+
data: {
|
|
31
|
+
/** Catalog-driven properties (validated against field schema). */
|
|
32
|
+
[fieldKey: string]: unknown;
|
|
33
|
+
/** Format-reserved — API-managed only. */
|
|
34
|
+
audit: WorkflowAuditEntry[];
|
|
35
|
+
tasks: WorkflowTask[];
|
|
36
|
+
};
|
|
37
|
+
modifiedAt: string;
|
|
38
|
+
modifiedBy?: string;
|
|
39
|
+
};
|
|
40
|
+
|
|
41
|
+
type WorkflowAuditEntry = {
|
|
42
|
+
id: string;
|
|
43
|
+
at: string;
|
|
44
|
+
actor: string;
|
|
45
|
+
action: string;
|
|
46
|
+
field?: string;
|
|
47
|
+
previous?: unknown;
|
|
48
|
+
next?: unknown;
|
|
49
|
+
};
|
|
50
|
+
|
|
51
|
+
type WorkflowTask = {
|
|
52
|
+
id: string;
|
|
53
|
+
dk: string;
|
|
54
|
+
title: string;
|
|
55
|
+
via?: string;
|
|
56
|
+
status: "open" | "done" | "cancelled";
|
|
57
|
+
createdAt: string;
|
|
58
|
+
createdBy: string;
|
|
59
|
+
completedAt?: string;
|
|
60
|
+
};
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
## Rules
|
|
64
|
+
|
|
65
|
+
1. Custom fields under `data` **must** be declared in `memorix-workflow-field-schemas`
|
|
66
|
+
for that `objectType` (when `rejectUnknownFields` is true, default).
|
|
67
|
+
2. `audit` and `tasks` are **never** writable via generic field PATCH — dedicated
|
|
68
|
+
API endpoints only.
|
|
69
|
+
3. Enum values are stored as English **keys**; UI maps display labels.
|
|
70
|
+
4. Do not put workflow state on snapshots, analysis, or decision documents.
|
|
71
|
+
|
|
72
|
+
## Field schema catalog
|
|
73
|
+
|
|
74
|
+
Catalog id: `memorix-workflow-field-schemas`
|
|
75
|
+
|
|
76
|
+
One item per object type (natural id e.g. `assets-workflow-fields`):
|
|
77
|
+
|
|
78
|
+
```ts
|
|
79
|
+
type WorkflowFieldSchemaItem = {
|
|
80
|
+
id: string;
|
|
81
|
+
objectType: string;
|
|
82
|
+
contentType: "workflow";
|
|
83
|
+
rejectUnknownFields: boolean;
|
|
84
|
+
fields: WorkflowFieldDef[];
|
|
85
|
+
};
|
|
86
|
+
|
|
87
|
+
type WorkflowFieldDef = {
|
|
88
|
+
key: string;
|
|
89
|
+
label: string;
|
|
90
|
+
description?: string;
|
|
91
|
+
valueType: "string" | "number" | "boolean" | "datetime" | "enum";
|
|
92
|
+
enumValues?: string[];
|
|
93
|
+
required?: boolean;
|
|
94
|
+
writable?: boolean;
|
|
95
|
+
defaultValue?: unknown;
|
|
96
|
+
list?: boolean;
|
|
97
|
+
item?: boolean;
|
|
98
|
+
section?: string;
|
|
99
|
+
order?: number;
|
|
100
|
+
};
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
## API
|
|
104
|
+
|
|
105
|
+
See `@x12i/memorix-workflow-api` — catalog-driven HTTP for schema list/get/upsert,
|
|
106
|
+
overview aggregation, document list/get/ensure, field PATCH, audit list, and
|
|
107
|
+
tasks CRUD. UI: `@x12i/memorix-workflow-ui`.
|
|
108
|
+
|
|
109
|
+
List/overview notes:
|
|
110
|
+
|
|
111
|
+
- `GET /api/workflow/schemas` — discover object types with field schemas
|
|
112
|
+
- `GET /api/workflow/:objectType` — list existing workflow overlays (slim `list:true` projection)
|
|
113
|
+
- `GET /api/workflow/overview` — totals and recent audit activity
|
|
114
|
+
- `GET .../audit` and `GET .../tasks` are read-only (404 when the document is missing; they do not create)
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
{
|
|
2
|
+
"_id": "6914a132880706529ece3da1",
|
|
3
|
+
"recordId": "69d216ed358778e93215787e",
|
|
4
|
+
"concept": {
|
|
5
|
+
"identifier": {
|
|
6
|
+
"kind": "pluginId",
|
|
7
|
+
"pluginId": "142594",
|
|
8
|
+
"identifierKey": "pluginId:142594"
|
|
9
|
+
},
|
|
10
|
+
"title": "Oracle WebLogic Server RCE (CVE-2020-14882)"
|
|
11
|
+
},
|
|
12
|
+
"data": {
|
|
13
|
+
"name": "Oracle WebLogic Server RCE (CVE-2020-14882)",
|
|
14
|
+
"cveId": "CVE-2020-14882",
|
|
15
|
+
"severity": "critical",
|
|
16
|
+
"pluginId": 142594
|
|
17
|
+
},
|
|
18
|
+
"associatedData": [
|
|
19
|
+
{ "assetIp": "10.100.24.31", "hostname": "BGAK105-002" },
|
|
20
|
+
{ "assetIp": "10.100.24.41", "hostname": "BGAK105-003" }
|
|
21
|
+
],
|
|
22
|
+
"analytics": {
|
|
23
|
+
"affectedAssets": { "total": 2 }
|
|
24
|
+
},
|
|
25
|
+
"enrichment": {
|
|
26
|
+
"cvss": { "baseScore": 9.8 },
|
|
27
|
+
"epss": { "epss": 0.94454 },
|
|
28
|
+
"kev": { "isKnownExploited": true }
|
|
29
|
+
},
|
|
30
|
+
"insights": {
|
|
31
|
+
"exploitability": { "level": "HIGH", "confidence": 0.92 }
|
|
32
|
+
},
|
|
33
|
+
"_system": {
|
|
34
|
+
"lifecycle": [
|
|
35
|
+
{ "process": "enriched", "processedAt": "2026-04-05T08:01:15.511Z" }
|
|
36
|
+
]
|
|
37
|
+
}
|
|
38
|
+
}
|
package/package.json
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@x12i/memorix-format",
|
|
3
|
+
"version": "1.0.0",
|
|
4
|
+
"description": "Memorix record format: types, validator, CLI, and specification docs.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"main": "./dist/index.js",
|
|
7
|
+
"types": "./dist/index.d.ts",
|
|
8
|
+
"bin": {
|
|
9
|
+
"memorix-format": "./dist/cli.js"
|
|
10
|
+
},
|
|
11
|
+
"exports": {
|
|
12
|
+
".": {
|
|
13
|
+
"types": "./dist/index.d.ts",
|
|
14
|
+
"import": "./dist/index.js"
|
|
15
|
+
},
|
|
16
|
+
"./validate": {
|
|
17
|
+
"types": "./dist/validate.d.ts",
|
|
18
|
+
"import": "./dist/validate.js"
|
|
19
|
+
},
|
|
20
|
+
"./types": {
|
|
21
|
+
"types": "./dist/types.d.ts",
|
|
22
|
+
"import": "./dist/types.js"
|
|
23
|
+
}
|
|
24
|
+
},
|
|
25
|
+
"files": [
|
|
26
|
+
"dist",
|
|
27
|
+
"docs",
|
|
28
|
+
"examples",
|
|
29
|
+
"README.md"
|
|
30
|
+
],
|
|
31
|
+
"sideEffects": false,
|
|
32
|
+
"scripts": {
|
|
33
|
+
"prebuild": "rm -rf dist",
|
|
34
|
+
"build": "tsc -p tsconfig.build.json",
|
|
35
|
+
"test": "vitest run",
|
|
36
|
+
"test:watch": "vitest",
|
|
37
|
+
"typecheck": "tsc --noEmit",
|
|
38
|
+
"prepack": "npm run build",
|
|
39
|
+
"prepublishOnly": "npm run build && npm test"
|
|
40
|
+
},
|
|
41
|
+
"engines": {
|
|
42
|
+
"node": ">=18.0.0"
|
|
43
|
+
},
|
|
44
|
+
"dependencies": {},
|
|
45
|
+
"devDependencies": {
|
|
46
|
+
"@types/node": "^20.14.0",
|
|
47
|
+
"typescript": "^5.6.0",
|
|
48
|
+
"vitest": "^2.1.0"
|
|
49
|
+
},
|
|
50
|
+
"publishConfig": {
|
|
51
|
+
"access": "public",
|
|
52
|
+
"registry": "https://registry.npmjs.org/"
|
|
53
|
+
},
|
|
54
|
+
"repository": {
|
|
55
|
+
"type": "git",
|
|
56
|
+
"url": "git+ssh://git@github.com/x12i/memorix-mono-repo.git",
|
|
57
|
+
"directory": "memorix-format"
|
|
58
|
+
},
|
|
59
|
+
"keywords": [
|
|
60
|
+
"memorix",
|
|
61
|
+
"format",
|
|
62
|
+
"validator",
|
|
63
|
+
"record",
|
|
64
|
+
"exellix"
|
|
65
|
+
],
|
|
66
|
+
"license": "exellix-license"
|
|
67
|
+
}
|