@x12i/memorix-descriptors 1.4.2 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -10
- package/dist/admin/create-admin.d.ts +20 -0
- package/dist/admin/create-admin.d.ts.map +1 -1
- package/dist/admin/create-admin.js +38 -2
- package/dist/admin/create-admin.js.map +1 -1
- package/dist/catalog/ids.d.ts +7 -0
- package/dist/catalog/ids.d.ts.map +1 -1
- package/dist/catalog/ids.js +13 -0
- package/dist/catalog/ids.js.map +1 -1
- package/dist/catalox/client.d.ts.map +1 -1
- package/dist/catalox/client.js +26 -1
- package/dist/catalox/client.js.map +1 -1
- package/dist/cli/index.js +146 -3
- package/dist/cli/index.js.map +1 -1
- package/dist/env/load-env-file.d.ts +7 -0
- package/dist/env/load-env-file.d.ts.map +1 -0
- package/dist/env/load-env-file.js +74 -0
- package/dist/env/load-env-file.js.map +1 -0
- package/dist/index.d.ts +9 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +5 -1
- package/dist/index.js.map +1 -1
- package/dist/migration/apply.d.ts +21 -0
- package/dist/migration/apply.d.ts.map +1 -0
- package/dist/migration/apply.js +102 -0
- package/dist/migration/apply.js.map +1 -0
- package/dist/migration/backup.d.ts +8 -0
- package/dist/migration/backup.d.ts.map +1 -0
- package/dist/migration/backup.js +27 -0
- package/dist/migration/backup.js.map +1 -0
- package/dist/migration/classify.d.ts +6 -0
- package/dist/migration/classify.d.ts.map +1 -0
- package/dist/migration/classify.js +67 -0
- package/dist/migration/classify.js.map +1 -0
- package/dist/migration/ensure-canonical-catalogs.d.ts +4 -0
- package/dist/migration/ensure-canonical-catalogs.d.ts.map +1 -0
- package/dist/migration/ensure-canonical-catalogs.js +101 -0
- package/dist/migration/ensure-canonical-catalogs.js.map +1 -0
- package/dist/migration/generate-descriptors.d.ts +22 -0
- package/dist/migration/generate-descriptors.d.ts.map +1 -0
- package/dist/migration/generate-descriptors.js +234 -0
- package/dist/migration/generate-descriptors.js.map +1 -0
- package/dist/migration/index.d.ts +13 -0
- package/dist/migration/index.d.ts.map +1 -0
- package/dist/migration/index.js +12 -0
- package/dist/migration/index.js.map +1 -0
- package/dist/migration/inspect.d.ts +9 -0
- package/dist/migration/inspect.d.ts.map +1 -0
- package/dist/migration/inspect.js +57 -0
- package/dist/migration/inspect.js.map +1 -0
- package/dist/migration/load-snapshot.d.ts +15 -0
- package/dist/migration/load-snapshot.d.ts.map +1 -0
- package/dist/migration/load-snapshot.js +83 -0
- package/dist/migration/load-snapshot.js.map +1 -0
- package/dist/migration/mongo-inventory.d.ts +20 -0
- package/dist/migration/mongo-inventory.d.ts.map +1 -0
- package/dist/migration/mongo-inventory.js +95 -0
- package/dist/migration/mongo-inventory.js.map +1 -0
- package/dist/migration/object-type-specs.d.ts +17 -0
- package/dist/migration/object-type-specs.d.ts.map +1 -0
- package/dist/migration/object-type-specs.js +58 -0
- package/dist/migration/object-type-specs.js.map +1 -0
- package/dist/migration/plan.d.ts +4 -0
- package/dist/migration/plan.d.ts.map +1 -0
- package/dist/migration/plan.js +136 -0
- package/dist/migration/plan.js.map +1 -0
- package/dist/migration/reset.d.ts +17 -0
- package/dist/migration/reset.d.ts.map +1 -0
- package/dist/migration/reset.js +73 -0
- package/dist/migration/reset.js.map +1 -0
- package/dist/migration/types.d.ts +76 -0
- package/dist/migration/types.d.ts.map +1 -0
- package/dist/migration/types.js +17 -0
- package/dist/migration/types.js.map +1 -0
- package/dist/migration/validate-plan.d.ts +15 -0
- package/dist/migration/validate-plan.d.ts.map +1 -0
- package/dist/migration/validate-plan.js +112 -0
- package/dist/migration/validate-plan.js.map +1 -0
- package/dist/migration/verify.d.ts +15 -0
- package/dist/migration/verify.d.ts.map +1 -0
- package/dist/migration/verify.js +27 -0
- package/dist/migration/verify.js.map +1 -0
- package/dist/mutations/common.d.ts +7 -4
- package/dist/mutations/common.d.ts.map +1 -1
- package/dist/mutations/common.js +54 -3
- package/dist/mutations/common.js.map +1 -1
- package/dist/mutations/entity.d.ts.map +1 -1
- package/dist/mutations/entity.js +5 -1
- package/dist/mutations/entity.js.map +1 -1
- package/dist/mutations/execute.d.ts.map +1 -1
- package/dist/mutations/execute.js +9 -1
- package/dist/mutations/execute.js.map +1 -1
- package/dist/mutations/item.d.ts.map +1 -1
- package/dist/mutations/item.js +5 -1
- package/dist/mutations/item.js.map +1 -1
- package/dist/mutations/list.js +8 -1
- package/dist/mutations/list.js.map +1 -1
- package/dist/source/action-drafts.d.ts +17 -0
- package/dist/source/action-drafts.d.ts.map +1 -0
- package/dist/source/action-drafts.js +191 -0
- package/dist/source/action-drafts.js.map +1 -0
- package/dist/source/aliases.d.ts +23 -0
- package/dist/source/aliases.d.ts.map +1 -0
- package/dist/source/aliases.js +60 -0
- package/dist/source/aliases.js.map +1 -0
- package/dist/source/catalog-health.d.ts +17 -0
- package/dist/source/catalog-health.d.ts.map +1 -0
- package/dist/source/catalog-health.js +87 -0
- package/dist/source/catalog-health.js.map +1 -0
- package/dist/source/envelope.d.ts +29 -0
- package/dist/source/envelope.d.ts.map +1 -0
- package/dist/source/envelope.js +125 -0
- package/dist/source/envelope.js.map +1 -0
- package/dist/source/graph.d.ts +12 -0
- package/dist/source/graph.d.ts.map +1 -0
- package/dist/source/graph.js +148 -0
- package/dist/source/graph.js.map +1 -0
- package/dist/source/guards.d.ts +7 -0
- package/dist/source/guards.d.ts.map +1 -0
- package/dist/source/guards.js +74 -0
- package/dist/source/guards.js.map +1 -0
- package/dist/source/index.d.ts +14 -0
- package/dist/source/index.d.ts.map +1 -0
- package/dist/source/index.js +14 -0
- package/dist/source/index.js.map +1 -0
- package/dist/source/inspectable.d.ts +11 -0
- package/dist/source/inspectable.d.ts.map +1 -0
- package/dist/source/inspectable.js +167 -0
- package/dist/source/inspectable.js.map +1 -0
- package/dist/source/reconcile-source.d.ts +10 -0
- package/dist/source/reconcile-source.d.ts.map +1 -0
- package/dist/source/reconcile-source.js +40 -0
- package/dist/source/reconcile-source.js.map +1 -0
- package/dist/source/registry.d.ts +9 -0
- package/dist/source/registry.d.ts.map +1 -0
- package/dist/source/registry.js +121 -0
- package/dist/source/registry.js.map +1 -0
- package/dist/source/snapshot.d.ts +15 -0
- package/dist/source/snapshot.d.ts.map +1 -0
- package/dist/source/snapshot.js +160 -0
- package/dist/source/snapshot.js.map +1 -0
- package/dist/source/types.d.ts +222 -0
- package/dist/source/types.d.ts.map +1 -0
- package/dist/source/types.js +2 -0
- package/dist/source/types.js.map +1 -0
- package/dist/source/validation.d.ts +12 -0
- package/dist/source/validation.d.ts.map +1 -0
- package/dist/source/validation.js +229 -0
- package/dist/source/validation.js.map +1 -0
- package/dist/tests/canonical-metadata.test.d.ts +2 -0
- package/dist/tests/canonical-metadata.test.d.ts.map +1 -0
- package/dist/tests/canonical-metadata.test.js +198 -0
- package/dist/tests/canonical-metadata.test.js.map +1 -0
- package/dist/tests/source-aware.test.d.ts +2 -0
- package/dist/tests/source-aware.test.d.ts.map +1 -0
- package/dist/tests/source-aware.test.js +227 -0
- package/dist/tests/source-aware.test.js.map +1 -0
- package/dist/types/canonical-v2.d.ts +164 -0
- package/dist/types/canonical-v2.d.ts.map +1 -0
- package/dist/types/canonical-v2.js +6 -0
- package/dist/types/canonical-v2.js.map +1 -0
- package/dist/types/index.d.ts +25 -2
- package/dist/types/index.d.ts.map +1 -1
- package/dist/validation/canonical-contamination.d.ts +6 -0
- package/dist/validation/canonical-contamination.d.ts.map +1 -0
- package/dist/validation/canonical-contamination.js +71 -0
- package/dist/validation/canonical-contamination.js.map +1 -0
- package/dist/validation/canonical-metadata-validation.d.ts +13 -0
- package/dist/validation/canonical-metadata-validation.d.ts.map +1 -0
- package/dist/validation/canonical-metadata-validation.js +248 -0
- package/dist/validation/canonical-metadata-validation.js.map +1 -0
- package/dist/validation/canonical-v2-validation.d.ts +8 -0
- package/dist/validation/canonical-v2-validation.d.ts.map +1 -0
- package/dist/validation/canonical-v2-validation.js +273 -0
- package/dist/validation/canonical-v2-validation.js.map +1 -0
- package/docs/MEMORIX-CATALOX-CATALOG-MAP.md +589 -0
- package/docs/MEMORIX-CATALOX-CONTRACTS.md +17 -10
- package/docs/MEMORIX-DATABASE-CONVENTIONS.md +10 -7
- package/docs/MEMORIX-OBJECT-TYPES-AND-TARGETS.md +207 -0
- package/package.json +9 -1
|
@@ -0,0 +1,589 @@
|
|
|
1
|
+
# Memorix Catalox catalog map
|
|
2
|
+
|
|
3
|
+
**Purpose:** Single reference for every Memorix `catalogId` stored in Catalox under app `memorix`, including catalog metadata, item schemas, shipped items, cross-links, and the **why** behind each choice.
|
|
4
|
+
|
|
5
|
+
**Terminology:** Catalog **object types** (`assets`, `vulnerabilities`, …) vs storage **targets** (`entity`, `event`, `knowledge`) vs overloaded “entity” names are defined in **[MEMORIX-OBJECT-TYPES-AND-TARGETS.md](./MEMORIX-OBJECT-TYPES-AND-TARGETS.md)**. Read that first if “entities vs objects” is unclear.
|
|
6
|
+
|
|
7
|
+
**Sources (2026-05-23):**
|
|
8
|
+
|
|
9
|
+
| Source | What it represents |
|
|
10
|
+
|--------|-------------------|
|
|
11
|
+
| `@x12i/memorix-descriptors` | Canonical catalog ids, `ensureCatalog` metadata, write/read paths, integrity rules |
|
|
12
|
+
| `@x12i/memorix-retrieval` → `catalox-seeds/inputs/` | Shipped descriptor JSON (expected Catalox contents after seed apply) |
|
|
13
|
+
| `docs/MEMORIX-CATALOX-CONTRACTS.md`, `docs/MEMORIX-DATABASE-CONVENTIONS.md` | Cross-package contracts |
|
|
14
|
+
| **Live Firebase** (project `x12i-493313`, app `memorix`) | Catalog registry + item counts via `@x12i/catalox` (`listCatalogs`, `listCatalogItems`) |
|
|
15
|
+
|
|
16
|
+
Reconcile local exports with production:
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
npx memorix-descriptors seed export ./catalox-seeds/inputs
|
|
20
|
+
# or: admin.loadSnapshot() / validateMemorixDescriptors()
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## 1. Namespace
|
|
26
|
+
|
|
27
|
+
| Field | Value | Why |
|
|
28
|
+
|-------|-------|-----|
|
|
29
|
+
| **App id** | `memorix` | Single namespace for all Memorix metadata catalogs so retrieval, Explorer, and descriptors share one discovery surface. |
|
|
30
|
+
| **Override** | `CATALOX_APP_ID` → `MEMORIX_APP_ID` | Allows non-default deployments without forking catalog ids. |
|
|
31
|
+
| **Rule** | Do not invent parallel catalog names | Fragmented truth breaks discovery (`discoverMemorixEntities` lists one catalog only) and seed apply. |
|
|
32
|
+
|
|
33
|
+
**Package-managed catalogs:** **5** (`memorix-descriptors` ensures these). Retrieval’s public contract documents the **first three** descriptor catalogs.
|
|
34
|
+
|
|
35
|
+
**Production Firebase (this deployment):** **7** Memorix-related native catalogs with item rows, plus legacy catalogs still present in Firestore (see §2.1).
|
|
36
|
+
|
|
37
|
+
### 1.1 Firestore physical layout (what you see in Firebase console)
|
|
38
|
+
|
|
39
|
+
Catalox stores **native catalog items** in top-level Firestore collections named:
|
|
40
|
+
|
|
41
|
+
```text
|
|
42
|
+
catalogData-{catalogId}-items/{itemId}
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
The Firebase console often shows the **suffix** with a leading `-`, e.g. `-memorix-entity-descriptors-items` is the tail of **`catalogData-memorix-entity-descriptors-items`**.
|
|
46
|
+
|
|
47
|
+
| Console suffix (your screenshot) | Full Firestore collection | `catalogId` |
|
|
48
|
+
|----------------------------------|---------------------------|-------------|
|
|
49
|
+
| `-memorix-entity-descriptors-items` | `catalogData-memorix-entity-descriptors-items` | `memorix-entity-descriptors` |
|
|
50
|
+
| `-memorix-list-descriptors-items` | `catalogData-memorix-list-descriptors-items` | `memorix-list-descriptors` |
|
|
51
|
+
| `-memorix-item-descriptors-items` | `catalogData-memorix-item-descriptors-items` | `memorix-item-descriptors` |
|
|
52
|
+
| `-memorix-entities-items` | `catalogData-memorix-entities-items` | `memorix-entities` (legacy — see §12) |
|
|
53
|
+
| `-memorix-entity_content_types-items` | `catalogData-memorix_entity_content_types-items` | `memorix_entity_content_types` (legacy — underscore in id) |
|
|
54
|
+
|
|
55
|
+
**Not the same as Mongo:** Mongo database `memorix-entities` is the **storage target `entity`** database (payload). Catalox catalog `memorix-entities` holds **legacy inferred schema samples** — unrelated catalog id. Catalog **`memorix-entity-descriptors`** registers **object types** for all targets (`entity`, `event`, `knowledge`), not only the entity tier. See [MEMORIX-OBJECT-TYPES-AND-TARGETS.md](./MEMORIX-OBJECT-TYPES-AND-TARGETS.md).
|
|
56
|
+
|
|
57
|
+
Other Catalox roots in the same project (not in your screenshot): `apps`, `catalogs`, `catalogBindings`, `catalogDescriptors`, legacy `catalogData/{catalogId}/items` subcollections (pre-migration). See `@x12i/catalox` README → `docs/firestore-data-model.md`.
|
|
58
|
+
|
|
59
|
+
```text
|
|
60
|
+
appId: memorix
|
|
61
|
+
│
|
|
62
|
+
┌──────────────────────┼──────────────────────┐
|
|
63
|
+
│ │ │
|
|
64
|
+
▼ ▼ ▼
|
|
65
|
+
memorix-entity- memorix-list- memorix-item-
|
|
66
|
+
descriptors descriptors descriptors
|
|
67
|
+
│ │ │
|
|
68
|
+
│ default-list / default-item edges │
|
|
69
|
+
└──────────────────────┴──────────────────────┘
|
|
70
|
+
│
|
|
71
|
+
┌────────────┴────────────┐
|
|
72
|
+
▼ ▼
|
|
73
|
+
memorix-completion-mappings memorix-inventory-policies
|
|
74
|
+
(optional, runtime CRUD) (single item: default)
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
## 2. Catalog index
|
|
80
|
+
|
|
81
|
+
### 2.1 Live Firebase inventory (app `memorix`, 2026-05-23)
|
|
82
|
+
|
|
83
|
+
| `catalogId` | Firestore items collection | Items (live) | Status | Primary consumer |
|
|
84
|
+
|-------------|------------------------------|--------------|--------|------------------|
|
|
85
|
+
| `memorix-entity-descriptors` | `catalogData-memorix-entity-descriptors-items` | **3** | **Current** — retrieval discovery | `@x12i/memorix-retrieval`, `@x12i/memorix-descriptors` |
|
|
86
|
+
| `memorix-list-descriptors` | `catalogData-memorix-list-descriptors-items` | **5** | **Current** | Retrieval lists / workspace; descriptor slices |
|
|
87
|
+
| `memorix-item-descriptors` | `catalogData-memorix-item-descriptors-items` | **3** | **Current** | Retrieval detail views |
|
|
88
|
+
| `memorix-completion-mappings` | `catalogData-memorix-completion-mappings-items` | ? (list blocked) | Registered; `listCatalogItems` → adapter error | `@x12i/memorix-completion` when wired |
|
|
89
|
+
| `memorix-inventory-policies` | `catalogData-memorix-inventory-policies-items` | **0** (catalog not registered) | Not deployed yet | `memorix-descriptors` reconcile |
|
|
90
|
+
| `memorix-entities` | `catalogData-memorix-entities-items` | **9** | **Legacy** — schema inference | Old Explorer / `catalog-memorix-entities` tooling |
|
|
91
|
+
| `memorix_entity_content_types` | `catalogData-memorix_entity_content_types-items` | **15** | **Legacy** — xmemory content types | `@x12i/xmemory-store` era; superseded by `contentTypes` on entity descriptors |
|
|
92
|
+
| `knowledge` | `catalogData-knowledge-items` | **3** | **Active** (parallel track) | Memorix knowledge / skills (not entity descriptors) |
|
|
93
|
+
|
|
94
|
+
**Live item ids (descriptor catalogs — match seeds):**
|
|
95
|
+
|
|
96
|
+
| Catalog | Item ids |
|
|
97
|
+
|---------|----------|
|
|
98
|
+
| `memorix-entity-descriptors` | `assets`, `variabilities-groups`, `vulnerabilities` |
|
|
99
|
+
| `memorix-list-descriptors` | `assets-main-list`, `critical-vulnerabilities-list`, `memorix-workspace-records-list`, `variabilities-groups-main-list`, `vulnerabilities-main-list` |
|
|
100
|
+
| `memorix-item-descriptors` | `asset-detail-item`, `variabilities-group-detail-item`, `vulnerability-detail-item` |
|
|
101
|
+
|
|
102
|
+
### 2.2 Package-managed catalog index (`memorix-descriptors`)
|
|
103
|
+
|
|
104
|
+
| # | `catalogId` | Group | `catalogType` | `sourceMode` | Item id field | Title field | In Firebase today |
|
|
105
|
+
|---|-------------|-------|---------------|--------------|---------------|-------------|-------------------|
|
|
106
|
+
| 1 | `memorix-entity-descriptors` | Descriptor | `memorix` | `native` | `id` | `entityName` | Yes (3 items) |
|
|
107
|
+
| 2 | `memorix-list-descriptors` | Descriptor | `memorix` | `native` | `id` | `title` | Yes (5 items) |
|
|
108
|
+
| 3 | `memorix-item-descriptors` | Descriptor | `memorix` | `native` | `id` | `title` | Yes (3 items) |
|
|
109
|
+
| 4 | `memorix-completion-mappings` | Control plane | `generic`* | `native` | `id` | `name` | Catalog exists; list adapter error |
|
|
110
|
+
| 5 | `memorix-inventory-policies` | System | `memorix` | `native` | `id` | `id` | Not registered |
|
|
111
|
+
|
|
112
|
+
\*Live registry shows `catalogType: "generic"` for completion mappings; seeds use `memorix`. Harmless for reads if items exist; align type on next `ensureCatalog` if desired.
|
|
113
|
+
|
|
114
|
+
**Why native + memorix type:** Catalox stores JSON documents as first-class items (not external DB mirrors). The `memorix` catalog type signals Memorix-specific validation and seed presets (`memorix-retrieval-descriptors`).
|
|
115
|
+
|
|
116
|
+
---
|
|
117
|
+
|
|
118
|
+
## 3. Shared item envelope
|
|
119
|
+
|
|
120
|
+
Every catalog item upserted through seeds or mutations follows:
|
|
121
|
+
|
|
122
|
+
```json
|
|
123
|
+
{
|
|
124
|
+
"id": "<same as Catalox itemId>",
|
|
125
|
+
"...descriptor or policy fields...",
|
|
126
|
+
"scope": {
|
|
127
|
+
"domains": ["network", "vulnerabilities"],
|
|
128
|
+
"agents": ["neo"]
|
|
129
|
+
}
|
|
130
|
+
}
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
| Aspect | Detail | Why |
|
|
134
|
+
|--------|--------|-----|
|
|
135
|
+
| **`id` in body** | Must match Catalox `itemId` | Stable joins for `getCatalogItem`, integrity graph, and seed round-trip. |
|
|
136
|
+
| **`scope`** | `domains[]`, `agents[]` | Filters which deployment/agent sees seeded descriptors; **not** a Mongo namespace. Workspace lists may use `scopeFilter: "inherit-seed-scope"` as a Catalox hint. |
|
|
137
|
+
| **Seed apply** | Scope merged into upsert `data` | Catalox seed CLI upserts `row.data` only; merge avoids losing scope until native top-level scope exists. |
|
|
138
|
+
| **Shipped scope** | `network`, `vulnerabilities` × agent `neo` | Matches Neo/network vulnerability product slice. |
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## 4. Catalog: `memorix-entity-descriptors` (object type descriptors)
|
|
143
|
+
|
|
144
|
+
> **Naming debt:** Catalog id and type name `MemorixEntityDescriptor` predate **event** and **knowledge** targets. Each item is a **catalog object type** descriptor (field `entityName`), not “an entity” in the Mongo `memorix-entities` sense. Proposed rename: `memorix-object-type-descriptors` — [roadmap](./MEMORIX-OBJECT-TYPES-AND-TARGETS.md#7-rename-roadmap-recommended--not-yet-applied-in-code).
|
|
145
|
+
|
|
146
|
+
### 4.1 Catalog metadata (Catalox)
|
|
147
|
+
|
|
148
|
+
| Property | Value |
|
|
149
|
+
|----------|-------|
|
|
150
|
+
| **Display name** | Memorix entity descriptors (display name is legacy) |
|
|
151
|
+
| **`catalogType`** | `memorix` |
|
|
152
|
+
| **`sourceMode`** | `native` |
|
|
153
|
+
| **`native.itemIdField`** | `id` |
|
|
154
|
+
| **`native.titleField`** | `entityName` |
|
|
155
|
+
| **Queryable fields** | `id`, `entityName`, `target` |
|
|
156
|
+
| **Descriptor capabilities** | list, get, create, edit, delete, import, export, validate, viewReferences (no sync) |
|
|
157
|
+
| **Ensured by** | `ensureMemorixDescriptorCatalogs()` |
|
|
158
|
+
|
|
159
|
+
### 4.2 Why this catalog exists
|
|
160
|
+
|
|
161
|
+
**Object type descriptors** (this catalog) are the **root of truth** for which domain types exist in Memorix:
|
|
162
|
+
|
|
163
|
+
- **Discovery:** `discoverMemorixEntities` lists this catalog only — returns **catalog object types** (field `entityName`); no env fallback (`MEMORIX_ENTITY_NAMES` is forbidden).
|
|
164
|
+
- **Mongo routing:** per-item `target` selects storage tier → `memorix-entities` | `memorix-events` | `memorix-knowledge`.
|
|
165
|
+
- **Schema:** Properties, content types, relations, and default list/item ids per object type.
|
|
166
|
+
- **Graph:** Relations are declared here; retrieval resolves joins at read time using Mongo paths, not Catalox edges.
|
|
167
|
+
|
|
168
|
+
### 4.3 Item schema (`MemorixEntityDescriptor`)
|
|
169
|
+
|
|
170
|
+
| Field | Required | Purpose |
|
|
171
|
+
|-------|----------|---------|
|
|
172
|
+
| `id` | yes | Catalog item id (usually equals `entityName`) |
|
|
173
|
+
| `entityName` | yes | **Catalog object type** id (kebab-case); rename candidate: `objectType` |
|
|
174
|
+
| `defaultListDescriptorId` | yes | Pointer into `memorix-list-descriptors` |
|
|
175
|
+
| `defaultItemDescriptorId` | yes | Pointer into `memorix-item-descriptors` |
|
|
176
|
+
| `target` | no (default `entity`) | `entity` \| `event` \| `knowledge` → database role |
|
|
177
|
+
| `collectionPrefix` | yes | `{prefix}-{postfix}` heuristic when `collection` omitted |
|
|
178
|
+
| `identity` | yes | `allowedIdFields`, `requiredExactlyOne`, `defaultIdField` |
|
|
179
|
+
| `defaults` | yes | Canonical content type, `dataRoot`, effective-date paths |
|
|
180
|
+
| `contentTypes` | yes | Named slices → Mongo collections |
|
|
181
|
+
| `properties` | yes | Field dictionary (paths, types, list/item flags) |
|
|
182
|
+
| `relations` | no | Cross-entity joins (Mongo path matching) |
|
|
183
|
+
| `contentDefaults` | no | Content-object storage defaults |
|
|
184
|
+
|
|
185
|
+
**Property rules:** List views only expose properties with `humanReadable: true`. Paths are dot paths from the Mongo document root (e.g. `data.assetIp`).
|
|
186
|
+
|
|
187
|
+
### 4.4 Shipped items (seed)
|
|
188
|
+
|
|
189
|
+
| Item id | `entityName` | `target` | Mongo DB | Canonical collection | Default list | Default item | Why this entity |
|
|
190
|
+
|---------|--------------|----------|----------|----------------------|--------------|--------------|-----------------|
|
|
191
|
+
| `assets` | `assets` | `entity` | `memorix-entities` | `assets-snapshots` | `assets-main-list` | `asset-detail-item` | XDR/network **assets** — stable inventory hosts; hub for vulnerability counts and `assetVulnerabilities` relation. |
|
|
192
|
+
| `vulnerabilities` | `vulnerabilities` | `event` | `memorix-events` | `vulnerabilities-snapshots` | `vulnerabilities-main-list` | `vulnerability-detail-item` | Vulnerability **findings** as events (timeline/detections), enriched fields (`data.enrichment.*`), MITRE/network context. |
|
|
193
|
+
| `variabilities-groups` | `variabilities-groups` | `entity` | `memorix-entities` | `variabilities-groups-snapshots` | `variabilities-groups-main-list` | `variabilities-group-detail-item` | Plugin-level **groups** aggregating many vulnerabilities; supports group ↔ vuln relation. |
|
|
194
|
+
|
|
195
|
+
**Target injection at seed build:** `MEMORIX_RETRIEVAL_ENTITY_TARGETS` in `memorix-descriptors` sets `target` when building manifests from retrieval seed files (assets/variabilities-groups → `entity`, vulnerabilities → `event`).
|
|
196
|
+
|
|
197
|
+
### 4.5 Shipped relations (why)
|
|
198
|
+
|
|
199
|
+
| Source entity | Relation key | Type | Target | Join (summary) | Why |
|
|
200
|
+
|---------------|--------------|------|--------|----------------|-----|
|
|
201
|
+
| `assets` | `assetVulnerabilities` | `oneToMany` | `vulnerabilities` | `data.ip_address` ↔ `data.assetIp` | Asset detail/list can show related vulns without storing edges in Mongo. |
|
|
202
|
+
| `vulnerabilities` | `affectedAsset` | `manyToOne` | `assets` | `data.assetIp` ↔ `data.ip_address` | Vuln detail enriches with host/XDR context from asset snapshots. |
|
|
203
|
+
| `vulnerabilities` | `vulnerabilityGroup` | `manyToOne` | `variabilities-groups` | `data.pluginId` ↔ `data.plugin_id` | Links finding to plugin-level group metadata. |
|
|
204
|
+
| `variabilities-groups` | `groupVulnerabilities` | `oneToMany` | `vulnerabilities` | `data.plugin_id` ↔ `data.pluginId` | Group detail lists member vulnerabilities. |
|
|
205
|
+
|
|
206
|
+
### 4.6 Property counts (shipped)
|
|
207
|
+
|
|
208
|
+
| Entity | Properties | Relations |
|
|
209
|
+
|--------|------------|-----------|
|
|
210
|
+
| `assets` | 17 | 1 |
|
|
211
|
+
| `vulnerabilities` | 27 | 2 |
|
|
212
|
+
| `variabilities-groups` | 18 | 1 |
|
|
213
|
+
|
|
214
|
+
---
|
|
215
|
+
|
|
216
|
+
## 5. Catalog: `memorix-list-descriptors`
|
|
217
|
+
|
|
218
|
+
### 5.1 Catalog metadata (Catalox)
|
|
219
|
+
|
|
220
|
+
| Property | Value |
|
|
221
|
+
|----------|-------|
|
|
222
|
+
| **Display name** | Memorix list descriptors |
|
|
223
|
+
| **`catalogType`** | `memorix` |
|
|
224
|
+
| **`sourceMode`** | `native` |
|
|
225
|
+
| **`native.itemIdField`** | `id` |
|
|
226
|
+
| **`native.titleField`** | `title` |
|
|
227
|
+
| **Queryable fields** | `id`, `entity`, `title` |
|
|
228
|
+
|
|
229
|
+
### 5.2 Why this catalog exists
|
|
230
|
+
|
|
231
|
+
List descriptors define **how** retrieval (and Explorer) query Mongo and shape tabular rows:
|
|
232
|
+
|
|
233
|
+
- Pagination driver (`leadingContentType`), default filters, sort allowlists.
|
|
234
|
+
- Column sets (`fields[]` → entity property keys).
|
|
235
|
+
- Optional extensions and `includeRelations` for list rows.
|
|
236
|
+
- **Workspace** and **slice** variants also live here (`kind: "workspace"` \| `"slice"`).
|
|
237
|
+
|
|
238
|
+
Without this catalog, hosts would query collections by name and invent columns — forbidden by contract.
|
|
239
|
+
|
|
240
|
+
### 5.3 Item kinds (discriminated union)
|
|
241
|
+
|
|
242
|
+
| `kind` | When | Key fields | Why separate |
|
|
243
|
+
|--------|------|------------|--------------|
|
|
244
|
+
| *(omit / `"entity"`)* | Per-entity table | `entity`, `leadingContentType`, `fields[]`, `pagination` | Standard entity list API (`fetchMemorixList`). |
|
|
245
|
+
| `"workspace"` | Cross-entity Records view | `fields[]`, optional `entities`, `listDescriptorByEntity` | Explorer workspace merges multiple entity lists in memory (v1). |
|
|
246
|
+
| `"slice"` | Filtered sub-view (admin) | `entityName`, `contentType`, `filter`, `sort` | Operational slices (e.g. critical subset) without new entities; written by `createSliceDescriptor`. |
|
|
247
|
+
|
|
248
|
+
### 5.4 Shipped items (seed)
|
|
249
|
+
|
|
250
|
+
| Item id | Kind | Entity | Title | Leading CT | Notable filters / sort | Why |
|
|
251
|
+
|---------|------|--------|-------|------------|------------------------|-----|
|
|
252
|
+
| `assets-main-list` | entity | `assets` | Assets | `snapshots` | Sort default: `ipAddress` asc | Primary asset inventory for Neo network view. |
|
|
253
|
+
| `vulnerabilities-main-list` | entity | `vulnerabilities` | Vulnerabilities | `snapshots` | Sort: `priorityScore` desc | Full vuln table with enrichment columns. |
|
|
254
|
+
| `critical-vulnerabilities-list` | entity | `vulnerabilities` | Critical Vulnerabilities | `snapshots` | Default filter `severityLevel >= 4` | Focused view for high-severity triage without separate entity. |
|
|
255
|
+
| `variabilities-groups-main-list` | entity | `variabilities-groups` | Vulnerability Groups | `snapshots` | Sort: `priorityScore` desc | Group-centric prioritization. |
|
|
256
|
+
| `memorix-workspace-records-list` | workspace | — | Workspace records | — | Sort: `lastUpdated` desc | Explorer **Records** when no entity filter; merges per-entity default lists. |
|
|
257
|
+
|
|
258
|
+
**Workspace column hints:** `entityName`, `recordTitle`, `recordId`, `source`, `lastUpdated`, `status` — synthetic/host-composed row fields, not all entity properties.
|
|
259
|
+
|
|
260
|
+
---
|
|
261
|
+
|
|
262
|
+
## 6. Catalog: `memorix-item-descriptors`
|
|
263
|
+
|
|
264
|
+
### 6.1 Catalog metadata (Catalox)
|
|
265
|
+
|
|
266
|
+
| Property | Value |
|
|
267
|
+
|----------|-------|
|
|
268
|
+
| **Display name** | Memorix item descriptors |
|
|
269
|
+
| **`catalogType`** | `memorix` |
|
|
270
|
+
| **`sourceMode`** | `native` |
|
|
271
|
+
| **`native.itemIdField`** | `id` |
|
|
272
|
+
| **`native.titleField`** | `title` |
|
|
273
|
+
| **Queryable fields** | `id`, `entity`, `title` |
|
|
274
|
+
|
|
275
|
+
### 6.2 Why this catalog exists
|
|
276
|
+
|
|
277
|
+
Item descriptors define **record detail** composition:
|
|
278
|
+
|
|
279
|
+
- Which content types to load per identity (`entityId` / `eventId`).
|
|
280
|
+
- `multiMatch` strategy when several Mongo docs share identity (`last` by `capturedAt` default).
|
|
281
|
+
- UI sections (`sections[].fields[]` → property keys).
|
|
282
|
+
- `includeRelations` for embedded related data on detail pages.
|
|
283
|
+
|
|
284
|
+
### 6.3 Item schema (summary)
|
|
285
|
+
|
|
286
|
+
| Field | Required | Purpose |
|
|
287
|
+
|-------|----------|---------|
|
|
288
|
+
| `id`, `entity`, `title` | yes | Identity and label |
|
|
289
|
+
| `identity.idField` | yes | `entityId` or `eventId` for fetch |
|
|
290
|
+
| `contentTypes[]` | yes | Per-type `required`, `multiMatch` |
|
|
291
|
+
| `sections[]` | yes | Grouped field layout |
|
|
292
|
+
| `includeRelations` | no | Relation keys from entity descriptor |
|
|
293
|
+
| `content` | no | Full content-object fetch policy |
|
|
294
|
+
|
|
295
|
+
### 6.4 Shipped items (seed)
|
|
296
|
+
|
|
297
|
+
| Item id | Entity | `identity.idField` | Sections (ids) | Relations included | Why |
|
|
298
|
+
|---------|--------|-------------------|----------------|-------------------|-----|
|
|
299
|
+
| `asset-detail-item` | `assets` | `entityId` | summary, vulnerabilitySummary, impact, xdr | `assetVulnerabilities` (array, limit 100) | Asset drill-down with vuln rollups and XDR raw block. |
|
|
300
|
+
| `vulnerability-detail-item` | `vulnerabilities` | `entityId` | summary, asset, risk, threatIntelligence, networkAndMitre | `affectedAsset`, `vulnerabilityGroup` | Single-finding analysis with asset + group context. |
|
|
301
|
+
| `variabilities-group-detail-item` | `variabilities-groups` | `entityId` | summary, affectedEntities, risk, threatIntelligence | `groupVulnerabilities` (array, limit 200) | Group drill-down with member vuln list. |
|
|
302
|
+
|
|
303
|
+
**Note:** Shipped vuln item uses `entityId` even though entity `target` is `event` — identity block allows both fields on entity descriptor; item fetch uses declared `idField`.
|
|
304
|
+
|
|
305
|
+
---
|
|
306
|
+
|
|
307
|
+
## 7. Catalog: `memorix-completion-mappings`
|
|
308
|
+
|
|
309
|
+
### 7.1 Catalog metadata (Catalox)
|
|
310
|
+
|
|
311
|
+
| Property | Value |
|
|
312
|
+
|----------|-------|
|
|
313
|
+
| **Display name** | Memorix completion mappings |
|
|
314
|
+
| **`catalogType`** | `memorix` |
|
|
315
|
+
| **`sourceMode`** | `native` |
|
|
316
|
+
| **`native.itemIdField`** | `id` |
|
|
317
|
+
| **`native.titleField`** | `name` |
|
|
318
|
+
| **Ensured by** | `ensureMemorixDescriptorCatalogs()` |
|
|
319
|
+
| **Read behavior** | If catalog missing or adapter error → **empty** (no throw) | Lets retrieval/descriptors run before mappings are seeded. |
|
|
320
|
+
|
|
321
|
+
### 7.2 Why this catalog exists
|
|
322
|
+
|
|
323
|
+
Completion pipelines enrich Memorix **Mongo `data`** from **source databases**. Storing mappings in Catalox (instead of repo-only JSON) gives:
|
|
324
|
+
|
|
325
|
+
- One control plane with entity descriptors (`entityType`, `targetCollection` validated against entity `contentTypes`).
|
|
326
|
+
- Runtime CRUD via `registerMapping` / `updateMapping` / `removeMapping`.
|
|
327
|
+
- Rename safety: `renameEntityDescriptor` updates `entityType` on affected mappings.
|
|
328
|
+
|
|
329
|
+
**Not in retrieval seeds:** `@x12i/memorix-retrieval` ships zero mapping files; mappings are deployment-specific.
|
|
330
|
+
|
|
331
|
+
### 7.3 Item schema (`MemorixCompletionMapping`)
|
|
332
|
+
|
|
333
|
+
| Field | Purpose | Why |
|
|
334
|
+
|-------|---------|-----|
|
|
335
|
+
| `id` | Stable mapping id | Catalog item key |
|
|
336
|
+
| `name` | Human label | Catalox title field |
|
|
337
|
+
| `entityType` | Must match an entity descriptor id / name | Ties mapping to schema and collections |
|
|
338
|
+
| `target` | `entity` \| `event` \| `knowledge` | Selects Memorix DB |
|
|
339
|
+
| `targetCollection` | Explicit Mongo collection | Must exist on entity `contentTypes` |
|
|
340
|
+
| `targetSelector` | `{ sourceCollection, idField }` | Provenance / join lineage on Memorix records |
|
|
341
|
+
| `source` | `{ databaseName, collection }` | Where to read enrichment from (often `{{ENV.*}}`) |
|
|
342
|
+
| `match` | `{ targetPath, sourcePath }` | Document join (e.g. `entityId` ↔ `vulnerabilityId`) |
|
|
343
|
+
| `writeRoot` | Always `"data"` | Never overwrite envelope fields |
|
|
344
|
+
| `fields[]` | `{ sourcePath, targetPath }` | Field-level copy rules |
|
|
345
|
+
| `onlyFillMissing` / `overwriteExisting` | Write mode | Default: fill missing only |
|
|
346
|
+
|
|
347
|
+
See `docs/MEMORIX-DATABASE-CONVENTIONS.md` § Completion mapping conventions for example JSON.
|
|
348
|
+
|
|
349
|
+
### 7.4 Shipped items
|
|
350
|
+
|
|
351
|
+
**None** in `memorix-retrieval` catalox-seeds. Expected count in fresh deploy: **0** until operators register mappings.
|
|
352
|
+
|
|
353
|
+
---
|
|
354
|
+
|
|
355
|
+
## 8. Catalog: `memorix-inventory-policies`
|
|
356
|
+
|
|
357
|
+
### 8.1 Catalog metadata (Catalox)
|
|
358
|
+
|
|
359
|
+
| Property | Value |
|
|
360
|
+
|----------|-------|
|
|
361
|
+
| **Display name** | Memorix inventory policies |
|
|
362
|
+
| **`catalogType`** | `memorix` |
|
|
363
|
+
| **`sourceMode`** | `native` |
|
|
364
|
+
| **`native.itemIdField`** | `id` |
|
|
365
|
+
| **`native.titleField`** | `id` |
|
|
366
|
+
| **Fixed item id** | `default` (`MEMORIX_INVENTORY_POLICY_ITEM_ID`) | Single policy document per app |
|
|
367
|
+
|
|
368
|
+
### 8.2 Why this catalog exists
|
|
369
|
+
|
|
370
|
+
Unified inventory compares **Mongo collections** to **descriptor-declared** collections. Operators need to suppress noise (legacy DBs, external collections, pending mappings) without deleting data:
|
|
371
|
+
|
|
372
|
+
- `ignoreCollection` / `unignoreCollection` append/remove entries.
|
|
373
|
+
- Reconcile marks ignored rows separately from true orphans.
|
|
374
|
+
|
|
375
|
+
**Why not in retrieval contract:** Read-only data tier does not manage inventory policy; `memorix-descriptors` owns it.
|
|
376
|
+
|
|
377
|
+
### 8.3 Item schema (`MemorixInventoryPolicy`)
|
|
378
|
+
|
|
379
|
+
```json
|
|
380
|
+
{
|
|
381
|
+
"id": "default",
|
|
382
|
+
"ignoredCollections": [
|
|
383
|
+
{
|
|
384
|
+
"target": "entity",
|
|
385
|
+
"collectionName": "some-legacy-col",
|
|
386
|
+
"reason": "legacy",
|
|
387
|
+
"databaseName": "optional-override",
|
|
388
|
+
"note": "optional",
|
|
389
|
+
"createdAt": "ISO-8601",
|
|
390
|
+
"createdBy": "operator"
|
|
391
|
+
}
|
|
392
|
+
]
|
|
393
|
+
}
|
|
394
|
+
```
|
|
395
|
+
|
|
396
|
+
| `reason` enum | Typical use |
|
|
397
|
+
|---------------|-------------|
|
|
398
|
+
| `legacy` | Old collections not yet mapped |
|
|
399
|
+
| `external` | Non-Memorix data in shared cluster |
|
|
400
|
+
| `reserved` | Planned future mapping |
|
|
401
|
+
| `temporary` | Migration in progress |
|
|
402
|
+
| `not-memorix` | Unrelated database artifact |
|
|
403
|
+
| `pending-mapping` | Known gap, mapping coming |
|
|
404
|
+
| `other` | Catch-all |
|
|
405
|
+
|
|
406
|
+
### 8.4 Shipped items
|
|
407
|
+
|
|
408
|
+
**None** required. Absent catalog/item → empty policy (`ignoredCollections: []`).
|
|
409
|
+
|
|
410
|
+
---
|
|
411
|
+
|
|
412
|
+
## 9. Cross-catalog reference graph
|
|
413
|
+
|
|
414
|
+
### 9.1 Default wiring (Option A)
|
|
415
|
+
|
|
416
|
+
```text
|
|
417
|
+
assets ──default-list──► assets-main-list
|
|
418
|
+
└──default-item──► asset-detail-item
|
|
419
|
+
|
|
420
|
+
vulnerabilities ──default-list──► vulnerabilities-main-list
|
|
421
|
+
└──default-item──► vulnerability-detail-item
|
|
422
|
+
└──alt list──────► critical-vulnerabilities-list
|
|
423
|
+
|
|
424
|
+
variabilities-groups ──default-list──► variabilities-groups-main-list
|
|
425
|
+
└──default-item──► variabilities-group-detail-item
|
|
426
|
+
|
|
427
|
+
(workspace) memorix-workspace-records-list ──► all discovered entities' default lists
|
|
428
|
+
```
|
|
429
|
+
|
|
430
|
+
### 9.2 Integrity edges checked by `memorix-descriptors`
|
|
431
|
+
|
|
432
|
+
| Edge kind | From | To | Why validated |
|
|
433
|
+
|-----------|------|-----|---------------|
|
|
434
|
+
| `default-list` | entity id | list id | `fetchMemorixListForEntity` must resolve |
|
|
435
|
+
| `default-item` | entity id | item id | `fetchMemorixItemForEntity` must resolve |
|
|
436
|
+
| `entity-list` | list id | entity name | List fields ⊆ entity properties |
|
|
437
|
+
| `entity-item` | item id | entity name | Item fields human-readable on entity |
|
|
438
|
+
| `relation:*` | entity | target entity | Graph tools and `includeRelations` |
|
|
439
|
+
| `mapping-entity` | mapping id | `entityType` | Completion target must exist |
|
|
440
|
+
|
|
441
|
+
### 9.3 Mermaid (shipped seed)
|
|
442
|
+
|
|
443
|
+
```mermaid
|
|
444
|
+
graph TD
|
|
445
|
+
assets["entity:assets"]
|
|
446
|
+
vulns["entity:vulnerabilities"]
|
|
447
|
+
groups["entity:variabilities-groups"]
|
|
448
|
+
assets_list["list:Assets"]
|
|
449
|
+
vulns_list["list:Vulnerabilities"]
|
|
450
|
+
crit_list["list:Critical Vulnerabilities"]
|
|
451
|
+
groups_list["list:Vulnerability Groups"]
|
|
452
|
+
workspace["list:Workspace records"]
|
|
453
|
+
asset_item["item:Asset Detail"]
|
|
454
|
+
vuln_item["item:Vulnerability Detail"]
|
|
455
|
+
group_item["item:Vulnerability Group Detail"]
|
|
456
|
+
|
|
457
|
+
assets -->|default-list| assets_list
|
|
458
|
+
assets -->|default-item| asset_item
|
|
459
|
+
vulns -->|default-list| vulns_list
|
|
460
|
+
vulns -->|default-item| vuln_item
|
|
461
|
+
groups -->|default-list| groups_list
|
|
462
|
+
groups -->|default-item| group_item
|
|
463
|
+
assets -->|relation:assetVulnerabilities| vulns
|
|
464
|
+
vulns -->|relation:affectedAsset| assets
|
|
465
|
+
vulns -->|relation:vulnerabilityGroup| groups
|
|
466
|
+
groups -->|relation:groupVulnerabilities| vulns
|
|
467
|
+
assets_list -->|entity-list| assets
|
|
468
|
+
vulns_list -->|entity-list| vulns
|
|
469
|
+
crit_list -->|entity-list| vulns
|
|
470
|
+
groups_list -->|entity-list| groups
|
|
471
|
+
asset_item -->|entity-item| assets
|
|
472
|
+
vuln_item -->|entity-item| vulns
|
|
473
|
+
group_item -->|entity-item| groups
|
|
474
|
+
```
|
|
475
|
+
|
|
476
|
+
---
|
|
477
|
+
|
|
478
|
+
## 10. Who reads / writes what
|
|
479
|
+
|
|
480
|
+
| Component | `entity` | `list` | `item` | `completion-mappings` | `inventory-policies` |
|
|
481
|
+
|-----------|----------|--------|--------|----------------------|----------------------|
|
|
482
|
+
| `@x12i/memorix-retrieval` | Read | Read | Read | — | — |
|
|
483
|
+
| `@x12i/memorix-descriptors` | R/W | R/W | R/W | R/W | R/W |
|
|
484
|
+
| `@x12i/memorix-completion` | — | — | — | Read (when wired) | — |
|
|
485
|
+
| `@x12i/memorix-explorer` | Via retrieval | Via retrieval | Via retrieval | — | — |
|
|
486
|
+
| Seed CLI (`memorix-retrieval`) | Apply | Apply | Apply | — | — |
|
|
487
|
+
| Seed export/import (`memorix-descriptors`) | Apply | Apply | Apply | Apply | Apply |
|
|
488
|
+
|
|
489
|
+
---
|
|
490
|
+
|
|
491
|
+
## 11. Seed preset metadata
|
|
492
|
+
|
|
493
|
+
| Field | Value |
|
|
494
|
+
|-------|-------|
|
|
495
|
+
| **Preset id** | `memorix-retrieval-descriptors` |
|
|
496
|
+
| **Preset version** | `1` |
|
|
497
|
+
| **Binding** | `{ appId: "memorix", catalogId, access: { canRead: true, canWrite: true } }` |
|
|
498
|
+
| **Input root** | `catalox-seeds/inputs/` (sibling `memorix-retrieval` or local copy) |
|
|
499
|
+
|
|
500
|
+
**Expected item counts after retrieval seed apply:**
|
|
501
|
+
|
|
502
|
+
| Catalog | Items |
|
|
503
|
+
|---------|-------|
|
|
504
|
+
| `memorix-entity-descriptors` | 3 |
|
|
505
|
+
| `memorix-list-descriptors` | 5 |
|
|
506
|
+
| `memorix-item-descriptors` | 3 |
|
|
507
|
+
| **Total descriptor items** | **11** |
|
|
508
|
+
|
|
509
|
+
---
|
|
510
|
+
|
|
511
|
+
## 12. Legacy & parallel Firebase catalogs (not retrieval seeds)
|
|
512
|
+
|
|
513
|
+
These appear in Firestore alongside the three descriptor catalogs. **`@x12i/memorix-retrieval` does not read them** for discovery or list/item APIs.
|
|
514
|
+
|
|
515
|
+
### 12.1 `memorix-entities` (legacy schema inference)
|
|
516
|
+
|
|
517
|
+
| Aspect | Detail |
|
|
518
|
+
|--------|--------|
|
|
519
|
+
| **Firestore** | `catalogData-memorix-entities-items` |
|
|
520
|
+
| **Why it existed** | Pre-descriptor pipeline: sample Mongo collections and publish inferred field catalogs for Explorer / catalog tooling (`scripts/catalog-memorix-entities.ts`, `schemaVersion: memorix.entity-descriptor.v1`). |
|
|
521
|
+
| **Item shape** | `entityName`, `displayName`, `documentCount`, `collectionFamilies[]`, `dataDescriptors[]` (inferred paths/types), `sourceCollections[]` — **not** `MemorixEntityDescriptor`. |
|
|
522
|
+
| **Live items (9)** | `any`, `assets`, `subnets`, `topology-cidr-graphs`, `topology-subnets`, `topology-vrs`, `topology-zones`, `vulnerabilities`, `vulnerability-groups` |
|
|
523
|
+
| **Why keep in Firebase** | Historical UI and inventory that still point at this catalog; safe to deprecate once nothing lists it. |
|
|
524
|
+
| **Why not use for new work** | Contracts forbid env/collection discovery bypass; retrieval uses `memorix-entity-descriptors` only. Relations belong on entity descriptors, not rows here (see retrieval `EXPLORER-HOST-APIS.md`). |
|
|
525
|
+
|
|
526
|
+
**Example (`assets` item):** ~42k Mongo docs sampled; families `assets-snapshots`, `assets-scoped`; 67 inferred `dataDescriptors` — parallel to but **not** the same JSON as `memorix-entity-descriptors/assets`.
|
|
527
|
+
|
|
528
|
+
**Naming trap:** Mongo DB **`memorix-entities`** ≠ Catalox catalog **`memorix-entities`**.
|
|
529
|
+
|
|
530
|
+
### 12.2 `memorix_entity_content_types` (legacy content-type registry)
|
|
531
|
+
|
|
532
|
+
| Aspect | Detail |
|
|
533
|
+
|--------|--------|
|
|
534
|
+
| **Firestore** | `catalogData-memorix_entity_content_types-items` (underscore in `catalogId`) |
|
|
535
|
+
| **Why it existed** | `@x12i/xmemory-store` registered suffix/content-type metadata (`sourcePackage`, `suffix`, `editorSlot`, …) before content types lived under each **entity descriptor** `contentTypes` block. |
|
|
536
|
+
| **Live items (15)** | `analytics`, `event_trigger`, `foresights`, `inference`, `inferences`, `narratives`, `raw`, `scoped`, `scoped_data`, `scoped_views`, `scoping_questions`, `snapshots`, `things`, `web`, `web-scoped` |
|
|
537
|
+
| **Why deprecated** | `MEMORIX-CATALOX-CONTRACTS.md` explicitly says: do not introduce `memorix_entity_content_types` for new work — use entity descriptor `contentTypes`. |
|
|
538
|
+
| **Console label** | May show as `-memorix-entity_content_types-items` (mixed hyphen/underscore from `catalogData-` + `memorix_entity_content_types`). |
|
|
539
|
+
|
|
540
|
+
### 12.3 `knowledge` (Memorix knowledge / skills)
|
|
541
|
+
|
|
542
|
+
| Aspect | Detail |
|
|
543
|
+
|--------|--------|
|
|
544
|
+
| **Firestore** | `catalogData-knowledge-items` |
|
|
545
|
+
| **`catalogType`** | `memorix` |
|
|
546
|
+
| **Why** | Curated knowledge artifacts (skills, playbooks) with `knowledgeId` — separate from entity/event **payload** descriptors. |
|
|
547
|
+
| **Live items (3)** | `skill-asset-context-analysis`, `skill-asset-exposure-analysis`, `skill-subnet-routed-reachability` |
|
|
548
|
+
| **Shape** | `knowledgeId`, `kind`, `title`, `content`, `status`, `metadata` |
|
|
549
|
+
| **Retrieval** | `target: "knowledge"` on entity descriptors is a coordinated extension; this catalog is not the same as the three descriptor catalogs. |
|
|
550
|
+
|
|
551
|
+
---
|
|
552
|
+
|
|
553
|
+
## 13. Operational notes
|
|
554
|
+
|
|
555
|
+
| Topic | Behavior |
|
|
556
|
+
|-------|----------|
|
|
557
|
+
| **Catalog ensure** | `MEMORIX_ENSURE_DESCRIPTOR_CATALOGS=1` on admin bootstrap calls `ensureMemorixDescriptorCatalogs`. |
|
|
558
|
+
| **Version checks** | `expectedVersion` on mutations reserved; Catalox optimistic version via `assertExpectedCatalogItemVersion`. |
|
|
559
|
+
| **Completion catalog** | Registered in Firebase; `listCatalogItems` may return adapter error — `memorix-descriptors` treats missing/adapter failure as empty when loading snapshot. |
|
|
560
|
+
| **Inventory policies** | Not in Firebase yet; `ignoreCollection` will create catalog + `default` item on first write. |
|
|
561
|
+
| **Slices** | Stored as list descriptors with `kind: "slice"`; same catalog id. |
|
|
562
|
+
| **Knowledge target** | Accepted on entity descriptors and mappings; full ecosystem support is coordinated across packages (see README). |
|
|
563
|
+
| **Contract drift** | `MEMORIX-CATALOX-CONTRACTS.md` lists three descriptor catalogs; this map adds completion + inventory + documents **legacy** Firebase catalogs. |
|
|
564
|
+
|
|
565
|
+
---
|
|
566
|
+
|
|
567
|
+
## 14. Quick lookup — catalogId → why (one line)
|
|
568
|
+
|
|
569
|
+
| `catalogId` | Why it exists |
|
|
570
|
+
|-------------|---------------|
|
|
571
|
+
| `memorix-entity-descriptors` | Defines **what** entities exist, how they map to Mongo, and how they relate. **Use this for retrieval.** |
|
|
572
|
+
| `memorix-list-descriptors` | Defines **how** to list and filter records in tables and workspace views. |
|
|
573
|
+
| `memorix-item-descriptors` | Defines **how** to compose record detail pages and related data. |
|
|
574
|
+
| `memorix-completion-mappings` | Defines **how** source DB rows enrich Memorix `data` without bypassing schema. |
|
|
575
|
+
| `memorix-inventory-policies` | Defines **which** Mongo collections to exclude from orphan/reconcile noise. |
|
|
576
|
+
| `memorix-entities` | **Legacy** — Mongo schema inference / catalog explorer; not descriptor-driven retrieval. |
|
|
577
|
+
| `memorix_entity_content_types` | **Legacy** — global content-type suffix registry (xmemory-store era). |
|
|
578
|
+
| `knowledge` | Memorix **knowledge/skills** documents (`knowledgeId`), not list/item descriptors. |
|
|
579
|
+
|
|
580
|
+
---
|
|
581
|
+
|
|
582
|
+
## 15. Related documents
|
|
583
|
+
|
|
584
|
+
- [MEMORIX-OBJECT-TYPES-AND-TARGETS.md](./MEMORIX-OBJECT-TYPES-AND-TARGETS.md) — **Object types vs targets vs “entity” overload** (read this for naming cleanup)
|
|
585
|
+
- [MEMORIX-CATALOX-CONTRACTS.md](./MEMORIX-CATALOX-CONTRACTS.md) — Descriptor JSON formats and retrieval behavior
|
|
586
|
+
- [MEMORIX-DATABASE-CONVENTIONS.md](./MEMORIX-DATABASE-CONVENTIONS.md) — Mongo databases, envelopes, completion mapping fields
|
|
587
|
+
- [../README.md](../README.md) — `MemorixDescriptorAdmin` API and CLI
|
|
588
|
+
|
|
589
|
+
**Code anchors:** `src/catalog/ids.ts`, `src/catalog/ensure-catalogs.ts`, `src/seeds/default-seed-spec.ts`, `src/catalox/client.ts` (`CATALOG_BY_KIND`).
|
|
@@ -2,10 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
**Canonical sync document** for all Memorix data packages and Memorix-consuming UIs (Explorer, completion pipelines, ingestion, graph tools).
|
|
4
4
|
|
|
5
|
+
**Terminology:** **Catalog object types** (e.g. `assets`, `vulnerabilities`) are declared in `memorix-entity-descriptors`. **Storage targets** (`entity`, `event`, `knowledge`) select Mongo databases — not the same as “entity” in the catalog id. Full glossary and rename plan: **[MEMORIX-OBJECT-TYPES-AND-TARGETS.md](./MEMORIX-OBJECT-TYPES-AND-TARGETS.md)**.
|
|
5
6
|
|
|
6
7
|
| Document | Scope |
|
|
7
8
|
|----------|--------|
|
|
8
9
|
| **This file** | Catalox catalogs, descriptor JSON formats, expected Mongo shape, cross-component behavior |
|
|
10
|
+
| [MEMORIX-OBJECT-TYPES-AND-TARGETS.md](./MEMORIX-OBJECT-TYPES-AND-TARGETS.md) | Object types vs targets; “entity” disambiguation; rename roadmap |
|
|
11
|
+
| [MEMORIX-CATALOX-CATALOG-MAP.md](./MEMORIX-CATALOX-CATALOG-MAP.md) | Per–catalog-id map (Firestore + live inventory) |
|
|
9
12
|
| [MEMORIX-DATABASE-CONVENTIONS.md](./MEMORIX-DATABASE-CONVENTIONS.md) | Database names, collection naming, env vars, record envelope |
|
|
10
13
|
| [DATA-TIER-CONTRACT.md](./DATA-TIER-CONTRACT.md) | What host apps may call (retrieval APIs only) |
|
|
11
14
|
|
|
@@ -67,15 +70,17 @@ If this document and a package disagree, **update both together**.
|
|
|
67
70
|
|
|
68
71
|
All Memorix descriptor catalogs live under this app. Peers must use the same `appId` unless a deployment explicitly namespaces apps. `createMemorixRetrieval`, `createMemorixRetrievalFromEnv`, and `createMemorixRetrievalStackFromEnv` resolve the app id via `resolveMemorixAppId()`.
|
|
69
72
|
|
|
70
|
-
### 2.2 Catalog ids (
|
|
73
|
+
### 2.2 Catalog ids (descriptor trio + control-plane catalogs)
|
|
71
74
|
|
|
72
75
|
| Catalog id | Item id field | Title field | Purpose |
|
|
73
76
|
|------------|---------------|-------------|---------|
|
|
74
|
-
| `memorix-entity-descriptors` | `id` | `entityName` |
|
|
75
|
-
| `memorix-list-descriptors` | `id` | `title` | Tabular list views |
|
|
76
|
-
| `memorix-item-descriptors` | `id` | `title` | Record detail layout |
|
|
77
|
+
| `memorix-entity-descriptors` | `id` | `entityName` | **Object type** schema: properties, content types, relations, `target`, default list/item (catalog id is legacy — holds event/knowledge types too) |
|
|
78
|
+
| `memorix-list-descriptors` | `id` | `title` | Tabular list views (per object type or workspace) |
|
|
79
|
+
| `memorix-item-descriptors` | `id` | `title` | Record detail layout (per object type) |
|
|
77
80
|
|
|
78
|
-
|
|
81
|
+
`@x12i/memorix-descriptors` also uses `memorix-completion-mappings` and `memorix-inventory-policies` (see [MEMORIX-CATALOX-CATALOG-MAP.md](./MEMORIX-CATALOX-CATALOG-MAP.md)).
|
|
82
|
+
|
|
83
|
+
Catalogs use **`sourceMode: "native"`** and **`catalogType: "memorix"`** in seed manifests. Do not introduce parallel catalog names (e.g. `memorix_entity_content_types`, legacy `memorix-entities` inference catalog) for new work.
|
|
79
84
|
|
|
80
85
|
### 2.3 Item shape in Catalox
|
|
81
86
|
|
|
@@ -116,14 +121,17 @@ If the catalog is empty or Catalox is unreachable, discovery returns `source: "n
|
|
|
116
121
|
|
|
117
122
|
Types below mirror `MemorixEntityDescriptor`, `MemorixListDescriptor`, and `MemorixItemDescriptor` in `@x12i/memorix-retrieval`.
|
|
118
123
|
|
|
119
|
-
### 3.1
|
|
124
|
+
### 3.1 Object type descriptor (`memorix-entity-descriptors`)
|
|
125
|
+
|
|
126
|
+
> TypeScript: `MemorixEntityDescriptor`. Field **`entityName`** = **catalog object type** id (not “only for target entity”). See [MEMORIX-OBJECT-TYPES-AND-TARGETS.md](./MEMORIX-OBJECT-TYPES-AND-TARGETS.md).
|
|
120
127
|
|
|
121
128
|
**Required top-level fields:**
|
|
122
129
|
|
|
123
130
|
| Field | Type | Rules |
|
|
124
131
|
|-------|------|-------|
|
|
125
|
-
| `id` | string | Stable catalog item id; usually equals `entityName` |
|
|
126
|
-
| `entityName` | string | Kebab-case
|
|
132
|
+
| `id` | string | Stable catalog item id; usually equals `entityName` (object type) |
|
|
133
|
+
| `entityName` | string | Kebab-case **object type** id (`vulnerabilities`, `variabilities-groups`, `assets`) |
|
|
134
|
+
| `target` | `"entity"` \| `"event"` \| `"knowledge"` | **Storage target** — which Memorix Mongo DB role (default `"entity"`) |
|
|
127
135
|
| `defaultListDescriptorId` | string | **Option A** — id in `memorix-list-descriptors` |
|
|
128
136
|
| `defaultItemDescriptorId` | string | **Option A** — id in `memorix-item-descriptors` |
|
|
129
137
|
| `collectionPrefix` | string | Used with content-type `postfix` when `collection` is omitted |
|
|
@@ -136,9 +144,8 @@ Types below mirror `MemorixEntityDescriptor`, `MemorixListDescriptor`, and `Memo
|
|
|
136
144
|
|
|
137
145
|
| Field | Type | Default |
|
|
138
146
|
|-------|------|---------|
|
|
139
|
-
| `target` | `"entity"` \| `"event"` | `"entity"` — selects `memorix-entities` vs `memorix-events` |
|
|
140
147
|
| `contentDefaults` | object | Default content-object storage/format |
|
|
141
|
-
| `relations` | object | Cross-
|
|
148
|
+
| `relations` | object | Cross–object-type joins (relation `targetEntity` is another object type name) |
|
|
142
149
|
|
|
143
150
|
**Identity block:**
|
|
144
151
|
|