@x12i/memorix-metadata 3.6.1 → 3.6.3

package/CHANGELOG.md

@@ -0,0 +1,70 @@
+## 3.6.3
+
+- Updated `docs/memorix-format-reference.md` (copied from `@x12i/memorix-format` at pack time). It now shows how a content type and its `profile` become the `ContentTypeProfileV2` that record validation uses, and how to validate a record against a pack offline. No validation change.
+
+## 3.6.2
+
+- Ship `docs/memorix-format-reference.md` and `CHANGELOG.md`. Object types, content types, and descriptors were already this package; the reference was not. No validation change.
+
+## 3.6.1
+
+- A discovery rule may set `contentType` and `sourceEndpoint` (`from` or `to`). `sourcePath` is the id inside that content document. Packs that omit both still validate.
+
+## 3.6.0
+
+- A content type may declare `writer`: one caller, or a map of discriminator value to caller (`""` when the content type has no discriminator). A content type may declare `privateMetadata`, the `_metadata` key names only that writer reads back. An empty writer map, a blank caller, or a private key that is not a single name (including `private`) is `META-TYPE`. Packs that omit both still validate.
+
+## 3.5.0
+
+- A content type profile may declare `deletion: { mode, keepBody? }`: `logical`, `tracked`, or `untracked`, with `keepBody` only on `tracked`. An unknown key or mode, a `keepBody` that is not a boolean, or `keepBody` on any other mode is `META-DELETION`. The same content type id on more than one object type must declare the same policy, and no declaration equals `tracked` without the body, so a pack that omits `deletion` still validates. `ContentTypeProfile` gains `deletion`.
+
+## 3.4.2
+
+- A content type may carry `schema`, a JSON Schema map keyed by discriminator value, and an optional `owner`. `validateMetadataPack({ generateStorySchemas: true })` fills a missing story schema from `inputSchemaFor(template)` for each story template. That option stays off until MS-1 is deployed: the current story writer would otherwise receive 400 and write no stories. An explicit schema already on the pack is kept.
+
+## 3.4.1
+
+- An object type may declare `functions`. Each entry has `id`, `question`, `page`, `composes`, `pageStory`, and `reads`, plus optional `template` and `kind` (`spine` or `threads`). Raw metadata and effective metadata return the array as it was installed, and a later install of the same pack keeps it. When `functions` is present, each `story` `presentation.values[].value` has exactly one function with that `template`. A pack that omits `functions` still validates, including one that already has a story presentation. More than one `pageStory` on the same page is kept as published. A content type may also declare `presentation` and `origin`.
+
+## 3.4.0
+
+- `analytics`: `scope` is optional (organization-wide counter). `where` accepts `in: [...]`, `exists: true`, or an array of conditions (all must hold). Groups take `subject` (label) and `split: { key, property, buckets, missing? }`. New `derived[]` (`sum` / `difference` / `ratio` over measures, earlier derived keys, or `group.BUCKET`). `measures[].windows` (`24h` / `7d` / `30d`, count only). `leaders[].countOf: { abstractType, property, where? }`; `sort` is optional when `countOf` is set.
+- Pseudo-property `@datasource` (the record's data source) works wherever a property does. A group over it may omit `buckets`. `ANALYTICS_PSEUDO_PROPERTIES` and `ANALYTICS_RESERVED_BUCKETS` are exported.
+- Buckets `""`, `none`, and `unknown` are reserved; only `split.missing` may use them.
+- Example pack `examples/analytics/review-decisions.pack.json`.
+
+## 3.3.0
+
+- Built-in kind `analytics`: snapshot-scoped rollup declarations (scope pair, target record id, measures, groups, ratios, splits, unwinds, leaders, coverage, deltas, transitions, trends).
+- Optional `enum` on abstract properties and object-type fields. Group/split buckets must be members of that enum.
+
+## 3.2.3
+
+- List and item views may store a remembered fetch: `properties`, `filter`, `sort`, `limit` (abstract property names).
+- Example packs: Nexum 2.0.1 — drop `nexum-*` contribution OTs; persist is `content`/`snapshots` (`proposedGraph`); explore URL `:8080`.
+
+## 3.2.2
+
+- KnowX example pack: Nexum replaced Graphifier (`knowx-nexum` + ingest step id `nexum` + write-record `mode: "nexum"`; drop `knowx-graphify` / `mode: "graphifier"` / write-graph-links / index-graph).
+- OpX example pack: drop leftover snapshot fields `data.graphDraft` / `data.patchPlan` (Nexum persist uses `nexum-*` object types).
+
+## 3.2.1
+
+- OpX example pack: Nexum replaced Graphifier (`nexum` + `opx-nexum-analyze`; drop `memorix-graphifier` / `opx-graphify-analyze`).
+
+## 3.1.0
+
+- `buildEffectiveMetadataTopology` / `EffectiveMetadataTopology`.
+- Required `abstractBindings` on metadata packs.
+- KnowX / OPX example packs: `aiService` skill bindings for extract / associate.
+
+## 3.0.2
+
+- Docs: README install pins + stack/live/core-service references.
+
+# Changelog
+
+## 3.0.1
+
+- Patch release: README/port-map alignment, live-view wiring where applicable.
+

package/README.md

@@ -4,10 +4,12 @@ Declarative **metadata contracts**, **multi-agent effective resolution**, and **
 
 This package is the **law** layer (what agents, types, views, mappings, and pipelines *are*). It does not install packs into Mongo, read subject records, or serve HTTP.
 
+Install it on its own. The only runtime dependency is `@x12i/memorix-mapping`. Record shape and validation are `@x12i/memorix-format`, which has no dependencies. Neither package pulls in the service or the database.
+
 ## Install
 
 ```bash
-npm install @x12i/memorix-metadata@3.0.2
+npm install @x12i/memorix-metadata@3.6.3
 ```
 
 | Audience | Use this package? |
@@ -81,6 +83,6 @@ Abstracts stay **virtual** — no abstract collections and no abstract `recordId
 
 ## Docs
 
-- Practical: [`documentation/metadata.md`](../documentation/metadata.md) · [`documentation/packages/memorix-metadata.md`](../documentation/packages/memorix-metadata.md)
-- Ebook: [Metadata, Agents, Abstracts](../memorix-docs/05-metadata-agents-abstracts/)
+- [`docs/memorix-format-reference.md`](./docs/memorix-format-reference.md) — records, object types, content types, descriptors, and mappings. Section 5 covers object types, content types, and the descriptor; section 5.7 covers validating a record against a pack.
+- [`examples/`](./examples/) — valid packs, and `examples/invalid/` with one failing rule each.
 - Hub: [https://docs.memorix.x12i.com](https://docs.memorix.x12i.com)

package/docs/memorix-format-reference.md

@@ -0,0 +1,673 @@
+# Memorix format reference: records, object types, content types, descriptors, and mappings
+
+This document puts in one place everything that defines how Memorix data is shaped:
+
+1. The **record envelope** (`memorix-record/2`): the shape of every stored document.
+2. The **metadata pack** (`memorix-metadata/1`): where an agent declares its **object types (OTs)**, their **content types (CTs)**, the CT **descriptor** (`profile`), fields, abstracts, and relationships.
+3. **Mappings** (the mapping AST): how provider payloads land on an OT/CT, and how concrete fields project to abstract properties.
+
+Each section names the package and export that is the law for it. When this document and the code disagree, the code wins.
+
+## Using these packages on their own
+
+Memorix depends on these packages. Anyone else can install them without the service, the database layer, or the pipeline runtime.
+
+```bash
+npm install @x12i/memorix-format     # records
+npm install @x12i/memorix-metadata   # object types, content types, descriptors (brings @x12i/memorix-mapping)
+npm install @x12i/memorix-mapping    # mappings only
+```
+
+| Package | What you get | Runtime dependencies |
+|---|---|---|
+| `@x12i/memorix-format` | The `memorix-record/2` envelope, `validateMemorixRecordV2`, `docs/`, and `meta/` | none |
+| `@x12i/memorix-metadata` | Object types, content types, the content-type descriptor (`profile`), abstracts, relationship declarations, and `validateMetadataPack` | `@x12i/memorix-mapping` only |
+| `@x12i/memorix-mapping` | The mapping AST and `executeMapping` | none |
+
+`@x12i/memorix-metadata` installs `@x12i/memorix-mapping` for you. Validating a record or a pack does not require `@x12i/memorix-data`, `@x12i/memorix-service`, or Mongo. Routing and the write path live in `@x12i/memorix-data`; they are listed below so the error codes make sense, and they are not needed to use the format or the descriptors.
+
+This file ships as `docs/memorix-format-reference.md` in all three packages. The copy in `@x12i/memorix-format` is the source; the other two are copied from it when they are packed.
+
+```ts
+import { validateMemorixRecordV2 } from "@x12i/memorix-format/v2";
+import {
+  validateMetadataPack,
+  type ObjectTypeDefinition,
+  type ContentTypeSpec,
+} from "@x12i/memorix-metadata";
+```
+
+| Layer | Package | Exports and files |
+|---|---|---|
+| Record envelope | `@x12i/memorix-format` | `MemorixRecordV2`, `validateMemorixRecordV2` (from `@x12i/memorix-format/v2`); `docs/memorix-record-v2.md` |
+| Format-level content-type profile | `@x12i/memorix-format` | `ContentTypeProfileV2`, `BUILTIN_CONTENT_PROFILES`, `resolveContentProfile` |
+| OT / CT / descriptor / abstracts / relationships / mapping declarations | `@x12i/memorix-metadata` | `MetadataPack`, `ObjectTypeDefinition`, `ContentTypeSpec`, `ContentTypeProfileDecl`, `validateMetadataPack` |
+| Platform interfaces (Entity / Event / Knowledge) | `@x12i/memorix-metadata` | `PLATFORM_INTERFACES` |
+| Mapping AST and execution | `@x12i/memorix-mapping` | `MappingDocument`, `MappingExpr`, `executeMapping`, `validateMappingDocument` |
+| Physical routing and write-time checks | `@x12i/memorix-data` (Memorix runtime; not needed to use the format) | `databaseNameFor`, `contentCollectionName`, the content writer |
+
+---
+
+## 1. Vocabulary
+
+| Term | Meaning |
+|---|---|
+| **Object type (OT)** | The kind of subject, such as `work-items`, `people`, `vulnerabilities`. Declared in `pack.objectTypes[]`. |
+| **Content type (CT)** | One facet of a subject, such as `snapshots`, `raw`, `analysis`, `workflow`. Declared inside an OT as `objectTypes[].contentTypes[]`. The same `recordId` across several CTs is one subject (composition), not a relationship. |
+| **Data category** | `entity`, `event`, or `knowledge`. Set on the OT. Picks the database. It is not a CT. |
+| **Descriptor** | The CT `profile` object. It is mandatory: `id` plus `profile` is the CT declaration. A write to a CT with no descriptor fails with `METADATA_DESCRIPTOR_MISSING`. See section 5.2. |
+| **Discriminator** | An optional path inside `data` (for example `data.template`) whose value splits one CT into variants. Schemas, writers, and tabs can be keyed by discriminator value. |
+| **Abstract type** | A metadata-only type that one or more OTs project into (for example `People`). Never has a collection or its own `recordId`. Views, agent services, and relationships speak abstracts. |
+| **Platform interface** | The sealed abstracts `Entity`, `Event`, `Knowledge`. Every OT implements the one that matches its data category. |
+| **Mapping** | A declarative transform written in the mapping AST. Purpose is `source`, `abstract`, or `completion`. |
+| **Managed descriptor (legacy)** | Older Catalox "entity descriptors" in `memorix-retrieval/catalox-seeds`. Only an inventory of them is kept, in `meta/inventory/managed-descriptors.json` of `@x12i/memorix-format`. They are not format law and not agent packs. |
+
+---
+
+## 2. Physical layout
+
+| Thing | Name | Notes |
+|---|---|---|
+| Database | `{orgId}-memorix-{entity\|event\|knowledge\|operational}` | The OT `dataCategory` picks one of the first three. |
+| Subject content collection | `{objectType}-{contentType}` | For example `work-items-snapshots`. Lives in the OT's category database. |
+| Relationships | `relationships` | Reserved name. One per category database. A CT cannot be named `relationships` (`ROUTE_RESERVED_RELATIONSHIPS`). |
+| Deletion ledger | `deletions` | One per category database. Holds `memorix-deletion/1` entries. No hyphen, so it never collides with `{ot}-{ct}`. |
+| Record directory | `record-directory` | In the `operational` database. |
+
+OT and CT ids must match `^[a-z][a-z0-9-]*$` (lowercase, digits, hyphen, starting with a letter). Otherwise routing fails with `ROUTE_UNSAFE_OBJECT_TYPE` or `ROUTE_UNSAFE_CONTENT_TYPE`.
+
+A write to an OT that is not in the caller's effective metadata fails with `ROUTE_UNKNOWN_OBJECT_TYPE`. A CT that is not declared on that OT fails with `ROUTE_UNKNOWN_CONTENT_TYPE`. Writes never create types; register them with `POST /api/metadata/ensure` first.
+
+---
+
+## 3. The record envelope: `memorix-record/2`
+
+Every document in an `{ot}-{ct}` collection has this shape. Validate with `validateMemorixRecordV2` from `@x12i/memorix-format/v2`, or `npx memorix-format validate-v2 <file|dir>`.
+
+```ts
+type MemorixRecordV2 = {
+  format: "memorix-record/2";
+  recordId: string;
+  revision: number;
+  objectType: string;
+  contentType: string;
+  dataCategory: "entity" | "event" | "knowledge";
+  concept: {
+    title: string;
+    identifiers: {
+      primary: { kind: string; value: string; issuer?: string; normalizedValue?: string };
+      alternates?: Array<{ kind: string; value: string; issuer?: string; normalizedValue?: string }>;
+    };
+  };
+  data: Record<string, unknown>;
+  subject?: { objectType: string; recordId: string };
+  context?: Record<string, unknown>;
+  _metadata?: Record<string, unknown>;
+  _markdown?: { content: string; source: "ai" | "json" | "content"; metadata?: Record<string, unknown> };
+  _audit?: Record<string, unknown>;
+  _history?: Array<Record<string, unknown>>;
+  analytics?: Record<string, unknown>;
+  enrichment?: Record<string, unknown>;
+  insights?: Record<string, unknown>;
+  // associated<Name>?: Array<{ relationshipDefinitionId: string; recordId: string; relationshipId?: string }>;
+  _system: {
+    state: "active" | "archived" | "superseded" | "deleted";
+    createdAt: string;
+    modifiedAt: string;
+    observedAt?: string;
+    writtenAt?: string;
+    lifecycle?: Array<{ process: string; processedAt: string }>;
+    provenance?: Record<string, unknown>;
+    _datasource?: {
+      service: string; ingress: string; basis: string; recordedAt: string;
+      origin?: string; channel?: string; originRecordId?: string; originAt?: string;
+      sources?: Array<{ system: string; name: string }>;
+    };
+    deletion?: { deletedAt: string; by: string; via: "content" | "patch"; patchId?: string; reason?: string };
+  };
+};
+```
+
+### 3.1 Envelope properties
+
+| Property | Required | Meaning and rules |
+|---|---|---|
+| `format` | yes | Always `"memorix-record/2"`. |
+| `recordId` | yes | Subject identity. Shared by every CT of the same subject. Never the Mongo `_id`. Source identifiers are adopted as `{issuer}/{kind}/{value}`, never a bare provider id. |
+| `revision` | yes | Starts at 1 and increments on every successful mutation. A stale `expectedRevision` is rejected. |
+| `objectType` | yes | Must be a declared OT. Must match the collection. |
+| `contentType` | yes | Must be a CT declared on that OT. |
+| `dataCategory` | yes | Must equal the OT's `dataCategory`. |
+| `concept.title` | yes | Human-readable title. Default source for the platform interface `title` property. |
+| `concept.identifiers.primary` | yes | `{ kind, value }`, optional `issuer` and `normalizedValue`. |
+| `concept.identifiers.alternates` | no | Omit when empty. |
+| `data` | yes | The subject content itself. JSON markdown is rendered from `data` only. |
+| `subject` | no | `{ objectType, recordId }` of the subject this row is about, when that subject is a different OT. The row keeps its own `recordId`. List filters use `subject.recordId`. |
+| `context` | no | Context payloads that are not the subject content. Omit when empty. Not rendered to markdown. |
+| `_metadata` | no | Producer context (for example `stepId`, `writtenBy`). A key here must not also be in `data`. Not `_system.provenance` and not `_markdown.metadata`. |
+| `_markdown` | no | Original text beside `data`. `source` is `ai`, `json`, or `content`. Writers fill `source: "json"` from `data` when it is omitted. Readers omit it unless the fetch asks for markdown. |
+| `_audit` | no | Audit of the record. When present, `provider`, `model`, and `reasoningEffort` must be non-empty strings. Other keys are open. |
+| `_history` | no | Earlier products of this record, oldest first. Each entry is an object stored as given. An empty array is valid and differs from an absent key. A write that removes fields appends `{ action: "patch", caller, modifiedAt, removed: string[] }`. |
+| `analytics` | no | Numeric rollups. Allowed only when the CT descriptor has `allowSnapshotBuckets: true`. |
+| `enrichment` | no | External or prior-system values as received. Same descriptor rule. |
+| `insights` | no | Judgments, scores, recommendations. Same descriptor rule. |
+| `associated<Name>` | no | Lightweight relationship refs only (`relationshipDefinitionId` + `recordId`, optional `relationshipId`). Copied related payloads are rejected. Authoritative relationships live in the `relationships` collection, not here. |
+| `_system.state` | yes | `active`, `archived`, `superseded`, or `deleted`. |
+| `_system.createdAt` / `modifiedAt` | yes | ISO timestamps. |
+| `_system.observedAt` | no | Default source for the `Event.occurredAt` interface property. |
+| `_system.writtenAt` | no | Clock time of the write that produced this revision. |
+| `_system._datasource` | no | Memorix-owned document source (`service`, `ingress`, `basis`, `recordedAt`, optional `origin`, `channel`, `sources`). Not part of `data`. |
+| `_system.lifecycle` | no | Appended only for meaningful process events. Omit when empty. Independent of `revision`. |
+| `_system.provenance` | no | Source attribution such as `sourceId` or `endpoint`. |
+| `_system.deletion` | conditional | Present if and only if `state` is `deleted` (`V2-DELETION`). |
+
+### 3.2 Envelope rules that come from the CT descriptor
+
+- Empty optional buckets are omitted. Never write `{}` or `[]`.
+- `analytics`, `enrichment`, and `insights` appear only on CTs whose descriptor sets `allowSnapshotBuckets: true`.
+- If the CT declares a discriminator with a `defaultValue` and the record lacks it, validation warns `V2-DISCRIMINATOR-DEFAULT` and the writer fills the default. If it is required with no default, validation fails with `V2-DISCRIMINATOR`.
+- What a delete does is declared on the CT (`profile.deletion`), see section 5.2.
+- Section 5.7 shows how the metadata descriptor becomes the profile that `validateMemorixRecordV2` checks.
+
+Golden examples live in `examples/v2/` of `@x12i/memorix-format`.
+
+---
+
+## 4. The metadata pack: `memorix-metadata/1`
+
+An agent's whole catalog is one pack. `POST /api/metadata/ensure` installs the pack you send and **replaces** the agent's catalog, so anything omitted becomes unknown. Always merge from `GET /api/metadata/raw?agentId=…`.
+
+```ts
+type MetadataPack = {
+  format: "memorix-metadata/1";
+  agentId: string;
+  version?: string;
+  agents: AgentDefinition[];
+  objectTypes: ObjectTypeDefinition[];          // section 5
+  abstractTypes: AbstractTypeDefinition[];      // section 6
+  relationshipDefinitions: RelationshipDefinition[]; // section 7
+  mappings: MappingDefinition[];                // section 8
+  views: ViewDefinition[];
+  writes: WriteDefinition[];
+  sources: SourceDefinition[];
+  services: ServiceDefinition[];
+  pipelines: PipelineDefinition[];
+  schedules: ScheduleDefinition[];
+  analytics?: AnalyticsDefinition[];
+  kindDeclarations?: KindDeclarationDefinition[];
+  extensions?: Record<string, Array<{ id: string } & Record<string, unknown>>>;
+};
+```
+
+Validate in-process with `validateMetadataPack(pack)` from `@x12i/memorix-metadata`. Issues come back as `{ path, code, message }` with codes such as `META-REQUIRED`, `META-CT-PROFILE`, `META-LEADING`, `META-REF-CONTENT`.
+
+This document covers `objectTypes`, `abstractTypes`, `relationshipDefinitions`, and `mappings`. Views, sources, services, pipelines, schedules, and analytics are typed in the same `types.ts`.
+
+---
+
+## 5. Object types and content types
+
+### 5.1 Object type (`ObjectTypeDefinition`)
+
+| Property | Required | Type | Meaning and rules |
+|---|---|---|---|
+| `id` | yes | string | OT id. Must be routing-safe (`^[a-z][a-z0-9-]*$`). Must not be `Entity`, `Event`, or `Knowledge` (`META-PLATFORM-RESERVED`). |
+| `name` | yes | string | Display name. |
+| `dataCategory` | yes | `entity` \| `event` \| `knowledge` | Picks the database. |
+| `implements` | no | string[] | Interfaces this OT implements. Must include the platform interface for its category (`entity` → `Entity`, and so on) and no other platform interface (`META-INTERFACE-CATEGORY`). Omitted means the category interface. |
+| `interfaceBindings` | no | `ObjectTypeInterfaceBinding[]` | Overrides the default path of an interface property. Each entry: `interfaceId`, `property`, `propertyPath`, optional `contentType` (defaults to the first CT that is not `relationships`). |
+| `abstractTypes` | no | string[] | Abstract types this OT participates in. |
+| `contentTypes` | yes | `ContentTypeSpec[]` | Non-empty. Ids unique within the OT. At most one may set `leading: true` (`META-LEADING`). See 5.2. |
+| `fields` | no | `ObjectTypeFieldSpec[]` | Declared concrete fields. See 5.3. |
+| `functions` | no | `ObjectTypeFunction[]` | Functions the skill API publishes on this OT. See 5.4. |
+
+### 5.2 Content type (`ContentTypeSpec`) and its descriptor
+
+| Property | Required | Type | Meaning and rules |
+|---|---|---|---|
+| `id` | yes | string | CT id. Routing-safe. Must not be `relationships`. |
+| `profile` | yes | `ContentTypeProfileDecl` | **The descriptor.** See below. |
+| `required` | no | boolean | Subjects of this OT should have this CT. When `true`, `requiredReason` is required. |
+| `requiredReason` | conditional | string | Why the CT is required. |
+| `notes` | no | string | Free text, for example who writes it. |
+| `retention` | no | string | Retention note. |
+| `leading` | no | boolean | Default page driver for the OT. At most one per OT. With none set, clients use the first CT that is not `relationships`. |
+| `discriminator` | no | `{ path, required?, defaultValue? }` | `path` is required when `discriminator` is set. `defaultValue` must be a non-empty string. |
+| `schema` | no | `Record<discriminatorValue, JSONSchema>` | JSON Schema for `data`, keyed by discriminator value. A failing write returns 422 `CONTENT_INVALID`. Omitted means only the discriminator is checked. |
+| `owner` | no | string | When set, a later write is refused with 409 `ADDRESS_OWNED` if the stored row's owner (`_metadata.stepId`, else `_metadata.writtenBy`) is different. |
+| `writer` | no | string \| `Record<discriminatorValue, caller>` | Which `x-memorix-caller` may write. A string owns every value. A map owns one value each (`""` stands for the CT when it has no discriminator). A different caller gets 409 `WRITER_NOT_OWNER`. |
+| `privateMetadata` | no | string[] | `_metadata` keys returned only to the caller that wrote the row. Non-empty; key names match `^[A-Za-z0-9_-]+$` and must not be `private`. |
+| `fieldMappings` | no | `FieldMappingProfile[]` | Assistant-published field presentation. Each: `id`, `format: "memorix-field-map/1"`, `namespace`, `fieldMap` (object), optional `discriminatorValue`, `standingRules`, `requiresCapabilities`, `provenance`. |
+| `presentation` | no | `{ label?, order?, tab?, values? }` | Client tab. `tab` is `always`, `whenPresent`, or `hidden`. `values[]` are `{ value, label?, order?, origin? }` where `value` matches `data` at the discriminator path. |
+| `origin` | no | `{ producer?, use?, record? }` | `producer` is `model`, `code`, or `mixed`. `use` is `always` or `conditional`. |
+
+**The descriptor (`profile`)** is metadata only; it is never stored on the record.
+
+```ts
+type ContentTypeProfileDecl = {
+  allowSnapshotBuckets: boolean;     // required
+  deletion?: {
+    mode: "logical" | "tracked" | "untracked";
+    keepBody?: boolean;              // tracked only, default false
+  };
+};
+```
+
+| Descriptor key | Rules |
+|---|---|
+| `allowSnapshotBuckets` | Required boolean. When `true`, records of this CT may carry `analytics`, `enrichment`, `insights`. The **same CT id must declare the same value on every OT** that uses it. |
+| `deletion.mode` | `logical`: the row stays with `_system.state: "deleted"` and reads hide it. `tracked`: the row is removed and a `memorix-deletion/1` entry is written to `deletions`. `untracked`: the row is removed and nothing is recorded. Omitted `deletion` means `{ mode: "tracked", keepBody: false }`. |
+| `deletion.keepBody` | Only valid with `tracked`. Keeps the removed body on the ledger entry. Purges and legacy-instance deletions never keep the body. A CT declares the same policy on every OT that uses it. |
+
+Write-time enforcement: if the effective catalog has no descriptor for the CT, the writer throws `METADATA_DESCRIPTOR_MISSING` (HTTP 400, with a `next` hint pointing at `/api/metadata/ensure`).
+
+### 5.3 Fields (`ObjectTypeFieldSpec`)
+
+Optional declared concrete fields. Used for metadata-only path and type checks, abstract binding validation, and write-time enum checks.
+
+| Property | Required | Meaning |
+|---|---|---|
+| `path` | yes | Property path inside the content document, for example `data.status`. |
+| `valueType` | yes | For example `string`, `number`, `datetime`, `recordId`. |
+| `contentType` | no | Which CT the field lives on. Must be declared on the same OT (`META-REF-CONTENT`). Omitted means all CTs of the OT. |
+| `enum` | no | Closed vocabulary. Writers reject any other string, including `""` (`WRITE_ENUM_INVALID`). A missing field stays absent; it is never coerced to `""`. |
+| `refersTo` | conditional | `{ objectType }`. Required when the field holds a `recordId` or an array of them. |
+
+### 5.4 Functions (`ObjectTypeFunction`)
+
+Published by the skill API and run with `POST /api/v1/agent-functions/agents/{agentId}/functions/{id}/run`.
+
+| Property | Required | Meaning |
+|---|---|---|
+| `id` | yes | Function id. |
+| `question` | yes | The question the function answers. |
+| `page` | yes | Page the function renders into. |
+| `composes` / `pageStory` | yes | Booleans. |
+| `reads` | yes | `{ objectType?, contentType?, recordId?, template?, sameSubject? }[]`: content loaded before the run. |
+| `template` | no | Matches `presentation.values[].value` and `data.template` on a story row. |
+| `kind` | no | `spine` or `threads`. |
+
+### 5.5 Content-type vocabulary in use
+
+Any routing-safe id may be declared. These are the CTs the platform and current packs use (`meta/content-types.json` in `@x12i/memorix-format`):
+
+| CT | Typical descriptor | Purpose |
+|---|---|---|
+| `snapshots` | `allowSnapshotBuckets: true` | Full readable view of the subject. The canonical CT. |
+| `raw` | `false` | Default landing CT for pulled provider payloads. |
+| `context` | `false` | Context fetched on demand. |
+| `analysis`, `decisions`, `discovery`, `scoped`, `legacy` | `false` | Skill and judgment products. Some older collections use the singular postfix `decision`. |
+| `narratives`, `narrative-steps` | `false` | Narrative records. |
+| `workflow` | `false` | Product workflow overlay (`memorix-workflow/1`). A sibling CT, not a separate document system. |
+| `extrated`, `chunks` | `false` | Document extraction and chunks. `extrated` is spelled as it is in the managed descriptors. |
+| `knowx`, `nexum`, `confirmations` | `false` | KnowX facets, Nexum contributions, accept or reject decisions. |
+| `ai-processing`, `suggested-plans`, `suggested-actions` | `false` | Optional product CTs. |
+
+### 5.6 Who declares what
+
+The lander declares OTs and their source CTs. A skill appends only its target CTs to an OT that already exists, and never re-declares source CTs. Because ensure replaces the catalog, a skill always merges into the full pack from `/api/metadata/raw` before it ensures.
+
+### 5.7 From metadata descriptor to format profile
+
+`@x12i/memorix-format` does not read metadata packs. `validateMemorixRecordV2` takes a `ContentTypeProfileV2`, the format-level descriptor:
+
+```ts
+type ContentTypeProfileV2 = {
+  contentType: string;
+  allowSnapshotBuckets: boolean;
+  discriminator?: { path: string; required: boolean; defaultValue?: string; allowedValues?: string[] };
+  allowedExtraRoots?: string[];
+  deletion?: { mode: "logical" | "tracked" | "untracked"; keepBody?: boolean };
+};
+```
+
+Memorix builds it from the metadata content type like this:
+
+| `ContentTypeProfileV2` | Comes from (`ContentTypeSpec`) |
+|---|---|
+| `contentType` | `id` |
+| `allowSnapshotBuckets` | `profile.allowSnapshotBuckets` |
+| `deletion` | `profile.deletion`, parsed with `parseDeletionPolicy`. An invalid declaration reads as absent. |
+| `discriminator.path` | `discriminator.path`. With no path, there is no discriminator. |
+| `discriminator.required` | `discriminator.required === true` (default `false`) |
+| `discriminator.defaultValue` | `discriminator.defaultValue` when it is a non-empty string |
+| `allowedValues`, `allowedExtraRoots` | Not set from metadata. Available to callers that validate offline. |
+
+The lookup is by content-type id across the caller's effective object types, and the first match wins. That is why one CT id must declare the same `allowSnapshotBuckets` and `deletion` on every OT (`META-CT-PROFILE`).
+
+When no profile is passed, `validateMemorixRecordV2` falls back to `resolveContentProfile(contentType)`, which uses `BUILTIN_CONTENT_PROFILES`:
+
+| CT | `allowSnapshotBuckets` | Built-in discriminator |
+|---|---|---|
+| `snapshots`, `knowledge` | `true` | none |
+| `raw`, `workflow`, `context`, `relationships`, `events` | `false` | none |
+| `discovery` | `false` | `data.discoveryType`, default `default` |
+| `analysis` | `false` | `data.analysisType`, default `default` |
+| `decisions` | `false` | `data.decisionType`, default `default` |
+| any other id | `false` | none |
+
+A pack's descriptor always replaces the built-in profile, including its discriminator.
+
+To validate a record offline against a pack, without Memorix:
+
+```ts
+import {
+  validateMemorixRecordV2,
+  parseDeletionPolicy,
+  type ContentTypeProfileV2,
+} from "@x12i/memorix-format/v2";
+import type { ContentTypeSpec } from "@x12i/memorix-metadata";
+
+function toProfile(ct: ContentTypeSpec): ContentTypeProfileV2 {
+  const deletion = ct.profile.deletion ? parseDeletionPolicy(ct.profile.deletion) : undefined;
+  return {
+    contentType: ct.id,
+    allowSnapshotBuckets: ct.profile.allowSnapshotBuckets,
+    ...(deletion?.ok ? { deletion: deletion.policy } : {}),
+    ...(ct.discriminator?.path
+      ? {
+          discriminator: {
+            path: ct.discriminator.path,
+            required: ct.discriminator.required === true,
+            ...(ct.discriminator.defaultValue ? { defaultValue: ct.discriminator.defaultValue } : {}),
+          },
+        }
+      : {}),
+  };
+}
+
+const ct = pack.objectTypes.find((o) => o.id === record.objectType)!
+  .contentTypes.find((c) => c.id === record.contentType)!;
+const result = validateMemorixRecordV2(record, { profile: toProfile(ct), enforceDiscriminator: true });
+```
+
+---
+
+## 6. Abstract types and platform interfaces
+
+### 6.1 Platform interfaces
+
+Injected into every effective-metadata result. Packs cannot redefine them.
+
+| Interface | Category | Properties (id → default path) |
+|---|---|---|
+| `Entity` | `entity` | `title` (string, required) → `concept.title` |
+| `Event` | `event` | `title` (string, required) → `concept.title`; `occurredAt` (datetime, required) → `_system.observedAt` |
+| `Knowledge` | `knowledge` | `title` (string, required) → `concept.title` |
+
+Override a default path with `interfaceBindings` on the OT.
+
+### 6.2 Abstract type (`AbstractTypeDefinition`)
+
+Abstracts exist only in metadata. Declaring a collection or an abstract `recordId` is rejected (`META-ABS-RECORD`).
+
+| Property | Required | Meaning |
+|---|---|---|
+| `id`, `name`, `meaning`, `category` | yes | Identity and description. |
+| `properties` | no | `AbstractPropertyDefinition[]`: `{ id, valueType, required?, name?, description?, enum? }`. `valueType` is one of `string`, `number`, `boolean`, `date`, `datetime`, `enum`, `object`, `array`, `content`, `recordId`, `unknown`. |
+| `members` | no | `AbstractMemberBinding[]`: which OTs participate and how each fills the properties. |
+| `union` | no | `true` when more than one OT participates. |
+| `platformOwned` | no | Only on the injected platform interfaces. Packs must omit it. |
+| `notes` | no | Free text. |
+
+### 6.3 Member and property bindings (OT → abstract mapping)
+
+```ts
+type AbstractMemberBinding = {
+  objectType: string;
+  contentType?: string;               // default CT for this member's properties
+  properties?: AbstractPropertyBinding[];
+};
+
+type AbstractPropertyBinding = {
+  abstractProperty: string;
+  contentType: string;
+  propertyPath?: string;              // direct binding, e.g. "data.displayName"
+  concreteValueType?: string;         // for direct type-compatibility checks
+  mappingId?: string;                 // converted binding: a mapping with purpose "abstract"
+  reverse?: { propertyPath?: string; mappingId?: string }; // write-back; absent = read-only
+};
+```
+
+Each property binding must set `propertyPath`, `mappingId`, or both (`META-BIND-PROP`). A `reverse` block must also name at least one of them (`META-BIND-REVERSE`). Without `reverse`, the abstract property cannot be written through.
+
+The legacy separate `abstractBindings[]` kind is folded into `abstractTypes[].members` on load and is always empty after normalization.
+
+---
+
+## 7. Relationship definitions
+
+Relationships are declared on abstracts, never on concrete collections. Storage is the central `relationships` collection; declaring storage on the definition is rejected (`META-REL-STORAGE`).
+
+| Property | Required | Meaning |
+|---|---|---|
+| `id`, `name`, `meaning` | yes | Identity. |
+| `directionality` | yes | `directed` or `undirected`. |
+| `from`, `to` | yes | `{ abstractType, path }`. `path` is the directional navigation path on that abstract (`META-REL-PATH`). |
+| `discoveryRules` | yes | At least one (`META-REL-RULES`). Each: `{ id, kind, sourcePath?, contentType?, sourceEndpoint?: "from" \| "to", notes? }`. `sourcePath` is inside the `data` of `contentType`. Without `contentType` the rule scans `raw` then `snapshots`. |
+
+Concrete OT-to-OT edges are derived, not authored: see `buildEffectiveMetadataTopology` and `GET /api/metadata/topology`.
+
+---
+
+## 8. Mappings
+
+### 8.1 Mapping declaration (`MappingDefinition`)
+
+| Property | Required | Meaning |
+|---|---|---|
+| `id` | yes | Mapping id. |
+| `purpose` | yes | `source`, `abstract`, or `completion` (`META-MAP-PURPOSE`). |
+| `objectType`, `contentType` | yes | The OT/CT the mapping lands on or reads from. |
+| `sourceId` | for `source` | The `sources[].id` the payload comes from. |
+| `endpoint` | no | Provider endpoint, for example `/work-items/:id/context`. |
+| `abstractType` | for `abstract` | The abstract whose properties the map produces. |
+| `mode`, `trigger` | no | For example `mode: "knowledge"` or `"context"`, `trigger: "on-demand"`. |
+| `map` | yes | Output path → expression. See 8.2. |
+| `reverse` | no | `Record<outputProperty, { expr, targetPath }>`. Explicit write-back for abstract outputs. Never inferred from the forward map. |
+
+Execution and schedule policy keys are rejected on mappings (`META-MAP-EXEC`). Completion policy belongs on the pipeline step that references the mapping.
+
+What the `map` keys mean depends on the purpose:
+
+| Purpose | Input (`path` reads from) | Output keys |
+|---|---|---|
+| `source` | The provider payload. `$` is the whole payload. | Record paths, such as `recordId`, `data.displayName`, or `data` (whole payload into `data`). |
+| `abstract` | The stored record, such as `data.status` or `concept.title`. | Abstract property names, such as `title`, `status`. |
+| `completion` | The stored record. | Record paths to fill. |
+
+### 8.2 Mapping AST
+
+Each `map` value is either a bare expression or `{ "expr": <expression>, "required": true | false }`. Missing optional outputs are omitted; a missing required output is an error.
+
+| Operation | Shape | Behavior |
+|---|---|---|
+| `path` | `{ op: "path", path }` | Read a value by dotted path. `$` is the input root; inside `array.*` it is the current element. |
+| `constant` | `{ op: "constant", value }` | A literal. Never a secret. |
+| `cast` | `{ op: "cast", expr, to: "string" \| "number" \| "boolean" }` | Type conversion. |
+| `date` | `{ op: "date", expr, acceptedFormats: [...] }` | Parse with explicit formats, for example `["iso"]`. |
+| `enum` | `{ op: "enum", expr, map: { source: target }, default? }` | Value translation. `default` applies when the input is present but not in `map`. |
+| `coalesce` | `{ op: "coalesce", exprs: [...] }` | First non-missing value. |
+| `conditional` | `{ op: "conditional", when, then, else? }` | Branch. |
+| `template` | `{ op: "template", template, vars: { name: expr }, requiredVars? }` | String template. `requiredVars` defaults to every var. |
+| `array.map` | `{ op: "array.map", expr, item }` | Map each element (`$` is the element). |
+| `array.filter` | `{ op: "array.filter", expr, predicate }` | Keep elements whose predicate is boolean `true`. |
+| `transform` | `{ op: "transform", name, version, args? }` | A registered transform, for example `lowercase@1`. See `createDefaultRegistry`. |
+
+Limits: depth 32, 512 nodes, 10,000 array items (`DEFAULT_MAPPING_LIMITS`).
+
+Run with `executeMapping`, `simulateMapping` (adds coverage), and `executeReverseMapping` (returns an update plan; the mapping never writes).
+
+---
+
+## 9. Worked example
+
+A pack fragment with one OT, two CTs, an abstract with a direct binding, a source mapping, and an abstract mapping. A complete real pack is `examples/opx/normalized/pack.json` in `@x12i/memorix-metadata`.
+
+```json
+{
+  "format": "memorix-metadata/1",
+  "agentId": "opx",
+  "agents": [{ "id": "opx", "name": "OPX" }],
+  "objectTypes": [
+    {
+      "id": "work-items",
+      "name": "Work item",
+      "dataCategory": "entity",
+      "abstractTypes": ["WorkItem"],
+      "contentTypes": [
+        {
+          "id": "snapshots",
+          "profile": { "allowSnapshotBuckets": true },
+          "leading": true
+        },
+        {
+          "id": "raw",
+          "profile": { "allowSnapshotBuckets": false, "deletion": { "mode": "untracked" } }
+        },
+        {
+          "id": "analysis",
+          "profile": { "allowSnapshotBuckets": false },
+          "discriminator": { "path": "data.template", "required": true },
+          "writer": "memorix-skill-analysis",
+          "presentation": {
+            "label": "Analysis",
+            "tab": "whenPresent",
+            "values": [{ "value": "risk", "label": "Risk" }]
+          }
+        }
+      ],
+      "fields": [
+        { "path": "data.status", "valueType": "string", "contentType": "snapshots",
+          "enum": ["open", "in-progress", "done"] },
+        { "path": "data.ownerId", "valueType": "recordId", "contentType": "snapshots",
+          "refersTo": { "objectType": "people" } }
+      ]
+    }
+  ],
+  "abstractTypes": [
+    {
+      "id": "WorkItem",
+      "name": "Work item",
+      "meaning": "A unit of work tracked by any connected system",
+      "category": "entity",
+      "properties": [
+        { "id": "title", "valueType": "string", "required": true },
+        { "id": "status", "valueType": "enum", "enum": ["open", "in-progress", "done"] }
+      ],
+      "members": [
+        {
+          "objectType": "work-items",
+          "contentType": "snapshots",
+          "properties": [
+            { "abstractProperty": "title", "contentType": "snapshots", "propertyPath": "concept.title" },
+            { "abstractProperty": "status", "contentType": "snapshots", "propertyPath": "data.status",
+              "reverse": { "propertyPath": "data.status" } }
+          ]
+        }
+      ]
+    }
+  ],
+  "mappings": [
+    {
+      "id": "opx-source-work-items-to-raw",
+      "purpose": "source",
+      "sourceId": "opx-source",
+      "endpoint": "/work-items",
+      "objectType": "work-items",
+      "contentType": "raw",
+      "mode": "knowledge",
+      "map": { "data": { "op": "path", "path": "$" } }
+    },
+    {
+      "id": "opx-work-items-abstract",
+      "purpose": "abstract",
+      "abstractType": "WorkItem",
+      "objectType": "work-items",
+      "contentType": "snapshots",
+      "map": {
+        "title": { "expr": { "op": "path", "path": "concept.title" }, "required": true },
+        "status": {
+          "op": "enum",
+          "expr": { "op": "path", "path": "data.state" },
+          "map": { "notStarted": "open", "active": "in-progress", "closed": "done" }
+        }
+      }
+    }
+  ],
+  "relationshipDefinitions": [],
+  "views": [], "writes": [], "sources": [], "services": [], "pipelines": [], "schedules": []
+}
+```
+
+A stored record for that OT/CT, in collection `work-items-snapshots` of database `{orgId}-memorix-entity`:
+
+```json
+{
+  "format": "memorix-record/2",
+  "recordId": "opx-source/workItemId/WI-100",
+  "revision": 3,
+  "objectType": "work-items",
+  "contentType": "snapshots",
+  "dataCategory": "entity",
+  "concept": {
+    "title": "Prepare quarterly operations review",
+    "identifiers": { "primary": { "kind": "workItemId", "value": "WI-100", "issuer": "opx-source" } }
+  },
+  "data": { "status": "in-progress", "ownerId": "hr-source/employeeId/E-7" },
+  "insights": { "riskScore": 0.42 },
+  "_system": {
+    "state": "active",
+    "createdAt": "2026-01-10T10:00:00.000Z",
+    "modifiedAt": "2026-01-15T12:00:00.000Z"
+  }
+}
+```
+
+`insights` is allowed here only because the `snapshots` descriptor sets `allowSnapshotBuckets: true`. The same bucket on an `analysis` row would be rejected.
+
+---
+
+## 10. Error codes you will meet
+
+| Code | Where | Cause |
+|---|---|---|
+| `ROUTE_UNKNOWN_OBJECT_TYPE` | write / read | OT not in the caller's effective metadata. |
+| `ROUTE_UNKNOWN_CONTENT_TYPE` | write / read | CT not declared on that OT. |
+| `ROUTE_UNSAFE_OBJECT_TYPE` / `ROUTE_UNSAFE_CONTENT_TYPE` | routing | Id fails `^[a-z][a-z0-9-]*$`. |
+| `ROUTE_RESERVED_RELATIONSHIPS` | routing | CT named `relationships`. |
+| `ROUTE_CATEGORY_MISMATCH` | routing | OT category is not `entity`, `event`, or `knowledge`, or differs from what was expected. |
+| `METADATA_DESCRIPTOR_MISSING` | write (400) | CT has no `profile` in the effective catalog. Ensure the pack first. |
+| `CONTENT_INVALID` | write (422) | `data` fails the CT `schema` for its discriminator value. |
+| `WRITE_ENUM_INVALID` | write | Value outside a declared field `enum`. |
+| `ADDRESS_OWNED` / `WRITER_NOT_OWNER` | write (409) | CT `owner` or `writer` check failed. |
+| `V2-DISCRIMINATOR` / `V2-DISCRIMINATOR-DEFAULT` | record validation | Required discriminator missing / default applied. |
+| `V2-DELETION` | record validation | `_system.deletion` present without `state: "deleted"`, or the reverse. |
+| `META-REQUIRED` | pack validation | Descriptor (`profile`) or `allowSnapshotBuckets` missing, or another required key missing. |
+| `META-CT-PROFILE` | pack validation | The same CT id declares `allowSnapshotBuckets` or `profile.deletion` differently on two OTs. |
+| `META-LEADING` | pack validation | More than one `leading` CT on an OT. |
+| `META-INTERFACE-CATEGORY` | pack validation | `implements` does not match `dataCategory`. |
+| `META-BIND-PROP` / `META-BIND-REVERSE` | pack validation | Abstract property binding with no `propertyPath` or `mappingId`. |
+| `META-MAP-PURPOSE` / `META-MAP-EXEC` | pack validation | Bad mapping purpose, or execution policy on a mapping. |
+
+---
+
+## Related documents
+
+In `@x12i/memorix-format`:
+
+- `docs/memorix-record-v2.md`: the full envelope contract and its locked decisions.
+- `docs/migrate-present-to-v2.md`: migrating the older present envelope to `memorix-record/2`.
+- `docs/memorix-workflow-format.md`: the `workflow` content type.
+- `docs/memorix-format.md`, `docs/memorix-format-requirements.md`, `docs/association-format.md`: the older present envelope. Read them only to migrate existing data.
+- `meta/`: envelope, open-bucket, and content-type vocabulary as JSON.
+- `examples/v2/`: valid `memorix-record/2` documents.
+
+In `@x12i/memorix-metadata`:
+
+- `examples/opx/`, `examples/knowx/`, `examples/analytics/`, `examples/custom-kinds/`: valid packs.
+- `examples/invalid/`: packs that fail validation, one rule each.
+
+In `@x12i/memorix-mapping`:
+
+- `examples/`: source, abstract, and completion mappings.

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@x12i/memorix-metadata",
-  "version": "3.6.1",
-  "description": "Canonical Memorix metadata contracts, effective multi-agent resolution, and metadata-only abstract unions (Phase 3).",
+  "version": "3.6.3",
+  "description": "Memorix object-type and content-type descriptors, metadata pack contracts, and validation. Depends only on @x12i/memorix-mapping.",
   "type": "module",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
@@ -15,14 +15,16 @@
     "dist",
     "examples",
     "docs",
-    "README.md"
+    "README.md",
+    "CHANGELOG.md"
   ],
   "scripts": {
     "prebuild": "rm -rf dist",
     "build": "tsc -p tsconfig.build.json",
     "test": "vitest run",
     "test:watch": "vitest",
-    "typecheck": "tsc --noEmit"
+    "typecheck": "tsc --noEmit",
+    "prepack": "node ../scripts/sync-format-reference.mjs && npm run build"
   },
   "engines": {
     "node": ">=18.0.0"