@clivly/core 0.9.0 → 0.9.2

package/README.md

@@ -14,6 +14,9 @@ the view compiler that turns a mapping into SQL. Its only runtime dependency is
 > to import it from. Reach for `@clivly/core` when you are building an adapter
 > or working with the entity config outside an SDK instance.
 
+Connection and presence transport live in `@clivly/sdk`, not this package.
+That includes the opt-in WebSocket presence transport and its HTTP fallback.
+
 ## Prerequisites
 
 | Requirement | Version / note |
@@ -37,12 +40,12 @@ peer), `/entity-heuristics`, `/mapping-score`, `/integration-state` or
 
 | Entry | Contents |
 | --- | --- |
-| `@clivly/core` | The re-exported surface: contracts, entity config, mapping form, sync engine, view compiler, shared types |
+| `@clivly/core` | The re-exported surface: contracts, entity config, mapping form, sync engine, view compiler, projections, shared types |
 | `@clivly/core/adapter` | `CRMAdapter` — the interface every ORM implementation fulfils (org-scoped; contacts, companies, deals, tasks, activities) |
 | `@clivly/core/auth-adapter` | `ClivlyAuthAdapter`, `ClivlyUser`, `authCustom()`, `MembershipResolver`, `CrmMembership` — pluggable host-app identity + the `crm_members` authorization layer |
 | `@clivly/core/entity-config` | `defineClivlyConfig`, `validateEntitiesConfig`, `entitiesConfigSchema`, `CANONICAL_FIELDS`, `CRM_CONCEPTS`, `ClivlyConfigError`, and the config types |
 | `@clivly/core/types` | Shared domain types (`Contact`, `Company`, `Deal`, `Task`, `Note`, `Activity`, inputs, filters, `CrmRole`) |
-| `@clivly/core/drizzle` | `discoverFromDrizzle(schema)` — derive the discovery shape from a Drizzle schema object. Requires the optional `drizzle-orm` peer |
+| `@clivly/core/drizzle` | `discoverFromDrizzle(schema)` — derive the discovery shape (including keys, types, nullability and FK edges) from a Drizzle schema object; plus `buildUniqueConstraints` / `buildColumnsMeta` for reflecting a single table. Requires the optional `drizzle-orm` peer |
 | `@clivly/core/view-compiler` | `compileEntityView`, `compileEntityViews`, `explainEntitiesConfig`, `DEFAULT_VIEW_PREFIX` |
 | `@clivly/core/mappings-config` | `mappingsToEntitiesConfig`, `EntityMappingRow` — turn `crm_entity_mappings` rows into an entities config |
 | `@clivly/core/mapping-form` | `buildFilterFromDiscriminator`, `buildRelationships`, `relationshipViaSignature` — mapping-UI helpers |
@@ -64,12 +67,30 @@ import { discoverFromDrizzle } from "@clivly/core/drizzle";
 import * as schema from "./db/schema";
 
 const tables = discoverFromDrizzle(schema);
-// → [{ name: "participants", columns: ["id", "full_name", "email", …] }, …]
+// → [
+//     {
+//       name: "participants",
+//       columns: ["id", "full_name", "email", …],
+//       columnsMeta: [
+//         { name: "id", type: "uuid", nullable: false, isPrimaryKey: true, isForeignKey: false },
+//         { name: "org_id", type: "uuid", nullable: false, isPrimaryKey: false,
+//           isForeignKey: true, references: { table: "organizations", column: "id" } },
+//         …
+//       ],
+//       uniqueConstraints: [["id"], ["email"]],
+//     },
+//     …
+//   ]
 ```
 
 Column names are the **real DB names** (`full_name`), not the Drizzle property
 keys (`fullName`).
 
+`columnsMeta` and `uniqueConstraints` are what [derived
+objects](#derived-objects-projections) validate against — without them a
+projection cannot be authored, so keep this wired up rather than hand-writing
+`{ name, columns }` pairs.
+
 ### Defining an entity config
 
 ```ts
@@ -101,6 +122,73 @@ function useAdapter(crm: CRMAdapter) {
 }
 ```
 
+## Derived objects (projections)
+
+An ordinary entity maps one table. A **projection** derives a custom object from
+a base table plus explicit joins, so an object that spans tables — an
+`Enrollment` built from `enrollments`, `students` and `courses` — can be
+expressed at all.
+
+```ts
+import { validateProjection, type ProjectionMapping } from "@clivly/core";
+
+const enrollment: ProjectionMapping = {
+  version: 1,
+  kind: "projection",
+  entityKey: "enrollment",
+  baseTable: "enrollments",
+  joins: [
+    {
+      key: "stu",
+      table: "students",
+      type: "left",
+      on: [{ left: { table: "stu", column: "id" }, op: "eq",
+             right: { table: "base", column: "student_id" } }],
+    },
+  ],
+  fieldBindings: {
+    student_name: { from: { table: "stu", column: "full_name" } },
+    term: { from: { table: "base", column: "term" } },
+  },
+  identity: { kind: "base_pk" },
+};
+
+const errors = validateProjection(enrollment, tables, "projection");
+```
+
+Refer to the base table as `base` and each join by its `key`. Validation is
+**fail-closed**: a table whose discovery payload lacks `columnsMeta` /
+`uniqueConstraints` is rejected rather than assumed safe, because identity
+uniqueness cannot be proven without them.
+
+### Fan-out
+
+A join that does not cover a unique key of the joined table can match many rows
+per base row. That is allowed — but the base table's primary key is then no
+longer unique per output row, so validation enforces: at most one fan-out join,
+no `base_pk` identity on a fanning join, and an identity that includes a unique
+key of the fanning table. `fanOutJoins` reports which joins fan out.
+
+Composite identities concatenate their parts with `::`, so every part must be
+`NOT NULL` (`CONCAT_WS` skips nulls) and non-text (a value containing `::` could
+collide).
+
+### Building one interactively
+
+`ProjectionDraft` and its helpers exist so a builder UI stays presentational —
+every rule is a pure function of draft + schema:
+
+| Helper | Purpose |
+| --- | --- |
+| `emptyProjectionDraft(baseTable)` | A blank draft |
+| `suggestJoinConditions(base, joinTable, schema, key)` | Pre-fill a join's `ON` from foreign-key metadata |
+| `draftToProjection(draft)` | `ProjectionMapping`, or `null` while incomplete |
+| `draftErrors(draft, schema)` | Validation errors for the current draft |
+| `describeCardinality(draft, schema)` | Per-join `{ joinKey, fansOut }` for inline hints |
+
+Full guide, including how to choose an identity and the current limits:
+[`docs/guides/derived-objects.md`](https://github.com/amani-joseph/clivly.com/blob/master/docs/guides/derived-objects.md).
+
 ## Advanced: the `$raw` escape hatch
 
 Entity `filter`s and relationship `via`s accept a `$raw` form — an opaque SQL

package/dist/drizzle.d.cts

@@ -1,4 +1,4 @@
-import { DiscoveredColumnMeta, DiscoveredSchemaTable } from "./entity-config.cjs";
+import { m as DiscoveredSchemaTable, p as DiscoveredColumnMeta } from "./entity-config-buOIVtMM.cjs";
 import { Table } from "drizzle-orm";
 //#region src/drizzle.d.ts
 /**

package/dist/drizzle.d.mts

@@ -1,4 +1,4 @@
-import { DiscoveredColumnMeta, DiscoveredSchemaTable } from "./entity-config.mjs";
+import { m as DiscoveredSchemaTable, p as DiscoveredColumnMeta } from "./entity-config-buOIVtMM.mjs";
 import { Table } from "drizzle-orm";
 //#region src/drizzle.d.ts
 /**

package/dist/{entity-config-PoOGzvpN.cjs → entity-config-Bfq3OkCU.cjs}

@@ -438,10 +438,23 @@ const entitySchema = zod.z.strictObject({
 	relationships: zod.z.record(zod.z.string(), relationshipSchema).optional(),
 	/**
 	* For a "custom" entity, the crm_object_types row this materializes into.
-	* Required for custom (enforced at the API layer), unused for standard concepts.
+	* Server-generated, so pasting one into config couples the file to a
+	* specific workspace. Prefer declaring `objectType` and letting
+	* `clivly push-schema` resolve the id by slug.
 	*/
 	targetObjectTypeId: zod.z.string().min(1).optional(),
 	/**
+	* Declares the custom object this entity materializes into, so config is
+	* self-contained: the entity key is the slug (unique per org and immutable),
+	* and `clivly push-schema` creates the object type if the workspace does not
+	* have it yet. An alternative to pasting a `targetObjectTypeId`.
+	*/
+	objectType: zod.z.strictObject({
+		name: zod.z.string().min(1),
+		pluralName: zod.z.string().min(1).optional(),
+		iconKey: zod.z.string().min(1).optional()
+	}).optional(),
+	/**
 	* Advanced mapping: derive this entity from a base table plus explicit joins
 	* instead of a single source table. Valid only for `concept: "custom"`.
 	*
@@ -483,19 +496,69 @@ function defaultWarn(message) {
 	console.warn(message);
 }
 /**
-* Structural helper for `clivly.config.ts`. Gives full type inference on the
-* literal you pass and eagerly parses it, so shape errors surface at config
-* load rather than at sync time.
-*
-* It also catches the silent-data-loss footgun: a non-canonical field key for a
-* materialised concept (e.g. `fullName` instead of `name` on a `contact`) has no
-* destination column, so the store would write an empty value. By default these
-* are reported via `onWarn` (a `console.warn`); pass `{ strict: true }` to throw
-* a `ClivlyConfigError` instead. Does NOT validate against the host schema — that
-* needs the discovered schema; see `validateEntitiesConfig`.
+* The custom objects a config declares, keyed by entity slug. Entities that
+* already carry an explicit `targetObjectTypeId` are omitted: their object type
+* exists server-side and re-declaring it would be a no-op at best.
+*/
+function customObjectDeclarations(config) {
+	const declarations = [];
+	for (const [slug, entity] of Object.entries(config.entities)) {
+		if (entity.concept !== "custom" || entity.targetObjectTypeId) continue;
+		if (!entity.objectType) continue;
+		declarations.push({
+			slug,
+			...entity.objectType
+		});
+	}
+	return declarations;
+}
+/**
+* The `fields` entry a projection binding implies, qualified as `ref.column`.
+* Qualified rather than bare: two bindings can pull the same column name from
+* different tables (`cl.full_name` and `adv.full_name`), which a bare column
+* name cannot tell apart.
+*/
+function projectionFields(projection) {
+	const fields = {};
+	for (const [name, binding] of Object.entries(projection.fieldBindings)) fields[name] = `${binding.from.table}.${binding.from.column}`;
+	return fields;
+}
+/**
+* `fields` duplicates `projection.fieldBindings`, and only the bindings are
+* compiled — so a hand-written `fields` that drifts is both silent and
+* invisible. Derive it when omitted, and reject it when it disagrees.
 */
+function normalizeProjectionFields(config) {
+	const entities = {};
+	for (const [key, entity] of Object.entries(config.entities ?? {})) {
+		const projection = entity?.projection;
+		if (!projection?.fieldBindings) {
+			entities[key] = entity;
+			continue;
+		}
+		const derived = projectionFields(projection);
+		const declared = entity.fields;
+		if (!declared) {
+			entities[key] = {
+				...entity,
+				fields: derived
+			};
+			continue;
+		}
+		const drifted = [.../* @__PURE__ */ new Set([...Object.keys(derived), ...Object.keys(declared)])].filter((name) => declared[name] !== derived[name]).sort();
+		if (drifted.length > 0) {
+			const detail = drifted.map((name) => `"${name}": fields has ${declared[name] === void 0 ? "no entry" : `"${declared[name]}"`}, bindings imply ${derived[name] === void 0 ? "no entry" : `"${derived[name]}"`}`).join("; ");
+			throw new Error(`Clivly config: entity "${key}" has a \`fields\` map that disagrees with \`projection.fieldBindings\` — ${detail}. Only the bindings are compiled, so the \`fields\` copy would be silently ignored. Omit \`fields\` entirely and it will be derived.`);
+		}
+		entities[key] = entity;
+	}
+	return {
+		...config,
+		entities
+	};
+}
 function defineClivlyConfig(config, options) {
-	const parsed = entitiesConfigSchema.parse(config);
+	const parsed = entitiesConfigSchema.parse(normalizeProjectionFields(config));
 	const issues = canonicalFieldIssues(parsed);
 	if (issues.length > 0) {
 		if (options?.strict) throw new ClivlyConfigError(issues);
@@ -588,7 +651,7 @@ function validateEntity(v, config, key, entity, schema) {
 	}
 	if (entity.projection) {
 		if (entity.concept !== "custom") v.report(`${base}.projection`, `projection mappings are only supported for concept "${CUSTOM_CONCEPT}"`);
-		if (!entity.targetObjectTypeId) v.report(`${base}.targetObjectTypeId`, "a projection mapping must target a custom object type");
+		if (!(entity.targetObjectTypeId || entity.objectType)) v.report(`${base}.targetObjectTypeId`, "a projection mapping must target a custom object type — declare `objectType: { name: \"…\" }` (recommended: the entity key becomes its slug) or set an explicit `targetObjectTypeId`");
 		if (entity.source !== entity.projection.baseTable) v.report(`${base}.source`, `"source" ("${entity.source}") must match "projection.baseTable" ("${entity.projection.baseTable}") — the view compiler reads baseTable and the sync engine reads source, so a mismatch makes them disagree about which table this entity is`);
 		for (const error of validateProjection(entity.projection, schema, base)) v.report(error.path, error.message);
 		validateReadOnly(v, base, entity);
@@ -660,6 +723,12 @@ Object.defineProperty(exports, "coversUniqueKey", {
 		return coversUniqueKey;
 	}
 });
+Object.defineProperty(exports, "customObjectDeclarations", {
+	enumerable: true,
+	get: function() {
+		return customObjectDeclarations;
+	}
+});
 Object.defineProperty(exports, "defineClivlyConfig", {
 	enumerable: true,
 	get: function() {
@@ -702,6 +771,12 @@ Object.defineProperty(exports, "primaryKeyColumns", {
 		return primaryKeyColumns;
 	}
 });
+Object.defineProperty(exports, "projectionFields", {
+	enumerable: true,
+	get: function() {
+		return projectionFields;
+	}
+});
 Object.defineProperty(exports, "relationshipSchema", {
 	enumerable: true,
 	get: function() {

package/dist/{entity-config-Tb5bbKLj.mjs → entity-config-DFJ8L9jm.mjs}

@@ -438,10 +438,23 @@ const entitySchema = z.strictObject({
 	relationships: z.record(z.string(), relationshipSchema).optional(),
 	/**
 	* For a "custom" entity, the crm_object_types row this materializes into.
-	* Required for custom (enforced at the API layer), unused for standard concepts.
+	* Server-generated, so pasting one into config couples the file to a
+	* specific workspace. Prefer declaring `objectType` and letting
+	* `clivly push-schema` resolve the id by slug.
 	*/
 	targetObjectTypeId: z.string().min(1).optional(),
 	/**
+	* Declares the custom object this entity materializes into, so config is
+	* self-contained: the entity key is the slug (unique per org and immutable),
+	* and `clivly push-schema` creates the object type if the workspace does not
+	* have it yet. An alternative to pasting a `targetObjectTypeId`.
+	*/
+	objectType: z.strictObject({
+		name: z.string().min(1),
+		pluralName: z.string().min(1).optional(),
+		iconKey: z.string().min(1).optional()
+	}).optional(),
+	/**
 	* Advanced mapping: derive this entity from a base table plus explicit joins
 	* instead of a single source table. Valid only for `concept: "custom"`.
 	*
@@ -483,19 +496,69 @@ function defaultWarn(message) {
 	console.warn(message);
 }
 /**
-* Structural helper for `clivly.config.ts`. Gives full type inference on the
-* literal you pass and eagerly parses it, so shape errors surface at config
-* load rather than at sync time.
-*
-* It also catches the silent-data-loss footgun: a non-canonical field key for a
-* materialised concept (e.g. `fullName` instead of `name` on a `contact`) has no
-* destination column, so the store would write an empty value. By default these
-* are reported via `onWarn` (a `console.warn`); pass `{ strict: true }` to throw
-* a `ClivlyConfigError` instead. Does NOT validate against the host schema — that
-* needs the discovered schema; see `validateEntitiesConfig`.
+* The custom objects a config declares, keyed by entity slug. Entities that
+* already carry an explicit `targetObjectTypeId` are omitted: their object type
+* exists server-side and re-declaring it would be a no-op at best.
+*/
+function customObjectDeclarations(config) {
+	const declarations = [];
+	for (const [slug, entity] of Object.entries(config.entities)) {
+		if (entity.concept !== "custom" || entity.targetObjectTypeId) continue;
+		if (!entity.objectType) continue;
+		declarations.push({
+			slug,
+			...entity.objectType
+		});
+	}
+	return declarations;
+}
+/**
+* The `fields` entry a projection binding implies, qualified as `ref.column`.
+* Qualified rather than bare: two bindings can pull the same column name from
+* different tables (`cl.full_name` and `adv.full_name`), which a bare column
+* name cannot tell apart.
+*/
+function projectionFields(projection) {
+	const fields = {};
+	for (const [name, binding] of Object.entries(projection.fieldBindings)) fields[name] = `${binding.from.table}.${binding.from.column}`;
+	return fields;
+}
+/**
+* `fields` duplicates `projection.fieldBindings`, and only the bindings are
+* compiled — so a hand-written `fields` that drifts is both silent and
+* invisible. Derive it when omitted, and reject it when it disagrees.
 */
+function normalizeProjectionFields(config) {
+	const entities = {};
+	for (const [key, entity] of Object.entries(config.entities ?? {})) {
+		const projection = entity?.projection;
+		if (!projection?.fieldBindings) {
+			entities[key] = entity;
+			continue;
+		}
+		const derived = projectionFields(projection);
+		const declared = entity.fields;
+		if (!declared) {
+			entities[key] = {
+				...entity,
+				fields: derived
+			};
+			continue;
+		}
+		const drifted = [.../* @__PURE__ */ new Set([...Object.keys(derived), ...Object.keys(declared)])].filter((name) => declared[name] !== derived[name]).sort();
+		if (drifted.length > 0) {
+			const detail = drifted.map((name) => `"${name}": fields has ${declared[name] === void 0 ? "no entry" : `"${declared[name]}"`}, bindings imply ${derived[name] === void 0 ? "no entry" : `"${derived[name]}"`}`).join("; ");
+			throw new Error(`Clivly config: entity "${key}" has a \`fields\` map that disagrees with \`projection.fieldBindings\` — ${detail}. Only the bindings are compiled, so the \`fields\` copy would be silently ignored. Omit \`fields\` entirely and it will be derived.`);
+		}
+		entities[key] = entity;
+	}
+	return {
+		...config,
+		entities
+	};
+}
 function defineClivlyConfig(config, options) {
-	const parsed = entitiesConfigSchema.parse(config);
+	const parsed = entitiesConfigSchema.parse(normalizeProjectionFields(config));
 	const issues = canonicalFieldIssues(parsed);
 	if (issues.length > 0) {
 		if (options?.strict) throw new ClivlyConfigError(issues);
@@ -588,7 +651,7 @@ function validateEntity(v, config, key, entity, schema) {
 	}
 	if (entity.projection) {
 		if (entity.concept !== "custom") v.report(`${base}.projection`, `projection mappings are only supported for concept "${CUSTOM_CONCEPT}"`);
-		if (!entity.targetObjectTypeId) v.report(`${base}.targetObjectTypeId`, "a projection mapping must target a custom object type");
+		if (!(entity.targetObjectTypeId || entity.objectType)) v.report(`${base}.targetObjectTypeId`, "a projection mapping must target a custom object type — declare `objectType: { name: \"…\" }` (recommended: the entity key becomes its slug) or set an explicit `targetObjectTypeId`");
 		if (entity.source !== entity.projection.baseTable) v.report(`${base}.source`, `"source" ("${entity.source}") must match "projection.baseTable" ("${entity.projection.baseTable}") — the view compiler reads baseTable and the sync engine reads source, so a mismatch makes them disagree about which table this entity is`);
 		for (const error of validateProjection(entity.projection, schema, base)) v.report(error.path, error.message);
 		validateReadOnly(v, base, entity);
@@ -618,4 +681,4 @@ function validateEntitiesConfig(config, schema) {
 	};
 }
 //#endregion
-export { hasRichMeta as _, MATERIALIZING_CONCEPTS as a, filterSchema as c, validateEntitiesConfig as d, validateProjection as f, coversUniqueKey as g, columnMetaFor as h, ClivlyConfigError as i, relationshipSchema as l, joinFansOut as m, CRM_CONCEPTS as n, defineClivlyConfig as o, fanOutJoins as p, CUSTOM_CONCEPT as r, entitiesConfigSchema as s, CANONICAL_FIELDS as t, relationshipsSchema as u, primaryKeyColumns as v };
+export { columnMetaFor as _, MATERIALIZING_CONCEPTS as a, primaryKeyColumns as b, entitiesConfigSchema as c, relationshipSchema as d, relationshipsSchema as f, joinFansOut as g, fanOutJoins as h, ClivlyConfigError as i, filterSchema as l, validateProjection as m, CRM_CONCEPTS as n, customObjectDeclarations as o, validateEntitiesConfig as p, CUSTOM_CONCEPT as r, defineClivlyConfig as s, CANONICAL_FIELDS as t, projectionFields as u, coversUniqueKey as v, hasRichMeta as y };