@twinfinity/twin 6.1.1 → 6.1.2-ci.31342

package/docs/media/twin-management.md

@@ -0,0 +1,197 @@
+# Twin Management
+
+## Overview
+
+Twins are the central entity in the Twin service. A twin represents a digital twin specification with versioned state, soft-delete support, and optimistic concurrency via etags. Twins reference containers and files, and their processed content (T8CM, T8GI formats) is available for download.
+
+## Storage and Versioning
+
+Twins use a two-table versioning pattern:
+
+- `twin.twins` — The `version` column points to the current version. The `etag` column provides optimistic concurrency control, regenerated on every write.
+- `twin.twin_versions` — Immutable version history.
+
+Every update creates a new row in `twin_versions` and bumps the version pointer in `twins`. Soft deletion sets `is_deleted = true` in a new version row rather than removing data.
+
+The twin's `name`, its WGS84 `location`, and the set of entity `classes` it contains are persisted on the version row, so the list endpoint and the version-history endpoint can surface them without reading the specification blob. `name` and `location` are **derived from the twin's root entity** (see [Root Entity](#root-entity)) at write time; `classes` is derived from the specification's spatial hierarchy. Existing twins were backfilled across all versions (current and historical), so every version reports its own `name`, `location`, and `classes`.
+
+## REST API
+
+Twins are managed through `TwinsController` at route `twins`.
+
+| Operation     | Method | Route                                | Permission |
+| ------------- | ------ | ------------------------------------ | ---------- |
+| List          | GET    | `/twins`                             | (filtered) |
+| Get           | GET    | `/twins/{twinId}`                    | View       |
+| List versions | GET    | `/twins/{twinId}/versions`           | View       |
+| Get version   | GET    | `/twins/{twinId}/versions/{version}` | View       |
+| Create/Update | PUT    | `/twins/{twinId}`                    | Edit       |
+| Delete        | DELETE | `/twins/{twinId}`                    | Delete     |
+| Restore       | PUT    | `/twins/{twinId}/restore`            | Delete     |
+
+Content and diagnostics are handled by `TwinContentController`:
+
+| Operation                | Method | Route                                                                | Permission |
+| ------------------------ | ------ | -------------------------------------------------------------------- | ---------- |
+| Get content (latest)     | GET    | `/twins/{twinId}/contents`                                           | View       |
+| Get version content      | GET    | `/twins/{twinId}/versions/{version}/contents`                        | View       |
+| Download content blob    | GET    | `/twins/{twinId}/versions/{version}/contents/{contentId}/content`    | View       |
+| Get diagnostics (latest) | GET    | `/twins/{twinId}/diagnostics`                                        | View       |
+| Get version diagnostics  | GET    | `/twins/{twinId}/versions/{version}/diagnostics`                     | View       |
+| Get version events       | GET    | `/twins/{twinId}/versions/{version}/events`                          | View       |
+| Get processor events     | GET    | `/twins/{twinId}/versions/{version}/processors/{processorId}/events` | View       |
+
+### Request/Response Types
+
+Request DTO (`TwinPut`): `rootEntity` (required; see [Root Entity](#root-entity)), `entities` (sites, facilities, spaces, and `twinReferences`), `propertySets`, `elementQuantities`, `relatedContainerIds`, `fileReferences` (array of `{fileId, fileVersion, twinNodeId?}`), `etag` (required for updates). The legacy top-level `site` field is deprecated; `rootEntity` is authoritative. `name` and `location` are **not** accepted on input — they are derived from the root entity (a client that still sends them has them ignored, not rejected).
+
+Response DTO (`TwinGet`): extends `TwinSpecification` with `id`, `etag`, `relatedContainerIds`, `version`, `isDeleted`, `createdBy`, `createdUtc`, `modifiedBy`, `modifiedUtc`, `fileReferences`, `computedPropertySets`, the two computed nested-twin fields `aggregatedTwins` and `parentTwin` (see [Nested Twins](#nested-twins-twin-references)), and the derived `name` and full `location` (the latter omitted when the root entity declares no location). `name`/`location` are read-only projections of the root entity, not part of the stored specification body.
+
+List item (`ListTwinItem`): subset with `id`, `name`, `etag`, `propertySets`, `computedPropertySets`, `elementQuantities`, `relatedContainerIds`, `version`, `isDeleted`, `createdBy`, `createdUtc`, `modifiedBy`, `modifiedUtc`, `aggregatedTwins`, `parentTwin`, `location`, `classes`.
+
+The list `location` is a **coordinates-only** view: `{ epsg, latitude, longitude }`, where `epsg` is always `"4326"`. It is present only when the twin's specification declares a location, and is omitted entirely otherwise. `elevation` and `trueNorth` are intentionally left off the list to keep the payload small; the full single-twin GET (`TwinGet`) continues to return the complete `location` (including `elevation` and `trueNorth`) from the specification.
+
+The list `classes` is the deduplicated, ordinally-sorted set of IFC entity classes the twin's specification declares across its spatial hierarchy (sites, facilities, facility parts, and spaces) — e.g. `["IfcBuilding", "IfcBuildingStorey", "IfcSite", "IfcSpace"]`. It lets applications see *what a twin contains* without reading the specification blob. It is **always present** as an array and is `[]` when the twin declares no such entities. The set is derived from the specification at write time; the full single-twin GET (`TwinGet`) does not repeat it, since that response already returns every entity with its `class` inline.
+
+Delete/Restore request: `{ etag: string }`.
+
+### Query Parameters
+
+**List twins:** `relatedTo` (container ID filter), `includeDeleted` (bool, default: false), `page` (pagination token), `limit` (default: 100, max: 1000).
+
+**List versions:** `includeDeleted` (bool), `page`, `limit` (default: 100, max: 1000).
+
+**Events:** `offset`, `limit` (default: 100, max: 500).
+
+## Optimistic Concurrency
+
+All mutating operations use the `etag` field for conflict detection:
+
+- **Create**: `etag` is omitted or null. If the twin already exists, the server treats it as an update and requires a matching etag.
+- **Update**: `etag` must match the server's current etag. On mismatch, the server returns **409 Conflict** with the current twin state in the response body.
+- **Delete/Restore**: `etag` is sent in the request body. Same 409 behavior on mismatch.
+
+The 409 response containing the current state allows clients to implement merge-and-retry logic without an extra round-trip.
+
+## Soft Delete and Restore
+
+- `DELETE /twins/{twinId}` marks the twin as deleted (new version with `isDeleted = true`).
+- `PUT /twins/{twinId}/restore` restores a soft-deleted twin.
+- Both operations require the `Delete` permission and the current etag.
+- Deleted twins return redacted responses: list results (`TwinEntry`) redact the name to `"[Redacted]"`, omit properties and `location`, and empty `classes` to `[]` (it is always present, so it cannot be omitted), while GET-by-id (`TwinGet`) keeps the real name but strips the rest of the specification and relationships — emptying `entities` to `{}`, emptying `aggregatedTwins`, `aggregatedSpaces`, `elementQuantities`, `propertySets` and `fileReferences` to `[]`, setting `location` and `site` to `null`, and omitting `parentTwin`.
+- List operations exclude deleted twins by default; use `includeDeleted=true` to include them.
+
+## Root Entity
+
+Every twin declares a `rootEntity` (required) naming the entity that roots its model tree. The service enforces (`RootEntityValidator`):
+
+- `rootEntity` must be a **site or facility** present in `entities`; facility parts and spaces are not allowed as roots. A value that is not a key in `entities.sites` or `entities.facilities` — including a `twinReferences` key — is rejected, so a nested twin reference can never be a root.
+- A **facility-rooted** twin must not contain a site (neither in `entities.sites` nor the legacy `site` field) and cannot aggregate twins. The converter aggregates only the root under the project, so a site or reference on a facility root would be orphaned from the model tree.
+- The root entity must have a **non-empty `name`**, for **both** site and facility roots. `Site.name` is required by the schema, but `Facility.name` is optional, so a nameless facility root is rejected — the root's name becomes the twin's name and a twin always has a name.
+
+The legacy top-level `site` field is deprecated; `rootEntity` is authoritative. A failing root is rejected with cause `InvalidRootEntity`.
+
+> **Back-compat note.** Requiring a name on the root is a behavioural tightening: before name/location moved onto the root, a client could author a top-level `name` and leave the root entity nameless. Such a write is now rejected — clients must put the name on the root entity (the site or facility named by `rootEntity`). This is intentional (the no-legacy-compat stance); there is no transition fallback that accepts a top-level name.
+
+### Name and location are derived from the root entity
+
+A twin's `name` and `location` are **owned by its root entity**, not authored at the top level of the specification. `Site` and `Facility` each carry an optional `location` (WGS84); the root entity's `name` and `location` are extracted at write time, persisted on the version row, and surfaced on `TwinGet` (full `location`) and the list (`name` + coordinates-only `location`). They are read-only on input: a `TwinPut` cannot set them, mirroring the computed nested-twin fields (`aggregatedTwins`/`parentTwin`). Non-root sites/facilities may also carry a `location`, but only the root's is the twin's location.
+
+The T8 client-model converter likewise reads the project name from the root entity. If a twin's `rootEntity` is unresolvable (names no site/facility in `entities` — an edge case the validator blocks on write, reachable only for pre-existing data), the converter produces a project with **no name** rather than falling back to a top-level value (there is none after migration). This graceful degradation only affects already-malformed twins.
+
+## Nested Twins (Twin References)
+
+A twin can nest other twins as children, forming a parent/child tree. This is the **twin reference** feature (TWINS-3703). The behavioural rationale — especially how references that involve deleted twins are handled — is recorded in the ADR [Twin References Are Spec-Declared Edges With Computed Parentage](../../../../decisions/twin-reference-lifecycle.md); this section summarises the schema surface and the rules the service enforces.
+
+### Schema surface
+
+A reference has two halves, both authored in the parent's specification blob (so both are versioned):
+
+- **Declaration** — `entities.twinReferences` is a `{ refId → { twin: <twinId> } }` map. A `TwinReference` is intentionally minimal: it carries only the referenced twin's id (no transform, no name).
+- **Placement** — a `Site`'s `aggregatedTwinReferences` lists the `refId`s nested directly under it. Placement is what gives a reference a position in the spatial tree.
+
+A child twin must hang on a site: **every declared reference must be placed under exactly one site**. A reference declared in `twinReferences` but placed by no site (a dangling reference) is rejected, as is placing a `refId` that is not declared, or placing the same `refId` under more than one site. The holding site must also be **reachable from `rootEntity`** through the site aggregation chain (`site → aggregatedSites`). Only a site aggregates sites — facilities, facility parts and spaces never aggregate a site — so a site is reachable only along that chain; a reference under a site disconnected from the root is rejected, since the converter would orphan it from the model tree. Because a reference is placed under a `Site`, a **facility-rooted twin cannot aggregate twins** (it has no site). These cross-field rules are enforced in the service, since JSON Schema (draft-06) cannot express them.
+
+### Computed fields
+
+From those declarations the service maintains two **computed** read-model fields, returned on `TwinGet` and `ListTwinItem`:
+
+- **`aggregatedTwins`** (`string[]`) — the ids of the twins this twin parents (the flat projection of its `twinReferences`).
+- **`parentTwin`** (`string?`) — the id of the twin that references this twin, or omitted for a root twin.
+
+These are *projections of the actual edges, not authored input*. A `TwinPut` that sends them has them ignored. The authoritative relational edge is `twin.twins.parent_twin_id`, maintained transactionally by `upsert_twin`; `parentTwin` reads it directly.
+
+### Spec-versions-vs-computed-doesn't
+
+This is the load-bearing rule:
+
+- Changing **specification** data (name, location, `entities` including `twinReferences`/placement) creates a new version, a new etag, and emits a `TwinUpdated` event.
+- Changing **computed** data (`parentTwin`, `aggregatedTwins`) is maintained in place — **no new version, no etag change, no event**.
+
+Because parentage is computed, re-parenting a child is *not an edit of the child* and never bumps the child's version. A single PUT that adds N references versions exactly one twin (the parent), not N+1.
+
+### Enforced invariants
+
+`upsert_twin` validates the references that are *newly added* in a write (references carried over from the previous version are left untouched) and enforces, transactionally:
+
+| Rule | SQL violation | HTTP `cause` |
+| ---- | --------- | --- |
+| A twin must not reference itself | `self_reference` | `SelfTwinReference` |
+| A referenced twin must exist | `referenced_row_not_found` | `ReferencedItemNotFound` |
+| A referenced twin must not be deleted (a **new** reference to a deleted twin is rejected) | `referenced_row_deleted` | `ReferencedItemDeleted` |
+| A twin has at most one parent — unless its current parent is itself deleted, in which case it may be re-parented | `twin_already_referenced` | `TwinAlreadyReferenced` |
+| References must not form a cycle | `cycle` | `CyclicTwinReference` |
+
+Further rules are enforced in C# *before* the transaction (returned as a 400 with the named `cause`), because the SQL function only sees a de-duplicated id array: a twin may be **referenced at most once** within a single specification (`DuplicateTwinReference`), and each `refId` must be **placed under exactly one site reachable from `rootEntity`** (`InvalidTwinReferencePlacement`).
+
+### Permission to reference
+
+Editing a twin requires `Edit` on that twin. Adding a reference additionally requires **`View` on the referenced twin**: a reference places the child in this twin's tree and repoints the child's computed parent, so the caller must be able to read it. The check applies only to references *newly added* by a write (those carried over from the current version are not re-checked, mirroring `upsert_twin`'s claim loop — re-saving a twin that still references a child it can no longer read is allowed). A newly referenced twin that is missing **or** not visible to the caller is reported identically, as `ReferencedItemNotFound`, so the response never reveals the existence of a twin the caller cannot read (consistent with the service's "no View → not found, not forbidden" convention).
+
+### Deleted twins
+
+A reference to a twin that is deleted *after* it was first referenced is **kept** (the parent may keep and re-save it), but a **new** reference to an already-deleted twin is rejected. Consequently a live twin's `aggregatedTwins`, or a live child's `parentTwin`, may name a deleted (and therefore redacted) twin — consumers walking the tree must expect that.
+
+A child whose current parent is *deleted* may be claimed by another twin (the "steal"). At steal time only the relational edge moves; the deleted parent's specification still declares the lost child, but this is invisible because deleted twins are redacted on read. The declaration is reconciled against the actual edges on **restore**. See the ADR for the full lifecycle.
+
+## Content and Processing
+
+After a twin is updated, processors generate content in various formats:
+
+- **t8cm** — T8 Client Model (BIM product hierarchy, properties, classifications)
+- **t8cm-ext** — T8 Client Model extensions
+- **t8gi** — T8 Geometry Info (mesh descriptors, styles, regions)
+- **t8gd** — T8 Geometry Data
+
+The content endpoint returns metadata about available content items and their associated processors. Content blobs are downloaded separately via the content blob endpoint, which supports HTTP range requests.
+
+### Version Content Response
+
+```typescript
+interface TwinVersionContentResponse {
+  twinId: string;
+  twinVersion: number;
+  status: string;
+  processors: TwinProcessorResponse[];
+  content: TwinContentResponse[];
+  computedPropertySets: TwinComputedPropertySetsResponse[];
+}
+```
+
+### Diagnostics
+
+Each twin version may have diagnostics from processing, returned as a summary (fatal/error/warning counts) and detailed diagnostics data.
+
+### Processor Events
+
+Processor events provide a timeline of processing activity for a twin version. Events can be queried for all processors or filtered to a specific processor.
+
+## Container Relationship
+
+- `relatedContainerIds` links a twin to one or more containers (projects/folders).
+- Container IDs are **immutable after creation** — they cannot be changed on subsequent updates.
+- When creating a twin linked to multiple containers, the service computes intersected ACLs across all container scopes.
+
+## Client SDK
+
+The `@twinfinity/twin` package (`packages/twinfinity-twin/`) provides a typed TypeScript client. See its [README](../../../../../packages/twinfinity-twin/README.md) for usage examples. For the conventions all TypeScript clients follow, see [TypeScript Client Conventions](../../../packages/client-conventions/typescript-client-conventions.md).