@thatopen/services 0.10.0 → 0.12.0

package/docs/ai-quickstart.md

@@ -283,4 +283,9 @@ component, once added to a project, is then triggered by an app or an automation
   four-point plan and waiting for approval to do what was just requested costs the user a whole turn
   and buys nothing. Stop and ask only when you are about to change files you did not create, when
   the request can be read two ways that mean different work, or when the next step is destructive.
+- **Never save to the platform on every change.** Drafts belong in `localStorage` /
+  IndexedDB; the platform gets an explicit save. Writes are capped at 30 per minute and a
+  `429` loses the write. See
+  `node_modules/@thatopen/services/docs/rate-limits.md` before you write any save, sync or
+  polling code.
 - The scaffold already works — **extend it, don't replace it.**

package/docs/builtin/paths.json

@@ -39,6 +39,10 @@
     "path": "src/built-in/ClashesManager/example.ts",
     "description": "ClashesManager — Structural engineers and BIM coordinators routinely detect geometric clashes between disciplines — structural steel penetrating ductwork, pipes clashing with slabs — and need to track the status of each issue through resolution without leaving their BIM environment."
   },
+  {
+    "path": "src/built-in/ProjectManager/example.ts",
+    "description": "ProjectManager — Project-level configuration and data — GIS origin, georeferencing, app settings, and other project-scoped concerns — need a single canonical place to live. Without it, every builtin that needs to persist something at the project level (not per-file, not per-user) ends up inventing its own file, its own save/load pattern, and its own place to put it."
+  },
   {
     "path": "src/built-in/UIManager/src/app/example.ts",
     "description": "top-app — Front-end developers building BIM web apps need to coordinate several async initialization steps before the UI can be shown — loading the engine, preparing fragment workers, registering loaders — and users need visible feedback on what is happening during that wait, rather than a blank screen."
@@ -94,5 +98,13 @@
   {
     "path": "src/built-in/UIManager/src/topics-panel/example.ts",
     "description": "topics-panel — Every coordination workflow needs a place to track issues, clashes, and action items against a 3D model — who's responsible, what still needs review, and where in the model it actually is. Building that from scratch means a BCF-compatible data model, a form, comment threads, viewpoint capture/navigation, document links, and a persistence layer — all before a single issue gets tracked."
+  },
+  {
+    "path": "src/built-in/UIManager/src/viewer/example.ts",
+    "description": "top-viewer — Every BIM app needs a 3D viewport: a scene, a camera, a grid, element highlighting, and the plumbing to keep loaded models framed and rendering efficiently as they stream in. Building that from scratch is a project on its own, before a single button or panel gets wired to it."
+  },
+  {
+    "path": "src/built-in/UIManager/src/viewer-toolbar/example.ts",
+    "description": "top-viewer-toolbar — A 3D viewport is only as useful as the controls around it: selecting a target set, hiding/isolating/ghosting elements, clipping, measuring, walking through the model. Building that toolbar — and keeping every button in sync with tool state and with each other — is real work, before you've customized anything."
   }
 ]

package/docs/rate-limits.md

@@ -0,0 +1,123 @@
+# Rate limits, and how to save data without hitting them
+
+The platform API is rate limited. Every app and cloud component shares the same
+budget, so **how often you write decides whether your app works**. This page has
+the numbers, the failure mode, and the pattern to use instead.
+
+> **The one rule:** never send a request on every change the user makes. No
+> autosave-per-keystroke, no write inside a render loop, no polling loop that
+> hammers a list endpoint. Keep work-in-progress in the browser and talk to the
+> server on an explicit save.
+
+---
+
+## The numbers
+
+Limits are per **rolling 60-second window**, counted per user (JWT), or per API
+token owner, or per IP — whichever identifies the caller.
+
+| Endpoint | Client method | Limit |
+|---|---|---|
+| `POST /api/item` | `createFile`, `createComponent`, `createApp` | **30 / min** |
+| `POST /api/item/:id/version` | `updateFile` / `updateComponent` **with a `file`** | **30 / min** |
+| `PUT /api/item/:id/version/:tag/metadata` | `updateFileVersionMetadata` | 30 / min |
+| `DELETE /api/item/:id/version/:tag/metadata` | `deleteFileVersionMetadata` | 30 / min |
+| `PUT /api/item/:id/version/:tag/archive` \| `/recover` | `archiveVersion`, `recoverVersion` | 30 / min |
+| `POST /api/item/hidden` \| `/hidden/batch` | `createHiddenFile`, `createHiddenFilesBatch` | 30 / min |
+| `POST /api/processor/:id/execute` | `executeComponent` | 20 / min |
+| `GET /api/item/folder/:id/download`, `POST /api/item/batch/download` | `downloadFolder` | 10 / min |
+| `POST /api/item/batch/versions` \| `/batch/version-metadata` \| `/batch/folders` | `listVersionsBatch`, `getFileVersionMetadataBatch`, `getFoldersBatch` | 60 / min |
+| `POST /api/item/hidden/signed-url/batch` | `getHiddenFileSignedUrlsBatch` | 100 / min |
+| `GET /api/item/hidden/:id/download` | `downloadHiddenFile` | 3000 / min |
+| Everything else | — | 100 / min |
+
+Two things follow from the table:
+
+- **30 writes per minute is one write every two seconds.** An autosave tied to
+  user input passes that in a few seconds of typing or dragging.
+- **Reads are cheap, but not free.** A viewer that mints one signed URL per tile
+  will exhaust 100/min quickly — use `getHiddenFileSignedUrlsBatch`, which signs
+  up to `STORAGE_BATCH_MAX` files per request.
+
+## What a rate-limited response looks like
+
+Status `429`, with a `Retry-After` header (seconds) and this body:
+
+```json
+{
+  "statusCode": 429,
+  "message": "Rate limit exceeded: max 30 requests per 60s for this endpoint. Retry after 12s.",
+  "code": "RATE_LIMITED",
+  "details": { "limit": 30, "windowSeconds": 60, "retryAfter": 12, "scope": "user" }
+}
+```
+
+The client surfaces it as a `RequestError`:
+
+```ts
+import { RequestError } from '@thatopen/services';
+
+try {
+  await client.updateFile(fileId, { file: blob, versionTag: tag });
+} catch (err) {
+  if (err instanceof RequestError && err.status === 429) {
+    // err.code === 'RATE_LIMITED'
+    // err.retryAfter — seconds to wait, from Retry-After or details.retryAfter
+    showToast(`Saving is paused for ${err.retryAfter}s — your work is kept locally.`);
+    return;
+  }
+  throw err;
+}
+```
+
+**A 429 means the write did not happen.** Nothing was saved. If the user's only
+copy of the change was in that request, it is gone — which is the real reason
+the local-draft pattern below matters.
+
+## The pattern: local drafts, explicit saves
+
+Keep every intermediate state in the browser. Write to the platform only when
+the user asks for it.
+
+```ts
+const draftKey = `draft:${fileId}`;
+
+function onChange(state: unknown) {
+  localStorage.setItem(draftKey, JSON.stringify({ state, at: Date.now() }));
+}
+
+async function onSave(state: unknown) {
+  const blob = new Blob([JSON.stringify(state)], { type: 'application/json' });
+  await client.updateFile(fileId, { file: blob, versionTag: `v${Date.now()}` });
+  localStorage.removeItem(draftKey);
+}
+```
+
+On load, if a draft exists for the file, offer to restore it. That gives crash
+recovery — the thing autosave was really for — at zero requests.
+
+Rules of thumb:
+
+- **Explicit save**, or a timer no faster than **once every 30 seconds**, and
+  only when something actually changed.
+- **One request per save**, not one per changed object. Batch the whole document.
+- **Never save while a save is in flight.** Keep a flag per file, and drop or
+  queue the second save. Overlapping writes also create versions that are hard
+  to reconcile.
+- **Big or binary work-in-progress** belongs in IndexedDB, not `localStorage`
+  (about 5 MB per origin).
+- There is **no draft-write method in the client on purpose.** Local storage is
+  the draft store; the platform stores versions the user chose to keep.
+
+## Retries
+
+The client does not retry by default. When you turn retries on, it backs off
+exponentially, adds jitter, and honours `Retry-After`:
+
+```ts
+const client = new EngineServicesClient(token, apiUrl, { retries: 3 });
+```
+
+Only network failures, `429` and `5xx` are retried. Other `4xx` fail straight
+away, because repeating them cannot help. Never write your own immediate retry
+loop around a 429 — that is what turns a throttled request into an outage.

package/package.json

@@ -15,7 +15,7 @@
     "access": "public"
   },
   "private": false,
-  "version": "0.10.0",
+  "version": "0.12.0",
   "main": "dist/index.cjs.js",
   "module": "dist/index.es.js",
   "types": "dist/index.d.ts",

package/resources/AGENTS.md

@@ -19,6 +19,7 @@ Do this before answering any question or writing any code. These are compact des
 | Platform built-ins | `node_modules/@thatopen/services/docs/builtin/paths.json` |
 | Platform client API | `node_modules/@thatopen/services/docs/client/paths.json` |
 | CLI commands | `node_modules/@thatopen/services/docs/cli/paths.json` |
+| Rate limits + how to save data | `node_modules/@thatopen/services/docs/rate-limits.md` |
 | Engine components (`OBC`, `OBF`) | `https://raw.githubusercontent.com/ThatOpen/engine_components/refs/heads/main/examples/paths.json` |
 | Fragments (`FRAGS`) | `https://raw.githubusercontent.com/ThatOpen/engine_fragment/refs/heads/main/examples/paths.json` |
 | UI components (`BUI`) | `https://raw.githubusercontent.com/ThatOpen/engine_ui-components/refs/heads/main/examples/paths.json` — **skip** entries whose path contains `packages/obc` or `bim-grid` |
@@ -38,4 +39,13 @@ Once you have these, you know everything available on the platform. Only then fe
 
 ## Hard rules (always apply)
 
-1. **All UI must be built with Lit**, using the web components from `@thatopen/ui` (`BUI`) — `bim-button`, `bim-panel`, `bim-panel-section`, `bim-toolbar`, `bim-dropdown`, `bim-input`, and the rest of `packages/core`. Always consult the design system before writing any UI: `https://raw.githubusercontent.com/ThatOpen/engine_ui-components/refs/heads/main/DESIGN.md`.
+1. **Never write to the platform on every change.** No autosave per keystroke,
+   per drag, or inside a render loop. Keep work in progress in `localStorage` /
+   IndexedDB and call the platform on an **explicit user save** (or a timer no
+   faster than once every 30 seconds). Writes are capped at **30 per minute**;
+   crossing that returns `429` and the write is **lost**, not queued.
+   If the user asks for autosave, build it against local storage and say so.
+   Read `node_modules/@thatopen/services/docs/rate-limits.md` before writing any
+   save, sync, or polling code — it has the per-endpoint limits and the pattern.
+
+2. **All UI must be built with Lit**, using the web components from `@thatopen/ui` (`BUI`) — `bim-button`, `bim-panel`, `bim-panel-section`, `bim-toolbar`, `bim-dropdown`, `bim-input`, and the rest of `packages/core`. Always consult the design system before writing any UI: `https://raw.githubusercontent.com/ThatOpen/engine_ui-components/refs/heads/main/DESIGN.md`.

package/src/built-in/ProjectManager/example.ts

@@ -0,0 +1,163 @@
+/* MD
+  ## ProjectManager
+  ---
+  Project-level configuration and data — GIS origin, georeferencing, app
+  settings, and other project-scoped concerns — need a single canonical place
+  to live. Without it, every builtin that needs to persist something at the
+  project level (not per-file, not per-user) ends up inventing its own file,
+  its own save/load pattern, and its own place to put it.
+
+  ProjectManager is that canonical store. It persists one JSON file
+  (`${ProjectManager.uuid}.json`) loose directly inside `__project_data` — the
+  same folder convention `GISManager` already used — rather than in a
+  per-app subfolder like `collider`/`cde`, which are reserved for secondary,
+  per-builtin data.
+
+  This tutorial covers initializing the manager from the platform client; the
+  `origin` and `graphicsSettings` singletons; the `bimCoordinates` sites
+  collection used for BIM georeferencing; the `assetCoordinates` collection
+  used for splats and point clouds; auto-save and its events; and the
+  read-only namespace introspection API.
+
+  ProjectManager has no built-in UI panel — it's a pure data layer. Apps that
+  need georeferencing or project-settings UI (like Aqualia's viewer) build it
+  on top of this.
+
+  ### 🖖 Importing our libraries
+*/
+
+import * as OBC from "@thatopen/components";
+import { PlatformClient, ProjectManager } from "@thatopen/services";
+
+
+/* MD
+  ### 🚀 Initializing ProjectManager
+  Like any other built-in, ProjectManager is set up through the platform
+  client. `client.setup(...)` resolves the shared `OBC.Components` instance
+  and gives us back everything we need.
+*/
+
+const client = PlatformClient.fromPlatformContext();
+
+const { components } = (await client.setup(
+  { OBC },
+  { uuid: ProjectManager.uuid },
+)) as { components: OBC.Components };
+
+const manager = components.get(ProjectManager);
+
+/* MD
+  `init(client)` loads the persisted JSON for the current project — or
+  creates it with the default schema if it doesn't exist yet. `manager.ready`
+  flips to `true` once that's done, and the entity accessors below are only
+  safe to use after that.
+*/
+
+await manager.init(client);
+console.log("ready:", manager.ready);
+console.log("namespaces:", manager.getNamespaces());
+
+manager.onSaveStart.add(() => console.log("[event] onSaveStart"));
+manager.onSaveComplete.add((success) => console.log("[event] onSaveComplete:", success));
+
+/* MD
+  ### 📍 origin — a singleton entity
+  `origin` holds the project's GIS origin: latitude, longitude, height, and
+  rotation. Singletons expose `get`/`set`/`update`; `set` replaces the whole
+  value, `update` merges a partial one. Both schedule a debounced auto-save —
+  call `flush()` to force it immediately and await completion.
+*/
+
+console.log("origin before:", manager.origin.get());
+
+await manager.origin.set({ lat: 40.4164, lon: -3.7038, height: 700, rotation: 0 });
+await manager.origin.update({ height: 710 });
+await manager.flush();
+
+console.log("origin after update+flush:", manager.origin.get());
+
+/* MD
+  ### 🗺️ bimCoordinates — the sites collection
+  Sites are how a project keeps georeferencing for more than one BIM model,
+  each with its own origin and transform matrix. Because sites are created by
+  users at runtime, their ids are opaque and generated internally —
+  `createSite` mints the id and returns it, while `setSite` only updates a
+  site that already exists (it throws otherwise). This split keeps storage
+  keys decoupled from mutable data like a site's label.
+*/
+
+const siteId = await manager.bimCoordinates.createSite({
+  label: "Site 1",
+  description: "Primary IFC export.",
+  lat: 40.41647848134075,
+  lon: -3.703451491164702,
+  height: 699.970526176243,
+  rotation: 0,
+  baseMatrix: [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, -498477.20216, -719.2375, 4389662.11289, 1],
+});
+
+await manager.bimCoordinates.setDefaultSite(siteId);
+await manager.flush();
+
+console.log("default site:", manager.bimCoordinates.getDefaultSite());
+console.log("all sites:", manager.bimCoordinates.getSites());
+
+/* MD
+  Updating an existing site goes through `setSite`, not `createSite`:
+*/
+
+await manager.bimCoordinates.setSite(siteId, {
+  ...manager.bimCoordinates.getSite(siteId)!,
+  label: "Site 1 (revised)",
+});
+await manager.flush();
+
+console.log("site after update:", manager.bimCoordinates.getSite(siteId));
+
+/* MD
+  ### 🖼️ assetCoordinates — the files collection
+  `assetCoordinates` georeferences individual files — splats, point clouds —
+  that live outside the BIM model tree. Unlike sites, entries are keyed by
+  the caller-supplied `fileId`, since that id already comes from the platform
+  and is stable by construction.
+*/
+
+await manager.assetCoordinates.set("splat_1", {
+  type: "splat",
+  lat: 40.416600807789145,
+  lon: -3.703739209785748,
+  height: 727.2700125737748,
+  rotation: 0,
+});
+await manager.flush();
+
+console.log("splats:", manager.assetCoordinates.getByType("splat"));
+console.log("splat_1:", manager.assetCoordinates.get("splat_1"));
+
+/* MD
+  ### ⚙️ graphicsSettings — another singleton
+  Same shape as `origin` — `get`/`set`/`update`, debounced auto-save.
+*/
+
+await manager.graphicsSettings.update({ /* project-wide graphics defaults */ } as any);
+await manager.flush();
+
+console.log("graphicsSettings:", manager.graphicsSettings.get());
+
+/* MD
+  ### 🔍 Namespaces — read-only introspection
+  `getNamespaces()` lists the closed set of namespaces the schema defines;
+  `getNamespace(key)` returns every entity registered under one. Useful for
+  debugging or building a generic project-data inspector — not for reading
+  values you already know the shape of, which is what the entity accessors
+  above are for.
+*/
+
+console.log("georeferencing namespace:", manager.getNamespace("georeferencing"));
+
+/* MD
+  ### 🤖 What's not covered here
+  ProjectManager ships without a UI panel — everything above is the data
+  layer only. Building georeferencing or project-settings UI on top of
+  `origin`/`bimCoordinates`/`assetCoordinates` is up to the consuming app.
+*/

package/src/built-in/UIManager/src/viewer/example.ts

@@ -0,0 +1,182 @@
+/* MD
+  ## top-viewer
+  ---
+  Every BIM app needs a 3D viewport: a scene, a camera, a grid, element
+  highlighting, and the plumbing to keep loaded models framed and rendering
+  efficiently as they stream in. Building that from scratch is a project on
+  its own, before a single button or panel gets wired to it.
+
+  `top-viewer` is a ready-made web component that delivers exactly that. Drop
+  it into a `top-app` layout and it creates the 3D world, wires camera
+  controls, sets up the highlighter, keeps every loaded fragment model framed
+  and culled correctly, and exposes four docking slots so toolbars and tools
+  can attach to it without any manual overlay/positioning code.
+
+  This tutorial covers the prerequisites; the minimal mount; what the
+  viewport does automatically; how docking works; and loading your first
+  model into it.
+
+  By the end, you'll have a fully working 3D viewport running in your
+  application with a single line of markup.
+*/
+
+import { html } from "lit";
+import * as THREE from "three";
+import * as OBC from "@thatopen/components";
+import * as OBF from "@thatopen/components-front";
+import * as FRAGS from "@thatopen/fragments";
+import * as BUI from "@thatopen/ui";
+import { PlatformClient, UIManager } from "@thatopen/services";
+import type { App } from "../app/index";
+
+/* MD
+  ### ✅ Prerequisites
+
+  Two conditions must be met before `<top-viewer>` does anything useful:
+
+  1. **`UIManager` must be in the setup call.** It registers all platform web
+     components, including `<top-viewer>` itself. Without it the element is
+     unknown to the browser and renders as an empty box.
+
+  2. **`OBC.FragmentsManager` must be initialized** (`fragments.init(...)`)
+     before you try to load any model. `top-viewer` itself doesn't need this
+     to mount — its world, camera, and grid are ready the moment it connects
+     — but nothing will load until the fragments worker is up.
+
+  Both are naturally satisfied when you do the setup shown below before
+  appending `top-app` to the document.
+*/
+
+const client = PlatformClient.fromPlatformContext();
+
+const { components } = (await client.setup(
+  { OBC, OBF, BUI, THREE, FRAGS },
+  { uuid: UIManager.uuid },
+)) as { components: OBC.Components };
+
+components.get(UIManager).init();
+
+/* MD
+  ### 🖥️ Minimal mount
+
+  `<top-viewer>` alone is a fully working, empty viewport — camera controls,
+  grid, and highlighter are already set up, no further code needed:
+
+  ```html
+  <top-viewer></top-viewer>
+  ```
+
+  It does **not** come with a toolbar. If you want the built-in one, add
+  `<top-viewer-toolbar>` as a child (see that component's own tutorial) — a
+  bare `<top-viewer>` is intentionally just the viewport.
+*/
+
+const app = document.createElement("top-app") as unknown as App;
+
+app.setup = (waitUntil) => {
+  waitUntil(
+    (async () => {
+      const fragments = components.get(OBC.FragmentsManager);
+      const workerUrl = await FRAGS.FragmentsModels.getWorker();
+      fragments.init(await FRAGS.toClassicWorker(workerUrl), { classicWorker: true });
+    })(),
+    "Fragments Core",
+  );
+  return { components, client };
+};
+
+app.elements = {
+  viewer: () => html`<top-viewer></top-viewer>`,
+};
+
+app.layouts = {
+  main: {
+    label: "Main",
+    icon: "solar:3d-square-bold",
+    template: `"viewer" 1fr / 1fr`,
+  },
+};
+
+app.layout = "main";
+
+const container = document.getElementById("that-open-app") ?? document.body;
+container.appendChild(app);
+document.body.style.margin = "0";
+
+/* MD
+  ### 🤖 What's handled automatically
+
+  Once mounted, `<top-viewer>` takes ownership of the following without any
+  further code from you:
+
+  **World + camera** — an `OBC.SimpleWorld` with a transparent-background
+  scene, an `OBC.OrthoPerspectiveCamera` (orbit controls, perspective by
+  default), and a reference grid.
+
+  **Highlighter** — `OBF.Highlighter` is set up for the world, ready for any
+  built-in or custom code to select/highlight elements.
+
+  **Model framing** — every model loaded through `OBC.FragmentsManager`
+  (anywhere in your app, not just inside this component) is automatically
+  added to the scene, told which camera to use for culling/LOD, and factored
+  into the camera's far-plane distance so large models never get clipped.
+  You never need to call `scene.add()` or `useCamera()` yourself.
+
+  **Navigation tools** — the navigation gizmo and active-tool HUD are baked
+  onto the viewport overlay the moment the world is ready.
+
+  **Context for descendants** — `<top-viewer>` provides `worldContext` (the
+  `OBC.World` instance) to any descendant, regardless of shadow-DOM
+  boundaries. `<top-viewer-toolbar>` and its standalone buttons consume it
+  this way — you never pass `world` around by hand.
+
+  **Cleanup** — disposes the world (and the highlighter) automatically when
+  the element is removed from the DOM.
+
+  ### 🧩 Docking slots
+
+  `<top-viewer>` exposes four named slots for toolbars/tools, arranged around
+  the viewport: `"bottom"`, `"top"`, `"left"`, `"right"`. Any element can use
+  them, not just `<top-viewer-toolbar>`:
+
+  ```html
+  <top-viewer>
+    <top-viewer-toolbar></top-viewer-toolbar>            <!-- no slot → auto-docks "bottom" -->
+    <my-custom-panel slot="right"></my-custom-panel>
+  </top-viewer>
+  ```
+
+  A `<top-viewer-toolbar>` added **without** a `slot` attribute is
+  auto-docked into `"bottom"` for you — that's the only slot with special
+  handling; every other element (or every other slot) needs its `slot`
+  attribute set explicitly.
+
+  ### ⚠️ Gotchas
+
+  - **No toolbar is added for you.** A bare `<top-viewer>` has no bottom bar,
+    no buttons — just the viewport. If your app expects one, you must add
+    `<top-viewer-toolbar>` yourself.
+  - **Only `"bottom"`, `"top"`, `"left"`, `"right"` are recognized slot
+    names.** Anything else (a typo, or an app-specific name) is not placed
+    into any docking area and simply doesn't render where you'd expect.
+  - **`worldContext` starts out `undefined`.** World creation is asynchronous
+    — any custom descendant consuming the context must handle the
+    not-yet-ready case, the same way `<top-viewer-toolbar>` does.
+  - **The host needs real, resolved dimensions from its parent.** `width:
+    100%; height: 100%` alone isn't enough if the parent's own size hasn't
+    resolved (e.g. an unsized grid area) — give the viewer's container an
+    explicit size.
+
+  ### 📥 Loading a model
+
+  Load any `.frag` through the shared `OBC.FragmentsManager` — `<top-viewer>`
+  picks it up and frames it automatically, no matter where in your app the
+  load happens:
+
+  ```ts
+  const fragments = components.get(OBC.FragmentsManager);
+  const file = await fetch("https://your-storage.example.com/model.frag");
+  const buffer = await file.arrayBuffer();
+  await fragments.core.load(buffer, { modelId: "example-model" });
+  ```
+*/