@x12i/static-memorix-explorer-api 1.0.0 → 1.1.1

package/guides/demo-app.md

@@ -0,0 +1,524 @@
+# Demo App — FlowState
+
+This repository ships with a complete reference demo application — **FlowState**,
+a small task manager — that exercises the Memorix Explorer API end-to-end. The
+demo is served at `GET /demo` and runs entirely in the browser with no build
+step.
+
+This document explains how every piece of the demo is implemented: the
+metadata that defines the schema, the seed data the demo reads, and the way the
+UI maps each user action (create / edit / delete / comment) onto Explorer API
+calls.
+
+---
+
+## TL;DR
+
+```bash
+npm run build
+npm start
+# open http://localhost:5030/demo
+```
+
+Every mutation you make in the UI is persisted back to `mocks/data/**/*.json`
+through the Explorer API. Restart the server and your edits are still there.
+
+---
+
+## What the demo proves
+
+The demo is intentionally small, but it touches every part of the Explorer API
+that matters for a real frontend:
+
+| Capability              | Endpoint used                                            |
+| ----------------------- | -------------------------------------------------------- |
+| List users              | `GET  /api/explorer/records/collection?entityName=users` |
+| List tasks per area     | `GET  /api/explorer/records/collection?entityName=<area>`|
+| Full-text search        | `?searchText=` / `?q=`                                   |
+| Equality filters        | `?filter=status:eq:Doing&filter=priority:eq:Urgent`      |
+| Create a task           | `POST /api/explorer/records/write` (`operation=add`)     |
+| Update a task           | `POST /api/explorer/records/write` (`operation=upsert`)  |
+| Delete a task           | `DELETE /api/explorer/records/item`                      |
+| Move a task between areas | `DELETE` on old collection + `POST …/write op=add` on new |
+| Object-type discovery   | `GET /api/explorer/object-types` (used for inventory)    |
+
+No private endpoints, no shortcuts: the demo is a normal API client.
+
+---
+
+## Architecture overview
+
+```
+┌─────────────────────────── Browser (/demo) ────────────────────────────┐
+│                                                                        │
+│  FlowState UI  ──►  fetch()  ──►  /api/explorer/*  (Explorer API)      │
+│                                                                        │
+└────────────────────────────────────────────────────────────────────────┘
+                                 │
+                                 ▼
+┌──────────────────────── Fastify mock server ───────────────────────────┐
+│                                                                        │
+│  routes/index.ts                                                       │
+│    ├── /demo                       serves mocks/demo.html              │
+│    ├── /records/collection         reads  ←  store.getCollection       │
+│    ├── /records/write              writes → store.setCollection        │
+│    └── /records/item (DELETE)      writes → store.setCollection        │
+│                                                                        │
+│  engine/write.ts     validates + executes add/upsert/patch/delete      │
+│  engine/records.ts   query pipeline (filter / sort / search / page)    │
+│                                                                        │
+│  storage/InMemoryStore.ts   in-memory + debounced disk flush           │
+│                                                                        │
+└────────────────────────────────────────────────────────────────────────┘
+                                 │
+                                 ▼
+                       mocks/metadata/**/*.json
+                       mocks/data/**/*.json
+```
+
+The UI never talks to the filesystem directly. Every mutation goes through
+`POST /api/explorer/records/write` (or `DELETE /api/explorer/records/item`),
+which mutates the in-memory store, which debounces a flush back to the JSON
+file on disk.
+
+---
+
+## Metadata model
+
+Metadata files (under `mocks/metadata/`) describe the **shape** of the system.
+The demo contributes the following metadata.
+
+### Object types — `mocks/metadata/object-types/*.json`
+
+Each area in the demo is its own object type:
+
+```json
+// mocks/metadata/object-types/product.json
+{
+  "name": "product",
+  "summary": "Building the thing.",
+  "catalogRelations": [],
+  "rootPropertyCatalog": []
+}
+```
+
+The same shape is repeated for `marketing.json`, `admin.json`. Plus a `users`
+object type so people are first-class queryable records:
+
+```json
+// mocks/metadata/object-types/users.json
+{
+  "name": "users",
+  "summary": "People who can be assigned to tasks.",
+  "catalogRelations": [
+    {
+      "sourceProperty": "assigneeId",
+      "targetObject": "product",
+      "targetProperty": "recordId",
+      "relationType": "assigned-to"
+    }
+  ],
+  "rootPropertyCatalog": []
+}
+```
+
+`catalogRelations` is what powers `GET /api/explorer/inventory/graph`.
+
+### Narratives — `mocks/metadata/narratives/*.json`
+
+Narratives are authored tags that get merged into the snapshot view. The demo
+ships with a few per area:
+
+```json
+// mocks/metadata/narratives/product.json
+{
+  "website-refresh": {
+    "key": "website-refresh",
+    "label": "Website Refresh",
+    "description": "Redesigning the public-facing site."
+  },
+  "launch-prep": {
+    "key": "launch-prep",
+    "label": "Launch Prep",
+    "description": "Getting ready for public launch."
+  }
+}
+```
+
+Records reference narratives via `doc.narratives["website-refresh"] = true`.
+You can filter on them with the virtual filter syntax:
+`?filter=narrativeId:eq:website-refresh`.
+
+### Lists — `mocks/metadata/lists/*.json`
+
+Saved queries. The demo ships with three:
+
+- `my-plate.json` — sort by priority desc
+- `this-week.json` — sort by dueDate asc
+- `backlog.json` — default ordering
+
+```json
+// mocks/metadata/lists/this-week.json
+{
+  "id": "this-week",
+  "entity": "product",
+  "entityName": "product",
+  "baseContentType": "snapshots",
+  "fields": ["recordId", "title", "status", "priority", "owner", "dueDate"],
+  "sort": { "dueDate": 1 },
+  "query": {},
+  "listId": "this-week"
+}
+```
+
+### Write descriptors — `mocks/metadata/write-descriptors/*.json`
+
+This is the contract between the UI's mutation layer and the server's write
+engine. The demo ships with one descriptor per area:
+
+```json
+// mocks/metadata/write-descriptors/product-task-write.json
+{
+  "writeDescriptorId": "product-task-write",
+  "targetCollection": "product/snapshots",
+  "schema": {
+    "type": "object",
+    "required": ["recordId", "title"],
+    "properties": {
+      "recordId":   { "type": "string" },
+      "title":      { "type": "string" },
+      "status":     { "type": "string" },
+      "priority":   { "type": "string" },
+      "owner":      { "type": "string" },
+      "area":       { "type": "string" },
+      "assigneeId": { "type": "string" },
+      "dueDate":    { "type": "string" },
+      "notes":      { "type": "string" },
+      "chat":       { "type": "array" }
+    }
+  }
+}
+```
+
+The same shape is mirrored for `marketing-task-write.json` and
+`admin-task-write.json`. The `targetCollection` field tells the write engine
+which `objectType/contentType` pair to mutate.
+
+**Required-field semantics:** for `operation: "add"` every field in `required`
+must be present. For `upsert`, `patch`, and `delete`, only the identity field
+(`recordId` for tasks, `memoryId` for the memory tier) is mandatory — the rest
+of the payload is treated as a partial update.
+
+---
+
+## Data model
+
+Data files (under `mocks/data/`) hold the actual records.
+
+### Users — `mocks/data/users/snapshots.json`
+
+```json
+[
+  { "recordId": "usr-1", "name": "Ami",    "role": "owner" },
+  { "recordId": "usr-2", "name": "Devran", "role": "engineer" },
+  { "recordId": "usr-3", "name": "Ilan",   "role": "growth" }
+]
+```
+
+The demo loads these with:
+
+```
+GET /api/explorer/records/collection?entityName=users&limit=500
+```
+
+…and uses `recordId` as the foreign key on each task's `assigneeId`.
+
+### Tasks — `mocks/data/{product,marketing,admin}/snapshots.json`
+
+Each area is its own collection. A typical task:
+
+```json
+{
+  "recordId": "t1",
+  "title": "Review new frontend styling for the landing page",
+  "status": "doing",
+  "priority": "urgent",
+  "owner": "Sam",
+  "area": "product",
+  "assigneeId": "usr-2",
+  "dueDate": "2026-07-21",
+  "notes": "Ensure the dark-mode contrast ratios meet the updated accessibility guidelines.",
+  "chat": [
+    {
+      "id": "cmt-1",
+      "authorId": "usr-1",
+      "timestamp": "2026-07-19T14:22:00Z",
+      "text": "Can you add tests for the refresh token flow as well?"
+    }
+  ],
+  "narratives": { "website-refresh": true }
+}
+```
+
+Notes and chat live **on the snapshot itself**. This keeps the data model
+simple for the demo. In a richer deployment you could split chat into the
+`events` content type and join via the list `extensions` mechanism.
+
+---
+
+## UI → API call map
+
+Every interactive element in `mocks/demo.html` is wired to a specific API
+call. The table below is the source of truth.
+
+### Reads
+
+| UI element                                | API call                                                                                                |
+| ----------------------------------------- | ------------------------------------------------------------------------------------------------------- |
+| Initial load                              | `GET /records/collection?entityName=users&limit=500` + `GET /records/collection?entityName=<area>&limit=500` (×3 areas, in parallel) |
+| Search box (`/` to focus)                 | `GET /records/collection?entityName=<area>&searchText=<q>`                                              |
+| "To Do" / "Doing" / "Done" filter pills   | `GET /records/collection?entityName=<area>&filter=status:eq:<Value>`                                    |
+| "Urgent" filter pill                      | `GET /records/collection?entityName=<area>&filter=priority:eq:Urgent`                                   |
+| Sidebar area / quick view selections      | client-side filter on the already-loaded rows (no extra round trip)                                     |
+
+Because tasks span 3 collections, the demo loads all of them up front and
+filters client-side. For larger datasets you would issue one `records/collection`
+per navigation and rely on `?filter=` and `?limit=` server-side.
+
+### Mutations
+
+Every mutation is a single API call. The UI uses optimistic updates with
+rollback on failure (the previous state is restored and a toast is shown).
+
+| UI action                       | API call                                                                                          |
+| ------------------------------- | ------------------------------------------------------------------------------------------------- |
+| **+ New Task** (modal)          | `POST /records/write` body: `{writeDescriptorId:"<area>-task-write", operation:"add", input:{…}}` |
+| **Status pill** (Todo/Doing/Done/Someday) | `POST /records/write` body: `{operation:"upsert", input:{recordId, status}}`             |
+| **Priority pill** (Urgent/Normal/Low)     | `POST /records/write` body: `{operation:"upsert", input:{recordId, priority}}`           |
+| **Area dropdown** (move task)   | Descriptor-verified `DELETE /records/item` then `POST /records/write operation:add` with the new area |
+| **Assignee dropdown**           | `POST /records/write` body: `{operation:"upsert", input:{recordId, assigneeId}}`                  |
+| **Due date input**              | `POST /records/write` body: `{operation:"upsert", input:{recordId, dueDate}}`                     |
+| **Notes textarea** (debounced)  | `POST /records/write` body: `{operation:"upsert", input:{recordId, notes}}`                       |
+| **Send comment**                | `POST /records/write` body: `{operation:"upsert", input:{recordId, chat:[…]}}` (full chat array)  |
+| **Delete** button               | `DELETE /records/item?entityName=<area>&recordId=<id>&writeDescriptorId=<area>-task-write`         |
+
+Two equivalent ways to delete exist:
+
+1. **REST-style:** `DELETE /api/explorer/records/item?entityName=…&recordId=…&writeDescriptorId=…`
+2. **Operation-style:** `POST /api/explorer/records/write` body:
+   `{writeDescriptorId:"…", operation:"delete", input:{recordId:"…"}}`
+
+The demo uses #1 (the REST form) because it doesn't require knowing the
+writeDescriptorId for the source collection.
+
+---
+
+## Create flow (modal)
+
+When the user clicks **+ New Task** (`Cmd/Ctrl-N` also works), a modal opens
+with fields for title, area, status, priority, and assignee. On submit:
+
+1. The UI generates a `recordId` of the form `t-<Date.now()>` to avoid
+   collisions with the seed data (`t1`…`t12`).
+2. A `POST /api/explorer/records/write` is sent with `operation:"add"` and the
+   descriptor for the chosen area (`product-task-write`, etc.).
+3. The server validates the payload against the descriptor's JSON schema.
+4. On success, the record is appended to the in-memory collection and the
+   debounced flush writes it to `mocks/data/<area>/snapshots.json` within 300 ms.
+5. The UI unshifts the returned record into `state.tasks`, selects it, and
+   re-renders.
+
+On validation failure, the server returns
+`{ok:false, validated:false, errors:[…]}` and the UI shows a toast.
+
+---
+
+## Edit flow
+
+Edit affordances are inline in the **Overview** tab of the detail panel:
+
+- **Status** — four pill buttons (Todo / Doing / Done / Someday). Clicking one
+  immediately fires an `upsert` with `{recordId, status}`.
+- **Priority** — three pill buttons (Urgent / Normal / Low). Same pattern.
+- **Area** — dropdown. **Moving areas is a delete + add** (because each area
+  is a separate collection on disk). The UI sequences:
+  1. `DELETE /records/item?entityName=<old>&recordId=<id>&writeDescriptorId=<old>-task-write`
+  2. `POST /records/write` with `operation:"add"` on the new area's descriptor.
+- **Assignee** — dropdown. `upsert` with `{recordId, assigneeId}`.
+- **Due date** — date input. `upsert` with `{recordId, dueDate}`.
+- **Notes** — textarea. Debounced 400 ms before sending `upsert`.
+
+All edits are **optimistic**: the local state is updated immediately, the UI
+re-renders, and the network call fires in the background. If the call fails,
+the previous state is restored and a red toast is shown.
+
+---
+
+## Delete flow
+
+The detail panel header has a **Delete** button. Clicking it shows a
+confirmation dialog explaining which endpoint will be called. On confirm:
+
+1. `DELETE /api/explorer/records/item?entityName=<area>&recordId=<id>&writeDescriptorId=<area>-task-write`
+2. The server removes the record from the in-memory collection, returns the
+   removed document, and schedules a disk flush.
+3. The UI removes the record from `state.tasks`, clears the selection, and
+   re-renders.
+
+---
+
+## Comment flow
+
+Comments live on the task's `chat` array. Adding a comment:
+
+1. The UI appends `{id:"cmt-<ts>", authorId, timestamp, text}` to a **copy** of
+   the task's existing `chat` array.
+2. Sends `POST /records/write` with `operation:"upsert"` and
+   `input:{recordId, chat:[…full array…]}`.
+
+This rewrites the whole `chat` array per comment, which is fine for the demo's
+scale. For higher volume you'd model comments as `events` records keyed by
+`eventId` and join them in via a list descriptor's `extensions` array.
+
+---
+
+## Why each area is a separate collection
+
+We could have put all tasks in one `tasks` collection with an `area` field.
+The demo deliberately splits them across `product/snapshots.json`,
+`marketing/snapshots.json`, and `admin/snapshots.json` to demonstrate:
+
+1. **Multi-entity applications** — typical real apps have more than one object
+   type. Showing 3 object types is more representative than 1.
+2. **Cross-collection navigation** — the demo's "All Tasks" view merges 3
+   parallel `records/collection` calls, the same pattern a real dashboard uses.
+3. **Cross-collection moves** — moving a task between areas exercises
+   `DELETE` + `add`, which is what you'd do when an entity changes type.
+4. **Per-object-type metadata** — each area can carry its own narratives,
+   catalog relations, and write descriptors. The demo ships per-area narratives
+   (`product.json`, `marketing.json`, `admin.json`) to show this.
+
+If you want a single-collection layout, just put all tasks in one
+`tasks/snapshots.json`, drop the area field, and update the demo's
+`loadAllTasks()` to fetch from one entity.
+
+---
+
+## Object identity
+
+Every record carries exactly one identity key. For the demo, that's always
+`recordId`. The mock server's `MemorixRecordArraySchema` validates this at
+startup — see `engine/identity.ts`. If a record is missing its identity key,
+the server logs a warning at boot but continues running.
+
+The other supported identity keys are `entityId`, `eventId`, `knowledgeId`,
+and `memoryId`. The memory tier (`mocks/data/memory/memory.json`) uses
+`memoryId` and is the only place in the demo where `recordId` is not the key.
+
+---
+
+## Persistence guarantees
+
+- **In-memory state** is updated synchronously on every write.
+- **Disk flush** is debounced by `DISK_FLUSH_DEBOUNCE_MS` (default 300 ms).
+- **On `SIGINT` / `SIGTERM`**, the server flushes everything before exiting —
+  see `server.ts`.
+- **Crash** (SIGKILL, panic) loses any writes that hadn't been flushed yet
+  (at most 300 ms of writes).
+
+This matches the production semantics of a write-behind cache. For tests that
+need to assert against on-disk JSON, wait > `DISK_FLUSH_DEBOUNCE_MS` after the
+write call.
+
+---
+
+## Feature flags
+
+`POST /records/write` and `DELETE /records/item` are **not** gated behind a
+feature flag — they're always on. This matches the production behavior where
+record writes are gated by the write-descriptor contract, not by an
+environment toggle.
+
+Mutations on **lists** and **narratives** (`POST/PATCH/PUT/DELETE` under
+`/lists` and `/narratives`) are gated behind `MEMORIX_EXPLORER_ENABLE_METADATA_WRITES=true`.
+The demo does not exercise those endpoints.
+
+---
+
+## Verifying it works
+
+A 30-second smoke test, runnable from the repo root:
+
+```bash
+# 1. Start the server
+npm run build && npm start &
+
+# 2. Read
+curl -s "localhost:5030/api/explorer/records/collection?entityName=users&limit=10"
+
+# 3. Create
+curl -s -X POST localhost:5030/api/explorer/records/write \
+  -H "Content-Type: application/json" \
+  -d '{"writeDescriptorId":"product-task-write","operation":"add","input":{"recordId":"t-smoke-1","title":"Smoke","area":"product","status":"todo","priority":"normal"}}'
+
+# 4. Update (partial)
+curl -s -X POST localhost:5030/api/explorer/records/write \
+  -H "Content-Type: application/json" \
+  -d '{"writeDescriptorId":"product-task-write","operation":"upsert","input":{"recordId":"t-smoke-1","status":"doing"}}'
+
+# 5. Delete
+curl -s -X DELETE "localhost:5030/api/explorer/records/item?entityName=product&recordId=t-smoke-1&writeDescriptorId=product-task-write"
+
+# 6. Open the UI
+open http://localhost:5030/demo
+```
+
+---
+
+## Extending the demo
+
+Common modifications:
+
+- **Add a new area** — create `mocks/metadata/object-types/<area>.json`,
+  `mocks/metadata/write-descriptors/<area>-task-write.json`,
+  `mocks/data/<area>/snapshots.json`, then add the area key to the `AREAS` map
+  at the top of `mocks/demo.html`.
+- **Add a new task field** — extend the write descriptor's `schema.properties`,
+  then render a new input in `renderDetail()` and send it via `updateTask()`.
+- **Switch to per-view server-side pagination** — replace the demo's
+  `loadAllTasks()` (3 parallel fetches) with a single fetch per navigation that
+  uses `?filter=`, `?limit=`, and `?offset=`.
+- **Add a list descriptor** — create a JSON file under
+  `mocks/metadata/lists/`, then call
+  `GET /api/explorer/lists/<listId>/records` from the UI.
+
+---
+
+## File map
+
+```
+mocks/
+├── demo.html                                  ← the UI
+├── metadata/
+│   ├── object-types/
+│   │   ├── product.json   marketing.json   admin.json
+│   │   └── users.json                          ← demo users (assignable)
+│   ├── narratives/
+│   │   ├── product.json   marketing.json   admin.json
+│   ├── lists/
+│   │   ├── my-plate.json   this-week.json   backlog.json
+│   ├── write-descriptors/
+│   │   ├── product-task-write.json
+│   │   ├── marketing-task-write.json
+│   │   └── admin-task-write.json
+│   └── agents.json                             ← still here for the assets demo
+└── data/
+    ├── users/snapshots.json                    ← 3 seed users
+    ├── product/snapshots.json                  ← 5 seed tasks
+    ├── marketing/snapshots.json                ← 4 seed tasks
+    └── admin/snapshots.json                    ← 3 seed tasks
+```
+
+The `assets/`, `findings/`, and `memory/` collections are not used by the
+demo UI; they exist to keep the Explorer API parity complete and to support
+the smoke tests in the guides.