@x12i/static-memorix-explorer-api 1.0.0 → 1.1.1

package/README.md

@@ -1,37 +1,81 @@
-# @x12i/mock-memorix-explorer-api
+# @x12i/static-memorix-explorer-api
 
-In-memory mock server that provides **full API parity** for the Memorix Explorer
+Static mock server that provides **full API parity** for the Memorix Explorer
 Fastify API. No MongoDB, Catalox, or Redis required — state lives in
 `./mocks/**/*.json` and is flushed to disk (debounced) on mutation.
 
+Ships with a **complete reference demo app** (FlowState — a small task
+manager) at `GET /demo` that exercises every read/write path in the API.
+See [guides/demo-app.md](./guides/demo-app.md) for the full walkthrough.
+
 ## Quick start
 
 ```bash
 npm install
 npm run build
-npm start                 # listens on :4300 (PORT / HOST / MOCKS_DIR env-overridable)
+npm start                 # listens on :5030 (PORT / HOST / MOCKS_DIR env-overridable)
 # dev (no build): npm run dev
+
+# open the demo UI in your browser
+open http://localhost:5030/demo
 ```
 
 Health:
 
 ```bash
-curl localhost:4300/health
-curl "localhost:4300/api/explorer/health?includeInventory=1"
+curl localhost:5030/health
+curl "localhost:5030/api/explorer/health?includeInventory=1"
+```
+
+## Run from CLI or code
+
+Installed CLI, using the default port `5030`:
+
+```bash
+npx @x12i/static-memorix-explorer-api
+```
+
+Override the port from the shell:
+
+```bash
+PORT=8080 npx @x12i/static-memorix-explorer-api
 ```
 
+Start it programmatically (explicit options take precedence over environment
+defaults):
+
+```js
+import { startServer } from "@x12i/static-memorix-explorer-api";
+
+const app = await startServer({ port: 5030, host: "127.0.0.1" });
+// await app.close();
+```
+
+For code-level tests without binding a port:
+
+```js
+import { buildServer } from "@x12i/static-memorix-explorer-api";
+
+const app = await buildServer();
+const response = await app.inject({ method: "GET", url: "/health" });
+await app.close();
+```
+
+See [guides/running-the-service.md](./guides/running-the-service.md) for all
+CLI, environment, lifecycle, and programmatic examples.
+
 ## Architecture
 
 ```
-Fastify Router (/api/explorer/* + /health + Authix/M2M shim)
+Fastify Router (/api/explorer/* + /health + Authix/M2M shim + /demo)
         │
 Query & Interceptor Engine (mingo + narrative virtual filters)
-   ┌────────────┬──────────────────┬──────────────┐
+    ┌────────────┬──────────────────┬──────────────┐
 Inventory/Lenses  Snapshots/Aliases   Records Writer
-   └────────────┴──────────────────┴──────────────┘
-              InMemoryStore (state matrix)
-                        │ debounced disk flush
-         ./mocks/metadata  &  ./mocks/data
+    └────────────┴──────────────────┴──────────────┘
+               InMemoryStore (state matrix)
+                         │ debounced disk flush
+          ./mocks/metadata  &  ./mocks/data
 ```
 
 ## Routes
@@ -41,13 +85,14 @@ Inventory/Lenses  Snapshots/Aliases   Records Writer
 | Health | `GET /health`, `GET /api/explorer/health?includeInventory=1` |
 | Inventory | `/inventory/collections`, `/summary`, `/issues`, `/graph` (query `sourceLens=db-first\|catalox-first`) |
 | Snapshots | `/snapshots/:objectType/:recordId[.../associated[/:propertyName]]`, `/snapshots/:objectType/associated-properties`, `/associations/plan\|apply\|verify` |
-| Records | `/records/collection`, `/full`, `/item`, `/content`, `/raw-collection`, `/raw-item`, `/workspace` |
+| Records | `/records/collection`, `/full`, `/item`, `/content`, `/raw-collection`, `/raw-item`, `/workspace`, `POST /records/write`, `DELETE /records/item` |
 | Lists | `/lists`, `/lists/:listId`, `/lists/:listId/records`, `/lists/suggest`, mutations `POST/PATCH/PUT/DELETE` |
 | Narratives | `/narratives`, `/:entity`, `/:entity/raw`, `/:entity/:key`, `/:entity/:key/records`, mutations `POST/PATCH/DELETE` |
 | Object Types | `/object-types`, `/:name`, `/:name/root-property-catalog`, `POST .../compute` |
-| Write | `POST /records/write` (validate / dryRun / upsert / patch / replace) |
+| Write | `POST /records/write` (validate / dryRun / add / upsert / patch / replace / **delete**) |
 | Agents | `GET /agents` |
 | Auth shim | `POST /api/explorer/auth/token` |
+| Demo | `GET /demo` — serves `mocks/demo.html` |
 
 ## Query conventions
 
@@ -79,11 +124,59 @@ Client `associated.data|discovery|analysis|{custom}` maps to storage
 associated{Custom}`. Raw `associated*` fields are stripped from the returned
 document and recomposed into a normalized `associated` bucket.
 
+## Write engine
+
+`POST /api/explorer/records/write` accepts an `operation` and routes the
+payload through the write descriptor's JSON schema.
+
+### Operations
+
+| Operation | Behavior |
+|-----------|----------|
+| `add`     | Append. Auto-generates an identity if missing. |
+| `upsert`  | Update existing by identity key, or append. |
+| `patch`   | Update existing by identity key. No-op if missing. |
+| `replace` | Replace the entire collection (use with caution). |
+| `delete`  | Remove every record matching the supplied identity keys. Returns the removed records. |
+
+### Required-field semantics
+
+The `schema.required` array is enforced **fully** for `add` and `replace`.
+For `upsert`, `patch`, and `delete`, only the identity field (`recordId`,
+or `memoryId` on the memory tier) is required. This allows partial updates
+like `{recordId, status}` without resupplying every required field — the
+pattern the demo UI uses for every inline edit.
+
+### REST-style delete
+
+For ergonomics, you can also delete a single record with:
+
+```
+DELETE /api/explorer/records/item?entityName=product&recordId=t1&writeDescriptorId=product-task-write
+```
+
+Returns `{ ok: true, count: <n>, removed: [<record>…] }` and flushes immediately.
+
+## Demo app
+
+The `mocks/demo.html` reference UI (served at `GET /demo`) is a complete task
+manager that uses the Explorer API for every read and mutation. It demonstrates:
+
+- Multi-collection reads (3 parallel `records/collection` calls + a `users` collection)
+- Filter / search / sort via query params
+- **Create** via modal → `POST /records/write op=add`
+- **Edit** via inline pills/dropdowns → `POST /records/write op=upsert` (partial updates)
+- **Delete** via button → `DELETE /records/item`
+- Cross-collection **move** (delete on old area + add on new area)
+- Optimistic UI with rollback on failure
+
+Full implementation walkthrough: [guides/demo-app.md](./guides/demo-app.md).
+
 ## Environment
 
 | Var | Default | Notes |
 |-----|---------|-------|
-| `PORT` | `4300` | Listen port |
+| `PORT` | `5030` | Listen port |
 | `HOST` | `0.0.0.0` | Listen host |
 | `MOCKS_DIR` | `<project>/mocks` | Fixture root |
 | `DISK_FLUSH_DEBOUNCE_MS` | `300` | Debounce before disk flush |
@@ -99,7 +192,7 @@ document and recomposed into a normalized `associated` bucket.
 The mock server does not connect to MongoDB, but it **tricks the frontend** into
 thinking it is talking to the routed tenant databases.
 
-- The `GET /api/explorer/health` response now includes a dynamic `memorixDb`
+- The `GET /api/explorer/health` response includes a dynamic `memorixDb`
   field constructed as `<orgId>-memorix-entities + <orgId>-memorix-events`.
 - The agent-scoped catalox DB is reflected in the `cataloxDb` field as
   `<agentId>-memorix-catalox`.
@@ -118,6 +211,9 @@ The mock server enforces the following feature-flag gates with error responses:
 - Registry writes (if implemented) require
   `MEMORIX_EXPLORER_ENABLE_REGISTRY_WRITES=true`.
 
+Record writes (`POST /records/write`, `DELETE /records/item`) are **not**
+feature-flagged — they're always on, gated only by the write descriptor contract.
+
 When a flag is disabled or unset, the mock returns `{ ok: false, error: "..." }`
 matching the production Fastify error shape (typically 403/404).
 
@@ -134,4 +230,42 @@ and `knowledgeId`.
 - Records engine accepts `target=memory` and supports `memoryId` for fetching.
 - Write engine validates and persists `memoryId` on memory-tier records.
 - Startup validation (`MemorixRecordArraySchema` mock) allows `memoryId` without
-  throwing schema errors.
+  throwing schema errors.
+
+## Documentation
+
+- [guides/running-the-service.md](./guides/running-the-service.md) — installation, configuration, env vars
+- [guides/using-the-api.md](./guides/using-the-api.md) — endpoint reference, query language, write operations
+- [guides/managing-json-files.md](./guides/managing-json-files.md) — fixture structure, schemas, how the store works
+- [guides/demo-app.md](./guides/demo-app.md) — FlowState reference UI walkthrough
+
+## Release history
+
+### v1.1.1
+- Changed the default listen port from `4300` to `5030`.
+- Added safe programmatic APIs: `buildServer()` and `startServer(options)`.
+- Importing the package no longer starts a listener as a side effect.
+- Added CLI, environment override, programmatic startup, and injected-test examples.
+
+### v1.1.0
+- **Added**: `delete` operation to the write engine
+  (`POST /records/write` body `operation:"delete"`).
+- **Added**: REST-style `DELETE /api/explorer/records/item` endpoint.
+- **Added**: Operation-aware schema validation — `upsert`/`patch`/`delete`
+  only require the identity field, not every `required` field.
+- **Added**: Complete demo application (`mocks/demo.html`) wired entirely
+  through the Explorer API — Create/Edit/Delete/Move with optimistic UI.
+- **Added**: `users` object type + collection.
+- **Added**: Per-area task write descriptors
+  (`product/marketing/admin-task-write`).
+- **Added**: Per-area narratives and enriched task snapshots
+  (`notes`, `chat`, `assigneeId`).
+- **Removed**: obsolete `/api/state` flow and `mocks/db.json`
+  (replaced by Explorer API writes).
+- **Docs**: full demo walkthrough (`guides/demo-app.md`) and refreshed
+  endpoint tables.
+
+### v1.0.0
+- Initial implementation of mock Memorix Explorer API.
+- Health, inventory, snapshots, records, lists, narratives, object types,
+  write engine (add/upsert/patch/replace), agents, auth shim.

package/dist/config.js

@@ -8,7 +8,7 @@ export const MOCKS_DIR = process.env.MOCKS_DIR
     : path.resolve(ROOT_DIR, "..", "mocks");
 export const METADATA_DIR = path.join(MOCKS_DIR, "metadata");
 export const DATA_DIR = path.join(MOCKS_DIR, "data");
-export const PORT = Number(process.env.PORT || 4300);
+export const PORT = Number(process.env.PORT || 5030);
 export const HOST = process.env.HOST || "0.0.0.0";
 export const DISK_FLUSH_DEBOUNCE_MS = Number(process.env.DISK_FLUSH_DEBOUNCE_MS || 300);
 // ---------------------------------------------------------------------------

package/dist/engine/identity.js

@@ -6,34 +6,36 @@
 export function resolveIdField(ct, sample) {
     if (ct === "memory")
         return "memoryId";
-    if (ct === "events")
-        return "eventId";
     if (sample) {
+        if (ct === "events" && "eventId" in sample)
+            return "eventId";
         if ("recordId" in sample)
             return "recordId";
         if ("entityId" in sample)
             return "entityId";
+        if ("eventId" in sample)
+            return "eventId";
         if ("id" in sample)
             return "id";
     }
+    if (ct === "events")
+        return "eventId";
     return "recordId";
 }
 /**
  * Mock of the production `MemorixRecordArraySchema`. Validates that each record
- * in a collection array carries exactly one recognized identity key. Previously
- * this threw at boot if a collection used `memoryId`; it now accepts it.
+ * in a collection array carries the collection's canonical identity key.
+ * Additional identity-shaped fields are allowed because they can represent
+ * relationships (for example a memory has `memoryId` and may reference an
+ * `entityId` or `eventId`).
  */
 export function validateRecordArray(objectType, ct, records) {
     const errors = [];
     const expected = resolveIdField(ct, records[0]);
     for (const rec of records) {
-        const present = ["recordId", "entityId", "eventId", "knowledgeId", "memoryId"].filter((k) => rec?.[k] !== undefined);
-        if (present.length === 0) {
+        if (rec?.[expected] === undefined) {
             errors.push(`${objectType}/${ct}: record missing identity key (expected ${expected})`);
         }
-        else if (present.length > 1) {
-            errors.push(`${objectType}/${ct}: record has multiple identity keys [${present.join(", ")}]`);
-        }
     }
     return { ok: errors.length === 0, errors };
 }

package/dist/engine/write.js

@@ -13,15 +13,31 @@ export function loadWriteDescriptor(writeDescriptorId) {
 }
 /**
  * Validate input against the write descriptor schema (lightweight AJV-free check).
+ *
+ * `operation` controls how `required` is enforced:
+ *   - `add` / `replace`: every field in `schema.required` must be present.
+ *   - `upsert` / `patch` / `delete`: only the collection's identity field
+ *     (e.g. `recordId`, `memoryId`) is required; other `required` entries are
+ *     ignored because partial updates are allowed.
+ *
+ * Type checks on whatever fields ARE present always run.
  */
-export function validateInputSchema(writeSpec, input) {
+export function validateInputSchema(writeSpec, input, operation = "add") {
     const errors = [];
     const schema = (writeSpec.schema || {});
     const payload = Array.isArray(input) ? input : input ? [input] : [];
+    const partial = operation === "upsert" || operation === "patch" || operation === "delete";
     if (schema.required && Array.isArray(schema.required)) {
+        // Determine the identity field for this descriptor based on targetCollection.
+        const targetCollection = writeSpec.targetCollection || "";
+        const [, ctRaw] = targetCollection.split("/");
+        const idField = ctRaw === "memory" ? "memoryId" : "recordId";
         for (const req of schema.required) {
             for (const item of payload) {
                 if (item?.[req] === undefined) {
+                    // In partial modes, only the identity field is mandatory.
+                    if (partial && req !== idField)
+                        continue;
                     errors.push(`Missing required field: ${req}`);
                 }
             }
@@ -30,7 +46,7 @@ export function validateInputSchema(writeSpec, input) {
     if (schema.properties) {
         for (const item of payload) {
             for (const [field, def] of Object.entries(schema.properties)) {
-                if (item[field] === undefined)
+                if (item[field] === undefined || item[field] === null)
                     continue;
                 if (def.type) {
                     const actualType = Array.isArray(item[field]) ? "array" : typeof item[field];
@@ -58,8 +74,11 @@ export function parseTargetCollection(target) {
 }
 export function executeOperation(params) {
     const { objectType, ct } = parseTargetCollection(params.target);
-    const collection = store.getCollection(objectType, ct);
+    // Work on a copy so dry runs cannot leak into the live in-memory store and
+    // later get persisted by an unrelated write.
+    const collection = [...store.getCollection(objectType, ct)];
     const payload = Array.isArray(params.data) ? params.data : [params.data];
+    const persist = params.persist !== false;
     // Resolve the identity field for this collection tier.
     // Memory tier uses memoryId; everything else defaults to recordId.
     const idField = ct === "memory" ? "memoryId" : "recordId";
@@ -95,10 +114,35 @@ export function executeOperation(params) {
             break;
         }
         case "replace": {
+            if (persist)
+                store.setCollection(objectType, ct, payload);
             return payload;
         }
+        case "delete": {
+            const removed = [];
+            for (const item of payload) {
+                const id = item[idField] ?? item.recordId ?? item.id;
+                if (id === undefined)
+                    continue;
+                let i = collection.length;
+                while (i--) {
+                    const candidate = collection[i];
+                    if ((candidate[idField] ?? candidate.recordId ?? candidate.id) === id) {
+                        removed.push(collection.splice(i, 1)[0]);
+                    }
+                }
+            }
+            if (persist)
+                store.setCollection(objectType, ct, collection);
+            return removed;
+        }
+        default:
+            throw Object.assign(new Error(`Unsupported write operation: ${params.op}`), {
+                statusCode: 400,
+            });
     }
-    store.setCollection(objectType, ct, collection);
+    if (persist)
+        store.setCollection(objectType, ct, collection);
     return payload;
 }
 function getNextId(collection, idField = "recordId") {

package/dist/routes/index.js

@@ -10,6 +10,9 @@ import { getMergedNarratives, getRawNarratives, queryNarrativeRecords, } from ".
 import { computeRootPropertyCatalog } from "../engine/objectTypes.js";
 import { loadWriteDescriptor, validateInputSchema, executeOperation, } from "../engine/write.js";
 import { planAssociations, applyAssociations, verifyAssociations, } from "../engine/associations.js";
+import { MOCKS_DIR } from "../config.js";
+import fs from "node:fs";
+import path from "node:path";
 export async function registerRoutes(app) {
     // ---------- A. Health & System Diagnostics ----------
     app.get("/health", async () => ({ ok: true }));
@@ -296,18 +299,24 @@ export async function registerRoutes(app) {
         return { objectType: name, computed: true, rootPropertyCatalog: entries };
     });
     // ---------- H. Record Write Engine ----------
-    app.post("/api/explorer/records/write", async (request) => {
+    app.post("/api/explorer/records/write", async (request, reply) => {
         const { writeDescriptorId, operation = "add", input, records, dryRun = false, validateOnly = false, } = (request.body || {});
         if (!writeDescriptorId)
-            return { error: "writeDescriptorId required", ok: false };
+            return reply.code(400).send({ error: "writeDescriptorId required", ok: false });
+        if (!["add", "upsert", "patch", "replace", "delete"].includes(operation)) {
+            return reply.code(400).send({ ok: false, error: `Unsupported operation: ${operation}` });
+        }
         const writeSpec = loadWriteDescriptor(writeDescriptorId);
         const targetCollection = writeSpec.targetCollection;
         if (!targetCollection)
-            return { error: "writeDescriptor missing targetCollection", ok: false };
-        const data = input || records;
-        const validation = validateInputSchema(writeSpec, data);
+            return reply.code(400).send({ error: "writeDescriptor missing targetCollection", ok: false });
+        const data = input ?? records;
+        if (data === undefined || data === null) {
+            return reply.code(400).send({ ok: false, error: "input or records required" });
+        }
+        const validation = validateInputSchema(writeSpec, data, operation);
         if (!validation.ok) {
-            return { ok: false, validated: false, errors: validation.errors };
+            return reply.code(400).send({ ok: false, validated: false, errors: validation.errors });
         }
         if (validateOnly)
             return { ok: true, validated: true };
@@ -315,11 +324,40 @@ export async function registerRoutes(app) {
             target: targetCollection,
             op: operation,
             data,
+            persist: !dryRun,
         });
         if (!dryRun) {
             await store.flushAll();
         }
-        return { ok: true, count: results.length, dryRun };
+        return { ok: true, count: results.length, operation, dryRun, results };
+    });
+    app.delete("/api/explorer/records/item", async (request, reply) => {
+        const { entityName, recordId, writeDescriptorId } = (request.query || {});
+        if (!entityName || !recordId || !writeDescriptorId) {
+            return reply.code(400).send({
+                ok: false,
+                error: "entityName, recordId, and writeDescriptorId query params are required",
+            });
+        }
+        const writeSpec = loadWriteDescriptor(writeDescriptorId);
+        const expectedTarget = `${entityName}/snapshots`;
+        if (writeSpec.targetCollection !== expectedTarget) {
+            return reply.code(400).send({
+                ok: false,
+                error: `Write descriptor targets ${writeSpec.targetCollection}, not ${expectedTarget}`,
+            });
+        }
+        const validation = validateInputSchema(writeSpec, { recordId }, "delete");
+        if (!validation.ok) {
+            return reply.code(400).send({ ok: false, validated: false, errors: validation.errors });
+        }
+        const removed = executeOperation({
+            target: writeSpec.targetCollection,
+            op: "delete",
+            data: { recordId },
+        });
+        await store.flushAll();
+        return { ok: true, count: removed.length, removed };
     });
     app.get("/api/explorer/object-types/:name", async (request, reply) => {
         const { name } = request.params;
@@ -339,6 +377,18 @@ export async function registerRoutes(app) {
         token_type: "Bearer",
         expires_in: 3600,
     }));
+    // ---- Demo UI ----
+    // The demo app served at /demo uses the Explorer API exclusively for its
+    // CRUD (records/collection, records/write, DELETE records/item). No separate
+    // /api/state flow is needed.
+    app.get("/demo", async (_request, reply) => {
+        const demoPath = path.join(MOCKS_DIR, "demo.html");
+        if (!fs.existsSync(demoPath)) {
+            return reply.code(404).send({ ok: false, error: "demo.html not found" });
+        }
+        const html = fs.readFileSync(demoPath, "utf8");
+        return reply.type("text/html").send(html);
+    });
 }
 function send404(reply, message) {
     return reply.code(404).send({ ok: false, error: message });

package/dist/server.js

@@ -1,36 +1,58 @@
 #!/usr/bin/env node
 import Fastify from "fastify";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
 import { store } from "./storage/InMemoryStore.js";
 import { registerRoutes } from "./routes/index.js";
 import { PORT, HOST, MOCKS_DIR } from "./config.js";
-async function main() {
-    const app = Fastify({ logger: false });
+/** Build a loaded Fastify instance without opening a network port. */
+export async function buildServer(options = {}) {
+    const app = Fastify({ logger: options.logger ?? false });
     // Global error handler matching predictable error shapes.
     app.setErrorHandler((error, _request, reply) => {
-        const status = error.statusCode && error.statusCode >= 400 ? error.statusCode : 500;
+        const normalized = error instanceof Error
+            ? error
+            : new Error("Internal Server Error");
+        const status = normalized.statusCode && normalized.statusCode >= 400
+            ? normalized.statusCode
+            : 500;
         reply.status(status).send({
             ok: false,
-            error: error.message || "Internal Server Error",
+            error: normalized.message || "Internal Server Error",
         });
     });
     await store.load();
     await registerRoutes(app);
-    const port = Number(process.env.PORT || PORT);
-    await app.listen({ port, host: HOST });
+    return app;
+}
+/** Build and listen. Options override PORT/HOST environment configuration. */
+export async function startServer(options = {}) {
+    const app = await buildServer(options);
+    const port = options.port ?? PORT;
+    const host = options.host ?? HOST;
+    await app.listen({ port, host });
     // eslint-disable-next-line no-console
-    console.log(`[mock-memorix-explorer-api] listening on ${HOST}:${port}`);
+    console.log(`[mock-memorix-explorer-api] listening on ${host}:${port}`);
     console.log(`[mock-memorix-explorer-api] mocks dir: ${MOCKS_DIR}`);
-    const shutdown = async () => {
-        console.log("\n[mock-memorix-explorer-api] flushing + shutting down");
-        await store.flushAll();
-        await app.close();
-        process.exit(0);
-    };
-    process.on("SIGINT", shutdown);
-    process.on("SIGTERM", shutdown);
+    if (options.installSignalHandlers ?? true) {
+        const shutdown = async () => {
+            console.log("\n[mock-memorix-explorer-api] flushing + shutting down");
+            await store.flushAll();
+            await app.close();
+            process.exit(0);
+        };
+        process.once("SIGINT", shutdown);
+        process.once("SIGTERM", shutdown);
+    }
+    return app;
+}
+const isCliEntry = process.argv[1]
+    ? path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)
+    : false;
+if (isCliEntry) {
+    startServer().catch((err) => {
+        // eslint-disable-next-line no-console
+        console.error(err);
+        process.exit(1);
+    });
 }
-main().catch((err) => {
-    // eslint-disable-next-line no-console
-    console.error(err);
-    process.exit(1);
-});