mikser-io-sdk-api 2.4.0 → 3.0.0

package/README.md

@@ -103,7 +103,105 @@ Operations outside the endpoint's allowlist return `403`; missing or wrong token
 
 ## Entities
 
-`mikser.entities(endpointName, { token })` returns a per-endpoint client. The endpoint name matches a key in your `api.endpoints` config on the server.
+`mikser.entities(endpointName, options)` returns a per-endpoint client. The endpoint name matches a key in your `api.endpoints` config on the server. Supported options:
+
+| Option   | Default | What it does                                                              |
+|---|---|---|
+| `token`  | `null`  | Bearer token for endpoints declared with a token. Sent on every request.  |
+| `data`   | `{}`    | Pairs the client with the mikser-io `data` plugin's static-file outputs. See below. |
+
+### `data` — pair with the `data` plugin for fast first paint
+
+If you have a known, predictable query that runs on every page load — a route table for an SPA, a navigation menu, a per-document body — paying an API round-trip for it on first paint is wasted work. The mikser-io `data` plugin lets you publish that data as static files at build/finalize time, and the SDK consumes them on the client side. The option names on both sides are the same — `catalog` and `entities` — so it reads as one config split across the network.
+
+Mikser's `data` plugin has two relevant blocks:
+
+```js
+// mikser.config.js
+data: {
+    catalog: {
+        // out/data/sitemap.json — one combined file for the whole catalog,
+        // projected to just the fields your router/nav needs.
+        sitemap: {
+            query: e => e.type === 'document' && e.meta?.published && e.meta?.component,
+            pick: ['id', 'destination', 'meta.component', 'meta.route', 'meta.title'],
+        },
+    },
+    entities: {
+        // out/data/<entity.name>.page.json — one file per published document,
+        // with full content. Consumed by useDocument(id) for first-paint
+        // single-doc reads.
+        page: {
+            query: e => e.type === 'document' && e.meta?.published,
+            pick: ['id', 'meta', 'content'],
+        },
+    },
+},
+
+api: {
+    endpoints: {
+        public: {
+            query: e => e.type === 'document' && e.meta?.published,
+            operations: ['list', 'subscribe'],
+            cache: true,
+        },
+    },
+},
+```
+
+On the client, name the same blocks. The names you pass match the keys you used on the server side:
+
+```js
+const documents = createClient({ baseUrl: 'https://cms.example.com' })
+    .entities('public', {
+        data: {
+            catalog:  'sitemap',   // /data/sitemap.json
+            entities: 'page',      // /data/<entry.name>.page.json
+        },
+    })
+```
+
+What gets used when:
+
+- **`live({id})` / `useDocument(id)`** — if `data.entities` is set, the SDK fetches the matching per-entity file instead of calling the API. Needs `data.catalog` to be loaded too, so the entry's `name` is known. The SSE subscribe still opens for live updates.
+- **`live(filter, onChange, options)` with no filter** / **`listAll()`** — if `data.catalog` is set, the SDK consults `/data/<catalog>.json` for first paint, then opens SSE.
+- **`list()`**, **`urlFor()`**, **`query()`**, **`update()`**, **`delete()`**, **`render()`**, **`subscribe()`** — unchanged, always hit the API.
+- **Any file fetch failure** — falls back to the live API for that call. No separate flag.
+
+The data plugin emits each entry as `{ refId, name, date, data: {...picked} }`. The SDK strips that wrapper automatically — `onChange` and `listAll` always see plain payloads regardless of source.
+
+This is **not** a cache. The data plugin's files are only consulted for the initial fill; ongoing changes come over SSE on the actual API endpoint. For runtime fail-safety on the live API, that's what the api plugin's `cache: true` is for — see [mikser-io's caching docs](https://github.com/almero-digital-marketing/mikser-io/blob/main/documentation/caching.md).
+
+**Edge case: first-paint flash for fast-changing snapshots.** First paint renders whatever the data plugin last wrote — which may be N seconds old — and the SSE stream then arrives and reconciles any changes since. For a route table that's invisible; routes don't move every second. For snapshots of fast-changing data (recent activity, in-stock badges, live counters) the user may briefly see stale content before SSE catches up. If that flash is visible UX, either design for it (subtle "syncing" indicator until the first SSE event arrives, or render a loading state when the snapshot age exceeds your tolerance) or skip `data.catalog` / `data.entities` for that particular client — paying the API roundtrip is the right tradeoff when the response can't be allowed to be stale.
+
+**Snapshot-bypass warning.** Snapshots only apply when the `live()` / `listAll()` call is trivial — no filter, no sort, no skip. Add any of those and the SDK silently falls back to the live API. Since that's the kind of thing a developer can change without noticing, the SDK emits a one-time `console.warn` per `(endpoint, call kind, what-was-set)` shape:
+
+```
+[mikser-sdk] data.catalog is set on "public" but this live() call uses filter+sort
+  — snapshot bypassed, falling back to live list().
+  Snapshots only apply when the call is trivial (no filter/sort/skip).
+  Either remove the filter+sort from this call, or accept the API roundtrip if
+  filtering is intentional.
+  Suppress: pass { quiet: true } on the call, or set MIKSER_QUIET=1.
+```
+
+Same suppression channels as the wide-list warning above.
+
+### Dev-mode warning: accidentally wide queries
+
+The common failure mode for a CMS-backed app is "I just wanted a nav menu but pulled every full document over the wire." To catch it at write time, `list()` and `live()` emit a one-time dev-mode `console.warn` when a response has more than 50 items and the query has no `fields:` projection:
+
+```
+[mikser-sdk] list() returned 247 items (~4.2 MB) from "public" with no `fields` projection.
+  Add { fields: [...] } to narrow it, or — if this query runs on every page load —
+  move it to a `data.catalog.<name>` snapshot on the mikser side and load via:
+    entities('public', { data: { catalog: '<name>' } })
+  Suppress: pass { quiet: true } on the call, or set MIKSER_QUIET=1.
+```
+
+The warning is deduped per `(endpoint, filter, sort)` shape, so an SSE-driven list that updates 30 times only fires once. It's silenced when `process.env.NODE_ENV === 'production'`, when `MIKSER_QUIET` is set, or per-call via `{ quiet: true }` on `list(query, opts)` / `live(filter, onChange, opts)`.
+
+The mikser-io server emits a matching warning in its logs for the same shape — useful when the wide query came from curl, another SDK, or an environment where `console` isn't visible.
 
 ### `list(query)` — body-based
 

package/index.d.ts

@@ -16,6 +16,37 @@ export interface ClientOptions {
 export interface EntityOptions {
     /** Bearer token sent on every request to this endpoint. */
     token?: string
+    /**
+     * Mirrors the mikser-io `data` plugin's config block. The SDK
+     * consumes the static JSON files the data plugin writes under
+     * `out/data/` so first-paint reads come from disk (CDN-cacheable)
+     * instead of hitting the live API.
+     *
+     * The names are the same as on the server:
+     *   - `data.catalog`  pairs with `data.catalog.<name>`  on mikser
+     *   - `data.entities` pairs with `data.entities.<name>` on mikser
+     *
+     * On a fetch failure the SDK silently falls back to the live API
+     * for that call — no separate flag.
+     */
+    data?: DataOptions
+}
+
+export interface DataOptions {
+    /**
+     * Name of a `data.catalog.<name>` block on the server. The SDK
+     * loads `/data/<name>.json` (one combined file) on first paint;
+     * `live()` and `listAll()` consult it before hitting the API.
+     */
+    catalog?: string
+    /**
+     * Name of a `data.entities.<name>` block on the server. When
+     * `live({id})` (the shape `useDocument` issues) fires, the SDK
+     * loads `/data/<entry.name>.<name>.json` (one file per entity)
+     * instead of calling the API. Requires `data.catalog` so the
+     * `entry.name` mapping is available.
+     */
+    entities?: string
 }
 
 /**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mikser-io-sdk-api",
-  "version": "2.4.0",
+  "version": "3.0.0",
   "description": "Client SDK for mikser-io's api plugin — query the document catalog from the browser or Node",
   "main": "index.js",
   "types": "index.d.ts",

package/src/entities.js

@@ -7,20 +7,234 @@ import { bearer, jsonOrThrow } from './http.js'
 import { joinUrl, sortToParam, filterToParams } from './url.js'
 import { parseSseEvent } from './sse.js'
 
+// Conservative URL-length ceiling for the GET form of list(). Real
+// browsers and proxies vary (Chrome ~32k, IIS ~16k, nginx default
+// 8k, some CDNs 4k), but the safest interop floor for list-with-
+// many-filter-params is well under 2k. Queries past this fall back
+// to POST automatically.
+const GET_MAX_URL = 1800
+
+// Wide-list warning — surfaces in dev mode when a list/live call
+// returns more than this without a `fields` projection. Catches the
+// common "I just wanted a nav menu but pulled every full document"
+// failure mode at the first place it manifests (developer's DevTools
+// console). Server-side has a matching warning that fires for all
+// clients regardless of SDK use.
+const WIDE_RESPONSE_ITEMS = 50
+const _warnedShapes = new Set()
+
+function isProductionEnv() {
+    try {
+        return typeof process !== 'undefined'
+            && process.env?.NODE_ENV === 'production'
+    } catch { return false }
+}
+function isQuiet() {
+    try {
+        return typeof process !== 'undefined' && process.env?.MIKSER_QUIET
+    } catch { return false }
+}
+
+function maybeWarnWide({ endpoint, query, envelopeOrItems, quiet }) {
+    if (quiet || isProductionEnv() || isQuiet()) return
+    const items = Array.isArray(envelopeOrItems)
+        ? envelopeOrItems
+        : envelopeOrItems?.items
+    if (!Array.isArray(items) || items.length <= WIDE_RESPONSE_ITEMS) return
+    const hasFields = Array.isArray(query?.fields) && query.fields.length > 0
+    if (hasFields) return
+    const shape = `${endpoint}|${JSON.stringify(query?.filter ?? null)}|${JSON.stringify(query?.sort ?? null)}`
+    if (_warnedShapes.has(shape)) return
+    _warnedShapes.add(shape)
+    let sizeNote = ''
+    try {
+        const bytes = JSON.stringify(items).length
+        sizeNote = bytes >= 1024 * 1024
+            ? ` (~${(bytes / 1024 / 1024).toFixed(1)} MB)`
+            : ` (~${Math.round(bytes / 1024)} KB)`
+    } catch { /* size note is best-effort */ }
+    console.warn(
+        `[mikser-sdk] list() returned ${items.length} items${sizeNote} from "${endpoint}" with no \`fields\` projection.\n` +
+        `  Add { fields: [...] } to narrow it, or — if this query runs on every page load —\n` +
+        `  move it to a \`data.catalog.<name>\` snapshot on the mikser side and load via:\n` +
+        `    entities('${endpoint}', { data: { catalog: '<name>' } })\n` +
+        `  Suppress: pass { quiet: true } on the call, or set MIKSER_QUIET=1.`,
+    )
+}
+
+// Snapshot-bypass warning — fires when `data.catalog` is configured
+// but the call is non-trivial (has filter / sort / skip), so the
+// snapshot can't be used and the SDK falls back to the live API. The
+// bypass is correct behavior, but it's silent: developers often set
+// data.catalog once and then add a sort to one of their useDocuments
+// calls without noticing the snapshot is no longer involved. Deduped
+// per (endpoint, kind, what-was-set) so a page with 3 filtered calls
+// produces 3 warnings, not 30.
+const _bypassedShapes = new Set()
+function maybeWarnSnapshotBypass({ endpoint, kind, filter, sort, skip, quiet }) {
+    if (quiet || isProductionEnv() || isQuiet()) return
+    const reasons = []
+    if (filter) reasons.push('filter')
+    if (sort)   reasons.push('sort')
+    if (skip != null) reasons.push('skip')
+    if (reasons.length === 0) return
+    const reasonLabel = reasons.join('+')
+    const shape = `${endpoint}|${kind}|${reasonLabel}`
+    if (_bypassedShapes.has(shape)) return
+    _bypassedShapes.add(shape)
+    const fallback = kind === 'live' ? 'live list()' : 'paginated fetch'
+    console.warn(
+        `[mikser-sdk] data.catalog is set on "${endpoint}" but this ${kind}() call uses ${reasonLabel} — snapshot bypassed, falling back to ${fallback}.\n` +
+        `  Snapshots only apply when the call is trivial (no filter/sort/skip).\n` +
+        `  Either remove the ${reasonLabel} from this call, or accept the API roundtrip if filtering is intentional.\n` +
+        `  Suppress: pass { quiet: true } on the call, or set MIKSER_QUIET=1.`,
+    )
+}
+
 export function createEntitiesClient({ baseUrl, basePath, fetch: doFetch, headers: defaultHeaders }) {
-    return function entities(name, { token } = {}) {
+    return function entities(name, opts = {}) {
+        const {
+            token,
+            // `data` mirrors the mikser-io `data` plugin's config block:
+            //
+            //   data: {
+            //       catalog:  'sitemap',   // pairs with data.catalog.sitemap
+            //       entities: 'page',      // pairs with data.entities.page
+            //   }
+            //
+            // On the server the data plugin writes:
+            //   - catalog.<name>  → out/data/<name>.json                   (one combined file)
+            //   - entities.<name> → out/data/<entity.name>.<name>.json     (one file per entity)
+            //
+            // On the client:
+            //   - `data.catalog` makes live() / listAll() consult
+            //     /data/<this>.json on first paint, falling back to a
+            //     fresh list() if the file is missing.
+            //   - `data.entities` makes live({id}) consult the per-entity
+            //     file /data/<entry.name>.<this>.json, falling back to a
+            //     fresh list({filter:{id}}) call. Requires `data.catalog`
+            //     to be loaded so the entity's `name` is known (the
+            //     mapping comes from the catalog wrapper, not the id).
+            //
+            // Default URL prefix is /data/ to match the data plugin's
+            // default `dataFolder`. If a project customizes `data.dataFolder`
+            // server-side, add a matching prefix here — but for now this
+            // is hardcoded.
+            data: dataConfig = {},
+        } = opts
+        const { catalog: catalogName, entities: entitiesName } = dataConfig
         const endpointBase = `${basePath}/${name}`
         const queryUrl     = joinUrl(baseUrl, `${endpointBase}/entities/query`)
         const listUrl      = joinUrl(baseUrl, `${endpointBase}/entities`)
         const subscribeUrl = joinUrl(baseUrl, `${endpointBase}/entities/subscribe`)
         const renderUrl    = joinUrl(baseUrl, `${endpointBase}/render`)
 
+        // Default URL prefix matches the data plugin's default folder.
+        const DATA_PREFIX = '/data'
+
+        const catalogUrl = catalogName
+            ? joinUrl(baseUrl, `${DATA_PREFIX}/${catalogName}.json`)
+            : null
+
+        // id → entity.name index, populated when the catalog snapshot
+        // loads. Lets `live({id})` derive the per-entity file URL
+        // without re-fetching the catalog or guessing.
+        const nameById = new Map()
+
+        // Cached snapshot promise — concurrent live() / listAll() calls
+        // share one fetch. Cleared on error so a flaky connection can
+        // recover on the next call.
+        let snapshotPromise = null
+        async function loadSnapshot() {
+            if (!catalogUrl) return null
+            if (snapshotPromise) return snapshotPromise
+            snapshotPromise = (async () => {
+                try {
+                    const res = await doFetch(catalogUrl, {
+                        method: 'GET',
+                        headers: { accept: 'application/json', ...defaultHeaders },
+                    })
+                    if (!res.ok) return null
+                    const payload = await res.json()
+                    return unwrapSnapshot(payload, nameById)
+                } catch (err) {
+                    // Clear the cached promise so the next call can retry.
+                    snapshotPromise = null
+                    return null
+                }
+            })()
+            return snapshotPromise
+        }
+
+        // Per-entity file fetch — for live({id}) when data.entities is
+        // configured. Always returns either the entity or null (null
+        // means "fall back to the API"). Never throws.
+        async function loadEntityFile(id) {
+            if (!entitiesName) return null
+            const entityName = nameById.get(id)
+            if (!entityName) {
+                // The catalog isn't loaded yet, or this id wasn't in it.
+                // Trigger a catalog load lazily; the next call will hit
+                // the populated map.
+                await loadSnapshot()
+                if (!nameById.has(id)) return null
+            }
+            const url = joinUrl(baseUrl, `${DATA_PREFIX}/${nameById.get(id)}.${entitiesName}.json`)
+            try {
+                const res = await doFetch(url, {
+                    method: 'GET',
+                    headers: { accept: 'application/json', ...defaultHeaders },
+                })
+                if (!res.ok) return null
+                const payload = await res.json()
+                // Per-entity files are single-object wrappers:
+                //   { refId, name, date, data: {...} }
+                // unwrapEntityFile pulls out `data`.
+                return unwrapEntityFile(payload)
+            } catch {
+                return null
+            }
+        }
+
         /**
-         * Body-based query. Send everything sift accepts —
-         * $and / $or / $regex, projections, sorts. Returns the standard
-         * envelope: { items, page, limit, total, totalPages, hasNext, hasPrev }.
+         * Run a list query. Returns the standard envelope:
+         * { items, page, limit, total, totalPages, hasNext, hasPrev }.
+         *
+         * Transport selection: tries GET first because GET responses
+         * are what the api plugin's per-query disk cache writes (and
+         * what a reverse proxy can serve as failover when mikser is
+         * down). Falls back to POST when:
+         *   - the encoded URL exceeds GET_MAX_URL (browsers and most
+         *     proxies start refusing past ~2KB; we use a conservative
+         *     limit of 1800)
+         *   - GET is explicitly disabled via opts.method = 'POST'
+         *
+         * GET can express anything sift accepts via the `.$op` URL-param
+         * suffix scheme (see urlFor). Use POST when you want to be
+         * explicit about not engaging the cache path, e.g. for queries
+         * with secrets in the body that shouldn't sit in proxy logs.
          */
-        async function list(query = {}) {
+        async function list(query = {}, opts = {}) {
+            const forcePost = opts.method === 'POST'
+
+            if (!forcePost) {
+                const url = urlFor(query)
+                if (url.length <= GET_MAX_URL) {
+                    const res = await doFetch(url, {
+                        method: 'GET',
+                        headers: {
+                            accept: 'application/json',
+                            ...defaultHeaders,
+                            ...bearer(token),
+                        },
+                    })
+                    const envelope = await jsonOrThrow(res, url)
+                    maybeWarnWide({ endpoint: name, query, envelopeOrItems: envelope, quiet: opts.quiet })
+                    return envelope
+                }
+            }
+
+            // POST fallback — long URLs and forced POST land here.
             const res = await doFetch(queryUrl, {
                 method: 'POST',
                 headers: {
@@ -30,7 +244,9 @@ export function createEntitiesClient({ baseUrl, basePath, fetch: doFetch, header
                 },
                 body: JSON.stringify(query),
             })
-            return jsonOrThrow(res, queryUrl)
+            const envelope = await jsonOrThrow(res, queryUrl)
+            maybeWarnWide({ endpoint: name, query, envelopeOrItems: envelope, quiet: opts.quiet })
+            return envelope
         }
 
         /**
@@ -78,6 +294,27 @@ export function createEntitiesClient({ baseUrl, basePath, fetch: doFetch, header
          * is wasteful. Use pages() directly and stream-process there.
          */
         async function listAll(query = {}) {
+            // Snapshot fast path: if no filter/sort/skip is requested
+            // beyond what the snapshot was built with, return the
+            // pre-built array directly. The caller's filter/sort would
+            // require re-querying the live endpoint anyway, so any
+            // non-trivial query falls through to the paginated fetch.
+            const trivial = !query.filter && !query.sort && !query.skip
+            if (trivial && catalogUrl) {
+                const snapshot = await loadSnapshot()
+                if (snapshot) {
+                    return query.fields
+                        ? snapshot.map(item => pickFields(item, query.fields))
+                        : snapshot
+                }
+                // Snapshot unavailable — fall through to paginated fetch.
+            }
+            if (!trivial && catalogUrl) {
+                maybeWarnSnapshotBypass({
+                    endpoint: name, kind: 'listAll',
+                    filter: query.filter, sort: query.sort, skip: query.skip,
+                })
+            }
             const items = []
             for await (const env of pages({ limit: 1000, ...query })) {
                 items.push(...env.items)
@@ -170,6 +407,7 @@ export function createEntitiesClient({ baseUrl, basePath, fetch: doFetch, header
         function live(filter, onChange, options = {}) {
             const {
                 sort, fields, limit, skip,
+                quiet,
                 signal: externalSignal,
                 onError = (err) => console.error('mikser-io-sdk-api live error:', err),
             } = options
@@ -185,10 +423,65 @@ export function createEntitiesClient({ baseUrl, basePath, fetch: doFetch, header
 
             const loop = (async () => {
                 try {
-                    const env = await list({ filter, sort, fields, limit, skip })
-                    if (disposed || ac.signal.aborted) return
-                    items = env.items
-                    onChange(items)
+                    let usedFastPath = false
+                    const trivial = !filter && !sort && !skip
+
+                    // Per-entity file fast path. When the call is a
+                    // single-id lookup (the shape useDocument issues)
+                    // and `data.entities` is configured, fetch the
+                    // pre-built file the data plugin wrote instead of
+                    // calling the API. Falls back to list() if the
+                    // file isn't there.
+                    const isSingleIdLookup = (
+                        entitiesName &&
+                        filter && typeof filter === 'object' &&
+                        Object.keys(filter).length === 1 &&
+                        'id' in filter && filter.id != null
+                    )
+                    if (isSingleIdLookup) {
+                        const entity = await loadEntityFile(filter.id)
+                        if (entity) {
+                            if (disposed || ac.signal.aborted) return
+                            items = fields ? [pickFields(entity, fields)] : [entity]
+                            onChange(items)
+                            usedFastPath = true
+                        }
+                    }
+
+                    // Catalog snapshot fast path. Used only when the
+                    // call is trivial enough that the pre-built array
+                    // reflects what list() would return — anything
+                    // more specific (filter, sort, skip) goes through
+                    // list() so the caller's intent is honored.
+                    if (!usedFastPath && trivial && catalogUrl) {
+                        const snapshot = await loadSnapshot()
+                        if (snapshot) {
+                            if (disposed || ac.signal.aborted) return
+                            items = fields ? snapshot.map(i => pickFields(i, fields)) : snapshot
+                            onChange(items)
+                            usedFastPath = true
+                        }
+                    }
+
+                    // Bypass warning for cases where the user opted into
+                    // a snapshot but their call doesn't fit either fast
+                    // path. The single-id lookup is its own valid shape,
+                    // so don't warn for it.
+                    if (!usedFastPath && !isSingleIdLookup && !trivial && catalogUrl) {
+                        maybeWarnSnapshotBypass({
+                            endpoint: name, kind: 'live',
+                            filter, sort, skip, quiet,
+                        })
+                    }
+
+                    if (!usedFastPath) {
+                        // Pass { quiet } so list()'s wide-warning honors
+                        // the live() caller's quiet opt.
+                        const env = await list({ filter, sort, fields, limit, skip }, { quiet })
+                        if (disposed || ac.signal.aborted) return
+                        items = env.items
+                        onChange(items)
+                    }
 
                     for await (const event of watch({ filter }, { signal: ac.signal })) {
                         if (disposed) return
@@ -288,3 +581,79 @@ export function createEntitiesClient({ baseUrl, basePath, fetch: doFetch, header
         return { list, listAll, urlFor, pages, watch, live, update, delete: remove, render }
     }
 }
+
+// Unwrap a snapshot payload into a plain array of entity-like objects.
+// Recognises three shapes:
+//   - data-plugin catalog output:  [{ refId, name, date, data }, ...]
+//   - plain array of entities:     [{ id, meta, ... }, ...]
+//   - list() envelope:             { items: [...], page, ... }
+// Returns null on unrecognised shapes so the caller can decide to fall
+// back to a fresh list() call.
+function unwrapSnapshot(payload, nameById) {
+    if (Array.isArray(payload)) {
+        if (payload.length === 0) return []
+        // Heuristic: data-plugin entries have `refId` + `data`; treat any
+        // object with `data` as a wrapped entry and unwrap. Anything else
+        // is a plain array.
+        if (payload[0] && typeof payload[0] === 'object' && 'data' in payload[0]) {
+            return payload
+                .map(entry => {
+                    if (!entry || !entry.data) return null
+                    // Side-table: stash entity.name keyed by entity.id so
+                    // live({id}) can compute per-entity file URLs later.
+                    if (nameById && entry.data.id != null && entry.name != null) {
+                        nameById.set(entry.data.id, entry.name)
+                    }
+                    return entry.data
+                })
+                .filter(Boolean)
+        }
+        return payload
+    }
+    if (payload && typeof payload === 'object' && Array.isArray(payload.items)) {
+        return payload.items
+    }
+    return null
+}
+
+// Per-entity files are single-object wrappers — `{ refId, name, date,
+// data: {...} }` — written by `data.entities.<name>` on the server.
+// Returns the unwrapped data, or null if the shape isn't recognised.
+function unwrapEntityFile(payload) {
+    if (!payload || typeof payload !== 'object') return null
+    if ('data' in payload && payload.data && typeof payload.data === 'object') {
+        return payload.data
+    }
+    // Plain entity (someone hand-wrote a JSON file, no wrapper) — accept it.
+    if ('id' in payload) return payload
+    return null
+}
+
+// Project an entity object to only the fields requested. Dotted paths
+// supported ("meta.title" → object with `meta.title` set, other meta
+// keys dropped). Used by the snapshot fast paths in list/live so a
+// caller asking for narrow fields still gets narrow data from the
+// snapshot — saves a re-fetch and keeps memory/bundles smaller.
+function pickFields(entity, fields) {
+    const out = {}
+    for (const field of fields) {
+        const parts = field.split('.')
+        let src = entity, dst = out
+        for (let i = 0; i < parts.length - 1; i++) {
+            const part = parts[i]
+            if (src == null || typeof src !== 'object') { src = undefined; break }
+            src = src[part]
+            if (dst[part] == null || typeof dst[part] !== 'object') dst[part] = {}
+            dst = dst[part]
+        }
+        if (src !== undefined) {
+            const last = parts[parts.length - 1]
+            if (src != null && typeof src === 'object' && last in src) {
+                dst[last] = src[last]
+            } else if (parts.length === 1 && entity[last] !== undefined) {
+                out[last] = entity[last]
+            }
+        }
+    }
+    return out
+}