domma-cms 1.6.59 → 1.7.0

package/docs/superpowers/specs/2026-10-09-content-api-design.md

@@ -0,0 +1,325 @@
+# Content API - headless read access to pages, menus and site content - design
+
+**Date:** 2026-10-09
+**Status:** Implemented on `feat/content-api` (unreleased) - see `docs/superpowers/plans/2026-10-09-content-api.md`
+**Builds on:** the external API surface (`/api/v1/:slug`, API tokens, API Builder) - see the
+`domma-api-surface` skill
+
+Adds a versioned, externally consumable **Content API** beside the existing **Data API**
+(`/api/v1/:slug`). The Data API exposes collection entries; the Content API exposes what the CMS
+*publishes*: pages, menus, a safe subset of site settings, and search. Its first consumers are
+headless front ends (a Next/Astro/mobile client), a static-site exporter and the Manager.
+
+## Goal
+
+1. A third-party client can list and fetch published pages, with frontmatter, as Markdown, rendered
+   HTML or both.
+2. **One access decision.** A page reachable over the API is reachable on the public site by the
+   same viewer, and the reverse - no second implementation of drafts, visibility or the project
+   kill-switch.
+3. The same credentials model as the Data API: public, a role (JWT), or a project-bound `dcms_`
+   token - with the same strictness rules.
+4. Cached and invalidated by the existing tag cache, with no new invalidation paths.
+
+Non-goals for v1: writes (create/update/delete pages), media upload, block/component source,
+theme CSS. See [Phase 2](#phase-2---writes).
+
+## What already exists
+
+Verified against 0.116.8:
+
+- `getPage(urlPath, {user})` / `listPages()` (`server/services/content.js`) return frontmatter,
+  `urlPath`, raw `content`, rendered `html`, `usedComponents`, `cacheTags`, `perViewer` - through
+  the same `parseMarkdown` pipeline the public site uses. `listPages()` also stamps
+  `resolvedProject`.
+- The full public access decision lives inline in the `/*` route at `server/routes/public.js:177`:
+  project kill-switch → 404, `status !== 'published'` → 404 unless `canPreviewDrafts`,
+  `checkVisibility()` → 403, cache key `page:<url>:m<mtime>:role<roles>:o<baseUrl>` with tags
+  `page:<url>`, `nav`, `site` plus the page's own.
+- `resolveViewer(request)` (`server/services/viewerSession.js`) - cookie/JWT viewer resolution.
+- `resolveLocation(slot, user, ctx)` (`server/services/menus.js`) - bound menu for a slot, items
+  already filtered for the viewer.
+- `searchPages(pages, query, {viewer, exclusions})` + `isSearchable()` (`server/services/search.js`).
+- `validateToken()`, `scopeAllows()` and `validateScopes()` (`server/services/apiTokens.js`) -
+  scopes are `{collection, verbs[]}` only.
+- `checkPublicAccess()` (`server/routes/api/collections.js`) - the token/role/public gate, keyed off
+  `schema.api.<verb>`. Pages have no schema, so it cannot be reused as-is.
+- The `onSend` hook in `server/server.js` that marks `/api/*` `no-store` already exempts
+  `/api/v1/` and `/api/x/`; the Content API needs the same exemption.
+
+## Findings that shape the design
+
+1. **`/api/v1/pages` is not free.** `/api/v1/:slug` matches any collection slug, and `slugify()`
+   happily produces `pages`, `menus` or `content`. Fastify's static-beats-param rule would silently
+   shadow an existing collection of that name. The Content API therefore gets its own prefix.
+2. **The access decision is not extractable by import** - it is inline in a route handler that
+   writes HTML replies. It must be lifted into a service first, or the API will drift from the site.
+3. **Two draft rules exist today.** `public.js` hides `status !== 'published'`; `isSearchable()`
+   hides only `status === 'draft'`. Any other status (e.g. `scheduled`, `archived`) is hidden on the
+   site but searchable. The Content API uses the public rule, and search should be brought into
+   line in the same change.
+4. **Rendered HTML is Domma-dependent.** Tabs, carousels, accordions, `[collection]` displays and
+   effects emit markup that needs Domma's CSS/JS (and `__CMS_DCONFIG__`) to behave. Clients need to
+   be told, not left to discover it.
+5. **Frontmatter is open-ended.** Plugins and authors add keys freely; `dconfig`, `sidebar` and
+   `seo` carry implementation detail. Output must be an allowlist, never a spread.
+6. **`site.json` holds SMTP credentials.** No endpoint may serialise it wholesale - including the
+   `__CMS_SITE__` object, whose shape should be checked before being reused as the settings payload.
+7. **Per-viewer renders exist.** `[menu]` filters by role and a row-scoped `[view]` renders each
+   user their own rows (`perViewer`). A token has no user.
+
+## URL surface
+
+Prefix: **`/api/content/v1`** (unused today; no collection slug can reach it).
+
+| Method | Path                               | Returns |
+|--------|------------------------------------|---------|
+| `GET`  | `/api/content/v1/pages`            | Paged list of page summaries |
+| `GET`  | `/api/content/v1/pages/*`          | One page by URL path (`/pages/` = home, `/pages/services/web-dev`) |
+| `GET`  | `/api/content/v1/menus/:location`  | The menu bound to a location, filtered for the caller |
+| `GET`  | `/api/content/v1/site`             | Public site settings subset |
+| `GET`  | `/api/content/v1/search?q=`        | Search results |
+
+### Pages list
+
+Query: `page`, `limit` (default 50, max 200 - and `limit=0` means the default, matching
+`/api/v1`), `prefix` (URL path prefix, e.g. `/services`), `tag`, `category`, `project`, `sort`
+(`sortOrder` default | `title` | `updatedAt` | `createdAt`), `order`.
+
+Response mirrors the Data API list shape:
+
+```json
+{
+  "pages": [
+    { "urlPath": "/about", "title": "About", "description": "…", "tags": ["company"],
+      "category": "general", "layout": "default", "updatedAt": "2026-09-30T10:12:00.000Z" }
+  ],
+  "total": 14, "page": 1, "limit": 50
+}
+```
+
+Summaries never include a body - the list must stay cheap enough to run over every page without
+rendering any.
+
+### Single page
+
+`GET /api/content/v1/pages/about?format=html|markdown|both` (default `html`).
+
+```json
+{
+  "urlPath": "/about",
+  "title": "About",
+  "description": "…",
+  "layout": "default",
+  "tags": ["company"],
+  "category": "general",
+  "seo": { "title": "…", "description": "…", "image": "…" },
+  "createdAt": "…", "updatedAt": "…",
+  "html": "<section>…</section>",
+  "markdown": "## About us\n…",
+  "requires": { "domma": true, "components": ["tabs", "carousel"] },
+  "perViewer": false
+}
+```
+
+- `html` is the body only (`page.html`), not the full rendered document with navbar and footer.
+- `requires.components` names the Domma markup in the body that `public/js/site.js` initialises
+  (tabs, accordion, carousel, countdown, tooltip, reveal, …), plus any custom `[component]`s.
+  `usedComponents` alone is not enough: it lists only custom components, and a page of tabs
+  reports none. `requires.domma` is true when the list is non-empty, so a client can decide
+  whether to load Domma or render the Markdown itself.
+- `markdown` is the raw source. It contains shortcodes, which are documented in
+  `docs/markdown-shortcodes.md`; a client choosing Markdown takes on interpreting them.
+- `seo` is allowlisted (`title`, `description`, `image`, `canonical`, `noindex`) - not passed through.
+
+Resolution follows the public route: `/services` falls back to `/services/index`.
+
+### Menus
+
+`GET /api/content/v1/menus/navbar?page=/about` - `resolveLocation(slot, viewer, {urlPath})`.
+Returns `{location, slug, name, items}` with items in the same nested `items` shape the navbar
+reads, each item allowlisted (`type`, `text`, `url`, `icon`, `target`, `size`, `badge`, `items`) -
+none of its gating. Structural items (`separator`, `spacer`) are kept: a client rendering them is
+better than indices shifting. The `admin-sidebar` and `overlay` slots, and any slot starting
+`admin`, are never served (404).
+
+### Site
+
+A fixed allowlist built in one function (`publicSiteSettings()`), never a filtered copy of
+`site.json`, holding exactly what every public page already shows: `title`, `brand` (text, logo,
+url), `footer` (hidden links removed, as the renderer does), `social`, the SEO defaults in the meta
+tags (`defaultTitle`, `titleSeparator`, `defaultDescription`) and the `theme` id. `tagline` and
+`url` never reach a public page, so they are left out. Anything added later is added to the
+allowlist explicitly.
+
+### Search
+
+`GET /api/content/v1/search?q=…&limit=10` - `searchPages(listPages(), q, {viewer, exclusions})`,
+returning `{results: [{urlPath, title, excerpt, score}]}`. Respects `config/search.json`
+`enabled: false` (404).
+
+## Access
+
+### Configuration
+
+New file **`config/content-api.json`** (yours; absence = disabled):
+
+```json
+{
+  "enabled": true,
+  "access": "public",
+  "resources": {
+    "pages":  { "enabled": true },
+    "menus":  { "enabled": true },
+    "site":   { "enabled": true },
+    "search": { "enabled": true }
+  },
+  "formats": ["html", "markdown"],
+  "listable": true
+}
+```
+
+- `access` is `"public"`, `"token"` or a role name - the same three modes and the same ladder /
+  unknown-role-fails-closed rule as `schema.api.<verb>.access`.
+- `formats` lets a site refuse Markdown (shortcode source can reveal collection slugs, view names
+  and form ids that the HTML does not).
+- `listable: false` keeps single-page fetch but turns the list off, for sites that do not want
+  their full URL inventory enumerable.
+- **Off by default.** An upgrade must not expose anything a site has not chosen to.
+
+Edited in the admin at **System → API Tokens**, on a new "Content API" tab, or a small screen of
+its own - see [Open questions](#open-questions).
+
+### The gate
+
+`checkContentAccess(resource, request, reply)` - a sibling of `checkPublicAccess`, reading
+`config/content-api.json` instead of a schema. The token branch is shared, not copied: factor the
+Bearer-parse → `validateToken` → project check → scope check out of `checkPublicAccess` into
+`authenticateApiToken(request, {project, scope})`, used by both gates.
+
+Strictness carries over unchanged: a token never satisfies a role mode, a JWT never satisfies
+token mode.
+
+### Tokens and projects
+
+- A token is bound to one project. For pages, the project is `getProjectForPage()`; a token for
+  `world-cup` lists and fetches only pages resolving to `world-cup` (`core` tokens see `core`
+  pages) - single pages, the list and search results alike; a page in another project is `404`.
+  Menus and site settings are site-wide and already shown on every public page, so any valid
+  token may read them. The gate binds no project itself (`authenticateApiToken` without
+  `project`); the routes bind per page.
+- **Scopes gain a second shape:** `{ "content": "pages" | "menus" | "site" | "search" }`. A scope
+  is `{collection, verbs}` OR `{content}`. `validateScopes()` accepts both; `scopeAllows()` gains a
+  `contentScopeAllows(scopes, resource)` sibling. Empty scopes still mean "everything in the
+  project" - which now includes content. That widens existing unscoped tokens, so it is gated on
+  the site enabling `access: "token"` - an unscoped token gets content only on sites that opted in.
+
+### Viewer and per-viewer content
+
+- Public and token callers render as **anonymous** (`user: null`). Role-mode callers render as
+  themselves.
+- Role mode accepts a Bearer **access** JWT only (a refresh token is refused) - never the
+  `dm_session` cookie, which only the page renderer reads, so no API answers to ambient
+  credentials.
+- A page with `perViewer: true` is served as rendered for that viewer, and the flag is returned so
+  a client knows the body is not shareable between users.
+- Drafts: never served in public or token mode. In role mode a caller holding `pages.read` may
+  pass `?preview=1` to receive drafts, mirroring `canPreviewDrafts` - responses are then
+  `no-store` and never cached.
+
+## The shared page resolver
+
+Lift the decision out of `public.js` into `server/services/pagePublishing.js`:
+
+```js
+/**
+ * @returns {Promise<
+ *   {status: 'ok', page, project, isDraft} |
+ *   {status: 'not-found'} | {status: 'forbidden'}
+ * >}
+ */
+export async function resolvePublishedPage(urlPath, viewer, {allowDrafts = false} = {})
+```
+
+It does the index fallback, the project kill-switch, the draft rule and `checkVisibility()`, and
+returns a verdict - no replies, no HTML. `public.js` keeps its rendering and cache key and calls
+this for the decision; the Content API calls it too. **This refactor lands first, on its own, with
+the existing public-route tests passing unchanged** - it is the step that makes "one access
+decision" true rather than aspirational.
+
+`listPages()` filtering for the list endpoint uses the same predicate (`isPubliclyListed(page,
+viewer)`), and `isSearchable()` is changed to call it - closing finding #3.
+
+## Caching
+
+- **No server-side response cache.** The access decision has to parse the page on every request -
+  exactly as the site's does - and the JSON is built straight from that parse, so caching the
+  payload would save nothing. (The site caches its *rendered document*, which is the expensive
+  part; the API has no document.)
+- Page writes clear a `pages:list` tag (not `pages`, which `cache-invalidation.test.js` pins as
+  carrying nothing), ready for when a list cache is worth having.
+- Token responses: `Cache-Control: private, no-cache` so intermediaries never share a
+  token-gated body.
+- Public responses: `Cache-Control: public, max-age=60` plus a weak `ETag` from mtime.
+- Role and preview responses: `no-store`.
+- Refusals (404/401/403/400): `no-store`, set by a hook scoped to the Content API routes - a 404
+  with no `Cache-Control` is heuristically cacheable and would outlive switching the API on.
+- `server.js` `onSend`: add `/api/content/v1/` to the exempt prefixes.
+
+## Errors
+
+Same as the Data API, with no oracle:
+
+| Case | Status |
+|------|--------|
+| Content API disabled, resource disabled, page missing, draft, disabled project | `404` |
+| Token/JWT missing or invalid | `401` |
+| Token wrong project or scope, visibility refused | `403` - except a page outside the token's project, which is `404` (the project is the token's whole world) |
+| Format not allowed | `400` |
+
+A visibility-gated page answers `403` exactly as the site does, so the API leaks no more than the
+site already does.
+
+## Testing
+
+- `pagePublishing.js` unit tests: every verdict, including index fallback, unknown role in
+  visibility, `=role` exact match, disabled project, non-`published` statuses.
+- Public route regression: the existing suite passes unchanged after the extraction.
+- Parity test: for a fixture site, every page × {anon, user, admin} gets the same allow/deny from
+  `GET /about` and `GET /api/content/v1/pages/about`.
+- Token tests: project binding, `{content}` scopes, unscoped token with Content API off/on,
+  JWT refused in token mode.
+- Leakage tests: unknown frontmatter keys never appear; `site` never contains `smtp`; Markdown
+  refused when `formats` excludes it.
+- Cache: a page save changes the next API response; a token response carries `private`.
+
+## Build order
+
+1. Extract `resolvePublishedPage` / `isPubliclyListed`; switch `public.js` and `isSearchable()`.
+2. Factor `authenticateApiToken()` out of `checkPublicAccess()`.
+3. `config/content-api.json` loader + `checkContentAccess()`.
+4. Routes (`server/routes/api/content-public.js`), pages first, then menus, site, search.
+5. Token scope shape `{content}` - service, admin token editor, scaffolder `apiTokens` block.
+6. Admin settings UI.
+7. Docs: `docs/api-reference.md` section, in-admin API reference page, `domma-api-surface` skill,
+   `FEATURES.md`, `CHANGELOG.md`.
+
+## Phase 2 - writes
+
+Not in v1, recorded so v1 does not paint it out:
+
+- `POST/PUT/DELETE /api/content/v1/pages/*` through `createPage` / `updatePage` / `deletePage`,
+  so versioning (`content/versions`), audit and cache invalidation come for free.
+- Token-only (no public writes, ever), with a `{content: "pages", verbs: [...]}` scope - which is
+  why the scope shape above is an object rather than a bare string.
+- Author attribution `token:<id>`, as the Data API does for entries.
+- Media upload, blocks and menus as separate, later decisions.
+
+## Settled questions
+
+Settled in the plan and kept through implementation:
+
+1. **Admin home:** its own screen, System › Content API (`#/content-api`, permission `settings`).
+2. **Per-page opt-out:** yes - front matter `api: false` (404, not listed, not searched).
+3. **Asset URLs in `requires`:** not in v1. `requires` names components only.
+4. **Edition:** the same as the Data API - free core (`docs/editions-and-licences.md`).