domma-cms 1.6.60 → 1.7.0

package/docs/superpowers/specs/2026-05-06-dashboard-rebuild-design.md

@@ -0,0 +1,257 @@
+# Dashboard Rebuild Design
+
+**Date:** 2026-05-06
+**Status:** Approved
+**Scope:** Replace the current admin dashboard with a site-aware command centre that surfaces traffic, journeys, spikes, content state, and combined site/server health.
+
+## Problem
+
+The existing admin dashboard (`admin/js/views/dashboard.js`) shows three count cards (Total / Published / Drafts) and a Recent Pages table. It tells the operator nothing about how the site is performing - no hits, no journeys, no spike detection, no health signal. For a single-site CMS this is the natural landing page; it should be the place the operator opens first to know whether the site is OK and where attention is needed.
+
+## Goals
+
+- One-glance answer to: *Is the site healthy? What's getting traffic? Where are spikes? What's stale?*
+- Single network call on mount; auto-refresh only the volatile tiles (KPIs + spike feed).
+- Each widget self-contained so we can add/remove without touching the orchestrator.
+- Privacy-preserving - no IP, no UA, no PII; honour DNT; sessionStorage-only session IDs.
+
+## Non-Goals (YAGNI)
+
+- Geographic / IP-derived data
+- Browser / device breakdown (no UA parsing)
+- Multi-day cohort retention or returning-visitor tracking
+- Per-user custom widget layouts
+- A pluggable widget framework (defer until a 2nd plugin actually wants to contribute widgets)
+
+## Layout
+
+12-column grid, four rows:
+
+| Row | Content                                                                               | Cols |
+|-----|---------------------------------------------------------------------------------------|------|
+| 1   | KPI strip: Hits today (Δ vs yesterday) · Live sessions · Submissions/week · Health    | 4×3  |
+| 2   | Traffic chart (30d trend + today hourly overlay, spike points annotated) · Spike feed | 8 / 4 |
+| 3   | Top pages (sparkline + Δ%) · User journeys (entry / exit / top 2-step paths)          | 6 / 6 |
+| 4   | Site & server health detail · Activity feed (edits, publishes, submissions)           | 6 / 6 |
+
+## Data Model - Analytics Plugin Extensions
+
+### Beacon
+
+`public/inject-body.html` currently posts `{url, dnt}`. Extend to:
+
+```json
+{ "url": "/about", "dnt": false, "ref": "https://example.com/", "sid": "<uuid>" }
+```
+
+- `sid` - generated client-side via `crypto.randomUUID()` on first hit, stored in `sessionStorage` under `dm_sid`. Clears on tab close. No cross-session tracking.
+- `ref` - `document.referrer || ''`. Stripped to origin + path (no query, no fragment) before send.
+
+### Storage
+
+Two new files in `plugins/analytics/`:
+
+```
+journeys.json     { "YYYY-MM-DD": [ { sid, t, url, ref } ] }
+spikes-cache.json { "lastBuiltAt": iso, "items": [ { url, ratio, hits, baseline } ] }
+```
+
+- `journeys.json` is daily-capped (configurable, default **5,000 events/day**). When the cap is hit for the day, further events are dropped silently - keeps disk bounded.
+- `spikes-cache.json` is rebuilt on demand by `/spikes` and cached for 60s; avoids re-scanning daily files on every dashboard refresh.
+
+Existing `lifetime.json` and `daily.json` are unchanged - extension is additive.
+
+### New Endpoints (mounted at `/api/plugins/analytics`)
+
+| Method | Path        | Auth  | Purpose                                                                          |
+|--------|-------------|-------|----------------------------------------------------------------------------------|
+| GET    | `/journeys` | Admin | `?range=7d` - top entry pages, top exit pages, top 2-step paths, bounce rate     |
+| GET    | `/realtime` | Admin | Distinct `sid` count active in last 5 min                                        |
+| GET    | `/spikes`   | Admin | Current-hour hits per URL vs trailing 7-day same-hour baseline; flag where ratio > 2σ |
+
+#### Journeys derivation
+
+For the requested range, group `journeys.json` events by `sid`, sort by `t`. Then:
+- **Entry page** = first URL in each session
+- **Exit page** = last URL in each session
+- **2-step path** = each consecutive `(url[i], url[i+1])` pair
+- **Bounce rate** = sessions of length 1 ÷ total sessions
+
+Aggregate; return top 10 of each category plus the bounce-rate scalar.
+
+#### Spike detection
+
+For each URL with hits in the current hour:
+- Baseline = mean hits at this hour-of-day across the previous 7 days
+- StdDev = sample stddev across same window
+- If `currentHits > baseline + 2*stddev` AND `currentHits >= 5` (noise floor), flag
+- Return up to 5 flagged URLs sorted by `currentHits / baseline` ratio descending
+
+### Privacy
+
+- `sid` lives only in `sessionStorage` - closes with the tab; no cookie banner triggered.
+- No IP logging, no UA, no fingerprinting.
+- DNT continues to short-circuit at the beacon.
+- Reset button (existing) wipes `journeys.json` along with the others.
+
+## Data Model - Core Dashboard Endpoint
+
+### `GET /api/dashboard/summary`
+
+**Auth:** authenticate + requireAdmin
+
+**Response:**
+
+```json
+{
+  "traffic": {
+    "today": 1234,
+    "yesterday": 980,
+    "deltaPct": 25.9,
+    "weekToDate": 8200,
+    "previousWeek": 7600,
+    "hourlyToday": [{ "hour": 0, "hits": 12 }, ...]
+  },
+  "topPages": [
+    { "url": "/about", "hits": 432, "deltaPct": 12.4, "spark": [10, 14, ...] }
+  ],
+  "journeys": {
+    "entry": [{ "url": "/", "count": 800 }, ...],
+    "exit":  [{ "url": "/contact", "count": 120 }, ...],
+    "paths": [{ "from": "/", "to": "/pricing", "count": 90 }, ...],
+    "bounceRate": 0.42
+  },
+  "spikes": [{ "url": "/pricing", "ratio": 4.12, "hits": 312, "baseline": 76 }],
+  "realtime": { "activeSessions": 7 },
+  "health": {
+    "status": "warn",
+    "checks": [
+      { "id": "uptime",        "level": "ok",   "label": "Uptime", "value": "3d 4h" },
+      { "id": "disk-content",  "level": "ok",   "label": "Content disk", "value": "142 MB" },
+      { "id": "memory",        "level": "ok",   "label": "Heap", "value": "84 / 512 MB" },
+      { "id": "stale-pages",   "level": "warn", "label": "Stale pages (>90d)", "value": 4, "detail": ["/about", "/legal", ...] },
+      { "id": "missing-meta",  "level": "warn", "label": "Pages missing description", "value": 2 },
+      { "id": "last-form",     "level": "ok",   "label": "Last form submission", "value": "2h ago" },
+      { "id": "smtp",          "level": "ok",   "label": "Last email send", "value": "2026-05-06 09:12" },
+      { "id": "plugin-load",   "level": "ok",   "label": "Plugin failures", "value": 0 }
+    ]
+  },
+  "activity": [
+    { "type": "edit",      "at": "2026-05-06T09:14:00Z", "actor": "darryl", "target": "/about" },
+    { "type": "publish",   "at": "...", "actor": "...", "target": "..." },
+    { "type": "form",      "at": "...", "form": "contact" }
+  ]
+}
+```
+
+### Aggregation sources
+
+| Section      | Source                                                            |
+|--------------|-------------------------------------------------------------------|
+| traffic      | analytics plugin `daily.json` (today, yesterday, weeks)           |
+| topPages     | analytics plugin `daily.json` (last 7d, sparkline = per-day)      |
+| journeys     | analytics plugin `journeys.json` via internal helper              |
+| spikes       | analytics plugin `spikes-cache.json`                              |
+| realtime     | analytics plugin `journeys.json` (sids in last 5 min)             |
+| health       | new `server/services/health.js` (see below)                       |
+| activity     | versions service + form submissions (no auth audit log exists yet) |
+
+### `server/services/health.js`
+
+Computes all health checks. Each check returns `{ id, level: 'ok'\|'warn'\|'fail', label, value, detail? }`. Top-level `status` = worst level across checks (`fail` > `warn` > `ok`).
+
+Checks implemented:
+
+- **uptime** - `process.uptime()`, formatted
+- **disk-content** - recursive size of `content/` directory
+- **memory** - `process.memoryUsage().heapUsed / heapTotal`
+- **stale-pages** - pages whose `updatedAt` is >90 days ago; counts + lists up to 10
+- **missing-meta** - pages without `description` in frontmatter
+- **last-form** - newest submission across `content/forms/*/submissions/`; warn if >7 days, fail if no submissions ever AND any forms exist
+- **smtp** - read last result from `server/services/email.js` (add a small `lastSendResult` export)
+- **plugin-load** - count of plugins that failed manifest validation at startup (already logged; expose via `plugins.js`)
+
+Cross-cutting: results cached for 30s in-memory inside the service to avoid recomputing on every poll.
+
+### Cross-plugin coupling
+
+The dashboard endpoint imports the analytics plugin's helpers directly. To avoid making `server/` depend on `plugins/analytics/` (which violates the upstream-replaced boundary), the analytics plugin **registers an internal aggregator on the Fastify instance**:
+
+```js
+fastify.decorate('analytics', {
+  getTraffic, getTopPages, getJourneys, getSpikes, getRealtime
+});
+```
+
+`/api/dashboard/summary` reads `request.server.analytics?.getTraffic(...)` and tolerates the analytics plugin being absent (returns `null` for those sections; the dashboard renders empty-state placeholders).
+
+## Frontend Architecture
+
+### File structure
+
+```
+admin/js/views/dashboard.js                  Orchestrator: fetch summary, mount widgets, schedule refresh
+admin/js/templates/dashboard.html            Grid skeleton with named slot divs
+admin/js/views/dashboard/widgets/
+  kpi-strip.js / kpi-strip.html              Row 1
+  traffic-chart.js / traffic-chart.html      Row 2 left
+  spike-feed.js / spike-feed.html            Row 2 right
+  top-pages.js / top-pages.html              Row 3 left
+  journeys.js / journeys.html                Row 3 right
+  health-detail.js / health-detail.html      Row 4 left
+  activity-feed.js / activity-feed.html      Row 4 right
+admin/css/dashboard.css                      Grid + tile styling
+```
+
+### Widget contract
+
+Each widget exports:
+
+```js
+export default {
+  selector: '#dash-kpi-strip',
+  templateUrl: '/admin/js/templates/dashboard/kpi-strip.html',
+  render(data) { /* receives the relevant slice of /summary; returns nothing, mutates DOM via Domma */ }
+};
+```
+
+The orchestrator loads the template into the slot once, then calls `render(slice)` on initial mount and each refresh. Widgets do not fetch their own data - the orchestrator hands them what they need.
+
+### Refresh
+
+Single `setInterval(30_000)` in the orchestrator hits `/api/dashboard/summary?lite=1` (which returns only `traffic`, `realtime`, `spikes`, `health.status`). Re-renders KPI strip, spike feed, and the health pill. Other widgets are static after mount.
+
+### Charts
+
+Reuse the existing analytics-plugin Chart.js cache pattern (CDN with one-time load + module-level promise so we don't double-load). Sparklines in the top-pages widget use inline SVG (lightweight; Chart.js for one mini chart is overkill).
+
+### Empty states
+
+Every widget must render a zero-state when its data slice is empty (no hits yet, no journeys yet, no spikes detected). Show "No data yet" with a hint, not an empty box.
+
+## Error Handling
+
+- `/api/dashboard/summary` returns 200 with partial data even if a sub-aggregator throws. Each section is wrapped in `try/catch`; on failure, that key is `null` and a `warnings: ["traffic: <message>"]` array is populated.
+- Frontend orchestrator surfaces warnings as a single dismissable banner above the grid, not per-widget toasts.
+- Widget render functions must tolerate `null` slices (render the empty state).
+
+## Testing
+
+- `server/services/health.js` - unit tests for each check with mocked filesystem / process state.
+- Analytics plugin journey aggregation - unit tests with seeded `journeys.json` fixtures (entry/exit/paths/bounce).
+- Spike detector - unit tests across edge cases (no baseline, all zeros, single high hour).
+- `/api/dashboard/summary` - integration test asserting graceful degradation when analytics plugin is disabled (sections are `null`, status is 200).
+- Frontend widget rendering - manual browser verification per the user's standing rule (no "verified" claims without a real browser load).
+
+## Implementation Order
+
+1. **Analytics extensions** - beacon update, `journeys.json` storage, `/journeys` `/realtime` `/spikes` endpoints, internal `fastify.analytics` decorator. Ship and verify with the existing analytics admin view (still works).
+2. **Health service** - `server/services/health.js` with all checks + caching.
+3. **Dashboard summary endpoint** - `server/routes/api/dashboard.js` aggregating all sources.
+4. **Dashboard frontend** - new template + orchestrator + 7 widgets + CSS. Last because it depends on the API surface above.
+
+Each step is independently verifiable.
+
+## Open Questions
+
+None - all scope decisions resolved during brainstorm.