@rubytech/create-sitedesk-code 0.1.523 → 0.1.525

package/payload/platform/docs/superpowers/plans/2026-07-14-graph-top-level-labels-ontology-single-source.md

@@ -1,458 +0,0 @@
-# Graph top-level label allowlist ontology single-source — Implementation Plan
-
-> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
-
-**Goal:** Make the `/graph` filter-popover chip allowlist (and the two other server routes that read the same set) derive from each vertical's `## Top-level node types` schema table instead of the hardcoded `FILTER_TOP_LEVEL_LABELS`, so a top-level label added to a vertical schema surfaces as a chip with no TypeScript edit, while preserving today's exact chip visibility.
-
-**Architecture:** A new server-only module `platform/ui/server/lib/top-level-labels.ts` reads every `platform/plugins/memory/references/schema-*.md`, extracts the `Neo4j Label` column of each file's `## Top-level node types` table, and unions those with an explicit `STATIC_TOP_LEVEL_LABELS` const (base-schema infra plus verticals lacking a top-level section). The three server routes (`graph-labels-in-graph`, `graph-subgraph`, `graph-default-view`) call `getTopLevelLabelAllowlist()` instead of importing `FILTER_TOP_LEVEL_LABELS`. Count-gating in the cypher is unchanged.
-
-**Tech Stack:** TypeScript (ES modules), Node `fs` (`readdirSync`/`readFileSync`), Vitest, Hono routes.
-
-## Global Constraints
-
-- Derived union MUST deep-equal today's `FILTER_TOP_LEVEL_LABELS` (48 members) against the real references dir — the byte-identical acceptance baseline.
-- No `schema:` namespace filter: every row of a `## Top-level node types` table contributes its `Neo4j Label`, so `WhatsAppGroup` (`cdm:Channel`) stays a chip.
-- The `Neo4j Label` column is resolved by header name, never by index (construction table has 5 columns, estate-agent 4).
-- The new module is server-only (reads files). `FILTER_TOP_LEVEL_LABELS` stays in `app/lib/graph-labels.ts` for the client `searchHelpers.ts`.
-- `getTopLevelLabelAllowlist()` returns a `ReadonlySet<string>` so `.has()` and spread work at every call site.
-- Observability line on computation: `op=top-level-labels source=ontology count=<n> files=<...> derived=<d> static=<s>`, written to stderr via `console.error`.
-- No em-dashes in any shipped log string. No task numbers in operator-visible strings or log lines.
-- Path resolution mirrors `server/index.ts`: `process.env.MAXY_PLATFORM_ROOT ?? join(__dirname, '..')`, then `plugins/memory/references`.
-
----
-
-### Task 1: The derivation module and its regression test
-
-**Files:**
-- Create: `platform/ui/server/lib/top-level-labels.ts`
-- Test: `platform/ui/server/lib/top-level-labels.test.ts`
-
-**Interfaces:**
-- Produces: `getTopLevelLabelAllowlist(opts?: { referencesDir?: string }): ReadonlySet<string>` and `STATIC_TOP_LEVEL_LABELS: ReadonlySet<string>`.
-- Consumes: nothing from earlier tasks. Reads `FILTER_TOP_LEVEL_LABELS` from `../../app/lib/graph-labels` in the test only.
-
-- [ ] **Step 1: Write the failing test**
-
-```ts
-// platform/ui/server/lib/top-level-labels.test.ts
-import { describe, it, expect } from 'vitest'
-import { resolve } from 'node:path'
-import { mkdtempSync, writeFileSync, mkdirSync } from 'node:fs'
-import { tmpdir } from 'node:os'
-import { getTopLevelLabelAllowlist } from './top-level-labels'
-import { FILTER_TOP_LEVEL_LABELS } from '../../app/lib/graph-labels'
-
-// The real shipped references directory (four hops up from this test file:
-// server/lib -> server -> ui -> platform -> plugins/memory/references).
-const REAL_REFERENCES = resolve(
-  import.meta.dirname,
-  '../../../plugins/memory/references',
-)
-
-describe('getTopLevelLabelAllowlist', () => {
-  it('deep-equals FILTER_TOP_LEVEL_LABELS against the real references dir', () => {
-    const derived = getTopLevelLabelAllowlist({ referencesDir: REAL_REFERENCES })
-    expect([...derived].sort()).toEqual([...FILTER_TOP_LEVEL_LABELS].sort())
-  })
-
-  it('surfaces a new top-level row with no TypeScript edit', () => {
-    const dir = mkdtempSync(resolve(tmpdir(), 'tll-'))
-    writeFileSync(
-      resolve(dir, 'schema-widgets.md'),
-      [
-        '# Widgets',
-        '',
-        '## Top-level node types (operator-entry, natural key)',
-        '',
-        '| Entity | Neo4j Label | Schema.org Type | Required Properties |',
-        '|--------|-------------|-----------------|---------------------|',
-        '| Widget | `Widget` | `schema:Product` | `accountId`, `sku` |',
-        '',
-      ].join('\n'),
-    )
-    const derived = getTopLevelLabelAllowlist({ referencesDir: dir })
-    expect(derived.has('Widget')).toBe(true)
-  })
-
-  it('includes a cdm:-namespaced top-level row (no namespace filter)', () => {
-    const dir = mkdtempSync(resolve(tmpdir(), 'tll-'))
-    writeFileSync(
-      resolve(dir, 'schema-transport.md'),
-      [
-        '## Top-level node types',
-        '',
-        '| Entity | Neo4j Label | Schema.org Type | Required Properties |',
-        '|--------|-------------|-----------------|---------------------|',
-        '| Channel | `SomeChannel` | `cdm:Channel` | `accountId`, `id` |',
-        '',
-      ].join('\n'),
-    )
-    const derived = getTopLevelLabelAllowlist({ referencesDir: dir })
-    expect(derived.has('SomeChannel')).toBe(true)
-  })
-
-  it('ignores tables that are not under a Top-level node types heading', () => {
-    const dir = mkdtempSync(resolve(tmpdir(), 'tll-'))
-    writeFileSync(
-      resolve(dir, 'schema-child.md'),
-      [
-        '## Child node types (reached via parent neighbourhood)',
-        '',
-        '| Entity | Neo4j Label | Schema.org Type | Required Properties |',
-        '|--------|-------------|-----------------|---------------------|',
-        '| Line | `QuoteLine` | `schema:Thing` | `accountId`, `id` |',
-        '',
-      ].join('\n'),
-    )
-    const derived = getTopLevelLabelAllowlist({ referencesDir: dir })
-    // Only the STATIC base labels survive; QuoteLine is a child, not top-level.
-    expect(derived.has('QuoteLine')).toBe(false)
-    expect(derived.has('Person')).toBe(true)
-  })
-})
-```
-
-- [ ] **Step 2: Run test to verify it fails**
-
-Run: `cd platform/ui && npx vitest run server/lib/top-level-labels.test.ts`
-Expected: FAIL — `getTopLevelLabelAllowlist` is not exported / module not found.
-
-- [ ] **Step 3: Write minimal implementation**
-
-```ts
-// platform/ui/server/lib/top-level-labels.ts
-/**
- * Server-side derivation of the /graph top-level chip allowlist from the
- * per-vertical ontology.
- *
- * The allowlist is the union of two sources:
- *   1. The `Neo4j Label` column of every `## Top-level node types` table in
- *      `plugins/memory/references/schema-*.md`. Today only schema-construction
- *      and schema-estate-agent carry that section; more arrive as verticals
- *      gain one.
- *   2. `STATIC_TOP_LEVEL_LABELS` — base-schema infra (schema-base.md has no
- *      top-level section; its Node Types table mixes in child types) plus any
- *      vertical whose schema has no top-level section yet.
- *
- * No namespace filter: every row contributes its label, so transport labels
- * like WhatsAppGroup (cdm:Channel) stay chips. This differs from the account
- * file-schema generator, which excludes non-schema: rows because a channel
- * has no file bucket. A channel is still a legitimate operator-entry graph node.
- */
-import { readdirSync, readFileSync } from 'node:fs'
-import { join, resolve } from 'node:path'
-
-/**
- * Top-level labels not (yet) sourced from a vertical `## Top-level node types`
- * section. This set shrinks as verticals gain a top-level section; each such
- * change moves its labels from here into the derived union.
- */
-export const STATIC_TOP_LEVEL_LABELS: ReadonlySet<string> = Object.freeze(
-  new Set([
-    // Base-schema infra (schema-base.md has no top-level section).
-    'LocalBusiness',
-    'Service',
-    'PriceSpecification',
-    'OpeningHoursSpecification',
-    'Organization',
-    'Person',
-    'UserProfile',
-    'Preference',
-    'AdminUser',
-    'AccessGrant',
-    'KnowledgeDocument',
-    'DigitalDocument',
-    'CreativeWork',
-    'Question',
-    'FAQPage',
-    'DefinedTerm',
-    'Review',
-    'ImageObject',
-    'Invoice',
-    'Task',
-    'Project',
-    'Event',
-    'Workflow',
-    'Email',
-    'EmailAccount',
-    'Agent',
-    // Knowledge-work vertical: schema-knowledge-work.md has no top-level
-    // section yet.
-    'Objective',
-    'KeyResult',
-    'Decision',
-    'Risk',
-    'Source',
-    'Finding',
-    'Hypothesis',
-  ]),
-) as ReadonlySet<string>
-
-/**
- * Split a pipe-delimited markdown row into trimmed cells, honouring backtick
- * code-spans so a `|` inside `` `A|B` `` does not split the row. Strips the
- * optional leading/trailing pipes.
- */
-function splitPipeRow(row: string): string[] {
-  let trimmed = row.trim()
-  if (trimmed.startsWith('|')) trimmed = trimmed.slice(1)
-  if (trimmed.endsWith('|')) trimmed = trimmed.slice(0, -1)
-  const cells: string[] = []
-  let inBacktick = false
-  let start = 0
-  for (let i = 0; i < trimmed.length; i++) {
-    const ch = trimmed[i]
-    if (ch === '`') inBacktick = !inBacktick
-    else if (ch === '|' && !inBacktick) {
-      cells.push(trimmed.slice(start, i).trim())
-      start = i + 1
-    }
-  }
-  cells.push(trimmed.slice(start).trim())
-  return cells
-}
-
-const stripBackticks = (s: string): string => s.replace(/`/g, '').trim()
-
-/**
- * Extract the `Neo4j Label` column of the first pipe-table beneath a
- * `## Top-level node types` heading. Returns [] when the file has no such
- * section. The heading is matched by case-insensitive prefix so the
- * `(operator-entry, natural key)` suffix is tolerated; parsing stops at the
- * next `## ` heading.
- */
-function parseTopLevelLabels(content: string): string[] {
-  const lines = content.split(/\r?\n/)
-  let i = 0
-  // Find the heading.
-  while (i < lines.length) {
-    const l = lines[i]
-    if (l.startsWith('## ') && l.toLowerCase().includes('top-level node types')) break
-    i++
-  }
-  if (i >= lines.length) return []
-  i++
-  // Find the header row of the first table before the next `## `.
-  for (; i < lines.length - 1; i++) {
-    const line = lines[i]
-    if (line.startsWith('## ')) return []
-    if (!line.includes('|')) continue
-    const sep = lines[i + 1]
-    if (!/^\s*\|?[\s:|-]+\|?\s*$/.test(sep) || !sep.includes('-')) continue
-    const headers = splitPipeRow(line)
-    const labelCol = headers.findIndex((h) => h.toLowerCase() === 'neo4j label')
-    if (labelCol === -1) return []
-    const out: string[] = []
-    let j = i + 2
-    for (; j < lines.length; j++) {
-      const row = lines[j]
-      if (!row.trim() || !row.includes('|')) break
-      const cells = splitPipeRow(row)
-      const label = stripBackticks(cells[labelCol] ?? '')
-      if (label) out.push(label)
-    }
-    return out
-  }
-  return []
-}
-
-function resolveReferencesDir(): string {
-  // Production: MAXY_PLATFORM_ROOT is `<install>/platform`, and the schema
-  // references live at `<platform>/plugins/memory/references` (same root
-  // schema-loader.ts resolves brand.json against).
-  if (process.env.MAXY_PLATFORM_ROOT) {
-    return join(process.env.MAXY_PLATFORM_ROOT, 'plugins', 'memory', 'references')
-  }
-  // Dev fallback (env unset, unbundled): three hops up from this source file,
-  // server/lib -> server -> ui -> platform, then plugins/memory/references.
-  return resolve(import.meta.dirname, '../../../plugins/memory/references')
-}
-
-let cached: ReadonlySet<string> | null = null
-
-/**
- * The /graph top-level chip allowlist, derived from the ontology. Cached at
- * module scope because schema files are static per install. An injected
- * `referencesDir` (tests) bypasses the cache and does not populate it.
- */
-export function getTopLevelLabelAllowlist(
-  opts: { referencesDir?: string } = {},
-): ReadonlySet<string> {
-  if (!opts.referencesDir && cached) return cached
-
-  const referencesDir = opts.referencesDir ?? resolveReferencesDir()
-  const fileNames = readdirSync(referencesDir)
-    .filter((f) => f.startsWith('schema-') && f.endsWith('.md'))
-    .sort()
-
-  const derived = new Set<string>()
-  const contributingFiles: string[] = []
-  for (const fileName of fileNames) {
-    const content = readFileSync(join(referencesDir, fileName), 'utf-8')
-    const labels = parseTopLevelLabels(content)
-    if (labels.length > 0) contributingFiles.push(fileName)
-    for (const label of labels) derived.add(label)
-  }
-
-  const derivedCount = derived.size
-  for (const label of STATIC_TOP_LEVEL_LABELS) derived.add(label)
-
-  const result = Object.freeze(derived) as ReadonlySet<string>
-  process.stderr.write(
-    `[graph-page] op=top-level-labels source=ontology count=${derived.size} ` +
-      `files=${contributingFiles.join('|') || 'none'} derived=${derivedCount} ` +
-      `static=${STATIC_TOP_LEVEL_LABELS.size}\n`,
-  )
-
-  if (!opts.referencesDir) cached = result
-  return result
-}
-```
-
-- [ ] **Step 4: Run test to verify it passes**
-
-Run: `cd platform/ui && npx vitest run server/lib/top-level-labels.test.ts`
-Expected: PASS — all four tests green. If the deep-equal test fails, the diff names the drifted label(s); reconcile `STATIC_TOP_LEVEL_LABELS` against `FILTER_TOP_LEVEL_LABELS` (do not weaken the assertion).
-
-- [ ] **Step 5: Commit**
-
-```bash
-git add platform/ui/server/lib/top-level-labels.ts platform/ui/server/lib/top-level-labels.test.ts
-git commit -m "feat: derive /graph top-level chip allowlist from vertical ontology (1624)"
-```
-
----
-
-### Task 2: Point the three server routes at the derived allowlist
-
-**Files:**
-- Modify: `platform/ui/server/routes/admin/graph-labels-in-graph.ts`
-- Modify: `platform/ui/server/routes/admin/graph-subgraph.ts`
-- Modify: `platform/ui/server/routes/admin/graph-default-view.ts`
-
-**Interfaces:**
-- Consumes: `getTopLevelLabelAllowlist(): ReadonlySet<string>` from Task 1.
-- Produces: nothing new; behaviour of the three routes is unchanged because the derived set deep-equals `FILTER_TOP_LEVEL_LABELS`.
-
-- [ ] **Step 1: Confirm the existing route tests pass before the swap (baseline)**
-
-Run: `cd platform/ui && npx vitest run app/graph/__tests__/labels-in-graph.test.ts server/routes/__tests__/graph-default-topLevel.test.ts`
-Expected: PASS. This is the regression baseline the swap must preserve.
-
-- [ ] **Step 2: Swap the popover route**
-
-In `graph-labels-in-graph.ts`, change the import block
-
-```ts
-import {
-  FILTER_TOP_LEVEL_LABELS,
-  AGENT_ACTION_LABELS,
-} from '../../../app/lib/graph-labels'
-```
-
-to
-
-```ts
-import { AGENT_ACTION_LABELS } from '../../../app/lib/graph-labels'
-import { getTopLevelLabelAllowlist } from '../../lib/top-level-labels'
-```
-
-and change the allowlist construction (currently `new Set<string>(FILTER_TOP_LEVEL_LABELS)`)
-
-```ts
-    const allowedSet = new Set<string>(getTopLevelLabelAllowlist())
-    if (includeAgentActions) {
-      for (const label of AGENT_ACTION_LABELS) allowedSet.add(label)
-    }
-```
-
-- [ ] **Step 3: Swap the default-view fetch route**
-
-In `graph-subgraph.ts`, remove `FILTER_TOP_LEVEL_LABELS` from the `../../../app/lib/graph-labels` import (keep `HIDDEN_BY_DEFAULT_LABELS` and any others it imports), add `import { getTopLevelLabelAllowlist } from '../../lib/top-level-labels'`, and change line ~280
-
-```ts
-  const labels = [...getTopLevelLabelAllowlist()].filter((l) => !HIDDEN_BY_DEFAULT_LABELS.has(l))
-```
-
-- [ ] **Step 4: Swap the eligibility route**
-
-In `graph-default-view.ts`, replace
-
-```ts
-import { FILTER_TOP_LEVEL_LABELS } from '../../../app/lib/graph-labels'
-```
-
-with
-
-```ts
-import { getTopLevelLabelAllowlist } from '../../lib/top-level-labels'
-```
-
-and change the eligibility check (line ~158)
-
-```ts
-    if (!getTopLevelLabelAllowlist().has(lbl)) {
-```
-
-Update the adjacent `reason` string to drop the parenthetical `(FILTER_TOP_LEVEL_LABELS)` naming so no internal set name leaks; keep the operator-facing text: `label "${lbl}" is not eligible for default view — outside the chip allowlist`.
-
-- [ ] **Step 5: Run the route tests to verify no behaviour change**
-
-Run: `cd platform/ui && npx vitest run app/graph/__tests__/labels-in-graph.test.ts server/routes/__tests__/graph-default-topLevel.test.ts`
-Expected: PASS — the derived set equals `FILTER_TOP_LEVEL_LABELS`, so every assertion still holds.
-
-- [ ] **Step 6: Typecheck the UI package**
-
-Run: `cd platform/ui && npx tsc --noEmit`
-Expected: no errors (no unused-import errors for the removed `FILTER_TOP_LEVEL_LABELS`).
-
-- [ ] **Step 7: Commit**
-
-```bash
-git add platform/ui/server/routes/admin/graph-labels-in-graph.ts platform/ui/server/routes/admin/graph-subgraph.ts platform/ui/server/routes/admin/graph-default-view.ts
-git commit -m "refactor: three /graph server routes read the ontology-derived allowlist (1624)"
-```
-
----
-
-### Task 3: Update the graph-labels doc comment and file the client follow-up
-
-**Files:**
-- Modify: `platform/ui/app/lib/graph-labels.ts` (comment only — the set stays for the client)
-- Create: `.tasks/NNNN-graph-searchhelpers-derive-top-level-from-ontology.md` (number assigned at land time from LANES)
-
-**Interfaces:**
-- Consumes: nothing. Documentation and task-tracking only.
-
-- [ ] **Step 1: Annotate `FILTER_TOP_LEVEL_LABELS` as the client-only mirror**
-
-Add a sentence to the `FILTER_TOP_LEVEL_LABELS` doc block in `graph-labels.ts` recording that the server routes now derive this set from the vertical ontology (`server/lib/top-level-labels.ts`), and that this constant remains only for the client `searchHelpers.ts`, which cannot read files. Do not change the set members.
-
-- [ ] **Step 2: Write the follow-up task file**
-
-Create the task file describing: plumb the ontology-derived allowlist to the client `app/data/searchHelpers.ts` so its `/data` search chip pre-filter stops reading the static `FILTER_TOP_LEVEL_LABELS`, closing the last second source. In scope: deliver the derived set to the client (via the `graph-labels-in-graph` response the page already fetches, or an equivalent). Out of scope: the server routes (done here). Include Testing and Observability sections per the `.tasks` format.
-
-- [ ] **Step 3: Commit**
-
-```bash
-git add platform/ui/app/lib/graph-labels.ts .tasks/NNNN-graph-searchhelpers-derive-top-level-from-ontology.md
-git commit -m "docs: mark FILTER_TOP_LEVEL_LABELS client-only; file searchHelpers follow-up (1624)"
-```
-
----
-
-## Self-Review
-
-**Spec coverage:**
-- New module + derivation → Task 1. ✅
-- Three server routes switch → Task 2. ✅
-- Byte-identical baseline → Task 1 Step 1 deep-equal test + Task 2 route tests. ✅
-- No namespace filter (WhatsAppGroup) → Task 1 Step 1 cdm test + the real-dir deep-equal (which contains WhatsAppGroup). ✅
-- Base list explicit → `STATIC_TOP_LEVEL_LABELS` in Task 1. ✅
-- Observability line → Task 1 Step 3 `process.stderr.write`. ✅
-- Client follow-up filed → Task 3. ✅
-- Count-gating unchanged → cypher untouched in Task 2; asserted by the pre-existing route tests. ✅
-
-**Placeholder scan:** The only deferred value is the follow-up task number (`NNNN`), assigned from LANES at archive time per sprint protocol — not a code placeholder.
-
-**Type consistency:** `getTopLevelLabelAllowlist(opts?: { referencesDir?: string }): ReadonlySet<string>` is used identically in Task 1 (definition), Task 1 test, and all three Task 2 call sites. `STATIC_TOP_LEVEL_LABELS` is a `ReadonlySet<string>`. Call sites use `.has()` and spread, both valid on `ReadonlySet`.