@focus-reactive/payload-plugin-seo 1.3.0 → 1.5.0

package/README.md

@@ -2,7 +2,7 @@
 
 Live SEO analysis for [Payload CMS](https://payloadcms.com/) v3 + Next.js, powered by [Yoast](https://github.com/Yoast/wordpress-seo). Adds a real-time SEO drawer to the document editor — keyphrase optimization, on-page checks, readability, inclusive language, content vitals, and a Google SERP preview — without adding a single field to your database.
 
-The plugin injects a button into the editor toolbar of each configured collection. Clicking it opens a drawer that reads the current (unsaved) form values, extracts the title, meta description, slug, body content, and images, and runs the Yoast analysis engine **entirely in the browser**. Nothing is persisted — there are zero new collections, globals, or fields.
+The plugin injects a button into the editor toolbar of each configured collection. Clicking it opens a drawer that reads the current (unsaved) form values, derives the title, meta description, and slug from dot-path config, runs **your** registered content extractor to build the body content, and runs the Yoast analysis engine **entirely in the browser**. Nothing is persisted — there are zero new collections, globals, or fields.
 
 ---
 
@@ -19,8 +19,11 @@ The plugin adds NO database fields, collections, or globals. It injects a button
 document editor toolbar (admin.components.edit.beforeDocumentControls) of each configured
 collection. The button opens a drawer that:
 - Reads the live (unsaved) form values for the document
-- Extracts title, meta description, slug, body content and images using dot-path config
-- Resolves upload/relationship media into <img> tags via the Payload REST API
+- Derives title, meta description, and slug from dot-path config
+- Runs YOUR registered content extractor to build the body content. Content extraction is
+  app-owned — there is no built-in walker. The extractor receives the raw form values plus a
+  toolkit ({ resolveDocs, helpers }); it fetches any referenced/upload docs it needs via
+  resolveDocs and returns a ContentNode[] the plugin serializes to HTML.
 - Runs the Yoast engine (yoastseo + @yoast/search-metadata-previews) in the browser
 - Shows tabs: Keyphrase, On-page SEO, Readability, Inclusive, Content vitals, SERP preview
 
@@ -43,8 +46,9 @@ seoPlugin({
         seoTitle: 'seoTitle',           // dot-path; falls back to useAsTitle / 'title'
         metaDescription: 'metaDescription',
         slug: 'slug',                   // default: 'slug'
-        content: 'sections',            // dot-path to the main content field (blocks/richText/textarea)
       },
+      // REQUIRED: lookup key for a content extractor you register (see Step 4).
+      extractContentPath: '@/seo/extractPageContent#default',
     },
   ],
   site: { name: 'My Site', baseUrl: 'https://example.com', faviconUrl: '/favicon.ico' },
@@ -65,16 +69,39 @@ const nextConfig = {
   transpilePackages: ['@yoast/search-metadata-previews', '@yoast/components'],
 }
 
+## Step 4 — Write and register a content extractor (REQUIRED)
+
+Content extraction is entirely yours — there is no built-in walker, and extractContentPath is
+required. Write an extractor and register it under the same key, from an admin-mounted client
+module:
+
+// src/seo/extractPageContent.ts
+import type { ContentExtractor } from '@focus-reactive/payload-plugin-seo/content'
+const extractPageContent: ContentExtractor = async (values, ctx, { resolveDocs, helpers }) => {
+  // 1. collect ids from the RAW values (relationship/upload fields are ids)
+  // 2. const docs = await resolveDocs([{ collection: 'media', ids, select: ['url','alt'] }])
+  // 3. build the IR with helpers
+  return helpers.compact([helpers.heading(1, values.title as string) /* … */])
+}
+export default extractPageContent
+
+// src/providers/SeoExtractorRegistrar.tsx   ("use client")
+import { registerContentExtractors } from '@focus-reactive/payload-plugin-seo/content'
+import extractPageContent from '@/seo/extractPageContent'
+registerContentExtractors({ '@/seo/extractPageContent#default': extractPageContent })
+// export a component that renders {children} and mount it via admin.components.providers
+
 ## Important notes
 
 - The plugin reads UNSAVED form values, so analysis updates live as you type (debounced ~1s).
-- `fields.content` should point at your primary body field. The built-in extractor walks
-  blocks, arrays, groups, tabs, lexical richText, and uploads, converting them to HTML.
-- If the built-in extractor can't reach your content shape, supply `extractContentPath`:
-  an importMap module path to a `(formValues) => string | Promise<string>` returning HTML.
+- Content extraction is done by YOUR registered extractor; extractContentPath is required and
+  there is no built-in fallback. The extractor receives raw form values, a ctx
+  ({ locale, apiRoute }), and a toolkit ({ resolveDocs, helpers }), and returns ContentNode[].
+- relationship/upload fields arrive as ids; use toolkit.resolveDocs(queries) to fetch only the
+  docs/fields you need (one parallel request per collection), then read them with store.get().
 - Non-English analysis requires the locale code in `supportedLocales`; the matching Yoast
   language pack is dynamically imported on demand.
-- No GA4, no API keys, no server calls except resolving media URLs from your own Payload API.
+- No GA4, no API keys, no server calls except the resolveDocs reads against your own Payload API.
 ```
 
 ---
@@ -90,9 +117,10 @@ Document editor (configured collection)
         ▼
    SEO Drawer (client-only)
         │
-        ├─ read live form values (title, description, slug, content, keyphrase)
-        ├─ collect upload/relationship refs → resolve via /api/{collection}?depth=0&locale=…
-        ├─ hydrate values → walk tree → build HTML (lexical → HTML, images → <img>)
+        ├─ read live form values (title, description, slug, keyphrase)
+        ├─ run your registered extractor(values, ctx, toolkit)
+        │     └─ toolkit.resolveDocs(): parallel, projected /api/{collection} fetches
+        ├─ extractor returns ContentNode[] → plugin serializes to HTML
         ▼
    Yoast engine (in browser): Paper + EnglishResearcher + SeoAssessor
         │
@@ -135,8 +163,9 @@ export default buildConfig({
             seoTitle: "seoTitle",
             metaDescription: "metaDescription",
             slug: "slug",
-            content: "sections",
           },
+          // Required: register a matching extractor (see "Content Extraction").
+          extractContentPath: "@/seo/extractPageContent#default",
         },
       ],
       site: {
@@ -174,6 +203,10 @@ const nextConfig = {
 export default nextConfig;
 ```
 
+### Step 4 — Write and register a content extractor
+
+Required — see [Content Extraction](#content-extraction).
+
 ---
 
 ## Configuration Reference
@@ -201,15 +234,21 @@ interface SeoPluginConfig {
 interface SeoCollectionConfig {
   /** Collection slug to attach the drawer to. */
   slug: string;
-  /** Dot-paths telling the plugin which fields hold the SEO inputs. */
+  /** Dot-paths telling the plugin which fields hold the title / meta description / slug. */
   fields?: SeoFieldPaths;
   /**
-   * importMap module-path to a custom client extractor
-   * `(formData) => string | Promise<string>` returning HTML.
-   * Example: "@/seo/my-extractor#default".
-   * Default: built-in smart extractor.
+   * REQUIRED. Lookup key for a registered ContentExtractor — the only content path;
+   * there is no built-in walker. Set it to the same string you pass as the key in
+   * registerContentExtractors(). Convention: the module path of the extractor file,
+   * e.g. "@/collections/Page/extractPageContent#default". The extractor runs in the
+   * browser on the raw form values and returns ContentNode[]. See "Content Extraction".
+   *
+   * A collection whose extractContentPath is missing/empty is dropped at plugin init
+   * (with a warning); if no collection has a valid extractContentPath the plugin no-ops.
+   * If the key is set but not registered at runtime, content analysis for that collection
+   * is empty (a one-time console error is logged) — there is no built-in fallback.
    */
-  extractContentPath?: string;
+  extractContentPath: string;
 }
 ```
 
@@ -224,12 +263,10 @@ interface SeoFieldPaths {
   metaDescription?: string;
   /** Dot-path to the slug. Default: 'slug' */
   slug?: string;
-  /** Dot-path to the primary content field (blocks / richText / textarea). */
-  content?: string;
 }
 ```
 
-Dot-paths support nesting, e.g. `"meta.description"` or `"content.body"`.
+Dot-paths support nesting, e.g. `"meta.description"` or `"content.body"`. Body content is **not** configured here — it is produced by your registered extractor (see below).
 
 ### SeoSiteConfig
 
@@ -244,48 +281,186 @@ interface SeoSiteConfig {
 }
 ```
 
-### Custom content extractor
+## Content Extraction
+
+Content extraction is **app-owned**: you register one `ContentExtractor` per collection. The plugin makes no assumptions about your document schema, relationships, link types, or URL construction — it hands your extractor the raw values plus a small, generic toolkit, and serializes whatever `ContentNode[]` you return.
+
+### The `ContentNode` Intermediate Representation
+
+The plugin represents page content as a flat array of typed nodes before serializing to HTML. This is the `ContentNode` union exported from `@focus-reactive/payload-plugin-seo/content`:
 
 ```ts
-type ExtractorFn = (data: Record<string, unknown>) => string | Promise<string>;
+type ContentNode =
+  | { type: "heading"; level: 1 | 2 | 3 | 4 | 5 | 6; text: string }
+  | { type: "paragraph"; text: string }
+  | { type: "link"; href: string; text: string }
+  | { type: "image"; src: string; alt?: string }
+  | { type: "video"; src: string; poster?: string }
+  | { type: "html"; html: string }; // lexical-converted or raw HTML escape hatch
 ```
 
-Provide one when the built-in extractor can't reconstruct your content shape. It receives the raw (unhydrated) form values and must return an HTML string. Reference it from config by importMap path:
+Serialization to HTML (for the Yoast engine) happens entirely inside the plugin. Extractors produce `ContentNode[]`; they never construct HTML strings directly.
+
+### Builder helpers
+
+The `/content` subpath exports pure builder functions. Each helper returns `null` for empty or missing input, and `compact` drops the nulls — so you can build sparse arrays and clean them in one pass. The same helpers are also handed to your extractor as `toolkit.helpers`, so you can use either the imports or the injected object.
 
 ```ts
-// payload.config.ts
+import {
+  heading, // heading(level: 1|2|3|4|5|6, text?: string | null): ContentNode | null
+  paragraph, // paragraph(text?: string | null): ContentNode | null
+  link, // link(href?: string | null, text?: string | null): ContentNode | null
+  image, // image(src?: string | null, alt?: string | null): ContentNode | null
+  video, // video(src?: string | null, poster?: string | null): ContentNode | null
+  richText, // richText(lexicalValue: unknown): ContentNode | null  (lexical → HTML; null when empty)
+  html, // html(raw?: string | null): ContentNode | null
+  compact, // compact(nodes: (ContentNode | null | undefined)[]): ContentNode[]
+} from "@focus-reactive/payload-plugin-seo/content";
+import type {
+  ContentNode,
+  HeadingLevel,
+} from "@focus-reactive/payload-plugin-seo/content";
+```
+
+### The extractor contract
+
+```ts
+type ContentExtractor = (
+  values: Record<string, unknown>, // RAW form values; relationship/upload fields are ids
+  ctx: ExtractContext, // { locale?: string; apiRoute?: string }
+  toolkit: ExtractToolkit, // { resolveDocs, helpers }
+) => ContentNode[] | Promise<ContentNode[]>;
+
+interface ExtractToolkit {
+  resolveDocs: (queries: DocQuery[]) => Promise<DocStore>;
+  helpers: ContentHelpers; // heading, paragraph, link, image, video, richText, html, compact
+}
+
+interface DocQuery {
+  collection: string;
+  ids: (string | number)[];
+  select?: string[]; // field projection → ?select[field]=true
+  depth?: number; // relationship population → ?depth=N (default 0)
+}
+
+interface DocStore {
+  get(collection: string, id: string | number): Record<string, unknown> | undefined;
+}
+```
+
+Your extractor:
+
+- Receives the **raw**, unsaved form values. Relationship and upload fields are **ids** (or id arrays / `{ relationTo, value }`), **not** populated objects — the plugin does no hydration.
+- Owns ref collection and any link/URL building. The plugin makes no assumptions about your link types (internal references, custom URLs, etc.) — you decide what to fetch and how to turn it into a node.
+- Uses `toolkit.resolveDocs(queries)` to fetch referenced/upload documents. You pass one query per collection with the `ids` you collected and an optional `select` projection (fetch only the fields you need) and `depth`. **All queries run in parallel.** Read results with `store.get(collection, id)`.
+- Returns `ContentNode[]` (built with the helpers); the plugin serializes it.
+
+```ts
+// src/collections/Page/extractPageContent.ts
+import { heading, image, paragraph, richText } from "@focus-reactive/payload-plugin-seo/content";
+import type {
+  ContentExtractor,
+  DocStore,
+} from "@focus-reactive/payload-plugin-seo/content";
+
+const extractPageContent: ContentExtractor = async (values, _ctx, { resolveDocs, helpers }) => {
+  const blocks = (values as { blocks?: Record<string, unknown>[] }).blocks ?? [];
+
+  // 1. Collect the ids you care about from the RAW values (you know your schema).
+  const mediaIds = blocks.flatMap((b) => (typeof b.image === "number" ? [b.image] : []));
+
+  // 2. Fetch them — one parallel request per collection, projected to only the fields you need.
+  const docs: DocStore = await resolveDocs([
+    { collection: "media", ids: mediaIds, select: ["url", "alt", "mimeType"] },
+  ]);
+
+  // 3. Build the Intermediate Representation.
+  return helpers.compact(
+    blocks.flatMap((b) => {
+      const media = typeof b.image === "number" ? docs.get("media", b.image) : undefined;
+      return [
+        heading(2, b.title as string),
+        paragraph(b.subtitle as string),
+        image((media as { url?: string })?.url, (media as { alt?: string })?.alt),
+        richText(b.content),
+      ];
+    }),
+  );
+};
+
+export default extractPageContent;
+```
+
+### The registry: why it exists and how to use it
+
+Payload 3.84 has no client-side import map — `admin.dependencies` resolves path strings to functions server-side only, and a resolved function cannot cross the server→client boundary as a prop. Because the Yoast analysis runs live in the browser, `extractContentPath` cannot be resolved by Payload machinery on the client.
+
+The plugin bridges this with a `globalThis`-backed registry. `extractContentPath` in config is the **lookup key**; the consuming app registers the actual function under the same key in an admin-mounted client module. The registry uses `globalThis` (not a bare module-level `Map`) so the key and function resolve to the same instance across separate bundle chunks.
+
+**Step 1 — Set `extractContentPath` in the plugin config (the lookup key):**
+
+```ts
+// payload.config.ts (or your plugins file)
+import { seoPlugin } from "@focus-reactive/payload-plugin-seo";
+
 seoPlugin({
   collections: [
-    { slug: "pages", extractContentPath: "@/seo/my-extractor#default" },
+    {
+      slug: "page",
+      fields: {
+        seoTitle: "meta.title",
+        metaDescription: "meta.description",
+        slug: "slug",
+      },
+      extractContentPath: "@/collections/Page/extractPageContent#default",
+    },
   ],
 });
 ```
 
-```ts
-// src/seo/my-extractor.ts
-export default function extractContent(data: Record<string, unknown>): string {
-  return `<h1>${data.title}</h1><p>${data.body}</p>`;
+**Step 2 — Create a client registrar component:**
+
+```tsx
+// src/providers/SeoExtractorRegistrar.tsx
+"use client";
+
+import { registerContentExtractors } from "@focus-reactive/payload-plugin-seo/content";
+import type { ReactNode } from "react";
+
+import extractPageContent from "@/collections/Page/extractPageContent";
+
+// registerContentExtractors runs once when this module loads in the admin bundle.
+// The key must exactly match the extractContentPath string in your plugin config.
+registerContentExtractors({
+  "@/collections/Page/extractPageContent#default": extractPageContent,
+});
+
+export function SeoExtractorRegistrar({ children }: { children: ReactNode }) {
+  return <>{children}</>;
 }
+
+export default SeoExtractorRegistrar;
 ```
 
----
+**Step 3 — Mount the registrar as an admin provider:**
 
-## Content Extraction
+```ts
+// payload.config.ts
+export default buildConfig({
+  admin: {
+    components: {
+      providers: ["/providers/SeoExtractorRegistrar"],
+    },
+  },
+  // ...
+});
+```
 
-When `extractContentPath` is **not** set, the plugin's built-in extractor:
+If the configured `extractContentPath` is set but the function is not registered (e.g. the provider is missing), the plugin logs a one-time console error and content analysis for that collection is empty — there is no built-in fallback.
 
-1. **Reads** the value at `fields.content` from the live form values.
-2. **Collects upload / relationship references** by walking the form schema (arrays, blocks, groups, tabs, rows, collapsibles, and lexical richText, including inline media nodes).
-3. **Resolves media** by calling your Payload REST API per collection:
-   `GET /api/{collection}?depth=0&locale={locale}&where[id][in][]=…` — fetching each doc's `url`, `mimeType`, and `alt`. Results are cached in-memory and invalidated when the drawer re-opens or content changes.
-4. **Hydrates** the value tree (upload IDs → full docs) and walks it to build HTML:
-   - Lexical richText → HTML via `@payloadcms/richtext-lexical/html`
-   - `{ url, mimeType: "image/*", alt? }` → `<img src="…" alt="…" />`
-   - `{ url, label | text | title }` → `<a href="…">…</a>`
-   - Strings → `<p>…</p>`
-   - Structural keys (`id`, `blockType`, `blockName`, `_template`, `order`) are skipped
+### Limitation: links and uploads embedded inside richText
 
-This means image checks (alt text, keyphrase in alt, image count) work against the real, resolved media — not raw relationship IDs.
+`helpers.richText(value)` serializes the lexical tree to HTML **as-is**. Internal-link nodes and upload nodes embedded inside richText *body content* are **not** resolved by the plugin — their `href`s / `src`s are left as the lexical tree provides them. This keeps the plugin fully schema-agnostic. If you need those resolved, walk the lexical tree yourself inside your extractor (its structure is standard Payload lexical), collect the referenced ids, fetch them with `resolveDocs`, and rewrite the nodes before building the IR.
 
 ---
 
@@ -293,14 +468,14 @@ This means image checks (alt text, keyphrase in alt, image count) work against t
 
 The drawer presents six tabs, all derived from a single in-browser Yoast analysis pass (a `Paper` analyzed by `SeoAssessor` with the language-appropriate `Researcher`):
 
-| Tab                   | What it checks                                                                 |
-| --------------------- | ------------------------------------------------------------------------------ |
-| **Keyphrase**         | Focus keyphrase usage — in title, slug, meta description, first paragraph, density, image alt, synonyms. Enter a keyphrase to unlock these checks. |
-| **On-page SEO**       | Title width, meta description presence/length, internal & outbound links, heading structure. |
-| **Readability**       | Sentence/paragraph length, transition words, passive voice, consecutive sentences. |
-| **Inclusive**         | Flags potentially exclusionary or non-inclusive language.                      |
-| **Content vitals**    | Word count, sentence/paragraph counts, image & video counts, reading time, prominent words. |
-| **Search result preview** | Live Google SERP preview (desktop + mobile) with keyphrase highlighting, built on `@yoast/search-metadata-previews`. |
+| Tab                       | What it checks                                                                                                                                     |
+| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Keyphrase**             | Focus keyphrase usage — in title, slug, meta description, first paragraph, density, image alt, synonyms. Enter a keyphrase to unlock these checks. |
+| **On-page SEO**           | Title width, meta description presence/length, internal & outbound links, heading structure.                                                       |
+| **Readability**           | Sentence/paragraph length, transition words, passive voice, consecutive sentences.                                                                 |
+| **Inclusive**             | Flags potentially exclusionary or non-inclusive language.                                                                                          |
+| **Content vitals**        | Word count, sentence/paragraph counts, image & video counts, reading time, prominent words.                                                        |
+| **Search result preview** | Live Google SERP preview (desktop + mobile) with keyphrase highlighting, built on `@yoast/search-metadata-previews`.                               |
 
 **Without a keyphrase:** the drawer still runs and the On-page, Readability, Inclusive, Content vitals, and SERP tabs all populate. Only the keyphrase-specific assessments wait until you type a focus keyphrase and analysis runs.
 
@@ -312,22 +487,29 @@ The drawer presents six tabs, all derived from a single in-browser Yoast analysi
 
 ```ts
 seoPlugin({
-  collections: [{ slug: "pages", fields: { content: "sections" } }],
+  collections: [
+    {
+      slug: "pages",
+      fields: { slug: "slug" },
+      extractContentPath: "@/seo/extractPageContent#default",
+    },
+  ],
   supportedLocales: ["en", "de", "fr", "es"],
 });
 ```
 
-The active locale is taken from the admin and normalized to Yoast's `xx_XX` form (e.g. `en` → `en_EN`). Media is resolved per-locale so localized URLs and alt text are analyzed correctly. A locale not listed in `supportedLocales` falls back to English processing.
+The active locale is taken from the admin and normalized to Yoast's `xx_XX` form (e.g. `en` → `en_EN`). The locale is passed to your extractor as `ctx.locale` and to `resolveDocs` (so projected fetches are locale-correct). A locale not listed in `supportedLocales` falls back to English processing.
 
 ---
 
 ## Exports Reference
 
-| Import path                                            | Exports                                                                                       |
-| ------------------------------------------------------ | --------------------------------------------------------------------------------------------- |
-| `@focus-reactive/payload-plugin-seo`                   | `seoPlugin`, and types `SeoPluginConfig`, `SeoCollectionConfig`, `SeoFieldPaths`, `SeoSiteConfig`, `ExtractorFn` |
-| `@focus-reactive/payload-plugin-seo/admin.css`         | Compiled admin styles for the drawer & button                                                 |
-| `@focus-reactive/payload-plugin-seo/components/SeoButton` | `SeoButton` — the toolbar button component (wired automatically by the plugin via the importMap; you normally never import this directly) |
+| Import path                                               | Exports                                                                                                                                                                                                                                              |
+| --------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `@focus-reactive/payload-plugin-seo`                      | `seoPlugin`, and types `SeoPluginConfig`, `SeoCollectionConfig`, `SeoFieldPaths`, `SeoSiteConfig`, `ContentExtractor`                                                                                                                                |
+| `@focus-reactive/payload-plugin-seo/content`              | Builder helpers `heading`, `paragraph`, `link`, `image`, `video`, `richText`, `html`, `compact`; `registerContentExtractors`, `resolveContentExtractor`; types `ContentNode`, `HeadingLevel`, `ContentExtractor`, `ExtractContext`, `ExtractToolkit`, `DocQuery`, `DocStore`, `ContentHelpers` |
+| `@focus-reactive/payload-plugin-seo/admin.css`            | Compiled admin styles for the drawer & button                                                                                                                                                                                                       |
+| `@focus-reactive/payload-plugin-seo/components/SeoButton` | `SeoButton` — the toolbar button component (wired automatically by the plugin via the importMap; you normally never import this directly)                                                                                                            |
 
 ---