@simmalugnt-se/payload-editor-assistant 0.5.0 → 0.6.0

package/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## 0.6.0
+
+- `media.view`: the assistant can look at the image of the open upload document, so it writes alt
+  texts from what the image shows. Allowed per upload collection. The server reads the saved file
+  with the editor's access (storage adapter handlers, then the upload directory, then a public
+  address fetched without credentials) and sends the image itself to the model, never a URL; the
+  smallest `imageSizes` entry at least 768px wide is used, and JPEG, PNG, WebP and GIF up to 5 MB.
+  The panel says "Looking at the image…" meanwhile. Without `media.view`, the assistant says it
+  cannot see an open image instead of describing it from the filename.
+
+- A reply that quotes its proposal is shown whole. A quote with a full stop or a word such as
+  "click", "save" or "spara" ("Click. Edit. Done.", "Spara 20 %") was split into sentences and
+  partly removed as if it claimed something about the panel.
+
 ## 0.5.0
 
 - `next` is a required peer dependency: the panel imports `next/navigation`, so it never worked

package/README.md

@@ -4,36 +4,36 @@ Context-aware AI assistant for Payload Admin. The editor asks for changes in a d
 proposes form changes the editor reviews and approves, within an allowlist of collections, globals,
 fields and capabilities.
 
-Requires Payload `>=3.87.1 <4`, Next 16, React 19, Node 22 and the AI SDK (`ai` 7) with a provider of
-your choice.
+Requires Payload `>=3.87.1 <4`, Next 16, React 19, Node 22 and the AI SDK (`ai` 7) with a model of
+your choice. Choose a model that reads images: the assistant is getting abilities, such as alt
+texts, that show it the image.
 
 ## Setup
 
-1. Install the plugin, the AI SDK and a provider (DeepSeek here; any AI SDK provider works):
+1. Install the plugin and the AI SDK. The AI SDK reaches models from many companies through Vercel
+   AI Gateway without another package; any other AI SDK provider works too:
 
    ```bash
-   pnpm add @simmalugnt-se/payload-editor-assistant ai @ai-sdk/deepseek
+   pnpm add @simmalugnt-se/payload-editor-assistant ai
    ```
 
-2. Put the provider's API key in the server environment, e.g. `.env.local` locally and the
-   hosting provider's environment variables in production. Never expose it to the browser
-   (no `NEXT_PUBLIC_` prefix):
+2. Put the gateway's API key in the server environment, e.g. `.env.local` locally. Never expose it
+   to the browser (no `NEXT_PUBLIC_` prefix). A site deployed on Vercel signs in to the gateway by
+   itself and needs no key:
 
    ```bash
-   DEEPSEEK_API_KEY=sk-...
+   AI_GATEWAY_API_KEY=...
    ```
 
-3. Create the model with that key and pass it to the plugin:
+3. Create the model and pass it to the plugin:
 
    ```ts
-   import { createDeepSeek } from "@ai-sdk/deepseek";
+   import { gateway } from "ai";
    import { editorAssistantPlugin } from "@simmalugnt-se/payload-editor-assistant";
 
-   const deepSeek = createDeepSeek({ apiKey: process.env.DEEPSEEK_API_KEY });
-
    plugins: [
      editorAssistantPlugin({
-       model: deepSeek("deepseek-chat"),
+       model: gateway("openai/gpt-6-luna"),
        collections: {
          pages: {
            capabilities: ["content.find", "content.findById", "form.read", "form.propose", "draft.create"],
@@ -51,21 +51,22 @@ your choice.
 ## API keys
 
 The plugin has no provider or key of its own: it only receives the `model` you create, and the model
-is called on the server. To switch provider, install another AI SDK provider and create the model
-with its key, e.g. `createAnthropic({ apiKey: process.env.ANTHROPIC_API_KEY })`.
-
-Providers also read a default environment variable (`DEEPSEEK_API_KEY`, `ANTHROPIC_API_KEY`, ...)
-when no `apiKey` is passed, but passing it explicitly keeps the connection visible in your config.
+is called on the server. To switch model, change the gateway model ID (see
+[the model list](https://vercel.com/ai-gateway/models)). To use a provider directly, install its AI
+SDK provider and create the model with its key, e.g.
+`createAnthropic({ apiKey: process.env.ANTHROPIC_API_KEY })`.
 
 If the key is wrong, the drawer says the AI service did not answer and the cause is logged on the
 server through Payload's logger.
 
 To keep the plugin installed without a key (its tables stay in your schema and migrations), let
 `model` be a function that returns `null` or `undefined` when there is nothing to call. The drawer then
-tells the editor that no AI model is set up, and a warning is logged:
+tells the editor that no AI model is set up, and a warning is logged. On Vercel the gateway needs no
+key, so check for that too:
 
 ```ts
-model: () => (process.env.DEEPSEEK_API_KEY ? deepSeek("deepseek-chat") : null),
+model: () =>
+  process.env.AI_GATEWAY_API_KEY || process.env.VERCEL ? gateway("openai/gpt-6-luna") : null,
 ```
 
 ## What the assistant may do
@@ -79,11 +80,29 @@ Nothing is reachable unless you list it. Each collection or global gets the capa
 | `form.read` | read the open edit form, including unsaved changes | no |
 | `form.propose` | put a change in the open form, where the editor keeps or undoes it | yes |
 | `draft.create` | create a new draft document (collections only) | yes |
+| `media.view` | look at the image of the open upload document (upload collections only) | no |
 
 The assistant never publishes. Form changes land in the open form and the editor saves as usual; an
 approved `draft.create` saves a new, unpublished draft. The `users` collection, Payload's own collections and fields that look like
 secrets (`password`, `token`, `apiKey`, ...) are always off limits.
 
+## Images and alt texts
+
+With `media.view` on an upload collection, the assistant can look at the open image and write its
+alt text, which the editor keeps or undoes like any other change:
+
+```ts
+collections: {
+  media: { capabilities: ["form.read", "form.propose", "media.view"] },
+},
+```
+
+The server reads the saved file with the editor's access (through the storage adapter, the upload
+directory, or the file's public address) and sends the image itself to the model, never its
+address. It uses the smallest of the collection's `imageSizes` at least 768px wide, else the
+original; JPEG, PNG, WebP and GIF up to 5 MB. Choose a model that reads images. Without
+`media.view` the assistant says it cannot see the image instead of guessing from the filename.
+
 ## Keep or undo
 
 A proposed form change goes straight into the open form, so the fields and Live Preview show it

package/dist/capabilities.d.ts

@@ -1,10 +1,10 @@
-export declare const BUILTIN_CAPABILITIES: readonly ["admin.navigate", "schema.discover", "content.find", "content.findById", "form.read", "form.propose", "draft.create"];
+export declare const BUILTIN_CAPABILITIES: readonly ["admin.navigate", "schema.discover", "content.find", "content.findById", "form.read", "form.propose", "draft.create", "media.view"];
 export type BuiltinCapability = (typeof BUILTIN_CAPABILITIES)[number];
 export type CapabilityRisk = "auto" | "approval" | "forbidden";
 export declare const BUILTIN_RISK: Record<BuiltinCapability, CapabilityRisk>;
-export declare const EMBEDDED_ONLY_CAPABILITIES: Set<"admin.navigate" | "schema.discover" | "content.find" | "content.findById" | "form.read" | "form.propose" | "draft.create">;
-export declare const MCP_ELIGIBLE_CAPABILITIES: Set<"admin.navigate" | "schema.discover" | "content.find" | "content.findById" | "form.read" | "form.propose" | "draft.create">;
-export declare const ENTITY_CAPABILITIES: readonly ["content.find", "content.findById", "form.read", "form.propose", "draft.create"];
+export declare const EMBEDDED_ONLY_CAPABILITIES: Set<"admin.navigate" | "schema.discover" | "content.find" | "content.findById" | "form.read" | "form.propose" | "draft.create" | "media.view">;
+export declare const MCP_ELIGIBLE_CAPABILITIES: Set<"admin.navigate" | "schema.discover" | "content.find" | "content.findById" | "form.read" | "form.propose" | "draft.create" | "media.view">;
+export declare const ENTITY_CAPABILITIES: readonly ["content.find", "content.findById", "form.read", "form.propose", "draft.create", "media.view"];
 export type EntityCapability = (typeof ENTITY_CAPABILITIES)[number];
 export declare function isEntityCapability(value: string): value is EntityCapability;
 export declare function isBuiltinCapability(value: string): value is BuiltinCapability;

package/dist/capabilities.js

@@ -6,6 +6,7 @@ export const BUILTIN_CAPABILITIES = [
     "form.read",
     "form.propose",
     "draft.create",
+    "media.view",
 ];
 export const BUILTIN_RISK = {
     "admin.navigate": "auto",
@@ -15,11 +16,13 @@ export const BUILTIN_RISK = {
     "form.read": "auto",
     "form.propose": "approval",
     "draft.create": "approval",
+    "media.view": "auto",
 };
 export const EMBEDDED_ONLY_CAPABILITIES = new Set([
     "admin.navigate",
     "form.read",
     "form.propose",
+    "media.view",
 ]);
 export const MCP_ELIGIBLE_CAPABILITIES = new Set([
     "schema.discover",
@@ -33,6 +36,7 @@ export const ENTITY_CAPABILITIES = [
     "form.read",
     "form.propose",
     "draft.create",
+    "media.view",
 ];
 const ENTITY_CAPABILITY_SET = new Set(ENTITY_CAPABILITIES);
 export function isEntityCapability(value) {

package/dist/components/copy.d.ts

@@ -19,6 +19,7 @@ export declare const COPY: {
             readonly searching: "Searching the content…";
             readonly reading: "Reading a document…";
             readonly reading_form: "Reading the form…";
+            readonly viewing: "Looking at the image…";
             readonly proposing: "Preparing the change…";
             readonly drafting: "Preparing the draft…";
             readonly navigating: "Finding the page…";
@@ -81,6 +82,7 @@ export declare const COPY: {
             readonly searching: "Söker bland innehållet…";
             readonly reading: "Läser ett dokument…";
             readonly reading_form: "Läser formuläret…";
+            readonly viewing: "Tittar på bilden…";
             readonly proposing: "Förbereder ändringen…";
             readonly drafting: "Förbereder utkastet…";
             readonly navigating: "Letar upp sidan…";

package/dist/components/copy.js

@@ -20,6 +20,7 @@ export const COPY = {
             searching: "Searching the content…",
             reading: "Reading a document…",
             reading_form: "Reading the form…",
+            viewing: "Looking at the image…",
             proposing: "Preparing the change…",
             drafting: "Preparing the draft…",
             navigating: "Finding the page…",
@@ -82,6 +83,7 @@ export const COPY = {
             searching: "Söker bland innehållet…",
             reading: "Läser ett dokument…",
             reading_form: "Läser formuläret…",
+            viewing: "Tittar på bilden…",
             proposing: "Förbereder ändringen…",
             drafting: "Förbereder utkastet…",
             navigating: "Letar upp sidan…",

package/dist/config/validate.js

@@ -39,6 +39,9 @@ function validateEntityMap(kind, map, rules) {
             if (capability === "draft.create" && !rules.allowDraftCreate) {
                 throw new Error(`payload-editor-assistant: globals cannot enable draft.create ("${slug}").`);
             }
+            if (capability === "media.view" && kind === "globals") {
+                throw new Error(`payload-editor-assistant: media.view is for upload collections, not globals ("${slug}").`);
+            }
         }
         validateFields(entry, `${kind} "${slug}"`);
     }

package/dist/domain/media.d.ts

@@ -0,0 +1,40 @@
+import type { PayloadRequest } from "payload";
+import type { ValidatedEditorAssistantOptions } from "../config/validate.ts";
+export declare const MAX_IMAGE_BYTES: number;
+export type ImageSource = {
+    filename: string;
+    mimeType: string;
+    width?: number;
+    url?: string;
+};
+export type MediaViewResult = {
+    ok: true;
+    data: {
+        filename: string;
+        width?: number;
+        height?: number;
+        alt?: unknown;
+    };
+    image: {
+        mediaType: string;
+        data: string;
+    };
+} | {
+    ok: false;
+    error: string;
+    message: string;
+};
+/**
+ * The image file to show the model: the smallest of Payload's `imageSizes` at least
+ * `PREFERRED_WIDTH` wide, otherwise the original.
+ */
+export declare function pickImageSource(doc: Record<string, unknown>): ImageSource | null;
+/**
+ * Reads the image of a saved upload document with the editor's access, the way Payload serves it:
+ * the storage adapter's handlers first, then the file in `staticDir`, then a public URL fetched
+ * without the editor's credentials.
+ */
+export declare function viewMedia(req: PayloadRequest, options: ValidatedEditorAssistantOptions, input: {
+    collection?: string;
+    documentId?: string | number;
+}): Promise<MediaViewResult>;

package/dist/domain/media.js

@@ -0,0 +1,165 @@
+import { readFile, stat } from "node:fs/promises";
+import path from "node:path";
+import { findCollectionConfig } from "../schema/discover.js";
+import { hasCapability, resolveAllowlist } from "./allowlist.js";
+import { localApi } from "./local-api.js";
+/** Image types the AI SDK's providers take; SVG, AVIF and HEIC are left out. */
+const READABLE_TYPES = new Set(["image/jpeg", "image/png", "image/webp", "image/gif"]);
+/** Enough detail for an alt text without sending a full-size original. */
+const PREFERRED_WIDTH = 768;
+export const MAX_IMAGE_BYTES = 5 * 1024 * 1024;
+const FETCH_TIMEOUT_MS = 10_000;
+/**
+ * The image file to show the model: the smallest of Payload's `imageSizes` at least
+ * `PREFERRED_WIDTH` wide, otherwise the original.
+ */
+export function pickImageSource(doc) {
+    const original = asSource(doc);
+    const sizes = Object.values(asRecord(doc.sizes) ?? {})
+        .map((size) => asSource(asRecord(size) ?? {}))
+        .filter((size) => size !== null && (size.width ?? 0) >= PREFERRED_WIDTH)
+        .sort((a, b) => (a.width ?? 0) - (b.width ?? 0));
+    return sizes[0] ?? original;
+}
+/**
+ * Reads the image of a saved upload document with the editor's access, the way Payload serves it:
+ * the storage adapter's handlers first, then the file in `staticDir`, then a public URL fetched
+ * without the editor's credentials.
+ */
+export async function viewMedia(req, options, input) {
+    const resolved = resolveAllowlist(options, { collection: input.collection });
+    if (!resolved || !hasCapability(resolved.allowlist, "media.view")) {
+        return {
+            ok: false,
+            error: "tool_denied",
+            message: "Images in this collection are not allowlisted.",
+        };
+    }
+    const config = input.collection ? findCollectionConfig(req, input.collection) : undefined;
+    if (!config?.upload) {
+        return { ok: false, error: "tool_denied", message: "This collection holds no uploads." };
+    }
+    if (input.documentId === undefined) {
+        return {
+            ok: false,
+            error: "unsupported_media",
+            message: "The image is not saved yet. Ask the editor to save it first.",
+        };
+    }
+    let doc;
+    try {
+        doc = await localApi(req).findByID({
+            collection: resolved.slug,
+            id: input.documentId,
+            depth: 0,
+            draft: true,
+            req,
+            user: req.user,
+            overrideAccess: false,
+        });
+    }
+    catch {
+        return { ok: false, error: "tool_denied", message: "Document is not accessible." };
+    }
+    const source = pickImageSource(doc);
+    if (!source || !READABLE_TYPES.has(source.mimeType)) {
+        return {
+            ok: false,
+            error: "unsupported_media",
+            message: "This file is not an image the assistant can look at (JPEG, PNG, WebP or GIF).",
+        };
+    }
+    const bytes = await readSource(req, resolved.slug, config.upload, source, doc.prefix);
+    if (!bytes.ok) {
+        return bytes;
+    }
+    return {
+        ok: true,
+        data: {
+            filename: String(doc.filename),
+            width: asNumber(doc.width),
+            height: asNumber(doc.height),
+            alt: doc.alt,
+        },
+        image: { mediaType: source.mimeType, data: bytes.data.toString("base64") },
+    };
+}
+async function readSource(req, collection, uploadConfig, source, prefix) {
+    const upload = (asRecord(uploadConfig) ?? {});
+    const tooLarge = {
+        ok: false,
+        error: "unsupported_media",
+        message: "The image is too large for the assistant to look at.",
+    };
+    const unreadable = {
+        ok: false,
+        error: "unsupported_media",
+        message: "The image file could not be read.",
+    };
+    try {
+        for (const handler of upload.handlers ?? []) {
+            const response = await handler(req, {
+                doc: undefined,
+                headers: new Headers(),
+                params: {
+                    collection,
+                    filename: source.filename,
+                    prefix: typeof prefix === "string" && prefix ? prefix : undefined,
+                },
+            });
+            if (response instanceof Response) {
+                return response.ok ? await readResponse(response, tooLarge) : unreadable;
+            }
+        }
+        if (!upload.disableLocalStorage) {
+            const dir = path.resolve(upload.staticDir || collection);
+            const file = path.resolve(dir, source.filename);
+            if (!file.startsWith(dir + path.sep)) {
+                return unreadable;
+            }
+            const info = await stat(file).catch(() => null);
+            if (info?.isFile()) {
+                return info.size > MAX_IMAGE_BYTES ? tooLarge : { ok: true, data: await readFile(file) };
+            }
+        }
+        // Storage without handlers serves files from a public address of its own.
+        if (source.url && /^https?:\/\//i.test(source.url)) {
+            const response = await fetch(source.url, {
+                credentials: "omit",
+                redirect: "error",
+                signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
+            });
+            return response.ok ? await readResponse(response, tooLarge) : unreadable;
+        }
+    }
+    catch {
+        return unreadable;
+    }
+    return unreadable;
+}
+async function readResponse(response, tooLarge) {
+    const length = Number(response.headers.get("content-length"));
+    if (length > MAX_IMAGE_BYTES) {
+        return tooLarge;
+    }
+    const data = Buffer.from(await response.arrayBuffer());
+    return data.length > MAX_IMAGE_BYTES ? tooLarge : { ok: true, data };
+}
+function asSource(record) {
+    if (typeof record.filename !== "string" || !record.filename)
+        return null;
+    if (typeof record.mimeType !== "string")
+        return null;
+    return {
+        filename: record.filename,
+        mimeType: record.mimeType,
+        width: asNumber(record.width),
+        url: typeof record.url === "string" ? record.url : undefined,
+    };
+}
+function asRecord(value) {
+    return value && typeof value === "object" ? value : undefined;
+}
+function asNumber(value) {
+    return typeof value === "number" ? value : undefined;
+}

package/dist/plugin.js

@@ -19,6 +19,10 @@ export function editorAssistantPlugin(options = {}) {
             if (collection.auth && validated.collections[collection.slug]) {
                 throw new Error(`payload-editor-assistant: auth collection "${collection.slug}" cannot be allowlisted.`);
             }
+            if (!collection.upload &&
+                validated.collections[collection.slug]?.capabilities.includes("media.view")) {
+                throw new Error(`payload-editor-assistant: media.view needs an upload collection; "${collection.slug}" is not one.`);
+            }
             if (validated.collections[collection.slug]) {
                 collections[index] = withDocumentBridge(collection);
             }

package/dist/runtime/agent.d.ts

@@ -33,7 +33,7 @@ type ChatMessage = {
     text: string;
 };
 /** What the assistant is doing right now, for the panel to show while the turn runs. */
-export type AgentStep = "thinking" | "searching" | "reading" | "reading_form" | "proposing" | "drafting" | "navigating" | "writing";
+export type AgentStep = "thinking" | "searching" | "reading" | "reading_form" | "viewing" | "proposing" | "drafting" | "navigating" | "writing";
 export type AgentEvent = {
     type: "step";
     step: AgentStep;

package/dist/runtime/agent.js

@@ -2,7 +2,7 @@ import { PACKAGE_NAME } from "../package-name.js";
 import { loadTurnContext } from "./context.js";
 import { needsProposeRetry } from "./propose-loop.js";
 import { buildSystemPrompt } from "./system-prompt.js";
-import { executeTool, fromProviderToolName, TOOL_DESCRIPTIONS, TOOL_JSON_SCHEMAS, toProviderToolName, } from "./tools.js";
+import { executeTool, fromProviderToolName, TOOL_DESCRIPTIONS, TOOL_JSON_SCHEMAS, toolModelOutput, toProviderToolName, } from "./tools.js";
 import { contentLanguageNote, editorFacingText } from "./voice.js";
 const MAX_MESSAGES = 20;
 const MAX_STEPS = 8;
@@ -12,6 +12,7 @@ const STEP_FOR_TOOL = {
     "content.find": "searching",
     "content.findById": "reading",
     "form.read": "reading_form",
+    "media.view": "viewing",
     "form.propose": "proposing",
     "draft.create": "drafting",
     "admin.navigate": "navigating",
@@ -228,7 +229,8 @@ function buildSdkTools(ai, ctx, captured, previews, emit) {
                         type: "step",
                         step: executed.ok && executed.pendingApproval ? "writing" : "thinking",
                     });
-                    captured.push(outcome);
+                    // The image goes to the model through toModelOutput; Admin never needs the bytes.
+                    captured.push(outcome.ok ? { ...outcome, image: undefined } : outcome);
                     return outcome;
                 }
                 catch {
@@ -242,6 +244,7 @@ function buildSdkTools(ai, ctx, captured, previews, emit) {
                     return outcome;
                 }
             },
+            toModelOutput: ({ output }) => toolModelOutput(output),
         });
     }
     return tools;

package/dist/runtime/system-prompt.js

@@ -30,10 +30,22 @@ function openDocumentLine(bridge, turnContext) {
         const sources = turnContext?.otherLanguages
             ? " Other language versions of this same page are in the quoted context. If the editor asks to fill or translate empty fields, call form.propose with text written in the open language. Translate from the source language; do not copy the source wording unless they asked to copy it unchanged. Do not ask them to paste. Do not say the source is missing if it is in the context. If a layout block already exists, set its empty text fields; do not insert a new block."
             : "";
-        return `The editor is looking at ${title} in ${language}. Do not ask them which document it is.${sources}${selectionLine(turnContext)}`;
+        return `The editor is looking at ${title} in ${language}. Do not ask them which document it is.${sources}${imageLine(bridge, turnContext)}${selectionLine(turnContext)}`;
     }
     return `${viewLine(turnContext)}No document is open. You can search, navigate, or create a draft. Form edits need an open document.`;
 }
+/** Without seeing the image, a model writes alt texts from the filename; say when it can look. */
+function imageLine(bridge, turnContext) {
+    const entry = turnContext?.allowlist.find((entity) => entity.kind === "collection" && entity.slug === bridge.collection);
+    if (entry?.capabilities.includes("media.view")) {
+        return ' The open document may be an image. You cannot see it until you call media.view. Before writing an alt text, a caption or anything about what the image shows, call media.view in this turn; never guess from the filename. An alt text says in one sentence, under 125 characters, what the image shows and what matters in it, without starting with "image of" or "bild av".';
+    }
+    const mimeType = turnContext?.form?.mimeType;
+    if (typeof mimeType === "string" && mimeType.startsWith("image/")) {
+        return " The open document is an image you cannot see. Never write an alt text or describe what it shows; tell the editor you cannot look at images here.";
+    }
+    return "";
+}
 function selectionLine(turnContext) {
     const selection = turnContext?.selection;
     if (!selection) {

package/dist/runtime/tools.d.ts

@@ -51,6 +51,11 @@ export type ToolSuccess = {
     };
     /** For Admin only; the agent keeps it out of what the model sees. */
     preview?: ProposalPreview;
+    /** For the model only, as a file part (see `toolModelOutput`); never sent to Admin. */
+    image?: {
+        mediaType: string;
+        data: string;
+    };
 };
 export type ToolFailure = {
     ok: false;
@@ -65,3 +70,24 @@ export declare const TOOL_DESCRIPTIONS: Record<string, string>;
 export declare function toProviderToolName(capability: string): string;
 export declare function fromProviderToolName(name: string): string;
 export declare function executeTool(name: string, input: Record<string, unknown>, ctx: ToolContext): Promise<ToolOutcome>;
+/** What the model gets back from a tool: an image as a file part next to the rest as text. */
+export declare function toolModelOutput(outcome: ToolOutcome): {
+    type: "json";
+    value: ToolOutcome;
+} | {
+    type: "content";
+    value: ({
+        type: "text";
+        text: string;
+        mediaType?: undefined;
+        data?: undefined;
+    } | {
+        type: "file";
+        mediaType: string;
+        data: {
+            type: "data";
+            data: string;
+        };
+        text?: undefined;
+    })[];
+};

package/dist/runtime/tools.js

@@ -1,5 +1,6 @@
 import { writeAudit } from "../domain/audit.js";
 import { findContent, findContentById } from "../domain/content.js";
+import { viewMedia } from "../domain/media.js";
 import { canNavigate, navigateAdmin } from "../domain/navigate.js";
 import { proposeChange } from "../domain/propose.js";
 import { discoverSchema, entityFields } from "../schema/discover.js";
@@ -31,6 +32,10 @@ export const TOOL_JSON_SCHEMAS = {
         type: "object",
         properties: {},
     },
+    "media.view": {
+        type: "object",
+        properties: {},
+    },
     "admin.navigate": {
         type: "object",
         additionalProperties: false,
@@ -66,6 +71,7 @@ export const TOOL_DESCRIPTIONS = {
     "content.find": "Search or list allowlisted documents by title, slug, or id. Omit query to list recent documents. Returns identity only, not a summary.",
     "content.findById": "Read one allowlisted document or global. Pass locale to read another language of the same document. For the currently open form, prefer form.read.",
     "form.read": "Read the currently open unsaved form in the current language. No document ID needed.",
+    "media.view": "Look at the image of the open media document. Call it before writing or judging an alt text, caption or anything else about what the image shows. No document ID needed.",
     "form.propose": 'Stage a structured patch against the open unsaved form in the current language. Call this as soon as you have the new field values. Do not ask the editor to confirm in chat; they approve in the panel. Example: {"op":"set","path":"layout.0.headline","value":"Välkommen"}. Requires editor approval.',
     "admin.navigate": "Navigate Admin to an allowlisted collection, global, or unique document match.",
     "draft.create": 'Stage creation of one draft document. Requires editor approval. Never publishes. The document starts empty: set top-level fields with {"op":"set","path":"title","value":"Start"}, and add each layout block with {"op":"blocks.insert","path":"layout","index":0,"blockType":"hero","data":{"headline":"Välkommen"}} using only block types and fields from the quoted context. Rich text fields take plain text.',
@@ -115,6 +121,17 @@ export async function executeTool(name, input, ctx) {
             const projected = projectRecord(ctx.formData ?? {}, fields, ctx.options.redact, ctx.bridge.collection ?? ctx.bridge.global);
             return { ok: true, capability: name, data: projected };
         }
+        case "media.view": {
+            // Only the open document: its id comes from the bridge, never from the model.
+            const viewed = await viewMedia(ctx.req, ctx.options, {
+                collection: ctx.bridge.collection,
+                documentId: ctx.bridge.documentId,
+            });
+            if (!viewed.ok) {
+                return { ok: false, capability: name, error: viewed.error, message: viewed.message };
+            }
+            return { ok: true, capability: name, data: viewed.data, image: viewed.image };
+        }
         case "admin.navigate": {
             if (!canNavigate(ctx.options)) {
                 return {
@@ -179,6 +196,24 @@ export async function executeTool(name, input, ctx) {
             return { ok: false, capability: name, error: "tool_denied", message: "Unknown tool." };
     }
 }
+/** What the model gets back from a tool: an image as a file part next to the rest as text. */
+export function toolModelOutput(outcome) {
+    if (!outcome.ok || !outcome.image) {
+        return { type: "json", value: outcome };
+    }
+    const { image, ...rest } = outcome;
+    return {
+        type: "content",
+        value: [
+            { type: "text", text: JSON.stringify(rest) },
+            {
+                type: "file",
+                mediaType: image.mediaType,
+                data: { type: "data", data: image.data },
+            },
+        ],
+    };
+}
 function wrap(capability, result) {
     if (!result.ok) {
         return { ok: false, capability, error: result.error, message: result.message };

package/dist/runtime/voice.js

@@ -36,7 +36,8 @@ const OVERCLAIM = /godkänn|godkann|approve|spara|sparat|sparad|sparats|publicer
 const MAX_OWN_TEXT = 600;
 export function withoutOverclaims(text) {
     const kept = splitSentences(text)
-        .filter((sentence) => !OVERCLAIM.test(sentence))
+        // A quoted proposal ("Spara 20 %") is the editor's content, not a claim about the panel.
+        .filter((sentence) => !OVERCLAIM.test(withoutQuotes(sentence)))
         .join(" ")
         .trim();
     return kept.length <= MAX_OWN_TEXT ? kept : "";
@@ -62,12 +63,42 @@ function doneCopy(locale) {
 function isSwedish(locale) {
     return locale === "sv" || locale.startsWith("sv-");
 }
+const QUOTES = '"“”„«»';
+const QUOTED = /["“”„«»][^"“”„«»]*["“”„«»]/g;
+/**
+ * Sentences, keeping a quoted proposal ("Click. Edit. Done.") inside the sentence that quotes it.
+ * A line with an unclosed quote is split as if it had none, so the quote cannot hide a claim.
+ */
 function splitSentences(text) {
     return text
-        .split(/(?<=[.!?])\s+|\n+/)
+        .split(/\n+/)
+        .flatMap((line) => {
+        const quotes = [...line].filter((char) => QUOTES.includes(char)).length;
+        if (quotes % 2 !== 0) {
+            return line.split(/(?<=[.!?])\s+/);
+        }
+        const sentences = [];
+        let start = 0;
+        let quoted = false;
+        for (let i = 0; i < line.length; i++) {
+            const char = line[i] ?? "";
+            if (QUOTES.includes(char)) {
+                quoted = !quoted;
+            }
+            else if (!quoted && /\s/.test(char) && /[.!?]["“”»]?$/.test(line.slice(start, i))) {
+                sentences.push(line.slice(start, i));
+                start = i + 1;
+            }
+        }
+        sentences.push(line.slice(start));
+        return sentences;
+    })
         .map((part) => part.trim())
         .filter(Boolean);
 }
+function withoutQuotes(sentence) {
+    return sentence.replace(QUOTED, "");
+}
 const CONTENT_LANGUAGE = {
     en: { sv: "engelska", en: "English" },
     sv: { sv: "svenska", en: "Swedish" },

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@simmalugnt-se/payload-editor-assistant",
-  "version": "0.5.0",
+  "version": "0.6.0",
   "description": "Context-aware editor assistant plugin for Payload Admin",
   "keywords": [
     "payload",