@framers/agentos-ext-vision-pipeline 0.2.2 → 0.3.0

package/SKILL.md

@@ -11,23 +11,38 @@ tools_required: [vision-pipeline]
 Extract text from images, documents, and handwritten notes with a progressive 3-tier pipeline: local OCR (PaddleOCR or Tesseract), then local vision models (TrOCR, Florence-2), then cloud vision.
 
 ## Tool
-**vision-pipeline** takes `imageUrl`, `mode` and `maxTier`. `imageUrl` is a `data:image/...` URL or an http(s) URL on a public host: local file paths and addresses on this machine or a private network are refused, so a model cannot send the machine's files, or what its network serves, to a cloud model. With AgentOS 0.13.16 or later, the tool fetches an http(s) image through `imageToBuffer`'s untrusted mode and hands the pipeline the bytes: every address the host resolves to is checked when the connection is made, each redirect is checked the same way, and the fetch stops at 50 MiB and 30 seconds. With an older AgentOS the check reads the URL's host as written, so the address a public name resolves to and a redirect's target are not checked.
+**vision-pipeline** takes `imageUrl`, `mode` and `maxTier`. `imageUrl` is one of:
+- an http(s) URL on a public host;
+- a `data:image/...` URL;
+- the `file:` URL of an image the image-generation or image-editing tools saved, for the caller it was saved for.
+
+Every other local file is refused, and so are addresses on this machine or a private network (for IPv6, everything outside `2000::/3`), so a model cannot send the machine's files, or what its network serves, to a cloud model. An http(s) image is fetched through AgentOS's `imageToBuffer` in its untrusted mode, and the pipeline gets the bytes: every address the host resolves to is checked when the connection is made, each redirect is checked the same way, and the image is held to 50 MiB and 30 seconds.
 
 | mode | what it returns | lowest maxTier |
 |------|-----------------|----------------|
 | `auto` (default) | the pipeline's best text | 1 |
 | `ocr` | printed text | 1 |
 | `handwriting` | handwritten text (TrOCR) | 2 |
-| `layout` | document text and structure (Florence-2) | 2 |
+| `layout` | the lines of text, each with its box, in `layout` (Florence-2) | 2 |
 | `embed` | a CLIP embedding vector | 2 |
 | `describe` | what the image shows (cloud vision) | 3 |
 
-`maxTier` 1 runs local OCR alone, 2 adds the local vision models, 3 (default) may use cloud vision.
+`maxTier` 1 runs local OCR alone, 2 adds the local vision models, 3 (default) may use cloud vision. Any other value is refused: it is not read as 3.
+
+`handwriting` and `layout` run the tier they are named for and nothing else. When its model is not installed the call fails with the reason; it does not come back with plain OCR text. `auto` lets the pipeline choose, and returns as soon as a tier is confident.
+
+A text result (every mode but `embed`) names the tiers that ran in `tiers`. A tier that was due to run and failed, such as a model that did not load, is listed in `failedTiers` with its error, and the text comes from the tiers that ran. `embed` returns the vector in `embedding` with its `dimensions`, or the error when the embedding tier cannot run.
+
+CLIP is loaded for `embed` alone, the first time it is asked for: no text call runs an embedding.
+
+Requires `@framers/agentos` 0.13.40 or later.
 
 ## Setup
-The local tiers use the optional packages AgentOS finds installed: `ppu-paddle-ocr` or `tesseract.js` for OCR, and `@huggingface/transformers` for handwriting, layout and embeddings. With an OpenAI key in the pack option `openaiApiKey` or the secret `openai.apiKey`, the cloud tier uses OpenAI with that key. Without one, it uses the first provider whose key is in the environment: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY` (or `GEMINI_API_KEY`), `OPENROUTER_API_KEY`. `describe` and the cloud tier need `@framers/agentos` 0.12.14 or later: earlier releases send the image to the cloud model as text, and read the key from the environment only.
+The local tiers use the optional packages AgentOS finds installed: `ppu-paddle-ocr` or `tesseract.js` for OCR, and `@huggingface/transformers` for handwriting, layout and embeddings. With an OpenAI key in the pack option `openaiApiKey` or the secret `openai.apiKey`, the cloud tier uses OpenAI with that key. Without one, it uses the first provider whose key is in the environment: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY` (or `GEMINI_API_KEY`), `OPENROUTER_API_KEY`.
+
+The images directory, for `file:` sources, is the pack option `imageDir`, else the environment variable `AGENTOS_IMAGE_DIR`, else `agentos-images-<uid>` in the temp directory: the same as the image-generation and image-editing packs', under the same rule: it belongs to the service's user, and so does every directory above it unless that one is root's (any that others can write must be sticky, as `/tmp` is). Give all three the same `imageDir`.
 
 ## Example
 "Read the text from this receipt"
 "What does this handwritten note say?"
-"Extract the table data from this PDF page"
+"List the lines of this scanned page with where each one sits"

package/dist/imageFiles.d.ts

@@ -0,0 +1,113 @@
+/**
+ * @fileoverview Where the image tools keep image data, and how a saved image
+ * comes back as a source.
+ *
+ * A provider that answers with image data (OpenAI's GPT Image models always
+ * do) used to have that data returned to the model as a `data:` URL: megabytes
+ * of base64 in the next request. The tools save the bytes to a file and return
+ * its `file:` URL, or hand them to the host's own saver. A saved file's URL is
+ * accepted as a source by the image-editing tools and by `vision-pipeline`.
+ * The image-generation, image-editing and vision-pipeline packs each carry
+ * this file, the same in all three, so they agree on the directory.
+ *
+ * The directory is private to the service's user, each caller has a
+ * subdirectory of its own, and no other user can change a directory above
+ * it. That is the trust boundary: only this user can put a file there, or
+ * swap the directory for another, so a file there with the saver's name
+ * pattern is one the service wrote.
+ */
+/** The most bytes of a saved image the tools read back as a source: 50 MiB. */
+export declare const MAX_SOURCE_BYTES: number;
+/**
+ * The images directory: the pack option `imageDir`, else the environment
+ * variable `AGENTOS_IMAGE_DIR`, else a folder in the user's temp directory,
+ * which the operating system clears (the packs delete nothing). On POSIX the
+ * default carries the user id, since `/tmp` is shared. On Windows it is the
+ * user's own temp folder whatever `TEMP` says: `os.tmpdir()` reads `TEMP`,
+ * which can name a folder every user shares.
+ */
+export declare function imageDirectory(option?: unknown): string;
+/**
+ * The name of a caller's subdirectory: `u-` and 32 hex characters of the
+ * SHA-256 of its user id, or `shared` for a call that carries none. A host
+ * with several users gives each call its user (`context.userContext.userId`),
+ * so one user's saved image is no source for another.
+ */
+export declare function scopeOf(context?: {
+    userContext?: {
+        userId?: unknown;
+    };
+}): string;
+/** The kind of image `bytes` are, by their signature: PNG, JPEG or WebP. */
+export declare function imageKind(bytes: Buffer): {
+    ext: 'png' | 'jpg' | 'webp';
+    mimeType: string;
+} | undefined;
+/**
+ * Saves image bytes as a new file in the caller's subdirectory of `dir` and
+ * returns its `file:` URL. The extension comes from the bytes' signature, and
+ * the name is random, so a file is never overwritten.
+ *
+ * @throws When the bytes are not a PNG, JPEG or WebP image, or when the
+ *   directory cannot be made, is not private to this user, or cannot be
+ *   written. The message names the directory and the option that sets it.
+ */
+export declare function saveImageFile(bytes: Buffer, dir: string, scope: string): Promise<string>;
+/**
+ * The bytes of the saved image that `source` names, or `undefined` when
+ * `source` is not the `file:` URL of an image the saver wrote for this
+ * caller. Every reason gives `undefined`, so the caller's one refusal does
+ * not tell which files exist.
+ *
+ * The file must carry the saver's name pattern and sit directly in the
+ * caller's subdirectory of the images directory's real path: the same
+ * directory by device and inode, itself a directory and not a link, private
+ * to this user, as the images directory is, with no directory above it that
+ * another user can change. It must be a regular file with one link and at
+ * most {@link MAX_SOURCE_BYTES}, and start with a PNG, JPEG or WebP
+ * signature. It is opened once, without following a link, and read from
+ * that handle.
+ */
+export declare function readSavedImage(source: string, dir: string, scope: string): Promise<Buffer | undefined>;
+/** An image as AgentOS's image functions return it. */
+export interface ReturnedImage {
+    url?: string;
+    dataUrl?: string;
+    base64?: string;
+    mimeType?: string;
+}
+/**
+ * A host's own saver: it stores the bytes where the host's clients can load
+ * them and returns that http(s) URL.
+ */
+export type SaveImage = (image: {
+    bytes: Buffer;
+    mimeType: string;
+    tool: string;
+    context?: unknown;
+}) => string | Promise<string>;
+/** Where the tools keep image data: the directory of the default saver, and a host's own saver when it gave one. */
+export interface ImageStore {
+    /** The images directory (see {@link imageDirectory}). */
+    dir: string;
+    /** The host's saver, which replaces the default. */
+    saveImage?: SaveImage;
+}
+/** The store for a pack's options: its `imageDir` and its `saveImage`, when that is a function. */
+export declare function imageStore(options?: {
+    imageDir?: unknown;
+    saveImage?: unknown;
+}): ImageStore;
+/**
+ * An image a provider returned, as the reference the caller gets: the
+ * provider's http(s) URL as it is, or, for image data, the URL under which
+ * it was saved. Image data is never returned itself: as a `data:` URL it
+ * would go into the model's next request, megabytes of base64. `undefined`
+ * when the provider returned neither a URL nor data.
+ *
+ * @throws When the data is not a PNG, JPEG or WebP image, when it cannot be
+ *   saved, or when the host's saver throws or returns anything but an
+ *   http(s) URL of at most 2,048 characters.
+ */
+export declare function storeImage(image: ReturnedImage | undefined, tool: string, store: ImageStore, context?: unknown): Promise<string | undefined>;
+//# sourceMappingURL=imageFiles.d.ts.map

package/dist/imageFiles.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"imageFiles.d.ts","sourceRoot":"","sources":["../src/imageFiles.ts"],"names":[],"mappings":"AACA;;;;;;;;;;;;;;;;;GAiBG;AASH,+EAA+E;AAC/E,eAAO,MAAM,gBAAgB,QAAmB,CAAC;AAQjD;;;;;;;GAOG;AACH,wBAAgB,cAAc,CAAC,MAAM,CAAC,EAAE,OAAO,GAAG,MAAM,CAOvD;AAED;;;;;GAKG;AACH,wBAAgB,OAAO,CAAC,OAAO,CAAC,EAAE;IAAE,WAAW,CAAC,EAAE;QAAE,MAAM,CAAC,EAAE,OAAO,CAAA;KAAE,CAAA;CAAE,GAAG,MAAM,CAKhF;AAED,4EAA4E;AAC5E,wBAAgB,SAAS,CAAC,KAAK,EAAE,MAAM,GAAG;IAAE,GAAG,EAAE,KAAK,GAAG,KAAK,GAAG,MAAM,CAAC;IAAC,QAAQ,EAAE,MAAM,CAAA;CAAE,GAAG,SAAS,CAStG;AAsCD;;;;;;;;GAQG;AACH,wBAAsB,aAAa,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAuB9F;AAED;;;;;;;;;;;;;;GAcG;AACH,wBAAsB,cAAc,CAAC,MAAM,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC,CA+C5G;AAED,uDAAuD;AACvD,MAAM,WAAW,aAAa;IAC5B,GAAG,CAAC,EAAE,MAAM,CAAC;IACb,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED;;;GAGG;AACH,MAAM,MAAM,SAAS,GAAG,CAAC,KAAK,EAAE;IAC9B,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE,MAAM,CAAC;IACjB,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB,KAAK,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC;AAE/B,oHAAoH;AACpH,MAAM,WAAW,UAAU;IACzB,yDAAyD;IACzD,GAAG,EAAE,MAAM,CAAC;IACZ,oDAAoD;IACpD,SAAS,CAAC,EAAE,SAAS,CAAC;CACvB;AAED,mGAAmG;AACnG,wBAAgB,UAAU,CAAC,OAAO,CAAC,EAAE;IAAE,QAAQ,CAAC,EAAE,OAAO,CAAC;IAAC,SAAS,CAAC,EAAE,OAAO,CAAA;CAAE,GAAG,UAAU,CAK5F;AAwBD;;;;;;;;;;GAUG;AACH,wBAAsB,UAAU,CAC9B,KAAK,EAAE,aAAa,GAAG,SAAS,EAChC,IAAI,EAAE,MAAM,EACZ,KAAK,EAAE,UAAU,EACjB,OAAO,CAAC,EAAE,OAAO,GAChB,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC,CAgB7B"}

package/dist/imageFiles.js

@@ -0,0 +1,282 @@
+// @ts-nocheck
+/**
+ * @fileoverview Where the image tools keep image data, and how a saved image
+ * comes back as a source.
+ *
+ * A provider that answers with image data (OpenAI's GPT Image models always
+ * do) used to have that data returned to the model as a `data:` URL: megabytes
+ * of base64 in the next request. The tools save the bytes to a file and return
+ * its `file:` URL, or hand them to the host's own saver. A saved file's URL is
+ * accepted as a source by the image-editing tools and by `vision-pipeline`.
+ * The image-generation, image-editing and vision-pipeline packs each carry
+ * this file, the same in all three, so they agree on the directory.
+ *
+ * The directory is private to the service's user, each caller has a
+ * subdirectory of its own, and no other user can change a directory above
+ * it. That is the trust boundary: only this user can put a file there, or
+ * swap the directory for another, so a file there with the saver's name
+ * pattern is one the service wrote.
+ */
+import { createHash, randomBytes } from 'node:crypto';
+import { constants as fsConstants } from 'node:fs';
+import * as fs from 'node:fs/promises';
+import * as os from 'node:os';
+import * as path from 'node:path';
+import { fileURLToPath, pathToFileURL } from 'node:url';
+/** The most bytes of a saved image the tools read back as a source: 50 MiB. */
+export const MAX_SOURCE_BYTES = 50 * 1024 * 1024;
+/** The name of a file the saver wrote. */
+const SAVED_NAME = /^agentos-image-[0-9a-f]{32}\.(png|jpg|webp)$/;
+/** The eight bytes every PNG starts with. */
+const PNG_SIGNATURE = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
+/**
+ * The images directory: the pack option `imageDir`, else the environment
+ * variable `AGENTOS_IMAGE_DIR`, else a folder in the user's temp directory,
+ * which the operating system clears (the packs delete nothing). On POSIX the
+ * default carries the user id, since `/tmp` is shared. On Windows it is the
+ * user's own temp folder whatever `TEMP` says: `os.tmpdir()` reads `TEMP`,
+ * which can name a folder every user shares.
+ */
+export function imageDirectory(option) {
+    const named = [option, process.env.AGENTOS_IMAGE_DIR]
+        .map((value) => (typeof value === 'string' ? value.trim() : ''))
+        .find(Boolean);
+    if (named)
+        return path.resolve(named);
+    if (typeof process.getuid === 'function')
+        return path.join(os.tmpdir(), `agentos-images-${process.getuid()}`);
+    return path.join(os.homedir(), 'AppData', 'Local', 'Temp', 'agentos-images');
+}
+/**
+ * The name of a caller's subdirectory: `u-` and 32 hex characters of the
+ * SHA-256 of its user id, or `shared` for a call that carries none. A host
+ * with several users gives each call its user (`context.userContext.userId`),
+ * so one user's saved image is no source for another.
+ */
+export function scopeOf(context) {
+    const userId = context?.userContext?.userId;
+    return typeof userId === 'string' && userId !== ''
+        ? `u-${createHash('sha256').update(userId).digest('hex').slice(0, 32)}`
+        : 'shared';
+}
+/** The kind of image `bytes` are, by their signature: PNG, JPEG or WebP. */
+export function imageKind(bytes) {
+    if (bytes.length >= 8 && bytes.subarray(0, 8).equals(PNG_SIGNATURE))
+        return { ext: 'png', mimeType: 'image/png' };
+    if (bytes.length >= 3 && bytes[0] === 0xff && bytes[1] === 0xd8 && bytes[2] === 0xff) {
+        return { ext: 'jpg', mimeType: 'image/jpeg' };
+    }
+    if (bytes.length >= 12 && bytes.toString('latin1', 0, 4) === 'RIFF' && bytes.toString('latin1', 8, 12) === 'WEBP') {
+        return { ext: 'webp', mimeType: 'image/webp' };
+    }
+    return undefined;
+}
+/**
+ * Whether a directory's `stat` (taken with `bigint: true`) shows it private
+ * to this user: a directory this user owns that neither its group nor others
+ * can write. Where the platform has no user ids (Windows) only the kind is
+ * checked: the default there is inside the user's own profile.
+ */
+function isPrivateDirectory(stat) {
+    if (!stat.isDirectory())
+        return false;
+    if (typeof process.getuid !== 'function')
+        return true;
+    return stat.uid === BigInt(process.getuid()) && (stat.mode & BigInt(0o022)) === BigInt(0);
+}
+/**
+ * Whether no other user can rename or replace `dir`, a real path, through a
+ * directory above it: each one is owned by this user or by root, and none
+ * can be written by its group or by others unless it is sticky (as `/tmp`
+ * is), where only an entry's owner can rename it. Where the platform has no
+ * user ids (Windows) nothing is checked.
+ */
+async function hasTrustedParents(dir) {
+    if (typeof process.getuid !== 'function')
+        return true;
+    const uid = BigInt(process.getuid());
+    for (let parent = path.dirname(dir);; parent = path.dirname(parent)) {
+        const stat = await fs.lstat(parent, { bigint: true });
+        const sticky = (stat.mode & BigInt(0o1000)) !== BigInt(0);
+        const writableByOthers = (stat.mode & BigInt(0o022)) !== BigInt(0);
+        if (!stat.isDirectory() || (stat.uid !== uid && stat.uid !== BigInt(0)) || (writableByOthers && !sticky))
+            return false;
+        if (path.dirname(parent) === parent)
+            return true;
+    }
+}
+/** The error for a directory that is not private to this user. */
+function notPrivate(dir) {
+    return new Error(`${dir} is not a directory that this user owns and no one else can write`);
+}
+/**
+ * Saves image bytes as a new file in the caller's subdirectory of `dir` and
+ * returns its `file:` URL. The extension comes from the bytes' signature, and
+ * the name is random, so a file is never overwritten.
+ *
+ * @throws When the bytes are not a PNG, JPEG or WebP image, or when the
+ *   directory cannot be made, is not private to this user, or cannot be
+ *   written. The message names the directory and the option that sets it.
+ */
+export async function saveImageFile(bytes, dir, scope) {
+    const kind = imageKind(bytes);
+    if (!kind)
+        throw new Error('The provider returned data that is not a PNG, JPEG or WebP image.');
+    try {
+        // The directory is written by its real path, which no other user can
+        // change once the directories above it are checked.
+        await fs.mkdir(dir, { recursive: true, mode: 0o700 });
+        const root = await fs.realpath(dir);
+        if (!isPrivateDirectory(await fs.stat(root, { bigint: true })))
+            throw notPrivate(root);
+        if (!(await hasTrustedParents(root)))
+            throw new Error(`another user can change a directory above ${root}`);
+        const scopeDir = path.join(root, scope);
+        await fs.mkdir(scopeDir, { recursive: true, mode: 0o700 });
+        // lstat: a caller's directory that is a link to another caller's is refused.
+        if (!isPrivateDirectory(await fs.lstat(scopeDir, { bigint: true })))
+            throw notPrivate(scopeDir);
+        const file = path.join(scopeDir, `agentos-image-${randomBytes(16).toString('hex')}.${kind.ext}`);
+        await fs.writeFile(file, bytes, { flag: 'wx', mode: 0o600 });
+        return pathToFileURL(file).href;
+    }
+    catch (error) {
+        const reason = error instanceof Error ? error.message : String(error);
+        throw new Error(`The image could not be saved under ${dir}: ${reason}. Set the pack's imageDir option, or AGENTOS_IMAGE_DIR, to a directory that the service's user owns and no one else can write.`);
+    }
+}
+/**
+ * The bytes of the saved image that `source` names, or `undefined` when
+ * `source` is not the `file:` URL of an image the saver wrote for this
+ * caller. Every reason gives `undefined`, so the caller's one refusal does
+ * not tell which files exist.
+ *
+ * The file must carry the saver's name pattern and sit directly in the
+ * caller's subdirectory of the images directory's real path: the same
+ * directory by device and inode, itself a directory and not a link, private
+ * to this user, as the images directory is, with no directory above it that
+ * another user can change. It must be a regular file with one link and at
+ * most {@link MAX_SOURCE_BYTES}, and start with a PNG, JPEG or WebP
+ * signature. It is opened once, without following a link, and read from
+ * that handle.
+ */
+export async function readSavedImage(source, dir, scope) {
+    let handle;
+    try {
+        const url = new URL(source);
+        if (url.protocol !== 'file:' || url.hostname !== '')
+            return undefined;
+        const named = fileURLToPath(url);
+        if (!SAVED_NAME.test(path.basename(named)))
+            return undefined;
+        // fs.promises.realpath asks the operating system. (fs.realpathSync and
+        // fs.realpath resolve ".." in the text first, as path.resolve does.)
+        const real = await fs.realpath(named);
+        if (!SAVED_NAME.test(path.basename(real)))
+            return undefined;
+        // The images directory as it is now, by its real path, so a directory
+        // swapped in for it is read as what it is; and the caller's directory in
+        // it by lstat, so a link to another caller's directory is refused.
+        const root = await fs.realpath(dir);
+        const [rootDir, parent, scopeDir] = await Promise.all([
+            fs.stat(root, { bigint: true }),
+            fs.stat(path.dirname(real), { bigint: true }),
+            fs.lstat(path.join(root, scope), { bigint: true }),
+        ]);
+        if (!isPrivateDirectory(rootDir) || !isPrivateDirectory(scopeDir))
+            return undefined;
+        if (parent.dev !== scopeDir.dev || parent.ino !== scopeDir.ino)
+            return undefined;
+        if (!(await hasTrustedParents(root)))
+            return undefined;
+        // A FIFO would hold an open until a writer came; only a regular file is opened.
+        if (!(await fs.lstat(real)).isFile())
+            return undefined;
+        handle = await fs.open(real, fsConstants.O_RDONLY | (fsConstants.O_NOFOLLOW ?? 0) | (fsConstants.O_NONBLOCK ?? 0));
+        const stat = await handle.stat();
+        if (!stat.isFile() || stat.nlink !== 1 || stat.size > MAX_SOURCE_BYTES)
+            return undefined;
+        // One byte more than the size at open: a file that has grown is refused.
+        const buffer = Buffer.alloc(stat.size + 1);
+        let length = 0;
+        while (length < buffer.length) {
+            const { bytesRead } = await handle.read(buffer, length, buffer.length - length, null);
+            if (bytesRead === 0)
+                break;
+            length += bytesRead;
+        }
+        if (length > stat.size)
+            return undefined;
+        const bytes = buffer.subarray(0, length);
+        return imageKind(bytes) ? bytes : undefined;
+    }
+    catch {
+        return undefined;
+    }
+    finally {
+        await handle?.close().catch(() => { });
+    }
+}
+/** The store for a pack's options: its `imageDir` and its `saveImage`, when that is a function. */
+export function imageStore(options) {
+    return {
+        dir: imageDirectory(options?.imageDir),
+        saveImage: typeof options?.saveImage === 'function' ? options.saveImage : undefined,
+    };
+}
+/** The longest reference a host's saver may return. */
+const MAX_REFERENCE_LENGTH = 2048;
+/** The bytes of a base64 `data:` URL, the form AgentOS's providers give image data in; `undefined` for anything else. */
+function dataUrlBytes(value) {
+    if (typeof value !== 'string' || !/^data:/i.test(value))
+        return undefined;
+    const comma = value.indexOf(',');
+    if (comma === -1 || !/; *base64 *$/i.test(value.slice(0, comma)))
+        return undefined;
+    return Buffer.from(value.slice(comma + 1), 'base64');
+}
+/** Whether a saver's result is an http(s) URL short enough to hand to the model. */
+function isHttpReference(value) {
+    if (typeof value !== 'string' || value.length === 0 || value.length > MAX_REFERENCE_LENGTH)
+        return false;
+    try {
+        const { protocol } = new URL(value);
+        return protocol === 'http:' || protocol === 'https:';
+    }
+    catch {
+        return false;
+    }
+}
+/**
+ * An image a provider returned, as the reference the caller gets: the
+ * provider's http(s) URL as it is, or, for image data, the URL under which
+ * it was saved. Image data is never returned itself: as a `data:` URL it
+ * would go into the model's next request, megabytes of base64. `undefined`
+ * when the provider returned neither a URL nor data.
+ *
+ * @throws When the data is not a PNG, JPEG or WebP image, when it cannot be
+ *   saved, or when the host's saver throws or returns anything but an
+ *   http(s) URL of at most 2,048 characters.
+ */
+export async function storeImage(image, tool, store, context) {
+    if (!image)
+        return undefined;
+    if (typeof image.url === 'string' && /^https?:\/\//i.test(image.url))
+        return image.url;
+    const bytes = typeof image.base64 === 'string' && image.base64 !== ''
+        ? Buffer.from(image.base64, 'base64')
+        : (dataUrlBytes(image.dataUrl) ?? dataUrlBytes(image.url));
+    if (!bytes || bytes.length === 0)
+        return undefined;
+    if (!store.saveImage)
+        return saveImageFile(bytes, store.dir, scopeOf(context));
+    const kind = imageKind(bytes);
+    if (!kind)
+        throw new Error('The provider returned data that is not a PNG, JPEG or WebP image.');
+    const reference = await store.saveImage({ bytes, mimeType: kind.mimeType, tool, context });
+    if (!isHttpReference(reference)) {
+        throw new Error("The host's saveImage must return an http(s) URL of at most 2,048 characters.");
+    }
+    return reference;
+}
+//# sourceMappingURL=imageFiles.js.map

package/dist/imageFiles.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"imageFiles.js","sourceRoot":"","sources":["../src/imageFiles.ts"],"names":[],"mappings":"AAAA,cAAc;AACd;;;;;;;;;;;;;;;;;GAiBG;AAEH,OAAO,EAAE,UAAU,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AACtD,OAAO,EAAE,SAAS,IAAI,WAAW,EAAE,MAAM,SAAS,CAAC;AACnD,OAAO,KAAK,EAAE,MAAM,kBAAkB,CAAC;AACvC,OAAO,KAAK,EAAE,MAAM,SAAS,CAAC;AAC9B,OAAO,KAAK,IAAI,MAAM,WAAW,CAAC;AAClC,OAAO,EAAE,aAAa,EAAE,aAAa,EAAE,MAAM,UAAU,CAAC;AAExD,+EAA+E;AAC/E,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,GAAG,IAAI,GAAG,IAAI,CAAC;AAEjD,0CAA0C;AAC1C,MAAM,UAAU,GAAG,8CAA8C,CAAC;AAElE,6CAA6C;AAC7C,MAAM,aAAa,GAAG,MAAM,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC,CAAC;AAEpF;;;;;;;GAOG;AACH,MAAM,UAAU,cAAc,CAAC,MAAgB;IAC7C,MAAM,KAAK,GAAG,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,CAAC,iBAAiB,CAAC;SAClD,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,OAAO,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;SAC/D,IAAI,CAAC,OAAO,CAAC,CAAC;IACjB,IAAI,KAAK;QAAE,OAAO,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;IACtC,IAAI,OAAO,OAAO,CAAC,MAAM,KAAK,UAAU;QAAE,OAAO,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,MAAM,EAAE,EAAE,kBAAkB,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IAC9G,OAAO,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,OAAO,EAAE,EAAE,SAAS,EAAE,OAAO,EAAE,MAAM,EAAE,gBAAgB,CAAC,CAAC;AAC/E,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,OAAO,CAAC,OAAgD;IACtE,MAAM,MAAM,GAAG,OAAO,EAAE,WAAW,EAAE,MAAM,CAAC;IAC5C,OAAO,OAAO,MAAM,KAAK,QAAQ,IAAI,MAAM,KAAK,EAAE;QAChD,CAAC,CAAC,KAAK,UAAU,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,EAAE;QACvE,CAAC,CAAC,QAAQ,CAAC;AACf,CAAC;AAED,4EAA4E;AAC5E,MAAM,UAAU,SAAS,CAAC,KAAa;IACrC,IAAI,KAAK,CAAC,MAAM,IAAI,CAAC,IAAI,KAAK,CAAC,QAAQ,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC;QAAE,OAAO,EAAE,GAAG,EAAE,KAAK,EAAE,QAAQ,EAAE,WAAW,EAAE,CAAC;IAClH,IAAI,KAAK,CAAC,MAAM,IAAI,CAAC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,IAAI,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,IAAI,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC;QACrF,OAAO,EAAE,GAAG,EAAE,KAAK,EAAE,QAAQ,EAAE,YAAY,EAAE,CAAC;IAChD,CAAC;IACD,IAAI,KAAK,CAAC,MAAM,IAAI,EAAE,IAAI,KAAK,CAAC,QAAQ,CAAC,QAAQ,EAAE,CAAC,EAAE,CAAC,CAAC,KAAK,MAAM,IAAI,KAAK,CAAC,QAAQ,CAAC,QAAQ,EAAE,CAAC,EAAE,EAAE,CAAC,KAAK,MAAM,EAAE,CAAC;QAClH,OAAO,EAAE,GAAG,EAAE,MAAM,EAAE,QAAQ,EAAE,YAAY,EAAE,CAAC;IACjD,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;;;;GAKG;AACH,SAAS,kBAAkB,CAAC,IAA2D;IACrF,IAAI,CAAC,IAAI,CAAC,WAAW,EAAE;QAAE,OAAO,KAAK,CAAC;IACtC,IAAI,OAAO,OAAO,CAAC,MAAM,KAAK,UAAU;QAAE,OAAO,IAAI,CAAC;IACtD,OAAO,IAAI,CAAC,GAAG,KAAK,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,KAAK,MAAM,CAAC,CAAC,CAAC,CAAC;AAC5F,CAAC;AAED;;;;;;GAMG;AACH,KAAK,UAAU,iBAAiB,CAAC,GAAW;IAC1C,IAAI,OAAO,OAAO,CAAC,MAAM,KAAK,UAAU;QAAE,OAAO,IAAI,CAAC;IACtD,MAAM,GAAG,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC;IACrC,KAAK,IAAI,MAAM,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,GAAI,MAAM,GAAG,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;QACrE,MAAM,IAAI,GAAG,MAAM,EAAE,CAAC,KAAK,CAAC,MAAM,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QACtD,MAAM,MAAM,GAAG,CAAC,IAAI,CAAC,IAAI,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC,KAAK,MAAM,CAAC,CAAC,CAAC,CAAC;QAC1D,MAAM,gBAAgB,GAAG,CAAC,IAAI,CAAC,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,KAAK,MAAM,CAAC,CAAC,CAAC,CAAC;QACnE,IAAI,CAAC,IAAI,CAAC,WAAW,EAAE,IAAI,CAAC,IAAI,CAAC,GAAG,KAAK,GAAG,IAAI,IAAI,CAAC,GAAG,KAAK,MAAM,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,gBAAgB,IAAI,CAAC,MAAM,CAAC;YAAE,OAAO,KAAK,CAAC;QACvH,IAAI,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,MAAM;YAAE,OAAO,IAAI,CAAC;IACnD,CAAC;AACH,CAAC;AAED,kEAAkE;AAClE,SAAS,UAAU,CAAC,GAAW;IAC7B,OAAO,IAAI,KAAK,CAAC,GAAG,GAAG,mEAAmE,CAAC,CAAC;AAC9F,CAAC;AAED;;;;;;;;GAQG;AACH,MAAM,CAAC,KAAK,UAAU,aAAa,CAAC,KAAa,EAAE,GAAW,EAAE,KAAa;IAC3E,MAAM,IAAI,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC;IAC9B,IAAI,CAAC,IAAI;QAAE,MAAM,IAAI,KAAK,CAAC,mEAAmE,CAAC,CAAC;IAChG,IAAI,CAAC;QACH,qEAAqE;QACrE,oDAAoD;QACpD,MAAM,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;QACtD,MAAM,IAAI,GAAG,MAAM,EAAE,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC;QACpC,IAAI,CAAC,kBAAkB,CAAC,MAAM,EAAE,CAAC,IAAI,CAAC,IAAI,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;YAAE,MAAM,UAAU,CAAC,IAAI,CAAC,CAAC;QACvF,IAAI,CAAC,CAAC,MAAM,iBAAiB,CAAC,IAAI,CAAC,CAAC;YAAE,MAAM,IAAI,KAAK,CAAC,6CAA6C,IAAI,EAAE,CAAC,CAAC;QAC3G,MAAM,QAAQ,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;QACxC,MAAM,EAAE,CAAC,KAAK,CAAC,QAAQ,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;QAC3D,6EAA6E;QAC7E,IAAI,CAAC,kBAAkB,CAAC,MAAM,EAAE,CAAC,KAAK,CAAC,QAAQ,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;YAAE,MAAM,UAAU,CAAC,QAAQ,CAAC,CAAC;QAChG,MAAM,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC,QAAQ,EAAE,iBAAiB,WAAW,CAAC,EAAE,CAAC,CAAC,QAAQ,CAAC,KAAK,CAAC,IAAI,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC;QACjG,MAAM,EAAE,CAAC,SAAS,CAAC,IAAI,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;QAC7D,OAAO,aAAa,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC;IAClC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,MAAM,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACtE,MAAM,IAAI,KAAK,CACb,sCAAsC,GAAG,KAAK,MAAM,gIAAgI,CACrL,CAAC;IACJ,CAAC;AACH,CAAC;AAED;;;;;;;;;;;;;;GAcG;AACH,MAAM,CAAC,KAAK,UAAU,cAAc,CAAC,MAAc,EAAE,GAAW,EAAE,KAAa;IAC7E,IAAI,MAAiC,CAAC;IACtC,IAAI,CAAC;QACH,MAAM,GAAG,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,CAAC;QAC5B,IAAI,GAAG,CAAC,QAAQ,KAAK,OAAO,IAAI,GAAG,CAAC,QAAQ,KAAK,EAAE;YAAE,OAAO,SAAS,CAAC;QACtE,MAAM,KAAK,GAAG,aAAa,CAAC,GAAG,CAAC,CAAC;QACjC,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC;YAAE,OAAO,SAAS,CAAC;QAC7D,uEAAuE;QACvE,qEAAqE;QACrE,MAAM,IAAI,GAAG,MAAM,EAAE,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC;QACtC,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;YAAE,OAAO,SAAS,CAAC;QAC5D,sEAAsE;QACtE,yEAAyE;QACzE,mEAAmE;QACnE,MAAM,IAAI,GAAG,MAAM,EAAE,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC;QACpC,MAAM,CAAC,OAAO,EAAE,MAAM,EAAE,QAAQ,CAAC,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC;YACpD,EAAE,CAAC,IAAI,CAAC,IAAI,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC;YAC/B,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC;YAC7C,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,KAAK,CAAC,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC;SACnD,CAAC,CAAC;QACH,IAAI,CAAC,kBAAkB,CAAC,OAAO,CAAC,IAAI,CAAC,kBAAkB,CAAC,QAAQ,CAAC;YAAE,OAAO,SAAS,CAAC;QACpF,IAAI,MAAM,CAAC,GAAG,KAAK,QAAQ,CAAC,GAAG,IAAI,MAAM,CAAC,GAAG,KAAK,QAAQ,CAAC,GAAG;YAAE,OAAO,SAAS,CAAC;QACjF,IAAI,CAAC,CAAC,MAAM,iBAAiB,CAAC,IAAI,CAAC,CAAC;YAAE,OAAO,SAAS,CAAC;QACvD,gFAAgF;QAChF,IAAI,CAAC,CAAC,MAAM,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,EAAE;YAAE,OAAO,SAAS,CAAC;QACvD,MAAM,GAAG,MAAM,EAAE,CAAC,IAAI,CACpB,IAAI,EACJ,WAAW,CAAC,QAAQ,GAAG,CAAC,WAAW,CAAC,UAAU,IAAI,CAAC,CAAC,GAAG,CAAC,WAAW,CAAC,UAAU,IAAI,CAAC,CAAC,CACrF,CAAC;QACF,MAAM,IAAI,GAAG,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC;QACjC,IAAI,CAAC,IAAI,CAAC,MAAM,EAAE,IAAI,IAAI,CAAC,KAAK,KAAK,CAAC,IAAI,IAAI,CAAC,IAAI,GAAG,gBAAgB;YAAE,OAAO,SAAS,CAAC;QACzF,yEAAyE;QACzE,MAAM,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,GAAG,CAAC,CAAC,CAAC;QAC3C,IAAI,MAAM,GAAG,CAAC,CAAC;QACf,OAAO,MAAM,GAAG,MAAM,CAAC,MAAM,EAAE,CAAC;YAC9B,MAAM,EAAE,SAAS,EAAE,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,GAAG,MAAM,EAAE,IAAI,CAAC,CAAC;YACtF,IAAI,SAAS,KAAK,CAAC;gBAAE,MAAM;YAC3B,MAAM,IAAI,SAAS,CAAC;QACtB,CAAC;QACD,IAAI,MAAM,GAAG,IAAI,CAAC,IAAI;YAAE,OAAO,SAAS,CAAC;QACzC,MAAM,KAAK,GAAG,MAAM,CAAC,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC;QACzC,OAAO,SAAS,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC;IAC9C,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,SAAS,CAAC;IACnB,CAAC;YAAS,CAAC;QACT,MAAM,MAAM,EAAE,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;IACxC,CAAC;AACH,CAAC;AA6BD,mGAAmG;AACnG,MAAM,UAAU,UAAU,CAAC,OAAqD;IAC9E,OAAO;QACL,GAAG,EAAE,cAAc,CAAC,OAAO,EAAE,QAAQ,CAAC;QACtC,SAAS,EAAE,OAAO,OAAO,EAAE,SAAS,KAAK,UAAU,CAAC,CAAC,CAAE,OAAO,CAAC,SAAuB,CAAC,CAAC,CAAC,SAAS;KACnG,CAAC;AACJ,CAAC;AAED,uDAAuD;AACvD,MAAM,oBAAoB,GAAG,IAAI,CAAC;AAElC,yHAAyH;AACzH,SAAS,YAAY,CAAC,KAAc;IAClC,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,KAAK,CAAC;QAAE,OAAO,SAAS,CAAC;IAC1E,MAAM,KAAK,GAAG,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IACjC,IAAI,KAAK,KAAK,CAAC,CAAC,IAAI,CAAC,eAAe,CAAC,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC;QAAE,OAAO,SAAS,CAAC;IACnF,OAAO,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,GAAG,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC;AACvD,CAAC;AAED,oFAAoF;AACpF,SAAS,eAAe,CAAC,KAAc;IACrC,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,IAAI,KAAK,CAAC,MAAM,GAAG,oBAAoB;QAAE,OAAO,KAAK,CAAC;IACzG,IAAI,CAAC;QACH,MAAM,EAAE,QAAQ,EAAE,GAAG,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC;QACpC,OAAO,QAAQ,KAAK,OAAO,IAAI,QAAQ,KAAK,QAAQ,CAAC;IACvD,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,KAAK,CAAC;IACf,CAAC;AACH,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,CAAC,KAAK,UAAU,UAAU,CAC9B,KAAgC,EAChC,IAAY,EACZ,KAAiB,EACjB,OAAiB;IAEjB,IAAI,CAAC,KAAK;QAAE,OAAO,SAAS,CAAC;IAC7B,IAAI,OAAO,KAAK,CAAC,GAAG,KAAK,QAAQ,IAAI,eAAe,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC;QAAE,OAAO,KAAK,CAAC,GAAG,CAAC;IACvF,MAAM,KAAK,GACT,OAAO,KAAK,CAAC,MAAM,KAAK,QAAQ,IAAI,KAAK,CAAC,MAAM,KAAK,EAAE;QACrD,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,EAAE,QAAQ,CAAC;QACrC,CAAC,CAAC,CAAC,YAAY,CAAC,KAAK,CAAC,OAAO,CAAC,IAAI,YAAY,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC;IAC/D,IAAI,CAAC,KAAK,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,SAAS,CAAC;IACnD,IAAI,CAAC,KAAK,CAAC,SAAS;QAAE,OAAO,aAAa,CAAC,KAAK,EAAE,KAAK,CAAC,GAAG,EAAE,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC;IAC/E,MAAM,IAAI,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC;IAC9B,IAAI,CAAC,IAAI;QAAE,MAAM,IAAI,KAAK,CAAC,mEAAmE,CAAC,CAAC;IAChG,MAAM,SAAS,GAAG,MAAM,KAAK,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,QAAQ,EAAE,IAAI,CAAC,QAAQ,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC,CAAC;IAC3F,IAAI,CAAC,eAAe,CAAC,SAAS,CAAC,EAAE,CAAC;QAChC,MAAM,IAAI,KAAK,CAAC,8EAA8E,CAAC,CAAC;IAClG,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC"}

package/dist/index.d.ts

@@ -12,14 +12,29 @@
  * The cloud tier reads OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY (or
  * GEMINI_API_KEY) or OPENROUTER_API_KEY from the environment.
  *
+ * The pipelines are built without the embedding tier. With it, every text
+ * call would run a CLIP embedding and wait for it, to return a result that
+ * has no place for the vector; the first such call downloads a 350 MB model.
+ * The `embed` mode loads CLIP when it is first asked for.
+ *
+ * An image is an http(s) URL, a data URL, or the `file:` URL of an image the
+ * image-generation or image-editing tools saved: the three packs share the
+ * images directory (`imageDir`, `AGENTOS_IMAGE_DIR`, or a folder in the
+ * user's temp directory).
+ *
  * @module @framers/agentos-ext-vision-pipeline
  */
 /** What the extension manager passes the pack factory. */
 export interface ExtensionContext {
-    /** `priority` of the tool descriptor (default 45); `openaiApiKey` for the cloud tier. */
+    /**
+     * `priority` of the tool descriptor (default 45); `openaiApiKey` for the
+     * cloud tier; `imageDir`, the images directory the image tools save to
+     * (else `AGENTOS_IMAGE_DIR`, else a folder in the user's temp directory).
+     */
     options?: {
         priority?: number;
         openaiApiKey?: string;
+        imageDir?: string;
     } & Record<string, unknown>;
     /** Reads a secret, such as `openai.apiKey`. */
     getSecret?: (key: string) => string | undefined;

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AACA;;;;;;;;;;;;;;;GAeG;AAQH,0DAA0D;AAC1D,MAAM,WAAW,gBAAgB;IAC/B,yFAAyF;IACzF,OAAO,CAAC,EAAE;QAAE,QAAQ,CAAC,EAAE,MAAM,CAAC;QAAC,YAAY,CAAC,EAAE,MAAM,CAAA;KAAE,GAAG,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACjF,+CAA+C;IAC/C,SAAS,CAAC,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,MAAM,GAAG,SAAS,CAAC;IAChD,MAAM,CAAC,EAAE;QAAE,IAAI,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,IAAI,CAAC;QAAC,IAAI,CAAC,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,IAAI,CAAA;KAAE,CAAC;CACxE;AAED,2EAA2E;AAC3E,MAAM,WAAW,aAAa;IAC5B,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,MAAM,CAAC;IAChB,WAAW,EAAE,KAAK,CAAC;QACjB,EAAE,EAAE,MAAM,CAAC;QACX,IAAI,EAAE,MAAM,CAAC;QACb,QAAQ,CAAC,EAAE,MAAM,CAAC;QAClB,OAAO,EAAE,OAAO,CAAC;QACjB,eAAe,CAAC,EAAE,KAAK,CAAC;YAAE,EAAE,EAAE,MAAM,CAAC;YAAC,QAAQ,CAAC,EAAE,OAAO,CAAA;SAAE,CAAC,CAAC;KAC7D,CAAC,CAAC;IACH,UAAU,CAAC,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IACjC,YAAY,CAAC,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;CACpC;AAED;;;;;;GAMG;AACH,wBAAgB,mBAAmB,CAAC,OAAO,GAAE,gBAAqB,GAAG,aAAa,CAiEjF;AAED,OAAO,EAAE,kBAAkB,EAAE,UAAU,EAAE,MAAM,2BAA2B,CAAC;AAC3E,YAAY,EAAE,mBAAmB,EAAE,oBAAoB,EAAE,UAAU,EAAE,MAAM,2BAA2B,CAAC;AACvG,eAAe,mBAAmB,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AACA;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AASH,0DAA0D;AAC1D,MAAM,WAAW,gBAAgB;IAC/B;;;;OAIG;IACH,OAAO,CAAC,EAAE;QAAE,QAAQ,CAAC,EAAE,MAAM,CAAC;QAAC,YAAY,CAAC,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,EAAE,MAAM,CAAA;KAAE,GAAG,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACpG,+CAA+C;IAC/C,SAAS,CAAC,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,MAAM,GAAG,SAAS,CAAC;IAChD,MAAM,CAAC,EAAE;QAAE,IAAI,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,IAAI,CAAC;QAAC,IAAI,CAAC,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,IAAI,CAAA;KAAE,CAAC;CACxE;AAED,2EAA2E;AAC3E,MAAM,WAAW,aAAa;IAC5B,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,MAAM,CAAC;IAChB,WAAW,EAAE,KAAK,CAAC;QACjB,EAAE,EAAE,MAAM,CAAC;QACX,IAAI,EAAE,MAAM,CAAC;QACb,QAAQ,CAAC,EAAE,MAAM,CAAC;QAClB,OAAO,EAAE,OAAO,CAAC;QACjB,eAAe,CAAC,EAAE,KAAK,CAAC;YAAE,EAAE,EAAE,MAAM,CAAC;YAAC,QAAQ,CAAC,EAAE,OAAO,CAAA;SAAE,CAAC,CAAC;KAC7D,CAAC,CAAC;IACH,UAAU,CAAC,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IACjC,YAAY,CAAC,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;CACpC;AAED;;;;;;GAMG;AACH,wBAAgB,mBAAmB,CAAC,OAAO,GAAE,gBAAqB,GAAG,aAAa,CAiEjF;AAED,OAAO,EAAE,kBAAkB,EAAE,UAAU,EAAE,MAAM,2BAA2B,CAAC;AAC3E,YAAY,EAAE,mBAAmB,EAAE,oBAAoB,EAAE,UAAU,EAAE,MAAM,2BAA2B,CAAC;AACvG,eAAe,mBAAmB,CAAC"}

package/dist/index.js

@@ -13,10 +13,21 @@
  * The cloud tier reads OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY (or
  * GEMINI_API_KEY) or OPENROUTER_API_KEY from the environment.
  *
+ * The pipelines are built without the embedding tier. With it, every text
+ * call would run a CLIP embedding and wait for it, to return a result that
+ * has no place for the vector; the first such call downloads a 350 MB model.
+ * The `embed` mode loads CLIP when it is first asked for.
+ *
+ * An image is an http(s) URL, a data URL, or the `file:` URL of an image the
+ * image-generation or image-editing tools saved: the three packs share the
+ * images directory (`imageDir`, `AGENTOS_IMAGE_DIR`, or a folder in the
+ * user's temp directory).
+ *
  * @module @framers/agentos-ext-vision-pipeline
  */
 import { createRequire } from 'node:module';
 import { createVisionPipeline } from '@framers/agentos';
+import { imageStore } from './imageFiles.js';
 import { VisionPipelineTool } from './tools/visionPipeline.js';
 const { version } = createRequire(import.meta.url)('../package.json');
 /**
@@ -43,7 +54,7 @@ export function createExtensionPack(context = {}) {
             return Promise.reject(new Error('The Vision & OCR Pipeline pack is deactivated.'));
         let pipeline = pipelines.get(strategy);
         if (!pipeline) {
-            pipeline = createVisionPipeline({ strategy, ...cloud });
+            pipeline = createVisionPipeline({ strategy, embedding: false, ...cloud });
             pipelines.set(strategy, pipeline);
             const built = pipeline;
             built.catch(() => {
@@ -53,7 +64,7 @@ export function createExtensionPack(context = {}) {
         }
         return pipeline;
     };
-    const tool = new VisionPipelineTool(pipelineFor);
+    const tool = new VisionPipelineTool(pipelineFor, { dir: imageStore({ imageDir: context.options?.imageDir }).dir });
     return {
         name: '@framers/agentos-ext-vision-pipeline',
         version,

package/dist/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,cAAc;AACd;;;;;;;;;;;;;;;GAeG;AAEH,OAAO,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AAC5C,OAAO,EAAE,oBAAoB,EAAE,MAAM,kBAAkB,CAAC;AACxD,OAAO,EAAE,kBAAkB,EAAuB,MAAM,2BAA2B,CAAC;AAEpF,MAAM,EAAE,OAAO,EAAE,GAAG,aAAa,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,iBAAiB,CAAC,CAAC;AA0BtE;;;;;;GAMG;AACH,MAAM,UAAU,mBAAmB,CAAC,UAA4B,EAAE;IAChE,8EAA8E;IAC9E,6EAA6E;IAC7E,4EAA4E;IAC5E,oDAAoD;IACpD,MAAM,QAAQ,GAAG,CAAC,GAAG,MAAiB,EAAE,EAAE,CACxC,MAAM,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,OAAO,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,SAAS,CAAC;IACpG,MAAM,SAAS,GAAG,QAAQ,CAAC,OAAO,CAAC,OAAO,EAAE,YAAY,EAAE,OAAO,CAAC,SAAS,EAAE,CAAC,eAAe,CAAC,CAAC,CAAC;IAChG,MAAM,KAAK,GAAG,SAAS,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,QAAQ,EAAE,WAAW,EAAE,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;IACnF,MAAM,SAAS,GAAG,IAAI,GAAG,EAAgC,CAAC;IAC1D,uEAAuE;IACvE,2CAA2C;IAC3C,IAAI,WAAW,GAAG,KAAK,CAAC;IACxB,MAAM,WAAW,GAAG,CAAC,QAAwB,EAAE,EAAE;QAC/C,IAAI,WAAW;YAAE,OAAO,OAAO,CAAC,MAAM,CAAC,IAAI,KAAK,CAAC,gDAAgD,CAAC,CAAC,CAAC;QACpG,IAAI,QAAQ,GAAG,SAAS,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;QACvC,IAAI,CAAC,QAAQ,EAAE,CAAC;YACd,QAAQ,GAAG,oBAAoB,CAAC,EAAE,QAAQ,EAAE,GAAG,KAAK,EAAE,CAAC,CAAC;YACxD,SAAS,CAAC,GAAG,CAAC,QAAQ,EAAE,QAAQ,CAAC,CAAC;YAClC,MAAM,KAAK,GAAG,QAAQ,CAAC;YACvB,KAAK,CAAC,KAAK,CAAC,GAAG,EAAE;gBACf,IAAI,SAAS,CAAC,GAAG,CAAC,QAAQ,CAAC,KAAK,KAAK;oBAAE,SAAS,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;YACpE,CAAC,CAAC,CAAC;QACL,CAAC;QACD,OAAO,QAAQ,CAAC;IAClB,CAAC,CAAC;IACF,MAAM,IAAI,GAAG,IAAI,kBAAkB,CAAC,WAAW,CAAC,CAAC;IAEjD,OAAO;QACL,IAAI,EAAE,sCAAsC;QAC5C,OAAO;QACP,WAAW,EAAE;YACX;gBACE,EAAE,EAAE,IAAI,CAAC,IAAI;gBACb,IAAI,EAAE,MAAM;gBACZ,QAAQ,EAAE,OAAO,CAAC,OAAO,EAAE,QAAQ,IAAI,EAAE;gBACzC,OAAO,EAAE,IAAI;gBACb,eAAe,EAAE,CAAC,EAAE,EAAE,EAAE,eAAe,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC;aAC3D;SACF;QACD,UAAU,EAAE,KAAK,IAAI,EAAE;YACrB,WAAW,GAAG,KAAK,CAAC;YACpB,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC,2CAA2C,CAAC,CAAC;QACpE,CAAC;QACD,YAAY,EAAE,KAAK,IAAI,EAAE;YACvB,WAAW,GAAG,IAAI,CAAC;YACnB,MAAM,KAAK,GAAG,CAAC,GAAG,SAAS,CAAC,MAAM,EAAE,CAAC,CAAC;YACtC,SAAS,CAAC,KAAK,EAAE,CAAC;YAClB,KAAK,MAAM,QAAQ,IAAI,KAAK,EAAE,CAAC;gBAC7B,IAAI,QAAQ,CAAC;gBACb,IAAI,CAAC;oBACH,QAAQ,GAAG,MAAM,QAAQ,CAAC;gBAC5B,CAAC;gBAAC,MAAM,CAAC;oBACP,SAAS,CAAC,0DAA0D;gBACtE,CAAC;gBACD,IAAI,CAAC;oBACH,MAAM,QAAQ,CAAC,OAAO,EAAE,CAAC;gBAC3B,CAAC;gBAAC,OAAO,KAAK,EAAE,CAAC;oBACf,MAAM,GAAG,GAAG,OAAO,CAAC,MAAM,EAAE,IAAI,IAAI,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC;oBACzD,GAAG,EAAE,IAAI,CAAC,OAAO,CAAC,MAAM,EAAE,0DAA0D,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;gBAChJ,CAAC;YACH,CAAC;YACD,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC,6CAA6C,CAAC,CAAC;QACtE,CAAC;KACF,CAAC;AACJ,CAAC;AAED,OAAO,EAAE,kBAAkB,EAAE,UAAU,EAAE,MAAM,2BAA2B,CAAC;AAE3E,eAAe,mBAAmB,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,cAAc;AACd;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AAEH,OAAO,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AAC5C,OAAO,EAAE,oBAAoB,EAAE,MAAM,kBAAkB,CAAC;AACxD,OAAO,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AAC7C,OAAO,EAAE,kBAAkB,EAAuB,MAAM,2BAA2B,CAAC;AAEpF,MAAM,EAAE,OAAO,EAAE,GAAG,aAAa,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,iBAAiB,CAAC,CAAC;AA8BtE;;;;;;GAMG;AACH,MAAM,UAAU,mBAAmB,CAAC,UAA4B,EAAE;IAChE,8EAA8E;IAC9E,6EAA6E;IAC7E,4EAA4E;IAC5E,oDAAoD;IACpD,MAAM,QAAQ,GAAG,CAAC,GAAG,MAAiB,EAAE,EAAE,CACxC,MAAM,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,OAAO,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,SAAS,CAAC;IACpG,MAAM,SAAS,GAAG,QAAQ,CAAC,OAAO,CAAC,OAAO,EAAE,YAAY,EAAE,OAAO,CAAC,SAAS,EAAE,CAAC,eAAe,CAAC,CAAC,CAAC;IAChG,MAAM,KAAK,GAAG,SAAS,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,QAAQ,EAAE,WAAW,EAAE,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;IACnF,MAAM,SAAS,GAAG,IAAI,GAAG,EAAgC,CAAC;IAC1D,uEAAuE;IACvE,2CAA2C;IAC3C,IAAI,WAAW,GAAG,KAAK,CAAC;IACxB,MAAM,WAAW,GAAG,CAAC,QAAwB,EAAE,EAAE;QAC/C,IAAI,WAAW;YAAE,OAAO,OAAO,CAAC,MAAM,CAAC,IAAI,KAAK,CAAC,gDAAgD,CAAC,CAAC,CAAC;QACpG,IAAI,QAAQ,GAAG,SAAS,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;QACvC,IAAI,CAAC,QAAQ,EAAE,CAAC;YACd,QAAQ,GAAG,oBAAoB,CAAC,EAAE,QAAQ,EAAE,SAAS,EAAE,KAAK,EAAE,GAAG,KAAK,EAAE,CAAC,CAAC;YAC1E,SAAS,CAAC,GAAG,CAAC,QAAQ,EAAE,QAAQ,CAAC,CAAC;YAClC,MAAM,KAAK,GAAG,QAAQ,CAAC;YACvB,KAAK,CAAC,KAAK,CAAC,GAAG,EAAE;gBACf,IAAI,SAAS,CAAC,GAAG,CAAC,QAAQ,CAAC,KAAK,KAAK;oBAAE,SAAS,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;YACpE,CAAC,CAAC,CAAC;QACL,CAAC;QACD,OAAO,QAAQ,CAAC;IAClB,CAAC,CAAC;IACF,MAAM,IAAI,GAAG,IAAI,kBAAkB,CAAC,WAAW,EAAE,EAAE,GAAG,EAAE,UAAU,CAAC,EAAE,QAAQ,EAAE,OAAO,CAAC,OAAO,EAAE,QAAQ,EAAE,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC;IAEnH,OAAO;QACL,IAAI,EAAE,sCAAsC;QAC5C,OAAO;QACP,WAAW,EAAE;YACX;gBACE,EAAE,EAAE,IAAI,CAAC,IAAI;gBACb,IAAI,EAAE,MAAM;gBACZ,QAAQ,EAAE,OAAO,CAAC,OAAO,EAAE,QAAQ,IAAI,EAAE;gBACzC,OAAO,EAAE,IAAI;gBACb,eAAe,EAAE,CAAC,EAAE,EAAE,EAAE,eAAe,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC;aAC3D;SACF;QACD,UAAU,EAAE,KAAK,IAAI,EAAE;YACrB,WAAW,GAAG,KAAK,CAAC;YACpB,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC,2CAA2C,CAAC,CAAC;QACpE,CAAC;QACD,YAAY,EAAE,KAAK,IAAI,EAAE;YACvB,WAAW,GAAG,IAAI,CAAC;YACnB,MAAM,KAAK,GAAG,CAAC,GAAG,SAAS,CAAC,MAAM,EAAE,CAAC,CAAC;YACtC,SAAS,CAAC,KAAK,EAAE,CAAC;YAClB,KAAK,MAAM,QAAQ,IAAI,KAAK,EAAE,CAAC;gBAC7B,IAAI,QAAQ,CAAC;gBACb,IAAI,CAAC;oBACH,QAAQ,GAAG,MAAM,QAAQ,CAAC;gBAC5B,CAAC;gBAAC,MAAM,CAAC;oBACP,SAAS,CAAC,0DAA0D;gBACtE,CAAC;gBACD,IAAI,CAAC;oBACH,MAAM,QAAQ,CAAC,OAAO,EAAE,CAAC;gBAC3B,CAAC;gBAAC,OAAO,KAAK,EAAE,CAAC;oBACf,MAAM,GAAG,GAAG,OAAO,CAAC,MAAM,EAAE,IAAI,IAAI,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC;oBACzD,GAAG,EAAE,IAAI,CAAC,OAAO,CAAC,MAAM,EAAE,0DAA0D,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;gBAChJ,CAAC;YACH,CAAC;YACD,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC,6CAA6C,CAAC,CAAC;QACtE,CAAC;KACF,CAAC;AACJ,CAAC;AAED,OAAO,EAAE,kBAAkB,EAAE,UAAU,EAAE,MAAM,2BAA2B,CAAC;AAE3E,eAAe,mBAAmB,CAAC"}

package/dist/tools/visionPipeline.d.ts

@@ -3,11 +3,12 @@
  * image description and CLIP embeddings through an AgentOS VisionPipeline.
  */
 import type { ITool, JSONSchemaObject, ToolExecutionContext, ToolExecutionResult } from '@framers/agentos';
+import { type ImageStore } from '../imageFiles.js';
 /** What the tool reads from an image; see {@link VisionPipelineTool}. */
 export type VisionMode = 'ocr' | 'handwriting' | 'layout' | 'describe' | 'embed' | 'auto';
 /** The vision-pipeline tool's input. */
 export interface VisionPipelineInput {
-    /** The image: a data:image URL, or an http(s) URL on a public host. */
+    /** The image: a data:image URL, an http(s) URL on a public host, or the file: URL of an image the image tools saved. */
     imageUrl: string;
     /** What to read (default `auto`). */
     mode?: VisionMode;
@@ -24,6 +25,13 @@ export interface VisionPipelineOutput {
     tiers?: string[];
     /** How many text regions the winning tier located. */
     regions?: number;
+    /** The document layout, when the layout tier ran: one page, a text block per line with its box. */
+    layout?: unknown;
+    /** The tiers that were due to run and failed, each with its error. */
+    failedTiers?: {
+        tier: string;
+        error: string;
+    }[];
     durationMs?: number;
     /** For embed: the CLIP vector. */
     embedding?: number[];
@@ -49,26 +57,37 @@ export type PipelineFor = (strategy: VisionStrategy) => Promise<{
  */
 export declare function isPrivateHost(hostname: string): boolean;
 /**
- * The image as the pipeline takes it: a data URL decoded to its bytes, an
- * http(s) URL whose host is not this machine or a private network as is, or
- * `undefined` for anything else. AgentOS also reads local file paths and
- * fetches any URL, but a tool the model calls must not send the machine's
- * files, or what its network serves, to a cloud vision model.
+ * The image a value names, when it is one a caller may hand the pipeline: a
+ * data URL decoded to its bytes, or an http(s) URL whose host is not this
+ * machine or a private network, written as the URL parser serializes it
+ * (scheme and host in lower case, dot segments resolved); `undefined` for
+ * anything else, a `file:` URL included. AgentOS also reads any local file
+ * path and fetches any URL, but a tool the model calls must not send the
+ * machine's files, or what its network serves, to a cloud vision model.
+ *
+ * The tool also takes the `file:` URL of an image the image tools saved,
+ * which it reads itself and holds to that directory (see {@link loadImage}).
  */
 export declare function imageInput(value: unknown): Buffer | string | undefined;
 /**
- * The image as the tool hands it to the pipeline. A Buffer goes on as it is. An
- * http(s) URL is fetched here through AgentOS's `imageToBuffer` with
+ * The bytes of the image a call names, for the pipeline. A Buffer goes on as
+ * it is.
+ *
+ * A `file:` URL must name an image that the image-generation or image-editing
+ * tools saved for this caller, in the images directory the packs share.
+ *
+ * An http(s) URL is fetched here through AgentOS's `imageToBuffer` with
  * `untrusted: true`, which connects only to public network addresses (every
- * address the host resolves to, and every redirect, is checked) and stops at
- * 50 MiB and 30 seconds; the bytes go on. With an AgentOS without that mode
- * (it has no `isPublicNetworkAddress` export) the pipeline gets the URL,
- * already checked against the host as written.
+ * address the host resolves to, and every redirect, is checked) and holds the
+ * image to 50 MiB and 30 seconds. An AgentOS without that mode (it has no
+ * `isPublicNetworkAddress` export; the mode came in 0.13.16) would fetch the
+ * URL unchecked, so an http(s) image is refused there.
  */
-export declare function loadImage(image: Buffer | string): Promise<Buffer | string>;
+export declare function loadImage(image: Buffer | string, store?: ImageStore, context?: unknown): Promise<Buffer>;
 /** The vision-pipeline tool, on an AgentOS `VisionPipeline` per strategy. */
 export declare class VisionPipelineTool implements ITool<VisionPipelineInput, VisionPipelineOutput> {
     private readonly pipelineFor;
+    private readonly store?;
     readonly id = "tool.vision-pipeline";
     readonly name = "vision-pipeline";
     readonly displayName = "Vision & OCR Pipeline";
@@ -76,7 +95,11 @@ export declare class VisionPipelineTool implements ITool<VisionPipelineInput, Vi
     readonly category = "media";
     readonly hasSideEffects = false;
     readonly inputSchema: JSONSchemaObject;
-    constructor(pipelineFor: PipelineFor);
-    execute(args: VisionPipelineInput, _context?: ToolExecutionContext): Promise<ToolExecutionResult<VisionPipelineOutput>>;
+    /**
+     * @param pipelineFor - Returns the pipeline for a strategy.
+     * @param store - The images directory, for `file:` sources. Without one, no `file:` URL is read.
+     */
+    constructor(pipelineFor: PipelineFor, store?: ImageStore | undefined);
+    execute(args: VisionPipelineInput, context?: ToolExecutionContext): Promise<ToolExecutionResult<VisionPipelineOutput>>;
 }
 //# sourceMappingURL=visionPipeline.d.ts.map

package/dist/tools/visionPipeline.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"visionPipeline.d.ts","sourceRoot":"","sources":["../../src/tools/visionPipeline.ts"],"names":[],"mappings":"AACA;;;GAGG;AAGH,OAAO,KAAK,EAAE,KAAK,EAAE,gBAAgB,EAAE,oBAAoB,EAAE,mBAAmB,EAAE,MAAM,kBAAkB,CAAC;AAE3G,yEAAyE;AACzE,MAAM,MAAM,UAAU,GAAG,KAAK,GAAG,aAAa,GAAG,QAAQ,GAAG,UAAU,GAAG,OAAO,GAAG,MAAM,CAAC;AAE1F,wCAAwC;AACxC,MAAM,WAAW,mBAAmB;IAClC,uEAAuE;IACvE,QAAQ,EAAE,MAAM,CAAC;IACjB,qCAAqC;IACrC,IAAI,CAAC,EAAE,UAAU,CAAC;IAClB,6FAA6F;IAC7F,OAAO,CAAC,EAAE,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;CACrB;AAED,oFAAoF;AACpF,MAAM,WAAW,oBAAoB;IACnC,IAAI,EAAE,UAAU,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,mDAAmD;IACnD,KAAK,CAAC,EAAE,MAAM,EAAE,CAAC;IACjB,sDAAsD;IACtD,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,kCAAkC;IAClC,SAAS,CAAC,EAAE,MAAM,EAAE,CAAC;IACrB,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB;AAED,wGAAwG;AACxG,MAAM,MAAM,cAAc,GAAG,aAAa,GAAG,YAAY,CAAC;AAE1D,8DAA8D;AAC9D,MAAM,MAAM,WAAW,GAAG,CAAC,QAAQ,EAAE,cAAc,KAAK,OAAO,CAAC;IAC9D,OAAO,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,EAAE,OAAO,CAAC,EAAE;QAAE,aAAa,CAAC,EAAE,MAAM,CAAC;QAAC,KAAK,CAAC,EAAE,MAAM,EAAE,CAAA;KAAE,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC;IACtG,KAAK,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC;CAClD,CAAC,CAAC;AAqFH;;;;;;;GAOG;AACH,wBAAgB,aAAa,CAAC,QAAQ,EAAE,MAAM,GAAG,OAAO,CAQvD;AAiCD;;;;;;GAMG;AACH,wBAAgB,UAAU,CAAC,KAAK,EAAE,OAAO,GAAG,MAAM,GAAG,MAAM,GAAG,SAAS,CAetE;AAED;;;;;;;;GAQG;AACH,wBAAsB,SAAS,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,MAAM,CAAC,CAIhF;AAED,6EAA6E;AAC7E,qBAAa,kBAAmB,YAAW,KAAK,CAAC,mBAAmB,EAAE,oBAAoB,CAAC;IA8B7E,OAAO,CAAC,QAAQ,CAAC,WAAW;IA7BxC,QAAQ,CAAC,EAAE,0BAA0B;IACrC,QAAQ,CAAC,IAAI,qBAAqB;IAClC,QAAQ,CAAC,WAAW,2BAA2B;IAC/C,QAAQ,CAAC,WAAW,SAE8F;IAClH,QAAQ,CAAC,QAAQ,WAAW;IAC5B,QAAQ,CAAC,cAAc,SAAS;IAEhC,QAAQ,CAAC,WAAW,EAAE,gBAAgB,CAkBpC;gBAE2B,WAAW,EAAE,WAAW;IAE/C,OAAO,CAAC,IAAI,EAAE,mBAAmB,EAAE,QAAQ,CAAC,EAAE,oBAAoB,GAAG,OAAO,CAAC,mBAAmB,CAAC,oBAAoB,CAAC,CAAC;CAmD9H"}
+{"version":3,"file":"visionPipeline.d.ts","sourceRoot":"","sources":["../../src/tools/visionPipeline.ts"],"names":[],"mappings":"AACA;;;GAGG;AAGH,OAAO,KAAK,EAAE,KAAK,EAAE,gBAAgB,EAAE,oBAAoB,EAAE,mBAAmB,EAAE,MAAM,kBAAkB,CAAC;AAE3G,OAAO,EAA2B,KAAK,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAE5E,yEAAyE;AACzE,MAAM,MAAM,UAAU,GAAG,KAAK,GAAG,aAAa,GAAG,QAAQ,GAAG,UAAU,GAAG,OAAO,GAAG,MAAM,CAAC;AAE1F,wCAAwC;AACxC,MAAM,WAAW,mBAAmB;IAClC,wHAAwH;IACxH,QAAQ,EAAE,MAAM,CAAC;IACjB,qCAAqC;IACrC,IAAI,CAAC,EAAE,UAAU,CAAC;IAClB,6FAA6F;IAC7F,OAAO,CAAC,EAAE,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;CACrB;AAED,oFAAoF;AACpF,MAAM,WAAW,oBAAoB;IACnC,IAAI,EAAE,UAAU,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,mDAAmD;IACnD,KAAK,CAAC,EAAE,MAAM,EAAE,CAAC;IACjB,sDAAsD;IACtD,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,mGAAmG;IACnG,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,sEAAsE;IACtE,WAAW,CAAC,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;IAChD,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,kCAAkC;IAClC,SAAS,CAAC,EAAE,MAAM,EAAE,CAAC;IACrB,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB;AAED,wGAAwG;AACxG,MAAM,MAAM,cAAc,GAAG,aAAa,GAAG,YAAY,CAAC;AAE1D,8DAA8D;AAC9D,MAAM,MAAM,WAAW,GAAG,CAAC,QAAQ,EAAE,cAAc,KAAK,OAAO,CAAC;IAC9D,OAAO,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,EAAE,OAAO,CAAC,EAAE;QAAE,aAAa,CAAC,EAAE,MAAM,CAAC;QAAC,KAAK,CAAC,EAAE,MAAM,EAAE,CAAA;KAAE,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC;IACtG,KAAK,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC;CAClD,CAAC,CAAC;AAqFH;;;;;;;GAOG;AACH,wBAAgB,aAAa,CAAC,QAAQ,EAAE,MAAM,GAAG,OAAO,CAQvD;AAiCD;;;;;;;;;;;GAWG;AACH,wBAAgB,UAAU,CAAC,KAAK,EAAE,OAAO,GAAG,MAAM,GAAG,MAAM,GAAG,SAAS,CAGtE;AA8BD;;;;;;;;;;;;;GAaG;AACH,wBAAsB,SAAS,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,EAAE,KAAK,CAAC,EAAE,UAAU,EAAE,OAAO,CAAC,EAAE,OAAO,GAAG,OAAO,CAAC,MAAM,CAAC,CAY9G;AAED,6EAA6E;AAC7E,qBAAa,kBAAmB,YAAW,KAAK,CAAC,mBAAmB,EAAE,oBAAoB,CAAC;IAsCvF,OAAO,CAAC,QAAQ,CAAC,WAAW;IAC5B,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAC;IAtCzB,QAAQ,CAAC,EAAE,0BAA0B;IACrC,QAAQ,CAAC,IAAI,qBAAqB;IAClC,QAAQ,CAAC,WAAW,2BAA2B;IAC/C,QAAQ,CAAC,WAAW,SAE8F;IAClH,QAAQ,CAAC,QAAQ,WAAW;IAC5B,QAAQ,CAAC,cAAc,SAAS;IAEhC,QAAQ,CAAC,WAAW,EAAE,gBAAgB,CAqBpC;IAEF;;;OAGG;gBAEgB,WAAW,EAAE,WAAW,EACxB,KAAK,CAAC,EAAE,UAAU,YAAA;IAG/B,OAAO,CAAC,IAAI,EAAE,mBAAmB,EAAE,OAAO,CAAC,EAAE,oBAAoB,GAAG,OAAO,CAAC,mBAAmB,CAAC,oBAAoB,CAAC,CAAC;CA6D7H"}

package/dist/tools/visionPipeline.js

@@ -4,6 +4,7 @@
  * image description and CLIP embeddings through an AgentOS VisionPipeline.
  */
 import * as agentos from '@framers/agentos';
+import { readSavedImage, scopeOf } from '../imageFiles.js';
 /**
  * The lowest tier each mode needs: 1 is local OCR, 2 local vision models
  * (TrOCR, Florence-2, CLIP), 3 cloud vision.
@@ -63,35 +64,32 @@ function ipv6Groups(address) {
     return groups.map((group) => parseInt(group, 16));
 }
 /**
- * Whether an IPv6 address is off the public internet, the IPv4 address it
- * carries included: unspecified, loopback, IPv4-compatible and IPv4-mapped,
- * NAT64 (64:ff9b::/96 and 64:ff9b:1::/48), 6to4 (2002::/16), discard and
- * dummy (100::/63), the IETF protocol assignments (2001::/23), documentation
- * (2001:db8::/32 and 3fff::/20), SRv6 segment identifiers (5f00::/16),
- * unique local, link-local,
- * site-local and multicast.
+ * Whether an IPv6 address is off the public internet. Only 2000::/3 is
+ * allocated for global unicast, so everything outside it is refused
+ * (unspecified, loopback, the deprecated IPv4-compatible form, discard, SRv6,
+ * unique local, link-local, site-local, multicast, and space no one has been
+ * given), except the two forms that reach an IPv4 address and are judged by
+ * it: IPv4-mapped (::ffff:0:0/96) and the NAT64 well-known prefix
+ * (64:ff9b::/96). Inside 2000::/3, 6to4 (2002::/16) is judged by the address
+ * it carries, and the IETF protocol assignments (2001::/23) and documentation
+ * (2001:db8::/32 and 3fff::/20) are refused. The same rule as AgentOS's
+ * `isPublicNetworkAddress`.
  */
 function isNonPublicIPv6(groups) {
     const carried = (high, low) => isNonPublicIPv4([high >> 8, high & 255, low >> 8, low & 255]);
-    const [g0, g1, g2, g3, g4, g5, g6, g7] = groups;
+    const [g0, g1, g2, , , g5, g6, g7] = groups;
     const zeros = (from, to) => groups.slice(from, to).every((group) => group === 0);
-    if (zeros(0, 6))
-        return (g6 === 0 && g7 <= 1) || carried(g6, g7);
     if (zeros(0, 5) && g5 === 0xffff)
         return carried(g6, g7);
-    if (g0 === 0x64 && g1 === 0xff9b)
-        return g2 === 1 || !zeros(2, 6) || carried(g6, g7);
+    if (g0 === 0x64 && g1 === 0xff9b && zeros(2, 6))
+        return carried(g6, g7);
+    if ((g0 & 0xe000) !== 0x2000)
+        return true;
     if (g0 === 0x2002)
         return carried(g1, g2);
-    if (g0 === 0x100 && g1 === 0 && g2 === 0 && g3 <= 1)
-        return true;
     if (g0 === 0x2001 && (g1 <= 0x01ff || g1 === 0xdb8))
         return true;
-    if (g0 === 0x3fff && g1 <= 0x0fff)
-        return true;
-    if (g0 === 0x5f00)
-        return true;
-    return (g0 & 0xfe00) === 0xfc00 || (g0 & 0xffc0) === 0xfe80 || (g0 & 0xffc0) === 0xfec0 || (g0 & 0xff00) === 0xff00;
+    return g0 === 0x3fff && g1 <= 0x0fff;
 }
 /**
  * Whether a URL's host is this machine or off the public internet: `localhost`
@@ -147,16 +145,28 @@ function percentDecode(payload) {
     return out.subarray(0, length);
 }
 /**
- * The image as the pipeline takes it: a data URL decoded to its bytes, an
- * http(s) URL whose host is not this machine or a private network as is, or
- * `undefined` for anything else. AgentOS also reads local file paths and
- * fetches any URL, but a tool the model calls must not send the machine's
- * files, or what its network serves, to a cloud vision model.
+ * The image a value names, when it is one a caller may hand the pipeline: a
+ * data URL decoded to its bytes, or an http(s) URL whose host is not this
+ * machine or a private network, written as the URL parser serializes it
+ * (scheme and host in lower case, dot segments resolved); `undefined` for
+ * anything else, a `file:` URL included. AgentOS also reads any local file
+ * path and fetches any URL, but a tool the model calls must not send the
+ * machine's files, or what its network serves, to a cloud vision model.
+ *
+ * The tool also takes the `file:` URL of an image the image tools saved,
+ * which it reads itself and holds to that directory (see {@link loadImage}).
  */
 export function imageInput(value) {
+    const source = sourceOf(value);
+    return typeof source === 'string' && /^file:/i.test(source) ? undefined : source;
+}
+/** The image a call names, as {@link imageInput} reads it, and a `file:` URL as it was written. */
+function sourceOf(value) {
     if (typeof value !== 'string')
         return undefined;
     const source = value.trim();
+    if (/^file:/i.test(source))
+        return source;
     if (/^https?:\/\//i.test(source)) {
         let url;
         try {
@@ -165,31 +175,53 @@ export function imageInput(value) {
         catch {
             return undefined;
         }
-        return isPrivateHost(url.hostname) ? undefined : source;
+        return isPrivateHost(url.hostname) ? undefined : url.href;
     }
-    const data = /^data:image\/[^,]*?(;base64)?,(.*)$/is.exec(source);
-    if (!data)
+    // A data URL as WHATWG Fetch reads one, and as AgentOS's imageToBuffer
+    // does: tabs and line breaks dropped first, then base64 when the media type
+    // ends in `;base64`, spaces around it allowed.
+    const url = source.replace(/[\t\n\r]/g, '');
+    const comma = url.indexOf(',');
+    if (!/^data:image\//i.test(url) || comma === -1)
         return undefined;
-    return data[1] ? Buffer.from(data[2], 'base64') : percentDecode(data[2]);
+    const payload = url.slice(comma + 1);
+    return /; *base64 *$/i.test(url.slice(0, comma)) ? Buffer.from(payload, 'base64') : percentDecode(payload);
 }
+/** The error for an input that is not an accepted image source. */
+const SOURCE_ERROR = 'imageUrl must be a data:image URL, an http(s) URL on a public host, or the file: URL of an image the image tools saved: other local files are not read, nor local or private network addresses.';
 /**
- * The image as the tool hands it to the pipeline. A Buffer goes on as it is. An
- * http(s) URL is fetched here through AgentOS's `imageToBuffer` with
+ * The bytes of the image a call names, for the pipeline. A Buffer goes on as
+ * it is.
+ *
+ * A `file:` URL must name an image that the image-generation or image-editing
+ * tools saved for this caller, in the images directory the packs share.
+ *
+ * An http(s) URL is fetched here through AgentOS's `imageToBuffer` with
  * `untrusted: true`, which connects only to public network addresses (every
- * address the host resolves to, and every redirect, is checked) and stops at
- * 50 MiB and 30 seconds; the bytes go on. With an AgentOS without that mode
- * (it has no `isPublicNetworkAddress` export) the pipeline gets the URL,
- * already checked against the host as written.
+ * address the host resolves to, and every redirect, is checked) and holds the
+ * image to 50 MiB and 30 seconds. An AgentOS without that mode (it has no
+ * `isPublicNetworkAddress` export; the mode came in 0.13.16) would fetch the
+ * URL unchecked, so an http(s) image is refused there.
  */
-export async function loadImage(image) {
-    const { imageToBuffer, isPublicNetworkAddress } = agentos;
-    if (typeof image !== 'string' || typeof isPublicNetworkAddress !== 'function')
+export async function loadImage(image, store, context) {
+    if (typeof image !== 'string')
         return image;
+    if (/^file:/i.test(image)) {
+        const bytes = store ? await readSavedImage(image, store.dir, scopeOf(context)) : undefined;
+        if (!bytes)
+            throw new Error(SOURCE_ERROR);
+        return bytes;
+    }
+    const { imageToBuffer, isPublicNetworkAddress } = agentos;
+    if (typeof isPublicNetworkAddress !== 'function') {
+        throw new Error('imageUrl is an http(s) URL, which needs @framers/agentos 0.13.16 or later: pass a data:image URL, or update AgentOS.');
+    }
     return imageToBuffer(image, { untrusted: true });
 }
 /** The vision-pipeline tool, on an AgentOS `VisionPipeline` per strategy. */
 export class VisionPipelineTool {
     pipelineFor;
+    store;
     id = 'tool.vision-pipeline';
     name = 'vision-pipeline';
     displayName = 'Vision & OCR Pipeline';
@@ -200,11 +232,14 @@ export class VisionPipelineTool {
     inputSchema = {
         type: 'object',
         properties: {
-            imageUrl: { type: 'string', description: 'The image: an http(s) URL or a data:image URL.' },
+            imageUrl: {
+                type: 'string',
+                description: 'The image: an http(s) URL, a data:image URL, or the file: URL of an image the image tools saved.',
+            },
             mode: {
                 type: 'string',
                 enum: ['ocr', 'handwriting', 'layout', 'describe', 'embed', 'auto'],
-                description: 'ocr for printed text, handwriting, layout for document structure, describe for what the image shows, ' +
+                description: 'ocr for printed text, handwriting, layout for the lines of text with their boxes, describe for what the image shows, ' +
                     'embed for a CLIP vector, or auto (default) to let the pipeline decide.',
             },
             maxTier: {
@@ -215,38 +250,53 @@ export class VisionPipelineTool {
         },
         required: ['imageUrl'],
     };
-    constructor(pipelineFor) {
+    /**
+     * @param pipelineFor - Returns the pipeline for a strategy.
+     * @param store - The images directory, for `file:` sources. Without one, no `file:` URL is read.
+     */
+    constructor(pipelineFor, store) {
         this.pipelineFor = pipelineFor;
+        this.store = store;
     }
-    async execute(args, _context) {
-        const image = imageInput(args.imageUrl);
-        if (image === undefined) {
-            return {
-                success: false,
-                error: 'imageUrl must be a data:image URL or an http(s) URL on a public host: local file paths are not read, nor local or private network addresses.',
-            };
-        }
+    async execute(args, context) {
+        const image = sourceOf(args.imageUrl);
+        if (image === undefined)
+            return { success: false, error: SOURCE_ERROR };
         const mode = args.mode ?? 'auto';
         if (typeof mode !== 'string' || !Object.hasOwn(TIER_NEEDED, mode)) {
             return { success: false, error: `mode must be one of ${Object.keys(TIER_NEEDED).join(', ')}.` };
         }
-        const maxTier = args.maxTier === 1 || args.maxTier === 2 ? args.maxTier : 3;
+        // The highest tier is the cloud: only an absent maxTier means it. A value
+        // the schema does not allow, null among them, is refused, not read as
+        // "no limit".
+        let maxTier = 3;
+        if (args.maxTier !== undefined) {
+            const given = typeof args.maxTier === 'string' && args.maxTier.trim() !== '' ? Number(args.maxTier) : args.maxTier;
+            if (given !== 1 && given !== 2 && given !== 3)
+                return { success: false, error: 'maxTier must be 1, 2 or 3.' };
+            maxTier = given;
+        }
         if (TIER_NEEDED[mode] > maxTier) {
             return { success: false, error: `${mode} needs maxTier ${TIER_NEEDED[mode]} or higher; this call allows ${maxTier}.` };
         }
         try {
-            const input = await loadImage(image);
+            const input = await loadImage(image, this.store, context);
             const pipeline = await this.pipelineFor(maxTier === 3 ? 'progressive' : 'local-only');
             if (mode === 'embed') {
                 const embedding = await pipeline.embed(input);
                 return { success: true, output: { mode, embedding, dimensions: embedding.length } };
             }
+            // handwriting and layout name their tier. With the category alone, a
+            // pipeline that may reach the cloud returns as soon as plain OCR is
+            // confident, before the tier the mode is named for has run; and a tier
+            // whose model is not installed is passed over without a word. A named
+            // tier runs, or fails with the reason.
             const options = maxTier === 1 || mode === 'ocr'
                 ? { tiers: ['ocr'] }
                 : mode === 'handwriting'
-                    ? { forceCategory: 'handwritten' }
+                    ? { tiers: ['handwriting'], forceCategory: 'handwritten' }
                     : mode === 'layout'
-                        ? { forceCategory: 'document-layout' }
+                        ? { tiers: ['document-ai'], forceCategory: 'document-layout' }
                         : mode === 'describe'
                             ? { tiers: ['cloud-vision'] }
                             : undefined;
@@ -260,6 +310,8 @@ export class VisionPipelineTool {
                     category: result.category,
                     tiers: result.tiers,
                     regions: result.regions?.length ?? 0,
+                    ...(result.layout ? { layout: result.layout } : {}),
+                    ...(result.failedTiers?.length ? { failedTiers: result.failedTiers } : {}),
                     durationMs: result.durationMs,
                 },
             };

package/dist/tools/visionPipeline.js.map

@@ -1 +1 @@
-{"version":3,"file":"visionPipeline.js","sourceRoot":"","sources":["../../src/tools/visionPipeline.ts"],"names":[],"mappings":"AAAA,cAAc;AACd;;;GAGG;AAEH,OAAO,KAAK,OAAO,MAAM,kBAAkB,CAAC;AAyC5C;;;GAGG;AACH,MAAM,WAAW,GAAkC;IACjD,IAAI,EAAE,CAAC;IACP,GAAG,EAAE,CAAC;IACN,WAAW,EAAE,CAAC;IACd,MAAM,EAAE,CAAC;IACT,KAAK,EAAE,CAAC;IACR,QAAQ,EAAE,CAAC;CACZ,CAAC;AAEF;;;;;;GAMG;AACH,SAAS,eAAe,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAW;IAC1C,OAAO,CACL,CAAC,KAAK,CAAC;QACP,CAAC,KAAK,EAAE;QACR,CAAC,KAAK,GAAG;QACT,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,IAAI,EAAE,IAAI,CAAC,IAAI,GAAG,CAAC;QAClC,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,GAAG,CAAC;QACxB,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,IAAI,EAAE,IAAI,CAAC,IAAI,EAAE,CAAC;QACjC,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC;QAC9C,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,KAAK,EAAE,CAAC;QACnC,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,GAAG,CAAC;QACxB,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,CAAC,KAAK,EAAE,IAAI,CAAC,KAAK,EAAE,CAAC,CAAC;QACrC,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,KAAK,GAAG,CAAC;QACpC,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,GAAG,CAAC;QACnC,CAAC,IAAI,GAAG,CACT,CAAC;AACJ,CAAC;AAED,2FAA2F;AAC3F,SAAS,UAAU,CAAC,OAAe;IACjC,IAAI,IAAI,GAAG,OAAO,CAAC;IACnB,yDAAyD;IACzD,MAAM,MAAM,GAAG,mDAAmD,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC9E,IAAI,MAAM,EAAE,CAAC;QACX,MAAM,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;QACjD,IAAI,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,GAAG,CAAC;YAAE,OAAO,SAAS,CAAC;QACxD,IAAI,GAAG,GAAG,MAAM,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,EAAE,CAAC;IACrF,CAAC;IACD,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAChC,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC;QAAE,OAAO,SAAS,CAAC;IACxC,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IACnD,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,KAAK,CAAC,IAAI,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IAC1E,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC;IACrE,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,SAAS,CAAC;IACzE,MAAM,MAAM,GAAG,CAAC,GAAG,IAAI,EAAE,GAAG,KAAK,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,GAAG,IAAI,CAAC,CAAC;IAC5D,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,iBAAiB,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;QAAE,OAAO,SAAS,CAAC;IAC9E,OAAO,MAAM,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,QAAQ,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,CAAC;AACpD,CAAC;AAED;;;;;;;;GAQG;AACH,SAAS,eAAe,CAAC,MAAgB;IACvC,MAAM,OAAO,GAAG,CAAC,IAAY,EAAE,GAAW,EAAE,EAAE,CAAC,eAAe,CAAC,CAAC,IAAI,IAAI,CAAC,EAAE,IAAI,GAAG,GAAG,EAAE,GAAG,IAAI,CAAC,EAAE,GAAG,GAAG,GAAG,CAAC,CAAC,CAAC;IAC7G,MAAM,CAAC,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,CAAC,GAAG,MAAM,CAAC;IAChD,MAAM,KAAK,GAAG,CAAC,IAAY,EAAE,EAAU,EAAE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,KAAK,CAAC,CAAC,CAAC;IACjG,IAAI,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC;QAAE,OAAO,CAAC,EAAE,KAAK,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC,IAAI,OAAO,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;IACjE,IAAI,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,EAAE,KAAK,MAAM;QAAE,OAAO,OAAO,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;IACzD,IAAI,EAAE,KAAK,IAAI,IAAI,EAAE,KAAK,MAAM;QAAE,OAAO,EAAE,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,OAAO,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;IACrF,IAAI,EAAE,KAAK,MAAM;QAAE,OAAO,OAAO,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;IAC1C,IAAI,EAAE,KAAK,KAAK,IAAI,EAAE,KAAK,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,EAAE,IAAI,CAAC;QAAE,OAAO,IAAI,CAAC;IACjE,IAAI,EAAE,KAAK,MAAM,IAAI,CAAC,EAAE,IAAI,MAAM,IAAI,EAAE,KAAK,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC;IACjE,IAAI,EAAE,KAAK,MAAM,IAAI,EAAE,IAAI,MAAM;QAAE,OAAO,IAAI,CAAC;IAC/C,IAAI,EAAE,KAAK,MAAM;QAAE,OAAO,IAAI,CAAC;IAC/B,OAAO,CAAC,EAAE,GAAG,MAAM,CAAC,KAAK,MAAM,IAAI,CAAC,EAAE,GAAG,MAAM,CAAC,KAAK,MAAM,IAAI,CAAC,EAAE,GAAG,MAAM,CAAC,KAAK,MAAM,IAAI,CAAC,EAAE,GAAG,MAAM,CAAC,KAAK,MAAM,CAAC;AACtH,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,UAAU,aAAa,CAAC,QAAgB;IAC5C,MAAM,IAAI,GAAG,QAAQ,CAAC,WAAW,EAAE,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC,OAAO,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC;IAC/E,IAAI,IAAI,KAAK,WAAW,IAAI,IAAI,CAAC,QAAQ,CAAC,YAAY,CAAC;QAAE,OAAO,IAAI,CAAC;IACrE,MAAM,EAAE,GAAG,8CAA8C,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACrE,IAAI,EAAE;QAAE,OAAO,eAAe,CAAC,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC;IACxD,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC;QAAE,OAAO,KAAK,CAAC;IACtC,MAAM,MAAM,GAAG,UAAU,CAAC,IAAI,CAAC,CAAC;IAChC,OAAO,MAAM,KAAK,SAAS,IAAI,eAAe,CAAC,MAAM,CAAC,CAAC;AACzD,CAAC;AAED,8CAA8C;AAC9C,SAAS,QAAQ,CAAC,IAAY;IAC5B,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI;QAAE,OAAO,IAAI,GAAG,IAAI,CAAC;IACrD,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI;QAAE,OAAO,IAAI,GAAG,IAAI,CAAC;IACrD,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI;QAAE,OAAO,IAAI,GAAG,IAAI,CAAC;IACrD,OAAO,CAAC,CAAC,CAAC;AACZ,CAAC;AAED;;;;GAIG;AACH,SAAS,aAAa,CAAC,OAAe;IACpC,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,OAAO,EAAE,MAAM,CAAC,CAAC;IAC1C,MAAM,GAAG,GAAG,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IAC5C,IAAI,MAAM,GAAG,CAAC,CAAC;IACf,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC;QACxC,MAAM,IAAI,GAAG,IAAI,CAAC,CAAC,CAAC,KAAK,IAAI,IAAI,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QAClF,MAAM,GAAG,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACnD,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC;YACb,GAAG,CAAC,MAAM,CAAC,GAAG,IAAI,GAAG,EAAE,GAAG,GAAG,CAAC;YAC9B,CAAC,IAAI,CAAC,CAAC;QACT,CAAC;aAAM,CAAC;YACN,GAAG,CAAC,MAAM,CAAC,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACxB,CAAC;QACD,MAAM,IAAI,CAAC,CAAC;IACd,CAAC;IACD,OAAO,GAAG,CAAC,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC;AACjC,CAAC;AAED;;;;;;GAMG;AACH,MAAM,UAAU,UAAU,CAAC,KAAc;IACvC,IAAI,OAAO,KAAK,KAAK,QAAQ;QAAE,OAAO,SAAS,CAAC;IAChD,MAAM,MAAM,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC;IAC5B,IAAI,eAAe,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;QACjC,IAAI,GAAQ,CAAC;QACb,IAAI,CAAC;YACH,GAAG,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,CAAC;QACxB,CAAC;QAAC,MAAM,CAAC;YACP,OAAO,SAAS,CAAC;QACnB,CAAC;QACD,OAAO,aAAa,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,MAAM,CAAC;IAC1D,CAAC;IACD,MAAM,IAAI,GAAG,uCAAuC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IAClE,IAAI,CAAC,IAAI;QAAE,OAAO,SAAS,CAAC;IAC5B,OAAO,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC;AAC3E,CAAC;AAED;;;;;;;;GAQG;AACH,MAAM,CAAC,KAAK,UAAU,SAAS,CAAC,KAAsB;IACpD,MAAM,EAAE,aAAa,EAAE,sBAAsB,EAAE,GAAG,OAAO,CAAC;IAC1D,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,OAAO,sBAAsB,KAAK,UAAU;QAAE,OAAO,KAAK,CAAC;IAC5F,OAAO,aAAa,CAAC,KAAK,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;AACnD,CAAC;AAED,6EAA6E;AAC7E,MAAM,OAAO,kBAAkB;IA8BA;IA7BpB,EAAE,GAAG,sBAAsB,CAAC;IAC5B,IAAI,GAAG,iBAAiB,CAAC;IACzB,WAAW,GAAG,uBAAuB,CAAC;IACtC,WAAW,GAClB,8GAA8G;QAC9G,+GAA+G,CAAC;IACzG,QAAQ,GAAG,OAAO,CAAC;IACnB,cAAc,GAAG,KAAK,CAAC;IAEvB,WAAW,GAAqB;QACvC,IAAI,EAAE,QAAQ;QACd,UAAU,EAAE;YACV,QAAQ,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,gDAAgD,EAAE;YAC3F,IAAI,EAAE;gBACJ,IAAI,EAAE,QAAQ;gBACd,IAAI,EAAE,CAAC,KAAK,EAAE,aAAa,EAAE,QAAQ,EAAE,UAAU,EAAE,OAAO,EAAE,MAAM,CAAC;gBACnE,WAAW,EACT,uGAAuG;oBACvG,wEAAwE;aAC3E;YACD,OAAO,EAAE;gBACP,IAAI,EAAE,QAAQ;gBACd,IAAI,EAAE,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC;gBACf,WAAW,EAAE,6FAA6F;aAC3G;SACF;QACD,QAAQ,EAAE,CAAC,UAAU,CAAC;KACvB,CAAC;IAEF,YAA6B,WAAwB;QAAxB,gBAAW,GAAX,WAAW,CAAa;IAAG,CAAC;IAEzD,KAAK,CAAC,OAAO,CAAC,IAAyB,EAAE,QAA+B;QACtE,MAAM,KAAK,GAAG,UAAU,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;QACxC,IAAI,KAAK,KAAK,SAAS,EAAE,CAAC;YACxB,OAAO;gBACL,OAAO,EAAE,KAAK;gBACd,KAAK,EAAE,8IAA8I;aACtJ,CAAC;QACJ,CAAC;QACD,MAAM,IAAI,GAAe,IAAI,CAAC,IAAI,IAAI,MAAM,CAAC;QAC7C,IAAI,OAAO,IAAI,KAAK,QAAQ,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,WAAW,EAAE,IAAI,CAAC,EAAE,CAAC;YAClE,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,uBAAuB,MAAM,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC;QAClG,CAAC;QACD,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,KAAK,CAAC,IAAI,IAAI,CAAC,OAAO,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;QAC5E,IAAI,WAAW,CAAC,IAAI,CAAC,GAAG,OAAO,EAAE,CAAC;YAChC,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,GAAG,IAAI,kBAAkB,WAAW,CAAC,IAAI,CAAC,gCAAgC,OAAO,GAAG,EAAE,CAAC;QACzH,CAAC;QAED,IAAI,CAAC;YACH,MAAM,KAAK,GAAG,MAAM,SAAS,CAAC,KAAK,CAAC,CAAC;YACrC,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,WAAW,CAAC,OAAO,KAAK,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC;YACtF,IAAI,IAAI,KAAK,OAAO,EAAE,CAAC;gBACrB,MAAM,SAAS,GAAG,MAAM,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;gBAC9C,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,UAAU,EAAE,SAAS,CAAC,MAAM,EAAE,EAAE,CAAC;YACtF,CAAC;YACD,MAAM,OAAO,GACX,OAAO,KAAK,CAAC,IAAI,IAAI,KAAK,KAAK;gBAC7B,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,KAAK,CAAC,EAAE;gBACpB,CAAC,CAAC,IAAI,KAAK,aAAa;oBACtB,CAAC,CAAC,EAAE,aAAa,EAAE,aAAa,EAAE;oBAClC,CAAC,CAAC,IAAI,KAAK,QAAQ;wBACjB,CAAC,CAAC,EAAE,aAAa,EAAE,iBAAiB,EAAE;wBACtC,CAAC,CAAC,IAAI,KAAK,UAAU;4BACnB,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,cAAc,CAAC,EAAE;4BAC7B,CAAC,CAAC,SAAS,CAAC;YACtB,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,OAAO,CAAC,KAAK,EAAE,OAAO,CAAC,CAAC;YACtD,OAAO;gBACL,OAAO,EAAE,IAAI;gBACb,MAAM,EAAE;oBACN,IAAI;oBACJ,IAAI,EAAE,MAAM,CAAC,IAAI;oBACjB,UAAU,EAAE,MAAM,CAAC,UAAU;oBAC7B,QAAQ,EAAE,MAAM,CAAC,QAAQ;oBACzB,KAAK,EAAE,MAAM,CAAC,KAAK;oBACnB,OAAO,EAAE,MAAM,CAAC,OAAO,EAAE,MAAM,IAAI,CAAC;oBACpC,UAAU,EAAE,MAAM,CAAC,UAAU;iBAC9B;aACF,CAAC;QACJ,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC;QAC3F,CAAC;IACH,CAAC;CACF"}
+{"version":3,"file":"visionPipeline.js","sourceRoot":"","sources":["../../src/tools/visionPipeline.ts"],"names":[],"mappings":"AAAA,cAAc;AACd;;;GAGG;AAEH,OAAO,KAAK,OAAO,MAAM,kBAAkB,CAAC;AAG5C,OAAO,EAAE,cAAc,EAAE,OAAO,EAAmB,MAAM,kBAAkB,CAAC;AA4C5E;;;GAGG;AACH,MAAM,WAAW,GAAkC;IACjD,IAAI,EAAE,CAAC;IACP,GAAG,EAAE,CAAC;IACN,WAAW,EAAE,CAAC;IACd,MAAM,EAAE,CAAC;IACT,KAAK,EAAE,CAAC;IACR,QAAQ,EAAE,CAAC;CACZ,CAAC;AAEF;;;;;;GAMG;AACH,SAAS,eAAe,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAW;IAC1C,OAAO,CACL,CAAC,KAAK,CAAC;QACP,CAAC,KAAK,EAAE;QACR,CAAC,KAAK,GAAG;QACT,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,IAAI,EAAE,IAAI,CAAC,IAAI,GAAG,CAAC;QAClC,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,GAAG,CAAC;QACxB,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,IAAI,EAAE,IAAI,CAAC,IAAI,EAAE,CAAC;QACjC,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC;QAC9C,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,KAAK,EAAE,CAAC;QACnC,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,GAAG,CAAC;QACxB,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,CAAC,KAAK,EAAE,IAAI,CAAC,KAAK,EAAE,CAAC,CAAC;QACrC,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,KAAK,GAAG,CAAC;QACpC,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,GAAG,CAAC;QACnC,CAAC,IAAI,GAAG,CACT,CAAC;AACJ,CAAC;AAED,2FAA2F;AAC3F,SAAS,UAAU,CAAC,OAAe;IACjC,IAAI,IAAI,GAAG,OAAO,CAAC;IACnB,yDAAyD;IACzD,MAAM,MAAM,GAAG,mDAAmD,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC9E,IAAI,MAAM,EAAE,CAAC;QACX,MAAM,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;QACjD,IAAI,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,GAAG,CAAC;YAAE,OAAO,SAAS,CAAC;QACxD,IAAI,GAAG,GAAG,MAAM,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,EAAE,CAAC;IACrF,CAAC;IACD,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAChC,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC;QAAE,OAAO,SAAS,CAAC;IACxC,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IACnD,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,KAAK,CAAC,IAAI,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IAC1E,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC;IACrE,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,SAAS,CAAC;IACzE,MAAM,MAAM,GAAG,CAAC,GAAG,IAAI,EAAE,GAAG,KAAK,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,GAAG,IAAI,CAAC,CAAC;IAC5D,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,iBAAiB,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;QAAE,OAAO,SAAS,CAAC;IAC9E,OAAO,MAAM,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,QAAQ,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,CAAC;AACpD,CAAC;AAED;;;;;;;;;;;GAWG;AACH,SAAS,eAAe,CAAC,MAAgB;IACvC,MAAM,OAAO,GAAG,CAAC,IAAY,EAAE,GAAW,EAAE,EAAE,CAAC,eAAe,CAAC,CAAC,IAAI,IAAI,CAAC,EAAE,IAAI,GAAG,GAAG,EAAE,GAAG,IAAI,CAAC,EAAE,GAAG,GAAG,GAAG,CAAC,CAAC,CAAC;IAC7G,MAAM,CAAC,EAAE,EAAE,EAAE,EAAE,EAAE,EAAE,AAAD,EAAG,AAAD,EAAG,EAAE,EAAE,EAAE,EAAE,EAAE,CAAC,GAAG,MAAM,CAAC;IAC5C,MAAM,KAAK,GAAG,CAAC,IAAY,EAAE,EAAU,EAAE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,KAAK,CAAC,CAAC,CAAC;IACjG,IAAI,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,EAAE,KAAK,MAAM;QAAE,OAAO,OAAO,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;IACzD,IAAI,EAAE,KAAK,IAAI,IAAI,EAAE,KAAK,MAAM,IAAI,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC;QAAE,OAAO,OAAO,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;IACxE,IAAI,CAAC,EAAE,GAAG,MAAM,CAAC,KAAK,MAAM;QAAE,OAAO,IAAI,CAAC;IAC1C,IAAI,EAAE,KAAK,MAAM;QAAE,OAAO,OAAO,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;IAC1C,IAAI,EAAE,KAAK,MAAM,IAAI,CAAC,EAAE,IAAI,MAAM,IAAI,EAAE,KAAK,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC;IACjE,OAAO,EAAE,KAAK,MAAM,IAAI,EAAE,IAAI,MAAM,CAAC;AACvC,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,UAAU,aAAa,CAAC,QAAgB;IAC5C,MAAM,IAAI,GAAG,QAAQ,CAAC,WAAW,EAAE,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC,OAAO,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC;IAC/E,IAAI,IAAI,KAAK,WAAW,IAAI,IAAI,CAAC,QAAQ,CAAC,YAAY,CAAC;QAAE,OAAO,IAAI,CAAC;IACrE,MAAM,EAAE,GAAG,8CAA8C,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACrE,IAAI,EAAE;QAAE,OAAO,eAAe,CAAC,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC;IACxD,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC;QAAE,OAAO,KAAK,CAAC;IACtC,MAAM,MAAM,GAAG,UAAU,CAAC,IAAI,CAAC,CAAC;IAChC,OAAO,MAAM,KAAK,SAAS,IAAI,eAAe,CAAC,MAAM,CAAC,CAAC;AACzD,CAAC;AAED,8CAA8C;AAC9C,SAAS,QAAQ,CAAC,IAAY;IAC5B,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI;QAAE,OAAO,IAAI,GAAG,IAAI,CAAC;IACrD,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI;QAAE,OAAO,IAAI,GAAG,IAAI,CAAC;IACrD,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI;QAAE,OAAO,IAAI,GAAG,IAAI,CAAC;IACrD,OAAO,CAAC,CAAC,CAAC;AACZ,CAAC;AAED;;;;GAIG;AACH,SAAS,aAAa,CAAC,OAAe;IACpC,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,OAAO,EAAE,MAAM,CAAC,CAAC;IAC1C,MAAM,GAAG,GAAG,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IAC5C,IAAI,MAAM,GAAG,CAAC,CAAC;IACf,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC;QACxC,MAAM,IAAI,GAAG,IAAI,CAAC,CAAC,CAAC,KAAK,IAAI,IAAI,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QAClF,MAAM,GAAG,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACnD,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC;YACb,GAAG,CAAC,MAAM,CAAC,GAAG,IAAI,GAAG,EAAE,GAAG,GAAG,CAAC;YAC9B,CAAC,IAAI,CAAC,CAAC;QACT,CAAC;aAAM,CAAC;YACN,GAAG,CAAC,MAAM,CAAC,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACxB,CAAC;QACD,MAAM,IAAI,CAAC,CAAC;IACd,CAAC;IACD,OAAO,GAAG,CAAC,QAAQ,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC;AACjC,CAAC;AAED;;;;;;;;;;;GAWG;AACH,MAAM,UAAU,UAAU,CAAC,KAAc;IACvC,MAAM,MAAM,GAAG,QAAQ,CAAC,KAAK,CAAC,CAAC;IAC/B,OAAO,OAAO,MAAM,KAAK,QAAQ,IAAI,SAAS,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,MAAM,CAAC;AACnF,CAAC;AAED,mGAAmG;AACnG,SAAS,QAAQ,CAAC,KAAc;IAC9B,IAAI,OAAO,KAAK,KAAK,QAAQ;QAAE,OAAO,SAAS,CAAC;IAChD,MAAM,MAAM,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC;IAC5B,IAAI,SAAS,CAAC,IAAI,CAAC,MAAM,CAAC;QAAE,OAAO,MAAM,CAAC;IAC1C,IAAI,eAAe,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;QACjC,IAAI,GAAQ,CAAC;QACb,IAAI,CAAC;YACH,GAAG,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,CAAC;QACxB,CAAC;QAAC,MAAM,CAAC;YACP,OAAO,SAAS,CAAC;QACnB,CAAC;QACD,OAAO,aAAa,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC;IAC5D,CAAC;IACD,uEAAuE;IACvE,4EAA4E;IAC5E,+CAA+C;IAC/C,MAAM,GAAG,GAAG,MAAM,CAAC,OAAO,CAAC,WAAW,EAAE,EAAE,CAAC,CAAC;IAC5C,MAAM,KAAK,GAAG,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC/B,IAAI,CAAC,gBAAgB,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,KAAK,KAAK,CAAC,CAAC;QAAE,OAAO,SAAS,CAAC;IAClE,MAAM,OAAO,GAAG,GAAG,CAAC,KAAK,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC;IACrC,OAAO,eAAe,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,OAAO,CAAC,CAAC;AAC7G,CAAC;AAED,mEAAmE;AACnE,MAAM,YAAY,GAChB,iMAAiM,CAAC;AAEpM;;;;;;;;;;;;;GAaG;AACH,MAAM,CAAC,KAAK,UAAU,SAAS,CAAC,KAAsB,EAAE,KAAkB,EAAE,OAAiB;IAC3F,IAAI,OAAO,KAAK,KAAK,QAAQ;QAAE,OAAO,KAAK,CAAC;IAC5C,IAAI,SAAS,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;QAC1B,MAAM,KAAK,GAAG,KAAK,CAAC,CAAC,CAAC,MAAM,cAAc,CAAC,KAAK,EAAE,KAAK,CAAC,GAAG,EAAE,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;QAC3F,IAAI,CAAC,KAAK;YAAE,MAAM,IAAI,KAAK,CAAC,YAAY,CAAC,CAAC;QAC1C,OAAO,KAAK,CAAC;IACf,CAAC;IACD,MAAM,EAAE,aAAa,EAAE,sBAAsB,EAAE,GAAG,OAAO,CAAC;IAC1D,IAAI,OAAO,sBAAsB,KAAK,UAAU,EAAE,CAAC;QACjD,MAAM,IAAI,KAAK,CAAC,sHAAsH,CAAC,CAAC;IAC1I,CAAC;IACD,OAAO,aAAa,CAAC,KAAK,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;AACnD,CAAC;AAED,6EAA6E;AAC7E,MAAM,OAAO,kBAAkB;IAsCV;IACA;IAtCV,EAAE,GAAG,sBAAsB,CAAC;IAC5B,IAAI,GAAG,iBAAiB,CAAC;IACzB,WAAW,GAAG,uBAAuB,CAAC;IACtC,WAAW,GAClB,8GAA8G;QAC9G,+GAA+G,CAAC;IACzG,QAAQ,GAAG,OAAO,CAAC;IACnB,cAAc,GAAG,KAAK,CAAC;IAEvB,WAAW,GAAqB;QACvC,IAAI,EAAE,QAAQ;QACd,UAAU,EAAE;YACV,QAAQ,EAAE;gBACR,IAAI,EAAE,QAAQ;gBACd,WAAW,EAAE,kGAAkG;aAChH;YACD,IAAI,EAAE;gBACJ,IAAI,EAAE,QAAQ;gBACd,IAAI,EAAE,CAAC,KAAK,EAAE,aAAa,EAAE,QAAQ,EAAE,UAAU,EAAE,OAAO,EAAE,MAAM,CAAC;gBACnE,WAAW,EACT,uHAAuH;oBACvH,wEAAwE;aAC3E;YACD,OAAO,EAAE;gBACP,IAAI,EAAE,QAAQ;gBACd,IAAI,EAAE,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC;gBACf,WAAW,EAAE,6FAA6F;aAC3G;SACF;QACD,QAAQ,EAAE,CAAC,UAAU,CAAC;KACvB,CAAC;IAEF;;;OAGG;IACH,YACmB,WAAwB,EACxB,KAAkB;QADlB,gBAAW,GAAX,WAAW,CAAa;QACxB,UAAK,GAAL,KAAK,CAAa;IAClC,CAAC;IAEJ,KAAK,CAAC,OAAO,CAAC,IAAyB,EAAE,OAA8B;QACrE,MAAM,KAAK,GAAG,QAAQ,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;QACtC,IAAI,KAAK,KAAK,SAAS;YAAE,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,YAAY,EAAE,CAAC;QACxE,MAAM,IAAI,GAAe,IAAI,CAAC,IAAI,IAAI,MAAM,CAAC;QAC7C,IAAI,OAAO,IAAI,KAAK,QAAQ,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,WAAW,EAAE,IAAI,CAAC,EAAE,CAAC;YAClE,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,uBAAuB,MAAM,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC;QAClG,CAAC;QACD,0EAA0E;QAC1E,sEAAsE;QACtE,cAAc;QACd,IAAI,OAAO,GAAG,CAAC,CAAC;QAChB,IAAI,IAAI,CAAC,OAAO,KAAK,SAAS,EAAE,CAAC;YAC/B,MAAM,KAAK,GAAG,OAAO,IAAI,CAAC,OAAO,KAAK,QAAQ,IAAI,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC;YACnH,IAAI,KAAK,KAAK,CAAC,IAAI,KAAK,KAAK,CAAC,IAAI,KAAK,KAAK,CAAC;gBAAE,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,4BAA4B,EAAE,CAAC;YAC9G,OAAO,GAAG,KAAK,CAAC;QAClB,CAAC;QACD,IAAI,WAAW,CAAC,IAAI,CAAC,GAAG,OAAO,EAAE,CAAC;YAChC,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,GAAG,IAAI,kBAAkB,WAAW,CAAC,IAAI,CAAC,gCAAgC,OAAO,GAAG,EAAE,CAAC;QACzH,CAAC;QAED,IAAI,CAAC;YACH,MAAM,KAAK,GAAG,MAAM,SAAS,CAAC,KAAK,EAAE,IAAI,CAAC,KAAK,EAAE,OAAO,CAAC,CAAC;YAC1D,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,WAAW,CAAC,OAAO,KAAK,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC;YACtF,IAAI,IAAI,KAAK,OAAO,EAAE,CAAC;gBACrB,MAAM,SAAS,GAAG,MAAM,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;gBAC9C,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,UAAU,EAAE,SAAS,CAAC,MAAM,EAAE,EAAE,CAAC;YACtF,CAAC;YACD,qEAAqE;YACrE,oEAAoE;YACpE,uEAAuE;YACvE,sEAAsE;YACtE,uCAAuC;YACvC,MAAM,OAAO,GACX,OAAO,KAAK,CAAC,IAAI,IAAI,KAAK,KAAK;gBAC7B,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,KAAK,CAAC,EAAE;gBACpB,CAAC,CAAC,IAAI,KAAK,aAAa;oBACtB,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,aAAa,CAAC,EAAE,aAAa,EAAE,aAAa,EAAE;oBAC1D,CAAC,CAAC,IAAI,KAAK,QAAQ;wBACjB,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,aAAa,CAAC,EAAE,aAAa,EAAE,iBAAiB,EAAE;wBAC9D,CAAC,CAAC,IAAI,KAAK,UAAU;4BACnB,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,cAAc,CAAC,EAAE;4BAC7B,CAAC,CAAC,SAAS,CAAC;YACtB,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,OAAO,CAAC,KAAK,EAAE,OAAO,CAAC,CAAC;YACtD,OAAO;gBACL,OAAO,EAAE,IAAI;gBACb,MAAM,EAAE;oBACN,IAAI;oBACJ,IAAI,EAAE,MAAM,CAAC,IAAI;oBACjB,UAAU,EAAE,MAAM,CAAC,UAAU;oBAC7B,QAAQ,EAAE,MAAM,CAAC,QAAQ;oBACzB,KAAK,EAAE,MAAM,CAAC,KAAK;oBACnB,OAAO,EAAE,MAAM,CAAC,OAAO,EAAE,MAAM,IAAI,CAAC;oBACpC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBACnD,GAAG,CAAC,MAAM,CAAC,WAAW,EAAE,MAAM,CAAC,CAAC,CAAC,EAAE,WAAW,EAAE,MAAM,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC1E,UAAU,EAAE,MAAM,CAAC,UAAU;iBAC9B;aACF,CAAC;QACJ,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC;QAC3F,CAAC;IACH,CAAC;CACF"}

package/manifest.json

@@ -2,16 +2,16 @@
   "$schema": "https://agentos.sh/schemas/extension-manifest-v1.json",
   "id": "com.framers.tools.vision-pipeline",
   "name": "Vision & OCR Pipeline",
-  "version": "0.2.2",
+  "version": "0.3.0",
   "description": "Progressive vision pipeline with PaddleOCR, TrOCR, Florence-2, CLIP, and cloud vision (GPT-4o, Claude).",
   "author": {
     "name": "Frame.dev",
     "email": "support@frame.dev",
     "url": "https://github.com/framerslab"
   },
-  "license": "MIT",
+  "license": "Apache-2.0",
   "keywords": ["vision", "ocr", "paddle", "trocr", "florence", "clip", "document-ai"],
-  "agentosVersion": ">=0.10.40",
+  "agentosVersion": ">=0.13.40",
   "categories": ["tool", "media"],
   "platforms": [],
   "sdk": "none",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@framers/agentos-ext-vision-pipeline",
-  "version": "0.2.2",
+  "version": "0.3.0",
   "description": "Vision & OCR pipeline extension for AgentOS — PaddleOCR, TrOCR, Florence-2, CLIP, and cloud vision",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
@@ -35,7 +35,7 @@
       "url": "https://github.com/jddunn"
     }
   ],
-  "license": "MIT",
+  "license": "Apache-2.0",
   "repository": {
     "type": "git",
     "url": "git+https://github.com/framerslab/agentos-extensions.git",
@@ -45,10 +45,10 @@
     "access": "public"
   },
   "peerDependencies": {
-    "@framers/agentos": ">=0.10.40"
+    "@framers/agentos": ">=0.13.40"
   },
   "devDependencies": {
-    "@framers/agentos": "^0.13.19",
+    "@framers/agentos": "^0.13.40",
     "@types/node": "^20.12.12",
     "rimraf": "^5.0.7",
     "typescript": "^5.4.5",