@proveanything/smartlinks 2.0.26 → 2.0.29

package/docs/ai.md

@@ -324,15 +324,40 @@ in `input`.
 | `image.searchStock` | Search real stock photos (Unsplash). |
 | `image.transform` | Resize / crop / rotate / grayscale / format-convert / compress → hosted URL. |
 | `pdf.create` | Render HTML → PDF → hosted URL. |
-| `pdf.fill` | Fill an AcroForm PDF's fields → hosted URL. |
-| `pdf.merge` | Merge several PDFs into one → hosted URL. |
+| `pdf.fill` | Fill an AcroForm PDF's fields (`{ field: value }`) → hosted URL. |
+| `pdf.merge` | Merge several PDFs into one, in order → hosted URL. |
+| `pdf.inspect` | Cheap, no-AI introspection: page count/sizes, which pages have a real text layer, raster present, and a routing hint (`text` vs `vision`). |
+| `pdf.render` | Rasterize one page to a PNG at a chosen DPI → hosted image URL (feed to vision, or screenshot a page). |
+| `pdf.extract` | PDF → typed JSON in one call (schema and/or prompt). Auto-routes text vs vision. Optional per-field `confidence`/`source` (`includeConfidence`) and `bbox` (`includeBoxes`) in `fieldsMeta`. |
+| `pdf.decodeBarcodes` | Deterministically decode barcodes/QR on a page (WASM, no AI) → value + symbology + page + bbox + confidence. Use this for barcode digits, never vision. |
+| `pdf.inspectGraphics` | Prepress inspection: per-page path/image/outlined-text counts + colour spaces, plus named SPOT colours (e.g. "PANTONE 871 C"). No AI. |
 | `http.request` | SSRF-guarded outbound HTTP(S) to a public URL (call a REST API). |
 | `translate` | Translate text into one or more languages (generic, model-based). |
 
+**Working with PDFs.** The PDF tools split into *read/analyse* and *produce*:
+
+- **Read a whole PDF as text** → `document.read` (Firecrawl; good for prose/decks).
+- **Turn a PDF into structured fields** → `pdf.extract` (give it a JSON schema and/or a prompt; it
+  returns typed JSON). It routes itself, but you can drive the route yourself: call `pdf.inspect`
+  first (deterministic, no AI) to see whether each page has a real text layer, then `pdf.extract`
+  (cheap text path) or `pdf.render` → `image.describe`/vision for curve-only or raster artwork.
+- **Zoom in on small print** (INCI/allergen lists) → `pdf.render` at a high DPI (e.g. 300), then read
+  the PNG with vision.
+- **Barcodes / QR** → `pdf.decodeBarcodes` (deterministic WASM decode) — never trust vision for barcode
+  digits; it hallucinates them.
+- **Prepress / print QA** (spot colours, colour spaces, vector vs raster) → `pdf.inspectGraphics`.
+- **A review UI that flags guessed fields** → `pdf.extract` with `includeConfidence` (per-field
+  confidence + source) and `includeBoxes` (per-field bbox on text-native pages) in `fieldsMeta`.
+- **Produce a PDF** → `pdf.create` (HTML → PDF), `pdf.fill` (populate an AcroForm's fields),
+  `pdf.merge` (combine several). These return a hosted `hostedUrl`.
+
+Every tool is also directly callable without the model loop via `ai.tools.run(collectionId, name,
+args)` — e.g. render a page or extract fields straight from a UI, no agent round-trip.
+
 Discover tools two ways:
 - **Design time (typed):** import `BUILTIN_AI_TOOLS`, `AI_TOOL_NAMES`, and the per-tool arg types
-  (`WebSearchArgs`, `DataExtractArgs`, …) from the SDK. This is the core set — stable, versioned,
-  documented here.
+  (`WebSearchArgs`, `DataExtractArgs`, `PdfExtractArgs`, …) from the SDK. This is the core set —
+  stable, versioned, documented here.
 - **Runtime (live):** `await ai.catalog(collectionId)` returns the registry as the server sees it,
   including any future app-contributed tools. The built-in set above is always present.
 

package/docs/server-functions.md

@@ -271,6 +271,96 @@ name (a first-party builtin still wins). Always prefer an appId.
 
 ---
 
+## Exposing a function to the AI agent
+
+An `http` function can be offered to the AI agent as a **callable tool**, alongside the built-in
+tools. The model calls it, the server runs it, and the result is fed back into the loop. Opt in from
+the manifest with an `agent` block (full reference: [agent-tools.md](agent-tools.md)):
+
+```jsonc
+{
+  "name": "getLoyaltyBalance",
+  "trigger": { "type": "http" },
+  "visibility": "admin",
+  "authority": "caller",
+  "capabilities": ["sl:records:read"],
+  "agent": {
+    "tool": true,
+    "title": "Loyalty balance",
+    "description": "Look up a member's current loyalty points balance.",
+    "input": {
+      "type": "object",
+      "properties": { "memberId": { "type": "string" } },
+      "required": ["memberId"]
+    },
+    "approval": "auto"                       // 'require' = human confirms before each call
+  }
+}
+```
+
+The agent becomes **just another caller surface** — your function's `visibility`, `authority`, and
+`capabilities` are enforced exactly as on the http route. Nothing new is granted. The agent's tool
+arguments arrive as the function's request `body`, and whatever you return becomes the tool result the
+model sees. `approval: "require"` tools are held back from the autonomous server-side loop until the
+human-approval UX ships.
+
+Include your app's functions in an agent run:
+
+```ts
+// Agentic Responses:
+await SL.ai.chat.responses.create(collectionId, {
+  model: 'balanced', input: 'What is member 42's balance?',
+  server_tools: true,                              // built-in tools
+  app_functions: { appId: 'my-loyalty-app' },      // + this app's ai.tool functions
+})
+
+// Or the one-shot agent loop:
+await SL.ai.agent.run(collectionId, {
+  input: '…', appFunctions: { appId: 'my-loyalty-app' },
+})
+```
+
+On the **consumer surface**, a `visibility: "public"` function reaches a public assistant via the
+public agent loop — the caller runs as the signed-in consumer (`'owner'`, send the authKit bearer) or
+anonymous (`'public'`):
+
+```ts
+await SL.ai.publicClient.agentRun(collectionId, {
+  input: '…',
+  appFunctions: { appId: 'my-app' },   // this app's public agent tools
+  server_tools: ['web.search'],         // built-ins are an explicit allowlist on the public surface
+})
+```
+
+`channel` (default `'stable'`, pass `'dev'` to test a dev build) and `only: string[]` narrow which
+functions are exposed. Only `http` functions with `agent.tool: true` are eligible; `event`/`cron`
+functions never are. A built-in tool of the same name wins the clash.
+
+### One unified toolbelt
+
+Rather than juggling `server_tools` + `app_functions`, declare everything in one `toolbelt` — built-in
+tools, one **or several** apps' functions, and (reserved, staged) front-end client tools:
+
+```ts
+await SL.ai.chat.responses.create(collectionId, {
+  input: '…',
+  toolbelt: {
+    builtins: ['web.search', 'document.read'],      // true = all, [names] = subset, omit = none
+    appFunctions: [{ appId: 'loyalty' }, { appId: 'catalog' }],  // several apps at once
+    // clientTools: [ … ]  // reserved — the client-tool bridge is staged
+  },
+})
+```
+
+The same `toolbelt` works on `ai.agent.run` and `ai.publicClient.agentRun` (on the public surface
+`builtins` is an explicit allowlist — `true` is treated as none). Precedence on a name clash: a
+built-in wins, then earlier `appFunctions` sources win over later ones.
+
+> Directly (no model): every function is also callable deterministically — `SL.functions.call` /
+> `callAdmin` (above), the same way the built-in tools are callable via `SL.ai.tools.run`.
+
+---
+
 ## Runtime — what your function can use
 
 Your function runs in a **web-standard sandbox** (think Cloudflare Workers / Deno), **not
@@ -310,6 +400,12 @@ Prefer the platform primitives above over a dependency; when you do need one, pi
 | Schema validation of inputs | no | `zod` |
 | JWT (sign/verify for a third-party API) | Web Crypto can, verbosely | `jose` |
 | Hash / HMAC / encrypt | **native** — `crypto.subtle` | — |
+| PDF **text** extraction (cheap first-pass, no AI) | no | `unpdf` — a serverless/WASM pdf.js build with no Node deps; `import { extractText } from 'unpdf'`. Text only. |
+
+> **PDFs in a server function:** for a cheap text-layer check use `unpdf` (edge-safe, WASM). For
+> anything heavier — rendering pages, vision extraction, **barcode/QR decode**, or prepress/spot-colour
+> inspection — call the platform tools `pdf.render` / `pdf.extract` / `pdf.decodeBarcodes` /
+> `pdf.inspectGraphics` via `ai.tools.run`; those run in the full platform runtime, not the sandbox.
 
 **The common actions, and how to do each:**
 

package/openapi.yaml

@@ -16,6 +16,8 @@ tags:
     description: Chat completion generation.
   - name: agent
     description: Agent orchestration and tool-calling.
+  - name: tools
+    description: tools API
   - name: skills
     description: Reusable agent skills.
   - name: models
@@ -1239,6 +1241,45 @@ paths:
           description: Unauthorized
         404:
           description: Not found
+  /admin/collection/{collectionId}/ai/tools/{name}/run:
+    post:
+      tags:
+        - tools
+      summary: tools.run
+      operationId: tools_run
+      security:
+        - bearerAuth: []
+      parameters:
+        - name: collectionId
+          in: path
+          required: true
+          schema:
+            type: string
+        - name: name
+          in: path
+          required: true
+          schema:
+            type: string
+      responses:
+        200:
+          description: Success
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/ToolRunResult"
+        400:
+          description: Bad request
+        401:
+          description: Unauthorized
+        404:
+          description: Not found
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              additionalProperties: true
   /admin/collection/{collectionId}/ai/tts:
     post:
       tags:
@@ -10390,6 +10431,38 @@ paths:
           description: Unauthorized
         404:
           description: Not found
+  /public/collection/{collectionId}/ai/agent/run:
+    post:
+      tags:
+        - publicClient
+      summary: Public agent loop — run the orchestration-neutral tool loop on the consumer surface.
+      operationId: publicClient_agentRun
+      security: []
+      parameters:
+        - name: collectionId
+          in: path
+          required: true
+          schema:
+            type: string
+      responses:
+        200:
+          description: Success
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/AgentRunResult"
+        400:
+          description: Bad request
+        401:
+          description: Unauthorized
+        404:
+          description: Not found
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: "#/components/schemas/PublicAgentRunRequest"
   /public/collection/{collectionId}/ai/chat:
     post:
       tags:
@@ -15499,6 +15572,10 @@ components:
             $ref: "#/components/schemas/AiToolName"
         maxSteps:
           type: number
+        app_functions:
+          $ref: "#/components/schemas/AgentAppFunctionsOption"
+        toolbelt:
+          $ref: "#/components/schemas/AgentToolbelt"
       required:
         - enabled
     ResponsesResult:
@@ -16530,6 +16607,162 @@ components:
           type: array
           items:
             type: string
+        appFunctions:
+          $ref: "#/components/schemas/AgentAppFunctionsOption"
+        toolbelt:
+          $ref: "#/components/schemas/AgentToolbelt"
+    AgentAppFunctionsOption:
+      type: object
+      properties:
+        appId:
+          type: string
+        channel:
+          type: string
+        only:
+          type: array
+          items:
+            type: string
+      required:
+        - appId
+    AgentToolbelt:
+      type: object
+      properties:
+        builtins:
+          type: array
+          items:
+            type: object
+            additionalProperties: true
+        appFunctions:
+          type: array
+          items:
+            $ref: "#/components/schemas/AgentAppFunctionsOption"
+        clientTools:
+          type: array
+          items:
+            $ref: "#/components/schemas/ClientToolDeclaration"
+    ClientToolDeclaration:
+      type: object
+      properties:
+        name:
+          type: string
+        description:
+          type: string
+        input:
+          type: object
+          additionalProperties: true
+      required:
+        - name
+    ClientToolCall:
+      type: object
+      properties:
+        callId:
+          type: string
+        name:
+          type: string
+        args:
+          type: object
+          additionalProperties: true
+      required:
+        - callId
+        - name
+        - args
+    RequiresActionResult:
+      type: object
+      properties:
+        status:
+          type: string
+          enum:
+            - requires_action
+        client_tool_calls:
+          type: array
+          items:
+            $ref: "#/components/schemas/ClientToolCall"
+        items:
+          type: array
+          items: {}
+        steps:
+          type: number
+        toolResults:
+          type: array
+          items:
+            $ref: "#/components/schemas/AgentToolResult"
+        availableTools:
+          type: array
+          items:
+            type: string
+      required:
+        - status
+        - client_tool_calls
+        - items
+    ClientTool:
+      type: object
+      properties:
+        declaration:
+          $ref: "#/components/schemas/ClientToolDeclaration"
+      required:
+        - declaration
+    RunWithClientToolsOptions:
+      type: object
+      properties:
+        input:
+          type: string
+        tools:
+          type: array
+          items:
+            $ref: "#/components/schemas/ClientTool"
+        surface:
+          type: string
+          enum:
+            - admin
+            - public
+        instructions:
+          type: string
+        model:
+          type: string
+        maxSteps:
+          type: number
+        toolbelt:
+          $ref: "#/components/schemas/AgentToolbelt"
+        maxRounds:
+          type: number
+      required:
+        - input
+        - tools
+    PublicAgentRunRequest:
+      type: object
+      properties:
+        input:
+          type: string
+        prompt:
+          type: string
+        instructions:
+          type: string
+        model:
+          type: string
+        maxSteps:
+          type: number
+        server_tools:
+          type: array
+          items:
+            $ref: "#/components/schemas/AiToolName"
+        only:
+          type: array
+          items:
+            $ref: "#/components/schemas/AiToolName"
+        exclude:
+          type: array
+          items:
+            $ref: "#/components/schemas/AiToolName"
+        allowCapabilities:
+          type: array
+          items:
+            $ref: "#/components/schemas/AiToolCapability"
+        appFunctions:
+          $ref: "#/components/schemas/AgentAppFunctionsOption"
+        toolbelt:
+          $ref: "#/components/schemas/AgentToolbelt"
+        userId:
+          type: string
     AgentToolResult:
       type: object
       properties:
@@ -16542,6 +16775,20 @@ components:
         - name
         - isError
         - result
+    ToolRunResult:
+      type: object
+      properties:
+        name:
+          type: string
+        isError:
+          type: boolean
+        result:
+          type: object
+          additionalProperties: true
+      required:
+        - name
+        - isError
+        - result
     AgentRunResult:
       type: object
       properties:
@@ -16793,6 +17040,46 @@ components:
             type: string
       required:
         - urls
+    PdfInspectArgs:
+      type: object
+      properties:
+        url:
+          type: object
+          additionalProperties: true
+      required:
+        - url
+    PdfRenderArgs:
+      type: object
+      properties:
+        url:
+          type: object
+          additionalProperties: true
+      required:
+        - url
+    PdfExtractArgs:
+      type: object
+      properties:
+        url:
+          type: object
+          additionalProperties: true
+      required:
+        - url
+    PdfDecodeBarcodesArgs:
+      type: object
+      properties:
+        url:
+          type: object
+          additionalProperties: true
+      required:
+        - url
+    PdfInspectGraphicsArgs:
+      type: object
+      properties:
+        url:
+          type: object
+          additionalProperties: true
+      required:
+        - url
     HttpRequestArgs:
       type: object
       properties:
@@ -16848,6 +17135,16 @@ components:
           $ref: "#/components/schemas/PdfFillArgs"
         pdf.merge:
           $ref: "#/components/schemas/PdfMergeArgs"
+        pdf.inspect:
+          $ref: "#/components/schemas/PdfInspectArgs"
+        pdf.render:
+          $ref: "#/components/schemas/PdfRenderArgs"
+        pdf.extract:
+          $ref: "#/components/schemas/PdfExtractArgs"
+        pdf.decodeBarcodes:
+          $ref: "#/components/schemas/PdfDecodeBarcodesArgs"
+        pdf.inspectGraphics:
+          $ref: "#/components/schemas/PdfInspectGraphicsArgs"
         http.request:
           $ref: "#/components/schemas/HttpRequestArgs"
         translate:
@@ -16868,6 +17165,11 @@ components:
         - pdf.create
         - pdf.fill
         - pdf.merge
+        - pdf.inspect
+        - pdf.render
+        - pdf.extract
+        - pdf.decodeBarcodes
+        - pdf.inspectGraphics
         - http.request
         - translate
     WebSearchResultItem:
@@ -16944,6 +17246,90 @@ components:
           additionalProperties: true
       required:
         - translations
+    PdfInspectPage:
+      type: object
+      properties:
+        page:
+          type: string
+          enum:
+            - text-native
+            - text+raster
+            - raster-only
+            - curve-only
+      required:
+        - page
+    PdfInspectResult:
+      type: object
+      properties:
+        url:
+          type: array
+          items:
+            type: string
+            enum:
+              - text
+              - vision
+      required:
+        - url
+    PdfRenderResult:
+      type: object
+      properties:
+        url:
+          type: object
+          additionalProperties: true
+      required:
+        - url
+    PdfFieldMeta:
+      type: object
+      properties:
+        confidence:
+          type: object
+          additionalProperties: true
+    PdfExtractResult:
+      type: object
+      properties:
+        url:
+          type: string
+          enum:
+            - text
+            - vision
+      required:
+        - url
+    PdfBarcode:
+      type: object
+      properties:
+        type:
+          type: object
+          additionalProperties: true
+      required:
+        - type
+    PdfDecodeBarcodesResult:
+      type: object
+      properties:
+        url:
+          type: array
+          items:
+            type: object
+            additionalProperties: true
+      required:
+        - url
+    PdfGraphicsPage:
+      type: object
+      properties:
+        page:
+          type: array
+          items:
+            type: object
+            additionalProperties: true
+      required:
+        - page
+    PdfInspectGraphicsResult:
+      type: object
+      properties:
+        url:
+          type: object
+          additionalProperties: true
+      required:
+        - url
     ResponsesAgentTrace:
       type: object
       properties:
@@ -17022,6 +17408,11 @@ components:
         - pdf.create
         - pdf.fill
         - pdf.merge
+        - pdf.inspect
+        - pdf.render
+        - pdf.extract
+        - pdf.decodeBarcodes
+        - pdf.inspectGraphics
         - http.request
         - translate
     AgentStreamEvent:
@@ -18264,9 +18655,30 @@ components:
           type: string
         handler:
           type: string
+        agent:
+          $ref: "#/components/schemas/AppFunctionAgentExposure"
       required:
         - name
         - trigger
+    AppFunctionAgentExposure:
+      type: object
+      properties:
+        tool:
+          type: boolean
+        title:
+          type: string
+        description:
+          type: string
+        input:
+          type: object
+          additionalProperties: true
+        approval:
+          type: string
+          enum:
+            - auto
+            - require
+      required:
+        - tool
     AppManifestFunctions:
       type: object
       properties:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@proveanything/smartlinks",
-  "version": "2.0.26",
+  "version": "2.0.29",
   "description": "Official JavaScript/TypeScript SDK for the Smartlinks API",
   "type": "module",
   "main": "dist/index.js",