@formio/uag 1.11.2 → 1.12.0

package/README.md

@@ -6,6 +6,13 @@ The Universal Agent Gateway (UAG) leverages the [Model Context Protocol (MCP)](h
 
 ![Diagram showing how the UAG provides an AI Agent the ability to read and understand a Form.io Form JSON. Similar to how the JavaScript renderer provides a Human the ability to visually see a rendered form.](./examples/images/uag-agent-rendering.png)
 
+Official Documentation
+--------------------------
+For the latest documentation, release information, and guides, always refer to the official Form.io Help Documentation available here:
+
+**[https://help.form.io](https://help.form.io/dev/ai/uag)**
+
+--------------------------
 There are two primary scenarios that the UAG enables:
 
  - **Unstructured content conversion to deterministic structured data**:  This capability uses the JSON schema of a Form.io form to produce a deterministic structured data object provided a Natural Language input. 
@@ -80,6 +87,8 @@ In order to a aid in the "agentification" of the UAG, we have also provided some
 
 ![Agentic process flow via single API call into Claude Integration](./examples/images/agent-flow.png)
 
+Each integration runs its own MCP client and connects *out* to the UAG, so the UAG does not need to be publicly accessible for these to work — it only needs to be reachable from the integration, such as over a private Docker network or on `localhost`.
+
 These integrations can be found within the **integrations** folder. The following **integrations** are provided to enable automated Agentic workflows.
 
   - [Claude Integration](./integrations/claude)
@@ -165,6 +174,40 @@ The following sections describe the process of creating a running instance of UA
 
 This documentation focuses on using UAG with Form.io Open Source. For documentation on using UAG with Form.io Enterprise, refer to [https://help.form.io/uag](https://help.form.io/uag).
 
+### How Agents Connect
+Before deploying, it is worth deciding **which side opens the connection** to the UAG's `/mcp` endpoint, because that single decision determines whether the UAG needs to be publicly accessible.
+
+| Connection model | Who runs the MCP client | Does the UAG need a public domain? |
+|------------------|-------------------------|------------------------------------|
+| **Local MCP client** | Your own application or integration, running alongside the UAG | **No.** The UAG only needs to be reachable from that process — `http://localhost:3200` or a private Docker network address is fine. |
+| **Remote MCP client** | A hosted agent that dials into your UAG (Claude.ai and Claude Desktop connectors, the Anthropic MCP connector, or any other remote MCP host) | **Yes.** The agent must be able to reach `/mcp` and the OAuth discovery endpoints over the public internet. |
+
+With a **local MCP client**, your application connects *out* to the UAG, calls the tools itself, and passes the results on to whichever model it is using. Nothing dials back in, so the UAG can stay entirely inside your network. This is how the [Claude Integration](./integrations/claude) works, and it is the recommended model for server-to-server and webhook-triggered automation — including running the whole stack on a developer machine.
+
+With a **remote MCP client**, the agent lives outside your network and connects in, so `/mcp` and the `.well-known` OAuth discovery endpoints must be publicly reachable over HTTPS, and `BASE_URL` must be that public URL. See [Domain and URL Configuration](#domain-and-url-configuration).
+
+Both models can be used against the same UAG deployment at the same time.
+
+#### Authenticating a local MCP client
+A remote agent authenticates through the browser-based OAuth (PKCE) flow. A local client has no browser, so it exchanges a server-side key for an access token using the `client_credentials` grant against `{{ BASE_URL }}/auth/token`, then sends that token as a `Bearer` token on its `/mcp` requests.
+
+The `client_id` must be prefixed with the project name — which is `formio-oss` for Open Source deployments:
+
+```
+POST /auth/token
+Content-Type: application/json
+
+{
+  "grant_type": "client_credentials",
+  "client_id": "formio-oss-x-admin-key",
+  "client_secret": "{{ ADMIN_KEY }}"
+}
+```
+
+For Enterprise, use the project name from your project endpoint together with the project API key — for example `"client_id": "myproject-x-token"` with `"client_secret": "{{ PROJECT_KEY }}"`.
+
+The response contains an `access_token`, which is passed to the UAG as `Authorization: Bearer {{ access_token }}`. Tokens expire according to `JWT_EXPIRE_TIME`, so long-running clients should cache the token and refresh it before it expires. See [`integrations/claude/auth.js`](./integrations/claude/auth.js) for a working implementation.
+
 ### Runtime Environments
 There are currently two run-time environments that work with the UAG:
  -[**Docker**](#docker)
@@ -300,8 +343,8 @@ This module can be configured in many ways. One of those ways is through the use
 | JWT_EXPIRE_TIME | The expiration for the jwt secret. | 3600 |
 | MONGO | (Enterprise Only) Allows you to connect the UAG directly to a mongo database, rather than having to redirect the submissions to the Form.io Submission APIs. | |
 | MONGO_CONFIG | JSON configuration for the Node.js Mongo Driver. | |
-| BASE_URL | The public URL that the UAG is hosted on. This allows for proper OIDC authentication and allows for the authentication callbacks to point to the correct url. | https://forms.yoursite.com |
-| LOGIN_FORM | The public URL to the Login Form JSON endpoint. | https://mysite.com/project/user/login |
+| BASE_URL | **Required.** The URL that the UAG is reachable at *by its clients*. It is published in the `.well-known` OIDC (PKCE) definitions and used for authentication callbacks. This must be a public URL when remote agents connect in; for a local MCP client, a private or localhost URL is fine. | https://forms.yoursite.com |
+| LOGIN_FORM | The URL to the Login Form JSON endpoint, loaded by the `/auth/authorize` page. Only used by the browser-based login flow, so it must be reachable by that browser. | https://mysite.com/project/user/login |
 | CORS | The cors domain, or the JSON configuration to configure the "cors" node.js module cross domain resource sharing. | *.* |
 
 ### Project Cache and TTL
@@ -315,8 +358,10 @@ Here is a table explaining how this parameter can be used.
 | PROJECT_TTL | 60 | Check for changes every minute |
 | PROJECT_TTL | 3600 | Check for changes every hour |
 
-### Running on Public Domain
-In order to run the UAG on a public domain, it is very important to provide the proper configurations so that any AI agent can properly authenticate. There are 3 different "domain" environment variables that matter, and it is important to understand how to configure them depending on your use case:
+### Domain and URL Configuration
+There are 3 different "domain" environment variables that matter, and it is important to understand how to configure them depending on your use case. Whether any of them need to be *public* depends on how agents connect — see [How Agents Connect](#how-agents-connect).
+
+If a remote agent connects into your UAG, these must be configured correctly and publicly, or the agent will not be able to authenticate. If you only use a local MCP client, all three can point at private or localhost addresses.
 
 #### PROJECT
 The ```PROJECT``` environment variable is used to establish a connection from the UAG server to the project endpoint (for Enterprise) or OSS base url. This does NOT need to be a public DNS entry, but rather a URL that connects the UAG container to the Server container.  The following examples illustrate how this would be configured.
@@ -400,14 +445,21 @@ services:
 ```
 
 #### BASE_URL
-The ```BASE_URL``` is used to communicate to the AI agent the public domain that is hosting the UAG server. This value is provided within the ```.well-known``` definitions for the OIDC (PKCE) authentication. If this is not correct, then the AI agent will not be able to authenticate into the UAG.
+The ```BASE_URL``` tells the UAG what URL it is reachable at. This value is published within the ```.well-known``` definitions for the OIDC (PKCE) authentication, and is used for the authentication callbacks.
+
+**```BASE_URL``` is always required**, including for local clients. The UAG's authentication provider is only enabled when it is set, so if you omit it, ```/mcp``` returns a 401 for every request and the ```/auth/token``` and ```/.well-known/*``` endpoints do not exist at all (404). A 404 on ```/auth/token``` is the clearest signal that ```BASE_URL``` is missing.
 
-**This needs to be the publicly accessible domain that you are hosting your UAG.**
+What it needs to be set to depends on how agents connect:
 
-For example, ```BASE_URL: https://forms.mysite.com```.
+ - **Remote agents connect in**: this must be the publicly accessible domain hosting your UAG, because that is the address the agent is told to authenticate against. For example, ```BASE_URL: https://forms.mysite.com```.
+ - **Local MCP client only**: set it to the address your client uses to reach the UAG. A private or localhost URL is fine — for example ```BASE_URL: http://localhost:3200``` or ```BASE_URL: http://formio-uag:3200``` within a Docker network.
+
+If a remote agent cannot authenticate, a mismatched ```BASE_URL``` is the most common cause: the agent reads this value from the discovery document and will follow it, so if it points somewhere the agent cannot reach, authentication fails even though the UAG itself is running correctly.
 
 #### LOGIN_FORM
-This is the publicly accessible URL to the Login form of your Project or OSS deployment. This provides the URL that is loaded when the user navigates to ```{{ BASE_URL }}/auth/authorize```.  If you navigate to this URL, and the page says that you cannot load the form, then this is because your LOGIN_FORM environment variable is not pointing to the correct form JSON endpoint of your project. 
+This is the URL to the Login form of your Project or OSS deployment. It provides the form that is loaded when a user navigates to ```{{ BASE_URL }}/auth/authorize```.  If you navigate to this URL, and the page says that you cannot load the form, then this is because your LOGIN_FORM environment variable is not pointing to the correct form JSON endpoint of your project. 
+
+This is only used by the browser-based login flow, so it needs to be reachable by the browser performing that login. A local MCP client authenticating with the `client_credentials` grant never loads this page.
 
 For example:
  - Enterprise Example: ```LOGIN_FORM: https://forms.mysite.com/myproject/user/login```

package/lib/UAGFormInterface.d.ts

@@ -89,7 +89,7 @@ export declare class UAGFormInterface extends FormInterface {
     }[] | {
         data: {};
     } | {
-        data?: undefined;
+        data?: {};
     } | null;
     /**
      * Override the process methods for the UAG.

package/lib/UAGProjectInterface.d.ts

@@ -13,22 +13,23 @@ export declare class UAGProjectInterface extends ProjectInterface {
     get config(): UAGConfig;
     constructor(endpoint?: string);
     initialize(): Promise<void>;
+    buildMcpServer(): Promise<McpServer>;
     addForm(form: Form, machineName: string): UAGFormInterface | undefined;
     uagRouter(): Router;
     authorizeRequest(req: SubmissionRequest, res: Response, next: NextFunction): Promise<Response<any, Record<string, any>> | undefined>;
     mcpJSONResponse(data?: object, isError?: boolean): {
-        isError?: boolean | undefined;
         content: {
-            type: "text";
+            type: 'text';
             text: string;
         }[];
         structuredContent: object;
+        isError?: boolean | undefined;
     };
     mcpResponse(templateName: ResponseTemplate, data?: object, isError?: boolean): {
-        isError?: boolean | undefined;
         content: {
-            type: "text";
+            type: 'text';
             text: string;
         }[];
+        isError?: boolean | undefined;
     };
 }

package/lib/UAGProjectInterface.js

@@ -20,6 +20,12 @@ class UAGProjectInterface extends appserver_1.ProjectInterface {
     async initialize() {
         await super.initialize();
         this.uagTemplate = new template_1.UAGTemplate(this.config?.responseTemplates || {});
+        this.mcpServer = await this.buildMcpServer();
+    }
+    // Build a fresh McpServer with all tools registered. A new instance is created
+    // per request so concurrent requests never share (and overwrite) a transport.
+    async buildMcpServer() {
+        const server = new mcp_js_1.McpServer({ name: 'formio-uag', version: '1.0.0' });
         // Get the standard UAG tools.
         const tools = await (0, tools_1.getTools)(this);
         // Add the custom tools from config.
@@ -29,13 +35,14 @@ class UAGProjectInterface extends appserver_1.ProjectInterface {
         // Iterate and register all the tools.
         for (const tool of tools) {
             if (tool?.name) {
-                this.mcpServer.registerTool(tool.name, {
+                server.registerTool(tool.name, {
                     title: tool.title,
                     description: tool.description,
                     inputSchema: tool.inputSchema
                 }, tool.execute);
             }
         }
+        return server;
     }
     addForm(form, machineName) {
         const formInterface = super.addForm(form, machineName);

package/lib/index.d.ts

@@ -7,7 +7,7 @@ export type UAGModule = ServerModule & {
 };
 export declare class UAGServer extends Server {
     constructor(config?: UAGConfig);
-    router(): Promise<import("express-serve-static-core").Router>;
+    router(): Promise<any>;
 }
 export * from '@formio/appserver';
 export * from './config';

package/lib/router.js

@@ -3,7 +3,6 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.UAGRouter = UAGRouter;
 const express_1 = require("express");
 const streamableHttp_js_1 = require("@modelcontextprotocol/sdk/server/streamableHttp.js");
-const types_js_1 = require("@modelcontextprotocol/sdk/types.js");
 // Helper function to create JSON RPC error responses
 const createJsonRpcErrorResponse = (code, message) => {
     return JSON.stringify({
@@ -19,6 +18,8 @@ const createJsonRpcErrorResponse = (code, message) => {
  * @returns True if the error was handled, false otherwise.
  */
 const handleResponseError = (error, res) => {
+    // A thrown fetch Response (the global Response, not Express's) carries the
+    // upstream status and headers that should be forwarded to the client.
     if (error instanceof Response) {
         const fixedHeaders = {};
         error.headers.forEach((value, key) => {
@@ -31,22 +32,29 @@ const handleResponseError = (error, res) => {
     res.status(500).json(JSON.parse(createJsonRpcErrorResponse(-32603, "Internal Server Error")));
     return false;
 };
+/**
+ * Build a per-request McpServer + transport (stateless mode) and tear both down
+ * when the response closes. A fresh instance per request keeps concurrent requests
+ * from overwriting each other's transport.
+ */
+const connectMcpTransport = async (project, res) => {
+    const server = await project.buildMcpServer();
+    const transport = new streamableHttp_js_1.StreamableHTTPServerTransport({
+        sessionIdGenerator: undefined, // Use "undefined" to trigger stateless mode.
+    });
+    res.on('close', () => {
+        transport.close();
+        server.close();
+    });
+    await server.connect(transport);
+    return transport;
+};
 function UAGRouter(project) {
     const router = (0, express_1.Router)();
     // Handles the MCP post requests.
     router.post('/', async (req, res) => {
         try {
-            const transport = new streamableHttp_js_1.StreamableHTTPServerTransport({
-                sessionIdGenerator: undefined, // Use "undefined" to trigger stateless mode.
-            });
-            // Close any existing connection before connecting to a new transport (required in SDK 1.26.0+)
-            await project.mcpServer.close();
-            await project.mcpServer.connect(transport);
-            if (!transport) {
-                res.status(500).json(JSON.parse(createJsonRpcErrorResponse(-32603, "Unable to create MCP transport.")));
-                return;
-            }
-            (0, types_js_1.isInitializeRequest)(req.body);
+            const transport = await connectMcpTransport(project, res);
             await transport.handleRequest(req, res, req.body);
         }
         catch (error) {
@@ -56,16 +64,7 @@ function UAGRouter(project) {
     // Handles the MCP get requests (for health checks, etc).
     router.get('/', async (req, res) => {
         try {
-            const transport = new streamableHttp_js_1.StreamableHTTPServerTransport({
-                sessionIdGenerator: undefined, // Use "undefined" to trigger stateless mode.
-            });
-            // Close any existing connection before connecting to a new transport (required in SDK 1.26.0+)
-            await project.mcpServer.close();
-            await project.mcpServer.connect(transport);
-            if (!transport) {
-                res.status(500).json(JSON.parse(createJsonRpcErrorResponse(-32603, "Unable to create MCP transport.")));
-                return;
-            }
+            const transport = await connectMcpTransport(project, res);
             await transport.handleRequest(req, res);
         }
         catch (err) {

package/lib/templates/agentProcessData.md

@@ -1,4 +1,4 @@
-Please read and understand the following **--- CRITERIA ---** section. This section provides instructions on how to analyze the **--- SUBMISSION ---** section that follows. Your objective is to provide values defined within the **--- REQUIRED FIELDS ---** section, and then to submit those values using the `agent_submit_data` tool.
+Please read and understand the following **--- CRITERIA ---** section. This section provides instructions on how to analyze the **--- SUBMISSION ---** section that follows. Your objective is to provide values defined within the **--- REQUIRED FIELDS ---** section, and then to submit those values using the `submission_update` tool.
 
 --- CRITERIA ---
 <%= criteria %>

package/lib/templates/submitValidationError.md

@@ -1,6 +1,6 @@
 There are some validation errors in the data collected:
 <% validationErrors.forEach(function(error) { %>
- - <% if (error.label && error.path) { %>**<%= error.label %> (<%= error.path %>)**: <% } %><%= error.message %><% }); %>
+ - <% if (error.label && error.path) { %>**<%= error.label %> (<%= error.path %>)**: <% } %><%= error.error || error.message %><% }); %>
 
 These validation errors are in one of the following format:
  - **Field Label (data_path)** - Error message:  If the error is attributed to a specific field value.

package/lib/tools/submitForm.js

@@ -21,13 +21,28 @@ const submitCompletedForm = async (project) => {
             try {
                 let submission = form.convertToSubmission(form_data);
                 const submitted = await form.submit(submission, extra.authInfo);
-                if (submitted)
+                if (submitted) {
+                    // The submission save proxies to the Form.io server, and a rejection
+                    // (e.g. failed validation) can come back through the save pipeline as
+                    // the error response body instead of a saved submission. A result
+                    // without an _id was not persisted — report the failure to the agent
+                    // instead of a false success.
+                    const result = submitted;
+                    if (!result._id) {
+                        const errors = (result.name === 'ValidationError' && result.details)
+                            ? form.convertToFormFieldErrors(result.details)
+                            : [];
+                        return project.mcpResponse(template_1.ResponseTemplate.submitValidationError, {
+                            validationErrors: errors.length ? errors : [{
+                                    message: result.message || 'The submission was not saved by the server. Verify that all required fields have been collected and that every value conforms to its field rules, then try again.'
+                                }]
+                        }, true);
+                    }
                     submission = submitted;
-                if (!submission) {
-                    return project.mcpResponse(template_1.ResponseTemplate.submitValidationError, {
-                        validationErrors: ['Unknown error during submission']
-                    }, true);
                 }
+                // A null result is still a valid outcome for "action" forms that have no
+                // Save Submission action (e.g. email or webhook only) — the data was
+                // processed but intentionally not persisted, so no submission ID exists.
                 debug(`Form submitted: ${form_name} with submission ID: ${submission._id}`);
                 return project.mcpResponse(template_1.ResponseTemplate.formSubmitted, {
                     form: form.form,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@formio/uag",
-  "version": "1.11.2",
+  "version": "1.12.0",
   "mcpName": "io.form/uag",
   "description": "The Form.io Universal Agent Gateway (UAG).",
   "main": "lib/index.js",
@@ -13,11 +13,12 @@
   ],
   "scripts": {
     "clean": "rm -rf lib",
-    "build:ts": "npx tsc",
+    "build:ts": "tsc",
     "build:copy-templates": "mkdir -p lib/templates && cp src/templates/*.md lib/templates/",
     "build": "npm run clean && npm run build:ts && npm run build:copy-templates",
     "build:docker": "docker build -t formio/uag$(node -p \"require('./package.json').version.includes('-rc.') ? ':rc' : ''\") --platform=linux/amd64 .",
-    "test": "TS_NODE_PROJECT=tsconfig.test.json mocha --no-node-snapshot",
+    "test": "tsc --noEmit -p tsconfig.test.json && mocha --no-node-snapshot",
+    "test:e2e": "mocha --config .mocharc.e2e.json",
     "dev": "tsx --no-node-snapshot index.js",
     "start": "node --no-node-snapshot index.js",
     "claude": "node --no-node-snapshot ./integrations/claude/index.js"
@@ -29,18 +30,18 @@
   "author": "Form.io Developers",
   "license": "MIT",
   "dependencies": {
-    "@formio/appserver": "^2.12.2",
-    "@formio/core": "2.5.1-dev.291.6557e4e",
-    "@modelcontextprotocol/sdk": "1.27.1",
+    "@formio/appserver": "^2.13.0",
+    "@formio/core": "2.8.0",
+    "@modelcontextprotocol/sdk": "1.30.0",
     "cors": "^2.8.6",
     "debug": "^4.4.3",
-    "dotenv": "^17.3.1",
+    "dotenv": "^17.4.2",
     "express": "^5.2.1",
     "jsonwebtoken": "^9.0.3",
-    "lodash": "^4.17.23",
+    "lodash": "^4.18.1",
     "node-cache": "^5.1.2",
     "node-html-markdown": "^2.0.0",
-    "zod": "^4.3.6"
+    "zod": "^4.4.3"
   },
   "resolutions": {
     "tar": ">=7.5.12",
@@ -51,22 +52,23 @@
     "dompurify": ">=3.3.3"
   },
   "devDependencies": {
+    "@anthropic-ai/sdk": "^0.115.0",
     "@types/chai": "^5.2.3",
     "@types/cors": "^2.8.19",
-    "@types/debug": "^4.1.12",
+    "@types/debug": "^4.1.13",
     "@types/dotenv": "^8.2.3",
     "@types/express": "^5.0.6",
     "@types/jsonwebtoken": "^9.0.10",
     "@types/lodash": "^4.17.24",
     "@types/mocha": "^10.0.10",
-    "@types/node": "^25.5.0",
-    "@types/supertest": "^7.2.0",
+    "@types/node": "^26.1.2",
+    "@types/supertest": "^7.2.1",
     "chai": "^6.2.2",
-    "mocha": "^11.7.5",
+    "mocha": "^11.7.6",
     "supertest": "^7.2.2",
-    "terser": "^5.46.1",
-    "ts-node": "^10.9.2",
-    "tsx": "^4.21.0",
-    "typescript": "^5.9.3"
-  }
+    "terser": "^5.49.0",
+    "tsx": "^4.23.1",
+    "typescript": "^7.0.2"
+  },
+  "packageManager": "yarn@4.17.1"
 }