@formio/uag 1.11.2 → 1.12.1

package/README.md

@@ -6,9 +6,16 @@ The Universal Agent Gateway (UAG) leverages the [Model Context Protocol (MCP)](h
 
 ![Diagram showing how the UAG provides an AI Agent the ability to read and understand a Form.io Form JSON. Similar to how the JavaScript renderer provides a Human the ability to visually see a rendered form.](./examples/images/uag-agent-rendering.png)
 
+Official Documentation
+--------------------------
+For the latest documentation, release information, and guides, always refer to the official Form.io Help Documentation available here:
+
+**[https://help.form.io](https://help.form.io/dev/ai/uag)**
+
+--------------------------
 There are two primary scenarios that the UAG enables:
 
- - **Unstructured content conversion to deterministic structured data**:  This capability uses the JSON schema of a Form.io form to produce a deterministic structured data object provided a Natural Language input. 
+ - **Unstructured 'chat' input to deterministic form submission**:  This capability uses the JSON schema of a Form.io form to produce a deterministic structured data object provided natural language conversational input. 
 
  ![Conversion of natural language to structured data using UAG.](./examples/images/uag-natural-language.png)
 
@@ -20,19 +27,43 @@ There are two primary scenarios that the UAG enables:
 
 ---
 
-One of the many benefits that the UAG has to offer is the ability to inject **Dynamic Context** into an AI processing cycle. This is achieved from the dynamic nature of the Form JSON schemas that can easily be modified and deployed to any enviornment without the need to update your application or re-train the agents within the process. Because of this, dynamic context changes can be injected seamlessly into an existing running process which will automatically update the behaviors of the Agents utilzing the UAG.
+One of the many benefits that the UAG has to offer is the ability to inject **Dynamic Context** into an AI processing cycle. This is achieved from the dynamic nature of the Form JSON schemas that can easily be modified and deployed to any environment without the need to update your application or re-train the agents within the process. Because of this, dynamic context changes can be injected seamlessly into an existing running process which will automatically update the behaviors of the Agents utilizing the UAG.
 
 ## Try it out!
-The quickest way to try out the UAG run the [Local Example](examples/local) which will spin up a local instance of Form.io Open Source and UAG. This can then be connected to an AI agent, which will have the ability to interact with the included example forms. The local example can be run entirely free, with no trial or license required.
+There are three working examples in this repo, each spinning up a Form.io server alongside the UAG with Docker Compose. Pick the one that matches what you are building.
+
+| Example | Pattern | Licensing |
+|---------|---------|-----------|
+| **[Custom Module](examples/custom-module)** | **Conversational, existing client.** Connect Claude Desktop to your forms and talk to them: "I would like to add a new customer." Built as a custom module, so roles and permissions decide what the agent may do. | Open Source — free |
+| **[Conversation Form](examples/conversation-form)** | **Conversational, your own UI.** A chat application whose questions come entirely from a form. About 200 lines, no field names in the code — edit the form and the conversation changes. | Open Source — free |
+| **[Agentic Workflow](examples/agentic-workflow)** | **Autonomous.** A submission triggers an agent that scores it against a rubric and writes its decision back, which in turn triggers a second agent. No human, no chat window. | Enterprise — the trigger needs a Webhook action that can set headers and transform its payload |
+
+There is also a [Flow Viewer](examples/flow-viewer) — a small proxy that shows every leg of an agentic run live, which is the fastest way to see what an agent is actually doing.
+
+**[Conversation »](examples/conversation-form)**  |  **[Claude Desktop »](examples/custom-module)**  |  **[Autonomous »](examples/agentic-workflow)**
+
+## Documentation in this repository
 
-**[Go to Local Example »](examples/local)**
+This file is the main reference: what the UAG is, how to configure a form for it, how to deploy it, and how to troubleshoot it. The rest of the documentation lives next to the code it describes.
+
+| Document | What it covers |
+|----------|----------------|
+| **README.md** (this file) | Concepts, [agentic workflow setup](#agentic-workflows), the [MCP tools](#pre-defined-mcp-tools-providing-dynamic-context), [deployment](#deploying-uag), [environment variables](#environment-variables), and [troubleshooting](#troubleshooting). |
+| [module/Readme.md](./module/Readme.md) | Building a **custom module** — adding your own MCP tools, form actions, pre-defined forms and resources, and configuration overrides, then mounting it into the Docker container. |
+| [integrations/claude/Readme.md](./integrations/claude/Readme.md) | The **Claude integration**: its API endpoints and commands, how to run it with Node or Docker, its environment variables, and how to trigger it from a Form.io Webhook action. |
+| [examples/custom-module/Readme.md](./examples/custom-module/Readme.md) | Example — a custom module driven conversationally from **Claude Desktop**, where roles and permissions govern what the agent may do. |
+| [examples/conversation-form/Readme.md](./examples/conversation-form/Readme.md) | Example — a **chat application of your own** whose questions come entirely from a form. |
+| [examples/agentic-workflow/Readme.md](./examples/agentic-workflow/Readme.md) | Example — an **autonomous, webhook-triggered workflow** with two chained agent personas. |
+| [examples/flow-viewer/Readme.md](./examples/flow-viewer/Readme.md) | A small proxy that shows **every leg of an agentic run live** — the token request, each tool call, and the write-back. The quickest way to see what an agent is doing. |
+| [examples/custom-module/module/templates/Readme.md](./examples/custom-module/module/templates/Readme.md) | Overriding the **response templates** the UAG returns to the agent, using Lodash templates. |
+| [test/e2e/Readme.md](./test/e2e/Readme.md) | The **end-to-end test suite** — its three layers, what each covers, and how to run it. |
 
 ---
 
 ## Agentic Workflows
 One of the more powerful features of the **UAG** is the `agent_provide_data` tool. This tool provides the ability to instruct a generically trained agent how to analyze existing submission data, and then produce its own data by following a configurable **Criteria**. This behavior historically could only be achieved using a specifically trained agent, which does not provide any benefits of dynamic configurability that the Form.io platform offers. This tool is able to achieve this goal by providing a generally trained agent with the necessary "context" it needs to accurately produce its own data as part of an automated workflow. This feature is particularly helpful if you wish to utilize the UAG within an Agentic Workflow, where the AI Agent is capable of understanding structured data, and then contribute its own data by following the configured **Criteria** "context" provided by the UAG. 
 
-For example, let's suppose you wish to automate the backend administration behind a College Application Process. In this example, a potential student submits an application that consists of many different fields of data, such as Academnics, Extra curricular activities, Honors, Volunteer work, as well as possibly written Essays. Historically, these applications would be reviewed by an administrator in order to assess the candidates qualifications for acceptance. With the `agent_provide_data` tool, it is now possible to automate this process as the following diagram illustrates.
+For example, let's suppose you wish to automate the backend administration behind a College Application Process. In this example, a potential student submits an application that consists of many different fields of data, such as Academics, Extra curricular activities, Honors, Volunteer work, as well as possibly written Essays. Historically, these applications would be reviewed by an administrator in order to assess the candidates qualifications for acceptance. With the `agent_provide_data` tool, it is now possible to automate this process as the following diagram illustrates.
 
 ![Process example of using the UAG `agent_provide_data` to achieve Agentic workflow automation](./examples/images/uag-agent-provide-data.png)
 
@@ -44,6 +75,24 @@ To achieve this feat, the `agent_provide_data` tool utilizes the following infor
 
 ### Setup
  To configure a form to use the `agent_provide_data` tool, you must first designate a section of your form that will be read and used by the AI Agent. This is similar to what you would see in a form that says "For Office Use Only", but instead of a Human contributing to the values of this section, it will be an automated AI Agent. There are two types of fields that can be added to a form to configure it for use by the `agent_provide_data` tool:  **Criteria** and **Agent Fields**
+
+ A complete setup is four steps, and all four are required:
+
+ 1. **Tag the form `uag`** (described below) — without this the UAG will not register the form at all.
+ 2. Add the **Criteria** content component that teaches the agent what to do.
+ 3. Add the **Agent Fields** the agent is allowed to fill.
+ 4. Add a **Webhook action** to trigger the agent on submission — see the [Claude Integration](./integrations/claude/Readme.md#formio-webhook-setup). This step requires the Enterprise Server, because the action has to set a request header and transform its payload; on Community Edition, trigger the integration from your own code after creating the submission.
+
+ For a working reference containing all four, see the [Agentic Workflow example](./examples/agentic-workflow).
+
+#### Tagging the form `uag`
+The UAG only registers forms that carry the `uag` tag. Any form without it is invisible to every agent and to every tool, including `get_forms`. In the Form Builder this is set under **Form Settings → Tags**; in a form's JSON it looks like this:
+
+```json
+"tags": ["uag"]
+```
+
+This is worth checking first whenever an agent insists that a form does not exist, because a missing tag produces no error anywhere — the form simply is not there as far as the agent is concerned. Note also that newly tagged forms are picked up on the next project cache expiry, so allow up to `PROJECT_TTL` seconds (see [Project Cache and TTL](#project-cache-and-ttl)).
  
 #### Agent "Criteria" component
 The first thing that needs to be configured is a special **Content Component** that instructs the AI Agent how to interpret the data. For example, here is a Criteria content block that was written to instruct an AI Agent on how to assess the submission of a College Application Essay.
@@ -73,13 +122,47 @@ It is also possible to "flag" a collection of fields with the `uag` property. Th
 ### Multiple Personas per form
 It is also possible to achieve multiple personas per-form. This is very helpful if there are several isolated evaluations that need to occur within a process. For example, if we look at the flow chart diagram shown above, we can see a College Application Form. The first agentic evaluation occurs when the form is submitted by the applicant, where their application is assessed to be accepted or not. From that point, IF the applicant is accepted, a completely different evaluation needs to occur to award Financial Aid or even Scholarships to the applicant. These would require completely different criteria to assess the submission data differently from one another.
 
-To achieve this, you simply need to "flag" the components `uag` property value differently with the persona that applies for that field. For the example above, you would add a Content component to assess the if the applicant should be accepted, and then provide the following `uag` property to that content.  `uag="application"`.  Any fields that need to be filled out by the AI Agent for application approval would also be flagged with the property `uag="application"`.  Then, in a different section of the form, you would create a separate Content component with the criteria for the financial admin AI Agent. This content would be flagged with the property `uag="finance"`.  Any fields that the finance admin AI Agent need to fill out would also be flagged with `uag="finance"`.  This provides a truely flexible and dynamic AI engagement where multiple generically trained agents can be "taught" dynamically how to read and interpret data that is only relevant to that part of the agentic process.
+To achieve this, you simply need to "flag" the components `uag` property value differently with the persona that applies for that field. For the example above, you would add a Content component to assess if the applicant should be accepted, and then provide the following `uag` property to that content.  `uag="application"`.  Any fields that need to be filled out by the AI Agent for application approval would also be flagged with the property `uag="application"`.  Then, in a different section of the form, you would create a separate Content component with the criteria for the financial admin AI Agent. This content would be flagged with the property `uag="finance"`.  Any fields that the finance admin AI Agent need to fill out would also be flagged with `uag="finance"`.  This provides a truly flexible and dynamic AI engagement where multiple generically trained agents can be "taught" dynamically how to read and interpret data that is only relevant to that part of the agentic process.
+
+Agent fields do not have to sit next to their criteria, either. A `status` field in one panel can be flagged `uag="application"` while the field beside it is flagged `uag="finance"`, which is how a single workflow field can be owned by whichever persona is responsible for that stage.
+
+#### Chaining personas into a workflow
+Multiple personas on a form are only useful if something triggers the later ones. Because each persona writes to the submission, the natural trigger for the second persona is the *first* persona's write.
+
+This is configured with a second Webhook action that fires on `update` rather than `create`, narrowed by a condition on a field that the first persona sets:
+
+```json
+{
+  "name": "webhook", "handler": ["after"], "method": ["update"], "priority": 0,
+  "condition": {
+    "conjunction": "all",
+    "conditions": [
+      { "component": "status", "operator": "isEqual", "value": "scholarshipEvaluation" }
+    ]
+  },
+  "settings": {
+    "method": "post",
+    "url": "http://formio-uag-claude:3300/agent/claude/agent_provide_data",
+    "headers": [{ "header": "x-token", "value": "<your project API key>" }],
+    "transform": "payload = {formName: 'application', submissionId: payload.submission._id, persona: 'finance'};",
+    "block": false
+  }
+}
+```
+
+In the college application workflow, the admissions persona sets `status` to `scholarshipEvaluation` only for applicants on the scholarship track, so only those submissions wake the finance persona. Applicants who were simply accepted or declined never trigger it.
+
+> **Avoid loops.** The second persona also writes to the submission, which fires the same `update` webhook again. Make sure each persona's own write cannot re-match the condition that triggered it — in the example above the finance persona sets `status` to `scholarshipGranted`, which no longer satisfies the condition, so the workflow terminates. A condition that stays true after the persona runs will loop indefinitely and consume API credits until you stop the container.
+
+A complete, runnable version of this two-persona workflow is in the [Agentic Workflow example](./examples/agentic-workflow).
 
 ## Integrations
-In order to a aid in the "agentification" of the UAG, we have also provided some "integrations" that can be used to directly communicate with the Generically trained AI Agents to utilize the UAG to accomplish automated AI Agent behaviors. This involves providing a single API endpoint that is capable of triggering the AI Agent process along with the connections to your deployed UAG. These API endpoints can then be triggered using a simple Webhook action within a form to start an Agentic process as the result of a Form submission. Here is a diagram of how specific integrations can be used to achieve a single API call to accomplish an agentic task.
+In order to aid in the "agentification" of the UAG, we have also provided some "integrations" that can be used to directly communicate with the Generically trained AI Agents to utilize the UAG to accomplish automated AI Agent behaviors. This involves providing a single API endpoint that is capable of triggering the AI Agent process along with the connections to your deployed UAG. These API endpoints can then be triggered using a simple Webhook action within a form to start an Agentic process as the result of a Form submission. Here is a diagram of how specific integrations can be used to achieve a single API call to accomplish an agentic task.
 
 ![Agentic process flow via single API call into Claude Integration](./examples/images/agent-flow.png)
 
+Each integration runs its own MCP client and connects *out* to the UAG, so the UAG does not need to be publicly accessible for these to work — it only needs to be reachable from the integration, such as over a private Docker network or on `localhost`.
+
 These integrations can be found within the **integrations** folder. The following **integrations** are provided to enable automated Agentic workflows.
 
   - [Claude Integration](./integrations/claude)
@@ -128,7 +211,7 @@ While the UAG can be used as a stand-alone system to enable the interaction betw
  2. **Custom Actions**: Actions can be thought of as composable middleware functions that allow for configurable Express.js middleware functions to be attached to any Form and Resource, and then to be executed once the data being submitted by an AI agent has passed through the proper data validations and sanitization. These can be used to provide custom integrations to backend processes and workflows.
  3. **Custom Tools**: In addition to providing custom actions, a Module can also introduce custom MCP Tools to provide bespoke interaction capabilities between the UAG and the AI agents for any industry specific purposes.
  4. **Custom Authentication**: A module can also modify the "authentication" landing page to assist with any custom requirements that are needed around Authentication. 
- 5. **Behavior Overrides**: A module also has the ability to "override" any default behavior from the UAG. This ensures that whomever is using a certain module can "tweak" certain tools and responses to achieve better results as it pertains their their industry specific needs.
+ 5. **Behavior Overrides**: A module also has the ability to "override" any default behavior from the UAG. This ensures that whomever is using a certain module can "tweak" certain tools and responses to achieve better results as it pertains to their industry specific needs.
 
 Form.io aims to enable developers to build and contribute their own custom modules to the broader community through GitHub and NPM. 
 
@@ -165,10 +248,46 @@ The following sections describe the process of creating a running instance of UA
 
 This documentation focuses on using UAG with Form.io Open Source. For documentation on using UAG with Form.io Enterprise, refer to [https://help.form.io/uag](https://help.form.io/uag).
 
+### How Agents Connect
+Before deploying, it is worth deciding **which side opens the connection** to the UAG's `/mcp` endpoint, because that single decision determines whether the UAG needs to be publicly accessible.
+
+| Connection model | Who runs the MCP client | Does the UAG need a public domain? |
+|------------------|-------------------------|------------------------------------|
+| **Local MCP client** | Your own application or integration, running alongside the UAG | **No.** The UAG only needs to be reachable from that process — `http://localhost:3200` or a private Docker network address is fine. |
+| **Remote MCP client** | A hosted agent that dials into your UAG (Claude.ai and Claude Desktop connectors, the Anthropic MCP connector, or any other remote MCP host) | **Yes.** The agent must be able to reach `/mcp` and the OAuth discovery endpoints over the public internet. |
+
+With a **local MCP client**, your application connects *out* to the UAG, calls the tools itself, and passes the results on to whichever model it is using. Nothing dials back in, so the UAG can stay entirely inside your network. This is how the [Claude Integration](./integrations/claude) works, and it is the recommended model for server-to-server and webhook-triggered automation — including running the whole stack on a developer machine.
+
+> If you are using the published Claude integration image, this requires **`formio/uag-claude:1.3.0` or newer**. Earlier images handed the MCP server URL to the Anthropic API as a *remote* connector, meaning the API dialed into your UAG and a public URL was mandatory. An older image pointed at a private address fails with a `400` from the Anthropic API reading `Connection error while communicating with MCP server`, which looks like a UAG fault but is not one.
+
+With a **remote MCP client**, the agent lives outside your network and connects in, so `/mcp` and the `.well-known` OAuth discovery endpoints must be publicly reachable over HTTPS, and `BASE_URL` must be that public URL. See [Domain and URL Configuration](#domain-and-url-configuration).
+
+Both models can be used against the same UAG deployment at the same time.
+
+#### Authenticating a local MCP client
+A remote agent authenticates through the browser-based OAuth (PKCE) flow. A local client has no browser, so it exchanges a server-side key for an access token using the `client_credentials` grant against `{{ BASE_URL }}/auth/token`, then sends that token as a `Bearer` token on its `/mcp` requests.
+
+The `client_id` must be prefixed with the project name — which is `formio-oss` for Open Source deployments:
+
+```
+POST /auth/token
+Content-Type: application/json
+
+{
+  "grant_type": "client_credentials",
+  "client_id": "formio-oss-x-admin-key",
+  "client_secret": "{{ ADMIN_KEY }}"
+}
+```
+
+For Enterprise, use the project name from your project endpoint together with the project API key — for example `"client_id": "myproject-x-token"` with `"client_secret": "{{ PROJECT_KEY }}"`.
+
+The response contains an `access_token`, which is passed to the UAG as `Authorization: Bearer {{ access_token }}`. Tokens expire according to `JWT_EXPIRE_TIME`, so long-running clients should cache the token and refresh it before it expires. See [`integrations/claude/auth.js`](./integrations/claude/auth.js) for a working implementation.
+
 ### Runtime Environments
 There are currently two run-time environments that work with the UAG:
- -[**Docker**](#docker)
- -[**Node.js (Express)**](#nodejs-express)
+ - [**Docker**](#docker)
+ - [**Node.js (Express)**](#nodejs-express)
 
 
 ### Docker
@@ -179,16 +298,15 @@ formio/uag
 ```
 
 This container can be run in two ways:  
-  - [**Docker-Compose.yml**](#docker-compose-recommended) - The recommended method, deploys with of a Docker Compose multicontainer.
+  - [**Docker-Compose.yml**](#docker-compose-recommended) - The recommended method, deploys as a multi-container Docker Compose stack.
   - [**Docker Run**](#docker-run) - a standalone container using the common `docker run` command.
 
 
 #### Docker Compose (Recommended)
-The recommended way to launching the UAG is through Docker Compose. This enables you to orchestrate several of the containers to run within a single instance to provide a more seamless and simple way of managing your deployments. Here is a simple example of how to run both the Form.io UAG + Form.io OSS server on the same instance.
+The recommended way to launch the UAG is through Docker Compose. This enables you to orchestrate several of the containers to run within a single instance to provide a more seamless and simple way of managing your deployments. Here is a simple example of how to run both the Form.io UAG + Form.io OSS server on the same instance.
 
 docker-compose.yml
 ```
-version: "3.8"
 services:
   mongo:
     image: mongo
@@ -228,8 +346,10 @@ services:
       ADMIN_KEY: CHANGEME
       JWT_SECRET: CHANGEME
       JWT_EXPIRE_TIME: 525600
-      LOGIN_FORM: http://localhost:3000/employee/login
-      BASE_URL: http://localhost:3000
+      LOGIN_FORM: http://localhost:3000/user/login
+      # The address clients reach the UAG at — this is the UAG's own port,
+      # not the Form.io server's. See the BASE_URL section below.
+      BASE_URL: http://localhost:3200
     ports:
       - "3200:3200"
 ```
@@ -259,7 +379,7 @@ docker run -d \
   formio/uag
 ```
 
-**UAG pointed to an Form.io Enterprise Server**
+**UAG pointed to a Form.io Enterprise Server**
 ```
 docker run -d \
   -e "PROJECT=https://forms.mysite.com/myproject" \
@@ -281,7 +401,7 @@ Once it is running, you can then navigate to the following to access both the OS
  - http://localhost:3000:  The Form.io OSS Server
  - http://localhost:3200:  The UAG Server
 
-In both the Node.js runtime environmnt as well as Docker, the way to control the UAG is through the use of **Environment Variables** and **Modules**.
+In both the Node.js runtime environment as well as Docker, the way to control the UAG is through the use of **Environment Variables** and **Modules**.
 
 ### Environment Variables
 This module can be configured in many ways. One of those ways is through the use of Environment Variables, which are documented as follows.
@@ -289,7 +409,7 @@ This module can be configured in many ways. One of those ways is through the use
 | Variable | Description | Example |
 |----------|-------------|---------|
 | PROJECT | The API Endpoint to either an Enterprise project endpoint, or the OSS server url. | http://localhost:3000 |
-| PROJECT_KEY | (Enterprise Only) Either a Project API Key (for Form.io Enterprise) or the ADMIN_KEY for Open Source. | CHANGEME |
+| PROJECT_KEY | (Enterprise Only) The API Key of the project this UAG is bound to, found in **Project Settings &rarr; API Keys**. Open Source deployments use `ADMIN_KEY` instead. | CHANGEME |
 | ADMIN_KEY | (Open Source Only) Allows you to provide the ADMIN_KEY to install and connect to the OSS Server. | CHANGEME |
 | UAG_LICENSE | The license to run the UAG against a Form.io Enterprise Deployment. | |
 | PROJECT_TTL | The project cache TTL in seconds. This controls the amount of time to check for any "changes" that have been made to the project to refresh any forms and resources within the UAG. | 900 |
@@ -300,12 +420,13 @@ This module can be configured in many ways. One of those ways is through the use
 | JWT_EXPIRE_TIME | The expiration for the jwt secret. | 3600 |
 | MONGO | (Enterprise Only) Allows you to connect the UAG directly to a mongo database, rather than having to redirect the submissions to the Form.io Submission APIs. | |
 | MONGO_CONFIG | JSON configuration for the Node.js Mongo Driver. | |
-| BASE_URL | The public URL that the UAG is hosted on. This allows for proper OIDC authentication and allows for the authentication callbacks to point to the correct url. | https://forms.yoursite.com |
-| LOGIN_FORM | The public URL to the Login Form JSON endpoint. | https://mysite.com/project/user/login |
+| BASE_URL | **Required.** The URL that the UAG is reachable at *by its clients*. It is published in the `.well-known` OIDC (PKCE) definitions and used for authentication callbacks. This must be a public URL when remote agents connect in; for a local MCP client, a private or localhost URL is fine. | https://forms.yoursite.com |
+| LOGIN_FORM | The URL to the Login Form JSON endpoint, loaded by the `/auth/authorize` page. Only used by the browser-based login flow, so it must be reachable by that browser. | https://mysite.com/project/user/login |
 | CORS | The cors domain, or the JSON configuration to configure the "cors" node.js module cross domain resource sharing. | *.* |
+| NODE_TLS_REJECT_UNAUTHORIZED | Standard Node.js variable. Set to `0` when `PROJECT` is an `https` URL secured by a self-signed or otherwise untrusted certificate, which is common for local and internal deployments — without it the UAG cannot connect to the server and every tool call fails. Never set this on a deployment reachable from outside your network, since it disables certificate verification for all outbound requests. | 0 |
 
 ### Project Cache and TTL
-By default, the UAG uses a TTL to "fetch" any of the project assets and register them and cache them into the UAG. What this means is that if you make any changes to your underlying project forms and resources (either directly or through a deployment), you need to wait a maximum of the TTL setting (in seconds) to see any updates that have been made. For example, if the TTL is set to 900 (or 15 minutes), and you make a change to any forms and resources, you must wait at most 15 minutes to see any of these changes take effect into the UAG. It should be noted that when a "refresh" occurs, an API call to the "export" method is called on the underlying project endpoint. This is a processor intensive API call, so it is not recommended to set the TTL to any value less than 60.  A value of 0 is interpreted as "No TTL", which means the only way to refresh the project forms and resourcs, a UAG reboot is necessary.
+By default, the UAG uses a TTL to "fetch" any of the project assets and register them and cache them into the UAG. What this means is that if you make any changes to your underlying project forms and resources (either directly or through a deployment), you need to wait a maximum of the TTL setting (in seconds) to see any updates that have been made. For example, if the TTL is set to 900 (or 15 minutes), and you make a change to any forms and resources, you must wait at most 15 minutes to see any of these changes take effect into the UAG. It should be noted that when a "refresh" occurs, an API call to the "export" method is called on the underlying project endpoint. This is a processor intensive API call, so it is not recommended to set the TTL to any value less than 60.  A value of 0 is interpreted as "No TTL", which means the only way to refresh the project forms and resources, a UAG reboot is necessary.
 
 Here is a table explaining how this parameter can be used.
 
@@ -315,8 +436,16 @@ Here is a table explaining how this parameter can be used.
 | PROJECT_TTL | 60 | Check for changes every minute |
 | PROJECT_TTL | 3600 | Check for changes every hour |
 
-### Running on Public Domain
-In order to run the UAG on a public domain, it is very important to provide the proper configurations so that any AI agent can properly authenticate. There are 3 different "domain" environment variables that matter, and it is important to understand how to configure them depending on your use case:
+This cache is the reason a form edit does not appear to take effect immediately. Anything that changes what the agent knows — adding the `uag` tag, rewriting a **Criteria** content component, adding or removing **Agent Fields** — only reaches the agent after the next expiry. When you are actively iterating on criteria, restarting the UAG re-registers the project immediately instead of waiting:
+
+```bash
+docker restart formio-uag
+```
+
+### Domain and URL Configuration
+There are 3 different "domain" environment variables that matter, and it is important to understand how to configure them depending on your use case. Whether any of them need to be *public* depends on how agents connect — see [How Agents Connect](#how-agents-connect).
+
+If a remote agent connects into your UAG, these must be configured correctly and publicly, or the agent will not be able to authenticate. If you only use a local MCP client, all three can point at private or localhost addresses.
 
 #### PROJECT
 The ```PROJECT``` environment variable is used to establish a connection from the UAG server to the project endpoint (for Enterprise) or OSS base url. This does NOT need to be a public DNS entry, but rather a URL that connects the UAG container to the Server container.  The following examples illustrate how this would be configured.
@@ -326,7 +455,6 @@ If you are using a local connection, such as within a Docker Compose file, you c
 
 docker-compose.yml
 ```
-version: "3.8"
 services:
   formio:
     image: formio/formio:rc
@@ -353,7 +481,6 @@ If you are using the Enterprise Form.io server within a local environment to the
 
 docker-compose.yml
 ```
-version: "3.8"
 services:
   formio-enterprise:
     image: formio/formio-enterprise
@@ -377,7 +504,6 @@ If your project does not reside on the same network as the UAG, you can provide
 
 docker-compose.yml: Connected to Enterprise (formio/formio-enterprise)
 ```
-version: "3.8"
 services:
   formio-uag:
     image: formio/uag
@@ -389,7 +515,6 @@ services:
 
 docker-compose.yml: Connected to Open Source (formio/formio)
 ```
-version: "3.8"
 services:
   formio-uag:
     image: formio/uag
@@ -400,14 +525,21 @@ services:
 ```
 
 #### BASE_URL
-The ```BASE_URL``` is used to communicate to the AI agent the public domain that is hosting the UAG server. This value is provided within the ```.well-known``` definitions for the OIDC (PKCE) authentication. If this is not correct, then the AI agent will not be able to authenticate into the UAG.
+The ```BASE_URL``` tells the UAG what URL it is reachable at. This value is published within the ```.well-known``` definitions for the OIDC (PKCE) authentication, and is used for the authentication callbacks.
+
+**```BASE_URL``` is always required**, including for local clients. The UAG's authentication provider is only enabled when it is set, so if you omit it, ```/mcp``` returns a 401 for every request and the ```/auth/token``` and ```/.well-known/*``` endpoints do not exist at all (404). A 404 on ```/auth/token``` is the clearest signal that ```BASE_URL``` is missing.
+
+What it needs to be set to depends on how agents connect:
 
-**This needs to be the publicly accessible domain that you are hosting your UAG.**
+ - **Remote agents connect in**: this must be the publicly accessible domain hosting your UAG, because that is the address the agent is told to authenticate against. For example, ```BASE_URL: https://forms.mysite.com```.
+ - **Local MCP client only**: set it to the address your client uses to reach the UAG. A private or localhost URL is fine — for example ```BASE_URL: http://localhost:3200``` or ```BASE_URL: http://formio-uag:3200``` within a Docker network.
 
-For example, ```BASE_URL: https://forms.mysite.com```.
+If a remote agent cannot authenticate, a mismatched ```BASE_URL``` is the most common cause: the agent reads this value from the discovery document and will follow it, so if it points somewhere the agent cannot reach, authentication fails even though the UAG itself is running correctly.
 
 #### LOGIN_FORM
-This is the publicly accessible URL to the Login form of your Project or OSS deployment. This provides the URL that is loaded when the user navigates to ```{{ BASE_URL }}/auth/authorize```.  If you navigate to this URL, and the page says that you cannot load the form, then this is because your LOGIN_FORM environment variable is not pointing to the correct form JSON endpoint of your project. 
+This is the URL to the Login form of your Project or OSS deployment. It provides the form that is loaded when a user navigates to ```{{ BASE_URL }}/auth/authorize```.  If you navigate to this URL, and the page says that you cannot load the form, then this is because your LOGIN_FORM environment variable is not pointing to the correct form JSON endpoint of your project. 
+
+This is only used by the browser-based login flow, so it needs to be reachable by the browser performing that login. A local MCP client authenticating with the `client_credentials` grant never loads this page.
 
 For example:
  - Enterprise Example: ```LOGIN_FORM: https://forms.mysite.com/myproject/user/login```
@@ -465,3 +597,23 @@ In order to run the UAG against an Enterprise Server, you need to provide a few
 
 Once you have these environment variables in place, you should be able to run the UAG pointed to your Enterprise Project.
 
+## Troubleshooting
+Most UAG problems are configuration, not code, and they tend to surface far from their cause — a missing tag looks like a missing form, and a missing `BASE_URL` looks like a permissions error. The table below maps the symptoms you are likely to see back to what actually produces them.
+
+| Symptom | Likely cause |
+|---------|--------------|
+| The agent says a form does not exist, or `get_forms` does not list it | The form is not tagged `uag`, or the change has not survived the project cache yet. See [Tagging the form `uag`](#tagging-the-form-uag) and [Project Cache and TTL](#project-cache-and-ttl). |
+| `/auth/token` returns `404`, and `/.well-known/*` is also missing | `BASE_URL` is not set. The authentication provider is only enabled when it is present, so those endpoints do not exist at all. See [BASE_URL](#base_url). |
+| Every `/mcp` request returns `401` | Either `BASE_URL` is unset (as above), or the client is not sending `Authorization: Bearer <token>`. For a server-side client, see [Authenticating a local MCP client](#authenticating-a-local-mcp-client). |
+| A remote agent (claude.ai, Claude Desktop connector) cannot authenticate | `BASE_URL` does not point at a publicly reachable HTTPS URL. The agent reads that value out of the discovery document and follows it, so it must be correct from the agent's perspective, not the server's. |
+| The Claude integration returns `400 ... Connection error while communicating with MCP server` | The `formio/uag-claude` image predates `1.3.0` and is treating the UAG as a *remote* MCP server, so the Anthropic API is trying to dial into a private address. Upgrade to `1.3.0` or newer. See [How Agents Connect](#how-agents-connect). |
+| The UAG cannot reach the server at all, and every tool fails | If `PROJECT` is an `https` URL with a self-signed certificate, set `NODE_TLS_REJECT_UNAUTHORIZED=0`. Otherwise check that `PROJECT` is resolvable *from inside the UAG container* — it is a container-to-container address, not a browser one. See [PROJECT](#project). |
+| Nothing happens when a submission is created | The Webhook action did not fire. Confirm its `handler` is `after` and `method` includes `create`, and that the API key header matches the integration's `PROJECT_KEY` (Enterprise) or `ADMIN_KEY` (OSS). |
+| A submission is created but the response hangs for a long time | The Webhook action has `block: true`. Agent runs take tens of seconds; set `block: false` so the submitter is not made to wait. |
+| An agent runs continuously and burns API credits | A chained persona is re-triggering itself. Its own write still satisfies the condition on the `update` webhook that started it. See [Chaining personas into a workflow](#chaining-personas-into-a-workflow). |
+| The agent fills in some fields but not others | Those fields are missing the `uag` property, or its value does not exactly match the persona on the **Criteria** component. Every field the agent may write must carry the same persona, either directly or by inheriting it from a parent component. |
+| The agent writes values the form rejects | The **Criteria** is not specific enough about formats and allowed values. State units, rounding, and exact option values, and name the field keys verbatim. |
+| A run fails and the logs are not enough to see where | Put the [Flow Viewer](./examples/flow-viewer) in front of the UAG. It shows the token request, `tools/list`, each tool call with arguments, and the write-back in order. |
+
+If none of these apply, run one of the [examples](#try-it-out) to confirm the environment itself is sound, then compare its configuration against yours.
+

package/lib/UAGFormInterface.d.ts

@@ -89,7 +89,7 @@ export declare class UAGFormInterface extends FormInterface {
     }[] | {
         data: {};
     } | {
-        data?: undefined;
+        data?: {};
     } | null;
     /**
      * Override the process methods for the UAG.

package/lib/UAGProjectInterface.d.ts

@@ -13,22 +13,23 @@ export declare class UAGProjectInterface extends ProjectInterface {
     get config(): UAGConfig;
     constructor(endpoint?: string);
     initialize(): Promise<void>;
+    buildMcpServer(): Promise<McpServer>;
     addForm(form: Form, machineName: string): UAGFormInterface | undefined;
     uagRouter(): Router;
     authorizeRequest(req: SubmissionRequest, res: Response, next: NextFunction): Promise<Response<any, Record<string, any>> | undefined>;
     mcpJSONResponse(data?: object, isError?: boolean): {
-        isError?: boolean | undefined;
         content: {
-            type: "text";
+            type: 'text';
             text: string;
         }[];
         structuredContent: object;
+        isError?: boolean | undefined;
     };
     mcpResponse(templateName: ResponseTemplate, data?: object, isError?: boolean): {
-        isError?: boolean | undefined;
         content: {
-            type: "text";
+            type: 'text';
             text: string;
         }[];
+        isError?: boolean | undefined;
     };
 }

package/lib/UAGProjectInterface.js

@@ -20,6 +20,12 @@ class UAGProjectInterface extends appserver_1.ProjectInterface {
     async initialize() {
         await super.initialize();
         this.uagTemplate = new template_1.UAGTemplate(this.config?.responseTemplates || {});
+        this.mcpServer = await this.buildMcpServer();
+    }
+    // Build a fresh McpServer with all tools registered. A new instance is created
+    // per request so concurrent requests never share (and overwrite) a transport.
+    async buildMcpServer() {
+        const server = new mcp_js_1.McpServer({ name: 'formio-uag', version: '1.0.0' });
         // Get the standard UAG tools.
         const tools = await (0, tools_1.getTools)(this);
         // Add the custom tools from config.
@@ -29,13 +35,14 @@ class UAGProjectInterface extends appserver_1.ProjectInterface {
         // Iterate and register all the tools.
         for (const tool of tools) {
             if (tool?.name) {
-                this.mcpServer.registerTool(tool.name, {
+                server.registerTool(tool.name, {
                     title: tool.title,
                     description: tool.description,
                     inputSchema: tool.inputSchema
                 }, tool.execute);
             }
         }
+        return server;
     }
     addForm(form, machineName) {
         const formInterface = super.addForm(form, machineName);

package/lib/index.d.ts

@@ -7,7 +7,7 @@ export type UAGModule = ServerModule & {
 };
 export declare class UAGServer extends Server {
     constructor(config?: UAGConfig);
-    router(): Promise<import("express-serve-static-core").Router>;
+    router(): Promise<any>;
 }
 export * from '@formio/appserver';
 export * from './config';

package/lib/router.js

@@ -3,7 +3,6 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.UAGRouter = UAGRouter;
 const express_1 = require("express");
 const streamableHttp_js_1 = require("@modelcontextprotocol/sdk/server/streamableHttp.js");
-const types_js_1 = require("@modelcontextprotocol/sdk/types.js");
 // Helper function to create JSON RPC error responses
 const createJsonRpcErrorResponse = (code, message) => {
     return JSON.stringify({
@@ -19,6 +18,8 @@ const createJsonRpcErrorResponse = (code, message) => {
  * @returns True if the error was handled, false otherwise.
  */
 const handleResponseError = (error, res) => {
+    // A thrown fetch Response (the global Response, not Express's) carries the
+    // upstream status and headers that should be forwarded to the client.
     if (error instanceof Response) {
         const fixedHeaders = {};
         error.headers.forEach((value, key) => {
@@ -31,22 +32,29 @@ const handleResponseError = (error, res) => {
     res.status(500).json(JSON.parse(createJsonRpcErrorResponse(-32603, "Internal Server Error")));
     return false;
 };
+/**
+ * Build a per-request McpServer + transport (stateless mode) and tear both down
+ * when the response closes. A fresh instance per request keeps concurrent requests
+ * from overwriting each other's transport.
+ */
+const connectMcpTransport = async (project, res) => {
+    const server = await project.buildMcpServer();
+    const transport = new streamableHttp_js_1.StreamableHTTPServerTransport({
+        sessionIdGenerator: undefined, // Use "undefined" to trigger stateless mode.
+    });
+    res.on('close', () => {
+        transport.close();
+        server.close();
+    });
+    await server.connect(transport);
+    return transport;
+};
 function UAGRouter(project) {
     const router = (0, express_1.Router)();
     // Handles the MCP post requests.
     router.post('/', async (req, res) => {
         try {
-            const transport = new streamableHttp_js_1.StreamableHTTPServerTransport({
-                sessionIdGenerator: undefined, // Use "undefined" to trigger stateless mode.
-            });
-            // Close any existing connection before connecting to a new transport (required in SDK 1.26.0+)
-            await project.mcpServer.close();
-            await project.mcpServer.connect(transport);
-            if (!transport) {
-                res.status(500).json(JSON.parse(createJsonRpcErrorResponse(-32603, "Unable to create MCP transport.")));
-                return;
-            }
-            (0, types_js_1.isInitializeRequest)(req.body);
+            const transport = await connectMcpTransport(project, res);
             await transport.handleRequest(req, res, req.body);
         }
         catch (error) {
@@ -56,16 +64,7 @@ function UAGRouter(project) {
     // Handles the MCP get requests (for health checks, etc).
     router.get('/', async (req, res) => {
         try {
-            const transport = new streamableHttp_js_1.StreamableHTTPServerTransport({
-                sessionIdGenerator: undefined, // Use "undefined" to trigger stateless mode.
-            });
-            // Close any existing connection before connecting to a new transport (required in SDK 1.26.0+)
-            await project.mcpServer.close();
-            await project.mcpServer.connect(transport);
-            if (!transport) {
-                res.status(500).json(JSON.parse(createJsonRpcErrorResponse(-32603, "Unable to create MCP transport.")));
-                return;
-            }
+            const transport = await connectMcpTransport(project, res);
             await transport.handleRequest(req, res);
         }
         catch (err) {

package/lib/templates/agentProcessData.md

@@ -1,4 +1,4 @@
-Please read and understand the following **--- CRITERIA ---** section. This section provides instructions on how to analyze the **--- SUBMISSION ---** section that follows. Your objective is to provide values defined within the **--- REQUIRED FIELDS ---** section, and then to submit those values using the `agent_submit_data` tool.
+Please read and understand the following **--- CRITERIA ---** section. This section provides instructions on how to analyze the **--- SUBMISSION ---** section that follows. Your objective is to provide values defined within the **--- REQUIRED FIELDS ---** section, and then to submit those values using the `submission_update` tool.
 
 --- CRITERIA ---
 <%= criteria %>

package/lib/templates/submitValidationError.md

@@ -1,6 +1,6 @@
 There are some validation errors in the data collected:
 <% validationErrors.forEach(function(error) { %>
- - <% if (error.label && error.path) { %>**<%= error.label %> (<%= error.path %>)**: <% } %><%= error.message %><% }); %>
+ - <% if (error.label && error.path) { %>**<%= error.label %> (<%= error.path %>)**: <% } %><%= error.error || error.message %><% }); %>
 
 These validation errors are in one of the following format:
  - **Field Label (data_path)** - Error message:  If the error is attributed to a specific field value.

package/lib/tools/submitForm.js

@@ -21,13 +21,28 @@ const submitCompletedForm = async (project) => {
             try {
                 let submission = form.convertToSubmission(form_data);
                 const submitted = await form.submit(submission, extra.authInfo);
-                if (submitted)
+                if (submitted) {
+                    // The submission save proxies to the Form.io server, and a rejection
+                    // (e.g. failed validation) can come back through the save pipeline as
+                    // the error response body instead of a saved submission. A result
+                    // without an _id was not persisted — report the failure to the agent
+                    // instead of a false success.
+                    const result = submitted;
+                    if (!result._id) {
+                        const errors = (result.name === 'ValidationError' && result.details)
+                            ? form.convertToFormFieldErrors(result.details)
+                            : [];
+                        return project.mcpResponse(template_1.ResponseTemplate.submitValidationError, {
+                            validationErrors: errors.length ? errors : [{
+                                    message: result.message || 'The submission was not saved by the server. Verify that all required fields have been collected and that every value conforms to its field rules, then try again.'
+                                }]
+                        }, true);
+                    }
                     submission = submitted;
-                if (!submission) {
-                    return project.mcpResponse(template_1.ResponseTemplate.submitValidationError, {
-                        validationErrors: ['Unknown error during submission']
-                    }, true);
                 }
+                // A null result is still a valid outcome for "action" forms that have no
+                // Save Submission action (e.g. email or webhook only) — the data was
+                // processed but intentionally not persisted, so no submission ID exists.
                 debug(`Form submitted: ${form_name} with submission ID: ${submission._id}`);
                 return project.mcpResponse(template_1.ResponseTemplate.formSubmitted, {
                     form: form.form,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@formio/uag",
-  "version": "1.11.2",
+  "version": "1.12.1",
   "mcpName": "io.form/uag",
   "description": "The Form.io Universal Agent Gateway (UAG).",
   "main": "lib/index.js",
@@ -13,11 +13,12 @@
   ],
   "scripts": {
     "clean": "rm -rf lib",
-    "build:ts": "npx tsc",
+    "build:ts": "tsc",
     "build:copy-templates": "mkdir -p lib/templates && cp src/templates/*.md lib/templates/",
     "build": "npm run clean && npm run build:ts && npm run build:copy-templates",
     "build:docker": "docker build -t formio/uag$(node -p \"require('./package.json').version.includes('-rc.') ? ':rc' : ''\") --platform=linux/amd64 .",
-    "test": "TS_NODE_PROJECT=tsconfig.test.json mocha --no-node-snapshot",
+    "test": "tsc --noEmit -p tsconfig.test.json && mocha --no-node-snapshot",
+    "test:e2e": "mocha --config .mocharc.e2e.json",
     "dev": "tsx --no-node-snapshot index.js",
     "start": "node --no-node-snapshot index.js",
     "claude": "node --no-node-snapshot ./integrations/claude/index.js"
@@ -29,18 +30,18 @@
   "author": "Form.io Developers",
   "license": "MIT",
   "dependencies": {
-    "@formio/appserver": "^2.12.2",
-    "@formio/core": "2.5.1-dev.291.6557e4e",
-    "@modelcontextprotocol/sdk": "1.27.1",
+    "@formio/appserver": "^2.13.1",
+    "@formio/core": "2.8.0",
+    "@modelcontextprotocol/sdk": "1.30.0",
     "cors": "^2.8.6",
     "debug": "^4.4.3",
-    "dotenv": "^17.3.1",
+    "dotenv": "^17.4.2",
     "express": "^5.2.1",
     "jsonwebtoken": "^9.0.3",
-    "lodash": "^4.17.23",
+    "lodash": "^4.18.1",
     "node-cache": "^5.1.2",
     "node-html-markdown": "^2.0.0",
-    "zod": "^4.3.6"
+    "zod": "^4.4.3"
   },
   "resolutions": {
     "tar": ">=7.5.12",
@@ -51,22 +52,23 @@
     "dompurify": ">=3.3.3"
   },
   "devDependencies": {
+    "@anthropic-ai/sdk": "^0.115.0",
     "@types/chai": "^5.2.3",
     "@types/cors": "^2.8.19",
-    "@types/debug": "^4.1.12",
+    "@types/debug": "^4.1.13",
     "@types/dotenv": "^8.2.3",
     "@types/express": "^5.0.6",
     "@types/jsonwebtoken": "^9.0.10",
     "@types/lodash": "^4.17.24",
     "@types/mocha": "^10.0.10",
-    "@types/node": "^25.5.0",
-    "@types/supertest": "^7.2.0",
+    "@types/node": "^26.1.2",
+    "@types/supertest": "^7.2.1",
     "chai": "^6.2.2",
-    "mocha": "^11.7.5",
+    "mocha": "^11.7.6",
     "supertest": "^7.2.2",
-    "terser": "^5.46.1",
-    "ts-node": "^10.9.2",
-    "tsx": "^4.21.0",
-    "typescript": "^5.9.3"
-  }
+    "terser": "^5.49.0",
+    "tsx": "^4.23.1",
+    "typescript": "^7.0.2"
+  },
+  "packageManager": "yarn@4.17.1"
 }