@formio/uag 1.4.1 → 1.4.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.md +117 -17
  2. package/lib/templates/getFormFields.md +4 -1
  3. package/package.json +6 -6
package/README.md CHANGED
@@ -1,10 +1,65 @@
1
1
  # The Form.io "Universal Agent Gateway" (UAG)
2
- The Universal Agent Gateway (uag) is an exciting new technology that enables the exposure of Form.io forms to AI Agents. Leveraging the power of MCP, this library provides AI Agents with dynamic context of how to interface with any backend system using Form.io JSON Forms as the common language between these agents and enterprise systems.
2
+ The Universal Agent Gateway (UAG) is an exciting new technology that brings JSON powered (model driven) [Form.io](https://form.io) forms to AI Agents. Leveraging the power of the [**Model Context Protocol (MCP)**](https://modelcontextprotocol.io/docs/getting-started/intro), this library provides AI Agents with **dynamic context** of how to interface with any backend system using Form.io JSON Forms as the common language between these agents and Enterprise systems.
3
3
 
4
- This library can be used in many ways, either by deploying it directly via the Docker container, or by extending the interfaces and introducing your own custom Actions and providers.
4
+ ---
5
+
6
+ ## Table of Contents
7
+
8
+ - [The Form.io "Universal Agent Gateway" (UAG)](#the-formio-universal-agent-gateway-uag)
9
+ - [Introduction](#introduction)
10
+ - [How it works](#how-it-works)
11
+ - [Technical Overview](#technical-overview)
12
+ - [MCP Tools](#pre-defined-mcp-tools-providing-dynamic-context)
13
+ - [Custom Modules](#custom-modules)
14
+ - [Module Documentation](./module)
15
+ - [Getting Started](#getting-started)
16
+ - [Example Local Deployment](./examples/local)
17
+ - [Runtime Environments](#runtime-environments)
18
+ - [Node.js (Express):](#nodejs-express)
19
+ - [Docker](#docker)
20
+ - [Environment Variables](#environment-variables)
21
+ - [Running on Public Domain](#running-on-public-domain)
22
+ - [Using with Form.io Enterprise Server](#using-with-formio-enterprise-server)
23
+
24
+ ---
25
+
26
+ ## Introduction
27
+ The core purpose of the UAG is to provide AI Agents the ability to understand dynamic data structures defined from JSON form definitions. It accomplishes this goal by combining two innovative Open Source technologies.
28
+
29
+ - [**Form.io JSON Forms**](https://form.io): The Form.io platform enables deterministic data collection through the use of JSON form definitions created with a drag-and-drop form builder interface. Once created, these forms serve as more than just a front-end interface for humans to "fill out", but rather serve as a standard data model contract between applications and any backend processes that depend on that data. It ensures that every system is speaking the same language and provides human-in-the-loop experiences (through rendered forms) to collect, audit, and report on the data exchanges between many complex data driven systems. Here is a visual graphic showing how the Form JSON serves as the "centerpiece" contract to ensure data integrity throughout your Enterprise.
30
+
31
+ ![](./examples/images/form-interfaces.png)
32
+
33
+ - [**Model Context Protocol (MCP)**](https://modelcontextprotocol.io/docs/getting-started/intro): The Model Context Protocol (MCP) is a standard for connecting AI Applications to external systems. It achieves this by providing AI Agents with "context" so that they can understand how to interact with the external systems. When combined with the Form.io's dynamic JSON forms, the "context" that is provided to the AI Agents is also dynamic in nature making it so that AI Agent interactions can easily be controlled via the Form JSON models that are defined by Form.io
5
34
 
6
35
  ## How it works
7
- The UAG leverages the [**Model Context Protocol (MCP)**](https://modelcontextprotocol.io/docs/getting-started/intro) to provide AI Agents the ability to understand the dynamic data structures provided from [Form.io](https://form.io) JSON form definitions. It achieves this though the introduction of new MCP "tools" to provide an AI Agent with dynamic "form" based context. This cabability allows for Developers to treat AI Agents as they would any other Human who is submitting a form, and ensure that the AI Agents stay "on the rails" without requiring extensive domain specific agent training methods to accomplish the same goal. It achieves this by exposing the following MCP tools to an AI Agent.
36
+
37
+ The UAG provides **dynamic context** to AI Agents though the introduction of new MCP "tools" that describe the dynamic form structures to AI Agents so that they understand how to properly structure the data being processed by those agents. This cabability allows for Developers to treat AI Agents as they would any other Human who is submitting a form, and ensure that the AI Agents stay "on the rails" without requiring extensive domain specific agent training methods to accomplish the same goal. But more imporantly, the UAG offers Enterprises the ability to create trustworthy data interfaces through the use of dynamic and composable JSON data model definitions created and managed through Form.io. This not only enforces deterministic data interactions from AI Agents, but also provides an auditable human-in-the-loop experience through the provided Form.io Open Source and Enterprise portal applications.
38
+
39
+ Here is a visual graphic of the layers provided by the UAG to achieve a trusted and deterministic interface between AI Agents to external systems through the UAG + Form.io Server.
40
+
41
+ ![](./examples/images/agents_to_formio.png)
42
+
43
+ ## Technical Overview
44
+ The UAG can be thought of as consisting of many "layers" of functionality, broken up between the actions that fetch dynamic context and the ones that submit deterministic data structures. The following layers are leveraged within the UAG.
45
+
46
+ 1. **MCP Interface**: The entry point of the UAG is the MCP interface that sits between the AI Agent and the tools that are executed on the server.
47
+ 2. **Authorization**: As part of the MCP specification, every interaction betwen the AI Agent and the backend service must pass through an Authorization process leveraging OIDC (PKCE) authentication. The UAG takes this process one step further, however, by adding an OIDC (PKCE) authentication layer on top of the existing authentication flexibilities that Form.io already provides. Because of this, it is possible to authenticate with the UAG if you are using SAML, Form.io Authentication, OIDC, or even Custom Authentication processes through the Form.io platform.
48
+ 3. [**Pre-defined MCP Tools providing Dynamic Context**](#pre-defined-mcp-tools-providing-dynamic-context): The UAG utilizes the MCP to provide a number of pre-defined tools that provide AI Agents with dynamic context based on the Forms and Resources within the Form.io platform. These tools are described in more detail in a section below.
49
+ 4. **Custom Tools** (provided by a [Custom Module](#custom-modules)): This enables developers to introduce any new tools that can leverage the Form.io forms and submission data structures to provide additional capabilities introduced to the AI Agents. This would allow for custom agent interactions to achieve certain goals for specific use cases.
50
+ 5. **Custom Actions** (provided by a [Custom Module](#custom-modules)): Enables developers to create composable middleware that will be executed once all data provided by the AI agent has been sanitized and validated. This provides the ability to interface the data collected from AI Agents with any backend system or workflow.
51
+ 6. **Built-in Actions**: In addition to any custom actions that my be introduced via a Module, the UAG also supports a handlful of existing Actions that can be assigned to any form or resource. Actions such as the **Webhook** that will send a REST api call to any endpoint once any submission has been made by an AI Agent. This can be used to point to any serverless function to instigate backend workflows that are triggered from the interactions from AI Agents.
52
+ 7. **Pre-defined Forms and Resources** (provided by a [Custom Module](#custom-modules)): Allows for a developer to release a "module" that provides default data structures to the AI Agent. This would be useful if a developer wishes to create a module that achieves a specific goal where the data structures required for that goal need to follow a pre-determined set of fields provided by a form or resource. For example, if a developer wishes to release a CRM module for the UAG, then it would make sense that the module would also provide the default **Customer** and **Company** resources that commonly accompany most CRM implementations.
53
+ 8. **Custom Database** (provided by a [Custom Module](#custom-modules)): By default, the UAG performs the data collection and retrieval via a "Submission Proxy", whereas the data is submitted via REST API to either the Enterprise Server project or the Open Source server. This, however, could be modified to send the data directly to and from a Database. This can be achieved via the Database Interface provided from the UAG. For more information, see the **Module** documentation. This feature does require that the UAG is connected to the Enterprise Server.
54
+
55
+ Here is a diagram to understand how all of these layers are organized to make up the UAG.
56
+
57
+ ![](./examples/images/uag-overview.png)
58
+
59
+ For each of these layers, the following documentation applies...
60
+
61
+ ### Pre-defined MCP Tools providing Dynamic Context
62
+ The following tools provided by the UAG can be described as follows:
8
63
 
9
64
  | Tool Name | Tool Description |
10
65
  |-------------------|----------------------------------------------------------------------------------|
@@ -17,6 +72,56 @@ The UAG leverages the [**Model Context Protocol (MCP)**](https://modelcontextpro
17
72
  | find_submissions | A very powerful tool that provides the AI Agent the ability to understand how to query for data within the form. This enables the user to say something like "What is the email address of Joe Smith who is the CTO of Microsoft?" and it will format the query responsible for finding that record within the Form.io database.
18
73
  | submission_update | Provides the AI agent the ability to update an existing record with any additional information. This also has the awareness to "append", "prepend", and perform any other operations on the data in congruence with how the user has requested the data be updated.
19
74
 
75
+ ### Custom Modules
76
+ While the UAG can be used as a stand-alone system to enable the interaction between AI Agents and dynamic JSON forms, the true power of this platform will be realized when developers extend the capabilities of this platform to solve industry specific use cases through the use of Custom Modules and Tools. It is possible for a developer to create a **Module** that introduces a number of custom tools, actions, and pre-defined resources and forms to achieve interactions with industry specific technologies.
77
+
78
+ For example, by creating a UAG Module for Salesforce, you can produce a system that can dynamically provide Customer and Company resource data structures to any AI Agent, and then through the use of Custom Actions, you can integrate that structured data directly to Salesforce via their own API's as the following image illustrates.
79
+
80
+ <sub>This is just an example.</sub>
81
+ ![](./examples/images/uag-custom-module.png)
82
+
83
+ Custom modules are able to achieve this custom integration capabilities through the use of the following mechanisms:
84
+
85
+ 1. **Pre-defined Form and Resources**: Using the Form.io template system, a custom module can include a template export of a Form.io project that includes a number of Forms, Resources, and Actions (as well as any Roles and Permissions associated with them). By using the module, this will then automatically "register" these assets so that the AI Agent is immediately aware of those assets. For example, if your module includes a Customer resource with "First Name", "Last Name", and "Email" and you use that module. If you then simply ask an AI Agent, "I would like to add a new Customer" the agent will automatically be aware of the context for the customer data structures and know exactly what your intent is when making that request.
86
+ 2. **Custom Actions**: Actions can be thought of as composable middleware functions that allow for configurable Express.js middleware function to be attached to any Form and Resource, and then be executed once the data being submitted by an AI Agent has already passed through the proper data validations and sanitizations. These can be used to provide custom integrations to backend processes and workflows.
87
+ 3. **Custom Tools**: In addition to providing custom actions, a Module can also introduce custom MCP Tools to provide bespoke interaction capabilities between the UAG and the AI Agents for any industry specific purposes.
88
+ 4. **Custom Authentication**: A module can also modify the "authentication" landing page to assist with any custom requirements that are needed around Authentication.
89
+ 5. **Behavior Overrides**: A module also has the ability to "override" any default behavior from the UAG. This ensures that whomever is using a certain module can "tweak" certain tools and responses to achieve better results as it pertains their their industry specific needs.
90
+
91
+ It is our goal to enable developers to build and contribute their own Custom modules to the broader community through Github and NPM. For example, once these modules proliferate, it may be possible to install the salesforce module for UAG with the following.
92
+
93
+ <sub>This is just an example.</sub>
94
+ ```
95
+ npm install salesforce-uag
96
+ ```
97
+
98
+ and then use it as follows.
99
+
100
+ <sub>This is just an example.</sub>
101
+ ```js
102
+ import { UAGServer } from '@formio/uag';
103
+ import { SalesforceUAG } from 'salesforce-uag';
104
+ import Express from 'express';
105
+ try {
106
+ (async function () {
107
+ const server = new UAGServer();
108
+ server.use(SalesforceUAG);
109
+ const app = Express();
110
+ app.use(await server.router());
111
+ const port = process.env.PORT || 3200;
112
+ app.listen(port, () => {
113
+ console.log(`Form.io UAG server running on port ${port}`);
114
+ console.log(`Visit http://localhost:${port} to access the application`);
115
+ });
116
+ })();
117
+ } catch (error) {
118
+ console.error('Failed to start server:', error);
119
+ process.exit(1);
120
+ }
121
+ ```
122
+
123
+ To get started in building your own custom module, go to the help documentation at [Modules Readme](./module/Readme.md).
124
+
20
125
  ## Getting Started
21
126
  The quickest way to become familar with the UAG is to first walk through our Local Example, and run it on your local machine. This will allow you to see how the UAG leverages the power of Form.io to connect with an AI Agent to provide dynamic context. All of this can be ran locally for free without any subscriptions required!
22
127
 
@@ -105,7 +210,7 @@ docker run -d \
105
210
  formio/uag:rc
106
211
  ```
107
212
 
108
- ### Docker Compose (Recommended)
213
+ #### Docker Compose (Recommended)
109
214
  The recommended way to launching the UAG is through Docker Compose. This enables you to orchestrate several of the containers to run within a single instance to provide a more seamless and simple way of managing your deployments. Here is a simple example of how to run both the Form.io UAG + Form.io OSS server on the same instance.
110
215
 
111
216
  docker-compose.yml
@@ -189,13 +294,13 @@ This module can be configured in many ways. One of those ways is through the use
189
294
  | LOGIN_FORM | The public URL to the Login Form JSON endpoint. | https://mysite.com/project/user/login |
190
295
  | CORS | The cors domain, or the JSON configuration to configure the "cors" node.js module cross domain resource sharing. | *.* |
191
296
 
192
- ## Running on Public Domain
297
+ ### Running on Public Domain
193
298
  In order to run the UAG on a public domain, it is very important to provide the proper configurations so that any AI Agent can properly authenticate. There are 3 different "domain" environment variables that matter, and it is important to understand how to configure them depending on your use case:
194
299
 
195
- ### PROJECT
300
+ #### PROJECT
196
301
  The ```PROJECT``` environment variable is used to establish a connetion from the UAG server to the project endpoint (for Enterprise) or OSS base url. This does NOT need to be a public DNS entry, but rather a URL that connects the UAG container to the Server container. The following examples illustrate how this would be configured.
197
302
 
198
- #### Local connection to OSS Server
303
+ ##### Local connection to OSS Server
199
304
  If you are using a local connection, such as within a Docker Compose file, you can configure the ```PROJECT``` environment variable to point directly to the local url as follows.
200
305
 
201
306
  docker-compose.yml
@@ -222,7 +327,7 @@ services:
222
327
 
223
328
  In this example, we have Docker Compose launching the OSS Form.io container with the ADMIN_KEY set for this deployment, the UAG is connected using ```http://formio:3000``` which is the local Docker network name (provided using the "links" property in the docker compose file)
224
329
 
225
- #### Local Connection to Enterprise Server Project
330
+ ##### Local Connection to Enterprise Server Project
226
331
  If you are using the Enterprise Form.io server within a local environment to the UAG, then you will need to ensure that the UAG connects to an independent project using the PROJECT_KEY as follows.
227
332
 
228
333
  docker-compose.yml
@@ -246,7 +351,7 @@ services:
246
351
  PROJECT_KEY: CHANGEME
247
352
  ```
248
353
 
249
- #### Public DNS
354
+ ##### Public DNS
250
355
  If your project does not reside on the same network as the UAG, you can provide the domain name as the PROJECT as follows.
251
356
 
252
357
  docker-compose.yml: Connected to Enterprise (formio/formio-enterprise)
@@ -273,25 +378,20 @@ services:
273
378
  ADMIN_KEY: CHANGEME
274
379
  ```
275
380
 
276
- ### BASE_URL
381
+ #### BASE_URL
277
382
  The ```BASE_URL``` is used to communicate to the AI Agent the public domain that is hosting the UAG server. This value is provided within the ```.well-known``` definitions for the OIDC (PKCE) authentication. If this is not correct, then the AI Agent will not be able to authenticate into the UAG.
278
383
 
279
384
  **This needs to be the publically accessible domain that you are hosting your UAG.**
280
385
 
281
386
  For example, ```BASE_URL: https://forms.mysite.com```.
282
387
 
283
- ### LOGIN_FORM
388
+ #### LOGIN_FORM
284
389
  This is the publically accessible URL to the Login form of your Project or OSS deployment. This provides the URL that is loaded when the user navigates to ```{{ BASE_URL }}/auth/authorize```. If you navigate to this URL, and the page says that you cannot load the form, then this is because your LOGIN_FORM environment variable is not pointing to the correct form JSON endpoint of your project.
285
390
 
286
391
  For example:
287
392
  - Enterprise Example: ```LOGIN_FORM: https://forms.mysite.com/myproject/user/login```
288
393
  - OSS Example: ```LOGIN_FORM: https://forms.mysite.com/user/login```
289
394
 
290
- ## Modules
291
- One of the more powerful ways to "control" and extend the UAG is through the use of Modules. This allows you to provide custom Resources, Forms, as well as custom actions and configurations to ensure that the UAG behaves exactly as you wish for it to behave for your use case.
292
-
293
- Extensive documentation for the UAG Module system can be found in the [Modules Readme](./module/Readme.md).
294
-
295
395
  ## Using with Form.io Enterprise Server
296
396
  If you are using the UAG with the Form.io Enterprise Server, you unlock several benefits with regards to managing the Forms and Resources within the UAG. Some of the features that you gain with our Enterprise Server are as follows:
297
397
 
@@ -330,7 +430,7 @@ Now that the UAG is connected, you can then navigate to any Forms and Resources.
330
430
  ### Deploying changes to your UAG using Stage Versions
331
431
  Next, you will simply use the existing Staging and Deployment system from your Developer portal to "deploy" any changes to your UAG. This will allow you to treat the UAG just like you would treat any other stage within your Enterprise deployment. This will allow you to track and any forms and resource changes using the Tag system, and then deploy new versions as well as "roll-back" to any previous versions if a change is made that does to perform as you would expect within the AI Agent enviornment. This is a stark contrast to what Enterprises must deal with "Trained Agents" where it is much harder to "roll back" any training that an agent has gone through.
332
432
 
333
- ### Custom Actions
433
+ ### Custom Actions within Developer Portal
334
434
  In addition to managing Tags and Versions within the Developer portal, you can also use the Developer Portal to add Custom Actions to any forms and resources. Within the stage that is connected to the UAG, you can navigate to any Form or Resource, and then click on **Actions**. From there, any actions that show up in the Drop-down list of Actions that you can add to this form, you will see any Custom Actions that are part of your **Module** that you can also attach to your Forms and Resources. From here, you can add custom configurations and settings for each Action instance. It can also be versioned just like any other standard action using the tagging and versioning system.
335
435
 
336
436
 
package/lib/templates/getFormFields.md CHANGED
@@ -5,4 +5,7 @@ All values for these fields should be stored within the `data_path`="<%= parentD
5
5
 
6
6
  <%= rules %>
7
7
 
8
- Once the user provides provides some context, use the `get_field_info` tool<% if (parent) { %> with the `parent_path`="<%= parent.data_path %>"<% } %> to understand how to validate and process the data that was collected. Make sure to set the `field_paths` to only the paths of the fields that the user has provided values for.
8
+ **Next Steps**:
9
+ 1. If a user has already provided values for any of these fields, use the `get_field_info` tool with the `field_paths` set to the provided fields<% if (parent) { %> along with the `parent_path`="<%= parent.data_path %>"<% } %> to determine if the values conform to the correct format and validation criteria. If so, then use the `collect_field_data` tool to collect the information already received from the user.
10
+
11
+ 2. For the fields the user has not provided values, ask them if they would like to provide values for those fields. Then follow the instruction above to collect that information.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@formio/uag",
3
- "version": "1.4.1",
3
+ "version": "1.4.3",
4
4
  "description": "The Form.io Universal Agent Gateway (UAG).",
5
5
  "main": "lib/index.js",
6
6
  "types": "lib/index.d.ts",
@@ -25,7 +25,7 @@
25
25
  "dependencies": {
26
26
  "@formio/appserver": "^2.3.2",
27
27
  "@formio/core": "2.5.1-dev.291.6557e4e",
28
- "@modelcontextprotocol/sdk": "^1.21.0",
28
+ "@modelcontextprotocol/sdk": "^1.21.1",
29
29
  "cors": "^2.8.5",
30
30
  "debug": "^4.4.1",
31
31
  "dotenv": "^17.2.1",
@@ -44,12 +44,12 @@
44
44
  "@types/jsonwebtoken": "^9.0.10",
45
45
  "@types/lodash": "^4.17.20",
46
46
  "@types/mocha": "^10.0.10",
47
- "@types/node": "^24.10.0",
47
+ "@types/node": "^24.10.1",
48
48
  "@types/supertest": "^6.0.3",
49
- "chai": "^6.0.1",
50
- "mocha": "^11.7.4",
49
+ "chai": "^6.2.1",
50
+ "mocha": "^11.7.5",
51
51
  "supertest": "^7.1.4",
52
- "terser": "^5.43.1",
52
+ "terser": "^5.44.1",
53
53
  "ts-node": "^10.9.2",
54
54
  "tsx": "^4.20.4",
55
55
  "typescript": "^5.9.3"