@formio/uag 1.4.3 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +175 -163
  2. package/package.json +4 -4
package/README.md CHANGED
@@ -1,22 +1,25 @@
1
1
  # The Form.io "Universal Agent Gateway" (UAG)
2
- The Universal Agent Gateway (UAG) is an exciting new technology that brings JSON powered (model driven) [Form.io](https://form.io) forms to AI Agents. Leveraging the power of the [**Model Context Protocol (MCP)**](https://modelcontextprotocol.io/docs/getting-started/intro), this library provides AI Agents with **dynamic context** of how to interface with any backend system using Form.io JSON Forms as the common language between these agents and Enterprise systems.
2
+ The Universal Agent Gateway (UAG) introduces the power of [Form.io](https://form.io) forms to AI agents.
3
+
4
+ The Form.io UAG uses the [**Model Context Protocol (MCP)**](https://modelcontextprotocol.io/docs/getting-started/intro) to enable Form.io interaction through an AI agent workflow. By providing AI agents with the **dynamic context** of how to use Form.io JSON forms, the UAG allows a user to interact with any aspect of their enterprise system served by the Form.io Platform directly through their AI agent.
3
5
 
4
6
  ---
5
7
 
6
8
  ## Table of Contents
7
9
 
8
10
  - [The Form.io "Universal Agent Gateway" (UAG)](#the-formio-universal-agent-gateway-uag)
9
- - [Introduction](#introduction)
10
- - [How it works](#how-it-works)
11
+ - [Introduction](#introduction)
12
+ - [The Model Context Protocol (MCP)](#the-model-context-protocol-mcp)
13
+ - [Dynamic Context](#dynamic-context)
14
+ - [The Role of the UAG](#the-role-of-the-uag)
11
15
  - [Technical Overview](#technical-overview)
12
16
  - [MCP Tools](#pre-defined-mcp-tools-providing-dynamic-context)
13
17
  - [Custom Modules](#custom-modules)
14
18
  - [Module Documentation](./module)
15
- - [Getting Started](#getting-started)
16
- - [Example Local Deployment](./examples/local)
17
- - [Runtime Environments](#runtime-environments)
18
- - [Node.js (Express):](#nodejs-express)
19
- - [Docker](#docker)
19
+ - [Deploying UAG](#deploying-uag)
20
+ - [Runtime Environments](#runtime-environments)
21
+ - [Docker](#docker)
22
+ - [Node.js (Express)](#nodejs-express)
20
23
  - [Environment Variables](#environment-variables)
21
24
  - [Running on Public Domain](#running-on-public-domain)
22
25
  - [Using with Form.io Enterprise Server](#using-with-formio-enterprise-server)
@@ -24,71 +27,81 @@ The Universal Agent Gateway (UAG) is an exciting new technology that brings JSON
24
27
  ---
25
28
 
26
29
  ## Introduction
27
- The core purpose of the UAG is to provide AI Agents the ability to understand dynamic data structures defined from JSON form definitions. It accomplishes this goal by combining two innovative Open Source technologies.
30
+ The core purpose of the UAG is to equip AI agents with the ability to fluidly apply a user's natural language instructions to the Form.io platform's capabilities as defined by selected forms and fields.
31
+
32
+ ### The Model Context Protocol (MCP)
33
+ The Model Context Protocol (MCP) is an open-source standard for connecting AI applications to external systems.
28
34
 
29
- - [**Form.io JSON Forms**](https://form.io): The Form.io platform enables deterministic data collection through the use of JSON form definitions created with a drag-and-drop form builder interface. Once created, these forms serve as more than just a front-end interface for humans to "fill out", but rather serve as a standard data model contract between applications and any backend processes that depend on that data. It ensures that every system is speaking the same language and provides human-in-the-loop experiences (through rendered forms) to collect, audit, and report on the data exchanges between many complex data driven systems. Here is a visual graphic showing how the Form JSON serves as the "centerpiece" contract to ensure data integrity throughout your Enterprise.
35
+ The MCP essentially puts an AI agent's hands on the controls of an external system. In the case of Form.io, the MCP describes a discrete set of tools the AI agent can use to interact with forms, fields, and submissions. While some tools are delivered off-the-shelf, this functionality can be extended, modified, or refined through custom built modules.
30
36
 
31
- ![](./examples/images/form-interfaces.png)
37
+ ### Dynamic Context
38
+ What does it mean to say MCP provides a dynamic context for how to use these tools?
32
39
 
33
- - [**Model Context Protocol (MCP)**](https://modelcontextprotocol.io/docs/getting-started/intro): The Model Context Protocol (MCP) is a standard for connecting AI Applications to external systems. It achieves this by providing AI Agents with "context" so that they can understand how to interact with the external systems. When combined with the Form.io's dynamic JSON forms, the "context" that is provided to the AI Agents is also dynamic in nature making it so that AI Agent interactions can easily be controlled via the Form JSON models that are defined by Form.io
40
+ The power of AI agents is their ability to parse natural language and infer a user's intent rather than take rote input like a command line. The dynamic context delivered by MCP gives the AI agent guidance on how to correlate the user's prompt to the tools and data available. When the AI agent receives a prompt, it uses this dynamic context to determine what tool it should use, what elements of the prompt are inputs to that tool, and what additional input might be necessary.
34
41
 
35
- ## How it works
42
+ ### The Role of the UAG
43
+ The UAG is the package that contains everything sitting between the Form.io Platform and the AI agent.
44
+ It contains the stock MCP server, the authorization infrastructure, and any custom tools or modules that extend the functionality.
36
45
 
37
- The UAG provides **dynamic context** to AI Agents though the introduction of new MCP "tools" that describe the dynamic form structures to AI Agents so that they understand how to properly structure the data being processed by those agents. This cabability allows for Developers to treat AI Agents as they would any other Human who is submitting a form, and ensure that the AI Agents stay "on the rails" without requiring extensive domain specific agent training methods to accomplish the same goal. But more imporantly, the UAG offers Enterprises the ability to create trustworthy data interfaces through the use of dynamic and composable JSON data model definitions created and managed through Form.io. This not only enforces deterministic data interactions from AI Agents, but also provides an auditable human-in-the-loop experience through the provided Form.io Open Source and Enterprise portal applications.
46
+ By equipping an AI agent with Form.io capabilities, the UAG brings the AI agent much deeper into the broader application ecosystem.
47
+ Form.io's drag-and-drop Form Builder simultaneously defines the look of the form and the structure of the data. This makes it easy to use data collected through Form.io forms elsewhere in an application or as an input to other enterprise systems.
48
+ ![](./examples/images/form-interfaces.png)
49
+
50
+ When the UAG is able to map the natural language inputs that an AI agent receives to the data model defined by a form, it means that data can be quickly supplied to the enterprise tools that depend on the Form.io platform without additional interpretation or transformation.
51
+ This allows the UAG and Form.io to serve as a reliable middleware between an AI agent and countless other systems.
38
52
 
39
- Here is a visual graphic of the layers provided by the UAG to achieve a trusted and deterministic interface between AI Agents to external systems through the UAG + Form.io Server.
53
+ Here is a visual graphic of the layers provided by the UAG to achieve a trusted and deterministic interface between AI Agents to external systems through the UAG + Form.io Server.
40
54
 
41
- ![](./examples/images/agents_to_formio.png)
55
+ ![](./examples/images/agents_to_formio.png)
42
56
 
43
57
  ## Technical Overview
44
- The UAG can be thought of as consisting of many "layers" of functionality, broken up between the actions that fetch dynamic context and the ones that submit deterministic data structures. The following layers are leveraged within the UAG.
45
-
46
- 1. **MCP Interface**: The entry point of the UAG is the MCP interface that sits between the AI Agent and the tools that are executed on the server.
47
- 2. **Authorization**: As part of the MCP specification, every interaction betwen the AI Agent and the backend service must pass through an Authorization process leveraging OIDC (PKCE) authentication. The UAG takes this process one step further, however, by adding an OIDC (PKCE) authentication layer on top of the existing authentication flexibilities that Form.io already provides. Because of this, it is possible to authenticate with the UAG if you are using SAML, Form.io Authentication, OIDC, or even Custom Authentication processes through the Form.io platform.
48
- 3. [**Pre-defined MCP Tools providing Dynamic Context**](#pre-defined-mcp-tools-providing-dynamic-context): The UAG utilizes the MCP to provide a number of pre-defined tools that provide AI Agents with dynamic context based on the Forms and Resources within the Form.io platform. These tools are described in more detail in a section below.
49
- 4. **Custom Tools** (provided by a [Custom Module](#custom-modules)): This enables developers to introduce any new tools that can leverage the Form.io forms and submission data structures to provide additional capabilities introduced to the AI Agents. This would allow for custom agent interactions to achieve certain goals for specific use cases.
50
- 5. **Custom Actions** (provided by a [Custom Module](#custom-modules)): Enables developers to create composable middleware that will be executed once all data provided by the AI agent has been sanitized and validated. This provides the ability to interface the data collected from AI Agents with any backend system or workflow.
51
- 6. **Built-in Actions**: In addition to any custom actions that my be introduced via a Module, the UAG also supports a handlful of existing Actions that can be assigned to any form or resource. Actions such as the **Webhook** that will send a REST api call to any endpoint once any submission has been made by an AI Agent. This can be used to point to any serverless function to instigate backend workflows that are triggered from the interactions from AI Agents.
52
- 7. **Pre-defined Forms and Resources** (provided by a [Custom Module](#custom-modules)): Allows for a developer to release a "module" that provides default data structures to the AI Agent. This would be useful if a developer wishes to create a module that achieves a specific goal where the data structures required for that goal need to follow a pre-determined set of fields provided by a form or resource. For example, if a developer wishes to release a CRM module for the UAG, then it would make sense that the module would also provide the default **Customer** and **Company** resources that commonly accompany most CRM implementations.
53
- 8. **Custom Database** (provided by a [Custom Module](#custom-modules)): By default, the UAG performs the data collection and retrieval via a "Submission Proxy", whereas the data is submitted via REST API to either the Enterprise Server project or the Open Source server. This, however, could be modified to send the data directly to and from a Database. This can be achieved via the Database Interface provided from the UAG. For more information, see the **Module** documentation. This feature does require that the UAG is connected to the Enterprise Server.
58
+ The UAG can be thought of as consisting of many layers of functionality. Each layer is broadly either serving to fetch dynamic context, or submit deterministic data structures. The following layers are leveraged within the UAG:
59
+
60
+ 1. **MCP Interface**: The entry point of the UAG is the MCP interface that sits between the AI agent and the tools that are executed on the server.
61
+ 2. **Authorization**: As part of the MCP specification, every interaction between the AI agent and the backend service must pass through an Authorization process leveraging OIDC (PKCE) authentication. The UAG takes this process one step further, however, by adding an OIDC (PKCE) authentication layer on top of the existing authentication flexibilities that Form.io already provides. Because of this, it is possible to authenticate with the UAG if you are using SAML, Form.io Authentication, OIDC, or even Custom Authentication processes through the Form.io platform.
62
+ 3. [**Pre-defined MCP Tools providing Dynamic Context**](#pre-defined-mcp-tools-providing-dynamic-context): The UAG utilizes the MCP to provide a number of pre-defined tools that provide AI agents with dynamic context based on the Forms and Resources marked as compatible within the Form.io platform. These tools are described in more detail in a section below.
63
+ 4. **Custom Tools** (provided by [Custom Modules](#custom-modules)): This enables developers to introduce any new tools that can leverage Form.io forms and submission data structures to provide additional capabilities introduced to the AI agents. This would allow for custom agent interactions to achieve certain goals for specific use cases.
64
+ 5. **Custom Actions** (provided by [Custom Modules](#custom-modules)): Enables developers to create composable middleware that will be executed once all data provided by the AI agent has been sanitized and validated. This provides the ability to interface the data collected from AI agents with any backend system or workflow.
65
+ 6. **Built-in Actions**: In addition to any custom actions introduced via a Module, the UAG supports a handful of existing Actions that can be assigned to any form or resource. Actions such as the **Webhook** that will send a REST API call to any endpoint once any submission has been made by an AI agent. This can be used to point to any serverless function to instigate backend workflows that are triggered from the interactions from AI agents.
66
+ 7. **Pre-defined Forms and Resources** (provided by [Custom Modules](#custom-modules)): Allows a developer to release a module that provides default data structures to the AI agent. This would be useful if a developer wishes to create a module that achieves a specific goal, where the data structures required for that goal need to follow a pre-determined set of fields provided by a form or resource. For example, if a developer wishes to release a CRM module for the UAG, then it would make sense that the module would also provide the default **Customer** and **Company** resources that commonly accompany most CRM implementations.
67
+ 8. **Custom Database** (provided by [Custom Module](#custom-modules)): By default, the UAG performs the data collection and retrieval via a "Submission Proxy", whereas the data is submitted via REST API to either the Enterprise Server project or the Open Source server. This, however, could be modified to send the data directly to and from a Database. This can be achieved via the Database Interface provided from the UAG. For more information, see the **Module** documentation. This feature does require that the UAG is connected to the Enterprise Server.
54
68
 
55
69
  Here is a diagram to understand how all of these layers are organized to make up the UAG.
56
70
 
57
71
  ![](./examples/images/uag-overview.png)
58
72
 
59
- For each of these layers, the following documentation applies...
60
-
61
73
  ### Pre-defined MCP Tools providing Dynamic Context
62
74
  The following tools provided by the UAG can be described as follows:
63
75
 
64
- | Tool Name | Tool Description |
65
- |-------------------|----------------------------------------------------------------------------------|
66
- | get_forms | Provides the AI Agent an understanding of what Forms are available. It will only return forms that have been tagged **uag**. |
67
- | get_form_fields | Once a form has been identified by the AI Agent that the user wishes to engage with (using natural language), this tool provides the AI Agent a high level overview of all the fields needed (along with the field data path) to submit that form. |
68
- | get_field_info | Once the fields have been determined using the `get_form_fields` tool, this tool is used to get specific information about the requested fields, such as validation, conditionals, input formats, etc. This tool provides the AI Agent the ability to undersand how to "format" and structure the data that is sent to the MCP server. |
69
- | collect_field_data | This tool provides the AI Agent a mechanism to dynamically collect the required information from the user. It is capable of collecting many fields at once, and also provides the AI agent the ability to understand what required fields are missing and what is needed to complete its goal. This tool supports complex, yet structured, data collecting through multiple "nested" data constructs such as Nested Forms and Table structured data (datagrid, editgrid). |
70
- | confirm_form_submission | This tool provides the AI Agent the ability to confirm all the information it has collected before a submission has been made. It allows the AI Agent the ability to get a confirmation from the user before it submits the form. |
71
- | submit_completed_form | This tool provides the AI Agent the ability to submit all of the data collected from the user to complete the form submission.
72
- | find_submissions | A very powerful tool that provides the AI Agent the ability to understand how to query for data within the form. This enables the user to say something like "What is the email address of Joe Smith who is the CTO of Microsoft?" and it will format the query responsible for finding that record within the Form.io database.
73
- | submission_update | Provides the AI agent the ability to update an existing record with any additional information. This also has the awareness to "append", "prepend", and perform any other operations on the data in congruence with how the user has requested the data be updated.
76
+ | Tool Name | Tool Description |
77
+ |-----------------|----------------------|
78
+ | get_forms | Provides the AI agent with a list of available forms. <br/>It will only return forms that have been tagged `uag`. |
79
+ | get_form_fields | When the AI agent infers the user intends to use a specific form, this tool provides the agent with a high level overview of all the fields needed (along with the field data path) to submit that form. |
80
+ | get_field_info | Once the fields have been determined using `get_form_fields`, this tool provides specific information about the requested fields, such as validation, conditionals, input formats, etc. <br/>This tool instructs the AI agent on how to format and structure the data that is sent to the MCP server. |
81
+ | collect_field_data | Provides the AI agent with a mechanism to dynamically collect the required information from the user. It can parse the user's input into multiple fields at once, and will identify additional inputs from the user if needed. <br/>This tool supports complex and structured data collection and is compatible with nested or multi-value fields like nested forms, data grids, etc. |
82
+ | confirm_form_submission | When the AI agent is ready to submit a form, this tool allows it to confirm all the information it has collected before a submission has been made to the form. |
83
+ | submit_completed_form | Provides the AI agent with the ability to submit all of the data collected from the user to create the form submission. |
84
+ | find_submissions | Enables the agent to parse a user's natural language request into a query for a submission, or a specific field of a particular submission. |
85
+ | submission_update | Provides the AI agent with the ability to update an existing submission, either by supplying unfilled fields or updating existing ones if allowed. Provides the AI agent with the context of the existing field values, allowing inline changes or edits. |
74
86
 
75
87
  ### Custom Modules
76
- While the UAG can be used as a stand-alone system to enable the interaction between AI Agents and dynamic JSON forms, the true power of this platform will be realized when developers extend the capabilities of this platform to solve industry specific use cases through the use of Custom Modules and Tools. It is possible for a developer to create a **Module** that introduces a number of custom tools, actions, and pre-defined resources and forms to achieve interactions with industry specific technologies.
88
+ While the UAG can be used as a stand-alone system to enable the interaction between AI agents and dynamic JSON forms, the true power of this platform will be realized when developers extend the capabilities of this platform to solve industry specific use cases through the use of Custom Modules and Tools. It is possible for a developer to create a **Module** that introduces a number of custom tools, actions, and pre-defined resources and forms to achieve interactions with industry specific technologies.
77
89
 
78
- For example, by creating a UAG Module for Salesforce, you can produce a system that can dynamically provide Customer and Company resource data structures to any AI Agent, and then through the use of Custom Actions, you can integrate that structured data directly to Salesforce via their own API's as the following image illustrates.
90
+ For example, by creating a UAG Module for Salesforce, you can produce a system that can dynamically provide Customer and Company resource data structures to any AI agent, and then through the use of Custom Actions, you can integrate that structured data directly to Salesforce via their own API's as the following image illustrates.
79
91
 
80
92
  <sub>This is just an example.</sub>
81
93
  ![](./examples/images/uag-custom-module.png)
82
94
 
83
95
  Custom modules are able to achieve this custom integration capabilities through the use of the following mechanisms:
84
96
 
85
- 1. **Pre-defined Form and Resources**: Using the Form.io template system, a custom module can include a template export of a Form.io project that includes a number of Forms, Resources, and Actions (as well as any Roles and Permissions associated with them). By using the module, this will then automatically "register" these assets so that the AI Agent is immediately aware of those assets. For example, if your module includes a Customer resource with "First Name", "Last Name", and "Email" and you use that module. If you then simply ask an AI Agent, "I would like to add a new Customer" the agent will automatically be aware of the context for the customer data structures and know exactly what your intent is when making that request.
86
- 2. **Custom Actions**: Actions can be thought of as composable middleware functions that allow for configurable Express.js middleware function to be attached to any Form and Resource, and then be executed once the data being submitted by an AI Agent has already passed through the proper data validations and sanitizations. These can be used to provide custom integrations to backend processes and workflows.
87
- 3. **Custom Tools**: In addition to providing custom actions, a Module can also introduce custom MCP Tools to provide bespoke interaction capabilities between the UAG and the AI Agents for any industry specific purposes.
97
+ 1. **Pre-defined Form and Resources**: Using the Form.io template system, a custom module can include a template export of a Form.io project that includes a number of Forms, Resources, and Actions (as well as any Roles and Permissions associated with them). By using the module, this will then automatically "register" these assets so that the AI agent is immediately aware of those assets.
98
+ For example, imagine a module includes a Customer resource with "First Name", "Last Name", and "Email" fields. If you then simply ask an AI agent, "I would like to add a new Customer" the agent will automatically be aware of the context for the customer data structures and know exactly what your intent is when making that request.
99
+ 2. **Custom Actions**: Actions can be thought of as composable middleware functions that allow for configurable Express.js middleware functions to be attached to any Form and Resource, and then to be executed once the data being submitted by an AI agent has passed through the proper data validations and sanitization. These can be used to provide custom integrations to backend processes and workflows.
100
+ 3. **Custom Tools**: In addition to providing custom actions, a Module can also introduce custom MCP Tools to provide bespoke interaction capabilities between the UAG and the AI agents for any industry specific purposes.
88
101
  4. **Custom Authentication**: A module can also modify the "authentication" landing page to assist with any custom requirements that are needed around Authentication.
89
102
  5. **Behavior Overrides**: A module also has the ability to "override" any default behavior from the UAG. This ensures that whomever is using a certain module can "tweak" certain tools and responses to achieve better results as it pertains their their industry specific needs.
90
103
 
91
- It is our goal to enable developers to build and contribute their own Custom modules to the broader community through Github and NPM. For example, once these modules proliferate, it may be possible to install the salesforce module for UAG with the following.
104
+ Form.io aims to enable developers to build and contribute their own custom modules to the broader community through GitHub and NPM. For example, once these modules proliferate, it may be possible to install the salesforce module for UAG with the following.
92
105
 
93
106
  <sub>This is just an example.</sub>
94
107
  ```
@@ -122,93 +135,33 @@ try {
122
135
 
123
136
  To get started in building your own custom module, go to the help documentation at [Modules Readme](./module/Readme.md).
124
137
 
125
- ## Getting Started
126
- The quickest way to become familar with the UAG is to first walk through our Local Example, and run it on your local machine. This will allow you to see how the UAG leverages the power of Form.io to connect with an AI Agent to provide dynamic context. All of this can be ran locally for free without any subscriptions required!
127
-
128
- **[Go to Local Example &raquo;](examples/local)**
138
+ ## Deploying UAG
139
+ The following sections describe the process of creating a running instance of UAG, connecting it to the Form.io Platform, and configuring an AI agent to use it.
129
140
 
130
- ## Runtime Environments
131
- There are currently two run-time environments that work with the UAG. **Node.js (Express)** and **Docker**.
141
+ This documentation focuses on using UAG with Form.io Open Source. For documentation on using UAG with Form.io Enterprise, refer to [Help.Form.io/UAG](https://help.form.io/uag).
132
142
 
133
- ### Node.js (Express):
134
- With the Node.js environment, you can import the UAG within a locally running Node.js and Express.js envioronment. This works by first importing the UAG module and "use"ing it within an Express.js application. First, you will install the uag inside of your Node.js Express application like the following.
143
+ ### Local Example
144
+ The quickest way to become familiar with the UAG is to first walk through our Local Example, which will spin up a local instance of Form.io Open Source and UAG. This can then be connected to an AI agent, which will have the ability to interact with the included example forms. The local example can be run entirely free, with no trial or license required.
135
145
 
136
- ```npm install --save @formio/uag```
146
+ **[Go to Local Example &raquo;](examples/local)**
137
147
 
138
- or
148
+ ### Runtime Environments
149
+ There are currently two run-time environments that work with the UAG:
150
+ -[**Docker**](#docker)
151
+ -[**Node.js (Express)**](#nodejs-express)
139
152
 
140
- ```yarn add @formio/uag```
141
-
142
- You can then mount the UAG within your Express application like the following example shows.
143
-
144
- ```js
145
- import 'dotenv/config';
146
- import Express from 'express';
147
- import { UAGServer } from '@formio/uag';
148
- try {
149
- (async function () {
150
- const server = new UAGServer();
151
- const app = Express();
152
- app.use(await server.router());
153
- const port = process.env.PORT || 3200;
154
- app.listen(port, () => {
155
- console.log(`Form.io UAG server running on port ${port}`);
156
- console.log(`Visit http://localhost:${port} to access the application`);
157
- });
158
- })();
159
- } catch (error) {
160
- console.error('Failed to start server:', error);
161
- process.exit(1);
162
- }
163
- ```
164
-
165
- There is also a way to extend the functionality of the UAG through the use of modules, which is documented in the [Modules Readme](./module/Readme.md)
166
153
 
167
154
  ### Docker
168
- In addition to running the UAG in Node.js, you can also run the UAG within the Docker environment. This enables a wide range of deployment options into common hosting environments such as AWS and Azure as well as allow for the use in common orchestration runtimes such as Kubernetes and Docker Compose. The container that you will use for running the UAG is as follows.
155
+ Running the UAG in Docker enables a wide range of deployment options into common hosting environments such as AWS and Azure. Additionally, it allows for the use of common orchestration runtimes such as Kubernetes and Docker Compose. The container that you will run the UAG is as follows:
169
156
 
170
157
  ```
171
158
  formio/uag
172
159
  ```
173
160
 
174
- This container can be ran as a standalone container using the common **docker run** command, or inside of a **docker-compose.yml** (for Docker Compose). To see an example of using UAG with Docker Compose, we recommend taking a look at the [Local Example](./examples/local).
175
-
176
- #### Docker Run
177
- Here are some examples of running the UAG using the ```docker run``` command.
178
-
179
- **UAG pointed to an Open Source server**
180
- ```
181
- docker run -d \
182
- -e "PROJECT=https://forms.mysite.com" \
183
- -e "ADMIN_KEY=CHANGEME" \
184
- -e "JWT_SECRET=CHANGEME" \
185
- -e "BASE_URL=https://forms.mysite.com" \
186
- -e "LOGIN_FORM=https://forms.mysite.com/user/login" \
187
- -e "PORT=3200" \
188
- --restart unless-stopped \
189
- --network formio \
190
- --name formio-uag \
191
- -p 3200:3200 \
192
- formio/uag:rc
193
- ```
161
+ This container can be run in two ways:
162
+ - [**Docker-Compose.yml**](#docker-compose-recommended) - The recommended method, deploys with of a Docker Compose multicontainer.
163
+ - [**Docker Run**](#docker-run) - a standalone container using the common `docker run` command.
194
164
 
195
- **UAG pointed to an Form.io Enterprise Server**
196
- ```
197
- docker run -d \
198
- -e "PROJECT=https://forms.mysite.com/myproject" \
199
- -e "PROJECT_KEY=CHANGEME" \
200
- -e "UAG_LICENSE=YOUR-LICENSE" \
201
- -e "JWT_SECRET=CHANGEME" \
202
- -e "PORTAL_SECRET=CHANGEME" \
203
- -e "BASE_URL=https://forms.mysite.com" \
204
- -e "LOGIN_FORM=https://forms.mysite.com/myproject/user/login" \
205
- -e "PORT=3200" \
206
- --restart unless-stopped \
207
- --network formio \
208
- --name formio-uag \
209
- -p 3200:3200 \
210
- formio/uag:rc
211
- ```
212
165
 
213
166
  #### Docker Compose (Recommended)
214
167
  The recommended way to launching the UAG is through Docker Compose. This enables you to orchestrate several of the containers to run within a single instance to provide a more seamless and simple way of managing your deployments. Here is a simple example of how to run both the Form.io UAG + Form.io OSS server on the same instance.
@@ -242,7 +195,7 @@ services:
242
195
  ports:
243
196
  - "3000:3000"
244
197
  formio-uag:
245
- image: formio/uag:rc
198
+ image: formio/uag
246
199
  restart: always
247
200
  links:
248
201
  - formio
@@ -267,11 +220,48 @@ This can be run by typing the following...
267
220
  docker compose up -d
268
221
  ```
269
222
 
223
+ #### Docker Run
224
+ Here are some examples of running the UAG using the ```docker run``` command.
225
+
226
+ **UAG pointed to an Open Source server**
227
+ ```
228
+ docker run -d \
229
+ -e "PROJECT=https://forms.mysite.com" \
230
+ -e "ADMIN_KEY=CHANGEME" \
231
+ -e "JWT_SECRET=CHANGEME" \
232
+ -e "BASE_URL=https://forms.mysite.com" \
233
+ -e "LOGIN_FORM=https://forms.mysite.com/user/login" \
234
+ -e "PORT=3200" \
235
+ --restart unless-stopped \
236
+ --network formio \
237
+ --name formio-uag \
238
+ -p 3200:3200 \
239
+ formio/uag
240
+ ```
241
+
242
+ **UAG pointed to an Form.io Enterprise Server**
243
+ ```
244
+ docker run -d \
245
+ -e "PROJECT=https://forms.mysite.com/myproject" \
246
+ -e "PROJECT_KEY=CHANGEME" \
247
+ -e "UAG_LICENSE=YOUR-LICENSE" \
248
+ -e "JWT_SECRET=CHANGEME" \
249
+ -e "PORTAL_SECRET=CHANGEME" \
250
+ -e "BASE_URL=https://forms.mysite.com" \
251
+ -e "LOGIN_FORM=https://forms.mysite.com/myproject/user/login" \
252
+ -e "PORT=3200" \
253
+ --restart unless-stopped \
254
+ --network formio \
255
+ --name formio-uag \
256
+ -p 3200:3200 \
257
+ formio/uag
258
+ ```
259
+
270
260
  Once it is running, you can then navigate to the following to access both the OSS Deployment + UAG Server
271
261
  - http://localhost:3000: The Form.io OSS Server
272
262
  - http://localhost:3200: The UAG Server
273
263
 
274
- In both the Node.js runtime environemnt as well as Docker, the way to control the UAG is thorugh the use of **Environment Variables** and **Modules**.
264
+ In both the Node.js runtime environmnt as well as Docker, the way to control the UAG is through the use of **Environment Variables** and **Modules**.
275
265
 
276
266
  ### Environment Variables
277
267
  This module can be configured in many ways. One of those ways is through the use of Environment Variables, which are documented as follows.
@@ -279,26 +269,38 @@ This module can be configured in many ways. One of those ways is through the use
279
269
  | Variable | Description | Example |
280
270
  |----------|-------------|---------|
281
271
  | PROJECT | The API Endpoint to either an Enterprise project endpoint, or the OSS server url. | http://localhost:3000 |
282
- | PROJECT_KEY | (Enterprise Only) Either a Project API Key (for Form.io Enterprise) or the ADMIN_KEY for Community Edition. | CHANGEME |
283
- | ADMIN_KEY | (OSS Only) Allows you to provide the ADMIN_KEY to install and connect to the OSS Server. | CHANGEME |
272
+ | PROJECT_KEY | (Enterprise Only) Either a Project API Key (for Form.io Enterprise) or the ADMIN_KEY for Open Source. | CHANGEME |
273
+ | ADMIN_KEY | (Open Source Only) Allows you to provide the ADMIN_KEY to install and connect to the OSS Server. | CHANGEME |
284
274
  | UAG_LICENSE | The license to run the UAG against a Form.io Enterprise Deployment. | |
275
+ | PROJECT_TTL | The project cache TTL in seconds. This controls the amount of time to check for any "changes" that have been made to the project to refresh any forms and resources within the UAG. | 900 |
285
276
  | PORT | The port you wish to run the server on. | 3200 |
286
277
  | DEBUG | Variable used to perform debug logs of server activity | formio.* |
287
278
  | PORTAL_SECRET | Enterprise Only: Allows you to connect to the UAG from the Form.io Enterprise Portal. | CHANGEME |
288
279
  | JWT_SECRET | A secret used to generate and validate JWT tokens generated through the authentication process of the UAG. This does not need to match the JWT_SECRET of the Enterprise Server that it is connected to. | CHANGEME |
289
280
  | PORTAL_SECRET | (Enterprise Only) Used to connect the UAG server with the Enterprise Portal | CHANGEME |
290
281
  | JWT_EXPIRE_TIME | The expiration for the jwt secret. | 3600 |
291
- | MONGO | (Enterprise Only) Allows you to connect the UAG directly to a mongo database vs. having to redirect the submissions to the Form.io Submission APIs. | |
282
+ | MONGO | (Enterprise Only) Allows you to connect the UAG directly to a mongo database, rather than having to redirect the submissions to the Form.io Submission APIs. | |
292
283
  | MONGO_CONFIG | JSON configuration for the Node.js Mongo Driver. | |
293
284
  | BASE_URL | The public URL that the UAG is hosted on. This allows for proper OIDC authentication and allows for the authentication callbacks to point to the correct url. | https://ai.onform.io |
294
285
  | LOGIN_FORM | The public URL to the Login Form JSON endpoint. | https://mysite.com/project/user/login |
295
286
  | CORS | The cors domain, or the JSON configuration to configure the "cors" node.js module cross domain resource sharing. | *.* |
296
287
 
288
+ ### Project Cache and TTL
289
+ By default, the UAG uses a TTL to "fetch" any of the project assets and register them and cache them into the UAG. What this means is that if you make any changes to your underlying project forms and resources (either directly or through a deployment), you need to wait a maximum of the TTL setting (in seconds) to see any updates that have been made. For example, if the TTL is set to 900 (or 15 minutes), and you make a change to any forms and resources, you must wait at most 15 minutes to see any of these changes take effect into the UAG. It should be noted that when a "refresh" occurs, an API call to the "export" method is called on the underlying project endpoint. This is a processor intensive API call, so it is not recommended to set the TTL to any value less than 60. A value of 0 is interpreted as "No TTL", which means the only way to refresh the project forms and resourcs, a UAG reboot is necessary.
290
+
291
+ Here is a table explaining how this parameter can be used.
292
+
293
+ | Environment Variable | Value | Result |
294
+ |----------|-------------|---------|
295
+ | PROJECT_TTL | 0 | No TTL (refresh) will occur. This can be used to "disable" any refresh and ensure the ONLY way to refetch updated forms and resources, you must reboot the UAG server. |
296
+ | PROJECT_TTL | 60 | Check for changes every minute |
297
+ | PROJECT_TTL | 3600 | Check for changes every hour |
298
+
297
299
  ### Running on Public Domain
298
- In order to run the UAG on a public domain, it is very important to provide the proper configurations so that any AI Agent can properly authenticate. There are 3 different "domain" environment variables that matter, and it is important to understand how to configure them depending on your use case:
300
+ In order to run the UAG on a public domain, it is very important to provide the proper configurations so that any AI agent can properly authenticate. There are 3 different "domain" environment variables that matter, and it is important to understand how to configure them depending on your use case:
299
301
 
300
302
  #### PROJECT
301
- The ```PROJECT``` environment variable is used to establish a connetion from the UAG server to the project endpoint (for Enterprise) or OSS base url. This does NOT need to be a public DNS entry, but rather a URL that connects the UAG container to the Server container. The following examples illustrate how this would be configured.
303
+ The ```PROJECT``` environment variable is used to establish a connection from the UAG server to the project endpoint (for Enterprise) or OSS base url. This does NOT need to be a public DNS entry, but rather a URL that connects the UAG container to the Server container. The following examples illustrate how this would be configured.
302
304
 
303
305
  ##### Local connection to OSS Server
304
306
  If you are using a local connection, such as within a Docker Compose file, you can configure the ```PROJECT``` environment variable to point directly to the local url as follows.
@@ -314,7 +316,7 @@ services:
314
316
  PORT: 3000
315
317
  ADMIN_KEY: CHANGEME
316
318
  formio-uag:
317
- image: formio/uag:rc
319
+ image: formio/uag
318
320
  restart: always
319
321
  links:
320
322
  - formio
@@ -325,7 +327,7 @@ services:
325
327
  ADMIN_KEY: CHANGEME
326
328
  ```
327
329
 
328
- In this example, we have Docker Compose launching the OSS Form.io container with the ADMIN_KEY set for this deployment, the UAG is connected using ```http://formio:3000``` which is the local Docker network name (provided using the "links" property in the docker compose file)
330
+ In this example, we have Docker Compose launching the OSS Form.io container with the ADMIN_KEY set for this deployment, the UAG is connected using ```http://formio:3000``` which is the local Docker network name (provided using the "links" property in the docker compose file).
329
331
 
330
332
  ##### Local Connection to Enterprise Server Project
331
333
  If you are using the Enterprise Form.io server within a local environment to the UAG, then you will need to ensure that the UAG connects to an independent project using the PROJECT_KEY as follows.
@@ -340,7 +342,7 @@ services:
340
342
  environment:
341
343
  PORT: 3000
342
344
  formio-uag:
343
- image: formio/uag:rc
345
+ image: formio/uag
344
346
  restart: always
345
347
  links:
346
348
  - formio-enterprise
@@ -359,7 +361,7 @@ docker-compose.yml: Connected to Enterprise (formio/formio-enterprise)
359
361
  version: "3.8"
360
362
  services:
361
363
  formio-uag:
362
- image: formio/uag:rc
364
+ image: formio/uag
363
365
  restart: always
364
366
  environment:
365
367
  PROJECT: https://forms.mydomain.com/myproject
@@ -371,7 +373,7 @@ docker-compose.yml: Connected to Open Source (formio/formio)
371
373
  version: "3.8"
372
374
  services:
373
375
  formio-uag:
374
- image: formio/uag:rc
376
+ image: formio/uag
375
377
  restart: always
376
378
  environment:
377
379
  PROJECT: https://forms.mydomain.com
@@ -379,58 +381,68 @@ services:
379
381
  ```
380
382
 
381
383
  #### BASE_URL
382
- The ```BASE_URL``` is used to communicate to the AI Agent the public domain that is hosting the UAG server. This value is provided within the ```.well-known``` definitions for the OIDC (PKCE) authentication. If this is not correct, then the AI Agent will not be able to authenticate into the UAG.
384
+ The ```BASE_URL``` is used to communicate to the AI agent the public domain that is hosting the UAG server. This value is provided within the ```.well-known``` definitions for the OIDC (PKCE) authentication. If this is not correct, then the AI agent will not be able to authenticate into the UAG.
383
385
 
384
- **This needs to be the publically accessible domain that you are hosting your UAG.**
386
+ **This needs to be the publicly accessible domain that you are hosting your UAG.**
385
387
 
386
388
  For example, ```BASE_URL: https://forms.mysite.com```.
387
389
 
388
390
  #### LOGIN_FORM
389
- This is the publically accessible URL to the Login form of your Project or OSS deployment. This provides the URL that is loaded when the user navigates to ```{{ BASE_URL }}/auth/authorize```. If you navigate to this URL, and the page says that you cannot load the form, then this is because your LOGIN_FORM environment variable is not pointing to the correct form JSON endpoint of your project.
391
+ This is the publicly accessible URL to the Login form of your Project or OSS deployment. This provides the URL that is loaded when the user navigates to ```{{ BASE_URL }}/auth/authorize```. If you navigate to this URL, and the page says that you cannot load the form, then this is because your LOGIN_FORM environment variable is not pointing to the correct form JSON endpoint of your project.
390
392
 
391
393
  For example:
392
394
  - Enterprise Example: ```LOGIN_FORM: https://forms.mysite.com/myproject/user/login```
393
395
  - OSS Example: ```LOGIN_FORM: https://forms.mysite.com/user/login```
394
396
 
395
- ## Using with Form.io Enterprise Server
396
- If you are using the UAG with the Form.io Enterprise Server, you unlock several benefits with regards to managing the Forms and Resources within the UAG. Some of the features that you gain with our Enterprise Server are as follows:
397
-
398
- - **Form Revisions** - Our form revision system provides a way to keep track of any changes made to any forms, and associate those changes with the submission data that is submitted against those revisions. This feature enables you to ensure that if any form schemas change, that the data submitted for that form correlates with the revision of the form in which it was submitted. In addition to this powerful feature, you can also leverage Form Revisions as a method to "roll-back" any mistakes or regressions caused by form changes. Instead of having to re-train or un-train an AI Agent with any schema, you simply revert to previous revisions and the AI Agent will adjust accordingly.
399
- - **Submission Revisions** - Submission revisions provides a way to keep track of any data changes that have been made to a submission and provides information on who changed the data. This combined with the power of the UAG provides any system the ability to audit any changes in data made by both AI Agents as well as humans in the workflow processes.
400
- - **Stage Versions and Deployments** - Stage versions provides a way to create a "tag" version accross all forms and resources within a project. This ensures that you can stamp a point in time where your AI Agent (which may consume many different forms and resources) interfaces with your whole project in a deterministic way. It also provides a much more elegant "roll-back" mechansism where the entire project of forms and resources can be versioned and deployed independently.
401
- - **Custom Actions** - Through the use of the Developer Portal and remote connections, you can use the Actions UI to manage and configure any custom actions that are configured within your UAG. Once a custom action is registered, via the **Modules** system, you can add that action to a form and configure it using the Form.io Developer Portal.
397
+ ### Node.js (Express):
398
+ With the Node.js environment, you can import the UAG within a locally running Node.js and Express.js environment. This works by first importing the UAG module and "use"ing it within an Express.js application. First, you will install the UAG inside of your Node.js Express application like the following.
402
399
 
403
- ### Running the UAG against the Enterprise Server
404
- In order to run the UAG against an Enterprise Server, you need to provide a few Environment variables that are different from the OSS runtime of this module. The following environment variables are required to run against an Enterprise Project.
400
+ ```npm install --save @formio/uag```
405
401
 
406
- | Variable | Description | Example |
407
- |----------|-------------|---------|
408
- | PROJECT | For the Enterprise Server, this points to the Project Endpoint you wish to bind the UAG to. | https://mydeployment.com/myproject |
409
- | PROJECT_KEY | An API Key for that project, provided within the Project Settings | CHANGEME |
410
- | UAG_LICENSE | This is the licnese provided to you from the Form.io License team. Contact support@form.io to acquire a "temporary" or full license. | |
411
- | PORTAL_SECRET | This enables you to "connect" your Form.io Developer Portal to the UAG so that you can view any custom actions as well as perform deployments to the UAG. | |
402
+ or
412
403
 
413
- Once you have these environment variables in place, you should be able to run the UAG pointed to your Enterprise Project. You can now connect to this UAG from the Developer Portal as follows.
404
+ ```yarn add @formio/uag```
414
405
 
415
- ### Connecting your Developer Portal to the UAG.
416
- Once you have the UAG running in your own environment with a PORTAL_SECRET, you will now create a new Stage within your Developer portal. We can call this UAG.
406
+ You can then mount the UAG within your Express application, as seen in the following example:
417
407
 
418
- <div align="center">
419
- <img src="./examples/images/uag-create-stage.png" alt="Create UAG Stage" width="600">
420
- </div>
408
+ ```js
409
+ import 'dotenv/config';
410
+ import Express from 'express';
411
+ import { UAGServer } from '@formio/uag';
412
+ try {
413
+ (async function () {
414
+ const server = new UAGServer();
415
+ const app = Express();
416
+ app.use(await server.router());
417
+ const port = process.env.PORT || 3200;
418
+ app.listen(port, () => {
419
+ console.log(`Form.io UAG server running on port ${port}`);
420
+ console.log(`Visit http://localhost:${port} to access the application`);
421
+ });
422
+ })();
423
+ } catch (error) {
424
+ console.error('Failed to start server:', error);
425
+ process.exit(1);
426
+ }
427
+ ```
421
428
 
422
- Next, you will click on **Staging** and then connect to your UAG server by providing the PORTAL_SECRET as follows.
429
+ There is also a way to extend the functionality of the UAG through the use of modules, which is documented in the [Modules Readme](./module/Readme.md)
423
430
 
424
- <div align="center">
425
- <img src="./examples/images/connect-uag.png" alt="Connect to UAG" width="600">
426
- </div>
431
+ ## Using with Form.io Enterprise Server
432
+ When using the UAG with the Form.io Enterprise Server, you unlock several benefits with regards to managing the Forms and Resources within the UAG. Some of the features that you gain with our Enterprise Server include:
427
433
 
428
- Now that the UAG is connected, you can then navigate to any Forms and Resources. These are the forms and resources hosted through the UAG.
434
+ - **Form Revisions** - Our form revision system provides a way to keep track of any changes made to any forms, and associate those changes with the submission data that is submitted against those revisions. This feature enables you to ensure that if any form schemas change, that the data submitted for that form correlates with the revision of the form in which it was submitted. In addition to this powerful feature, you can also leverage Form Revisions as a method to "roll-back" any mistakes or regressions caused by form changes. Instead of having to re-train or un-train an AI agent with any schema, you simply revert to previous revisions and the AI agent will adjust accordingly.
435
+ - **Submission Revisions** - Submission revisions provides a way to keep track of any data changes that have been made to a submission and provides information on who changed the data. This combined with the power of the UAG provides any system the ability to audit any changes in data made by both AI agents as well as humans in the workflow processes.
436
+ - **Stage Versions and Deployments** - Stage versions provides a way to create a "tag" version across all forms and resources within a project. This ensures that you can stamp a point in time where your AI agent (which may consume many different forms and resources) interfaces with your whole project in a deterministic way. It also provides a much more elegant "roll-back" mechanism where the entire project of forms and resources can be versioned and deployed independently.
429
437
 
430
- ### Deploying changes to your UAG using Stage Versions
431
- Next, you will simply use the existing Staging and Deployment system from your Developer portal to "deploy" any changes to your UAG. This will allow you to treat the UAG just like you would treat any other stage within your Enterprise deployment. This will allow you to track and any forms and resource changes using the Tag system, and then deploy new versions as well as "roll-back" to any previous versions if a change is made that does to perform as you would expect within the AI Agent enviornment. This is a stark contrast to what Enterprises must deal with "Trained Agents" where it is much harder to "roll back" any training that an agent has gone through.
438
+ ### Running the UAG against the Enterprise Server
439
+ In order to run the UAG against an Enterprise Server, you need to provide a few Environment variables that are different from the OSS runtime of this module. The following environment variables are required to run against an Enterprise Project.
432
440
 
433
- ### Custom Actions within Developer Portal
434
- In addition to managing Tags and Versions within the Developer portal, you can also use the Developer Portal to add Custom Actions to any forms and resources. Within the stage that is connected to the UAG, you can navigate to any Form or Resource, and then click on **Actions**. From there, any actions that show up in the Drop-down list of Actions that you can add to this form, you will see any Custom Actions that are part of your **Module** that you can also attach to your Forms and Resources. From here, you can add custom configurations and settings for each Action instance. It can also be versioned just like any other standard action using the tagging and versioning system.
441
+ | Variable | Description | Example |
442
+ |----------|-------------|---------|
443
+ | PROJECT | For the Enterprise Server, this points to the Project Endpoint you wish to bind the UAG to. | https://mydeployment.com/myproject |
444
+ | PROJECT_KEY | An API Key for that project, provided within the Project Settings | CHANGEME |
445
+ | UAG_LICENSE | This is the license provided to you from the Form.io License team. Contact support@form.io to acquire a "temporary" or full license. | |
435
446
 
447
+ Once you have these environment variables in place, you should be able to run the UAG pointed to your Enterprise Project.
436
448
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@formio/uag",
3
- "version": "1.4.3",
3
+ "version": "1.6.0",
4
4
  "description": "The Form.io Universal Agent Gateway (UAG).",
5
5
  "main": "lib/index.js",
6
6
  "types": "lib/index.d.ts",
@@ -15,7 +15,7 @@
15
15
  "build:ts": "tsc",
16
16
  "build:copy-templates": "mkdir -p lib/templates && cp src/templates/*.md lib/templates/",
17
17
  "build": "npm run clean && npm run build:ts && npm run build:copy-templates",
18
- "build:docker": "npm run build && docker build -t formio/uag:rc --platform=linux/amd64 .",
18
+ "build:docker": "npm run build && docker build -t formio/uag --platform=linux/amd64 .",
19
19
  "test": "TS_NODE_PROJECT=tsconfig.test.json mocha --no-node-snapshot",
20
20
  "dev": "tsx --no-node-snapshot index.js",
21
21
  "start": "node --no-node-snapshot index.js"
@@ -23,9 +23,9 @@
23
23
  "author": "",
24
24
  "license": "MIT",
25
25
  "dependencies": {
26
- "@formio/appserver": "^2.3.2",
26
+ "@formio/appserver": "^2.5.0",
27
27
  "@formio/core": "2.5.1-dev.291.6557e4e",
28
- "@modelcontextprotocol/sdk": "^1.21.1",
28
+ "@modelcontextprotocol/sdk": "^1.22.0",
29
29
  "cors": "^2.8.5",
30
30
  "debug": "^4.4.1",
31
31
  "dotenv": "^17.2.1",