@testomatio/mcp 1.0.3 → 1.0.5

package/README.md

@@ -112,13 +112,79 @@ Then add this to your Cursor MCP settings:
 #### Tests
 * `get_tests` – Get all tests (params: `plan`, `query`, `state`, `suite_id`, `tag`, `labels`) — api: GET `/tests`
 * `search_tests` – Search tests (params: `query`, `tql`, `labels`, `state`, `priority`, `filter`, `page`) — api: GET `/tests`
-* `create_test` – Create a new test (params: `suite_id`, `title`, `description`, `code`, `file`, `state`, `tags`, `jira_issues`, `assigned_to`, `labels_ids`) — api: POST `/tests`
-* `update_test` – Update an existing test (params: `test_id`, `suite_id`, `title`, `description`, `code`, `file`, `state`, `tags`, `jira_issues`, `assigned_to`, `labels_ids`) — api: PUT `/tests/{test_id}`
+* `create_test` – Create a new test (params: `suite_id`, `title`, `description`, `code`, `file`, `state`, `tags`, `jira_issues`, `assigned_to`, `labels_ids`, `fields`) — api: POST `/tests`
+* `update_test` – Update an existing test (params: `test_id`, `suite_id`, `title`, `description`, `code`, `file`, `state`, `tags`, `jira_issues`, `assigned_to`, `labels_ids`, `fields`) — api: PUT `/tests/{test_id}`
 
 #### Test Suites
 * `search_suites` – Search suites (params: `query`, `labels`, `state`, `priority`, `page`) — api: GET `/suites`
 * `get_root_suites` – List root-level suites (no params) — api: GET `/suites`
 * `get_suite` – Get one suite (params: `suite_id`) — api: GET `/suites/{suite_id}`
+* `create_suite` – Create a new suite (params: `title`, `description`, `parent_id`, `fields`) — api: POST `/suites`
+* `create_folder` – Create a new folder (params: `title`, `description`, `parent_id`, `fields`) — api: POST `/suites`
+
+#### Labels
+* `create_label` – Create a new label with optional custom field (params: `title`, `color`, `scope`, `visibility`, `field`) — api: POST `/labels`
+
+### Custom Fields and Labels
+
+The MCP server provides two distinct ways to assign values to tests, suites, and folders:
+
+#### 1. Using `labels_ids` with label:value syntax
+```javascript
+{
+  "labels_ids": ["priority:high", "severity:critical", "type:regression"]
+}
+```
+- Direct label assignment with values using `label:value` format
+- Good for simple label assignments
+- Works with existing Testomatio labels
+
+#### 2. Using `fields` parameter (structured custom fields)
+```javascript
+{
+  "fields": {
+    "priority": "high",
+    "severity": "critical",
+    "risk_score": "8.5",
+    "team": "backend"
+  }
+}
+```
+- Structured way to set custom fields
+- Cleaner syntax for AI assistants
+- Supports any custom field defined in your Testomatio project
+- Maps to Testomatio's custom-fields API
+
+**Available for:**
+- `create_test` and `update_test` - Test custom fields
+- `create_suite` - Suite custom fields
+- `create_folder` - Folder custom fields
+
+#### Example Usage
+```javascript
+// Create a test with custom fields
+{
+  "tool": "create_test",
+  "arguments": {
+    "suite_id": "123",
+    "title": "Login Test",
+    "fields": {
+      "priority": "high",
+      "severity": "critical",
+      "team": "backend"
+    }
+  }
+}
+
+// Update a test with label:value syntax
+{
+  "tool": "update_test",
+  "arguments": {
+    "test_id": "456",
+    "labels_ids": ["priority:high", "severity:critical"]
+  }
+}
+```
 
 #### Test Runs
 * `get_runs` – List all runs (no params) — api: GET `/runs`
@@ -141,6 +207,12 @@ Once configured, you can ask your AI assistant questions like:
 - "Get all test plans for this project"
 - "Create a new test called 'Login validation' in suite suite-123"
 - "Update test test-456 to change its description and add @regression tag"
+- "Create a test with custom fields: priority='high', severity='critical', team='backend'"
+- "Update test test-789 to set custom fields for risk score and assigned team"
+- "Create a new suite called 'Authentication Tests' with description 'All login and signup related tests'"
+- "Create a suite with custom fields for team ownership and priority level"
+- "Create a folder called 'API Tests' to organize API-related test suites with custom fields"
+- "Create a label called 'Severity' with color '#ffe9ad' and predefined values like 'Blocker', 'Critical', 'Major', 'Minor', 'Normal', 'Trivial'"
 
 ## Query Patterns
 
@@ -160,6 +232,31 @@ These queries allow creating and updating tests:
 - **"Create a new test called 'Login validation' in suite suite-123"** → `create_test` tool with `title: "Login validation"`, `suite_id: "suite-123"`
 - **"Update test test-456 to change its description"** → `update_test` tool with `test_id: "test-456"`, `description: "new description"`
 - **"Create an automated test with @smoke tag"** → `create_test` tool with `state: "automated"`, `tags: ["smoke"]`
+- **"Create a test with custom fields: priority='high', severity='critical'"** → `create_test` tool with `title: "Test Title"`, `suite_id: "suite-123"`, `fields: { "priority": "high", "severity": "critical" }`
+- **"Update test test-789 to set custom fields for risk score and team"** → `update_test` tool with `test_id: "test-789"`, `fields: { "risk_score": "8.5", "team": "backend" }`
+
+### Suite and Folder Management Queries
+
+These queries help organize your test structure:
+
+- **"Create a new suite called 'Authentication Tests'"** → `create_suite` tool with `title: "Authentication Tests"`
+- **"Create a suite for login tests with description"** → `create_suite` tool with `title: "Login Tests"`, `description: "All login related test cases"`
+- **"Create a suite with custom fields for team ownership and priority level"** → `create_suite` tool with `title: "Backend Tests"`, `fields: { "team": "backend", "priority": "high" }`
+- **"Create a folder called 'API Tests' under parent suite-123"** → `create_folder` tool with `title: "API Tests"`, `parent_id: "suite-123"`
+- **"Create a folder with custom fields for team and project"** → `create_folder` tool with `title: "Integration Tests"`, `fields: { "team": "qa", "project": "mobile-app" }`
+- **"Create a test suite for payment features"** → `create_suite` tool with `title: "Payment Features", description: "Tests covering payment processing"`
+- **"Create a folder to organize integration tests"** → `create_folder` tool with `title: "Integration Tests"`
+
+**Note**: Suites can only contain other suites, while folders can contain both suites and folders (but no tests).
+
+### Label Creation Queries
+
+These queries help create custom labels for better test categorization:
+
+- **"Create a label called 'Severity' with red color"** → `create_label` tool with `title: "Severity"`, `color: "#ff6b6b"`
+- **"Create a severity label with predefined values"** → `create_label` tool with `title: "Severity"`, `color: "#ffe9ad"`, `field: { "type": "list", "short": true, "value": "Blocker\nCritical\nMajor\nNormal\nMinor\nTrivial" }`
+- **"Create a simple label for test types"** → `create_label` tool with `title: "Test Type"`, `scope: ["tests", "suites"]`
+- **"Create a label visible in test lists"** → `create_label` tool with `title: "Category"`, `visibility: ["list"]`
 
 ### Specific Item Queries
 
@@ -174,9 +271,8 @@ These queries target specific entities by ID:
 These queries use advanced filtering capabilities:
 
 - **"List all automated tests with the @smoke tag"** → `search_tests` tool with `query: "@smoke"`, `state: "automated"`
-- **"Find tests with priority high"** → `search_tests` tool with `priority: "high"`
 - **"Search for tests containing 'login'"** → `search_tests` tool with `query: "login"`
-- **"List tests tagged @critical or labelled 'ux' with high priority"** → `search_tests` tool with `tql: "tag == 'critical' or label == 'ux' and priority == 'high'"`
+- **"List tests tagged @critical or labelled 'ux' with critical severity"** → `search_tests` tool with `tql: "tag == 'critical' or label == 'ux' and severity == 'critical'"`
 - **"Find tests linked to JIRA-123"** → `search_tests` tool with `tql: jira == 'BDCP-2'`
 
 ### Advanced Query Syntax
@@ -187,7 +283,7 @@ The `search_tests` tool supports TQL for complex filtering:
 
 ```
 "tag == 'smoke' and state == 'manual'"
-"priority == 'high' or label == 'ux'"
+"severity == 'critical' or label == 'ux'"
 ```
 
 #### Tag-Based Searches
@@ -247,6 +343,74 @@ For detailed information about the underlying Testomat.io API, refer to the [Tes
 
 Contributions are welcome! Please feel free to submit a Pull Request.
 
+### Development Setup
+
+```bash
+# Clone the repository
+git clone https://github.com/testomatio/mcp.git
+cd mcp
+
+# Install dependencies
+npm install
+
+# Run unit tests
+npm test
+
+# Run integration tests (requires environment variables)
+npm run test:integration
+
+# Run all tests
+npm run test:all
+```
+
+### Testing
+
+The project includes comprehensive test coverage:
+
+- **Unit Tests**: Fast tests with mocked dependencies
+- **Integration Tests**: Real API tests against Testomat.io
+
+#### Running Tests Locally
+
+```bash
+# Unit tests only
+npm run test:unit
+
+# Integration tests (requires .env file)
+npm run test:integration
+
+# With coverage
+npm run test:coverage
+npm run test:coverage:integration
+```
+
+#### Environment Setup for Integration Tests
+
+Create a `.env` file:
+
+```bash
+TESTOMATIO_API_TOKEN=testomat_your_token_here
+TESTOMATIO_PROJECT_ID=your_project_id
+TESTOMATIO_BASE_URL=https://app.testomat.io  # optional
+```
+
+### CI/CD
+
+This project uses GitHub Actions for continuous integration:
+
+- ✅ **Unit Tests**: Run on every push/PR across Node.js 18, 20, 22
+- ✅ **Integration Tests**: Run daily and on main branch merges
+- ✅ **Coverage Reports**: Automatic upload to Codecov
+- ✅ **Security**: Secrets management for API credentials
+
+
+### Code Quality
+
+- Follow existing code style patterns
+- Add tests for new functionality
+- Update documentation when needed
+- Ensure all tests pass before submitting PRs
+
 ## License
 
 This project is licensed under the MIT License - see the LICENSE file for details.

package/index.js

@@ -41,7 +41,8 @@ class TestomatioMCPServer {
     });
 
     if (!response.ok) {
-      throw new Error(`Authentication failed: HTTP ${response.status}: ${response.statusText}`);
+      const errorText = await response.text();
+      throw new Error(`Authentication failed: HTTP ${response.status}: ${response.statusText}. Response: ${errorText}`);
     }
 
     const data = await response.json();
@@ -317,7 +318,14 @@ class TestomatioMCPServer {
                 labels_ids: {
                   type: 'array',
                   items: { type: 'string' },
-                  description: 'Slugs of labels to assign to the test',
+                  description: 'Slugs of labels to assign to the test. Supports label:value format (e.g., ["priority:high", "severity:critical"])',
+                },
+                fields: {
+                  type: 'object',
+                  description: 'Set custom fields for the test. Object with field names as keys and values as properties (e.g., {"priority": "high", "severity": "critical"})',
+                  additionalProperties: {
+                    type: 'string'
+                  },
                 },
               },
               required: ['suite_id', 'title'],
@@ -358,6 +366,11 @@ class TestomatioMCPServer {
                   enum: ['manual', 'automated'],
                   description: 'State of the test',
                 },
+                priority: {
+                  type: 'string',
+                  enum: ['low', 'normal', 'high', 'critical'],
+                  description: 'Priority level of the test',
+                },
                 tags: {
                   type: 'array',
                   items: { type: 'string' },
@@ -375,12 +388,129 @@ class TestomatioMCPServer {
                 labels_ids: {
                   type: 'array',
                   items: { type: 'string' },
-                  description: 'Slugs of labels to assign to the test',
+                  description: 'Slugs of labels to assign to the test. Supports label:value format (e.g., ["priority:high", "severity:critical"])',
+                },
+                fields: {
+                  type: 'object',
+                  description: 'Set custom fields for the test. Object with field names as keys and values as properties (e.g., {"priority": "high", "severity": "critical"})',
+                  additionalProperties: {
+                    type: 'string'
+                  },
                 },
               },
               required: ['test_id'],
             },
           },
+          {
+            name: 'create_suite',
+            description: 'Create a new suite. Suites can only contain other suites (no tests or folders)',
+            inputSchema: {
+              type: 'object',
+              properties: {
+                title: {
+                  type: 'string',
+                  description: 'Suite title',
+                },
+                description: {
+                  type: 'string',
+                  description: 'Suite description',
+                },
+                parent_id: {
+                  type: 'string',
+                  description: 'Parent suite ID to create this suite under',
+                },
+                fields: {
+                  type: 'object',
+                  description: 'Set custom fields for the suite. Object with field names as keys and values as properties (e.g., {"priority": "high", "team": "backend"})',
+                  additionalProperties: {
+                    type: 'string'
+                  },
+                },
+              },
+              required: ['title'],
+            },
+          },
+          {
+            name: 'create_folder',
+            description: 'Create a new folder. Folders can contain suites and folders (but no tests)',
+            inputSchema: {
+              type: 'object',
+              properties: {
+                title: {
+                  type: 'string',
+                  description: 'Folder title',
+                },
+                description: {
+                  type: 'string',
+                  description: 'Folder description',
+                },
+                parent_id: {
+                  type: 'string',
+                  description: 'Parent folder or suite ID to create this folder under',
+                },
+                fields: {
+                  type: 'object',
+                  description: 'Set custom fields for the folder. Object with field names as keys and values as properties (e.g., {"priority": "high", "team": "backend"})',
+                  additionalProperties: {
+                    type: 'string'
+                  },
+                },
+              },
+              required: ['title'],
+            },
+          },
+          {
+            name: 'create_label',
+            description: 'Create a new label with optional custom field configuration. Labels can be used to tag and categorize tests and suites',
+            inputSchema: {
+              type: 'object',
+              properties: {
+                title: {
+                  type: 'string',
+                  description: 'Label title (e.g., "Severity", "Priority", "Type")',
+                },
+                color: {
+                  type: 'string',
+                  description: 'Label color in hex format (e.g., "#ffe9ad")',
+                },
+                scope: {
+                  type: 'array',
+                  items: {
+                    type: 'string',
+                    enum: ['tests', 'suites']
+                  },
+                  description: 'Where this label can be used (e.g., ["tests", "suites"])',
+                },
+                visibility: {
+                  type: 'array',
+                  items: {
+                    type: 'string'
+                  },
+                  description: 'Where the label is visible (e.g., ["list"])',
+                },
+                field: {
+                  type: 'object',
+                  description: 'Custom field configuration for labels with predefined values',
+                  properties: {
+                    type: {
+                      type: 'string',
+                      description: 'Field type (e.g., "list", "string", "number")',
+                    },
+                    short: {
+                      type: 'boolean',
+                      description: 'Whether to display short version',
+                    },
+                    value: {
+                      type: 'string',
+                      description: 'Predefined values for the field (newline separated for list type)',
+                    },
+                  },
+                  required: ['type'],
+                },
+              },
+              required: ['title'],
+            },
+          },
         ],
       };
     });
@@ -414,6 +544,12 @@ class TestomatioMCPServer {
             return await this.createTest(args);
           case 'update_test':
             return await this.updateTest(args);
+          case 'create_suite':
+            return await this.createSuite(args);
+          case 'create_folder':
+            return await this.createFolder(args);
+          case 'create_label':
+            return await this.createLabel(args);
           default:
             throw new Error(`Unknown tool: ${name}`);
         }
@@ -914,14 +1050,19 @@ class TestomatioMCPServer {
   }
 
   async createTest(args) {
-    const { suite_id, labels_ids, ...attributes } = args;
+    const { suite_id, labels_ids, fields, ...attributes } = args;
+
+    // Handle fields parameter for custom fields
     const requestData = {
       data: {
         type: 'tests',
-        attributes: Object.fromEntries(Object.entries(attributes).map(([k, v]) => [k.replace(/_/g, '-'), v]))
+        attributes: {
+          ...Object.fromEntries(Object.entries(attributes).map(([k, v]) => [k.replace(/_/g, '-'), v])),
+          ...(fields && { 'custom-fields': fields })
+        }
       }
     };
-    
+
     if (suite_id) {
       requestData.data.relationships = {
         suite: {
@@ -932,7 +1073,7 @@ class TestomatioMCPServer {
         }
       };
     }
-    
+
     if (labels_ids) requestData.labels_ids = labels_ids;
 
     const data = await this.makePostRequest('/tests', requestData);
@@ -952,14 +1093,19 @@ class TestomatioMCPServer {
   }
 
   async updateTest(args) {
-    const { test_id, suite_id, labels_ids, ...attributes } = args;
+    const { test_id, suite_id, labels_ids, fields, ...attributes } = args;
+
+    // Handle fields parameter for custom fields
     const requestData = {
       data: {
         type: 'tests',
-        attributes: Object.fromEntries(Object.entries(attributes).map(([k, v]) => [k.replace(/_/g, '-'), v]))
+        attributes: {
+          ...Object.fromEntries(Object.entries(attributes).map(([k, v]) => [k.replace(/_/g, '-'), v])),
+          ...(fields && { 'custom-fields': fields })
+        }
       }
     };
-    
+
     if (suite_id) {
       requestData.data.relationships = {
         suite: {
@@ -970,7 +1116,7 @@ class TestomatioMCPServer {
         }
       };
     }
-    
+
     if (labels_ids) requestData.labels_ids = labels_ids;
 
     const data = await this.makePutRequest(`/tests/${test_id}`, requestData);
@@ -989,6 +1135,124 @@ class TestomatioMCPServer {
     };
   }
 
+  async createSuite(args) {
+    const { parent_id, fields, ...attributes } = args;
+    const requestData = {
+      data: {
+        type: 'suites',
+        attributes: {
+          ...Object.fromEntries(Object.entries(attributes).map(([k, v]) => [k.replace(/_/g, '-'), v])),
+          'file-type': 'file',
+          ...(fields && { 'custom-fields': fields })
+        }
+      }
+    };
+
+    if (parent_id) {
+      requestData.data.relationships = {
+        parent: {
+          data: {
+            type: 'suites',
+            id: parent_id
+          }
+        }
+      };
+    }
+
+    const data = await this.makePostRequest('/suites', requestData);
+    const formattedSuite = this.formatModel(data.data, 'suite', [
+      'title', 'description', 'test-count', 'is-root', 'file-type'
+    ]);
+
+    return {
+      content: [
+        {
+          type: 'text',
+          text: `Successfully created suite:\n\n${formattedSuite}`,
+        },
+      ],
+    };
+  }
+
+  async createFolder(args) {
+    const { parent_id, fields, ...attributes } = args;
+    const requestData = {
+      data: {
+        type: 'suites',
+        attributes: {
+          ...Object.fromEntries(Object.entries(attributes).map(([k, v]) => [k.replace(/_/g, '-'), v])),
+          'file-type': 'folder',
+          ...(fields && { 'custom-fields': fields })
+        }
+      }
+    };
+
+    if (parent_id) {
+      requestData.data.relationships = {
+        parent: {
+          data: {
+            type: 'suites',
+            id: parent_id
+          }
+        }
+      };
+    }
+
+    const data = await this.makePostRequest('/suites', requestData);
+    const formattedSuite = this.formatModel(data.data, 'suite', [
+      'title', 'description', 'test-count', 'is-root', 'file-type'
+    ]);
+
+    return {
+      content: [
+        {
+          type: 'text',
+          text: `Successfully created folder:\n\n${formattedSuite}`,
+        },
+      ],
+    };
+  }
+
+  async createLabel(args) {
+    const { title, color, scope, visibility, field, ...otherAttributes } = args;
+
+    const requestData = {
+      data: {
+        type: 'labels',
+        attributes: {
+          ...Object.fromEntries(Object.entries(otherAttributes).map(([k, v]) => [k.replace(/_/g, '-'), v])),
+          title,
+          color,
+          scope,
+          visibility
+        }
+      }
+    };
+
+    // Add field configuration if provided
+    if (field) {
+      requestData.data.attributes.field = {
+        type: field.type,
+        ...(field.short !== undefined && { short: field.short }),
+        ...(field.value && { value: field.value })
+      };
+    }
+
+    const data = await this.makePostRequest('/labels', requestData);
+    const formattedLabel = this.formatModel(data.data, 'label', [
+      'title', 'color', 'scope', 'visibility', 'field'
+    ]);
+
+    return {
+      content: [
+        {
+          type: 'text',
+          text: `Successfully created label:\n\n${formattedLabel}`,
+        },
+      ],
+    };
+  }
+
   async run() {
     // Test authentication on startup
     try {
@@ -1005,6 +1269,9 @@ class TestomatioMCPServer {
   }
 }
 
+// Export the class for testing
+export { TestomatioMCPServer };
+
 // Parse command line arguments using commander
 function parseArgs() {
   program
@@ -1019,7 +1286,7 @@ function parseArgs() {
   const options = program.opts();
 
   const token = options.token || process.env.TESTOMATIO_API_TOKEN;
-  const projectId = options.project;
+  const projectId = options.project || process.env.TESTOMATIO_PROJECT_ID;
   const baseUrl = options.baseUrl || process.env.TESTOMATIO_BASE_URL || 'https://app.testomat.io';
 
   if (!token) {
@@ -1028,7 +1295,7 @@ function parseArgs() {
   }
 
   if (!projectId) {
-    console.error('Error: Project ID is required. Use --project <project_id>');
+    console.error('Error: Project ID is required. Use --project <project_id> or set TESTOMATIO_PROJECT_ID environment variable');
     process.exit(1);
   }
 
@@ -1047,4 +1314,7 @@ async function main() {
   }
 }
 
-main().catch(console.error);
+// Only run main() if this file is executed directly (not imported)
+if (import.meta.url === `file://${process.argv[1]}`) {
+  main().catch(console.error);
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@testomatio/mcp",
-  "version": "1.0.3",
+  "version": "1.0.5",
   "description": "Model Context Protocol server for Testomatio API",
   "main": "index.js",
   "bin": {
@@ -10,7 +10,14 @@
   "scripts": {
     "start": "node index.js",
     "dev": "node index.js",
-    "test": "echo \"Error: no test specified\" && exit 1"
+    "test": "node --experimental-vm-modules node_modules/.bin/jest",
+    "test:unit": "node --experimental-vm-modules node_modules/.bin/jest",
+    "test:integration": "node --experimental-vm-modules node_modules/.bin/jest --config jest.integration.config.js",
+    "test:watch": "node --experimental-vm-modules node_modules/.bin/jest --watch",
+    "test:integration:watch": "node --experimental-vm-modules node_modules/.bin/jest --config jest.integration.config.js --watch",
+    "test:coverage": "node --experimental-vm-modules node_modules/.bin/jest --coverage",
+    "test:coverage:integration": "node --experimental-vm-modules node_modules/.bin/jest --config jest.integration.config.js --coverage",
+    "test:all": "node --experimental-vm-modules node_modules/.bin/jest --config jest.integration.config.js"
   },
   "keywords": [
     "testomatio",
@@ -24,7 +31,8 @@
   "license": "MIT",
   "dependencies": {
     "@modelcontextprotocol/sdk": "^0.4.0",
-    "commander": "^12.0.0"
+    "commander": "^12.0.0",
+    "dotenv": "^17.2.3"
   },
   "files": [
     "index.js",
@@ -32,5 +40,8 @@
   ],
   "engines": {
     "node": ">=18.0.0"
+  },
+  "devDependencies": {
+    "jest": "^30.2.0"
   }
 }