@delorenj/mcp-server-trello 0.2.1 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +362 -28
- package/build/evals/evals.d.ts +5 -0
- package/build/evals/evals.js +50 -0
- package/build/evals/evals.js.map +1 -0
- package/build/index.js +1 -337
- package/build/index.js.map +1 -1
- package/build/rate-limiter.d.ts +14 -0
- package/build/rate-limiter.js +15 -1
- package/build/rate-limiter.js.map +1 -1
- package/build/trello-client.d.ts +57 -11
- package/build/trello-client.js +170 -19
- package/build/trello-client.js.map +1 -1
- package/build/types.d.ts +20 -1
- package/build/validators.d.ts +21 -0
- package/build/validators.js +38 -1
- package/build/validators.js.map +1 -1
- package/package.json +7 -3
package/README.md
CHANGED
|
@@ -1,14 +1,45 @@
|
|
|
1
1
|
# MCP Server Trello
|
|
2
|
-
[](https://smithery.ai/server/@modelcontextprotocol/mcp-server-trello)
|
|
3
2
|
|
|
4
|
-
|
|
3
|
+
[](https://mseep.ai/app/27359682-7632-4ba7-981d-7dfecadf1c4b)
|
|
5
4
|
|
|
6
5
|
<a href="https://glama.ai/mcp/servers/klqkamy7wt"><img width="380" height="200" src="https://glama.ai/mcp/servers/klqkamy7wt/badge" alt="Server Trello MCP server" /></a>
|
|
7
6
|
|
|
7
|
+
A Model Context Protocol (MCP) server that provides tools for interacting with Trello boards. This server enables seamless integration with Trello's API while handling rate limiting, type safety, and error handling automatically.
|
|
8
|
+
|
|
8
9
|
## Changelog
|
|
9
10
|
|
|
10
|
-
### 0.
|
|
11
|
+
### 1.0.0
|
|
12
|
+
|
|
13
|
+
- Fixed MCP protocol compatibility by removing all console output that interfered with JSON-RPC communication
|
|
14
|
+
- Improved pnpx support - now works seamlessly with `pnpx @delorenj/mcp-server-trello`
|
|
15
|
+
- Updated installation docs to feature pnpx as the primary installation method
|
|
16
|
+
- Added mise installation instructions for convenient tool management
|
|
17
|
+
- Production-ready release with stable API
|
|
18
|
+
|
|
19
|
+
### 0.3.0
|
|
20
|
+
|
|
21
|
+
- Added multi-board support - all methods now accept optional `boardId` parameter (thanks @blackoutnet!)
|
|
22
|
+
- `TRELLO_BOARD_ID` environment variable is now optional and serves as default board
|
|
23
|
+
- Added board and workspace management capabilities:
|
|
24
|
+
- `list_boards` - List all boards the user has access to
|
|
25
|
+
- `set_active_board` - Set the active board for future operations
|
|
26
|
+
- `list_workspaces` - List all workspaces the user has access to
|
|
27
|
+
- `set_active_workspace` - Set the active workspace for future operations
|
|
28
|
+
- `list_boards_in_workspace` - List all boards in a specific workspace
|
|
29
|
+
- `get_active_board_info` - Get information about the currently active board
|
|
30
|
+
- Added persistent configuration storage to remember active board/workspace
|
|
31
|
+
- Improved error handling with MCP-specific error types
|
|
32
|
+
- Full backward compatibility maintained
|
|
11
33
|
|
|
34
|
+
### 0.2.1
|
|
35
|
+
|
|
36
|
+
- Added detailed JSDoc comments to rate limiter functions
|
|
37
|
+
- Improved error handling for image attachment functionality
|
|
38
|
+
- Updated documentation for attach_image_to_card tool
|
|
39
|
+
|
|
40
|
+
### 0.2.0
|
|
41
|
+
|
|
42
|
+
- Added `attach_image_to_card` tool to attach images to cards from URLs
|
|
12
43
|
- Added Docker support with multi-stage build
|
|
13
44
|
- Improved security by moving environment variables to `.env`
|
|
14
45
|
- Added Docker Compose configuration
|
|
@@ -30,29 +61,68 @@ A Model Context Protocol (MCP) server that provides tools for interacting with T
|
|
|
30
61
|
- **Type-Safe Implementation**: Written in TypeScript with comprehensive type definitions
|
|
31
62
|
- **Input Validation**: Robust validation for all API inputs
|
|
32
63
|
- **Error Handling**: Graceful error handling with informative messages
|
|
64
|
+
- **Dynamic Board Selection**: Switch between boards and workspaces without restarting
|
|
33
65
|
|
|
34
66
|
## Installation
|
|
35
67
|
|
|
36
|
-
###
|
|
68
|
+
### Quick Start with pnpx (Recommended)
|
|
37
69
|
|
|
38
|
-
The easiest way to
|
|
70
|
+
The easiest way to use the Trello MCP server is with `pnpx`, which doesn't require a global install:
|
|
39
71
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
72
|
+
```json
|
|
73
|
+
{
|
|
74
|
+
"mcpServers": {
|
|
75
|
+
"trello": {
|
|
76
|
+
"command": "pnpx",
|
|
77
|
+
"args": ["@delorenj/mcp-server-trello"],
|
|
78
|
+
"env": {
|
|
79
|
+
"TRELLO_API_KEY": "your-api-key",
|
|
80
|
+
"TRELLO_TOKEN": "your-token"
|
|
81
|
+
}
|
|
82
|
+
}
|
|
83
|
+
}
|
|
84
|
+
}
|
|
44
85
|
```
|
|
45
86
|
|
|
46
|
-
|
|
87
|
+
Or if you're using mise:
|
|
88
|
+
|
|
89
|
+
```json
|
|
90
|
+
{
|
|
91
|
+
"mcpServers": {
|
|
92
|
+
"trello": {
|
|
93
|
+
"command": "mise",
|
|
94
|
+
"args": ["x", "--", "pnpx", "@delorenj/mcp-server-trello"],
|
|
95
|
+
"env": {
|
|
96
|
+
"TRELLO_API_KEY": "your-api-key",
|
|
97
|
+
"TRELLO_TOKEN": "your-token"
|
|
98
|
+
}
|
|
99
|
+
}
|
|
100
|
+
}
|
|
101
|
+
}
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
#### Don't have pnpm?
|
|
105
|
+
|
|
106
|
+
The simplest way to get `pnpm` (and thus `pnpx`) is through [mise](https://mise.jdx.dev/):
|
|
107
|
+
|
|
47
108
|
```bash
|
|
48
|
-
|
|
109
|
+
# Install mise (if you don't have it)
|
|
110
|
+
curl https://mise.run | sh
|
|
111
|
+
|
|
112
|
+
# Install pnpm with mise
|
|
113
|
+
mise install pnpm
|
|
49
114
|
```
|
|
50
115
|
|
|
51
|
-
|
|
116
|
+
### Installing via npm
|
|
117
|
+
|
|
118
|
+
If you prefer using npm directly:
|
|
119
|
+
|
|
52
120
|
```bash
|
|
53
|
-
|
|
121
|
+
npm install -g @delorenj/mcp-server-trello
|
|
54
122
|
```
|
|
55
123
|
|
|
124
|
+
Then use `mcp-server-trello` as the command in your MCP configuration.
|
|
125
|
+
|
|
56
126
|
### Installing via Smithery
|
|
57
127
|
|
|
58
128
|
To install Trello Server for Claude Desktop automatically via [Smithery](https://smithery.ai/server/@modelcontextprotocol/mcp-server-trello):
|
|
@@ -61,25 +131,120 @@ To install Trello Server for Claude Desktop automatically via [Smithery](https:/
|
|
|
61
131
|
npx -y @smithery/cli install @modelcontextprotocol/mcp-server-trello --client claude
|
|
62
132
|
```
|
|
63
133
|
|
|
64
|
-
###
|
|
134
|
+
### Docker Installation
|
|
135
|
+
|
|
136
|
+
For containerized environments:
|
|
137
|
+
|
|
138
|
+
1. Clone the repository:
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
git clone https://github.com/delorenj/mcp-server-trello
|
|
142
|
+
cd mcp-server-trello
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
2. Copy the environment template and fill in your Trello credentials:
|
|
146
|
+
|
|
147
|
+
```bash
|
|
148
|
+
cp .env.template .env
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
3. Build and run with Docker Compose:
|
|
152
|
+
|
|
65
153
|
```bash
|
|
66
|
-
|
|
154
|
+
docker compose up --build
|
|
67
155
|
```
|
|
68
156
|
|
|
69
157
|
## Configuration
|
|
70
158
|
|
|
159
|
+
### Environment Variables
|
|
160
|
+
|
|
71
161
|
The server can be configured using environment variables. Create a `.env` file in the root directory with the following variables:
|
|
72
162
|
|
|
73
163
|
```env
|
|
164
|
+
# Required: Your Trello API credentials
|
|
74
165
|
TRELLO_API_KEY=your-api-key
|
|
75
166
|
TRELLO_TOKEN=your-token
|
|
167
|
+
|
|
168
|
+
# Optional (Deprecated): Default board ID (can be changed later using set_active_board)
|
|
76
169
|
TRELLO_BOARD_ID=your-board-id
|
|
170
|
+
|
|
171
|
+
# Optional: Initial workspace ID (can be changed later using set_active_workspace)
|
|
172
|
+
TRELLO_WORKSPACE_ID=your-workspace-id
|
|
77
173
|
```
|
|
78
174
|
|
|
79
175
|
You can get these values from:
|
|
80
|
-
|
|
176
|
+
|
|
177
|
+
- API Key: <https://trello.com/app-key>
|
|
81
178
|
- Token: Generate using your API key
|
|
82
|
-
- Board ID: Found in the board URL
|
|
179
|
+
- Board ID (optional, deprecated): Found in the board URL (e.g., <https://trello.com/b/BOARD_ID/board-name>)
|
|
180
|
+
- Workspace ID: Found in workspace settings or using `list_workspaces` tool
|
|
181
|
+
|
|
182
|
+
### Board and Workspace Management
|
|
183
|
+
|
|
184
|
+
Starting with version 0.3.0, the MCP server supports multiple ways to work with boards:
|
|
185
|
+
|
|
186
|
+
1. **Multi-board support**: All methods now accept an optional `boardId` parameter
|
|
187
|
+
- Omit `TRELLO_BOARD_ID` and provide `boardId` in each API call
|
|
188
|
+
- Set `TRELLO_BOARD_ID` as default and optionally override with `boardId` parameter
|
|
189
|
+
|
|
190
|
+
2. **Dynamic board selection**: Use workspace management tools
|
|
191
|
+
- The `TRELLO_BOARD_ID` in your `.env` file is used as the initial/default board ID
|
|
192
|
+
- You can change the active board at any time using the `set_active_board` tool
|
|
193
|
+
- The selected board persists between server restarts (stored in `~/.trello-mcp/config.json`)
|
|
194
|
+
- Similarly, you can set and persist an active workspace using `set_active_workspace`
|
|
195
|
+
|
|
196
|
+
This allows you to work with multiple boards and workspaces without restarting the server.
|
|
197
|
+
|
|
198
|
+
#### Example Workflow
|
|
199
|
+
|
|
200
|
+
1. Start by listing available boards:
|
|
201
|
+
|
|
202
|
+
```typescript
|
|
203
|
+
{
|
|
204
|
+
name: 'list_boards',
|
|
205
|
+
arguments: {}
|
|
206
|
+
}
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
2. Set your active board:
|
|
210
|
+
|
|
211
|
+
```typescript
|
|
212
|
+
{
|
|
213
|
+
name: 'set_active_board',
|
|
214
|
+
arguments: {
|
|
215
|
+
boardId: "abc123" // ID from list_boards response
|
|
216
|
+
}
|
|
217
|
+
}
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
3. List workspaces if needed:
|
|
221
|
+
|
|
222
|
+
```typescript
|
|
223
|
+
{
|
|
224
|
+
name: 'list_workspaces',
|
|
225
|
+
arguments: {}
|
|
226
|
+
}
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
4. Set active workspace if needed:
|
|
230
|
+
|
|
231
|
+
```typescript
|
|
232
|
+
{
|
|
233
|
+
name: 'set_active_workspace',
|
|
234
|
+
arguments: {
|
|
235
|
+
workspaceId: "xyz789" // ID from list_workspaces response
|
|
236
|
+
}
|
|
237
|
+
}
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
5. Check current active board info:
|
|
241
|
+
|
|
242
|
+
```typescript
|
|
243
|
+
{
|
|
244
|
+
name: 'get_active_board_info',
|
|
245
|
+
arguments: {}
|
|
246
|
+
}
|
|
247
|
+
```
|
|
83
248
|
|
|
84
249
|
## Available Tools
|
|
85
250
|
|
|
@@ -91,31 +256,35 @@ Fetch all cards from a specific list.
|
|
|
91
256
|
{
|
|
92
257
|
name: 'get_cards_by_list_id',
|
|
93
258
|
arguments: {
|
|
94
|
-
|
|
259
|
+
boardId?: string, // Optional: ID of the board (uses default if not provided)
|
|
260
|
+
listId: string // ID of the Trello list
|
|
95
261
|
}
|
|
96
262
|
}
|
|
97
263
|
```
|
|
98
264
|
|
|
99
265
|
### get_lists
|
|
100
266
|
|
|
101
|
-
Retrieve all lists from
|
|
267
|
+
Retrieve all lists from a board.
|
|
102
268
|
|
|
103
269
|
```typescript
|
|
104
270
|
{
|
|
105
271
|
name: 'get_lists',
|
|
106
|
-
arguments: {
|
|
272
|
+
arguments: {
|
|
273
|
+
boardId?: string // Optional: ID of the board (uses default if not provided)
|
|
274
|
+
}
|
|
107
275
|
}
|
|
108
276
|
```
|
|
109
277
|
|
|
110
278
|
### get_recent_activity
|
|
111
279
|
|
|
112
|
-
Fetch recent activity on
|
|
280
|
+
Fetch recent activity on a board.
|
|
113
281
|
|
|
114
282
|
```typescript
|
|
115
283
|
{
|
|
116
284
|
name: 'get_recent_activity',
|
|
117
285
|
arguments: {
|
|
118
|
-
|
|
286
|
+
boardId?: string, // Optional: ID of the board (uses default if not provided)
|
|
287
|
+
limit?: number // Optional: Number of activities to fetch (default: 10)
|
|
119
288
|
}
|
|
120
289
|
}
|
|
121
290
|
```
|
|
@@ -128,6 +297,7 @@ Add a new card to a specified list.
|
|
|
128
297
|
{
|
|
129
298
|
name: 'add_card_to_list',
|
|
130
299
|
arguments: {
|
|
300
|
+
boardId?: string, // Optional: ID of the board (uses default if not provided)
|
|
131
301
|
listId: string, // ID of the list to add the card to
|
|
132
302
|
name: string, // Name of the card
|
|
133
303
|
description?: string, // Optional: Description of the card
|
|
@@ -145,6 +315,7 @@ Update an existing card's details.
|
|
|
145
315
|
{
|
|
146
316
|
name: 'update_card_details',
|
|
147
317
|
arguments: {
|
|
318
|
+
boardId?: string, // Optional: ID of the board (uses default if not provided)
|
|
148
319
|
cardId: string, // ID of the card to update
|
|
149
320
|
name?: string, // Optional: New name for the card
|
|
150
321
|
description?: string, // Optional: New description
|
|
@@ -162,20 +333,22 @@ Send a card to the archive.
|
|
|
162
333
|
{
|
|
163
334
|
name: 'archive_card',
|
|
164
335
|
arguments: {
|
|
165
|
-
|
|
336
|
+
boardId?: string, // Optional: ID of the board (uses default if not provided)
|
|
337
|
+
cardId: string // ID of the card to archive
|
|
166
338
|
}
|
|
167
339
|
}
|
|
168
340
|
```
|
|
169
341
|
|
|
170
342
|
### add_list_to_board
|
|
171
343
|
|
|
172
|
-
Add a new list to
|
|
344
|
+
Add a new list to a board.
|
|
173
345
|
|
|
174
346
|
```typescript
|
|
175
347
|
{
|
|
176
348
|
name: 'add_list_to_board',
|
|
177
349
|
arguments: {
|
|
178
|
-
|
|
350
|
+
boardId?: string, // Optional: ID of the board (uses default if not provided)
|
|
351
|
+
name: string // Name of the new list
|
|
179
352
|
}
|
|
180
353
|
}
|
|
181
354
|
```
|
|
@@ -188,7 +361,8 @@ Send a list to the archive.
|
|
|
188
361
|
{
|
|
189
362
|
name: 'archive_list',
|
|
190
363
|
arguments: {
|
|
191
|
-
|
|
364
|
+
boardId?: string, // Optional: ID of the board (uses default if not provided)
|
|
365
|
+
listId: string // ID of the list to archive
|
|
192
366
|
}
|
|
193
367
|
}
|
|
194
368
|
```
|
|
@@ -212,12 +386,165 @@ Move a card to a different list.
|
|
|
212
386
|
{
|
|
213
387
|
name: 'move_card',
|
|
214
388
|
arguments: {
|
|
215
|
-
|
|
216
|
-
|
|
389
|
+
boardId?: string, // Optional: ID of the target board (uses default if not provided)
|
|
390
|
+
cardId: string, // ID of the card to move
|
|
391
|
+
listId: string // ID of the target list
|
|
392
|
+
}
|
|
393
|
+
}
|
|
394
|
+
```
|
|
395
|
+
|
|
396
|
+
### attach_image_to_card
|
|
397
|
+
|
|
398
|
+
Attach an image to a card directly from a URL.
|
|
399
|
+
|
|
400
|
+
```typescript
|
|
401
|
+
{
|
|
402
|
+
name: 'attach_image_to_card',
|
|
403
|
+
arguments: {
|
|
404
|
+
boardId?: string, // Optional: ID of the board (uses default if not provided)
|
|
405
|
+
cardId: string, // ID of the card to attach the image to
|
|
406
|
+
imageUrl: string, // URL of the image to attach
|
|
407
|
+
name?: string // Optional: Name for the attachment (defaults to "Image Attachment")
|
|
408
|
+
}
|
|
409
|
+
}
|
|
410
|
+
```
|
|
411
|
+
|
|
412
|
+
### list_boards
|
|
413
|
+
|
|
414
|
+
List all boards the user has access to.
|
|
415
|
+
|
|
416
|
+
```typescript
|
|
417
|
+
{
|
|
418
|
+
name: 'list_boards',
|
|
419
|
+
arguments: {}
|
|
420
|
+
}
|
|
421
|
+
```
|
|
422
|
+
|
|
423
|
+
### set_active_board
|
|
424
|
+
|
|
425
|
+
Set the active board for future operations.
|
|
426
|
+
|
|
427
|
+
```typescript
|
|
428
|
+
{
|
|
429
|
+
name: 'set_active_board',
|
|
430
|
+
arguments: {
|
|
431
|
+
boardId: string // ID of the board to set as active
|
|
432
|
+
}
|
|
433
|
+
}
|
|
434
|
+
```
|
|
435
|
+
|
|
436
|
+
### list_workspaces
|
|
437
|
+
|
|
438
|
+
List all workspaces the user has access to.
|
|
439
|
+
|
|
440
|
+
```typescript
|
|
441
|
+
{
|
|
442
|
+
name: 'list_workspaces',
|
|
443
|
+
arguments: {}
|
|
444
|
+
}
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
### set_active_workspace
|
|
448
|
+
|
|
449
|
+
Set the active workspace for future operations.
|
|
450
|
+
|
|
451
|
+
```typescript
|
|
452
|
+
{
|
|
453
|
+
name: 'set_active_workspace',
|
|
454
|
+
arguments: {
|
|
455
|
+
workspaceId: string // ID of the workspace to set as active
|
|
217
456
|
}
|
|
218
457
|
}
|
|
219
458
|
```
|
|
220
459
|
|
|
460
|
+
### list_boards_in_workspace
|
|
461
|
+
|
|
462
|
+
List all boards in a specific workspace.
|
|
463
|
+
|
|
464
|
+
```typescript
|
|
465
|
+
{
|
|
466
|
+
name: 'list_boards_in_workspace',
|
|
467
|
+
arguments: {
|
|
468
|
+
workspaceId: string // ID of the workspace to list boards from
|
|
469
|
+
}
|
|
470
|
+
}
|
|
471
|
+
```
|
|
472
|
+
|
|
473
|
+
### get_active_board_info
|
|
474
|
+
|
|
475
|
+
Get information about the currently active board.
|
|
476
|
+
|
|
477
|
+
```typescript
|
|
478
|
+
{
|
|
479
|
+
name: 'get_active_board_info',
|
|
480
|
+
arguments: {}
|
|
481
|
+
}
|
|
482
|
+
```
|
|
483
|
+
|
|
484
|
+
## Integration Examples
|
|
485
|
+
|
|
486
|
+
### 🎨 Pairing with Ideogram MCP Server
|
|
487
|
+
|
|
488
|
+
The Trello MCP server pairs beautifully with [@flowluap/ideogram-mcp-server](https://github.com/flowluap/ideogram-mcp-server) for AI-powered visual content creation. Generate images with Ideogram and attach them directly to your Trello cards!
|
|
489
|
+
|
|
490
|
+

|
|
491
|
+
|
|
492
|
+
#### Example Workflow
|
|
493
|
+
|
|
494
|
+
1. **Generate an image with Ideogram:**
|
|
495
|
+
```typescript
|
|
496
|
+
// Using ideogram-mcp-server
|
|
497
|
+
{
|
|
498
|
+
name: 'generate_image',
|
|
499
|
+
arguments: {
|
|
500
|
+
prompt: "A futuristic dashboard design with neon accents",
|
|
501
|
+
aspect_ratio: "16:9"
|
|
502
|
+
}
|
|
503
|
+
}
|
|
504
|
+
// Returns: { image_url: "https://..." }
|
|
505
|
+
```
|
|
506
|
+
|
|
507
|
+
2. **Attach the generated image to a Trello card:**
|
|
508
|
+
```typescript
|
|
509
|
+
// Using trello-mcp-server
|
|
510
|
+
{
|
|
511
|
+
name: 'attach_image_to_card',
|
|
512
|
+
arguments: {
|
|
513
|
+
cardId: "your-card-id",
|
|
514
|
+
imageUrl: "https://...", // URL from Ideogram
|
|
515
|
+
name: "Dashboard Mockup v1"
|
|
516
|
+
}
|
|
517
|
+
}
|
|
518
|
+
```
|
|
519
|
+
|
|
520
|
+
#### Setting up both servers
|
|
521
|
+
|
|
522
|
+
Add both servers to your Claude Desktop configuration:
|
|
523
|
+
|
|
524
|
+
```json
|
|
525
|
+
{
|
|
526
|
+
"mcpServers": {
|
|
527
|
+
"trello": {
|
|
528
|
+
"command": "pnpx",
|
|
529
|
+
"args": ["@delorenj/mcp-server-trello"],
|
|
530
|
+
"env": {
|
|
531
|
+
"TRELLO_API_KEY": "your-trello-api-key",
|
|
532
|
+
"TRELLO_TOKEN": "your-trello-token"
|
|
533
|
+
}
|
|
534
|
+
},
|
|
535
|
+
"ideogram": {
|
|
536
|
+
"command": "pnpx",
|
|
537
|
+
"args": ["@flowluap/ideogram-mcp-server"],
|
|
538
|
+
"env": {
|
|
539
|
+
"IDEOGRAM_API_KEY": "your-ideogram-api-key"
|
|
540
|
+
}
|
|
541
|
+
}
|
|
542
|
+
}
|
|
543
|
+
}
|
|
544
|
+
```
|
|
545
|
+
|
|
546
|
+
Now you can seamlessly create visual content and organize it in Trello, all within Claude!
|
|
547
|
+
|
|
221
548
|
## Rate Limiting
|
|
222
549
|
|
|
223
550
|
The server implements a token bucket algorithm for rate limiting to comply with Trello's API limits:
|
|
@@ -265,6 +592,13 @@ npm install
|
|
|
265
592
|
npm run build
|
|
266
593
|
```
|
|
267
594
|
|
|
595
|
+
## Running evals
|
|
596
|
+
|
|
597
|
+
The evals package loads an mcp client that then runs the index.ts file, so there is no need to rebuild between tests. You can load environment variables by prefixing the npx command. Full documentation can be found [here](https://www.mcpevals.io/docs).
|
|
598
|
+
|
|
599
|
+
```bash
|
|
600
|
+
OPENAI_API_KEY=your-key npx mcp-eval src/evals/evals.ts src/index.ts
|
|
601
|
+
```
|
|
268
602
|
|
|
269
603
|
## Contributing
|
|
270
604
|
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
//evals.ts
|
|
2
|
+
import { openai } from "@ai-sdk/openai";
|
|
3
|
+
import { grade } from "mcp-evals";
|
|
4
|
+
const get_cards_by_list_idEval = {
|
|
5
|
+
name: 'get_cards_by_list_id Tool Evaluation',
|
|
6
|
+
description: 'Evaluates the get_cards_by_list_id tool functionality',
|
|
7
|
+
run: async () => {
|
|
8
|
+
const result = await grade(openai("gpt-4"), "Can you fetch all cards from the Trello list with ID abc123?");
|
|
9
|
+
return JSON.parse(result);
|
|
10
|
+
}
|
|
11
|
+
};
|
|
12
|
+
const get_listsEval = {
|
|
13
|
+
name: 'get_lists Tool Evaluation',
|
|
14
|
+
description: 'Evaluates the get_lists tool by retrieving all lists from a specified board',
|
|
15
|
+
run: async () => {
|
|
16
|
+
const result = await grade(openai("gpt-4"), "Please retrieve all lists from the board with ID 12345 and provide their names.");
|
|
17
|
+
return JSON.parse(result);
|
|
18
|
+
}
|
|
19
|
+
};
|
|
20
|
+
const get_recent_activityEvalFunction = {
|
|
21
|
+
name: 'get_recent_activity Tool Evaluation',
|
|
22
|
+
description: 'Evaluates the ability to fetch recent activity on the Trello board',
|
|
23
|
+
run: async () => {
|
|
24
|
+
const result = await grade(openai("gpt-4"), "Fetch the recent activity on the Trello board, limit it to 5 items");
|
|
25
|
+
return JSON.parse(result);
|
|
26
|
+
}
|
|
27
|
+
};
|
|
28
|
+
const add_card_to_listEval = {
|
|
29
|
+
name: 'add_card_to_listEval',
|
|
30
|
+
description: 'Evaluates the add_card_to_list tool',
|
|
31
|
+
run: async () => {
|
|
32
|
+
const result = await grade(openai("gpt-4"), "Please add a new card named 'Demo Card' to the list with ID 'abc123', with a description of 'This is a test card', due date '2023-12-31T12:00:00Z', and a label 'priority'.");
|
|
33
|
+
return JSON.parse(result);
|
|
34
|
+
}
|
|
35
|
+
};
|
|
36
|
+
const update_card_detailsEval = {
|
|
37
|
+
name: 'update_card_details Evaluation',
|
|
38
|
+
description: 'Evaluates the update_card_details tool functionality',
|
|
39
|
+
run: async () => {
|
|
40
|
+
const result = await grade(openai("gpt-4"), "Please update the card with ID 'abc123' to have the name 'Updated Card Name', the description 'New description for the card', a due date of '2024-01-01T10:00:00Z', and labels ['priority','review'].");
|
|
41
|
+
return JSON.parse(result);
|
|
42
|
+
}
|
|
43
|
+
};
|
|
44
|
+
const config = {
|
|
45
|
+
model: openai("gpt-4"),
|
|
46
|
+
evals: [get_cards_by_list_idEval, get_listsEval, get_recent_activityEvalFunction, add_card_to_listEval, update_card_detailsEval]
|
|
47
|
+
};
|
|
48
|
+
export default config;
|
|
49
|
+
export const evals = [get_cards_by_list_idEval, get_listsEval, get_recent_activityEvalFunction, add_card_to_listEval, update_card_detailsEval];
|
|
50
|
+
//# sourceMappingURL=evals.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"evals.js","sourceRoot":"","sources":["../../src/evals/evals.ts"],"names":[],"mappings":"AAAA,UAAU;AAGV,OAAO,EAAE,MAAM,EAAE,MAAM,gBAAgB,CAAC;AACxC,OAAO,EAAE,KAAK,EAAgB,MAAM,WAAW,CAAC;AAEhD,MAAM,wBAAwB,GAAiB;IAC3C,IAAI,EAAE,sCAAsC;IAC5C,WAAW,EAAE,uDAAuD;IACpE,GAAG,EAAE,KAAK,IAAI,EAAE;QACZ,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,8DAA8D,CAAC,CAAC;QAC5G,OAAO,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;IAC9B,CAAC;CACJ,CAAC;AAEF,MAAM,aAAa,GAAiB;IAChC,IAAI,EAAE,2BAA2B;IACjC,WAAW,EAAE,6EAA6E;IAC1F,GAAG,EAAE,KAAK,IAAI,EAAE;QACZ,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,iFAAiF,CAAC,CAAC;QAC/H,OAAO,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;IAC9B,CAAC;CACJ,CAAC;AAEF,MAAM,+BAA+B,GAAiB;IAClD,IAAI,EAAE,qCAAqC;IAC3C,WAAW,EAAE,oEAAoE;IACjF,GAAG,EAAE,KAAK,IAAI,EAAE;QACZ,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,oEAAoE,CAAC,CAAC;QAClH,OAAO,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;IAC9B,CAAC;CACJ,CAAC;AAEF,MAAM,oBAAoB,GAAiB;IACvC,IAAI,EAAE,sBAAsB;IAC5B,WAAW,EAAE,qCAAqC;IAClD,GAAG,EAAE,KAAK,IAAI,EAAE;QACZ,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,6KAA6K,CAAC,CAAC;QAC3N,OAAO,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;IAC9B,CAAC;CACJ,CAAC;AAEF,MAAM,uBAAuB,GAAiB;IAC1C,IAAI,EAAE,gCAAgC;IACtC,WAAW,EAAE,sDAAsD;IACnE,GAAG,EAAE,KAAK,IAAI,EAAE;QACZ,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,uMAAuM,CAAC,CAAC;QACrP,OAAO,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;IAC9B,CAAC;CACJ,CAAC;AAEF,MAAM,MAAM,GAAe;IACvB,KAAK,EAAE,MAAM,CAAC,OAAO,CAAC;IACtB,KAAK,EAAE,CAAC,wBAAwB,EAAE,aAAa,EAAE,+BAA+B,EAAE,oBAAoB,EAAE,uBAAuB,CAAC;CACnI,CAAC;AAEF,eAAe,MAAM,CAAC;AAEtB,MAAM,CAAC,MAAM,KAAK,GAAG,CAAC,wBAAwB,EAAE,aAAa,EAAE,+BAA+B,EAAE,oBAAoB,EAAE,uBAAuB,CAAC,CAAC"}
|