s3db.js 10.0.3 โ 10.0.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/s3db-cli.js +0 -0
- package/dist/s3db.cjs.js +549 -2
- package/dist/s3db.cjs.js.map +1 -1
- package/dist/s3db.es.js +549 -2
- package/dist/s3db.es.js.map +1 -1
- package/mcp/README.md +1728 -0
- package/package.json +24 -22
- package/src/plugins/eventual-consistency.plugin.js +686 -2
package/mcp/README.md
ADDED
|
@@ -0,0 +1,1728 @@
|
|
|
1
|
+
# ๐๏ธ S3DB MCP Server
|
|
2
|
+
|
|
3
|
+
<p align="center">
|
|
4
|
+
<img width="200" src="https://img.icons8.com/fluency/200/database.png" alt="S3DB MCP Server">
|
|
5
|
+
</p>
|
|
6
|
+
|
|
7
|
+
<p align="center">
|
|
8
|
+
<strong>Model Context Protocol (MCP) server for S3DB</strong><br>
|
|
9
|
+
<em>Transform AWS S3 into a powerful document database accessible by AI agents</em>
|
|
10
|
+
</p>
|
|
11
|
+
|
|
12
|
+
<p align="center">
|
|
13
|
+
<a href="https://www.npmjs.com/package/s3db-mcp-server"><img src="https://img.shields.io/npm/v/s3db-mcp-server.svg?style=flat&color=brightgreen" alt="npm version"></a>
|
|
14
|
+
|
|
15
|
+
<a href="https://github.com/forattini-dev/s3db.js"><img src="https://img.shields.io/github/stars/forattini-dev/s3db.js?style=flat&color=yellow" alt="GitHub stars"></a>
|
|
16
|
+
|
|
17
|
+
<a href="https://github.com/forattini-dev/s3db.js/blob/main/UNLICENSE"><img src="https://img.shields.io/badge/license-Unlicense-blue.svg?style=flat" alt="License"></a>
|
|
18
|
+
</p>
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## ๐ Quick Start with Claude Desktop
|
|
23
|
+
|
|
24
|
+
### Prerequisites
|
|
25
|
+
Before starting, ensure you have:
|
|
26
|
+
1. **Claude Desktop** installed (version 0.7.0 or later)
|
|
27
|
+
2. **Node.js** installed (version 18+ recommended)
|
|
28
|
+
3. **S3 Bucket** or S3-compatible storage (MinIO, etc.)
|
|
29
|
+
4. **Credentials** for your S3 service
|
|
30
|
+
|
|
31
|
+
### Step 1: Locate Claude Desktop Configuration
|
|
32
|
+
|
|
33
|
+
The configuration file location depends on your operating system:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
# macOS
|
|
37
|
+
~/Library/Application Support/Claude/claude_desktop_config.json
|
|
38
|
+
|
|
39
|
+
# Windows
|
|
40
|
+
%APPDATA%\Claude\claude_desktop_config.json
|
|
41
|
+
|
|
42
|
+
# Linux
|
|
43
|
+
~/.config/Claude/claude_desktop_config.json
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
### Step 2: Configure Claude Desktop
|
|
47
|
+
|
|
48
|
+
Open the configuration file and add the S3DB MCP server:
|
|
49
|
+
|
|
50
|
+
```json
|
|
51
|
+
{
|
|
52
|
+
"mcpServers": {
|
|
53
|
+
"s3db": {
|
|
54
|
+
"command": "npx",
|
|
55
|
+
"args": ["s3db-mcp-server", "--transport=sse"],
|
|
56
|
+
"env": {
|
|
57
|
+
"S3DB_CONNECTION_STRING": "s3://ACCESS_KEY:SECRET_KEY@bucket/databases/myapp",
|
|
58
|
+
"S3DB_CACHE_ENABLED": "true",
|
|
59
|
+
"S3DB_CACHE_DRIVER": "memory",
|
|
60
|
+
"S3DB_CACHE_MAX_SIZE": "1000",
|
|
61
|
+
"S3DB_CACHE_TTL": "300000",
|
|
62
|
+
"MCP_SERVER_PORT": "17500"
|
|
63
|
+
}
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Step 3: Restart Claude Desktop
|
|
70
|
+
|
|
71
|
+
After saving the configuration:
|
|
72
|
+
1. Completely quit Claude Desktop (not just close the window)
|
|
73
|
+
2. Restart Claude Desktop
|
|
74
|
+
3. The MCP server should start automatically
|
|
75
|
+
|
|
76
|
+
### Step 4: Test Your Setup
|
|
77
|
+
|
|
78
|
+
In Claude Desktop, test with these commands:
|
|
79
|
+
|
|
80
|
+
```javascript
|
|
81
|
+
// 1. Test connection
|
|
82
|
+
"Can you connect to the S3DB database and show me the status?"
|
|
83
|
+
|
|
84
|
+
// 2. Create a resource
|
|
85
|
+
"Please create a new resource called 'users' with these fields:
|
|
86
|
+
- name (string, required)
|
|
87
|
+
- email (string, required, unique)
|
|
88
|
+
- age (number)
|
|
89
|
+
- active (boolean, default true)"
|
|
90
|
+
|
|
91
|
+
// 3. Insert data
|
|
92
|
+
"Insert a new user with name 'John Doe', email 'john@example.com', age 30"
|
|
93
|
+
|
|
94
|
+
// 4. Query data
|
|
95
|
+
"Show me all users in the database"
|
|
96
|
+
|
|
97
|
+
// 5. Check statistics
|
|
98
|
+
"Can you show me the database statistics including cache hits and S3 costs?"
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
---
|
|
102
|
+
|
|
103
|
+
## ๐ Table of Contents
|
|
104
|
+
|
|
105
|
+
- [๐ Quick Start with Claude Desktop](#-quick-start-with-claude-desktop)
|
|
106
|
+
- [๐พ Alternative Installation Methods](#-alternative-installation-methods)
|
|
107
|
+
- [โ๏ธ Configuration Examples](#๏ธ-configuration-examples)
|
|
108
|
+
- [๐ ๏ธ Available Tools](#๏ธ-available-tools)
|
|
109
|
+
- [๐ Command Line Examples](#-command-line-examples)
|
|
110
|
+
- [๐๏ธ Partitions & Performance](#๏ธ-partitions--performance)
|
|
111
|
+
- [๐ณ Docker Deployment](#-docker-deployment)
|
|
112
|
+
- [๐ง Advanced Configuration](#-advanced-configuration)
|
|
113
|
+
- [๐ Security](#-security)
|
|
114
|
+
- [๐จ Troubleshooting](#-troubleshooting)
|
|
115
|
+
|
|
116
|
+
---
|
|
117
|
+
|
|
118
|
+
## ๐พ Alternative Installation Methods
|
|
119
|
+
|
|
120
|
+
### Option 1: NPX (Recommended)
|
|
121
|
+
```bash
|
|
122
|
+
# SSE transport (web clients) - Default port: 17500
|
|
123
|
+
npx s3db-mcp-server --transport=sse
|
|
124
|
+
|
|
125
|
+
# STDIO transport (desktop clients)
|
|
126
|
+
npx s3db-mcp-server --transport=stdio
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
### Option 2: Global Installation
|
|
130
|
+
```bash
|
|
131
|
+
npm install -g s3db-mcp-server
|
|
132
|
+
s3db-mcp --transport=sse
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
### Option 3: Docker
|
|
136
|
+
```bash
|
|
137
|
+
docker run -p 17500:8000 -e S3DB_CONNECTION_STRING="s3://key:secret@bucket/db" s3db-mcp-server
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
### Option 4: Standalone Server
|
|
141
|
+
```bash
|
|
142
|
+
# Start the server (it will run in the foreground)
|
|
143
|
+
npx s3db-mcp-server --transport=sse
|
|
144
|
+
|
|
145
|
+
# With environment variables
|
|
146
|
+
cat > s3db-mcp.env << EOF
|
|
147
|
+
S3DB_CONNECTION_STRING=s3://ACCESS_KEY:SECRET_KEY@bucket-name/databases/myapp
|
|
148
|
+
S3DB_CACHE_ENABLED=true
|
|
149
|
+
S3DB_CACHE_DRIVER=memory
|
|
150
|
+
S3DB_CACHE_MAX_SIZE=1000
|
|
151
|
+
EOF
|
|
152
|
+
|
|
153
|
+
env $(cat s3db-mcp.env | xargs) npx s3db-mcp-server --transport=sse
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
### Option 5: Background Process
|
|
157
|
+
```bash
|
|
158
|
+
# Using nohup (Linux/macOS)
|
|
159
|
+
nohup npx s3db-mcp-server --transport=sse > s3db-mcp.log 2>&1 &
|
|
160
|
+
echo $! > s3db-mcp.pid
|
|
161
|
+
|
|
162
|
+
# To stop later
|
|
163
|
+
kill $(cat s3db-mcp.pid)
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
### Option 6: Using PM2 Process Manager
|
|
167
|
+
```bash
|
|
168
|
+
# Install PM2 globally
|
|
169
|
+
npm install -g pm2
|
|
170
|
+
|
|
171
|
+
# Start with PM2
|
|
172
|
+
pm2 start npx --name "s3db-mcp" -- s3db-mcp-server --transport=sse
|
|
173
|
+
|
|
174
|
+
# View logs
|
|
175
|
+
pm2 logs s3db-mcp
|
|
176
|
+
|
|
177
|
+
# Stop/restart
|
|
178
|
+
pm2 stop s3db-mcp
|
|
179
|
+
pm2 restart s3db-mcp
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## โ๏ธ Configuration Examples
|
|
185
|
+
|
|
186
|
+
### ๐ Local Development with MinIO
|
|
187
|
+
|
|
188
|
+
```json
|
|
189
|
+
{
|
|
190
|
+
"mcpServers": {
|
|
191
|
+
"s3db-local": {
|
|
192
|
+
"command": "npx",
|
|
193
|
+
"args": ["s3db-mcp-server", "--transport=sse"],
|
|
194
|
+
"env": {
|
|
195
|
+
"S3DB_CONNECTION_STRING": "s3://minioadmin:minioadmin123@localhost:9000/dev-bucket?forcePathStyle=true",
|
|
196
|
+
"S3DB_VERBOSE": "true",
|
|
197
|
+
"S3DB_CACHE_ENABLED": "true",
|
|
198
|
+
"S3DB_COSTS_ENABLED": "false",
|
|
199
|
+
"NODE_ENV": "development"
|
|
200
|
+
}
|
|
201
|
+
}
|
|
202
|
+
}
|
|
203
|
+
}
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
### โ๏ธ Production with AWS S3
|
|
207
|
+
|
|
208
|
+
```json
|
|
209
|
+
{
|
|
210
|
+
"mcpServers": {
|
|
211
|
+
"s3db-prod": {
|
|
212
|
+
"command": "npx",
|
|
213
|
+
"args": ["s3db-mcp-server", "--transport=sse"],
|
|
214
|
+
"env": {
|
|
215
|
+
"S3DB_CONNECTION_STRING": "s3://prod-data-bucket/databases/main",
|
|
216
|
+
"AWS_REGION": "us-east-1",
|
|
217
|
+
"AWS_ACCESS_KEY_ID": "AKIA...",
|
|
218
|
+
"AWS_SECRET_ACCESS_KEY": "wJal...",
|
|
219
|
+
"S3DB_CACHE_DRIVER": "filesystem",
|
|
220
|
+
"S3DB_CACHE_DIRECTORY": "/var/cache/s3db",
|
|
221
|
+
"S3DB_CACHE_TTL": "1800000",
|
|
222
|
+
"S3DB_PASSPHRASE": "your-strong-passphrase-here",
|
|
223
|
+
"S3DB_VERSIONING_ENABLED": "true",
|
|
224
|
+
"NODE_ENV": "production"
|
|
225
|
+
}
|
|
226
|
+
}
|
|
227
|
+
}
|
|
228
|
+
}
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
### ๐ DigitalOcean Spaces
|
|
232
|
+
|
|
233
|
+
```json
|
|
234
|
+
{
|
|
235
|
+
"mcpServers": {
|
|
236
|
+
"s3db-do": {
|
|
237
|
+
"command": "npx",
|
|
238
|
+
"args": ["s3db-mcp-server", "--transport=sse"],
|
|
239
|
+
"env": {
|
|
240
|
+
"S3DB_CONNECTION_STRING": "s3://DO_ACCESS_KEY:DO_SECRET_KEY@nyc3.digitaloceanspaces.com/space-name/databases/app",
|
|
241
|
+
"S3_ENDPOINT": "https://nyc3.digitaloceanspaces.com",
|
|
242
|
+
"S3DB_CACHE_ENABLED": "true"
|
|
243
|
+
}
|
|
244
|
+
}
|
|
245
|
+
}
|
|
246
|
+
}
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
### ๐ง Multiple Environments
|
|
250
|
+
|
|
251
|
+
```json
|
|
252
|
+
{
|
|
253
|
+
"mcpServers": {
|
|
254
|
+
"s3db-dev": {
|
|
255
|
+
"command": "npx",
|
|
256
|
+
"args": ["s3db-mcp-server", "--transport=sse", "--port=17500"],
|
|
257
|
+
"env": {
|
|
258
|
+
"S3DB_CONNECTION_STRING": "s3://minioadmin:minioadmin123@localhost:9000/dev-bucket?forcePathStyle=true"
|
|
259
|
+
}
|
|
260
|
+
},
|
|
261
|
+
"s3db-staging": {
|
|
262
|
+
"command": "npx",
|
|
263
|
+
"args": ["s3db-mcp-server", "--transport=sse", "--port=17501"],
|
|
264
|
+
"env": {
|
|
265
|
+
"S3DB_CONNECTION_STRING": "s3://staging-bucket/databases/staging",
|
|
266
|
+
"AWS_REGION": "us-east-1"
|
|
267
|
+
}
|
|
268
|
+
},
|
|
269
|
+
"s3db-prod": {
|
|
270
|
+
"command": "npx",
|
|
271
|
+
"args": ["s3db-mcp-server", "--transport=sse", "--port=17502"],
|
|
272
|
+
"env": {
|
|
273
|
+
"S3DB_CONNECTION_STRING": "s3://prod-bucket/databases/production",
|
|
274
|
+
"AWS_REGION": "us-east-1",
|
|
275
|
+
"S3DB_CACHE_DRIVER": "filesystem"
|
|
276
|
+
}
|
|
277
|
+
}
|
|
278
|
+
}
|
|
279
|
+
}
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
## โ๏ธ Configuration Reference
|
|
283
|
+
|
|
284
|
+
### ๐ Configuration Overview
|
|
285
|
+
|
|
286
|
+
The S3DB MCP Server can be configured through multiple methods (in order of precedence):
|
|
287
|
+
1. **Command-line arguments** (highest priority)
|
|
288
|
+
2. **Environment variables**
|
|
289
|
+
3. **`.env` file**
|
|
290
|
+
4. **Default values** (lowest priority)
|
|
291
|
+
|
|
292
|
+
### ๐ Server Configuration
|
|
293
|
+
|
|
294
|
+
#### **Core Server Settings**
|
|
295
|
+
|
|
296
|
+
| Variable | Default | Description | Example | Notes |
|
|
297
|
+
|----------|---------|-------------|---------|-------|
|
|
298
|
+
| `NODE_ENV` | `development` | Environment mode | `production`, `development`, `test` | Affects logging verbosity and error details |
|
|
299
|
+
| `MCP_SERVER_HOST` | `0.0.0.0` | Server bind address | `localhost`, `127.0.0.1`, `0.0.0.0` | Use `0.0.0.0` to accept connections from any interface |
|
|
300
|
+
| `MCP_SERVER_PORT` | `17500` | Server port | Any port 1024-65535 | Changed from 8000 to avoid conflicts |
|
|
301
|
+
| `MCP_TRANSPORT` | `sse` | Transport method | `sse`, `stdio` | SSE for web clients, stdio for CLI tools |
|
|
302
|
+
|
|
303
|
+
#### **Transport Modes Explained**
|
|
304
|
+
|
|
305
|
+
- **SSE (Server-Sent Events)**:
|
|
306
|
+
- Best for: Web-based AI clients, Claude Desktop, Cursor IDE
|
|
307
|
+
- Protocol: HTTP/HTTPS
|
|
308
|
+
- URL format: `http://localhost:17500/sse`
|
|
309
|
+
|
|
310
|
+
- **STDIO (Standard Input/Output)**:
|
|
311
|
+
- Best for: CLI tools, shell scripts, pipe-based communication
|
|
312
|
+
- Protocol: JSON-RPC over stdin/stdout
|
|
313
|
+
- No network port required
|
|
314
|
+
|
|
315
|
+
### ๐๏ธ S3DB Core Configuration
|
|
316
|
+
|
|
317
|
+
#### **Essential Database Settings**
|
|
318
|
+
|
|
319
|
+
| Variable | Default | Required | Description | Example Values |
|
|
320
|
+
|----------|---------|----------|-------------|----------------|
|
|
321
|
+
| `S3DB_CONNECTION_STRING` | - | โ
Yes | Complete S3 connection URL | See [Connection String Formats](#connection-string-formats) below |
|
|
322
|
+
| `S3DB_VERBOSE` | `false` | No | Enable detailed operation logs | `true` for debugging, `false` for production |
|
|
323
|
+
| `S3DB_PARALLELISM` | `10` | No | Max concurrent S3 operations | `5` (conservative), `20` (aggressive), `50` (high-performance) |
|
|
324
|
+
| `S3DB_PASSPHRASE` | `secret` | No | Encryption key for sensitive fields | Any strong passphrase (min 12 chars recommended) |
|
|
325
|
+
| `S3DB_VERSIONING_ENABLED` | `false` | No | Track resource schema versions | `true` for production, `false` for development |
|
|
326
|
+
|
|
327
|
+
#### **Performance Tuning Guidelines**
|
|
328
|
+
|
|
329
|
+
```bash
|
|
330
|
+
# Development (fast iteration, verbose logging)
|
|
331
|
+
S3DB_VERBOSE=true
|
|
332
|
+
S3DB_PARALLELISM=5
|
|
333
|
+
S3DB_VERSIONING_ENABLED=false
|
|
334
|
+
|
|
335
|
+
# Staging (balanced performance)
|
|
336
|
+
S3DB_VERBOSE=false
|
|
337
|
+
S3DB_PARALLELISM=10
|
|
338
|
+
S3DB_VERSIONING_ENABLED=true
|
|
339
|
+
|
|
340
|
+
# Production (optimized for scale)
|
|
341
|
+
S3DB_VERBOSE=false
|
|
342
|
+
S3DB_PARALLELISM=20
|
|
343
|
+
S3DB_VERSIONING_ENABLED=true
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
### ๐ Plugin Configuration
|
|
347
|
+
|
|
348
|
+
#### **Cache Plugin Settings**
|
|
349
|
+
|
|
350
|
+
| Variable | Default | Description | When to Change | Impact |
|
|
351
|
+
|----------|---------|-------------|----------------|--------|
|
|
352
|
+
| `S3DB_CACHE_ENABLED` | `true` | Master cache toggle | Set `false` only for debugging | 70-90% performance improvement when enabled |
|
|
353
|
+
| `S3DB_CACHE_DRIVER` | `memory` | Cache storage backend | Use `filesystem` for persistent cache | Memory: faster, Filesystem: survives restarts |
|
|
354
|
+
| `S3DB_CACHE_MAX_SIZE` | `1000` | Max cached items (memory only) | Increase for read-heavy workloads | Each item ~1-10KB RAM |
|
|
355
|
+
| `S3DB_CACHE_TTL` | `300000` | Cache lifetime (ms) | Decrease for frequently changing data | 5 min default, 0 = no expiry |
|
|
356
|
+
| `S3DB_CACHE_DIRECTORY` | `./cache` | Filesystem cache location | Use SSD path for best performance | Only for filesystem driver |
|
|
357
|
+
| `S3DB_CACHE_PREFIX` | `s3db` | Cache file prefix | Change for multiple instances | Prevents cache conflicts |
|
|
358
|
+
|
|
359
|
+
#### **Cache Strategy Examples**
|
|
360
|
+
|
|
361
|
+
```bash
|
|
362
|
+
# High-traffic read-heavy API
|
|
363
|
+
S3DB_CACHE_DRIVER=memory
|
|
364
|
+
S3DB_CACHE_MAX_SIZE=5000
|
|
365
|
+
S3DB_CACHE_TTL=600000 # 10 minutes
|
|
366
|
+
|
|
367
|
+
# Data analytics workload
|
|
368
|
+
S3DB_CACHE_DRIVER=filesystem
|
|
369
|
+
S3DB_CACHE_DIRECTORY=/mnt/ssd/cache
|
|
370
|
+
S3DB_CACHE_TTL=3600000 # 1 hour
|
|
371
|
+
|
|
372
|
+
# Real-time application
|
|
373
|
+
S3DB_CACHE_DRIVER=memory
|
|
374
|
+
S3DB_CACHE_MAX_SIZE=500
|
|
375
|
+
S3DB_CACHE_TTL=30000 # 30 seconds
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
#### **Cost Tracking Plugin**
|
|
379
|
+
|
|
380
|
+
| Variable | Default | Description | Use Case |
|
|
381
|
+
|----------|---------|-------------|----------|
|
|
382
|
+
| `S3DB_COSTS_ENABLED` | `true` | Track S3 API costs | Disable for local MinIO/testing |
|
|
383
|
+
|
|
384
|
+
Cost tracking provides:
|
|
385
|
+
- Per-operation cost breakdown
|
|
386
|
+
- Daily/monthly projections
|
|
387
|
+
- Request type statistics
|
|
388
|
+
- Data transfer metrics
|
|
389
|
+
|
|
390
|
+
### ๐ AWS & S3-Compatible Configuration
|
|
391
|
+
|
|
392
|
+
#### **AWS Credentials**
|
|
393
|
+
|
|
394
|
+
| Variable | Default | Description | Priority Order |
|
|
395
|
+
|----------|---------|-------------|----------------|
|
|
396
|
+
| `AWS_ACCESS_KEY_ID` | - | AWS access key | 1. Env var, 2. IAM role, 3. Connection string |
|
|
397
|
+
| `AWS_SECRET_ACCESS_KEY` | - | AWS secret key | Required if using access key |
|
|
398
|
+
| `AWS_SESSION_TOKEN` | - | Temporary credentials | For STS/assumed roles |
|
|
399
|
+
| `AWS_REGION` | `us-east-1` | AWS region | Must match bucket region |
|
|
400
|
+
|
|
401
|
+
#### **S3-Compatible Services**
|
|
402
|
+
|
|
403
|
+
| Variable | Default | Description | Services |
|
|
404
|
+
|----------|---------|-------------|----------|
|
|
405
|
+
| `S3_ENDPOINT` | - | Custom S3 API endpoint | MinIO, DigitalOcean, Backblaze, Wasabi |
|
|
406
|
+
| `S3_FORCE_PATH_STYLE` | `false` | URL style | Required for MinIO, LocalStack |
|
|
407
|
+
|
|
408
|
+
### ๐ Connection String Formats
|
|
409
|
+
|
|
410
|
+
#### **Anatomy of a Connection String**
|
|
411
|
+
|
|
412
|
+
```
|
|
413
|
+
s3://[ACCESS_KEY:SECRET_KEY@]BUCKET[/PATH][?PARAMS]
|
|
414
|
+
```
|
|
415
|
+
|
|
416
|
+
Components:
|
|
417
|
+
- `ACCESS_KEY:SECRET_KEY` - Optional inline credentials
|
|
418
|
+
- `BUCKET` - S3 bucket name
|
|
419
|
+
- `PATH` - Optional path prefix for organization
|
|
420
|
+
- `PARAMS` - Query parameters for advanced config
|
|
421
|
+
|
|
422
|
+
#### **Real-World Examples**
|
|
423
|
+
|
|
424
|
+
```bash
|
|
425
|
+
# AWS S3 - Production with IAM role (recommended)
|
|
426
|
+
S3DB_CONNECTION_STRING="s3://my-prod-bucket/databases/main"
|
|
427
|
+
|
|
428
|
+
# AWS S3 - Development with credentials
|
|
429
|
+
S3DB_CONNECTION_STRING="s3://AKIA...:wJal...@my-dev-bucket/databases/dev"
|
|
430
|
+
|
|
431
|
+
# MinIO - Local development
|
|
432
|
+
S3DB_CONNECTION_STRING="s3://minioadmin:minioadmin123@localhost:17998/s3db?forcePathStyle=true"
|
|
433
|
+
|
|
434
|
+
# DigitalOcean Spaces
|
|
435
|
+
S3DB_CONNECTION_STRING="s3://DO_KEY:DO_SECRET@nyc3.digitaloceanspaces.com/space-name/databases/prod"
|
|
436
|
+
|
|
437
|
+
# Backblaze B2
|
|
438
|
+
S3DB_CONNECTION_STRING="s3://KEY_ID:APP_KEY@s3.us-west-002.backblazeb2.com/bucket-name/db"
|
|
439
|
+
|
|
440
|
+
# Wasabi
|
|
441
|
+
S3DB_CONNECTION_STRING="s3://ACCESS_KEY:SECRET_KEY@s3.wasabisys.com/bucket-name/databases/app"
|
|
442
|
+
|
|
443
|
+
# LocalStack (testing)
|
|
444
|
+
S3DB_CONNECTION_STRING="s3://test:test@localhost:4566/test-bucket/db?forcePathStyle=true"
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
### ๐ Complete Configuration Examples
|
|
448
|
+
|
|
449
|
+
#### **Development Setup (.env)**
|
|
450
|
+
|
|
451
|
+
```bash
|
|
452
|
+
# Server
|
|
453
|
+
NODE_ENV=development
|
|
454
|
+
MCP_SERVER_PORT=17500
|
|
455
|
+
MCP_TRANSPORT=sse
|
|
456
|
+
|
|
457
|
+
# S3DB
|
|
458
|
+
S3DB_CONNECTION_STRING=s3://minioadmin:minioadmin123@localhost:9000/dev-bucket/db
|
|
459
|
+
S3DB_VERBOSE=true
|
|
460
|
+
S3DB_PARALLELISM=5
|
|
461
|
+
|
|
462
|
+
# Cache
|
|
463
|
+
S3DB_CACHE_ENABLED=true
|
|
464
|
+
S3DB_CACHE_DRIVER=memory
|
|
465
|
+
S3DB_CACHE_MAX_SIZE=100
|
|
466
|
+
S3DB_CACHE_TTL=60000
|
|
467
|
+
|
|
468
|
+
# Costs
|
|
469
|
+
S3DB_COSTS_ENABLED=false
|
|
470
|
+
```
|
|
471
|
+
|
|
472
|
+
#### **Production Setup (.env)**
|
|
473
|
+
|
|
474
|
+
```bash
|
|
475
|
+
# Server
|
|
476
|
+
NODE_ENV=production
|
|
477
|
+
MCP_SERVER_PORT=17500
|
|
478
|
+
MCP_TRANSPORT=sse
|
|
479
|
+
|
|
480
|
+
# S3DB (using IAM role)
|
|
481
|
+
S3DB_CONNECTION_STRING=s3://prod-data-bucket/databases/main
|
|
482
|
+
S3DB_VERBOSE=false
|
|
483
|
+
S3DB_PARALLELISM=20
|
|
484
|
+
S3DB_PASSPHRASE=${SECRET_PASSPHRASE}
|
|
485
|
+
S3DB_VERSIONING_ENABLED=true
|
|
486
|
+
|
|
487
|
+
# Cache
|
|
488
|
+
S3DB_CACHE_ENABLED=true
|
|
489
|
+
S3DB_CACHE_DRIVER=filesystem
|
|
490
|
+
S3DB_CACHE_DIRECTORY=/var/cache/s3db
|
|
491
|
+
S3DB_CACHE_TTL=1800000
|
|
492
|
+
S3DB_CACHE_PREFIX=prod
|
|
493
|
+
|
|
494
|
+
# Costs
|
|
495
|
+
S3DB_COSTS_ENABLED=true
|
|
496
|
+
|
|
497
|
+
# AWS
|
|
498
|
+
AWS_REGION=us-east-1
|
|
499
|
+
```
|
|
500
|
+
|
|
501
|
+
### ๐ Command Line Options
|
|
502
|
+
|
|
503
|
+
```bash
|
|
504
|
+
# Basic usage
|
|
505
|
+
npx s3db-mcp-server [OPTIONS]
|
|
506
|
+
|
|
507
|
+
# Transport selection
|
|
508
|
+
npx s3db-mcp-server --transport=sse # Web clients (default)
|
|
509
|
+
npx s3db-mcp-server --transport=stdio # CLI/pipe communication
|
|
510
|
+
|
|
511
|
+
# Network configuration
|
|
512
|
+
npx s3db-mcp-server --host=0.0.0.0 --port=17500
|
|
513
|
+
|
|
514
|
+
# Override environment variables
|
|
515
|
+
npx s3db-mcp-server --transport=sse \
|
|
516
|
+
--host=127.0.0.1 \
|
|
517
|
+
--port=18000
|
|
518
|
+
|
|
519
|
+
# Combined with environment variables
|
|
520
|
+
S3DB_CONNECTION_STRING="s3://..." \
|
|
521
|
+
S3DB_CACHE_DRIVER=filesystem \
|
|
522
|
+
npx s3db-mcp-server --transport=sse
|
|
523
|
+
|
|
524
|
+
# Debug mode with verbose output
|
|
525
|
+
S3DB_VERBOSE=true \
|
|
526
|
+
NODE_ENV=development \
|
|
527
|
+
npx s3db-mcp-server --transport=stdio
|
|
528
|
+
```
|
|
529
|
+
|
|
530
|
+
### ๐ Configuration Validation
|
|
531
|
+
|
|
532
|
+
The server validates configuration on startup and will:
|
|
533
|
+
1. Check for required `S3DB_CONNECTION_STRING`
|
|
534
|
+
2. Test S3 connectivity
|
|
535
|
+
3. Verify bucket permissions
|
|
536
|
+
4. Initialize cache directory (if using filesystem)
|
|
537
|
+
5. Report configuration summary
|
|
538
|
+
|
|
539
|
+
Example startup log:
|
|
540
|
+
```
|
|
541
|
+
S3DB MCP Server v1.0.0 started
|
|
542
|
+
Transport: sse
|
|
543
|
+
Port: 17500
|
|
544
|
+
Cache: memory (1000 items, 5 min TTL)
|
|
545
|
+
Costs tracking: enabled
|
|
546
|
+
Connected to: s3://my-bucket/databases/main
|
|
547
|
+
Ready for connections...
|
|
548
|
+
```
|
|
549
|
+
|
|
550
|
+
---
|
|
551
|
+
|
|
552
|
+
## ๐ ๏ธ Available Tools
|
|
553
|
+
|
|
554
|
+
### Database Management
|
|
555
|
+
|
|
556
|
+
| Tool | Description | Parameters |
|
|
557
|
+
|------|-------------|------------|
|
|
558
|
+
| `dbConnect` | Connect to S3DB database with costs & cache | `connectionString`, `verbose?`, `parallelism?`, `passphrase?`, `versioningEnabled?`, `enableCache?`, `enableCosts?`, `cacheDriver?`, `cacheMaxSize?`, `cacheTtl?`, `cacheDirectory?`, `cachePrefix?` |
|
|
559
|
+
| `dbDisconnect` | Disconnect from database | - |
|
|
560
|
+
| `dbStatus` | Get connection status | - |
|
|
561
|
+
| `dbCreateResource` | Create resource/collection | `name`, `attributes`, `behavior?`, `timestamps?`, `partitions?`, `paranoid?` |
|
|
562
|
+
| `dbListResources` | List all resources | - |
|
|
563
|
+
| `dbGetStats` | Get database statistics (costs, cache, resources) | - |
|
|
564
|
+
| `dbClearCache` | Clear cache data | `resourceName?` (optional - clears all if not provided) |
|
|
565
|
+
|
|
566
|
+
### Document Operations
|
|
567
|
+
|
|
568
|
+
| Tool | Description | Parameters |
|
|
569
|
+
|------|-------------|------------|
|
|
570
|
+
| `resourceInsert` | Insert single document | `resourceName`, `data` |
|
|
571
|
+
| `resourceInsertMany` | Insert multiple documents | `resourceName`, `data[]` |
|
|
572
|
+
| `resourceGet` | Get document by ID | `resourceName`, `id` |
|
|
573
|
+
| `resourceGetMany` | Get multiple documents | `resourceName`, `ids[]` |
|
|
574
|
+
| `resourceUpdate` | Update document | `resourceName`, `id`, `data` |
|
|
575
|
+
| `resourceUpsert` | Insert or update | `resourceName`, `data` |
|
|
576
|
+
| `resourceDelete` | Delete document | `resourceName`, `id` |
|
|
577
|
+
| `resourceDeleteMany` | Delete multiple documents | `resourceName`, `ids[]` |
|
|
578
|
+
|
|
579
|
+
### Query Operations
|
|
580
|
+
|
|
581
|
+
| Tool | Description | Parameters |
|
|
582
|
+
|------|-------------|------------|
|
|
583
|
+
| `resourceExists` | Check if document exists | `resourceName`, `id` |
|
|
584
|
+
| `resourceList` | List with pagination | `resourceName`, `limit?`, `offset?`, `partition?`, `partitionValues?` |
|
|
585
|
+
| `resourceListIds` | List document IDs only | `resourceName`, `limit?`, `offset?` |
|
|
586
|
+
| `resourceCount` | Count documents | `resourceName`, `partition?`, `partitionValues?` |
|
|
587
|
+
| `resourceGetAll` | Get all documents | `resourceName` |
|
|
588
|
+
| `resourceDeleteAll` | Delete all documents | `resourceName`, `confirm: true` |
|
|
589
|
+
|
|
590
|
+
---
|
|
591
|
+
|
|
592
|
+
## ๐ Command Line Examples
|
|
593
|
+
|
|
594
|
+
### ๐ Basic CRUD Operations
|
|
595
|
+
|
|
596
|
+
```bash
|
|
597
|
+
# Connect to database
|
|
598
|
+
"Please connect to the S3DB database using the connection string: s3://my-bucket/databases/app"
|
|
599
|
+
|
|
600
|
+
# Create a blog posts resource
|
|
601
|
+
"Create a resource named 'posts' with fields: title (string, required), content (string), author (string), published (boolean), tags (array of strings)"
|
|
602
|
+
|
|
603
|
+
# Insert a blog post
|
|
604
|
+
"Insert a new post with title 'Getting Started with S3DB', content 'S3DB is amazing...', author 'john-doe', published true, and tags ['tutorial', 's3db']"
|
|
605
|
+
|
|
606
|
+
# Query posts
|
|
607
|
+
"List all published posts"
|
|
608
|
+
"Find posts by author 'john-doe'"
|
|
609
|
+
"Count how many posts have the tag 'tutorial'"
|
|
610
|
+
|
|
611
|
+
# Update a post
|
|
612
|
+
"Update the post with ID 'post-123' to set published to false"
|
|
613
|
+
|
|
614
|
+
# Delete a post
|
|
615
|
+
"Delete the post with ID 'post-456'"
|
|
616
|
+
```
|
|
617
|
+
|
|
618
|
+
### ๐ Advanced Queries with Partitions
|
|
619
|
+
|
|
620
|
+
```bash
|
|
621
|
+
# Create partitioned resource
|
|
622
|
+
"Create an 'orders' resource with:
|
|
623
|
+
- orderId (string, required, unique)
|
|
624
|
+
- customerId (string, required)
|
|
625
|
+
- amount (number, required)
|
|
626
|
+
- status (string: pending, processing, shipped, delivered)
|
|
627
|
+
- region (string)
|
|
628
|
+
- orderDate (date)
|
|
629
|
+
And create partitions by status and region"
|
|
630
|
+
|
|
631
|
+
# Query specific partition
|
|
632
|
+
"Show me all orders with status 'pending' in the 'north' region"
|
|
633
|
+
|
|
634
|
+
# Count by partition
|
|
635
|
+
"Count orders by status"
|
|
636
|
+
|
|
637
|
+
# List with pagination
|
|
638
|
+
"List the first 10 orders, then get the next 10"
|
|
639
|
+
```
|
|
640
|
+
|
|
641
|
+
### ๐ ๏ธ Database Management
|
|
642
|
+
|
|
643
|
+
```bash
|
|
644
|
+
# Check connection status
|
|
645
|
+
"Show me the current database connection status"
|
|
646
|
+
|
|
647
|
+
# List all resources
|
|
648
|
+
"What resources/collections exist in the database?"
|
|
649
|
+
|
|
650
|
+
# Get statistics
|
|
651
|
+
"Show database statistics including cache performance and S3 costs"
|
|
652
|
+
|
|
653
|
+
# Clear cache
|
|
654
|
+
"Clear the cache for the 'users' resource"
|
|
655
|
+
"Clear all cached data"
|
|
656
|
+
|
|
657
|
+
# Bulk operations
|
|
658
|
+
"Insert 100 test users with random data"
|
|
659
|
+
"Delete all users where active is false"
|
|
660
|
+
```
|
|
661
|
+
|
|
662
|
+
### ๐ Reporting and Analytics
|
|
663
|
+
|
|
664
|
+
```bash
|
|
665
|
+
# Cost analysis
|
|
666
|
+
"How much are we spending on S3 operations today?"
|
|
667
|
+
"Show me a breakdown of S3 costs by operation type"
|
|
668
|
+
|
|
669
|
+
# Performance metrics
|
|
670
|
+
"What's the cache hit ratio?"
|
|
671
|
+
"Which queries are slowest?"
|
|
672
|
+
|
|
673
|
+
# Data overview
|
|
674
|
+
"Give me a summary of all resources: total records, size, last modified"
|
|
675
|
+
"Which partitions have the most data?"
|
|
676
|
+
```
|
|
677
|
+
|
|
678
|
+
## ๐ API Usage Examples
|
|
679
|
+
|
|
680
|
+
### Basic CRUD Operations
|
|
681
|
+
|
|
682
|
+
```javascript
|
|
683
|
+
// 1. Connect to database with automatic cache and costs tracking
|
|
684
|
+
await agent.callTool('dbConnect', {
|
|
685
|
+
connectionString: 's3://ACCESS_KEY:SECRET_KEY@bucket/databases/blog',
|
|
686
|
+
verbose: false,
|
|
687
|
+
parallelism: 10,
|
|
688
|
+
enableCache: true, // Cache enabled by default
|
|
689
|
+
enableCosts: true, // Costs tracking enabled by default
|
|
690
|
+
cacheDriver: 'memory', // 'memory' or 'filesystem'
|
|
691
|
+
cacheMaxSize: 1000, // Cache up to 1000 items (memory only)
|
|
692
|
+
cacheTtl: 300000, // 5 minute cache TTL
|
|
693
|
+
cacheDirectory: './cache', // Directory for filesystem cache
|
|
694
|
+
cachePrefix: 's3db' // Prefix for cache files
|
|
695
|
+
});
|
|
696
|
+
|
|
697
|
+
// 2. Create a resource with schema validation
|
|
698
|
+
await agent.callTool('dbCreateResource', {
|
|
699
|
+
name: 'posts',
|
|
700
|
+
attributes: {
|
|
701
|
+
title: 'string|required|min:3|max:200',
|
|
702
|
+
content: 'string|required',
|
|
703
|
+
author: 'string|required',
|
|
704
|
+
tags: 'array|items:string',
|
|
705
|
+
published: 'boolean',
|
|
706
|
+
publishDate: 'date',
|
|
707
|
+
metadata: {
|
|
708
|
+
views: 'number|positive',
|
|
709
|
+
likes: 'number|positive'
|
|
710
|
+
}
|
|
711
|
+
},
|
|
712
|
+
behavior: 'user-managed',
|
|
713
|
+
timestamps: true, // Adds createdAt/updatedAt automatically
|
|
714
|
+
paranoid: true // Soft deletes
|
|
715
|
+
});
|
|
716
|
+
|
|
717
|
+
// 3. Insert a blog post
|
|
718
|
+
const post = await agent.callTool('resourceInsert', {
|
|
719
|
+
resourceName: 'posts',
|
|
720
|
+
data: {
|
|
721
|
+
title: 'Getting Started with S3DB MCP',
|
|
722
|
+
content: 'S3DB transforms AWS S3 into a powerful document database...',
|
|
723
|
+
author: 'john-doe',
|
|
724
|
+
tags: ['tutorial', 's3db', 'mcp', 'ai'],
|
|
725
|
+
published: true,
|
|
726
|
+
publishDate: '2024-01-15',
|
|
727
|
+
metadata: {
|
|
728
|
+
views: 0,
|
|
729
|
+
likes: 0
|
|
730
|
+
}
|
|
731
|
+
}
|
|
732
|
+
});
|
|
733
|
+
|
|
734
|
+
// 4. Update the post
|
|
735
|
+
await agent.callTool('resourceUpdate', {
|
|
736
|
+
resourceName: 'posts',
|
|
737
|
+
id: post.data.id,
|
|
738
|
+
data: {
|
|
739
|
+
metadata: {
|
|
740
|
+
views: 150,
|
|
741
|
+
likes: 12
|
|
742
|
+
}
|
|
743
|
+
}
|
|
744
|
+
});
|
|
745
|
+
|
|
746
|
+
// 5. Query posts with pagination
|
|
747
|
+
const posts = await agent.callTool('resourceList', {
|
|
748
|
+
resourceName: 'posts',
|
|
749
|
+
limit: 10,
|
|
750
|
+
offset: 0
|
|
751
|
+
});
|
|
752
|
+
|
|
753
|
+
// 6. Check if post exists
|
|
754
|
+
const exists = await agent.callTool('resourceExists', {
|
|
755
|
+
resourceName: 'posts',
|
|
756
|
+
id: post.data.id
|
|
757
|
+
});
|
|
758
|
+
|
|
759
|
+
// 7. Check performance statistics
|
|
760
|
+
const stats = await agent.callTool('dbGetStats');
|
|
761
|
+
console.log('Cache hits:', stats.stats.cache.size);
|
|
762
|
+
console.log('S3 costs:', stats.stats.costs.estimatedCostUSD);
|
|
763
|
+
console.log('Total requests:', stats.stats.costs.totalRequests);
|
|
764
|
+
|
|
765
|
+
// 8. Clear cache if needed
|
|
766
|
+
await agent.callTool('dbClearCache', {
|
|
767
|
+
resourceName: 'posts' // Clear cache for specific resource
|
|
768
|
+
});
|
|
769
|
+
```
|
|
770
|
+
|
|
771
|
+
### Performance & Costs Monitoring
|
|
772
|
+
|
|
773
|
+
```javascript
|
|
774
|
+
// Monitor database performance and costs
|
|
775
|
+
const stats = await agent.callTool('dbGetStats');
|
|
776
|
+
|
|
777
|
+
console.log('Database Stats:', {
|
|
778
|
+
resources: stats.stats.database.resources,
|
|
779
|
+
totalCosts: `$${stats.stats.costs.estimatedCostUSD.toFixed(6)}`,
|
|
780
|
+
cacheHitRate: `${stats.stats.cache.keyCount}/${stats.stats.cache.maxSize}`,
|
|
781
|
+
s3Operations: stats.stats.costs.requestsByType
|
|
782
|
+
});
|
|
783
|
+
|
|
784
|
+
// Cache performance
|
|
785
|
+
if (stats.stats.cache.enabled) {
|
|
786
|
+
console.log('Cache Performance:', {
|
|
787
|
+
driver: stats.stats.cache.driver,
|
|
788
|
+
itemsCached: stats.stats.cache.size,
|
|
789
|
+
maxCapacity: stats.stats.cache.maxSize,
|
|
790
|
+
ttl: `${stats.stats.cache.ttl / 1000}s`,
|
|
791
|
+
sampleKeys: stats.stats.cache.sampleKeys
|
|
792
|
+
});
|
|
793
|
+
}
|
|
794
|
+
|
|
795
|
+
// S3 costs breakdown
|
|
796
|
+
if (stats.stats.costs) {
|
|
797
|
+
console.log('S3 Costs Breakdown:', {
|
|
798
|
+
totalRequests: stats.stats.costs.totalRequests,
|
|
799
|
+
getRequests: stats.stats.costs.requestsByType.get,
|
|
800
|
+
putRequests: stats.stats.costs.requestsByType.put,
|
|
801
|
+
listRequests: stats.stats.costs.requestsByType.list,
|
|
802
|
+
estimatedCost: `$${stats.stats.costs.estimatedCostUSD.toFixed(6)}`
|
|
803
|
+
});
|
|
804
|
+
}
|
|
805
|
+
|
|
806
|
+
// Clear cache for performance reset
|
|
807
|
+
await agent.callTool('dbClearCache'); // Clear all cache
|
|
808
|
+
// or
|
|
809
|
+
await agent.callTool('dbClearCache', { resourceName: 'posts' }); // Clear specific resource
|
|
810
|
+
```
|
|
811
|
+
|
|
812
|
+
### Batch Operations
|
|
813
|
+
|
|
814
|
+
```javascript
|
|
815
|
+
// Insert multiple documents at once
|
|
816
|
+
await agent.callTool('resourceInsertMany', {
|
|
817
|
+
resourceName: 'posts',
|
|
818
|
+
data: [
|
|
819
|
+
{
|
|
820
|
+
title: 'AI and Databases',
|
|
821
|
+
content: 'Exploring the intersection...',
|
|
822
|
+
author: 'jane-smith',
|
|
823
|
+
published: true
|
|
824
|
+
},
|
|
825
|
+
{
|
|
826
|
+
title: 'S3DB Performance Tips',
|
|
827
|
+
content: 'Best practices for...',
|
|
828
|
+
author: 'bob-wilson',
|
|
829
|
+
published: false
|
|
830
|
+
}
|
|
831
|
+
]
|
|
832
|
+
});
|
|
833
|
+
|
|
834
|
+
// Get multiple documents by ID
|
|
835
|
+
const multiplePosts = await agent.callTool('resourceGetMany', {
|
|
836
|
+
resourceName: 'posts',
|
|
837
|
+
ids: ['post_123', 'post_456', 'post_789']
|
|
838
|
+
});
|
|
839
|
+
|
|
840
|
+
// Delete multiple documents
|
|
841
|
+
await agent.callTool('resourceDeleteMany', {
|
|
842
|
+
resourceName: 'posts',
|
|
843
|
+
ids: ['post_old1', 'post_old2']
|
|
844
|
+
});
|
|
845
|
+
```
|
|
846
|
+
|
|
847
|
+
### E-commerce Example with Complex Schema
|
|
848
|
+
|
|
849
|
+
```javascript
|
|
850
|
+
// Create products resource
|
|
851
|
+
await agent.callTool('dbCreateResource', {
|
|
852
|
+
name: 'products',
|
|
853
|
+
attributes: {
|
|
854
|
+
sku: 'string|required|unique',
|
|
855
|
+
name: 'string|required|min:2|max:200',
|
|
856
|
+
description: 'string|required',
|
|
857
|
+
price: 'number|positive|required',
|
|
858
|
+
category: 'string|required',
|
|
859
|
+
subcategory: 'string|optional',
|
|
860
|
+
inStock: 'boolean',
|
|
861
|
+
inventory: {
|
|
862
|
+
quantity: 'number|integer|min:0',
|
|
863
|
+
reserved: 'number|integer|min:0',
|
|
864
|
+
warehouse: 'string|required'
|
|
865
|
+
},
|
|
866
|
+
specifications: {
|
|
867
|
+
weight: 'number|positive|optional',
|
|
868
|
+
dimensions: {
|
|
869
|
+
length: 'number|positive',
|
|
870
|
+
width: 'number|positive',
|
|
871
|
+
height: 'number|positive'
|
|
872
|
+
},
|
|
873
|
+
color: 'string|optional',
|
|
874
|
+
material: 'string|optional'
|
|
875
|
+
},
|
|
876
|
+
pricing: {
|
|
877
|
+
cost: 'number|positive',
|
|
878
|
+
markup: 'number|positive',
|
|
879
|
+
discountPercent: 'number|min:0|max:100'
|
|
880
|
+
},
|
|
881
|
+
tags: 'array|items:string',
|
|
882
|
+
images: 'array|items:url'
|
|
883
|
+
},
|
|
884
|
+
partitions: {
|
|
885
|
+
byCategory: {
|
|
886
|
+
fields: { category: 'string' },
|
|
887
|
+
description: 'Partition products by main category'
|
|
888
|
+
},
|
|
889
|
+
byCategoryAndSubcategory: {
|
|
890
|
+
fields: {
|
|
891
|
+
category: 'string',
|
|
892
|
+
subcategory: 'string'
|
|
893
|
+
},
|
|
894
|
+
description: 'Fine-grained category partitioning'
|
|
895
|
+
}
|
|
896
|
+
},
|
|
897
|
+
timestamps: true
|
|
898
|
+
});
|
|
899
|
+
|
|
900
|
+
// Insert a complex product
|
|
901
|
+
await agent.callTool('resourceInsert', {
|
|
902
|
+
resourceName: 'products',
|
|
903
|
+
data: {
|
|
904
|
+
sku: 'LAP-GAMING-001',
|
|
905
|
+
name: 'Gaming Laptop Pro 15"',
|
|
906
|
+
description: 'High-performance gaming laptop with RTX graphics',
|
|
907
|
+
price: 1299.99,
|
|
908
|
+
category: 'electronics',
|
|
909
|
+
subcategory: 'laptops',
|
|
910
|
+
inStock: true,
|
|
911
|
+
inventory: {
|
|
912
|
+
quantity: 25,
|
|
913
|
+
reserved: 3,
|
|
914
|
+
warehouse: 'US-WEST-1'
|
|
915
|
+
},
|
|
916
|
+
specifications: {
|
|
917
|
+
weight: 2.3,
|
|
918
|
+
dimensions: {
|
|
919
|
+
length: 35.5,
|
|
920
|
+
width: 25.0,
|
|
921
|
+
height: 2.2
|
|
922
|
+
},
|
|
923
|
+
color: 'black',
|
|
924
|
+
material: 'aluminum'
|
|
925
|
+
},
|
|
926
|
+
pricing: {
|
|
927
|
+
cost: 850.00,
|
|
928
|
+
markup: 0.53,
|
|
929
|
+
discountPercent: 0
|
|
930
|
+
},
|
|
931
|
+
tags: ['gaming', 'laptop', 'rtx', 'high-performance'],
|
|
932
|
+
images: [
|
|
933
|
+
'https://example.com/laptop-1.jpg',
|
|
934
|
+
'https://example.com/laptop-2.jpg'
|
|
935
|
+
]
|
|
936
|
+
}
|
|
937
|
+
});
|
|
938
|
+
```
|
|
939
|
+
|
|
940
|
+
---
|
|
941
|
+
|
|
942
|
+
## ๐๏ธ Partitions & Performance
|
|
943
|
+
|
|
944
|
+
Partitions organize data for better performance and logical separation.
|
|
945
|
+
|
|
946
|
+
### Creating Partitioned Resources
|
|
947
|
+
|
|
948
|
+
```javascript
|
|
949
|
+
await agent.callTool('dbCreateResource', {
|
|
950
|
+
name: 'orders',
|
|
951
|
+
attributes: {
|
|
952
|
+
orderId: 'string|required|unique',
|
|
953
|
+
customerId: 'string|required',
|
|
954
|
+
amount: 'number|positive|required',
|
|
955
|
+
status: 'string|enum:pending,paid,shipped,delivered,cancelled',
|
|
956
|
+
region: 'string|required',
|
|
957
|
+
orderDate: 'date|required',
|
|
958
|
+
items: 'array|items:object'
|
|
959
|
+
},
|
|
960
|
+
partitions: {
|
|
961
|
+
// Single field partitions
|
|
962
|
+
byRegion: {
|
|
963
|
+
fields: { region: 'string' },
|
|
964
|
+
description: 'Geographic distribution'
|
|
965
|
+
},
|
|
966
|
+
byStatus: {
|
|
967
|
+
fields: { status: 'string' },
|
|
968
|
+
description: 'Order status tracking'
|
|
969
|
+
},
|
|
970
|
+
byMonth: {
|
|
971
|
+
fields: { orderDate: 'date|maxlength:7' }, // YYYY-MM format
|
|
972
|
+
description: 'Monthly order archives'
|
|
973
|
+
},
|
|
974
|
+
|
|
975
|
+
// Multi-field partitions
|
|
976
|
+
byRegionAndStatus: {
|
|
977
|
+
fields: {
|
|
978
|
+
region: 'string',
|
|
979
|
+
status: 'string'
|
|
980
|
+
},
|
|
981
|
+
description: 'Regional status tracking'
|
|
982
|
+
},
|
|
983
|
+
byRegionAndMonth: {
|
|
984
|
+
fields: {
|
|
985
|
+
region: 'string',
|
|
986
|
+
orderDate: 'date|maxlength:7'
|
|
987
|
+
},
|
|
988
|
+
description: 'Regional monthly reports'
|
|
989
|
+
}
|
|
990
|
+
},
|
|
991
|
+
timestamps: true
|
|
992
|
+
});
|
|
993
|
+
```
|
|
994
|
+
|
|
995
|
+
### Querying with Partitions
|
|
996
|
+
|
|
997
|
+
```javascript
|
|
998
|
+
// Query specific partition - much faster than full scan
|
|
999
|
+
const northernOrders = await agent.callTool('resourceList', {
|
|
1000
|
+
resourceName: 'orders',
|
|
1001
|
+
partition: 'byRegion',
|
|
1002
|
+
partitionValues: { region: 'north' },
|
|
1003
|
+
limit: 100
|
|
1004
|
+
});
|
|
1005
|
+
|
|
1006
|
+
// Multi-field partition query
|
|
1007
|
+
const northPendingOrders = await agent.callTool('resourceList', {
|
|
1008
|
+
resourceName: 'orders',
|
|
1009
|
+
partition: 'byRegionAndStatus',
|
|
1010
|
+
partitionValues: {
|
|
1011
|
+
region: 'north',
|
|
1012
|
+
status: 'pending'
|
|
1013
|
+
}
|
|
1014
|
+
});
|
|
1015
|
+
|
|
1016
|
+
// Time-based partition query
|
|
1017
|
+
const januaryOrders = await agent.callTool('resourceList', {
|
|
1018
|
+
resourceName: 'orders',
|
|
1019
|
+
partition: 'byMonth',
|
|
1020
|
+
partitionValues: { orderDate: '2024-01' }
|
|
1021
|
+
});
|
|
1022
|
+
|
|
1023
|
+
// Count documents in partition
|
|
1024
|
+
const pendingCount = await agent.callTool('resourceCount', {
|
|
1025
|
+
resourceName: 'orders',
|
|
1026
|
+
partition: 'byStatus',
|
|
1027
|
+
partitionValues: { status: 'pending' }
|
|
1028
|
+
});
|
|
1029
|
+
```
|
|
1030
|
+
|
|
1031
|
+
### Automatic Partition Migration (v9.2.2+)
|
|
1032
|
+
|
|
1033
|
+
**๐ฏ NEW FEATURE**: Records automatically move between partitions when you update partition fields!
|
|
1034
|
+
|
|
1035
|
+
```javascript
|
|
1036
|
+
// 1. Insert order with status 'pending' - goes to 'pending' partition
|
|
1037
|
+
const order = await agent.callTool('resourceInsert', {
|
|
1038
|
+
resourceName: 'orders',
|
|
1039
|
+
data: {
|
|
1040
|
+
orderId: 'ORD-001',
|
|
1041
|
+
customerId: 'CUST-123',
|
|
1042
|
+
amount: 299.99,
|
|
1043
|
+
status: 'pending', // Goes to 'pending' partition
|
|
1044
|
+
region: 'north'
|
|
1045
|
+
}
|
|
1046
|
+
});
|
|
1047
|
+
|
|
1048
|
+
// 2. Update status to 'shipped' - AUTOMATICALLY moves to 'shipped' partition!
|
|
1049
|
+
await agent.callTool('resourceUpdate', {
|
|
1050
|
+
resourceName: 'orders',
|
|
1051
|
+
id: order.id,
|
|
1052
|
+
data: {
|
|
1053
|
+
...order,
|
|
1054
|
+
status: 'shipped' // Automatically moved from 'pending' to 'shipped' partition
|
|
1055
|
+
}
|
|
1056
|
+
});
|
|
1057
|
+
|
|
1058
|
+
// The record is now:
|
|
1059
|
+
// โ
In the 'shipped' partition
|
|
1060
|
+
// โ NOT in the 'pending' partition anymore (automatically cleaned up!)
|
|
1061
|
+
```
|
|
1062
|
+
|
|
1063
|
+
### Partition Best Practices
|
|
1064
|
+
|
|
1065
|
+
**Common Partition Patterns:**
|
|
1066
|
+
- **By Date**: `{ orderDate: 'date|maxlength:10' }` (YYYY-MM-DD)
|
|
1067
|
+
- **By Month**: `{ orderDate: 'date|maxlength:7' }` (YYYY-MM)
|
|
1068
|
+
- **By Category**: `{ category: 'string' }`
|
|
1069
|
+
- **By User**: `{ userId: 'string' }`
|
|
1070
|
+
- **By Status**: `{ status: 'string' }`
|
|
1071
|
+
- **By Geographic Region**: `{ region: 'string', country: 'string' }`
|
|
1072
|
+
|
|
1073
|
+
**Performance Benefits:**
|
|
1074
|
+
- โก **Faster queries** - scans only relevant partition
|
|
1075
|
+
- ๐ฐ **Lower S3 costs** - fewer requests and data transfer
|
|
1076
|
+
- ๐ **Better analytics** - efficient aggregations
|
|
1077
|
+
- ๐ **Easier maintenance** - targeted operations
|
|
1078
|
+
|
|
1079
|
+
---
|
|
1080
|
+
|
|
1081
|
+
## ๐ณ Docker Deployment
|
|
1082
|
+
|
|
1083
|
+
### Quick Start with Docker Compose
|
|
1084
|
+
|
|
1085
|
+
```bash
|
|
1086
|
+
# 1. Create project directory
|
|
1087
|
+
mkdir s3db-mcp && cd s3db-mcp
|
|
1088
|
+
|
|
1089
|
+
# 2. Create docker-compose.yml
|
|
1090
|
+
curl -o docker-compose.yml https://raw.githubusercontent.com/forattini-dev/s3db.js/main/mcp-server/docker-compose.yml
|
|
1091
|
+
|
|
1092
|
+
# 3. Create .env file
|
|
1093
|
+
curl -o .env.example https://raw.githubusercontent.com/forattini-dev/s3db.js/main/mcp-server/.env.example
|
|
1094
|
+
cp .env.example .env
|
|
1095
|
+
|
|
1096
|
+
# 4. Edit .env with your configuration
|
|
1097
|
+
# 5. Start services
|
|
1098
|
+
docker compose up
|
|
1099
|
+
```
|
|
1100
|
+
|
|
1101
|
+
### Production Docker Setup
|
|
1102
|
+
|
|
1103
|
+
```yaml
|
|
1104
|
+
services:
|
|
1105
|
+
s3db-mcp-server:
|
|
1106
|
+
image: s3db-mcp-server:latest
|
|
1107
|
+
restart: unless-stopped
|
|
1108
|
+
environment:
|
|
1109
|
+
- NODE_ENV=production
|
|
1110
|
+
- S3DB_CONNECTION_STRING=s3://bucket/databases/prod
|
|
1111
|
+
- MCP_TRANSPORT=sse
|
|
1112
|
+
- MCP_SERVER_PORT=17500
|
|
1113
|
+
ports:
|
|
1114
|
+
- "17500:8000"
|
|
1115
|
+
healthcheck:
|
|
1116
|
+
test: ["CMD", "curl", "-f", "http://localhost:8001/health"]
|
|
1117
|
+
interval: 30s
|
|
1118
|
+
timeout: 10s
|
|
1119
|
+
retries: 3
|
|
1120
|
+
logging:
|
|
1121
|
+
driver: "json-file"
|
|
1122
|
+
options:
|
|
1123
|
+
max-size: "10m"
|
|
1124
|
+
max-file: "3"
|
|
1125
|
+
```
|
|
1126
|
+
|
|
1127
|
+
### Local Development with MinIO
|
|
1128
|
+
|
|
1129
|
+
```bash
|
|
1130
|
+
# Start with MinIO for local S3 testing
|
|
1131
|
+
docker compose --profile local-testing up
|
|
1132
|
+
|
|
1133
|
+
# Access:
|
|
1134
|
+
# - MCP Server: http://localhost:17500/sse
|
|
1135
|
+
# - MinIO Console: http://localhost:9001 (minioadmin/minioadmin)
|
|
1136
|
+
# - Health Check: http://localhost:8001/health
|
|
1137
|
+
```
|
|
1138
|
+
|
|
1139
|
+
### Docker Environment Variables
|
|
1140
|
+
|
|
1141
|
+
All the configuration variables mentioned above can be used in Docker:
|
|
1142
|
+
|
|
1143
|
+
```bash
|
|
1144
|
+
docker run -p 17500:8000 \
|
|
1145
|
+
-e S3DB_CONNECTION_STRING="s3://key:secret@bucket/db" \
|
|
1146
|
+
-e S3DB_VERBOSE=true \
|
|
1147
|
+
-e S3DB_PARALLELISM=20 \
|
|
1148
|
+
-e MCP_TRANSPORT=sse \
|
|
1149
|
+
s3db-mcp-server
|
|
1150
|
+
```
|
|
1151
|
+
|
|
1152
|
+
---
|
|
1153
|
+
|
|
1154
|
+
## ๐ค AI Agent Integration
|
|
1155
|
+
|
|
1156
|
+
### Claude Desktop Integration
|
|
1157
|
+
|
|
1158
|
+
1. **Locate config file:**
|
|
1159
|
+
- macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
|
|
1160
|
+
- Windows: `%APPDATA%\Claude\claude_desktop_config.json`
|
|
1161
|
+
|
|
1162
|
+
2. **Add S3DB MCP server:**
|
|
1163
|
+
```json
|
|
1164
|
+
{
|
|
1165
|
+
"mcpServers": {
|
|
1166
|
+
"s3db": {
|
|
1167
|
+
"transport": "sse",
|
|
1168
|
+
"url": "http://localhost:17500/sse"
|
|
1169
|
+
}
|
|
1170
|
+
}
|
|
1171
|
+
}
|
|
1172
|
+
```
|
|
1173
|
+
|
|
1174
|
+
3. **For STDIO transport:**
|
|
1175
|
+
```json
|
|
1176
|
+
{
|
|
1177
|
+
"mcpServers": {
|
|
1178
|
+
"s3db": {
|
|
1179
|
+
"transport": "stdio",
|
|
1180
|
+
"command": "npx",
|
|
1181
|
+
"args": ["s3db-mcp-server", "--transport=stdio"],
|
|
1182
|
+
"env": {
|
|
1183
|
+
"S3DB_CONNECTION_STRING": "s3://bucket/databases/myapp"
|
|
1184
|
+
}
|
|
1185
|
+
}
|
|
1186
|
+
}
|
|
1187
|
+
}
|
|
1188
|
+
```
|
|
1189
|
+
|
|
1190
|
+
### Cursor IDE Integration
|
|
1191
|
+
|
|
1192
|
+
Add to your MCP settings:
|
|
1193
|
+
```json
|
|
1194
|
+
{
|
|
1195
|
+
"mcpServers": {
|
|
1196
|
+
"s3db": {
|
|
1197
|
+
"url": "http://localhost:17500/sse"
|
|
1198
|
+
}
|
|
1199
|
+
}
|
|
1200
|
+
}
|
|
1201
|
+
```
|
|
1202
|
+
|
|
1203
|
+
### AI Agent Usage Rules
|
|
1204
|
+
|
|
1205
|
+
**Before any task:**
|
|
1206
|
+
1. Always use `dbConnect` first to establish connection (cache and costs tracking are enabled by default)
|
|
1207
|
+
2. Use `dbStatus` to verify connection and see resources
|
|
1208
|
+
3. Use `dbListResources` to see available collections
|
|
1209
|
+
|
|
1210
|
+
**For data operations:**
|
|
1211
|
+
1. Use `resourceExists` to check if documents exist before operations
|
|
1212
|
+
2. Prefer batch operations (`resourceInsertMany`, `resourceGetMany`) for efficiency
|
|
1213
|
+
3. Use partitions for performance when querying large datasets
|
|
1214
|
+
4. Always use pagination (`resourceList` with `limit`/`offset`) for large results
|
|
1215
|
+
|
|
1216
|
+
**Schema design:**
|
|
1217
|
+
- Define validation rules: `"email": "email|required|unique"`
|
|
1218
|
+
- Use nested objects for complex data structures
|
|
1219
|
+
- Enable timestamps for audit trails
|
|
1220
|
+
- Consider partitioning strategy upfront
|
|
1221
|
+
|
|
1222
|
+
**Performance monitoring:**
|
|
1223
|
+
- Use `dbGetStats` to monitor S3 costs and cache performance
|
|
1224
|
+
- Cache is automatically enabled for read operations (get, list, count)
|
|
1225
|
+
- Use `dbClearCache` to reset cache when needed
|
|
1226
|
+
- Monitor costs to optimize S3 usage patterns
|
|
1227
|
+
|
|
1228
|
+
**Error handling:**
|
|
1229
|
+
- Check connection status before operations
|
|
1230
|
+
- Validate data structure matches schema
|
|
1231
|
+
- Handle resource not found errors gracefully
|
|
1232
|
+
- Use appropriate error messages for users
|
|
1233
|
+
|
|
1234
|
+
---
|
|
1235
|
+
|
|
1236
|
+
## ๐ง Advanced Usage
|
|
1237
|
+
|
|
1238
|
+
### ๐ Auto-restart on Failure
|
|
1239
|
+
|
|
1240
|
+
Using systemd (Linux):
|
|
1241
|
+
```ini
|
|
1242
|
+
# /etc/systemd/system/s3db-mcp.service
|
|
1243
|
+
[Unit]
|
|
1244
|
+
Description=S3DB MCP Server
|
|
1245
|
+
After=network.target
|
|
1246
|
+
|
|
1247
|
+
[Service]
|
|
1248
|
+
Type=simple
|
|
1249
|
+
User=youruser
|
|
1250
|
+
WorkingDirectory=/home/youruser
|
|
1251
|
+
ExecStart=/usr/bin/npx s3db-mcp-server --transport=sse
|
|
1252
|
+
Restart=always
|
|
1253
|
+
RestartSec=10
|
|
1254
|
+
Environment="S3DB_CONNECTION_STRING=s3://..."
|
|
1255
|
+
|
|
1256
|
+
[Install]
|
|
1257
|
+
WantedBy=multi-user.target
|
|
1258
|
+
```
|
|
1259
|
+
|
|
1260
|
+
```bash
|
|
1261
|
+
sudo systemctl enable s3db-mcp
|
|
1262
|
+
sudo systemctl start s3db-mcp
|
|
1263
|
+
sudo systemctl status s3db-mcp
|
|
1264
|
+
```
|
|
1265
|
+
|
|
1266
|
+
### ๐ Secure Credential Management
|
|
1267
|
+
|
|
1268
|
+
Instead of hardcoding credentials:
|
|
1269
|
+
|
|
1270
|
+
**Option 1: Use AWS Credentials File**
|
|
1271
|
+
```json
|
|
1272
|
+
{
|
|
1273
|
+
"mcpServers": {
|
|
1274
|
+
"s3db": {
|
|
1275
|
+
"command": "npx",
|
|
1276
|
+
"args": ["s3db-mcp-server", "--transport=sse"],
|
|
1277
|
+
"env": {
|
|
1278
|
+
"S3DB_CONNECTION_STRING": "s3://bucket/databases/app",
|
|
1279
|
+
"AWS_PROFILE": "production"
|
|
1280
|
+
}
|
|
1281
|
+
}
|
|
1282
|
+
}
|
|
1283
|
+
}
|
|
1284
|
+
```
|
|
1285
|
+
|
|
1286
|
+
**Option 2: Use Environment Variables**
|
|
1287
|
+
```bash
|
|
1288
|
+
# Set in your shell profile
|
|
1289
|
+
export S3DB_CONNECTION_STRING="s3://..."
|
|
1290
|
+
export AWS_ACCESS_KEY_ID="..."
|
|
1291
|
+
export AWS_SECRET_ACCESS_KEY="..."
|
|
1292
|
+
```
|
|
1293
|
+
|
|
1294
|
+
```json
|
|
1295
|
+
{
|
|
1296
|
+
"mcpServers": {
|
|
1297
|
+
"s3db": {
|
|
1298
|
+
"command": "npx",
|
|
1299
|
+
"args": ["s3db-mcp-server", "--transport=sse"]
|
|
1300
|
+
// No env section - uses system environment
|
|
1301
|
+
}
|
|
1302
|
+
}
|
|
1303
|
+
}
|
|
1304
|
+
```
|
|
1305
|
+
|
|
1306
|
+
### ๐ฏ Performance Optimization
|
|
1307
|
+
|
|
1308
|
+
For production workloads:
|
|
1309
|
+
```json
|
|
1310
|
+
{
|
|
1311
|
+
"mcpServers": {
|
|
1312
|
+
"s3db-optimized": {
|
|
1313
|
+
"command": "npx",
|
|
1314
|
+
"args": ["s3db-mcp-server", "--transport=sse"],
|
|
1315
|
+
"env": {
|
|
1316
|
+
"S3DB_PARALLELISM": "20",
|
|
1317
|
+
"S3DB_CACHE_DRIVER": "filesystem",
|
|
1318
|
+
"S3DB_CACHE_DIRECTORY": "/mnt/ssd/s3db-cache",
|
|
1319
|
+
"S3DB_CACHE_MAX_SIZE": "10000",
|
|
1320
|
+
"S3DB_CACHE_TTL": "3600000",
|
|
1321
|
+
"NODE_ENV": "production"
|
|
1322
|
+
}
|
|
1323
|
+
}
|
|
1324
|
+
}
|
|
1325
|
+
}
|
|
1326
|
+
```
|
|
1327
|
+
|
|
1328
|
+
---
|
|
1329
|
+
|
|
1330
|
+
## ๐ Security
|
|
1331
|
+
|
|
1332
|
+
### AWS IAM Policy
|
|
1333
|
+
|
|
1334
|
+
Minimal S3 permissions required:
|
|
1335
|
+
|
|
1336
|
+
```json
|
|
1337
|
+
{
|
|
1338
|
+
"Version": "2012-10-17",
|
|
1339
|
+
"Statement": [
|
|
1340
|
+
{
|
|
1341
|
+
"Effect": "Allow",
|
|
1342
|
+
"Action": [
|
|
1343
|
+
"s3:GetObject",
|
|
1344
|
+
"s3:PutObject",
|
|
1345
|
+
"s3:DeleteObject",
|
|
1346
|
+
"s3:ListBucket",
|
|
1347
|
+
"s3:HeadObject"
|
|
1348
|
+
],
|
|
1349
|
+
"Resource": [
|
|
1350
|
+
"arn:aws:s3:::your-s3db-bucket",
|
|
1351
|
+
"arn:aws:s3:::your-s3db-bucket/*"
|
|
1352
|
+
]
|
|
1353
|
+
}
|
|
1354
|
+
]
|
|
1355
|
+
}
|
|
1356
|
+
```
|
|
1357
|
+
|
|
1358
|
+
### Security Best Practices
|
|
1359
|
+
|
|
1360
|
+
1. **Use IAM roles** when possible instead of access keys
|
|
1361
|
+
2. **Rotate credentials** regularly
|
|
1362
|
+
3. **Use environment variables** never hardcode credentials
|
|
1363
|
+
4. **Enable S3 bucket encryption** and versioning
|
|
1364
|
+
5. **Monitor access logs** and set up CloudTrail
|
|
1365
|
+
6. **Use strong passphrases** for S3DB encryption
|
|
1366
|
+
7. **Restrict network access** with security groups
|
|
1367
|
+
8. **Enable HTTPS** for all connections
|
|
1368
|
+
|
|
1369
|
+
### Field-Level Encryption
|
|
1370
|
+
|
|
1371
|
+
```javascript
|
|
1372
|
+
await agent.callTool('dbCreateResource', {
|
|
1373
|
+
name: 'sensitive_data',
|
|
1374
|
+
attributes: {
|
|
1375
|
+
publicInfo: 'string',
|
|
1376
|
+
privateData: 'secret', // Automatically encrypted
|
|
1377
|
+
ssn: 'secret', // Encrypted with passphrase
|
|
1378
|
+
creditCard: 'secret' // Encrypted
|
|
1379
|
+
}
|
|
1380
|
+
});
|
|
1381
|
+
```
|
|
1382
|
+
|
|
1383
|
+
---
|
|
1384
|
+
|
|
1385
|
+
## ๐จ Troubleshooting
|
|
1386
|
+
|
|
1387
|
+
### โ Common Issues and Solutions
|
|
1388
|
+
|
|
1389
|
+
#### 1. MCP Server Not Starting
|
|
1390
|
+
|
|
1391
|
+
**Issue**: Claude Desktop can't connect to MCP server
|
|
1392
|
+
|
|
1393
|
+
**Solution**:
|
|
1394
|
+
```bash
|
|
1395
|
+
# Check if port is already in use
|
|
1396
|
+
lsof -i :17500 # macOS/Linux
|
|
1397
|
+
netstat -an | findstr :17500 # Windows
|
|
1398
|
+
|
|
1399
|
+
# Use a different port
|
|
1400
|
+
{
|
|
1401
|
+
"mcpServers": {
|
|
1402
|
+
"s3db": {
|
|
1403
|
+
"command": "npx",
|
|
1404
|
+
"args": ["s3db-mcp-server", "--transport=sse", "--port=18000"],
|
|
1405
|
+
"env": {
|
|
1406
|
+
"MCP_SERVER_PORT": "18000"
|
|
1407
|
+
}
|
|
1408
|
+
}
|
|
1409
|
+
}
|
|
1410
|
+
}
|
|
1411
|
+
```
|
|
1412
|
+
|
|
1413
|
+
#### 2. Connection String Issues
|
|
1414
|
+
|
|
1415
|
+
**Issue**: "Invalid connection string" error
|
|
1416
|
+
|
|
1417
|
+
**Solution**:
|
|
1418
|
+
```bash
|
|
1419
|
+
# Test connection string directly
|
|
1420
|
+
export S3DB_CONNECTION_STRING="s3://..."
|
|
1421
|
+
npx s3db-mcp-server --transport=sse
|
|
1422
|
+
|
|
1423
|
+
# Common fixes:
|
|
1424
|
+
# 1. URL-encode special characters in credentials
|
|
1425
|
+
# 2. Add forcePathStyle=true for MinIO
|
|
1426
|
+
# 3. Ensure bucket exists
|
|
1427
|
+
```
|
|
1428
|
+
|
|
1429
|
+
#### 3. Permission Errors
|
|
1430
|
+
|
|
1431
|
+
**Issue**: "Access Denied" when accessing S3
|
|
1432
|
+
|
|
1433
|
+
**Solution**:
|
|
1434
|
+
```json
|
|
1435
|
+
{
|
|
1436
|
+
"env": {
|
|
1437
|
+
// Ensure credentials are correct
|
|
1438
|
+
"AWS_ACCESS_KEY_ID": "your-key",
|
|
1439
|
+
"AWS_SECRET_ACCESS_KEY": "your-secret",
|
|
1440
|
+
"AWS_REGION": "correct-region"
|
|
1441
|
+
}
|
|
1442
|
+
}
|
|
1443
|
+
```
|
|
1444
|
+
|
|
1445
|
+
#### 4. Cache Not Working
|
|
1446
|
+
|
|
1447
|
+
**Issue**: Slow performance, no cache hits
|
|
1448
|
+
|
|
1449
|
+
**Solution**:
|
|
1450
|
+
```json
|
|
1451
|
+
{
|
|
1452
|
+
"env": {
|
|
1453
|
+
"S3DB_CACHE_ENABLED": "true", // Must be string "true"
|
|
1454
|
+
"S3DB_CACHE_DRIVER": "memory",
|
|
1455
|
+
"S3DB_CACHE_MAX_SIZE": "5000", // Increase size
|
|
1456
|
+
"S3DB_CACHE_TTL": "600000" // 10 minutes
|
|
1457
|
+
}
|
|
1458
|
+
}
|
|
1459
|
+
```
|
|
1460
|
+
|
|
1461
|
+
### ๐ Debugging
|
|
1462
|
+
|
|
1463
|
+
Enable verbose logging:
|
|
1464
|
+
```json
|
|
1465
|
+
{
|
|
1466
|
+
"mcpServers": {
|
|
1467
|
+
"s3db": {
|
|
1468
|
+
"command": "npx",
|
|
1469
|
+
"args": ["s3db-mcp-server", "--transport=sse"],
|
|
1470
|
+
"env": {
|
|
1471
|
+
"S3DB_VERBOSE": "true",
|
|
1472
|
+
"NODE_ENV": "development",
|
|
1473
|
+
"DEBUG": "true"
|
|
1474
|
+
}
|
|
1475
|
+
}
|
|
1476
|
+
}
|
|
1477
|
+
}
|
|
1478
|
+
```
|
|
1479
|
+
|
|
1480
|
+
View MCP logs in Claude Desktop:
|
|
1481
|
+
1. Open Developer Tools: `Cmd+Option+I` (Mac) or `Ctrl+Shift+I` (Windows)
|
|
1482
|
+
2. Go to Console tab
|
|
1483
|
+
3. Filter for "mcp" messages
|
|
1484
|
+
|
|
1485
|
+
### S3 Access Issues
|
|
1486
|
+
|
|
1487
|
+
```bash
|
|
1488
|
+
# Test S3 connection
|
|
1489
|
+
aws s3 ls s3://your-bucket
|
|
1490
|
+
|
|
1491
|
+
# Check credentials
|
|
1492
|
+
aws sts get-caller-identity
|
|
1493
|
+
|
|
1494
|
+
# Test with MinIO
|
|
1495
|
+
mc alias set local http://localhost:9000 minioadmin minioadmin
|
|
1496
|
+
mc ls local
|
|
1497
|
+
```
|
|
1498
|
+
|
|
1499
|
+
**Performance Issues:**
|
|
1500
|
+
- Increase `S3DB_PARALLELISM` for better throughput
|
|
1501
|
+
- Use partitions to reduce query scope
|
|
1502
|
+
- Implement proper pagination
|
|
1503
|
+
- Monitor S3 request patterns
|
|
1504
|
+
|
|
1505
|
+
**Memory Issues:**
|
|
1506
|
+
- Avoid `resourceGetAll` on large datasets
|
|
1507
|
+
- Use `resourceList` with pagination instead
|
|
1508
|
+
- Increase Docker memory limits if needed
|
|
1509
|
+
|
|
1510
|
+
### Error Messages
|
|
1511
|
+
|
|
1512
|
+
| Error | Cause | Solution |
|
|
1513
|
+
|-------|-------|----------|
|
|
1514
|
+
| "Database not connected" | No `dbConnect` called | Call `dbConnect` tool first |
|
|
1515
|
+
| "Resource not found" | Invalid resource name | Check with `dbListResources` |
|
|
1516
|
+
| "Validation failed" | Data doesn't match schema | Review attribute definitions |
|
|
1517
|
+
| "Connection string invalid" | Malformed connection string | Check format: `s3://key:secret@bucket/path` |
|
|
1518
|
+
| "Health check failed" | Server not responding | Check if process is running on correct port |
|
|
1519
|
+
|
|
1520
|
+
### Debug Mode
|
|
1521
|
+
|
|
1522
|
+
Enable verbose logging:
|
|
1523
|
+
```bash
|
|
1524
|
+
# Environment variable
|
|
1525
|
+
export S3DB_VERBOSE=true
|
|
1526
|
+
|
|
1527
|
+
# Command line
|
|
1528
|
+
s3db-mcp --transport=sse
|
|
1529
|
+
|
|
1530
|
+
# Docker
|
|
1531
|
+
docker run -e S3DB_VERBOSE=true s3db-mcp-server
|
|
1532
|
+
```
|
|
1533
|
+
|
|
1534
|
+
### Health Monitoring
|
|
1535
|
+
|
|
1536
|
+
```bash
|
|
1537
|
+
# Check server health
|
|
1538
|
+
curl http://localhost:8001/health
|
|
1539
|
+
|
|
1540
|
+
# Response includes:
|
|
1541
|
+
{
|
|
1542
|
+
"status": "healthy",
|
|
1543
|
+
"database": {
|
|
1544
|
+
"connected": true,
|
|
1545
|
+
"bucket": "my-bucket",
|
|
1546
|
+
"resourceCount": 5
|
|
1547
|
+
},
|
|
1548
|
+
"memory": { "rss": 45000000 },
|
|
1549
|
+
"uptime": 3600
|
|
1550
|
+
}
|
|
1551
|
+
```
|
|
1552
|
+
|
|
1553
|
+
### ๐ Success Indicators
|
|
1554
|
+
|
|
1555
|
+
You know your setup is working when:
|
|
1556
|
+
1. โ
Claude Desktop shows "Connected to s3db" in the MCP status
|
|
1557
|
+
2. โ
You can run database commands without errors
|
|
1558
|
+
3. โ
Cache statistics show hits after repeated queries
|
|
1559
|
+
4. โ
Cost tracking shows S3 operation counts
|
|
1560
|
+
|
|
1561
|
+
### ๐ก Pro Tips
|
|
1562
|
+
|
|
1563
|
+
1. **Start simple**: Test with MinIO locally before using AWS S3
|
|
1564
|
+
2. **Monitor costs**: Use `dbGetStats` regularly to track S3 expenses
|
|
1565
|
+
3. **Optimize partitions**: Design partitions based on query patterns
|
|
1566
|
+
4. **Cache wisely**: Filesystem cache survives restarts, memory cache is faster
|
|
1567
|
+
5. **Use batch operations**: `insertMany` is much faster than multiple `insert` calls
|
|
1568
|
+
|
|
1569
|
+
---
|
|
1570
|
+
|
|
1571
|
+
## ๐ Built-in Performance Features
|
|
1572
|
+
|
|
1573
|
+
The S3DB MCP Server includes **automatic performance optimizations**:
|
|
1574
|
+
|
|
1575
|
+
### **๐๏ธ Configurable Cache (Enabled by Default)**
|
|
1576
|
+
- **Two cache drivers**: Memory (fast, temporary) and Filesystem (persistent)
|
|
1577
|
+
- **Automatic caching** of read operations (get, list, count, exists)
|
|
1578
|
+
- **Partition-aware** caching for optimized queries
|
|
1579
|
+
- **Configurable TTL** and size limits
|
|
1580
|
+
- **Cache invalidation** on write operations
|
|
1581
|
+
- **Performance monitoring** via `dbGetStats`
|
|
1582
|
+
|
|
1583
|
+
#### **Memory Cache**
|
|
1584
|
+
- โก **Fastest performance** for frequently accessed data
|
|
1585
|
+
- ๐ **Lost on restart** - ideal for temporary caching
|
|
1586
|
+
- ๐ **Size-limited** by number of items
|
|
1587
|
+
|
|
1588
|
+
#### **Filesystem Cache**
|
|
1589
|
+
- ๐พ **Persistent across restarts** - cache survives server restarts
|
|
1590
|
+
- ๐๏ธ **Automatic compression** to save disk space
|
|
1591
|
+
- ๐งน **Automatic cleanup** of expired files
|
|
1592
|
+
- ๐ **Configurable directory** and file naming
|
|
1593
|
+
|
|
1594
|
+
### **๐ฐ Costs Tracking (Enabled by Default)**
|
|
1595
|
+
- **Real-time S3 costs** calculation
|
|
1596
|
+
- **Request counting** by operation type
|
|
1597
|
+
- **Cost estimation** in USD
|
|
1598
|
+
- **Performance analytics** for optimization
|
|
1599
|
+
|
|
1600
|
+
### **Configuration Options**
|
|
1601
|
+
```javascript
|
|
1602
|
+
// Connect with memory cache (fast, temporary)
|
|
1603
|
+
await agent.callTool('dbConnect', {
|
|
1604
|
+
connectionString: 's3://...',
|
|
1605
|
+
enableCache: true, // Default: true
|
|
1606
|
+
enableCosts: true, // Default: true
|
|
1607
|
+
cacheDriver: 'memory', // Fast but lost on restart
|
|
1608
|
+
cacheMaxSize: 2000, // Default: 1000
|
|
1609
|
+
cacheTtl: 600000 // Default: 300000 (5 min)
|
|
1610
|
+
});
|
|
1611
|
+
|
|
1612
|
+
// Connect with filesystem cache (persistent)
|
|
1613
|
+
await agent.callTool('dbConnect', {
|
|
1614
|
+
connectionString: 's3://...',
|
|
1615
|
+
enableCache: true,
|
|
1616
|
+
cacheDriver: 'filesystem', // Survives restarts
|
|
1617
|
+
cacheDirectory: './data/cache',
|
|
1618
|
+
cachePrefix: 'myapp',
|
|
1619
|
+
cacheTtl: 1800000 // 30 minutes
|
|
1620
|
+
});
|
|
1621
|
+
|
|
1622
|
+
// Monitor performance
|
|
1623
|
+
const stats = await agent.callTool('dbGetStats');
|
|
1624
|
+
console.log('Cache size:', stats.stats.cache.size);
|
|
1625
|
+
console.log('Cache driver:', stats.stats.cache.driver);
|
|
1626
|
+
console.log('S3 costs:', stats.stats.costs.estimatedCostUSD);
|
|
1627
|
+
|
|
1628
|
+
// Clear cache when needed
|
|
1629
|
+
await agent.callTool('dbClearCache', { resourceName: 'users' });
|
|
1630
|
+
```
|
|
1631
|
+
|
|
1632
|
+
### **Environment Variables**
|
|
1633
|
+
```bash
|
|
1634
|
+
S3DB_CACHE_ENABLED=true # Enable/disable cache
|
|
1635
|
+
S3DB_CACHE_DRIVER=memory # Cache driver: 'memory' or 'filesystem'
|
|
1636
|
+
S3DB_CACHE_MAX_SIZE=1000 # Cache capacity (memory driver)
|
|
1637
|
+
S3DB_CACHE_TTL=300000 # 5 minute TTL
|
|
1638
|
+
S3DB_CACHE_DIRECTORY=./cache # Filesystem cache directory
|
|
1639
|
+
S3DB_CACHE_PREFIX=s3db # Filesystem cache file prefix
|
|
1640
|
+
S3DB_COSTS_ENABLED=true # Enable/disable costs tracking
|
|
1641
|
+
```
|
|
1642
|
+
|
|
1643
|
+
---
|
|
1644
|
+
|
|
1645
|
+
## ๐ **Cache Strategy Guide**
|
|
1646
|
+
|
|
1647
|
+
Choose the right cache driver for your use case:
|
|
1648
|
+
|
|
1649
|
+
### **When to Use Memory Cache**
|
|
1650
|
+
- โก **Development & Testing** - fastest performance, no setup required
|
|
1651
|
+
- ๐ **Short-lived processes** - containers that restart frequently
|
|
1652
|
+
- ๐ **High-frequency reads** - when you need maximum speed
|
|
1653
|
+
- ๐ฐ **Cost optimization** - minimize S3 requests for hot data
|
|
1654
|
+
- โ ๏ธ **Limitation**: Cache is lost on restart
|
|
1655
|
+
|
|
1656
|
+
### **When to Use Filesystem Cache**
|
|
1657
|
+
- ๐พ **Production environments** - cache survives server restarts
|
|
1658
|
+
- ๐ **Long-running processes** - persistent data across deployments
|
|
1659
|
+
- ๐ฆ **Containerized deployments** - mount cache volume for persistence
|
|
1660
|
+
- ๐ง **Development consistency** - maintain cache between code changes
|
|
1661
|
+
- ๐๏ธ **Large datasets** - no memory size limitations
|
|
1662
|
+
|
|
1663
|
+
### **Configuration Examples**
|
|
1664
|
+
|
|
1665
|
+
```javascript
|
|
1666
|
+
// High-performance temporary cache
|
|
1667
|
+
await agent.callTool('dbConnect', {
|
|
1668
|
+
cacheDriver: 'memory',
|
|
1669
|
+
cacheMaxSize: 5000,
|
|
1670
|
+
cacheTtl: 600000 // 10 minutes
|
|
1671
|
+
});
|
|
1672
|
+
|
|
1673
|
+
// Production persistent cache
|
|
1674
|
+
await agent.callTool('dbConnect', {
|
|
1675
|
+
cacheDriver: 'filesystem',
|
|
1676
|
+
cacheDirectory: './data/cache',
|
|
1677
|
+
cachePrefix: 'prod',
|
|
1678
|
+
cacheTtl: 3600000 // 1 hour
|
|
1679
|
+
});
|
|
1680
|
+
```
|
|
1681
|
+
|
|
1682
|
+
### **Docker Volume Setup**
|
|
1683
|
+
```yaml
|
|
1684
|
+
# docker-compose.yml
|
|
1685
|
+
volumes:
|
|
1686
|
+
- ./cache-data:/app/cache # Persistent filesystem cache
|
|
1687
|
+
environment:
|
|
1688
|
+
- S3DB_CACHE_DRIVER=filesystem
|
|
1689
|
+
- S3DB_CACHE_DIRECTORY=/app/cache
|
|
1690
|
+
```
|
|
1691
|
+
|
|
1692
|
+
---
|
|
1693
|
+
|
|
1694
|
+
## ๐ Performance Tips
|
|
1695
|
+
|
|
1696
|
+
1. **Choose appropriate cache** - memory for speed, filesystem for persistence
|
|
1697
|
+
2. **Leverage built-in cache** - read operations are automatically cached
|
|
1698
|
+
3. **Use partitions** for large datasets to improve cache efficiency
|
|
1699
|
+
4. **Monitor costs** with `dbGetStats` to optimize S3 usage
|
|
1700
|
+
5. **Batch operations** when possible to reduce S3 requests
|
|
1701
|
+
6. **Proper pagination** - don't load everything at once
|
|
1702
|
+
7. **Connection reuse** - keep connections alive
|
|
1703
|
+
8. **Appropriate parallelism** - tune `S3DB_PARALLELISM`
|
|
1704
|
+
|
|
1705
|
+
---
|
|
1706
|
+
|
|
1707
|
+
## ๐ Additional Resources
|
|
1708
|
+
|
|
1709
|
+
- [S3DB Documentation](https://github.com/forattini-dev/s3db.js)
|
|
1710
|
+
- [MCP Protocol Specification](https://modelcontextprotocol.io)
|
|
1711
|
+
- [Claude Desktop Downloads](https://claude.ai/download)
|
|
1712
|
+
- [GitHub Issues](https://github.com/forattini-dev/s3db.js/issues)
|
|
1713
|
+
- [NPM Package](https://www.npmjs.com/package/s3db-mcp-server)
|
|
1714
|
+
|
|
1715
|
+
Need help? Check the [Troubleshooting](#troubleshooting) section or file an issue on [GitHub](https://github.com/forattini-dev/s3db.js/issues).
|
|
1716
|
+
|
|
1717
|
+
---
|
|
1718
|
+
|
|
1719
|
+
## ๐ License
|
|
1720
|
+
|
|
1721
|
+
This project is licensed under the same license as the parent S3DB project.
|
|
1722
|
+
|
|
1723
|
+
---
|
|
1724
|
+
|
|
1725
|
+
<p align="center">
|
|
1726
|
+
<strong>๐ Ready to supercharge your AI agents with persistent data storage!</strong><br>
|
|
1727
|
+
<em>Start building with S3DB MCP Server today</em>
|
|
1728
|
+
</p>
|