s3db.js 10.0.7 โ†’ 10.0.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/mcp/README.md DELETED
@@ -1,1728 +0,0 @@
1
- # ๐Ÿ—ƒ๏ธ S3DB MCP Server
2
-
3
- <p align="center">
4
- <img width="200" src="https://img.icons8.com/fluency/200/database.png" alt="S3DB MCP Server">
5
- </p>
6
-
7
- <p align="center">
8
- <strong>Model Context Protocol (MCP) server for S3DB</strong><br>
9
- <em>Transform AWS S3 into a powerful document database accessible by AI agents</em>
10
- </p>
11
-
12
- <p align="center">
13
- <a href="https://www.npmjs.com/package/s3db-mcp-server"><img src="https://img.shields.io/npm/v/s3db-mcp-server.svg?style=flat&color=brightgreen" alt="npm version"></a>
14
- &nbsp;
15
- <a href="https://github.com/forattini-dev/s3db.js"><img src="https://img.shields.io/github/stars/forattini-dev/s3db.js?style=flat&color=yellow" alt="GitHub stars"></a>
16
- &nbsp;
17
- <a href="https://github.com/forattini-dev/s3db.js/blob/main/UNLICENSE"><img src="https://img.shields.io/badge/license-Unlicense-blue.svg?style=flat" alt="License"></a>
18
- </p>
19
-
20
- ---
21
-
22
- ## ๐Ÿš€ Quick Start with Claude Desktop
23
-
24
- ### Prerequisites
25
- Before starting, ensure you have:
26
- 1. **Claude Desktop** installed (version 0.7.0 or later)
27
- 2. **Node.js** installed (version 18+ recommended)
28
- 3. **S3 Bucket** or S3-compatible storage (MinIO, etc.)
29
- 4. **Credentials** for your S3 service
30
-
31
- ### Step 1: Locate Claude Desktop Configuration
32
-
33
- The configuration file location depends on your operating system:
34
-
35
- ```bash
36
- # macOS
37
- ~/Library/Application Support/Claude/claude_desktop_config.json
38
-
39
- # Windows
40
- %APPDATA%\Claude\claude_desktop_config.json
41
-
42
- # Linux
43
- ~/.config/Claude/claude_desktop_config.json
44
- ```
45
-
46
- ### Step 2: Configure Claude Desktop
47
-
48
- Open the configuration file and add the S3DB MCP server:
49
-
50
- ```json
51
- {
52
- "mcpServers": {
53
- "s3db": {
54
- "command": "npx",
55
- "args": ["s3db-mcp-server", "--transport=sse"],
56
- "env": {
57
- "S3DB_CONNECTION_STRING": "s3://ACCESS_KEY:SECRET_KEY@bucket/databases/myapp",
58
- "S3DB_CACHE_ENABLED": "true",
59
- "S3DB_CACHE_DRIVER": "memory",
60
- "S3DB_CACHE_MAX_SIZE": "1000",
61
- "S3DB_CACHE_TTL": "300000",
62
- "MCP_SERVER_PORT": "17500"
63
- }
64
- }
65
- }
66
- }
67
- ```
68
-
69
- ### Step 3: Restart Claude Desktop
70
-
71
- After saving the configuration:
72
- 1. Completely quit Claude Desktop (not just close the window)
73
- 2. Restart Claude Desktop
74
- 3. The MCP server should start automatically
75
-
76
- ### Step 4: Test Your Setup
77
-
78
- In Claude Desktop, test with these commands:
79
-
80
- ```javascript
81
- // 1. Test connection
82
- "Can you connect to the S3DB database and show me the status?"
83
-
84
- // 2. Create a resource
85
- "Please create a new resource called 'users' with these fields:
86
- - name (string, required)
87
- - email (string, required, unique)
88
- - age (number)
89
- - active (boolean, default true)"
90
-
91
- // 3. Insert data
92
- "Insert a new user with name 'John Doe', email 'john@example.com', age 30"
93
-
94
- // 4. Query data
95
- "Show me all users in the database"
96
-
97
- // 5. Check statistics
98
- "Can you show me the database statistics including cache hits and S3 costs?"
99
- ```
100
-
101
- ---
102
-
103
- ## ๐Ÿ“‹ Table of Contents
104
-
105
- - [๐Ÿš€ Quick Start with Claude Desktop](#-quick-start-with-claude-desktop)
106
- - [๐Ÿ’พ Alternative Installation Methods](#-alternative-installation-methods)
107
- - [โš™๏ธ Configuration Examples](#๏ธ-configuration-examples)
108
- - [๐Ÿ› ๏ธ Available Tools](#๏ธ-available-tools)
109
- - [๐Ÿ“– Command Line Examples](#-command-line-examples)
110
- - [๐Ÿ—‚๏ธ Partitions & Performance](#๏ธ-partitions--performance)
111
- - [๐Ÿณ Docker Deployment](#-docker-deployment)
112
- - [๐Ÿ”ง Advanced Configuration](#-advanced-configuration)
113
- - [๐Ÿ”’ Security](#-security)
114
- - [๐Ÿšจ Troubleshooting](#-troubleshooting)
115
-
116
- ---
117
-
118
- ## ๐Ÿ’พ Alternative Installation Methods
119
-
120
- ### Option 1: NPX (Recommended)
121
- ```bash
122
- # SSE transport (web clients) - Default port: 17500
123
- npx s3db-mcp-server --transport=sse
124
-
125
- # STDIO transport (desktop clients)
126
- npx s3db-mcp-server --transport=stdio
127
- ```
128
-
129
- ### Option 2: Global Installation
130
- ```bash
131
- npm install -g s3db-mcp-server
132
- s3db-mcp --transport=sse
133
- ```
134
-
135
- ### Option 3: Docker
136
- ```bash
137
- docker run -p 17500:8000 -e S3DB_CONNECTION_STRING="s3://key:secret@bucket/db" s3db-mcp-server
138
- ```
139
-
140
- ### Option 4: Standalone Server
141
- ```bash
142
- # Start the server (it will run in the foreground)
143
- npx s3db-mcp-server --transport=sse
144
-
145
- # With environment variables
146
- cat > s3db-mcp.env << EOF
147
- S3DB_CONNECTION_STRING=s3://ACCESS_KEY:SECRET_KEY@bucket-name/databases/myapp
148
- S3DB_CACHE_ENABLED=true
149
- S3DB_CACHE_DRIVER=memory
150
- S3DB_CACHE_MAX_SIZE=1000
151
- EOF
152
-
153
- env $(cat s3db-mcp.env | xargs) npx s3db-mcp-server --transport=sse
154
- ```
155
-
156
- ### Option 5: Background Process
157
- ```bash
158
- # Using nohup (Linux/macOS)
159
- nohup npx s3db-mcp-server --transport=sse > s3db-mcp.log 2>&1 &
160
- echo $! > s3db-mcp.pid
161
-
162
- # To stop later
163
- kill $(cat s3db-mcp.pid)
164
- ```
165
-
166
- ### Option 6: Using PM2 Process Manager
167
- ```bash
168
- # Install PM2 globally
169
- npm install -g pm2
170
-
171
- # Start with PM2
172
- pm2 start npx --name "s3db-mcp" -- s3db-mcp-server --transport=sse
173
-
174
- # View logs
175
- pm2 logs s3db-mcp
176
-
177
- # Stop/restart
178
- pm2 stop s3db-mcp
179
- pm2 restart s3db-mcp
180
- ```
181
-
182
- ---
183
-
184
- ## โš™๏ธ Configuration Examples
185
-
186
- ### ๐Ÿ  Local Development with MinIO
187
-
188
- ```json
189
- {
190
- "mcpServers": {
191
- "s3db-local": {
192
- "command": "npx",
193
- "args": ["s3db-mcp-server", "--transport=sse"],
194
- "env": {
195
- "S3DB_CONNECTION_STRING": "s3://minioadmin:minioadmin123@localhost:9000/dev-bucket?forcePathStyle=true",
196
- "S3DB_VERBOSE": "true",
197
- "S3DB_CACHE_ENABLED": "true",
198
- "S3DB_COSTS_ENABLED": "false",
199
- "NODE_ENV": "development"
200
- }
201
- }
202
- }
203
- }
204
- ```
205
-
206
- ### โ˜๏ธ Production with AWS S3
207
-
208
- ```json
209
- {
210
- "mcpServers": {
211
- "s3db-prod": {
212
- "command": "npx",
213
- "args": ["s3db-mcp-server", "--transport=sse"],
214
- "env": {
215
- "S3DB_CONNECTION_STRING": "s3://prod-data-bucket/databases/main",
216
- "AWS_REGION": "us-east-1",
217
- "AWS_ACCESS_KEY_ID": "AKIA...",
218
- "AWS_SECRET_ACCESS_KEY": "wJal...",
219
- "S3DB_CACHE_DRIVER": "filesystem",
220
- "S3DB_CACHE_DIRECTORY": "/var/cache/s3db",
221
- "S3DB_CACHE_TTL": "1800000",
222
- "S3DB_PASSPHRASE": "your-strong-passphrase-here",
223
- "S3DB_VERSIONING_ENABLED": "true",
224
- "NODE_ENV": "production"
225
- }
226
- }
227
- }
228
- }
229
- ```
230
-
231
- ### ๐ŸŒŠ DigitalOcean Spaces
232
-
233
- ```json
234
- {
235
- "mcpServers": {
236
- "s3db-do": {
237
- "command": "npx",
238
- "args": ["s3db-mcp-server", "--transport=sse"],
239
- "env": {
240
- "S3DB_CONNECTION_STRING": "s3://DO_ACCESS_KEY:DO_SECRET_KEY@nyc3.digitaloceanspaces.com/space-name/databases/app",
241
- "S3_ENDPOINT": "https://nyc3.digitaloceanspaces.com",
242
- "S3DB_CACHE_ENABLED": "true"
243
- }
244
- }
245
- }
246
- }
247
- ```
248
-
249
- ### ๐Ÿ”ง Multiple Environments
250
-
251
- ```json
252
- {
253
- "mcpServers": {
254
- "s3db-dev": {
255
- "command": "npx",
256
- "args": ["s3db-mcp-server", "--transport=sse", "--port=17500"],
257
- "env": {
258
- "S3DB_CONNECTION_STRING": "s3://minioadmin:minioadmin123@localhost:9000/dev-bucket?forcePathStyle=true"
259
- }
260
- },
261
- "s3db-staging": {
262
- "command": "npx",
263
- "args": ["s3db-mcp-server", "--transport=sse", "--port=17501"],
264
- "env": {
265
- "S3DB_CONNECTION_STRING": "s3://staging-bucket/databases/staging",
266
- "AWS_REGION": "us-east-1"
267
- }
268
- },
269
- "s3db-prod": {
270
- "command": "npx",
271
- "args": ["s3db-mcp-server", "--transport=sse", "--port=17502"],
272
- "env": {
273
- "S3DB_CONNECTION_STRING": "s3://prod-bucket/databases/production",
274
- "AWS_REGION": "us-east-1",
275
- "S3DB_CACHE_DRIVER": "filesystem"
276
- }
277
- }
278
- }
279
- }
280
- ```
281
-
282
- ## โš™๏ธ Configuration Reference
283
-
284
- ### ๐Ÿ“ Configuration Overview
285
-
286
- The S3DB MCP Server can be configured through multiple methods (in order of precedence):
287
- 1. **Command-line arguments** (highest priority)
288
- 2. **Environment variables**
289
- 3. **`.env` file**
290
- 4. **Default values** (lowest priority)
291
-
292
- ### ๐ŸŒ Server Configuration
293
-
294
- #### **Core Server Settings**
295
-
296
- | Variable | Default | Description | Example | Notes |
297
- |----------|---------|-------------|---------|-------|
298
- | `NODE_ENV` | `development` | Environment mode | `production`, `development`, `test` | Affects logging verbosity and error details |
299
- | `MCP_SERVER_HOST` | `0.0.0.0` | Server bind address | `localhost`, `127.0.0.1`, `0.0.0.0` | Use `0.0.0.0` to accept connections from any interface |
300
- | `MCP_SERVER_PORT` | `17500` | Server port | Any port 1024-65535 | Changed from 8000 to avoid conflicts |
301
- | `MCP_TRANSPORT` | `sse` | Transport method | `sse`, `stdio` | SSE for web clients, stdio for CLI tools |
302
-
303
- #### **Transport Modes Explained**
304
-
305
- - **SSE (Server-Sent Events)**:
306
- - Best for: Web-based AI clients, Claude Desktop, Cursor IDE
307
- - Protocol: HTTP/HTTPS
308
- - URL format: `http://localhost:17500/sse`
309
-
310
- - **STDIO (Standard Input/Output)**:
311
- - Best for: CLI tools, shell scripts, pipe-based communication
312
- - Protocol: JSON-RPC over stdin/stdout
313
- - No network port required
314
-
315
- ### ๐Ÿ—„๏ธ S3DB Core Configuration
316
-
317
- #### **Essential Database Settings**
318
-
319
- | Variable | Default | Required | Description | Example Values |
320
- |----------|---------|----------|-------------|----------------|
321
- | `S3DB_CONNECTION_STRING` | - | โœ… Yes | Complete S3 connection URL | See [Connection String Formats](#connection-string-formats) below |
322
- | `S3DB_VERBOSE` | `false` | No | Enable detailed operation logs | `true` for debugging, `false` for production |
323
- | `S3DB_PARALLELISM` | `10` | No | Max concurrent S3 operations | `5` (conservative), `20` (aggressive), `50` (high-performance) |
324
- | `S3DB_PASSPHRASE` | `secret` | No | Encryption key for sensitive fields | Any strong passphrase (min 12 chars recommended) |
325
- | `S3DB_VERSIONING_ENABLED` | `false` | No | Track resource schema versions | `true` for production, `false` for development |
326
-
327
- #### **Performance Tuning Guidelines**
328
-
329
- ```bash
330
- # Development (fast iteration, verbose logging)
331
- S3DB_VERBOSE=true
332
- S3DB_PARALLELISM=5
333
- S3DB_VERSIONING_ENABLED=false
334
-
335
- # Staging (balanced performance)
336
- S3DB_VERBOSE=false
337
- S3DB_PARALLELISM=10
338
- S3DB_VERSIONING_ENABLED=true
339
-
340
- # Production (optimized for scale)
341
- S3DB_VERBOSE=false
342
- S3DB_PARALLELISM=20
343
- S3DB_VERSIONING_ENABLED=true
344
- ```
345
-
346
- ### ๐Ÿ”Œ Plugin Configuration
347
-
348
- #### **Cache Plugin Settings**
349
-
350
- | Variable | Default | Description | When to Change | Impact |
351
- |----------|---------|-------------|----------------|--------|
352
- | `S3DB_CACHE_ENABLED` | `true` | Master cache toggle | Set `false` only for debugging | 70-90% performance improvement when enabled |
353
- | `S3DB_CACHE_DRIVER` | `memory` | Cache storage backend | Use `filesystem` for persistent cache | Memory: faster, Filesystem: survives restarts |
354
- | `S3DB_CACHE_MAX_SIZE` | `1000` | Max cached items (memory only) | Increase for read-heavy workloads | Each item ~1-10KB RAM |
355
- | `S3DB_CACHE_TTL` | `300000` | Cache lifetime (ms) | Decrease for frequently changing data | 5 min default, 0 = no expiry |
356
- | `S3DB_CACHE_DIRECTORY` | `./cache` | Filesystem cache location | Use SSD path for best performance | Only for filesystem driver |
357
- | `S3DB_CACHE_PREFIX` | `s3db` | Cache file prefix | Change for multiple instances | Prevents cache conflicts |
358
-
359
- #### **Cache Strategy Examples**
360
-
361
- ```bash
362
- # High-traffic read-heavy API
363
- S3DB_CACHE_DRIVER=memory
364
- S3DB_CACHE_MAX_SIZE=5000
365
- S3DB_CACHE_TTL=600000 # 10 minutes
366
-
367
- # Data analytics workload
368
- S3DB_CACHE_DRIVER=filesystem
369
- S3DB_CACHE_DIRECTORY=/mnt/ssd/cache
370
- S3DB_CACHE_TTL=3600000 # 1 hour
371
-
372
- # Real-time application
373
- S3DB_CACHE_DRIVER=memory
374
- S3DB_CACHE_MAX_SIZE=500
375
- S3DB_CACHE_TTL=30000 # 30 seconds
376
- ```
377
-
378
- #### **Cost Tracking Plugin**
379
-
380
- | Variable | Default | Description | Use Case |
381
- |----------|---------|-------------|----------|
382
- | `S3DB_COSTS_ENABLED` | `true` | Track S3 API costs | Disable for local MinIO/testing |
383
-
384
- Cost tracking provides:
385
- - Per-operation cost breakdown
386
- - Daily/monthly projections
387
- - Request type statistics
388
- - Data transfer metrics
389
-
390
- ### ๐Ÿ” AWS & S3-Compatible Configuration
391
-
392
- #### **AWS Credentials**
393
-
394
- | Variable | Default | Description | Priority Order |
395
- |----------|---------|-------------|----------------|
396
- | `AWS_ACCESS_KEY_ID` | - | AWS access key | 1. Env var, 2. IAM role, 3. Connection string |
397
- | `AWS_SECRET_ACCESS_KEY` | - | AWS secret key | Required if using access key |
398
- | `AWS_SESSION_TOKEN` | - | Temporary credentials | For STS/assumed roles |
399
- | `AWS_REGION` | `us-east-1` | AWS region | Must match bucket region |
400
-
401
- #### **S3-Compatible Services**
402
-
403
- | Variable | Default | Description | Services |
404
- |----------|---------|-------------|----------|
405
- | `S3_ENDPOINT` | - | Custom S3 API endpoint | MinIO, DigitalOcean, Backblaze, Wasabi |
406
- | `S3_FORCE_PATH_STYLE` | `false` | URL style | Required for MinIO, LocalStack |
407
-
408
- ### ๐Ÿ”— Connection String Formats
409
-
410
- #### **Anatomy of a Connection String**
411
-
412
- ```
413
- s3://[ACCESS_KEY:SECRET_KEY@]BUCKET[/PATH][?PARAMS]
414
- ```
415
-
416
- Components:
417
- - `ACCESS_KEY:SECRET_KEY` - Optional inline credentials
418
- - `BUCKET` - S3 bucket name
419
- - `PATH` - Optional path prefix for organization
420
- - `PARAMS` - Query parameters for advanced config
421
-
422
- #### **Real-World Examples**
423
-
424
- ```bash
425
- # AWS S3 - Production with IAM role (recommended)
426
- S3DB_CONNECTION_STRING="s3://my-prod-bucket/databases/main"
427
-
428
- # AWS S3 - Development with credentials
429
- S3DB_CONNECTION_STRING="s3://AKIA...:wJal...@my-dev-bucket/databases/dev"
430
-
431
- # MinIO - Local development
432
- S3DB_CONNECTION_STRING="s3://minioadmin:minioadmin123@localhost:17998/s3db?forcePathStyle=true"
433
-
434
- # DigitalOcean Spaces
435
- S3DB_CONNECTION_STRING="s3://DO_KEY:DO_SECRET@nyc3.digitaloceanspaces.com/space-name/databases/prod"
436
-
437
- # Backblaze B2
438
- S3DB_CONNECTION_STRING="s3://KEY_ID:APP_KEY@s3.us-west-002.backblazeb2.com/bucket-name/db"
439
-
440
- # Wasabi
441
- S3DB_CONNECTION_STRING="s3://ACCESS_KEY:SECRET_KEY@s3.wasabisys.com/bucket-name/databases/app"
442
-
443
- # LocalStack (testing)
444
- S3DB_CONNECTION_STRING="s3://test:test@localhost:4566/test-bucket/db?forcePathStyle=true"
445
- ```
446
-
447
- ### ๐Ÿ“ Complete Configuration Examples
448
-
449
- #### **Development Setup (.env)**
450
-
451
- ```bash
452
- # Server
453
- NODE_ENV=development
454
- MCP_SERVER_PORT=17500
455
- MCP_TRANSPORT=sse
456
-
457
- # S3DB
458
- S3DB_CONNECTION_STRING=s3://minioadmin:minioadmin123@localhost:9000/dev-bucket/db
459
- S3DB_VERBOSE=true
460
- S3DB_PARALLELISM=5
461
-
462
- # Cache
463
- S3DB_CACHE_ENABLED=true
464
- S3DB_CACHE_DRIVER=memory
465
- S3DB_CACHE_MAX_SIZE=100
466
- S3DB_CACHE_TTL=60000
467
-
468
- # Costs
469
- S3DB_COSTS_ENABLED=false
470
- ```
471
-
472
- #### **Production Setup (.env)**
473
-
474
- ```bash
475
- # Server
476
- NODE_ENV=production
477
- MCP_SERVER_PORT=17500
478
- MCP_TRANSPORT=sse
479
-
480
- # S3DB (using IAM role)
481
- S3DB_CONNECTION_STRING=s3://prod-data-bucket/databases/main
482
- S3DB_VERBOSE=false
483
- S3DB_PARALLELISM=20
484
- S3DB_PASSPHRASE=${SECRET_PASSPHRASE}
485
- S3DB_VERSIONING_ENABLED=true
486
-
487
- # Cache
488
- S3DB_CACHE_ENABLED=true
489
- S3DB_CACHE_DRIVER=filesystem
490
- S3DB_CACHE_DIRECTORY=/var/cache/s3db
491
- S3DB_CACHE_TTL=1800000
492
- S3DB_CACHE_PREFIX=prod
493
-
494
- # Costs
495
- S3DB_COSTS_ENABLED=true
496
-
497
- # AWS
498
- AWS_REGION=us-east-1
499
- ```
500
-
501
- ### ๐Ÿš€ Command Line Options
502
-
503
- ```bash
504
- # Basic usage
505
- npx s3db-mcp-server [OPTIONS]
506
-
507
- # Transport selection
508
- npx s3db-mcp-server --transport=sse # Web clients (default)
509
- npx s3db-mcp-server --transport=stdio # CLI/pipe communication
510
-
511
- # Network configuration
512
- npx s3db-mcp-server --host=0.0.0.0 --port=17500
513
-
514
- # Override environment variables
515
- npx s3db-mcp-server --transport=sse \
516
- --host=127.0.0.1 \
517
- --port=18000
518
-
519
- # Combined with environment variables
520
- S3DB_CONNECTION_STRING="s3://..." \
521
- S3DB_CACHE_DRIVER=filesystem \
522
- npx s3db-mcp-server --transport=sse
523
-
524
- # Debug mode with verbose output
525
- S3DB_VERBOSE=true \
526
- NODE_ENV=development \
527
- npx s3db-mcp-server --transport=stdio
528
- ```
529
-
530
- ### ๐Ÿ” Configuration Validation
531
-
532
- The server validates configuration on startup and will:
533
- 1. Check for required `S3DB_CONNECTION_STRING`
534
- 2. Test S3 connectivity
535
- 3. Verify bucket permissions
536
- 4. Initialize cache directory (if using filesystem)
537
- 5. Report configuration summary
538
-
539
- Example startup log:
540
- ```
541
- S3DB MCP Server v1.0.0 started
542
- Transport: sse
543
- Port: 17500
544
- Cache: memory (1000 items, 5 min TTL)
545
- Costs tracking: enabled
546
- Connected to: s3://my-bucket/databases/main
547
- Ready for connections...
548
- ```
549
-
550
- ---
551
-
552
- ## ๐Ÿ› ๏ธ Available Tools
553
-
554
- ### Database Management
555
-
556
- | Tool | Description | Parameters |
557
- |------|-------------|------------|
558
- | `dbConnect` | Connect to S3DB database with costs & cache | `connectionString`, `verbose?`, `parallelism?`, `passphrase?`, `versioningEnabled?`, `enableCache?`, `enableCosts?`, `cacheDriver?`, `cacheMaxSize?`, `cacheTtl?`, `cacheDirectory?`, `cachePrefix?` |
559
- | `dbDisconnect` | Disconnect from database | - |
560
- | `dbStatus` | Get connection status | - |
561
- | `dbCreateResource` | Create resource/collection | `name`, `attributes`, `behavior?`, `timestamps?`, `partitions?`, `paranoid?` |
562
- | `dbListResources` | List all resources | - |
563
- | `dbGetStats` | Get database statistics (costs, cache, resources) | - |
564
- | `dbClearCache` | Clear cache data | `resourceName?` (optional - clears all if not provided) |
565
-
566
- ### Document Operations
567
-
568
- | Tool | Description | Parameters |
569
- |------|-------------|------------|
570
- | `resourceInsert` | Insert single document | `resourceName`, `data` |
571
- | `resourceInsertMany` | Insert multiple documents | `resourceName`, `data[]` |
572
- | `resourceGet` | Get document by ID | `resourceName`, `id` |
573
- | `resourceGetMany` | Get multiple documents | `resourceName`, `ids[]` |
574
- | `resourceUpdate` | Update document | `resourceName`, `id`, `data` |
575
- | `resourceUpsert` | Insert or update | `resourceName`, `data` |
576
- | `resourceDelete` | Delete document | `resourceName`, `id` |
577
- | `resourceDeleteMany` | Delete multiple documents | `resourceName`, `ids[]` |
578
-
579
- ### Query Operations
580
-
581
- | Tool | Description | Parameters |
582
- |------|-------------|------------|
583
- | `resourceExists` | Check if document exists | `resourceName`, `id` |
584
- | `resourceList` | List with pagination | `resourceName`, `limit?`, `offset?`, `partition?`, `partitionValues?` |
585
- | `resourceListIds` | List document IDs only | `resourceName`, `limit?`, `offset?` |
586
- | `resourceCount` | Count documents | `resourceName`, `partition?`, `partitionValues?` |
587
- | `resourceGetAll` | Get all documents | `resourceName` |
588
- | `resourceDeleteAll` | Delete all documents | `resourceName`, `confirm: true` |
589
-
590
- ---
591
-
592
- ## ๐Ÿ“– Command Line Examples
593
-
594
- ### ๐Ÿ“ Basic CRUD Operations
595
-
596
- ```bash
597
- # Connect to database
598
- "Please connect to the S3DB database using the connection string: s3://my-bucket/databases/app"
599
-
600
- # Create a blog posts resource
601
- "Create a resource named 'posts' with fields: title (string, required), content (string), author (string), published (boolean), tags (array of strings)"
602
-
603
- # Insert a blog post
604
- "Insert a new post with title 'Getting Started with S3DB', content 'S3DB is amazing...', author 'john-doe', published true, and tags ['tutorial', 's3db']"
605
-
606
- # Query posts
607
- "List all published posts"
608
- "Find posts by author 'john-doe'"
609
- "Count how many posts have the tag 'tutorial'"
610
-
611
- # Update a post
612
- "Update the post with ID 'post-123' to set published to false"
613
-
614
- # Delete a post
615
- "Delete the post with ID 'post-456'"
616
- ```
617
-
618
- ### ๐Ÿ” Advanced Queries with Partitions
619
-
620
- ```bash
621
- # Create partitioned resource
622
- "Create an 'orders' resource with:
623
- - orderId (string, required, unique)
624
- - customerId (string, required)
625
- - amount (number, required)
626
- - status (string: pending, processing, shipped, delivered)
627
- - region (string)
628
- - orderDate (date)
629
- And create partitions by status and region"
630
-
631
- # Query specific partition
632
- "Show me all orders with status 'pending' in the 'north' region"
633
-
634
- # Count by partition
635
- "Count orders by status"
636
-
637
- # List with pagination
638
- "List the first 10 orders, then get the next 10"
639
- ```
640
-
641
- ### ๐Ÿ› ๏ธ Database Management
642
-
643
- ```bash
644
- # Check connection status
645
- "Show me the current database connection status"
646
-
647
- # List all resources
648
- "What resources/collections exist in the database?"
649
-
650
- # Get statistics
651
- "Show database statistics including cache performance and S3 costs"
652
-
653
- # Clear cache
654
- "Clear the cache for the 'users' resource"
655
- "Clear all cached data"
656
-
657
- # Bulk operations
658
- "Insert 100 test users with random data"
659
- "Delete all users where active is false"
660
- ```
661
-
662
- ### ๐Ÿ“Š Reporting and Analytics
663
-
664
- ```bash
665
- # Cost analysis
666
- "How much are we spending on S3 operations today?"
667
- "Show me a breakdown of S3 costs by operation type"
668
-
669
- # Performance metrics
670
- "What's the cache hit ratio?"
671
- "Which queries are slowest?"
672
-
673
- # Data overview
674
- "Give me a summary of all resources: total records, size, last modified"
675
- "Which partitions have the most data?"
676
- ```
677
-
678
- ## ๐Ÿ“– API Usage Examples
679
-
680
- ### Basic CRUD Operations
681
-
682
- ```javascript
683
- // 1. Connect to database with automatic cache and costs tracking
684
- await agent.callTool('dbConnect', {
685
- connectionString: 's3://ACCESS_KEY:SECRET_KEY@bucket/databases/blog',
686
- verbose: false,
687
- parallelism: 10,
688
- enableCache: true, // Cache enabled by default
689
- enableCosts: true, // Costs tracking enabled by default
690
- cacheDriver: 'memory', // 'memory' or 'filesystem'
691
- cacheMaxSize: 1000, // Cache up to 1000 items (memory only)
692
- cacheTtl: 300000, // 5 minute cache TTL
693
- cacheDirectory: './cache', // Directory for filesystem cache
694
- cachePrefix: 's3db' // Prefix for cache files
695
- });
696
-
697
- // 2. Create a resource with schema validation
698
- await agent.callTool('dbCreateResource', {
699
- name: 'posts',
700
- attributes: {
701
- title: 'string|required|min:3|max:200',
702
- content: 'string|required',
703
- author: 'string|required',
704
- tags: 'array|items:string',
705
- published: 'boolean',
706
- publishDate: 'date',
707
- metadata: {
708
- views: 'number|positive',
709
- likes: 'number|positive'
710
- }
711
- },
712
- behavior: 'user-managed',
713
- timestamps: true, // Adds createdAt/updatedAt automatically
714
- paranoid: true // Soft deletes
715
- });
716
-
717
- // 3. Insert a blog post
718
- const post = await agent.callTool('resourceInsert', {
719
- resourceName: 'posts',
720
- data: {
721
- title: 'Getting Started with S3DB MCP',
722
- content: 'S3DB transforms AWS S3 into a powerful document database...',
723
- author: 'john-doe',
724
- tags: ['tutorial', 's3db', 'mcp', 'ai'],
725
- published: true,
726
- publishDate: '2024-01-15',
727
- metadata: {
728
- views: 0,
729
- likes: 0
730
- }
731
- }
732
- });
733
-
734
- // 4. Update the post
735
- await agent.callTool('resourceUpdate', {
736
- resourceName: 'posts',
737
- id: post.data.id,
738
- data: {
739
- metadata: {
740
- views: 150,
741
- likes: 12
742
- }
743
- }
744
- });
745
-
746
- // 5. Query posts with pagination
747
- const posts = await agent.callTool('resourceList', {
748
- resourceName: 'posts',
749
- limit: 10,
750
- offset: 0
751
- });
752
-
753
- // 6. Check if post exists
754
- const exists = await agent.callTool('resourceExists', {
755
- resourceName: 'posts',
756
- id: post.data.id
757
- });
758
-
759
- // 7. Check performance statistics
760
- const stats = await agent.callTool('dbGetStats');
761
- console.log('Cache hits:', stats.stats.cache.size);
762
- console.log('S3 costs:', stats.stats.costs.estimatedCostUSD);
763
- console.log('Total requests:', stats.stats.costs.totalRequests);
764
-
765
- // 8. Clear cache if needed
766
- await agent.callTool('dbClearCache', {
767
- resourceName: 'posts' // Clear cache for specific resource
768
- });
769
- ```
770
-
771
- ### Performance & Costs Monitoring
772
-
773
- ```javascript
774
- // Monitor database performance and costs
775
- const stats = await agent.callTool('dbGetStats');
776
-
777
- console.log('Database Stats:', {
778
- resources: stats.stats.database.resources,
779
- totalCosts: `$${stats.stats.costs.estimatedCostUSD.toFixed(6)}`,
780
- cacheHitRate: `${stats.stats.cache.keyCount}/${stats.stats.cache.maxSize}`,
781
- s3Operations: stats.stats.costs.requestsByType
782
- });
783
-
784
- // Cache performance
785
- if (stats.stats.cache.enabled) {
786
- console.log('Cache Performance:', {
787
- driver: stats.stats.cache.driver,
788
- itemsCached: stats.stats.cache.size,
789
- maxCapacity: stats.stats.cache.maxSize,
790
- ttl: `${stats.stats.cache.ttl / 1000}s`,
791
- sampleKeys: stats.stats.cache.sampleKeys
792
- });
793
- }
794
-
795
- // S3 costs breakdown
796
- if (stats.stats.costs) {
797
- console.log('S3 Costs Breakdown:', {
798
- totalRequests: stats.stats.costs.totalRequests,
799
- getRequests: stats.stats.costs.requestsByType.get,
800
- putRequests: stats.stats.costs.requestsByType.put,
801
- listRequests: stats.stats.costs.requestsByType.list,
802
- estimatedCost: `$${stats.stats.costs.estimatedCostUSD.toFixed(6)}`
803
- });
804
- }
805
-
806
- // Clear cache for performance reset
807
- await agent.callTool('dbClearCache'); // Clear all cache
808
- // or
809
- await agent.callTool('dbClearCache', { resourceName: 'posts' }); // Clear specific resource
810
- ```
811
-
812
- ### Batch Operations
813
-
814
- ```javascript
815
- // Insert multiple documents at once
816
- await agent.callTool('resourceInsertMany', {
817
- resourceName: 'posts',
818
- data: [
819
- {
820
- title: 'AI and Databases',
821
- content: 'Exploring the intersection...',
822
- author: 'jane-smith',
823
- published: true
824
- },
825
- {
826
- title: 'S3DB Performance Tips',
827
- content: 'Best practices for...',
828
- author: 'bob-wilson',
829
- published: false
830
- }
831
- ]
832
- });
833
-
834
- // Get multiple documents by ID
835
- const multiplePosts = await agent.callTool('resourceGetMany', {
836
- resourceName: 'posts',
837
- ids: ['post_123', 'post_456', 'post_789']
838
- });
839
-
840
- // Delete multiple documents
841
- await agent.callTool('resourceDeleteMany', {
842
- resourceName: 'posts',
843
- ids: ['post_old1', 'post_old2']
844
- });
845
- ```
846
-
847
- ### E-commerce Example with Complex Schema
848
-
849
- ```javascript
850
- // Create products resource
851
- await agent.callTool('dbCreateResource', {
852
- name: 'products',
853
- attributes: {
854
- sku: 'string|required|unique',
855
- name: 'string|required|min:2|max:200',
856
- description: 'string|required',
857
- price: 'number|positive|required',
858
- category: 'string|required',
859
- subcategory: 'string|optional',
860
- inStock: 'boolean',
861
- inventory: {
862
- quantity: 'number|integer|min:0',
863
- reserved: 'number|integer|min:0',
864
- warehouse: 'string|required'
865
- },
866
- specifications: {
867
- weight: 'number|positive|optional',
868
- dimensions: {
869
- length: 'number|positive',
870
- width: 'number|positive',
871
- height: 'number|positive'
872
- },
873
- color: 'string|optional',
874
- material: 'string|optional'
875
- },
876
- pricing: {
877
- cost: 'number|positive',
878
- markup: 'number|positive',
879
- discountPercent: 'number|min:0|max:100'
880
- },
881
- tags: 'array|items:string',
882
- images: 'array|items:url'
883
- },
884
- partitions: {
885
- byCategory: {
886
- fields: { category: 'string' },
887
- description: 'Partition products by main category'
888
- },
889
- byCategoryAndSubcategory: {
890
- fields: {
891
- category: 'string',
892
- subcategory: 'string'
893
- },
894
- description: 'Fine-grained category partitioning'
895
- }
896
- },
897
- timestamps: true
898
- });
899
-
900
- // Insert a complex product
901
- await agent.callTool('resourceInsert', {
902
- resourceName: 'products',
903
- data: {
904
- sku: 'LAP-GAMING-001',
905
- name: 'Gaming Laptop Pro 15"',
906
- description: 'High-performance gaming laptop with RTX graphics',
907
- price: 1299.99,
908
- category: 'electronics',
909
- subcategory: 'laptops',
910
- inStock: true,
911
- inventory: {
912
- quantity: 25,
913
- reserved: 3,
914
- warehouse: 'US-WEST-1'
915
- },
916
- specifications: {
917
- weight: 2.3,
918
- dimensions: {
919
- length: 35.5,
920
- width: 25.0,
921
- height: 2.2
922
- },
923
- color: 'black',
924
- material: 'aluminum'
925
- },
926
- pricing: {
927
- cost: 850.00,
928
- markup: 0.53,
929
- discountPercent: 0
930
- },
931
- tags: ['gaming', 'laptop', 'rtx', 'high-performance'],
932
- images: [
933
- 'https://example.com/laptop-1.jpg',
934
- 'https://example.com/laptop-2.jpg'
935
- ]
936
- }
937
- });
938
- ```
939
-
940
- ---
941
-
942
- ## ๐Ÿ—‚๏ธ Partitions & Performance
943
-
944
- Partitions organize data for better performance and logical separation.
945
-
946
- ### Creating Partitioned Resources
947
-
948
- ```javascript
949
- await agent.callTool('dbCreateResource', {
950
- name: 'orders',
951
- attributes: {
952
- orderId: 'string|required|unique',
953
- customerId: 'string|required',
954
- amount: 'number|positive|required',
955
- status: 'string|enum:pending,paid,shipped,delivered,cancelled',
956
- region: 'string|required',
957
- orderDate: 'date|required',
958
- items: 'array|items:object'
959
- },
960
- partitions: {
961
- // Single field partitions
962
- byRegion: {
963
- fields: { region: 'string' },
964
- description: 'Geographic distribution'
965
- },
966
- byStatus: {
967
- fields: { status: 'string' },
968
- description: 'Order status tracking'
969
- },
970
- byMonth: {
971
- fields: { orderDate: 'date|maxlength:7' }, // YYYY-MM format
972
- description: 'Monthly order archives'
973
- },
974
-
975
- // Multi-field partitions
976
- byRegionAndStatus: {
977
- fields: {
978
- region: 'string',
979
- status: 'string'
980
- },
981
- description: 'Regional status tracking'
982
- },
983
- byRegionAndMonth: {
984
- fields: {
985
- region: 'string',
986
- orderDate: 'date|maxlength:7'
987
- },
988
- description: 'Regional monthly reports'
989
- }
990
- },
991
- timestamps: true
992
- });
993
- ```
994
-
995
- ### Querying with Partitions
996
-
997
- ```javascript
998
- // Query specific partition - much faster than full scan
999
- const northernOrders = await agent.callTool('resourceList', {
1000
- resourceName: 'orders',
1001
- partition: 'byRegion',
1002
- partitionValues: { region: 'north' },
1003
- limit: 100
1004
- });
1005
-
1006
- // Multi-field partition query
1007
- const northPendingOrders = await agent.callTool('resourceList', {
1008
- resourceName: 'orders',
1009
- partition: 'byRegionAndStatus',
1010
- partitionValues: {
1011
- region: 'north',
1012
- status: 'pending'
1013
- }
1014
- });
1015
-
1016
- // Time-based partition query
1017
- const januaryOrders = await agent.callTool('resourceList', {
1018
- resourceName: 'orders',
1019
- partition: 'byMonth',
1020
- partitionValues: { orderDate: '2024-01' }
1021
- });
1022
-
1023
- // Count documents in partition
1024
- const pendingCount = await agent.callTool('resourceCount', {
1025
- resourceName: 'orders',
1026
- partition: 'byStatus',
1027
- partitionValues: { status: 'pending' }
1028
- });
1029
- ```
1030
-
1031
- ### Automatic Partition Migration (v9.2.2+)
1032
-
1033
- **๐ŸŽฏ NEW FEATURE**: Records automatically move between partitions when you update partition fields!
1034
-
1035
- ```javascript
1036
- // 1. Insert order with status 'pending' - goes to 'pending' partition
1037
- const order = await agent.callTool('resourceInsert', {
1038
- resourceName: 'orders',
1039
- data: {
1040
- orderId: 'ORD-001',
1041
- customerId: 'CUST-123',
1042
- amount: 299.99,
1043
- status: 'pending', // Goes to 'pending' partition
1044
- region: 'north'
1045
- }
1046
- });
1047
-
1048
- // 2. Update status to 'shipped' - AUTOMATICALLY moves to 'shipped' partition!
1049
- await agent.callTool('resourceUpdate', {
1050
- resourceName: 'orders',
1051
- id: order.id,
1052
- data: {
1053
- ...order,
1054
- status: 'shipped' // Automatically moved from 'pending' to 'shipped' partition
1055
- }
1056
- });
1057
-
1058
- // The record is now:
1059
- // โœ… In the 'shipped' partition
1060
- // โŒ NOT in the 'pending' partition anymore (automatically cleaned up!)
1061
- ```
1062
-
1063
- ### Partition Best Practices
1064
-
1065
- **Common Partition Patterns:**
1066
- - **By Date**: `{ orderDate: 'date|maxlength:10' }` (YYYY-MM-DD)
1067
- - **By Month**: `{ orderDate: 'date|maxlength:7' }` (YYYY-MM)
1068
- - **By Category**: `{ category: 'string' }`
1069
- - **By User**: `{ userId: 'string' }`
1070
- - **By Status**: `{ status: 'string' }`
1071
- - **By Geographic Region**: `{ region: 'string', country: 'string' }`
1072
-
1073
- **Performance Benefits:**
1074
- - โšก **Faster queries** - scans only relevant partition
1075
- - ๐Ÿ’ฐ **Lower S3 costs** - fewer requests and data transfer
1076
- - ๐Ÿ“Š **Better analytics** - efficient aggregations
1077
- - ๐Ÿ”„ **Easier maintenance** - targeted operations
1078
-
1079
- ---
1080
-
1081
- ## ๐Ÿณ Docker Deployment
1082
-
1083
- ### Quick Start with Docker Compose
1084
-
1085
- ```bash
1086
- # 1. Create project directory
1087
- mkdir s3db-mcp && cd s3db-mcp
1088
-
1089
- # 2. Create docker-compose.yml
1090
- curl -o docker-compose.yml https://raw.githubusercontent.com/forattini-dev/s3db.js/main/mcp-server/docker-compose.yml
1091
-
1092
- # 3. Create .env file
1093
- curl -o .env.example https://raw.githubusercontent.com/forattini-dev/s3db.js/main/mcp-server/.env.example
1094
- cp .env.example .env
1095
-
1096
- # 4. Edit .env with your configuration
1097
- # 5. Start services
1098
- docker compose up
1099
- ```
1100
-
1101
- ### Production Docker Setup
1102
-
1103
- ```yaml
1104
- services:
1105
- s3db-mcp-server:
1106
- image: s3db-mcp-server:latest
1107
- restart: unless-stopped
1108
- environment:
1109
- - NODE_ENV=production
1110
- - S3DB_CONNECTION_STRING=s3://bucket/databases/prod
1111
- - MCP_TRANSPORT=sse
1112
- - MCP_SERVER_PORT=17500
1113
- ports:
1114
- - "17500:8000"
1115
- healthcheck:
1116
- test: ["CMD", "curl", "-f", "http://localhost:8001/health"]
1117
- interval: 30s
1118
- timeout: 10s
1119
- retries: 3
1120
- logging:
1121
- driver: "json-file"
1122
- options:
1123
- max-size: "10m"
1124
- max-file: "3"
1125
- ```
1126
-
1127
- ### Local Development with MinIO
1128
-
1129
- ```bash
1130
- # Start with MinIO for local S3 testing
1131
- docker compose --profile local-testing up
1132
-
1133
- # Access:
1134
- # - MCP Server: http://localhost:17500/sse
1135
- # - MinIO Console: http://localhost:9001 (minioadmin/minioadmin)
1136
- # - Health Check: http://localhost:8001/health
1137
- ```
1138
-
1139
- ### Docker Environment Variables
1140
-
1141
- All the configuration variables mentioned above can be used in Docker:
1142
-
1143
- ```bash
1144
- docker run -p 17500:8000 \
1145
- -e S3DB_CONNECTION_STRING="s3://key:secret@bucket/db" \
1146
- -e S3DB_VERBOSE=true \
1147
- -e S3DB_PARALLELISM=20 \
1148
- -e MCP_TRANSPORT=sse \
1149
- s3db-mcp-server
1150
- ```
1151
-
1152
- ---
1153
-
1154
- ## ๐Ÿค– AI Agent Integration
1155
-
1156
- ### Claude Desktop Integration
1157
-
1158
- 1. **Locate config file:**
1159
- - macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
1160
- - Windows: `%APPDATA%\Claude\claude_desktop_config.json`
1161
-
1162
- 2. **Add S3DB MCP server:**
1163
- ```json
1164
- {
1165
- "mcpServers": {
1166
- "s3db": {
1167
- "transport": "sse",
1168
- "url": "http://localhost:17500/sse"
1169
- }
1170
- }
1171
- }
1172
- ```
1173
-
1174
- 3. **For STDIO transport:**
1175
- ```json
1176
- {
1177
- "mcpServers": {
1178
- "s3db": {
1179
- "transport": "stdio",
1180
- "command": "npx",
1181
- "args": ["s3db-mcp-server", "--transport=stdio"],
1182
- "env": {
1183
- "S3DB_CONNECTION_STRING": "s3://bucket/databases/myapp"
1184
- }
1185
- }
1186
- }
1187
- }
1188
- ```
1189
-
1190
- ### Cursor IDE Integration
1191
-
1192
- Add to your MCP settings:
1193
- ```json
1194
- {
1195
- "mcpServers": {
1196
- "s3db": {
1197
- "url": "http://localhost:17500/sse"
1198
- }
1199
- }
1200
- }
1201
- ```
1202
-
1203
- ### AI Agent Usage Rules
1204
-
1205
- **Before any task:**
1206
- 1. Always use `dbConnect` first to establish connection (cache and costs tracking are enabled by default)
1207
- 2. Use `dbStatus` to verify connection and see resources
1208
- 3. Use `dbListResources` to see available collections
1209
-
1210
- **For data operations:**
1211
- 1. Use `resourceExists` to check if documents exist before operations
1212
- 2. Prefer batch operations (`resourceInsertMany`, `resourceGetMany`) for efficiency
1213
- 3. Use partitions for performance when querying large datasets
1214
- 4. Always use pagination (`resourceList` with `limit`/`offset`) for large results
1215
-
1216
- **Schema design:**
1217
- - Define validation rules: `"email": "email|required|unique"`
1218
- - Use nested objects for complex data structures
1219
- - Enable timestamps for audit trails
1220
- - Consider partitioning strategy upfront
1221
-
1222
- **Performance monitoring:**
1223
- - Use `dbGetStats` to monitor S3 costs and cache performance
1224
- - Cache is automatically enabled for read operations (get, list, count)
1225
- - Use `dbClearCache` to reset cache when needed
1226
- - Monitor costs to optimize S3 usage patterns
1227
-
1228
- **Error handling:**
1229
- - Check connection status before operations
1230
- - Validate data structure matches schema
1231
- - Handle resource not found errors gracefully
1232
- - Use appropriate error messages for users
1233
-
1234
- ---
1235
-
1236
- ## ๐Ÿ”ง Advanced Usage
1237
-
1238
- ### ๐Ÿ”„ Auto-restart on Failure
1239
-
1240
- Using systemd (Linux):
1241
- ```ini
1242
- # /etc/systemd/system/s3db-mcp.service
1243
- [Unit]
1244
- Description=S3DB MCP Server
1245
- After=network.target
1246
-
1247
- [Service]
1248
- Type=simple
1249
- User=youruser
1250
- WorkingDirectory=/home/youruser
1251
- ExecStart=/usr/bin/npx s3db-mcp-server --transport=sse
1252
- Restart=always
1253
- RestartSec=10
1254
- Environment="S3DB_CONNECTION_STRING=s3://..."
1255
-
1256
- [Install]
1257
- WantedBy=multi-user.target
1258
- ```
1259
-
1260
- ```bash
1261
- sudo systemctl enable s3db-mcp
1262
- sudo systemctl start s3db-mcp
1263
- sudo systemctl status s3db-mcp
1264
- ```
1265
-
1266
- ### ๐Ÿ” Secure Credential Management
1267
-
1268
- Instead of hardcoding credentials:
1269
-
1270
- **Option 1: Use AWS Credentials File**
1271
- ```json
1272
- {
1273
- "mcpServers": {
1274
- "s3db": {
1275
- "command": "npx",
1276
- "args": ["s3db-mcp-server", "--transport=sse"],
1277
- "env": {
1278
- "S3DB_CONNECTION_STRING": "s3://bucket/databases/app",
1279
- "AWS_PROFILE": "production"
1280
- }
1281
- }
1282
- }
1283
- }
1284
- ```
1285
-
1286
- **Option 2: Use Environment Variables**
1287
- ```bash
1288
- # Set in your shell profile
1289
- export S3DB_CONNECTION_STRING="s3://..."
1290
- export AWS_ACCESS_KEY_ID="..."
1291
- export AWS_SECRET_ACCESS_KEY="..."
1292
- ```
1293
-
1294
- ```json
1295
- {
1296
- "mcpServers": {
1297
- "s3db": {
1298
- "command": "npx",
1299
- "args": ["s3db-mcp-server", "--transport=sse"]
1300
- // No env section - uses system environment
1301
- }
1302
- }
1303
- }
1304
- ```
1305
-
1306
- ### ๐ŸŽฏ Performance Optimization
1307
-
1308
- For production workloads:
1309
- ```json
1310
- {
1311
- "mcpServers": {
1312
- "s3db-optimized": {
1313
- "command": "npx",
1314
- "args": ["s3db-mcp-server", "--transport=sse"],
1315
- "env": {
1316
- "S3DB_PARALLELISM": "20",
1317
- "S3DB_CACHE_DRIVER": "filesystem",
1318
- "S3DB_CACHE_DIRECTORY": "/mnt/ssd/s3db-cache",
1319
- "S3DB_CACHE_MAX_SIZE": "10000",
1320
- "S3DB_CACHE_TTL": "3600000",
1321
- "NODE_ENV": "production"
1322
- }
1323
- }
1324
- }
1325
- }
1326
- ```
1327
-
1328
- ---
1329
-
1330
- ## ๐Ÿ”’ Security
1331
-
1332
- ### AWS IAM Policy
1333
-
1334
- Minimal S3 permissions required:
1335
-
1336
- ```json
1337
- {
1338
- "Version": "2012-10-17",
1339
- "Statement": [
1340
- {
1341
- "Effect": "Allow",
1342
- "Action": [
1343
- "s3:GetObject",
1344
- "s3:PutObject",
1345
- "s3:DeleteObject",
1346
- "s3:ListBucket",
1347
- "s3:HeadObject"
1348
- ],
1349
- "Resource": [
1350
- "arn:aws:s3:::your-s3db-bucket",
1351
- "arn:aws:s3:::your-s3db-bucket/*"
1352
- ]
1353
- }
1354
- ]
1355
- }
1356
- ```
1357
-
1358
- ### Security Best Practices
1359
-
1360
- 1. **Use IAM roles** when possible instead of access keys
1361
- 2. **Rotate credentials** regularly
1362
- 3. **Use environment variables** never hardcode credentials
1363
- 4. **Enable S3 bucket encryption** and versioning
1364
- 5. **Monitor access logs** and set up CloudTrail
1365
- 6. **Use strong passphrases** for S3DB encryption
1366
- 7. **Restrict network access** with security groups
1367
- 8. **Enable HTTPS** for all connections
1368
-
1369
- ### Field-Level Encryption
1370
-
1371
- ```javascript
1372
- await agent.callTool('dbCreateResource', {
1373
- name: 'sensitive_data',
1374
- attributes: {
1375
- publicInfo: 'string',
1376
- privateData: 'secret', // Automatically encrypted
1377
- ssn: 'secret', // Encrypted with passphrase
1378
- creditCard: 'secret' // Encrypted
1379
- }
1380
- });
1381
- ```
1382
-
1383
- ---
1384
-
1385
- ## ๐Ÿšจ Troubleshooting
1386
-
1387
- ### โŒ Common Issues and Solutions
1388
-
1389
- #### 1. MCP Server Not Starting
1390
-
1391
- **Issue**: Claude Desktop can't connect to MCP server
1392
-
1393
- **Solution**:
1394
- ```bash
1395
- # Check if port is already in use
1396
- lsof -i :17500 # macOS/Linux
1397
- netstat -an | findstr :17500 # Windows
1398
-
1399
- # Use a different port
1400
- {
1401
- "mcpServers": {
1402
- "s3db": {
1403
- "command": "npx",
1404
- "args": ["s3db-mcp-server", "--transport=sse", "--port=18000"],
1405
- "env": {
1406
- "MCP_SERVER_PORT": "18000"
1407
- }
1408
- }
1409
- }
1410
- }
1411
- ```
1412
-
1413
- #### 2. Connection String Issues
1414
-
1415
- **Issue**: "Invalid connection string" error
1416
-
1417
- **Solution**:
1418
- ```bash
1419
- # Test connection string directly
1420
- export S3DB_CONNECTION_STRING="s3://..."
1421
- npx s3db-mcp-server --transport=sse
1422
-
1423
- # Common fixes:
1424
- # 1. URL-encode special characters in credentials
1425
- # 2. Add forcePathStyle=true for MinIO
1426
- # 3. Ensure bucket exists
1427
- ```
1428
-
1429
- #### 3. Permission Errors
1430
-
1431
- **Issue**: "Access Denied" when accessing S3
1432
-
1433
- **Solution**:
1434
- ```json
1435
- {
1436
- "env": {
1437
- // Ensure credentials are correct
1438
- "AWS_ACCESS_KEY_ID": "your-key",
1439
- "AWS_SECRET_ACCESS_KEY": "your-secret",
1440
- "AWS_REGION": "correct-region"
1441
- }
1442
- }
1443
- ```
1444
-
1445
- #### 4. Cache Not Working
1446
-
1447
- **Issue**: Slow performance, no cache hits
1448
-
1449
- **Solution**:
1450
- ```json
1451
- {
1452
- "env": {
1453
- "S3DB_CACHE_ENABLED": "true", // Must be string "true"
1454
- "S3DB_CACHE_DRIVER": "memory",
1455
- "S3DB_CACHE_MAX_SIZE": "5000", // Increase size
1456
- "S3DB_CACHE_TTL": "600000" // 10 minutes
1457
- }
1458
- }
1459
- ```
1460
-
1461
- ### ๐Ÿ” Debugging
1462
-
1463
- Enable verbose logging:
1464
- ```json
1465
- {
1466
- "mcpServers": {
1467
- "s3db": {
1468
- "command": "npx",
1469
- "args": ["s3db-mcp-server", "--transport=sse"],
1470
- "env": {
1471
- "S3DB_VERBOSE": "true",
1472
- "NODE_ENV": "development",
1473
- "DEBUG": "true"
1474
- }
1475
- }
1476
- }
1477
- }
1478
- ```
1479
-
1480
- View MCP logs in Claude Desktop:
1481
- 1. Open Developer Tools: `Cmd+Option+I` (Mac) or `Ctrl+Shift+I` (Windows)
1482
- 2. Go to Console tab
1483
- 3. Filter for "mcp" messages
1484
-
1485
- ### S3 Access Issues
1486
-
1487
- ```bash
1488
- # Test S3 connection
1489
- aws s3 ls s3://your-bucket
1490
-
1491
- # Check credentials
1492
- aws sts get-caller-identity
1493
-
1494
- # Test with MinIO
1495
- mc alias set local http://localhost:9000 minioadmin minioadmin
1496
- mc ls local
1497
- ```
1498
-
1499
- **Performance Issues:**
1500
- - Increase `S3DB_PARALLELISM` for better throughput
1501
- - Use partitions to reduce query scope
1502
- - Implement proper pagination
1503
- - Monitor S3 request patterns
1504
-
1505
- **Memory Issues:**
1506
- - Avoid `resourceGetAll` on large datasets
1507
- - Use `resourceList` with pagination instead
1508
- - Increase Docker memory limits if needed
1509
-
1510
- ### Error Messages
1511
-
1512
- | Error | Cause | Solution |
1513
- |-------|-------|----------|
1514
- | "Database not connected" | No `dbConnect` called | Call `dbConnect` tool first |
1515
- | "Resource not found" | Invalid resource name | Check with `dbListResources` |
1516
- | "Validation failed" | Data doesn't match schema | Review attribute definitions |
1517
- | "Connection string invalid" | Malformed connection string | Check format: `s3://key:secret@bucket/path` |
1518
- | "Health check failed" | Server not responding | Check if process is running on correct port |
1519
-
1520
- ### Debug Mode
1521
-
1522
- Enable verbose logging:
1523
- ```bash
1524
- # Environment variable
1525
- export S3DB_VERBOSE=true
1526
-
1527
- # Command line
1528
- s3db-mcp --transport=sse
1529
-
1530
- # Docker
1531
- docker run -e S3DB_VERBOSE=true s3db-mcp-server
1532
- ```
1533
-
1534
- ### Health Monitoring
1535
-
1536
- ```bash
1537
- # Check server health
1538
- curl http://localhost:8001/health
1539
-
1540
- # Response includes:
1541
- {
1542
- "status": "healthy",
1543
- "database": {
1544
- "connected": true,
1545
- "bucket": "my-bucket",
1546
- "resourceCount": 5
1547
- },
1548
- "memory": { "rss": 45000000 },
1549
- "uptime": 3600
1550
- }
1551
- ```
1552
-
1553
- ### ๐ŸŽ‰ Success Indicators
1554
-
1555
- You know your setup is working when:
1556
- 1. โœ… Claude Desktop shows "Connected to s3db" in the MCP status
1557
- 2. โœ… You can run database commands without errors
1558
- 3. โœ… Cache statistics show hits after repeated queries
1559
- 4. โœ… Cost tracking shows S3 operation counts
1560
-
1561
- ### ๐Ÿ’ก Pro Tips
1562
-
1563
- 1. **Start simple**: Test with MinIO locally before using AWS S3
1564
- 2. **Monitor costs**: Use `dbGetStats` regularly to track S3 expenses
1565
- 3. **Optimize partitions**: Design partitions based on query patterns
1566
- 4. **Cache wisely**: Filesystem cache survives restarts, memory cache is faster
1567
- 5. **Use batch operations**: `insertMany` is much faster than multiple `insert` calls
1568
-
1569
- ---
1570
-
1571
- ## ๐Ÿ”Œ Built-in Performance Features
1572
-
1573
- The S3DB MCP Server includes **automatic performance optimizations**:
1574
-
1575
- ### **๐ŸŽ๏ธ Configurable Cache (Enabled by Default)**
1576
- - **Two cache drivers**: Memory (fast, temporary) and Filesystem (persistent)
1577
- - **Automatic caching** of read operations (get, list, count, exists)
1578
- - **Partition-aware** caching for optimized queries
1579
- - **Configurable TTL** and size limits
1580
- - **Cache invalidation** on write operations
1581
- - **Performance monitoring** via `dbGetStats`
1582
-
1583
- #### **Memory Cache**
1584
- - โšก **Fastest performance** for frequently accessed data
1585
- - ๐Ÿ”„ **Lost on restart** - ideal for temporary caching
1586
- - ๐Ÿ“Š **Size-limited** by number of items
1587
-
1588
- #### **Filesystem Cache**
1589
- - ๐Ÿ’พ **Persistent across restarts** - cache survives server restarts
1590
- - ๐Ÿ—œ๏ธ **Automatic compression** to save disk space
1591
- - ๐Ÿงน **Automatic cleanup** of expired files
1592
- - ๐Ÿ“ **Configurable directory** and file naming
1593
-
1594
- ### **๐Ÿ’ฐ Costs Tracking (Enabled by Default)**
1595
- - **Real-time S3 costs** calculation
1596
- - **Request counting** by operation type
1597
- - **Cost estimation** in USD
1598
- - **Performance analytics** for optimization
1599
-
1600
- ### **Configuration Options**
1601
- ```javascript
1602
- // Connect with memory cache (fast, temporary)
1603
- await agent.callTool('dbConnect', {
1604
- connectionString: 's3://...',
1605
- enableCache: true, // Default: true
1606
- enableCosts: true, // Default: true
1607
- cacheDriver: 'memory', // Fast but lost on restart
1608
- cacheMaxSize: 2000, // Default: 1000
1609
- cacheTtl: 600000 // Default: 300000 (5 min)
1610
- });
1611
-
1612
- // Connect with filesystem cache (persistent)
1613
- await agent.callTool('dbConnect', {
1614
- connectionString: 's3://...',
1615
- enableCache: true,
1616
- cacheDriver: 'filesystem', // Survives restarts
1617
- cacheDirectory: './data/cache',
1618
- cachePrefix: 'myapp',
1619
- cacheTtl: 1800000 // 30 minutes
1620
- });
1621
-
1622
- // Monitor performance
1623
- const stats = await agent.callTool('dbGetStats');
1624
- console.log('Cache size:', stats.stats.cache.size);
1625
- console.log('Cache driver:', stats.stats.cache.driver);
1626
- console.log('S3 costs:', stats.stats.costs.estimatedCostUSD);
1627
-
1628
- // Clear cache when needed
1629
- await agent.callTool('dbClearCache', { resourceName: 'users' });
1630
- ```
1631
-
1632
- ### **Environment Variables**
1633
- ```bash
1634
- S3DB_CACHE_ENABLED=true # Enable/disable cache
1635
- S3DB_CACHE_DRIVER=memory # Cache driver: 'memory' or 'filesystem'
1636
- S3DB_CACHE_MAX_SIZE=1000 # Cache capacity (memory driver)
1637
- S3DB_CACHE_TTL=300000 # 5 minute TTL
1638
- S3DB_CACHE_DIRECTORY=./cache # Filesystem cache directory
1639
- S3DB_CACHE_PREFIX=s3db # Filesystem cache file prefix
1640
- S3DB_COSTS_ENABLED=true # Enable/disable costs tracking
1641
- ```
1642
-
1643
- ---
1644
-
1645
- ## ๐Ÿš€ **Cache Strategy Guide**
1646
-
1647
- Choose the right cache driver for your use case:
1648
-
1649
- ### **When to Use Memory Cache**
1650
- - โšก **Development & Testing** - fastest performance, no setup required
1651
- - ๐Ÿ”„ **Short-lived processes** - containers that restart frequently
1652
- - ๐Ÿ“Š **High-frequency reads** - when you need maximum speed
1653
- - ๐Ÿ’ฐ **Cost optimization** - minimize S3 requests for hot data
1654
- - โš ๏ธ **Limitation**: Cache is lost on restart
1655
-
1656
- ### **When to Use Filesystem Cache**
1657
- - ๐Ÿ’พ **Production environments** - cache survives server restarts
1658
- - ๐Ÿ”„ **Long-running processes** - persistent data across deployments
1659
- - ๐Ÿ“ฆ **Containerized deployments** - mount cache volume for persistence
1660
- - ๐Ÿ”ง **Development consistency** - maintain cache between code changes
1661
- - ๐Ÿ—‚๏ธ **Large datasets** - no memory size limitations
1662
-
1663
- ### **Configuration Examples**
1664
-
1665
- ```javascript
1666
- // High-performance temporary cache
1667
- await agent.callTool('dbConnect', {
1668
- cacheDriver: 'memory',
1669
- cacheMaxSize: 5000,
1670
- cacheTtl: 600000 // 10 minutes
1671
- });
1672
-
1673
- // Production persistent cache
1674
- await agent.callTool('dbConnect', {
1675
- cacheDriver: 'filesystem',
1676
- cacheDirectory: './data/cache',
1677
- cachePrefix: 'prod',
1678
- cacheTtl: 3600000 // 1 hour
1679
- });
1680
- ```
1681
-
1682
- ### **Docker Volume Setup**
1683
- ```yaml
1684
- # docker-compose.yml
1685
- volumes:
1686
- - ./cache-data:/app/cache # Persistent filesystem cache
1687
- environment:
1688
- - S3DB_CACHE_DRIVER=filesystem
1689
- - S3DB_CACHE_DIRECTORY=/app/cache
1690
- ```
1691
-
1692
- ---
1693
-
1694
- ## ๐Ÿ“Š Performance Tips
1695
-
1696
- 1. **Choose appropriate cache** - memory for speed, filesystem for persistence
1697
- 2. **Leverage built-in cache** - read operations are automatically cached
1698
- 3. **Use partitions** for large datasets to improve cache efficiency
1699
- 4. **Monitor costs** with `dbGetStats` to optimize S3 usage
1700
- 5. **Batch operations** when possible to reduce S3 requests
1701
- 6. **Proper pagination** - don't load everything at once
1702
- 7. **Connection reuse** - keep connections alive
1703
- 8. **Appropriate parallelism** - tune `S3DB_PARALLELISM`
1704
-
1705
- ---
1706
-
1707
- ## ๐Ÿ“š Additional Resources
1708
-
1709
- - [S3DB Documentation](https://github.com/forattini-dev/s3db.js)
1710
- - [MCP Protocol Specification](https://modelcontextprotocol.io)
1711
- - [Claude Desktop Downloads](https://claude.ai/download)
1712
- - [GitHub Issues](https://github.com/forattini-dev/s3db.js/issues)
1713
- - [NPM Package](https://www.npmjs.com/package/s3db-mcp-server)
1714
-
1715
- Need help? Check the [Troubleshooting](#troubleshooting) section or file an issue on [GitHub](https://github.com/forattini-dev/s3db.js/issues).
1716
-
1717
- ---
1718
-
1719
- ## ๐Ÿ“„ License
1720
-
1721
- This project is licensed under the same license as the parent S3DB project.
1722
-
1723
- ---
1724
-
1725
- <p align="center">
1726
- <strong>๐ŸŽ‰ Ready to supercharge your AI agents with persistent data storage!</strong><br>
1727
- <em>Start building with S3DB MCP Server today</em>
1728
- </p>