db-mcp 1.1.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.gitleaks.toml +9 -0
- package/.trivyignore +8 -0
- package/README.md +193 -120
- package/dist/{chunk-DWJXQEFA.js → chunk-5Y42NPBP.js} +4168 -2411
- package/dist/{chunk-AOUL5SHS.js → chunk-645ZEFLA.js} +70 -20
- package/dist/chunk-OKOVZ5QE.js +28 -0
- package/dist/chunk-SFJQCNG7.js +131 -0
- package/dist/{chunk-S5IDDSSB.js → chunk-VIDSICEL.js} +12 -1
- package/dist/chunk-WBER5YY4.js +2053 -0
- package/dist/{chunk-4IA3DB5C.js → chunk-X3MUUOWM.js} +19 -2
- package/dist/{chunk-Z2GFQU3G.js → chunk-Z7C2TM4L.js} +114 -21
- package/dist/cli.js +79 -5
- package/dist/{http-VSB7DBJR.js → http-6KF4ULDI.js} +199 -77
- package/dist/index.d.ts +231 -8
- package/dist/index.js +6 -5
- package/dist/{sqlite-VHBA4ABV.js → sqlite-U5KSYQXK.js} +63 -140
- package/dist/{sqlite-native-D2LH5ZT7.js → sqlite-native-JXMCFQBA.js} +576 -114
- package/dist/worker-script.js +34 -10
- package/logs/.gitkeep +1 -0
- package/mcp-config-example.json +83 -0
- package/package.json +9 -8
- package/playwright.config.ts +1 -1
- package/scripts/update-badges.ts +99 -0
- package/server.json +7 -5
- package/test-server/README.md +20 -23
- package/test-server/code-map.md +45 -33
- package/test-server/reset-database.ps1 +59 -17
- package/test-server/scripts/README.md +27 -0
- package/test-server/{test-help-resources.mjs → scripts/test-help-resources.mjs} +12 -5
- package/test-server/scripts/test-prompts.mjs +251 -0
- package/test-server/{test-tool-annotations.mjs → scripts/test-tool-annotations.mjs} +9 -4
- package/test-server/test-advanced/README.md +70 -0
- package/test-server/test-advanced/test-codemode-advanced-admin.md +174 -0
- package/test-server/test-advanced/test-codemode-advanced-core.md +193 -0
- package/test-server/test-advanced/test-codemode-advanced-geo.md +157 -0
- package/test-server/test-advanced/test-codemode-advanced-introspection.md +171 -0
- package/test-server/test-advanced/test-codemode-advanced-json.md +169 -0
- package/test-server/test-advanced/test-codemode-advanced-migration.md +160 -0
- package/test-server/test-advanced/test-codemode-advanced-stats.md +185 -0
- package/test-server/test-advanced/test-codemode-advanced-text.md +183 -0
- package/test-server/test-advanced/test-codemode-advanced-transactions.md +136 -0
- package/test-server/test-advanced/test-codemode-advanced-vector.md +141 -0
- package/test-server/test-codemode/README.md +121 -0
- package/test-server/test-codemode/test-codemode-admin.md +223 -0
- package/test-server/test-codemode/test-codemode-core.md +286 -0
- package/test-server/test-codemode/test-codemode-geo.md +177 -0
- package/test-server/test-codemode/test-codemode-introspection.md +190 -0
- package/test-server/test-codemode/test-codemode-json.md +237 -0
- package/test-server/test-codemode/test-codemode-migration.md +278 -0
- package/test-server/test-codemode/test-codemode-sandbox.md +413 -0
- package/test-server/test-codemode/test-codemode-stats.md +232 -0
- package/test-server/test-codemode/test-codemode-text.md +237 -0
- package/test-server/test-codemode/test-codemode-transactions.md +236 -0
- package/test-server/test-codemode/test-codemode-vector.md +244 -0
- package/test-server/test-codemode/test-codemode-wasm-degradation.md +394 -0
- package/test-server/test-database.sql +37 -1
- package/test-server/test-resources.md +43 -16
- package/test-server/test-tool-groups/README.md +100 -0
- package/test-server/test-tool-groups/test-admin-core.md +165 -0
- package/test-server/test-tool-groups/test-admin-extensions.md +133 -0
- package/test-server/{test-tools.md → test-tool-groups/test-core-data.md} +103 -17
- package/test-server/test-tool-groups/test-core-schema.md +240 -0
- package/test-server/test-tool-groups/test-geo-haversine.md +130 -0
- package/test-server/test-tool-groups/test-geo-spatialite.md +110 -0
- package/test-server/test-tool-groups/test-introspection-diagnostics.md +123 -0
- package/test-server/test-tool-groups/test-introspection-schema.md +133 -0
- package/test-server/test-tool-groups/test-json-read.md +219 -0
- package/test-server/test-tool-groups/test-json-write.md +157 -0
- package/test-server/test-tool-groups/test-migration.md +193 -0
- package/test-server/test-tool-groups/test-stats-advanced.md +118 -0
- package/test-server/test-tool-groups/test-stats-basic.md +156 -0
- package/test-server/test-tool-groups/test-text-advanced.md +169 -0
- package/test-server/test-tool-groups/test-text-basic.md +177 -0
- package/test-server/test-tool-groups/test-transactions.md +179 -0
- package/test-server/test-tool-groups/test-vector-read.md +130 -0
- package/test-server/test-tool-groups/test-vector-write.md +115 -0
- package/test-server/tool-reference.md +79 -58
- package/tsconfig.build.json +6 -0
- package/dist/chunk-DZQLDEQS.js +0 -879
- package/test-server/test-agent-experience.md +0 -243
- package/test-server/test-group-tools.md +0 -861
- package/test-server/test-tools-advanced-1.md +0 -517
- package/test-server/test-tools-advanced-2.md +0 -487
- package/test-server/test-tools-codemode.md +0 -629
|
@@ -0,0 +1,193 @@
|
|
|
1
|
+
# db-mcp Tool Group Testing: [migration]
|
|
2
|
+
|
|
3
|
+
> [!IMPORTANT]
|
|
4
|
+
> **Do not track progress in this file.** Track your test progress, coverage matrix, and findings in your internal task tracking system (artifact). However, you SHOULD edit this file to fix any factual errors, broken code, or incorrect assertions in the test prompts.
|
|
5
|
+
> If there is nothing to fix, don't update UNRELEASED.md.
|
|
6
|
+
> We're currently testing Native mode.
|
|
7
|
+
|
|
8
|
+
## WASM Mode
|
|
9
|
+
|
|
10
|
+
> When testing against a **WASM backend** (`sqlite-wasm` / sql.js): All 6 migration tools are fully WASM-compatible. No items to skip or adjust.
|
|
11
|
+
|
|
12
|
+
**Step 1:** Confirm you read the server help content sourced from `C:\Users\chris\Desktop\db-mcp\src\constants\server-instructions\gotchas.md` using `view_file` (not grep or search) — to understand documented behaviors, edge cases, and response structures for this tool group.
|
|
13
|
+
|
|
14
|
+
**Step 2:** Please conduct an exhaustive test of the **migration** tool group specified in the group-specific checklist below using live MCP server tool calls directly — not scripts/terminal.
|
|
15
|
+
|
|
16
|
+
**Step 3:** The agent should update C:\Users\chris\Desktop\db-mcp\UNRELEASED.md with any/all changes/fixes.
|
|
17
|
+
|
|
18
|
+
**Note** If temp tables are present from a previous test pass, it's because the database is locked. Ignore them.
|
|
19
|
+
|
|
20
|
+
## Reporting Format
|
|
21
|
+
|
|
22
|
+
- ❌ Fail: Tool errors or produces incorrect results (include error message)
|
|
23
|
+
- ⚠️ Issue: Unexpected behavior or improvement opportunity
|
|
24
|
+
- 📦 Payload: Unnecessarily large response that should be optimized — **blocking, equally important as ❌ bugs**. Report the response size in KB and suggest a concrete optimization.
|
|
25
|
+
|
|
26
|
+
## Test Database Schema
|
|
27
|
+
|
|
28
|
+
| Table | Rows | Columns | JSON Columns |
|
|
29
|
+
| ----------------- | ---- | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
|
30
|
+
| test_products | 16 | id, name, description, price, category, created_at | — |
|
|
31
|
+
| test_orders | 20 | id, product_id (FK), customer_name, quantity, total_price, order_date, status | — |
|
|
32
|
+
| test_jsonb_docs | 6 | id, doc, metadata, tags, created_at | **doc**, **metadata** (nested), **tags** (array) |
|
|
33
|
+
| test_articles | 8 | id, title, body, author, category, published_at | — |
|
|
34
|
+
| test_users | 9 | id, username, email, phone, bio, created_at | — |
|
|
35
|
+
| test_measurements | 200 | id, sensor_id, temperature, humidity, pressure, measured_at | — |
|
|
36
|
+
| test_embeddings | 20 | id, content, category, embedding | **embedding** (8-dim float array); category values: database, fitness, food, tech, travel |
|
|
37
|
+
| test_locations | 15 | id, name, city, latitude, longitude, type | — |
|
|
38
|
+
| test_categories | 17 | id, name, path, level | — |
|
|
39
|
+
| test_events | 100 | id, event_type, user_id (INT, 8 values), payload, event_date | **payload** (JSON) |
|
|
40
|
+
|
|
41
|
+
> **Note:** When testing `sqlite_execute_code`, do **not** pass `readonly: true` unless specifically testing read-only filtering.
|
|
42
|
+
|
|
43
|
+
## Testing Requirements
|
|
44
|
+
|
|
45
|
+
> [!CAUTION]
|
|
46
|
+
> **Zero tolerance for raw MCP errors.** ANY response that is a raw MCP error (e.g., `-32602`, `isError: true`, no `success` field) is a **bug that must be reported and fixed**.
|
|
47
|
+
|
|
48
|
+
1. Use existing `test_*` tables for read operations
|
|
49
|
+
2. Report all failures, unexpected behaviors, or unnecessarily large payloads
|
|
50
|
+
3. Do not mention what already works well or issues documented in help resources
|
|
51
|
+
4. **Error path testing**: For **every** tool, test (a) domain error and (b) Zod validation error (`{}`). Both must return `{success: false, error: "..."}`.
|
|
52
|
+
5. **Output schema testing**: For tools with `outputSchema`, confirm valid calls return structured JSON.
|
|
53
|
+
6. **Deterministic checklist first**: Complete ALL items before freeform exploration.
|
|
54
|
+
7. **Code Over Docs**: Fix the handler code if standards (Structured Errors/Zod) are violated. Do NOT change docs/prompts to accommodate broken code.
|
|
55
|
+
|
|
56
|
+
## Structured Error Response Pattern
|
|
57
|
+
|
|
58
|
+
```json
|
|
59
|
+
{ "success": false, "error": "Human-readable error message" }
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
| Type | Source | What you see | Verdict |
|
|
63
|
+
| -------------------- | ------------------------------------------------------------------ | ----------------------------------------------------------- | ------------------ |
|
|
64
|
+
| **Handler error** ✅ | Handler catches error and returns `{success: false, error: "..."}` | Parseable JSON object with `success` and `error` fields | Correct |
|
|
65
|
+
| **MCP error** ❌ | Uncaught throw propagates to MCP framework | Raw text error string, `isError: true` — no `success` field | Bug — report as ❌ |
|
|
66
|
+
|
|
67
|
+
### Zod Validation Errors
|
|
68
|
+
|
|
69
|
+
**Zod refinement leak pattern:** `.partial()` does NOT strip `.min(N)` / `.max(N)` refinements. **Fix:** Remove refinements from schema, validate inside handler.
|
|
70
|
+
|
|
71
|
+
- Raw MCP error (no `success` field) → report as ❌
|
|
72
|
+
- `{success: false, error: "..."}` → correct
|
|
73
|
+
- Successful response for invalid input → report as ⚠️
|
|
74
|
+
|
|
75
|
+
### Output Schema Validation Errors
|
|
76
|
+
|
|
77
|
+
If valid inputs return raw MCP `-32602` mentioning "output schema", report as ❌.
|
|
78
|
+
|
|
79
|
+
### Error Consistency Audit
|
|
80
|
+
|
|
81
|
+
1. Raw error instead of `{success: false}` → ❌
|
|
82
|
+
2. Must use `error` field name
|
|
83
|
+
3. Orphaned/inline output schemas → ⚠️
|
|
84
|
+
|
|
85
|
+
### Split Schema Pattern Verification
|
|
86
|
+
|
|
87
|
+
Verify parameter visibility and alias acceptance.
|
|
88
|
+
|
|
89
|
+
---
|
|
90
|
+
|
|
91
|
+
## Group Focus: migration
|
|
92
|
+
|
|
93
|
+
> **Instructions**: Execute every numbered checklist item with the exact inputs shown. Compare responses against the expected results. Report any deviation.
|
|
94
|
+
|
|
95
|
+
> **⚠️ After testing, run `Set-Location C:\Users\chris\Desktop\db-mcp\test-server; .\reset-database.ps1`** — migration testing creates the `_mcp_migrations` tracking table and modifies the database schema. Always reset the database after completing migration tests.
|
|
96
|
+
|
|
97
|
+
### Built-in Tools (3)
|
|
98
|
+
|
|
99
|
+
1. server_info
|
|
100
|
+
2. server_health
|
|
101
|
+
3. list_adapters
|
|
102
|
+
|
|
103
|
+
### migration Group Tools (6)
|
|
104
|
+
|
|
105
|
+
4. sqlite_migration_init
|
|
106
|
+
5. sqlite_migration_record
|
|
107
|
+
6. sqlite_migration_apply
|
|
108
|
+
7. sqlite_migration_rollback
|
|
109
|
+
8. sqlite_migration_history
|
|
110
|
+
9. sqlite_migration_status
|
|
111
|
+
10. sqlite_execute_code
|
|
112
|
+
|
|
113
|
+
**Checklist — Initialization & Recording:**
|
|
114
|
+
|
|
115
|
+
1. `sqlite_migration_init` → `{success: true}`, creates `_mcp_migrations` table (idempotent — safe to call multiple times)
|
|
116
|
+
2. `sqlite_migration_init` → call again to verify idempotency (should succeed without error)
|
|
117
|
+
3. `sqlite_migration_status` → verify empty state (no migrations applied yet)
|
|
118
|
+
4. `sqlite_migration_record({version: "1.0.0", description: "Create temp table", sql: "CREATE TABLE temp_migration_test (id INTEGER PRIMARY KEY, name TEXT)"})` → recorded (not executed — only logged)
|
|
119
|
+
5. `sqlite_migration_status` → verify migration count incremented
|
|
120
|
+
6. `sqlite_migration_history` → verify migration 1.0.0 listed with status "recorded"
|
|
121
|
+
|
|
122
|
+
**Checklist — Apply & Execute:**
|
|
123
|
+
|
|
124
|
+
7. `sqlite_migration_apply({version: "1.0.1", description: "Create temp migration table", sql: "CREATE TABLE temp_migration_applied (id INTEGER PRIMARY KEY, value TEXT)"})` → applied (SQL executed AND recorded)
|
|
125
|
+
8. Verify: `sqlite_read_query({query: "SELECT name FROM sqlite_master WHERE name = 'temp_migration_applied'"})` → table exists
|
|
126
|
+
9. `sqlite_migration_history` → verify both 1.0.0 and 1.0.1 listed
|
|
127
|
+
10. `sqlite_migration_status` → verify count shows 2 migrations
|
|
128
|
+
|
|
129
|
+
**Checklist — SHA-256 Deduplication:**
|
|
130
|
+
|
|
131
|
+
11. `sqlite_migration_record({version: "1.0.2", description: "Duplicate test", sql: "CREATE TABLE temp_migration_test (id INTEGER PRIMARY KEY, name TEXT)"})` → should fail (duplicate SQL detected via SHA-256 hash)
|
|
132
|
+
|
|
133
|
+
**Checklist — Rollback:**
|
|
134
|
+
|
|
135
|
+
12. `sqlite_migration_rollback({version: "1.0.1"})` → report behavior (rollback requires rollback SQL to have been recorded; if none was provided during apply, rollback should return an informative error)
|
|
136
|
+
13. `sqlite_migration_apply({version: "1.0.3", description: "With rollback", sql: "CREATE TABLE temp_migration_rollback (id INTEGER PRIMARY KEY)", rollbackSql: "DROP TABLE IF EXISTS temp_migration_rollback"})` → applied with rollback SQL stored
|
|
137
|
+
14. `sqlite_migration_rollback({version: "1.0.3"})` → should execute the rollback SQL
|
|
138
|
+
15. Verify: `sqlite_read_query({query: "SELECT name FROM sqlite_master WHERE name = 'temp_migration_rollback'"})` → table should NOT exist
|
|
139
|
+
16. `sqlite_migration_rollback({version: "1.0.3", dryRun: true})` → report behavior (preview mode)
|
|
140
|
+
|
|
141
|
+
**Code mode testing:**
|
|
142
|
+
|
|
143
|
+
17. `sqlite_execute_code({code: "const result = await sqlite.migration.migrationStatus(); return result;"})` → migration status summary
|
|
144
|
+
18. `sqlite_execute_code({code: "const result = await sqlite.migration.migrationHistory(); return result;"})` → migration history list
|
|
145
|
+
|
|
146
|
+
**Error path testing:**
|
|
147
|
+
|
|
148
|
+
🔴 19. `sqlite_migration_apply({version: "bad version!", description: "Invalid", sql: "SELECT 1"})` → report behavior (invalid version format)
|
|
149
|
+
🔴 20. `sqlite_migration_rollback({version: "nonexistent_version"})` → structured error
|
|
150
|
+
|
|
151
|
+
**Zod validation sweep** — call each tool with `{}` (empty params). Must return handler error, NOT raw MCP error:
|
|
152
|
+
|
|
153
|
+
🔴 21. `sqlite_migration_init({})` → handler error (or success if no required params)
|
|
154
|
+
🔴 22. `sqlite_migration_record({})` → handler error
|
|
155
|
+
🔴 23. `sqlite_migration_apply({})` → handler error
|
|
156
|
+
🔴 24. `sqlite_migration_rollback({})` → handler error
|
|
157
|
+
🔴 25. `sqlite_migration_history({})` → handler error (or success if no required params)
|
|
158
|
+
🔴 26. `sqlite_migration_status({})` → handler error (or success if no required params)
|
|
159
|
+
|
|
160
|
+
---
|
|
161
|
+
|
|
162
|
+
## Post-Test Procedures
|
|
163
|
+
|
|
164
|
+
### After Testing
|
|
165
|
+
|
|
166
|
+
1. **⚠️ Reset database**: Run `Set-Location C:\Users\chris\Desktop\db-mcp\test-server; .\reset-database.ps1` to clean up migration artifacts (`_mcp_migrations` table and any `temp_migration_*` tables)
|
|
167
|
+
2. **Triage findings**: Create implementation plan if issues found
|
|
168
|
+
3. **Scope of fixes**: Handler code, server-instructions, test database, this prompt
|
|
169
|
+
4. **Validate**: Instruct the user to run the test suite (Vitest/Playwright), lint, and typecheck. Do NOT run them yourself.
|
|
170
|
+
5. **Commit**: Stage and commit — do NOT push
|
|
171
|
+
6. **Live re-test**: After server rebuild
|
|
172
|
+
7. **Final summary**: After testing/re-testing
|
|
173
|
+
|
|
174
|
+
---
|
|
175
|
+
|
|
176
|
+
## Troubleshooting
|
|
177
|
+
|
|
178
|
+
### Database is locked / file in use
|
|
179
|
+
|
|
180
|
+
1. Check for Node.js processes: `Get-CimInstance Win32_Process -Filter "Name = 'node.exe'"`
|
|
181
|
+
2. WAL/journal files are normal
|
|
182
|
+
|
|
183
|
+
### Migration state is dirty
|
|
184
|
+
|
|
185
|
+
If previous migration test left the database in a bad state:
|
|
186
|
+
|
|
187
|
+
1. Run `Set-Location C:\Users\chris\Desktop\db-mcp\test-server; .\reset-database.ps1` to regenerate the test database from `test-database.sql`
|
|
188
|
+
2. This drops and recreates all tables, including removing `_mcp_migrations`
|
|
189
|
+
|
|
190
|
+
### Reset script fails
|
|
191
|
+
|
|
192
|
+
1. Run with `-Verbose`: `.\reset-database.ps1 -Verbose`
|
|
193
|
+
2. If `sqlite3` is not in PATH, the script falls back to Node.js with `better-sqlite3`
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
# db-mcp Tool Group Testing: [stats-advanced]
|
|
2
|
+
|
|
3
|
+
> [!IMPORTANT]
|
|
4
|
+
> **Do not track progress in this file.** Track your test progress, coverage matrix, and findings in your internal task tracking system (artifact). However, you SHOULD edit this file to fix any factual errors, broken code, or incorrect assertions in the test prompts.
|
|
5
|
+
> If there is nothing to fix, don't update UNRELEASED.md.
|
|
6
|
+
> We're currently testing Native mode.
|
|
7
|
+
|
|
8
|
+
## WASM Mode
|
|
9
|
+
|
|
10
|
+
> When testing against a **WASM backend** (`sqlite-wasm` / sql.js), apply these adjustments:
|
|
11
|
+
>
|
|
12
|
+
> - **Skip window function tools** (items 1-6: `sqlite_window_row_number`, `sqlite_window_rank`, `sqlite_window_lag_lead`, `sqlite_window_running_total`, `sqlite_window_moving_avg`, `sqlite_window_ntile`) — `[NATIVE ONLY]`. These tools are not registered in WASM.
|
|
13
|
+
> - **Skip checklist items** 1-6.
|
|
14
|
+
> - **Skip Zod items** 7-12.
|
|
15
|
+
|
|
16
|
+
**Step 1:** Confirm you read the server help content sourced from `C:\Users\chris\Desktop\db-mcp\src\constants\server-instructions\gotchas.md` using `view_file` (not grep or search) — to understand documented behaviors, edge cases, and response structures for this tool group.
|
|
17
|
+
|
|
18
|
+
**Step 2:** Please conduct an exhaustive test of the **stats-advanced** tool group specified in the group-specific checklist below using live MCP server tool calls directly — not scripts/terminal.
|
|
19
|
+
|
|
20
|
+
**Step 3:** The agent should update C:\Users\chris\Desktop\db-mcp\UNRELEASED.md with any/all changes/fixes.
|
|
21
|
+
|
|
22
|
+
**Note** If temp tables are present from a previous test pass, it's because the database is locked. Ignore them.
|
|
23
|
+
|
|
24
|
+
## Reporting Format
|
|
25
|
+
|
|
26
|
+
- ❌ Fail: Tool errors or produces incorrect results (include error message)
|
|
27
|
+
- ⚠️ Issue: Unexpected behavior or improvement opportunity
|
|
28
|
+
- 📦 Payload: Unnecessarily large response that should be optimized.
|
|
29
|
+
|
|
30
|
+
## Test Database Schema
|
|
31
|
+
|
|
32
|
+
| Table | Rows | Columns | JSON Columns |
|
|
33
|
+
| ----------------- | ---- | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
|
34
|
+
| test_products | 16 | id, name, description, price, category, created_at | — |
|
|
35
|
+
| test_orders | 20 | id, product_id (FK), customer_name, quantity, total_price, order_date, status | — |
|
|
36
|
+
| test_jsonb_docs | 6 | id, doc, metadata, tags, created_at | **doc**, **metadata** (nested), **tags** (array) |
|
|
37
|
+
| test_articles | 8 | id, title, body, author, category, published_at | — |
|
|
38
|
+
| test_users | 9 | id, username, email, phone, bio, created_at | — |
|
|
39
|
+
| test_measurements | 200 | id, sensor_id, temperature, humidity, pressure, measured_at | — |
|
|
40
|
+
| test_embeddings | 20 | id, content, category, embedding | **embedding** (8-dim float array); category values: database, fitness, food, tech, travel |
|
|
41
|
+
| test_locations | 15 | id, name, city, latitude, longitude, type | — |
|
|
42
|
+
| test_categories | 17 | id, name, path, level | — |
|
|
43
|
+
| test_events | 100 | id, event_type, user_id (INT, 8 values), payload, event_date | **payload** (JSON) |
|
|
44
|
+
|
|
45
|
+
## Testing Requirements
|
|
46
|
+
|
|
47
|
+
> [!CAUTION]
|
|
48
|
+
> **Zero tolerance for raw MCP errors.** ANY response that is a raw MCP error (e.g., `-32602`, `isError: true`, no `success` field) is a **bug that must be reported and fixed**.
|
|
49
|
+
|
|
50
|
+
1. Use existing `test_*` tables for read operations
|
|
51
|
+
2. Report all failures, unexpected behaviors, improvement opportunities, or unnecessarily large payloads
|
|
52
|
+
3. **Error path testing**: For **every** tool, test at least **two** invalid inputs: (a) domain error and (b) Zod validation error (`{}`). Both must return `{success: false, error: "..."}` — NOT a raw MCP error frame.
|
|
53
|
+
4. **Output schema testing**: For **every** tool with an `outputSchema`, confirm valid happy-path calls return structured JSON — NOT a raw MCP `-32602` error.
|
|
54
|
+
|
|
55
|
+
## Structured Error Response Pattern
|
|
56
|
+
|
|
57
|
+
```json
|
|
58
|
+
{ "success": false, "error": "Human-readable error message" }
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
| Type | Source | What you see | Verdict |
|
|
62
|
+
| -------------------- | ------------------------------------------------------------------ | ----------------------------------------------------------- | ------------------ |
|
|
63
|
+
| **Handler error** ✅ | Handler catches error and returns `{success: false, error: "..."}` | Parseable JSON object with `success` and `error` fields | Correct |
|
|
64
|
+
| **MCP error** ❌ | Uncaught throw propagates to MCP framework | Raw text error string, `isError: true` — no `success` field | Bug — report as ❌ |
|
|
65
|
+
|
|
66
|
+
### Zod Validation Errors
|
|
67
|
+
|
|
68
|
+
**Zod refinement leak pattern:** `.partial()` does NOT strip `.min(N)` / `.max(N)` refinements. **Fix:** Remove refinements from schema, validate inside handler.
|
|
69
|
+
|
|
70
|
+
### Output Schema Validation Errors
|
|
71
|
+
|
|
72
|
+
If valid inputs return raw MCP `-32602` mentioning "output schema", report as ❌.
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## Group Focus: stats-advanced
|
|
77
|
+
|
|
78
|
+
> **Instructions**: Execute every numbered checklist item with the exact inputs shown. Compare responses against the expected results. Report any deviation.
|
|
79
|
+
|
|
80
|
+
### stats-advanced Group Tools (6)
|
|
81
|
+
|
|
82
|
+
1. sqlite_window_row_number `[NATIVE ONLY]`
|
|
83
|
+
2. sqlite_window_rank `[NATIVE ONLY]`
|
|
84
|
+
3. sqlite_window_lag_lead `[NATIVE ONLY]`
|
|
85
|
+
4. sqlite_window_running_total `[NATIVE ONLY]`
|
|
86
|
+
5. sqlite_window_moving_avg `[NATIVE ONLY]`
|
|
87
|
+
6. sqlite_window_ntile `[NATIVE ONLY]`
|
|
88
|
+
|
|
89
|
+
**Checklist:**
|
|
90
|
+
|
|
91
|
+
**Window functions `[NATIVE ONLY]`:**
|
|
92
|
+
|
|
93
|
+
1. `sqlite_window_row_number({table: "test_products", orderBy: "price DESC"})` → products ranked by price
|
|
94
|
+
2. `sqlite_window_rank({table: "test_products", orderBy: "price DESC"})` → rank with ties
|
|
95
|
+
3. `sqlite_window_running_total({table: "test_orders", column: "total_price", orderBy: "order_date"})` → cumulative totals
|
|
96
|
+
4. `sqlite_window_moving_avg({table: "test_measurements", column: "temperature", windowSize: 5, orderBy: "measured_at"})` → moving averages
|
|
97
|
+
5. `sqlite_window_lag_lead({table: "test_orders", column: "total_price", direction: "lag", orderBy: "order_date"})` → lag/lead values
|
|
98
|
+
6. `sqlite_window_ntile({table: "test_products", buckets: 4, orderBy: "price"})` → quartile assignments
|
|
99
|
+
|
|
100
|
+
**Zod validation sweep** — call each tool with `{}` (empty params). Must return handler error, NOT raw MCP error:
|
|
101
|
+
|
|
102
|
+
🔴 7. `sqlite_window_row_number({})` `[NATIVE ONLY]` → handler error
|
|
103
|
+
🔴 8. `sqlite_window_rank({})` `[NATIVE ONLY]` → handler error
|
|
104
|
+
🔴 9. `sqlite_window_lag_lead({})` `[NATIVE ONLY]` → handler error
|
|
105
|
+
🔴 10. `sqlite_window_running_total({})` `[NATIVE ONLY]` → handler error
|
|
106
|
+
🔴 11. `sqlite_window_moving_avg({})` `[NATIVE ONLY]` → handler error
|
|
107
|
+
🔴 12. `sqlite_window_ntile({})` `[NATIVE ONLY]` → handler error
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
## Post-Test Procedures
|
|
112
|
+
|
|
113
|
+
1. **Triage findings**: Create implementation plan if issues found
|
|
114
|
+
2. **Scope of fixes**: Handler code, server-instructions, test database, this prompt
|
|
115
|
+
3. **Validate**: Instruct the user to run the test suite (Vitest/Playwright), lint, and typecheck. Do NOT run them yourself.
|
|
116
|
+
4. **Commit**: Stage and commit — do NOT push
|
|
117
|
+
5. **Live re-test**: After server rebuild
|
|
118
|
+
6. **Final summary**: After testing/re-testing
|
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
# db-mcp Tool Group Testing: [stats-basic]
|
|
2
|
+
|
|
3
|
+
> [!IMPORTANT]
|
|
4
|
+
> **Do not track progress in this file.** Track your test progress, coverage matrix, and findings in your internal task tracking system (artifact). However, you SHOULD edit this file to fix any factual errors, broken code, or incorrect assertions in the test prompts.
|
|
5
|
+
> If there is nothing to fix, don't update UNRELEASED.md.
|
|
6
|
+
> We're currently testing Native mode.
|
|
7
|
+
|
|
8
|
+
## WASM Mode
|
|
9
|
+
|
|
10
|
+
> When testing against a **WASM backend** (`sqlite-wasm` / sql.js): All 16 tools in this basic stats suite are fully WASM-compatible. No items to skip or adjust.
|
|
11
|
+
|
|
12
|
+
**Step 1:** Confirm you read the server help content sourced from `C:\Users\chris\Desktop\db-mcp\src\constants\server-instructions\gotchas.md` using `view_file` (not grep or search) — to understand documented behaviors, edge cases, and response structures for this tool group.
|
|
13
|
+
|
|
14
|
+
**Step 2:** Please conduct an exhaustive test of the **stats-basic** tool group specified in the group-specific checklist below using live MCP server tool calls directly — not scripts/terminal.
|
|
15
|
+
|
|
16
|
+
**Step 3:** The agent should update C:\Users\chris\Desktop\db-mcp\UNRELEASED.md with any/all changes/fixes.
|
|
17
|
+
|
|
18
|
+
**Note** If temp tables are present from a previous test pass, it's because the database is locked. Ignore them.
|
|
19
|
+
|
|
20
|
+
## Reporting Format
|
|
21
|
+
|
|
22
|
+
- ❌ Fail: Tool errors or produces incorrect results (include error message)
|
|
23
|
+
- ⚠️ Issue: Unexpected behavior or improvement opportunity
|
|
24
|
+
- 📦 Payload: Unnecessarily large response that should be optimized — **blocking, equally important as ❌ bugs**. Report the response size in KB and suggest a concrete optimization.
|
|
25
|
+
|
|
26
|
+
## Test Database Schema
|
|
27
|
+
|
|
28
|
+
| Table | Rows | Columns | JSON Columns |
|
|
29
|
+
| ----------------- | ---- | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
|
30
|
+
| test_products | 16 | id, name, description, price, category, created_at | — |
|
|
31
|
+
| test_orders | 20 | id, product_id (FK), customer_name, quantity, total_price, order_date, status | — |
|
|
32
|
+
| test_jsonb_docs | 6 | id, doc, metadata, tags, created_at | **doc**, **metadata** (nested), **tags** (array) |
|
|
33
|
+
| test_articles | 8 | id, title, body, author, category, published_at | — |
|
|
34
|
+
| test_users | 9 | id, username, email, phone, bio, created_at | — |
|
|
35
|
+
| test_measurements | 200 | id, sensor_id, temperature, humidity, pressure, measured_at | — |
|
|
36
|
+
| test_embeddings | 20 | id, content, category, embedding | **embedding** (8-dim float array); category values: database, fitness, food, tech, travel |
|
|
37
|
+
| test_locations | 15 | id, name, city, latitude, longitude, type | — |
|
|
38
|
+
| test_categories | 17 | id, name, path, level | — |
|
|
39
|
+
| test_events | 100 | id, event_type, user_id (INT, 8 values), payload, event_date | **payload** (JSON) |
|
|
40
|
+
|
|
41
|
+
## Testing Requirements
|
|
42
|
+
|
|
43
|
+
> [!CAUTION]
|
|
44
|
+
> **Zero tolerance for raw MCP errors.** ANY response that is a raw MCP error (e.g., `-32602`, `isError: true`, no `success` field) is a **bug that must be reported and fixed**.
|
|
45
|
+
|
|
46
|
+
1. Use existing `test_*` tables for read operations
|
|
47
|
+
2. Create temporary tables with `temp_*` prefix for write operations
|
|
48
|
+
3. Report all failures, unexpected behaviors, improvement opportunities, or unnecessarily large payloads
|
|
49
|
+
4. Do not mention what already works well or issues well documented in help resources
|
|
50
|
+
5. **Error path testing**: For **every** tool, test at least **two** invalid inputs: (a) domain error and (b) Zod validation error (`{}`). Both must return `{success: false, error: "..."}` — NOT a raw MCP error frame.
|
|
51
|
+
6. **Output schema testing**: For **every** tool with an `outputSchema`, confirm valid happy-path calls return structured JSON — NOT a raw MCP `-32602` error.
|
|
52
|
+
7. **Deterministic checklist first**: Complete ALL items before freeform exploration.
|
|
53
|
+
8. **Code Over Docs**: Fix the handler code if standards (Structured Errors/Zod) are violated. Do NOT change docs/prompts to accommodate broken code.
|
|
54
|
+
|
|
55
|
+
## Structured Error Response Pattern
|
|
56
|
+
|
|
57
|
+
```json
|
|
58
|
+
{ "success": false, "error": "Human-readable error message" }
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
| Type | Source | What you see | Verdict |
|
|
62
|
+
| -------------------- | ------------------------------------------------------------------ | ----------------------------------------------------------- | ------------------ |
|
|
63
|
+
| **Handler error** ✅ | Handler catches error and returns `{success: false, error: "..."}` | Parseable JSON object with `success` and `error` fields | Correct |
|
|
64
|
+
| **MCP error** ❌ | Uncaught throw propagates to MCP framework | Raw text error string, `isError: true` — no `success` field | Bug — report as ❌ |
|
|
65
|
+
|
|
66
|
+
### Zod Validation Errors
|
|
67
|
+
|
|
68
|
+
**Zod refinement leak pattern:** `.partial()` does NOT strip `.min(N)` / `.max(N)` refinements. **Fix:** Remove refinements from schema, validate inside handler.
|
|
69
|
+
|
|
70
|
+
### Output Schema Validation Errors
|
|
71
|
+
|
|
72
|
+
If valid inputs return raw MCP `-32602` mentioning "output schema", report as ❌.
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## Group Focus: stats-basic
|
|
77
|
+
|
|
78
|
+
> **Instructions**: Execute every numbered checklist item with the exact inputs shown. Compare responses against the expected results. Report any deviation.
|
|
79
|
+
|
|
80
|
+
### stats-basic Group Tools (16)
|
|
81
|
+
|
|
82
|
+
1. sqlite_stats_basic
|
|
83
|
+
2. sqlite_stats_count
|
|
84
|
+
3. sqlite_stats_group_by
|
|
85
|
+
4. sqlite_stats_histogram
|
|
86
|
+
5. sqlite_stats_percentile
|
|
87
|
+
6. sqlite_stats_correlation
|
|
88
|
+
7. sqlite_stats_top_n
|
|
89
|
+
8. sqlite_stats_distinct
|
|
90
|
+
9. sqlite_stats_summary
|
|
91
|
+
10. sqlite_stats_frequency
|
|
92
|
+
11. sqlite_stats_outliers
|
|
93
|
+
12. sqlite_stats_regression
|
|
94
|
+
13. sqlite_stats_hypothesis
|
|
95
|
+
14. sqlite_stats_detect_anomalies
|
|
96
|
+
15. sqlite_stats_detect_bloat
|
|
97
|
+
16. sqlite_stats_detect_schema_risks
|
|
98
|
+
17. sqlite_execute_code
|
|
99
|
+
|
|
100
|
+
**Checklist:**
|
|
101
|
+
|
|
102
|
+
1. `sqlite_stats_basic({table: "test_measurements", column: "temperature"})` → verify `count: 200`, `min`, `max`, `avg` present
|
|
103
|
+
2. `sqlite_stats_count({table: "test_products"})` → `{count: 16}`
|
|
104
|
+
3. `sqlite_stats_count({table: "test_products", column: "category", distinct: true})` → distinct category count (electronics, accessories, office = 3)
|
|
105
|
+
4. `sqlite_stats_group_by({table: "test_measurements", groupByColumn: "sensor_id", valueColumn: "temperature", stat: "avg"})` → 5 groups (sensor_id 1-5) with average temperatures
|
|
106
|
+
5. `sqlite_stats_histogram({table: "test_measurements", column: "temperature", buckets: 5})` → 5 buckets
|
|
107
|
+
6. `sqlite_stats_percentile({table: "test_measurements", column: "temperature", percentiles: [25, 50, 75, 90]})` → 4 percentile values
|
|
108
|
+
7. `sqlite_stats_correlation({table: "test_measurements", column1: "temperature", column2: "humidity"})` → correlation value between -1 and 1
|
|
109
|
+
8. `sqlite_stats_top_n({table: "test_products", column: "price", n: 3, orderDirection: "desc"})` → top 3 most expensive products
|
|
110
|
+
9. `sqlite_stats_distinct({table: "test_locations", column: "city"})` → distinct city count (6)
|
|
111
|
+
10. `sqlite_stats_summary({table: "test_measurements", columns: ["temperature", "humidity", "pressure"]})` → summaries array with 3 entries
|
|
112
|
+
11. `sqlite_stats_frequency({table: "test_events", column: "event_type"})` → distribution
|
|
113
|
+
12. `sqlite_stats_outliers({table: "test_measurements", column: "temperature"})` → outlier detection result
|
|
114
|
+
13. `sqlite_stats_regression({table: "test_measurements", xColumn: "temperature", yColumn: "humidity", degree: 1})` → regression coefficients
|
|
115
|
+
14. `sqlite_stats_hypothesis({table: "test_measurements", column: "temperature", testType: "ttest_one", expectedMean: 25})` → verify `statistic` and `pValue` present
|
|
116
|
+
|
|
117
|
+
**Code mode testing:**
|
|
118
|
+
|
|
119
|
+
15. `sqlite_execute_code({code: "const result = await sqlite.stats.statsBasic({table: 'test_measurements', column: 'temperature'}); return result;"})` → verify `count: 200`, `min`, `max`, `avg` present
|
|
120
|
+
16. `sqlite_execute_code({code: "const result = await sqlite.stats.statsPercentile({table: 'test_measurements', column: 'temperature', percentiles: [50]}); return result;"})` → median value
|
|
121
|
+
|
|
122
|
+
**Error path testing:**
|
|
123
|
+
|
|
124
|
+
🔴 17. `sqlite_stats_basic({table: "nonexistent_table_xyz", column: "x"})` → structured error
|
|
125
|
+
🔴 18. `sqlite_stats_correlation({table: "test_products", column1: "name", column2: "description"})` → error about non-numeric columns
|
|
126
|
+
|
|
127
|
+
**Zod validation sweep** — call each tool with `{}` (empty params). Must return handler error, NOT raw MCP error:
|
|
128
|
+
|
|
129
|
+
🔴 19. `sqlite_stats_basic({})` → handler error
|
|
130
|
+
🔴 20. `sqlite_stats_count({})` → handler error
|
|
131
|
+
🔴 21. `sqlite_stats_group_by({})` → handler error
|
|
132
|
+
🔴 22. `sqlite_stats_histogram({})` → handler error
|
|
133
|
+
🔴 23. `sqlite_stats_percentile({})` → handler error
|
|
134
|
+
🔴 24. `sqlite_stats_correlation({})` → handler error
|
|
135
|
+
🔴 25. `sqlite_stats_top_n({})` → handler error
|
|
136
|
+
🔴 26. `sqlite_stats_distinct({})` → handler error
|
|
137
|
+
🔴 27. `sqlite_stats_summary({})` → handler error
|
|
138
|
+
🔴 28. `sqlite_stats_frequency({})` → handler error
|
|
139
|
+
🔴 29. `sqlite_stats_outliers({})` → handler error
|
|
140
|
+
🔴 30. `sqlite_stats_regression({})` → handler error
|
|
141
|
+
🔴 31. `sqlite_stats_hypothesis({})` → handler error
|
|
142
|
+
🔴 32. `sqlite_stats_detect_anomalies({})` → handler error
|
|
143
|
+
✅ 33. `sqlite_stats_detect_bloat({})` → success (no required params)
|
|
144
|
+
✅ 34. `sqlite_stats_detect_schema_risks({})` → success (no required params)
|
|
145
|
+
🔴 35. `sqlite_execute_code({})` → handler error
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## Post-Test Procedures
|
|
150
|
+
|
|
151
|
+
1. **Triage findings**: Create implementation plan if issues found
|
|
152
|
+
2. **Scope of fixes**: Handler code, server-instructions, test database, this prompt
|
|
153
|
+
3. **Validate**: Instruct the user to run the test suite (Vitest/Playwright), lint, and typecheck. Do NOT run them yourself.
|
|
154
|
+
4. **Commit**: Stage and commit — do NOT push
|
|
155
|
+
5. **Live re-test**: After server rebuild
|
|
156
|
+
6. **Final summary**: After testing/re-testing
|
|
@@ -0,0 +1,169 @@
|
|
|
1
|
+
# db-mcp Tool Group Testing: [text-advanced]
|
|
2
|
+
|
|
3
|
+
> [!IMPORTANT]
|
|
4
|
+
> **Do not track progress in this file.** Track your test progress, coverage matrix, and findings in your internal task tracking system (artifact). However, you SHOULD edit this file to fix any factual errors, broken code, or incorrect assertions in the test prompts.
|
|
5
|
+
> If there is nothing to fix, don't update UNRELEASED.md.
|
|
6
|
+
> We're currently testing Native mode.
|
|
7
|
+
|
|
8
|
+
## WASM Mode
|
|
9
|
+
|
|
10
|
+
> When testing against a **WASM backend** (`sqlite-wasm` / sql.js), apply these adjustments:
|
|
11
|
+
>
|
|
12
|
+
> - **Skip FTS5 tools** (items 5-9: `sqlite_fts_create`, `sqlite_fts_search`, `sqlite_fts_rebuild`, `sqlite_fts_match_info`, `sqlite_fts_headline`) — `[NATIVE ONLY]`. These tools are not registered in WASM and direct calls will fail with "unknown tool".
|
|
13
|
+
> - **Skip FTS5 checklist items** 6-11 — all require FTS5 tools.
|
|
14
|
+
> - **Skip error items** for FTS5 (item 14).
|
|
15
|
+
> - **Skip Zod items** for FTS5 (items 20-24).
|
|
16
|
+
|
|
17
|
+
**Step 1:** Confirm you read the server help content sourced from `C:\Users\chris\Desktop\db-mcp\src\constants\server-instructions\gotchas.md` using `view_file` (not grep or search) — to understand documented behaviors, edge cases, and response structures for this tool group.
|
|
18
|
+
|
|
19
|
+
**Step 2:** Please conduct an exhaustive test of the **text-advanced** tool group specified in the group-specific checklist below using live MCP server tool calls directly — not scripts/terminal.
|
|
20
|
+
|
|
21
|
+
**Step 3:** The agent should update C:\Users\chris\Desktop\db-mcp\UNRELEASED.md with any/all changes/fixes.
|
|
22
|
+
|
|
23
|
+
**Note** If temp tables are present from a previous test pass, it's because the database is locked. Ignore them.
|
|
24
|
+
|
|
25
|
+
## Reporting Format
|
|
26
|
+
|
|
27
|
+
- ❌ Fail: Tool errors or produces incorrect results (include error message)
|
|
28
|
+
- ⚠️ Issue: Unexpected behavior or improvement opportunity
|
|
29
|
+
- 📦 Payload: Unnecessarily large response that should be optimized.
|
|
30
|
+
|
|
31
|
+
## Test Database Schema
|
|
32
|
+
|
|
33
|
+
The test database (test-server/test.db) contains these tables with JSON-relevant columns:
|
|
34
|
+
|
|
35
|
+
| Table | Rows | Columns | JSON Columns |
|
|
36
|
+
| ----------------- | ---- | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
|
37
|
+
| test_products | 16 | id, name, description, price, category, created_at | — |
|
|
38
|
+
| test_orders | 20 | id, product_id (FK), customer_name, quantity, total_price, order_date, status | — |
|
|
39
|
+
| test_jsonb_docs | 6 | id, doc, metadata, tags, created_at | **doc**, **metadata** (nested), **tags** (array) |
|
|
40
|
+
| test_articles | 8 | id, title, body, author, category, published_at | — |
|
|
41
|
+
| test_users | 9 | id, username, email, phone, bio, created_at | — |
|
|
42
|
+
| test_measurements | 200 | id, sensor_id, temperature, humidity, pressure, measured_at | — |
|
|
43
|
+
| test_embeddings | 20 | id, content, category, embedding | **embedding** (8-dim float array); category values: database, fitness, food, tech, travel |
|
|
44
|
+
| test_locations | 15 | id, name, city, latitude, longitude, type | — |
|
|
45
|
+
| test_categories | 17 | id, name, path, level | — |
|
|
46
|
+
| test_events | 100 | id, event_type, user_id (INT, 8 values), payload, event_date | **payload** (JSON) |
|
|
47
|
+
|
|
48
|
+
## Testing Requirements
|
|
49
|
+
|
|
50
|
+
> [!CAUTION]
|
|
51
|
+
> **Zero tolerance for raw MCP errors.** ANY response that is a raw MCP error (e.g., `-32602`, `isError: true`, no `success` field) is a **bug that must be reported and fixed** — never an acceptable design choice, SDK limitation, or expected behavior. If you see one, report it as ❌ immediately. Do not rationalize it as "the SDK rejecting at the boundary" or "by design for range-constrained params." The handler MUST catch it.
|
|
52
|
+
|
|
53
|
+
1. Use existing `test_*` tables for read operations.
|
|
54
|
+
2. Create temporary tables with `temp_*` prefix for write operations.
|
|
55
|
+
3. Test each tool with realistic inputs based on the schema above.
|
|
56
|
+
4. Report all failures, unexpected behaviors, improvement opportunities, or unnecessarily large payloads.
|
|
57
|
+
5. **Error path testing**: For **every** tool, test at least **two** invalid inputs: (a) a domain error and (b) a **Zod validation error** (call the tool with `{}` empty params). Both must return a **structured handler error** (`{success: false, error: "..."}`) — NOT a raw MCP error frame.
|
|
58
|
+
6. **Output schema testing**: For **every** tool that has an `outputSchema`, confirm that at least one valid happy-path call returns a structured JSON response — NOT a raw MCP `-32602` "output schema" error. Output schema mismatches produce the same `-32602` code as input errors but are only caught with valid inputs.
|
|
59
|
+
7. **Code Over Docs**: Fix the handler code if standards (Structured Errors/Zod) are violated. Do NOT change docs/prompts to accommodate broken code.
|
|
60
|
+
|
|
61
|
+
## Structured Error Response Pattern
|
|
62
|
+
|
|
63
|
+
All tools should return errors as structured objects instead of throwing. The expected pattern:
|
|
64
|
+
|
|
65
|
+
```json
|
|
66
|
+
{ "success": false, "error": "Human-readable error message" }
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Handler Error vs MCP Error — How to Distinguish
|
|
70
|
+
|
|
71
|
+
| Type | Source | What you see | Verdict |
|
|
72
|
+
| -------------------- | ------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | ------------------ |
|
|
73
|
+
| **Handler error** ✅ | Handler catches error and returns `{success: false, error: "..."}` | Parseable JSON object with `success` and `error` fields | Correct |
|
|
74
|
+
| **MCP error** ❌ | Uncaught throw propagates to MCP framework | Raw text error string, often prefixed with `Error:`, wrapped in an `isError: true` content block — no `success` field | Bug — report as ❌ |
|
|
75
|
+
|
|
76
|
+
### Zod Validation Errors
|
|
77
|
+
|
|
78
|
+
Calling a tool with wrong parameter types or missing required fields triggers a Zod validation error. If the handler has no outer `try/catch`, this surfaces as a raw MCP error (often `-32602`). Test every tool with `{}` (empty params) if it has required parameters — the response must be a handler error, not an MCP error.
|
|
79
|
+
|
|
80
|
+
**Fix:** Remove ALL `.min(N)` / `.max(N)` refinements from the schema and validate inside the handler instead.
|
|
81
|
+
|
|
82
|
+
### Output Schema Validation Errors
|
|
83
|
+
|
|
84
|
+
The MCP SDK enforces `additionalProperties: false` on **output** schemas. If a handler returns fields not declared in its output schema, the SDK rejects the response with a raw `-32602` error.
|
|
85
|
+
|
|
86
|
+
**How to detect:** If a tool call with **correct, valid inputs** returns a raw MCP `-32602` mentioning "does not match the tool's output schema" or "additional properties", report as ❌ with both the tool name and the missing field(s).
|
|
87
|
+
|
|
88
|
+
### Error Consistency Audit
|
|
89
|
+
|
|
90
|
+
1. **Throw-vs-return**: If a tool throws a raw error instead of returning `{success: false}`, report as ❌.
|
|
91
|
+
2. **Error field name**: All `{success: false}` responses should use `error` as the field name.
|
|
92
|
+
3. **Zod validation leaks**: If calling a tool with missing required field produces a raw MCP `-32602` error instead of a structured response, report as ❌.
|
|
93
|
+
4. **Output schema leaks**: If calling a tool with valid inputs produces a raw MCP `-32602` mentioning "output schema", report as ❌.
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Group Focus: text-advanced
|
|
98
|
+
|
|
99
|
+
> **Instructions**: Execute every numbered checklist item with the exact inputs shown. Compare responses against the expected results. Report any deviation.
|
|
100
|
+
|
|
101
|
+
> **FTS Testing Notes:** After `sqlite_fts_create`, always call `sqlite_fts_rebuild` before searching. `test_articles` searchable terms: `SQLite`, `database`, `JSON`, `FTS`, `vector`, `API`, `search`, `MCP`.
|
|
102
|
+
|
|
103
|
+
### text-advanced Group Tools (9)
|
|
104
|
+
|
|
105
|
+
1. sqlite_fuzzy_match
|
|
106
|
+
2. sqlite_phonetic_match
|
|
107
|
+
3. sqlite_advanced_search
|
|
108
|
+
4. sqlite_text_sentiment
|
|
109
|
+
5. sqlite_fts_create `[NATIVE ONLY]`
|
|
110
|
+
6. sqlite_fts_search `[NATIVE ONLY]`
|
|
111
|
+
7. sqlite_fts_rebuild `[NATIVE ONLY]`
|
|
112
|
+
8. sqlite_fts_match_info `[NATIVE ONLY]`
|
|
113
|
+
9. sqlite_fts_headline `[NATIVE ONLY]`
|
|
114
|
+
|
|
115
|
+
**Checklist:**
|
|
116
|
+
|
|
117
|
+
1. `sqlite_fuzzy_match({table: "test_products", column: "name", search: "Laptp", maxDistance: 3})` → results include `Laptop Pro 15`
|
|
118
|
+
2. `sqlite_fuzzy_match({table: "test_products", column: "name", search: "Laptp", tokenize: false})` → verify behavior without tokenization
|
|
119
|
+
3. `sqlite_phonetic_match({table: "test_products", column: "name", search: "Labtop"})` → should find `Laptop Pro 15` via Soundex (both produce L131)
|
|
120
|
+
4. `sqlite_phonetic_match({table: "test_products", column: "name", search: "Labtop", algorithm: "metaphone"})` → test metaphone algorithm
|
|
121
|
+
5. `sqlite_advanced_search({table: "test_products", column: "name", searchTerm: "keyboard", techniques: ["exact", "fuzzy", "phonetic"]})` → should find `Mechanical Keyboard`
|
|
122
|
+
6. `sqlite_advanced_search({table: "test_products", column: "name", searchTerm: "Labtop", techniques: ["phonetic"]})` → test with single technique
|
|
123
|
+
|
|
124
|
+
**FTS5 tools `[NATIVE ONLY]`:**
|
|
125
|
+
|
|
126
|
+
7. `sqlite_fts_create({sourceTable: "test_users", columns: ["username", "bio"], ftsTable: "temp_users_fts"})` → FTS5 virtual table created
|
|
127
|
+
8. `sqlite_fts_rebuild({table: "temp_users_fts"})` → rebuild index before searching
|
|
128
|
+
9. `sqlite_fts_search({table: "temp_users_fts", query: "test*"})` → verify results from test_users data (prefix query needed since no standalone "test" token exists)
|
|
129
|
+
10. Cleanup: `sqlite_drop_table({table: "temp_users_fts"})` (drop the temp FTS table using sqlite_write_query or core drop_table)
|
|
130
|
+
11. `sqlite_fts_search({table: "test_articles_fts", query: "SQLite"})` → at least 1 result (article 1: "Introduction to SQLite")
|
|
131
|
+
12. `sqlite_fts_search({table: "test_articles_fts", query: "MCP protocol"})` → matches article 3: "The Model Context Protocol Explained"
|
|
132
|
+
13. `sqlite_fts_search({table: "test_articles_fts", query: "nonexistent_term_xyz"})` → 0 results
|
|
133
|
+
14. `sqlite_fts_match_info({table: "test_articles_fts", query: "database"})` → match info with scoring data
|
|
134
|
+
15. `sqlite_fts_rebuild({table: "test_articles_fts"})` → success
|
|
135
|
+
|
|
136
|
+
**Error path testing:**
|
|
137
|
+
|
|
138
|
+
🔴 16. `sqlite_fuzzy_match({table: "test_users", column: "nonexistent_col", search: "test"})` → structured error with code `COLUMN_NOT_FOUND`
|
|
139
|
+
🔴 17. `sqlite_fts_search({table: "nonexistent_fts_xyz", query: "test"})` `[NATIVE ONLY]` → structured error
|
|
140
|
+
|
|
141
|
+
**Zod validation sweep** — call each tool with `{}` (empty params). Must return handler error, NOT raw MCP error:
|
|
142
|
+
|
|
143
|
+
🔴 18. `sqlite_fuzzy_match({})` → handler error
|
|
144
|
+
🔴 19. `sqlite_phonetic_match({})` → handler error
|
|
145
|
+
🔴 20. `sqlite_advanced_search({})` → handler error
|
|
146
|
+
🔴 21. `sqlite_text_sentiment({})` → handler error
|
|
147
|
+
🔴 22. `sqlite_fts_create({})` `[NATIVE ONLY]` → handler error
|
|
148
|
+
🔴 23. `sqlite_fts_search({})` `[NATIVE ONLY]` → handler error
|
|
149
|
+
🔴 24. `sqlite_fts_rebuild({})` `[NATIVE ONLY]` → handler error
|
|
150
|
+
🔴 25. `sqlite_fts_match_info({})` `[NATIVE ONLY]` → handler error
|
|
151
|
+
🔴 26. `sqlite_fts_headline({})` `[NATIVE ONLY]` → handler error
|
|
152
|
+
|
|
153
|
+
---
|
|
154
|
+
|
|
155
|
+
## Post-Test Procedures
|
|
156
|
+
|
|
157
|
+
### Reporting Rules
|
|
158
|
+
|
|
159
|
+
- Use ✅ only in inline notes during testing; omit from Final Summary
|
|
160
|
+
- Do not mention what already works well or issues already documented in help resources and runtime hints
|
|
161
|
+
|
|
162
|
+
### After Testing
|
|
163
|
+
|
|
164
|
+
1. **Triage findings**: If issues were found, create an implementation plan. If the plan requires no user decisions, proceed directly to implementation
|
|
165
|
+
2. **Scope of fixes** includes corrections to handler code, `src/constants/server-instructions/*.md`, test database, or this prompt
|
|
166
|
+
3. **Validate**: Instruct the user to run the test suite (Vitest/Playwright), lint, and typecheck. Do NOT run them yourself.
|
|
167
|
+
4. **Commit**: Stage and commit — do NOT push
|
|
168
|
+
5. **Live re-test**: Test fixes with direct MCP tool calls after server rebuild
|
|
169
|
+
6. **Final summary**: Provide summary after testing/re-testing confirms fixes
|