sqlew 5.0.7 → 5.0.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +100 -242
- package/dist/cli/hooks/pr-adr.d.ts +22 -0
- package/dist/cli/hooks/pr-adr.d.ts.map +1 -0
- package/dist/cli/hooks/pr-adr.js +72 -0
- package/dist/cli/hooks/pr-adr.js.map +1 -0
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +8 -1
- package/dist/cli.js.map +1 -1
- package/dist/index.js +2 -2
- package/dist/index.js.map +1 -1
- package/docs/ADR_CONCEPTS.md +66 -58
- package/docs/CLI_USAGE.md +472 -0
- package/docs/CONFIGURATION.md +152 -922
- package/docs/CROSS_DATABASE.md +1 -88
- package/docs/DATABASE_AUTH.md +24 -47
- package/docs/HOOKS_GUIDE.md +40 -112
- package/docs/MIGRATION_TO_SAAS.md +1 -1
- package/package.json +1 -1
- package/docs/HOW_TO_UNINSTALL.md +0 -199
- package/docs/MIGRATION_CLEANUP_GUIDE.md +0 -212
- package/docs/SLASH_COMMANDS.md +0 -329
- package/docs/cli/DATABASE_MIGRATION.md +0 -290
- package/docs/cli/DATA_EXPORT_IMPORT.md +0 -701
- package/docs/cli/README.md +0 -276
|
@@ -1,290 +0,0 @@
|
|
|
1
|
-
# Database Migration Guide
|
|
2
|
-
|
|
3
|
-
This guide explains how to generate complete SQL dumps from your mcp-sqlew database for backup and restore operations.
|
|
4
|
-
|
|
5
|
-
## ⚠️ BREAKING CHANGE (v4.0.2)
|
|
6
|
-
|
|
7
|
-
**SQL dump no longer supports cross-database format conversion.**
|
|
8
|
-
|
|
9
|
-
Starting from v4.0.2, `db:dump` generates SQL for the **same database type only**:
|
|
10
|
-
- SQLite → SQLite ✅
|
|
11
|
-
- MySQL → MySQL ✅
|
|
12
|
-
- PostgreSQL → PostgreSQL ✅
|
|
13
|
-
- SQLite → MySQL ❌ (use JSON instead)
|
|
14
|
-
- SQLite → PostgreSQL ❌ (use JSON instead)
|
|
15
|
-
|
|
16
|
-
**For cross-database migrations**, use JSON export/import:
|
|
17
|
-
```bash
|
|
18
|
-
sqlew db:export backup.json # Export to JSON
|
|
19
|
-
sqlew db:import backup.json # Import to target database
|
|
20
|
-
```
|
|
21
|
-
|
|
22
|
-
See [DATA_EXPORT_IMPORT.md](DATA_EXPORT_IMPORT.md) for complete cross-database migration guide.
|
|
23
|
-
|
|
24
|
-
---
|
|
25
|
-
|
|
26
|
-
## Overview
|
|
27
|
-
|
|
28
|
-
The `db:dump` CLI tool generates complete SQL dumps (CREATE TABLE + INSERT statements) for **same-database-type backup and restore operations**. The generated SQL can be imported directly into an empty database of the same type.
|
|
29
|
-
|
|
30
|
-
## Usage
|
|
31
|
-
|
|
32
|
-
**No installation required!** Use directly via npx:
|
|
33
|
-
|
|
34
|
-
```bash
|
|
35
|
-
sqlew db:dump <format> [output-file] [key=value ...]
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
**Parameters:**
|
|
39
|
-
|
|
40
|
-
| Parameter | Description | Default |
|
|
41
|
-
|-----------|-------------|---------|
|
|
42
|
-
| `<format>` | Target SQL format: sqlite, mysql, postgresql | **Required** |
|
|
43
|
-
| `[output-file]` | Output file path | stdout |
|
|
44
|
-
| `from=<source>` | Source database type | sqlite |
|
|
45
|
-
| `tables=<list>` | Comma-separated table names | all tables |
|
|
46
|
-
| `chunk-size=<n>` | Rows per INSERT statement | 100 |
|
|
47
|
-
| `on-conflict=<mode>` | error, ignore, replace | error |
|
|
48
|
-
| `exclude-schema=true` | Data-only dump (no CREATE TABLE) | false |
|
|
49
|
-
| `db-path=<path>` | SQLite database path | .sqlew/sqlew.db |
|
|
50
|
-
|
|
51
|
-
**Note:** By default, the dump includes both schema (CREATE TABLE) and data (INSERT) for complete migration.
|
|
52
|
-
|
|
53
|
-
### Generate SQL Dumps (Same-Database Backup)
|
|
54
|
-
|
|
55
|
-
**SQLite Backup:**
|
|
56
|
-
|
|
57
|
-
```bash
|
|
58
|
-
# Backup SQLite database (default)
|
|
59
|
-
sqlew db:dump sqlite backup-sqlite.sql
|
|
60
|
-
```
|
|
61
|
-
|
|
62
|
-
**MySQL Backup:**
|
|
63
|
-
|
|
64
|
-
```bash
|
|
65
|
-
# Configure connection via .sqlew/config.toml or environment variables
|
|
66
|
-
export MYSQL_HOST=127.0.0.1
|
|
67
|
-
export MYSQL_PORT=3306
|
|
68
|
-
export MYSQL_USER=youruser
|
|
69
|
-
export MYSQL_PASSWORD=yourpass
|
|
70
|
-
export MYSQL_DATABASE=mcp_context
|
|
71
|
-
|
|
72
|
-
# Backup MySQL database
|
|
73
|
-
sqlew db:dump mysql backup-mysql.sql from=mysql
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
**PostgreSQL Backup:**
|
|
77
|
-
|
|
78
|
-
```bash
|
|
79
|
-
# Configure connection via .sqlew/config.toml or environment variables
|
|
80
|
-
export PG_HOST=localhost
|
|
81
|
-
export PG_PORT=5432
|
|
82
|
-
export PG_USER=postgres
|
|
83
|
-
export PG_PASSWORD=yourpass
|
|
84
|
-
export PG_DATABASE=mcp_context
|
|
85
|
-
|
|
86
|
-
# Backup PostgreSQL database
|
|
87
|
-
sqlew db:dump postgresql backup-pg.sql from=postgresql
|
|
88
|
-
```
|
|
89
|
-
|
|
90
|
-
> **Note**: For cross-database migrations (e.g., SQLite → MySQL), use JSON export/import instead.
|
|
91
|
-
> See [DATA_EXPORT_IMPORT.md](DATA_EXPORT_IMPORT.md) for the complete guide.
|
|
92
|
-
|
|
93
|
-
### Selective Table Export
|
|
94
|
-
|
|
95
|
-
Export only specific tables:
|
|
96
|
-
|
|
97
|
-
```bash
|
|
98
|
-
sqlew db:dump mysql partial.sql tables=v4_decisions,v4_tasks,v4_files
|
|
99
|
-
```
|
|
100
|
-
|
|
101
|
-
### Custom Chunk Size
|
|
102
|
-
|
|
103
|
-
For large tables, adjust INSERT batch size:
|
|
104
|
-
|
|
105
|
-
```bash
|
|
106
|
-
sqlew db:dump mysql dump.sql chunk-size=500
|
|
107
|
-
```
|
|
108
|
-
|
|
109
|
-
### Data-Only Dumps
|
|
110
|
-
|
|
111
|
-
For advanced use cases where you manage schema separately:
|
|
112
|
-
|
|
113
|
-
```bash
|
|
114
|
-
sqlew db:dump mysql data-only.sql exclude-schema=true
|
|
115
|
-
```
|
|
116
|
-
|
|
117
|
-
This generates INSERT statements without CREATE TABLE, useful for:
|
|
118
|
-
|
|
119
|
-
- Importing into databases with existing schema
|
|
120
|
-
- Backup/restore of data only
|
|
121
|
-
- Data transfer between identical schemas
|
|
122
|
-
|
|
123
|
-
### Conflict Resolution
|
|
124
|
-
|
|
125
|
-
Handle duplicate keys when importing into existing databases:
|
|
126
|
-
|
|
127
|
-
```bash
|
|
128
|
-
# Ignore duplicates (safe for adding new data)
|
|
129
|
-
sqlew db:dump mysql dump.sql on-conflict=ignore
|
|
130
|
-
|
|
131
|
-
# Update existing rows (sync/overwrite mode)
|
|
132
|
-
sqlew db:dump mysql dump.sql on-conflict=replace
|
|
133
|
-
|
|
134
|
-
# Fail on duplicates (default, strict mode)
|
|
135
|
-
sqlew db:dump mysql dump.sql on-conflict=error
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
**Modes:**
|
|
139
|
-
|
|
140
|
-
- `error` (default): Standard INSERT, fails if duplicate keys exist
|
|
141
|
-
- `ignore`: Skip duplicate rows (INSERT IGNORE / ON CONFLICT DO NOTHING)
|
|
142
|
-
- `replace`: Update existing rows with new values (ON DUPLICATE KEY UPDATE / ON CONFLICT DO UPDATE)
|
|
143
|
-
|
|
144
|
-
**Use cases:**
|
|
145
|
-
|
|
146
|
-
- `ignore`: Importing into a database that may already have some data
|
|
147
|
-
- `replace`: Synchronizing data from one database to another
|
|
148
|
-
- `error`: Fresh database migration where duplicates indicate a problem
|
|
149
|
-
|
|
150
|
-
---
|
|
151
|
-
|
|
152
|
-
## Supported Operations (v4.0.2+)
|
|
153
|
-
|
|
154
|
-
The `db:dump` tool supports **same-database-type backup and restore** only:
|
|
155
|
-
|
|
156
|
-
| Source | Target | Command | Supported |
|
|
157
|
-
|------------|------------|----------------------------------------------|-----------|
|
|
158
|
-
| SQLite | SQLite | `--format sqlite` | ✅ |
|
|
159
|
-
| MySQL | MySQL | `--from mysql --format mysql` | ✅ |
|
|
160
|
-
| PostgreSQL | PostgreSQL | `--from postgresql --format postgresql` | ✅ |
|
|
161
|
-
| SQLite | MySQL | N/A - Use JSON export/import | ❌ |
|
|
162
|
-
| SQLite | PostgreSQL | N/A - Use JSON export/import | ❌ |
|
|
163
|
-
| MySQL | PostgreSQL | N/A - Use JSON export/import | ❌ |
|
|
164
|
-
|
|
165
|
-
**For cross-database migrations**, see [DATA_EXPORT_IMPORT.md](DATA_EXPORT_IMPORT.md).
|
|
166
|
-
|
|
167
|
-
---
|
|
168
|
-
|
|
169
|
-
## Data Type Mappings
|
|
170
|
-
|
|
171
|
-
The tool automatically handles database-specific data type conversions:
|
|
172
|
-
|
|
173
|
-
### Boolean Values
|
|
174
|
-
|
|
175
|
-
| SQLite | MySQL | PostgreSQL |
|
|
176
|
-
|--------|---------------|------------|
|
|
177
|
-
| 0/1 | 0/1 (TINYINT) | FALSE/TRUE |
|
|
178
|
-
|
|
179
|
-
### Binary Data
|
|
180
|
-
|
|
181
|
-
| SQLite | MySQL | PostgreSQL |
|
|
182
|
-
|------------|---------------|----------------|
|
|
183
|
-
| BLOB (hex) | X'hex' (BLOB) | '\xhex'::bytea |
|
|
184
|
-
|
|
185
|
-
### Identifiers
|
|
186
|
-
|
|
187
|
-
| SQLite | MySQL | PostgreSQL |
|
|
188
|
-
|---------|-----------|------------|
|
|
189
|
-
| "table" | \`table\` | "table" |
|
|
190
|
-
|
|
191
|
-
### Text and Numeric Types
|
|
192
|
-
|
|
193
|
-
All databases handle TEXT, VARCHAR, INTEGER, and REAL types consistently. The tool preserves:
|
|
194
|
-
|
|
195
|
-
- Unix epoch timestamps (stored as INTEGER)
|
|
196
|
-
- UTF-8 text encoding
|
|
197
|
-
- NULL values
|
|
198
|
-
|
|
199
|
-
---
|
|
200
|
-
|
|
201
|
-
## Importing Generated SQL
|
|
202
|
-
|
|
203
|
-
The generated SQL is complete and self-contained. Import it directly into an **empty database**:
|
|
204
|
-
|
|
205
|
-
**SQLite:**
|
|
206
|
-
|
|
207
|
-
```bash
|
|
208
|
-
sqlite3 your-database.db < dump-sqlite.sql
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
**MySQL/MariaDB:**
|
|
212
|
-
|
|
213
|
-
```bash
|
|
214
|
-
# Create empty database first
|
|
215
|
-
mysql -e "CREATE DATABASE mydb CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"
|
|
216
|
-
# Import dump
|
|
217
|
-
mysql mydb < dump-mysql.sql
|
|
218
|
-
```
|
|
219
|
-
|
|
220
|
-
**PostgreSQL:**
|
|
221
|
-
|
|
222
|
-
```bash
|
|
223
|
-
# Create empty database first
|
|
224
|
-
createdb mydb
|
|
225
|
-
# Import dump
|
|
226
|
-
psql -d mydb -f dump-pg.sql
|
|
227
|
-
```
|
|
228
|
-
|
|
229
|
-
**For existing databases with data:**
|
|
230
|
-
Use `--on-conflict ignore` or `--on-conflict replace` when generating the dump (see Conflict Resolution section above).
|
|
231
|
-
|
|
232
|
-
## Transaction Safety
|
|
233
|
-
|
|
234
|
-
All generated SQL dumps are wrapped in database transactions:
|
|
235
|
-
|
|
236
|
-
- **MySQL/MariaDB**: `START TRANSACTION;` ... `COMMIT;`
|
|
237
|
-
- **PostgreSQL**: `BEGIN;` ... `COMMIT;`
|
|
238
|
-
- **SQLite**: `BEGIN TRANSACTION;` ... `COMMIT;`
|
|
239
|
-
|
|
240
|
-
If the import fails at any point, all changes are automatically rolled back, leaving the database in its original state.
|
|
241
|
-
This prevents partial imports that would leave the database in an inconsistent state.
|
|
242
|
-
|
|
243
|
-
**Benefits:**
|
|
244
|
-
|
|
245
|
-
- Atomic imports: Either all data is imported successfully or nothing is changed
|
|
246
|
-
- Safe to retry: Failed imports don't leave partial data that needs cleanup
|
|
247
|
-
- Consistent state: Database is never left in an intermediate state
|
|
248
|
-
|
|
249
|
-
---
|
|
250
|
-
|
|
251
|
-
## Environment Variables
|
|
252
|
-
|
|
253
|
-
### MySQL Configuration
|
|
254
|
-
|
|
255
|
-
```bash
|
|
256
|
-
MYSQL_HOST=127.0.0.1 # Default: 127.0.0.1
|
|
257
|
-
MYSQL_PORT=3306 # Default: 3306
|
|
258
|
-
MYSQL_USER=root # Default: root
|
|
259
|
-
MYSQL_PASSWORD= # Default: empty
|
|
260
|
-
MYSQL_DATABASE=mcp_context # Default: mcp_context
|
|
261
|
-
```
|
|
262
|
-
|
|
263
|
-
### PostgreSQL Configuration
|
|
264
|
-
|
|
265
|
-
```bash
|
|
266
|
-
PG_HOST=localhost # Default: localhost
|
|
267
|
-
PG_PORT=5432 # Default: 5432
|
|
268
|
-
PG_USER=postgres # Default: postgres
|
|
269
|
-
PG_PASSWORD= # Default: empty
|
|
270
|
-
PG_DATABASE=mcp_context # Default: mcp_context
|
|
271
|
-
```
|
|
272
|
-
|
|
273
|
-
---
|
|
274
|
-
|
|
275
|
-
## Best Practices
|
|
276
|
-
|
|
277
|
-
1. **Test on a copy** - Always test migrations on a database copy first
|
|
278
|
-
2. **Create schema first** - Run Knex migrations before importing data
|
|
279
|
-
3. **Review SQL before import** - Inspect generated SQL file for correctness
|
|
280
|
-
4. **Verify row counts** - Compare source and target table row counts after import
|
|
281
|
-
5. **Backup original data** - Keep backups before performing migrations
|
|
282
|
-
|
|
283
|
-
---
|
|
284
|
-
|
|
285
|
-
## Support
|
|
286
|
-
|
|
287
|
-
For issues or questions:
|
|
288
|
-
|
|
289
|
-
- GitHub Issues: https://github.com/sqlew-io/sqlew/issues
|
|
290
|
-
- Documentation: `/docs` directory in repository
|