sqlew 5.0.7 → 5.0.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,290 +0,0 @@
1
- # Database Migration Guide
2
-
3
- This guide explains how to generate complete SQL dumps from your mcp-sqlew database for backup and restore operations.
4
-
5
- ## ⚠️ BREAKING CHANGE (v4.0.2)
6
-
7
- **SQL dump no longer supports cross-database format conversion.**
8
-
9
- Starting from v4.0.2, `db:dump` generates SQL for the **same database type only**:
10
- - SQLite → SQLite ✅
11
- - MySQL → MySQL ✅
12
- - PostgreSQL → PostgreSQL ✅
13
- - SQLite → MySQL ❌ (use JSON instead)
14
- - SQLite → PostgreSQL ❌ (use JSON instead)
15
-
16
- **For cross-database migrations**, use JSON export/import:
17
- ```bash
18
- sqlew db:export backup.json # Export to JSON
19
- sqlew db:import backup.json # Import to target database
20
- ```
21
-
22
- See [DATA_EXPORT_IMPORT.md](DATA_EXPORT_IMPORT.md) for complete cross-database migration guide.
23
-
24
- ---
25
-
26
- ## Overview
27
-
28
- The `db:dump` CLI tool generates complete SQL dumps (CREATE TABLE + INSERT statements) for **same-database-type backup and restore operations**. The generated SQL can be imported directly into an empty database of the same type.
29
-
30
- ## Usage
31
-
32
- **No installation required!** Use directly via npx:
33
-
34
- ```bash
35
- sqlew db:dump <format> [output-file] [key=value ...]
36
- ```
37
-
38
- **Parameters:**
39
-
40
- | Parameter | Description | Default |
41
- |-----------|-------------|---------|
42
- | `<format>` | Target SQL format: sqlite, mysql, postgresql | **Required** |
43
- | `[output-file]` | Output file path | stdout |
44
- | `from=<source>` | Source database type | sqlite |
45
- | `tables=<list>` | Comma-separated table names | all tables |
46
- | `chunk-size=<n>` | Rows per INSERT statement | 100 |
47
- | `on-conflict=<mode>` | error, ignore, replace | error |
48
- | `exclude-schema=true` | Data-only dump (no CREATE TABLE) | false |
49
- | `db-path=<path>` | SQLite database path | .sqlew/sqlew.db |
50
-
51
- **Note:** By default, the dump includes both schema (CREATE TABLE) and data (INSERT) for complete migration.
52
-
53
- ### Generate SQL Dumps (Same-Database Backup)
54
-
55
- **SQLite Backup:**
56
-
57
- ```bash
58
- # Backup SQLite database (default)
59
- sqlew db:dump sqlite backup-sqlite.sql
60
- ```
61
-
62
- **MySQL Backup:**
63
-
64
- ```bash
65
- # Configure connection via .sqlew/config.toml or environment variables
66
- export MYSQL_HOST=127.0.0.1
67
- export MYSQL_PORT=3306
68
- export MYSQL_USER=youruser
69
- export MYSQL_PASSWORD=yourpass
70
- export MYSQL_DATABASE=mcp_context
71
-
72
- # Backup MySQL database
73
- sqlew db:dump mysql backup-mysql.sql from=mysql
74
- ```
75
-
76
- **PostgreSQL Backup:**
77
-
78
- ```bash
79
- # Configure connection via .sqlew/config.toml or environment variables
80
- export PG_HOST=localhost
81
- export PG_PORT=5432
82
- export PG_USER=postgres
83
- export PG_PASSWORD=yourpass
84
- export PG_DATABASE=mcp_context
85
-
86
- # Backup PostgreSQL database
87
- sqlew db:dump postgresql backup-pg.sql from=postgresql
88
- ```
89
-
90
- > **Note**: For cross-database migrations (e.g., SQLite → MySQL), use JSON export/import instead.
91
- > See [DATA_EXPORT_IMPORT.md](DATA_EXPORT_IMPORT.md) for the complete guide.
92
-
93
- ### Selective Table Export
94
-
95
- Export only specific tables:
96
-
97
- ```bash
98
- sqlew db:dump mysql partial.sql tables=v4_decisions,v4_tasks,v4_files
99
- ```
100
-
101
- ### Custom Chunk Size
102
-
103
- For large tables, adjust INSERT batch size:
104
-
105
- ```bash
106
- sqlew db:dump mysql dump.sql chunk-size=500
107
- ```
108
-
109
- ### Data-Only Dumps
110
-
111
- For advanced use cases where you manage schema separately:
112
-
113
- ```bash
114
- sqlew db:dump mysql data-only.sql exclude-schema=true
115
- ```
116
-
117
- This generates INSERT statements without CREATE TABLE, useful for:
118
-
119
- - Importing into databases with existing schema
120
- - Backup/restore of data only
121
- - Data transfer between identical schemas
122
-
123
- ### Conflict Resolution
124
-
125
- Handle duplicate keys when importing into existing databases:
126
-
127
- ```bash
128
- # Ignore duplicates (safe for adding new data)
129
- sqlew db:dump mysql dump.sql on-conflict=ignore
130
-
131
- # Update existing rows (sync/overwrite mode)
132
- sqlew db:dump mysql dump.sql on-conflict=replace
133
-
134
- # Fail on duplicates (default, strict mode)
135
- sqlew db:dump mysql dump.sql on-conflict=error
136
- ```
137
-
138
- **Modes:**
139
-
140
- - `error` (default): Standard INSERT, fails if duplicate keys exist
141
- - `ignore`: Skip duplicate rows (INSERT IGNORE / ON CONFLICT DO NOTHING)
142
- - `replace`: Update existing rows with new values (ON DUPLICATE KEY UPDATE / ON CONFLICT DO UPDATE)
143
-
144
- **Use cases:**
145
-
146
- - `ignore`: Importing into a database that may already have some data
147
- - `replace`: Synchronizing data from one database to another
148
- - `error`: Fresh database migration where duplicates indicate a problem
149
-
150
- ---
151
-
152
- ## Supported Operations (v4.0.2+)
153
-
154
- The `db:dump` tool supports **same-database-type backup and restore** only:
155
-
156
- | Source | Target | Command | Supported |
157
- |------------|------------|----------------------------------------------|-----------|
158
- | SQLite | SQLite | `--format sqlite` | ✅ |
159
- | MySQL | MySQL | `--from mysql --format mysql` | ✅ |
160
- | PostgreSQL | PostgreSQL | `--from postgresql --format postgresql` | ✅ |
161
- | SQLite | MySQL | N/A - Use JSON export/import | ❌ |
162
- | SQLite | PostgreSQL | N/A - Use JSON export/import | ❌ |
163
- | MySQL | PostgreSQL | N/A - Use JSON export/import | ❌ |
164
-
165
- **For cross-database migrations**, see [DATA_EXPORT_IMPORT.md](DATA_EXPORT_IMPORT.md).
166
-
167
- ---
168
-
169
- ## Data Type Mappings
170
-
171
- The tool automatically handles database-specific data type conversions:
172
-
173
- ### Boolean Values
174
-
175
- | SQLite | MySQL | PostgreSQL |
176
- |--------|---------------|------------|
177
- | 0/1 | 0/1 (TINYINT) | FALSE/TRUE |
178
-
179
- ### Binary Data
180
-
181
- | SQLite | MySQL | PostgreSQL |
182
- |------------|---------------|----------------|
183
- | BLOB (hex) | X'hex' (BLOB) | '\xhex'::bytea |
184
-
185
- ### Identifiers
186
-
187
- | SQLite | MySQL | PostgreSQL |
188
- |---------|-----------|------------|
189
- | "table" | \`table\` | "table" |
190
-
191
- ### Text and Numeric Types
192
-
193
- All databases handle TEXT, VARCHAR, INTEGER, and REAL types consistently. The tool preserves:
194
-
195
- - Unix epoch timestamps (stored as INTEGER)
196
- - UTF-8 text encoding
197
- - NULL values
198
-
199
- ---
200
-
201
- ## Importing Generated SQL
202
-
203
- The generated SQL is complete and self-contained. Import it directly into an **empty database**:
204
-
205
- **SQLite:**
206
-
207
- ```bash
208
- sqlite3 your-database.db < dump-sqlite.sql
209
- ```
210
-
211
- **MySQL/MariaDB:**
212
-
213
- ```bash
214
- # Create empty database first
215
- mysql -e "CREATE DATABASE mydb CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"
216
- # Import dump
217
- mysql mydb < dump-mysql.sql
218
- ```
219
-
220
- **PostgreSQL:**
221
-
222
- ```bash
223
- # Create empty database first
224
- createdb mydb
225
- # Import dump
226
- psql -d mydb -f dump-pg.sql
227
- ```
228
-
229
- **For existing databases with data:**
230
- Use `--on-conflict ignore` or `--on-conflict replace` when generating the dump (see Conflict Resolution section above).
231
-
232
- ## Transaction Safety
233
-
234
- All generated SQL dumps are wrapped in database transactions:
235
-
236
- - **MySQL/MariaDB**: `START TRANSACTION;` ... `COMMIT;`
237
- - **PostgreSQL**: `BEGIN;` ... `COMMIT;`
238
- - **SQLite**: `BEGIN TRANSACTION;` ... `COMMIT;`
239
-
240
- If the import fails at any point, all changes are automatically rolled back, leaving the database in its original state.
241
- This prevents partial imports that would leave the database in an inconsistent state.
242
-
243
- **Benefits:**
244
-
245
- - Atomic imports: Either all data is imported successfully or nothing is changed
246
- - Safe to retry: Failed imports don't leave partial data that needs cleanup
247
- - Consistent state: Database is never left in an intermediate state
248
-
249
- ---
250
-
251
- ## Environment Variables
252
-
253
- ### MySQL Configuration
254
-
255
- ```bash
256
- MYSQL_HOST=127.0.0.1 # Default: 127.0.0.1
257
- MYSQL_PORT=3306 # Default: 3306
258
- MYSQL_USER=root # Default: root
259
- MYSQL_PASSWORD= # Default: empty
260
- MYSQL_DATABASE=mcp_context # Default: mcp_context
261
- ```
262
-
263
- ### PostgreSQL Configuration
264
-
265
- ```bash
266
- PG_HOST=localhost # Default: localhost
267
- PG_PORT=5432 # Default: 5432
268
- PG_USER=postgres # Default: postgres
269
- PG_PASSWORD= # Default: empty
270
- PG_DATABASE=mcp_context # Default: mcp_context
271
- ```
272
-
273
- ---
274
-
275
- ## Best Practices
276
-
277
- 1. **Test on a copy** - Always test migrations on a database copy first
278
- 2. **Create schema first** - Run Knex migrations before importing data
279
- 3. **Review SQL before import** - Inspect generated SQL file for correctness
280
- 4. **Verify row counts** - Compare source and target table row counts after import
281
- 5. **Backup original data** - Keep backups before performing migrations
282
-
283
- ---
284
-
285
- ## Support
286
-
287
- For issues or questions:
288
-
289
- - GitHub Issues: https://github.com/sqlew-io/sqlew/issues
290
- - Documentation: `/docs` directory in repository