resplite 1.2.2 → 1.2.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -176,11 +176,121 @@ const srv = await createRESPlite({
176
176
 
177
177
  ## Migration from Redis
178
178
 
179
- RESPLite is a good fit for migrating **non-replicated Redis** instances that have **grown large** (e.g. tens of GB) and where RESPLite's latency is acceptable. The SPEC_F flow (dirty-key tracker, bulk import, cutover) is designed for that scenario with minimal downtime.
179
+ RESPLite is a good fit for migrating **non-replicated Redis** instances that have **grown large** (e.g. tens of GB) and where RESPLite's latency is acceptable. The flow (dirty-key tracker, bulk import, cutover) is designed for that scenario with minimal downtime.
180
180
 
181
181
  Migration supports two modes:
182
182
 
183
- ### Simple one-shot import (legacy)
183
+ ### Programmatic migration API (JavaScript)
184
+
185
+ As an alternative to the CLI, the full migration flow is available as a JavaScript API via `resplite/migration`. Useful for embedding the migration inside your own scripts or automation pipelines.
186
+
187
+ ```javascript
188
+ import { createMigration } from 'resplite/migration';
189
+
190
+ const m = createMigration({
191
+ from: 'redis://127.0.0.1:6379', // source Redis URL (default)
192
+ to: './resplite.db', // destination SQLite DB path (required)
193
+ runId: 'my-migration-1', // unique run ID (required for bulk/status/applyDirty)
194
+
195
+ // optional — same defaults as the CLI:
196
+ scanCount: 1000,
197
+ batchKeys: 200,
198
+ batchBytes: 64 * 1024 * 1024, // 64 MB
199
+ maxRps: 0, // 0 = unlimited
200
+ pragmaTemplate: 'default',
201
+
202
+ // If your Redis deployment renamed CONFIG for security:
203
+ // configCommand: 'MYCONFIG',
204
+ });
205
+
206
+ // Step 0 — Preflight: inspect Redis before starting
207
+ const info = await m.preflight();
208
+ console.log('keys (estimate):', info.keyCountEstimate);
209
+ console.log('type distribution:', info.typeDistribution);
210
+ console.log('notify-keyspace-events:', info.notifyKeyspaceEvents);
211
+ console.log('CONFIG available:', info.configCommandAvailable); // false if renamed
212
+ console.log('recommended params:', info.recommended);
213
+
214
+ // Step 0b — Enable keyspace notifications (required for dirty-key tracking)
215
+ // Reads the current value and merges the new flags — existing flags are preserved.
216
+ const ks = await m.enableKeyspaceNotifications();
217
+ // → { ok: true, previous: '', applied: 'KEA' }
218
+ // If CONFIG is renamed and configCommand was not set, ok=false and error explains how to fix it.
219
+
220
+ // Step 0c — Start dirty tracking (in-process, same script)
221
+ await m.startDirtyTracker({
222
+ onProgress: (p) => {
223
+ // one callback per keyspace event tracked during bulk/cutover
224
+ console.log(`[dirty ${p.totalEvents}] event=${p.event} key=${p.key}`);
225
+ },
226
+ });
227
+
228
+ // Step 1 — Bulk import (checkpointed, resumable). Same script to start or continue.
229
+ // Use keyCountEstimate from preflight to show progress % (estimate; actual count may change).
230
+ const total = info.keyCountEstimate || 1;
231
+ await m.bulk({
232
+ resume: true,
233
+ onProgress: (r) => {
234
+ const pct = total ? ((r.scanned_keys / total) * 100).toFixed(1) : '—';
235
+ console.log(
236
+ `scanned=${r.scanned_keys} migrated=${r.migrated_keys} errors=${r.error_keys} progress=${pct}%`
237
+ );
238
+ },
239
+ });
240
+
241
+ // Check status at any point (synchronous, no Redis needed)
242
+ const { run, dirty } = m.status();
243
+ console.log('bulk status:', run.status, '— dirty counts:', dirty);
244
+
245
+ // Step 2 — Apply dirty keys that changed in Redis during bulk
246
+ await m.applyDirty();
247
+
248
+ // Step 2b — Stop tracker after cutover
249
+ await m.stopDirtyTracker();
250
+
251
+ // Step 3 — Verify a sample of keys match between Redis and the destination
252
+ const result = await m.verify({ samplePct: 0.5, maxSample: 10000 });
253
+ console.log(`verified ${result.sampled} keys — mismatches: ${result.mismatches.length}`);
254
+
255
+ // Disconnect Redis when done
256
+ await m.close();
257
+ ```
258
+
259
+ **Bult: Automatic resume (default)**
260
+ `resume` defaults to `true`. It doesn't matter whether it's the first run or a resume: the same script works for both starting and continuing. The first run starts from cursor 0; if the process is interrupted (Ctrl+C, crash, etc.), running the script again continues from the last checkpoint. You don't need to pass `resume: false` on the first run or change anything to resume.
261
+
262
+ **Graceful shutdown**
263
+ On SIGINT (Ctrl+C) or SIGTERM, the bulk importer checkpoints progress, sets the run status to `aborted`, closes the SQLite database cleanly (so WAL is checkpointed and the file is not left open), then exits. You can safely interrupt a long-running bulk and resume later.
264
+
265
+ The JS API can run the dirty-key tracker in-process via `m.startDirtyTracker()` / `m.stopDirtyTracker()`, so the full flow can run from a single script. You can still use `npx resplite-dirty-tracker start|stop` if you prefer a separate process.
266
+
267
+ #### Renamed CONFIG command
268
+
269
+ If your Redis instance has the `CONFIG` command renamed (a common hardening practice), pass the new name to `createMigration`:
270
+
271
+ ```javascript
272
+ const m = createMigration({
273
+ from: 'redis://10.0.0.10:6379',
274
+ to: './resplite.db',
275
+ runId: 'run_001',
276
+ configCommand: 'MYCONFIG', // the renamed command
277
+ });
278
+
279
+ // preflight will use MYCONFIG GET notify-keyspace-events
280
+ const info = await m.preflight();
281
+ // info.configCommandAvailable → false if the name is wrong
282
+
283
+ // enableKeyspaceNotifications will use MYCONFIG SET notify-keyspace-events KEA
284
+ const result = await m.enableKeyspaceNotifications({ value: 'KEA' });
285
+ ```
286
+
287
+ The same flag is available in the CLI:
288
+
289
+ ```bash
290
+ npx resplite-dirty-tracker start --run-id run_001 --to ./resplite.db \
291
+ --from redis://10.0.0.10:6379 --config-command MYCONFIG
292
+ ```
293
+ ### Simple one-shot import
184
294
 
185
295
  For small datasets or when downtime is acceptable:
186
296
 
@@ -211,14 +321,17 @@ Examples:
211
321
  # One-shot import from authenticated Redis
212
322
  npm run import-from-redis -- --db ./migrated.db --redis-url "redis://:mysecret@127.0.0.1:6379"
213
323
 
214
- # SPEC_F flow: use --from with the full URL (or set RESPLITE_IMPORT_FROM)
324
+ # flow: use --from with the full URL (or set RESPLITE_IMPORT_FROM)
215
325
  npx resplite-import preflight --from "redis://:mysecret@10.0.0.10:6379" --to ./resplite.db
216
326
  npx resplite-dirty-tracker start --run-id run_001 --from "redis://:mysecret@10.0.0.10:6379" --to ./resplite.db
217
327
  ```
218
328
 
219
329
  For one-shot import, authentication is only available when using `--redis-url`; the `--host` / `--port` options do not support a password.
220
330
 
221
- ### Minimal-downtime migration (SPEC_F)
331
+ **Search indices (FT.\*)**
332
+ The KV bulk migration imports only the Redis keyspace (strings, hashes, sets, lists, zsets). RediSearch index schemas and documents are migrated separately with the `migrate-search` step — see [Migrating RediSearch indices](#migrating-redisearch-indices) below.
333
+
334
+ ### Minimal-downtime migration
222
335
 
223
336
  For large datasets (~30 GB), use the Dirty Key Registry flow so the bulk of the migration runs online and only a short cutover is needed.
224
337
 
@@ -278,97 +391,6 @@ notify-keyspace-events KEA
278
391
 
279
392
  Then start RespLite with the migrated DB: `RESPLITE_DB=./resplite.db npm start`.
280
393
 
281
- ### Programmatic migration API (JavaScript)
282
-
283
- As an alternative to the CLI, the full migration flow is available as a JavaScript API via `resplite/migration`. Useful for embedding the migration inside your own scripts or automation pipelines.
284
-
285
- ```javascript
286
- import { createMigration } from 'resplite/migration';
287
-
288
- const m = createMigration({
289
- from: 'redis://127.0.0.1:6379', // source Redis URL (default)
290
- to: './resplite.db', // destination SQLite DB path (required)
291
- runId: 'my-migration-1', // unique run ID (required for bulk/status/applyDirty)
292
-
293
- // optional — same defaults as the CLI:
294
- scanCount: 1000,
295
- batchKeys: 200,
296
- batchBytes: 64 * 1024 * 1024, // 64 MB
297
- maxRps: 0, // 0 = unlimited
298
- pragmaTemplate: 'default',
299
-
300
- // If your Redis deployment renamed CONFIG for security:
301
- // configCommand: 'MYCONFIG',
302
- });
303
-
304
- // Step 0 — Preflight: inspect Redis before starting
305
- const info = await m.preflight();
306
- console.log('keys (estimate):', info.keyCountEstimate);
307
- console.log('type distribution:', info.typeDistribution);
308
- console.log('notify-keyspace-events:', info.notifyKeyspaceEvents);
309
- console.log('CONFIG available:', info.configCommandAvailable); // false if renamed
310
- console.log('recommended params:', info.recommended);
311
-
312
- // Step 0b — Enable keyspace notifications (required for dirty-key tracking)
313
- // Reads the current value and merges the new flags — existing flags are preserved.
314
- const ks = await m.enableKeyspaceNotifications();
315
- // → { ok: true, previous: '', applied: 'KEA' }
316
- // If CONFIG is renamed and configCommand was not set, ok=false and error explains how to fix it.
317
-
318
- // Step 1 — Bulk import (checkpointed, resumable). Same script to start or continue.
319
- await m.bulk({
320
- onProgress: (r) => console.log(
321
- `scanned=${r.scanned_keys} migrated=${r.migrated_keys} errors=${r.error_keys}`
322
- ),
323
- });
324
-
325
- // Check status at any point (synchronous, no Redis needed)
326
- const { run, dirty } = m.status();
327
- console.log('bulk status:', run.status, '— dirty counts:', dirty);
328
-
329
- // Step 2 — Apply dirty keys that changed in Redis during bulk
330
- await m.applyDirty();
331
-
332
- // Step 3 — Verify a sample of keys match between Redis and the destination
333
- const result = await m.verify({ samplePct: 0.5, maxSample: 10000 });
334
- console.log(`verified ${result.sampled} keys — mismatches: ${result.mismatches.length}`);
335
-
336
- // Disconnect Redis when done
337
- await m.close();
338
- ```
339
-
340
- **Resume automático (por defecto)**
341
- `resume` viene en `true` por defecto. Da igual si es la primera vez o una reanudación: el mismo script sirve para empezar y para continuar. La primera ejecución arranca desde cursor 0; si el proceso se corta (Ctrl+C, crash, etc.), al volver a ejecutar el script continúa desde el último checkpoint. No hace falta pasar `resume: false` la primera vez ni cambiar nada para reanudar.
342
-
343
- The dirty-key tracker (to capture writes during bulk) still runs as a separate process via `npx resplite-dirty-tracker`. The API above handles everything else in a single script.
344
-
345
- #### Renamed CONFIG command
346
-
347
- If your Redis instance has the `CONFIG` command renamed (a common hardening practice), pass the new name to `createMigration`:
348
-
349
- ```javascript
350
- const m = createMigration({
351
- from: 'redis://10.0.0.10:6379',
352
- to: './resplite.db',
353
- runId: 'run_001',
354
- configCommand: 'MYCONFIG', // the renamed command
355
- });
356
-
357
- // preflight will use MYCONFIG GET notify-keyspace-events
358
- const info = await m.preflight();
359
- // info.configCommandAvailable → false if the name is wrong
360
-
361
- // enableKeyspaceNotifications will use MYCONFIG SET notify-keyspace-events KEA
362
- const result = await m.enableKeyspaceNotifications({ value: 'KEA' });
363
- ```
364
-
365
- The same flag is available in the CLI:
366
-
367
- ```bash
368
- npx resplite-dirty-tracker start --run-id run_001 --to ./resplite.db \
369
- --from redis://10.0.0.10:6379 --config-command MYCONFIG
370
- ```
371
-
372
394
  #### Low-level re-exports
373
395
 
374
396
  If you need more control, the individual functions and registry helpers are also exported:
@@ -551,9 +573,66 @@ await c2.quit();
551
573
  await srv2.close();
552
574
  ```
553
575
 
554
- ## Compatibility matrix
576
+ ### Migrating RediSearch indices
577
+
578
+ If your Redis source uses **RediSearch** (Redis Stack or the `redis/search` module), run `migrate-search` after (or during) the KV bulk import. It reads index schemas with `FT.INFO`, creates them in RespLite, and imports documents by scanning the matching hash keys.
579
+
580
+ **CLI:**
581
+
582
+ ```bash
583
+ # Migrate all indices
584
+ npx resplite-import migrate-search \
585
+ --from redis://10.0.0.10:6379 \
586
+ --to ./resplite.db
587
+
588
+ # Migrate specific indices only
589
+ npx resplite-import migrate-search \
590
+ --from redis://10.0.0.10:6379 \
591
+ --to ./resplite.db \
592
+ --index products \
593
+ --index articles
594
+
595
+ # Options
596
+ # --scan-count N SCAN COUNT hint (default 500)
597
+ # --max-rps N throttle Redis reads
598
+ # --batch-docs N docs per SQLite transaction (default 200)
599
+ # --max-suggestions N cap for suggestion import (default 10000)
600
+ # --no-skip overwrite if the index already exists in RespLite
601
+ # --no-suggestions skip suggestion import
602
+ ```
603
+
604
+ **Programmatic API:**
605
+
606
+ ```javascript
607
+ const m = createMigration({ from, to, runId });
608
+
609
+ const result = await m.migrateSearch({
610
+ onlyIndices: ['products', 'articles'], // omit to migrate all
611
+ batchDocs: 200,
612
+ maxSuggestions: 10000,
613
+ skipExisting: true, // default
614
+ withSuggestions: true, // default
615
+ onProgress: (r) => console.log(r.name, r.docsImported, r.warnings),
616
+ });
617
+ // result.indices: [{ name, created, skipped, docsImported, docsSkipped, docErrors, sugsImported, warnings, error? }]
618
+ // result.aborted: true if interrupted by SIGINT/SIGTERM
619
+ ```
620
+
621
+ **What gets migrated:**
622
+
623
+ | RediSearch type | RespLite | Notes |
624
+ |---|---|---|
625
+ | TEXT | TEXT | Direct |
626
+ | TAG | TEXT | Values preserved; TAG filtering lost |
627
+ | NUMERIC | TEXT | Stored as string; numeric range queries not supported |
628
+ | GEO, VECTOR, … | skipped | Warning emitted per field |
629
+
630
+ - Only **HASH**-based indices are supported. JSON (RedisJSON) indices are skipped.
631
+ - A `payload` field is added automatically if none of the source fields maps to it.
632
+ - Suggestions are imported via `FT.SUGGET "" MAX n WITHSCORES` (no cursor; capped at `maxSuggestions`).
633
+ - Graceful shutdown: Ctrl+C finishes the current document, closes SQLite cleanly, and exits with a non-zero code.
555
634
 
556
- RESPLite implements **47 core Redis commands** (~19% of the ~246 commands in Redis 7). The uncovered 81% is mostly entire subsystems that are out of scope by design: pub/sub (~10 commands), Streams (~20), cluster/replication (~30), Lua scripting (~5), server admin (~40), and extended variants of data-structure commands. For typical single-node application workloads — strings, hashes, sets, lists, sorted sets, key TTLs — coverage is close to the commands developers reach for daily.
635
+ ## Compatibility matrix
557
636
 
558
637
  ### Supported (v1)
559
638
 
@@ -594,7 +673,7 @@ Unsupported commands return: `ERR command not supported yet`.
594
673
  | `npm run test:stress` | Stress tests |
595
674
  | `npm run benchmark` | Comparative benchmark Redis vs RESPLite |
596
675
  | `npm run import-from-redis` | One-shot import from Redis into a SQLite DB |
597
- | `npx resplite-import` (preflight, bulk, status, apply-dirty, verify) | Migration CLI (SPEC_F minimal-downtime flow) |
676
+ | `npx resplite-import` (preflight, bulk, status, apply-dirty, verify) | Migration CLI (minimal-downtime flow) |
598
677
  | `npx resplite-dirty-tracker <start\|stop>` | Dirty-key tracker for migration cutover |
599
678
 
600
679
  ## Specification
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "resplite",
3
- "version": "1.2.2",
3
+ "version": "1.2.6",
4
4
  "description": "A RESP2 server with practical Redis compatibility, backed by SQLite",
5
5
  "type": "module",
6
6
  "main": "src/index.js",
package/spec/SPEC_F.md CHANGED
@@ -13,6 +13,7 @@
13
13
  * Perfect change-data-capture guarantees equivalent to replication logs.
14
14
  * Distributed migration across multiple import workers with strict ordering semantics.
15
15
  * Full fidelity for unsupported Redis data types (streams, modules, Lua scripts, etc.).
16
+ * **Search indices (FT.\*):** Keyspace migration (`bulk` / `apply-dirty`) copies only the Redis KV data (strings, hashes, sets, lists, zsets). RediSearch index schemas and documents are migrated separately via the `migrate-search` step (§F.10).
16
17
 
17
18
  ---
18
19
 
@@ -258,6 +259,18 @@ Persist:
258
259
 
259
260
  If interrupted, `--resume` restarts from the stored cursor.
260
261
 
262
+ ## F.7.2.1 Graceful shutdown (SIGINT / SIGTERM)
263
+
264
+ When the bulk import process receives SIGINT or SIGTERM it must:
265
+
266
+ * Stop the import loop after the current key (no partial key).
267
+ * Write a final checkpoint (cursor and counters) so progress is persisted.
268
+ * Set run status to `aborted`.
269
+ * Close the SQLite database handle so WAL is checkpointed and the file is not left open.
270
+ * Remove signal handlers and exit (rethrow so the process exits non-zero).
271
+
272
+ This ensures the destination DB is always closed cleanly when the process is killed or interrupted; the next run with `--resume` continues from the last checkpoint.
273
+
261
274
  ## F.7.3 Throughput controls
262
275
 
263
276
  The importer must support:
@@ -497,6 +510,87 @@ If dirty tracker disconnects:
497
510
 
498
511
  ---
499
512
 
513
+ # F.10 Search Index Migration (FT.* / RediSearch)
514
+
515
+ ## F.10.1 Overview
516
+
517
+ When the source is a Redis instance with **RediSearch** (Redis Stack or the `redis/search` module), search indices can be migrated with the `migrate-search` step. This step is independent of the KV bulk import and can be run at any time (before or after `bulk`).
518
+
519
+ ## F.10.2 Algorithm
520
+
521
+ For each index in the source:
522
+
523
+ 1. **`FT._LIST`** → enumerate all index names.
524
+ 2. **`FT.INFO <name>`** → read `index_definition` (key type, prefix patterns) and `attributes` (field names and types).
525
+ 3. **Schema mapping** (see §F.10.3).
526
+ 4. **`FT.CREATE`** in RespLite with the mapped schema. Skip if already exists (controlled by `skipExisting`).
527
+ 5. **SCAN** keys matching each index prefix → **HGETALL** → `addDocument` in SQLite batches.
528
+ 6. **`FT.SUGGET "" MAX n WITHSCORES`** → import suggestions into RespLite.
529
+
530
+ ## F.10.3 Field type mapping
531
+
532
+ | RediSearch type | RespLite type | Notes |
533
+ |-----------------|---------------|-------|
534
+ | TEXT | TEXT | Direct mapping |
535
+ | TAG | TEXT | Values preserved as-is; TAG filtering semantics lost |
536
+ | NUMERIC | TEXT | Values stored as strings; numeric range queries not supported |
537
+ | GEO, VECTOR, … | — | Skipped with warning |
538
+
539
+ RespLite requires a `payload` TEXT field. If none of the source fields maps to `payload`, a `payload` field is added automatically and synthesised at import time by concatenating all other text values.
540
+
541
+ ## F.10.4 Constraints
542
+
543
+ * Only **HASH**-based indices are supported (`key_type = HASH`). JSON indices (RedisJSON) are skipped with an error.
544
+ * Index names must match `[A-Za-z][A-Za-z0-9:_-]{0,63}`. Indices with invalid names are skipped with an error.
545
+ * `FT.SUGGET` has no cursor; suggestions are imported up to `maxSuggestions` (default 10 000).
546
+ * Document score is read from the `__score` or `score` hash field if present; defaults to `1.0`.
547
+
548
+ ## F.10.5 Graceful shutdown
549
+
550
+ Same pattern as `bulk` (§F.7.2.1): SIGINT/SIGTERM finishes the current document, closes the SQLite DB cleanly, and exits with a non-zero code.
551
+
552
+ ## F.10.6 CLI
553
+
554
+ ```bash
555
+ # Migrate all RediSearch indices
556
+ resplite-import migrate-search \
557
+ --from redis://10.0.0.10:6379 \
558
+ --to ./resplite.db
559
+
560
+ # Migrate specific indices only
561
+ resplite-import migrate-search \
562
+ --from redis://10.0.0.10:6379 \
563
+ --to ./resplite.db \
564
+ --index products \
565
+ --index articles
566
+
567
+ # Options
568
+ # --scan-count N SCAN COUNT hint (default 500)
569
+ # --max-rps N throttle (default unlimited)
570
+ # --batch-docs N docs per SQLite transaction (default 200)
571
+ # --max-suggestions N cap for FT.SUGGET (default 10000)
572
+ # --no-skip overwrite if index already exists
573
+ # --no-suggestions skip suggestion import
574
+ ```
575
+
576
+ ## F.10.7 Programmatic API
577
+
578
+ ```javascript
579
+ const m = createMigration({ from, to, runId });
580
+
581
+ const result = await m.migrateSearch({
582
+ onlyIndices: ['products', 'articles'], // omit for all
583
+ batchDocs: 200,
584
+ maxSuggestions: 10000,
585
+ skipExisting: true,
586
+ withSuggestions: true,
587
+ onProgress: (r) => console.log(r.name, r.docsImported, r.warnings),
588
+ });
589
+ // result: { indices: [{ name, created, skipped, docsImported, docsSkipped, docErrors, sugsImported, warnings, error? }], aborted }
590
+ ```
591
+
592
+ ---
593
+
500
594
  # F.12 Operational Guidance (Large datasets)
501
595
 
502
596
  * Use a dedicated Redis replica for reads if possible to reduce load on primary.
@@ -11,9 +11,10 @@ import { runPreflight } from '../migration/preflight.js';
11
11
  import { runBulkImport } from '../migration/bulk.js';
12
12
  import { runApplyDirty } from '../migration/apply-dirty.js';
13
13
  import { runVerify } from '../migration/verify.js';
14
+ import { runMigrateSearch } from '../migration/migrate-search.js';
14
15
  import { getRun, getDirtyCounts } from '../migration/registry.js';
15
16
 
16
- const SUBCOMMANDS = ['preflight', 'bulk', 'status', 'apply-dirty', 'verify'];
17
+ const SUBCOMMANDS = ['preflight', 'bulk', 'status', 'apply-dirty', 'verify', 'migrate-search'];
17
18
 
18
19
  function parseArgs(argv = process.argv.slice(2)) {
19
20
  const args = { _: [] };
@@ -53,6 +54,17 @@ function parseArgs(argv = process.argv.slice(2)) {
53
54
  args.pragmaTemplate = argv[++i];
54
55
  } else if (arg === '--sample' && argv[i + 1]) {
55
56
  args.sample = argv[++i];
57
+ } else if (arg === '--index' && argv[i + 1]) {
58
+ if (!args.index) args.index = [];
59
+ args.index.push(argv[++i]);
60
+ } else if (arg === '--batch-docs' && argv[i + 1]) {
61
+ args.batchDocs = parseInt(argv[++i], 10);
62
+ } else if (arg === '--max-suggestions' && argv[i + 1]) {
63
+ args.maxSuggestions = parseInt(argv[++i], 10);
64
+ } else if (arg === '--no-skip') {
65
+ args.noSkip = true;
66
+ } else if (arg === '--no-suggestions') {
67
+ args.noSuggestions = true;
56
68
  } else if (arg.startsWith('--')) {
57
69
  args[arg.slice(2).replace(/-/g, '')] = argv[i + 1] ?? true;
58
70
  if (argv[i + 1] && !argv[i + 1].startsWith('--')) i++;
@@ -200,28 +212,73 @@ async function cmdVerify(args) {
200
212
  }
201
213
  }
202
214
 
215
+ async function cmdMigrateSearch(args) {
216
+ const redisUrl = getRedisUrl(args);
217
+ const dbPath = getDbPath(args);
218
+ const client = createClient({ url: redisUrl });
219
+ client.on('error', (e) => console.error('Redis:', e.message));
220
+ await client.connect();
221
+ try {
222
+ const onlyIndices = args.index
223
+ ? (Array.isArray(args.index) ? args.index : [args.index])
224
+ : null;
225
+
226
+ const result = await runMigrateSearch(client, dbPath, {
227
+ pragmaTemplate: args.pragmaTemplate || 'default',
228
+ onlyIndices,
229
+ scanCount: args.scanCount || 500,
230
+ maxRps: args.maxRps || 0,
231
+ batchDocs: args.batchDocs || 200,
232
+ maxSuggestions: args.maxSuggestions || 10000,
233
+ skipExisting: args.noSkip ? false : true,
234
+ withSuggestions: args.noSuggestions ? false : true,
235
+ onProgress: (r) => {
236
+ const status = r.error ? `ERROR: ${r.error}` : (r.skipped ? 'skipped (already exists)' : 'created');
237
+ console.log(` [${r.name}] ${status} — docs=${r.docsImported} skipped=${r.docsSkipped} errors=${r.docErrors} sugs=${r.sugsImported}`);
238
+ if (r.warnings?.length) r.warnings.forEach((w) => console.log(` WARN: ${w}`));
239
+ },
240
+ });
241
+
242
+ if (result.aborted) console.log('Migration aborted by signal.');
243
+ console.log(`Done. Indices processed: ${result.indices.length}`);
244
+ const errors = result.indices.filter((i) => i.error);
245
+ if (errors.length) {
246
+ console.error(` ${errors.length} index(es) failed:`);
247
+ errors.forEach((i) => console.error(` - ${i.name}: ${i.error}`));
248
+ }
249
+ } finally {
250
+ await client.quit();
251
+ }
252
+ }
253
+
203
254
  async function main() {
204
255
  const args = parseArgs();
205
256
  const sub = args._[0];
206
257
  if (!SUBCOMMANDS.includes(sub)) {
207
- console.error('Usage: resplite-import <preflight|bulk|status|apply-dirty|verify> [options]');
208
- console.error(' --from <redis-url> (default: redis://127.0.0.1:6379)');
209
- console.error(' --to <db-path> (required for bulk, status, apply-dirty, verify)');
210
- console.error(' --run-id <id> (required for bulk, status, apply-dirty)');
211
- console.error(' --scan-count N (bulk, default 1000)');
212
- console.error(' --max-rps N (bulk, apply-dirty)');
213
- console.error(' --batch-keys N (default 200)');
214
- console.error(' --batch-bytes N[MB|KB|GB] (default 64MB)');
215
- console.error(' --resume / --no-resume (bulk: default resume=on; use --no-resume to always start from 0)');
216
- console.error(' --sample 0.5% (verify, default 0.5%)');
258
+ console.error('Usage: resplite-import <preflight|bulk|status|apply-dirty|verify|migrate-search> [options]');
259
+ console.error(' --from <redis-url> (default: redis://127.0.0.1:6379)');
260
+ console.error(' --to <db-path> (required for bulk, status, apply-dirty, verify, migrate-search)');
261
+ console.error(' --run-id <id> (required for bulk, status, apply-dirty)');
262
+ console.error(' --scan-count N (bulk / migrate-search, default 1000 / 500)');
263
+ console.error(' --max-rps N (bulk, apply-dirty, migrate-search)');
264
+ console.error(' --batch-keys N (default 200)');
265
+ console.error(' --batch-bytes N[MB|KB|GB](default 64MB)');
266
+ console.error(' --resume / --no-resume (bulk: default resume=on)');
267
+ console.error(' --sample 0.5% (verify, default 0.5%)');
268
+ console.error(' --index <name> (migrate-search: repeat for multiple; omit for all indices)');
269
+ console.error(' --batch-docs N (migrate-search: docs per SQLite tx, default 200)');
270
+ console.error(' --max-suggestions N (migrate-search: cap for FT.SUGGET, default 10000)');
271
+ console.error(' --no-skip (migrate-search: overwrite if index exists)');
272
+ console.error(' --no-suggestions (migrate-search: skip suggestion import)');
217
273
  process.exit(1);
218
274
  }
219
275
  try {
220
- if (sub === 'preflight') await cmdPreflight(args);
221
- else if (sub === 'bulk') await cmdBulk(args);
222
- else if (sub === 'status') await cmdStatus(args);
223
- else if (sub === 'apply-dirty') await cmdApplyDirty(args);
224
- else if (sub === 'verify') await cmdVerify(args);
276
+ if (sub === 'preflight') await cmdPreflight(args);
277
+ else if (sub === 'bulk') await cmdBulk(args);
278
+ else if (sub === 'status') await cmdStatus(args);
279
+ else if (sub === 'apply-dirty')await cmdApplyDirty(args);
280
+ else if (sub === 'verify') await cmdVerify(args);
281
+ else if (sub === 'migrate-search') await cmdMigrateSearch(args);
225
282
  } catch (err) {
226
283
  console.error(err);
227
284
  process.exit(1);
@@ -236,4 +293,4 @@ if (isMain) {
236
293
  });
237
294
  }
238
295
 
239
- export { parseArgs, cmdPreflight, cmdBulk, cmdStatus, cmdApplyDirty, cmdVerify };
296
+ export { parseArgs, cmdPreflight, cmdBulk, cmdStatus, cmdApplyDirty, cmdVerify, cmdMigrateSearch };