@fazetitans/fscopy 1.1.3 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (26) hide show
  1. package/README.md +76 -33
  2. package/package.json +3 -3
  3. package/src/cli.ts +71 -621
  4. package/src/config/defaults.ts +4 -0
  5. package/src/config/parser.ts +4 -0
  6. package/src/config/validator.ts +52 -0
  7. package/src/firebase/index.ts +82 -0
  8. package/src/interactive.ts +59 -56
  9. package/src/orchestrator.ts +407 -0
  10. package/src/output/display.ts +221 -0
  11. package/src/state/index.ts +188 -1
  12. package/src/transfer/clear.ts +162 -104
  13. package/src/transfer/count.ts +83 -44
  14. package/src/transfer/transfer.ts +487 -156
  15. package/src/transform/loader.ts +31 -0
  16. package/src/types.ts +18 -0
  17. package/src/utils/credentials.ts +9 -4
  18. package/src/utils/doc-size.ts +41 -70
  19. package/src/utils/errors.ts +1 -1
  20. package/src/utils/index.ts +2 -1
  21. package/src/utils/integrity.ts +122 -0
  22. package/src/utils/logger.ts +59 -3
  23. package/src/utils/output.ts +265 -0
  24. package/src/utils/progress.ts +102 -0
  25. package/src/utils/rate-limiter.ts +4 -2
  26. package/src/webhook/index.ts +6 -6
package/README.md CHANGED
@@ -353,39 +353,42 @@ fscopy --init config.json
353
353
 
354
354
  ## CLI Reference
355
355
 
356
- | Option | Alias | Type | Default | Description |
357
- | -------------------------- | ----- | ------- | ------- | --------------------------------- |
358
- | `--init` | | string | | Generate config template |
359
- | `--config` | `-f` | string | | Path to config file |
360
- | `--source-project` | | string | | Source Firebase project |
361
- | `--dest-project` | | string | | Destination project |
362
- | `--collections` | `-c` | array | | Collections to transfer |
363
- | `--include-subcollections` | `-s` | boolean | `false` | Include subcollections |
364
- | `--where` | `-w` | array | | Filter documents |
365
- | `--exclude` | `-x` | array | | Exclude subcollections |
366
- | `--merge` | `-m` | boolean | `false` | Merge instead of overwrite |
367
- | `--parallel` | `-p` | number | `1` | Parallel transfers |
368
- | `--dry-run` | `-d` | boolean | `true` | Preview without writing |
369
- | `--batch-size` | `-b` | number | `500` | Documents per batch |
370
- | `--limit` | `-l` | number | `0` | Limit docs (0 = no limit) |
371
- | `--retries` | | number | `3` | Retries on error |
372
- | `--log` | | string | | Log file path |
373
- | `--quiet` | `-q` | boolean | `false` | No progress bar |
374
- | `--yes` | `-y` | boolean | `false` | Skip confirmation |
375
- | `--clear` | | boolean | `false` | Clear destination before transfer |
376
- | `--delete-missing` | | boolean | `false` | Delete dest docs not in source |
377
- | `--interactive` | `-i` | boolean | `false` | Interactive mode with prompts |
378
- | `--transform` | `-t` | string | | Path to JS/TS transform file |
379
- | `--rename-collection` | `-r` | array | | Rename collection (source:dest) |
380
- | `--id-prefix` | | string | | Add prefix to document IDs |
381
- | `--id-suffix` | | string | | Add suffix to document IDs |
382
- | `--webhook` | | string | | Webhook URL for notifications |
383
- | `--resume` | | boolean | `false` | Resume from saved state |
384
- | `--state-file` | | string | `.fscopy-state.json` | State file path |
385
- | `--verify` | | boolean | `false` | Verify counts after transfer |
386
- | `--rate-limit` | | number | `0` | Limit docs/second (0 = unlimited) |
387
- | `--skip-oversized` | | boolean | `false` | Skip documents > 1MB |
388
- | `--json` | | boolean | `false` | JSON output for CI/CD |
356
+ | Option | Alias | Type | Default | Description |
357
+ | -------------------------- | ----- | ------- | -------------------- | --------------------------------------- |
358
+ | `--init` | | string | | Generate config template |
359
+ | `--config` | `-f` | string | | Path to config file |
360
+ | `--source-project` | | string | | Source Firebase project |
361
+ | `--dest-project` | | string | | Destination project |
362
+ | `--collections` | `-c` | array | | Collections to transfer |
363
+ | `--include-subcollections` | `-s` | boolean | `false` | Include subcollections |
364
+ | `--where` | `-w` | array | | Filter documents |
365
+ | `--exclude` | `-x` | array | | Exclude subcollections |
366
+ | `--merge` | `-m` | boolean | `false` | Merge instead of overwrite |
367
+ | `--parallel` | `-p` | number | `1` | Parallel transfers |
368
+ | `--dry-run` | `-d` | boolean | `true` | Preview without writing |
369
+ | `--batch-size` | `-b` | number | `500` | Documents per batch |
370
+ | `--limit` | `-l` | number | `0` | Limit docs (0 = no limit) |
371
+ | `--retries` | | number | `3` | Retries on error |
372
+ | `--log` | | string | | Log file path |
373
+ | `--quiet` | `-q` | boolean | `false` | No progress bar |
374
+ | `--yes` | `-y` | boolean | `false` | Skip confirmation |
375
+ | `--clear` | | boolean | `false` | Clear destination before transfer |
376
+ | `--delete-missing` | | boolean | `false` | Delete dest docs not in source |
377
+ | `--interactive` | `-i` | boolean | `false` | Interactive mode with prompts |
378
+ | `--transform` | `-t` | string | | Path to JS/TS transform file |
379
+ | `--rename-collection` | `-r` | array | | Rename collection (source:dest) |
380
+ | `--id-prefix` | | string | | Add prefix to document IDs |
381
+ | `--id-suffix` | | string | | Add suffix to document IDs |
382
+ | `--webhook` | | string | | Webhook URL for notifications |
383
+ | `--resume` | | boolean | `false` | Resume from saved state |
384
+ | `--state-file` | | string | `.fscopy-state.json` | State file path |
385
+ | `--verify` | | boolean | `false` | Verify counts after transfer |
386
+ | `--rate-limit` | | number | `0` | Limit docs/second (0 = unlimited) |
387
+ | `--skip-oversized` | | boolean | `false` | Skip documents > 1MB |
388
+ | `--json` | | boolean | `false` | JSON output for CI/CD |
389
+ | `--max-depth` | | number | `0` | Max subcollection depth (0 = unlimited) |
390
+ | `--detect-conflicts` | | boolean | `false` | Detect concurrent modifications |
391
+ | `--verify-integrity` | | boolean | `false` | Verify document integrity with hash |
389
392
 
390
393
  ## How It Works
391
394
 
@@ -415,6 +418,46 @@ fscopy --init config.json
415
418
  - **Transform applies to all** - Transform function is applied to both root and subcollection docs
416
419
  - **Same project allowed** - Source and destination can be the same project when using `--rename-collection` or `--id-prefix`/`--id-suffix`
417
420
 
421
+ ## Limitations
422
+
423
+ ### Firestore Special Types
424
+
425
+ When reading documents, Firestore sentinel values are resolved to their actual values:
426
+
427
+ | Sentinel | Behavior |
428
+ | -------- | -------- |
429
+ | `serverTimestamp()` | Resolved to actual `Timestamp` value |
430
+ | `increment()` | Resolved to current numeric value |
431
+ | `arrayUnion()` / `arrayRemove()` | Resolved to current array value |
432
+
433
+ These sentinels are **write-time operations**, not persistent values. fscopy transfers the resolved data, which is the expected behavior for data migration.
434
+
435
+ ### Document References
436
+
437
+ `DocumentReference` fields are transferred as-is. If the reference points to a document in the source project, it will still point to the source after transfer. Consider using `--transform` to update references if needed.
438
+
439
+ ### Collection and Document IDs
440
+
441
+ fscopy validates IDs according to Firestore rules:
442
+
443
+ - Cannot be empty
444
+ - Cannot be `.` or `..`
445
+ - Cannot match `__*__` pattern (reserved by Firestore)
446
+
447
+ Unicode characters, special characters (`#`, `$`, `[`, `]`), and forward slashes in nested paths are all supported.
448
+
449
+ ### Subcollection Depth
450
+
451
+ Use `--max-depth` to limit recursion when copying deeply nested subcollections:
452
+
453
+ ```bash
454
+ # Copy only first level of subcollections
455
+ fscopy -f config.ini -s --max-depth 1
456
+
457
+ # Copy up to 3 levels deep
458
+ fscopy -f config.ini -s --max-depth 3
459
+ ```
460
+
418
461
  ## Development
419
462
 
420
463
  ```bash
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@fazetitans/fscopy",
3
- "version": "1.1.3",
3
+ "version": "1.2.0",
4
4
  "description": "Fast CLI tool to copy Firestore collections between Firebase projects with filtering, parallel transfers, and subcollection support",
5
5
  "type": "module",
6
6
  "bin": {
@@ -12,8 +12,8 @@
12
12
  "test": "bun test",
13
13
  "test:watch": "bun test --watch",
14
14
  "type-check": "tsc --noEmit",
15
- "lint": "eslint src/**/*.ts",
16
- "lint:fix": "eslint src/**/*.ts --fix",
15
+ "lint": "eslint src/**/*.ts --ignore-pattern 'src/__tests__/**' --no-warn-ignored",
16
+ "lint:fix": "eslint src/**/*.ts --ignore-pattern 'src/__tests__/**' --no-warn-ignored --fix",
17
17
  "format": "prettier --write src/**/*.ts",
18
18
  "format:check": "prettier --check src/**/*.ts",
19
19
  "prepublishOnly": "bun run type-check && bun run lint && bun test"