@arela/uploader 1.0.23 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (85) hide show
  1. package/docs/AUTO_PROCESSING_PIPELINE.md +258 -0
  2. package/docs/COMPLETE_USAGE_GUIDE.md +1363 -0
  3. package/docs/DATABASESERVICE_IMPROVEMENTS.md +546 -0
  4. package/docs/PASO_2_TEST_RESULTS.md +298 -0
  5. package/docs/PASO_3_PLAN.md +385 -0
  6. package/docs/PHASE_1_FILE_DETECTION.md +366 -0
  7. package/docs/PHASE_2_API_INTEGRATION.md +426 -0
  8. package/docs/PHASE_3_DATABASE_MANAGEMENT.md +480 -0
  9. package/docs/PHASE_4_FILE_OPERATIONS.md +448 -0
  10. package/docs/PHASE_5_WATCH_MODE.md +450 -0
  11. package/docs/PHASE_6_SIGNAL_HANDLING.md +472 -0
  12. package/docs/PHASE_7_ADVANCED_FEATURES.md +560 -0
  13. package/docs/PLAN_WATCH_FEATURE.md +417 -0
  14. package/docs/README.md +480 -0
  15. package/docs/SCHEMA_ALIGNMENT_SUMMARY.md +301 -0
  16. package/docs/SMARTWATCH_DATABASE_REFACTORING.md +181 -0
  17. package/docs/SMART_WATCH_DATABASE_CHANGES.md +502 -0
  18. package/docs/TESTING_WATCH_MODE.md +212 -0
  19. package/docs/WATCHER_API_IMPLEMENTATION.md +520 -0
  20. package/docs/WATCHER_API_INTEGRATION.md +562 -0
  21. package/docs/WATCHER_SETUP_GUIDE.md +614 -0
  22. package/docs/WATCH_ARCHITECTURE.md +395 -0
  23. package/docs/WATCH_AUTO_PIPELINE.md +334 -0
  24. package/docs/WATCH_CONFIGURATION.md +267 -0
  25. package/docs/WATCH_USAGE_GUIDE.md +567 -0
  26. package/docs/commands.md +14 -0
  27. package/package.json +1 -1
  28. package/scripts/scoring-compare.js +243 -0
  29. package/scripts/scoring-phase4-check.js +96 -0
  30. package/src/commands/IdentifyCommand.js +36 -0
  31. package/src/config/config.js +2 -2
  32. package/src/file-detection.js +71 -4
  33. package/src/scoring/db-matcher-adapter.js +98 -0
  34. package/src/scoring/matchers-seed.js +386 -0
  35. package/src/scoring/scoring-engine.js +246 -0
  36. package/src/services/ScanApiService.js +14 -0
  37. package/tests/unit/scoring-engine.test.js +221 -0
  38. package/.vscode/settings.json +0 -1
  39. package/coverage/IdentifyCommand.js.html +0 -1462
  40. package/coverage/PropagateCommand.js.html +0 -1507
  41. package/coverage/PushCommand.js.html +0 -1504
  42. package/coverage/ScanCommand.js.html +0 -1654
  43. package/coverage/UploadCommand.js.html +0 -1846
  44. package/coverage/WatchCommand.js.html +0 -4111
  45. package/coverage/base.css +0 -224
  46. package/coverage/block-navigation.js +0 -87
  47. package/coverage/favicon.png +0 -0
  48. package/coverage/index.html +0 -191
  49. package/coverage/lcov-report/IdentifyCommand.js.html +0 -1462
  50. package/coverage/lcov-report/PropagateCommand.js.html +0 -1507
  51. package/coverage/lcov-report/PushCommand.js.html +0 -1504
  52. package/coverage/lcov-report/ScanCommand.js.html +0 -1654
  53. package/coverage/lcov-report/UploadCommand.js.html +0 -1846
  54. package/coverage/lcov-report/WatchCommand.js.html +0 -4111
  55. package/coverage/lcov-report/base.css +0 -224
  56. package/coverage/lcov-report/block-navigation.js +0 -87
  57. package/coverage/lcov-report/favicon.png +0 -0
  58. package/coverage/lcov-report/index.html +0 -191
  59. package/coverage/lcov-report/prettify.css +0 -1
  60. package/coverage/lcov-report/prettify.js +0 -2
  61. package/coverage/lcov-report/sort-arrow-sprite.png +0 -0
  62. package/coverage/lcov-report/sorter.js +0 -210
  63. package/coverage/lcov.info +0 -1937
  64. package/coverage/prettify.css +0 -1
  65. package/coverage/prettify.js +0 -2
  66. package/coverage/sort-arrow-sprite.png +0 -0
  67. package/coverage/sorter.js +0 -210
  68. package/docs/API_ENDPOINTS_FOR_DETECTION.md +0 -647
  69. package/docs/API_RETRY_MECHANISM.md +0 -338
  70. package/docs/ARELA_IDENTIFY_IMPLEMENTATION.md +0 -489
  71. package/docs/ARELA_IDENTIFY_QUICKREF.md +0 -186
  72. package/docs/ARELA_PROPAGATE_IMPLEMENTATION.md +0 -581
  73. package/docs/ARELA_PROPAGATE_QUICKREF.md +0 -272
  74. package/docs/ARELA_PUSH_IMPLEMENTATION.md +0 -577
  75. package/docs/ARELA_PUSH_QUICKREF.md +0 -322
  76. package/docs/ARELA_SCAN_IMPLEMENTATION.md +0 -373
  77. package/docs/ARELA_SCAN_QUICKREF.md +0 -139
  78. package/docs/CROSS_PLATFORM_PATH_HANDLING.md +0 -597
  79. package/docs/DETECTION_ATTEMPT_TRACKING.md +0 -414
  80. package/docs/MIGRATION_UPLOADER_TO_FILE_STATS.md +0 -1020
  81. package/docs/MULTI_LEVEL_DIRECTORY_SCANNING.md +0 -494
  82. package/docs/QUICK_REFERENCE_API_DETECTION.md +0 -264
  83. package/docs/REFACTORING_SUMMARY_DETECT_PEDIMENTOS.md +0 -200
  84. package/docs/STATS_COMMAND_SEQUENCE_DIAGRAM.md +0 -287
  85. package/docs/STATS_COMMAND_SIMPLE.md +0 -93
@@ -1,139 +0,0 @@
1
- # Arela Scan Quick Reference
2
-
3
- ## Setup
4
-
5
- ### 1. Configure Backend
6
-
7
- Add `cli_registry` entity to TypeORM and run migration:
8
-
9
- ```bash
10
- cd arela-api
11
- npm run migration:generate -- -n CreateCliRegistry
12
- npm run migration:run
13
- ```
14
-
15
- ### 2. Configure CLI
16
-
17
- Set environment variables in `.env`:
18
-
19
- ```bash
20
- # Required
21
- ARELA_COMPANY_SLUG=your_company
22
- ARELA_SERVER_ID=server01
23
- UPLOAD_BASE_PATH=/path/to/files
24
- UPLOAD_SOURCES=2023|2024|2025
25
-
26
- # Optional
27
- ARELA_BASE_PATH_LABEL=data
28
- SCAN_EXCLUDE_PATTERNS=.DS_Store,Thumbs.db,desktop.ini
29
- SCAN_BATCH_SIZE=2000
30
-
31
- # API Configuration
32
- ARELA_API_URL=http://localhost:3010
33
- ARELA_API_TOKEN=your-token
34
- ```
35
-
36
- ## Commands
37
-
38
- ### Scan Filesystem
39
-
40
- ```bash
41
- # Basic scan with throughput display
42
- arela scan
43
-
44
- # Scan with percentage progress (counts files first)
45
- arela scan --count-first
46
-
47
- # Scan to specific API target
48
- arela scan --api cliente
49
- ```
50
-
51
- ### View Scan Instances
52
-
53
- ```bash
54
- # List all registered instances
55
- curl -H "x-api-key: $TOKEN" \
56
- http://localhost:3010/api/uploader/scan/instances
57
-
58
- # Get stale instances (no scan > 90 days)
59
- curl -H "x-api-key: $TOKEN" \
60
- "http://localhost:3010/api/uploader/scan/stale-instances?days=90"
61
- ```
62
-
63
- ### Deactivate Instance
64
-
65
- ```bash
66
- curl -X PATCH \
67
- -H "x-api-key: $TOKEN" \
68
- -H "Content-Type: application/json" \
69
- -d '{"tableName":"file_stats_company_server_path"}' \
70
- http://localhost:3010/api/uploader/scan/deactivate
71
- ```
72
-
73
- ## Files Modified
74
-
75
- ### Backend
76
-
77
- - ✅ `src/uploader/entities/cli-registry.entity.ts` - New entity
78
- - ✅ `src/uploader/services/file-stats-table-manager.service.ts` - New service
79
- - ✅ `src/uploader/services/uploader.service.ts` - Added scan methods
80
- - ✅ `src/uploader/controllers/uploader.controller.ts` - Added scan endpoints
81
- - ✅ `src/uploader/uploader.module.ts` - Updated imports
82
-
83
- ### CLI
84
-
85
- - ✅ `src/commands/ScanCommand.js` - New command
86
- - ✅ `src/services/ScanApiService.js` - New API service
87
- - ✅ `src/config/config.js` - Added scan configuration
88
- - ✅ `src/index.js` - Registered scan command
89
- - ✅ `.env.template` - Added scan variables
90
- - ✅ `docs/ARELA_SCAN_IMPLEMENTATION.md` - Documentation
91
-
92
- ## Table Schema
93
-
94
- Each scan instance creates a table:
95
-
96
- ```sql
97
- file_stats_<company>_<server>_<path>
98
- ├── id (uuid)
99
- ├── file_name (varchar)
100
- ├── file_extension (varchar)
101
- ├── directory_path (text)
102
- ├── relative_path (text)
103
- ├── absolute_path (text) [unique]
104
- ├── size_bytes (bigint)
105
- ├── modified_at (timestamp)
106
- ├── scan_timestamp (timestamp)
107
- └── created_at (timestamp)
108
- ```
109
-
110
- ## Troubleshooting
111
-
112
- ### Error: Missing configuration
113
-
114
- **Cause**: Required env vars not set
115
- **Fix**: Set `ARELA_COMPANY_SLUG` and `ARELA_SERVER_ID`
116
-
117
- ### Error: Table name collision
118
-
119
- **Cause**: Same identifiers used with different base path
120
- **Fix**: Change server ID or base path label
121
-
122
- ### Error: Cannot connect to API
123
-
124
- **Cause**: Backend not running or wrong URL
125
- **Fix**: Verify `ARELA_API_URL` and start backend
126
-
127
- ## Next Steps
128
-
129
- 1. ✅ Implement `arela scan` (DONE)
130
- 2. ⏳ Implement `arela identify` (detect pedimentos from PDFs)
131
- 3. ⏳ Implement `arela propagate` (propagate arela_path)
132
- 4. ⏳ Implement `arela push` (upload files by RFC)
133
-
134
- ## Performance Tips
135
-
136
- - Increase `SCAN_BATCH_SIZE` for better throughput (default: 2000)
137
- - Use `MAX_API_CONNECTIONS` to match backend replicas
138
- - Run scans during off-peak hours for large datasets
139
- - Use `--count-first` only when progress percentage is needed