@arela/uploader 0.2.13 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.template +66 -0
- package/README.md +263 -62
- package/docs/API_ENDPOINTS_FOR_DETECTION.md +647 -0
- package/docs/QUICK_REFERENCE_API_DETECTION.md +264 -0
- package/docs/REFACTORING_SUMMARY_DETECT_PEDIMENTOS.md +200 -0
- package/package.json +3 -2
- package/scripts/cleanup-ds-store.js +109 -0
- package/scripts/cleanup-system-files.js +69 -0
- package/scripts/tests/phase-7-features.test.js +415 -0
- package/scripts/tests/signal-handling.test.js +275 -0
- package/scripts/tests/smart-watch-integration.test.js +554 -0
- package/scripts/tests/watch-service-integration.test.js +584 -0
- package/src/commands/UploadCommand.js +31 -4
- package/src/commands/WatchCommand.js +1342 -0
- package/src/config/config.js +270 -2
- package/src/document-type-shared.js +2 -0
- package/src/document-types/support-document.js +200 -0
- package/src/file-detection.js +9 -1
- package/src/index.js +163 -4
- package/src/services/AdvancedFilterService.js +505 -0
- package/src/services/AutoProcessingService.js +749 -0
- package/src/services/BenchmarkingService.js +381 -0
- package/src/services/DatabaseService.js +1019 -539
- package/src/services/ErrorMonitor.js +275 -0
- package/src/services/LoggingService.js +419 -1
- package/src/services/MonitoringService.js +401 -0
- package/src/services/PerformanceOptimizer.js +511 -0
- package/src/services/ReportingService.js +511 -0
- package/src/services/SignalHandler.js +255 -0
- package/src/services/SmartWatchDatabaseService.js +527 -0
- package/src/services/WatchService.js +783 -0
- package/src/services/upload/ApiUploadService.js +447 -3
- package/src/services/upload/MultiApiUploadService.js +233 -0
- package/src/services/upload/SupabaseUploadService.js +12 -5
- package/src/services/upload/UploadServiceFactory.js +24 -0
- package/src/utils/CleanupManager.js +262 -0
- package/src/utils/FileOperations.js +44 -0
- package/src/utils/WatchEventHandler.js +522 -0
- package/supabase/migrations/001_create_initial_schema.sql +366 -0
- package/supabase/migrations/002_align_with_arela_api_schema.sql +145 -0
- package/.envbackup +0 -37
- package/SUPABASE_UPLOAD_FIX.md +0 -157
- package/commands.md +0 -14
package/SUPABASE_UPLOAD_FIX.md
DELETED
|
@@ -1,157 +0,0 @@
|
|
|
1
|
-
# Supabase Upload Issue Fix
|
|
2
|
-
|
|
3
|
-
## Problem Description
|
|
4
|
-
|
|
5
|
-
When running the CLI with `upload --force-supabase`, the logs displayed that files were being uploaded successfully, but no files appeared in the Supabase bucket.
|
|
6
|
-
|
|
7
|
-
## Root Cause
|
|
8
|
-
|
|
9
|
-
The issue was in `/src/commands/UploadCommand.js` in the `#processFile` method. The code was **silently ignoring upload failures** from Supabase:
|
|
10
|
-
|
|
11
|
-
```javascript
|
|
12
|
-
// ❌ BEFORE - Upload result was ignored
|
|
13
|
-
if (uploadService.getServiceName() === 'Arela API') {
|
|
14
|
-
result = await uploadService.upload([fileObject], { ...options, uploadPath });
|
|
15
|
-
} else {
|
|
16
|
-
// Supabase direct upload
|
|
17
|
-
await uploadService.upload([fileObject], { uploadPath });
|
|
18
|
-
// Result ignored! No error checking!
|
|
19
|
-
}
|
|
20
|
-
|
|
21
|
-
logger.info(`SUCCESS: ${path.basename(filePath)} -> ${uploadPath}`);
|
|
22
|
-
```
|
|
23
|
-
|
|
24
|
-
Even if the Supabase upload failed and returned `{ success: false, error: "..." }`, the code would:
|
|
25
|
-
1. Ignore the error
|
|
26
|
-
2. Log "SUCCESS"
|
|
27
|
-
3. Continue processing
|
|
28
|
-
|
|
29
|
-
This created the illusion that files were being uploaded when they weren't.
|
|
30
|
-
|
|
31
|
-
## Changes Made
|
|
32
|
-
|
|
33
|
-
### 1. Fixed Error Handling in `UploadCommand.js`
|
|
34
|
-
|
|
35
|
-
**File:** `/src/commands/UploadCommand.js`
|
|
36
|
-
|
|
37
|
-
Now properly captures and validates the Supabase upload result:
|
|
38
|
-
|
|
39
|
-
```javascript
|
|
40
|
-
// ✅ AFTER - Upload result is captured and validated
|
|
41
|
-
if (uploadService.getServiceName() === 'Arela API') {
|
|
42
|
-
result = await uploadService.upload([fileObject], { ...options, uploadPath });
|
|
43
|
-
} else {
|
|
44
|
-
// Supabase direct upload
|
|
45
|
-
const uploadResult = await uploadService.upload([fileObject], { uploadPath });
|
|
46
|
-
|
|
47
|
-
// Check if upload was successful
|
|
48
|
-
if (!uploadResult.success) {
|
|
49
|
-
throw new Error(`Supabase upload failed: ${uploadResult.error}`);
|
|
50
|
-
}
|
|
51
|
-
|
|
52
|
-
result = { successCount: 1 };
|
|
53
|
-
}
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
### 2. Enhanced Logging in `SupabaseUploadService.js`
|
|
57
|
-
|
|
58
|
-
**File:** `/src/services/upload/SupabaseUploadService.js`
|
|
59
|
-
|
|
60
|
-
Added detailed logging to help diagnose upload issues:
|
|
61
|
-
|
|
62
|
-
#### Connection Initialization
|
|
63
|
-
```javascript
|
|
64
|
-
console.log(`🔌 Connecting to Supabase: ${appConfig.supabase.url}`);
|
|
65
|
-
console.log(`📦 Using bucket: ${this.bucket}`);
|
|
66
|
-
console.log(`🧪 Testing bucket access...`);
|
|
67
|
-
console.log(`✅ Successfully connected to Supabase bucket: ${this.bucket}`);
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
#### Upload Operations
|
|
71
|
-
```javascript
|
|
72
|
-
if (error) {
|
|
73
|
-
console.error(`❌ Supabase upload failed for ${normalizedPath}:`, error.message);
|
|
74
|
-
return { success: false, error: error.message };
|
|
75
|
-
}
|
|
76
|
-
|
|
77
|
-
console.log(`✅ Supabase upload successful: ${normalizedPath}`);
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
#### Better Error Messages
|
|
81
|
-
```javascript
|
|
82
|
-
if (!appConfig.supabase.url || !appConfig.supabase.key || !this.bucket) {
|
|
83
|
-
const missing = [];
|
|
84
|
-
if (!appConfig.supabase.url) missing.push('SUPABASE_URL');
|
|
85
|
-
if (!appConfig.supabase.key) missing.push('SUPABASE_KEY');
|
|
86
|
-
if (!this.bucket) missing.push('SUPABASE_BUCKET');
|
|
87
|
-
throw new Error(`Missing Supabase configuration: ${missing.join(', ')}`);
|
|
88
|
-
}
|
|
89
|
-
```
|
|
90
|
-
|
|
91
|
-
## Expected Behavior After Fix
|
|
92
|
-
|
|
93
|
-
### On Successful Upload
|
|
94
|
-
```
|
|
95
|
-
🔌 Connecting to Supabase: https://your-project.supabase.co
|
|
96
|
-
📦 Using bucket: your-bucket-name
|
|
97
|
-
🧪 Testing bucket access...
|
|
98
|
-
✅ Successfully connected to Supabase bucket: your-bucket-name
|
|
99
|
-
✅ Supabase upload successful: 2023/2000601/file.xml
|
|
100
|
-
```
|
|
101
|
-
|
|
102
|
-
### On Failed Upload
|
|
103
|
-
```
|
|
104
|
-
❌ Supabase upload failed for 2023/2000601/file.xml: [error details]
|
|
105
|
-
Error: Supabase upload failed: [error details]
|
|
106
|
-
```
|
|
107
|
-
|
|
108
|
-
### On Configuration Error
|
|
109
|
-
```
|
|
110
|
-
Error: Missing Supabase configuration: SUPABASE_URL, SUPABASE_KEY
|
|
111
|
-
```
|
|
112
|
-
|
|
113
|
-
## Common Issues to Check
|
|
114
|
-
|
|
115
|
-
If uploads still fail after this fix, check:
|
|
116
|
-
|
|
117
|
-
1. **Environment Variables**
|
|
118
|
-
```bash
|
|
119
|
-
echo $SUPABASE_URL
|
|
120
|
-
echo $SUPABASE_KEY # Should be anon or service_role key
|
|
121
|
-
echo $SUPABASE_BUCKET
|
|
122
|
-
```
|
|
123
|
-
|
|
124
|
-
2. **Bucket Permissions**
|
|
125
|
-
- Check if the bucket exists in Supabase Storage
|
|
126
|
-
- Verify the API key has write permissions to the bucket
|
|
127
|
-
- Check bucket policies (public vs private)
|
|
128
|
-
|
|
129
|
-
3. **Network/Firewall**
|
|
130
|
-
- Ensure the server can reach Supabase (check firewall rules)
|
|
131
|
-
- Test connection: `curl https://your-project.supabase.co`
|
|
132
|
-
|
|
133
|
-
4. **File Size Limits**
|
|
134
|
-
- Check Supabase storage limits for your plan
|
|
135
|
-
- Verify file sizes are within allowed limits
|
|
136
|
-
|
|
137
|
-
5. **Path Format**
|
|
138
|
-
- Ensure paths don't contain invalid characters
|
|
139
|
-
- Paths are normalized (forward slashes only)
|
|
140
|
-
|
|
141
|
-
## Testing the Fix
|
|
142
|
-
|
|
143
|
-
Run the upload command with verbose logging:
|
|
144
|
-
|
|
145
|
-
```bash
|
|
146
|
-
arela upload --force-supabase -v
|
|
147
|
-
```
|
|
148
|
-
|
|
149
|
-
You should now see:
|
|
150
|
-
- Clear connection status messages
|
|
151
|
-
- Per-file upload success/failure messages
|
|
152
|
-
- Proper error messages if uploads fail
|
|
153
|
-
- Accurate success/error counts in the summary
|
|
154
|
-
|
|
155
|
-
## Migration Notes
|
|
156
|
-
|
|
157
|
-
No database migrations required. This is a code-only fix that improves error handling and logging.
|
package/commands.md
DELETED
|
@@ -1,14 +0,0 @@
|
|
|
1
|
-
node src/index.js --stats-only
|
|
2
|
-
node src/index.js --detect-pdfs
|
|
3
|
-
node src/index.js --propagate-arela-path
|
|
4
|
-
node src/index.js --upload-by-rfc --folder-structure palco
|
|
5
|
-
|
|
6
|
-
UPLOAD_RFCS="RFC1|RFC2" node src/index.js --upload-by-rfc --folder-structure target-folder
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
node src/index.js stats --stats-only
|
|
12
|
-
node src/index.js detect --detect-pdfs
|
|
13
|
-
node src/index.js detect --propagate-arela-path
|
|
14
|
-
node src/index.js upload --upload-by-rfc --folder-structure palco
|