hazo_files 1.4.7 → 1.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGE_LOG.md +486 -0
- package/README.md +158 -0
- package/SETUP_CHECKLIST.md +1309 -0
- package/dist/background-upload/index.d.mts +166 -0
- package/dist/background-upload/index.d.ts +166 -0
- package/dist/background-upload/index.js +301 -0
- package/dist/background-upload/index.mjs +271 -0
- package/dist/background-upload/react/index.d.mts +149 -0
- package/dist/background-upload/react/index.d.ts +149 -0
- package/dist/background-upload/react/index.js +474 -0
- package/dist/background-upload/react/index.mjs +433 -0
- package/package.json +26 -4
- package/docs/SETUP_CHECKLIST.md +0 -260
package/CHANGE_LOG.md
ADDED
|
@@ -0,0 +1,486 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [1.5.2] - 2026-05-18
|
|
9
|
+
|
|
10
|
+
### Fixed
|
|
11
|
+
- `useFileUpload` no longer emits the React dev warning *"The result of getServerSnapshot should be cached to avoid an infinite loop"*. The third argument to `useSyncExternalStore` now points at a module-level frozen `EMPTY_ACTIVE_JOBS` constant instead of allocating a fresh `[]` per call, so React's reference identity check holds across renders.
|
|
12
|
+
|
|
13
|
+
## 1.5.0
|
|
14
|
+
|
|
15
|
+
- **NEW**: `background_upload` module — UploadManager, Job, PipelineExecutor for pipelines that survive React component unmount.
|
|
16
|
+
- Subpath: `hazo_files/background-upload` (core, framework-agnostic)
|
|
17
|
+
- Subpath: `hazo_files/background-upload/react` (Provider, useFileUpload, useJobStatus, sonner toast bridge)
|
|
18
|
+
|
|
19
|
+
## [1.4.7] - 2026-05-10
|
|
20
|
+
|
|
21
|
+
### Changed
|
|
22
|
+
- Aligned dev dependencies to workspace canonical versions: `@types/node` (`^20.14.10`), `@types/react` (`^18.3.3`), `@types/react-dom` (`^18.3.0`), `typescript` (`^5.7.2`)
|
|
23
|
+
- Tightened peer dependency ranges from loose `>=` to pinned major.minor: `hazo_connect` (`^2.4.0`), `hazo_debug` (`^2.0.0`), `hazo_llm_api` (`^1.2.0`), `hazo_logs` (`^1.0.13`)
|
|
24
|
+
- test-app migrated to Tailwind CSS v4: replaced `tailwindcss-animate` with `tw-animate-css`, removed `autoprefixer` (Lightning CSS in `@tailwindcss/postcss` handles prefixing), updated `globals.css` to use `@import "tailwindcss"` syntax with `@config` directive
|
|
25
|
+
- test-app `clsx` (`^2.1.1`) and `tailwind-merge` (`^3.5.0`) bumped to canonical versions
|
|
26
|
+
|
|
27
|
+
### Notes
|
|
28
|
+
- Consumers using the optional `hazo_debug` integration must be on `hazo_debug ^2.0.0` (previously `>=1.0.0`)
|
|
29
|
+
- No public API changes; consumer setup remains unchanged
|
|
30
|
+
|
|
31
|
+
## [1.4.6] - 2026-03-30
|
|
32
|
+
|
|
33
|
+
### Added
|
|
34
|
+
- Structured `file_operation` log entries with timing (`duration_ms`), storage provider, and operation type emitted on every file operation via the `logger` option
|
|
35
|
+
- `logger` option on `FileManagerOptions` — enables structured logging at the storage level even without database tracking
|
|
36
|
+
- Logger passthrough from `createHazoFilesServer` → `TrackedFileManager` → `FileMetadataService` for end-to-end operation logging
|
|
37
|
+
- `hazo_debug` as optional peer dependency for visual file operation debugging
|
|
38
|
+
- Debug integration section in README documenting how to bridge hazo_files logs to hazo_debug
|
|
39
|
+
|
|
40
|
+
## [1.4.5] - 2026-03-27
|
|
41
|
+
|
|
42
|
+
### Security
|
|
43
|
+
- Fix path traversal vulnerability in `renameFile` and `renameFolder` — user-supplied `newName` is now validated to reject path separators and `..`
|
|
44
|
+
- Fix Google Drive API query injection — single quotes in file/folder names are now properly escaped in Drive API queries
|
|
45
|
+
- Fix SQL injection in schema helper functions (`getSchemaForTable`, `getMigrationForTable`, `getNamingSchemaForTable`, `getMigrationV3ForTable`) — custom table names are now validated against `^[a-z_][a-z0-9_]*$`
|
|
46
|
+
|
|
47
|
+
### Fixed
|
|
48
|
+
- `initializeSync` now throws a clear error instead of silently creating a broken state (storage modules require async initialization)
|
|
49
|
+
- `hasFileChanged` no longer hashes file path strings — returns `null` (unknown) when download returns a path instead of content
|
|
50
|
+
- `exchangeCodeForTokens` handles nullable `refresh_token` with fallback instead of non-null assertion
|
|
51
|
+
- `server-only` guard in `hazo_files/server` now logs a warning instead of silently swallowing the check
|
|
52
|
+
|
|
53
|
+
### Changed
|
|
54
|
+
- **Breaking (peer deps)**: `googleapis`, `dropbox`, and `xxhash-wasm` moved from `dependencies` to optional `peerDependencies` — consumers must now install the SDKs for the storage providers they use
|
|
55
|
+
- Live `hazo_files_config.ini` no longer published to npm (prevents accidental credential exposure)
|
|
56
|
+
|
|
57
|
+
### Added
|
|
58
|
+
- `tsconfig.build.json` — separate build config per workspace standard
|
|
59
|
+
- `config/hazo_files_config.ini.sample` — configuration template with all options documented including Dropbox
|
|
60
|
+
- `design/architecture.md` — architecture overview and design decisions
|
|
61
|
+
- `migrations/` directory with SQL files for all schema versions (001, 002, 003)
|
|
62
|
+
|
|
63
|
+
## [1.4.1] - 2026-02-09
|
|
64
|
+
|
|
65
|
+
### Fixed
|
|
66
|
+
- Extension validation mismatch in LocalStorageModule: config extensions with leading dots (e.g., `.pdf`) were not matching extracted extensions without dots (e.g., `pdf`), causing all uploads to fail when `allowed_extensions` was configured with dotted format
|
|
67
|
+
- Extensions are now normalized at initialization (trimmed, dot-stripped, lowercased), making validation dot-agnostic and case-insensitive
|
|
68
|
+
|
|
69
|
+
## [1.0.0] - 2025-12-09
|
|
70
|
+
|
|
71
|
+
### Added
|
|
72
|
+
|
|
73
|
+
#### Core Features
|
|
74
|
+
- Initial release of hazo_files package
|
|
75
|
+
- Unified FileManager service providing consistent API across storage providers
|
|
76
|
+
- Modular architecture for easy extension with custom storage providers
|
|
77
|
+
- Full TypeScript support with comprehensive type definitions
|
|
78
|
+
|
|
79
|
+
#### Storage Providers
|
|
80
|
+
- **LocalStorageModule**: Direct filesystem operations using Node.js `fs` module
|
|
81
|
+
- Extension filtering via `allowedExtensions` configuration
|
|
82
|
+
- File size limits via `maxFileSize` configuration
|
|
83
|
+
- Progress tracking for upload/download operations
|
|
84
|
+
- Recursive directory operations
|
|
85
|
+
- **GoogleDriveModule**: Google Drive integration with OAuth 2.0
|
|
86
|
+
- Full Google Drive API v3 implementation
|
|
87
|
+
- OAuth authentication with automatic token refresh
|
|
88
|
+
- Token persistence via callback system
|
|
89
|
+
- Configurable root folder support
|
|
90
|
+
|
|
91
|
+
#### File Operations
|
|
92
|
+
- Create and remove directories (with recursive option)
|
|
93
|
+
- Upload files from local path, Buffer, or ReadableStream
|
|
94
|
+
- Download files to local path or return as Buffer
|
|
95
|
+
- Move and rename files and folders
|
|
96
|
+
- Delete files
|
|
97
|
+
- List directory contents with filtering options
|
|
98
|
+
- Get file/folder metadata
|
|
99
|
+
- Check file/folder existence
|
|
100
|
+
- Generate folder tree structure
|
|
101
|
+
|
|
102
|
+
#### UI Components
|
|
103
|
+
- **FileBrowser**: Complete drop-in file browser component
|
|
104
|
+
- Folder tree navigation with lazy loading
|
|
105
|
+
- File list with grid and list view modes
|
|
106
|
+
- Breadcrumb navigation
|
|
107
|
+
- File preview for images, text, and PDFs
|
|
108
|
+
- Action toolbar with upload, download, create, rename, delete
|
|
109
|
+
- Drag-and-drop support (via API adapter)
|
|
110
|
+
- **Individual Components**: PathBreadcrumb, FolderTree, FileList, FilePreview, FileActions
|
|
111
|
+
- **Dialogs**: CreateFolderDialog, RenameDialog, DeleteConfirmDialog, UploadDialog
|
|
112
|
+
- **Hooks**: useFileBrowser, useFileOperations, useMultiFileOperations
|
|
113
|
+
|
|
114
|
+
#### Configuration System
|
|
115
|
+
- INI file configuration (`hazo_files_config.ini`)
|
|
116
|
+
- Environment variable support with automatic override
|
|
117
|
+
- Programmatic configuration via code
|
|
118
|
+
- Configuration validation and defaults
|
|
119
|
+
- Sample configuration generator
|
|
120
|
+
|
|
121
|
+
#### Common Utilities
|
|
122
|
+
- **Error Types**: 12 specific error classes for different failure scenarios
|
|
123
|
+
- FileNotFoundError, DirectoryNotFoundError
|
|
124
|
+
- FileExistsError, DirectoryExistsError
|
|
125
|
+
- DirectoryNotEmptyError, PermissionDeniedError
|
|
126
|
+
- InvalidPathError, FileTooLargeError
|
|
127
|
+
- InvalidExtensionError, AuthenticationError
|
|
128
|
+
- ConfigurationError, OperationError
|
|
129
|
+
- **Path Utilities**: Normalization, joining, validation, sanitization
|
|
130
|
+
- **MIME Type Detection**: 50+ file types with category classification
|
|
131
|
+
- **Helper Functions**: ID generation, byte formatting, item sorting/filtering
|
|
132
|
+
|
|
133
|
+
#### Developer Experience
|
|
134
|
+
- Comprehensive documentation
|
|
135
|
+
- README with quick start and usage examples
|
|
136
|
+
- Technical documentation (TECHDOC.md)
|
|
137
|
+
- AI-optimized reference guide (CLAUDE.md)
|
|
138
|
+
- Setup checklist (SETUP_CHECKLIST.md)
|
|
139
|
+
- Module creation guide (docs/ADDING_MODULES.md)
|
|
140
|
+
- Complete TypeScript definitions
|
|
141
|
+
- Example Next.js test application
|
|
142
|
+
- Progress callbacks for upload/download operations
|
|
143
|
+
- OperationResult pattern for consistent error handling
|
|
144
|
+
|
|
145
|
+
### Design Decisions
|
|
146
|
+
|
|
147
|
+
#### Virtual Path System
|
|
148
|
+
**Decision**: Use Unix-style virtual paths (`/folder/file.pdf`) across all storage providers.
|
|
149
|
+
|
|
150
|
+
**Rationale**:
|
|
151
|
+
- Provides consistent API regardless of storage backend
|
|
152
|
+
- Prevents Windows/Unix path separator conflicts
|
|
153
|
+
- Simplifies path manipulation and validation
|
|
154
|
+
- Easier to implement access control and path restrictions
|
|
155
|
+
|
|
156
|
+
#### OperationResult Pattern
|
|
157
|
+
**Decision**: Return `{ success: boolean, data?: T, error?: string }` instead of throwing exceptions.
|
|
158
|
+
|
|
159
|
+
**Rationale**:
|
|
160
|
+
- Expected failures (file not found, permission denied) shouldn't be exceptions
|
|
161
|
+
- Easier error handling in async code
|
|
162
|
+
- Consistent pattern across all operations
|
|
163
|
+
- Better for API responses (JSON-serializable)
|
|
164
|
+
- Allows partial success in batch operations
|
|
165
|
+
|
|
166
|
+
#### Module System
|
|
167
|
+
**Decision**: BaseStorageModule abstract class with StorageModule interface.
|
|
168
|
+
|
|
169
|
+
**Rationale**:
|
|
170
|
+
- Enforces consistent interface across providers
|
|
171
|
+
- Provides common utilities (path normalization, result helpers)
|
|
172
|
+
- Reduces code duplication
|
|
173
|
+
- Makes adding new providers straightforward
|
|
174
|
+
- Separates provider-specific logic from common functionality
|
|
175
|
+
|
|
176
|
+
#### Separate UI Package Export
|
|
177
|
+
**Decision**: Export UI components separately (`hazo_files/ui`)
|
|
178
|
+
|
|
179
|
+
**Rationale**:
|
|
180
|
+
- Core package has no React dependency
|
|
181
|
+
- Allows using file management without UI
|
|
182
|
+
- Reduces bundle size for server-only usage
|
|
183
|
+
- Clear separation of concerns
|
|
184
|
+
- Different peer dependencies (React only for UI)
|
|
185
|
+
|
|
186
|
+
#### Configuration Priority
|
|
187
|
+
**Decision**: Code > Environment Variables > INI File > Defaults
|
|
188
|
+
|
|
189
|
+
**Rationale**:
|
|
190
|
+
- Code config for programmatic control (tests, dynamic config)
|
|
191
|
+
- Environment variables for deployment (Docker, cloud platforms)
|
|
192
|
+
- INI file for local development and simple deployments
|
|
193
|
+
- Defaults prevent configuration errors from breaking initialization
|
|
194
|
+
|
|
195
|
+
### Security
|
|
196
|
+
|
|
197
|
+
- Path validation prevents directory traversal attacks
|
|
198
|
+
- Filename sanitization removes dangerous characters
|
|
199
|
+
- Extension whitelisting prevents unwanted file types
|
|
200
|
+
- File size limits prevent resource exhaustion
|
|
201
|
+
- OAuth 2.0 for Google Drive with automatic token refresh
|
|
202
|
+
- No client-side credential exposure
|
|
203
|
+
|
|
204
|
+
### Performance
|
|
205
|
+
|
|
206
|
+
- Local storage: Sub-10ms operations for most actions
|
|
207
|
+
- Google Drive: 200-1000ms operations (network-dependent)
|
|
208
|
+
- Stream-based transfers for memory efficiency
|
|
209
|
+
- Lazy loading for folder tree
|
|
210
|
+
- Progress tracking without blocking
|
|
211
|
+
|
|
212
|
+
### Dependencies
|
|
213
|
+
|
|
214
|
+
#### Runtime
|
|
215
|
+
- `googleapis` (^140.0.1): Google Drive API client
|
|
216
|
+
- `ini` (^4.1.3): INI file parsing
|
|
217
|
+
|
|
218
|
+
#### Development
|
|
219
|
+
- `typescript` (^5.3.3): TypeScript compiler
|
|
220
|
+
- `tsup` (^8.0.1): Build tool
|
|
221
|
+
- `vitest` (^1.1.0): Testing framework
|
|
222
|
+
|
|
223
|
+
#### Peer Dependencies
|
|
224
|
+
- `react` (^18.0.0): For UI components
|
|
225
|
+
- `react-dom` (^18.0.0): For UI components
|
|
226
|
+
|
|
227
|
+
### Known Limitations
|
|
228
|
+
|
|
229
|
+
- Google Drive path resolution requires multiple API calls (O(n) where n = path depth)
|
|
230
|
+
- No built-in caching for Google Drive path-to-ID mappings
|
|
231
|
+
- UI components do not implement virtual scrolling for large directories
|
|
232
|
+
- File preview loads entire file into memory
|
|
233
|
+
- No batch operation support for Google Drive
|
|
234
|
+
- Local storage requires shared filesystem for horizontal scaling
|
|
235
|
+
|
|
236
|
+
### Migration Notes
|
|
237
|
+
|
|
238
|
+
This is the initial release, no migration required.
|
|
239
|
+
|
|
240
|
+
### Contributors
|
|
241
|
+
|
|
242
|
+
- Pubs Abayasiri (Creator and maintainer)
|
|
243
|
+
|
|
244
|
+
---
|
|
245
|
+
|
|
246
|
+
## [Unreleased]
|
|
247
|
+
|
|
248
|
+
### Added
|
|
249
|
+
|
|
250
|
+
#### FileBrowser Drag-and-Drop File Moving (2026-01-19)
|
|
251
|
+
|
|
252
|
+
**Feature**: Native drag-and-drop functionality for moving files and folders within the FileBrowser component.
|
|
253
|
+
|
|
254
|
+
**User-Facing Features**:
|
|
255
|
+
- Drag files and folders from the file list (grid or list view)
|
|
256
|
+
- Drop onto folders in either the sidebar tree or main file list
|
|
257
|
+
- Visual feedback with opacity and colored borders during drag
|
|
258
|
+
- Real-time validation prevents invalid operations
|
|
259
|
+
- Shows dragged item preview (icon + name) following cursor
|
|
260
|
+
|
|
261
|
+
**Visual Feedback**:
|
|
262
|
+
- **Dragging**: Dragged item becomes semi-transparent (opacity-50)
|
|
263
|
+
- **Valid drop target**: Green ring border (`ring-2 ring-green-500`) and light green background (`bg-green-50`)
|
|
264
|
+
- **Invalid drop target**: No highlighting (drop is ignored)
|
|
265
|
+
- **Drag preview**: White card with shadow showing file/folder icon and name (max width 200px)
|
|
266
|
+
|
|
267
|
+
**Drop Validation** (automatic, no user action needed):
|
|
268
|
+
- Prevents dropping item onto itself
|
|
269
|
+
- Prevents dropping into current parent directory (no-op)
|
|
270
|
+
- Prevents dropping folder into its own descendant (circular reference prevention)
|
|
271
|
+
- All other moves are allowed
|
|
272
|
+
|
|
273
|
+
**Technical Implementation**:
|
|
274
|
+
- Uses `@dnd-kit/core` library (already a peer dependency for NamingRuleConfigurator)
|
|
275
|
+
- Single top-level DndContext in FileBrowser component
|
|
276
|
+
- PointerSensor with 8px activation distance prevents accidental drags
|
|
277
|
+
- `closestCenter` collision detection algorithm
|
|
278
|
+
|
|
279
|
+
**New Component** (`src/ui/components/DragPreview.tsx`):
|
|
280
|
+
- `DragPreview` - Visual preview component shown during drag
|
|
281
|
+
- Props: `item: FileSystemItem`, `className?: string`
|
|
282
|
+
- Displays file/folder icon and name with styling
|
|
283
|
+
|
|
284
|
+
**API Integration**:
|
|
285
|
+
- Requires `FileBrowserAPI.moveItem(sourcePath, destinationPath)` method
|
|
286
|
+
- Automatically refreshes directory list and folder tree after successful move
|
|
287
|
+
- Error handling via `onError` callback prop
|
|
288
|
+
|
|
289
|
+
**ID Patterns Used**:
|
|
290
|
+
- Draggable items: `file-item-{path}` (all files and folders in FileList)
|
|
291
|
+
- Drop targets in tree: `folder-drop-tree-{path}` (folders in FolderTree sidebar)
|
|
292
|
+
- Drop targets in list: `folder-drop-list-{path}` (folders in FileList)
|
|
293
|
+
|
|
294
|
+
**State Management**:
|
|
295
|
+
- `draggedItem`: Currently dragged FileSystemItem
|
|
296
|
+
- `dropTargetPath`: Path of valid drop target being hovered
|
|
297
|
+
- `isDragging`: Boolean flag for drag in progress
|
|
298
|
+
|
|
299
|
+
**Files Changed**:
|
|
300
|
+
- `src/ui/components/FileBrowser.tsx` - Added DndContext, drag handlers, state management
|
|
301
|
+
- `src/ui/components/DragPreview.tsx` - New component for drag preview
|
|
302
|
+
- `src/ui/components/FileList.tsx` - Added useDraggable and useDroppable hooks
|
|
303
|
+
- `src/ui/components/FolderTree.tsx` - Added useDroppable hook
|
|
304
|
+
- `src/ui/index.ts` - Export DragPreview component and DragPreviewProps type
|
|
305
|
+
|
|
306
|
+
**Dependencies**:
|
|
307
|
+
- No new dependencies (uses existing `@dnd-kit/core` peer dependency)
|
|
308
|
+
|
|
309
|
+
**Design Decisions**:
|
|
310
|
+
|
|
311
|
+
**Single DndContext Pattern**:
|
|
312
|
+
- **Decision**: Use single top-level DndContext in FileBrowser, not in child components
|
|
313
|
+
- **Rationale**: Nested DndContext blocks drag events from parent handlers (same pattern as NamingRuleConfigurator). Child components use only useDroppable/useDraggable hooks.
|
|
314
|
+
|
|
315
|
+
**ID Prefix Convention**:
|
|
316
|
+
- **Decision**: Use prefixed IDs (`file-item-`, `folder-drop-tree-`, `folder-drop-list-`)
|
|
317
|
+
- **Rationale**: Allows distinguishing between draggable items and drop targets. Necessary because folders are both draggable and droppable.
|
|
318
|
+
|
|
319
|
+
**Validation Before API Call**:
|
|
320
|
+
- **Decision**: Validate drop targets twice: during drag (for visual feedback) and before API call
|
|
321
|
+
- **Rationale**: Prevents unnecessary API calls for invalid drops, provides immediate visual feedback
|
|
322
|
+
|
|
323
|
+
**Performance Optimization**:
|
|
324
|
+
- 8px activation distance prevents accidental drags during normal clicking
|
|
325
|
+
- CSS-based visual feedback (no re-renders)
|
|
326
|
+
- DragOverlay renders outside main component tree (isolated updates)
|
|
327
|
+
|
|
328
|
+
**Use Cases**:
|
|
329
|
+
- Reorganizing files into folders
|
|
330
|
+
- Moving files between project folders
|
|
331
|
+
- Quick file organization without copy/paste
|
|
332
|
+
- Dragging from file list to sidebar tree for easy navigation
|
|
333
|
+
|
|
334
|
+
#### Naming Rules Configuration System (2025-12-09)
|
|
335
|
+
|
|
336
|
+
**Feature**: Comprehensive naming rules system for generating consistent file and folder names.
|
|
337
|
+
|
|
338
|
+
**Core Types** (`src/types/naming.ts`):
|
|
339
|
+
- `NamingVariable` - Define custom and system variables with descriptions and examples
|
|
340
|
+
- `PatternSegment` - Building blocks for naming patterns (variable or literal)
|
|
341
|
+
- `NamingRuleSchema` - Complete schema with file and folder patterns, metadata
|
|
342
|
+
- `GeneratedNameResult` - Result object from name generation
|
|
343
|
+
- `NameGenerationOptions` - Options for controlling generation behavior
|
|
344
|
+
|
|
345
|
+
**Utility Functions** (`src/common/naming-utils.ts`):
|
|
346
|
+
- `hazo_files_generate_file_name()` - Generate file names from schemas
|
|
347
|
+
- `hazo_files_generate_folder_name()` - Generate folder paths from schemas
|
|
348
|
+
- `validateNamingRuleSchema()` - Validate schema structure
|
|
349
|
+
- `parsePatternString()` - Parse "{var}text" strings to segments
|
|
350
|
+
- `patternToString()` - Convert segments back to strings
|
|
351
|
+
- `formatDateToken()` - Format dates to various tokens (YYYY, MM, DD, etc.)
|
|
352
|
+
- `formatCounter()` - Format counter with zero padding
|
|
353
|
+
- `createVariableSegment()`, `createLiteralSegment()` - Segment builders
|
|
354
|
+
|
|
355
|
+
**System Variables**:
|
|
356
|
+
- **Date variables**: YYYY, YY, MM, M, DD, D, MMM, MMMM, YYYY-MM-DD, YYYY-MMM-DD, DD-MM-YYYY, MM-DD-YYYY
|
|
357
|
+
- **File metadata**: original_name, extension, ext
|
|
358
|
+
- **Counter**: counter (auto-incrementing with padding)
|
|
359
|
+
- Exported as constants: `SYSTEM_DATE_VARIABLES`, `SYSTEM_FILE_VARIABLES`, `SYSTEM_COUNTER_VARIABLES`, `ALL_SYSTEM_VARIABLES`
|
|
360
|
+
|
|
361
|
+
**UI Components** (`src/ui/components/naming/`):
|
|
362
|
+
- `NamingRuleConfigurator` - Main drop-in component for visual pattern building
|
|
363
|
+
- `VariableList` - Category tabs (User/Date/File/Counter) with draggable variables
|
|
364
|
+
- `PatternBuilder` - Drag-and-drop zones for file and folder patterns
|
|
365
|
+
- `PatternPreview` - Live preview of generated names with example values
|
|
366
|
+
- `PatternSegmentItem` - Individual segments in patterns (editable/removable)
|
|
367
|
+
- `DraggableVariable` - Draggable variable chips
|
|
368
|
+
- `SeparatorPicker` - Quick-add common separators (-, _, space, /)
|
|
369
|
+
|
|
370
|
+
**Hooks** (`src/ui/hooks/useNamingRule.ts`):
|
|
371
|
+
- `useNamingRule` - State management for naming patterns with:
|
|
372
|
+
- Undo/redo support (max 50 history entries)
|
|
373
|
+
- Keyboard shortcuts (Ctrl+Z, Ctrl+Y, Ctrl+Shift+Z)
|
|
374
|
+
- Pattern manipulation functions (add, remove, update, reorder, clear)
|
|
375
|
+
- Schema import/export
|
|
376
|
+
- isDirty tracking
|
|
377
|
+
|
|
378
|
+
**Configuration** (`hazo_files_config.ini`):
|
|
379
|
+
- New `[naming]` section
|
|
380
|
+
- `date_formats` setting for customizing available date format tokens
|
|
381
|
+
|
|
382
|
+
**Dependencies**:
|
|
383
|
+
- Added peer dependencies: `@dnd-kit/core`, `@dnd-kit/sortable`, `@dnd-kit/utilities` (for drag-and-drop UI)
|
|
384
|
+
|
|
385
|
+
**Design Decisions**:
|
|
386
|
+
|
|
387
|
+
**Variable-based Pattern System**:
|
|
388
|
+
- **Decision**: Use segment-based patterns instead of regex or string templates
|
|
389
|
+
- **Rationale**:
|
|
390
|
+
- Type-safe with full IDE support
|
|
391
|
+
- Easier to manipulate programmatically
|
|
392
|
+
- Works well with React drag-and-drop
|
|
393
|
+
- Clear separation between variables and literals
|
|
394
|
+
- Enables visual editing without string parsing complexity
|
|
395
|
+
|
|
396
|
+
**System vs User Variables**:
|
|
397
|
+
- **Decision**: Separate system variables (dates, file metadata) from user variables
|
|
398
|
+
- **Rationale**:
|
|
399
|
+
- System variables work automatically without configuration
|
|
400
|
+
- User variables are application-specific
|
|
401
|
+
- Category-based UI organization improves UX
|
|
402
|
+
- Clear distinction prevents naming conflicts
|
|
403
|
+
|
|
404
|
+
**Undo/Redo System**:
|
|
405
|
+
- **Decision**: Implement history with keyboard shortcuts in the hook
|
|
406
|
+
- **Rationale**:
|
|
407
|
+
- Essential for pattern building workflow
|
|
408
|
+
- Users expect standard shortcuts (Ctrl+Z)
|
|
409
|
+
- Separates history logic from UI components
|
|
410
|
+
- 50-entry limit prevents memory issues
|
|
411
|
+
|
|
412
|
+
**Prefix Convention**:
|
|
413
|
+
- **Decision**: Use `hazo_files_` prefix for main generation functions
|
|
414
|
+
- **Rationale**:
|
|
415
|
+
- Prevents naming conflicts with user code
|
|
416
|
+
- Makes functions easily discoverable
|
|
417
|
+
- Consistent with package naming
|
|
418
|
+
- Clear these are primary API functions
|
|
419
|
+
|
|
420
|
+
**Use Cases**:
|
|
421
|
+
- Document management systems requiring consistent naming
|
|
422
|
+
- Automated file organization workflows
|
|
423
|
+
- Multi-tenant applications with client-specific naming
|
|
424
|
+
- Date-based archival systems
|
|
425
|
+
- Sequential numbering for versioned documents
|
|
426
|
+
|
|
427
|
+
### Fixed
|
|
428
|
+
|
|
429
|
+
#### NamingRuleConfigurator Drag-and-Drop Issues (2025-12-10)
|
|
430
|
+
|
|
431
|
+
**Fixed drag-and-drop for variables into patterns**:
|
|
432
|
+
- **Problem**: Variables from VariableList could not be dragged into PatternBuilder drop zones
|
|
433
|
+
- **Root Cause**: PatternBuilder had a nested DndContext that blocked drag events from the parent NamingRuleConfigurator
|
|
434
|
+
- **Solution**: Removed nested DndContext from PatternBuilder. Now uses single parent DndContext in NamingRuleConfigurator with SortableContext and useDroppable only in PatternBuilder
|
|
435
|
+
- **Impact**: Variables can now be dragged from the variable list into file and folder pattern areas
|
|
436
|
+
- **Files Changed**: `src/ui/components/naming/PatternBuilder.tsx`, `src/ui/components/naming/NamingRuleConfigurator.tsx`
|
|
437
|
+
|
|
438
|
+
**Why Nested DndContext Fails**: @dnd-kit prevents nested DndContext to avoid event bubbling conflicts. Child contexts block drag events from reaching parent handlers. Always use a single top-level DndContext with droppable/sortable children.
|
|
439
|
+
|
|
440
|
+
**Fixed segment reordering within patterns**:
|
|
441
|
+
- **Problem**: Segments could not be reordered by dragging within the same pattern
|
|
442
|
+
- **Root Cause**: handleDragEnd in NamingRuleConfigurator didn't handle the reordering case, only handled new variable drops
|
|
443
|
+
- **Solution**: Added Case 2 logic to handleDragEnd that detects when dragging between segments in the same pattern and calls reorderFilePattern or reorderFolderPattern with the correct indices
|
|
444
|
+
- **Impact**: Users can now reorder segments by dragging them to new positions within file or folder patterns
|
|
445
|
+
- **Files Changed**: `src/ui/components/naming/NamingRuleConfigurator.tsx`
|
|
446
|
+
|
|
447
|
+
**Improved drag activation sensitivity**:
|
|
448
|
+
- **Change**: Added PointerSensor with 8px activation distance
|
|
449
|
+
- **Rationale**: Prevents accidental drags when clicking on variables or segments
|
|
450
|
+
- **Impact**: More intentional drag interactions, better UX
|
|
451
|
+
- **Files Changed**: `src/ui/components/naming/NamingRuleConfigurator.tsx`
|
|
452
|
+
|
|
453
|
+
**Added visual drag feedback**:
|
|
454
|
+
- **Change**: Implemented DragOverlay showing the dragged variable during drag operation
|
|
455
|
+
- **Rationale**: Provides clear visual feedback of what is being dragged
|
|
456
|
+
- **Impact**: Users can see the variable chip following their cursor during drag
|
|
457
|
+
- **Files Changed**: `src/ui/components/naming/NamingRuleConfigurator.tsx`
|
|
458
|
+
|
|
459
|
+
**Made NamingRuleConfigurator content scrollable**:
|
|
460
|
+
- **Problem**: With many variables or long patterns, content overflowed the container with no way to scroll
|
|
461
|
+
- **Solution**: Wrapped Available Variables, Pattern Builders, and Preview sections in a scrollable container with `flex-1 overflow-y-auto min-h-0`, keeping Action Bar fixed at bottom
|
|
462
|
+
- **Impact**: Component now works well in fixed-height containers, all content is accessible via scroll
|
|
463
|
+
- **Files Changed**: `src/ui/components/naming/NamingRuleConfigurator.tsx`
|
|
464
|
+
|
|
465
|
+
### Planned Features
|
|
466
|
+
|
|
467
|
+
- Amazon S3 storage module
|
|
468
|
+
- Dropbox storage module
|
|
469
|
+
- OneDrive storage module
|
|
470
|
+
- WebDAV support
|
|
471
|
+
- Path caching for Google Drive (performance optimization)
|
|
472
|
+
- Batch operations for Google Drive
|
|
473
|
+
- Virtual scrolling for large file lists
|
|
474
|
+
- File versioning support
|
|
475
|
+
- Sharing and permissions system
|
|
476
|
+
- Advanced search and filtering
|
|
477
|
+
- Thumbnail generation
|
|
478
|
+
- File compression/decompression
|
|
479
|
+
- Trash/recycle bin functionality
|
|
480
|
+
|
|
481
|
+
---
|
|
482
|
+
|
|
483
|
+
**Note**: This changelog follows the principles of [Keep a Changelog](https://keepachangelog.com). Each version should be documented with Added, Changed, Deprecated, Removed, Fixed, and Security sections as applicable.
|
|
484
|
+
|
|
485
|
+
[1.0.0]: https://github.com/pub12/hazo_files/releases/tag/v1.0.0
|
|
486
|
+
[Unreleased]: https://github.com/pub12/hazo_files/compare/v1.0.0...HEAD
|
package/README.md
CHANGED
|
@@ -20,6 +20,7 @@ A powerful, modular file management package for Node.js and React applications w
|
|
|
20
20
|
- **File Change Detection**: xxHash-based content hashing for efficient change detection
|
|
21
21
|
- **Content Tagging**: Optional LLM-based content classification at upload time or on-demand via `content_tag` field
|
|
22
22
|
- **Schema Migrations**: Built-in V2/V3 migration utilities for adding reference tracking and content tagging to existing databases
|
|
23
|
+
- **Background Upload Pipelines**: Framework-agnostic `UploadManager` + React `HazoFileUploadProvider` for multi-step upload pipelines that survive component unmount, with optional sonner toast bridge
|
|
23
24
|
- **TypeScript**: Full type safety and IntelliSense support
|
|
24
25
|
- **OAuth Integration**: Built-in Google Drive and Dropbox OAuth authentication
|
|
25
26
|
- **Prompt Cache Invalidation**: Passthrough for hazo_llm_api prompt cache management via server instance
|
|
@@ -61,6 +62,12 @@ npm install server-only # Server-side safety (recommended)
|
|
|
61
62
|
npm install xxhash-wasm # File change detection (optional)
|
|
62
63
|
```
|
|
63
64
|
|
|
65
|
+
For the background-upload sonner toast bridge (optional):
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
npm install sonner # Toast notifications for background upload pipelines
|
|
69
|
+
```
|
|
70
|
+
|
|
64
71
|
### Tailwind CSS v4 Setup (Required for UI Components)
|
|
65
72
|
|
|
66
73
|
If you're using Tailwind CSS v4 with the UI components, you must add a `@source` directive to your CSS file to ensure Tailwind scans the package's files for utility classes.
|
|
@@ -1601,6 +1608,157 @@ const { fileManager, metadataService, namingService, extractionService, uploadEx
|
|
|
1601
1608
|
invalidatePromptCache?.('classification', 'classify_document');
|
|
1602
1609
|
```
|
|
1603
1610
|
|
|
1611
|
+
## Background Upload Pipelines
|
|
1612
|
+
|
|
1613
|
+
A framework-agnostic upload pipeline engine that survives React component unmount. Useful when uploads include multi-step server work (upload → LLM extract → user confirmation → DB commit) and the user may navigate away mid-flight.
|
|
1614
|
+
|
|
1615
|
+
Two subpath exports:
|
|
1616
|
+
|
|
1617
|
+
- `hazo_files/background-upload` — core, no React dependency (`UploadManager`, `Job`, `PipelineExecutor`, `TypedEventEmitter`, all types)
|
|
1618
|
+
- `hazo_files/background-upload/react` — React bindings (`HazoFileUploadProvider`, `useFileUpload`, `useJobStatus`, `useFileUploadToasts`)
|
|
1619
|
+
|
|
1620
|
+
### Core API (framework-agnostic)
|
|
1621
|
+
|
|
1622
|
+
```typescript
|
|
1623
|
+
import { UploadManager } from 'hazo_files/background-upload';
|
|
1624
|
+
import type { PipelineStep, PipelineContext, JobHandle } from 'hazo_files/background-upload';
|
|
1625
|
+
|
|
1626
|
+
const uploadStep: PipelineStep = {
|
|
1627
|
+
name: 'upload',
|
|
1628
|
+
async execute(ctx: PipelineContext, handle: JobHandle) {
|
|
1629
|
+
handle.set_status('uploading');
|
|
1630
|
+
for (let i = 0; i < ctx.files.length; i++) {
|
|
1631
|
+
// ... POST file to your /api/files/upload route
|
|
1632
|
+
handle.set_progress(i + 1, ctx.files.length);
|
|
1633
|
+
}
|
|
1634
|
+
},
|
|
1635
|
+
};
|
|
1636
|
+
|
|
1637
|
+
const extractStep: PipelineStep = {
|
|
1638
|
+
name: 'extract',
|
|
1639
|
+
async execute(ctx, handle) {
|
|
1640
|
+
handle.set_status('processing');
|
|
1641
|
+
const extracted = await fetch('/api/extract', { /* ... */ }).then(r => r.json());
|
|
1642
|
+
ctx.extracted_data = extracted;
|
|
1643
|
+
},
|
|
1644
|
+
};
|
|
1645
|
+
|
|
1646
|
+
const manager = new UploadManager({
|
|
1647
|
+
max_concurrent: 2,
|
|
1648
|
+
default_pipeline_steps: [uploadStep, extractStep],
|
|
1649
|
+
});
|
|
1650
|
+
|
|
1651
|
+
manager.on('job:completed', ({ job }) => console.log('done', job.job_id));
|
|
1652
|
+
|
|
1653
|
+
const batch_id = manager.submit_batch({
|
|
1654
|
+
files: [file1, file2],
|
|
1655
|
+
group_id: 'project-123',
|
|
1656
|
+
group_label: 'Q4 Tax Documents',
|
|
1657
|
+
});
|
|
1658
|
+
```
|
|
1659
|
+
|
|
1660
|
+
### React Provider + Hooks
|
|
1661
|
+
|
|
1662
|
+
```tsx
|
|
1663
|
+
// app/layout.tsx (Next.js) or your app root
|
|
1664
|
+
'use client';
|
|
1665
|
+
import { HazoFileUploadProvider } from 'hazo_files/background-upload/react';
|
|
1666
|
+
import { Toaster } from 'sonner';
|
|
1667
|
+
|
|
1668
|
+
export default function RootLayout({ children }: { children: React.ReactNode }) {
|
|
1669
|
+
return (
|
|
1670
|
+
<HazoFileUploadProvider config={{ max_concurrent: 2 }}>
|
|
1671
|
+
<Toaster richColors />
|
|
1672
|
+
{children}
|
|
1673
|
+
</HazoFileUploadProvider>
|
|
1674
|
+
);
|
|
1675
|
+
}
|
|
1676
|
+
```
|
|
1677
|
+
|
|
1678
|
+
```tsx
|
|
1679
|
+
'use client';
|
|
1680
|
+
import { useFileUpload, useJobStatus } from 'hazo_files/background-upload/react';
|
|
1681
|
+
|
|
1682
|
+
export function UploadButton() {
|
|
1683
|
+
const { submit_batch, active_jobs } = useFileUpload();
|
|
1684
|
+
|
|
1685
|
+
function onPick(files: FileList) {
|
|
1686
|
+
submit_batch({
|
|
1687
|
+
files: Array.from(files),
|
|
1688
|
+
group_id: 'project-123',
|
|
1689
|
+
group_label: 'Project 123',
|
|
1690
|
+
pipeline_steps: [/* your PipelineStep[] */],
|
|
1691
|
+
});
|
|
1692
|
+
}
|
|
1693
|
+
|
|
1694
|
+
return (
|
|
1695
|
+
<>
|
|
1696
|
+
<input type="file" multiple onChange={(e) => onPick(e.target.files!)} />
|
|
1697
|
+
<ul>
|
|
1698
|
+
{active_jobs.map((j) => (
|
|
1699
|
+
<li key={j.job_id}>{j.group_label}: {j.status}</li>
|
|
1700
|
+
))}
|
|
1701
|
+
</ul>
|
|
1702
|
+
</>
|
|
1703
|
+
);
|
|
1704
|
+
}
|
|
1705
|
+
|
|
1706
|
+
// Track a single job
|
|
1707
|
+
export function JobBadge({ job_id }: { job_id: string }) {
|
|
1708
|
+
const job = useJobStatus(job_id);
|
|
1709
|
+
if (!job) return null;
|
|
1710
|
+
return <span>{job.status} {job.progress && `${job.progress.current}/${job.progress.total}`}</span>;
|
|
1711
|
+
}
|
|
1712
|
+
```
|
|
1713
|
+
|
|
1714
|
+
### Confirmation Steps (user-in-the-loop)
|
|
1715
|
+
|
|
1716
|
+
```typescript
|
|
1717
|
+
const confirmStep: PipelineStep = {
|
|
1718
|
+
name: 'confirm',
|
|
1719
|
+
async execute(ctx, handle) {
|
|
1720
|
+
handle.set_status('awaiting_confirmation');
|
|
1721
|
+
const result = await handle.request_confirmation({
|
|
1722
|
+
conflicts: ctx.extracted_data.conflicts,
|
|
1723
|
+
});
|
|
1724
|
+
if (!result.confirmed) throw new Error('User cancelled');
|
|
1725
|
+
// ctx.extracted_data now reflects user choices via result.data
|
|
1726
|
+
},
|
|
1727
|
+
};
|
|
1728
|
+
```
|
|
1729
|
+
|
|
1730
|
+
In the UI, subscribe to `job:confirmation_needed` (or read jobs in `awaiting_confirmation` status), render a dialog, then:
|
|
1731
|
+
|
|
1732
|
+
```typescript
|
|
1733
|
+
const { resolve_confirmation } = useFileUpload();
|
|
1734
|
+
resolve_confirmation(job_id, { confirmed: true, data: userChoices });
|
|
1735
|
+
```
|
|
1736
|
+
|
|
1737
|
+
### Sonner Toast Bridge
|
|
1738
|
+
|
|
1739
|
+
The provider mounts a `ToastBridge` by default (`enable_toasts={true}`) that uses sonner to notify on `job:completed`, `job:error`, `job:confirmation_needed`, and `batch:completed`. Sonner is a soft optional peer dependency — if it isn't installed, the bridge is a no-op. Set `enable_toasts={false}` on the provider to opt out, or wire `useFileUploadToasts(manager)` yourself for custom toast behavior.
|
|
1740
|
+
|
|
1741
|
+
### Events
|
|
1742
|
+
|
|
1743
|
+
| Event | Payload | Fired when |
|
|
1744
|
+
|-------|---------|------------|
|
|
1745
|
+
| `job:created` | `{ job }` | Job enters the queue |
|
|
1746
|
+
| `job:status_changed` | `{ job, previous_status }` | Status transitions (queued → uploading → processing → ...) |
|
|
1747
|
+
| `job:progress` | `{ job }` | `handle.set_progress` called inside a pipeline step |
|
|
1748
|
+
| `job:completed` | `{ job }` | All pipeline steps finished successfully |
|
|
1749
|
+
| `job:error` | `{ job, error }` | A pipeline step threw |
|
|
1750
|
+
| `job:confirmation_needed` | `{ job, payload }` | `handle.request_confirmation` called |
|
|
1751
|
+
| `job:confirmation_resolved` | `{ job, result }` | `resolve_confirmation` called |
|
|
1752
|
+
| `batch:progress` | `{ batch }` | Any job in the batch settles |
|
|
1753
|
+
| `batch:completed` | `{ batch }` | All jobs in the batch are `done` or `error` |
|
|
1754
|
+
|
|
1755
|
+
### Design Notes
|
|
1756
|
+
|
|
1757
|
+
- **Survives unmount**: `UploadManager` lives on a `useRef`; pipelines run on the manager, not on React state. Navigating away does not abort uploads.
|
|
1758
|
+
- **Single source of truth**: `useFileUpload` / `useJobStatus` subscribe via `useSyncExternalStore` against the manager's event emitter, so multiple components stay consistent.
|
|
1759
|
+
- **Concurrency**: `max_concurrent` controls how many jobs the executor runs in parallel; the rest wait in the FIFO queue.
|
|
1760
|
+
- **No DB writes**: This module is purely an in-memory pipeline runner — your pipeline steps own all server I/O.
|
|
1761
|
+
|
|
1604
1762
|
## API Reference
|
|
1605
1763
|
|
|
1606
1764
|
### FileManager
|