dazzle-filekit 0.2.3__tar.gz → 0.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/PKG-INFO +110 -57
- dazzle_filekit-0.3.0/README.md +318 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit/__init__.py +79 -10
- dazzle_filekit-0.3.0/dazzle_filekit/_fallback.py +158 -0
- dazzle_filekit-0.3.0/dazzle_filekit/content.py +74 -0
- dazzle_filekit-0.3.0/dazzle_filekit/links.py +318 -0
- dazzle_filekit-0.3.0/dazzle_filekit/metadata.py +928 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit/operations.py +373 -227
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit/paths.py +228 -87
- dazzle_filekit-0.3.0/dazzle_filekit/platform/windows.py +149 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit/utils/__init__.py +2 -2
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit/utils/compat.py +88 -89
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit/utils/validation.py +165 -32
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit.egg-info/PKG-INFO +110 -57
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit.egg-info/SOURCES.txt +17 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit.egg-info/requires.txt +5 -3
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/pyproject.toml +22 -3
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/setup.py +1 -1
- dazzle_filekit-0.3.0/tests/test_ads_and_junction_v024.py +187 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/tests/test_compat.py +19 -4
- dazzle_filekit-0.3.0/tests/test_content_and_process.py +90 -0
- dazzle_filekit-0.3.0/tests/test_fallback.py +118 -0
- dazzle_filekit-0.3.0/tests/test_import_stability.py +225 -0
- dazzle_filekit-0.3.0/tests/test_link_primitives.py +252 -0
- dazzle_filekit-0.3.0/tests/test_metadata_v024_roundtrip.py +368 -0
- dazzle_filekit-0.3.0/tests/test_operations_primitives_v024.py +280 -0
- dazzle_filekit-0.3.0/tests/test_paths_platform_simulation.py +270 -0
- dazzle_filekit-0.3.0/tests/test_paths_v024_enrichments.py +290 -0
- dazzle_filekit-0.3.0/tests/test_phase_b_absorb_and_v9.py +90 -0
- dazzle_filekit-0.3.0/tests/test_phase_c_renames.py +127 -0
- dazzle_filekit-0.3.0/tests/test_phase_d_typeddicts.py +75 -0
- dazzle_filekit-0.3.0/tests/test_resolver_ops.py +108 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/tests/test_smoke.py +4 -3
- dazzle_filekit-0.2.3/README.md +0 -266
- dazzle_filekit-0.2.3/dazzle_filekit/platform/windows.py +0 -48
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/LICENSE +0 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit/platform/__init__.py +0 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit/platform/unix.py +0 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit/utils/disk.py +0 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit/utils/logger.py +0 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit/verification.py +0 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit.egg-info/dependency_links.txt +0 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/dazzle_filekit.egg-info/top_level.txt +0 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/setup.cfg +0 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/tests/test_auto_naming.py +0 -0
- {dazzle_filekit-0.2.3 → dazzle_filekit-0.3.0}/tests/test_disk.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: dazzle-filekit
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.3.0
|
|
4
4
|
Summary: Cross-platform file operations toolkit with path handling, verification, and metadata preservation
|
|
5
5
|
Author: Dustin
|
|
6
6
|
Author-email: Dustin Darcy <6962246+djdarcy@users.noreply.github.com>
|
|
@@ -28,8 +28,9 @@ Requires-Python: >=3.6
|
|
|
28
28
|
Description-Content-Type: text/markdown
|
|
29
29
|
License-File: LICENSE
|
|
30
30
|
Requires-Dist: pathlib; python_version < "3.4"
|
|
31
|
-
|
|
32
|
-
Requires-Dist: unctools>=0.
|
|
31
|
+
Requires-Dist: dazzle-lib>=0.2.0
|
|
32
|
+
Requires-Dist: unctools>=0.2.2
|
|
33
|
+
Requires-Dist: pywin32>=305; sys_platform == "win32"
|
|
33
34
|
Provides-Extra: dev
|
|
34
35
|
Requires-Dist: pytest>=7.0.0; extra == "dev"
|
|
35
36
|
Requires-Dist: pytest-cov>=3.0.0; extra == "dev"
|
|
@@ -41,11 +42,13 @@ Dynamic: requires-python
|
|
|
41
42
|
|
|
42
43
|
# dazzle-filekit
|
|
43
44
|
|
|
44
|
-
[](https://github.com/DazzleLib/dazzle-filekit/releases)
|
|
46
|
+
[](https://pypi.org/project/dazzle-filekit/)
|
|
47
|
+
[](https://pepy.tech/projects/dazzle-filekit)
|
|
48
|
+
[](https://www.python.org/downloads/)
|
|
49
|
+
[](https://opensource.org/licenses/MIT)
|
|
48
50
|
[](https://github.com/DazzleLib/dazzle-filekit/discussions)
|
|
51
|
+
[](docs/platform-support.md)
|
|
49
52
|
|
|
50
53
|
> **Cross-platform file operations with path handling, verification, and metadata preservation.**
|
|
51
54
|
|
|
@@ -53,12 +56,18 @@ A Python toolkit for reliable file operations across Windows, Linux, and macOS.
|
|
|
53
56
|
|
|
54
57
|
## Features
|
|
55
58
|
|
|
56
|
-
- **Cross-Platform Paths** - Normalize between Git Bash (`/c/...`), WSL (`/mnt/c/...`), and native Windows/Unix paths
|
|
59
|
+
- **Cross-Platform Paths** - Normalize between Git Bash (`/c/...`), WSL (`/mnt/c/...`), and native Windows/Unix paths via a single canonical `normalize_cross_platform_path(path, *, resolve=False)` entry point
|
|
60
|
+
- **Rich Metadata Preservation** - `dazzle_filekit.metadata` module captures Windows SDDL ACLs (JSON-serializable), NTFS creation time, Unix extended attributes, and attribute flag booleans; restore-on-recovery preserves everything via `pywin32.SetFileTime` for ctime
|
|
57
61
|
- **File Operations** - Copy, move, and manage files with metadata preservation
|
|
62
|
+
- **Atomic Write Primitives** - `atomic_write_text` / `atomic_write_json` use tmp+rename for crash-safe config and manifest writes
|
|
63
|
+
- **Link-Safe Tree Copy** - `copy_tree_preserving_links` wraps `shutil.copytree(symlinks=True)` with documented intent (never traverses junctions on Windows)
|
|
64
|
+
- **NTFS ADS Detection** - `platform.windows.detect_alternate_streams` enumerates alternate data streams via `FindFirstStreamW`; `has_significant_ads` filters out browser Zone.Identifier noise
|
|
65
|
+
- **Correct Junction Detection** - `is_junction` uses `DeviceIoControl(FSCTL_GET_REPARSE_POINT)` to distinguish real junctions (`IO_REPARSE_TAG_MOUNT_POINT`) from directory symlinks
|
|
58
66
|
- **File Verification** - Calculate and verify file hashes (MD5, SHA1, SHA256, SHA512)
|
|
59
67
|
- **Disk Space Checking** - Pre-flight space verification before operations
|
|
60
|
-
- **Platform Support** - Windows, Linux,
|
|
61
|
-
- **UNC Path
|
|
68
|
+
- **Platform Support** - Windows, Linux, and macOS with platform-specific optimizations
|
|
69
|
+
- **UNC Path Detection** - Native `is_unc_path` / `classify_fs_object` helpers; built on [UNCtools](https://github.com/DazzleLib/UNCtools) (a hard dependency as of 0.3.0) for UNC ↔ drive-letter translation and the optional path-variant resolver edge (see [docs/unctools-integration.md](docs/unctools-integration.md))
|
|
70
|
+
- **Link Primitives** - `analyze_link` / `create_junction` / `create_hardlink` / `read_link_target` (intrinsic link analysis; junctions via PowerShell, no `cmd`)
|
|
62
71
|
|
|
63
72
|
## Why dazzle-filekit?
|
|
64
73
|
|
|
@@ -79,14 +88,17 @@ dazzle-filekit was designed for applications requiring reliable file operations
|
|
|
79
88
|
pip install dazzle-filekit
|
|
80
89
|
```
|
|
81
90
|
|
|
91
|
+
This pulls in the stack bedrock and path-identity layer automatically:
|
|
92
|
+
`dazzle-lib` (shared Protocols/types) and `unctools` (UNC ↔ drive-letter
|
|
93
|
+
identity). As of 0.3.0 both are **required** -- `unctools` backs the
|
|
94
|
+
fallback-aware file operations (`copy_file(..., try_path_variants=True)`,
|
|
95
|
+
`open_file`, etc.) via the `PathVariantResolver` seam.
|
|
96
|
+
|
|
82
97
|
### Optional Dependencies
|
|
83
98
|
|
|
84
99
|
```bash
|
|
85
|
-
# Windows UNC path support
|
|
86
|
-
pip install dazzle-filekit[unctools]
|
|
87
|
-
|
|
88
100
|
# Development tools
|
|
89
|
-
pip install dazzle-filekit[dev]
|
|
101
|
+
pip install 'dazzle-filekit[dev]'
|
|
90
102
|
```
|
|
91
103
|
|
|
92
104
|
## Quick Start
|
|
@@ -120,10 +132,11 @@ if path_exists_cross_platform("/c/Users/foo/file.txt"):
|
|
|
120
132
|
### Path Operations
|
|
121
133
|
|
|
122
134
|
```python
|
|
123
|
-
from dazzle_filekit import
|
|
135
|
+
from dazzle_filekit import normalize_cross_platform_path, find_files, is_unc_path
|
|
124
136
|
|
|
125
|
-
# Normalize paths (returns Path object)
|
|
126
|
-
|
|
137
|
+
# Normalize paths (returns Path object). resolve=True follows symlinks;
|
|
138
|
+
# the default resolve=False is link-safe.
|
|
139
|
+
path = normalize_cross_platform_path("/some/path/../file.txt")
|
|
127
140
|
print(path) # PosixPath('/some/file.txt') or WindowsPath('C:/some/file.txt')
|
|
128
141
|
|
|
129
142
|
# Find files with patterns (returns list of path strings)
|
|
@@ -139,14 +152,15 @@ if is_unc_path(r"\\server\share"):
|
|
|
139
152
|
```python
|
|
140
153
|
from dazzle_filekit import copy_file, collect_file_metadata, create_symlink
|
|
141
154
|
|
|
142
|
-
# Copy file with
|
|
143
|
-
success = copy_file("source.txt", "dest.txt",
|
|
155
|
+
# Copy file with attribute preservation (timestamps, permissions, etc.)
|
|
156
|
+
success = copy_file("source.txt", "dest.txt", preserve_attrs=True)
|
|
144
157
|
|
|
145
|
-
# Collect file metadata
|
|
158
|
+
# Collect file metadata (v0.2.4: returns SDDL ACLs on Windows,
|
|
159
|
+
# xattrs on Linux/macOS, ctime, and ISO timestamps alongside the raw floats)
|
|
146
160
|
metadata = collect_file_metadata("file.txt")
|
|
147
|
-
print(f"Size: {metadata['size']}, Modified: {metadata['
|
|
161
|
+
print(f"Size: {metadata['size']}, Modified: {metadata['timestamps']['modified_iso']}")
|
|
148
162
|
|
|
149
|
-
# Create symbolic link (cross-platform)
|
|
163
|
+
# Create symbolic link (cross-platform, with Windows fallbacks)
|
|
150
164
|
success = create_symlink("/path/to/target", "/path/to/link")
|
|
151
165
|
|
|
152
166
|
# Force replace existing link
|
|
@@ -188,51 +202,74 @@ hash_value = calculate_file_hash("file.txt", algorithm="sha256")
|
|
|
188
202
|
is_valid = verify_file_hash("file.txt", expected_hash, algorithm="sha256")
|
|
189
203
|
```
|
|
190
204
|
|
|
191
|
-
|
|
205
|
+
### Atomic Writes (v0.2.4)
|
|
206
|
+
|
|
207
|
+
```python
|
|
208
|
+
from dazzle_filekit import atomic_write_text, atomic_write_json
|
|
209
|
+
|
|
210
|
+
# Atomic text write (tmp + os.replace). Crash mid-write leaves the
|
|
211
|
+
# original file intact; readers see either the old or the new contents.
|
|
212
|
+
atomic_write_text("config.ini", "[section]\nkey=value\n")
|
|
213
|
+
|
|
214
|
+
# Atomic JSON write with sensible defaults. default=str handles
|
|
215
|
+
# datetime, Path, and other non-JSON-native types out of the box.
|
|
216
|
+
atomic_write_json("manifest.json", {
|
|
217
|
+
"version": "1.0",
|
|
218
|
+
"created_at": datetime.datetime.now(),
|
|
219
|
+
"root": Path("/data"),
|
|
220
|
+
})
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
### Rich Metadata (v0.2.4)
|
|
192
224
|
|
|
193
|
-
|
|
225
|
+
```python
|
|
226
|
+
from dazzle_filekit import metadata
|
|
194
227
|
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
228
|
+
# Collect rich metadata. On Windows this captures SDDL ACL strings
|
|
229
|
+
# (JSON-serializable), creation time, file attribute flags, and owner.
|
|
230
|
+
# On Linux/macOS it captures extended attributes (xattrs) as base64.
|
|
231
|
+
md = metadata.collect_file_metadata("important.txt")
|
|
199
232
|
|
|
200
|
-
|
|
233
|
+
# Save it as JSON alongside the file
|
|
234
|
+
import json
|
|
235
|
+
with open("important.txt.meta.json", "w") as f:
|
|
236
|
+
json.dump(metadata.metadata_to_json(md), f, indent=2)
|
|
201
237
|
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
- `split_drive_letter(path)` - Split drive letter from path (Windows)
|
|
205
|
-
- `is_unc_path(path)` - Check if path is UNC format
|
|
206
|
-
- `get_relative_path(path, base)` - Get relative path from base
|
|
207
|
-
- `find_files(directory, patterns, exclude)` - Find files matching patterns
|
|
208
|
-
- `get_path_type(path)` - Detect path type (unc, network, subst, local)
|
|
238
|
+
# Later, restore metadata to a copy (including Windows ctime)
|
|
239
|
+
metadata.apply_file_metadata("restored.txt", md)
|
|
209
240
|
|
|
210
|
-
|
|
241
|
+
# Check if the richer Windows code path is available
|
|
242
|
+
if metadata.is_win32_available():
|
|
243
|
+
print("pywin32 present -- full SDDL/ctime/ADS support")
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
### Link-Safe Tree Copy (v0.2.4)
|
|
211
247
|
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
- `collect_file_metadata(path)` - Collect file metadata
|
|
215
|
-
- `apply_file_metadata(path, metadata)` - Apply metadata to file
|
|
216
|
-
- `create_directory_structure(path)` - Create directory tree
|
|
217
|
-
- `remove_file(path)` - Remove file safely
|
|
218
|
-
- `remove_directory(path, recursive)` - Remove directory
|
|
219
|
-
- `create_symlink(target, link, force)` - Create symbolic link with cross-platform support
|
|
248
|
+
```python
|
|
249
|
+
from dazzle_filekit import copy_tree_preserving_links
|
|
220
250
|
|
|
221
|
-
|
|
251
|
+
# Copies the tree, preserving symlinks and junctions as links (never
|
|
252
|
+
# traversing them). Safe for copying source trees that may contain
|
|
253
|
+
# self-referential junctions on Windows.
|
|
254
|
+
copy_tree_preserving_links("src_tree", "dst_tree", dirs_exist_ok=True)
|
|
255
|
+
```
|
|
222
256
|
|
|
223
|
-
|
|
224
|
-
- `check_disk_space(dest, required, margin)` - Check if space is sufficient
|
|
225
|
-
- `calculate_total_size(paths)` - Calculate total size of files/directories
|
|
226
|
-
- `ensure_disk_space(dest, sources, margin)` - Verify space for copy operation
|
|
257
|
+
## API at a glance
|
|
227
258
|
|
|
228
|
-
|
|
259
|
+
The Quick Start above covers the common 90% of what most users need.
|
|
260
|
+
For the full function-by-function reference, see
|
|
261
|
+
**[docs/api-reference.md](docs/api-reference.md)**.
|
|
229
262
|
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
263
|
+
| Area | Key entry points |
|
|
264
|
+
|------|------------------|
|
|
265
|
+
| **Paths** | `normalize_cross_platform_path(path, *, resolve=False)` (canonical), `resolve_cross_platform_path`, `path_exists_cross_platform`, `is_wsl()` |
|
|
266
|
+
| **File ops** | `copy_file`, `move_file`, `create_symlink`, `copy_tree_preserving_links`, `atomic_write_text`, `atomic_write_json` |
|
|
267
|
+
| **Metadata** | `dazzle_filekit.metadata` -- `collect_file_metadata`, `apply_file_metadata`, `restore_windows_creation_time`, `compare_metadata`, `is_win32_available` |
|
|
268
|
+
| **Platform (Windows)** | `dazzle_filekit.platform.windows` -- `detect_alternate_streams`, `has_significant_ads`, `is_admin` |
|
|
269
|
+
| **Disk space** | `get_disk_usage`, `check_disk_space`, `calculate_total_size`, `ensure_disk_space` |
|
|
270
|
+
| **Verification** | `calculate_file_hash`, `verify_file_hash`, `verify_files_with_manifest`, `compare_directories` |
|
|
271
|
+
| **UNC detection** | `is_unc_path`, `classify_fs_object` (UNCtools is the hard-dep L0 layer for translation + the resolver edge -- see [docs/unctools-integration.md](docs/unctools-integration.md)) |
|
|
272
|
+
| **Link primitives** | `analyze_link`, `detect_link_type`, `read_link_target`, `create_junction`, `create_hardlink` |
|
|
236
273
|
|
|
237
274
|
## Platform Support
|
|
238
275
|
|
|
@@ -274,7 +311,11 @@ pip install -e ".[dev]"
|
|
|
274
311
|
### Run Tests
|
|
275
312
|
|
|
276
313
|
```bash
|
|
314
|
+
# Standard run
|
|
277
315
|
pytest tests/ -v --cov=dazzle_filekit
|
|
316
|
+
|
|
317
|
+
# Cross-platform cross-check (Windows + WSL Ubuntu from one command)
|
|
318
|
+
./scripts/run-cross-platform-tests.sh
|
|
278
319
|
```
|
|
279
320
|
|
|
280
321
|
### Code Formatting
|
|
@@ -284,6 +325,18 @@ black dazzle_filekit tests
|
|
|
284
325
|
flake8 dazzle_filekit tests
|
|
285
326
|
```
|
|
286
327
|
|
|
328
|
+
## Documentation
|
|
329
|
+
|
|
330
|
+
- **[docs/api-reference.md](docs/api-reference.md)** -- full function-by-function reference
|
|
331
|
+
- **[docs/api-stability.md](docs/api-stability.md)** -- locked public API surface and known external callers. Changes to symbols listed here require a migration plan.
|
|
332
|
+
- **[docs/platform-support.md](docs/platform-support.md)** -- platform matrix, test counts, and platform-specific feature breakdown
|
|
333
|
+
- **[docs/preservelib-integration.md](docs/preservelib-integration.md)** -- how preservelib composes with filekit and the layering contract
|
|
334
|
+
- **[docs/unctools-integration.md](docs/unctools-integration.md)** -- UNCtools composition patterns (filekit does UNC detection; unctools does UNC translation; user code composes them)
|
|
335
|
+
- **[BREAKING_CHANGES.md](BREAKING_CHANGES.md)** -- forward-looking log of version-specific breaks and the Phase 4+ migration procedure
|
|
336
|
+
- **[CHANGELOG.md](CHANGELOG.md)** -- version history
|
|
337
|
+
|
|
338
|
+
`tests/test_import_stability.py` is the automated canary that enforces `docs/api-stability.md`. If you rename or remove a locked symbol, that test will fail.
|
|
339
|
+
|
|
287
340
|
## Contributing
|
|
288
341
|
|
|
289
342
|
Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
|
|
@@ -0,0 +1,318 @@
|
|
|
1
|
+
# dazzle-filekit
|
|
2
|
+
|
|
3
|
+
[](https://github.com/DazzleLib/dazzle-filekit/releases)
|
|
4
|
+
[](https://pypi.org/project/dazzle-filekit/)
|
|
5
|
+
[](https://pepy.tech/projects/dazzle-filekit)
|
|
6
|
+
[](https://www.python.org/downloads/)
|
|
7
|
+
[](https://opensource.org/licenses/MIT)
|
|
8
|
+
[](https://github.com/DazzleLib/dazzle-filekit/discussions)
|
|
9
|
+
[](docs/platform-support.md)
|
|
10
|
+
|
|
11
|
+
> **Cross-platform file operations with path handling, verification, and metadata preservation.**
|
|
12
|
+
|
|
13
|
+
A Python toolkit for reliable file operations across Windows, Linux, and macOS. Handles path normalization between Git Bash, WSL, and native formats, file verification with multiple hash algorithms, and metadata-preserving copy/move operations.
|
|
14
|
+
|
|
15
|
+
## Features
|
|
16
|
+
|
|
17
|
+
- **Cross-Platform Paths** - Normalize between Git Bash (`/c/...`), WSL (`/mnt/c/...`), and native Windows/Unix paths via a single canonical `normalize_cross_platform_path(path, *, resolve=False)` entry point
|
|
18
|
+
- **Rich Metadata Preservation** - `dazzle_filekit.metadata` module captures Windows SDDL ACLs (JSON-serializable), NTFS creation time, Unix extended attributes, and attribute flag booleans; restore-on-recovery preserves everything via `pywin32.SetFileTime` for ctime
|
|
19
|
+
- **File Operations** - Copy, move, and manage files with metadata preservation
|
|
20
|
+
- **Atomic Write Primitives** - `atomic_write_text` / `atomic_write_json` use tmp+rename for crash-safe config and manifest writes
|
|
21
|
+
- **Link-Safe Tree Copy** - `copy_tree_preserving_links` wraps `shutil.copytree(symlinks=True)` with documented intent (never traverses junctions on Windows)
|
|
22
|
+
- **NTFS ADS Detection** - `platform.windows.detect_alternate_streams` enumerates alternate data streams via `FindFirstStreamW`; `has_significant_ads` filters out browser Zone.Identifier noise
|
|
23
|
+
- **Correct Junction Detection** - `is_junction` uses `DeviceIoControl(FSCTL_GET_REPARSE_POINT)` to distinguish real junctions (`IO_REPARSE_TAG_MOUNT_POINT`) from directory symlinks
|
|
24
|
+
- **File Verification** - Calculate and verify file hashes (MD5, SHA1, SHA256, SHA512)
|
|
25
|
+
- **Disk Space Checking** - Pre-flight space verification before operations
|
|
26
|
+
- **Platform Support** - Windows, Linux, and macOS with platform-specific optimizations
|
|
27
|
+
- **UNC Path Detection** - Native `is_unc_path` / `classify_fs_object` helpers; built on [UNCtools](https://github.com/DazzleLib/UNCtools) (a hard dependency as of 0.3.0) for UNC ↔ drive-letter translation and the optional path-variant resolver edge (see [docs/unctools-integration.md](docs/unctools-integration.md))
|
|
28
|
+
- **Link Primitives** - `analyze_link` / `create_junction` / `create_hardlink` / `read_link_target` (intrinsic link analysis; junctions via PowerShell, no `cmd`)
|
|
29
|
+
|
|
30
|
+
## Why dazzle-filekit?
|
|
31
|
+
|
|
32
|
+
While Python's standard library (`shutil`, `pathlib`, `os`) provides basic file operations, dazzle-filekit offers:
|
|
33
|
+
|
|
34
|
+
- **Metadata Preservation**: Automatic preservation of timestamps, permissions, and extended attributes across platforms
|
|
35
|
+
- **Hash Verification**: Built-in file verification with multiple hash algorithms (MD5, SHA1, SHA256, SHA512)
|
|
36
|
+
- **Cross-Platform Path Handling**: Unified API for handling Windows UNC paths, network drives, and Unix paths
|
|
37
|
+
- **Batch Operations**: Process entire directory trees with pattern matching and filtering
|
|
38
|
+
- **Safe Operations**: Built-in conflict resolution, unique path generation, and error handling
|
|
39
|
+
- **Directory Comparison**: Compare directory contents and verify file integrity across locations
|
|
40
|
+
|
|
41
|
+
dazzle-filekit was designed for applications requiring reliable file operations with verification, such as backup tools, file synchronization, and data preservation systems (like the [preserve](https://github.com/DazzleTools/preserve) project).
|
|
42
|
+
|
|
43
|
+
## Installation
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
pip install dazzle-filekit
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
This pulls in the stack bedrock and path-identity layer automatically:
|
|
50
|
+
`dazzle-lib` (shared Protocols/types) and `unctools` (UNC ↔ drive-letter
|
|
51
|
+
identity). As of 0.3.0 both are **required** -- `unctools` backs the
|
|
52
|
+
fallback-aware file operations (`copy_file(..., try_path_variants=True)`,
|
|
53
|
+
`open_file`, etc.) via the `PathVariantResolver` seam.
|
|
54
|
+
|
|
55
|
+
### Optional Dependencies
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
# Development tools
|
|
59
|
+
pip install 'dazzle-filekit[dev]'
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
## Quick Start
|
|
63
|
+
|
|
64
|
+
### Cross-Platform Path Handling
|
|
65
|
+
|
|
66
|
+
```python
|
|
67
|
+
from dazzle_filekit import (
|
|
68
|
+
normalize_cross_platform_path,
|
|
69
|
+
resolve_cross_platform_path,
|
|
70
|
+
path_exists_cross_platform,
|
|
71
|
+
)
|
|
72
|
+
|
|
73
|
+
# Convert Git Bash style paths to native format
|
|
74
|
+
# On Windows: /c/Users/foo -> C:\Users\foo
|
|
75
|
+
# On Unix: C:\Users\foo -> /c/Users/foo
|
|
76
|
+
path = normalize_cross_platform_path("/c/Users/foo/file.txt")
|
|
77
|
+
|
|
78
|
+
# Also handles WSL paths: /mnt/c/Users/...
|
|
79
|
+
path = normalize_cross_platform_path("/mnt/c/Users/foo/file.txt")
|
|
80
|
+
|
|
81
|
+
# Resolve with probing: if the normalized path doesn't exist,
|
|
82
|
+
# tries alternate platform formats (WSL, MSYS, Windows)
|
|
83
|
+
path = resolve_cross_platform_path("/mnt/c/Users/foo/file.txt")
|
|
84
|
+
|
|
85
|
+
# Check if a cross-platform path exists (uses resolve internally)
|
|
86
|
+
if path_exists_cross_platform("/c/Users/foo/file.txt"):
|
|
87
|
+
print("File exists!")
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
### Path Operations
|
|
91
|
+
|
|
92
|
+
```python
|
|
93
|
+
from dazzle_filekit import normalize_cross_platform_path, find_files, is_unc_path
|
|
94
|
+
|
|
95
|
+
# Normalize paths (returns Path object). resolve=True follows symlinks;
|
|
96
|
+
# the default resolve=False is link-safe.
|
|
97
|
+
path = normalize_cross_platform_path("/some/path/../file.txt")
|
|
98
|
+
print(path) # PosixPath('/some/file.txt') or WindowsPath('C:/some/file.txt')
|
|
99
|
+
|
|
100
|
+
# Find files with patterns (returns list of path strings)
|
|
101
|
+
files = find_files("/directory", patterns=["*.py", "*.txt"])
|
|
102
|
+
|
|
103
|
+
# Check UNC paths
|
|
104
|
+
if is_unc_path(r"\\server\share"):
|
|
105
|
+
print("This is a UNC path")
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
### File Operations
|
|
109
|
+
|
|
110
|
+
```python
|
|
111
|
+
from dazzle_filekit import copy_file, collect_file_metadata, create_symlink
|
|
112
|
+
|
|
113
|
+
# Copy file with attribute preservation (timestamps, permissions, etc.)
|
|
114
|
+
success = copy_file("source.txt", "dest.txt", preserve_attrs=True)
|
|
115
|
+
|
|
116
|
+
# Collect file metadata (v0.2.4: returns SDDL ACLs on Windows,
|
|
117
|
+
# xattrs on Linux/macOS, ctime, and ISO timestamps alongside the raw floats)
|
|
118
|
+
metadata = collect_file_metadata("file.txt")
|
|
119
|
+
print(f"Size: {metadata['size']}, Modified: {metadata['timestamps']['modified_iso']}")
|
|
120
|
+
|
|
121
|
+
# Create symbolic link (cross-platform, with Windows fallbacks)
|
|
122
|
+
success = create_symlink("/path/to/target", "/path/to/link")
|
|
123
|
+
|
|
124
|
+
# Force replace existing link
|
|
125
|
+
success = create_symlink("/new/target", "/path/to/link", force=True)
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
### Disk Space Checking
|
|
129
|
+
|
|
130
|
+
```python
|
|
131
|
+
from dazzle_filekit import get_disk_usage, check_disk_space, ensure_disk_space
|
|
132
|
+
|
|
133
|
+
# Get disk usage statistics
|
|
134
|
+
usage = get_disk_usage("/path/to/check")
|
|
135
|
+
print(f"Total: {usage.total}, Free: {usage.free}, Used: {usage.used_percent:.1f}%")
|
|
136
|
+
|
|
137
|
+
# Check if space is available for an operation
|
|
138
|
+
has_space, required, available, message = check_disk_space(
|
|
139
|
+
"/destination",
|
|
140
|
+
required_bytes=1_000_000_000, # 1GB
|
|
141
|
+
safety_margin=0.1 # 10% extra margin
|
|
142
|
+
)
|
|
143
|
+
|
|
144
|
+
# Check space for a list of source files
|
|
145
|
+
has_space, message = ensure_disk_space(
|
|
146
|
+
dest_path="/destination",
|
|
147
|
+
source_paths=["/path/to/file1.zip", "/path/to/dir/"]
|
|
148
|
+
)
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
### File Verification
|
|
152
|
+
|
|
153
|
+
```python
|
|
154
|
+
from dazzle_filekit import calculate_file_hash, verify_file_hash
|
|
155
|
+
|
|
156
|
+
# Calculate hash
|
|
157
|
+
hash_value = calculate_file_hash("file.txt", algorithm="sha256")
|
|
158
|
+
|
|
159
|
+
# Verify hash
|
|
160
|
+
is_valid = verify_file_hash("file.txt", expected_hash, algorithm="sha256")
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
### Atomic Writes (v0.2.4)
|
|
164
|
+
|
|
165
|
+
```python
|
|
166
|
+
from dazzle_filekit import atomic_write_text, atomic_write_json
|
|
167
|
+
|
|
168
|
+
# Atomic text write (tmp + os.replace). Crash mid-write leaves the
|
|
169
|
+
# original file intact; readers see either the old or the new contents.
|
|
170
|
+
atomic_write_text("config.ini", "[section]\nkey=value\n")
|
|
171
|
+
|
|
172
|
+
# Atomic JSON write with sensible defaults. default=str handles
|
|
173
|
+
# datetime, Path, and other non-JSON-native types out of the box.
|
|
174
|
+
atomic_write_json("manifest.json", {
|
|
175
|
+
"version": "1.0",
|
|
176
|
+
"created_at": datetime.datetime.now(),
|
|
177
|
+
"root": Path("/data"),
|
|
178
|
+
})
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
### Rich Metadata (v0.2.4)
|
|
182
|
+
|
|
183
|
+
```python
|
|
184
|
+
from dazzle_filekit import metadata
|
|
185
|
+
|
|
186
|
+
# Collect rich metadata. On Windows this captures SDDL ACL strings
|
|
187
|
+
# (JSON-serializable), creation time, file attribute flags, and owner.
|
|
188
|
+
# On Linux/macOS it captures extended attributes (xattrs) as base64.
|
|
189
|
+
md = metadata.collect_file_metadata("important.txt")
|
|
190
|
+
|
|
191
|
+
# Save it as JSON alongside the file
|
|
192
|
+
import json
|
|
193
|
+
with open("important.txt.meta.json", "w") as f:
|
|
194
|
+
json.dump(metadata.metadata_to_json(md), f, indent=2)
|
|
195
|
+
|
|
196
|
+
# Later, restore metadata to a copy (including Windows ctime)
|
|
197
|
+
metadata.apply_file_metadata("restored.txt", md)
|
|
198
|
+
|
|
199
|
+
# Check if the richer Windows code path is available
|
|
200
|
+
if metadata.is_win32_available():
|
|
201
|
+
print("pywin32 present -- full SDDL/ctime/ADS support")
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
### Link-Safe Tree Copy (v0.2.4)
|
|
205
|
+
|
|
206
|
+
```python
|
|
207
|
+
from dazzle_filekit import copy_tree_preserving_links
|
|
208
|
+
|
|
209
|
+
# Copies the tree, preserving symlinks and junctions as links (never
|
|
210
|
+
# traversing them). Safe for copying source trees that may contain
|
|
211
|
+
# self-referential junctions on Windows.
|
|
212
|
+
copy_tree_preserving_links("src_tree", "dst_tree", dirs_exist_ok=True)
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
## API at a glance
|
|
216
|
+
|
|
217
|
+
The Quick Start above covers the common 90% of what most users need.
|
|
218
|
+
For the full function-by-function reference, see
|
|
219
|
+
**[docs/api-reference.md](docs/api-reference.md)**.
|
|
220
|
+
|
|
221
|
+
| Area | Key entry points |
|
|
222
|
+
|------|------------------|
|
|
223
|
+
| **Paths** | `normalize_cross_platform_path(path, *, resolve=False)` (canonical), `resolve_cross_platform_path`, `path_exists_cross_platform`, `is_wsl()` |
|
|
224
|
+
| **File ops** | `copy_file`, `move_file`, `create_symlink`, `copy_tree_preserving_links`, `atomic_write_text`, `atomic_write_json` |
|
|
225
|
+
| **Metadata** | `dazzle_filekit.metadata` -- `collect_file_metadata`, `apply_file_metadata`, `restore_windows_creation_time`, `compare_metadata`, `is_win32_available` |
|
|
226
|
+
| **Platform (Windows)** | `dazzle_filekit.platform.windows` -- `detect_alternate_streams`, `has_significant_ads`, `is_admin` |
|
|
227
|
+
| **Disk space** | `get_disk_usage`, `check_disk_space`, `calculate_total_size`, `ensure_disk_space` |
|
|
228
|
+
| **Verification** | `calculate_file_hash`, `verify_file_hash`, `verify_files_with_manifest`, `compare_directories` |
|
|
229
|
+
| **UNC detection** | `is_unc_path`, `classify_fs_object` (UNCtools is the hard-dep L0 layer for translation + the resolver edge -- see [docs/unctools-integration.md](docs/unctools-integration.md)) |
|
|
230
|
+
| **Link primitives** | `analyze_link`, `detect_link_type`, `read_link_target`, `create_junction`, `create_hardlink` |
|
|
231
|
+
|
|
232
|
+
## Platform Support
|
|
233
|
+
|
|
234
|
+
See [docs/platform-support.md](docs/platform-support.md) for the full platform support matrix and platform-specific features.
|
|
235
|
+
|
|
236
|
+
| Platform | Status |
|
|
237
|
+
|----------|--------|
|
|
238
|
+
| Windows 10/11 | Tested |
|
|
239
|
+
| Linux | Tested |
|
|
240
|
+
| WSL / WSL2 | Tested |
|
|
241
|
+
| macOS | Expected to work |
|
|
242
|
+
| BSD | Expected to work |
|
|
243
|
+
|
|
244
|
+
## Configuration
|
|
245
|
+
|
|
246
|
+
### Logging
|
|
247
|
+
|
|
248
|
+
```python
|
|
249
|
+
from dazzle_filekit import configure_logging, enable_verbose_logging
|
|
250
|
+
import logging
|
|
251
|
+
|
|
252
|
+
# Configure logging level
|
|
253
|
+
configure_logging(level=logging.DEBUG, log_file="dazzle-filekit.log")
|
|
254
|
+
|
|
255
|
+
# Or enable verbose logging
|
|
256
|
+
enable_verbose_logging()
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
## Development
|
|
260
|
+
|
|
261
|
+
### Setup Development Environment
|
|
262
|
+
|
|
263
|
+
```bash
|
|
264
|
+
git clone https://github.com/DazzleLib/dazzle-filekit.git
|
|
265
|
+
cd dazzle-filekit
|
|
266
|
+
pip install -e ".[dev]"
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
### Run Tests
|
|
270
|
+
|
|
271
|
+
```bash
|
|
272
|
+
# Standard run
|
|
273
|
+
pytest tests/ -v --cov=dazzle_filekit
|
|
274
|
+
|
|
275
|
+
# Cross-platform cross-check (Windows + WSL Ubuntu from one command)
|
|
276
|
+
./scripts/run-cross-platform-tests.sh
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
### Code Formatting
|
|
280
|
+
|
|
281
|
+
```bash
|
|
282
|
+
black dazzle_filekit tests
|
|
283
|
+
flake8 dazzle_filekit tests
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
## Documentation
|
|
287
|
+
|
|
288
|
+
- **[docs/api-reference.md](docs/api-reference.md)** -- full function-by-function reference
|
|
289
|
+
- **[docs/api-stability.md](docs/api-stability.md)** -- locked public API surface and known external callers. Changes to symbols listed here require a migration plan.
|
|
290
|
+
- **[docs/platform-support.md](docs/platform-support.md)** -- platform matrix, test counts, and platform-specific feature breakdown
|
|
291
|
+
- **[docs/preservelib-integration.md](docs/preservelib-integration.md)** -- how preservelib composes with filekit and the layering contract
|
|
292
|
+
- **[docs/unctools-integration.md](docs/unctools-integration.md)** -- UNCtools composition patterns (filekit does UNC detection; unctools does UNC translation; user code composes them)
|
|
293
|
+
- **[BREAKING_CHANGES.md](BREAKING_CHANGES.md)** -- forward-looking log of version-specific breaks and the Phase 4+ migration procedure
|
|
294
|
+
- **[CHANGELOG.md](CHANGELOG.md)** -- version history
|
|
295
|
+
|
|
296
|
+
`tests/test_import_stability.py` is the automated canary that enforces `docs/api-stability.md`. If you rename or remove a locked symbol, that test will fail.
|
|
297
|
+
|
|
298
|
+
## Contributing
|
|
299
|
+
|
|
300
|
+
Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
|
|
301
|
+
|
|
302
|
+
Like the project?
|
|
303
|
+
|
|
304
|
+
[](https://www.buymeacoffee.com/djdarcy)
|
|
305
|
+
|
|
306
|
+
## License
|
|
307
|
+
|
|
308
|
+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
309
|
+
|
|
310
|
+
## Part of DazzleLib
|
|
311
|
+
|
|
312
|
+
dazzle-filekit is part of the [DazzleLib](https://github.com/DazzleLib) ecosystem of Python file manipulation tools.
|
|
313
|
+
|
|
314
|
+
### Related Projects
|
|
315
|
+
|
|
316
|
+
- [UNCtools](https://github.com/DazzleLib/UNCtools) - Windows UNC path utilities
|
|
317
|
+
- [preserve](https://github.com/DazzleTools/preserve) - File preservation with manifest tracking and restoration
|
|
318
|
+
- [dazzle-tree-lib](https://github.com/DazzleLib/dazzle-tree-lib) - Tree structure utilities
|