hashio 0.3.0__tar.gz → 0.3.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {hashio-0.3.0 → hashio-0.3.2}/PKG-INFO +52 -3
- {hashio-0.3.0 → hashio-0.3.2}/README.md +51 -2
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio/__init__.py +1 -1
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio/cli.py +50 -34
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio/encoder.py +38 -1
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio/worker.py +29 -9
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio.egg-info/PKG-INFO +52 -3
- {hashio-0.3.0 → hashio-0.3.2}/setup.py +1 -1
- {hashio-0.3.0 → hashio-0.3.2}/LICENSE +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio/config.py +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio/exporter.py +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio/logger.py +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio/utils.py +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio.egg-info/SOURCES.txt +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio.egg-info/dependency_links.txt +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio.egg-info/entry_points.txt +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio.egg-info/not-zip-safe +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio.egg-info/requires.txt +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/lib/hashio.egg-info/top_level.txt +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/setup.cfg +0 -0
- {hashio-0.3.0 → hashio-0.3.2}/tests/test_hashio.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: hashio
|
|
3
|
-
Version: 0.3.
|
|
3
|
+
Version: 0.3.2
|
|
4
4
|
Summary: Custom file and directory checksum tool
|
|
5
5
|
Home-page: http://github.com/rsgalloway/hashio
|
|
6
6
|
Author: Ryan Galloway
|
|
@@ -33,6 +33,7 @@ Custom file and directory checksum and verification tool.
|
|
|
33
33
|
- multiple hash algos: c4, crc32, md5, sha256, sha512, xxh64
|
|
34
34
|
- recursively runs checksums on files in directory trees
|
|
35
35
|
- ignores predefined file name patterns
|
|
36
|
+
- collects important file stat metadata
|
|
36
37
|
- caches results for better performance
|
|
37
38
|
|
|
38
39
|
## Installation
|
|
@@ -59,6 +60,18 @@ results to a hash.json file:
|
|
|
59
60
|
$ hashio <DIR>
|
|
60
61
|
```
|
|
61
62
|
|
|
63
|
+
#### Quick usage with uvx
|
|
64
|
+
|
|
65
|
+
You can run `hashio` instantly using uvx:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
$ uvx hashio <DIR>
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
This downloads and runs `hashio` in a temporary, isolated environment — no
|
|
72
|
+
installs, no virtualenvs, no cleanup needed. Perfect for quick hash verification
|
|
73
|
+
tasks on large directories.
|
|
74
|
+
|
|
62
75
|
#### Ignorable files
|
|
63
76
|
|
|
64
77
|
Note that files matching patterns defined in `config.IGNORABLE` will be skipped,
|
|
@@ -80,10 +93,17 @@ $ hashio --verify hash.json
|
|
|
80
93
|
|
|
81
94
|
#### Portability
|
|
82
95
|
|
|
83
|
-
To make a portable hash file, use
|
|
96
|
+
To make a portable hash file, use `-or` to make the paths relative to the
|
|
97
|
+
`hash.json` file :
|
|
84
98
|
|
|
85
99
|
```bash
|
|
86
|
-
$ hashio <DIR>
|
|
100
|
+
$ hashio <DIR> -or hash.json
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
or use `--start` to make them relative to the `<START>` value
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
$ hashio <DIR> -o hash.json --start <START>
|
|
87
107
|
```
|
|
88
108
|
|
|
89
109
|
To verify the data in the hash file, run hashio from the parent dir of the data,
|
|
@@ -115,6 +135,35 @@ $ vi debug.env # make edits
|
|
|
115
135
|
$ ./debug.env -- hashio
|
|
116
136
|
```
|
|
117
137
|
|
|
138
|
+
## Metadata
|
|
139
|
+
|
|
140
|
+
By default `hashio` collects the following file metadata:
|
|
141
|
+
|
|
142
|
+
| Key | Value |
|
|
143
|
+
|-------|-------------|
|
|
144
|
+
| name | file name |
|
|
145
|
+
| atime | file access time (st_atime) |
|
|
146
|
+
| ctime | file creattion time (st_ctime) |
|
|
147
|
+
| mtime | file modify time (st_mtime) |
|
|
148
|
+
| ino | file inode (st_ino) |
|
|
149
|
+
| dev | filesystem device (st_dev) |
|
|
150
|
+
| size | file size in bytes |
|
|
151
|
+
| type | path type - (f)ile or (d)irectory |
|
|
152
|
+
|
|
153
|
+
To walk a directory and collect metadata without peforming file checksums, use
|
|
154
|
+
the "null" hash algo:
|
|
155
|
+
|
|
156
|
+
```bash
|
|
157
|
+
$ hashio <DIR> -a null
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
To make "null" the default, update `${HASHIO_ALGO}` in the environment or the
|
|
161
|
+
`hashio.env` file.
|
|
162
|
+
|
|
163
|
+
```bash
|
|
164
|
+
$ export HASHIO_ALGO=null
|
|
165
|
+
```
|
|
166
|
+
|
|
118
167
|
## Python API
|
|
119
168
|
|
|
120
169
|
Generate a `hash.json` file for a given path (Default is the current working
|
|
@@ -8,6 +8,7 @@ Custom file and directory checksum and verification tool.
|
|
|
8
8
|
- multiple hash algos: c4, crc32, md5, sha256, sha512, xxh64
|
|
9
9
|
- recursively runs checksums on files in directory trees
|
|
10
10
|
- ignores predefined file name patterns
|
|
11
|
+
- collects important file stat metadata
|
|
11
12
|
- caches results for better performance
|
|
12
13
|
|
|
13
14
|
## Installation
|
|
@@ -34,6 +35,18 @@ results to a hash.json file:
|
|
|
34
35
|
$ hashio <DIR>
|
|
35
36
|
```
|
|
36
37
|
|
|
38
|
+
#### Quick usage with uvx
|
|
39
|
+
|
|
40
|
+
You can run `hashio` instantly using uvx:
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
$ uvx hashio <DIR>
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
This downloads and runs `hashio` in a temporary, isolated environment — no
|
|
47
|
+
installs, no virtualenvs, no cleanup needed. Perfect for quick hash verification
|
|
48
|
+
tasks on large directories.
|
|
49
|
+
|
|
37
50
|
#### Ignorable files
|
|
38
51
|
|
|
39
52
|
Note that files matching patterns defined in `config.IGNORABLE` will be skipped,
|
|
@@ -55,10 +68,17 @@ $ hashio --verify hash.json
|
|
|
55
68
|
|
|
56
69
|
#### Portability
|
|
57
70
|
|
|
58
|
-
To make a portable hash file, use
|
|
71
|
+
To make a portable hash file, use `-or` to make the paths relative to the
|
|
72
|
+
`hash.json` file :
|
|
59
73
|
|
|
60
74
|
```bash
|
|
61
|
-
$ hashio <DIR>
|
|
75
|
+
$ hashio <DIR> -or hash.json
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
or use `--start` to make them relative to the `<START>` value
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
$ hashio <DIR> -o hash.json --start <START>
|
|
62
82
|
```
|
|
63
83
|
|
|
64
84
|
To verify the data in the hash file, run hashio from the parent dir of the data,
|
|
@@ -90,6 +110,35 @@ $ vi debug.env # make edits
|
|
|
90
110
|
$ ./debug.env -- hashio
|
|
91
111
|
```
|
|
92
112
|
|
|
113
|
+
## Metadata
|
|
114
|
+
|
|
115
|
+
By default `hashio` collects the following file metadata:
|
|
116
|
+
|
|
117
|
+
| Key | Value |
|
|
118
|
+
|-------|-------------|
|
|
119
|
+
| name | file name |
|
|
120
|
+
| atime | file access time (st_atime) |
|
|
121
|
+
| ctime | file creattion time (st_ctime) |
|
|
122
|
+
| mtime | file modify time (st_mtime) |
|
|
123
|
+
| ino | file inode (st_ino) |
|
|
124
|
+
| dev | filesystem device (st_dev) |
|
|
125
|
+
| size | file size in bytes |
|
|
126
|
+
| type | path type - (f)ile or (d)irectory |
|
|
127
|
+
|
|
128
|
+
To walk a directory and collect metadata without peforming file checksums, use
|
|
129
|
+
the "null" hash algo:
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
$ hashio <DIR> -a null
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
To make "null" the default, update `${HASHIO_ALGO}` in the environment or the
|
|
136
|
+
`hashio.env` file.
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
$ export HASHIO_ALGO=null
|
|
140
|
+
```
|
|
141
|
+
|
|
93
142
|
## Python API
|
|
94
143
|
|
|
95
144
|
Generate a `hash.json` file for a given path (Default is the current working
|
|
@@ -34,9 +34,11 @@ Contains command line interface for hashio.
|
|
|
34
34
|
"""
|
|
35
35
|
|
|
36
36
|
import argparse
|
|
37
|
-
import multiprocessing
|
|
38
37
|
import os
|
|
39
38
|
import sys
|
|
39
|
+
import threading
|
|
40
|
+
import time
|
|
41
|
+
from tqdm import tqdm
|
|
40
42
|
|
|
41
43
|
from hashio import __version__, config, utils
|
|
42
44
|
from hashio.encoder import get_encoder_class, verify_caches, verify_checksums
|
|
@@ -44,9 +46,29 @@ from hashio.logger import logger
|
|
|
44
46
|
from hashio.worker import HashWorker
|
|
45
47
|
|
|
46
48
|
|
|
49
|
+
def normalize_args(argv):
|
|
50
|
+
"""Rewrite `-or value` to `-o value -r value`."""
|
|
51
|
+
new_argv = []
|
|
52
|
+
skip = False
|
|
53
|
+
for i, arg in enumerate(argv):
|
|
54
|
+
if skip:
|
|
55
|
+
skip = False
|
|
56
|
+
continue
|
|
57
|
+
if arg == "-or" and i + 1 < len(argv):
|
|
58
|
+
val = argv[i + 1]
|
|
59
|
+
new_argv.extend(["-o", val, "-r", os.path.dirname(val) or "."])
|
|
60
|
+
skip = True
|
|
61
|
+
else:
|
|
62
|
+
new_argv.append(arg)
|
|
63
|
+
return new_argv
|
|
64
|
+
|
|
65
|
+
|
|
47
66
|
def parse_args():
|
|
48
67
|
"""Parse command line arguments."""
|
|
49
68
|
|
|
69
|
+
# normalize arguments to handle -or value as -o value -r value
|
|
70
|
+
argv = normalize_args(sys.argv[1:])
|
|
71
|
+
|
|
50
72
|
parser = argparse.ArgumentParser(
|
|
51
73
|
description=__doc__, formatter_class=argparse.RawTextHelpFormatter
|
|
52
74
|
)
|
|
@@ -67,6 +89,7 @@ def parse_args():
|
|
|
67
89
|
default=config.CACHE_FILENAME,
|
|
68
90
|
)
|
|
69
91
|
parser.add_argument(
|
|
92
|
+
"-p",
|
|
70
93
|
"--procs",
|
|
71
94
|
type=int,
|
|
72
95
|
metavar="PROCS",
|
|
@@ -74,7 +97,7 @@ def parse_args():
|
|
|
74
97
|
default=config.MAX_PROCS,
|
|
75
98
|
)
|
|
76
99
|
parser.add_argument(
|
|
77
|
-
"-
|
|
100
|
+
"-r",
|
|
78
101
|
"--start",
|
|
79
102
|
type=str,
|
|
80
103
|
metavar="START",
|
|
@@ -114,41 +137,33 @@ def parse_args():
|
|
|
114
137
|
help="verify checksums from a previously created hash file",
|
|
115
138
|
)
|
|
116
139
|
|
|
117
|
-
args = parser.parse_args()
|
|
140
|
+
args = parser.parse_args(argv)
|
|
118
141
|
return args
|
|
119
142
|
|
|
120
143
|
|
|
121
|
-
def
|
|
122
|
-
"""
|
|
144
|
+
def start_progress_thread(worker, update_interval=0.2):
|
|
145
|
+
"""Start a thread to monitor worker.progress.value and update tqdm.
|
|
123
146
|
|
|
124
|
-
:param worker: HashWorker instance to monitor
|
|
147
|
+
:param worker: The HashWorker instance to monitor.
|
|
148
|
+
:param update_interval: How often to update the progress bar (in seconds).
|
|
149
|
+
:return: The thread that updates the progress bar.
|
|
125
150
|
"""
|
|
151
|
+
pbar = tqdm(desc="hashing files", unit="file", dynamic_ncols=True)
|
|
126
152
|
|
|
127
|
-
|
|
128
|
-
|
|
153
|
+
def watch():
|
|
154
|
+
while not worker.is_done():
|
|
155
|
+
pbar.n = worker.progress_count()
|
|
156
|
+
pbar.refresh()
|
|
157
|
+
time.sleep(update_interval)
|
|
129
158
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
)
|
|
135
|
-
last = 0
|
|
136
|
-
|
|
137
|
-
while not worker.done.is_set():
|
|
138
|
-
with worker.progress.get_lock():
|
|
139
|
-
current = worker.progress.value
|
|
140
|
-
delta = current - last
|
|
141
|
-
if delta:
|
|
142
|
-
pbar.update(delta)
|
|
143
|
-
last = current
|
|
144
|
-
time.sleep(0.2)
|
|
159
|
+
# final update to catch any remaining progress
|
|
160
|
+
pbar.n = worker.progress_count()
|
|
161
|
+
pbar.refresh()
|
|
162
|
+
pbar.close()
|
|
145
163
|
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
pbar.update(final - last)
|
|
151
|
-
pbar.close()
|
|
164
|
+
thread = threading.Thread(target=watch, daemon=True)
|
|
165
|
+
thread.start()
|
|
166
|
+
return thread
|
|
152
167
|
|
|
153
168
|
|
|
154
169
|
def main():
|
|
@@ -204,15 +219,16 @@ def main():
|
|
|
204
219
|
)
|
|
205
220
|
|
|
206
221
|
try:
|
|
207
|
-
|
|
208
|
-
|
|
222
|
+
if sys.stdout.isatty():
|
|
223
|
+
progress_thread = start_progress_thread(worker)
|
|
209
224
|
worker.run()
|
|
210
|
-
|
|
225
|
+
if sys.stdout.isatty():
|
|
226
|
+
progress_thread.join()
|
|
211
227
|
|
|
212
228
|
except KeyboardInterrupt:
|
|
213
|
-
print("stopping...")
|
|
214
229
|
worker.stop()
|
|
215
|
-
|
|
230
|
+
print("\nstopping...")
|
|
231
|
+
sys.exit(0)
|
|
216
232
|
|
|
217
233
|
finally:
|
|
218
234
|
logger.debug("done in %s seconds", worker.total_time)
|
|
@@ -160,8 +160,35 @@ def checksum_gen(
|
|
|
160
160
|
yield (algo, value, path)
|
|
161
161
|
|
|
162
162
|
|
|
163
|
+
class NullChecksum(object):
|
|
164
|
+
"""A no-op checksum class that does nothing."""
|
|
165
|
+
|
|
166
|
+
def update(self, data: bytes):
|
|
167
|
+
"""No-op update method."""
|
|
168
|
+
return self # for chaining
|
|
169
|
+
|
|
170
|
+
def digest(self):
|
|
171
|
+
"""Return an empty byte string."""
|
|
172
|
+
return b""
|
|
173
|
+
|
|
174
|
+
def hexdigest(self):
|
|
175
|
+
"""Return an empty hex string."""
|
|
176
|
+
return ""
|
|
177
|
+
|
|
178
|
+
def copy(self):
|
|
179
|
+
"""Return a new instance of NullChecksum."""
|
|
180
|
+
return NullChecksum()
|
|
181
|
+
|
|
182
|
+
def reset(self):
|
|
183
|
+
"""No-op reset method."""
|
|
184
|
+
return self
|
|
185
|
+
|
|
186
|
+
|
|
163
187
|
class CRC32(object):
|
|
164
|
-
"""
|
|
188
|
+
"""
|
|
189
|
+
Simple implementation of CRC32 checksum class. Used by the
|
|
190
|
+
CRC32Encoder class.
|
|
191
|
+
"""
|
|
165
192
|
|
|
166
193
|
def __init__(self):
|
|
167
194
|
self._initial = 0
|
|
@@ -221,6 +248,16 @@ class Encoder(object):
|
|
|
221
248
|
self.hash.update(data)
|
|
222
249
|
|
|
223
250
|
|
|
251
|
+
class NullEncoder(Encoder):
|
|
252
|
+
"""Null Encoder class."""
|
|
253
|
+
|
|
254
|
+
name = "null"
|
|
255
|
+
|
|
256
|
+
def __init__(self):
|
|
257
|
+
super(NullEncoder, self).__init__()
|
|
258
|
+
self.hash = NullChecksum()
|
|
259
|
+
|
|
260
|
+
|
|
224
261
|
class MD5Encoder(Encoder):
|
|
225
262
|
"""MD5 Encoder class."""
|
|
226
263
|
|
|
@@ -65,11 +65,11 @@ def writer_process(
|
|
|
65
65
|
:param flush_interval: time interval in seconds to flush data
|
|
66
66
|
:param batch_size: number of items to collect before flushing
|
|
67
67
|
"""
|
|
68
|
+
from queue import Empty
|
|
69
|
+
|
|
68
70
|
buffer = []
|
|
69
71
|
last_flush = time.time()
|
|
70
72
|
|
|
71
|
-
from queue import Empty
|
|
72
|
-
|
|
73
73
|
# ensure the exporter is open
|
|
74
74
|
while True:
|
|
75
75
|
try:
|
|
@@ -79,6 +79,8 @@ def writer_process(
|
|
|
79
79
|
buffer.append(item)
|
|
80
80
|
except Empty:
|
|
81
81
|
pass
|
|
82
|
+
except (KeyboardInterrupt, EOFError):
|
|
83
|
+
break # clean exit
|
|
82
84
|
except Exception as e:
|
|
83
85
|
logger.error("write error: %s", e)
|
|
84
86
|
|
|
@@ -98,7 +100,6 @@ class HashWorker:
|
|
|
98
100
|
|
|
99
101
|
>>> w = HashWorker(path, outfile="hash.json")
|
|
100
102
|
>>> w.run()
|
|
101
|
-
>>> pprint(w.results)
|
|
102
103
|
"""
|
|
103
104
|
|
|
104
105
|
def __init__(
|
|
@@ -122,6 +123,7 @@ class HashWorker:
|
|
|
122
123
|
:param verbose: if True, print verbose output
|
|
123
124
|
"""
|
|
124
125
|
self.path = path
|
|
126
|
+
self.algo = algo
|
|
125
127
|
self.outfile = outfile
|
|
126
128
|
self.procs = procs
|
|
127
129
|
self.force = force
|
|
@@ -132,7 +134,6 @@ class HashWorker:
|
|
|
132
134
|
self.verbose = verbose
|
|
133
135
|
self.progress = Value("i", 0) # shared int for progress
|
|
134
136
|
self.start = start or os.path.relpath(path)
|
|
135
|
-
self.encoder = get_encoder_class(algo)()
|
|
136
137
|
self.exporter = get_exporter_class(os.path.splitext(outfile)[1])(outfile)
|
|
137
138
|
self.queue = Queue() # task queue
|
|
138
139
|
self.result_queue = Queue() # write queue
|
|
@@ -185,14 +186,17 @@ class HashWorker:
|
|
|
185
186
|
|
|
186
187
|
:param path: file path
|
|
187
188
|
"""
|
|
188
|
-
|
|
189
|
+
encoder = get_encoder_class(self.algo)()
|
|
190
|
+
value = checksum_file(path, encoder)
|
|
189
191
|
|
|
190
192
|
# normalize path to be relative to the start directory
|
|
191
193
|
npath = normalize_path(path, start=self.start)
|
|
192
194
|
|
|
193
195
|
# get metadata for the file
|
|
194
196
|
metadata = get_metadata(path)
|
|
195
|
-
|
|
197
|
+
|
|
198
|
+
# add the checksum value to metadata
|
|
199
|
+
metadata.update({self.algo: value})
|
|
196
200
|
|
|
197
201
|
# print progress to stdout
|
|
198
202
|
if self.verbose:
|
|
@@ -229,11 +233,25 @@ class HashWorker:
|
|
|
229
233
|
self.done.set()
|
|
230
234
|
|
|
231
235
|
def stop(self):
|
|
232
|
-
"""Stops worker."""
|
|
233
|
-
|
|
234
|
-
|
|
236
|
+
"""Stops the worker and cleans up resources."""
|
|
237
|
+
if self.pool:
|
|
238
|
+
self.pool.terminate()
|
|
239
|
+
self.pool.join()
|
|
240
|
+
if self.writer:
|
|
241
|
+
self.writer.terminate()
|
|
242
|
+
self.writer.join()
|
|
235
243
|
self.exporter.close()
|
|
236
244
|
self.total_time = time.time() - self.start_time
|
|
245
|
+
self.done.set()
|
|
246
|
+
logger.debug("stopping %s", multiprocessing.current_process())
|
|
247
|
+
|
|
248
|
+
def progress_count(self):
|
|
249
|
+
"""Returns the current progress count."""
|
|
250
|
+
return self.progress.value
|
|
251
|
+
|
|
252
|
+
def is_done(self):
|
|
253
|
+
"""Checks if the worker has completed its tasks."""
|
|
254
|
+
return self.done.is_set()
|
|
237
255
|
|
|
238
256
|
@staticmethod
|
|
239
257
|
def main(worker):
|
|
@@ -251,6 +269,8 @@ class HashWorker:
|
|
|
251
269
|
worker.do_hash(path)
|
|
252
270
|
except queue.Empty:
|
|
253
271
|
break
|
|
272
|
+
except (KeyboardInterrupt, EOFError):
|
|
273
|
+
break # clean exit
|
|
254
274
|
except Exception as err:
|
|
255
275
|
logger.error(err)
|
|
256
276
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: hashio
|
|
3
|
-
Version: 0.3.
|
|
3
|
+
Version: 0.3.2
|
|
4
4
|
Summary: Custom file and directory checksum tool
|
|
5
5
|
Home-page: http://github.com/rsgalloway/hashio
|
|
6
6
|
Author: Ryan Galloway
|
|
@@ -33,6 +33,7 @@ Custom file and directory checksum and verification tool.
|
|
|
33
33
|
- multiple hash algos: c4, crc32, md5, sha256, sha512, xxh64
|
|
34
34
|
- recursively runs checksums on files in directory trees
|
|
35
35
|
- ignores predefined file name patterns
|
|
36
|
+
- collects important file stat metadata
|
|
36
37
|
- caches results for better performance
|
|
37
38
|
|
|
38
39
|
## Installation
|
|
@@ -59,6 +60,18 @@ results to a hash.json file:
|
|
|
59
60
|
$ hashio <DIR>
|
|
60
61
|
```
|
|
61
62
|
|
|
63
|
+
#### Quick usage with uvx
|
|
64
|
+
|
|
65
|
+
You can run `hashio` instantly using uvx:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
$ uvx hashio <DIR>
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
This downloads and runs `hashio` in a temporary, isolated environment — no
|
|
72
|
+
installs, no virtualenvs, no cleanup needed. Perfect for quick hash verification
|
|
73
|
+
tasks on large directories.
|
|
74
|
+
|
|
62
75
|
#### Ignorable files
|
|
63
76
|
|
|
64
77
|
Note that files matching patterns defined in `config.IGNORABLE` will be skipped,
|
|
@@ -80,10 +93,17 @@ $ hashio --verify hash.json
|
|
|
80
93
|
|
|
81
94
|
#### Portability
|
|
82
95
|
|
|
83
|
-
To make a portable hash file, use
|
|
96
|
+
To make a portable hash file, use `-or` to make the paths relative to the
|
|
97
|
+
`hash.json` file :
|
|
84
98
|
|
|
85
99
|
```bash
|
|
86
|
-
$ hashio <DIR>
|
|
100
|
+
$ hashio <DIR> -or hash.json
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
or use `--start` to make them relative to the `<START>` value
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
$ hashio <DIR> -o hash.json --start <START>
|
|
87
107
|
```
|
|
88
108
|
|
|
89
109
|
To verify the data in the hash file, run hashio from the parent dir of the data,
|
|
@@ -115,6 +135,35 @@ $ vi debug.env # make edits
|
|
|
115
135
|
$ ./debug.env -- hashio
|
|
116
136
|
```
|
|
117
137
|
|
|
138
|
+
## Metadata
|
|
139
|
+
|
|
140
|
+
By default `hashio` collects the following file metadata:
|
|
141
|
+
|
|
142
|
+
| Key | Value |
|
|
143
|
+
|-------|-------------|
|
|
144
|
+
| name | file name |
|
|
145
|
+
| atime | file access time (st_atime) |
|
|
146
|
+
| ctime | file creattion time (st_ctime) |
|
|
147
|
+
| mtime | file modify time (st_mtime) |
|
|
148
|
+
| ino | file inode (st_ino) |
|
|
149
|
+
| dev | filesystem device (st_dev) |
|
|
150
|
+
| size | file size in bytes |
|
|
151
|
+
| type | path type - (f)ile or (d)irectory |
|
|
152
|
+
|
|
153
|
+
To walk a directory and collect metadata without peforming file checksums, use
|
|
154
|
+
the "null" hash algo:
|
|
155
|
+
|
|
156
|
+
```bash
|
|
157
|
+
$ hashio <DIR> -a null
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
To make "null" the default, update `${HASHIO_ALGO}` in the environment or the
|
|
161
|
+
`hashio.env` file.
|
|
162
|
+
|
|
163
|
+
```bash
|
|
164
|
+
$ export HASHIO_ALGO=null
|
|
165
|
+
```
|
|
166
|
+
|
|
118
167
|
## Python API
|
|
119
168
|
|
|
120
169
|
Generate a `hash.json` file for a given path (Default is the current working
|
|
@@ -40,7 +40,7 @@ with open(os.path.join(here, "README.md")) as f:
|
|
|
40
40
|
|
|
41
41
|
setup(
|
|
42
42
|
name="hashio",
|
|
43
|
-
version="0.3.
|
|
43
|
+
version="0.3.2",
|
|
44
44
|
description="Custom file and directory checksum tool",
|
|
45
45
|
long_description=long_description,
|
|
46
46
|
long_description_content_type="text/markdown",
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|