opex-manifest-generator 1.3.0__tar.gz → 1.3.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/PKG-INFO +39 -36
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/README.md +32 -32
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/cli.py +23 -23
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/opex_manifest.py +40 -40
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator.egg-info/PKG-INFO +39 -36
- opex_manifest_generator-1.3.2/opex_manifest_generator.egg-info/requires.txt +7 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/pyproject.toml +11 -4
- opex_manifest_generator-1.3.0/opex_manifest_generator.egg-info/requires.txt +0 -4
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/.github/workflows/codeql.yml +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/.github/workflows/pypi-publish.yml +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/.gitignore +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/LICENSE.md +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/assets/Column Headers.png +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/assets/FullName Column.png +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/assets/Hash Headers.png +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/assets/Identifiers Headers.png +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/assets/XML Headers.png +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/__init__.py +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/common.py +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/hash.py +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/metadata/DublinCore Template.xml +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/metadata/EAD Template.xml +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/metadata/GDPR Template.xml +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/metadata/MODS Template.xml +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/options/options.properties +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator.egg-info/SOURCES.txt +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator.egg-info/dependency_links.txt +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator.egg-info/entry_points.txt +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator.egg-info/top_level.txt +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/Sample File 1.jpg.opex +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/Sample File 2.gif.opex +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/Sample Folder Manifest.opex +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/opex_manifest_generator_AutoClass.xlsx +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/spreads/dctemplate.xlsx +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/spreads/eadtemplate.xlsx +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/spreads/gdprtemplate.xlsx +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/spreads/modstemplate.xlsx +0 -0
- {opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/setup.cfg +0 -0
|
@@ -1,20 +1,23 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: opex_manifest_generator
|
|
3
|
-
Version: 1.3.
|
|
4
|
-
Summary: Opex Manifest Generator
|
|
3
|
+
Version: 1.3.2
|
|
4
|
+
Summary: An Opex Manifest Generator tool for use with OPEX Files, as designed by Preservica
|
|
5
5
|
Author-email: Christopher Prince <c.pj.prince@gmail.com>
|
|
6
|
+
License-Expression: Apache-2.0
|
|
6
7
|
Project-URL: Homepage, https://github.com/CPJPRINCE/opex_manifest_generator
|
|
7
8
|
Project-URL: Issues, https://github.com/CPJPRINCE/opex_manifest_generator/issues
|
|
9
|
+
Keywords: archiving,archives,digital archiving,opex,Preservica,opex generator
|
|
8
10
|
Classifier: Programming Language :: Python :: 3
|
|
9
|
-
Classifier: License :: OSI Approved :: Apache Software License
|
|
10
11
|
Classifier: Operating System :: OS Independent
|
|
11
12
|
Classifier: Topic :: System :: Archiving
|
|
12
13
|
Description-Content-Type: text/markdown
|
|
13
14
|
License-File: LICENSE.md
|
|
14
|
-
Requires-Dist:
|
|
15
|
+
Requires-Dist: auto_reference_generator
|
|
15
16
|
Requires-Dist: pandas
|
|
16
17
|
Requires-Dist: openpyxl
|
|
17
18
|
Requires-Dist: lxml
|
|
19
|
+
Provides-Extra: addex
|
|
20
|
+
Requires-Dist: odfpy; extra == "addex"
|
|
18
21
|
Dynamic: license-file
|
|
19
22
|
|
|
20
23
|
# Opex Manifest Generator Tool
|
|
@@ -41,7 +44,7 @@ There are a number of features including:
|
|
|
41
44
|
- OPEX's can be cleared out, for repeated / ease of use.
|
|
42
45
|
- OPEX's can be zipped with the file, for imports use with Starter/UX2/Manual ingest methods.
|
|
43
46
|
|
|
44
|
-
The Program also makes use of the Auto
|
|
47
|
+
The Program also makes use of the Auto Reference Generator, allowing for:
|
|
45
48
|
- Reference's can be automatically generated and embedded into the Opex, with assignable prefixes.
|
|
46
49
|
- This can be utilised either in Catalog or Accession modes, or both.
|
|
47
50
|
- Clear and log empty folders.
|
|
@@ -50,7 +53,7 @@ The Program also makes use of the Auto Classification Generator, allowing for:
|
|
|
50
53
|
- Sorting!
|
|
51
54
|
- Keyword assignment!
|
|
52
55
|
|
|
53
|
-
A key feature of the program, is that the Auto
|
|
56
|
+
A key feature of the program, is that the Auto Ref spreadsheet can also act as an input, meaning you can utilise the generated spreadsheet to assign metadata to your files and folders. Currently this allows:
|
|
54
57
|
- Assignment of title, description, and security status fields.
|
|
55
58
|
- Assignment of standard and custom xml metadata templates.
|
|
56
59
|
- These fields are all 'drop-in', so only the fields as they are required need to be added.
|
|
@@ -62,7 +65,7 @@ All these features can be combined to create extensive and robust Opex files for
|
|
|
62
65
|
Python Version 3.8+ is recommended; the program is OS independent and works on Windows, MacOS and Linux.
|
|
63
66
|
|
|
64
67
|
The following modules are utilised and installed with the package:
|
|
65
|
-
-
|
|
68
|
+
- auto_reference_generator
|
|
66
69
|
- pandas
|
|
67
70
|
- openpyxl
|
|
68
71
|
- lxml
|
|
@@ -146,23 +149,23 @@ Hidden files and directories can be included by utilising the `--hidden` option.
|
|
|
146
149
|
|
|
147
150
|
## Note on 'meta' folders
|
|
148
151
|
|
|
149
|
-
Meta folders will be generated automatically when used with the `--fixity` and `-rme` options, as well as when some options from the Auto
|
|
152
|
+
Meta folders will be generated automatically when used with the `--fixity` and `-rme` options, as well as when some options from the Auto Reference Generator. You can redirect the path of the generated folder using the `-o` option: `-fx -o {/path/to/meta/output}`. Or you can also disable the generation of 'meta' folder using the `-dmd` option.
|
|
150
153
|
|
|
151
|
-
## Use with the Auto
|
|
154
|
+
## Use with the Auto Reference Generator
|
|
152
155
|
|
|
153
|
-
The Opex Manifest generator becomes much more powerful when utilised with another tool: the Auto
|
|
156
|
+
The Opex Manifest generator becomes much more powerful when utilised with another tool: the Auto Reference Generator, see [here](https://github.com/CPJPRINCE/auto_reference_generator) for further details.
|
|
154
157
|
|
|
155
158
|
This is built-in to the Opex Manifest Generator and can be utilised to embed identifiers and metadata directly to an Opex or through the use of an Excel spreadsheet or CSV file.
|
|
156
159
|
|
|
157
|
-
The Opex Manifest Generator makes use of the
|
|
160
|
+
The Opex Manifest Generator makes use of the auto_reference_generator as a module, therefore it's behaviour differs a little different when compared to utilising the standalone command `auto_ref.exe`.
|
|
158
161
|
|
|
159
162
|
### Identifier Generation
|
|
160
163
|
|
|
161
|
-
To generate an auto
|
|
164
|
+
To generate an auto reference code, call on `-c` option with `catalog` choice. You can also assign a prefix using `-p "ARCH"`:
|
|
162
165
|
|
|
163
166
|
`opex_generate -c catalog -p "ARCH" C:\Users\Christopher\Downloads`
|
|
164
167
|
|
|
165
|
-
This will generate Opex's with an identifier `code` for each of the files / folders. As described in the Auto
|
|
168
|
+
This will generate Opex's with an identifier `code` for each of the files / folders. As described in the Auto Ref module, the reference codes will take the hierarchy of the directories. You can also use the `-s` option to set a starting reference.
|
|
166
169
|
|
|
167
170
|
You can alternatively utilise the "Accession" / running number mode of generating a code using `-c accession` with the prefix "2024". You can also utilise the `--accession-mode` option to determine whether to have a running number for `file, folder, both`.
|
|
168
171
|
|
|
@@ -176,7 +179,7 @@ You can also combine the generic options, like so: `catalog-generic, accession-g
|
|
|
176
179
|
|
|
177
180
|
## Use of Input Override option.
|
|
178
181
|
|
|
179
|
-
This program also supports utilising an Auto
|
|
182
|
+
This program also supports utilising an Auto Ref spreadsheet as an 'input override', utilising the data added into said spreadsheet instead of generating them ad hoc like above.
|
|
180
183
|
|
|
181
184
|
Using this method XIP Metadata fields can be set on Ingest, including:
|
|
182
185
|
|
|
@@ -195,9 +198,9 @@ XML metadata template data, from both the default templates and custom templates
|
|
|
195
198
|
|
|
196
199
|
### XIP metadata - Title, Description and Security Status
|
|
197
200
|
|
|
198
|
-
To use an input override, we need to first create a spreadsheet with the path of. You can utilise the `
|
|
201
|
+
To use an input override, we need to first create a spreadsheet with the path of. You can utilise the `auto_ref` tool installed alongside the Opex Generator, like so:
|
|
199
202
|
|
|
200
|
-
`
|
|
203
|
+
`auto_ref -p "ARCH" "C:\Users\Christopher\Downloads"`
|
|
201
204
|
|
|
202
205
|
In the resultant spreadsheet, add in "Title", "Description", and "Security" as new columns. The column headers are case-sensitive and have to match exactly. These fields would then be filled in with the relevant data.
|
|
203
206
|
|
|
@@ -205,7 +208,7 @@ In the resultant spreadsheet, add in "Title", "Description", and "Security" as n
|
|
|
205
208
|
|
|
206
209
|
Once the cells are filled in with data, run a generation like so: `opex_generate -i "{/path/to/your/spreadsheet.xlsx}" "{/path/to/root/directory}"`
|
|
207
210
|
|
|
208
|
-
Ensure that the root directory matches the original directory of the export. In the above case this would be: `opex_generate -i "C:\Users\Christopher\Downloads\meta\
|
|
211
|
+
Ensure that the root directory matches the original directory of the export. In the above case this would be: `opex_generate -i "C:\Users\Christopher\Downloads\meta\Downloads_AutoRef.xlsx" "C:\Users\Christopher\Downloads"`
|
|
209
212
|
|
|
210
213
|
### Headers Note
|
|
211
214
|
|
|
@@ -223,7 +226,7 @@ Custom Identifiers can be added by adding the columns: `"Archive_Reference", "Ac
|
|
|
223
226
|
|
|
224
227
|

|
|
225
228
|
|
|
226
|
-
`Archive_Reference` or `Identifier` will default to the keyname `code`; `Accession_Reference` will default to `accref`. When using the Auto
|
|
229
|
+
`Archive_Reference` or `Identifier` will default to the keyname `code`; `Accession_Reference` will default to `accref`. When using the Auto Reference Generator it will always generate a column called `Archive_Reference`, but you can simply rename or remove this column as necessary.
|
|
227
230
|
|
|
228
231
|
To add a custom identifier import, do so like: `Identifier:{YourIdentifierName}`, without the curly brackets IE: `Identifier:MyCode`. Multiple identifiers can be added as needed.
|
|
229
232
|
|
|
@@ -231,17 +234,17 @@ No additional parameter's need to be set in the command line when using Identifi
|
|
|
231
234
|
|
|
232
235
|
### XIP Metadata - Hashes
|
|
233
236
|
|
|
234
|
-
If you utilise the Auto
|
|
237
|
+
If you utilise the Auto Reference's tool for generating Hashes; when utilising the `-fx` option in combination with `-i`, if both the columns `Hash` and `Algorithm` are present, the program will read the hashes from the spreadsheet instead of generating them.
|
|
235
238
|
|
|
236
239
|

|
|
237
240
|
|
|
238
|
-
*Be aware that interruption / resuming is not currently supported with the Auto
|
|
241
|
+
*Be aware that interruption / resuming is not currently supported with the Auto Ref Tool; also doesn't support multiple hashes*
|
|
239
242
|
|
|
240
243
|
### XML Metadata - Basic Templates
|
|
241
244
|
|
|
242
245
|
DC, MODS, GPDR, and EAD templates are supported alongside installation of the package.
|
|
243
246
|
|
|
244
|
-
After exporting an Auto
|
|
247
|
+
After exporting an Auto Ref spreadsheet, you can add in additional columns to the spreadsheet and fill it out with data for an import. Like the XIP data, all fields are optional, and can added on a 'drop-in' basis.
|
|
245
248
|
|
|
246
249
|

|
|
247
250
|
|
|
@@ -284,7 +287,7 @@ The default location will be in the installation path of the program, typically
|
|
|
284
287
|
|
|
285
288
|
After the xml is added to that directory, all that's required is to add the matching column headers into your spreadsheet. You can also utilise `--print-xmls` to obtain this.
|
|
286
289
|
|
|
287
|
-
### Additional Information for Auto
|
|
290
|
+
### Additional Information for Auto Reference
|
|
288
291
|
#### SourceID
|
|
289
292
|
|
|
290
293
|
A SourceID can also be set by adding a `SourceID` header. The behaviour of this is not fully tested, likely won't be as I don't really utilise SourceIDs in my work :\).
|
|
@@ -327,7 +330,7 @@ ALGORITHM_FIELD = Algorithm
|
|
|
327
330
|
```
|
|
328
331
|
#### Custom Spreadsheets - Quick Note
|
|
329
332
|
|
|
330
|
-
You technically don't have to utilise the
|
|
333
|
+
You technically don't have to utilise the AutoRef tool at all. Any old spreadsheet will do!
|
|
331
334
|
|
|
332
335
|
The only requirement to use the input override, is the presence of the `FullName` column. With an accurate list of paths.
|
|
333
336
|
|
|
@@ -387,10 +390,10 @@ Options:
|
|
|
387
390
|
-opt --options-file Specify an 'options.properties' file to change set [PATH/TO/FILE]
|
|
388
391
|
presets for column headers for input.
|
|
389
392
|
|
|
390
|
-
Auto
|
|
393
|
+
Auto Reference Options:
|
|
391
394
|
|
|
392
|
-
-
|
|
393
|
-
module to generate an Auto
|
|
395
|
+
-r, --autoref This will utilise the auto_reference_generator [{catalog, accession,both,
|
|
396
|
+
module to generate an Auto Ref spreadsheet. generic, catalog-generic,
|
|
394
397
|
accession-generic,
|
|
395
398
|
There are several options, {catalog} will generate both-generic}]
|
|
396
399
|
a Archival Reference following; {accession}
|
|
@@ -406,13 +409,13 @@ Options:
|
|
|
406
409
|
--accession-mode Sets whether to have the running tally be for {file,folder,both}
|
|
407
410
|
files, folders or both,
|
|
408
411
|
when utilising the Accession option with
|
|
409
|
-
|
|
412
|
+
autoref. Default is file.
|
|
410
413
|
|
|
411
|
-
-p, --prefix Assign a prefix to the Auto
|
|
414
|
+
-p, --prefix Assign a prefix to the Auto Reference, [PREFIX]
|
|
412
415
|
when utilising {both} fill in like:
|
|
413
416
|
"catalog-prefix","accession-prefix".
|
|
414
417
|
|
|
415
|
-
-s --suffix Assign a suffix to the Auto
|
|
418
|
+
-s --suffix Assign a suffix to the Auto Reference [SUFFIX]
|
|
416
419
|
program. By Default only applies to Files
|
|
417
420
|
|
|
418
421
|
--suffix-options Set the Suffix assignment options {apply_to_files, apply_to_folders,
|
|
@@ -422,16 +425,16 @@ Options:
|
|
|
422
425
|
Log will bee exported to 'meta' / output folder
|
|
423
426
|
|
|
424
427
|
-o, --output Set's the output of the 'meta' folder when [PATH/TO/FOLDER]
|
|
425
|
-
utilising
|
|
428
|
+
utilising AutoRef.
|
|
426
429
|
|
|
427
|
-
-s, --start-ref Sets the starting Reference in the Auto
|
|
430
|
+
-s, --start-ref Sets the starting Reference in the Auto Ref [int]
|
|
428
431
|
process.
|
|
429
432
|
|
|
430
|
-
-i --input Set whether to use an Auto
|
|
433
|
+
-i --input Set whether to use an Auto Ref spreadsheet as an [PATH/TO/FILE]
|
|
431
434
|
input. The input needs to be the (relative or
|
|
432
435
|
absolute) path of the spreadsheet.
|
|
433
436
|
|
|
434
|
-
This allows for use of the Auto
|
|
437
|
+
This allows for use of the Auto Ref spreadsheet
|
|
435
438
|
to customise the XIP metadata (and custom xml
|
|
436
439
|
metadata).
|
|
437
440
|
|
|
@@ -467,7 +470,7 @@ Options:
|
|
|
467
470
|
--disable-meta-dir Will disable the creation of the 'meta' folder. [boolean flag]
|
|
468
471
|
Can also be enabled with output.
|
|
469
472
|
|
|
470
|
-
-ex --export Set whether to export any Auto
|
|
473
|
+
-ex --export Set whether to export any Auto Ref generation [boolean flag]
|
|
471
474
|
to a spreadsheet
|
|
472
475
|
|
|
473
476
|
-fmt, --format Set whether to export as a CSV or XLSX file. {csv,xlsx}
|
|
@@ -530,8 +533,8 @@ Options:
|
|
|
530
533
|
|
|
531
534
|
- ~~Customisable Filtering~~ *Added!*
|
|
532
535
|
- ~~Adjust Accession so the different modes can utilised from Opex.~~ *Added!*
|
|
533
|
-
- ~~Add SourceID as option for use with Auto
|
|
534
|
-
- ~~Allow for multiple Identifier's to be added with Auto
|
|
536
|
+
- ~~Add SourceID as option for use with Auto Ref Spreadsheets.~~ *Added!*
|
|
537
|
+
- ~~Allow for multiple Identifier's to be added with Auto Ref Spreadsheets. Currently only 1 or 2 identifiers can be added at a time, under "Archive_Reference" or "Accession_Reference". These are also tied to be either "code" or "accref". An Option needs to be added to allow custom setting of identifier...~~ *Added!*
|
|
535
538
|
- ~~Add an option / make it a default for Metadata XML's to be located in a specified directory rather than in the package.~~ *Added!*
|
|
536
539
|
- Zipping to conform to PAX - Last on the check list; it technically does...
|
|
537
540
|
- In theory, this tool should be compatible with any system that makes use of the OPEX standard... But in theory Communism works, in theory...
|
|
@@ -22,7 +22,7 @@ There are a number of features including:
|
|
|
22
22
|
- OPEX's can be cleared out, for repeated / ease of use.
|
|
23
23
|
- OPEX's can be zipped with the file, for imports use with Starter/UX2/Manual ingest methods.
|
|
24
24
|
|
|
25
|
-
The Program also makes use of the Auto
|
|
25
|
+
The Program also makes use of the Auto Reference Generator, allowing for:
|
|
26
26
|
- Reference's can be automatically generated and embedded into the Opex, with assignable prefixes.
|
|
27
27
|
- This can be utilised either in Catalog or Accession modes, or both.
|
|
28
28
|
- Clear and log empty folders.
|
|
@@ -31,7 +31,7 @@ The Program also makes use of the Auto Classification Generator, allowing for:
|
|
|
31
31
|
- Sorting!
|
|
32
32
|
- Keyword assignment!
|
|
33
33
|
|
|
34
|
-
A key feature of the program, is that the Auto
|
|
34
|
+
A key feature of the program, is that the Auto Ref spreadsheet can also act as an input, meaning you can utilise the generated spreadsheet to assign metadata to your files and folders. Currently this allows:
|
|
35
35
|
- Assignment of title, description, and security status fields.
|
|
36
36
|
- Assignment of standard and custom xml metadata templates.
|
|
37
37
|
- These fields are all 'drop-in', so only the fields as they are required need to be added.
|
|
@@ -43,7 +43,7 @@ All these features can be combined to create extensive and robust Opex files for
|
|
|
43
43
|
Python Version 3.8+ is recommended; the program is OS independent and works on Windows, MacOS and Linux.
|
|
44
44
|
|
|
45
45
|
The following modules are utilised and installed with the package:
|
|
46
|
-
-
|
|
46
|
+
- auto_reference_generator
|
|
47
47
|
- pandas
|
|
48
48
|
- openpyxl
|
|
49
49
|
- lxml
|
|
@@ -127,23 +127,23 @@ Hidden files and directories can be included by utilising the `--hidden` option.
|
|
|
127
127
|
|
|
128
128
|
## Note on 'meta' folders
|
|
129
129
|
|
|
130
|
-
Meta folders will be generated automatically when used with the `--fixity` and `-rme` options, as well as when some options from the Auto
|
|
130
|
+
Meta folders will be generated automatically when used with the `--fixity` and `-rme` options, as well as when some options from the Auto Reference Generator. You can redirect the path of the generated folder using the `-o` option: `-fx -o {/path/to/meta/output}`. Or you can also disable the generation of 'meta' folder using the `-dmd` option.
|
|
131
131
|
|
|
132
|
-
## Use with the Auto
|
|
132
|
+
## Use with the Auto Reference Generator
|
|
133
133
|
|
|
134
|
-
The Opex Manifest generator becomes much more powerful when utilised with another tool: the Auto
|
|
134
|
+
The Opex Manifest generator becomes much more powerful when utilised with another tool: the Auto Reference Generator, see [here](https://github.com/CPJPRINCE/auto_reference_generator) for further details.
|
|
135
135
|
|
|
136
136
|
This is built-in to the Opex Manifest Generator and can be utilised to embed identifiers and metadata directly to an Opex or through the use of an Excel spreadsheet or CSV file.
|
|
137
137
|
|
|
138
|
-
The Opex Manifest Generator makes use of the
|
|
138
|
+
The Opex Manifest Generator makes use of the auto_reference_generator as a module, therefore it's behaviour differs a little different when compared to utilising the standalone command `auto_ref.exe`.
|
|
139
139
|
|
|
140
140
|
### Identifier Generation
|
|
141
141
|
|
|
142
|
-
To generate an auto
|
|
142
|
+
To generate an auto reference code, call on `-c` option with `catalog` choice. You can also assign a prefix using `-p "ARCH"`:
|
|
143
143
|
|
|
144
144
|
`opex_generate -c catalog -p "ARCH" C:\Users\Christopher\Downloads`
|
|
145
145
|
|
|
146
|
-
This will generate Opex's with an identifier `code` for each of the files / folders. As described in the Auto
|
|
146
|
+
This will generate Opex's with an identifier `code` for each of the files / folders. As described in the Auto Ref module, the reference codes will take the hierarchy of the directories. You can also use the `-s` option to set a starting reference.
|
|
147
147
|
|
|
148
148
|
You can alternatively utilise the "Accession" / running number mode of generating a code using `-c accession` with the prefix "2024". You can also utilise the `--accession-mode` option to determine whether to have a running number for `file, folder, both`.
|
|
149
149
|
|
|
@@ -157,7 +157,7 @@ You can also combine the generic options, like so: `catalog-generic, accession-g
|
|
|
157
157
|
|
|
158
158
|
## Use of Input Override option.
|
|
159
159
|
|
|
160
|
-
This program also supports utilising an Auto
|
|
160
|
+
This program also supports utilising an Auto Ref spreadsheet as an 'input override', utilising the data added into said spreadsheet instead of generating them ad hoc like above.
|
|
161
161
|
|
|
162
162
|
Using this method XIP Metadata fields can be set on Ingest, including:
|
|
163
163
|
|
|
@@ -176,9 +176,9 @@ XML metadata template data, from both the default templates and custom templates
|
|
|
176
176
|
|
|
177
177
|
### XIP metadata - Title, Description and Security Status
|
|
178
178
|
|
|
179
|
-
To use an input override, we need to first create a spreadsheet with the path of. You can utilise the `
|
|
179
|
+
To use an input override, we need to first create a spreadsheet with the path of. You can utilise the `auto_ref` tool installed alongside the Opex Generator, like so:
|
|
180
180
|
|
|
181
|
-
`
|
|
181
|
+
`auto_ref -p "ARCH" "C:\Users\Christopher\Downloads"`
|
|
182
182
|
|
|
183
183
|
In the resultant spreadsheet, add in "Title", "Description", and "Security" as new columns. The column headers are case-sensitive and have to match exactly. These fields would then be filled in with the relevant data.
|
|
184
184
|
|
|
@@ -186,7 +186,7 @@ In the resultant spreadsheet, add in "Title", "Description", and "Security" as n
|
|
|
186
186
|
|
|
187
187
|
Once the cells are filled in with data, run a generation like so: `opex_generate -i "{/path/to/your/spreadsheet.xlsx}" "{/path/to/root/directory}"`
|
|
188
188
|
|
|
189
|
-
Ensure that the root directory matches the original directory of the export. In the above case this would be: `opex_generate -i "C:\Users\Christopher\Downloads\meta\
|
|
189
|
+
Ensure that the root directory matches the original directory of the export. In the above case this would be: `opex_generate -i "C:\Users\Christopher\Downloads\meta\Downloads_AutoRef.xlsx" "C:\Users\Christopher\Downloads"`
|
|
190
190
|
|
|
191
191
|
### Headers Note
|
|
192
192
|
|
|
@@ -204,7 +204,7 @@ Custom Identifiers can be added by adding the columns: `"Archive_Reference", "Ac
|
|
|
204
204
|
|
|
205
205
|

|
|
206
206
|
|
|
207
|
-
`Archive_Reference` or `Identifier` will default to the keyname `code`; `Accession_Reference` will default to `accref`. When using the Auto
|
|
207
|
+
`Archive_Reference` or `Identifier` will default to the keyname `code`; `Accession_Reference` will default to `accref`. When using the Auto Reference Generator it will always generate a column called `Archive_Reference`, but you can simply rename or remove this column as necessary.
|
|
208
208
|
|
|
209
209
|
To add a custom identifier import, do so like: `Identifier:{YourIdentifierName}`, without the curly brackets IE: `Identifier:MyCode`. Multiple identifiers can be added as needed.
|
|
210
210
|
|
|
@@ -212,17 +212,17 @@ No additional parameter's need to be set in the command line when using Identifi
|
|
|
212
212
|
|
|
213
213
|
### XIP Metadata - Hashes
|
|
214
214
|
|
|
215
|
-
If you utilise the Auto
|
|
215
|
+
If you utilise the Auto Reference's tool for generating Hashes; when utilising the `-fx` option in combination with `-i`, if both the columns `Hash` and `Algorithm` are present, the program will read the hashes from the spreadsheet instead of generating them.
|
|
216
216
|
|
|
217
217
|

|
|
218
218
|
|
|
219
|
-
*Be aware that interruption / resuming is not currently supported with the Auto
|
|
219
|
+
*Be aware that interruption / resuming is not currently supported with the Auto Ref Tool; also doesn't support multiple hashes*
|
|
220
220
|
|
|
221
221
|
### XML Metadata - Basic Templates
|
|
222
222
|
|
|
223
223
|
DC, MODS, GPDR, and EAD templates are supported alongside installation of the package.
|
|
224
224
|
|
|
225
|
-
After exporting an Auto
|
|
225
|
+
After exporting an Auto Ref spreadsheet, you can add in additional columns to the spreadsheet and fill it out with data for an import. Like the XIP data, all fields are optional, and can added on a 'drop-in' basis.
|
|
226
226
|
|
|
227
227
|

|
|
228
228
|
|
|
@@ -265,7 +265,7 @@ The default location will be in the installation path of the program, typically
|
|
|
265
265
|
|
|
266
266
|
After the xml is added to that directory, all that's required is to add the matching column headers into your spreadsheet. You can also utilise `--print-xmls` to obtain this.
|
|
267
267
|
|
|
268
|
-
### Additional Information for Auto
|
|
268
|
+
### Additional Information for Auto Reference
|
|
269
269
|
#### SourceID
|
|
270
270
|
|
|
271
271
|
A SourceID can also be set by adding a `SourceID` header. The behaviour of this is not fully tested, likely won't be as I don't really utilise SourceIDs in my work :\).
|
|
@@ -308,7 +308,7 @@ ALGORITHM_FIELD = Algorithm
|
|
|
308
308
|
```
|
|
309
309
|
#### Custom Spreadsheets - Quick Note
|
|
310
310
|
|
|
311
|
-
You technically don't have to utilise the
|
|
311
|
+
You technically don't have to utilise the AutoRef tool at all. Any old spreadsheet will do!
|
|
312
312
|
|
|
313
313
|
The only requirement to use the input override, is the presence of the `FullName` column. With an accurate list of paths.
|
|
314
314
|
|
|
@@ -368,10 +368,10 @@ Options:
|
|
|
368
368
|
-opt --options-file Specify an 'options.properties' file to change set [PATH/TO/FILE]
|
|
369
369
|
presets for column headers for input.
|
|
370
370
|
|
|
371
|
-
Auto
|
|
371
|
+
Auto Reference Options:
|
|
372
372
|
|
|
373
|
-
-
|
|
374
|
-
module to generate an Auto
|
|
373
|
+
-r, --autoref This will utilise the auto_reference_generator [{catalog, accession,both,
|
|
374
|
+
module to generate an Auto Ref spreadsheet. generic, catalog-generic,
|
|
375
375
|
accession-generic,
|
|
376
376
|
There are several options, {catalog} will generate both-generic}]
|
|
377
377
|
a Archival Reference following; {accession}
|
|
@@ -387,13 +387,13 @@ Options:
|
|
|
387
387
|
--accession-mode Sets whether to have the running tally be for {file,folder,both}
|
|
388
388
|
files, folders or both,
|
|
389
389
|
when utilising the Accession option with
|
|
390
|
-
|
|
390
|
+
autoref. Default is file.
|
|
391
391
|
|
|
392
|
-
-p, --prefix Assign a prefix to the Auto
|
|
392
|
+
-p, --prefix Assign a prefix to the Auto Reference, [PREFIX]
|
|
393
393
|
when utilising {both} fill in like:
|
|
394
394
|
"catalog-prefix","accession-prefix".
|
|
395
395
|
|
|
396
|
-
-s --suffix Assign a suffix to the Auto
|
|
396
|
+
-s --suffix Assign a suffix to the Auto Reference [SUFFIX]
|
|
397
397
|
program. By Default only applies to Files
|
|
398
398
|
|
|
399
399
|
--suffix-options Set the Suffix assignment options {apply_to_files, apply_to_folders,
|
|
@@ -403,16 +403,16 @@ Options:
|
|
|
403
403
|
Log will bee exported to 'meta' / output folder
|
|
404
404
|
|
|
405
405
|
-o, --output Set's the output of the 'meta' folder when [PATH/TO/FOLDER]
|
|
406
|
-
utilising
|
|
406
|
+
utilising AutoRef.
|
|
407
407
|
|
|
408
|
-
-s, --start-ref Sets the starting Reference in the Auto
|
|
408
|
+
-s, --start-ref Sets the starting Reference in the Auto Ref [int]
|
|
409
409
|
process.
|
|
410
410
|
|
|
411
|
-
-i --input Set whether to use an Auto
|
|
411
|
+
-i --input Set whether to use an Auto Ref spreadsheet as an [PATH/TO/FILE]
|
|
412
412
|
input. The input needs to be the (relative or
|
|
413
413
|
absolute) path of the spreadsheet.
|
|
414
414
|
|
|
415
|
-
This allows for use of the Auto
|
|
415
|
+
This allows for use of the Auto Ref spreadsheet
|
|
416
416
|
to customise the XIP metadata (and custom xml
|
|
417
417
|
metadata).
|
|
418
418
|
|
|
@@ -448,7 +448,7 @@ Options:
|
|
|
448
448
|
--disable-meta-dir Will disable the creation of the 'meta' folder. [boolean flag]
|
|
449
449
|
Can also be enabled with output.
|
|
450
450
|
|
|
451
|
-
-ex --export Set whether to export any Auto
|
|
451
|
+
-ex --export Set whether to export any Auto Ref generation [boolean flag]
|
|
452
452
|
to a spreadsheet
|
|
453
453
|
|
|
454
454
|
-fmt, --format Set whether to export as a CSV or XLSX file. {csv,xlsx}
|
|
@@ -511,8 +511,8 @@ Options:
|
|
|
511
511
|
|
|
512
512
|
- ~~Customisable Filtering~~ *Added!*
|
|
513
513
|
- ~~Adjust Accession so the different modes can utilised from Opex.~~ *Added!*
|
|
514
|
-
- ~~Add SourceID as option for use with Auto
|
|
515
|
-
- ~~Allow for multiple Identifier's to be added with Auto
|
|
514
|
+
- ~~Add SourceID as option for use with Auto Ref Spreadsheets.~~ *Added!*
|
|
515
|
+
- ~~Allow for multiple Identifier's to be added with Auto Ref Spreadsheets. Currently only 1 or 2 identifiers can be added at a time, under "Archive_Reference" or "Accession_Reference". These are also tied to be either "code" or "accref". An Option needs to be added to allow custom setting of identifier...~~ *Added!*
|
|
516
516
|
- ~~Add an option / make it a default for Metadata XML's to be located in a specified directory rather than in the package.~~ *Added!*
|
|
517
517
|
- Zipping to conform to PAX - Last on the check list; it technically does...
|
|
518
518
|
- In theory, this tool should be compatible with any system that makes use of the OPEX standard... But in theory Communism works, in theory...
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/cli.py
RENAMED
|
@@ -31,14 +31,14 @@ def parse_args():
|
|
|
31
31
|
parser.add_argument("--hidden", required = False, action = 'store_true', default = False,
|
|
32
32
|
help="Set whether to include hidden files and folders")
|
|
33
33
|
parser.add_argument("-o", "--output", required = False, nargs = 1,
|
|
34
|
-
help = "Sets the output to send any generated files (Remove Empty, Fixity List,
|
|
34
|
+
help = "Sets the output to send any generated files (Remove Empty, Fixity List, Autoref Export) to. Will not affect creation of a meta dir.")
|
|
35
35
|
parser.add_argument("-clr", "--clear-opex", required = False, action = 'store_true', default = False,
|
|
36
36
|
help = """Clears existing opex files from a directory. If set with no further options will only clear opexes;
|
|
37
37
|
if multiple options are set will clear opexes and then run the program""")
|
|
38
38
|
parser.add_argument("-opt","--options-file", required = False, default=os.path.join(os.path.dirname(__file__),'options','options.properties'),
|
|
39
39
|
help="Specify a custom Options file, changing the set presets for column headers (Title,Description,etc)")
|
|
40
|
-
parser.add_argument("--
|
|
41
|
-
help="Specify a custom Auto
|
|
40
|
+
parser.add_argument("--autoref-options", required = False, default = None,
|
|
41
|
+
help="Specify a custom Auto Reference Options file, changing the set presets for Auto Reference Generator")
|
|
42
42
|
parser.add_argument("--disable-meta-dir", required = False, action = 'store_false',
|
|
43
43
|
help = """Set whether to disable the creation of a 'meta' directory for generated files,
|
|
44
44
|
default behaviour is to always generate this directory""")
|
|
@@ -56,11 +56,11 @@ def parse_args():
|
|
|
56
56
|
parser.add_argument("--print-xmls", required = False, action = "store_true", default = False,
|
|
57
57
|
help="Prints the elements from your xmls to the consoles")
|
|
58
58
|
|
|
59
|
-
# Auto
|
|
60
|
-
parser.add_argument("-
|
|
59
|
+
# Auto Reference Options
|
|
60
|
+
parser.add_argument("-r", "--autoref", required = False,
|
|
61
61
|
choices = ['catalog', 'c', 'accession', 'a', 'both', 'b', 'generic', 'g', 'catalog-generic', 'cg', "accession-generic", "ag", "both-generic", "bg"],
|
|
62
62
|
type = str.lower,
|
|
63
|
-
help="""Toggles whether to utilise the
|
|
63
|
+
help="""Toggles whether to utilise the auto_reference_generator
|
|
64
64
|
to generate an on the fly Reference listing.
|
|
65
65
|
|
|
66
66
|
There are several options, {catalog} will generate
|
|
@@ -72,22 +72,22 @@ def parse_args():
|
|
|
72
72
|
{generic-catalog,generic-accession, generic-both} it will do both simultaneously.
|
|
73
73
|
""")
|
|
74
74
|
parser.add_argument("-p", "--prefix", required = False, nargs = '+',
|
|
75
|
-
help= """Assign a prefix when utilising the --
|
|
75
|
+
help= """Assign a prefix when utilising the --autoref option. Prefix will append any text before all generated text.
|
|
76
76
|
When utilising the {both} option fill in like: [catalog-prefix, accession-prefix] without square brackets.
|
|
77
77
|
""")
|
|
78
78
|
parser.add_argument("-s", "--suffix", required = False, nargs = '?', default = '',
|
|
79
|
-
help= "Assign a suffix when utilising the --
|
|
79
|
+
help= "Assign a suffix when utilising the --autoref option. Suffix will append any text after all generated text.")
|
|
80
80
|
parser.add_argument("--suffix-option", required = False, choices= ['apply_to_files','apply_to_folders','apply_to_both'], default = 'apply_to_files',
|
|
81
|
-
help = "Set whether to apply the suffix to files, folders or both when utilising the --
|
|
81
|
+
help = "Set whether to apply the suffix to files, folders or both when utilising the --autoref option.")
|
|
82
82
|
parser.add_argument("--accession-mode", nargs = '?', required=False, const='file', default=None, choices=["file",'directory','both'],
|
|
83
|
-
help="""Set the mode when utilising the Accession option in
|
|
83
|
+
help="""Set the mode when utilising the Accession option in autoref.
|
|
84
84
|
file - only adds on files, folder - only adds on folders, both - adds on files and folders""")
|
|
85
85
|
parser.add_argument("-str", "--start-ref", required = False, nargs = '?', default = 1,
|
|
86
|
-
help="Set a custom Starting reference for the Auto
|
|
87
|
-
parser.add_argument("-ex", "--export-
|
|
88
|
-
help="Set whether to export the generated
|
|
86
|
+
help="Set a custom Starting reference for the Auto Reference Generator. The generated reference will")
|
|
87
|
+
parser.add_argument("-ex", "--export-autoref", required = False, action = 'store_true', default = False,
|
|
88
|
+
help="Set whether to export the generated references to an AutoRef spreadsheet")
|
|
89
89
|
parser.add_argument("-fmt", "--output-format", required = False, default = "xlsx", choices = ['xlsx', 'csv','json','ods','xml'],
|
|
90
|
-
help="Set whether to export
|
|
90
|
+
help="Set whether to export AutoRef Spreadsheet to: xlsx, csv, json, ods or xml format")
|
|
91
91
|
parser.add_argument("-dlm", "--delimiter", required=False,nargs = '?', type = str, default = '/',
|
|
92
92
|
help="Set a custom delimiter for generated references, default is '/'")
|
|
93
93
|
parser.add_argument("-key","--keywords", nargs = '*', default = None,
|
|
@@ -119,8 +119,8 @@ def run_cli():
|
|
|
119
119
|
else:
|
|
120
120
|
args.output = os.path.abspath(args.output[0])
|
|
121
121
|
print(f'Output path set to: {args.output}')
|
|
122
|
-
if args.input and args.
|
|
123
|
-
print(f'Both Input and Auto
|
|
122
|
+
if args.input and args.autoref:
|
|
123
|
+
print(f'Both Input and Auto ref options have been selected, please use only one...')
|
|
124
124
|
time.sleep(5); raise SystemExit()
|
|
125
125
|
if args.remove and not args.input:
|
|
126
126
|
print('Removal flag has been given without input, please ensure an input file is utilised when using this option.')
|
|
@@ -131,10 +131,10 @@ def run_cli():
|
|
|
131
131
|
if args.print_xmls:
|
|
132
132
|
OpexManifestGenerator(root = args.root, metadata_dir=args.metadata_dir).print_descriptive_xmls()
|
|
133
133
|
acc_prefix = None
|
|
134
|
-
if args.
|
|
134
|
+
if args.autoref in {"accession", "a", "accession-generic", "ag", "both", "b", "both-generic", "bg"} and args.accession_mode is None:
|
|
135
135
|
args.accession_mode = "file"
|
|
136
136
|
if args.prefix:
|
|
137
|
-
if args.
|
|
137
|
+
if args.autoref in {"both", "b", "both-generic", "bg"}:
|
|
138
138
|
if len(args.prefix) < 2 or len(args.prefix) > 2:
|
|
139
139
|
print('"Both" option is selected, please pass only two prefixes: [-p CATALOG_PREFIX ACCESSION_PREFIX]');
|
|
140
140
|
time.sleep(3); raise SystemExit
|
|
@@ -144,16 +144,16 @@ def run_cli():
|
|
|
144
144
|
elif n == 1:
|
|
145
145
|
acc_prefix = str(a)
|
|
146
146
|
print(f"Prefixes are set as: \t Catalog: {args.prefix} \t Acc: {acc_prefix}")
|
|
147
|
-
elif args.
|
|
147
|
+
elif args.autoref in {"accession", "a", "accession-generic", "ag"}:
|
|
148
148
|
for a in args.prefix:
|
|
149
149
|
acc_prefix = str(a)
|
|
150
150
|
print('Prefix is set as: ' + acc_prefix)
|
|
151
|
-
elif args.
|
|
151
|
+
elif args.autoref in {"catalog", "c", "catalog-generic", "cg"}:
|
|
152
152
|
acc_prefix = None
|
|
153
153
|
for a in args.prefix:
|
|
154
154
|
args.prefix = str(a)
|
|
155
155
|
print('Prefix is set as: ' + args.prefix)
|
|
156
|
-
elif args.
|
|
156
|
+
elif args.autoref in {"generic", "g"}:
|
|
157
157
|
pass
|
|
158
158
|
else:
|
|
159
159
|
print('''An invalid option has been selected, please select a valid option:
|
|
@@ -187,7 +187,7 @@ def run_cli():
|
|
|
187
187
|
time.sleep(3)
|
|
188
188
|
OpexManifestGenerator(root = args.root,
|
|
189
189
|
output_path = args.output,
|
|
190
|
-
|
|
190
|
+
autoref_flag = args.autoref,
|
|
191
191
|
prefix = args.prefix,
|
|
192
192
|
accession_mode=args.accession_mode,
|
|
193
193
|
acc_prefix = acc_prefix,
|
|
@@ -198,7 +198,7 @@ def run_cli():
|
|
|
198
198
|
pax_fixity= args.pax_fixity,
|
|
199
199
|
fixity_export_flag = args.fixity_export,
|
|
200
200
|
start_ref = args.start_ref,
|
|
201
|
-
export_flag = args.
|
|
201
|
+
export_flag = args.export_autoref,
|
|
202
202
|
meta_dir_flag = args.disable_meta_dir,
|
|
203
203
|
metadata_flag = args.metadata,
|
|
204
204
|
metadata_dir = args.metadata_dir,
|
|
@@ -10,8 +10,8 @@ license: Apache License 2.0"
|
|
|
10
10
|
import lxml.etree as ET
|
|
11
11
|
import pandas as pd
|
|
12
12
|
import os, time, datetime
|
|
13
|
-
from
|
|
14
|
-
from
|
|
13
|
+
from auto_reference_generator import ReferenceGenerator
|
|
14
|
+
from auto_reference_generator.common import export_list_txt, export_xl, export_csv, export_json, export_ods, export_xml,define_output_file
|
|
15
15
|
from pandas.api.types import is_datetime64_any_dtype
|
|
16
16
|
from opex_manifest_generator.hash import HashGenerator
|
|
17
17
|
from opex_manifest_generator.common import *
|
|
@@ -26,18 +26,18 @@ class OpexManifestGenerator():
|
|
|
26
26
|
:param meta_dir_flag: set whether to generate a 'meta' directory
|
|
27
27
|
:param metadata_flag: set whether to incorporate metadata into opex
|
|
28
28
|
:param metadata_dir: set the metadata directory to pull xml data from
|
|
29
|
-
:param
|
|
29
|
+
:param autoref_flag: set whether to generate an auto reference reference using autoref. Has a number of 'modes' {catalog, accession, both, generic}
|
|
30
30
|
:param prefix: set a prefix to append to generated references
|
|
31
|
-
:param accession_mode: if using accession in
|
|
31
|
+
:param accession_mode: if using accession in autoref_flag set the mode to count {file, folder, both}
|
|
32
32
|
:param acc_prefix: set an accession prefix
|
|
33
33
|
:param start_ref: set to set the starting reference number
|
|
34
34
|
:param algorithm: set whether to generate fixities and the algorithm to use {MD5, SHA-1, SHA-256, SHA-512}
|
|
35
35
|
:param empty_flag: set whether to delete and log empty directories
|
|
36
36
|
:param removal_flag: set whether to enable removals; data must also contain removals column and cell be set to True
|
|
37
37
|
:param clear_opex_flag: set whether clear existing opexes
|
|
38
|
-
:param export_flag: set whether to export the spreadsheet when using
|
|
39
|
-
:param output_format: set output format when using
|
|
40
|
-
:param input: set whether to use an
|
|
38
|
+
:param export_flag: set whether to export the spreadsheet when using autoref
|
|
39
|
+
:param output_format: set output format when using autoref {xlsx, csv,ods,json,lxml}
|
|
40
|
+
:param input: set whether to use an autoref spreadsheet / dataframe to establish data.
|
|
41
41
|
:param zip_flag: set whether to zip files and opexes together
|
|
42
42
|
:param hidden_flag: set to include hidden files/directories
|
|
43
43
|
:param print_xmls_flag: set to print all
|
|
@@ -54,7 +54,7 @@ class OpexManifestGenerator():
|
|
|
54
54
|
meta_dir_flag: bool = True,
|
|
55
55
|
metadata_dir: str = os.path.join(os.path.dirname(os.path.realpath(__file__)), "metadata"),
|
|
56
56
|
metadata_flag: str = 'none',
|
|
57
|
-
|
|
57
|
+
autoref_flag: str = None,
|
|
58
58
|
prefix: str = None,
|
|
59
59
|
suffix: str = None,
|
|
60
60
|
suffix_option: str|None = 'apply_to_files',
|
|
@@ -81,7 +81,7 @@ class OpexManifestGenerator():
|
|
|
81
81
|
keywords_abbreviation_number: int = 3,
|
|
82
82
|
sort_key = lambda x: (os.path.isfile(x), str.casefold(x)),
|
|
83
83
|
delimiter = "/",
|
|
84
|
-
|
|
84
|
+
autoref_options: str|None = None) -> None:
|
|
85
85
|
|
|
86
86
|
self.root = os.path.abspath(root)
|
|
87
87
|
# Base Parameters
|
|
@@ -111,9 +111,9 @@ class OpexManifestGenerator():
|
|
|
111
111
|
self.metadata_dir = metadata_dir
|
|
112
112
|
self.print_xmls_flag = print_xmls_flag
|
|
113
113
|
|
|
114
|
-
# Parameters for Auto
|
|
115
|
-
self.
|
|
116
|
-
self.
|
|
114
|
+
# Parameters for Auto Reference
|
|
115
|
+
self.autoref_flag = autoref_flag
|
|
116
|
+
self.autoref_options = autoref_options
|
|
117
117
|
self.prefix = prefix
|
|
118
118
|
self.suffix = suffix
|
|
119
119
|
self.suffix_option = suffix_option
|
|
@@ -214,8 +214,8 @@ class OpexManifestGenerator():
|
|
|
214
214
|
time.sleep(3)
|
|
215
215
|
|
|
216
216
|
def init_df(self) -> None:
|
|
217
|
-
if self.
|
|
218
|
-
ac =
|
|
217
|
+
if self.autoref_flag:
|
|
218
|
+
ac = ReferenceGenerator(self.root,
|
|
219
219
|
output_path = self.output_path,
|
|
220
220
|
prefix = self.prefix,
|
|
221
221
|
accprefix = self.acc_prefix,
|
|
@@ -234,7 +234,7 @@ class OpexManifestGenerator():
|
|
|
234
234
|
options_file = os.path.join(os.path.dirname(__file__),'options','options.properties')
|
|
235
235
|
)
|
|
236
236
|
self.df = ac.init_dataframe()
|
|
237
|
-
if self.
|
|
237
|
+
if self.autoref_flag in {"accession", "a", "accession-generic", "ag"}:
|
|
238
238
|
self.df = self.df.drop(ARCREF_FIELD, axis=1)
|
|
239
239
|
self.column_headers = self.df.columns.values.tolist()
|
|
240
240
|
self.set_input_flags()
|
|
@@ -277,7 +277,7 @@ class OpexManifestGenerator():
|
|
|
277
277
|
print(f'Cleared Opex: {file_path}')
|
|
278
278
|
|
|
279
279
|
def index_df_lookup(self, path: str) -> pd.Index:
|
|
280
|
-
idx = self.df
|
|
280
|
+
idx = self.df.loc[self.df[INDEX_FIELD ==path], INDEX_FIELD].index
|
|
281
281
|
return idx
|
|
282
282
|
|
|
283
283
|
def xip_df_lookup(self, idx: pd.Index) -> tuple:
|
|
@@ -289,11 +289,11 @@ class OpexManifestGenerator():
|
|
|
289
289
|
pass
|
|
290
290
|
else:
|
|
291
291
|
if self.title_flag:
|
|
292
|
-
title = check_nan(self.df
|
|
292
|
+
title = check_nan(self.df.loc[idx,TITLE_FIELD].item())
|
|
293
293
|
if self.description_flag:
|
|
294
|
-
description = check_nan(self.df
|
|
294
|
+
description = check_nan(self.df.loc[idx,DESCRIPTION_FIELD].item())
|
|
295
295
|
if self.security_flag:
|
|
296
|
-
security = check_nan(self.df
|
|
296
|
+
security = check_nan(self.df.loc[idx,SECURITY_FIELD].item())
|
|
297
297
|
return title,description,security
|
|
298
298
|
except Exception as e:
|
|
299
299
|
print('Error Looking up XIP Metadata')
|
|
@@ -304,7 +304,7 @@ class OpexManifestGenerator():
|
|
|
304
304
|
if idx.empty:
|
|
305
305
|
return False
|
|
306
306
|
else:
|
|
307
|
-
remove = check_nan(self.df
|
|
307
|
+
remove = check_nan(self.df.loc[idx,REMOVAL_FIELD].item())
|
|
308
308
|
if remove is not None:
|
|
309
309
|
return True
|
|
310
310
|
else:
|
|
@@ -318,7 +318,7 @@ class OpexManifestGenerator():
|
|
|
318
318
|
if idx.empty:
|
|
319
319
|
return False
|
|
320
320
|
else:
|
|
321
|
-
ignore = check_nan(self.df
|
|
321
|
+
ignore = check_nan(self.df.loc[idx,IGNORE_FIELD].item())
|
|
322
322
|
return bool(ignore)
|
|
323
323
|
except Exception as e:
|
|
324
324
|
print('Error looking up Ignore')
|
|
@@ -329,7 +329,7 @@ class OpexManifestGenerator():
|
|
|
329
329
|
if idx.empty:
|
|
330
330
|
pass
|
|
331
331
|
else:
|
|
332
|
-
sourceid = check_nan(self.df
|
|
332
|
+
sourceid = check_nan(self.df.loc[idx,SOURCEID_FIELD].item())
|
|
333
333
|
if sourceid:
|
|
334
334
|
source_xml = ET.SubElement(xml_element,f"{{{self.opexns}}}SourceID")
|
|
335
335
|
source_xml.text = str(sourceid)
|
|
@@ -344,8 +344,8 @@ class OpexManifestGenerator():
|
|
|
344
344
|
else:
|
|
345
345
|
for algorithm_type in self.algorithm:
|
|
346
346
|
self.fixity = ET.SubElement(xml_fixities,f"{{{self.opexns}}}Fixity")
|
|
347
|
-
self.hash = self.df
|
|
348
|
-
self.algorithm = self.df
|
|
347
|
+
self.hash = self.df.loc[idx,HASH_FIELD].item()
|
|
348
|
+
self.algorithm = self.df.loc[idx,ALGORITHM_FIELD].item()
|
|
349
349
|
self.fixity.set('type', algorithm_type)
|
|
350
350
|
self.fixity.set('value',self.hash)
|
|
351
351
|
except Exception as e:
|
|
@@ -370,7 +370,7 @@ class OpexManifestGenerator():
|
|
|
370
370
|
key_name = ACCREF_CODE
|
|
371
371
|
else:
|
|
372
372
|
key_name = IDENTIFIER_DEFAULT
|
|
373
|
-
ident = check_nan(self.df
|
|
373
|
+
ident = check_nan(self.df.loc[idx,header].item())
|
|
374
374
|
if ident:
|
|
375
375
|
self.identifier = ET.SubElement(self.identifiers, f"{{{self.opexns}}}Identifier")
|
|
376
376
|
self.identifier.set("type", key_name)
|
|
@@ -431,10 +431,10 @@ class OpexManifestGenerator():
|
|
|
431
431
|
ns = elem_dict.get('Namespace')
|
|
432
432
|
try:
|
|
433
433
|
if self.metadata_flag in {'e', 'exact'}:
|
|
434
|
-
val_series = self.df
|
|
434
|
+
val_series = self.df.loc[idx,path]
|
|
435
435
|
val = check_nan(val_series.item())
|
|
436
436
|
elif self.metadata_flag in {'f', 'flat'}:
|
|
437
|
-
val_series = self.df
|
|
437
|
+
val_series = self.df.loc[idx,name]
|
|
438
438
|
val = check_nan(val_series.item())
|
|
439
439
|
if val is None:
|
|
440
440
|
continue
|
|
@@ -476,7 +476,7 @@ class OpexManifestGenerator():
|
|
|
476
476
|
if security:
|
|
477
477
|
self.securityxml = ET.SubElement(self.properties, f"{{{self.opexns}}}SecurityDescriptor")
|
|
478
478
|
self.securityxml.text = str(security)
|
|
479
|
-
if self.
|
|
479
|
+
if self.autoref_flag not in {"generic", "g"} or self.input:
|
|
480
480
|
self.identifiers = ET.SubElement(self.properties, f"{{{self.opexns}}}Identifiers")
|
|
481
481
|
self.ident_df_lookup(idx)
|
|
482
482
|
if self.properties is None:
|
|
@@ -512,15 +512,15 @@ class OpexManifestGenerator():
|
|
|
512
512
|
print(f"Start time: {self.start_time}")
|
|
513
513
|
if self.clear_opex_flag:
|
|
514
514
|
self.clear_opex()
|
|
515
|
-
if self.
|
|
515
|
+
if self.autoref_flag or self.algorithm or self.input:
|
|
516
516
|
pass
|
|
517
517
|
else:
|
|
518
518
|
print_running_time(self.start_time)
|
|
519
519
|
print('Cleared OPEXES. No additional arguments passed, so ending program.'); time.sleep(3)
|
|
520
520
|
raise SystemExit()
|
|
521
521
|
if self.empty_flag:
|
|
522
|
-
|
|
523
|
-
if not self.
|
|
522
|
+
ReferenceGenerator(self.root, self.output_path, meta_dir_flag = self.meta_dir_flag).remove_empty_directories()
|
|
523
|
+
if not self.autoref_flag in {"g", "generic"}:
|
|
524
524
|
self.init_df()
|
|
525
525
|
self.count = 1
|
|
526
526
|
if not self.metadata_flag in {'none', 'n'}:
|
|
@@ -545,7 +545,7 @@ class OpexDir(OpexManifestGenerator):
|
|
|
545
545
|
else:
|
|
546
546
|
self.folder_path = folder_path
|
|
547
547
|
if any([self.OMG.input,
|
|
548
|
-
self.OMG.
|
|
548
|
+
self.OMG.autoref_flag in {"c","catalog","a","accession","b","both","cg","catalog-generic","ag","accession-generic","bg","both-generic"},
|
|
549
549
|
self.OMG.ignore_flag,
|
|
550
550
|
self.OMG.removal_flag,
|
|
551
551
|
self.OMG.sourceid_flag,
|
|
@@ -553,7 +553,7 @@ class OpexDir(OpexManifestGenerator):
|
|
|
553
553
|
self.OMG.description_flag,
|
|
554
554
|
self.OMG.security_flag]):
|
|
555
555
|
index = self.OMG.index_df_lookup(self.folder_path)
|
|
556
|
-
elif self.OMG.
|
|
556
|
+
elif self.OMG.autoref_flag in {None, "g","generic"}:
|
|
557
557
|
index = None
|
|
558
558
|
else:
|
|
559
559
|
index = None
|
|
@@ -575,7 +575,7 @@ class OpexDir(OpexManifestGenerator):
|
|
|
575
575
|
self.files = ET.SubElement(self.manifest, f"{{{self.opexns}}}Files")
|
|
576
576
|
if self.OMG.title_flag or self.OMG.description_flag or self.OMG.security_flag:
|
|
577
577
|
self.title, self.description, self.security = self.OMG.xip_df_lookup(index)
|
|
578
|
-
elif self.OMG.
|
|
578
|
+
elif self.OMG.autoref_flag in {"generic", "g", "catalog-generic", "cg", "accession-generic", "ag", "both-generic", "bg"}:
|
|
579
579
|
if title is not None:
|
|
580
580
|
self.title = title
|
|
581
581
|
else:
|
|
@@ -608,7 +608,7 @@ class OpexDir(OpexManifestGenerator):
|
|
|
608
608
|
file.set("type", "content")
|
|
609
609
|
file.set("size", str(os.path.getsize(abs_file)))
|
|
610
610
|
file.text = str(rel_file)
|
|
611
|
-
if self.OMG.
|
|
611
|
+
if self.OMG.autoref_flag or self.OMG.input:
|
|
612
612
|
self.OMG.generate_opex_properties(self.xmlroot, index,
|
|
613
613
|
title = self.title,
|
|
614
614
|
description = self.description,
|
|
@@ -710,7 +710,7 @@ class OpexFile(OpexManifestGenerator):
|
|
|
710
710
|
self.file_path = file_path
|
|
711
711
|
if check_opex(self.file_path):
|
|
712
712
|
if any([self.OMG.input,
|
|
713
|
-
self.OMG.
|
|
713
|
+
self.OMG.autoref_flag in {"c","catalog","a","accession","b","both","cg","catalog-generic","ag","accession-generic","bg","both-generic"},
|
|
714
714
|
self.OMG.ignore_flag,
|
|
715
715
|
self.OMG.removal_flag,
|
|
716
716
|
self.OMG.sourceid_flag,
|
|
@@ -718,7 +718,7 @@ class OpexFile(OpexManifestGenerator):
|
|
|
718
718
|
self.OMG.description_flag,
|
|
719
719
|
self.OMG.security_flag]):
|
|
720
720
|
index = self.OMG.index_df_lookup(self.file_path)
|
|
721
|
-
elif self.OMG.
|
|
721
|
+
elif self.OMG.autoref_flag is None or self.OMG.autoref_flag in {"g","generic"}:
|
|
722
722
|
index = None
|
|
723
723
|
else:
|
|
724
724
|
index = None
|
|
@@ -734,7 +734,7 @@ class OpexFile(OpexManifestGenerator):
|
|
|
734
734
|
return
|
|
735
735
|
if self.OMG.title_flag or self.OMG.description_flag or self.OMG.security_flag:
|
|
736
736
|
self.title, self.description, self.security = self.OMG.xip_df_lookup(index)
|
|
737
|
-
elif self.OMG.
|
|
737
|
+
elif self.OMG.autoref_flag in {"generic", "g", "catalog-generic", "cg", "accession-generic", "ag", "both-generic", "bg"}:
|
|
738
738
|
if title is not None:
|
|
739
739
|
self.title = title
|
|
740
740
|
else:
|
|
@@ -751,7 +751,7 @@ class OpexFile(OpexManifestGenerator):
|
|
|
751
751
|
self.title = title
|
|
752
752
|
self.description = description
|
|
753
753
|
self.security = security
|
|
754
|
-
if self.OMG.algorithm or self.OMG.
|
|
754
|
+
if self.OMG.algorithm or self.OMG.autoref_flag or self.OMG.input:
|
|
755
755
|
self.xmlroot = ET.Element(f"{{{self.opexns}}}OPEXMetadata", nsmap={"opex":self.opexns})
|
|
756
756
|
self.transfer = ET.SubElement(self.xmlroot, f"{{{self.opexns}}}Transfer")
|
|
757
757
|
if self.OMG.sourceid_flag:
|
|
@@ -767,7 +767,7 @@ class OpexFile(OpexManifestGenerator):
|
|
|
767
767
|
self.generate_opex_fixity(self.file_path)
|
|
768
768
|
if self.transfer is None:
|
|
769
769
|
self.xmlroot.remove(self.transfer)
|
|
770
|
-
if self.OMG.
|
|
770
|
+
if self.OMG.autoref_flag or self.OMG.input:
|
|
771
771
|
self.OMG.generate_opex_properties(self.xmlroot, index,
|
|
772
772
|
title = self.title,
|
|
773
773
|
description = self.description,
|
|
@@ -1,20 +1,23 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: opex_manifest_generator
|
|
3
|
-
Version: 1.3.
|
|
4
|
-
Summary: Opex Manifest Generator
|
|
3
|
+
Version: 1.3.2
|
|
4
|
+
Summary: An Opex Manifest Generator tool for use with OPEX Files, as designed by Preservica
|
|
5
5
|
Author-email: Christopher Prince <c.pj.prince@gmail.com>
|
|
6
|
+
License-Expression: Apache-2.0
|
|
6
7
|
Project-URL: Homepage, https://github.com/CPJPRINCE/opex_manifest_generator
|
|
7
8
|
Project-URL: Issues, https://github.com/CPJPRINCE/opex_manifest_generator/issues
|
|
9
|
+
Keywords: archiving,archives,digital archiving,opex,Preservica,opex generator
|
|
8
10
|
Classifier: Programming Language :: Python :: 3
|
|
9
|
-
Classifier: License :: OSI Approved :: Apache Software License
|
|
10
11
|
Classifier: Operating System :: OS Independent
|
|
11
12
|
Classifier: Topic :: System :: Archiving
|
|
12
13
|
Description-Content-Type: text/markdown
|
|
13
14
|
License-File: LICENSE.md
|
|
14
|
-
Requires-Dist:
|
|
15
|
+
Requires-Dist: auto_reference_generator
|
|
15
16
|
Requires-Dist: pandas
|
|
16
17
|
Requires-Dist: openpyxl
|
|
17
18
|
Requires-Dist: lxml
|
|
19
|
+
Provides-Extra: addex
|
|
20
|
+
Requires-Dist: odfpy; extra == "addex"
|
|
18
21
|
Dynamic: license-file
|
|
19
22
|
|
|
20
23
|
# Opex Manifest Generator Tool
|
|
@@ -41,7 +44,7 @@ There are a number of features including:
|
|
|
41
44
|
- OPEX's can be cleared out, for repeated / ease of use.
|
|
42
45
|
- OPEX's can be zipped with the file, for imports use with Starter/UX2/Manual ingest methods.
|
|
43
46
|
|
|
44
|
-
The Program also makes use of the Auto
|
|
47
|
+
The Program also makes use of the Auto Reference Generator, allowing for:
|
|
45
48
|
- Reference's can be automatically generated and embedded into the Opex, with assignable prefixes.
|
|
46
49
|
- This can be utilised either in Catalog or Accession modes, or both.
|
|
47
50
|
- Clear and log empty folders.
|
|
@@ -50,7 +53,7 @@ The Program also makes use of the Auto Classification Generator, allowing for:
|
|
|
50
53
|
- Sorting!
|
|
51
54
|
- Keyword assignment!
|
|
52
55
|
|
|
53
|
-
A key feature of the program, is that the Auto
|
|
56
|
+
A key feature of the program, is that the Auto Ref spreadsheet can also act as an input, meaning you can utilise the generated spreadsheet to assign metadata to your files and folders. Currently this allows:
|
|
54
57
|
- Assignment of title, description, and security status fields.
|
|
55
58
|
- Assignment of standard and custom xml metadata templates.
|
|
56
59
|
- These fields are all 'drop-in', so only the fields as they are required need to be added.
|
|
@@ -62,7 +65,7 @@ All these features can be combined to create extensive and robust Opex files for
|
|
|
62
65
|
Python Version 3.8+ is recommended; the program is OS independent and works on Windows, MacOS and Linux.
|
|
63
66
|
|
|
64
67
|
The following modules are utilised and installed with the package:
|
|
65
|
-
-
|
|
68
|
+
- auto_reference_generator
|
|
66
69
|
- pandas
|
|
67
70
|
- openpyxl
|
|
68
71
|
- lxml
|
|
@@ -146,23 +149,23 @@ Hidden files and directories can be included by utilising the `--hidden` option.
|
|
|
146
149
|
|
|
147
150
|
## Note on 'meta' folders
|
|
148
151
|
|
|
149
|
-
Meta folders will be generated automatically when used with the `--fixity` and `-rme` options, as well as when some options from the Auto
|
|
152
|
+
Meta folders will be generated automatically when used with the `--fixity` and `-rme` options, as well as when some options from the Auto Reference Generator. You can redirect the path of the generated folder using the `-o` option: `-fx -o {/path/to/meta/output}`. Or you can also disable the generation of 'meta' folder using the `-dmd` option.
|
|
150
153
|
|
|
151
|
-
## Use with the Auto
|
|
154
|
+
## Use with the Auto Reference Generator
|
|
152
155
|
|
|
153
|
-
The Opex Manifest generator becomes much more powerful when utilised with another tool: the Auto
|
|
156
|
+
The Opex Manifest generator becomes much more powerful when utilised with another tool: the Auto Reference Generator, see [here](https://github.com/CPJPRINCE/auto_reference_generator) for further details.
|
|
154
157
|
|
|
155
158
|
This is built-in to the Opex Manifest Generator and can be utilised to embed identifiers and metadata directly to an Opex or through the use of an Excel spreadsheet or CSV file.
|
|
156
159
|
|
|
157
|
-
The Opex Manifest Generator makes use of the
|
|
160
|
+
The Opex Manifest Generator makes use of the auto_reference_generator as a module, therefore it's behaviour differs a little different when compared to utilising the standalone command `auto_ref.exe`.
|
|
158
161
|
|
|
159
162
|
### Identifier Generation
|
|
160
163
|
|
|
161
|
-
To generate an auto
|
|
164
|
+
To generate an auto reference code, call on `-c` option with `catalog` choice. You can also assign a prefix using `-p "ARCH"`:
|
|
162
165
|
|
|
163
166
|
`opex_generate -c catalog -p "ARCH" C:\Users\Christopher\Downloads`
|
|
164
167
|
|
|
165
|
-
This will generate Opex's with an identifier `code` for each of the files / folders. As described in the Auto
|
|
168
|
+
This will generate Opex's with an identifier `code` for each of the files / folders. As described in the Auto Ref module, the reference codes will take the hierarchy of the directories. You can also use the `-s` option to set a starting reference.
|
|
166
169
|
|
|
167
170
|
You can alternatively utilise the "Accession" / running number mode of generating a code using `-c accession` with the prefix "2024". You can also utilise the `--accession-mode` option to determine whether to have a running number for `file, folder, both`.
|
|
168
171
|
|
|
@@ -176,7 +179,7 @@ You can also combine the generic options, like so: `catalog-generic, accession-g
|
|
|
176
179
|
|
|
177
180
|
## Use of Input Override option.
|
|
178
181
|
|
|
179
|
-
This program also supports utilising an Auto
|
|
182
|
+
This program also supports utilising an Auto Ref spreadsheet as an 'input override', utilising the data added into said spreadsheet instead of generating them ad hoc like above.
|
|
180
183
|
|
|
181
184
|
Using this method XIP Metadata fields can be set on Ingest, including:
|
|
182
185
|
|
|
@@ -195,9 +198,9 @@ XML metadata template data, from both the default templates and custom templates
|
|
|
195
198
|
|
|
196
199
|
### XIP metadata - Title, Description and Security Status
|
|
197
200
|
|
|
198
|
-
To use an input override, we need to first create a spreadsheet with the path of. You can utilise the `
|
|
201
|
+
To use an input override, we need to first create a spreadsheet with the path of. You can utilise the `auto_ref` tool installed alongside the Opex Generator, like so:
|
|
199
202
|
|
|
200
|
-
`
|
|
203
|
+
`auto_ref -p "ARCH" "C:\Users\Christopher\Downloads"`
|
|
201
204
|
|
|
202
205
|
In the resultant spreadsheet, add in "Title", "Description", and "Security" as new columns. The column headers are case-sensitive and have to match exactly. These fields would then be filled in with the relevant data.
|
|
203
206
|
|
|
@@ -205,7 +208,7 @@ In the resultant spreadsheet, add in "Title", "Description", and "Security" as n
|
|
|
205
208
|
|
|
206
209
|
Once the cells are filled in with data, run a generation like so: `opex_generate -i "{/path/to/your/spreadsheet.xlsx}" "{/path/to/root/directory}"`
|
|
207
210
|
|
|
208
|
-
Ensure that the root directory matches the original directory of the export. In the above case this would be: `opex_generate -i "C:\Users\Christopher\Downloads\meta\
|
|
211
|
+
Ensure that the root directory matches the original directory of the export. In the above case this would be: `opex_generate -i "C:\Users\Christopher\Downloads\meta\Downloads_AutoRef.xlsx" "C:\Users\Christopher\Downloads"`
|
|
209
212
|
|
|
210
213
|
### Headers Note
|
|
211
214
|
|
|
@@ -223,7 +226,7 @@ Custom Identifiers can be added by adding the columns: `"Archive_Reference", "Ac
|
|
|
223
226
|
|
|
224
227
|

|
|
225
228
|
|
|
226
|
-
`Archive_Reference` or `Identifier` will default to the keyname `code`; `Accession_Reference` will default to `accref`. When using the Auto
|
|
229
|
+
`Archive_Reference` or `Identifier` will default to the keyname `code`; `Accession_Reference` will default to `accref`. When using the Auto Reference Generator it will always generate a column called `Archive_Reference`, but you can simply rename or remove this column as necessary.
|
|
227
230
|
|
|
228
231
|
To add a custom identifier import, do so like: `Identifier:{YourIdentifierName}`, without the curly brackets IE: `Identifier:MyCode`. Multiple identifiers can be added as needed.
|
|
229
232
|
|
|
@@ -231,17 +234,17 @@ No additional parameter's need to be set in the command line when using Identifi
|
|
|
231
234
|
|
|
232
235
|
### XIP Metadata - Hashes
|
|
233
236
|
|
|
234
|
-
If you utilise the Auto
|
|
237
|
+
If you utilise the Auto Reference's tool for generating Hashes; when utilising the `-fx` option in combination with `-i`, if both the columns `Hash` and `Algorithm` are present, the program will read the hashes from the spreadsheet instead of generating them.
|
|
235
238
|
|
|
236
239
|

|
|
237
240
|
|
|
238
|
-
*Be aware that interruption / resuming is not currently supported with the Auto
|
|
241
|
+
*Be aware that interruption / resuming is not currently supported with the Auto Ref Tool; also doesn't support multiple hashes*
|
|
239
242
|
|
|
240
243
|
### XML Metadata - Basic Templates
|
|
241
244
|
|
|
242
245
|
DC, MODS, GPDR, and EAD templates are supported alongside installation of the package.
|
|
243
246
|
|
|
244
|
-
After exporting an Auto
|
|
247
|
+
After exporting an Auto Ref spreadsheet, you can add in additional columns to the spreadsheet and fill it out with data for an import. Like the XIP data, all fields are optional, and can added on a 'drop-in' basis.
|
|
245
248
|
|
|
246
249
|

|
|
247
250
|
|
|
@@ -284,7 +287,7 @@ The default location will be in the installation path of the program, typically
|
|
|
284
287
|
|
|
285
288
|
After the xml is added to that directory, all that's required is to add the matching column headers into your spreadsheet. You can also utilise `--print-xmls` to obtain this.
|
|
286
289
|
|
|
287
|
-
### Additional Information for Auto
|
|
290
|
+
### Additional Information for Auto Reference
|
|
288
291
|
#### SourceID
|
|
289
292
|
|
|
290
293
|
A SourceID can also be set by adding a `SourceID` header. The behaviour of this is not fully tested, likely won't be as I don't really utilise SourceIDs in my work :\).
|
|
@@ -327,7 +330,7 @@ ALGORITHM_FIELD = Algorithm
|
|
|
327
330
|
```
|
|
328
331
|
#### Custom Spreadsheets - Quick Note
|
|
329
332
|
|
|
330
|
-
You technically don't have to utilise the
|
|
333
|
+
You technically don't have to utilise the AutoRef tool at all. Any old spreadsheet will do!
|
|
331
334
|
|
|
332
335
|
The only requirement to use the input override, is the presence of the `FullName` column. With an accurate list of paths.
|
|
333
336
|
|
|
@@ -387,10 +390,10 @@ Options:
|
|
|
387
390
|
-opt --options-file Specify an 'options.properties' file to change set [PATH/TO/FILE]
|
|
388
391
|
presets for column headers for input.
|
|
389
392
|
|
|
390
|
-
Auto
|
|
393
|
+
Auto Reference Options:
|
|
391
394
|
|
|
392
|
-
-
|
|
393
|
-
module to generate an Auto
|
|
395
|
+
-r, --autoref This will utilise the auto_reference_generator [{catalog, accession,both,
|
|
396
|
+
module to generate an Auto Ref spreadsheet. generic, catalog-generic,
|
|
394
397
|
accession-generic,
|
|
395
398
|
There are several options, {catalog} will generate both-generic}]
|
|
396
399
|
a Archival Reference following; {accession}
|
|
@@ -406,13 +409,13 @@ Options:
|
|
|
406
409
|
--accession-mode Sets whether to have the running tally be for {file,folder,both}
|
|
407
410
|
files, folders or both,
|
|
408
411
|
when utilising the Accession option with
|
|
409
|
-
|
|
412
|
+
autoref. Default is file.
|
|
410
413
|
|
|
411
|
-
-p, --prefix Assign a prefix to the Auto
|
|
414
|
+
-p, --prefix Assign a prefix to the Auto Reference, [PREFIX]
|
|
412
415
|
when utilising {both} fill in like:
|
|
413
416
|
"catalog-prefix","accession-prefix".
|
|
414
417
|
|
|
415
|
-
-s --suffix Assign a suffix to the Auto
|
|
418
|
+
-s --suffix Assign a suffix to the Auto Reference [SUFFIX]
|
|
416
419
|
program. By Default only applies to Files
|
|
417
420
|
|
|
418
421
|
--suffix-options Set the Suffix assignment options {apply_to_files, apply_to_folders,
|
|
@@ -422,16 +425,16 @@ Options:
|
|
|
422
425
|
Log will bee exported to 'meta' / output folder
|
|
423
426
|
|
|
424
427
|
-o, --output Set's the output of the 'meta' folder when [PATH/TO/FOLDER]
|
|
425
|
-
utilising
|
|
428
|
+
utilising AutoRef.
|
|
426
429
|
|
|
427
|
-
-s, --start-ref Sets the starting Reference in the Auto
|
|
430
|
+
-s, --start-ref Sets the starting Reference in the Auto Ref [int]
|
|
428
431
|
process.
|
|
429
432
|
|
|
430
|
-
-i --input Set whether to use an Auto
|
|
433
|
+
-i --input Set whether to use an Auto Ref spreadsheet as an [PATH/TO/FILE]
|
|
431
434
|
input. The input needs to be the (relative or
|
|
432
435
|
absolute) path of the spreadsheet.
|
|
433
436
|
|
|
434
|
-
This allows for use of the Auto
|
|
437
|
+
This allows for use of the Auto Ref spreadsheet
|
|
435
438
|
to customise the XIP metadata (and custom xml
|
|
436
439
|
metadata).
|
|
437
440
|
|
|
@@ -467,7 +470,7 @@ Options:
|
|
|
467
470
|
--disable-meta-dir Will disable the creation of the 'meta' folder. [boolean flag]
|
|
468
471
|
Can also be enabled with output.
|
|
469
472
|
|
|
470
|
-
-ex --export Set whether to export any Auto
|
|
473
|
+
-ex --export Set whether to export any Auto Ref generation [boolean flag]
|
|
471
474
|
to a spreadsheet
|
|
472
475
|
|
|
473
476
|
-fmt, --format Set whether to export as a CSV or XLSX file. {csv,xlsx}
|
|
@@ -530,8 +533,8 @@ Options:
|
|
|
530
533
|
|
|
531
534
|
- ~~Customisable Filtering~~ *Added!*
|
|
532
535
|
- ~~Adjust Accession so the different modes can utilised from Opex.~~ *Added!*
|
|
533
|
-
- ~~Add SourceID as option for use with Auto
|
|
534
|
-
- ~~Allow for multiple Identifier's to be added with Auto
|
|
536
|
+
- ~~Add SourceID as option for use with Auto Ref Spreadsheets.~~ *Added!*
|
|
537
|
+
- ~~Allow for multiple Identifier's to be added with Auto Ref Spreadsheets. Currently only 1 or 2 identifiers can be added at a time, under "Archive_Reference" or "Accession_Reference". These are also tied to be either "code" or "accref". An Option needs to be added to allow custom setting of identifier...~~ *Added!*
|
|
535
538
|
- ~~Add an option / make it a default for Metadata XML's to be located in a specified directory rather than in the package.~~ *Added!*
|
|
536
539
|
- Zipping to conform to PAX - Last on the check list; it technically does...
|
|
537
540
|
- In theory, this tool should be compatible with any system that makes use of the OPEX standard... But in theory Communism works, in theory...
|
|
@@ -4,19 +4,26 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "opex_manifest_generator"
|
|
7
|
-
version = "1.3.
|
|
7
|
+
version = "1.3.2"
|
|
8
8
|
authors = [
|
|
9
9
|
{name="Christopher Prince", email="c.pj.prince@gmail.com"}
|
|
10
10
|
]
|
|
11
|
-
description = "Opex Manifest Generator
|
|
11
|
+
description = "An Opex Manifest Generator tool for use with OPEX Files, as designed by Preservica"
|
|
12
12
|
readme = "README.md"
|
|
13
|
+
license = "Apache-2.0"
|
|
14
|
+
license-files = ["LICEN[CS]E*"]
|
|
15
|
+
keywords = ["archiving","archives","digital archiving","opex","Preservica","opex generator"]
|
|
16
|
+
|
|
13
17
|
classifiers = [
|
|
14
18
|
"Programming Language :: Python :: 3",
|
|
15
|
-
"License :: OSI Approved :: Apache Software License",
|
|
16
19
|
"Operating System :: OS Independent",
|
|
17
20
|
"Topic :: System :: Archiving"
|
|
18
21
|
]
|
|
19
|
-
dependencies=["
|
|
22
|
+
dependencies=["auto_reference_generator","pandas","openpyxl","lxml"]
|
|
23
|
+
|
|
24
|
+
[project.optional-dependencies]
|
|
25
|
+
addex = ["odfpy"]
|
|
26
|
+
|
|
20
27
|
[project.urls]
|
|
21
28
|
Homepage = "https://github.com/CPJPRINCE/opex_manifest_generator"
|
|
22
29
|
Issues = "https://github.com/CPJPRINCE/opex_manifest_generator/issues"
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/.github/workflows/codeql.yml
RENAMED
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/.github/workflows/pypi-publish.yml
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/assets/Identifiers Headers.png
RENAMED
|
File without changes
|
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/__init__.py
RENAMED
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/common.py
RENAMED
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/opex_manifest_generator/hash.py
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/Sample File 1.jpg.opex
RENAMED
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/Sample File 2.gif.opex
RENAMED
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/Sample Folder Manifest.opex
RENAMED
|
File without changes
|
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/spreads/dctemplate.xlsx
RENAMED
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/spreads/eadtemplate.xlsx
RENAMED
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/spreads/gdprtemplate.xlsx
RENAMED
|
File without changes
|
{opex_manifest_generator-1.3.0 → opex_manifest_generator-1.3.2}/samples/spreads/modstemplate.xlsx
RENAMED
|
File without changes
|
|
File without changes
|