libseraph 0.1.1__py3-none-any.whl

libseraph-0.1.1.dist-info/METADATA

@@ -0,0 +1,157 @@
+Metadata-Version: 2.4
+Name: libseraph
+Version: 0.1.1
+Summary: A multimedia dataset management tool for ML training
+Author-email: Ryan Quinn <ryan.quinn@certusinnovations.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/Stonewall-Defense/libseraph
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Natural Language :: English
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: click~=8.2
+Requires-Dist: numpy~=2.3
+Requires-Dist: requests~=2.32
+Requires-Dist: rich~=14.1
+Requires-Dist: soundfile~=0.13
+Requires-Dist: tqdm~=4.67
+Requires-Dist: tinytag~=2.1
+Requires-Dist: torch~=2.8
+Requires-Dist: torchaudio~=2.8
+Requires-Dist: As-A-Person~=0.1
+Dynamic: license-file
+
+# libseraph
+
+A hot new dataset management tool that's crazy easy!
+
+## Motivation
+
+There is no generally accepted metadata standard for multimedia data, and no tooling for multimedia dataset (meta)data management. At the outset of TEAM-ML, creating a training dataset was an error-prone process that typically required 8-20 hours of work from an ML expert, at a total cost of $1,200-$3,000.
+
+To expedite the creation and refinement of training datasets, we developed an absolute minimum metadata standard for our multimedia datasets and a management tool that covers all our common use cases.
+
+In our experience, it requires 0.5-3 hours to prepare a component dataset for Seraph management with a bespoke script. This is a non-recurring cost that depends on the original dataset format and the delta between that format and the Seraph format. Once all component datasets are configured, a training dataset can be assembled in 5-30 _minutes_, depending on (1) the user’s understanding of the tooling and the desired end composition and (2) whether the component dataset(s) can be copied in as-is or if the data needs to be resampled/resized/etc. Seraph is particularly useful for creating special-purpose or exploratory datasets from existing components; we were able to create a “9mm Parabellum Cartridge Dataset” at a cost of <$10.
+
+Seraph currently supports only audio datasets, as anything else is out of scope for TEAM-ML.
+
+## Installation
+
+```bash
+conda create --name seraph python=3.12
+conda activate seraph
+
+pip install -r requirements.txt
+pip install .
+
+conda deactivate
+```
+
+### Compatibility Note
+
+Other Python ML libraries from Certus Innovations use PyTorch 2.10 and the `torchcodec` library for loading audio. Unfortunately, the `torchcodec` library does not support enough options to save files for `seraph`, so it currently must rely on `torchaudio`, which limits the PyTorch version to 2.8. Using Conda or `venv` this isn't too hard to work around, but we are actively working on a path to upgrade this library for compatibility of our packages.
+
+## Usage
+
+The most used features of the Seraph tool are:
+
+- Audio
+  - Import audio data from other datasets, including allowing class selection and exclusion
+  - Generate duration metadata
+  - Clip audio data to a set length while preserving original track identity data
+  - Resample audio
+  - Prune empty audio files
+- Classes
+  - Switch class columns
+  - Rename, merge, regex merge, and drop classes by name
+  - Check class balance, including by fold/split
+  - Compose class metadata from existing column(s)
+- Metadata
+  - Initialize a new seraph dataset
+  - Verify all data items against the dataset contract specified in the metadata file
+- Provenance
+  - Prototype OpenIRIS integration for showing and submitting provenance
+- Prune
+  - Remove records with no corresponding files and vice versa
+  - Drop data by row value
+  - Drop metadata columns
+- Splits
+  - Automatically generate train/test/validate splits or cross-validation folds with respect to class balance and optionally avoid pseudoreplication
+- Version
+  - Prototype dataset version management by at least one [community standard](https://github.com/dslp/dslp/blob/main/semantic-versioning.md)
+- Integrations
+  -Prototype Fuel AI metadata format export
+
+### Examples
+
+```bash
+# Activate environment
+conda activate seraph
+
+# Initialize new dataset
+seraph meta init
+
+# Import audio datasets
+seraph audio import --import_dir ~/Desktop/Kaggle_Gunshots/
+seraph audio import --import_dir ~/Desktop/Cadre_Forensics/ --channel_merge_strat mix_down --sample_rate_merge_strat mix_down
+
+# Switch classes from `gun_type` to `caliber`
+seraph classes switch --new_class_col caliber --new_name_for_current_class_col gun_type
+
+# Merge degenerate classes
+seraph classes merge --target_class_name 9x19 --classes_to_merge "9mm Luger" --classes_to_merge "9mm"
+
+# Add durations to columns and clip to 1 sec
+seraph audio duration --metadata_column_conflict_strat replace
+seraph audio clip --clip_duration_secs 1 --dry_run
+
+# Show provenance data (WIP)
+seraph prov show
+seraph prov submit --activity_label "Make new gunshot dataset"
+
+# Show verioning data (WIP)
+seraph version show
+
+# Cleanup
+conda deactivate
+```
+
+## Testing
+
+```bash
+python3 -m coverage run -m unittest discover -s test -p "*_test.py" && python -m coverage report --skip-covered
+python -m coverage html
+```
+
+### Tests to Write
+
+- No Coverage
+  - integrations
+  - provenance
+- Partial Coverage
+  - meta
+  - version
+
+## Feature Wish-List
+
+- **IDEMPOTENCE**
+  - Prevent a dataset from being "double-tapped"
+- Pipe dreams
+  - Undo
+
+## Versioning
+
+We use [SemVer](http://semver.org/) for versioning. For the versions available, see the [tags on this repository](https://github.com/Stonewall-Defense/libseraph/tags).
+
+## Authors
+
+- **Ryan Quinn** - _Initial work_
+
+## License
+
+MIT.

libseraph-0.1.1.dist-info/RECORD

@@ -0,0 +1,28 @@
+libseraph-0.1.1.dist-info/licenses/LICENSE,sha256=ZGxGE7xzaYRti8pi4TDgcCHpYwx5QjObSWpUp7H-AE0,1084
+seraph/__init__.py,sha256=SjD55C4w8RGyn2wqcaCsEM6qjlC9FIAYBLT8BdkAJ84,256
+seraph/main.py,sha256=cdMLthF0bGp8mT8SMhmNxDyiBpszwOhUkDYpPhAbVKc,1242
+seraph/exec/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+seraph/exec/audio.py,sha256=uWFwf3OXCuVPco5TRUaJl4eJkHXepsT-JFnTw6YFQ74,41844
+seraph/exec/classes.py,sha256=CKrpBKWE7fiJfd_n0QlygAHsGCtOovtOO9ysLH7m1J4,16500
+seraph/exec/integrations.py,sha256=AN7FHrNfnvADeWp1QpMpI73bbgzqJAXSm6aFT_V5ROQ,10225
+seraph/exec/meta.py,sha256=Loyy4wO3LH4Ngx_BnYH2OKN8JQ0uEGdSlLbm8ImC4B8,11699
+seraph/exec/provenance.py,sha256=IZwI7swGXU1qbCcT8F0y1zULeCf7bTGa_ZnPUyKpZII,7121
+seraph/exec/prune.py,sha256=-7UvkNEra5kEi5YozU-fxsUvP4hZnaPeplit3L_IBpc,8404
+seraph/exec/splits.py,sha256=I0P3HwJFpC9ccfS-uC65n3wpLVMiNW_dVCAwM_L-pGA,11605
+seraph/exec/version.py,sha256=nEKj9Yb9Yp7UAgycHUAZgYJpzAEn6tKkpAhIh6uo4R4,10883
+seraph/lib/__init__.py,sha256=ZHZAOQgxeJ7z7o8i1QO-u4YRq5VhQYZnKhlT-XWiiXI,973
+seraph/lib/author.py,sha256=kCVuR0eiv9VLIJj_bGoUTfe9dRWgPrQ-7HzPjE_nI1c,4865
+seraph/lib/common.py,sha256=JwxIpLYOWWbze4jjqSr6Os45dCftTzMw11sF3L3oJ8Y,4260
+seraph/lib/dataset.py,sha256=4PLdNLPecgSG2YTRF-dSr5MiPp5glvju7oHOdgHJylM,14786
+seraph/lib/history.py,sha256=O-_wY2Z5de3RMx4x29wtNOaglgx5ISGnl_QZineBDtU,11088
+seraph/lib/license.py,sha256=zglBCLMGy2uAVqYisF0jf8FXqp5ypCimCDpVxUisPq4,4778
+seraph/lib/media_type.py,sha256=kkjlB2ae73FYGkpsDrmxRja0S0GdzoW7t_caLKEq570,4379
+seraph/lib/data/media_types/audio.csv,sha256=4oCPCQwvjcH5C1ROmVWsXEefJoh9cOtrusU_TFscOe0,6476
+seraph/lib/data/media_types/image.csv,sha256=d6xxX3p_bWNuL0xQmEUsQQiHiAqoJTp9ESgDUQhjICo,4080
+seraph/lib/data/media_types/text.csv,sha256=Qxprf_vZnZp5Qq0K2xkk7HZ7M7YFEpxMhRXWFsLgAd0,4527
+seraph/lib/data/media_types/video.csv,sha256=Zf62LvoW-xNUyVH9zWVA7MUvkvdmi_nSUEDjc3MH2LM,4336
+libseraph-0.1.1.dist-info/METADATA,sha256=TG1TSOZp9EKNCWBMfnrv70cZjlJmlQzi_BqBknTQ9UI,5773
+libseraph-0.1.1.dist-info/WHEEL,sha256=aeYiig01lYGDzBgS8HxWXOg3uV61G9ijOsup-k9o1sk,91
+libseraph-0.1.1.dist-info/entry_points.txt,sha256=gtcSZlnkiVTdSvjppidnGgms2s2MGxRL5AmUiRkGkMc,43
+libseraph-0.1.1.dist-info/top_level.txt,sha256=WRG4cJzgKBBsFPMc6BtV8kXeOIIFLnq9Z07pbu9ZDdE,7
+libseraph-0.1.1.dist-info/RECORD,,

libseraph-0.1.1.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (82.0.1)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

libseraph-0.1.1.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+seraph = seraph.main:cli

libseraph-0.1.1.dist-info/licenses/LICENSE

@@ -0,0 +1,20 @@
+The MIT License (MIT)
+Copyright (c) 2026 Certus Innovations
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

libseraph-0.1.1.dist-info/top_level.txt

@@ -0,0 +1 @@
+seraph

seraph/__init__.py

@@ -0,0 +1,3 @@
+from .lib import read_csv, read_json, write_csv, write_json  # noqa
+from .lib import DatasetAuthor, SeraphMetadata, SeraphDataset, SeraphMetadataError  # noqa
+from .lib import HistoryManager, ChangeType, VersionBumpType, ImportRecord, ChangeRecord  # noqa