wavewash 0.1.0a1__tar.gz

wavewash-0.1.0a1/.github/workflows/tests.yml

@@ -0,0 +1,21 @@
+name: tests
+on:
+  push:
+  pull_request:
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+        python: ["3.10", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python }}
+      - run: python -m pip install --upgrade pip
+      - run: python -m pip install -e ".[dev]"
+      - run: wavewash doctor
+      - run: python -m pytest -q

wavewash-0.1.0a1/LICENSE

@@ -0,0 +1,29 @@
+MIT License
+
+Copyright (c) 2026 Boobalan Arjunan
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+---
+This project depends on third-party libraries (numpy, scipy, soundfile,
+librosa, pyloudnorm, scikit-learn, tqdm, langdetect, matplotlib, and
+optionally openai-whisper, torch/torchaudio). Each carries its own license
+(mostly BSD/MIT/Apache-2.0) -- check requirements.txt and each package's own
+license before redistributing, especially if your GTV program has its own
+licensing rules.

wavewash-0.1.0a1/MANIFEST.in

@@ -0,0 +1,9 @@
+include README.md LICENSE
+recursive-include src/wavewash/static *
+recursive-include frontend *
+prune frontend/node_modules
+prune frontend/dist
+global-exclude __pycache__ *.pyc .DS_Store
+recursive-include tests *.py
+recursive-include docs *.md *.png
+include .github/workflows/tests.yml

wavewash-0.1.0a1/PKG-INFO

@@ -0,0 +1,153 @@
+Metadata-Version: 2.4
+Name: wavewash
+Version: 0.1.0a1
+Summary: WaveWash: audio QC & data prep - check, repair, dedupe and audit audio datasets for ML training (ASR, TTS, speaker ID, music generation), with a local web UI.
+Author-email: Boobalan Arjunan <boobalan.it@gmail.com>
+Maintainer-email: Boobalan Arjunan <boobalan.it@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/BoobalanArjunan/wavewash
+Project-URL: Source, https://github.com/BoobalanArjunan/wavewash
+Project-URL: Issues, https://github.com/BoobalanArjunan/wavewash/issues
+Keywords: audio,dataset,quality-control,asr,tts,machine-learning,data-preparation
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
+Classifier: Intended Audience :: Science/Research
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.23
+Requires-Dist: scipy>=1.9
+Requires-Dist: soundfile>=0.12
+Requires-Dist: librosa>=0.10
+Requires-Dist: pyloudnorm>=0.1.1
+Requires-Dist: scikit-learn>=1.2
+Requires-Dist: tqdm>=4.65
+Requires-Dist: fastapi>=0.110
+Requires-Dist: uvicorn>=0.29
+Provides-Extra: lang
+Requires-Dist: langdetect>=1.0.9; extra == "lang"
+Provides-Extra: plots
+Requires-Dist: matplotlib>=3.5; extra == "plots"
+Provides-Extra: vad
+Requires-Dist: torch>=2.0; extra == "vad"
+Requires-Dist: torchaudio>=2.0; extra == "vad"
+Provides-Extra: whisper
+Requires-Dist: openai-whisper; extra == "whisper"
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: httpx>=0.25; extra == "dev"
+Provides-Extra: all
+Requires-Dist: langdetect>=1.0.9; extra == "all"
+Requires-Dist: matplotlib>=3.5; extra == "all"
+Dynamic: license-file
+
+# WaveWash
+
+[![tests](https://github.com/BoobalanArjunan/wavewash/actions/workflows/tests.yml/badge.svg)](https://github.com/BoobalanArjunan/wavewash/actions/workflows/tests.yml)
+![status](https://img.shields.io/badge/status-alpha-orange)
+![license](https://img.shields.io/badge/license-MIT-blue)
+
+**Check, repair and clean audio datasets before you train on them.**
+
+Point WaveWash at a folder of audio and tell it what you are training (speech recognition, voice synthesis, speaker ID, music generation). It checks every file, repairs what it can *verify* is fixed, moves the rest to an error folder with a labeled reason, sets duplicates aside, and writes a full audit log. Your original files are never modified.
+
+> **Status: alpha (0.1.0a1).** It works on the cases in its test suite, but it has not yet been validated on large real-world speech corpora. Treat its verdicts as a first pass and listen to a sample of what it keeps and rejects. See [Limitations](#limitations).
+
+![WaveWash app](https://raw.githubusercontent.com/BoobalanArjunan/wavewash/main/docs/screenshot-setup.png)
+
+## Why
+
+Models inherit the flaws of their audio: clipped or distorted takes, DC offset, background noise, low-quality copies of better recordings, duplicates that quietly inflate test scores, empty or broken files. These are usually found by hand (listening to thousands of files) or not at all. WaveWash puts the common checks in one tool and tells you *why* each file was kept, repaired or rejected.
+
+## Install
+
+```bash
+python3 -m venv ~/wavewash-env
+source ~/wavewash-env/bin/activate      # Windows: wavewash-env\Scripts\activate
+pip install wavewash
+wavewash doctor                         # confirms the install works
+```
+
+Python 3.10 or newer. No Node.js needed: the web app ships inside the package.
+
+Optional extras: `pip install "wavewash[lang]"` (transcript language check), `[plots]` (spectrograms in review samples), `[vad]` (Silero VAD, pulls in PyTorch), `[whisper]` (WER check).
+
+## Use it
+
+**Web app** (runs only on your machine, at `http://127.0.0.1:8420`):
+
+```bash
+wavewash app
+```
+
+Pick a purpose, choose an input folder (subfolders are included) and an output folder, click run.
+
+**Command line:**
+
+```bash
+wavewash run --input ./raw_audio --output ./out/clean --purpose asr
+wavewash run --help
+```
+
+Purposes: `asr`, `tts`, `speaker_id`, `music_gen`, `custom`.
+
+## What you get
+
+```
+out/
+  clean/          audio that passed, or was repaired and re-checked
+  _error/         rejected files, in folders named by reason (low_snr/, clipping/, corrupt_or_unreadable/ ...)
+  _duplicates/    duplicates, copied here (nothing is deleted)
+  _logs/          audit_log_*.csv and .json (one row per file: what was found, what was fixed, why rejected), summary_*.md
+```
+
+## What it checks and repairs
+
+| Checks | Repairs (only kept if a re-check shows they helped) |
+|---|---|
+| Unreadable, empty or truncated files | DC offset removal |
+| Too short / too long for the purpose | De-clipping of short clipped runs (up to 8 ms, spline interpolation) |
+| Signal-to-noise ratio (estimate) | Noise reduction (spectral subtraction, kept only if the SNR estimate improves) |
+| Clipping, DC offset, loudness | Loudness normalisation, resampling, channel conversion to the purpose's targets |
+| Silence: mostly-silent files, long gaps | Trimming silence at the edges |
+| Lossy or low-pass sources (sharp spectral cliff) | |
+| Exact duplicates, same-samples duplicates, near-duplicates (same take, different level or format) | |
+| Multiple speakers (heuristic) | |
+| Optional: transcript language, WER against Whisper, speaker/session train-validation split | |
+
+Repair is classical signal processing, **not** machine learning.
+
+## Limitations
+
+Please read these before trusting a result.
+
+- **Alpha, tested on synthetic data.** The test suite plants known defects (DC offset, clipping, duplicates, broken, empty and stereo files, other sample rates) and checks the routing, that originals are never modified, the local server's safety rules and the command line. It has *not* been validated on large real speech corpora; thresholds are reasoned defaults.
+- **"Repaired" means "passed the re-check", not "restored".** De-clipping interpolates over saturated samples. On heavily clipped audio the re-check can pass even though the result is far from the original recording. Listen to repaired files.
+- **Reverb (RT60) is a warning only.** Measured against synthetic rooms of known reverb time, the blind estimate could not tell dry from reverberant audio, so it never rejects a file.
+- **Heuristics, not trained models:** the SNR estimate, the multi-speaker check and the near-duplicate threshold. Check their verdicts on your own data.
+- **Music:** mid-track silence is not flagged, and moderate noise can go unnoticed.
+- **Not yet measured:** whether cleaned data improves a trained model; speed and memory on very large datasets; real Whisper and Silero runs.
+- The web app has **no login** and binds to `127.0.0.1` only. Do not expose it to a network.
+
+## Development
+
+```bash
+git clone https://github.com/BoobalanArjunan/wavewash.git
+cd wavewash
+python3 -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+python -m pytest -q
+```
+
+The front-end source is in `frontend/` (React + Vite). The built app is committed in `src/wavewash/static/`; to rebuild it: `cd frontend && npm ci && npm run build`, then copy `dist/` over `src/wavewash/static/`.
+
+Documentation: [step-by-step guide](docs/STEP_BY_STEP.md) · [user manual](docs/USER_MANUAL.md)
+
+## Contributing
+
+Bug reports with a small example file are the most useful contribution. If WaveWash rejects a good file or passes a bad one, please open an issue with the log row for that file (from `_logs/audit_log_*.csv`).
+
+## License
+
+MIT

wavewash-0.1.0a1/README.md

@@ -0,0 +1,109 @@
+# WaveWash
+
+[![tests](https://github.com/BoobalanArjunan/wavewash/actions/workflows/tests.yml/badge.svg)](https://github.com/BoobalanArjunan/wavewash/actions/workflows/tests.yml)
+![status](https://img.shields.io/badge/status-alpha-orange)
+![license](https://img.shields.io/badge/license-MIT-blue)
+
+**Check, repair and clean audio datasets before you train on them.**
+
+Point WaveWash at a folder of audio and tell it what you are training (speech recognition, voice synthesis, speaker ID, music generation). It checks every file, repairs what it can *verify* is fixed, moves the rest to an error folder with a labeled reason, sets duplicates aside, and writes a full audit log. Your original files are never modified.
+
+> **Status: alpha (0.1.0a1).** It works on the cases in its test suite, but it has not yet been validated on large real-world speech corpora. Treat its verdicts as a first pass and listen to a sample of what it keeps and rejects. See [Limitations](#limitations).
+
+![WaveWash app](https://raw.githubusercontent.com/BoobalanArjunan/wavewash/main/docs/screenshot-setup.png)
+
+## Why
+
+Models inherit the flaws of their audio: clipped or distorted takes, DC offset, background noise, low-quality copies of better recordings, duplicates that quietly inflate test scores, empty or broken files. These are usually found by hand (listening to thousands of files) or not at all. WaveWash puts the common checks in one tool and tells you *why* each file was kept, repaired or rejected.
+
+## Install
+
+```bash
+python3 -m venv ~/wavewash-env
+source ~/wavewash-env/bin/activate      # Windows: wavewash-env\Scripts\activate
+pip install wavewash
+wavewash doctor                         # confirms the install works
+```
+
+Python 3.10 or newer. No Node.js needed: the web app ships inside the package.
+
+Optional extras: `pip install "wavewash[lang]"` (transcript language check), `[plots]` (spectrograms in review samples), `[vad]` (Silero VAD, pulls in PyTorch), `[whisper]` (WER check).
+
+## Use it
+
+**Web app** (runs only on your machine, at `http://127.0.0.1:8420`):
+
+```bash
+wavewash app
+```
+
+Pick a purpose, choose an input folder (subfolders are included) and an output folder, click run.
+
+**Command line:**
+
+```bash
+wavewash run --input ./raw_audio --output ./out/clean --purpose asr
+wavewash run --help
+```
+
+Purposes: `asr`, `tts`, `speaker_id`, `music_gen`, `custom`.
+
+## What you get
+
+```
+out/
+  clean/          audio that passed, or was repaired and re-checked
+  _error/         rejected files, in folders named by reason (low_snr/, clipping/, corrupt_or_unreadable/ ...)
+  _duplicates/    duplicates, copied here (nothing is deleted)
+  _logs/          audit_log_*.csv and .json (one row per file: what was found, what was fixed, why rejected), summary_*.md
+```
+
+## What it checks and repairs
+
+| Checks | Repairs (only kept if a re-check shows they helped) |
+|---|---|
+| Unreadable, empty or truncated files | DC offset removal |
+| Too short / too long for the purpose | De-clipping of short clipped runs (up to 8 ms, spline interpolation) |
+| Signal-to-noise ratio (estimate) | Noise reduction (spectral subtraction, kept only if the SNR estimate improves) |
+| Clipping, DC offset, loudness | Loudness normalisation, resampling, channel conversion to the purpose's targets |
+| Silence: mostly-silent files, long gaps | Trimming silence at the edges |
+| Lossy or low-pass sources (sharp spectral cliff) | |
+| Exact duplicates, same-samples duplicates, near-duplicates (same take, different level or format) | |
+| Multiple speakers (heuristic) | |
+| Optional: transcript language, WER against Whisper, speaker/session train-validation split | |
+
+Repair is classical signal processing, **not** machine learning.
+
+## Limitations
+
+Please read these before trusting a result.
+
+- **Alpha, tested on synthetic data.** The test suite plants known defects (DC offset, clipping, duplicates, broken, empty and stereo files, other sample rates) and checks the routing, that originals are never modified, the local server's safety rules and the command line. It has *not* been validated on large real speech corpora; thresholds are reasoned defaults.
+- **"Repaired" means "passed the re-check", not "restored".** De-clipping interpolates over saturated samples. On heavily clipped audio the re-check can pass even though the result is far from the original recording. Listen to repaired files.
+- **Reverb (RT60) is a warning only.** Measured against synthetic rooms of known reverb time, the blind estimate could not tell dry from reverberant audio, so it never rejects a file.
+- **Heuristics, not trained models:** the SNR estimate, the multi-speaker check and the near-duplicate threshold. Check their verdicts on your own data.
+- **Music:** mid-track silence is not flagged, and moderate noise can go unnoticed.
+- **Not yet measured:** whether cleaned data improves a trained model; speed and memory on very large datasets; real Whisper and Silero runs.
+- The web app has **no login** and binds to `127.0.0.1` only. Do not expose it to a network.
+
+## Development
+
+```bash
+git clone https://github.com/BoobalanArjunan/wavewash.git
+cd wavewash
+python3 -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+python -m pytest -q
+```
+
+The front-end source is in `frontend/` (React + Vite). The built app is committed in `src/wavewash/static/`; to rebuild it: `cd frontend && npm ci && npm run build`, then copy `dist/` over `src/wavewash/static/`.
+
+Documentation: [step-by-step guide](docs/STEP_BY_STEP.md) · [user manual](docs/USER_MANUAL.md)
+
+## Contributing
+
+Bug reports with a small example file are the most useful contribution. If WaveWash rejects a good file or passes a bad one, please open an issue with the log row for that file (from `_logs/audit_log_*.csv`).
+
+## License
+
+MIT

wavewash-0.1.0a1/docs/STEP_BY_STEP.md

@@ -0,0 +1,85 @@
+# WaveWash: step by step, from download to published
+
+Everything below is for a Mac. Replace `BoobalanArjunan` with your own.
+
+## Part A. Put it on your computer and try it
+
+1. **Find the download.** Files you downloaded are in your `Downloads` folder.
+   You need `wavewash-source.zip`.
+2. **Unzip it.** Double-click the zip. You get a folder called `wavewash-tool`.
+3. **Move the folder** somewhere permanent, for example `Documents`.
+4. **Open Terminal** (press Cmd+Space, type Terminal, press Enter).
+5. **Go into the folder:**
+   `cd ~/Documents/wavewash-tool`
+6. **Make a private Python space** (done once):
+   `python3 -m venv .venv`
+   then switch it on (every time you open a new Terminal):
+   `source .venv/bin/activate`
+7. **Install WaveWash and the test tools:**
+   `pip install -e ".[dev]"`
+8. **Run the tests.** This is the "does it work on my Mac?" check:
+   `python -m pytest -q`
+   You want to see `23 passed`. If something fails, copy the message and send it to me.
+9. **Try the app:** `wavewash app`. Your browser opens. Choose an input folder with .wav files and
+   an output folder that is NOT inside the input folder, then click Run.
+10. **Or use the command line:**
+    `wavewash run --input ~/Desktop/my_wavs --output ~/Desktop/my_wavs_clean/clean --purpose asr`
+
+## Part B. Test it on real speech (the most important step)
+
+11. Download a free sample of real speech, for example LibriSpeech **dev-clean** from openslr.org/12.
+12. Run WaveWash on a few hundred files (step 10 with that folder).
+13. Open the results. Look at the first 50 files it rejected and ask: *was it really bad?*
+    Write down: files checked, files rejected, how many of the rejects were really bad.
+    These real numbers go in the README. If many good files were rejected, tell me and we fix the settings.
+
+## Part C. Put the code on GitHub
+
+14. Make a free account at github.com.
+15. Click **New repository**. Name it `wavewash`. Choose Public. Do not tick any extra files.
+16. In Terminal, inside `wavewash-tool`:
+    ```
+    git init
+    git add .
+    git commit -m "WaveWash 0.1.0a1"
+    git branch -M main
+    git remote add origin https://github.com/BoobalanArjunan/wavewash.git
+    git push -u origin main
+    ```
+17. In the README, replace `BoobalanArjunan` in the badge line.
+18. On GitHub open the **Actions** tab. The tests now run by themselves on Mac, Windows and Linux.
+    Green tick means good. Red means something broke on that system, and we fix it.
+
+## Part D. Publish to PyPI (so anyone can `pip install wavewash`)
+
+18b. **Put your name on it.** Run this once, with your own details (the email is optional and becomes PUBLIC):
+    `python scripts/set_owner.py --name "Your Full Name" --github your-github-name`
+    It sets you as author and maintainer, fixes the project links and the licence holder, and replaces every
+    `BoobalanArjunan`. On PyPI you are the *owner* of the project because you upload it from your PyPI account.
+
+19. Check the name is free: open pypi.org/project/wavewash. If the page says "not found", it is free.
+20. Make accounts at **test.pypi.org** and **pypi.org** (turn on two-factor login on both).
+    On each, create an **API token** (Account settings, API tokens).
+21. Build the files (inside `wavewash-tool`, with the venv switched on):
+    ```
+    pip install build twine
+    python -m build
+    ```
+    The files appear in the `dist` folder.
+22. **Practice upload first** (uses the test site; username is `__token__`, password is your token):
+    `python -m twine upload --repository testpypi dist/*`
+23. **Check it installs** in a fresh space:
+    ```
+    python3 -m venv ~/try-wavewash && source ~/try-wavewash/bin/activate
+    pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple wavewash
+    wavewash doctor
+    ```
+24. **Real upload.** You can't undo it, so only after step 23 works:
+    `python -m twine upload dist/*`
+25. Anyone can now run `pip install wavewash`.
+
+## What each part is for
+- **Tests** protect users. A change that breaks something shows up before it reaches them.
+- **Real-speech numbers** show it works on real data.
+- **GitHub and the green tick** give people a reason to trust it.
+- **TestPyPI first** because a mistake on real PyPI can't be erased.