audio-classifier-tool 1.2.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,123 @@
1
+ Metadata-Version: 2.5
2
+ Name: audio-classifier-tool
3
+ Version: 1.2.0
4
+ Summary: Detect and cut target segments (ads by default) from audio, config-driven and LLM-verified
5
+ Project-URL: Homepage, https://github.com/jon-fox/audio-classifier-tool
6
+ Project-URL: Repository, https://github.com/jon-fox/audio-classifier-tool
7
+ Author: Jon Fox
8
+ License-Expression: MIT
9
+ License-File: LICENSE
10
+ Keywords: ad-removal,ads,audio,classifier,podcast,whisper
11
+ Classifier: Development Status :: 4 - Beta
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: Programming Language :: Python :: 3.13
14
+ Classifier: Topic :: Multimedia :: Sound/Audio
15
+ Requires-Python: >=3.13
16
+ Requires-Dist: backoff>=2.2.1
17
+ Requires-Dist: boto3>=1.40.21
18
+ Requires-Dist: ctranslate2==4.8.2
19
+ Requires-Dist: faster-whisper==1.2.1
20
+ Requires-Dist: numpy>=2
21
+ Requires-Dist: nvidia-cublas-cu12; sys_platform == 'linux'
22
+ Requires-Dist: nvidia-cudnn-cu12==9.*; sys_platform == 'linux'
23
+ Requires-Dist: openai>=2
24
+ Requires-Dist: pydantic-ai-slim[openai]>=2
25
+ Requires-Dist: pydantic>=2.11.7
26
+ Requires-Dist: python-toon>=0.1.3
27
+ Requires-Dist: requests>=2.32.5
28
+ Requires-Dist: scikit-learn>=1.9.1
29
+ Requires-Dist: soundfile>=0.14.0
30
+ Description-Content-Type: text/markdown
31
+
32
+ # AudioClassifier - Dynamic Classifier used for Audio Segment Identification and Removal
33
+
34
+ Runs locally by default: one audio file in, filtered audio out. Needs an OpenAI API key and `ffmpeg`.
35
+
36
+ ## Quick Start
37
+
38
+ ```bash
39
+ uv sync
40
+ export OPENAI_API_KEY=<your-key>
41
+
42
+ PAYLOAD='{"source": "My Show", "name": "Episode 1", "audio_url": "<mp3-url>"}' \
43
+ uv run audioclassifier --detection examples/configs/ads.toon
44
+ ```
45
+
46
+ Results land in `output/<source>/<name>/`: the original mp3, the cleaned `*_filtered.mp3`, and a `transcripts/` dir with what was transcribed and each LLM cut/keep decision (with reasoning). Or with Docker:
47
+
48
+ ```bash
49
+ docker build -t audioclassifier-app .
50
+ docker run --gpus all \
51
+ -e OPENAI_API_KEY=<your-key> \
52
+ -e DETECTION_CONFIG=ads \
53
+ -e PAYLOAD='{"source": "My Show", "name": "Episode 1", "audio_url": "<mp3-url>"}' \
54
+ -v "$(pwd)/output:/app/output" \
55
+ audioclassifier-app
56
+ ```
57
+
58
+ Drop `--gpus all` to run on CPU (slower). The same container runs on any GPU box — RunPod, Modal, a gaming PC.
59
+
60
+ ## As a Library
61
+
62
+ ```bash
63
+ uv add audio-classifier-tool # pip install audio-classifier-tool
64
+ ```
65
+
66
+ ```python
67
+ import audioclassifier
68
+
69
+ result = audioclassifier.process_audio(
70
+ source="My Show",
71
+ name="Episode 1",
72
+ audio_url="<mp3-url>",
73
+ detection="examples/configs/ads.toon", # a .toon path or a name in ./configs
74
+ detection_instructions="...", # or pass the prompt directly
75
+ detection_keywords=["use code", ...], # and the keyword gate
76
+ )
77
+ print(result["output_path"], result["seconds_removed"])
78
+ ```
79
+
80
+ Importing has no side effects; configure the `"audioclassifier"` logger to see progress.
81
+
82
+ For a real end-to-end run with live console output and a decision summary:
83
+
84
+ ```bash
85
+ uv run python examples/process_audio.py "<mp3-url>"
86
+ ```
87
+
88
+ ## How It Works
89
+
90
+ - Downloads the audio from the `PAYLOAD` JSON (`source`, `name`, `audio_url`, optional `description`)
91
+ - Transcribes with faster-whisper (GPU-accelerated, CPU works too)
92
+ - Detects target segments with OpenAI — ads by default
93
+ - Cuts them and re-assembles the audio
94
+
95
+ ## Detection
96
+
97
+ The classifier is fully yours to define — nothing is bundled. A [TOON](https://github.com/toon-format/spec) config supplies the keywords and prompts: select one with `--detection <name|path>` or `DETECTION_CONFIG` (names resolve from `./configs/`), or pass the prompt and keywords directly (`detection_instructions`/`detection_keywords` in the API, `DETECTION_INSTRUCTIONS`/`DETECTION_KEYWORDS` env vars). Complete examples live in `examples/configs/` (ads, politics).
98
+
99
+ ## Text Classifier (optional, opt-in)
100
+
101
+ A self-distilled local classifier can add a third detection signal alongside keywords and audio analysis. Every processed audio file writes LLM-labeled decisions under `output/**/transcripts/` — that corpus is the classifier's training data, and it grows with each run.
102
+
103
+ Enable with `USE_TEXT_CLASSIFIER=true`: the classifier's flagged ranges join the LLM prompt (advisory only — the LLM still decides), and it retrains automatically after each run from the **full accumulated history** (retraining is from scratch, in seconds, so nothing is ever forgotten). Refresh manually anytime:
104
+
105
+ ```bash
106
+ uv run python examples/train_classifier.py # or audioclassifier.train_text_classifier()
107
+ ```
108
+
109
+ How it propagates: the durable memory is the decision files in `output/` — the model file (`local_models/text_classifier.joblib`) is a disposable cache rebuilt from them. Editing a decision file's `cut_ranges_seconds` after listening feeds your correction into the next training pass. In Docker, mount `local_models/` alongside `output/` to carry the model between containers (the data already survives via the `output/` mount).
110
+
111
+ ## Parallelism
112
+
113
+ One audio file per `process_audio` call. Multiple processes are safe, even in the same directory — downloads and segment audio live in per-run temp dirs, and the classifier model is written atomically. Within one process, run audio files sequentially (Whisper models load once and are reused); concurrent runs in threads are supported only with one shared detection config. Don't feed the same audio to two processes at once — they'd write the same output files.
114
+
115
+ ## Options
116
+
117
+ - `LLM_MODEL` — any [pydantic-ai model string](https://ai.pydantic.dev/models/) (default `openai:gpt-5.6`; e.g. `openai:gpt-5-nano` for cheapest, `anthropic:claude-sonnet-4-6`, `ollama:qwen3` — non-OpenAI providers may need their extra installed)
118
+ - `DISCORD_ALERTS=true DISCORD_WEBHOOK_URL=<url>` — processing alerts in Discord
119
+ - `storage="s3://bucket/prefix"` (API) or `"storage"` in `PAYLOAD` — upload the run's outputs (audio + transcripts + decisions) to S3 under `<prefix>/<source>/<name>/`, using ambient AWS credentials
120
+
121
+ ## License
122
+
123
+ [MIT](LICENSE)
@@ -0,0 +1,25 @@
1
+ audioclassifier/__init__.py,sha256=AyO7Ro0619IofMvR4qF-kmGBV1lnWDxhhGHuxr8I78Q,2096
2
+ audioclassifier/__main__.py,sha256=D2TLiTegR9xmSHp8vdjc_I9IBy2H-TbqEZ57YdHaGSw,76
3
+ audioclassifier/cli.py,sha256=f1oYzovV5xKqBcuRMUJYxhf3RrT7EwRcc_kRTJW8PVQ,4168
4
+ audioclassifier/util.py,sha256=Nj4SjhPSwFRLtVIBBZ98GNDWbXmFtNw6vhSYlJdVPGU,227
5
+ audioclassifier/alerts/__init__.py,sha256=-kPUIrzcbg4jY6f405jL5idMv9hhnABcwpIha1hF65M,44
6
+ audioclassifier/alerts/discord_alerts.py,sha256=kHvtMf23-CHnmTrJrvbdby1RFZSkCj_uT3bP3Sa0sqw,6929
7
+ audioclassifier/cloud/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
8
+ audioclassifier/cloud/storage.py,sha256=46QAuB6ZGlZ1W-gdkF9Qz8YgH4cA-RWkaKHjCwUfNmE,1267
9
+ audioclassifier/config/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
10
+ audioclassifier/config/constants.py,sha256=KY5WSZgkpetZaG9k7cVWpMWfvatTcFx8gXcaogwy0sc,1788
11
+ audioclassifier/config/detection_config.py,sha256=pugr_IozCjTeJgtWgIYahFWhp-3fEepYh4An4EA6Dg4,4692
12
+ audioclassifier/detection/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
13
+ audioclassifier/detection/llm_detector.py,sha256=NDJiBa86TnxnO8iWIPq39inUlaUA1HYMEVzBndleMYs,7292
14
+ audioclassifier/detection/text_classifier.py,sha256=T2Z-LJb26FUpF_D0MNcDIuB1Tlsh9WKtTceVgMx4nXc,4906
15
+ audioclassifier/logger/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
16
+ audioclassifier/logger/logger_setup.py,sha256=Lh31aHSRJd2J_YSLFLjVZ5HRjfmS1aFcpE8Ra2DVm_4,546
17
+ audioclassifier/processing/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
18
+ audioclassifier/processing/audio_processor.py,sha256=KXjELSZuo-B6wU_IuL-EhXfnPPJ93ziu42NCIzcx6N0,13520
19
+ audioclassifier/processing/download_mp3.py,sha256=K0NYZPBzhmCK5lIpOPqqJ5p2kzABLr1D66KKuoSFdUc,2333
20
+ audioclassifier/processing/mp3_handler.py,sha256=MdHyOCKk3FPrsI0YY1_oTd6JgLfILQQe0Etc0VQzWWI,2864
21
+ audio_classifier_tool-1.2.0.dist-info/METADATA,sha256=2puSsHfpU7Aic1gN2o5uC6Bpbr_CarosHVWDvMRpcQU,5962
22
+ audio_classifier_tool-1.2.0.dist-info/WHEEL,sha256=THafob7ofN-NsuMN7Mg4qZyHaQI7KkD-QlcQatYhXPo,87
23
+ audio_classifier_tool-1.2.0.dist-info/entry_points.txt,sha256=vSN6GeXl6Eq7TOab5nXDLcpDqMGt5hJuukhP_aqn-8M,61
24
+ audio_classifier_tool-1.2.0.dist-info/licenses/LICENSE,sha256=masymspiVA2InuFLMBG--KV1VliTPJqIIm-q-O_Tj-M,1069
25
+ audio_classifier_tool-1.2.0.dist-info/RECORD,,
@@ -0,0 +1,4 @@
1
+ Wheel-Version: 1.0
2
+ Generator: hatchling 1.32.3
3
+ Root-Is-Purelib: true
4
+ Tag: py3-none-any
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ audioclassifier = audioclassifier.cli:main
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2024-2026 Jon Fox
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,59 @@
1
+ """AudioClassifier: detect and cut target segments from audio (ads by default)."""
2
+
3
+
4
+ def process_audio(
5
+ source,
6
+ name,
7
+ audio_url,
8
+ detection=None,
9
+ detection_instructions=None,
10
+ detection_keywords=None,
11
+ description=None,
12
+ storage=None,
13
+ ):
14
+ """Process one audio file: download, transcribe, detect, and cut.
15
+
16
+ source groups outputs (output/<source>/<name>/); name is used for the
17
+ output filenames. detection: a config name (in ./configs) or a .toon path;
18
+ detection_instructions / detection_keywords override the selected config
19
+ directly. description is optional context about the audio for the
20
+ detection config's context-extraction prompt. storage: optional
21
+ s3://bucket/prefix — outputs are uploaded under <prefix>/<source>/<name>/.
22
+ Returns a dict with output_path, filtered_duration, original_duration,
23
+ seconds_removed, and (with storage) the uploaded S3 URIs.
24
+
25
+ Configure the "audioclassifier" logger to see progress; set OPENAI_API_KEY
26
+ (or the provider key matching LLM_MODEL) before calling.
27
+ """
28
+ if detection or detection_instructions or detection_keywords:
29
+ from audioclassifier.config.detection_config import set_detection_config
30
+
31
+ set_detection_config(
32
+ detection,
33
+ instructions=detection_instructions,
34
+ keywords=detection_keywords,
35
+ )
36
+
37
+ from audioclassifier.cli import process_payload
38
+
39
+ return process_payload(
40
+ {
41
+ "source": source,
42
+ "name": name,
43
+ "audio_url": audio_url,
44
+ "description": description,
45
+ "storage": storage,
46
+ }
47
+ )
48
+
49
+
50
+ def train_text_classifier(output_dir=None):
51
+ """Train the self-distilled ad classifier from past runs' decision files.
52
+
53
+ Returns training metrics. The trained model is picked up automatically by
54
+ subsequent runs as an extra detection signal.
55
+ """
56
+ from audioclassifier.config.constants import FINISHED_MP3_DIR
57
+ from audioclassifier.detection import text_classifier
58
+
59
+ return text_classifier.train(output_dir or FINISHED_MP3_DIR)
@@ -0,0 +1,4 @@
1
+ from audioclassifier.cli import main
2
+
3
+ if __name__ == "__main__":
4
+ main()
@@ -0,0 +1 @@
1
+ # Empty __init__.py file for alerts package
@@ -0,0 +1,206 @@
1
+ import os
2
+
3
+ import requests
4
+ import traceback
5
+ from datetime import datetime
6
+ from audioclassifier.config.constants import DISCORD_ALERTS_ENABLED
7
+ from audioclassifier.logger.logger_setup import logger
8
+
9
+
10
+ class DiscordAlerter:
11
+ def __init__(self):
12
+ self.webhook_url = None
13
+
14
+ if not DISCORD_ALERTS_ENABLED:
15
+ logger.info("Discord alerts disabled (set DISCORD_ALERTS=true to enable)")
16
+ return
17
+
18
+ self.webhook_url = os.getenv("DISCORD_WEBHOOK_URL")
19
+ if not self.webhook_url:
20
+ logger.warning(
21
+ "DISCORD_ALERTS is enabled but DISCORD_WEBHOOK_URL is not set, alerts disabled"
22
+ )
23
+
24
+ def send_error_alert(
25
+ self, error, context="", name="", source="", additional_info=None
26
+ ):
27
+ """
28
+ Send a detailed error alert to Discord
29
+
30
+ Args:
31
+ error: The exception object or error message
32
+ context: Additional context about where the error occurred
33
+ name: Name of the audio being processed when error occurred
34
+ source: Source the audio belongs to
35
+ additional_info: Dictionary of additional information to include
36
+ """
37
+ if not self.webhook_url:
38
+ logger.debug("Discord alerts disabled, skipping alert")
39
+ return False
40
+
41
+ try:
42
+ # Build the error message
43
+ timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S UTC")
44
+
45
+ # Extract error details
46
+ if isinstance(error, Exception):
47
+ error_type = type(error).__name__
48
+ error_message = str(error)
49
+ # Get traceback if available
50
+ tb_str = (
51
+ traceback.format_exc()
52
+ if hasattr(error, "__traceback__")
53
+ else "No traceback available"
54
+ )
55
+ else:
56
+ error_type = "Error"
57
+ error_message = str(error)
58
+ tb_str = "No traceback available"
59
+
60
+ # Build the alert message
61
+ message_parts = [
62
+ "🚨 **AUDIOCLASSIFIER APP ERROR ALERT** 🚨",
63
+ f"**Timestamp:** {timestamp}",
64
+ ]
65
+
66
+ if source:
67
+ message_parts.append(f"**Source:** {source}")
68
+
69
+ if name:
70
+ message_parts.append(f"**Audio:** {name}")
71
+
72
+ if context:
73
+ message_parts.append(f"**Context:** {context}")
74
+
75
+ message_parts.extend(
76
+ [
77
+ f"**Error Type:** {error_type}",
78
+ f"**Error Message:** {error_message}",
79
+ ]
80
+ )
81
+
82
+ if additional_info:
83
+ message_parts.append("**Additional Info:**")
84
+ for key, value in additional_info.items():
85
+ message_parts.append(f" • {key}: {value}")
86
+
87
+ # Add truncated traceback (Discord has message limits)
88
+ if tb_str and tb_str != "No traceback available":
89
+ # Truncate traceback to avoid Discord message limits (2000 chars)
90
+ tb_lines = tb_str.split("\n")
91
+ if len(tb_str) > 1000:
92
+ tb_str = "\n".join(tb_lines[:10]) + "\n... (traceback truncated)"
93
+
94
+ message_parts.append(f"**Traceback:**\n```\n{tb_str}\n```")
95
+
96
+ final_message = "\n".join(message_parts)
97
+
98
+ # Ensure message doesn't exceed Discord's 2000 character limit
99
+ if len(final_message) > 1900:
100
+ final_message = final_message[:1900] + "\n... (message truncated)"
101
+
102
+ # Send to Discord
103
+ payload = {"content": final_message}
104
+ response = requests.post(self.webhook_url, json=payload, timeout=10)
105
+
106
+ if response.status_code == 204:
107
+ logger.info("Discord error alert sent successfully")
108
+ return True
109
+ else:
110
+ logger.error(
111
+ f"Failed to send Discord alert. Status: {response.status_code}, Response: {response.text}"
112
+ )
113
+ return False
114
+
115
+ except Exception as e:
116
+ logger.error(f"Error sending Discord alert: {e}")
117
+ return False
118
+
119
+ def send_processing_alert(
120
+ self,
121
+ message_type,
122
+ source="",
123
+ name="",
124
+ additional_info=None,
125
+ alert_title="PROCESSING",
126
+ ):
127
+ """
128
+ Send processing status alerts (success, start, etc.)
129
+
130
+ Args:
131
+ message_type: Type of message (success, started, warning, etc.)
132
+ source: Source the audio belongs to
133
+ name: Name of the audio
134
+ additional_info: Dictionary of additional information
135
+ """
136
+ if not self.webhook_url:
137
+ logger.debug("Discord alerts disabled, skipping alert")
138
+ return False
139
+
140
+ try:
141
+ timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S UTC")
142
+
143
+ emoji_map = {
144
+ "success": "✅",
145
+ "started": "🚀",
146
+ "warning": "⚠️",
147
+ "info": "ℹ️",
148
+ }
149
+
150
+ emoji = emoji_map.get(message_type, "📢")
151
+
152
+ message_parts = [
153
+ f"{emoji} **AUDIOCLASSIFIER {alert_title} UPDATE**",
154
+ f"**Type:** {message_type.upper()}",
155
+ f"**Timestamp:** {timestamp}",
156
+ ]
157
+
158
+ if source:
159
+ message_parts.append(f"**Source:** {source}")
160
+
161
+ if name:
162
+ message_parts.append(f"**Audio:** {name}")
163
+
164
+ if additional_info:
165
+ for key, value in additional_info.items():
166
+ message_parts.append(f"**{key}:** {value}")
167
+
168
+ final_message = "\n".join(message_parts)
169
+
170
+ payload = {"content": final_message}
171
+ response = requests.post(self.webhook_url, json=payload, timeout=10)
172
+
173
+ if response.status_code == 204:
174
+ logger.info(f"Discord {message_type} alert sent successfully")
175
+ return True
176
+ else:
177
+ logger.error(
178
+ f"Failed to send Discord {message_type} alert. Status: {response.status_code}"
179
+ )
180
+ return False
181
+
182
+ except Exception as e:
183
+ logger.error(f"Error sending Discord processing alert: {e}")
184
+ return False
185
+
186
+
187
+ # Global instance for easy access
188
+ discord_alerter = DiscordAlerter()
189
+
190
+
191
+ def send_error_alert(error, context="", name="", source="", additional_info=None):
192
+ """
193
+ Convenience function to send error alerts
194
+ """
195
+ return discord_alerter.send_error_alert(
196
+ error, context, name, source, additional_info
197
+ )
198
+
199
+
200
+ def send_processing_alert(message_type, source="", name="", additional_info=None):
201
+ """
202
+ Convenience function to send processing alerts
203
+ """
204
+ return discord_alerter.send_processing_alert(
205
+ message_type, source, name, additional_info
206
+ )
audioclassifier/cli.py ADDED
@@ -0,0 +1,147 @@
1
+ import argparse
2
+ import os
3
+ import sys
4
+
5
+ from audioclassifier.logger.logger_setup import configure_logging, logger
6
+ from audioclassifier.processing.mp3_handler import mp3_handler
7
+ from audioclassifier.config.detection_config import (
8
+ get_detection_config,
9
+ set_detection_config,
10
+ )
11
+ import json
12
+ from audioclassifier.config.constants import USE_TEXT_CLASSIFIER
13
+ from audioclassifier.detection.llm_detector import _ensure_api_key
14
+ from audioclassifier.util import generate_hash, sanitize_name
15
+ from audioclassifier.alerts.discord_alerts import (
16
+ send_error_alert,
17
+ send_processing_alert,
18
+ )
19
+
20
+
21
+ def prepare_mp3_file(source, description, audio_hash, audio_url, name):
22
+ logger.info(f"Fetching MP3 file::{audio_url}")
23
+
24
+ try:
25
+ result = mp3_handler(
26
+ source,
27
+ description,
28
+ audio_hash,
29
+ audio_url,
30
+ name=name,
31
+ )
32
+ logger.info(f"MP3 file processed::{result}")
33
+ return result
34
+ except Exception as e:
35
+ logger.error(f"Error processing MP3 file::{e}")
36
+ send_error_alert(
37
+ error=e,
38
+ context="MP3 file processing failed in prepare_mp3_file",
39
+ name=name,
40
+ source=source,
41
+ additional_info={
42
+ "audio_url": audio_url,
43
+ "audio_hash": audio_hash,
44
+ },
45
+ )
46
+ raise e
47
+
48
+
49
+ def process_payload(payload={}):
50
+ logger.info(f"Received payload::{payload}")
51
+
52
+ if not payload:
53
+ logger.info("No payload received")
54
+ return
55
+
56
+ source = sanitize_name(payload.get("source"))
57
+ name = payload.get("name")
58
+ audio_url = payload.get("audio_url")
59
+ logger.info(f"Processing source: {source}, name: {name}, audio_url: {audio_url}")
60
+ audio_hash = generate_hash(source, name)
61
+
62
+ description = payload.get("description") or ""
63
+ if description:
64
+ logger.info(f"Description provided for processing::{description}")
65
+
66
+ result = prepare_mp3_file(
67
+ source=source,
68
+ description=description,
69
+ audio_hash=audio_hash,
70
+ audio_url=audio_url,
71
+ name=name,
72
+ )
73
+ logger.info(f"Processing result::{result}")
74
+
75
+ storage = payload.get("storage")
76
+ if storage:
77
+ from audioclassifier.cloud.storage import upload_outputs
78
+
79
+ result["uploaded"] = upload_outputs(
80
+ os.path.dirname(result["output_path"]),
81
+ f"{storage.rstrip('/')}/{source}/{sanitize_name(name)}",
82
+ )
83
+
84
+ # Send success alert
85
+ send_processing_alert(
86
+ message_type="success",
87
+ source=source,
88
+ name=name,
89
+ additional_info={
90
+ "Audio Hash": audio_hash,
91
+ "Processed Length": f"{result['filtered_duration']} seconds",
92
+ },
93
+ )
94
+
95
+ # This run's decisions are new training data; refresh the classifier
96
+ if USE_TEXT_CLASSIFIER:
97
+ from audioclassifier.detection.text_classifier import retrain_after_run
98
+
99
+ retrain_after_run()
100
+
101
+ return result
102
+
103
+
104
+ def main():
105
+ configure_logging()
106
+
107
+ parser = argparse.ArgumentParser(
108
+ description="AudioClassifier: detect and cut content from audio"
109
+ )
110
+ parser.add_argument(
111
+ "--detection",
112
+ help="detection config: a name in ./configs or a path to a .toon file",
113
+ )
114
+ args = parser.parse_args()
115
+ try:
116
+ config = (
117
+ set_detection_config(args.detection)
118
+ if args.detection
119
+ else get_detection_config()
120
+ )
121
+ except (RuntimeError, FileNotFoundError, ValueError) as e:
122
+ logger.error(str(e))
123
+ print(e, file=sys.stderr)
124
+ sys.exit(1)
125
+ logger.info(f"Using detection config: {config.name}")
126
+
127
+ try:
128
+ _ensure_api_key()
129
+ except RuntimeError as e:
130
+ logger.error(str(e))
131
+ print(e, file=sys.stderr)
132
+ sys.exit(1)
133
+
134
+ payload = os.getenv("PAYLOAD")
135
+ if not payload:
136
+ message = (
137
+ 'Set PAYLOAD to a JSON object like {"source": ..., '
138
+ '"name": ..., "audio_url": ...} — see README.md'
139
+ )
140
+ logger.error(message)
141
+ print(message, file=sys.stderr)
142
+ sys.exit(1)
143
+ process_payload(json.loads(payload))
144
+
145
+
146
+ if __name__ == "__main__":
147
+ main()
File without changes
@@ -0,0 +1,36 @@
1
+ """Optional cloud storage: push a run's outputs to an s3:// location."""
2
+
3
+ import os
4
+
5
+ from audioclassifier.logger.logger_setup import logger
6
+
7
+
8
+ def upload_outputs(output_dir, storage_uri):
9
+ """Upload everything under output_dir to storage_uri (s3://bucket/prefix).
10
+
11
+ Uses ambient AWS credentials (env/profile/role). Returns the uploaded
12
+ s3:// URIs.
13
+ """
14
+ import boto3
15
+
16
+ bucket, prefix = _parse_s3_uri(storage_uri)
17
+ client = boto3.client("s3")
18
+ uploaded = []
19
+ for root, _, files in os.walk(output_dir):
20
+ for filename in files:
21
+ local_path = os.path.join(root, filename)
22
+ relative = os.path.relpath(local_path, output_dir).replace(os.sep, "/")
23
+ key = f"{prefix}/{relative}" if prefix else relative
24
+ client.upload_file(local_path, bucket, key)
25
+ uploaded.append(f"s3://{bucket}/{key}")
26
+ logger.info(f"Uploaded {relative} to s3://{bucket}/{key}")
27
+ return uploaded
28
+
29
+
30
+ def _parse_s3_uri(uri):
31
+ if not uri.startswith("s3://"):
32
+ raise ValueError(f"storage must be an s3:// URI, got {uri!r}")
33
+ bucket, _, prefix = uri[5:].partition("/")
34
+ if not bucket:
35
+ raise ValueError(f"storage URI is missing a bucket: {uri!r}")
36
+ return bucket, prefix.strip("/")
File without changes