vitosa-speech-II 0.0.1__tar.gz

vitosa_speech_ii-0.0.1/PKG-INFO

@@ -0,0 +1,146 @@
+Metadata-Version: 2.4
+Name: vitosa-speech-II
+Version: 0.0.1
+Summary: A library for Robust Vietnamese Audio-Based Toxic Span Detection and Censoring
+Author: Vy Le-Phuong Huynh, Huy Ba Do and Luan Thanh Nguyen
+Author-email: luannt@uit.edu.vn
+Project-URL: Model (Hugging Face), https://huggingface.co/UIT-ViToSA/PhoWhisper-BiLSTM-CRF
+Keywords: audio-processing,toxic-span-detection,vietnamese,asr,speech-recognition,censoring,phowhisper
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
+Classifier: Natural Language :: Vietnamese
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
+Requires-Dist: torch>=1.13.0
+Requires-Dist: transformers>=4.28.0
+Requires-Dist: librosa
+Requires-Dist: pydub
+Requires-Dist: huggingface_hub
+Requires-Dist: pytorch-crf
+Requires-Dist: numpy
+Requires-Dist: tqdm
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: keywords
+Dynamic: project-url
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# ViToSA 2.0: A MULTI-TASK APPROACH TOWARDS ROBUST VIETNAMESE AUDIO-BASED TOXIC SPAN DETECTION | ICASSP 2026
+
+**Official implementation** of the paper:  
+**“A Multi-Task Approach Towards Robust Vietnamese Audio-Based Toxic Span Detection”** (ICASSP 2026).
+
+This package provides an end-to-end pipeline for **Vietnamese speech-based toxic span detection**, combining **ASR and toxic span detection** in a unified model. It also supports **automatic audio censoring**, replacing toxic spans with beep sounds in the output waveform.
+
+---
+
+## Key Features
+
+* **Automated Audio Censoring**: Takes an input audio file containing toxic language and outputs a **clean `.wav` file** where profanity is masked with a beep.
+* **Unified Multi-Task Architecture**: Integrates ASR and Toxic Span Detection (TSD) into a single model for high speed.
+* **SOTA Performance**: Achieves **F1-macro 0.9212** on the ViToSA-v2 dataset using **PhoWhisper + BiLSTM-CRF + Knowledge Distillation**.
+* **High Efficiency**: Reduces inference latency by over **56%** compared to traditional pipelines.
+
+## Installation
+
+```bash
+pip install vitosa-speech
+```
+
+### System requirements
+
+This package relies on `pydub` for audio processing, which requires **ffmpeg** to be installed.
+
+- **Ubuntu / Debian**
+  ```bash
+  sudo apt-get install ffmpeg
+  ```
+
+- **macOS (Homebrew)**
+  ```bash
+  brew install ffmpeg
+  ```
+
+- **Windows**
+  Download ffmpeg from https://ffmpeg.org and add it to your system `PATH`.
+
+---
+
+
+## Quick Start
+This library allows you to input a raw audio file and get a censored audio file as the output.
+
+### 1. Load the Model
+```python
+The model is pre-trained on the ViToSA-v2 dataset
+
+import torch
+from vitosa-speech-II import load_my_model
+# Automatically detect device (CUDA/CPU)
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+# Load the pre-trained model
+model, processor = load_my_model(device)
+```
+
+### 2. Run Inference (Detect & Censor)
+```python
+from vitosa-speech-II import return_labels, censor_audio_with_beep
+from IPython.display import Audio, display # Optional: to play in notebook
+
+# Path to your input file
+input_audio = "samples/toxic_speech.wav"
+
+# Step 1: Detect toxic spans
+words_with_labels = return_labels(input_audio, model, processor, device)
+
+# Step 2: Generate Censored Audio
+# This function creates a new audio file with beeps over toxic words
+output_audio_path = censor_audio_with_beep(
+    audio_path=input_audio, 
+    model=model, 
+    processor=processor, 
+    words_with_labels=words_with_labels, 
+    device=device
+)
+
+# Result
+print(f"✅ Censored audio saved to: {output_audio_path}")
+
+# Optional: Play the result (if in Jupyter/Colab)
+# display(Audio(output_audio_path))
+``` 
+
+## Methodology
+Our system works in two steps:
+
+1. Detection: The multi-task model (PhoWhisper + BiLSTM-CRF) processes the audio to identify the exact start and end timestamps of toxic words.
+2. Censoring: We reconstruct the audio by keeping safe segments and generating a sine wave (beep) to overlay exactly where the toxic tokens occur, ensuring the rest of the sentence remains intelligible.
+
+<!-- ## Citation
+If you use this tool or our findings, please cite:
+```bibtex
+@inproceedings{huynh2026multitask,
+  title={A Multi-Task Approach Towards Robust Vietnamese Audio-Based Toxic Span Detection},
+  author={Huynh, Vy Le-Phuong and Do, Huy Ba and Nguyen, Luan Thanh},
+  booktitle={Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
+  year={2026}
+}
+``` -->
+
+## Contact
+For more information: luannt@uit.edu.vn
+

vitosa_speech_ii-0.0.1/README.md

@@ -0,0 +1,105 @@
+# ViToSA 2.0: A MULTI-TASK APPROACH TOWARDS ROBUST VIETNAMESE AUDIO-BASED TOXIC SPAN DETECTION | ICASSP 2026
+
+**Official implementation** of the paper:  
+**“A Multi-Task Approach Towards Robust Vietnamese Audio-Based Toxic Span Detection”** (ICASSP 2026).
+
+This package provides an end-to-end pipeline for **Vietnamese speech-based toxic span detection**, combining **ASR and toxic span detection** in a unified model. It also supports **automatic audio censoring**, replacing toxic spans with beep sounds in the output waveform.
+
+---
+
+## Key Features
+
+* **Automated Audio Censoring**: Takes an input audio file containing toxic language and outputs a **clean `.wav` file** where profanity is masked with a beep.
+* **Unified Multi-Task Architecture**: Integrates ASR and Toxic Span Detection (TSD) into a single model for high speed.
+* **SOTA Performance**: Achieves **F1-macro 0.9212** on the ViToSA-v2 dataset using **PhoWhisper + BiLSTM-CRF + Knowledge Distillation**.
+* **High Efficiency**: Reduces inference latency by over **56%** compared to traditional pipelines.
+
+## Installation
+
+```bash
+pip install vitosa-speech
+```
+
+### System requirements
+
+This package relies on `pydub` for audio processing, which requires **ffmpeg** to be installed.
+
+- **Ubuntu / Debian**
+  ```bash
+  sudo apt-get install ffmpeg
+  ```
+
+- **macOS (Homebrew)**
+  ```bash
+  brew install ffmpeg
+  ```
+
+- **Windows**
+  Download ffmpeg from https://ffmpeg.org and add it to your system `PATH`.
+
+---
+
+
+## Quick Start
+This library allows you to input a raw audio file and get a censored audio file as the output.
+
+### 1. Load the Model
+```python
+The model is pre-trained on the ViToSA-v2 dataset
+
+import torch
+from vitosa-speech-II import load_my_model
+# Automatically detect device (CUDA/CPU)
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+# Load the pre-trained model
+model, processor = load_my_model(device)
+```
+
+### 2. Run Inference (Detect & Censor)
+```python
+from vitosa-speech-II import return_labels, censor_audio_with_beep
+from IPython.display import Audio, display # Optional: to play in notebook
+
+# Path to your input file
+input_audio = "samples/toxic_speech.wav"
+
+# Step 1: Detect toxic spans
+words_with_labels = return_labels(input_audio, model, processor, device)
+
+# Step 2: Generate Censored Audio
+# This function creates a new audio file with beeps over toxic words
+output_audio_path = censor_audio_with_beep(
+    audio_path=input_audio, 
+    model=model, 
+    processor=processor, 
+    words_with_labels=words_with_labels, 
+    device=device
+)
+
+# Result
+print(f"✅ Censored audio saved to: {output_audio_path}")
+
+# Optional: Play the result (if in Jupyter/Colab)
+# display(Audio(output_audio_path))
+``` 
+
+## Methodology
+Our system works in two steps:
+
+1. Detection: The multi-task model (PhoWhisper + BiLSTM-CRF) processes the audio to identify the exact start and end timestamps of toxic words.
+2. Censoring: We reconstruct the audio by keeping safe segments and generating a sine wave (beep) to overlay exactly where the toxic tokens occur, ensuring the rest of the sentence remains intelligible.
+
+<!-- ## Citation
+If you use this tool or our findings, please cite:
+```bibtex
+@inproceedings{huynh2026multitask,
+  title={A Multi-Task Approach Towards Robust Vietnamese Audio-Based Toxic Span Detection},
+  author={Huynh, Vy Le-Phuong and Do, Huy Ba and Nguyen, Luan Thanh},
+  booktitle={Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
+  year={2026}
+}
+``` -->
+
+## Contact
+For more information: luannt@uit.edu.vn
+

vitosa_speech_ii-0.0.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

vitosa_speech_ii-0.0.1/setup.py

@@ -0,0 +1,65 @@
+from setuptools import setup, find_packages
+
+with open("README.md", "r", encoding="utf-8") as fh:
+    long_description = fh.read()
+
+setup(
+    name="vitosa-speech-II", 
+    version="0.0.1",
+    
+    author="Vy Le-Phuong Huynh, Huy Ba Do and Luan Thanh Nguyen", 
+    author_email="luannt@uit.edu.vn",
+    
+    description="A library for Robust Vietnamese Audio-Based Toxic Span Detection and Censoring",
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    
+    # Link GitHub chính thức (quan trọng)
+    # url="https://github.com/ViToSAResearch/PhoWhisper-BiLSTM-CRF",
+    
+    # Tạo các liên kết bổ sung ở cột bên trái trang PyPI
+    project_urls={
+        # "Bug Tracker": "https://github.com/ViToSAResearch/PhoWhisper-BiLSTM-CRF/issues",
+        "Model (Hugging Face)": "https://huggingface.co/UIT-ViToSA/PhoWhisper-BiLSTM-CRF"
+    },
+    
+    packages=find_packages(exclude=("tests", "docs")),
+    
+    install_requires=[
+        "torch>=1.13.0",      
+        "transformers>=4.28.0",
+        "librosa",
+        "pydub",
+        "huggingface_hub",
+        "pytorch-crf",     
+        "numpy",
+        "tqdm"
+    ],
+    
+    keywords=[
+        "audio-processing", 
+        "toxic-span-detection", 
+        "vietnamese", 
+        "asr", 
+        "speech-recognition", 
+        "censoring",
+        "phowhisper"
+    ],
+    
+    classifiers=[
+        "Development Status :: 4 - Beta",
+        "Intended Audience :: Developers",
+        "Intended Audience :: Science/Research",
+        "Programming Language :: Python :: 3",
+        "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: 3.9",
+        "Programming Language :: Python :: 3.10",
+        "License :: OSI Approved :: MIT License",
+        "Operating System :: OS Independent",
+        "Topic :: Scientific/Engineering :: Artificial Intelligence",
+        "Topic :: Multimedia :: Sound/Audio :: Speech",
+        "Natural Language :: Vietnamese",
+    ],
+    
+    python_requires='>=3.7',
+)

vitosa_speech_ii-0.0.1/vitosa_speech_II/__init__.py

@@ -0,0 +1,5 @@
+from .inference import load_my_model, return_labels, toxic_span_asr_inference
+from .audio import censor_audio_with_beep
+from .model import WhisperToxicSpansKDModel
+
+__version__ = "0.1.0"

vitosa_speech_ii-0.0.1/vitosa_speech_II/audio.py

@@ -0,0 +1,75 @@
+import librosa
+from pydub import AudioSegment
+from pydub.generators import Sine
+from transformers import pipeline
+
+def censor_audio_with_beep(audio_path, model, processor, words_with_labels, device):
+    
+    print("-"*10 "Calculating time (alignment)" "-"*10)
+    pipe = pipeline(
+        "automatic-speech-recognition",
+        model=model.student,
+        tokenizer=processor.tokenizer,
+        feature_extractor=processor.feature_extractor,
+        device=device,
+        return_timestamps="word"
+    )
+    
+    result = pipe(audio_path)
+    chunks = result['chunks']
+    
+    toxic_timestamps = []
+    chunk_idx = 0
+    num_chunks = len(chunks)
+    
+    for word, label in words_with_labels:
+        clean_word = word.replace('<|startoftranscript|>', '').replace('<|transcribe|>', '').strip()
+        if not clean_word: continue
+            
+        if chunk_idx < num_chunks:
+            chunk = chunks[chunk_idx]
+            timestamp = chunk['timestamp']
+            
+            if label == 1:
+                toxic_timestamps.append(timestamp)
+            
+            chunk_idx += 1
+            
+    print('\nDone ✓\n')
+    print("-"*10 "Processing cutting and merging audio" "-"*10)
+    
+    try:
+        original_audio = AudioSegment.from_wav(audio_path)
+    except:
+        print("Error: Fail to load audio")
+        return None
+
+    final_audio = AudioSegment.empty()
+    
+    current_pos_ms = 0
+    
+    toxic_timestamps.sort(key=lambda x: x[0])
+    
+    for start, end in toxic_timestamps:
+        start_ms = start * 1000
+        end_ms = end * 1000
+        
+        if start_ms > current_pos_ms:
+            clean_segment = original_audio[current_pos_ms:start_ms]
+            final_audio += clean_segment
+        
+        duration_ms = end_ms - start_ms
+        if duration_ms > 0:
+            beep = Sine(1000).to_audio_segment(duration=duration_ms).apply_gain(-5)
+            final_audio += beep
+        
+        current_pos_ms = max(current_pos_ms, end_ms)
+
+    if current_pos_ms < len(original_audio):
+        remaining_audio = original_audio[current_pos_ms:]
+        final_audio += remaining_audio
+
+    output_path = "censored_audio_clean.wav"
+    final_audio.export(output_path, format="wav")
+    print(f"\nDone ✓ \n File result save path: {output_path}")
+    return output_path

vitosa_speech_ii-0.0.1/vitosa_speech_II/inference.py

@@ -0,0 +1,59 @@
+import torch
+import time
+import librosa
+from huggingface_hub import hf_hub_download
+from transformers import WhisperProcessor
+from .model import WhisperToxicSpansKDModel
+from .utils import group_tokens_into_words_corrected
+
+def load_my_model(device, repo_id="ViToSAResearch/PhoWhisper-BiLSTM-CRF", model_filename="model.pth"):
+    print(f"Loading model from {repo_id}...")
+    checkpoint_path = hf_hub_download(repo_id=repo_id, filename=model_filename)
+    
+    model = WhisperToxicSpansKDModel(use_crf=True, kd_layers=[4, 8, 12])
+    processor = WhisperProcessor.from_pretrained("Huydb/phowhisper-toxic", language="vietnamese", task="transcribe")
+    
+    state_dict = torch.load(checkpoint_path, map_location=device)
+    new_state_dict = {k: v for k, v in state_dict.items() if 'teacher.' not in k}
+    model.load_state_dict(new_state_dict, strict=False)
+    model.to(device)
+    model.eval()
+    return model, processor
+
+def toxic_span_asr_inference(audio_path: str, model, whisper_processor, device):
+    timings = {}
+    print( "-"*10 f"Processing audio file: {audio_path} " "-"*10)
+    
+    start_time = time.perf_counter()
+    speech_array, sampling_rate = librosa.load(audio_path, sr=16000)
+    input_features = whisper_processor(speech_array, sampling_rate=sampling_rate, return_tensors="pt").input_features
+    
+    with torch.no_grad():
+        start_time = time.perf_counter()
+        predicted_ids = model.student.generate(input_features.to(device))[0] 
+        if device.type == 'cuda':
+            torch.cuda.synchronize()
+        timings["2_asr_inference"] = time.perf_counter() - start_time
+    
+    transcribed_text = whisper_processor.tokenizer.decode(predicted_ids, skip_special_tokens=True)
+    
+    input_ids_for_toxic = predicted_ids.unsqueeze(0).to(device)
+    attention_mask_for_toxic = torch.ones_like(input_ids_for_toxic)
+    
+    with torch.no_grad():
+        start_time = time.perf_counter()
+        toxic_labels_list = model.predict(input_ids_for_toxic, attention_mask_for_toxic)
+        if device.type == 'cuda':
+            torch.cuda.synchronize()
+        timings["3_toxic_span_inference"] = time.perf_counter() - start_time
+        
+        toxic_labels = toxic_labels_list[0] if isinstance(toxic_labels_list, list) else toxic_labels_list
+
+    return transcribed_text, predicted_ids, toxic_labels, timings
+
+
+def return_labels(audio_file,model,processor,device):
+    text, pred_ids, labels, execution_times = toxic_span_asr_inference(audio_file, model, processor, device)
+
+    words_with_labels = group_tokens_into_words_corrected(pred_ids, labels, processor.tokenizer)
+    return words_with_labels

vitosa_speech_ii-0.0.1/vitosa_speech_II/model.py

@@ -0,0 +1,139 @@
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from transformers import WhisperConfig, WhisperTokenizerFast, WhisperForConditionalGeneration
+from torchcrf import CRF
+
+class WhisperToxicSpansKDModel(nn.Module):
+    def __init__(
+        self,
+        whisper_name: str = "Huydb/phowhisper-toxic",
+        use_crf: bool = True,
+        use_bidirectional = True,
+        lstm_hidden: int = 512,
+        dropout: float = 0.4,
+        alpha_span: float = 0.5,
+        alpha_kd: float = 0.5,
+        kd_temp: float = 2.0,
+        kd_layers: list = None,
+        teacher_model=None
+    ):
+        super().__init__()
+        # -- Student setup
+        self.tokenizer = WhisperTokenizerFast.from_pretrained(
+            whisper_name, language="vietnamese", task="transcribe", use_fast=True
+        )
+        config = WhisperConfig.from_pretrained(whisper_name)
+        config.output_hidden_states = True  # enable hidden states
+        self.student = WhisperForConditionalGeneration.from_pretrained(
+            whisper_name, config=config
+        )
+
+        # -- KD teacher (CPU)
+        self.teacher = teacher_model
+
+        # -- Heads
+        d_model = self.student.config.d_model
+        self.dropout = nn.Dropout(dropout)
+        self.use_crf = use_crf
+        self.use_bidirectional = use_bidirectional
+        if use_crf:
+            if use_bidirectional:
+                self.bilstm = nn.LSTM(
+                    d_model, lstm_hidden // 2,
+                    num_layers=1, batch_first=True, bidirectional=True
+                )
+                classifier_in_dim = lstm_hidden
+            else:
+                self.bilstm = nn.LSTM(d_model, lstm_hidden,
+                      num_layers=1, batch_first=True, bidirectional=False)
+                classifier_in_dim = lstm_hidden
+            
+            self.classifier = nn.Linear(classifier_in_dim, 2)
+            self.crf = CRF(2, batch_first=True)
+        else:
+            self.classifier = nn.Linear(d_model, 2)
+
+        # -- KD hyperparams
+        self.alpha_span = alpha_span
+        self.alpha_kd = alpha_kd
+        self.temperature = kd_temp
+        self.kd_layers = kd_layers or [4, 8, 12]
+
+    def forward(
+        self,
+        input_ids,
+        attention_mask,
+        labels=None,
+        teacher_input_ids=None,
+        teacher_attention_mask=None
+    ):
+        device = next(self.student.parameters()).device
+        ids = input_ids.to(device)
+        mask = attention_mask.to(device)
+        lab = labels.to(device) if labels is not None else None
+
+        # 1) Student forward to get hidden states
+        student_outputs = self.student.model.decoder(
+            input_ids=ids,
+            attention_mask=mask,
+            output_hidden_states=True,
+            return_dict=True
+        )
+        student_hiddens = student_outputs.hidden_states  # tuple of layer outputs
+
+        # use top hidden for classification pipeline
+        top_hidden = student_hiddens[-1]
+        h = self.dropout(top_hidden)
+        if self.use_crf:
+            h, _ = self.bilstm(h)
+        logits = self.classifier(h)
+
+        loss = None
+        if lab is not None:
+            # span loss
+            if self.use_crf:
+                m = mask.clone().bool()
+                m[:, 0] = True
+                tags = lab.clone()
+                tags[tags == -100] = 0
+                span_loss = -self.crf(logits, tags, mask=m, reduction='mean')
+            else:
+                span_loss = nn.CrossEntropyLoss(ignore_index=-100)(
+                    logits.view(-1, 2), lab.view(-1)
+                )
+
+            # KD loss: multi-depth
+            kd_loss = 0.0
+            if teacher_input_ids is not None and teacher_attention_mask is not None:
+                # Teacher on CPU
+                with torch.no_grad():
+                    tch_out = self.teacher(
+                        input_ids=teacher_input_ids,
+                        attention_mask=teacher_attention_mask,
+                        output_hidden_states=True,
+                        return_dict=True
+                    )
+                teacher_hiddens = tch_out.hidden_states
+                # compute layer-wise MSE
+                for i in self.kd_layers:
+                    s_feat = student_hiddens[i]
+                    t_feat = teacher_hiddens[i]
+                    # interpolate or project to same size if needed
+                    kd_loss += F.mse_loss(s_feat, t_feat.to(device))
+                kd_loss = kd_loss / len(self.kd_layers)
+                loss = self.alpha_span * span_loss + self.alpha_kd * kd_loss
+            else:
+                loss = span_loss
+
+        return {'loss': loss, 'logits': logits}
+
+    def predict(self, input_ids, attention_mask):
+        self.eval()
+        with torch.no_grad():
+            out = self.forward(input_ids, attention_mask)
+        logits = out['logits']
+        if self.use_crf:
+            m = attention_mask.bool().to(next(self.student.parameters()).device)
+            return self.crf.decode(F.log_softmax(logits, dim=-1), mask=m)
+        return logits.argmax(dim=-1)

vitosa_speech_ii-0.0.1/vitosa_speech_II/utils.py

@@ -0,0 +1,44 @@
+import torch
+
+def group_tokens_into_words_corrected(predicted_ids, toxic_labels, tokenizer):
+    """
+    Group token IDs to (decode)
+    """
+    words_with_labels = []
+    
+    if isinstance(predicted_ids, torch.Tensor):
+        predicted_ids = predicted_ids.tolist()
+    if isinstance(toxic_labels, torch.Tensor):
+        toxic_labels = toxic_labels.tolist()
+    
+    raw_tokens = tokenizer.convert_ids_to_tokens(predicted_ids)
+    
+    current_word_ids = []
+    current_label = -1 
+
+    min_len = min(len(predicted_ids), len(toxic_labels))
+    
+    for i in range(min_len):
+        token_id = predicted_ids[i]
+        label = toxic_labels[i]
+        raw_token = raw_tokens[i]
+        
+        if raw_token.startswith('Ġ') or i == 0:
+            
+            if current_word_ids:
+                decoded_word = tokenizer.decode(current_word_ids).strip()
+                if decoded_word: 
+                    words_with_labels.append((decoded_word, current_label))
+            
+            current_word_ids = [token_id]
+            current_label = label
+        else:
+            current_word_ids.append(token_id)
+            current_label = max(current_label, label)
+
+    if current_word_ids:
+        decoded_word = tokenizer.decode(current_word_ids).strip()
+        if decoded_word:
+            words_with_labels.append((decoded_word, current_label))
+        
+    return words_with_labels

vitosa_speech_ii-0.0.1/vitosa_speech_II.egg-info/PKG-INFO

@@ -0,0 +1,146 @@
+Metadata-Version: 2.4
+Name: vitosa-speech-II
+Version: 0.0.1
+Summary: A library for Robust Vietnamese Audio-Based Toxic Span Detection and Censoring
+Author: Vy Le-Phuong Huynh, Huy Ba Do and Luan Thanh Nguyen
+Author-email: luannt@uit.edu.vn
+Project-URL: Model (Hugging Face), https://huggingface.co/UIT-ViToSA/PhoWhisper-BiLSTM-CRF
+Keywords: audio-processing,toxic-span-detection,vietnamese,asr,speech-recognition,censoring,phowhisper
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
+Classifier: Natural Language :: Vietnamese
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
+Requires-Dist: torch>=1.13.0
+Requires-Dist: transformers>=4.28.0
+Requires-Dist: librosa
+Requires-Dist: pydub
+Requires-Dist: huggingface_hub
+Requires-Dist: pytorch-crf
+Requires-Dist: numpy
+Requires-Dist: tqdm
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: keywords
+Dynamic: project-url
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# ViToSA 2.0: A MULTI-TASK APPROACH TOWARDS ROBUST VIETNAMESE AUDIO-BASED TOXIC SPAN DETECTION | ICASSP 2026
+
+**Official implementation** of the paper:  
+**“A Multi-Task Approach Towards Robust Vietnamese Audio-Based Toxic Span Detection”** (ICASSP 2026).
+
+This package provides an end-to-end pipeline for **Vietnamese speech-based toxic span detection**, combining **ASR and toxic span detection** in a unified model. It also supports **automatic audio censoring**, replacing toxic spans with beep sounds in the output waveform.
+
+---
+
+## Key Features
+
+* **Automated Audio Censoring**: Takes an input audio file containing toxic language and outputs a **clean `.wav` file** where profanity is masked with a beep.
+* **Unified Multi-Task Architecture**: Integrates ASR and Toxic Span Detection (TSD) into a single model for high speed.
+* **SOTA Performance**: Achieves **F1-macro 0.9212** on the ViToSA-v2 dataset using **PhoWhisper + BiLSTM-CRF + Knowledge Distillation**.
+* **High Efficiency**: Reduces inference latency by over **56%** compared to traditional pipelines.
+
+## Installation
+
+```bash
+pip install vitosa-speech
+```
+
+### System requirements
+
+This package relies on `pydub` for audio processing, which requires **ffmpeg** to be installed.
+
+- **Ubuntu / Debian**
+  ```bash
+  sudo apt-get install ffmpeg
+  ```
+
+- **macOS (Homebrew)**
+  ```bash
+  brew install ffmpeg
+  ```
+
+- **Windows**
+  Download ffmpeg from https://ffmpeg.org and add it to your system `PATH`.
+
+---
+
+
+## Quick Start
+This library allows you to input a raw audio file and get a censored audio file as the output.
+
+### 1. Load the Model
+```python
+The model is pre-trained on the ViToSA-v2 dataset
+
+import torch
+from vitosa-speech-II import load_my_model
+# Automatically detect device (CUDA/CPU)
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+# Load the pre-trained model
+model, processor = load_my_model(device)
+```
+
+### 2. Run Inference (Detect & Censor)
+```python
+from vitosa-speech-II import return_labels, censor_audio_with_beep
+from IPython.display import Audio, display # Optional: to play in notebook
+
+# Path to your input file
+input_audio = "samples/toxic_speech.wav"
+
+# Step 1: Detect toxic spans
+words_with_labels = return_labels(input_audio, model, processor, device)
+
+# Step 2: Generate Censored Audio
+# This function creates a new audio file with beeps over toxic words
+output_audio_path = censor_audio_with_beep(
+    audio_path=input_audio, 
+    model=model, 
+    processor=processor, 
+    words_with_labels=words_with_labels, 
+    device=device
+)
+
+# Result
+print(f"✅ Censored audio saved to: {output_audio_path}")
+
+# Optional: Play the result (if in Jupyter/Colab)
+# display(Audio(output_audio_path))
+``` 
+
+## Methodology
+Our system works in two steps:
+
+1. Detection: The multi-task model (PhoWhisper + BiLSTM-CRF) processes the audio to identify the exact start and end timestamps of toxic words.
+2. Censoring: We reconstruct the audio by keeping safe segments and generating a sine wave (beep) to overlay exactly where the toxic tokens occur, ensuring the rest of the sentence remains intelligible.
+
+<!-- ## Citation
+If you use this tool or our findings, please cite:
+```bibtex
+@inproceedings{huynh2026multitask,
+  title={A Multi-Task Approach Towards Robust Vietnamese Audio-Based Toxic Span Detection},
+  author={Huynh, Vy Le-Phuong and Do, Huy Ba and Nguyen, Luan Thanh},
+  booktitle={Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
+  year={2026}
+}
+``` -->
+
+## Contact
+For more information: luannt@uit.edu.vn
+

vitosa_speech_ii-0.0.1/vitosa_speech_II.egg-info/SOURCES.txt

@@ -0,0 +1,12 @@
+README.md
+setup.py
+vitosa_speech_II/__init__.py
+vitosa_speech_II/audio.py
+vitosa_speech_II/inference.py
+vitosa_speech_II/model.py
+vitosa_speech_II/utils.py
+vitosa_speech_II.egg-info/PKG-INFO
+vitosa_speech_II.egg-info/SOURCES.txt
+vitosa_speech_II.egg-info/dependency_links.txt
+vitosa_speech_II.egg-info/requires.txt
+vitosa_speech_II.egg-info/top_level.txt

vitosa_speech_ii-0.0.1/vitosa_speech_II.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

vitosa_speech_ii-0.0.1/vitosa_speech_II.egg-info/requires.txt

@@ -0,0 +1,8 @@
+torch>=1.13.0
+transformers>=4.28.0
+librosa
+pydub
+huggingface_hub
+pytorch-crf
+numpy
+tqdm

vitosa_speech_ii-0.0.1/vitosa_speech_II.egg-info/top_level.txt

@@ -0,0 +1 @@
+vitosa_speech_II