praasper 0.1.0.dev1__tar.gz

praasper-0.1.0.dev1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Tony Liu
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

praasper-0.1.0.dev1/PKG-INFO

@@ -0,0 +1,73 @@
+Metadata-Version: 2.4
+Name: praasper
+Version: 0.1.0.dev1
+Summary: VAD-Enhanced ASR with Word- and Phoneme-Level Timestamps
+Home-page: https://github.com/ParadeLuxe/Praasper
+Author: Tony Liu
+Author-email: paradeluxe3726@gmail.com
+License: MIT
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: openai-whisper
+Requires-Dist: textgrid
+Requires-Dist: pypinyin
+Requires-Dist: librosa
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# Praasper
+![Python](https://img.shields.io/badge/python-3.10-blue.svg)
+![GitHub License](https://img.shields.io/github/license/Paradeluxe/Praasper)
+
+
+**Praasper** is an Automatic Speech Recognition (ASR) application designed help researchers transribe audio files to both **word-** and **phoneme-level** text.
+
+![mechanism](promote/mechanism.png)
+
+
+
+# Mechanism
+In **Praasper**, we adopt a rather simple and straightforward pipeline to extract phoneme-level information from audio files.
+
+**Whisper** ([repo](https://github.com/openai/whisper)) is used to transcribe the audio file to **word-level text**. At this point, speech onsets and offsets exhibit time deviations in seconds.
+
+```Python
+model = whisper.load_model("large-v3-turbo", device="cuda")
+result = model.transcribe(wav, word_timestamps=True)
+```
+
+[**Praditor**](https://github.com/Paradeluxe/Praditor) is applied to perform **Voice Activity Detection (VAD)** algorithm to trim the currently existing word/character-level timestamps (at millisecond level). It is a Speech Onset Detection (SOT) algorithm we developed for langauge researchers.
+
+To extract phoneme boundaries, we designed an **edge detection algorithm**. 
+- The audio file is first resampled to **16 kHz** as to remove noise in the high-frequency domain. 
+- A kernel,`[-1, 0, 1]`, is then applied to the frequency domain to enhance the edge(s) between phonetic segments.
+- The most prominent **n** peaks are then selected so as to match the wanted number of phonemes.
+
+# Support
+
+| Precision | Completed  | Developing  |
+| :---: | :---: | :---: |
+| Word  | Mandarin  |  Cantonese, English |
+|  Phoneme |  Mandarin |  Cantonese, English |
+
+# Setup
+
+## pip installation
+
+```bash
+pip install praasper
+```
+
+`uv` is also highly recommended

praasper-0.1.0.dev1/README.md

@@ -0,0 +1,44 @@
+# Praasper
+![Python](https://img.shields.io/badge/python-3.10-blue.svg)
+![GitHub License](https://img.shields.io/github/license/Paradeluxe/Praasper)
+
+
+**Praasper** is an Automatic Speech Recognition (ASR) application designed help researchers transribe audio files to both **word-** and **phoneme-level** text.
+
+![mechanism](promote/mechanism.png)
+
+
+
+# Mechanism
+In **Praasper**, we adopt a rather simple and straightforward pipeline to extract phoneme-level information from audio files.
+
+**Whisper** ([repo](https://github.com/openai/whisper)) is used to transcribe the audio file to **word-level text**. At this point, speech onsets and offsets exhibit time deviations in seconds.
+
+```Python
+model = whisper.load_model("large-v3-turbo", device="cuda")
+result = model.transcribe(wav, word_timestamps=True)
+```
+
+[**Praditor**](https://github.com/Paradeluxe/Praditor) is applied to perform **Voice Activity Detection (VAD)** algorithm to trim the currently existing word/character-level timestamps (at millisecond level). It is a Speech Onset Detection (SOT) algorithm we developed for langauge researchers.
+
+To extract phoneme boundaries, we designed an **edge detection algorithm**. 
+- The audio file is first resampled to **16 kHz** as to remove noise in the high-frequency domain. 
+- A kernel,`[-1, 0, 1]`, is then applied to the frequency domain to enhance the edge(s) between phonetic segments.
+- The most prominent **n** peaks are then selected so as to match the wanted number of phonemes.
+
+# Support
+
+| Precision | Completed  | Developing  |
+| :---: | :---: | :---: |
+| Word  | Mandarin  |  Cantonese, English |
+|  Phoneme |  Mandarin |  Cantonese, English |
+
+# Setup
+
+## pip installation
+
+```bash
+pip install praasper
+```
+
+`uv` is also highly recommended

praasper-0.1.0.dev1/praasper/process.py

@@ -0,0 +1,272 @@
+import os
+import whisper
+from textgrid import TextGrid, IntervalTier
+import librosa
+import numpy as np
+from scipy.signal import convolve2d, find_peaks
+from tool import *
+
+
+# defs
+def transcribe_wav_file(wav, vad):
+    """
+    使用 Whisper 模型转录 .wav 文件
+    
+    :param file_path: .wav 文件的路径
+    :param path_vad: VAD TextGrid 文件的路径
+    :return: 转录结果
+    """
+    # 加载最佳模型（large-v3）并指定使用设备
+    # device = "cuda" if torch.cuda.is_available() else "cpu"
+    model = whisper.load_model("large-v3-turbo")
+    print(f"Model loaded successfully. Current device in use: {model.device if hasattr(model, 'device') else 'Unknown'}")
+    
+    # 转录音频文件
+    result = model.transcribe(wav, word_timestamps=True)
+    language = result["language"]
+
+    print(result)
+
+
+    # 加载 path_vad 对应的 TextGrid 文件
+    try:
+        vad_tg = TextGrid.fromFile(vad)
+    except FileNotFoundError:
+        print(f"错误：未找到文件 {vad}")
+        raise
+
+    # 提取所有 mark 为空字符串的 interval 的起止时间
+    vad_intervals = []
+    empty_mark_intervals = []
+    for tier in vad_tg:
+        for interval in tier:
+            if interval.mark == "":
+                empty_mark_intervals.append((interval.minTime, interval.maxTime))
+            else:
+                vad_intervals.append((interval.minTime, interval.maxTime))
+
+
+
+    tg = TextGrid()
+    tier = IntervalTier(name='word', minTime=0.0, maxTime=vad_tg.tiers[0].maxTime)
+    
+    for segment in result["segments"]:
+        for idx, word in enumerate(segment["words"]):
+            start_time = word["start"]
+            end_time = word["end"]
+            
+            text = word["word"]
+
+            for empty_mark_interval in empty_mark_intervals:
+                if empty_mark_interval[0] <= end_time <= empty_mark_interval[1]:
+                    end_time = empty_mark_interval[0]
+                
+                if empty_mark_interval[0] <= start_time <= empty_mark_interval[1]:
+                    start_time = empty_mark_interval[1]
+                
+                if start_time < empty_mark_interval[0] < empty_mark_interval[1] < end_time:
+                    pass
+
+            print(start_time, end_time, text)
+            tier.add(start_time, end_time, text)
+
+    for vad_interval in vad_intervals:
+        # 找到距离 vad_interval[0] 最近的 interval.minTime
+        closest_interval = min(tier.intervals, key=lambda x: abs(x.minTime - vad_interval[0]))
+
+        if closest_interval.minTime - vad_interval[0] != 0:
+            closest_interval.minTime = vad_interval[0]
+
+        # 找到距离 vad_interval[1] 最近的 interval.maxTime
+        closest_interval = min(tier.intervals, key=lambda x: abs(x.maxTime - vad_interval[1]))
+
+        if closest_interval.maxTime - vad_interval[1] != 0:
+            closest_interval.maxTime = vad_interval[1]
+
+
+    tg.append(tier)
+    tg.write(wav.replace(".wav", "_whisper.TextGrid"))
+
+
+def word_timestamp(wav, tg_path):
+
+    # 加载音频文件
+    y, sr = librosa.load(wav, sr=16000)
+
+    # 创建一个新的IntervalTier
+    max_time = librosa.core.get_duration(y=y, sr=sr)
+    print(max_time)
+
+    # 加载 TextGrid 文件
+    tg = TextGrid.fromFile(tg_path)
+    word_tier = [tier for tier in tg if tier.name == 'word'][0]
+
+    # 计算 tg 的 segment 中 mark 不为空的 interval 的平均时长
+    non_empty_intervals = [interval.maxTime - interval.minTime for tier in tg for interval in tier if interval.mark != ""]
+    average_word_duration = np.mean(non_empty_intervals) if non_empty_intervals else 0
+    print(f"Speech rate (word dur) is {average_word_duration:.4f} seconds")
+
+    # adjacent_pairs = []
+    
+    word_intervals = [interval for interval in word_tier.intervals if interval.mark != ""]
+    for i in range(len(word_intervals) - 1):
+        current_interval = word_intervals[i]
+        next_interval = word_intervals[i + 1]
+        # 检查两个 interval 是否相粘着（前一个的结束时间等于后一个的开始时间）
+
+        if current_interval.maxTime == next_interval.minTime:
+            target_boundary = current_interval.maxTime - current_interval.minTime
+
+            start_sample = int(current_interval.minTime * sr)
+            end_sample = int(next_interval.maxTime * sr)
+            y_vad = y[start_sample:end_sample]
+
+            # 计算频谱图
+            spectrogram = librosa.stft(y_vad, n_fft=2048, win_length=1024, center=True)
+            spectrogram_db = librosa.amplitude_to_db(abs(spectrogram), ref=1.0)  # 使用librosa.amplitude_to_db已将y值转换为对数刻度，top_db=None确保不限制最大分贝值
+            
+            kernel = np.array([[-1, 0, 1]])
+            convolved_spectrogram = convolve2d(spectrogram_db, kernel, mode='same', boundary='symm')
+            convolved_spectrogram = np.where(np.abs(convolved_spectrogram) < 15, 0, convolved_spectrogram)
+
+            # 按频率轴求和，保持维度以方便后续绘图
+            convolved_spectrogram = np.sum(np.abs(convolved_spectrogram), axis=0, keepdims=False)
+            # 在保持输出信号长度不变的情况下，对卷积后的频谱图求一阶导
+            # convolved_spectrogram = np.gradient(convolved_spectrogram)
+            time_axis = np.linspace(0, len(convolved_spectrogram) * librosa.core.get_duration(y=y_vad, sr=sr) / len(convolved_spectrogram), len(convolved_spectrogram))
+
+            # 找到所有的波峰和波谷
+            peaks, _ = find_peaks(convolved_spectrogram, prominence=(10, None))
+            valleys, _ = find_peaks(-convolved_spectrogram, prominence=(10, None))
+
+
+            # 只保留波峰和波谷绝对值大于100的点
+            valid_peaks = peaks[np.abs(convolved_spectrogram[peaks]) > 0]
+            valid_valleys = valleys[np.abs(convolved_spectrogram[valleys]) > 0]
+
+            # 提取有效波峰和波谷对应的时间和值
+            peak_times = time_axis[valid_peaks]
+            peak_values = convolved_spectrogram[valid_peaks]
+
+            valley_times = time_axis[valid_valleys]
+            valley_values = convolved_spectrogram[valid_valleys]    
+
+            # 筛选出不在 current_interval.minTime 到 current_interval.minTime + 0.05s 之间的波峰
+            valid_peak_times = [t for t in peak_times if t >= 0.05 and (target_boundary -  average_word_duration/2 <= t <= target_boundary + average_word_duration * 3/4)]
+
+            if valid_peak_times:
+                # 找到距离 target_boundary 最近且最大的波峰
+                # 获取波峰对应的数值
+                peak_values_nearby = [convolved_spectrogram[int((t / librosa.core.get_duration(y=y_vad, sr=sr)) * len(convolved_spectrogram))] for t in valid_peak_times]
+                # 找到最大波峰对应的时间
+                closest_peak_time = valid_peak_times[np.argmax(peak_values_nearby)]
+            else:
+                closest_peak_time = target_boundary
+            
+            # 找到之后，开始写入
+            target_boundary = closest_peak_time + current_interval.minTime
+
+            current_interval.maxTime = target_boundary
+            next_interval.minTime = target_boundary
+    
+
+    phon_tier = IntervalTier(name="phoneme", minTime=0, maxTime=word_tier.maxTime)
+
+    for interval in word_intervals:
+        print()
+        con, vow, tone = get_pinyin_info(interval.mark)
+        expected_num = len(vow) + 1 if con else len(vow)
+        phon_series = [con] + vow if con else vow
+        print(expected_num)
+
+
+
+        start_sample = int(interval.minTime * sr)
+        end_sample = int(interval.maxTime * sr)
+        print(interval.mark, interval.minTime, interval.maxTime)
+
+        y_vad = y[start_sample:end_sample]
+
+
+        # 计算频谱图
+        spectrogram = librosa.stft(y_vad, n_fft=2048, win_length=1024, center=True)
+        spectrogram_db = librosa.amplitude_to_db(abs(spectrogram), ref=1.0)  # 使用librosa.amplitude_to_db已将y值转换为对数刻度，top_db=None确保不限制最大分贝值
+        
+        kernel = np.array([[-1, 0, 1]])
+        convolved_spectrogram = convolve2d(spectrogram_db, kernel, mode='same', boundary='symm')
+        convolved_spectrogram = np.where(np.abs(convolved_spectrogram) < 20, 0, convolved_spectrogram)
+
+        # 按频率轴求和，保持维度以方便后续绘图
+        convolved_spectrogram = np.sum(np.abs(convolved_spectrogram), axis=0, keepdims=False)
+        # 在保持输出信号长度不变的情况下，对卷积后的频谱图求一阶导
+        # convolved_spectrogram = np.gradient(convolved_spectrogram)
+        time_axis = np.linspace(0, len(convolved_spectrogram) * librosa.core.get_duration(y=y_vad, sr=sr) / len(convolved_spectrogram), len(convolved_spectrogram))
+
+        # 找到所有峰值，指定最小峰值高度为 0，后续再筛选最大的前几个
+        peaks, _ = find_peaks(convolved_spectrogram)
+
+        if con in ["k", 'b', 't', 'p', 'd']:
+            valid_peaks = [p for p in peaks if time_axis[p] <= len(y_vad)/sr - 0.05]
+
+            # if "i" == vow[0]:
+            #     vow = vow[0] + vow
+            #     expected_num -= 1
+            #     phon_series = [con] + vow if con else vow
+        
+        else:
+            # 忽略掉所有头0.05s和后0.05s的peak
+            valid_peaks = [p for p in peaks if time_axis[p] >= 0.05 and time_axis[p] <= len(y_vad)/sr - 0.05]
+        
+        peaks = np.array(valid_peaks)
+
+        # 按峰值大小对峰值索引进行排序
+        sorted_peaks = sorted(peaks, key=lambda x: convolved_spectrogram[x], reverse=True)
+        # 假设前 5 个峰值最大，可根据实际需求修改数量
+        peaks = sorted_peaks[:expected_num-1]
+        
+        # 获取波峰对应的时间戳
+        peak_times = time_axis[peaks]
+
+        peak_timestamps = [interval.minTime] + [pt + interval.minTime for pt in peak_times] + [interval.maxTime]
+
+        peak_timestamps.sort()
+
+        print(peak_timestamps)
+        for t, time_stamp in enumerate(peak_timestamps):
+            if t == 0:
+                continue
+            phon_tier.add(peak_timestamps[t-1], peak_timestamps[t], phon_series[t-1])
+    print(tg.maxTime)
+    tg.append(phon_tier)
+
+
+    # 保存修改后的 TextGrid 文件
+    new_tg_path = os.path.join("output", os.path.basename(wav).replace(".wav", ".TextGrid"))
+    tg.write(new_tg_path)
+
+
+
+
+
+
+if __name__ == "__main__":
+    data_path = os.path.abspath("data")
+    # input_dir = os.path.abspath("input")
+    # output_dir = os.path.abspath("output")
+
+    fnames = [os.path.splitext(f)[0] for f in os.listdir(data_path) if f.endswith('.wav')]
+
+
+    for fname in fnames:
+        wav_path = os.path.join(data_path, fname + ".wav")
+        tg_path = wav_path.replace(".wav", "_whisper.TextGrid")
+        vad_path = wav_path.replace(".wav", "_VAD.TextGrid")
+
+        transcribe_wav_file(wav_path, vad=vad_path)
+        word_timestamp(wav_path, tg_path)
+
+
+
+
+
+

praasper-0.1.0.dev1/praasper/tool.py

@@ -0,0 +1,54 @@
+import pypinyin
+
+def get_pinyin_info(character):
+    """
+    给定一个中文单字，返回其对应的拼音（声母、韵母、声调）
+    
+    :param character: 单个中文字符
+    :return: 包含声母、韵母、声调的字典
+    """
+    if len(character) != 1:
+        raise ValueError("只能输入单个中文字符")
+    
+    # 获取拼音信息
+    pinyin_result = pypinyin.pinyin(character, style=pypinyin.TONE3, heteronym=False)[0][0]
+    
+    # 提取声母、韵母和声调
+    initial = pypinyin.pinyin(character, style=pypinyin.INITIALS, heteronym=False)[0][0]
+    final_with_tone = pypinyin.pinyin(character, style=pypinyin.FINALS_TONE3, heteronym=False)[0][0]
+    
+    # 分离韵母和声调
+    tone = ''
+    final = final_with_tone
+    for char in final_with_tone:
+        if char.isdigit():
+            tone = char
+            final = final_with_tone.replace(char, '')
+            break
+    
+    
+
+    final = list(final)
+    # 如果n和g相邻，合并成ng
+    new_final = []
+    i = 0
+    while i < len(final):
+        if i < len(final) - 1 and final[i] == 'n' and final[i+1] == 'g':
+            new_final.append('ng')
+            i += 2
+        else:
+            new_final.append(final[i])
+            i += 1
+    final = new_final
+
+
+    
+    return (
+        initial if initial else '',
+        final,
+        tone if tone else ''
+    )
+
+
+if __name__ == '__main__':
+    print(get_pinyin_info('边'))

praasper-0.1.0.dev1/praasper.egg-info/PKG-INFO

@@ -0,0 +1,73 @@
+Metadata-Version: 2.4
+Name: praasper
+Version: 0.1.0.dev1
+Summary: VAD-Enhanced ASR with Word- and Phoneme-Level Timestamps
+Home-page: https://github.com/ParadeLuxe/Praasper
+Author: Tony Liu
+Author-email: paradeluxe3726@gmail.com
+License: MIT
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: openai-whisper
+Requires-Dist: textgrid
+Requires-Dist: pypinyin
+Requires-Dist: librosa
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# Praasper
+![Python](https://img.shields.io/badge/python-3.10-blue.svg)
+![GitHub License](https://img.shields.io/github/license/Paradeluxe/Praasper)
+
+
+**Praasper** is an Automatic Speech Recognition (ASR) application designed help researchers transribe audio files to both **word-** and **phoneme-level** text.
+
+![mechanism](promote/mechanism.png)
+
+
+
+# Mechanism
+In **Praasper**, we adopt a rather simple and straightforward pipeline to extract phoneme-level information from audio files.
+
+**Whisper** ([repo](https://github.com/openai/whisper)) is used to transcribe the audio file to **word-level text**. At this point, speech onsets and offsets exhibit time deviations in seconds.
+
+```Python
+model = whisper.load_model("large-v3-turbo", device="cuda")
+result = model.transcribe(wav, word_timestamps=True)
+```
+
+[**Praditor**](https://github.com/Paradeluxe/Praditor) is applied to perform **Voice Activity Detection (VAD)** algorithm to trim the currently existing word/character-level timestamps (at millisecond level). It is a Speech Onset Detection (SOT) algorithm we developed for langauge researchers.
+
+To extract phoneme boundaries, we designed an **edge detection algorithm**. 
+- The audio file is first resampled to **16 kHz** as to remove noise in the high-frequency domain. 
+- A kernel,`[-1, 0, 1]`, is then applied to the frequency domain to enhance the edge(s) between phonetic segments.
+- The most prominent **n** peaks are then selected so as to match the wanted number of phonemes.
+
+# Support
+
+| Precision | Completed  | Developing  |
+| :---: | :---: | :---: |
+| Word  | Mandarin  |  Cantonese, English |
+|  Phoneme |  Mandarin |  Cantonese, English |
+
+# Setup
+
+## pip installation
+
+```bash
+pip install praasper
+```
+
+`uv` is also highly recommended

praasper-0.1.0.dev1/praasper.egg-info/SOURCES.txt

@@ -0,0 +1,11 @@
+LICENSE
+README.md
+setup.py
+praasper/__init__.py
+praasper/process.py
+praasper/tool.py
+praasper.egg-info/PKG-INFO
+praasper.egg-info/SOURCES.txt
+praasper.egg-info/dependency_links.txt
+praasper.egg-info/requires.txt
+praasper.egg-info/top_level.txt

praasper-0.1.0.dev1/praasper.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

praasper-0.1.0.dev1/praasper.egg-info/requires.txt

@@ -0,0 +1,4 @@
+openai-whisper
+textgrid
+pypinyin
+librosa

praasper-0.1.0.dev1/praasper.egg-info/top_level.txt

@@ -0,0 +1 @@
+praasper

praasper-0.1.0.dev1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

praasper-0.1.0.dev1/setup.py

@@ -0,0 +1,25 @@
+from setuptools import setup, find_packages
+
+setup(
+    name='praasper',      # 在PyPI上唯一的包名
+    version='0.1.0.dev1',              # 版本号，遵循语义化版本规范
+    description='VAD-Enhanced ASR with Word- and Phoneme-Level Timestamps',
+    long_description=open('README.md').read(),
+    long_description_content_type='text/markdown',
+    author='Tony Liu',
+    author_email='paradeluxe3726@gmail.com',
+    url='https://github.com/ParadeLuxe/Praasper',
+    packages=find_packages(),     # 自动查找包
+    install_requires=[            # 你的项目所依赖的第三方库
+        'openai-whisper',
+        'textgrid',
+        'pypinyin',
+        'librosa',
+    ],
+    license='MIT',
+    classifiers=[
+        'Programming Language :: Python :: 3',
+        'Operating System :: OS Independent',
+    ],
+    python_requires='>=3.8',      # 指定Python版本要求
+)