hyper-animator-codex 0.1.0 → 0.2.0

package/README.md

@@ -43,6 +43,24 @@ npx hyper-animator-codex install --target /path/to/codex/skills
 - `skills/hyper-animator-codex/references/`: HyperFrames catalog map, workflow guide, pseudocode, and request examples.
 - `skills/hyper-animator-codex/scripts/validate_hyperframes_html.py`: static pre-render HTML quality gate.
 
+## Optional Beat Detection
+
+The skill can analyze background music and guide Codex to align HyperFrames/GSAP transitions to beats, bars, energy peaks, and detected music segments.
+
+Install optional Python dependencies:
+
+```bash
+python3 -m pip install librosa pydub numpy click
+```
+
+Analyze WAV/FLAC/OGG without ffmpeg. MP3/M4A/AAC/WMA require system `ffmpeg`.
+
+Run the bundled analyzer from an installed skill directory or repository checkout:
+
+```bash
+python3 skills/hyper-animator-codex/scripts/analyze_music_beats.py path/to/music.wav -o beat-map.json --fps 60 --pretty
+```
+
 ## Development
 
 ```bash

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "hyper-animator-codex",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "description": "Install the Hyper Animator Codex skill for Codex.",
   "type": "module",
   "bin": {

package/skills/hyper-animator-codex/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: hyper-animator-codex
-description: Use when a user asks Codex to create, plan, author, customize, validate, preview, or render a HyperFrames, HTML, or GSAP animation/video from natural-language requirements, including product demos, code demos, data videos, podcast captions, social shorts, catalog block/component assembly, or new HyperFrames HTML.
+description: Use when a user asks Codex to create, plan, author, customize, validate, preview, or render a HyperFrames, HTML, or GSAP animation/video from natural-language requirements, including product demos, code demos, data videos, podcast captions, social shorts, catalog assembly, new HyperFrames HTML, sound effects, background music, rhythm, BPM, or beat-synced transitions.
 ---
 
 # Hyper Animator Codex
@@ -18,11 +18,13 @@ Turn a natural-language animation or video brief into a validated HyperFrames HT
 5. Decide generation mode:
    - `generate_new_hyperframes_html` when the user asks to write HTML, create a new effect, customize style, match a brand, use complex animation, or when component snippets are only paste placeholders.
    - `assemble_existing_catalog_items` only when the user asks to use existing catalog items or quickly compose installed blocks/components.
-6. Ask the second clarification round with candidate context: visual direction, motion rhythm, generation mode, and any candidate tradeoffs.
-7. Write or assemble HTML.
-8. Run pre-render quality gates.
-9. Show a concise plan summary and preview path or HTML file to the user. Ask for confirmation before video render.
-10. Render only after user confirmation, then report output path and any caveats.
+6. Clarify audio: ask whether to add animation/transition sound effects and whether to add background music.
+7. If background music is used, read `references/beat-sync-workflow.md`, ask for the local audio path, and run `scripts/analyze_music_beats.py` when the file is available.
+8. Ask the second clarification round with candidate context: visual direction, motion rhythm, generation mode, audio choices, and any candidate tradeoffs.
+9. Write or assemble HTML. When beat-sync is enabled, align major reveals, cuts, transitions, camera moves, and visual accents to the beat map instead of arbitrary timestamps.
+10. Run pre-render quality gates.
+11. Show a concise plan summary and preview path or HTML file to the user. Ask for confirmation before video render.
+12. Render only after user confirmation, then report output path and any caveats.
 
 ## Interactive Questions
 
@@ -30,8 +32,8 @@ Prefer Codex interactive question tools such as `AskUserQuestion` or `request_us
 
 Use two rounds:
 
-- Round 1: purpose, format, duration, platform, required content, brand assets, and whether video render is expected in this turn.
-- Round 2: after catalog scoring, ask about style, motion, candidate selection, and generation mode.
+- Round 1: purpose, format, duration, platform, required content, brand assets, sound effects, background music, and whether video render is expected in this turn.
+- Round 2: after catalog scoring, ask about style, motion, candidate selection, generation mode, and beat-sync assumptions when background music is present.
 
 Do not ask everything upfront when the brief is already specific. Ask only for missing or decision-changing information.
 
@@ -40,6 +42,7 @@ Do not ask everything upfront when the brief is already specific. Ask only for m
 - Read `references/hyperframes-intent-workflow.md` for the full AskUserQuestion workflow, generation-mode rules, scoring model, and render confirmation requirements.
 - Read `references/hyperframes-catalog-map.json` whenever selecting catalog candidates or determining visual references.
 - Read `references/hyperframes-agent-pseudocode.ts` when implementing the end-to-end loop or when the correct sequence is ambiguous.
+- Read `references/beat-sync-workflow.md` when sound effects, background music, soundtrack, beat sync, rhythm, BPM, audio-reactive animation, or transition timing to music is mentioned.
 - Use `references/examples/*.json` for sanity checks against common request shapes.
 
 ## Catalog Rules
@@ -79,6 +82,8 @@ Before rendering, summarize:
 
 - generation mode;
 - selected or referenced catalog items;
+- sound effects and background music choices;
+- beat map path, BPM, duration, and timing assumptions when beat-sync is enabled;
 - dimensions and duration;
 - content assumptions;
 - preview location;

package/skills/hyper-animator-codex/references/beat-sync-workflow.md

@@ -0,0 +1,51 @@
+# Beat Sync Workflow
+
+Use this when the user mentions sound effects, background music, soundtrack, beat sync, rhythm, BPM, transitions to music, or audio-reactive animation.
+
+## Clarify Audio
+
+Ask:
+
+- Should animation or transition sound effects be added?
+- Should background music be added?
+- If background music is used, what is the local audio path?
+- May scene timing, transition timing, and animation accents be adjusted to match the music?
+- Should the music be trimmed, looped, or should video duration follow the selected music region?
+
+## Analyze Music
+
+When a background music file is provided:
+
+```bash
+python3 scripts/analyze_music_beats.py path/to/music.wav -o path/to/beat-map.json --fps 60 --pretty
+```
+
+If dependencies are missing, tell the user:
+
+```bash
+python3 -m pip install librosa pydub numpy click
+```
+
+MP3/M4A/AAC/WMA require system `ffmpeg`; WAV/FLAC/OGG are safer.
+
+## Map Beats To Animation
+
+- Use `meta.bpm` for global pacing.
+- Use beats with `beat_in_bar: 1` for major scene changes, title reveals, camera moves, and transitions.
+- Use `energy_level: strong` for visual accents, scale pulses, light sweeps, flashes, or cut emphasis.
+- Use weak beats for secondary motion only.
+- Use `structure.segments` to shape intro, main body, and outro.
+- Use `structure.energy_peaks` for high-impact moments.
+- Keep `data-duration` consistent with the selected music region or state trim/loop assumptions.
+
+## GSAP Timing
+
+Convert beat times from milliseconds to seconds:
+
+```js
+const beatSeconds = beatMap.beats.map((beat) => beat.time_ms / 1000);
+tl.addLabel("downbeat_1", beatSeconds[0]);
+tl.to(".hero", { opacity: 1, y: 0, duration: 0.35 }, "downbeat_1");
+```
+
+Do not use wall-clock audio playback to drive render progress. The render timeline remains a paused GSAP timeline registered in `window.__timelines`.

package/skills/hyper-animator-codex/scripts/analyze_music_beats.py

@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+sys.dont_write_bytecode = True
+
+SCRIPT_DIR = Path(__file__).resolve().parent
+VENDOR_ROOT = SCRIPT_DIR.parent / "vendor" / "music-beat-detector"
+sys.path.insert(0, str(VENDOR_ROOT))
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description="Analyze background music and write a beat map JSON for HyperFrames timing."
+    )
+    parser.add_argument("audio_file", help="Path to WAV, FLAC, OGG, MP3, M4A, AAC, or WMA audio")
+    parser.add_argument("-o", "--output", help="Output JSON path. Prints JSON to stdout when omitted.")
+    parser.add_argument("--fps", type=int, default=60, help="Timeline frame rate for frame indexes. Default: 60")
+    parser.add_argument("--pretty", action="store_true", help="Pretty-print JSON")
+    parser.add_argument(
+        "--log-level",
+        default="error",
+        choices=["debug", "info", "warning", "error"],
+        help="Detector log level",
+    )
+    return parser.parse_args()
+
+
+def main() -> int:
+    args = parse_args()
+    audio_path = Path(args.audio_file)
+    if not audio_path.exists():
+        print(f"Error: audio file does not exist: {audio_path}", file=sys.stderr)
+        return 1
+
+    try:
+        from beat_detector import analyze
+        from beat_detector.errors import BeatDetectorError, FFmpegRequiredError
+    except ModuleNotFoundError as exc:
+        print(
+            "Error: missing Python dependency for beat detection. "
+            "Install optional dependencies with: python3 -m pip install librosa pydub numpy click",
+            file=sys.stderr,
+        )
+        print(f"Missing module: {exc.name}", file=sys.stderr)
+        return 10
+
+    try:
+        result = analyze(str(audio_path), fps=args.fps, log_level=args.log_level)
+        if args.output:
+            result.save(args.output, pretty=args.pretty)
+            print(f"Beat map saved to: {args.output}")
+        else:
+            print(result.to_json(pretty=args.pretty))
+        return 0
+    except FFmpegRequiredError as exc:
+        print(f"Error: {exc}", file=sys.stderr)
+        print("Install ffmpeg or use WAV/FLAC/OGG audio for fewer runtime dependencies.", file=sys.stderr)
+        return 4
+    except BeatDetectorError as exc:
+        message = getattr(exc, "message", str(exc))
+        print(f"Error: {message}", file=sys.stderr)
+        return 3
+    except ModuleNotFoundError as exc:
+        print(
+            "Error: missing Python dependency for beat detection. "
+            "Install optional dependencies with: python3 -m pip install librosa pydub numpy click",
+            file=sys.stderr,
+        )
+        print(f"Missing module: {exc.name}", file=sys.stderr)
+        return 10
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

package/skills/hyper-animator-codex/vendor/music-beat-detector/README.md

@@ -0,0 +1,13 @@
+# Vendored music-beat-detector
+
+Trimmed source from `git@github.com:realpkuasule/music-beat-detector.git`, commit `29b081fbe3bb38f0fa8cb569fa3150d7cfdb18cb`.
+
+Only `beat_detector/` is vendored. Sample media, ffmpeg archives, development contracts, and upstream tests are intentionally omitted from the npm package.
+
+Optional runtime dependencies:
+
+```bash
+python3 -m pip install librosa pydub numpy click
+```
+
+MP3/M4A/AAC/WMA files require system `ffmpeg`; WAV/FLAC/OGG are preferred for fewer dependencies.

package/skills/hyper-animator-codex/vendor/music-beat-detector/beat_detector/__init__.py

@@ -0,0 +1,33 @@
+"""Music Beat Detector - 音乐自动打点工具"""
+
+from .analyzer import analyze, AnalysisResult, MetaInfo
+from .beat import Beat, BeatResult, EnergyLevel
+from .structure import Segment, EnergyPeak, SilenceRegion, StructureResult
+from .errors import BeatDetectorError, FileNotFoundError, UnsupportedFormatError, AnalysisError, FFmpegRequiredError
+
+__version__ = "0.1.0"
+
+__all__ = [
+    # 主要 API
+    'analyze',
+    'AnalysisResult',
+    'MetaInfo',
+
+    # 节拍检测
+    'Beat',
+    'BeatResult',
+    'EnergyLevel',
+
+    # 结构检测
+    'StructureResult',
+    'Segment',
+    'EnergyPeak',
+    'SilenceRegion',
+
+    # 异常
+    'BeatDetectorError',
+    'FileNotFoundError',
+    'UnsupportedFormatError',
+    'FFmpegRequiredError',
+    'AnalysisError',
+]

package/skills/hyper-animator-codex/vendor/music-beat-detector/beat_detector/analyzer.py

@@ -0,0 +1,129 @@
+"""分析器模块"""
+from dataclasses import dataclass, field, asdict
+from typing import List, Callable, Optional, Any
+import json
+import logging
+import numpy as np
+
+from .beat import Beat, BeatResult, detect_beats
+from .structure import StructureResult, detect_structure
+from .utils import load_audio, setup_logging
+from .errors import (
+    FileNotFoundError as BeatDetectorFileNotFoundError,
+    FFmpegRequiredError,
+    UnsupportedFormatError
+)
+
+
+@dataclass
+class MetaInfo:
+    """元信息"""
+    file: str
+    duration_ms: int
+    sample_rate: int
+    bpm: float
+    time_signature: str = "4/4"
+
+
+@dataclass
+class AnalysisResult:
+    """完整分析结果"""
+    meta: MetaInfo
+    beats: List[Beat] = field(default_factory=list)
+    structure: Optional[StructureResult] = None
+
+    def to_dict(self) -> dict:
+        """转换为字典"""
+        result = {
+            'meta': asdict(self.meta),
+            'beats': [
+                {
+                    'time_ms': beat.time_ms,
+                    'frame': beat.frame,
+                    'beat_in_bar': beat.beat_in_bar,
+                    'energy_level': beat.energy_level.value
+                }
+                for beat in self.beats
+            ],
+            'structure': {
+                'segments': [asdict(seg) for seg in self.structure.segments],
+                'energy_peaks': [asdict(peak) for peak in self.structure.energy_peaks],
+                'silence_regions': [asdict(region) for region in self.structure.silence_regions],
+            }
+        }
+        return result
+
+    def to_json(self, pretty: bool = False) -> str:
+        """导出为 JSON"""
+        indent = 2 if pretty else None
+        return json.dumps(self.to_dict(), indent=indent, ensure_ascii=False)
+
+    def save(self, path: str, pretty: bool = False) -> None:
+        """保存到文件"""
+        with open(path, 'w', encoding='utf-8') as f:
+            f.write(self.to_json(pretty=pretty))
+
+
+def analyze(
+    file_path: str,
+    fps: int = 30,
+    log_level: str = "info",
+    on_progress: Optional[Callable[[float], None]] = None
+) -> AnalysisResult:
+    """分析音频文件"""
+
+    # 设置日志
+    setup_logging(log_level)
+    logger = logging.getLogger(__name__)
+
+    # 进度辅助函数
+    def report_progress(stage_progress: float):
+        if on_progress:
+            on_progress(stage_progress)
+
+    # 加载音频
+    logger.info(f"Loading audio file: {file_path}")
+    try:
+        y, sr = load_audio(file_path)
+    except (BeatDetectorFileNotFoundError, FFmpegRequiredError, UnsupportedFormatError):
+        raise
+    except Exception as e:
+        raise BeatDetectorFileNotFoundError(file_path)
+
+    report_progress(10.0)
+
+    # 计算时长
+    duration_ms = int(len(y) / sr * 1000)
+    logger.info(f"Audio duration: {duration_ms/1000:.1f}s, sample rate: {sr}Hz")
+
+    # 节拍检测
+    logger.info("Detecting beats...")
+    beat_result = detect_beats(y, sr, fps=fps, on_progress=lambda p: report_progress(10 + p * 0.45))
+    logger.info(f"Detected BPM: {beat_result.bpm:.1f}")
+
+    report_progress(55.0)
+
+    # 结构检测
+    logger.info("Analyzing structure...")
+    structure_result = detect_structure(y, sr, fps=fps, on_progress=lambda p: report_progress(55 + p * 0.45))
+    logger.info(f"Found {len(structure_result.segments)} segments")
+
+    report_progress(100.0)
+
+    # 构建结果
+    meta = MetaInfo(
+        file=file_path,
+        duration_ms=duration_ms,
+        sample_rate=sr,
+        bpm=beat_result.bpm,
+        time_signature=beat_result.time_signature
+    )
+
+    result = AnalysisResult(
+        meta=meta,
+        beats=beat_result.beats,
+        structure=structure_result
+    )
+
+    logger.info("Analysis complete")
+    return result

package/skills/hyper-animator-codex/vendor/music-beat-detector/beat_detector/beat.py

@@ -0,0 +1,133 @@
+"""节拍检测模块"""
+from dataclasses import dataclass
+from typing import List, Callable, Optional
+from enum import Enum
+import numpy as np
+import librosa
+
+from .utils import ms_to_frame
+
+
+class EnergyLevel(Enum):
+    """节拍能量级别"""
+    WEAK = "weak"
+    MEDIUM = "medium"
+    STRONG = "strong"
+
+
+@dataclass
+class Beat:
+    """单个节拍"""
+    time_ms: int
+    frame: int
+    beat_in_bar: int
+    energy_level: EnergyLevel = EnergyLevel.MEDIUM
+
+
+@dataclass
+class BeatResult:
+    """节拍检测结果"""
+    bpm: float
+    beats: List[Beat]
+    time_signature: str = "4/4"
+
+
+def detect_beats(
+    y: np.ndarray,
+    sr: int,
+    fps: int = 30,
+    on_progress: Optional[Callable[[float], None]] = None
+) -> BeatResult:
+    """检测节拍"""
+
+    if on_progress:
+        on_progress(10.0)
+
+    # 使用 librosa 检测节拍
+    tempo, beat_frames = librosa.beat.beat_track(y=y, sr=sr)
+    bpm = float(np.asarray(tempo).reshape(-1)[0])
+
+    if on_progress:
+        on_progress(40.0)
+
+    # 转换为时间（毫秒）
+    beat_times = librosa.frames_to_time(beat_frames, sr=sr)
+    beat_times_ms = [int(t * 1000) for t in beat_times]
+
+    # 计算每个beat的能量
+    beat_energies = _compute_beat_energies(y, sr, beat_frames)
+
+    if on_progress:
+        on_progress(70.0)
+
+    # 根据能量分布分级
+    energy_levels = _classify_energy_levels(beat_energies)
+
+    if on_progress:
+        on_progress(80.0)
+
+    # 构建 Beat 列表
+    beats = []
+    beats_per_bar = 4  # 假设 4/4 拍
+    for i, time_ms in enumerate(beat_times_ms):
+        frame = ms_to_frame(time_ms, fps)
+        beat_in_bar = (i % beats_per_bar) + 1
+        beats.append(Beat(
+            time_ms=time_ms,
+            frame=frame,
+            beat_in_bar=beat_in_bar,
+            energy_level=energy_levels[i]
+        ))
+
+    if on_progress:
+        on_progress(100.0)
+
+    return BeatResult(bpm=bpm, beats=beats)
+
+
+def _compute_beat_energies(
+    y: np.ndarray,
+    sr: int,
+    beat_frames: np.ndarray,
+    window_ms: int = 50
+) -> np.ndarray:
+    """计算每个beat时刻的局部能量"""
+    window_samples = int(sr * window_ms / 1000)
+    energies = []
+
+    for frame in beat_frames:
+        # 将librosa帧转换为样本索引
+        sample = librosa.frames_to_samples(frame)
+        start = max(0, sample - window_samples // 2)
+        end = min(len(y), sample + window_samples // 2)
+
+        if end > start:
+            # 使用RMS作为能量度量
+            segment = y[start:end]
+            rms = np.sqrt(np.mean(segment ** 2))
+            energies.append(rms)
+        else:
+            energies.append(0.0)
+
+    return np.array(energies)
+
+
+def _classify_energy_levels(energies: np.ndarray) -> List[EnergyLevel]:
+    """根据能量分布分类为弱/中/强三档"""
+    if len(energies) == 0:
+        return []
+
+    # 使用百分位数划分阈值
+    weak_threshold = np.percentile(energies, 33)
+    strong_threshold = np.percentile(energies, 66)
+
+    levels = []
+    for energy in energies:
+        if energy <= weak_threshold:
+            levels.append(EnergyLevel.WEAK)
+        elif energy >= strong_threshold:
+            levels.append(EnergyLevel.STRONG)
+        else:
+            levels.append(EnergyLevel.MEDIUM)
+
+    return levels

package/skills/hyper-animator-codex/vendor/music-beat-detector/beat_detector/cli.py

@@ -0,0 +1,74 @@
+"""命令行接口"""
+import json
+import sys
+import click
+
+from .analyzer import analyze
+from .errors import (
+    BeatDetectorError,
+    FileNotFoundError,
+    UnsupportedFormatError,
+    AnalysisError,
+    FFmpegRequiredError
+)
+
+
+@click.command()
+@click.argument('input_file', type=click.Path(exists=False))
+@click.option('-o', '--output', type=click.Path(), help='Output file path')
+@click.option('--fps', default=30, help='Frame rate (default: 30)')
+@click.option('--log-level', default='info',
+              type=click.Choice(['debug', 'info', 'warning', 'error']),
+              help='Log level (default: info)')
+@click.option('--pretty', is_flag=True, help='Pretty print JSON')
+def main(input_file: str, output: str, fps: int, log_level: str, pretty: bool) -> None:
+    """
+    Music beat detector CLI
+
+    Analyze audio file and output beat detection results in JSON format.
+
+    Supported formats (no extra dependencies):
+        .wav, .flac, .ogg
+
+    Formats requiring ffmpeg:
+        .mp3, .m4a, .aac, .wma
+
+    Usage:
+        beat-detector input.mp3 -o output.json
+        beat-detector input.wav --pretty
+    """
+    try:
+        result = analyze(input_file, fps=fps, log_level=log_level)
+        json_output = result.to_json(pretty=pretty)
+
+        if output:
+            result.save(output, pretty=pretty)
+            click.echo(f"Output saved to: {output}")
+        else:
+            click.echo(json_output)
+
+        sys.exit(0)
+
+    except FileNotFoundError as e:
+        click.echo(f"Error: {e.message}", err=True)
+        sys.exit(1)
+
+    except UnsupportedFormatError as e:
+        click.echo(f"Error: {e.message}", err=True)
+        sys.exit(2)
+
+    except FFmpegRequiredError as e:
+        click.echo(f"Error: {e.message}", err=True)
+        sys.exit(4)
+
+    except AnalysisError as e:
+        click.echo(f"Error: {e.message}", err=True)
+        sys.exit(3)
+
+    except Exception as e:
+        click.echo(f"Unexpected error: {str(e)}", err=True)
+        sys.exit(3)
+
+
+if __name__ == '__main__':
+    main()

package/skills/hyper-animator-codex/vendor/music-beat-detector/beat_detector/errors.py

@@ -0,0 +1,49 @@
+"""错误类型"""
+from typing import Optional
+
+
+class BeatDetectorError(Exception):
+    """基础异常类"""
+
+    def __init__(self, message: str, code: int = 1):
+        self.message = message
+        self.code = code
+        super().__init__(self.message)
+
+
+class FileNotFoundError(BeatDetectorError):
+    """文件不存在"""
+
+    def __init__(self, path: str):
+        super().__init__(f"File not found: {path}", code=1)
+
+
+class UnsupportedFormatError(BeatDetectorError):
+    """不支持的音频格式"""
+
+    def __init__(self, format: str):
+        super().__init__(f"Unsupported audio format: {format}", code=2)
+
+
+class FFmpegRequiredError(BeatDetectorError):
+    """需要安装 ffmpeg"""
+
+    def __init__(self, format: str):
+        message = (
+            f"Format '{format}' requires ffmpeg. "
+            f"Please install ffmpeg:\n"
+            f"  Ubuntu/Debian: sudo apt install ffmpeg\n"
+            f"  macOS: brew install ffmpeg\n"
+            f"  Windows: choco install ffmpeg\n"
+            f"\n"
+            f"Or use native formats: .wav, .flac, .ogg"
+        )
+        super().__init__(message, code=4)
+
+
+class AnalysisError(BeatDetectorError):
+    """分析失败"""
+
+    def __init__(self, message: str, cause: Optional[Exception] = None):
+        self.cause = cause
+        super().__init__(message, code=3)

package/skills/hyper-animator-codex/vendor/music-beat-detector/beat_detector/structure.py

@@ -0,0 +1,171 @@
+"""结构检测模块"""
+from dataclasses import dataclass
+from typing import List, Callable, Optional
+import numpy as np
+import librosa
+
+from .utils import ms_to_frame
+
+
+@dataclass
+class Segment:
+    """音乐段落"""
+    type: str
+    start_ms: int
+    end_ms: int
+    start_frame: int
+    end_frame: int
+    confidence: float
+
+
+@dataclass
+class EnergyPeak:
+    """能量高峰"""
+    time_ms: int
+    frame: int
+    intensity: float
+
+
+@dataclass
+class SilenceRegion:
+    """静音区域"""
+    start_ms: int
+    end_ms: int
+    start_frame: int
+    end_frame: int
+
+
+@dataclass
+class StructureResult:
+    """结构检测结果"""
+    segments: List[Segment]
+    energy_peaks: List[EnergyPeak]
+    silence_regions: List[SilenceRegion]
+
+
+def detect_structure(
+    y: np.ndarray,
+    sr: int,
+    fps: int = 30,
+    on_progress: Optional[Callable[[float], None]] = None
+) -> StructureResult:
+    """检测音乐结构"""
+
+    if on_progress:
+        on_progress(10.0)
+
+    # 计算能量
+    rms = librosa.feature.rms(y=y)[0]
+    rms_frames = librosa.frames_to_time(range(len(rms)), sr=sr)
+
+    if on_progress:
+        on_progress(30.0)
+
+    # 检测能量高峰
+    energy_peaks = _detect_energy_peaks(rms, rms_frames, fps)
+
+    if on_progress:
+        on_progress(50.0)
+
+    # 检测静音区域
+    silence_regions = _detect_silence_regions(rms, rms_frames, fps, sr)
+
+    if on_progress:
+        on_progress(70.0)
+
+    # 简单的段落检测（基于能量变化）
+    segments = _detect_segments(y, sr, rms, rms_frames, fps)
+
+    if on_progress:
+        on_progress(100.0)
+
+    return StructureResult(
+        segments=segments,
+        energy_peaks=energy_peaks,
+        silence_regions=silence_regions
+    )
+
+
+def _detect_energy_peaks(rms: np.ndarray, times: np.ndarray, fps: int) -> List[EnergyPeak]:
+    """检测能量高峰"""
+    peaks = []
+    threshold = np.mean(rms) + np.std(rms)
+
+    for i in range(1, len(rms) - 1):
+        if rms[i] > threshold and rms[i] > rms[i-1] and rms[i] > rms[i+1]:
+            time_ms = int(times[i] * 1000)
+            frame = ms_to_frame(time_ms, fps)
+            intensity = float(rms[i] / np.max(rms)) if np.max(rms) > 0 else 0.0
+            peaks.append(EnergyPeak(
+                time_ms=time_ms,
+                frame=frame,
+                intensity=intensity
+            ))
+
+    return peaks
+
+
+def _detect_silence_regions(rms: np.ndarray, times: np.ndarray, fps: int, sr: int) -> List[SilenceRegion]:
+    """检测静音区域"""
+    regions = []
+    threshold = np.mean(rms) * 0.1  # 静音阈值
+    min_silence_duration = 0.5  # 最小静音时长（秒）
+
+    in_silence = False
+    silence_start = 0
+
+    for i, energy in enumerate(rms):
+        if energy < threshold:
+            if not in_silence:
+                in_silence = True
+                silence_start = i
+        else:
+            if in_silence:
+                silence_duration = times[i] - times[silence_start]
+                if silence_duration >= min_silence_duration:
+                    start_ms = int(times[silence_start] * 1000)
+                    end_ms = int(times[i] * 1000)
+                    regions.append(SilenceRegion(
+                        start_ms=start_ms,
+                        end_ms=end_ms,
+                        start_frame=ms_to_frame(start_ms, fps),
+                        end_frame=ms_to_frame(end_ms, fps)
+                    ))
+                in_silence = False
+
+    return regions
+
+
+def _detect_segments(y: np.ndarray, sr: int, rms: np.ndarray, times: np.ndarray, fps: int) -> List[Segment]:
+    """简单的段落检测"""
+    segments = []
+    duration_ms = int(len(y) / sr * 1000)
+
+    # 简单策略：将音频分成 intro, main, outro
+    num_sections = 3
+    section_duration = duration_ms // num_sections
+
+    # 计算每个部分的平均能量
+    rms_per_section = np.array_split(rms, num_sections)
+    avg_energies = [np.mean(s) for s in rms_per_section]
+
+    section_types = ['intro', 'verse', 'outro']
+
+    for i in range(num_sections):
+        start_ms = i * section_duration
+        end_ms = (i + 1) * section_duration if i < num_sections - 1 else duration_ms
+
+        # 根据相对能量计算置信度
+        max_energy = max(avg_energies) if max(avg_energies) > 0 else 1
+        confidence = float(avg_energies[i] / max_energy)
+
+        segments.append(Segment(
+            type=section_types[i],
+            start_ms=start_ms,
+            end_ms=end_ms,
+            start_frame=ms_to_frame(start_ms, fps),
+            end_frame=ms_to_frame(end_ms, fps),
+            confidence=confidence
+        ))
+
+    return segments

package/skills/hyper-animator-codex/vendor/music-beat-detector/beat_detector/utils.py

@@ -0,0 +1,73 @@
+"""工具函数"""
+from typing import Tuple
+import os
+import logging
+import numpy as np
+
+from .errors import FileNotFoundError, UnsupportedFormatError, FFmpegRequiredError
+
+# 原生支持的格式（无需额外依赖）
+NATIVE_FORMATS = {'.wav', '.flac', '.ogg'}
+
+# 需要 ffmpeg 的格式
+FFMPEG_FORMATS = {'.mp3', '.m4a', '.aac', '.wma'}
+
+# 所有支持的格式
+SUPPORTED_FORMATS = NATIVE_FORMATS | FFMPEG_FORMATS
+
+
+def ms_to_frame(ms: int, fps: int) -> int:
+    """毫秒转帧号"""
+    return int(ms * fps / 1000)
+
+
+def frame_to_ms(frame: int, fps: int) -> int:
+    """帧号转毫秒"""
+    return int(frame * 1000 / fps)
+
+
+def check_ffmpeg_available() -> bool:
+    """检查 ffmpeg 是否可用"""
+    import shutil
+    return shutil.which('ffmpeg') is not None
+
+
+def load_audio(file_path: str) -> Tuple[np.ndarray, int]:
+    """加载音频文件
+
+    原生支持: .wav, .flac, .ogg (无需额外依赖)
+    需要 ffmpeg: .mp3, .m4a, .aac, .wma
+    """
+    import librosa
+
+    # 检查文件是否存在
+    if not os.path.exists(file_path):
+        raise FileNotFoundError(file_path)
+
+    # 检查格式
+    ext = os.path.splitext(file_path)[1].lower()
+    if ext not in SUPPORTED_FORMATS:
+        raise UnsupportedFormatError(ext)
+
+    # 检查是否需要 ffmpeg
+    if ext in FFMPEG_FORMATS and not check_ffmpeg_available():
+        raise FFmpegRequiredError(ext)
+
+    # 加载音频
+    y, sr = librosa.load(file_path, sr=None)
+    return y, sr
+
+
+def setup_logging(level: str = "info") -> None:
+    """配置日志"""
+    level_map = {
+        "debug": logging.DEBUG,
+        "info": logging.INFO,
+        "warning": logging.WARNING,
+        "error": logging.ERROR,
+    }
+    logging.basicConfig(
+        level=level_map.get(level, logging.INFO),
+        format="[%(levelname)s] %(message)s",
+        force=True
+    )