@happyvertical/smrt-voice 0.30.0

package/AGENTS.md

@@ -0,0 +1,18 @@
+# @happyvertical/smrt-voice
+
+TTS voice profiles with two creation modes: AI design or audio cloning. Word-level timing output for lip-sync.
+
+## Models
+
+- **VoiceProfile**: two mutually exclusive modes — `designPrompt` (AI-generated from description) XOR `sampleAssetId` (cloned from audio). Status: `pending → processing → ready/failed`. `voiceData` is opaque provider-specific storage. `defaultSpeed` (0.5-2.0), `defaultPitch` (-20 to 20 semitones).
+- **VoiceSample**: audio training data. `duration`, `transcription`, `quality` (low/medium/high), `sampleRate`, `format`. Validation: `meetsMinDuration` (≥3 sec), `isSuitableForCloning` (≥3 sec AND quality ≠ low).
+- **VoiceOutput** (extends Content): generated TTS audio. `sourceText`, `audioAssetId`, `wordTimings` array `[{word, start, end}]` in seconds for lip-sync. `audioMetadata` (sampleRate, format, channels, bitDepth, provider, model). Computed: `wordCount`, `wordsPerSecond`, `getWordAtTime(seconds)`.
+
+## Gotchas
+
+- **Default provider hardcoded**: 'qwen3-tts' — no provider abstraction layer
+- **Sample minimum not enforced in constructor**: 3-sec minimum documented but not validated on create
+- **WordTiming from external provider**: framework doesn't generate timings — populated by TTS service
+- **Status transitions not enforced**: can manually set status without triggering generation workflow
+- **voiceData is opaque**: `{ [key: string]: any }` — provider-specific, no schema
+- **Optional tenancy**: tenantId=null for global/default voices

package/CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md

package/LICENSE

@@ -0,0 +1,7 @@
+Copyright <2025> <Happy Vertical Corporation>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

package/README.md

@@ -0,0 +1,106 @@
+# @happyvertical/smrt-voice
+
+Voice profile management for AI-powered voice synthesis and cloning in the SMRT ecosystem. Manages voice profiles, cloning samples, and TTS output with word-level timing for lip-sync.
+
+## Installation
+
+```bash
+pnpm add @happyvertical/smrt-voice
+```
+
+## Usage
+
+```typescript
+import { VoiceProfile, VoiceSample, VoiceOutput } from '@happyvertical/smrt-voice';
+
+// Two mutually exclusive creation modes:
+
+// 1. Voice design -- AI generates voice from a natural language prompt
+const designed = new VoiceProfile({
+  name: 'News Anchor',
+  language: 'en-US',
+  gender: 'male',
+  designPrompt: 'Warm, authoritative male voice with clear enunciation',
+  defaultSpeed: 1.0,   // 0.5 - 2.0
+  defaultPitch: 0,     // -20 to 20 semitones
+});
+await designed.save();
+
+// 2. Voice cloning -- replicate voice from audio sample(s)
+const cloned = new VoiceProfile({
+  name: 'Custom Voice',
+  language: 'en-US',
+  sampleAssetId: 'asset-123',
+});
+await cloned.save();
+
+// Add training samples for cloning (minimum 3 seconds, quality != low)
+const sample = new VoiceSample({
+  voiceProfileId: cloned.id,
+  assetId: 'asset-456',
+  duration: 5.2,
+  transcription: 'Hello, this is a test recording for voice cloning.',
+  quality: 'high',
+  sampleRate: 48000,
+  format: 'wav',
+  isPrimary: true,
+});
+await sample.save();
+
+// TTS output with word-level timing for lip-sync
+const output = new VoiceOutput({
+  voiceProfileId: designed.id,
+  sourceText: 'Welcome to the evening news.',
+  audioAssetId: 'asset-789',
+  duration: 2.8,
+  wordTimings: [
+    { word: 'Welcome', start: 0.0, end: 0.4 },
+    { word: 'to', start: 0.4, end: 0.5 },
+    { word: 'the', start: 0.5, end: 0.6 },
+    { word: 'evening', start: 0.6, end: 1.0 },
+    { word: 'news', start: 1.0, end: 1.3 },
+  ],
+});
+// Look up which word is being spoken at a given timestamp
+output.getWordAtTime(0.7); // { word: 'evening', start: 0.6, end: 1.0 }
+```
+
+## API
+
+### Models
+
+| Export | Description |
+|--------|------------|
+| `VoiceProfile` | Voice identity with two modes: `designPrompt` (AI-generated) or `sampleAssetId` (cloned) |
+| `VoiceSample` | Audio training data for voice cloning with quality rating |
+| `VoiceOutput` | Generated TTS audio (extends Content) with word-level timing for lip-sync |
+
+### Types
+
+| Export | Description |
+|--------|------------|
+| `VoiceProfileStatus` | Lifecycle status: `pending`, `processing`, `ready`, `failed` |
+| `VoiceGender` | Gender classification: `male`, `female`, `neutral` |
+| `SampleQuality` | Audio quality rating: `low`, `medium`, `high` |
+| `WordTiming` | Per-word timing entry: `{ word, start, end }` (seconds) |
+| `VoiceOutputMetadata` | Audio metadata: sampleRate, format, channels, bitDepth, provider, model |
+| `VoiceProfileOptions` | Profile creation options |
+| `VoiceSampleOptions` | Sample creation options |
+| `VoiceOutputOptions` | Output creation options |
+
+### Key Computed Properties
+
+- `VoiceProfile.isCloned` / `isDesigned` -- which creation mode is active
+- `VoiceProfile.isReady` -- status equals `ready`
+- `VoiceSample.meetsMinDuration` -- duration >= 3 seconds
+- `VoiceSample.isSuitableForCloning` -- meets min duration AND quality != low
+- `VoiceOutput.wordCount` / `wordsPerSecond` -- computed from sourceText and duration
+- `VoiceOutput.getWordAtTime(seconds)` -- look up word being spoken at a timestamp
+
+## Dependencies
+
+- `@happyvertical/smrt-core` -- ORM and code generation
+- `@happyvertical/smrt-assets` -- base asset management
+- `@happyvertical/smrt-config` -- configuration loading
+- `@happyvertical/smrt-content` -- content models (VoiceOutput extends Content)
+- `@happyvertical/smrt-tenancy` -- multi-tenant scoping

package/dist/index.d.ts

@@ -0,0 +1,504 @@
+import { Content } from '@happyvertical/smrt-content';
+import { ContentOptions } from '@happyvertical/smrt-content';
+import { SmrtObject } from '@happyvertical/smrt-core';
+import { SmrtObjectOptions } from '@happyvertical/smrt-core';
+
+/**
+ * Audio sample quality rating
+ */
+export declare type SampleQuality = 'low' | 'medium' | 'high';
+
+/**
+ * Voice gender classification
+ */
+export declare type VoiceGender = 'male' | 'female' | 'neutral';
+
+/**
+ * Generated audio output from text-to-speech synthesis
+ *
+ * VoiceOutput extends Content to represent audio generated from
+ * text using a VoiceProfile. It includes word-level timing information
+ * for lip-sync alignment in video production.
+ *
+ * @example
+ * ```typescript
+ * import { VoiceOutput } from '@happyvertical/smrt-voice';
+ *
+ * const output = new VoiceOutput({
+ *   voiceProfileId: 'voice-123',
+ *   sourceText: 'Welcome to the evening news broadcast.',
+ *   audioAssetId: 'asset-789',
+ *   duration: 3.5,
+ *   wordTimings: [
+ *     { word: 'Welcome', start: 0.0, end: 0.4 },
+ *     { word: 'to', start: 0.4, end: 0.5 },
+ *     { word: 'the', start: 0.5, end: 0.6 },
+ *     { word: 'evening', start: 0.6, end: 1.0 },
+ *     { word: 'news', start: 1.0, end: 1.3 },
+ *     { word: 'broadcast', start: 1.3, end: 1.9 },
+ *   ],
+ *   audioMetadata: {
+ *     sampleRate: 48000,
+ *     format: 'wav',
+ *     channels: 1,
+ *     provider: 'qwen3-tts',
+ *   },
+ * });
+ * ```
+ */
+export declare class VoiceOutput extends Content {
+    /**
+     * Voice profile used for synthesis
+     */
+    voiceProfileId: string | null;
+    /**
+     * Original text that was synthesized
+     */
+    sourceText: string;
+    /**
+     * Asset ID of the generated audio file
+     */
+    audioAssetId: string | null;
+    /**
+     * Duration of the generated audio in seconds
+     */
+    duration: number;
+    /**
+     * Word-level timing information for lip-sync alignment
+     */
+    wordTimings: WordTiming[] | null;
+    /**
+     * Audio metadata (sample rate, format, etc.)
+     */
+    audioMetadata: VoiceOutputMetadata;
+    constructor(options?: VoiceOutputOptions);
+    /**
+     * Get the word count of the source text
+     */
+    get wordCount(): number;
+    /**
+     * Get the average words per second rate
+     */
+    get wordsPerSecond(): number;
+    /**
+     * Check if word timing data is available for lip-sync
+     */
+    get hasWordTimings(): boolean;
+    /**
+     * Get the word at a specific timestamp
+     */
+    getWordAtTime(seconds: number): WordTiming | null;
+}
+
+/**
+ * Voice output metadata
+ */
+export declare interface VoiceOutputMetadata {
+    /**
+     * Sample rate in Hz
+     */
+    sampleRate?: number;
+    /**
+     * Audio format (e.g., 'wav', 'mp3', 'ogg')
+     */
+    format?: string;
+    /**
+     * Number of audio channels
+     */
+    channels?: number;
+    /**
+     * Bit depth (e.g., 16, 24, 32)
+     */
+    bitDepth?: number;
+    /**
+     * File size in bytes
+     */
+    fileSize?: number;
+    /**
+     * TTS provider used
+     */
+    provider?: string;
+    /**
+     * Model used for synthesis
+     */
+    model?: string;
+    /**
+     * Speech speed used (1.0 = normal)
+     */
+    speed?: number;
+    /**
+     * Pitch adjustment used (semitones)
+     */
+    pitch?: number;
+}
+
+/**
+ * Voice output creation options
+ */
+export declare interface VoiceOutputOptions extends ContentOptions {
+    /**
+     * Voice profile used for synthesis
+     */
+    voiceProfileId?: string | null;
+    /**
+     * Original text that was synthesized
+     */
+    sourceText?: string;
+    /**
+     * Asset ID of the generated audio file
+     */
+    audioAssetId?: string | null;
+    /**
+     * Duration of the generated audio in seconds
+     */
+    duration?: number;
+    /**
+     * Word-level timing information for lip-sync
+     */
+    wordTimings?: WordTiming[] | null;
+    /**
+     * Audio metadata
+     */
+    audioMetadata?: VoiceOutputMetadata;
+}
+
+/**
+ * Voice profile for AI-powered speech synthesis
+ *
+ * VoiceProfile represents a configured voice identity that can be used
+ * for text-to-speech synthesis. Voices can be created through:
+ * - Voice design: Natural language description of desired voice characteristics
+ * - Voice cloning: 3+ second audio sample for voice replication
+ *
+ * @example
+ * ```typescript
+ * import { VoiceProfile } from '@happyvertical/smrt-voice';
+ *
+ * // Create a designed voice
+ * const anchorVoice = new VoiceProfile({
+ *   name: 'News Anchor',
+ *   description: 'Professional news anchor voice with clear enunciation',
+ *   language: 'en-US',
+ *   gender: 'male',
+ *   designPrompt: 'Warm, authoritative male voice with slight gravitas, suitable for news broadcasts',
+ *   provider: 'qwen3-tts',
+ * });
+ *
+ * // Create a cloned voice
+ * const clonedVoice = new VoiceProfile({
+ *   name: 'Custom Voice',
+ *   description: 'Cloned from user sample',
+ *   language: 'en-US',
+ *   sampleAssetId: 'asset-123',
+ *   provider: 'qwen3-tts',
+ * });
+ * ```
+ */
+export declare class VoiceProfile extends SmrtObject {
+    /**
+     * Tenant ID for multi-tenant isolation
+     * Nullable to support global/default voices
+     */
+    tenantId: string | null;
+    /**
+     * Human-readable name for the voice profile
+     */
+    name: string;
+    /**
+     * Description of the voice characteristics
+     */
+    description: string | null;
+    /**
+     * ISO language code (e.g., 'en-US', 'zh-CN')
+     */
+    language: string;
+    /**
+     * Voice gender classification
+     */
+    gender: VoiceGender;
+    /**
+     * Natural language description for voice design
+     * Used when creating a voice from scratch via AI
+     */
+    designPrompt: string | null;
+    /**
+     * Asset ID of the audio sample for voice cloning
+     * Should be at least 3 seconds of clear speech
+     */
+    sampleAssetId: string | null;
+    /**
+     * Provider-specific voice data (ID, embedding, etc.)
+     * Stored after voice creation/cloning is complete
+     */
+    voiceData: Record<string, any> | null;
+    /**
+     * Default speech speed multiplier (0.5 - 2.0)
+     * 1.0 = normal speed
+     */
+    defaultSpeed: number;
+    /**
+     * Default pitch adjustment in semitones (-20 to 20)
+     * 0 = no adjustment
+     */
+    defaultPitch: number;
+    /**
+     * Voice profile status
+     * - pending: Profile created but voice not yet generated
+     * - processing: Voice generation/cloning in progress
+     * - ready: Voice is ready for use
+     * - failed: Voice generation failed
+     */
+    status: VoiceProfileStatus;
+    /**
+     * TTS provider that created/manages this voice
+     */
+    provider: string;
+    /**
+     * Error message if status is 'failed'
+     */
+    errorMessage: string | null;
+    constructor(options?: VoiceProfileOptions);
+    /**
+     * Check if this voice profile uses voice cloning
+     */
+    get isCloned(): boolean;
+    /**
+     * Check if this voice profile uses voice design
+     */
+    get isDesigned(): boolean;
+    /**
+     * Check if the voice is ready for use
+     */
+    get isReady(): boolean;
+    /**
+     * Check if this is a global (default) voice
+     */
+    get isGlobal(): boolean;
+}
+
+/**
+ * Voice profile creation options
+ */
+export declare interface VoiceProfileOptions extends SmrtObjectOptions {
+    /**
+     * Human-readable name for the voice profile
+     */
+    name?: string;
+    /**
+     * Description of the voice characteristics
+     */
+    description?: string | null;
+    /**
+     * ISO language code (e.g., 'en-US', 'zh-CN')
+     */
+    language?: string;
+    /**
+     * Voice gender classification
+     * @default 'neutral'
+     */
+    gender?: VoiceGender;
+    /**
+     * Natural language description for voice design
+     * Used when creating a voice from scratch
+     */
+    designPrompt?: string | null;
+    /**
+     * Asset ID of the audio sample for voice cloning
+     * Should be at least 3 seconds of clear speech
+     */
+    sampleAssetId?: string | null;
+    /**
+     * Provider-specific voice data (ID, embedding, etc.)
+     * Stored after voice creation/cloning
+     */
+    voiceData?: Record<string, any> | null;
+    /**
+     * Default speech speed multiplier
+     * @default 1.0
+     */
+    defaultSpeed?: number;
+    /**
+     * Default pitch adjustment in semitones
+     * @default 0
+     */
+    defaultPitch?: number;
+    /**
+     * Voice profile status
+     * @default 'pending'
+     */
+    status?: VoiceProfileStatus;
+    /**
+     * TTS provider that created this voice
+     * @default 'qwen3-tts'
+     */
+    provider?: string;
+    /**
+     * Error message if status is 'failed'
+     */
+    errorMessage?: string | null;
+    /**
+     * Tenant ID for multi-tenant isolation
+     * Null for global/default voices
+     */
+    tenantId?: string | null;
+}
+
+/**
+ * Voice profile status
+ */
+export declare type VoiceProfileStatus = 'pending' | 'processing' | 'ready' | 'failed';
+
+/**
+ * Audio sample for voice cloning
+ *
+ * VoiceSample represents an audio recording used as source material
+ * for voice cloning. For best results, samples should be:
+ * - At least 3 seconds long
+ * - Clear speech without background noise
+ * - Single speaker only
+ * - High quality (44.1kHz or higher)
+ *
+ * Multiple samples can be associated with a single VoiceProfile
+ * to improve voice cloning quality.
+ *
+ * @example
+ * ```typescript
+ * import { VoiceSample } from '@happyvertical/smrt-voice';
+ *
+ * const sample = new VoiceSample({
+ *   voiceProfileId: 'voice-123',
+ *   assetId: 'asset-456',
+ *   duration: 5.2,
+ *   transcription: 'Hello, this is a test recording for voice cloning.',
+ *   quality: 'high',
+ *   sampleRate: 48000,
+ *   channels: 1,
+ *   format: 'wav',
+ *   isPrimary: true,
+ * });
+ * ```
+ */
+export declare class VoiceSample extends SmrtObject {
+    /**
+     * Tenant ID for multi-tenant isolation
+     */
+    tenantId: string | null;
+    /**
+     * Voice profile this sample belongs to
+     */
+    voiceProfileId: string | null;
+    /**
+     * Asset ID of the audio file
+     * References an Asset in smrt-assets
+     */
+    assetId: string | null;
+    /**
+     * Sample duration in seconds
+     */
+    duration: number;
+    /**
+     * Transcription of what was said in the sample
+     * Used for alignment and quality verification
+     */
+    transcription: string | null;
+    /**
+     * Quality rating based on audio analysis
+     * - low: Noisy or short samples
+     * - medium: Acceptable quality
+     * - high: Clear audio, good length
+     */
+    quality: SampleQuality;
+    /**
+     * Sample rate in Hz
+     */
+    sampleRate: number | null;
+    /**
+     * Number of audio channels
+     */
+    channels: number | null;
+    /**
+     * Audio format
+     */
+    format: string | null;
+    /**
+     * Whether this is the primary sample for the voice profile
+     */
+    isPrimary: boolean;
+    constructor(options?: VoiceSampleOptions);
+    /**
+     * Check if sample meets minimum duration requirement (3 seconds)
+     */
+    get meetsMinDuration(): boolean;
+    /**
+     * Check if sample is high quality and suitable for cloning
+     */
+    get isSuitableForCloning(): boolean;
+}
+
+/**
+ * Voice sample creation options
+ */
+export declare interface VoiceSampleOptions extends SmrtObjectOptions {
+    /**
+     * Voice profile this sample belongs to
+     */
+    voiceProfileId?: string | null;
+    /**
+     * Asset ID of the audio file
+     */
+    assetId?: string | null;
+    /**
+     * Sample duration in seconds
+     */
+    duration?: number;
+    /**
+     * Transcription of what was said in the sample
+     */
+    transcription?: string | null;
+    /**
+     * Quality rating based on audio analysis
+     * @default 'medium'
+     */
+    quality?: SampleQuality;
+    /**
+     * Sample rate in Hz (e.g., 44100, 48000)
+     */
+    sampleRate?: number | null;
+    /**
+     * Number of audio channels (1 = mono, 2 = stereo)
+     */
+    channels?: number | null;
+    /**
+     * Audio format (e.g., 'wav', 'mp3', 'ogg')
+     */
+    format?: string | null;
+    /**
+     * Whether this is the primary sample for the voice profile
+     * @default false
+     */
+    isPrimary?: boolean;
+    /**
+     * Tenant ID for multi-tenant isolation
+     */
+    tenantId?: string | null;
+}
+
+/**
+ * Word timing information for lip-sync alignment
+ */
+export declare interface WordTiming {
+    /**
+     * The word
+     */
+    word: string;
+    /**
+     * Start time in seconds
+     */
+    start: number;
+    /**
+     * End time in seconds
+     */
+    end: number;
+}
+
+export { }