@pie-players/tts-server-google 0.3.74 → 1.0.0-beta.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,24 +1,16 @@
1
1
  # @pie-players/tts-server-google
2
2
 
3
- Google Cloud Text-to-Speech provider for server-side text-to-speech with speech marks support.
4
-
5
- For the cross-package TTS architecture and browser/server flow, see
6
- [TTS Architecture](../../docs/accessibility/tts-architecture.md). This README
7
- focuses on Google Cloud provider configuration and behavior.
8
-
9
- ## Overview
10
-
11
- This package provides a server-side TTS provider that uses Google Cloud Text-to-Speech to generate high-quality neural speech with millisecond-precise word timing through SSML mark injection.
12
-
13
- ## Features
14
-
15
- - ✅ **Speech Marks Support** - Millisecond-accurate word timing via SSML marks + timepoints
16
- - ✅ **WaveNet Neural Voices** - High-quality neural TTS with Google's WaveNet technology
17
- - ✅ **50+ Languages** - Extensive language support
18
- - ✅ **Full SSML** - Supports Speech Synthesis Markup Language 1.1
19
- - ✅ **Single API Call** - Audio and speech marks in one request (more efficient than AWS Polly)
20
- - ✅ **200+ Voices** - Multiple voice types per language (Standard, WaveNet, Studio)
21
- - ✅ **Flexible Authentication** - Service account, API key, or Application Default Credentials
3
+ `GoogleCloudTTSProvider`, the Google Cloud Text-to-Speech provider for
4
+ server-side text-to-speech (TTS), built on `@pie-players/tts-server-core`. It
5
+ returns audio with word timings, which it gets by injecting SSML `<mark>` tags
6
+ and reading Google's timepoints; the timings drive read-aloud highlighting in
7
+ the browser. This README covers the provider's configuration and behavior for
8
+ developers writing a host's TTS routes; the
9
+ [integration guide](./examples/INTEGRATION-GUIDE.md) builds those routes, and
10
+ [TTS Architecture](../../docs/accessibility/tts-architecture.md) covers the
11
+ browser and server flow across packages.
12
+
13
+ The package runs on Node.js 20 or later.
22
14
 
23
15
  ## Installation
24
16
 
@@ -37,15 +29,23 @@ const provider = new GoogleCloudTTSProvider();
37
29
 
38
30
  await provider.initialize({
39
31
  projectId: 'my-gcp-project',
40
- credentials: '/path/to/service-account.json', // Or use other auth methods
41
- voiceType: 'wavenet', // 'wavenet', 'standard', or 'studio'
32
+ credentials: '/path/to/service-account.json', // or another authentication method
33
+ voiceType: 'wavenet', // 'wavenet', 'standard', or 'studio' (Studio returns no speech marks)
42
34
  defaultVoice: 'en-US-Wavenet-A',
43
35
  });
44
36
  ```
45
37
 
38
+ `initialize` validates the config and creates the Google client without calling
39
+ Google. A missing `projectId` throws a `TTSError` with `INITIALIZATION_ERROR`.
40
+
46
41
  ### Authentication Methods
47
42
 
48
- #### 1. Service Account JSON File (Recommended for Production)
43
+ The project needs the Cloud Text-to-Speech API enabled, with billing, as
44
+ Google's [setup page](https://docs.cloud.google.com/text-to-speech/docs/get-started)
45
+ describes. The provider reads no environment variables; `credentials` selects
46
+ the method.
47
+
48
+ #### Service Account Key File
49
49
 
50
50
  ```typescript
51
51
  await provider.initialize({
@@ -54,7 +54,9 @@ await provider.initialize({
54
54
  });
55
55
  ```
56
56
 
57
- #### 2. Service Account Object (For Containers/Serverless)
57
+ #### Service Account Key Object
58
+
59
+ For containers and serverless platforms that hold the key as a secret:
58
60
 
59
61
  ```typescript
60
62
  await provider.initialize({
@@ -66,7 +68,7 @@ await provider.initialize({
66
68
  });
67
69
  ```
68
70
 
69
- #### 3. API Key (For Simple Applications)
71
+ #### API Key
70
72
 
71
73
  ```typescript
72
74
  await provider.initialize({
@@ -77,10 +79,15 @@ await provider.initialize({
77
79
  });
78
80
  ```
79
81
 
80
- #### 4. Application Default Credentials (For Local Development)
82
+ #### Application Default Credentials
83
+
84
+ Omitting `credentials` uses
85
+ [Application Default Credentials](https://docs.cloud.google.com/docs/authentication/application-default-credentials):
86
+ `gcloud auth application-default login` locally, the file named by
87
+ `GOOGLE_APPLICATION_CREDENTIALS`, or the attached service account on Google
88
+ Cloud.
81
89
 
82
90
  ```typescript
83
- // Omit credentials to use gcloud auth application-default login
84
91
  await provider.initialize({
85
92
  projectId: 'my-project',
86
93
  });
@@ -91,13 +98,13 @@ await provider.initialize({
91
98
  ```typescript
92
99
  const result = await provider.synthesize({
93
100
  text: 'Hello world, this is a test of Google Cloud Text to Speech.',
94
- voice: 'en-US-Wavenet-A', // Optional, uses defaultVoice if not specified
101
+ voice: 'en-US-Wavenet-A', // optional; see Voice Selection
95
102
  includeSpeechMarks: true,
96
103
  });
97
104
 
98
105
  console.log('Audio:', result.audio); // Buffer
99
- console.log('Speech marks:', result.speechMarks); // Array of word timings
100
- console.log('Duration:', result.metadata.duration, 'seconds');
106
+ console.log('Speech marks:', result.speechMarks); // word timings
107
+ console.log('Synthesis time:', result.metadata.duration, 'seconds');
101
108
  ```
102
109
 
103
110
  ### List Available Voices
@@ -116,6 +123,10 @@ const femaleVoices = await provider.getVoices({ gender: 'female' });
116
123
  const neuralVoices = await provider.getVoices({ quality: 'neural' });
117
124
  ```
118
125
 
126
+ `getVoices` passes `language` to Google and filters by gender and quality
127
+ locally. Quality follows the voice name: `Wavenet` is `neural`, `Studio` is
128
+ `premium`, and every other voice is `standard`.
129
+
119
130
  ### Speech Marks Example
120
131
 
121
132
  ```typescript
@@ -147,6 +158,9 @@ const result = await provider.synthesize({
147
158
  });
148
159
  ```
149
160
 
161
+ The [TTS authoring guide](../../docs/accessibility/tts-authoring-guide.md#ssml-provider-support)
162
+ covers how much of authored SSML Google voices.
163
+
150
164
  ## Configuration
151
165
 
152
166
  ### GoogleCloudTTSConfig
@@ -154,84 +168,97 @@ const result = await provider.synthesize({
154
168
  ```typescript
155
169
  interface GoogleCloudTTSConfig {
156
170
  projectId: string; // Google Cloud project ID (required)
157
- credentials?: // Authentication (optional if using ADC)
158
- | string // Path to service account JSON
159
- | { // Service account object
171
+ credentials?: // omitted: Application Default Credentials
172
+ | string // Path to a service account key file
173
+ | { // Service account key object
160
174
  client_email: string;
161
175
  private_key: string;
176
+ project_id?: string;
162
177
  }
163
178
  | { apiKey: string }; // API key
164
- voiceType?: 'wavenet' | 'standard' | 'studio'; // Voice type (default: 'wavenet')
165
- defaultVoice?: string; // Default voice (default: 'en-US-Wavenet-A')
166
- audioEncoding?: 'MP3' | 'LINEAR16' | 'OGG_OPUS'; // Audio format (default: 'MP3')
167
- enableLogging?: boolean; // Debug logging (default: false)
179
+ voiceType?: 'wavenet' | 'standard' | 'studio'; // default: 'wavenet'
180
+ defaultVoice?: string; // default: 'en-US-Wavenet-A'
181
+ audioEncoding?: 'MP3' | 'LINEAR16' | 'OGG_OPUS'; // output encoding (default: 'MP3')
182
+ enableLogging?: boolean; // default: false
168
183
  }
169
184
  ```
170
185
 
171
- ### Environment Variables
172
-
173
- ```bash
174
- GOOGLE_CLOUD_PROJECT=my-project
175
- GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
176
- ```
186
+ ## Synthesis Behavior
187
+
188
+ - **Requests.** One SynthesizeSpeech call returns the audio and, with speech
189
+ marks requested, the timepoints of the injected marks.
190
+ - **Output.** `audioEncoding` fixes the output for the instance: `MP3` returns
191
+ `audio/mpeg`, `LINEAR16` `audio/wav`, `OGG_OPUS` `audio/ogg`. A request's
192
+ `format` is only validated against `mp3`, `wav` and `ogg`. `sampleRate`
193
+ defaults to 24000 Hz. The language code sent to Google comes from the voice
194
+ name.
195
+ - **Prosody.** Plain text with a `rate` or `pitch` other than 1 is escaped and
196
+ wrapped in `<prosody>`. Authored SSML goes to Google with its markup
197
+ unchanged, so `rate` and `pitch` apply to plain text only. A Studio voice
198
+ drops `pitch`.
199
+ - **Limits.** The provider accepts 5000 characters per request.
200
+ `ServerTTSProvider` declares 3000, and the toolkit splits longer reads to fit,
201
+ so requests from the browser stay under the provider's limit.
202
+
203
+ ### Voice Selection
204
+
205
+ A request's `voice` wins. Without one, the provider picks a voice for the
206
+ request's `language` from Google's voice listing, preferring voices of the
207
+ configured `voiceType`, and falls back to `defaultVoice`; the core README's
208
+ [voice resolution](../tts-server-core/README.md#voice-resolution) section has the
209
+ matching rules.
210
+
211
+ ### Speech Marks
212
+
213
+ Speech marks are on unless the request sets `includeSpeechMarks: false`, and are
214
+ word marks only.
215
+
216
+ 1. The provider places a `<mark>` before each word: a run of letters, combining
217
+ marks, digits and apostrophes, so accented words stay whole.
218
+ 2. Google returns a timepoint for each mark.
219
+ 3. The provider converts the timepoints to speech marks in milliseconds, sorted
220
+ by time.
221
+
222
+ Plain-text offsets are UTF-16 indexes into the request text. Authored SSML keeps
223
+ its markup and takes marks in its text nodes, so its offsets index the SSML.
224
+ Entities are never read as words, and an element whose content the engine
225
+ replaces (`<sub>`, `<say-as>`, `<phoneme>`) takes one mark before its opening
226
+ tag, valued with its text.
227
+
228
+ Studio voices take no `<mark>` tags. A voice whose name contains `Studio`, or
229
+ one Google answers with its Studio `<mark>`-unsupported error, is synthesized as
230
+ audio only and returns `speechMarks: []`, so `ServerTTSProvider` reports no word
231
+ boundaries for it.
177
232
 
178
233
  ## Capabilities
179
234
 
180
235
  | Feature | Support |
181
236
  |---------|---------|
182
- | Speech Marks | ✅ Via SSML marks |
183
- | SSML | ✅ Full 1.1 |
184
- | Pitch Control | ✅ SSML |
185
- | Rate Control | ✅ SSML |
186
- | Volume Control | ❌ Client-side |
187
- | Max Text Length | 5000 chars |
188
- | Audio Formats | MP3, WAV, OGG |
237
+ | Speech Marks | Word marks via SSML marks; none for Studio voices |
238
+ | SSML | Google's supported subset |
239
+ | Pitch Control | Plain text; none for Studio voices |
240
+ | Rate Control | Plain text, through `<prosody>` |
241
+ | Volume Control | Not supported |
242
+ | Max Text Length | 5000 characters |
243
+ | Audio Formats | MP3, WAV (LINEAR16), OGG (Opus) |
244
+ | Sample Rate | Per request (default 24000 Hz) |
245
+
246
+ The [provider comparison](../../docs/accessibility/tts-architecture.md#provider-comparison)
247
+ sets these against Polly and the custom transport.
189
248
 
190
249
  ## Cost
191
250
 
192
- - **Standard voices:** $4 per 1M characters
193
- - **WaveNet (neural) voices:** $16 per 1M characters
194
- - **Studio voices:** $16 per 1M characters
195
- - **Speech marks (timepoints):** Included (no extra charge)
196
-
197
- Pricing is competitive with AWS Polly.
198
-
199
- ## Supported Voices
200
-
201
- Popular voices include:
251
+ Current rates: <https://cloud.google.com/text-to-speech/pricing>
202
252
 
203
- ### Standard Voices
204
- - **English (US):** en-US-Standard-A/B/C/D/E/F/G/H/I/J
205
- - **English (UK):** en-GB-Standard-A/B/C/D/F
206
- - **Spanish:** es-ES-Standard-A/B/C/D
207
- - **French:** fr-FR-Standard-A/B/C/D/E
208
- - **German:** de-DE-Standard-A/B/C/D/E/F
209
- - **Italian:** it-IT-Standard-A/B/C/D
210
- - **Portuguese:** pt-BR-Standard-A/B/C
253
+ ## Voices
211
254
 
212
- ### WaveNet (Neural) Voices
213
- - **English (US):** en-US-Wavenet-A/B/C/D/E/F/G/H/I/J
214
- - **English (UK):** en-GB-Wavenet-A/B/C/D/F
215
- - **Spanish:** es-ES-Wavenet-B/C/D
216
- - **French:** fr-FR-Wavenet-A/B/C/D/E
217
- - **German:** de-DE-Wavenet-A/B/C/D/E/F
218
- - **Italian:** it-IT-Wavenet-A/B/C/D
219
- - **Portuguese:** pt-BR-Wavenet-A/B/C
255
+ Google's [voice list](https://docs.cloud.google.com/text-to-speech/docs/list-voices-and-types)
256
+ names the voices per language and type; `getVoices()` returns those the project
257
+ can use. Names follow `{languageCode}-{voiceType}-{variant}`:
220
258
 
221
- ### Studio Voices
222
- - **English (US):** en-US-Studio-O/Q
223
- - **English (UK):** en-GB-Studio-B/C
224
-
225
- Use `getVoices()` for the complete list of 200+ voices.
226
-
227
- ## Voice Naming Convention
228
-
229
- Google Cloud voices follow the pattern: `{languageCode}-{voiceType}-{variant}`
230
-
231
- Examples:
232
- - `en-US-Wavenet-A` - US English, WaveNet (neural), variant A
233
- - `es-ES-Standard-B` - Spanish (Spain), Standard, variant B
234
- - `fr-FR-Studio-A` - French, Studio (premium), variant A
259
+ - `en-US-Wavenet-A`: US English, WaveNet (neural), variant A
260
+ - `es-ES-Standard-B`: Spanish (Spain), Standard, variant B
261
+ - `fr-FR-Studio-A`: French, Studio (premium), variant A
235
262
 
236
263
  ## Error Handling
237
264
 
@@ -256,52 +283,20 @@ try {
256
283
  }
257
284
  ```
258
285
 
259
- ## Google Cloud IAM Permissions
260
-
261
- Required IAM permissions for the service account:
262
-
263
- ```json
264
- {
265
- "role": "roles/cloudtexttospeech.user",
266
- "permissions": [
267
- "texttospeech.operations.get",
268
- "texttospeech.voices.list",
269
- "texttospeech.voices.synthesize"
270
- ]
271
- }
272
- ```
273
-
274
- Or use the predefined role:
275
-
276
- ```bash
277
- gcloud projects add-iam-policy-binding PROJECT_ID \
278
- --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \
279
- --role="roles/cloudtexttospeech.user"
280
- ```
281
-
282
- ## Comparison with AWS Polly
283
-
284
- | Feature | Google Cloud TTS | AWS Polly |
285
- |---------|------------------|-----------|
286
- | **Voices** | 200+ voices | 60+ voices |
287
- | **Languages** | 50+ languages | 25+ languages |
288
- | **Speech Marks** | Via SSML marks | Native |
289
- | **API Calls** | Single call | Two parallel calls |
290
- | **Max Text** | 5000 chars | 3000 chars |
291
- | **Neural Cost** | $16/1M chars | $16/1M chars |
292
- | **Standard Cost** | $4/1M chars | $4/1M chars |
293
- | **Authentication** | Flexible (4 methods) | AWS credentials |
294
- | **Region** | Global service | Region-specific |
295
-
296
- ## How Speech Marks Work
297
-
298
- Unlike AWS Polly (which provides native speech marks), Google Cloud TTS requires SSML mark injection:
286
+ `synthesize` maps Google's gRPC status codes:
299
287
 
300
- 1. The provider automatically parses your text and injects `<mark>` tags before each word
301
- 2. Google Cloud TTS returns timepoints corresponding to these marks
302
- 3. The provider converts timepoints to the unified speech mark format
288
+ | Google failure | `TTSErrorCode` |
289
+ |----------------|----------------|
290
+ | `PERMISSION_DENIED` (7) | `AUTHENTICATION_ERROR` |
291
+ | `RESOURCE_EXHAUSTED` (8) | `RATE_LIMIT_EXCEEDED` |
292
+ | `INVALID_ARGUMENT` (3) | `INVALID_REQUEST` |
293
+ | Anything else, a response without audio included | `PROVIDER_ERROR` |
303
294
 
304
- This process is transparent to the user - just set `includeSpeechMarks: true`.
295
+ Every `getVoices` failure is a `PROVIDER_ERROR`. Request validation runs before
296
+ any Google call and rejects with a `TTSError`: `TEXT_TOO_LONG` for text over
297
+ 5000 characters, `INVALID_REQUEST` for empty text, an unsupported format, or a
298
+ rate or pitch out of range. A call before `initialize` throws a `TTSError` with
299
+ `INITIALIZATION_ERROR`.
305
300
 
306
301
  ## Advanced Configuration
307
302
 
@@ -319,7 +314,7 @@ await provider.initialize({
319
314
  ```typescript
320
315
  await provider.initialize({
321
316
  projectId: 'my-project',
322
- enableLogging: true, // Logs SSML injection and speech marks extraction
317
+ enableLogging: true, // Logs SSML detection and the marks injected and extracted
323
318
  });
324
319
  ```
325
320
 
@@ -2,6 +2,7 @@
2
2
  * Google Cloud Text-to-Speech server-side TTS provider
3
3
  * @module @pie-players/tts-server-google
4
4
  */
5
+ import { v1beta1 } from "@google-cloud/text-to-speech";
5
6
  import { BaseTTSProvider, type GetVoicesOptions, type ServerProviderCapabilities, type SynthesizeRequest, type SynthesizeResponse, type TTSServerConfig, type Voice } from "@pie-players/tts-server-core";
6
7
  /**
7
8
  * Google Cloud Text-to-Speech provider configuration.
@@ -41,7 +42,7 @@ export interface GoogleCloudTTSConfig extends TTSServerConfig {
41
42
  * Voice type: 'wavenet' (neural), 'standard', or 'studio' (premium)
42
43
  *
43
44
  * @default 'wavenet'
44
- * @note WaveNet: $16/1M chars, Standard: $4/1M chars, Studio: $16/1M chars
45
+ * @see https://cloud.google.com/text-to-speech/pricing
45
46
  */
46
47
  voiceType?: "wavenet" | "standard" | "studio";
47
48
  /**
@@ -74,7 +75,7 @@ export interface GoogleCloudTTSConfig extends TTSServerConfig {
74
75
  * Features:
75
76
  * - Speech marks support via SSML mark injection (millisecond precision)
76
77
  * - WaveNet (neural), Standard, and Studio voice types
77
- * - 200+ voices across 50+ languages
78
+ * - Voices across many languages, listed by `getVoices`
78
79
  * - Full SSML support
79
80
  * - Single API call for audio + speech marks
80
81
  */
@@ -99,12 +100,15 @@ export declare class GoogleCloudTTSProvider extends BaseTTSProvider {
99
100
  * @performance Completes in ~10-50ms
100
101
  */
101
102
  initialize(config: GoogleCloudTTSConfig): Promise<void>;
103
+ /** The Google client every request goes through; a test subclass supplies a fake. */
104
+ protected createClient(clientConfig: ConstructorParameters<typeof v1beta1.TextToSpeechClient>[0]): v1beta1.TextToSpeechClient;
102
105
  /**
103
106
  * Synthesize speech with Google Cloud TTS
104
107
  */
105
108
  synthesize(request: SynthesizeRequest): Promise<SynthesizeResponse>;
106
109
  /**
107
- * Synthesize audio stream only (no speech marks)
110
+ * Synthesize audio stream only (no speech marks). A Studio voice takes
111
+ * `<prosody>` rate but not pitch, so its request drops pitch.
108
112
  */
109
113
  private synthesizeAudio;
110
114
  /**
@@ -116,16 +120,17 @@ export declare class GoogleCloudTTSProvider extends BaseTTSProvider {
116
120
  */
117
121
  private injectSSMLMarks;
118
122
  /**
119
- * Extract words from existing SSML (simplified version for v1)
123
+ * Inject a mark before each word of authored SSML, keeping its markup.
124
+ * Offsets index the SSML itself. An element whose content the engine
125
+ * replaces (`<sub>`, `<say-as>`, `<phoneme>`) takes one mark before its
126
+ * opening tag, valued with its text, since a mark may not sit inside it.
120
127
  */
121
128
  private extractWordsFromSSML;
129
+ private markTextNode;
122
130
  /**
123
131
  * Extract speech marks from Google's timepoints
124
132
  */
125
133
  private extractSpeechMarksFromTimepoints;
126
- /**
127
- * Detect if text contains SSML markup
128
- */
129
134
  /**
130
135
  * Get available voices from Google Cloud TTS
131
136
  */
@@ -4,6 +4,44 @@
4
4
  */
5
5
  import { v1beta1, protos } from "@google-cloud/text-to-speech";
6
6
  import { BaseTTSProvider, TTSError, TTSErrorCode, } from "@pie-players/tts-server-core";
7
+ const DEFAULT_SAMPLE_RATE_HERTZ = 24000;
8
+ const CONTENT_TYPE_BY_ENCODING = {
9
+ MP3: "audio/mpeg",
10
+ LINEAR16: "audio/wav",
11
+ OGG_OPUS: "audio/ogg",
12
+ };
13
+ // The segment of a Google voice name that identifies its voice type.
14
+ const VOICE_TYPE_NAME_SEGMENT = {
15
+ wavenet: "Wavenet",
16
+ standard: "Standard",
17
+ studio: "Studio",
18
+ };
19
+ // gRPC status codes the Google client reports in `error.code`.
20
+ const GRPC_STATUS = {
21
+ INVALID_ARGUMENT: 3,
22
+ PERMISSION_DENIED: 7,
23
+ RESOURCE_EXHAUSTED: 8,
24
+ };
25
+ // Google voice names start with their locale: "en-US-Wavenet-A" → "en-US".
26
+ const languageCodeOf = (voice) => voice.split("-").slice(0, 2).join("-");
27
+ // Letters, digits and combining marks, so accented words stay whole.
28
+ const WORD_PATTERN = /[\p{L}\p{M}\p{N}'\u2019]+/gu;
29
+ const SSML_TEXT_TOKEN_PATTERN = /&(?:#\d+|#x[\da-f]+|[a-z]+);|[\p{L}\p{M}\p{N}'\u2019]+/giu;
30
+ const SSML_ENTITIES = {
31
+ amp: "&",
32
+ lt: "<",
33
+ gt: ">",
34
+ quot: '"',
35
+ apos: "'",
36
+ };
37
+ const decodeSSMLText = (text) => text.replace(/&(#\d+|#x[\da-f]+|[a-z]+);/gi, (entity, name) => {
38
+ if (name[0] !== "#")
39
+ return SSML_ENTITIES[name.toLowerCase()] ?? entity;
40
+ const code = name[1] === "x" || name[1] === "X"
41
+ ? Number.parseInt(name.slice(2), 16)
42
+ : Number.parseInt(name.slice(1), 10);
43
+ return Number.isFinite(code) ? String.fromCodePoint(code) : entity;
44
+ });
7
45
  /**
8
46
  * Google Cloud Text-to-Speech Server Provider
9
47
  *
@@ -13,7 +51,7 @@ import { BaseTTSProvider, TTSError, TTSErrorCode, } from "@pie-players/tts-serve
13
51
  * Features:
14
52
  * - Speech marks support via SSML mark injection (millisecond precision)
15
53
  * - WaveNet (neural), Standard, and Studio voice types
16
- * - 200+ voices across 50+ languages
54
+ * - Voices across many languages, listed by `getVoices`
17
55
  * - Full SSML support
18
56
  * - Single API call for audio + speech marks
19
57
  */
@@ -58,7 +96,6 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
58
96
  this.audioEncoding = config.audioEncoding || "MP3";
59
97
  this.enableLogging = config.enableLogging || false;
60
98
  try {
61
- // Initialize Google Cloud TTS client
62
99
  const clientConfig = {
63
100
  projectId: config.projectId,
64
101
  };
@@ -78,7 +115,7 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
78
115
  }
79
116
  }
80
117
  // Else: Use Application Default Credentials (ADC)
81
- this.client = new v1beta1.TextToSpeechClient(clientConfig);
118
+ this.client = this.createClient(clientConfig);
82
119
  this.initialized = true;
83
120
  if (this.enableLogging) {
84
121
  console.log("[GoogleCloudTTS] Initialized successfully");
@@ -88,6 +125,10 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
88
125
  throw new TTSError(TTSErrorCode.INITIALIZATION_ERROR, `Failed to initialize Google Cloud TTS: ${error instanceof Error ? error.message : String(error)}`, { error }, this.providerId);
89
126
  }
90
127
  }
128
+ /** The Google client every request goes through; a test subclass supplies a fake. */
129
+ createClient(clientConfig) {
130
+ return new v1beta1.TextToSpeechClient(clientConfig);
131
+ }
91
132
  /**
92
133
  * Synthesize speech with Google Cloud TTS
93
134
  */
@@ -95,172 +136,112 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
95
136
  this.ensureInitialized();
96
137
  const capabilities = this.getCapabilities();
97
138
  this.validateRequest(request, capabilities);
98
- const voice = request.voice || this.defaultVoice;
139
+ const voiceType = VOICE_TYPE_NAME_SEGMENT[this.voiceType];
140
+ const voice = await this.resolveRequestVoice(request, this.defaultVoice, (candidate) => candidate.id.includes(voiceType));
99
141
  const startTime = Date.now();
142
+ const respond = (result, speechMarks) => ({
143
+ audio: result.audio,
144
+ contentType: result.contentType,
145
+ speechMarks,
146
+ metadata: {
147
+ providerId: this.providerId,
148
+ voice,
149
+ duration: (Date.now() - startTime) / 1000,
150
+ charCount: request.text.length,
151
+ cached: false,
152
+ timestamp: new Date().toISOString(),
153
+ },
154
+ });
100
155
  try {
101
- // Check if speech marks are requested
102
156
  if (request.includeSpeechMarks !== false && !this.isStudioVoice(voice)) {
103
- // Use SSML marks injection for precise word timing
104
157
  const result = await this.synthesizeWithSpeechMarks(request, voice);
105
- const duration = (Date.now() - startTime) / 1000;
106
- return {
107
- audio: result.audio,
108
- contentType: result.contentType,
109
- speechMarks: result.speechMarks,
110
- metadata: {
111
- providerId: this.providerId,
112
- voice,
113
- duration,
114
- charCount: request.text.length,
115
- cached: false,
116
- timestamp: new Date().toISOString(),
117
- },
118
- };
119
- }
120
- else {
121
- // Audio only (no speech marks)
122
- const result = await this.synthesizeAudio(request, voice);
123
- const duration = (Date.now() - startTime) / 1000;
124
- return {
125
- audio: result.audio,
126
- contentType: result.contentType,
127
- speechMarks: [],
128
- metadata: {
129
- providerId: this.providerId,
130
- voice,
131
- duration,
132
- charCount: request.text.length,
133
- cached: false,
134
- timestamp: new Date().toISOString(),
135
- },
136
- };
158
+ return respond(result, result.speechMarks);
137
159
  }
160
+ return respond(await this.synthesizeAudio(request, voice), []);
138
161
  }
139
162
  catch (error) {
140
163
  // Studio voices do not support SSML <mark> tags used for timing extraction.
141
164
  // Fall back to audio-only synthesis rather than failing the request.
142
165
  if (request.includeSpeechMarks !== false &&
143
166
  this.isStudioMarkUnsupportedError(error)) {
144
- const result = await this.synthesizeAudio(request, voice);
145
- const duration = (Date.now() - startTime) / 1000;
146
- return {
147
- audio: result.audio,
148
- contentType: result.contentType,
149
- speechMarks: [],
150
- metadata: {
151
- providerId: this.providerId,
152
- voice,
153
- duration,
154
- charCount: request.text.length,
155
- cached: false,
156
- timestamp: new Date().toISOString(),
157
- },
158
- };
167
+ try {
168
+ return respond(await this.synthesizeAudio(request, voice, true), []);
169
+ }
170
+ catch (fallbackError) {
171
+ throw this.mapGoogleErrorToTTSError(fallbackError);
172
+ }
159
173
  }
160
174
  throw this.mapGoogleErrorToTTSError(error);
161
175
  }
162
176
  }
163
177
  /**
164
- * Synthesize audio stream only (no speech marks)
178
+ * Synthesize audio stream only (no speech marks). A Studio voice takes
179
+ * `<prosody>` rate but not pitch, so its request drops pitch.
165
180
  */
166
- async synthesizeAudio(request, voice) {
181
+ async synthesizeAudio(request, voice, studio = this.isStudioVoice(voice)) {
167
182
  if (this.detectSSML(request.text) && this.enableLogging) {
168
183
  console.log("[GoogleCloudTTS] Detected SSML content");
169
184
  }
170
- const { text, isSsml } = this.applyProsody(request.text, request);
171
- // Parse voice name to extract language code
172
- const languageCode = voice.split("-").slice(0, 2).join("-"); // e.g., "en-US" from "en-US-Wavenet-A"
173
- // Map our audio encoding to Google's enum
174
- const audioEncodingMap = {
175
- MP3: "MP3",
176
- LINEAR16: "LINEAR16",
177
- OGG_OPUS: "OGG_OPUS",
178
- };
185
+ const { text, isSsml } = this.applyProsody(request.text, studio ? { ...request, pitch: undefined } : request);
179
186
  const [response] = await this.client.synthesizeSpeech({
180
187
  input: isSsml ? { ssml: text } : { text },
181
188
  voice: {
182
- languageCode,
189
+ languageCode: languageCodeOf(voice),
183
190
  name: voice,
184
191
  },
185
192
  audioConfig: {
186
- audioEncoding: audioEncodingMap[this.audioEncoding],
187
- sampleRateHertz: request.sampleRate || 24000,
193
+ audioEncoding: this.audioEncoding,
194
+ sampleRateHertz: request.sampleRate || DEFAULT_SAMPLE_RATE_HERTZ,
188
195
  },
189
196
  });
190
197
  if (!response.audioContent) {
191
198
  throw new Error("No audio content received from Google Cloud TTS");
192
199
  }
193
- // Convert Uint8Array to Buffer
194
- const audioBuffer = Buffer.from(response.audioContent);
195
- const contentTypeMap = {
196
- MP3: "audio/mpeg",
197
- LINEAR16: "audio/wav",
198
- OGG_OPUS: "audio/ogg",
199
- };
200
200
  return {
201
- audio: audioBuffer,
202
- contentType: contentTypeMap[this.audioEncoding],
201
+ audio: Buffer.from(response.audioContent),
202
+ contentType: CONTENT_TYPE_BY_ENCODING[this.audioEncoding],
203
203
  };
204
204
  }
205
205
  /**
206
206
  * Synthesize with speech marks using SSML mark injection
207
207
  */
208
208
  async synthesizeWithSpeechMarks(request, voice) {
209
- // Check if the text is already SSML
210
209
  const isUserSSML = this.detectSSML(request.text);
211
- // If user provided SSML, we need to inject marks within the existing SSML
212
- // For simplicity in v1, we'll inject marks for plain text only. Already-SSML
213
- // input skips prosody wrapping too: injecting <prosody> into markup the
214
- // caller authored themselves would require parsing it.
210
+ // Authored SSML keeps its markup and takes marks in its text nodes. It
211
+ // skips prosody wrapping: injecting <prosody> into markup the caller
212
+ // authored would require parsing it.
215
213
  const { ssml, wordMap } = isUserSSML
216
214
  ? this.extractWordsFromSSML(request.text)
217
215
  : this.injectSSMLMarks(request.text, this.buildProsodyAttrs(request));
218
216
  if (this.enableLogging) {
219
217
  console.log(`[GoogleCloudTTS] Injected ${wordMap.length} SSML marks`);
220
218
  }
221
- // Parse voice name to extract language code
222
- const languageCode = voice.split("-").slice(0, 2).join("-");
223
- // Map our audio encoding to Google's enum
224
- const audioEncodingMap = {
225
- MP3: "MP3",
226
- LINEAR16: "LINEAR16",
227
- OGG_OPUS: "OGG_OPUS",
228
- };
229
219
  // Single API call with timepoint tracking enabled
230
- const responseArray = await this.client.synthesizeSpeech({
220
+ const [response] = await this.client.synthesizeSpeech({
231
221
  input: { ssml },
232
222
  voice: {
233
- languageCode,
223
+ languageCode: languageCodeOf(voice),
234
224
  name: voice,
235
225
  },
236
226
  audioConfig: {
237
- audioEncoding: audioEncodingMap[this.audioEncoding],
238
- sampleRateHertz: request.sampleRate || 24000,
227
+ audioEncoding: this.audioEncoding,
228
+ sampleRateHertz: request.sampleRate || DEFAULT_SAMPLE_RATE_HERTZ,
239
229
  },
240
230
  enableTimePointing: [
241
231
  protos.google.cloud.texttospeech.v1beta1.SynthesizeSpeechRequest
242
232
  .TimepointType.SSML_MARK,
243
233
  ],
244
234
  });
245
- const response = responseArray[0];
246
235
  if (!response.audioContent) {
247
236
  throw new Error("No audio content received from Google Cloud TTS");
248
237
  }
249
- // Convert Uint8Array to Buffer
250
- const audioBuffer = Buffer.from(response.audioContent);
251
- const contentTypeMap = {
252
- MP3: "audio/mpeg",
253
- LINEAR16: "audio/wav",
254
- OGG_OPUS: "audio/ogg",
255
- };
256
- // Extract speech marks from timepoints
257
238
  const speechMarks = this.extractSpeechMarksFromTimepoints(response.timepoints || [], wordMap);
258
239
  if (this.enableLogging) {
259
240
  console.log(`[GoogleCloudTTS] Extracted ${speechMarks.length} speech marks`);
260
241
  }
261
242
  return {
262
- audio: audioBuffer,
263
- contentType: contentTypeMap[this.audioEncoding],
243
+ audio: Buffer.from(response.audioContent),
244
+ contentType: CONTENT_TYPE_BY_ENCODING[this.audioEncoding],
264
245
  speechMarks,
265
246
  };
266
247
  }
@@ -269,27 +250,16 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
269
250
  */
270
251
  injectSSMLMarks(text, prosodyAttrs = "") {
271
252
  const words = [];
272
- const wordRegex = /\b[\w']+\b/g;
273
- let match;
274
- let markIndex = 0;
275
- while ((match = wordRegex.exec(text)) !== null) {
276
- const word = match[0];
277
- const start = match.index;
278
- const end = start + word.length;
279
- const markName = `w${markIndex++}`;
280
- words.push({ word, start, end, markName });
281
- }
282
- // Build SSML with marks
283
253
  let ssml = prosodyAttrs ? `<speak><prosody ${prosodyAttrs}>` : "<speak>";
284
254
  let lastEnd = 0;
285
- for (const { word, start, end, markName } of words) {
286
- // Add text before word (including whitespace and punctuation)
287
- ssml += this.escapeSSML(text.slice(lastEnd, start));
288
- // Add marked word
289
- ssml += `<mark name="${markName}"/>${this.escapeSSML(word)}`;
290
- lastEnd = end;
255
+ for (const match of text.matchAll(WORD_PATTERN)) {
256
+ const word = match[0];
257
+ const start = match.index;
258
+ const markName = `w${words.length}`;
259
+ words.push({ word, start, end: start + word.length, markName });
260
+ ssml += `${this.escapeSSML(text.slice(lastEnd, start))}<mark name="${markName}"/>${this.escapeSSML(word)}`;
261
+ lastEnd = start + word.length;
291
262
  }
292
- // Add remaining text
293
263
  ssml +=
294
264
  this.escapeSSML(text.slice(lastEnd)) +
295
265
  (prosodyAttrs ? "</prosody>" : "") +
@@ -297,16 +267,63 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
297
267
  return { ssml, wordMap: words };
298
268
  }
299
269
  /**
300
- * Extract words from existing SSML (simplified version for v1)
270
+ * Inject a mark before each word of authored SSML, keeping its markup.
271
+ * Offsets index the SSML itself. An element whose content the engine
272
+ * replaces (`<sub>`, `<say-as>`, `<phoneme>`) takes one mark before its
273
+ * opening tag, valued with its text, since a mark may not sit inside it.
301
274
  */
302
275
  extractWordsFromSSML(ssmlText) {
303
- // For now, just strip SSML tags and inject marks
304
- // More sophisticated SSML parsing can be added in future versions
305
- const plainText = ssmlText
306
- .replace(/<[^>]+>/g, " ") // Remove all tags
307
- .replace(/\s+/g, " ") // Normalize whitespace
308
- .trim();
309
- return this.injectSSMLMarks(plainText);
276
+ const words = [];
277
+ const mark = (word, start, end) => {
278
+ const markName = `w${words.length}`;
279
+ words.push({ word, start, end, markName });
280
+ return `<mark name="${markName}"/>`;
281
+ };
282
+ let ssml = "";
283
+ let cursor = 0;
284
+ for (const tag of ssmlText.matchAll(/<[^>]*>/g)) {
285
+ if (tag.index < cursor)
286
+ continue;
287
+ ssml += this.markTextNode(ssmlText, cursor, tag.index, mark);
288
+ const opaque = /^<\s*(sub|say-as|phoneme)\b(?![^>]*\/>)[^>]*>$/i.exec(tag[0]);
289
+ const close = opaque
290
+ ? new RegExp(`</\\s*${opaque[1]}\\s*>`, "i")
291
+ : null;
292
+ const closeMatch = close?.exec(ssmlText.slice(tag.index));
293
+ if (opaque && closeMatch) {
294
+ const contentStart = tag.index + tag[0].length;
295
+ const contentEnd = tag.index + closeMatch.index;
296
+ const raw = ssmlText.slice(contentStart, contentEnd);
297
+ const inner = raw.trim();
298
+ const innerStart = contentStart + raw.indexOf(inner);
299
+ const elementEnd = contentEnd + closeMatch[0].length;
300
+ ssml += inner
301
+ ? mark(decodeSSMLText(inner), innerStart, innerStart + inner.length)
302
+ : "";
303
+ ssml += ssmlText.slice(tag.index, elementEnd);
304
+ cursor = elementEnd;
305
+ continue;
306
+ }
307
+ ssml += tag[0];
308
+ cursor = tag.index + tag[0].length;
309
+ }
310
+ ssml += this.markTextNode(ssmlText, cursor, ssmlText.length, mark);
311
+ return { ssml, wordMap: words };
312
+ }
313
+ // Marks each word of an SSML text node, which is already escaped, so
314
+ // entities stay whole and are never read as words.
315
+ markTextNode(ssmlText, from, to, mark) {
316
+ const segment = ssmlText.slice(from, to);
317
+ let out = "";
318
+ let lastEnd = 0;
319
+ for (const match of segment.matchAll(SSML_TEXT_TOKEN_PATTERN)) {
320
+ if (match[0].startsWith("&"))
321
+ continue;
322
+ const start = from + match.index;
323
+ out += `${segment.slice(lastEnd, match.index)}${mark(match[0], start, start + match[0].length)}${match[0]}`;
324
+ lastEnd = match.index + match[0].length;
325
+ }
326
+ return out + segment.slice(lastEnd);
310
327
  }
311
328
  /**
312
329
  * Extract speech marks from Google's timepoints
@@ -317,7 +334,6 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
317
334
  }
318
335
  const speechMarks = [];
319
336
  for (const timepoint of timepoints) {
320
- // Find corresponding word in our map
321
337
  const wordInfo = wordMap.find((w) => w.markName === timepoint.markName);
322
338
  if (wordInfo &&
323
339
  timepoint.timeSeconds !== undefined &&
@@ -331,12 +347,8 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
331
347
  });
332
348
  }
333
349
  }
334
- // Sort by time
335
350
  return speechMarks.sort((a, b) => a.time - b.time);
336
351
  }
337
- /**
338
- * Detect if text contains SSML markup
339
- */
340
352
  /**
341
353
  * Get available voices from Google Cloud TTS
342
354
  */
@@ -352,7 +364,6 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
352
364
  return response.voices
353
365
  .map((voice) => this.mapGoogleVoiceToVoice(voice))
354
366
  .filter((voice) => {
355
- // Apply filters
356
367
  if (options?.gender && voice.gender !== options.gender) {
357
368
  return false;
358
369
  }
@@ -413,10 +424,10 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
413
424
  // W3C Standard features
414
425
  standard: {
415
426
  supportsSSML: true, // ✅ Full SSML 1.1 support
416
- supportsPitch: true, // ✅ Via SSML <prosody pitch>
427
+ supportsPitch: this.voiceType !== "studio", // Via SSML <prosody pitch>; Studio voices lack it
417
428
  supportsRate: true, // ✅ Via SSML <prosody rate>
418
429
  supportsVolume: false, // ❌ Not supported (handle client-side)
419
- supportsMultipleVoices: true, // ✅ 200+ voices across 50+ languages
430
+ supportsMultipleVoices: true, // ✅ Listed by getVoices
420
431
  maxTextLength: 5000, // Google Cloud TTS limit per request
421
432
  },
422
433
  // Provider-specific extensions
@@ -427,8 +438,6 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
427
438
  // Google Cloud-specific features
428
439
  providerSpecific: {
429
440
  voiceTypes: ["standard", "wavenet", "studio"],
430
- voicesCount: 200, // ~200+ voices available
431
- languagesCount: 50, // 50+ languages supported
432
441
  supportsAudioProfiles: true, // Audio device profiles
433
442
  supportsEffects: false, // No built-in effects
434
443
  supportsEmotions: false, // No emotion control
@@ -441,21 +450,17 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
441
450
  * Map Google Cloud errors to TTSError codes
442
451
  */
443
452
  mapGoogleErrorToTTSError(error) {
444
- const message = error.message || String(error);
445
- // Check for specific Google Cloud error codes
446
- if (error.code === 7) {
447
- // PERMISSION_DENIED
453
+ const err = error;
454
+ const message = err.message || String(error);
455
+ if (err.code === GRPC_STATUS.PERMISSION_DENIED) {
448
456
  return new TTSError(TTSErrorCode.AUTHENTICATION_ERROR, `Google Cloud authentication failed: ${message}`, { error }, this.providerId);
449
457
  }
450
- if (error.code === 8) {
451
- // RESOURCE_EXHAUSTED
458
+ if (err.code === GRPC_STATUS.RESOURCE_EXHAUSTED) {
452
459
  return new TTSError(TTSErrorCode.RATE_LIMIT_EXCEEDED, `Google Cloud rate limit exceeded: ${message}`, { error }, this.providerId);
453
460
  }
454
- if (error.code === 3) {
455
- // INVALID_ARGUMENT
461
+ if (err.code === GRPC_STATUS.INVALID_ARGUMENT) {
456
462
  return new TTSError(TTSErrorCode.INVALID_REQUEST, `Invalid request to Google Cloud TTS: ${message}`, { error }, this.providerId);
457
463
  }
458
- // Default to provider error
459
464
  return new TTSError(TTSErrorCode.PROVIDER_ERROR, `Google Cloud TTS error: ${message}`, { error }, this.providerId);
460
465
  }
461
466
  /**
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@pie-players/tts-server-google",
3
- "version": "0.3.74",
3
+ "version": "1.0.0-beta.0",
4
4
  "description": "Google Cloud Text-to-Speech provider for server-side TTS with speech marks support",
5
5
  "type": "module",
6
6
  "main": "./dist/index.js",
@@ -17,8 +17,7 @@
17
17
  "scripts": {
18
18
  "build": "tsc",
19
19
  "dev": "tsc --watch",
20
- "test": "vitest run --passWithNoTests",
21
- "test:coverage": "vitest --coverage"
20
+ "test": "vitest run"
22
21
  },
23
22
  "keywords": [
24
23
  "tts",
@@ -40,7 +39,7 @@
40
39
  },
41
40
  "dependencies": {
42
41
  "@google-cloud/text-to-speech": "^7.1.1",
43
- "@pie-players/tts-server-core": "0.3.74"
42
+ "@pie-players/tts-server-core": "1.0.0-beta.0"
44
43
  },
45
44
  "devDependencies": {
46
45
  "typescript": "^5.9.3",