@pie-players/tts-server-google 0.3.75 → 1.0.0-beta.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,24 +1,16 @@
1
1
  # @pie-players/tts-server-google
2
2
 
3
- Google Cloud Text-to-Speech provider for server-side text-to-speech with speech marks support.
4
-
5
- For the cross-package TTS architecture and browser/server flow, see
6
- [TTS Architecture](../../docs/accessibility/tts-architecture.md). This README
7
- focuses on Google Cloud provider configuration and behavior.
8
-
9
- ## Overview
10
-
11
- This package provides a server-side TTS provider that uses Google Cloud Text-to-Speech to generate high-quality neural speech with millisecond-precise word timing through SSML mark injection.
12
-
13
- ## Features
14
-
15
- - ✅ **Speech Marks Support** - Millisecond-accurate word timing via SSML marks + timepoints
16
- - ✅ **WaveNet Neural Voices** - High-quality neural TTS with Google's WaveNet technology
17
- - ✅ **50+ Languages** - Extensive language support
18
- - ✅ **Full SSML** - Supports Speech Synthesis Markup Language 1.1
19
- - ✅ **Single API Call** - Audio and speech marks in one request (more efficient than AWS Polly)
20
- - ✅ **200+ Voices** - Multiple voice types per language (Standard, WaveNet, Studio)
21
- - ✅ **Flexible Authentication** - Service account, API key, or Application Default Credentials
3
+ `GoogleCloudTTSProvider`, the Google Cloud Text-to-Speech provider for
4
+ server-side text-to-speech (TTS), built on `@pie-players/tts-server-core`. It
5
+ returns audio with word timings, which it gets by injecting SSML `<mark>` tags
6
+ and reading Google's timepoints; the timings drive read-aloud highlighting in
7
+ the browser. This README covers the provider's configuration and behavior for
8
+ developers writing a host's TTS routes; the
9
+ [integration guide](./examples/INTEGRATION-GUIDE.md) builds those routes, and
10
+ [TTS Architecture](../../docs/accessibility/tts-architecture.md) covers the
11
+ browser and server flow across packages.
12
+
13
+ The package runs on Node.js 20 or later.
22
14
 
23
15
  ## Installation
24
16
 
@@ -37,15 +29,23 @@ const provider = new GoogleCloudTTSProvider();
37
29
 
38
30
  await provider.initialize({
39
31
  projectId: 'my-gcp-project',
40
- credentials: '/path/to/service-account.json', // Or use other auth methods
41
- voiceType: 'wavenet', // 'wavenet', 'standard', or 'studio'
32
+ credentials: '/path/to/service-account.json', // or another authentication method
33
+ voiceType: 'wavenet', // 'wavenet', 'standard', or 'studio' (Studio returns no speech marks)
42
34
  defaultVoice: 'en-US-Wavenet-A',
43
35
  });
44
36
  ```
45
37
 
38
+ `initialize` validates the config and creates the Google client without calling
39
+ Google. A missing `projectId` throws a `TTSError` with `INITIALIZATION_ERROR`.
40
+
46
41
  ### Authentication Methods
47
42
 
48
- #### 1. Service Account JSON File (Recommended for Production)
43
+ The project needs the Cloud Text-to-Speech API enabled, with billing, as
44
+ Google's [setup page](https://docs.cloud.google.com/text-to-speech/docs/get-started)
45
+ describes. The provider reads no environment variables; `credentials` selects
46
+ the method.
47
+
48
+ #### Service Account Key File
49
49
 
50
50
  ```typescript
51
51
  await provider.initialize({
@@ -54,7 +54,9 @@ await provider.initialize({
54
54
  });
55
55
  ```
56
56
 
57
- #### 2. Service Account Object (For Containers/Serverless)
57
+ #### Service Account Key Object
58
+
59
+ For containers and serverless platforms that hold the key as a secret:
58
60
 
59
61
  ```typescript
60
62
  await provider.initialize({
@@ -66,7 +68,7 @@ await provider.initialize({
66
68
  });
67
69
  ```
68
70
 
69
- #### 3. API Key (For Simple Applications)
71
+ #### API Key
70
72
 
71
73
  ```typescript
72
74
  await provider.initialize({
@@ -77,10 +79,15 @@ await provider.initialize({
77
79
  });
78
80
  ```
79
81
 
80
- #### 4. Application Default Credentials (For Local Development)
82
+ #### Application Default Credentials
83
+
84
+ Omitting `credentials` uses
85
+ [Application Default Credentials](https://docs.cloud.google.com/docs/authentication/application-default-credentials):
86
+ `gcloud auth application-default login` locally, the file named by
87
+ `GOOGLE_APPLICATION_CREDENTIALS`, or the attached service account on Google
88
+ Cloud.
81
89
 
82
90
  ```typescript
83
- // Omit credentials to use gcloud auth application-default login
84
91
  await provider.initialize({
85
92
  projectId: 'my-project',
86
93
  });
@@ -91,13 +98,13 @@ await provider.initialize({
91
98
  ```typescript
92
99
  const result = await provider.synthesize({
93
100
  text: 'Hello world, this is a test of Google Cloud Text to Speech.',
94
- voice: 'en-US-Wavenet-A', // Optional, uses defaultVoice if not specified
101
+ voice: 'en-US-Wavenet-A', // optional; see Voice Selection
95
102
  includeSpeechMarks: true,
96
103
  });
97
104
 
98
105
  console.log('Audio:', result.audio); // Buffer
99
- console.log('Speech marks:', result.speechMarks); // Array of word timings
100
- console.log('Duration:', result.metadata.duration, 'seconds');
106
+ console.log('Speech marks:', result.speechMarks); // word timings
107
+ console.log('Synthesis time:', result.metadata.duration, 'seconds');
101
108
  ```
102
109
 
103
110
  ### List Available Voices
@@ -116,6 +123,10 @@ const femaleVoices = await provider.getVoices({ gender: 'female' });
116
123
  const neuralVoices = await provider.getVoices({ quality: 'neural' });
117
124
  ```
118
125
 
126
+ `getVoices` passes `language` to Google and filters by gender and quality
127
+ locally. Quality follows the voice name: `Wavenet` is `neural`, `Studio` is
128
+ `premium`, and every other voice is `standard`.
129
+
119
130
  ### Speech Marks Example
120
131
 
121
132
  ```typescript
@@ -147,6 +158,9 @@ const result = await provider.synthesize({
147
158
  });
148
159
  ```
149
160
 
161
+ The [TTS authoring guide](../../docs/accessibility/tts-authoring-guide.md#ssml-provider-support)
162
+ covers how much of authored SSML Google voices.
163
+
150
164
  ## Configuration
151
165
 
152
166
  ### GoogleCloudTTSConfig
@@ -154,84 +168,97 @@ const result = await provider.synthesize({
154
168
  ```typescript
155
169
  interface GoogleCloudTTSConfig {
156
170
  projectId: string; // Google Cloud project ID (required)
157
- credentials?: // Authentication (optional if using ADC)
158
- | string // Path to service account JSON
159
- | { // Service account object
171
+ credentials?: // omitted: Application Default Credentials
172
+ | string // Path to a service account key file
173
+ | { // Service account key object
160
174
  client_email: string;
161
175
  private_key: string;
176
+ project_id?: string;
162
177
  }
163
178
  | { apiKey: string }; // API key
164
- voiceType?: 'wavenet' | 'standard' | 'studio'; // Voice type (default: 'wavenet')
165
- defaultVoice?: string; // Default voice (default: 'en-US-Wavenet-A')
166
- audioEncoding?: 'MP3' | 'LINEAR16' | 'OGG_OPUS'; // Audio format (default: 'MP3')
167
- enableLogging?: boolean; // Debug logging (default: false)
179
+ voiceType?: 'wavenet' | 'standard' | 'studio'; // default: 'wavenet'
180
+ defaultVoice?: string; // default: 'en-US-Wavenet-A'
181
+ audioEncoding?: 'MP3' | 'LINEAR16' | 'OGG_OPUS'; // output encoding (default: 'MP3')
182
+ enableLogging?: boolean; // default: false
168
183
  }
169
184
  ```
170
185
 
171
- ### Environment Variables
172
-
173
- ```bash
174
- GOOGLE_CLOUD_PROJECT=my-project
175
- GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
176
- ```
186
+ ## Synthesis Behavior
187
+
188
+ - **Requests.** One SynthesizeSpeech call returns the audio and, with speech
189
+ marks requested, the timepoints of the injected marks.
190
+ - **Output.** `audioEncoding` fixes the output for the instance: `MP3` returns
191
+ `audio/mpeg`, `LINEAR16` `audio/wav`, `OGG_OPUS` `audio/ogg`. A request's
192
+ `format` is only validated against `mp3`, `wav` and `ogg`. `sampleRate`
193
+ defaults to 24000 Hz. The language code sent to Google comes from the voice
194
+ name.
195
+ - **Prosody.** Plain text with a `rate` or `pitch` other than 1 is escaped and
196
+ wrapped in `<prosody>`. Authored SSML goes to Google with its markup
197
+ unchanged, so `rate` and `pitch` apply to plain text only. A Studio voice
198
+ drops `pitch`.
199
+ - **Limits.** The provider accepts 5000 characters per request.
200
+ `ServerTTSProvider` declares 3000, and the toolkit splits longer reads to fit,
201
+ so requests from the browser stay under the provider's limit.
202
+
203
+ ### Voice Selection
204
+
205
+ A request's `voice` wins. Without one, the provider picks a voice for the
206
+ request's `language` from Google's voice listing, preferring voices of the
207
+ configured `voiceType`, and falls back to `defaultVoice`; the core README's
208
+ [voice resolution](../tts-server-core/README.md#voice-resolution) section has the
209
+ matching rules.
210
+
211
+ ### Speech Marks
212
+
213
+ Speech marks are on unless the request sets `includeSpeechMarks: false`, and are
214
+ word marks only.
215
+
216
+ 1. The provider places a `<mark>` before each word: a run of letters, combining
217
+ marks, digits and apostrophes, so accented words stay whole.
218
+ 2. Google returns a timepoint for each mark.
219
+ 3. The provider converts the timepoints to speech marks in milliseconds, sorted
220
+ by time.
221
+
222
+ Plain-text offsets are UTF-16 indexes into the request text. Authored SSML keeps
223
+ its markup and takes marks in its text nodes, so its offsets index the SSML.
224
+ Entities are never read as words, and an element whose content the engine
225
+ replaces (`<sub>`, `<say-as>`, `<phoneme>`) takes one mark before its opening
226
+ tag, valued with its text.
227
+
228
+ Studio voices take no `<mark>` tags. A voice whose name contains `Studio`, or
229
+ one Google answers with its Studio `<mark>`-unsupported error, is synthesized as
230
+ audio only and returns `speechMarks: []`, so `ServerTTSProvider` reports no word
231
+ boundaries for it.
177
232
 
178
233
  ## Capabilities
179
234
 
180
235
  | Feature | Support |
181
236
  |---------|---------|
182
- | Speech Marks | ✅ Via SSML marks |
183
- | SSML | ✅ Full 1.1 |
184
- | Pitch Control | ✅ SSML |
185
- | Rate Control | ✅ SSML |
186
- | Volume Control | ❌ Client-side |
187
- | Max Text Length | 5000 chars |
188
- | Audio Formats | MP3, WAV, OGG |
237
+ | Speech Marks | Word marks via SSML marks; none for Studio voices |
238
+ | SSML | Google's supported subset |
239
+ | Pitch Control | Plain text; none for Studio voices |
240
+ | Rate Control | Plain text, through `<prosody>` |
241
+ | Volume Control | Not supported |
242
+ | Max Text Length | 5000 characters |
243
+ | Audio Formats | MP3, WAV (LINEAR16), OGG (Opus) |
244
+ | Sample Rate | Per request (default 24000 Hz) |
245
+
246
+ The [provider comparison](../../docs/accessibility/tts-architecture.md#provider-comparison)
247
+ sets these against Polly and the custom transport.
189
248
 
190
249
  ## Cost
191
250
 
192
- - **Standard voices:** $4 per 1M characters
193
- - **WaveNet (neural) voices:** $16 per 1M characters
194
- - **Studio voices:** $16 per 1M characters
195
- - **Speech marks (timepoints):** Included (no extra charge)
196
-
197
- Pricing is competitive with AWS Polly.
198
-
199
- ## Supported Voices
200
-
201
- Popular voices include:
251
+ Current rates: <https://cloud.google.com/text-to-speech/pricing>
202
252
 
203
- ### Standard Voices
204
- - **English (US):** en-US-Standard-A/B/C/D/E/F/G/H/I/J
205
- - **English (UK):** en-GB-Standard-A/B/C/D/F
206
- - **Spanish:** es-ES-Standard-A/B/C/D
207
- - **French:** fr-FR-Standard-A/B/C/D/E
208
- - **German:** de-DE-Standard-A/B/C/D/E/F
209
- - **Italian:** it-IT-Standard-A/B/C/D
210
- - **Portuguese:** pt-BR-Standard-A/B/C
253
+ ## Voices
211
254
 
212
- ### WaveNet (Neural) Voices
213
- - **English (US):** en-US-Wavenet-A/B/C/D/E/F/G/H/I/J
214
- - **English (UK):** en-GB-Wavenet-A/B/C/D/F
215
- - **Spanish:** es-ES-Wavenet-B/C/D
216
- - **French:** fr-FR-Wavenet-A/B/C/D/E
217
- - **German:** de-DE-Wavenet-A/B/C/D/E/F
218
- - **Italian:** it-IT-Wavenet-A/B/C/D
219
- - **Portuguese:** pt-BR-Wavenet-A/B/C
255
+ Google's [voice list](https://docs.cloud.google.com/text-to-speech/docs/list-voices-and-types)
256
+ names the voices per language and type; `getVoices()` returns those the project
257
+ can use. Names follow `{languageCode}-{voiceType}-{variant}`:
220
258
 
221
- ### Studio Voices
222
- - **English (US):** en-US-Studio-O/Q
223
- - **English (UK):** en-GB-Studio-B/C
224
-
225
- Use `getVoices()` for the complete list of 200+ voices.
226
-
227
- ## Voice Naming Convention
228
-
229
- Google Cloud voices follow the pattern: `{languageCode}-{voiceType}-{variant}`
230
-
231
- Examples:
232
- - `en-US-Wavenet-A` - US English, WaveNet (neural), variant A
233
- - `es-ES-Standard-B` - Spanish (Spain), Standard, variant B
234
- - `fr-FR-Studio-A` - French, Studio (premium), variant A
259
+ - `en-US-Wavenet-A`: US English, WaveNet (neural), variant A
260
+ - `es-ES-Standard-B`: Spanish (Spain), Standard, variant B
261
+ - `fr-FR-Studio-A`: French, Studio (premium), variant A
235
262
 
236
263
  ## Error Handling
237
264
 
@@ -256,52 +283,20 @@ try {
256
283
  }
257
284
  ```
258
285
 
259
- ## Google Cloud IAM Permissions
260
-
261
- Required IAM permissions for the service account:
262
-
263
- ```json
264
- {
265
- "role": "roles/cloudtexttospeech.user",
266
- "permissions": [
267
- "texttospeech.operations.get",
268
- "texttospeech.voices.list",
269
- "texttospeech.voices.synthesize"
270
- ]
271
- }
272
- ```
273
-
274
- Or use the predefined role:
275
-
276
- ```bash
277
- gcloud projects add-iam-policy-binding PROJECT_ID \
278
- --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \
279
- --role="roles/cloudtexttospeech.user"
280
- ```
281
-
282
- ## Comparison with AWS Polly
283
-
284
- | Feature | Google Cloud TTS | AWS Polly |
285
- |---------|------------------|-----------|
286
- | **Voices** | 200+ voices | 60+ voices |
287
- | **Languages** | 50+ languages | 25+ languages |
288
- | **Speech Marks** | Via SSML marks | Native |
289
- | **API Calls** | Single call | Two parallel calls |
290
- | **Max Text** | 5000 chars | 3000 chars |
291
- | **Neural Cost** | $16/1M chars | $16/1M chars |
292
- | **Standard Cost** | $4/1M chars | $4/1M chars |
293
- | **Authentication** | Flexible (4 methods) | AWS credentials |
294
- | **Region** | Global service | Region-specific |
295
-
296
- ## How Speech Marks Work
297
-
298
- Unlike AWS Polly (which provides native speech marks), Google Cloud TTS requires SSML mark injection:
286
+ `synthesize` maps Google's gRPC status codes:
299
287
 
300
- 1. The provider automatically parses your text and injects `<mark>` tags before each word
301
- 2. Google Cloud TTS returns timepoints corresponding to these marks
302
- 3. The provider converts timepoints to the unified speech mark format
288
+ | Google failure | `TTSErrorCode` |
289
+ |----------------|----------------|
290
+ | `PERMISSION_DENIED` (7) | `AUTHENTICATION_ERROR` |
291
+ | `RESOURCE_EXHAUSTED` (8) | `RATE_LIMIT_EXCEEDED` |
292
+ | `INVALID_ARGUMENT` (3) | `INVALID_REQUEST` |
293
+ | Anything else, a response without audio included | `PROVIDER_ERROR` |
303
294
 
304
- This process is transparent to the user - just set `includeSpeechMarks: true`.
295
+ Every `getVoices` failure is a `PROVIDER_ERROR`. Request validation runs before
296
+ any Google call and rejects with a `TTSError`: `TEXT_TOO_LONG` for text over
297
+ 5000 characters, `INVALID_REQUEST` for empty text, an unsupported format, or a
298
+ rate or pitch out of range. A call before `initialize` throws a `TTSError` with
299
+ `INITIALIZATION_ERROR`.
305
300
 
306
301
  ## Advanced Configuration
307
302
 
@@ -319,7 +314,7 @@ await provider.initialize({
319
314
  ```typescript
320
315
  await provider.initialize({
321
316
  projectId: 'my-project',
322
- enableLogging: true, // Logs SSML injection and speech marks extraction
317
+ enableLogging: true, // Logs SSML detection and the marks injected and extracted
323
318
  });
324
319
  ```
325
320
 
@@ -2,6 +2,7 @@
2
2
  * Google Cloud Text-to-Speech server-side TTS provider
3
3
  * @module @pie-players/tts-server-google
4
4
  */
5
+ import { v1beta1 } from "@google-cloud/text-to-speech";
5
6
  import { BaseTTSProvider, type GetVoicesOptions, type ServerProviderCapabilities, type SynthesizeRequest, type SynthesizeResponse, type TTSServerConfig, type Voice } from "@pie-players/tts-server-core";
6
7
  /**
7
8
  * Google Cloud Text-to-Speech provider configuration.
@@ -41,7 +42,7 @@ export interface GoogleCloudTTSConfig extends TTSServerConfig {
41
42
  * Voice type: 'wavenet' (neural), 'standard', or 'studio' (premium)
42
43
  *
43
44
  * @default 'wavenet'
44
- * @note WaveNet: $16/1M chars, Standard: $4/1M chars, Studio: $16/1M chars
45
+ * @see https://cloud.google.com/text-to-speech/pricing
45
46
  */
46
47
  voiceType?: "wavenet" | "standard" | "studio";
47
48
  /**
@@ -74,7 +75,7 @@ export interface GoogleCloudTTSConfig extends TTSServerConfig {
74
75
  * Features:
75
76
  * - Speech marks support via SSML mark injection (millisecond precision)
76
77
  * - WaveNet (neural), Standard, and Studio voice types
77
- * - 200+ voices across 50+ languages
78
+ * - Voices across many languages, listed by `getVoices`
78
79
  * - Full SSML support
79
80
  * - Single API call for audio + speech marks
80
81
  */
@@ -99,12 +100,15 @@ export declare class GoogleCloudTTSProvider extends BaseTTSProvider {
99
100
  * @performance Completes in ~10-50ms
100
101
  */
101
102
  initialize(config: GoogleCloudTTSConfig): Promise<void>;
103
+ /** The Google client every request goes through; a test subclass supplies a fake. */
104
+ protected createClient(clientConfig: ConstructorParameters<typeof v1beta1.TextToSpeechClient>[0]): v1beta1.TextToSpeechClient;
102
105
  /**
103
106
  * Synthesize speech with Google Cloud TTS
104
107
  */
105
108
  synthesize(request: SynthesizeRequest): Promise<SynthesizeResponse>;
106
109
  /**
107
- * Synthesize audio stream only (no speech marks)
110
+ * Synthesize audio stream only (no speech marks). A Studio voice takes
111
+ * `<prosody>` rate but not pitch, so its request drops pitch.
108
112
  */
109
113
  private synthesizeAudio;
110
114
  /**
@@ -116,16 +120,17 @@ export declare class GoogleCloudTTSProvider extends BaseTTSProvider {
116
120
  */
117
121
  private injectSSMLMarks;
118
122
  /**
119
- * Extract words from existing SSML (simplified version for v1)
123
+ * Inject a mark before each word of authored SSML, keeping its markup.
124
+ * Offsets index the SSML itself. An element whose content the engine
125
+ * replaces (`<sub>`, `<say-as>`, `<phoneme>`) takes one mark before its
126
+ * opening tag, valued with its text, since a mark may not sit inside it.
120
127
  */
121
128
  private extractWordsFromSSML;
129
+ private markTextNode;
122
130
  /**
123
131
  * Extract speech marks from Google's timepoints
124
132
  */
125
133
  private extractSpeechMarksFromTimepoints;
126
- /**
127
- * Detect if text contains SSML markup
128
- */
129
134
  /**
130
135
  * Get available voices from Google Cloud TTS
131
136
  */
@@ -4,6 +4,44 @@
4
4
  */
5
5
  import { v1beta1, protos } from "@google-cloud/text-to-speech";
6
6
  import { BaseTTSProvider, TTSError, TTSErrorCode, } from "@pie-players/tts-server-core";
7
+ const DEFAULT_SAMPLE_RATE_HERTZ = 24000;
8
+ const CONTENT_TYPE_BY_ENCODING = {
9
+ MP3: "audio/mpeg",
10
+ LINEAR16: "audio/wav",
11
+ OGG_OPUS: "audio/ogg",
12
+ };
13
+ // The segment of a Google voice name that identifies its voice type.
14
+ const VOICE_TYPE_NAME_SEGMENT = {
15
+ wavenet: "Wavenet",
16
+ standard: "Standard",
17
+ studio: "Studio",
18
+ };
19
+ // gRPC status codes the Google client reports in `error.code`.
20
+ const GRPC_STATUS = {
21
+ INVALID_ARGUMENT: 3,
22
+ PERMISSION_DENIED: 7,
23
+ RESOURCE_EXHAUSTED: 8,
24
+ };
25
+ // Google voice names start with their locale: "en-US-Wavenet-A" → "en-US".
26
+ const languageCodeOf = (voice) => voice.split("-").slice(0, 2).join("-");
27
+ // Letters, digits and combining marks, so accented words stay whole.
28
+ const WORD_PATTERN = /[\p{L}\p{M}\p{N}'\u2019]+/gu;
29
+ const SSML_TEXT_TOKEN_PATTERN = /&(?:#\d+|#x[\da-f]+|[a-z]+);|[\p{L}\p{M}\p{N}'\u2019]+/giu;
30
+ const SSML_ENTITIES = {
31
+ amp: "&",
32
+ lt: "<",
33
+ gt: ">",
34
+ quot: '"',
35
+ apos: "'",
36
+ };
37
+ const decodeSSMLText = (text) => text.replace(/&(#\d+|#x[\da-f]+|[a-z]+);/gi, (entity, name) => {
38
+ if (name[0] !== "#")
39
+ return SSML_ENTITIES[name.toLowerCase()] ?? entity;
40
+ const code = name[1] === "x" || name[1] === "X"
41
+ ? Number.parseInt(name.slice(2), 16)
42
+ : Number.parseInt(name.slice(1), 10);
43
+ return Number.isFinite(code) ? String.fromCodePoint(code) : entity;
44
+ });
7
45
  /**
8
46
  * Google Cloud Text-to-Speech Server Provider
9
47
  *
@@ -13,7 +51,7 @@ import { BaseTTSProvider, TTSError, TTSErrorCode, } from "@pie-players/tts-serve
13
51
  * Features:
14
52
  * - Speech marks support via SSML mark injection (millisecond precision)
15
53
  * - WaveNet (neural), Standard, and Studio voice types
16
- * - 200+ voices across 50+ languages
54
+ * - Voices across many languages, listed by `getVoices`
17
55
  * - Full SSML support
18
56
  * - Single API call for audio + speech marks
19
57
  */
@@ -58,7 +96,6 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
58
96
  this.audioEncoding = config.audioEncoding || "MP3";
59
97
  this.enableLogging = config.enableLogging || false;
60
98
  try {
61
- // Initialize Google Cloud TTS client
62
99
  const clientConfig = {
63
100
  projectId: config.projectId,
64
101
  };
@@ -78,7 +115,7 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
78
115
  }
79
116
  }
80
117
  // Else: Use Application Default Credentials (ADC)
81
- this.client = new v1beta1.TextToSpeechClient(clientConfig);
118
+ this.client = this.createClient(clientConfig);
82
119
  this.initialized = true;
83
120
  if (this.enableLogging) {
84
121
  console.log("[GoogleCloudTTS] Initialized successfully");
@@ -88,6 +125,10 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
88
125
  throw new TTSError(TTSErrorCode.INITIALIZATION_ERROR, `Failed to initialize Google Cloud TTS: ${error instanceof Error ? error.message : String(error)}`, { error }, this.providerId);
89
126
  }
90
127
  }
128
+ /** The Google client every request goes through; a test subclass supplies a fake. */
129
+ createClient(clientConfig) {
130
+ return new v1beta1.TextToSpeechClient(clientConfig);
131
+ }
91
132
  /**
92
133
  * Synthesize speech with Google Cloud TTS
93
134
  */
@@ -95,173 +136,112 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
95
136
  this.ensureInitialized();
96
137
  const capabilities = this.getCapabilities();
97
138
  this.validateRequest(request, capabilities);
98
- const voiceType = { wavenet: "Wavenet", standard: "Standard", studio: "Studio" }[this.voiceType];
139
+ const voiceType = VOICE_TYPE_NAME_SEGMENT[this.voiceType];
99
140
  const voice = await this.resolveRequestVoice(request, this.defaultVoice, (candidate) => candidate.id.includes(voiceType));
100
141
  const startTime = Date.now();
142
+ const respond = (result, speechMarks) => ({
143
+ audio: result.audio,
144
+ contentType: result.contentType,
145
+ speechMarks,
146
+ metadata: {
147
+ providerId: this.providerId,
148
+ voice,
149
+ duration: (Date.now() - startTime) / 1000,
150
+ charCount: request.text.length,
151
+ cached: false,
152
+ timestamp: new Date().toISOString(),
153
+ },
154
+ });
101
155
  try {
102
- // Check if speech marks are requested
103
156
  if (request.includeSpeechMarks !== false && !this.isStudioVoice(voice)) {
104
- // Use SSML marks injection for precise word timing
105
157
  const result = await this.synthesizeWithSpeechMarks(request, voice);
106
- const duration = (Date.now() - startTime) / 1000;
107
- return {
108
- audio: result.audio,
109
- contentType: result.contentType,
110
- speechMarks: result.speechMarks,
111
- metadata: {
112
- providerId: this.providerId,
113
- voice,
114
- duration,
115
- charCount: request.text.length,
116
- cached: false,
117
- timestamp: new Date().toISOString(),
118
- },
119
- };
120
- }
121
- else {
122
- // Audio only (no speech marks)
123
- const result = await this.synthesizeAudio(request, voice);
124
- const duration = (Date.now() - startTime) / 1000;
125
- return {
126
- audio: result.audio,
127
- contentType: result.contentType,
128
- speechMarks: [],
129
- metadata: {
130
- providerId: this.providerId,
131
- voice,
132
- duration,
133
- charCount: request.text.length,
134
- cached: false,
135
- timestamp: new Date().toISOString(),
136
- },
137
- };
158
+ return respond(result, result.speechMarks);
138
159
  }
160
+ return respond(await this.synthesizeAudio(request, voice), []);
139
161
  }
140
162
  catch (error) {
141
163
  // Studio voices do not support SSML <mark> tags used for timing extraction.
142
164
  // Fall back to audio-only synthesis rather than failing the request.
143
165
  if (request.includeSpeechMarks !== false &&
144
166
  this.isStudioMarkUnsupportedError(error)) {
145
- const result = await this.synthesizeAudio(request, voice);
146
- const duration = (Date.now() - startTime) / 1000;
147
- return {
148
- audio: result.audio,
149
- contentType: result.contentType,
150
- speechMarks: [],
151
- metadata: {
152
- providerId: this.providerId,
153
- voice,
154
- duration,
155
- charCount: request.text.length,
156
- cached: false,
157
- timestamp: new Date().toISOString(),
158
- },
159
- };
167
+ try {
168
+ return respond(await this.synthesizeAudio(request, voice, true), []);
169
+ }
170
+ catch (fallbackError) {
171
+ throw this.mapGoogleErrorToTTSError(fallbackError);
172
+ }
160
173
  }
161
174
  throw this.mapGoogleErrorToTTSError(error);
162
175
  }
163
176
  }
164
177
  /**
165
- * Synthesize audio stream only (no speech marks)
178
+ * Synthesize audio stream only (no speech marks). A Studio voice takes
179
+ * `<prosody>` rate but not pitch, so its request drops pitch.
166
180
  */
167
- async synthesizeAudio(request, voice) {
181
+ async synthesizeAudio(request, voice, studio = this.isStudioVoice(voice)) {
168
182
  if (this.detectSSML(request.text) && this.enableLogging) {
169
183
  console.log("[GoogleCloudTTS] Detected SSML content");
170
184
  }
171
- const { text, isSsml } = this.applyProsody(request.text, request);
172
- // Parse voice name to extract language code
173
- const languageCode = voice.split("-").slice(0, 2).join("-"); // e.g., "en-US" from "en-US-Wavenet-A"
174
- // Map our audio encoding to Google's enum
175
- const audioEncodingMap = {
176
- MP3: "MP3",
177
- LINEAR16: "LINEAR16",
178
- OGG_OPUS: "OGG_OPUS",
179
- };
185
+ const { text, isSsml } = this.applyProsody(request.text, studio ? { ...request, pitch: undefined } : request);
180
186
  const [response] = await this.client.synthesizeSpeech({
181
187
  input: isSsml ? { ssml: text } : { text },
182
188
  voice: {
183
- languageCode,
189
+ languageCode: languageCodeOf(voice),
184
190
  name: voice,
185
191
  },
186
192
  audioConfig: {
187
- audioEncoding: audioEncodingMap[this.audioEncoding],
188
- sampleRateHertz: request.sampleRate || 24000,
193
+ audioEncoding: this.audioEncoding,
194
+ sampleRateHertz: request.sampleRate || DEFAULT_SAMPLE_RATE_HERTZ,
189
195
  },
190
196
  });
191
197
  if (!response.audioContent) {
192
198
  throw new Error("No audio content received from Google Cloud TTS");
193
199
  }
194
- // Convert Uint8Array to Buffer
195
- const audioBuffer = Buffer.from(response.audioContent);
196
- const contentTypeMap = {
197
- MP3: "audio/mpeg",
198
- LINEAR16: "audio/wav",
199
- OGG_OPUS: "audio/ogg",
200
- };
201
200
  return {
202
- audio: audioBuffer,
203
- contentType: contentTypeMap[this.audioEncoding],
201
+ audio: Buffer.from(response.audioContent),
202
+ contentType: CONTENT_TYPE_BY_ENCODING[this.audioEncoding],
204
203
  };
205
204
  }
206
205
  /**
207
206
  * Synthesize with speech marks using SSML mark injection
208
207
  */
209
208
  async synthesizeWithSpeechMarks(request, voice) {
210
- // Check if the text is already SSML
211
209
  const isUserSSML = this.detectSSML(request.text);
212
- // If user provided SSML, we need to inject marks within the existing SSML
213
- // For simplicity in v1, we'll inject marks for plain text only. Already-SSML
214
- // input skips prosody wrapping too: injecting <prosody> into markup the
215
- // caller authored themselves would require parsing it.
210
+ // Authored SSML keeps its markup and takes marks in its text nodes. It
211
+ // skips prosody wrapping: injecting <prosody> into markup the caller
212
+ // authored would require parsing it.
216
213
  const { ssml, wordMap } = isUserSSML
217
214
  ? this.extractWordsFromSSML(request.text)
218
215
  : this.injectSSMLMarks(request.text, this.buildProsodyAttrs(request));
219
216
  if (this.enableLogging) {
220
217
  console.log(`[GoogleCloudTTS] Injected ${wordMap.length} SSML marks`);
221
218
  }
222
- // Parse voice name to extract language code
223
- const languageCode = voice.split("-").slice(0, 2).join("-");
224
- // Map our audio encoding to Google's enum
225
- const audioEncodingMap = {
226
- MP3: "MP3",
227
- LINEAR16: "LINEAR16",
228
- OGG_OPUS: "OGG_OPUS",
229
- };
230
219
  // Single API call with timepoint tracking enabled
231
- const responseArray = await this.client.synthesizeSpeech({
220
+ const [response] = await this.client.synthesizeSpeech({
232
221
  input: { ssml },
233
222
  voice: {
234
- languageCode,
223
+ languageCode: languageCodeOf(voice),
235
224
  name: voice,
236
225
  },
237
226
  audioConfig: {
238
- audioEncoding: audioEncodingMap[this.audioEncoding],
239
- sampleRateHertz: request.sampleRate || 24000,
227
+ audioEncoding: this.audioEncoding,
228
+ sampleRateHertz: request.sampleRate || DEFAULT_SAMPLE_RATE_HERTZ,
240
229
  },
241
230
  enableTimePointing: [
242
231
  protos.google.cloud.texttospeech.v1beta1.SynthesizeSpeechRequest
243
232
  .TimepointType.SSML_MARK,
244
233
  ],
245
234
  });
246
- const response = responseArray[0];
247
235
  if (!response.audioContent) {
248
236
  throw new Error("No audio content received from Google Cloud TTS");
249
237
  }
250
- // Convert Uint8Array to Buffer
251
- const audioBuffer = Buffer.from(response.audioContent);
252
- const contentTypeMap = {
253
- MP3: "audio/mpeg",
254
- LINEAR16: "audio/wav",
255
- OGG_OPUS: "audio/ogg",
256
- };
257
- // Extract speech marks from timepoints
258
238
  const speechMarks = this.extractSpeechMarksFromTimepoints(response.timepoints || [], wordMap);
259
239
  if (this.enableLogging) {
260
240
  console.log(`[GoogleCloudTTS] Extracted ${speechMarks.length} speech marks`);
261
241
  }
262
242
  return {
263
- audio: audioBuffer,
264
- contentType: contentTypeMap[this.audioEncoding],
243
+ audio: Buffer.from(response.audioContent),
244
+ contentType: CONTENT_TYPE_BY_ENCODING[this.audioEncoding],
265
245
  speechMarks,
266
246
  };
267
247
  }
@@ -270,27 +250,16 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
270
250
  */
271
251
  injectSSMLMarks(text, prosodyAttrs = "") {
272
252
  const words = [];
273
- const wordRegex = /\b[\w']+\b/g;
274
- let match;
275
- let markIndex = 0;
276
- while ((match = wordRegex.exec(text)) !== null) {
277
- const word = match[0];
278
- const start = match.index;
279
- const end = start + word.length;
280
- const markName = `w${markIndex++}`;
281
- words.push({ word, start, end, markName });
282
- }
283
- // Build SSML with marks
284
253
  let ssml = prosodyAttrs ? `<speak><prosody ${prosodyAttrs}>` : "<speak>";
285
254
  let lastEnd = 0;
286
- for (const { word, start, end, markName } of words) {
287
- // Add text before word (including whitespace and punctuation)
288
- ssml += this.escapeSSML(text.slice(lastEnd, start));
289
- // Add marked word
290
- ssml += `<mark name="${markName}"/>${this.escapeSSML(word)}`;
291
- lastEnd = end;
255
+ for (const match of text.matchAll(WORD_PATTERN)) {
256
+ const word = match[0];
257
+ const start = match.index;
258
+ const markName = `w${words.length}`;
259
+ words.push({ word, start, end: start + word.length, markName });
260
+ ssml += `${this.escapeSSML(text.slice(lastEnd, start))}<mark name="${markName}"/>${this.escapeSSML(word)}`;
261
+ lastEnd = start + word.length;
292
262
  }
293
- // Add remaining text
294
263
  ssml +=
295
264
  this.escapeSSML(text.slice(lastEnd)) +
296
265
  (prosodyAttrs ? "</prosody>" : "") +
@@ -298,16 +267,63 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
298
267
  return { ssml, wordMap: words };
299
268
  }
300
269
  /**
301
- * Extract words from existing SSML (simplified version for v1)
270
+ * Inject a mark before each word of authored SSML, keeping its markup.
271
+ * Offsets index the SSML itself. An element whose content the engine
272
+ * replaces (`<sub>`, `<say-as>`, `<phoneme>`) takes one mark before its
273
+ * opening tag, valued with its text, since a mark may not sit inside it.
302
274
  */
303
275
  extractWordsFromSSML(ssmlText) {
304
- // For now, just strip SSML tags and inject marks
305
- // More sophisticated SSML parsing can be added in future versions
306
- const plainText = ssmlText
307
- .replace(/<[^>]+>/g, " ") // Remove all tags
308
- .replace(/\s+/g, " ") // Normalize whitespace
309
- .trim();
310
- return this.injectSSMLMarks(plainText);
276
+ const words = [];
277
+ const mark = (word, start, end) => {
278
+ const markName = `w${words.length}`;
279
+ words.push({ word, start, end, markName });
280
+ return `<mark name="${markName}"/>`;
281
+ };
282
+ let ssml = "";
283
+ let cursor = 0;
284
+ for (const tag of ssmlText.matchAll(/<[^>]*>/g)) {
285
+ if (tag.index < cursor)
286
+ continue;
287
+ ssml += this.markTextNode(ssmlText, cursor, tag.index, mark);
288
+ const opaque = /^<\s*(sub|say-as|phoneme)\b(?![^>]*\/>)[^>]*>$/i.exec(tag[0]);
289
+ const close = opaque
290
+ ? new RegExp(`</\\s*${opaque[1]}\\s*>`, "i")
291
+ : null;
292
+ const closeMatch = close?.exec(ssmlText.slice(tag.index));
293
+ if (opaque && closeMatch) {
294
+ const contentStart = tag.index + tag[0].length;
295
+ const contentEnd = tag.index + closeMatch.index;
296
+ const raw = ssmlText.slice(contentStart, contentEnd);
297
+ const inner = raw.trim();
298
+ const innerStart = contentStart + raw.indexOf(inner);
299
+ const elementEnd = contentEnd + closeMatch[0].length;
300
+ ssml += inner
301
+ ? mark(decodeSSMLText(inner), innerStart, innerStart + inner.length)
302
+ : "";
303
+ ssml += ssmlText.slice(tag.index, elementEnd);
304
+ cursor = elementEnd;
305
+ continue;
306
+ }
307
+ ssml += tag[0];
308
+ cursor = tag.index + tag[0].length;
309
+ }
310
+ ssml += this.markTextNode(ssmlText, cursor, ssmlText.length, mark);
311
+ return { ssml, wordMap: words };
312
+ }
313
+ // Marks each word of an SSML text node, which is already escaped, so
314
+ // entities stay whole and are never read as words.
315
+ markTextNode(ssmlText, from, to, mark) {
316
+ const segment = ssmlText.slice(from, to);
317
+ let out = "";
318
+ let lastEnd = 0;
319
+ for (const match of segment.matchAll(SSML_TEXT_TOKEN_PATTERN)) {
320
+ if (match[0].startsWith("&"))
321
+ continue;
322
+ const start = from + match.index;
323
+ out += `${segment.slice(lastEnd, match.index)}${mark(match[0], start, start + match[0].length)}${match[0]}`;
324
+ lastEnd = match.index + match[0].length;
325
+ }
326
+ return out + segment.slice(lastEnd);
311
327
  }
312
328
  /**
313
329
  * Extract speech marks from Google's timepoints
@@ -318,7 +334,6 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
318
334
  }
319
335
  const speechMarks = [];
320
336
  for (const timepoint of timepoints) {
321
- // Find corresponding word in our map
322
337
  const wordInfo = wordMap.find((w) => w.markName === timepoint.markName);
323
338
  if (wordInfo &&
324
339
  timepoint.timeSeconds !== undefined &&
@@ -332,12 +347,8 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
332
347
  });
333
348
  }
334
349
  }
335
- // Sort by time
336
350
  return speechMarks.sort((a, b) => a.time - b.time);
337
351
  }
338
- /**
339
- * Detect if text contains SSML markup
340
- */
341
352
  /**
342
353
  * Get available voices from Google Cloud TTS
343
354
  */
@@ -353,7 +364,6 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
353
364
  return response.voices
354
365
  .map((voice) => this.mapGoogleVoiceToVoice(voice))
355
366
  .filter((voice) => {
356
- // Apply filters
357
367
  if (options?.gender && voice.gender !== options.gender) {
358
368
  return false;
359
369
  }
@@ -414,10 +424,10 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
414
424
  // W3C Standard features
415
425
  standard: {
416
426
  supportsSSML: true, // ✅ Full SSML 1.1 support
417
- supportsPitch: true, // ✅ Via SSML <prosody pitch>
427
+ supportsPitch: this.voiceType !== "studio", // Via SSML <prosody pitch>; Studio voices lack it
418
428
  supportsRate: true, // ✅ Via SSML <prosody rate>
419
429
  supportsVolume: false, // ❌ Not supported (handle client-side)
420
- supportsMultipleVoices: true, // ✅ 200+ voices across 50+ languages
430
+ supportsMultipleVoices: true, // ✅ Listed by getVoices
421
431
  maxTextLength: 5000, // Google Cloud TTS limit per request
422
432
  },
423
433
  // Provider-specific extensions
@@ -428,8 +438,6 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
428
438
  // Google Cloud-specific features
429
439
  providerSpecific: {
430
440
  voiceTypes: ["standard", "wavenet", "studio"],
431
- voicesCount: 200, // ~200+ voices available
432
- languagesCount: 50, // 50+ languages supported
433
441
  supportsAudioProfiles: true, // Audio device profiles
434
442
  supportsEffects: false, // No built-in effects
435
443
  supportsEmotions: false, // No emotion control
@@ -442,21 +450,17 @@ export class GoogleCloudTTSProvider extends BaseTTSProvider {
442
450
  * Map Google Cloud errors to TTSError codes
443
451
  */
444
452
  mapGoogleErrorToTTSError(error) {
445
- const message = error.message || String(error);
446
- // Check for specific Google Cloud error codes
447
- if (error.code === 7) {
448
- // PERMISSION_DENIED
453
+ const err = error;
454
+ const message = err.message || String(error);
455
+ if (err.code === GRPC_STATUS.PERMISSION_DENIED) {
449
456
  return new TTSError(TTSErrorCode.AUTHENTICATION_ERROR, `Google Cloud authentication failed: ${message}`, { error }, this.providerId);
450
457
  }
451
- if (error.code === 8) {
452
- // RESOURCE_EXHAUSTED
458
+ if (err.code === GRPC_STATUS.RESOURCE_EXHAUSTED) {
453
459
  return new TTSError(TTSErrorCode.RATE_LIMIT_EXCEEDED, `Google Cloud rate limit exceeded: ${message}`, { error }, this.providerId);
454
460
  }
455
- if (error.code === 3) {
456
- // INVALID_ARGUMENT
461
+ if (err.code === GRPC_STATUS.INVALID_ARGUMENT) {
457
462
  return new TTSError(TTSErrorCode.INVALID_REQUEST, `Invalid request to Google Cloud TTS: ${message}`, { error }, this.providerId);
458
463
  }
459
- // Default to provider error
460
464
  return new TTSError(TTSErrorCode.PROVIDER_ERROR, `Google Cloud TTS error: ${message}`, { error }, this.providerId);
461
465
  }
462
466
  /**
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@pie-players/tts-server-google",
3
- "version": "0.3.75",
3
+ "version": "1.0.0-beta.0",
4
4
  "description": "Google Cloud Text-to-Speech provider for server-side TTS with speech marks support",
5
5
  "type": "module",
6
6
  "main": "./dist/index.js",
@@ -17,8 +17,7 @@
17
17
  "scripts": {
18
18
  "build": "tsc",
19
19
  "dev": "tsc --watch",
20
- "test": "vitest run --passWithNoTests",
21
- "test:coverage": "vitest --coverage"
20
+ "test": "vitest run"
22
21
  },
23
22
  "keywords": [
24
23
  "tts",
@@ -40,7 +39,7 @@
40
39
  },
41
40
  "dependencies": {
42
41
  "@google-cloud/text-to-speech": "^7.1.1",
43
- "@pie-players/tts-server-core": "0.3.75"
42
+ "@pie-players/tts-server-core": "1.0.0-beta.0"
44
43
  },
45
44
  "devDependencies": {
46
45
  "typescript": "^5.9.3",