@pyai/sdk 0.2.3 → 0.4.0

package/README.md

@@ -8,10 +8,10 @@ dependencies; runs in the browser and Node 18+.
 ## PyAI products
 
 - **[Hear](https://pyai.com/models/hear)**, Lightning-fast, telephony-native **speech-to-text**. Whisper-compatible transcription tuned for real phone-call audio, with live streaming partials so your app reacts mid-sentence, plus async batch transcription for big archives. `POST /v1/audio/transcriptions`
-- **[Speak](https://pyai.com/models/speak)**, Ultra-realistic **text-to-speech** that starts speaking in tens of milliseconds. Stream lifelike, expressive voices, choose from 36 studio-quality presets, or clone any voice instantly, for free. `POST /v1/audio/speech`
+- **[Speak](https://pyai.com/models/speak)**, Ultra-realistic **text-to-speech** that starts speaking in tens of milliseconds. Stream lifelike, expressive voices, choose from 144 stock voices, or clone any voice instantly, for free. `POST /v1/audio/speech`
 - **[Omni](https://pyai.com/models/omni)** _(flagship)_, One **API for a complete, end-to-end voice AI agent**. A single WebSocket where your agent listens, thinks, and speaks, grounded in your knowledge bases and tools, with human-like turn-taking and instant barge-in, no STT, LLM, or TTS to stitch together yourself. `wss://api.pyai.com/v1/omni`
 - **[Trace](https://pyai.com/models/trace)** _(flagship)_, The **compliance API that keeps your AI agents safe**. Trace automatically checks every call for HIPAA, TCPA, and PII risks (plus your own brand-voice rules), flags the exact rule broken, redacts sensitive data, and seals each call with a tamper-evident audit trail, so a risky conversation never slips through. `GET /v1/trace/interactions`
-- **[Cue](https://pyai.com/models/cue)**, Realtime **turn detection + knowledge-grounded context** for your own stack. Bring your own LLM and voice; Cue nails the hard part, knowing the instant a speaker finishes and surfacing the right context. `wss://api.pyai.com/v1/audio/transcriptions/stream`
+- **Cue**, reserved turn-detection and grounding fields on Hear streaming. Grounding is not active on the serving route yet.
 - **[AMD](https://pyai.com/models/amd)**, **Answering-machine detection** that tells your dialer who or what answered, human, voicemail, IVR, iPhone/Google screening, dead number, fax, in a fraction of Twilio's dead-air dwell, with the reason. A one-line-TwiML Twilio Media Streams drop-in; billed per answered call (first 5,000/month free). `wss://api.pyai.com/v1/amd/stream`
 - **[Telephony](https://pyai.com/models/telephony)**, Instant **managed phone numbers** for your voice agents. Provision a US number and route live calls straight into an Omni agent, no carrier contracts, no telephony glue. `POST /v1/telephony/numbers`
 
@@ -93,6 +93,16 @@ omni.sendDtmf("5");
 omni.close();
 ```
 
+Live `0x02` transcript bodies are plain UTF-8 caller-text deltas, not JSON.
+`onTranscript` receives the normalized
+`{ event:"transcript", role:"user", text, final:false, mode:"delta" }` shape.
+Coalesce successive deltas for the current caller turn. Bounded direct JSON
+bodies remain accepted for older bridges.
+
+`rate` configures caller input. A `rate: 16000` session still receives 24 kHz
+agent audio; `rate: 8000` receives 8 kHz. Read `hello.audio_out` before playback.
+Omni has no commit frame—keep streaming silence during caller pauses.
+
 From the browser, mint an ephemeral token server-side with
 `pyai.omni.createSession({ allowedOrigins })` and pass it as `token` so the page
 never holds a secret key:
@@ -101,34 +111,70 @@ never holds a secret key:
 const omni = pyai.omni.connect({ token: session.token, configure: { voice_id, persona } });
 ```
 
-> Omni uses the native `wss://api.pyai.com/v1/omni` surface and is **zero-state**
->, no agent to create, `sessionLabel` is an optional opaque tag (never
-> required). Need the raw socket? `pyai.realtimeURL({ product: "omni" })` +
-> `pyai.realtimeSubprotocol()` (or `pyai.connectRealtime()`) still work;
-> `product: "flow"` uses `/v1/realtime`. The older `/v2/omni/chat` URL and the
-> `agentId` option are deprecated but still work.
+> Omni connects only to `wss://api.pyai.com/v1/omni` and is **zero-state**, no
+> agent to create. `sessionLabel` is an optional opaque tag (never required).
+> Need the raw socket? Use `pyai.realtimeURL({ sessionLabel })` with
+> `pyai.realtimeSubprotocol()` (or `pyai.connectRealtime()`).
+> The raw URL helper accepts canonical `format`, `rate`, and `api_key` query
+> parameters; retired connect aliases, model selectors, and token query names
+> throw instead of being translated.
 
-## Streaming speech-to-text (Hear / Cue)
+## Streaming speech-to-text (Hear)
 
 `transcriptions.stream()` hides the WebSocket frame protocol behind callbacks.
-It opens `wss://api.pyai.com/v1/audio/transcriptions/stream` (key carried as the
+It opens `wss://api.pyai.com/v1/audio/transcriptions/stream?protocol=pyai-hear-v1` (key carried as the
 WS subprotocol, so it works in the browser), routes the wire frames to
-`onPartial`/`onFinal`/`onError`, and gives you `sendAudio`, `commit()`, and
-`close()`:
+`onConfigAck`/`onPartial`/`onFinal`/`onError`, and gives you `sendAudio`,
+`configureEndpointing()`, `commit()`, and `close()`:
 
 ```ts
 const hear = pyai.audio.transcriptions.stream({
   sampleRate: 16000,
+  endpointingMs: 800, // minimum trailing pause; may wait up to max(800, 1500) ms
+  vocabulary: ["Nguyen", "SKU-99"],
+  onConfigAck: (ack) => {
+    if (ack.warnings.length) throw new Error(JSON.stringify(ack.warnings));
+  },
   onPartial: (f) => console.log("…", f.text),
-  onFinal: (f) => console.log("✓", f.text, `(${f.audio_ms}ms)`),
+  onFinal: (f) => console.log("✓", f.text, f.endpoint_reason),
   onError: (e) => console.error(e),
 });
 
-micChunks.on("data", (pcm16) => hear.sendAudio(pcm16)); // binary frames
-vad.on("end", () => hear.commit());                     // force-finalize an utterance
+micChunks.on("data", (pcm16) => hear.sendAudio(pcm16)); // keep sending silence through pauses
+hear.configureEndpointing(950);                         // update without reconnecting
+vad.on("end", () => hear.commit());                     // optional forced final
 // hear.close() also flushes a final for any buffered audio
 ```
 
+Streaming uses up to five sanitized vocabulary terms. To store organization
+suggestions, use a key with `hear:configure` and set explicit activation
+profiles first:
+
+```ts
+await pyai.hear.vocabulary.set({
+  terms: ["Nguyen", "SKU-99"],
+  enabledFor: ["batch", "hear_stream"],
+});
+```
+
+Request-level terms come first. Stored suggestions fill remaining slots up to
+five. The effective list is fixed when a stream opens or a batch job is created.
+Organization Hear terms are not used by Omni and are not populated
+automatically from CRM or dialer data. A managed Agent can opt in with its own
+list:
+
+```ts
+const agent = await pyai.agents.create({
+  name: "Front desk",
+  vocabulary: ["Nguyen", "Acme Dental", "SKU-99"],
+});
+
+await pyai.agents.update(agent.agent_id, { vocabulary: [] });
+```
+
+The Agent list is sanitized to at most five effective terms and fixed when a
+new session starts. An empty list turns the feature off.
+
 Frame `type`s, WS close codes, and error `code`s are exported as named
 constants so you never hardcode a magic string:
 
@@ -163,8 +209,8 @@ const ulaw = await pyai.audio.speech({
 
 | `response_format` | sample rates (Hz) | Content-Type |
 |---|---|---|
-| `mp3` (default) | 8000 / 16000 / 24000 / 48000 | `audio/mpeg` |
-| `wav` | 8000 / 16000 / 24000 / 48000 | `audio/wav` |
+| `wav` (default) | 8000 / 16000 / 24000 / 48000 | `audio/wav` |
+| `mp3` | 8000 / 16000 / 24000 / 48000 | `audio/mpeg` |
 | `opus` | 8000 / 16000 / 24000 / 48000 | `audio/ogg` |
 | `aac` | 8000 / 16000 / 24000 / 48000 | `audio/aac` |
 | `flac` | 8000 / 16000 / 24000 / 48000 | `audio/flac` |
@@ -175,7 +221,7 @@ const ulaw = await pyai.audio.speech({
 `sample_rate` is optional, omit it for the engine's native 24 kHz (`g711_*` is
 always 8 kHz). The set is typed (`SpeechFormat`) and exported as `SPEECH_FORMATS`
 / `SPEECH_SAMPLE_RATES` for dropdowns and validation. Any other value is a
-`400 unsupported_format`; omit `response_format` for the default `mp3`.
+`400 unsupported_format`; omit `response_format` for the default `wav`.
 
 > See [`examples/speak-telephony-formats`](../../examples/speak-telephony-formats)
 > for the full before/after: ~120 lines of resampler + μ-law replaced by one
@@ -292,7 +338,7 @@ Speak→Hear round-trip, and prints remediation hints for any failure:
 ```bash
 export PYAI_API_KEY=pyai_test_...
 npx pyai doctor
-# PASS  key (/v1/me), env=test; 3 scope(s): hear:transcribe, voice:synthesize, hear:stream
+# PASS  key (/v1/me), env=test; 3 scope(s): hear:transcribe, speak:synthesize, hear:stream
 # PASS  models.list, 12 models
 # PASS  voices.list, 38 voices
 # PASS  speak→hear round-trip, synth 45210 bytes → "the quick brown fox…"