@mindstudio-ai/remy 0.1.264 → 0.1.265
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -78,6 +78,25 @@ Classify by how the method actually behaves, not by what it is named. A "lookup"
|
|
|
78
78
|
external service is `slow`. When in doubt between `fast` and `slow`, pick `slow` — a needless
|
|
79
79
|
preamble is mildly chatty; an unexplained silence feels broken.
|
|
80
80
|
|
|
81
|
+
### Forwarding results to the screen (`forwardResult`)
|
|
82
|
+
|
|
83
|
+
A tool block may declare `forwardResult: true`. On completion, the platform then delivers the
|
|
84
|
+
tool's raw return value to the session's browser on the SDK's `toolCall` event (`result` field) —
|
|
85
|
+
so the UI can render what the agent just did (the citation it found, the record it pulled up, the
|
|
86
|
+
booking it made) in lockstep with the spoken answer. No polling, no key-threading, no model
|
|
87
|
+
involvement: the correlation is platform-guaranteed and scoped to that one session's client.
|
|
88
|
+
|
|
89
|
+
Opt in deliberately, per tool. The forwarded payload is the method's raw return — the same data
|
|
90
|
+
the model sees — so only enable it on tools whose returns are safe to render for the user in the
|
|
91
|
+
call (no internal fields you wouldn't show on screen). Payloads over ~32KB serialized arrive as
|
|
92
|
+
`resultTruncated: true` with no data — keep forwarded returns compact, or have the UI fetch big
|
|
93
|
+
data itself. Failed calls never forward anything.
|
|
94
|
+
|
|
95
|
+
For backend-side correlation (writing results to a table keyed by the call, custom channels), the
|
|
96
|
+
method itself can read `session.voiceSessionId` / `session.visitorId` from the agent SDK
|
|
97
|
+
(`import { session } from '@mindstudio-ai/agent'`) — the same id the browser holds as
|
|
98
|
+
`session.sessionId`, guaranteed by the platform rather than echoed by the model.
|
|
99
|
+
|
|
81
100
|
### Tool descriptions say results out loud
|
|
82
101
|
|
|
83
102
|
Follow the agent-interface principles for tool descriptions (when to use and when not, parameter
|
|
@@ -115,7 +134,8 @@ Two shapes, one `model` field:
|
|
|
115
134
|
can be *any* chat model — the right choice when the app's reasoning demands a specific model, or
|
|
116
135
|
when the agent interface already uses one and the voice should think identically. The blessed
|
|
117
136
|
streaming pairing is `"stt": "deepgram-nova-3", "tts": "cartesia-sonic-3"` — the lowest-latency
|
|
118
|
-
combination the platform wires; prefer it unless there's a reason not to
|
|
137
|
+
combination the platform wires; prefer it unless there's a reason not to (ElevenLabs TTS,
|
|
138
|
+
`"tts": "elevenlabs-tts"`, is also wired when its voice library fits better). One nuance: cascaded
|
|
119
139
|
engines speak the `greeting` verbatim (they have a real TTS); speech-to-speech engines have the
|
|
120
140
|
model say it, so it may paraphrase slightly.
|
|
121
141
|
|
|
@@ -138,8 +158,8 @@ someone reading chat aloud.
|
|
|
138
158
|
- Exposing the whole method surface. Voice is the most curated interface the app has.
|
|
139
159
|
- A generic greeting ("Hello! How can I assist you today?"). The greeting is the first thing anyone
|
|
140
160
|
hears; make it the character's.
|
|
141
|
-
- Writing your own current-user placeholder — the platform appends a `## Current User` block
|
|
142
|
-
roles) to every system prompt at runtime.
|
|
161
|
+
- Writing your own current-user placeholder — the platform appends a `## Current User` block
|
|
162
|
+
(email, phone, roles) to every system prompt at runtime.
|
|
143
163
|
|
|
144
164
|
## Compiling the Voice Spec
|
|
145
165
|
|
|
@@ -184,7 +204,9 @@ session.on('stateChange', (state) => { }); // on() returns an unsubscri
|
|
|
184
204
|
// far (never a delta) — render by upserting on segmentId, not appending.
|
|
185
205
|
session.on('transcript', ({ role, segmentId, text, final }) => { });
|
|
186
206
|
|
|
187
|
-
|
|
207
|
+
// status: 'running' | 'done' | 'failed'. Tools declared with `forwardResult: true`
|
|
208
|
+
// carry their return value in `result` on 'done' (or `resultTruncated: true` if >~32KB).
|
|
209
|
+
session.on('toolCall', ({ method, status, result }) => { });
|
|
188
210
|
session.on('error', (err) => { });
|
|
189
211
|
|
|
190
212
|
session.mute(); session.unmute(); session.isMuted;
|
|
@@ -254,7 +276,7 @@ export async function callMeAboutMyOrder(input: { phone: string }) {
|
|
|
254
276
|
of which number was dialed (the user types any number into a field; identity comes from their
|
|
255
277
|
session, not the phone). Omitted/false → anonymous call; role-gated tools decline.
|
|
256
278
|
System/cron invocations have no human identity and always run anonymously.
|
|
257
|
-
- **Production needs a dedicated phone number.** The app owner attaches one ($
|
|
279
|
+
- **Production needs a dedicated phone number.** The app owner attaches one ($1/month) via the
|
|
258
280
|
dashboard or `mindstudio-prod voice numbers` (see "Managing the phone side from the CLI"
|
|
259
281
|
below) — it becomes the caller ID for every call, in dev sessions too, so users always see
|
|
260
282
|
the same number. Without one, deployed calls throw `phone_out_requires_dedicated_number`, and
|
|
@@ -322,17 +344,18 @@ The `mindstudio-prod voice` family covers numbers, the call log, and voice polic
|
|
|
322
344
|
|
|
323
345
|
```bash
|
|
324
346
|
mindstudio-prod voice numbers search --area-code 310 # available numbers to offer the user
|
|
325
|
-
mindstudio-prod voice numbers buy +13105551234 # buy + attach ($
|
|
347
|
+
mindstudio-prod voice numbers buy +13105551234 # buy + attach ($1/month — see below)
|
|
326
348
|
mindstudio-prod voice numbers release +13105551234 # permanent; no refund, ~15-day quarantine
|
|
327
349
|
mindstudio-prod voice sessions list --limit 10 # call log: web / phone-out / phone-in
|
|
328
350
|
mindstudio-prod voice sessions get <sessionId> # full transcript + cost breakdown
|
|
329
351
|
```
|
|
330
352
|
|
|
331
|
-
Also `voice numbers list`, `voice
|
|
332
|
-
|
|
353
|
+
Also `voice numbers list`, `voice numbers set-name` (outbound caller-ID display
|
|
354
|
+
name; 12-72h carrier propagation), `voice settings get`/`set` (concurrency, per-visitor,
|
|
355
|
+
max duration — `set` merges: only the settings you pass change). `--help` for flags.
|
|
333
356
|
|
|
334
357
|
**Never buy a number without the user's explicit confirmation** — it starts a recurring
|
|
335
|
-
$
|
|
358
|
+
$1/month workspace charge. Search first, present the options with the price, and only run
|
|
336
359
|
`numbers buy` after they've picked one and said yes.
|
|
337
360
|
|
|
338
361
|
Transcripts are how you iterate on a voice persona: after the user test-calls the agent, read
|
|
@@ -366,8 +389,7 @@ Frontmatter fields:
|
|
|
366
389
|
- `description` — one-liner for listings
|
|
367
390
|
- `model` — JSON string, two shapes: native speech-to-speech `{"model": <realtime model id>,
|
|
368
391
|
"voice": <voice id>}`, or cascaded `{"llm": <chat model id>, "stt": <transcription model id>,
|
|
369
|
-
"tts": <speech model id>, "voice": <voice id>}`.
|
|
370
|
-
Ids via `askMindStudioSdk`.
|
|
392
|
+
"tts": <speech model id>, "voice": <voice id>}`. Ids via `askMindStudioSdk`.
|
|
371
393
|
- `turnDetection` — optional; `{"eagerness": "low" | "medium" | "high"}` — how quickly the platform
|
|
372
394
|
decides the user finished speaking. High is snappier; low is more patient (users dictating
|
|
373
395
|
numbers or addresses). Default `medium`.
|
|
@@ -475,11 +497,11 @@ Current User block — don't re-fetch it in the context method.
|
|
|
475
497
|
## Platform Behavior
|
|
476
498
|
|
|
477
499
|
- Input schemas are derived from each method's contract — never hand-written.
|
|
478
|
-
- The platform appends a `## Current User` block (
|
|
479
|
-
never author a placeholder for it.
|
|
500
|
+
- The platform appends a `## Current User` block (email, phone, roles) to the system prompt at
|
|
501
|
+
runtime; never author a placeholder for it.
|
|
480
502
|
- Turn detection, barge-in (interruption truncates the agent's context to the audio the user
|
|
481
503
|
actually heard), and background-noise handling are platform-managed; `turnDetection.eagerness` is
|
|
482
|
-
the only knob.
|
|
504
|
+
the only knob (not yet wired on Gemini realtime engines — it's a no-op there).
|
|
483
505
|
- Sessions have a per-app concurrency limit and a maximum duration, both configurable in the app's
|
|
484
506
|
settings; an idle session is ended gracefully after a prompt. Voice minutes and model usage are
|
|
485
507
|
metered.
|
|
@@ -81,6 +81,6 @@ You have access to the `mindstudio` CLI, which exposes every SDK action as a com
|
|
|
81
81
|
### Production App Management
|
|
82
82
|
You have access to `mindstudio-prod`, a CLI for managing the user's production app. Use it via your bash tool. All output is JSON. Run `mindstudio-prod --help` or `mindstudio-prod <command> --help` to discover usage and available options.
|
|
83
83
|
|
|
84
|
-
Available commands: `requests` (server-side request logs, error rates, latency), `crashes` (frontend browser errors — grouped issues + drill-down to individual events), `analytics` (traffic, top pages/referrers/geo, AI-referral attribution, live counters), `releases` (deploy status, history), `diagnostics` (post-deploy Lighthouse audit), `domains` (custom subdomains and fully custom domains), `users` (list, set roles), `db` (query production sql db), `data` (live db operations like lift-from-dev), `methods` (list, invoke), `secrets` (list, get, set, delete), `files` (upload + manage CDN files), `datasources` (build and query document corpora), `prerender` (bot/crawler snapshots), `voice` (dedicated phone numbers: search/buy/release — buying bills $
|
|
84
|
+
Available commands: `requests` (server-side request logs, error rates, latency), `crashes` (frontend browser errors — grouped issues + drill-down to individual events), `analytics` (traffic, top pages/referrers/geo, AI-referral attribution, live counters), `releases` (deploy status, history), `diagnostics` (post-deploy Lighthouse audit), `domains` (custom subdomains and fully custom domains), `users` (list, set roles), `db` (query production sql db), `data` (live db operations like lift-from-dev), `methods` (list, invoke), `secrets` (list, get, set, delete), `files` (upload + manage CDN files), `datasources` (build and query document corpora), `prerender` (bot/crawler snapshots), `voice` (dedicated phone numbers: search/buy/release — buying bills $1/month, so never buy without the user's explicit confirmation; call log + transcripts; voice policy settings), `issues` (reference and manage externally-reported bugs and issues. Do not use this to track work you are doing with the user - only read from it and resolve issues if the user asks for help fixing a bug from the issues tracker).
|
|
85
85
|
|
|
86
86
|
Use when the user asks about production behavior (server errors via `requests`, browser crashes via `crashes`, traffic/engagement via `analytics`), wants to manage their live app (domains, users, roles), needs to seed or query production data, or wants to check release status.
|