@vellumai/assistant 0.4.11 → 0.4.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ARCHITECTURE.md +401 -385
- package/package.json +1 -1
- package/src/__tests__/guardian-verify-setup-skill-regression.test.ts +75 -61
- package/src/__tests__/registry.test.ts +235 -187
- package/src/__tests__/secure-keys.test.ts +27 -0
- package/src/__tests__/session-agent-loop.test.ts +521 -256
- package/src/__tests__/session-surfaces-task-progress.test.ts +1 -0
- package/src/__tests__/session-tool-setup-app-refresh.test.ts +1 -0
- package/src/__tests__/session-tool-setup-memory-scope.test.ts +1 -0
- package/src/__tests__/session-tool-setup-side-effect-flag.test.ts +1 -0
- package/src/__tests__/skills.test.ts +334 -276
- package/src/__tests__/slack-skill.test.ts +124 -0
- package/src/__tests__/starter-task-flow.test.ts +7 -17
- package/src/agent/loop.ts +10 -3
- package/src/config/bundled-skills/chatgpt-import/tools/chatgpt-import.ts +449 -0
- package/src/config/bundled-skills/doordash/SKILL.md +171 -0
- package/src/config/bundled-skills/doordash/__tests__/doordash-client.test.ts +203 -0
- package/src/config/bundled-skills/doordash/__tests__/doordash-session.test.ts +164 -0
- package/src/config/bundled-skills/doordash/doordash-cli.ts +1193 -0
- package/src/config/bundled-skills/doordash/doordash-entry.ts +22 -0
- package/src/config/bundled-skills/doordash/lib/cart-queries.ts +787 -0
- package/src/config/bundled-skills/doordash/lib/client.ts +1071 -0
- package/src/config/bundled-skills/doordash/lib/order-queries.ts +85 -0
- package/src/config/bundled-skills/doordash/lib/queries.ts +28 -0
- package/src/config/bundled-skills/doordash/lib/query-extractor.ts +94 -0
- package/src/config/bundled-skills/doordash/lib/search-queries.ts +203 -0
- package/src/config/bundled-skills/doordash/lib/session.ts +93 -0
- package/src/config/bundled-skills/doordash/lib/shared/errors.ts +61 -0
- package/src/config/bundled-skills/doordash/lib/shared/ipc.ts +32 -0
- package/src/config/bundled-skills/doordash/lib/shared/network-recorder.ts +380 -0
- package/src/config/bundled-skills/doordash/lib/shared/platform.ts +35 -0
- package/src/config/bundled-skills/doordash/lib/shared/recording-store.ts +43 -0
- package/src/config/bundled-skills/doordash/lib/shared/recording-types.ts +49 -0
- package/src/config/bundled-skills/doordash/lib/shared/truncate.ts +6 -0
- package/src/config/bundled-skills/doordash/lib/store-queries.ts +246 -0
- package/src/config/bundled-skills/doordash/lib/types.ts +367 -0
- package/src/config/bundled-skills/google-calendar/SKILL.md +4 -5
- package/src/config/bundled-skills/google-oauth-setup/SKILL.md +41 -41
- package/src/config/bundled-skills/messaging/SKILL.md +59 -42
- package/src/config/bundled-skills/messaging/TOOLS.json +14 -92
- package/src/config/bundled-skills/messaging/tools/gmail-archive-by-query.ts +5 -1
- package/src/config/bundled-skills/messaging/tools/gmail-batch-archive.ts +11 -2
- package/src/config/bundled-skills/messaging/tools/gmail-outreach-scan.ts +8 -1
- package/src/config/bundled-skills/messaging/tools/gmail-sender-digest.ts +12 -4
- package/src/config/bundled-skills/messaging/tools/gmail-unsubscribe.ts +5 -1
- package/src/config/bundled-skills/messaging/tools/messaging-archive-by-sender.ts +5 -1
- package/src/config/bundled-skills/messaging/tools/messaging-sender-digest.ts +5 -2
- package/src/config/bundled-skills/notion/SKILL.md +240 -0
- package/src/config/bundled-skills/notion-oauth-setup/SKILL.md +127 -0
- package/src/config/bundled-skills/oauth-setup/SKILL.md +144 -0
- package/src/config/bundled-skills/phone-calls/SKILL.md +76 -45
- package/src/config/bundled-skills/skills-catalog/SKILL.md +32 -29
- package/src/config/bundled-skills/slack/SKILL.md +49 -0
- package/src/config/bundled-skills/slack/TOOLS.json +167 -0
- package/src/config/bundled-skills/slack/tools/shared.ts +23 -0
- package/src/config/bundled-skills/{messaging → slack}/tools/slack-add-reaction.ts +2 -5
- package/src/config/bundled-skills/slack/tools/slack-channel-details.ts +33 -0
- package/src/config/bundled-skills/slack/tools/slack-configure-channels.ts +75 -0
- package/src/config/bundled-skills/{messaging → slack}/tools/slack-delete-message.ts +2 -5
- package/src/config/bundled-skills/{messaging → slack}/tools/slack-leave-channel.ts +2 -5
- package/src/config/bundled-skills/slack/tools/slack-scan-digest.ts +193 -0
- package/src/config/{vellum-skills → bundled-skills}/sms-setup/SKILL.md +29 -22
- package/src/config/{vellum-skills → bundled-skills}/telegram-setup/SKILL.md +17 -14
- package/src/config/{vellum-skills → bundled-skills}/twilio-setup/SKILL.md +20 -5
- package/src/config/bundled-tool-registry.ts +292 -267
- package/src/config/schema.ts +1 -1
- package/src/daemon/handlers/skills.ts +334 -234
- package/src/daemon/ipc-contract/messages.ts +2 -0
- package/src/daemon/ipc-contract/surfaces.ts +2 -0
- package/src/daemon/lifecycle.ts +358 -221
- package/src/daemon/response-tier.ts +2 -0
- package/src/daemon/server.ts +453 -193
- package/src/daemon/session-agent-loop-handlers.ts +43 -2
- package/src/daemon/session-agent-loop.ts +3 -0
- package/src/daemon/session-lifecycle.ts +3 -0
- package/src/daemon/session-process.ts +1 -0
- package/src/daemon/session-surfaces.ts +22 -20
- package/src/daemon/session-tool-setup.ts +1 -0
- package/src/daemon/session.ts +5 -2
- package/src/messaging/outreach-classifier.ts +12 -5
- package/src/messaging/provider-types.ts +5 -0
- package/src/messaging/provider.ts +1 -1
- package/src/messaging/providers/gmail/adapter.ts +11 -5
- package/src/messaging/providers/gmail/client.ts +2 -0
- package/src/messaging/providers/slack/adapter.ts +1 -0
- package/src/messaging/providers/slack/client.ts +8 -0
- package/src/messaging/providers/slack/types.ts +5 -0
- package/src/runtime/http-errors.ts +33 -20
- package/src/runtime/http-server.ts +706 -291
- package/src/runtime/http-types.ts +26 -16
- package/src/runtime/routes/secret-routes.ts +57 -2
- package/src/runtime/routes/surface-action-routes.ts +66 -0
- package/src/runtime/routes/trust-rules-routes.ts +140 -0
- package/src/security/keychain-to-encrypted-migration.ts +59 -0
- package/src/security/secure-keys.ts +17 -0
- package/src/skills/frontmatter.ts +9 -7
- package/src/tools/apps/executors.ts +2 -1
- package/src/tools/tool-manifest.ts +44 -42
- package/src/tools/types.ts +9 -0
- package/src/__tests__/skill-mirror-parity.test.ts +0 -176
- package/src/config/vellum-skills/catalog.json +0 -63
- package/src/config/vellum-skills/chatgpt-import/tools/chatgpt-import.ts +0 -295
- package/src/skills/vellum-catalog-remote.ts +0 -166
- package/src/tools/skills/vellum-catalog.ts +0 -168
- /package/src/config/{vellum-skills → bundled-skills}/chatgpt-import/SKILL.md +0 -0
- /package/src/config/{vellum-skills → bundled-skills}/chatgpt-import/TOOLS.json +0 -0
- /package/src/config/{vellum-skills → bundled-skills}/deploy-fullstack-vercel/SKILL.md +0 -0
- /package/src/config/{vellum-skills → bundled-skills}/document-writer/SKILL.md +0 -0
- /package/src/config/{vellum-skills → bundled-skills}/guardian-verify-setup/SKILL.md +0 -0
- /package/src/config/{vellum-skills → bundled-skills}/slack-oauth-setup/SKILL.md +0 -0
- /package/src/config/{vellum-skills → bundled-skills}/trusted-contacts/SKILL.md +0 -0
|
@@ -0,0 +1,144 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: "OAuth Setup"
|
|
3
|
+
description: "Connect any OAuth service — create app credentials and authorize via browser automation"
|
|
4
|
+
user-invocable: true
|
|
5
|
+
includes: ["browser"]
|
|
6
|
+
metadata: {"vellum": {"emoji": "🔑"}}
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
You are helping the user connect an OAuth-based service to Vellum. This is a generic setup flow that works for any provider with a well-known OAuth config.
|
|
10
|
+
|
|
11
|
+
**Tone:** Be friendly and reassuring throughout. Narrate what you're doing in plain language so the user always knows what's happening. After each step, briefly confirm what was accomplished before moving on.
|
|
12
|
+
|
|
13
|
+
## Input
|
|
14
|
+
|
|
15
|
+
You will be given a `service` name (e.g., "discord", "linear", "spotify"). If not provided, ask the user which service they want to connect.
|
|
16
|
+
|
|
17
|
+
## Step 1: Read the service config
|
|
18
|
+
|
|
19
|
+
Use `credential_store` with `action: "describe"` and `service: "<name>"` to get the well-known OAuth config. This returns:
|
|
20
|
+
- `scopes` — permissions to request
|
|
21
|
+
- `redirectUri` — the callback URL to register
|
|
22
|
+
- `callbackTransport` — loopback or gateway
|
|
23
|
+
- `requiresClientSecret` — whether a client secret is needed
|
|
24
|
+
- `authUrl` / `tokenUrl` — OAuth endpoints
|
|
25
|
+
|
|
26
|
+
It may also include a `setup` object with rich metadata:
|
|
27
|
+
- `setup.displayName` — the provider's name
|
|
28
|
+
- `setup.dashboardUrl` — where to create the app
|
|
29
|
+
- `setup.appType` — what kind of app to create
|
|
30
|
+
- `setup.requiresClientSecret` — whether a client secret is needed
|
|
31
|
+
- `setup.notes` — provider-specific guidance
|
|
32
|
+
|
|
33
|
+
If no config is found, tell the user this service doesn't have a pre-configured setup and offer to help them configure it manually via `oauth2_connect`.
|
|
34
|
+
|
|
35
|
+
## Step 2: Choose the flow based on setup metadata
|
|
36
|
+
|
|
37
|
+
### If `setup` metadata is present (rich flow)
|
|
38
|
+
|
|
39
|
+
Continue to Step 3 (Rich Flow).
|
|
40
|
+
|
|
41
|
+
### If `setup` metadata is absent (manual flow)
|
|
42
|
+
|
|
43
|
+
The provider has OAuth config (endpoints, scopes) but no setup automation metadata. Guide the user through a manual app creation:
|
|
44
|
+
|
|
45
|
+
1. Tell the user: "I have the OAuth endpoints and scopes for this service, but I don't have developer dashboard automation for it. You'll need to create an OAuth app manually."
|
|
46
|
+
2. Provide the details they need:
|
|
47
|
+
- **Scopes to request:** list the `scopes` from the config
|
|
48
|
+
- **Redirect URI:** show the `redirectUri` value
|
|
49
|
+
- **Whether a client secret is required:** use the top-level `requiresClientSecret` field
|
|
50
|
+
3. Ask the user to:
|
|
51
|
+
- Go to the provider's developer dashboard
|
|
52
|
+
- Create an OAuth app (name it "Vellum Assistant")
|
|
53
|
+
- Set the redirect URI
|
|
54
|
+
- Configure the required scopes
|
|
55
|
+
- Copy the Client ID (and Client Secret if required)
|
|
56
|
+
4. Once they provide the credentials, skip to **Step 8: Connect**.
|
|
57
|
+
|
|
58
|
+
---
|
|
59
|
+
|
|
60
|
+
## Rich Flow (when `setup` is present)
|
|
61
|
+
|
|
62
|
+
### Step 3: Tell the user what's happening
|
|
63
|
+
|
|
64
|
+
Tell the user:
|
|
65
|
+
- "I'll walk you through creating a {setup.appType} so Vellum can connect to {setup.displayName}. The whole process takes a few minutes."
|
|
66
|
+
- "I'll be automating the {setup.displayName} developer dashboard in the browser — you'll be able to see everything I'm doing."
|
|
67
|
+
- "I'll ask for your approval before each major step, so nothing happens without your say-so."
|
|
68
|
+
|
|
69
|
+
### Step 4: Navigate to the developer dashboard
|
|
70
|
+
|
|
71
|
+
Use `browser_navigate` to go to `setup.dashboardUrl`.
|
|
72
|
+
|
|
73
|
+
Take a `browser_screenshot` and `browser_snapshot`:
|
|
74
|
+
- **If a sign-in page appears:** Tell the user to sign in and wait for confirmation.
|
|
75
|
+
- **If the dashboard loads:** Continue to Step 5.
|
|
76
|
+
|
|
77
|
+
### Step 5: Create an app
|
|
78
|
+
|
|
79
|
+
**Ask for approval before proceeding.** Use `ui_show` with `surface_type: "confirmation"` explaining what you're about to create.
|
|
80
|
+
|
|
81
|
+
Once approved:
|
|
82
|
+
1. Find and click the "Create App" / "New Application" / "New Integration" button (adapt to the provider's UI)
|
|
83
|
+
2. Name it "Vellum Assistant"
|
|
84
|
+
3. Follow any guidance from `setup.notes`
|
|
85
|
+
4. Complete the creation flow
|
|
86
|
+
|
|
87
|
+
Take a `browser_screenshot` to confirm.
|
|
88
|
+
|
|
89
|
+
### Step 6: Configure scopes/permissions
|
|
90
|
+
|
|
91
|
+
**Ask for approval before proceeding.** List the `scopes` from the config and explain what each grants.
|
|
92
|
+
|
|
93
|
+
Once approved, navigate to the OAuth/permissions section and add each scope. Follow `setup.notes` for any provider-specific guidance (e.g., "User Token Scopes" vs "Bot Token Scopes").
|
|
94
|
+
|
|
95
|
+
Take a `browser_screenshot` after adding all scopes.
|
|
96
|
+
|
|
97
|
+
### Step 7: Set redirect URL
|
|
98
|
+
|
|
99
|
+
Check the `redirectUri` from the config:
|
|
100
|
+
- If it mentions "not currently configured", `GATEWAY_BASE_URL`, or `INGRESS_PUBLIC_BASE_URL` — the user needs a public gateway URL configured. Check if one is set; if not, load the `public-ingress` skill first.
|
|
101
|
+
- If it says "automatic" — skip this step entirely (no redirect URI needed).
|
|
102
|
+
- Otherwise, enter the `redirectUri` exactly as provided.
|
|
103
|
+
|
|
104
|
+
Take a `browser_snapshot` to confirm.
|
|
105
|
+
|
|
106
|
+
### Step 7b: Extract credentials
|
|
107
|
+
|
|
108
|
+
Navigate to the app's credentials/settings section.
|
|
109
|
+
|
|
110
|
+
Use `browser_extract` to read:
|
|
111
|
+
1. **Client ID** (or equivalent)
|
|
112
|
+
2. **Client Secret** (if `requiresClientSecret` is true) — click "Show"/"Reveal" first if needed
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## Step 8: Connect
|
|
117
|
+
|
|
118
|
+
Tell the user you're opening the authorization page.
|
|
119
|
+
|
|
120
|
+
Use `credential_store` with:
|
|
121
|
+
```
|
|
122
|
+
action: "oauth2_connect"
|
|
123
|
+
service: "<service name>"
|
|
124
|
+
client_id: "<extracted client ID>"
|
|
125
|
+
client_secret: "<extracted client secret>" (if required)
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
Everything else (endpoints, scopes, params) is auto-filled from the well-known config. Wait for the flow to complete.
|
|
129
|
+
|
|
130
|
+
## Step 9: Celebrate!
|
|
131
|
+
|
|
132
|
+
Once connected, tell the user:
|
|
133
|
+
- If `setup` is present: "**{setup.displayName} is connected!** You're all set."
|
|
134
|
+
- If `setup` is absent: "**{service} is connected!** You're all set."
|
|
135
|
+
|
|
136
|
+
Summarize what was accomplished.
|
|
137
|
+
|
|
138
|
+
## Error Handling
|
|
139
|
+
|
|
140
|
+
- **Page load failures:** Retry once. If it still fails, ask the user to check their connection.
|
|
141
|
+
- **Element not found:** Take a fresh `browser_snapshot`. The provider's UI may have changed — adapt dynamically.
|
|
142
|
+
- **User declines an approval gate:** Don't push back. Explain why it matters, offer to retry or cancel.
|
|
143
|
+
- **OAuth flow timeout or failure:** Offer to retry. The app is already created, so only the connect step needs to be re-run.
|
|
144
|
+
- **Any unexpected state:** Take a `browser_screenshot` and `browser_snapshot`, describe what you see, and ask for guidance.
|
|
@@ -2,7 +2,8 @@
|
|
|
2
2
|
name: "Phone Calls"
|
|
3
3
|
description: "Set up Twilio for AI-powered voice calls — both outgoing calls on behalf of the user and incoming calls where the assistant answers as a receptionist"
|
|
4
4
|
user-invocable: true
|
|
5
|
-
metadata:
|
|
5
|
+
metadata:
|
|
6
|
+
{ "vellum": { "emoji": "📞", "requires": { "config": ["calls.enabled"] } } }
|
|
6
7
|
includes: ["public-ingress"]
|
|
7
8
|
---
|
|
8
9
|
|
|
@@ -34,6 +35,7 @@ When someone dials the assistant's Twilio phone number:
|
|
|
34
35
|
6. The assistant converses naturally, using ASK_GUARDIAN to consult the user when needed, just like outbound calls.
|
|
35
36
|
|
|
36
37
|
Three voice quality modes are available:
|
|
38
|
+
|
|
37
39
|
- **`twilio_standard`** (default) — Fully supported. Standard Twilio TTS with Google voices. No extra setup required.
|
|
38
40
|
- **`twilio_elevenlabs_tts`** — Fully supported. Uses ElevenLabs voices through Twilio ConversationRelay for more natural speech.
|
|
39
41
|
- **`elevenlabs_agent`** — **Experimental/restricted.** Full ElevenLabs conversational agent mode. Consultation bridging (`waiting_on_user`) is not yet supported in this mode; the runtime guard blocks it before any ElevenLabs API calls are made. See the "Runtime behavior" section below for fallback and strict-fail details.
|
|
@@ -60,8 +62,7 @@ If `hasCredentials` is `true`, `phoneNumber` is set, and `calls.enabled` is `tru
|
|
|
60
62
|
|
|
61
63
|
If Twilio is not yet configured, load the **twilio-setup** skill — it handles credential storage, phone number provisioning, and public ingress setup:
|
|
62
64
|
|
|
63
|
-
- Call `
|
|
64
|
-
- Then call `skill_load` with `skill: "twilio-setup"`.
|
|
65
|
+
- Call `skill_load` with `skill_id: "twilio-setup"` to load the dependency skill.
|
|
65
66
|
|
|
66
67
|
Once twilio-setup completes, return here to enable calls.
|
|
67
68
|
|
|
@@ -74,6 +75,7 @@ vellum config set calls.enabled true
|
|
|
74
75
|
```
|
|
75
76
|
|
|
76
77
|
Verify:
|
|
78
|
+
|
|
77
79
|
```bash
|
|
78
80
|
vellum config get calls.enabled
|
|
79
81
|
```
|
|
@@ -108,10 +110,10 @@ credential_store action=store service=twilio field=user_phone_number value=+1415
|
|
|
108
110
|
|
|
109
111
|
### Configuration reference
|
|
110
112
|
|
|
111
|
-
| Setting
|
|
112
|
-
|
|
113
|
-
| `calls.callerIdentity.allowPerCallOverride` | Whether per-call mode selection is allowed
|
|
114
|
-
| `calls.callerIdentity.userNumber`
|
|
113
|
+
| Setting | Description | Default |
|
|
114
|
+
| ------------------------------------------- | ------------------------------------------------------------------------------------------------ | --------- |
|
|
115
|
+
| `calls.callerIdentity.allowPerCallOverride` | Whether per-call mode selection is allowed | `true` |
|
|
116
|
+
| `calls.callerIdentity.userNumber` | Optional E.164 phone number for user-number mode (alternative to storing via `credential_store`) | _(empty)_ |
|
|
115
117
|
|
|
116
118
|
## DTMF Callee Verification
|
|
117
119
|
|
|
@@ -127,10 +129,10 @@ An optional verification step where the callee must enter a numeric code via the
|
|
|
127
129
|
|
|
128
130
|
### Configuration
|
|
129
131
|
|
|
130
|
-
| Setting
|
|
131
|
-
|
|
132
|
-
| `calls.verification.enabled`
|
|
133
|
-
| `calls.verification.codeLength` | Number of digits in the verification code | `6`
|
|
132
|
+
| Setting | Description | Default |
|
|
133
|
+
| ------------------------------- | ----------------------------------------- | ------- |
|
|
134
|
+
| `calls.verification.enabled` | Enable DTMF callee verification | `false` |
|
|
135
|
+
| `calls.verification.codeLength` | Number of digits in the verification code | `6` |
|
|
134
136
|
|
|
135
137
|
## Optional: Higher Quality Voice with ElevenLabs
|
|
136
138
|
|
|
@@ -218,6 +220,7 @@ vellum config set calls.voice.mode twilio_standard
|
|
|
218
220
|
## Making Outbound Calls
|
|
219
221
|
|
|
220
222
|
Use the `call_start` tool to place outbound calls. Every call requires:
|
|
223
|
+
|
|
221
224
|
- **phone_number**: The number to call in E.164 format (e.g. `+14155551234`)
|
|
222
225
|
- **task**: What the call should accomplish — this becomes the AI voice agent's objective
|
|
223
226
|
- **context** (optional): Additional background information for the conversation
|
|
@@ -225,16 +228,19 @@ Use the `call_start` tool to place outbound calls. Every call requires:
|
|
|
225
228
|
### Example calls:
|
|
226
229
|
|
|
227
230
|
**Making a reservation:**
|
|
231
|
+
|
|
228
232
|
```
|
|
229
233
|
call_start phone_number="+14155551234" task="Make a dinner reservation for 2 people tonight at 7pm" context="The user's name is John Smith. Prefer a table by the window if available."
|
|
230
234
|
```
|
|
231
235
|
|
|
232
236
|
**Calling a business:**
|
|
237
|
+
|
|
233
238
|
```
|
|
234
239
|
call_start phone_number="+18005551234" task="Check if they have a specific product in stock" context="Looking for a 65-inch Samsung OLED TV, model QN65S95D. Ask about availability and price."
|
|
235
240
|
```
|
|
236
241
|
|
|
237
242
|
**Following up on an appointment:**
|
|
243
|
+
|
|
238
244
|
```
|
|
239
245
|
call_start phone_number="+12125551234" task="Confirm the dentist appointment scheduled for next Tuesday at 2pm" context="The appointment is under the name Jane Doe, DOB 03/15/1990."
|
|
240
246
|
```
|
|
@@ -244,11 +250,13 @@ call_start phone_number="+12125551234" task="Confirm the dentist appointment sch
|
|
|
244
250
|
Implicit calls always use the assistant's Twilio number (`assistant_number`). Only specify `caller_identity_mode` when the user explicitly requests a different identity for a specific call.
|
|
245
251
|
|
|
246
252
|
**Default call (assistant number):**
|
|
253
|
+
|
|
247
254
|
```
|
|
248
255
|
call_start phone_number="+14155551234" task="Check store hours for today"
|
|
249
256
|
```
|
|
250
257
|
|
|
251
258
|
**Call from the user's own number:**
|
|
259
|
+
|
|
252
260
|
```
|
|
253
261
|
call_start phone_number="+14155551234" task="Check store hours for today" caller_identity_mode="user_number"
|
|
254
262
|
```
|
|
@@ -258,6 +266,7 @@ call_start phone_number="+14155551234" task="Check store hours for today" caller
|
|
|
258
266
|
### Phone number format
|
|
259
267
|
|
|
260
268
|
Phone numbers MUST be in E.164 format: `+` followed by country code and number with no spaces, dashes, or parentheses.
|
|
269
|
+
|
|
261
270
|
- US/Canada: `+1XXXXXXXXXX` (e.g. `+14155551234`)
|
|
262
271
|
- UK: `+44XXXXXXXXXX` (e.g. `+442071234567`)
|
|
263
272
|
- International: `+{country_code}{number}`
|
|
@@ -267,6 +276,7 @@ If the user provides a number in a different format, convert it to E.164 before
|
|
|
267
276
|
### Trial account limitations
|
|
268
277
|
|
|
269
278
|
On Twilio trial accounts, outbound calls can ONLY be made to **verified numbers**. If a call fails with a "not verified" error:
|
|
279
|
+
|
|
270
280
|
1. Tell the user they need to verify the number at https://console.twilio.com/us1/develop/phone-numbers/manage/verified
|
|
271
281
|
2. Or upgrade to a paid Twilio account to call any number
|
|
272
282
|
|
|
@@ -283,7 +293,7 @@ No additional configuration is needed beyond Twilio setup and `calls.enabled` be
|
|
|
283
293
|
|
|
284
294
|
### Guardian voice verification for inbound calls
|
|
285
295
|
|
|
286
|
-
For guardian verification setup,
|
|
296
|
+
For guardian verification setup, load the skill by calling `skill_load` with `skill_id: "guardian-verify-setup"`. This skill handles the full outbound verification flow; `phone-calls` does not orchestrate it inline. Do not use `call_start` to place guardian verification calls — the guardian outbound verification endpoints already place those calls.
|
|
287
297
|
|
|
288
298
|
Once a guardian binding exists for the voice channel, inbound callers may be prompted for verification before calls proceed. The relay server detects pending challenges and prompts callers: "Please enter your six-digit verification code using your keypad, or speak the digits now." If verification fails after 3 attempts, the call ends with "Verification failed. Goodbye."
|
|
289
299
|
|
|
@@ -345,6 +355,7 @@ The instruction is injected into the AI voice agent's conversation context as hi
|
|
|
345
355
|
### Ending a call early
|
|
346
356
|
|
|
347
357
|
Use `call_end` with the call session ID to terminate an active call:
|
|
358
|
+
|
|
348
359
|
```
|
|
349
360
|
call_end call_session_id="<session_id>" reason="User requested to end the call"
|
|
350
361
|
```
|
|
@@ -371,6 +382,7 @@ sqlite3 ~/.vellum/workspace/data/db/assistant.db \
|
|
|
371
382
|
```
|
|
372
383
|
|
|
373
384
|
This returns all messages in chronological order with:
|
|
385
|
+
|
|
374
386
|
- `role: "user"` — caller speech (prefixed with `[SPEAKER]` tags) and system events
|
|
375
387
|
- `role: "assistant"` — assistant responses, including `text` content and any `tool_use`/`tool_result` blocks
|
|
376
388
|
|
|
@@ -385,22 +397,22 @@ sqlite3 ~/.vellum/workspace/data/db/assistant.db \
|
|
|
385
397
|
|
|
386
398
|
### Additional tables for call metadata
|
|
387
399
|
|
|
388
|
-
| Table
|
|
389
|
-
|
|
390
|
-
| `call_sessions`
|
|
391
|
-
| `call_events`
|
|
392
|
-
| `notification_decisions`
|
|
393
|
-
| `notification_deliveries` | Notification delivery attempts
|
|
400
|
+
| Table | What it contains |
|
|
401
|
+
| ------------------------- | -------------------------------------------------------------- |
|
|
402
|
+
| `call_sessions` | Session metadata (start time, duration, phone numbers, status) |
|
|
403
|
+
| `call_events` | Granular event log for the call lifecycle |
|
|
404
|
+
| `notification_decisions` | Whether notifications were evaluated during the call |
|
|
405
|
+
| `notification_deliveries` | Notification delivery attempts |
|
|
394
406
|
|
|
395
407
|
### Key paths
|
|
396
408
|
|
|
397
|
-
| Resource
|
|
398
|
-
|
|
409
|
+
| Resource | Path |
|
|
410
|
+
| ------------------------------------------ | ------------------------------------------ |
|
|
399
411
|
| Daemon logs (caller-side transcripts only) | `~/.vellum/workspace/data/logs/vellum.log` |
|
|
400
|
-
| Full conversation database
|
|
401
|
-
| Messages table
|
|
402
|
-
| Call sessions table
|
|
403
|
-
| Call events table
|
|
412
|
+
| Full conversation database | `~/.vellum/workspace/data/db/assistant.db` |
|
|
413
|
+
| Messages table | `messages` (keyed by `conversation_id`) |
|
|
414
|
+
| Call sessions table | `call_sessions` |
|
|
415
|
+
| Call events table | `call_events` |
|
|
404
416
|
|
|
405
417
|
### Important
|
|
406
418
|
|
|
@@ -429,6 +441,7 @@ The `context` field is powerful — use it to give the agent background that hel
|
|
|
429
441
|
### Things the AI voice agent handles well
|
|
430
442
|
|
|
431
443
|
**Outbound calls:**
|
|
444
|
+
|
|
432
445
|
- Making reservations and appointments
|
|
433
446
|
- Checking business hours, availability, or pricing
|
|
434
447
|
- Confirming or rescheduling existing appointments
|
|
@@ -437,6 +450,7 @@ The `context` field is powerful — use it to give the agent background that hel
|
|
|
437
450
|
- Leaving voicemails (it will speak the message if voicemail picks up)
|
|
438
451
|
|
|
439
452
|
**Inbound calls:**
|
|
453
|
+
|
|
440
454
|
- Answering as a receptionist and routing caller requests to the user via ASK_GUARDIAN
|
|
441
455
|
- Taking messages when the user is unavailable
|
|
442
456
|
- Answering questions the assistant already knows from memory/context
|
|
@@ -453,27 +467,27 @@ The `context` field is powerful — use it to give the agent background that hel
|
|
|
453
467
|
|
|
454
468
|
All call-related settings can be managed via `vellum config`:
|
|
455
469
|
|
|
456
|
-
| Setting
|
|
457
|
-
|
|
458
|
-
| `calls.enabled`
|
|
459
|
-
| `calls.provider`
|
|
460
|
-
| `calls.maxDurationSeconds`
|
|
461
|
-
| `calls.userConsultTimeoutSeconds`
|
|
462
|
-
| `calls.disclosure.enabled`
|
|
463
|
-
| `calls.disclosure.text`
|
|
464
|
-
| `calls.model`
|
|
465
|
-
| `calls.callerIdentity.allowPerCallOverride` | Allow per-call caller identity selection
|
|
466
|
-
| `calls.callerIdentity.userNumber`
|
|
467
|
-
| `calls.voice.mode`
|
|
468
|
-
| `calls.voice.language`
|
|
469
|
-
| `calls.voice.transcriptionProvider`
|
|
470
|
-
| `calls.voice.fallbackToStandardOnError`
|
|
471
|
-
| `calls.voice.elevenlabs.voiceId`
|
|
472
|
-
| `calls.voice.elevenlabs.voiceModelId`
|
|
473
|
-
| `calls.voice.elevenlabs.agentId`
|
|
474
|
-
| `calls.voice.elevenlabs.speed`
|
|
475
|
-
| `calls.voice.elevenlabs.stability`
|
|
476
|
-
| `calls.voice.elevenlabs.similarityBoost`
|
|
470
|
+
| Setting | Description | Default |
|
|
471
|
+
| ------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
|
|
472
|
+
| `calls.enabled` | Master switch for the calling feature | `false` |
|
|
473
|
+
| `calls.provider` | Voice provider (currently only `twilio`) | `twilio` |
|
|
474
|
+
| `calls.maxDurationSeconds` | Maximum call length in seconds | `3600` (1 hour) |
|
|
475
|
+
| `calls.userConsultTimeoutSeconds` | How long to wait for user answers | `120` (2 min) |
|
|
476
|
+
| `calls.disclosure.enabled` | Whether the AI announces itself at call start | `true` |
|
|
477
|
+
| `calls.disclosure.text` | The disclosure message spoken at call start | `"At the very beginning of the call, introduce yourself as an assistant calling on behalf of my human."` |
|
|
478
|
+
| `calls.model` | Override LLM model for call orchestration | _(uses default model)_ |
|
|
479
|
+
| `calls.callerIdentity.allowPerCallOverride` | Allow per-call caller identity selection | `true` |
|
|
480
|
+
| `calls.callerIdentity.userNumber` | E.164 phone number for user-number mode | _(empty)_ |
|
|
481
|
+
| `calls.voice.mode` | Voice quality mode (`twilio_standard`, `twilio_elevenlabs_tts`, `elevenlabs_agent`) | `twilio_standard` |
|
|
482
|
+
| `calls.voice.language` | Language code for TTS and transcription | `en-US` |
|
|
483
|
+
| `calls.voice.transcriptionProvider` | Speech-to-text provider (`Deepgram`, `Google`) | `Deepgram` |
|
|
484
|
+
| `calls.voice.fallbackToStandardOnError` | Auto-fallback to standard Twilio TTS on ElevenLabs errors | `true` |
|
|
485
|
+
| `calls.voice.elevenlabs.voiceId` | Advanced/internal ElevenLabs voice identifier. Usually set by the assistant based on requested voice style | _(empty)_ |
|
|
486
|
+
| `calls.voice.elevenlabs.voiceModelId` | Optional Twilio ConversationRelay model suffix. Leave empty to send bare `voiceId` | _(empty)_ |
|
|
487
|
+
| `calls.voice.elevenlabs.agentId` | ElevenLabs agent ID (for `elevenlabs_agent` mode) | _(empty)_ |
|
|
488
|
+
| `calls.voice.elevenlabs.speed` | Playback speed (`0.7` – `1.2`) | `1.0` |
|
|
489
|
+
| `calls.voice.elevenlabs.stability` | Voice stability (`0.0` – `1.0`) | `0.5` |
|
|
490
|
+
| `calls.voice.elevenlabs.similarityBoost` | Voice similarity boost (`0.0` – `1.0`) | `0.75` |
|
|
477
491
|
|
|
478
492
|
### Adjusting settings
|
|
479
493
|
|
|
@@ -494,60 +508,77 @@ vellum config set calls.userConsultTimeoutSeconds 300
|
|
|
494
508
|
## Troubleshooting
|
|
495
509
|
|
|
496
510
|
### "Twilio credentials not configured"
|
|
511
|
+
|
|
497
512
|
Load the `twilio-setup` skill to store your Account SID and Auth Token.
|
|
498
513
|
|
|
499
514
|
### "Calls feature is disabled"
|
|
515
|
+
|
|
500
516
|
Run `vellum config set calls.enabled true`.
|
|
501
517
|
|
|
502
518
|
### "No public base URL configured"
|
|
519
|
+
|
|
503
520
|
Run the **public-ingress** skill to set up ngrok and configure `ingress.publicBaseUrl`.
|
|
504
521
|
|
|
505
522
|
### Call fails immediately after initiating
|
|
523
|
+
|
|
506
524
|
- Check that the phone number is in E.164 format
|
|
507
525
|
- Verify Twilio credentials are correct (wrong auth token causes API errors)
|
|
508
526
|
- On trial accounts, ensure the destination number is verified
|
|
509
527
|
- Check that the ngrok tunnel is still running (`curl -s http://127.0.0.1:4040/api/tunnels`)
|
|
510
528
|
|
|
511
529
|
### Call connects but no audio / one-way audio
|
|
530
|
+
|
|
512
531
|
- The ConversationRelay WebSocket may not be connecting. Check that `ingress.publicBaseUrl` is correct and the tunnel is active
|
|
513
532
|
- Verify the gateway is running at `$GATEWAY_BASE_URL`
|
|
514
533
|
|
|
515
534
|
### "Number not eligible for caller identity"
|
|
535
|
+
|
|
516
536
|
The user's phone number is not owned by or verified with the Twilio account. The number must be either purchased through Twilio or added as a verified caller ID at https://console.twilio.com/us1/develop/phone-numbers/manage/verified.
|
|
517
537
|
|
|
518
538
|
### "Per-call caller identity override is disabled"
|
|
539
|
+
|
|
519
540
|
The setting `calls.callerIdentity.allowPerCallOverride` is set to `false`, so per-call `caller_identity_mode` selection is not allowed. Re-enable overrides with `vellum config set calls.callerIdentity.allowPerCallOverride true`.
|
|
520
541
|
|
|
521
542
|
### Caller identity call fails on trial account
|
|
543
|
+
|
|
522
544
|
Twilio trial accounts can only place calls to verified numbers, regardless of caller identity mode. The user's phone number must also be verified with Twilio. Upgrade to a paid account or verify both the source and destination numbers.
|
|
523
545
|
|
|
524
546
|
### "This phone number is not allowed to be called"
|
|
547
|
+
|
|
525
548
|
Emergency numbers (911, 112, 999, 000, 110, 119) are permanently blocked for safety.
|
|
526
549
|
|
|
527
550
|
### ngrok tunnel URL changed
|
|
551
|
+
|
|
528
552
|
If you restarted ngrok, the public URL has changed. Update it:
|
|
553
|
+
|
|
529
554
|
```bash
|
|
530
555
|
vellum config set ingress.publicBaseUrl "<new-url>"
|
|
531
556
|
```
|
|
557
|
+
|
|
532
558
|
Or re-run the public-ingress skill to auto-detect and save the new URL.
|
|
533
559
|
|
|
534
560
|
### Call drops after 30 seconds of silence
|
|
561
|
+
|
|
535
562
|
The system has a 30-second silence timeout. If nobody speaks for 30 seconds, the agent will ask "Are you still there?" This is expected behavior.
|
|
536
563
|
|
|
537
564
|
### Call quality didn't improve after enabling ElevenLabs
|
|
565
|
+
|
|
538
566
|
- Verify `calls.voice.mode` is set to `twilio_elevenlabs_tts` or `elevenlabs_agent` (not still `twilio_standard`)
|
|
539
567
|
- Ask for the desired voice style again and try a different voice selection
|
|
540
568
|
- If configuring manually: check that `calls.voice.elevenlabs.voiceId` contains a valid ElevenLabs voice ID
|
|
541
569
|
- If mode is `elevenlabs_agent`, ensure `calls.voice.elevenlabs.agentId` is also set
|
|
542
570
|
|
|
543
571
|
### Twilio says "application error" right after answer
|
|
572
|
+
|
|
544
573
|
- This often means ConversationRelay rejected voice configuration after TwiML fetch
|
|
545
574
|
- Keep `calls.voice.elevenlabs.voiceModelId` empty first (bare `voiceId` mode)
|
|
546
575
|
- If you set `voiceModelId`, try clearing it and retesting:
|
|
547
576
|
`vellum config set calls.voice.elevenlabs.voiceModelId ""`
|
|
548
577
|
|
|
549
578
|
### ElevenLabs mode falls back to standard
|
|
579
|
+
|
|
550
580
|
When `calls.voice.fallbackToStandardOnError` is `true` (the default), the system silently falls back to standard Twilio TTS if ElevenLabs encounters an error or restriction. Check:
|
|
581
|
+
|
|
551
582
|
- For `elevenlabs_agent` mode: this mode is currently restricted (consultation bridging not yet supported) and will always fall back to standard when fallback is enabled. If fallback is disabled, the voice webhook returns HTTP 501.
|
|
552
583
|
- For `twilio_elevenlabs_tts` mode: verify `calls.voice.elevenlabs.voiceId` is set to a valid voice ID
|
|
553
584
|
- For invalid configs (missing voiceId/agentId): if fallback is disabled, the voice webhook returns HTTP 500 with the config error
|
|
@@ -1,52 +1,55 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: "Skills Catalog"
|
|
3
|
-
description: "
|
|
3
|
+
description: "Discover bundled skills and search/install community skills from Clawhub"
|
|
4
4
|
user-invocable: true
|
|
5
|
-
metadata: {"vellum": {"emoji": "🧩"}}
|
|
5
|
+
metadata: { "vellum": { "emoji": "🧩" } }
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
You can help the user discover
|
|
8
|
+
You can help the user discover what skills are available and find community skills to extend the assistant's capabilities.
|
|
9
9
|
|
|
10
|
-
##
|
|
10
|
+
## Bundled skills (first-party)
|
|
11
11
|
|
|
12
|
-
|
|
13
|
-
|
|
12
|
+
First-party skills are **bundled** with the assistant — they are compiled in and always available. They do not need to be installed or downloaded. To activate a bundled skill, use the `skill_load` tool:
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
skill_load skill=<skill-id>
|
|
14
16
|
```
|
|
15
17
|
|
|
16
|
-
|
|
17
|
-
- `id` — the skill identifier (used for install)
|
|
18
|
-
- `name` — human-readable name
|
|
19
|
-
- `description` — what the skill does
|
|
20
|
-
- `emoji` — optional emoji
|
|
21
|
-
- `includes` — optional list of dependency skill IDs
|
|
22
|
-
- `version` — optional version string
|
|
18
|
+
The skill catalog shown in the system prompt lists all bundled skills with their IDs. When a user asks about capabilities, refer to this list to find relevant bundled skills and load them as needed.
|
|
23
19
|
|
|
24
|
-
##
|
|
20
|
+
## Community skills (Clawhub)
|
|
25
21
|
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
22
|
+
Community skills are published on [Clawhub](https://clawhub.com) and can be searched and installed on demand.
|
|
23
|
+
|
|
24
|
+
### Searching for community skills
|
|
25
|
+
|
|
26
|
+
Use the `skill_load` tool to search the catalog, or check the system prompt's available skills list. The IPC `skills_search` message searches community skills on Clawhub. Bundled skills are already listed in the system prompt.
|
|
27
|
+
|
|
28
|
+
### Installing a community skill
|
|
29
|
+
|
|
30
|
+
Community skills are installed via the IPC `skills_install` message with a `slug` parameter. Once installed, they appear in `~/.vellum/workspace/skills/<slug>/` and can be loaded with `skill_load` like any other skill.
|
|
29
31
|
|
|
30
|
-
|
|
32
|
+
### Inspecting a community skill
|
|
31
33
|
|
|
32
|
-
|
|
34
|
+
Before installing, you can inspect a community skill via the IPC `skills_inspect` message with a `slug` parameter. This returns metadata (author, stats, version) and optionally the skill's SKILL.md content so the user can review it.
|
|
33
35
|
|
|
34
36
|
## Typical flow
|
|
35
37
|
|
|
36
38
|
1. **User asks about capabilities** — "Can you order food?" or "What can you do?"
|
|
37
|
-
-
|
|
39
|
+
- Check the bundled skills list in the system prompt
|
|
38
40
|
- Present relevant skills to the user
|
|
41
|
+
- Load any that match with `skill_load`
|
|
39
42
|
|
|
40
|
-
2. **User wants a
|
|
41
|
-
-
|
|
42
|
-
-
|
|
43
|
-
-
|
|
43
|
+
2. **User wants a capability not covered by bundled skills** — "Can you do X?"
|
|
44
|
+
- Search for community skills that provide the capability
|
|
45
|
+
- Present matching results with descriptions and install counts
|
|
46
|
+
- Install the chosen skill, then load it with `skill_load`
|
|
44
47
|
|
|
45
|
-
3. **Skill has dependencies** — if `includes` lists other skill IDs,
|
|
48
|
+
3. **Skill has dependencies** — if `includes` lists other skill IDs, load those first with `skill_load`
|
|
46
49
|
|
|
47
50
|
## Notes
|
|
48
51
|
|
|
49
|
-
-
|
|
50
|
-
-
|
|
51
|
-
- After installing
|
|
52
|
-
-
|
|
52
|
+
- Bundled skills are always available and do not need installation
|
|
53
|
+
- Community skills are installed to `~/.vellum/workspace/skills/<slug>/`
|
|
54
|
+
- After installing a community skill, it may need to be enabled in settings
|
|
55
|
+
- Skills can be enabled or disabled via feature flags without uninstalling them
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: "Slack"
|
|
3
|
+
description: "Scan channels, summarize threads, and manage Slack with privacy guardrails"
|
|
4
|
+
user-invocable: true
|
|
5
|
+
metadata: {"vellum": {"emoji": "💬"}}
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
You are a Slack assistant that helps users stay on top of their Slack workspace. Use the slack tools for channel scanning, thread summarization, and Slack-specific operations.
|
|
9
|
+
|
|
10
|
+
## Channel Scanning
|
|
11
|
+
|
|
12
|
+
When the user says "scan my Slack", "what's happening on Slack", or similar:
|
|
13
|
+
|
|
14
|
+
1. Call `slack_scan_digest` immediately. If preferred channels are configured, scan those; otherwise scan top active channels.
|
|
15
|
+
2. Present results progressively: overview first (channel names, message counts, top threads), then offer to drill into specific threads.
|
|
16
|
+
3. For threads the user wants to explore, use `messaging_read` with `thread_id` to fetch full content, then summarize with attribution (who said what, decisions made, open questions).
|
|
17
|
+
|
|
18
|
+
## Thread Summarization
|
|
19
|
+
|
|
20
|
+
When summarizing threads surfaced by the digest:
|
|
21
|
+
|
|
22
|
+
- Include attribution: who said what, what decisions were made, what questions remain open
|
|
23
|
+
- Note the thread's channel and whether it's private
|
|
24
|
+
- Keep summaries concise but complete
|
|
25
|
+
|
|
26
|
+
## Context Sharing (Privacy Rules)
|
|
27
|
+
|
|
28
|
+
**This is critical.** Channel privacy must be respected at all times:
|
|
29
|
+
|
|
30
|
+
- Content from `isPrivate: true` channels MUST NEVER be shared to other channels, DMs, or external destinations
|
|
31
|
+
- Before sharing any content, always check the source channel's `isPrivate` flag in the digest data
|
|
32
|
+
- If the user asks to share private channel content, explain that the content is from a private channel and cannot be shared externally, then offer alternatives (e.g., summarize the topic without quoting, ask the user to share manually)
|
|
33
|
+
- Public channel content can be shared with attribution ("From #channel: ...")
|
|
34
|
+
- Always confirm with the user before sending content to any destination
|
|
35
|
+
|
|
36
|
+
## Channel Preferences
|
|
37
|
+
|
|
38
|
+
Use `slack_configure_channels` to save and load preferred channels for scanning.
|
|
39
|
+
|
|
40
|
+
- After a first scan, suggest configuring defaults: "Want me to remember these channels for future scans?"
|
|
41
|
+
- Saved preferences are used automatically by `slack_scan_digest` when no specific channels are requested
|
|
42
|
+
|
|
43
|
+
## Watcher Integration
|
|
44
|
+
|
|
45
|
+
For real-time monitoring (not just on-demand scanning), the user can set up a Slack watcher using the watcher skill with the same channel IDs. Mention this if the user wants ongoing monitoring.
|
|
46
|
+
|
|
47
|
+
## Connection
|
|
48
|
+
|
|
49
|
+
Before using any Slack tool, verify that Slack is connected. If not connected, guide the user through the Slack setup flow described in the messaging skill.
|