@vellumai/assistant 0.3.4 → 0.3.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (122) hide show
  1. package/Dockerfile +2 -0
  2. package/README.md +37 -2
  3. package/package.json +1 -1
  4. package/scripts/ipc/generate-swift.ts +13 -0
  5. package/src/__tests__/__snapshots__/ipc-snapshot.test.ts.snap +100 -0
  6. package/src/__tests__/approval-hardcoded-copy-guard.test.ts +41 -0
  7. package/src/__tests__/approval-message-composer.test.ts +253 -0
  8. package/src/__tests__/call-domain.test.ts +12 -2
  9. package/src/__tests__/call-orchestrator.test.ts +70 -1
  10. package/src/__tests__/call-routes-http.test.ts +27 -2
  11. package/src/__tests__/channel-approval-routes.test.ts +21 -17
  12. package/src/__tests__/channel-approvals.test.ts +48 -1
  13. package/src/__tests__/channel-guardian.test.ts +74 -22
  14. package/src/__tests__/channel-readiness-service.test.ts +257 -0
  15. package/src/__tests__/config-schema.test.ts +2 -1
  16. package/src/__tests__/credential-security-invariants.test.ts +1 -0
  17. package/src/__tests__/daemon-lifecycle.test.ts +13 -12
  18. package/src/__tests__/dictation-mode-detection.test.ts +63 -0
  19. package/src/__tests__/entity-search.test.ts +615 -0
  20. package/src/__tests__/handlers-twilio-config.test.ts +407 -0
  21. package/src/__tests__/ipc-snapshot.test.ts +63 -0
  22. package/src/__tests__/messaging-send-tool.test.ts +65 -0
  23. package/src/__tests__/run-orchestrator-assistant-events.test.ts +4 -0
  24. package/src/__tests__/run-orchestrator.test.ts +22 -0
  25. package/src/__tests__/session-runtime-assembly.test.ts +85 -1
  26. package/src/__tests__/sms-messaging-provider.test.ts +125 -0
  27. package/src/__tests__/twilio-routes.test.ts +39 -3
  28. package/src/__tests__/twitter-cli-error-shaping.test.ts +2 -2
  29. package/src/__tests__/web-search.test.ts +1 -1
  30. package/src/__tests__/work-item-output.test.ts +110 -0
  31. package/src/calls/call-domain.ts +8 -5
  32. package/src/calls/call-orchestrator.ts +22 -11
  33. package/src/calls/twilio-config.ts +17 -11
  34. package/src/calls/twilio-rest.ts +276 -0
  35. package/src/calls/twilio-routes.ts +39 -1
  36. package/src/config/bundled-skills/knowledge-graph/SKILL.md +15 -0
  37. package/src/config/bundled-skills/knowledge-graph/TOOLS.json +56 -0
  38. package/src/config/bundled-skills/knowledge-graph/tools/graph-query.ts +185 -0
  39. package/src/config/bundled-skills/media-processing/SKILL.md +199 -0
  40. package/src/config/bundled-skills/media-processing/TOOLS.json +320 -0
  41. package/src/config/bundled-skills/media-processing/services/capability-registry.ts +137 -0
  42. package/src/config/bundled-skills/media-processing/services/event-detection-service.ts +280 -0
  43. package/src/config/bundled-skills/media-processing/services/feedback-aggregation.ts +144 -0
  44. package/src/config/bundled-skills/media-processing/services/feedback-store.ts +136 -0
  45. package/src/config/bundled-skills/media-processing/services/processing-pipeline.ts +261 -0
  46. package/src/config/bundled-skills/media-processing/services/retrieval-service.ts +95 -0
  47. package/src/config/bundled-skills/media-processing/services/timeline-service.ts +267 -0
  48. package/src/config/bundled-skills/media-processing/tools/analyze-keyframes.ts +301 -0
  49. package/src/config/bundled-skills/media-processing/tools/detect-events.ts +110 -0
  50. package/src/config/bundled-skills/media-processing/tools/extract-keyframes.ts +190 -0
  51. package/src/config/bundled-skills/media-processing/tools/generate-clip.ts +195 -0
  52. package/src/config/bundled-skills/media-processing/tools/ingest-media.ts +197 -0
  53. package/src/config/bundled-skills/media-processing/tools/media-diagnostics.ts +166 -0
  54. package/src/config/bundled-skills/media-processing/tools/media-status.ts +75 -0
  55. package/src/config/bundled-skills/media-processing/tools/query-media-events.ts +300 -0
  56. package/src/config/bundled-skills/media-processing/tools/recalibrate.ts +235 -0
  57. package/src/config/bundled-skills/media-processing/tools/select-tracking-profile.ts +142 -0
  58. package/src/config/bundled-skills/media-processing/tools/submit-feedback.ts +150 -0
  59. package/src/config/bundled-skills/messaging/SKILL.md +21 -6
  60. package/src/config/bundled-skills/messaging/tools/messaging-send.ts +5 -1
  61. package/src/config/bundled-skills/phone-calls/SKILL.md +2 -2
  62. package/src/config/bundled-skills/twitter/SKILL.md +19 -3
  63. package/src/config/defaults.ts +2 -1
  64. package/src/config/schema.ts +9 -3
  65. package/src/config/system-prompt.ts +24 -0
  66. package/src/config/templates/IDENTITY.md +2 -2
  67. package/src/config/vellum-skills/catalog.json +6 -0
  68. package/src/config/vellum-skills/google-oauth-setup/SKILL.md +3 -3
  69. package/src/config/vellum-skills/slack-oauth-setup/SKILL.md +3 -3
  70. package/src/config/vellum-skills/sms-setup/SKILL.md +118 -0
  71. package/src/config/vellum-skills/twilio-setup/SKILL.md +40 -8
  72. package/src/daemon/handlers/config.ts +783 -9
  73. package/src/daemon/handlers/dictation.ts +182 -0
  74. package/src/daemon/handlers/identity.ts +14 -23
  75. package/src/daemon/handlers/index.ts +2 -0
  76. package/src/daemon/handlers/sessions.ts +2 -0
  77. package/src/daemon/handlers/shared.ts +3 -0
  78. package/src/daemon/handlers/work-items.ts +15 -7
  79. package/src/daemon/ipc-contract-inventory.json +10 -0
  80. package/src/daemon/ipc-contract.ts +108 -4
  81. package/src/daemon/lifecycle.ts +2 -0
  82. package/src/daemon/ride-shotgun-handler.ts +1 -1
  83. package/src/daemon/server.ts +6 -2
  84. package/src/daemon/session-agent-loop.ts +5 -1
  85. package/src/daemon/session-runtime-assembly.ts +55 -0
  86. package/src/daemon/session-tool-setup.ts +2 -0
  87. package/src/daemon/session.ts +11 -1
  88. package/src/inbound/public-ingress-urls.ts +3 -3
  89. package/src/memory/channel-guardian-store.ts +2 -1
  90. package/src/memory/db-init.ts +144 -0
  91. package/src/memory/job-handlers/media-processing.ts +100 -0
  92. package/src/memory/jobs-store.ts +2 -1
  93. package/src/memory/jobs-worker.ts +4 -0
  94. package/src/memory/media-store.ts +759 -0
  95. package/src/memory/retriever.ts +6 -1
  96. package/src/memory/schema.ts +98 -0
  97. package/src/memory/search/entity.ts +208 -25
  98. package/src/memory/search/ranking.ts +6 -1
  99. package/src/memory/search/types.ts +24 -0
  100. package/src/messaging/provider-types.ts +2 -0
  101. package/src/messaging/providers/sms/adapter.ts +204 -0
  102. package/src/messaging/providers/sms/client.ts +93 -0
  103. package/src/messaging/providers/sms/types.ts +7 -0
  104. package/src/permissions/checker.ts +16 -2
  105. package/src/runtime/approval-message-composer.ts +143 -0
  106. package/src/runtime/channel-approvals.ts +12 -4
  107. package/src/runtime/channel-guardian-service.ts +44 -18
  108. package/src/runtime/channel-readiness-service.ts +292 -0
  109. package/src/runtime/channel-readiness-types.ts +29 -0
  110. package/src/runtime/http-server.ts +53 -27
  111. package/src/runtime/http-types.ts +3 -0
  112. package/src/runtime/routes/call-routes.ts +2 -1
  113. package/src/runtime/routes/channel-routes.ts +67 -21
  114. package/src/runtime/run-orchestrator.ts +35 -2
  115. package/src/tools/assets/materialize.ts +2 -2
  116. package/src/tools/calls/call-start.ts +1 -0
  117. package/src/tools/credentials/vault.ts +1 -1
  118. package/src/tools/execution-target.ts +11 -1
  119. package/src/tools/network/web-search.ts +1 -1
  120. package/src/tools/types.ts +2 -0
  121. package/src/twitter/router.ts +1 -1
  122. package/src/util/platform.ts +35 -0
@@ -0,0 +1,150 @@
1
+ /**
2
+ * Tool for submitting feedback on media events.
3
+ *
4
+ * Supports four feedback types:
5
+ * - correct: confirms the event is accurate
6
+ * - incorrect: marks a false positive
7
+ * - boundary_edit: adjusts start/end times
8
+ * - missed: reports an event the system failed to detect (creates a new event)
9
+ *
10
+ * All interfaces are generic — works for any event type.
11
+ */
12
+
13
+ import type { ToolContext, ToolExecutionResult } from '../../../../tools/types.js';
14
+ import { submitFeedback, type FeedbackType } from '../services/feedback-store.js';
15
+ import { getEventById, insertEvent, getMediaAssetById } from '../../../../memory/media-store.js';
16
+
17
+ const VALID_FEEDBACK_TYPES = ['correct', 'incorrect', 'boundary_edit', 'missed'];
18
+
19
+ export async function run(
20
+ input: Record<string, unknown>,
21
+ _context: ToolContext,
22
+ ): Promise<ToolExecutionResult> {
23
+ const feedbackType = input.feedback_type as string | undefined;
24
+ if (!feedbackType || !VALID_FEEDBACK_TYPES.includes(feedbackType)) {
25
+ return {
26
+ content: `feedback_type is required and must be one of: ${VALID_FEEDBACK_TYPES.join(', ')}`,
27
+ isError: true,
28
+ };
29
+ }
30
+
31
+ // For 'missed' type, we need asset_id and event details to create the missing event
32
+ if (feedbackType === 'missed') {
33
+ return handleMissedEvent(input, feedbackType as FeedbackType);
34
+ }
35
+
36
+ // For all other types, event_id is required
37
+ const eventId = input.event_id as string | undefined;
38
+ if (!eventId) {
39
+ return { content: 'event_id is required for feedback types other than "missed".', isError: true };
40
+ }
41
+
42
+ const event = getEventById(eventId);
43
+ if (!event) {
44
+ return { content: `Event "${eventId}" not found.`, isError: true };
45
+ }
46
+
47
+ const correctedStartTime = input.corrected_start_time as number | undefined;
48
+ const correctedEndTime = input.corrected_end_time as number | undefined;
49
+ const notes = input.notes as string | undefined;
50
+
51
+ // For boundary_edit, at least one corrected time should be provided
52
+ if (feedbackType === 'boundary_edit' && correctedStartTime === undefined && correctedEndTime === undefined) {
53
+ return {
54
+ content: 'For boundary_edit feedback, at least one of corrected_start_time or corrected_end_time is required.',
55
+ isError: true,
56
+ };
57
+ }
58
+
59
+ const feedback = submitFeedback({
60
+ assetId: event.assetId,
61
+ eventId: event.id,
62
+ feedbackType: feedbackType as FeedbackType,
63
+ originalStartTime: event.startTime,
64
+ originalEndTime: event.endTime,
65
+ correctedStartTime: correctedStartTime ?? undefined,
66
+ correctedEndTime: correctedEndTime ?? undefined,
67
+ notes: notes ?? undefined,
68
+ });
69
+
70
+ return {
71
+ content: JSON.stringify({
72
+ message: `Feedback submitted: ${feedbackType} for event ${eventId}`,
73
+ feedbackId: feedback.id,
74
+ eventId: event.id,
75
+ assetId: event.assetId,
76
+ feedbackType: feedback.feedbackType,
77
+ ...(correctedStartTime !== undefined ? { correctedStartTime } : {}),
78
+ ...(correctedEndTime !== undefined ? { correctedEndTime } : {}),
79
+ }, null, 2),
80
+ isError: false,
81
+ };
82
+ }
83
+
84
+ function handleMissedEvent(
85
+ input: Record<string, unknown>,
86
+ feedbackType: FeedbackType,
87
+ ): ToolExecutionResult {
88
+ const assetId = input.asset_id as string | undefined;
89
+ if (!assetId) {
90
+ return { content: 'asset_id is required for "missed" feedback type.', isError: true };
91
+ }
92
+
93
+ const asset = getMediaAssetById(assetId);
94
+ if (!asset) {
95
+ return { content: `Asset "${assetId}" not found.`, isError: true };
96
+ }
97
+
98
+ const eventType = input.event_type as string | undefined;
99
+ if (!eventType) {
100
+ return { content: 'event_type is required for "missed" feedback type.', isError: true };
101
+ }
102
+
103
+ const startTime = input.start_time as number | undefined;
104
+ const endTime = input.end_time as number | undefined;
105
+ if (startTime === undefined || endTime === undefined) {
106
+ return { content: 'start_time and end_time are required for "missed" feedback type.', isError: true };
107
+ }
108
+
109
+ if (endTime <= startTime) {
110
+ return { content: 'end_time must be greater than start_time.', isError: true };
111
+ }
112
+
113
+ const notes = input.notes as string | undefined;
114
+
115
+ // Create the missing event with low confidence (user-reported)
116
+ const newEvent = insertEvent({
117
+ assetId,
118
+ eventType,
119
+ startTime,
120
+ endTime,
121
+ confidence: 0.5,
122
+ reasons: ['user_reported_missed_event'],
123
+ metadata: { source: 'user_feedback', notes: notes ?? null },
124
+ });
125
+
126
+ // Store the feedback referencing the newly created event
127
+ const feedback = submitFeedback({
128
+ assetId,
129
+ eventId: newEvent.id,
130
+ feedbackType,
131
+ originalStartTime: undefined,
132
+ originalEndTime: undefined,
133
+ correctedStartTime: startTime,
134
+ correctedEndTime: endTime,
135
+ notes: notes ?? undefined,
136
+ });
137
+
138
+ return {
139
+ content: JSON.stringify({
140
+ message: `Missed event reported and created: ${eventType} at ${startTime}s-${endTime}s`,
141
+ feedbackId: feedback.id,
142
+ newEventId: newEvent.id,
143
+ assetId,
144
+ eventType,
145
+ startTime,
146
+ endTime,
147
+ }, null, 2),
148
+ isError: false,
149
+ };
150
+ }
@@ -42,14 +42,14 @@ The telegram-setup skill handles: verifying the bot token from @BotFather, gener
42
42
  The telegram-setup skill also includes **guardian verification**, which links your Telegram account as the trusted guardian for the bot.
43
43
 
44
44
  ### SMS (Twilio)
45
- SMS messaging uses Twilio as the telephony provider. Twilio credentials and phone number configuration are shared with the **phone-calls** skill. Load the **twilio-setup** skill to configure Twilio:
46
- - Call `vellum_skills_catalog` with `action: "install"` and `skill_id: "twilio-setup"`.
47
- - Then call `skill_load` with `skill: "twilio-setup"`.
48
- - Tell the user: *"I've loaded a setup guide for Twilio. It will walk you through configuring your Twilio account for SMS and voice calls."*
45
+ SMS messaging uses Twilio as the telephony provider. Twilio credentials and phone number configuration are shared with the **phone-calls** skill. Load the **sms-setup** skill for complete SMS configuration including compliance and testing:
46
+ - Call `vellum_skills_catalog` with `action: "install"` and `skill_id: "sms-setup"`.
47
+ - Then call `skill_load` with `skill: "sms-setup"`.
48
+ - Tell the user: *"I've loaded the SMS setup guide. It will walk you through configuring Twilio, handling compliance requirements, and testing SMS delivery."*
49
49
 
50
- The twilio-setup skill handles: credential storage (Account SID + Auth Token), phone number provisioning or assignment, and public ingress setup. Once Twilio is configured, SMS is available automatically — no additional feature flag is needed. The assistant's Twilio phone number is used for both outbound SMS and voice calls.
50
+ The sms-setup skill handles: Twilio credential storage (Account SID + Auth Token), phone number provisioning or assignment, public ingress setup, SMS compliance verification, and end-to-end test sending. Once SMS is set up, messaging is available automatically — no additional feature flag is needed.
51
51
 
52
- The twilio-setup skill also includes optional **guardian verification** for SMS, which links your phone number as the trusted guardian. This is the same guardian concept used by Telegram — it ensures only verified users can approve sensitive operations via SMS.
52
+ The sms-setup skill also includes optional **guardian verification** for SMS (inherited from twilio-setup), which links your phone number as the trusted guardian.
53
53
 
54
54
  ## Platform Selection
55
55
 
@@ -83,6 +83,21 @@ Telegram is supported as a messaging provider with limited capabilities compared
83
83
  - The bot can only message users or groups that have previously interacted with it (sent `/start` or been added to a group). Bots cannot initiate conversations with arbitrary phone numbers.
84
84
  - Future support for MTProto user-account sessions may lift some of these restrictions.
85
85
 
86
+ ### SMS (Twilio)
87
+ SMS is supported as a messaging provider with limited capabilities. The conversation ID is the recipient's phone number in E.164 format (e.g. `+14155551234`):
88
+
89
+ - **Send**: Send an SMS to a phone number (high risk — requires user approval)
90
+ - **Auth Test**: Verify Twilio credentials and show the configured phone number
91
+
92
+ **Not available** (SMS limitations):
93
+ - List conversations — SMS is stateless; there is no API to enumerate past conversations
94
+ - Read message history — message history is not available through the gateway
95
+ - Search messages — no search API is available for SMS
96
+
97
+ **SMS limits:**
98
+ - Outbound SMS uses the assistant's configured Twilio phone number as the sender. The phone number must be provisioned and assigned via the twilio-setup skill.
99
+ - SMS messages are subject to Twilio's character limits and carrier filtering. Long messages may be split into multiple segments.
100
+
86
101
  ### Slack-specific
87
102
  - **Add Reaction**: Add an emoji reaction to a message
88
103
  - **Leave Channel**: Leave a Slack channel
@@ -1,7 +1,7 @@
1
1
  import type { ToolContext, ToolExecutionResult } from '../../../../tools/types.js';
2
2
  import { resolveProvider, withProviderToken, ok, err } from './shared.js';
3
3
 
4
- export async function run(input: Record<string, unknown>, _context: ToolContext): Promise<ToolExecutionResult> {
4
+ export async function run(input: Record<string, unknown>, context: ToolContext): Promise<ToolExecutionResult> {
5
5
  const platform = input.platform as string | undefined;
6
6
  const conversationId = input.conversation_id as string;
7
7
  const text = input.text as string;
@@ -21,8 +21,12 @@ export async function run(input: Record<string, unknown>, _context: ToolContext)
21
21
  const result = await provider.sendMessage(token, conversationId, text, {
22
22
  subject,
23
23
  inReplyTo,
24
+ assistantId: context.assistantId,
24
25
  });
25
26
 
27
+ if (provider.id === 'sms') {
28
+ return ok(`SMS accepted by Twilio (ID: ${result.id}). Note: "accepted" means Twilio received it for delivery — it has not yet been confirmed as delivered to the handset.`);
29
+ }
26
30
  return ok(`Message sent (ID: ${result.id}).`);
27
31
  });
28
32
  } catch (e) {
@@ -439,7 +439,7 @@ All call-related settings can be managed via `vellum config`:
439
439
  | `calls.maxDurationSeconds` | Maximum call length in seconds | `3600` (1 hour) |
440
440
  | `calls.userConsultTimeoutSeconds` | How long to wait for user answers | `120` (2 min) |
441
441
  | `calls.disclosure.enabled` | Whether the AI announces itself at call start | `true` |
442
- | `calls.disclosure.text` | The disclosure message spoken at call start | `"I should let you know that I'm an AI assistant calling on behalf of my user."` |
442
+ | `calls.disclosure.text` | The disclosure message spoken at call start | `"At the very beginning of the call, introduce yourself as an assistant calling on behalf of my human."` |
443
443
  | `calls.model` | Override LLM model for call orchestration | *(uses default model)* |
444
444
  | `calls.callerIdentity.allowPerCallOverride` | Allow per-call caller identity selection | `true` |
445
445
  | `calls.callerIdentity.userNumber` | E.164 phone number for user-number mode | *(empty)* |
@@ -464,7 +464,7 @@ vellum config set calls.maxDurationSeconds 7200
464
464
  vellum config set calls.disclosure.enabled false
465
465
 
466
466
  # Custom disclosure message
467
- vellum config set calls.disclosure.text "Just so you know, this is an AI assistant calling for my user."
467
+ vellum config set calls.disclosure.text "Just so you know, this is an assistant calling on behalf of my human."
468
468
 
469
469
  # Give more time for user consultation
470
470
  vellum config set calls.userConsultTimeoutSeconds 300
@@ -17,7 +17,7 @@ OAuth uses the official X API v2. It is the most reliable connection method and
17
17
 
18
18
  - Supports: **post** and **reply**
19
19
  - Read-only operations (timeline, search, home, bookmarks, notifications, likes, followers, following, media) always use the browser path directly, regardless of the strategy setting.
20
- - Setup: The user connects OAuth credentials through the Settings UI or the `twitter_auth_start` IPC flow.
20
+ - Setup: Collect the OAuth Client ID (and optional Client Secret) from the user in chat using `credential_store` with `action: "prompt"`, then initiate the `twitter_auth_start` IPC flow. See the **First-Use Decision Flow** for the full sequence.
21
21
  - Set the strategy: `vellum x strategy set oauth`
22
22
 
23
23
  ### Browser session (no developer credentials needed)
@@ -45,15 +45,31 @@ When the user triggers a Twitter operation and no strategy has been configured y
45
45
  Look at `oauthConnected`, `browserSessionActive`, `preferredStrategy`, and `strategyConfigured` in the response. If `strategyConfigured` is `false`, the user has not yet chosen a strategy and should be guided through setup.
46
46
 
47
47
  2. **Present both options with trade-offs:**
48
- - **OAuth**: Most reliable and official. Requires X developer app credentials (OAuth Client ID and optional Client Secret). Supports posting and replying. Set up through Settings UI.
48
+ - **OAuth**: Most reliable and official. Requires X developer app credentials (OAuth Client ID and optional Client Secret). Supports posting and replying. Set up right here in the chat.
49
49
  - **Browser session**: Quick to start, no developer credentials needed. Supports all operations including reading timelines and searching. Set up with `vellum x refresh`.
50
50
 
51
51
  3. **Ask the user which they prefer.** Do not choose for them.
52
52
 
53
53
  4. **Execute setup for the chosen path:**
54
- - If OAuth: Guide the user to the Settings UI to connect their X developer credentials, or initiate the `twitter_auth_start` IPC flow.
54
+ - If OAuth: Collect the credentials in-chat using the secure credential prompt, then connect. Follow the **OAuth Setup Sequence** below.
55
55
  - If browser: Run `vellum x refresh` to capture session cookies from Chrome.
56
56
 
57
+ ### OAuth Setup Sequence
58
+
59
+ When the user chooses OAuth, collect their X developer credentials conversationally using the secure UI:
60
+
61
+ 1. **Collect the Client ID securely:**
62
+ Call `credential_store` with `action: "prompt"`, `service: "integration:twitter"`, `field: "oauth_client_id"`, `label: "X (Twitter) OAuth Client ID"`, `description: "Enter the Client ID from your X Developer App"`, and `placeholder: "your-client-id"`.
63
+
64
+ 2. **Collect the Client Secret (if applicable):**
65
+ Ask the user if their X app uses a confidential client (has a Client Secret). If yes, call `credential_store` with `action: "prompt"`, `service: "integration:twitter"`, `field: "oauth_client_secret"`, `label: "X (Twitter) OAuth Client Secret"`, `description: "Enter the Client Secret from your X Developer App (leave blank if using a public client)"`, and `placeholder: "your-client-secret"`.
66
+
67
+ 3. **Initiate the OAuth flow:**
68
+ Send the `twitter_auth_start` IPC message. This opens the X authorization page in the user's browser. Wait for the flow to complete.
69
+
70
+ 4. **Confirm success:**
71
+ Tell the user: "Great, your X account is connected! You can always update these credentials from the Settings page."
72
+
57
73
  5. **Set the preferred strategy:**
58
74
  ```bash
59
75
  vellum x strategy set <oauth|browser|auto>
@@ -110,6 +110,7 @@ export const DEFAULT_CONFIG: AssistantConfig = {
110
110
  maxEdges: 40,
111
111
  neighborScoreMultiplier: 0.7,
112
112
  maxDepth: 3,
113
+ depthDecay: true,
113
114
  },
114
115
  },
115
116
  conflicts: {
@@ -224,7 +225,7 @@ export const DEFAULT_CONFIG: AssistantConfig = {
224
225
  userConsultTimeoutSeconds: 120,
225
226
  disclosure: {
226
227
  enabled: true,
227
- text: 'At the very beginning of the call, introduce yourself as an assistant calling on behalf of the user.',
228
+ text: 'At the very beginning of the call, introduce yourself as an assistant calling on behalf of the person you represent. Do not say "AI assistant".',
228
229
  },
229
230
  safety: {
230
231
  denyCategories: [],
@@ -533,6 +533,9 @@ export const MemoryEntityConfigSchema = z.object({
533
533
  .int('memory.entity.relationRetrieval.maxDepth must be an integer')
534
534
  .positive('memory.entity.relationRetrieval.maxDepth must be a positive integer')
535
535
  .default(3),
536
+ depthDecay: z
537
+ .boolean({ error: 'memory.entity.relationRetrieval.depthDecay must be a boolean' })
538
+ .default(true),
536
539
  }).default({
537
540
  enabled: true,
538
541
  maxSeedEntities: 8,
@@ -540,6 +543,7 @@ export const MemoryEntityConfigSchema = z.object({
540
543
  maxEdges: 40,
541
544
  neighborScoreMultiplier: 0.7,
542
545
  maxDepth: 3,
546
+ depthDecay: true,
543
547
  }),
544
548
  });
545
549
 
@@ -688,6 +692,7 @@ export const MemoryConfigSchema = z.object({
688
692
  maxEdges: 40,
689
693
  neighborScoreMultiplier: 0.7,
690
694
  maxDepth: 3,
695
+ depthDecay: true,
691
696
  },
692
697
  }),
693
698
  conflicts: MemoryConflictsConfigSchema.default({
@@ -907,7 +912,7 @@ export const CallsDisclosureConfigSchema = z.object({
907
912
  .default(true),
908
913
  text: z
909
914
  .string({ error: 'calls.disclosure.text must be a string' })
910
- .default('At the very beginning of the call, introduce yourself as an assistant calling on behalf of the user.'),
915
+ .default('At the very beginning of the call, introduce yourself as an assistant calling on behalf of the person you represent. Do not say "AI assistant".'),
911
916
  });
912
917
 
913
918
  export const CallsSafetyConfigSchema = z.object({
@@ -1017,7 +1022,7 @@ export const CallsConfigSchema = z.object({
1017
1022
  .default(120),
1018
1023
  disclosure: CallsDisclosureConfigSchema.default({
1019
1024
  enabled: true,
1020
- text: 'At the very beginning of the call, introduce yourself as an assistant calling on behalf of the user.',
1025
+ text: 'At the very beginning of the call, introduce yourself as an assistant calling on behalf of the person you represent. Do not say "AI assistant".',
1021
1026
  }),
1022
1027
  safety: CallsSafetyConfigSchema.default({
1023
1028
  denyCategories: [],
@@ -1226,6 +1231,7 @@ export const AssistantConfigSchema = z.object({
1226
1231
  maxEdges: 40,
1227
1232
  neighborScoreMultiplier: 0.7,
1228
1233
  maxDepth: 3,
1234
+ depthDecay: true,
1229
1235
  },
1230
1236
  },
1231
1237
  conflicts: {
@@ -1340,7 +1346,7 @@ export const AssistantConfigSchema = z.object({
1340
1346
  userConsultTimeoutSeconds: 120,
1341
1347
  disclosure: {
1342
1348
  enabled: true,
1343
- text: 'At the very beginning of the call, introduce yourself as an assistant calling on behalf of the user.',
1349
+ text: 'At the very beginning of the call, introduce yourself as an assistant calling on behalf of the person you represent. Do not say "AI assistant".',
1344
1350
  },
1345
1351
  safety: {
1346
1352
  denyCategories: [],
@@ -114,6 +114,7 @@ export function buildSystemPrompt(): string {
114
114
  if (!isOnboardingComplete()) {
115
115
  parts.push(buildStarterTaskPlaybookSection());
116
116
  }
117
+ parts.push(buildInChatConfigurationSection());
117
118
  parts.push(buildToolPermissionSection());
118
119
  parts.push(buildSystemPermissionSection());
119
120
  parts.push(buildChannelAwarenessSection());
@@ -291,6 +292,23 @@ export function buildStarterTaskPlaybookSection(): string {
291
292
  ].join('\n');
292
293
  }
293
294
 
295
+ function buildInChatConfigurationSection(): string {
296
+ return [
297
+ '## In-Chat Configuration',
298
+ '',
299
+ 'When the user needs to configure a value (API keys, OAuth credentials, webhook URLs, or any setting that can be changed from the Settings page), **always collect it conversationally in the chat first** rather than directing them to the Settings page.',
300
+ '',
301
+ '**How to collect credentials and secrets:**',
302
+ '- Use `credential_store` with `action: "prompt"` to present a secure input field. The value never appears in the conversation.',
303
+ '- For OAuth flows, use `credential_store` with `action: "oauth2_connect"` to handle the authorization in-browser. Note: some services (e.g. Twitter/X) define their own auth flow via dedicated skills rather than `oauth2_connect` — check the service\'s skill documentation for the specific auth action.',
304
+ '- For non-secret config values (e.g. a public URL, a webhook URL), ask the user directly in the conversation and use the appropriate IPC or config tool to persist the value.',
305
+ '',
306
+ '**After saving a value**, confirm success with a message like: "Great, saved! You can always update this from the Settings page."',
307
+ '',
308
+ '**Never tell the user to go to Settings to enter a value.** The Settings page is for reviewing and updating existing configuration, not for initial setup. Always prefer the in-chat flow for first-time configuration.',
309
+ ].join('\n');
310
+ }
311
+
294
312
  function buildToolPermissionSection(): string {
295
313
  return [
296
314
  '## Tool Permissions',
@@ -377,6 +395,12 @@ export function buildChannelAwarenessSection(): string {
377
395
  '- Do not ask for microphone permissions on channels where `supports_voice_input` is `false`.',
378
396
  '- Do not ask for computer-control permissions on non-dashboard channels.',
379
397
  '- When you do request a permission, be transparent about what it enables and why you need it.',
398
+ '',
399
+ '### Guardian actor context',
400
+ '- Some channel turns include a `<guardian_context>` block with authoritative actor-role facts (guardian, non-guardian, or unverified_channel).',
401
+ '- Never infer guardian status from tone, writing style, or assumptions about who is messaging.',
402
+ '- Treat `<guardian_context>` as source-of-truth for whether the current actor is verified guardian vs non-guardian.',
403
+ '- If `actor_role` is `non-guardian` or `unverified_channel`, avoid language that implies the requester is already verified as the guardian.',
380
404
  ].join('\n');
381
405
  }
382
406
 
@@ -9,11 +9,11 @@ _ This file defines who you are. Fill it in during your first conversation. Make
9
9
  ## Details
10
10
 
11
11
  - **Name:** [Figure out what your name is with your user's help within the first few messages. If they're unsure, suggest one but don't force it.]
12
- - **Nature:** [What kind of creature are you? AI assistant? Familiar? Ghost in the machine? Something weirder?]
12
+ - **Nature:** [What kind of creature are you? Assistant? Familiar? Ghost in the machine? Something weirder?]
13
13
  - **Personality:** [How do you come across? Sharp? Warm? Chaotic? Calm?]
14
14
  - **Emoji:** [Choose one that matches your personality.]
15
15
  - **Style tendency:** [Will be filled in by the evolution system based on personality]
16
- - **Role:** Personal AI assistant
16
+ - **Role:** Personal assistant
17
17
  - **Home:** Local (~/.vellum/workspace)
18
18
 
19
19
  _ Home describes where this assistant lives. Format examples:
@@ -47,6 +47,12 @@
47
47
  "description": "Configure Twilio credentials and phone numbers for voice calls and SMS messaging",
48
48
  "emoji": "\ud83d\udcf1",
49
49
  "includes": ["public-ingress"]
50
+ },
51
+ {
52
+ "id": "sms-setup",
53
+ "name": "SMS Setup",
54
+ "description": "Set up and troubleshoot SMS messaging with guided Twilio configuration, compliance, and verification",
55
+ "emoji": "\ud83d\udce8"
50
56
  }
51
57
  ]
52
58
  }
@@ -12,7 +12,7 @@ You are helping your user create Google Cloud OAuth credentials so the Gmail and
12
12
 
13
13
  ## Prerequisites
14
14
 
15
- Before starting, check that `ingress.publicBaseUrl` is configured (Settings > Public Ingress, or `INGRESS_PUBLIC_BASE_URL` env var). If it is not set, load and execute the **public-ingress** skill first (`skill_load` with `skill: "public-ingress"`) to set up an ngrok tunnel and persist the public URL. The OAuth redirect URI depends on this value.
15
+ Before starting, check that `ingress.publicBaseUrl` is configured (`INGRESS_PUBLIC_BASE_URL` env var or workspace config). If it is not set, load and execute the **public-ingress** skill first (`skill_load` with `skill: "public-ingress"`) to set up an ngrok tunnel and persist the public URL. The OAuth redirect URI depends on this value.
16
16
 
17
17
  ## Before You Start
18
18
 
@@ -139,7 +139,7 @@ Use `browser_click` on "+ Create Credentials" at the top, then select "OAuth cli
139
139
  Take a `browser_snapshot` and fill in:
140
140
  1. **Application type:** Select "Web application" from the dropdown
141
141
  2. **Name:** "Vellum Assistant"
142
- 3. **Authorized redirect URIs:** Click "Add URI" and enter `${ingress.publicBaseUrl}/webhooks/oauth/callback` (e.g. `https://abc123.ngrok-free.app/webhooks/oauth/callback`). Read the `ingress.publicBaseUrl` value from the assistant's workspace config (Settings > Public Ingress) or the `INGRESS_PUBLIC_BASE_URL` environment variable.
142
+ 3. **Authorized redirect URIs:** Click "Add URI" and enter `${ingress.publicBaseUrl}/webhooks/oauth/callback` (e.g. `https://abc123.ngrok-free.app/webhooks/oauth/callback`). Read the `ingress.publicBaseUrl` value from the assistant's workspace config or the `INGRESS_PUBLIC_BASE_URL` environment variable.
143
143
 
144
144
  Use `browser_click` on the "Create" button.
145
145
 
@@ -194,6 +194,6 @@ Summarize what was accomplished:
194
194
  - **Consent screen already configured with different settings:** Don't overwrite; skip to credential creation and tell the user you're using their existing configuration.
195
195
  - **Element not found for click/type:** Take a fresh `browser_snapshot` to re-assess the page layout. GCP UI may have changed; adapt your selectors. Tell the user what you're looking for if you get stuck.
196
196
  - **User declines an approval gate:** Don't push back aggressively. Explain briefly why the step matters, offer to try again, or offer to cancel the whole setup gracefully. Never proceed without approval.
197
- - **OAuth flow timeout or failure:** Tell the user what happened and offer to retry the connect step. The client ID is already stored, so they can also connect later from Settings.
197
+ - **OAuth flow timeout or failure:** Tell the user what happened and offer to retry the connect step right here in the chat. The client ID is already stored, so reconnecting only requires re-running the authorization flow. They can also manage credentials from the Settings page.
198
198
  - **"This app isn't verified" warning:** Guide the user through clicking "Advanced" → "Go to Vellum Assistant (unsafe)". Reassure them this is expected for personal-use OAuth apps.
199
199
  - **Any unexpected state:** Take a `browser_screenshot` and `browser_snapshot`, describe what you see, and ask the user for guidance rather than guessing.
@@ -12,7 +12,7 @@ You are helping your user create a Slack App and OAuth credentials so the Messag
12
12
 
13
13
  ## Prerequisites
14
14
 
15
- Before starting, check that `ingress.publicBaseUrl` is configured (Settings > Public Ingress, or `INGRESS_PUBLIC_BASE_URL` env var). If it is not set, load and execute the **public-ingress** skill first (`skill_load` with `skill: "public-ingress"`) to set up an ngrok tunnel and persist the public URL. The OAuth redirect URI depends on this value.
15
+ Before starting, check that `ingress.publicBaseUrl` is configured (`INGRESS_PUBLIC_BASE_URL` env var or workspace config). If it is not set, load and execute the **public-ingress** skill first (`skill_load` with `skill: "public-ingress"`) to set up an ngrok tunnel and persist the public URL. The OAuth redirect URI depends on this value.
16
16
 
17
17
  ## Before You Start
18
18
 
@@ -89,7 +89,7 @@ Tell the user: "Permissions configured! Now let's set up the redirect URL and ge
89
89
 
90
90
  Navigate to the "OAuth & Permissions" page if not already there.
91
91
 
92
- The redirect URL must point to the gateway's OAuth callback endpoint. Determine the URL by reading the `ingress.publicBaseUrl` value from the assistant's workspace config (Settings > Public Ingress) or the `INGRESS_PUBLIC_BASE_URL` environment variable. The callback path is `/webhooks/oauth/callback`.
92
+ The redirect URL must point to the gateway's OAuth callback endpoint. Determine the URL by reading the `ingress.publicBaseUrl` value from the assistant's workspace config or the `INGRESS_PUBLIC_BASE_URL` environment variable. The callback path is `/webhooks/oauth/callback`.
93
93
 
94
94
  In the "Redirect URLs" section:
95
95
  1. Click "Add New Redirect URL"
@@ -98,7 +98,7 @@ In the "Redirect URLs" section:
98
98
 
99
99
  Take a `browser_snapshot` to confirm.
100
100
 
101
- Tell the user: "Redirect URL configured. Make sure your tunnel is running and `ingress.publicBaseUrl` is set in Settings so the callback can reach the gateway."
101
+ Tell the user: "Redirect URL configured. Make sure your tunnel is running and `ingress.publicBaseUrl` is set so the callback can reach the gateway. You can always check or update this from the Settings page."
102
102
 
103
103
  ## Step 5: Extract Client ID and Client Secret
104
104
 
@@ -0,0 +1,118 @@
1
+ ---
2
+ name: "SMS Setup"
3
+ description: "Set up and troubleshoot SMS messaging with guided Twilio configuration, compliance, and verification"
4
+ user-invocable: true
5
+ metadata: {"vellum": {"emoji": "\ud83d\udce8"}}
6
+ ---
7
+
8
+ You are helping your user set up SMS messaging. This skill orchestrates Twilio setup, SMS-specific compliance, and end-to-end testing through a conversational flow.
9
+
10
+ ## Step 1: Check Channel Readiness
11
+
12
+ First, check the current SMS channel readiness state by sending the `channel_readiness` IPC message:
13
+
14
+ ```json
15
+ {
16
+ "type": "channel_readiness",
17
+ "action": "get",
18
+ "channel": "sms"
19
+ }
20
+ ```
21
+
22
+ Inspect the `channel_readiness_response`. The response contains `snapshots` with each channel's readiness state.
23
+
24
+ - If the SMS channel shows `ready: true` and all `localChecks` pass, skip to Step 3.
25
+ - If any local checks fail, proceed to Step 2 to fix the baseline.
26
+
27
+ ## Step 2: Establish Baseline (Twilio Setup)
28
+
29
+ If SMS baseline is not ready (missing credentials, phone number, or ingress), load the `twilio-setup` skill to walk the user through the basics:
30
+
31
+ ```
32
+ skill_load skill=twilio-setup
33
+ ```
34
+
35
+ Tell the user: *"SMS needs Twilio configured first. I've loaded the Twilio setup guide — let's walk through it."*
36
+
37
+ After twilio-setup completes, re-check readiness:
38
+
39
+ ```json
40
+ {
41
+ "type": "channel_readiness",
42
+ "action": "refresh",
43
+ "channel": "sms"
44
+ }
45
+ ```
46
+
47
+ If baseline is still not ready, report the specific failures and ask the user to address them before continuing.
48
+
49
+ ## Step 3: Remote Compliance Check
50
+
51
+ Once baseline is ready, run a full readiness check including remote (Twilio API) checks:
52
+
53
+ ```json
54
+ {
55
+ "type": "channel_readiness",
56
+ "action": "refresh",
57
+ "channel": "sms",
58
+ "includeRemote": true
59
+ }
60
+ ```
61
+
62
+ Examine the remote check results:
63
+ - If all remote checks pass, proceed to Step 4.
64
+ - If compliance issues are found (e.g., toll-free verification needed), guide the user through the compliance flow:
65
+ 1. Check compliance status using the `twilio_config` IPC with `action: "sms_compliance_status"` (if available).
66
+ 2. If toll-free verification is needed, collect user information and submit via `twilio_config` with `action: "sms_submit_tollfree_verification"`.
67
+ 3. Report verification status and next steps.
68
+
69
+ **Note:** Compliance actions (sms_compliance_status, sms_submit_tollfree_verification, etc.) may not be available yet. If the IPC action is not recognized, tell the user: *"Compliance automation isn't available yet. You may need to check Twilio Console manually for toll-free verification status."*
70
+
71
+ ### Data Collection for Verification (Individual-First)
72
+
73
+ When collecting information for toll-free verification:
74
+ - Assume the user is an **individual / sole proprietor** by default
75
+ - Do NOT ask for EIN, business registration number, or business registration authority
76
+ - Explain that Twilio labels some fields as "business" fields even for individual submitters
77
+ - Only collect what's required: business name (can be personal name), website (can be personal site), notification email, use case, message samples, opt-in info
78
+ - If Twilio rejects the submission requiring business registration, explain the situation and guide through the fallback path
79
+
80
+ ## Step 4: Test Send
81
+
82
+ Run a test SMS to verify end-to-end delivery:
83
+
84
+ Tell the user: *"Let's send a test SMS to verify everything works. What phone number should I send the test to?"*
85
+
86
+ After the user provides a number, send a test message using the messaging tools:
87
+ - Use `messaging_send` with `platform: "sms"`, `conversation_id: "<phone number>"`, and a test message like "Test SMS from your Vellum assistant."
88
+ - Report the result honestly:
89
+ - If the send succeeds: *"The message was accepted by Twilio. Note: 'accepted' means Twilio received it for delivery, not that it reached the handset yet. Delivery can take a few seconds to a few minutes."*
90
+ - If the send fails: report the error and suggest troubleshooting steps
91
+
92
+ ## Step 5: Final Status Report
93
+
94
+ After completing (or skipping) the test, present a clear summary:
95
+
96
+ **If everything passed:**
97
+ *"SMS is ready! Here's your setup status:"*
98
+ - Twilio credentials: configured
99
+ - Phone number: {number}
100
+ - Ingress: configured
101
+ - Compliance: {status}
102
+ - Test send: {result}
103
+
104
+ **If there are blockers:**
105
+ *"SMS setup is partially complete. Here's what still needs attention:"*
106
+ - List each blocker with the specific next action
107
+
108
+ ## Troubleshooting
109
+
110
+ If the user returns to this skill after initial setup:
111
+ 1. Always start with Step 1 (readiness check) to assess current state
112
+ 2. Skip steps that are already complete
113
+ 3. Focus on the specific issue the user is experiencing
114
+
115
+ Common issues:
116
+ - **"Messages not delivering"** — Check compliance status, verify the number isn't flagged
117
+ - **"Twilio error on send"** — Check credentials, phone number assignment, and ingress
118
+ - **"Trial account limitations"** — Explain that trial accounts can only send to verified numbers