@vellumai/assistant 0.3.0

package/src/config/vellum-skills/slack-oauth-setup/SKILL.md

@@ -0,0 +1,153 @@
+---
+name: "Slack OAuth Setup"
+description: "Create Slack App and OAuth credentials for Slack integration using browser automation"
+user-invocable: true
+includes: ["browser", "public-ingress"]
+metadata: {"vellum": {"emoji": "🔑"}}
+---
+
+You are helping your user create a Slack App and OAuth credentials so the Messaging integration can connect to their Slack workspace. Walk through each step below using `browser_navigate`, `browser_snapshot`, `browser_screenshot`, `browser_click`, `browser_type`, and `browser_extract` tools.
+
+**Tone:** Be friendly and reassuring throughout. Narrate what you're doing in plain language so the user always knows what's happening. After each step, briefly confirm what was accomplished before moving on.
+
+## Prerequisites
+
+Before starting, check that `ingress.publicBaseUrl` is configured (Settings > Public Ingress, or `INGRESS_PUBLIC_BASE_URL` env var). If it is not set, load and execute the **public-ingress** skill first (`skill_load` with `skill: "public-ingress"`) to set up an ngrok tunnel and persist the public URL. The OAuth redirect URI depends on this value.
+
+## Before You Start
+
+Tell the user:
+- "I'll walk you through creating a Slack App so Vellum can connect to your workspace. The whole process takes a few minutes."
+- "I'll be automating the Slack API website in the browser — you'll be able to see everything I'm doing."
+- "I'll ask for your approval before each major step, so nothing happens without your say-so."
+- "No sensitive credentials will be shown in the conversation."
+
+## Step 1: Navigate to Slack API
+
+Tell the user: "First, let me open the Slack API dashboard."
+
+Use `browser_navigate` to go to `https://api.slack.com/apps`.
+
+Take a `browser_screenshot` to show the user what loaded, then take a `browser_snapshot` to check the page state:
+- **If a sign-in page appears:** Tell the user "Please sign in to your Slack account in the browser window. Let me know when you're done." Wait for their confirmation, then take another snapshot.
+- **If the apps dashboard loads:** Tell the user "Slack API dashboard is loaded. Let's create your app!" and continue to Step 2.
+
+## Step 2: Create a Slack App
+
+**Ask for approval before proceeding.** Use `ui_show` with `surface_type: "confirmation"` and this message:
+
+> **Create a Slack App**
+>
+> I'm about to create a new Slack App called "Vellum Assistant" in your workspace. This app will only have the permissions you approve in the next step. It won't post anything or access any data until you explicitly authorize it.
+
+Wait for the user to approve. If they decline, explain that the app is required for OAuth and offer to try again or cancel.
+
+Once approved:
+1. Click "Create New App"
+2. Select "From scratch"
+3. Enter app name: "Vellum Assistant"
+4. Select the user's workspace from the dropdown
+5. Click "Create App"
+
+Take a `browser_screenshot` to show the result.
+
+Tell the user: "App created! Now let's configure the permissions it needs."
+
+## Step 3: Configure OAuth Scopes
+
+**Ask for approval before proceeding.** Use `ui_show` with `surface_type: "confirmation"` and this message:
+
+> **Configure Slack Permissions**
+>
+> I'm about to add the following User Token Scopes to your Slack App. These let Vellum read your messages and channels, send messages on your behalf, search your message history, and add emoji reactions:
+>
+> - `channels:read` — View basic channel info
+> - `channels:history` — Read message history in public channels
+> - `groups:read` — View private channel info
+> - `groups:history` — Read message history in private channels
+> - `im:read` — View direct message info
+> - `im:history` — Read direct message history
+> - `mpim:read` — View group DM info
+> - `mpim:history` — Read group DM history
+> - `users:read` — View user profiles
+> - `chat:write` — Send messages
+> - `search:read` — Search messages
+> - `reactions:write` — Add emoji reactions
+
+Wait for the user to approve.
+
+Once approved:
+1. Navigate to "OAuth & Permissions" in the left sidebar (or go to the app's OAuth page directly)
+2. Scroll to "User Token Scopes"
+3. Add each scope listed above using the "Add an OAuth Scope" button
+
+Take a `browser_screenshot` after adding all scopes.
+
+Tell the user: "Permissions configured! Now let's set up the redirect URL and get the credentials."
+
+## Step 4: Add Redirect URL
+
+Navigate to the "OAuth & Permissions" page if not already there.
+
+The redirect URL must point to the gateway's OAuth callback endpoint. Determine the URL by reading the `ingress.publicBaseUrl` value from the assistant's workspace config (Settings > Public Ingress) or the `INGRESS_PUBLIC_BASE_URL` environment variable. The callback path is `/webhooks/oauth/callback`.
+
+In the "Redirect URLs" section:
+1. Click "Add New Redirect URL"
+2. Enter `${ingress.publicBaseUrl}/webhooks/oauth/callback` (e.g. `https://abc123.ngrok-free.app/webhooks/oauth/callback`)
+3. Click "Add" then "Save URLs"
+
+Take a `browser_snapshot` to confirm.
+
+Tell the user: "Redirect URL configured. Make sure your tunnel is running and `ingress.publicBaseUrl` is set in Settings so the callback can reach the gateway."
+
+## Step 5: Extract Client ID and Client Secret
+
+Navigate to "Basic Information" in the left sidebar.
+
+Use `browser_extract` to read:
+1. **Client ID** from the "App Credentials" section
+2. **Client Secret** — click "Show" first, then extract the value
+
+**Important:** Slack requires a client secret for the token exchange (unlike Google which uses PKCE). Both values are needed.
+
+Tell the user: "Credentials extracted! Now let's connect your Slack workspace."
+
+## Step 6: Connect Slack
+
+Tell the user: "Opening Slack authorization so you can grant Vellum access to your workspace. You'll see a Slack consent page — just click 'Allow'."
+
+Use the `credential_store` tool to connect Slack via OAuth2:
+
+```
+action: "oauth2_connect"
+service: "integration:slack"
+client_id: "<the extracted Client ID>"
+client_secret: "<the extracted Client Secret>"
+auth_url: "https://slack.com/oauth/v2/authorize"
+token_url: "https://slack.com/api/oauth.v2.access"
+scopes: ["channels:read", "channels:history", "groups:read", "groups:history", "im:read", "im:history", "im:write", "mpim:read", "mpim:history", "users:read", "chat:write", "search:read", "reactions:write"]
+extra_params: {"user_scope": "channels:read,channels:history,groups:read,groups:history,im:read,im:history,mpim:read,mpim:history,users:read,chat:write,search:read,reactions:write"}
+```
+
+This will open the Slack authorization page in the user's browser. Wait for the flow to complete.
+
+## Step 7: Celebrate!
+
+Once connected, tell the user:
+
+"**Slack is connected!** You're all set. You can now read channels, search messages, send messages, and manage your Slack workspace through Vellum. Try asking me to check your unread Slack messages!"
+
+Summarize what was accomplished:
+- Created a Slack App called "Vellum Assistant"
+- Configured User Token Scopes for reading, writing, and searching
+- Set up the OAuth redirect URL
+- Connected your Slack workspace
+
+## Error Handling
+
+- **Page load failures:** Retry navigation once. If it still fails, tell the user and ask them to check their internet connection.
+- **Workspace selection issues:** The user may need admin permissions to install apps. Explain this clearly.
+- **Element not found for click/type:** Take a fresh `browser_snapshot` to re-assess the page layout. Slack's UI may have changed; adapt your selectors.
+- **User declines an approval gate:** Don't push back. Explain briefly why the step matters, offer to try again, or cancel gracefully.
+- **OAuth flow timeout or failure:** Tell the user what happened and offer to retry. The app is already created, so they only need to re-run the connect step.
+- **Any unexpected state:** Take a `browser_screenshot` and `browser_snapshot`, describe what you see, and ask the user for guidance.

package/src/config/vellum-skills/telegram-setup/SKILL.md

@@ -0,0 +1,143 @@
+---
+name: "Telegram Setup"
+description: "Connect a Telegram bot to the Vellum Assistant gateway with automated webhook registration and credential storage"
+user-invocable: true
+includes: ["public-ingress"]
+metadata: {"vellum": {"emoji": "\ud83e\udd16"}}
+---
+
+You are helping your user connect a Telegram bot to the Vellum Assistant gateway. Telegram webhooks are received exclusively by the gateway (the public ingress boundary) — they never hit the assistant runtime directly. When this skill is invoked, walk through each step below using only existing tools.
+
+## What You Need
+
+1. **Bot token** from Telegram's @BotFather (the user provides this)
+2. **Gateway webhook URL** — derived from the canonical ingress setting: `${ingress.publicBaseUrl}/webhooks/telegram`. The gateway is the only publicly reachable endpoint; Telegram sends webhooks to the gateway, which validates and forwards them to the assistant runtime internally. If `ingress.publicBaseUrl` is not configured, load and execute the **public-ingress** skill first (`skill_load` with `skill: "public-ingress"`) to set up an ngrok tunnel and persist the URL before continuing.
+
+**IMPORTANT — Secure credential collection only:** Never use a bot token that was pasted in plaintext chat. Always collect the bot token through the secure credential prompt flow using `credential_store` with `action: "prompt"` and `service: "telegram"`, `field: "bot_token"`. If the user has already pasted a token in the conversation, inform them that for security reasons you cannot use tokens shared in chat and must collect it through the secure prompt instead.
+
+## Setup Steps
+
+### Step 1: Collect the Bot Token Securely
+
+Collect the bot token through the secure credential prompt:
+- Call `credential_store` with `action: "prompt"`, `service: "telegram"`, `field: "bot_token"`, `label: "Telegram Bot Token"`, `description: "Enter the bot token you received from @BotFather"`, and `placeholder: "123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11"`.
+
+The token is collected securely via a system-level prompt and is never exposed in plaintext chat.
+
+### Step 2: Configure via Daemon
+
+After the token is collected, send it to the daemon's `telegram_config` handler which validates, stores, and configures everything in one step:
+
+- Send the `telegram_config` IPC message with `action: "set"`. The daemon retrieves the token from secure storage internally when `botToken` is not provided in the message — you do not need to retrieve it yourself.
+
+The daemon's `telegram_config set` handler automatically:
+- Validates the token by calling the Telegram `getMe` API
+- Stores the bot token in secure storage with bot username metadata
+- Generates a webhook secret if one does not already exist
+- Triggers an immediate gateway webhook reconcile
+
+If the token is invalid, the daemon returns an error. Tell the user and ask them to re-enter the token via the secure prompt.
+
+### Step 3: Webhook Registration (Automatic)
+
+Manual webhook registration is no longer required. The gateway automatically reconciles the Telegram webhook on startup and whenever credentials change. It compares the current webhook URL against `${INGRESS_PUBLIC_BASE_URL}/webhooks/telegram` and updates it if needed, including the webhook secret and allowed updates.
+
+If the webhook secret changes (e.g., secret rotation), the gateway's credential watcher detects the change and re-registers the webhook automatically. If the ingress URL changes (e.g., tunnel restart), the assistant daemon triggers an immediate internal reconcile so the webhook re-registers automatically without a gateway restart.
+
+### Step 4: Register Bot Commands
+
+Send the `telegram_config` IPC message with `action: "set_commands"` to register the `/new` and `/guardian_verify` commands:
+
+```json
+{
+  "type": "telegram_config",
+  "action": "set_commands",
+  "commands": [
+    { "command": "new", "description": "Start a new conversation" },
+    { "command": "guardian_verify", "description": "Verify your guardian identity" }
+  ]
+}
+```
+
+The daemon handles token retrieval from secure storage internally — you do not need to retrieve it yourself.
+
+### Step 5: Verify Guardian Identity
+
+Now link the user's Telegram account as the trusted guardian for this bot. Tell the user: "Now let's verify your guardian identity. This links your Telegram account as the trusted guardian for this bot."
+
+1. Send the `guardian_verification` IPC message with `action: "create_challenge"` to generate a verification challenge:
+
+```json
+{
+  "type": "guardian_verification",
+  "action": "create_challenge"
+}
+```
+
+2. The daemon returns a `guardian_verification_response` with `success: true`, `secret`, and `instruction`. Display the instruction to the user. It will look like: "Send `/guardian_verify <secret>` to your bot from your Telegram account within 10 minutes."
+
+3. Wait for the user to confirm they have sent the command. The verification happens automatically when the bot receives the `/guardian_verify` message — the channel inbound handler validates the token and creates the guardian binding.
+
+4. If the user confirms success: "Guardian verified! Your Telegram account is now the trusted guardian for this bot."
+
+5. If the user reports failure or the challenge times out (10 minutes): "The verification code may have expired. Let's generate a new one." Then repeat from substep 1.
+
+**Note:** Guardian verification is optional but recommended. If the user declines or wants to skip, proceed to Step 6 without blocking.
+
+### Step 6: Validate Routing Configuration
+
+Verify that the gateway routing is configured to deliver inbound messages to the assistant:
+
+- In **single-assistant mode** (the default local deployment), routing is automatically configured. The CLI sets `GATEWAY_UNMAPPED_POLICY=default` and `GATEWAY_DEFAULT_ASSISTANT_ID` to the current assistant's ID when starting the gateway, so no manual routing configuration is needed.
+- In **multi-assistant mode**, the operator must set `GATEWAY_ASSISTANT_ROUTING_JSON` to map specific chat IDs or user IDs to assistant IDs, or configure a default assistant via `GATEWAY_DEFAULT_ASSISTANT_ID` with `GATEWAY_UNMAPPED_POLICY=default`.
+
+If routing is misconfigured, inbound Telegram messages will be rejected and the gateway will send a visible notice to the chat explaining the issue (rate-limited to once per 5 minutes per chat).
+
+### Step 7: Verify Binding State
+
+Before reporting success, confirm the guardian binding was actually created. Send a `guardian_verification` IPC message with `action: "status"` (or query the guardian binding via the `getGuardianBinding` service call) to check whether a binding exists for the `telegram` channel. If the binding is absent and the user said they completed the verification:
+
+1. Tell the user the verification does not appear to have succeeded.
+2. Offer to generate a new challenge (repeat Step 5, substep 1).
+3. Only proceed to Step 8 once binding state is confirmed or the user explicitly skips guardian verification.
+
+### Step 8: Report Success
+
+Summarize what was done:
+- Bot verified and credentials stored securely via daemon
+- Webhook registration: handled automatically by the gateway
+- Bot commands registered: /new, /guardian_verify
+- Guardian identity verified (if completed and binding confirmed)
+- Routing configuration validated
+
+The gateway automatically detects credentials from the vault, reconciles the Telegram webhook registration, and begins accepting Telegram webhooks shortly. In single-assistant mode, routing is automatically configured — no manual environment variable configuration or webhook registration is needed. If the webhook secret changes later, the gateway's credential watcher will automatically re-register the webhook. If the ingress URL changes (e.g., tunnel restart), the assistant daemon triggers an immediate internal reconcile so the webhook re-registers automatically without a gateway restart.
+
+## Bot-Account Limitations
+
+Telegram bot accounts have inherent limitations imposed by the Bot API:
+
+- **No arbitrary messaging**: Bots cannot initiate conversations with users who have not first interacted with the bot (sent `/start` or added it to a group). Messaging arbitrary phone numbers is not possible.
+- **No conversation listing**: The Bot API does not expose a method to enumerate the chats a bot belongs to.
+- **No message history retrieval**: Bots cannot fetch past messages from a chat.
+- **No message search**: No search API is available for bots.
+
+These limitations apply to all Telegram bots regardless of configuration. Future support for MTProto user-account sessions may lift some of these restrictions.
+
+## Automated vs Manual Steps
+
+The following steps are now **automated** by the gateway and CLI:
+
+| Step | Status | Details |
+|------|--------|---------|
+| Webhook registration | Automated | The gateway reconciles the webhook URL on startup and when credentials change |
+| Routing configuration | Automated (single-assistant) | The CLI sets `GATEWAY_UNMAPPED_POLICY=default` and `GATEWAY_DEFAULT_ASSISTANT_ID` automatically |
+| Credential detection | Automated | The gateway watches the credential vault for changes |
+
+The following steps still require **manual** action:
+
+| Step | Details |
+|------|---------|
+| Bot token from @BotFather | User must create a bot and provide the token via secure prompt |
+| Bot command registration | Registered via the setup skill (Step 4 above) |
+| Guardian verification | User sends `/guardian_verify <secret>` to the bot (Step 5 above) |
+| Multi-assistant routing | Requires manual `GATEWAY_ASSISTANT_ROUTING_JSON` configuration |

package/src/config/vellum-skills/twilio-setup/SKILL.md

@@ -0,0 +1,213 @@
+---
+name: "Twilio Setup"
+description: "Configure Twilio credentials and phone numbers for voice calls and SMS messaging"
+user-invocable: true
+includes: ["public-ingress"]
+metadata: {"vellum": {"emoji": "\ud83d\udcf1"}}
+---
+
+You are helping your user configure Twilio for voice calls and SMS messaging. Twilio is the shared telephony provider for both the **phone-calls** and **SMS messaging** capabilities. When this skill is invoked, walk through each step below using the `twilio_config` IPC contract and existing tools.
+
+## Overview
+
+This skill manages the full Twilio lifecycle:
+- **Credential storage** — Account SID and Auth Token
+- **Phone number provisioning** — Buy a new number directly from Twilio
+- **Phone number assignment** — Assign an existing Twilio number to the assistant
+- **Status checking** — Verify credentials and assigned number
+
+All operations go through the `twilio_config` IPC handler on the daemon, which validates inputs, stores credentials securely, and manages phone number state.
+
+## Step 1: Check Current Configuration
+
+First, check whether Twilio is already configured by sending the `twilio_config` IPC message with `action: "get"`:
+
+```json
+{
+  "type": "twilio_config",
+  "action": "get"
+}
+```
+
+The daemon returns a `twilio_config_response` with:
+- `hasCredentials` — whether Account SID and Auth Token are stored
+- `phoneNumber` — the currently assigned phone number (if any)
+
+If both are present, tell the user Twilio is already configured and offer to show the current status or reconfigure.
+
+## Step 2: Collect and Store Credentials
+
+If credentials are not yet stored, guide the user through Twilio account setup:
+
+1. Tell the user: **"You'll need a Twilio account. Sign up at https://www.twilio.com/try-twilio — it's free to start and includes trial credit."**
+2. Once they have an account, they need two pieces of information:
+   - **Account SID** — found on the Twilio Console dashboard at https://console.twilio.com
+   - **Auth Token** — found on the same dashboard (click "Show" to reveal it)
+
+**IMPORTANT — Secure credential collection only:** Never use credentials pasted in plaintext chat. Always collect credentials through the secure credential prompt flow:
+
+- Call `credential_store` with `action: "prompt"`, `service: "twilio"`, `field: "account_sid"`, `label: "Twilio Account SID"`, `description: "Enter your Account SID from the Twilio Console dashboard"`, and `placeholder: "ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"`.
+- Call `credential_store` with `action: "prompt"`, `service: "twilio"`, `field: "auth_token"`, `label: "Twilio Auth Token"`, `description: "Enter your Auth Token from the Twilio Console dashboard"`, and `placeholder: "your_auth_token"`.
+
+After both credentials are collected, retrieve them from secure storage and pass them to the daemon:
+
+```json
+{
+  "type": "twilio_config",
+  "action": "set_credentials",
+  "accountSid": "<value from credential_store for twilio/account_sid>",
+  "authToken": "<value from credential_store for twilio/auth_token>"
+}
+```
+
+Both `accountSid` and `authToken` are required — the daemon validates the credentials against the Twilio API before storing them. If credentials are invalid, the daemon returns an error. Tell the user and ask them to re-enter via the secure prompt.
+
+## Step 3: Get a Phone Number
+
+The assistant needs a phone number to make calls and send SMS. There are two paths:
+
+### Option A: Provision a New Number
+
+If the user wants to buy a new number through Twilio, send:
+
+```json
+{
+  "type": "twilio_config",
+  "action": "provision_number",
+  "areaCode": "415",
+  "country": "US"
+}
+```
+
+- `areaCode` is optional — ask the user if they have a preferred area code
+- `country` defaults to `"US"` — ask if they want a different country (ISO 3166-1 alpha-2)
+
+The daemon provisions the number via the Twilio API and automatically assigns it to the assistant. The response includes the new `phoneNumber`.
+
+**Trial account note:** Twilio trial accounts come with one free phone number. Check "Active Numbers" in the Twilio Console first before provisioning.
+
+### Option B: Assign an Existing Number
+
+If the user already has a Twilio phone number, first list available numbers:
+
+```json
+{
+  "type": "twilio_config",
+  "action": "list_numbers"
+}
+```
+
+The response includes a `numbers` array with each number's `phoneNumber`, `friendlyName`, and `capabilities` (voice, SMS). Present these to the user and let them choose.
+
+Then assign the chosen number:
+
+```json
+{
+  "type": "twilio_config",
+  "action": "assign_number",
+  "phoneNumber": "+14155551234"
+}
+```
+
+The phone number must be in E.164 format.
+
+### Option C: Manual Entry
+
+If the user wants to enter a number directly (e.g., they know it already), store it via credential store:
+
+```
+credential_store action=store service=twilio field=phone_number value=+14155551234
+```
+
+Then assign it through the IPC:
+
+```json
+{
+  "type": "twilio_config",
+  "action": "assign_number",
+  "phoneNumber": "+14155551234"
+}
+```
+
+## Step 4: Set Up Public Ingress
+
+Twilio needs a publicly reachable URL for voice webhooks, ConversationRelay WebSocket, and SMS delivery reports. The **public-ingress** skill handles this via ngrok.
+
+Check if ingress is already configured:
+
+```bash
+vellum config get ingress.publicBaseUrl
+vellum config get ingress.enabled
+```
+
+If not configured, load and run the public-ingress skill:
+
+```
+skill_load skill=public-ingress
+```
+
+**Twilio webhook endpoints (handled automatically by the gateway):**
+- Voice webhook: `{publicBaseUrl}/webhooks/twilio/voice`
+- Voice status callback: `{publicBaseUrl}/webhooks/twilio/status`
+- ConversationRelay WebSocket: `{publicBaseUrl}/webhooks/twilio/relay` (wss://)
+- SMS webhook: `{publicBaseUrl}/webhooks/twilio/sms`
+
+No manual Twilio webhook configuration is needed — webhook URLs are registered dynamically.
+
+## Step 5: Verify Setup
+
+After configuration, verify by sending `twilio_config` with `action: "get"` again.
+
+Confirm:
+- `hasCredentials` is `true`
+- `phoneNumber` is set to the expected number
+
+Tell the user: **"Twilio is configured. Your assistant's phone number is {phoneNumber}. This number is used for both voice calls and SMS messaging."**
+
+## Step 6: Enable Features
+
+Now that Twilio is configured, the user can enable the features that depend on it:
+
+**For voice calls:**
+```bash
+vellum config set calls.enabled true
+```
+
+**For SMS messaging:**
+SMS is available automatically once Twilio is configured — no additional feature flag is needed.
+
+## Clearing Credentials
+
+If the user wants to disconnect Twilio, send:
+
+```json
+{
+  "type": "twilio_config",
+  "action": "clear_credentials"
+}
+```
+
+This removes the stored Account SID and Auth Token. Your phone number assignment will be preserved. Voice calls and SMS will stop working until credentials are reconfigured.
+
+## Troubleshooting
+
+### "Twilio credentials not configured"
+Run Steps 2 and 3 to store credentials and assign a phone number.
+
+### "No phone number assigned"
+Run Step 3 to provision or assign a phone number.
+
+### Phone number provisioning fails
+- Verify Twilio credentials are correct
+- On trial accounts, you may already have a free number — check "Active Numbers" in the Console
+- Ensure the Twilio account has sufficient balance for paid accounts
+
+### Calls/SMS fail after setup
+- Verify public ingress is running (`ingress.publicBaseUrl` must be set)
+- For calls, ensure `calls.enabled` is `true`
+- On trial accounts, outbound calls and SMS can only reach verified numbers
+
+### "Number not found" when assigning
+- The number must be owned by the same Twilio account
+- Use `list_numbers` to see available numbers
+- Ensure the number is in E.164 format (`+` followed by country code and number)