@conversionpros/aiva 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +148 -0
- package/auto-deploy.js +190 -0
- package/bin/aiva.js +81 -0
- package/cli-sync.js +126 -0
- package/d2a-prompt-template.txt +106 -0
- package/diagnostics-api.js +304 -0
- package/docs/ara-dedup-fix-scope.md +112 -0
- package/docs/ara-fix-round2-scope.md +61 -0
- package/docs/ara-greeting-fix-scope.md +70 -0
- package/docs/calendar-date-fix-scope.md +28 -0
- package/docs/getting-started.md +115 -0
- package/docs/network-architecture-rollout-scope.md +43 -0
- package/docs/scope-google-oauth-integration.md +351 -0
- package/docs/settings-page-scope.md +50 -0
- package/docs/xai-imagine-scope.md +116 -0
- package/docs/xai-voice-integration-scope.md +115 -0
- package/docs/xai-voice-tools-scope.md +165 -0
- package/email-router.js +512 -0
- package/follow-up-handler.js +606 -0
- package/gateway-monitor.js +158 -0
- package/google-email.js +379 -0
- package/google-oauth.js +310 -0
- package/grok-imagine.js +97 -0
- package/health-reporter.js +287 -0
- package/invisible-prefix-base.txt +206 -0
- package/invisible-prefix-owner.txt +26 -0
- package/invisible-prefix-slim.txt +10 -0
- package/invisible-prefix.txt +43 -0
- package/knowledge-base.js +472 -0
- package/lib/cli.js +19 -0
- package/lib/config.js +124 -0
- package/lib/health.js +57 -0
- package/lib/process.js +207 -0
- package/lib/server.js +42 -0
- package/lib/setup.js +472 -0
- package/meta-capi.js +206 -0
- package/meta-leads.js +411 -0
- package/notion-oauth.js +323 -0
- package/package.json +61 -0
- package/public/agent-config.html +241 -0
- package/public/aiva-avatar-anime.png +0 -0
- package/public/css/docs.css.bak +688 -0
- package/public/css/onboarding.css +543 -0
- package/public/diagrams/claude-subscription-pool.html +329 -0
- package/public/diagrams/claude-subscription-pool.png +0 -0
- package/public/docs-icon.png +0 -0
- package/public/escalation.html +237 -0
- package/public/group-config.html +300 -0
- package/public/icon-192.png +0 -0
- package/public/icon-512.png +0 -0
- package/public/icons/agents.svg +1 -0
- package/public/icons/attach.svg +1 -0
- package/public/icons/characters.svg +1 -0
- package/public/icons/chat.svg +1 -0
- package/public/icons/docs.svg +1 -0
- package/public/icons/heartbeat.svg +1 -0
- package/public/icons/messages.svg +1 -0
- package/public/icons/mic.svg +1 -0
- package/public/icons/notes.svg +1 -0
- package/public/icons/settings.svg +1 -0
- package/public/icons/tasks.svg +1 -0
- package/public/images/onboarding/p0-communication-layer.png +0 -0
- package/public/images/onboarding/p0-infinite-surface.png +0 -0
- package/public/images/onboarding/p0-learning-model.png +0 -0
- package/public/images/onboarding/p0-meet-aiva.png +0 -0
- package/public/images/onboarding/p4-contact-intelligence.png +0 -0
- package/public/images/onboarding/p4-context-compounds.png +0 -0
- package/public/images/onboarding/p4-message-router.png +0 -0
- package/public/images/onboarding/p4-per-contact-rules.png +0 -0
- package/public/images/onboarding/p4-send-messages.png +0 -0
- package/public/images/onboarding/p6-be-precise.png +0 -0
- package/public/images/onboarding/p6-review-escalations.png +0 -0
- package/public/images/onboarding/p6-voice-input.png +0 -0
- package/public/images/onboarding/p7-completion.png +0 -0
- package/public/index.html +11594 -0
- package/public/js/onboarding.js +699 -0
- package/public/manifest.json +24 -0
- package/public/messages-v2.html +2824 -0
- package/public/permission-approve.html.bak +107 -0
- package/public/permissions.html +150 -0
- package/public/styles/design-system.css +68 -0
- package/router-db.js +604 -0
- package/router-utils.js +28 -0
- package/router-v2/adapters/imessage.js +191 -0
- package/router-v2/adapters/quo.js +82 -0
- package/router-v2/adapters/whatsapp.js +192 -0
- package/router-v2/contact-manager.js +234 -0
- package/router-v2/conversation-engine.js +498 -0
- package/router-v2/data/knowledge-base.json +176 -0
- package/router-v2/data/router-v2.db +0 -0
- package/router-v2/data/router-v2.db-shm +0 -0
- package/router-v2/data/router-v2.db-wal +0 -0
- package/router-v2/data/router.db +0 -0
- package/router-v2/db.js +457 -0
- package/router-v2/escalation-bridge.js +540 -0
- package/router-v2/follow-up-engine.js +347 -0
- package/router-v2/index.js +441 -0
- package/router-v2/ingestion.js +213 -0
- package/router-v2/knowledge-base.js +231 -0
- package/router-v2/lead-qualifier.js +152 -0
- package/router-v2/learning-loop.js +202 -0
- package/router-v2/outbound-sender.js +160 -0
- package/router-v2/package.json +13 -0
- package/router-v2/permission-gate.js +86 -0
- package/router-v2/playbook.js +177 -0
- package/router-v2/prompts/base.js +52 -0
- package/router-v2/prompts/first-contact.js +38 -0
- package/router-v2/prompts/lead-qualification.js +37 -0
- package/router-v2/prompts/scheduling.js +72 -0
- package/router-v2/prompts/style-overrides.js +22 -0
- package/router-v2/scheduler.js +301 -0
- package/router-v2/scripts/migrate-v1-to-v2.js +215 -0
- package/router-v2/scripts/seed-faq.js +67 -0
- package/router-v2/seed-knowledge-base.js +39 -0
- package/router-v2/utils/ai.js +129 -0
- package/router-v2/utils/phone.js +52 -0
- package/router-v2/utils/response-validator.js +98 -0
- package/router-v2/utils/sanitize.js +222 -0
- package/router.js +5005 -0
- package/routes/google-calendar.js +186 -0
- package/scripts/deploy.sh +62 -0
- package/scripts/macos-calendar.sh +232 -0
- package/scripts/onboard-device.sh +466 -0
- package/server.js +5131 -0
- package/start.sh +24 -0
- package/templates/AGENTS.md +548 -0
- package/templates/IDENTITY.md +15 -0
- package/templates/docs-agents.html +132 -0
- package/templates/docs-app.html +130 -0
- package/templates/docs-home.html +83 -0
- package/templates/docs-imessage.html +121 -0
- package/templates/docs-tasks.html +123 -0
- package/templates/docs-tips.html +175 -0
- package/templates/getting-started.html +809 -0
- package/templates/invisible-prefix-base.txt +171 -0
- package/templates/invisible-prefix-owner.txt +282 -0
- package/templates/invisible-prefix.txt +338 -0
- package/templates/manifest.json +61 -0
- package/templates/memory-org/clients.md +7 -0
- package/templates/memory-org/credentials.md +9 -0
- package/templates/memory-org/devices.md +7 -0
- package/templates/updates.html +464 -0
- package/templates/workspace/AGENTS.md.tmpl +161 -0
- package/templates/workspace/HEARTBEAT.md.tmpl +17 -0
- package/templates/workspace/IDENTITY.md.tmpl +15 -0
- package/templates/workspace/MEMORY.md.tmpl +16 -0
- package/templates/workspace/SOUL.md.tmpl +51 -0
- package/templates/workspace/USER.md.tmpl +25 -0
- package/tts-proxy.js +96 -0
- package/voice-call-local.js +731 -0
- package/voice-call.js +732 -0
- package/wa-listener.js +354 -0
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# Network Architecture Rollout — Policy Gate Model
|
|
2
|
+
|
|
3
|
+
## Machines to Update
|
|
4
|
+
1. nates-mac-mini (100.95.26.113, user: aiva)
|
|
5
|
+
2. james-mac-mini (100.88.226.81, user: jamesbrown)
|
|
6
|
+
3. conversion-mac-mini (100.72.95.78, user: brandonburgan)
|
|
7
|
+
|
|
8
|
+
## Changes Per Machine
|
|
9
|
+
|
|
10
|
+
### 1. Update openclaw.json agent list
|
|
11
|
+
- Add "assistant" agent as `default: true` with 20m heartbeat
|
|
12
|
+
- Change "main" agent to `default: false`, workspace `~/.openclaw/workspace-main-gate`, 30m heartbeat
|
|
13
|
+
- Keep all other agents (content, email, outreach, peekaboo, research) as-is
|
|
14
|
+
- Reference: `/Users/brandonburgan/.openclaw/workspace/sops/openclaw-config-reference.md`
|
|
15
|
+
|
|
16
|
+
### 2. Create main gate workspace
|
|
17
|
+
On each machine, create `~/.openclaw/workspace-main-gate/` with:
|
|
18
|
+
- Copy SOUL.md from `/Users/brandonburgan/.openclaw/workspace/templates/client-main-agent/SOUL.md`
|
|
19
|
+
- Copy AGENTS.md from `/Users/brandonburgan/.openclaw/workspace/templates/client-main-agent/AGENTS.md`
|
|
20
|
+
- Create `memory/` directory
|
|
21
|
+
|
|
22
|
+
### 3. Update assistant workspace
|
|
23
|
+
The existing `~/.openclaw/workspace/` becomes the assistant's workspace. Add/update:
|
|
24
|
+
- Copy SOUL.md from `/Users/brandonburgan/.openclaw/workspace/templates/client-assistant-agent/SOUL.md`
|
|
25
|
+
BUT preserve any existing USER.md, TOOLS.md, MEMORY.md, IDENTITY.md (machine-specific)
|
|
26
|
+
|
|
27
|
+
### 4. Restart gateway and verify
|
|
28
|
+
- Restart gateway on each machine
|
|
29
|
+
- Verify both main and assistant agents are running
|
|
30
|
+
- Send test message through each machine's AIVA app: "[System Test After Restart] Architecture update complete. Confirm receipt."
|
|
31
|
+
|
|
32
|
+
## SSH Access
|
|
33
|
+
- nates-mac-mini: `ssh aiva@100.95.26.113`
|
|
34
|
+
- james-mac-mini: `ssh jamesbrown@100.88.226.81` (password: Basketballev13!)
|
|
35
|
+
- conversion-mac-mini: `ssh brandonburgan@100.72.95.78` (password: CMP2024!)
|
|
36
|
+
|
|
37
|
+
All have PATH at /opt/homebrew/bin
|
|
38
|
+
|
|
39
|
+
## IMPORTANT
|
|
40
|
+
- Read the OpenClaw config reference at `sops/openclaw-config-reference.md` before making ANY config changes
|
|
41
|
+
- Make sure the gateway restarts properly after each change
|
|
42
|
+
- If a gateway fails to restart, revert the config change immediately
|
|
43
|
+
- Preserve all existing auth profiles, gateway tokens, and machine-specific settings
|
|
@@ -0,0 +1,351 @@
|
|
|
1
|
+
# Project Scope: Google OAuth Integration for AIVA App
|
|
2
|
+
|
|
3
|
+
**Author:** AIVA
|
|
4
|
+
**Date:** 2026-02-23
|
|
5
|
+
**Status:** Draft — Pending Brandon's Review
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## 1. Overview
|
|
10
|
+
|
|
11
|
+
### What We're Building
|
|
12
|
+
Native Google OAuth 2.0 integration in the AIVA app, providing centralized email (Gmail) and calendar access through server-side API endpoints.
|
|
13
|
+
|
|
14
|
+
### Why
|
|
15
|
+
The current approach uses **GOG CLI**, which requires:
|
|
16
|
+
- Per-device installation of the `gog` binary
|
|
17
|
+
- Per-device OAuth browser flow and keyring setup (`GOG_KEYRING_PASSWORD`)
|
|
18
|
+
- Each OpenClaw agent instance to have local filesystem access to credentials
|
|
19
|
+
- Manual re-auth when tokens expire or devices change
|
|
20
|
+
|
|
21
|
+
This creates friction for multi-device setups and makes the system brittle. The new approach:
|
|
22
|
+
- **Authenticate once** in the AIVA app settings UI
|
|
23
|
+
- **Tokens managed server-side** (encrypted, auto-refreshed)
|
|
24
|
+
- **OpenClaw agents call AIVA API endpoints** instead of local CLI commands
|
|
25
|
+
- **Zero per-device setup** for email and calendar access
|
|
26
|
+
- **Works with both Gmail (personal) and Google Workspace accounts**
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## 2. Architecture
|
|
31
|
+
|
|
32
|
+
### 2.1 Google Cloud Console Setup
|
|
33
|
+
|
|
34
|
+
1. **Create OAuth 2.0 credentials** in the existing Google Cloud project (or a new `aiva-app` project)
|
|
35
|
+
2. **OAuth consent screen** configured as:
|
|
36
|
+
- App name: "AIVA" (or "AIVA by CMP")
|
|
37
|
+
- Type: **External** (supports both Gmail and Workspace accounts)
|
|
38
|
+
- Authorized domains: the AIVA app domain
|
|
39
|
+
3. **OAuth client ID** — type: Web Application
|
|
40
|
+
- Authorized redirect URI: `https://<aiva-app-domain>/api/integrations/google/callback`
|
|
41
|
+
- For local dev: `http://localhost:<port>/api/integrations/google/callback`
|
|
42
|
+
4. Store `client_id` and `client_secret` in environment variables (never in code)
|
|
43
|
+
|
|
44
|
+
### 2.2 OAuth 2.0 Flow (Authorization Code Grant with PKCE)
|
|
45
|
+
|
|
46
|
+
```
|
|
47
|
+
User clicks "Connect Google Account"
|
|
48
|
+
│
|
|
49
|
+
▼
|
|
50
|
+
Frontend requests → GET /api/integrations/google/auth-url
|
|
51
|
+
│
|
|
52
|
+
▼
|
|
53
|
+
Backend generates:
|
|
54
|
+
- code_verifier (random 128 chars)
|
|
55
|
+
- code_challenge = SHA256(code_verifier)
|
|
56
|
+
- state token (CSRF protection, stored in session)
|
|
57
|
+
- Returns Google OAuth URL with scopes, redirect_uri, code_challenge
|
|
58
|
+
│
|
|
59
|
+
▼
|
|
60
|
+
User redirected to Google → signs in → grants consent
|
|
61
|
+
│
|
|
62
|
+
▼
|
|
63
|
+
Google redirects to → GET /api/integrations/google/callback?code=...&state=...
|
|
64
|
+
│
|
|
65
|
+
▼
|
|
66
|
+
Backend:
|
|
67
|
+
1. Validates state token
|
|
68
|
+
2. Exchanges auth code + code_verifier for access_token + refresh_token
|
|
69
|
+
3. Fetches user profile (email, name) via Google userinfo endpoint
|
|
70
|
+
4. Encrypts tokens → stores in oauth-tokens.json
|
|
71
|
+
5. Redirects user back to settings page with success status
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
### 2.3 Token Storage
|
|
75
|
+
|
|
76
|
+
**File:** `oauth-tokens.json` (separate from existing `tokens.json` to avoid coupling)
|
|
77
|
+
|
|
78
|
+
```json
|
|
79
|
+
{
|
|
80
|
+
"google": {
|
|
81
|
+
"accounts": [
|
|
82
|
+
{
|
|
83
|
+
"id": "uuid-v4",
|
|
84
|
+
"email": "brandon@example.com",
|
|
85
|
+
"name": "Brandon Burgan",
|
|
86
|
+
"type": "personal",
|
|
87
|
+
"access_token": "<AES-256-GCM encrypted>",
|
|
88
|
+
"refresh_token": "<AES-256-GCM encrypted>",
|
|
89
|
+
"token_expiry": "2026-02-23T08:53:00Z",
|
|
90
|
+
"scopes": ["gmail.modify", "calendar", "contacts.readonly"],
|
|
91
|
+
"connected_at": "2026-02-23T07:53:00Z"
|
|
92
|
+
}
|
|
93
|
+
]
|
|
94
|
+
}
|
|
95
|
+
}
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
- Encryption matches existing `tokens.json` pattern (AES-256-GCM with key from env)
|
|
99
|
+
- File added to `.gitignore`
|
|
100
|
+
|
|
101
|
+
### 2.4 Token Refresh
|
|
102
|
+
|
|
103
|
+
- On every API call, check `token_expiry`
|
|
104
|
+
- If expiring within **5 minutes**, refresh proactively using the `refresh_token`
|
|
105
|
+
- If refresh fails (revoked, expired), mark account as `disconnected` and notify via status endpoint
|
|
106
|
+
- No user interaction needed for refresh — fully automatic
|
|
107
|
+
|
|
108
|
+
### 2.5 Required OAuth Scopes
|
|
109
|
+
|
|
110
|
+
| Scope | Purpose |
|
|
111
|
+
|---|---|
|
|
112
|
+
| `https://www.googleapis.com/auth/gmail.modify` | Read, send, archive, mark read/unread |
|
|
113
|
+
| `https://www.googleapis.com/auth/calendar` | Read/write calendar events |
|
|
114
|
+
| `https://www.googleapis.com/auth/contacts.readonly` | Read contacts for autocomplete |
|
|
115
|
+
| `https://www.googleapis.com/auth/userinfo.email` | Get authenticated user's email |
|
|
116
|
+
| `https://www.googleapis.com/auth/userinfo.profile` | Get user's display name |
|
|
117
|
+
|
|
118
|
+
**Note:** `gmail.modify` is used instead of the broader `mail.google.com` to minimize scope. It covers read, send, archive, and label management.
|
|
119
|
+
|
|
120
|
+
---
|
|
121
|
+
|
|
122
|
+
## 3. Frontend (Settings Page)
|
|
123
|
+
|
|
124
|
+
### 3.1 UI Components
|
|
125
|
+
|
|
126
|
+
**Settings → Integrations → Google Account**
|
|
127
|
+
|
|
128
|
+
- **"Connect Google Account" button** — initiates OAuth flow
|
|
129
|
+
- **Connected account card** showing:
|
|
130
|
+
- Profile picture (from Google)
|
|
131
|
+
- Email address
|
|
132
|
+
- Account type badge: "Personal" or "Workspace"
|
|
133
|
+
- Connection status: Connected ✅ / Expired ⚠️ / Disconnected ❌
|
|
134
|
+
- Last synced timestamp
|
|
135
|
+
- **"Disconnect" button** — with confirmation dialog
|
|
136
|
+
- **"Add Another Account" button** — for connecting multiple accounts
|
|
137
|
+
|
|
138
|
+
### 3.2 Multiple Account Support
|
|
139
|
+
|
|
140
|
+
Users can connect both personal Gmail and Google Workspace accounts simultaneously. Each account is independently managed with its own tokens. The email and calendar tabs show a combined view with account selector/filter.
|
|
141
|
+
|
|
142
|
+
### 3.3 State Handling
|
|
143
|
+
|
|
144
|
+
- Poll `/api/integrations/google/status` on settings page load
|
|
145
|
+
- After OAuth callback redirect, show success/error toast
|
|
146
|
+
- If token refresh fails, show warning banner in email/calendar tabs
|
|
147
|
+
|
|
148
|
+
---
|
|
149
|
+
|
|
150
|
+
## 4. Backend API Endpoints
|
|
151
|
+
|
|
152
|
+
All endpoints require AIVA app authentication (`x-aiva-internal` header or session auth).
|
|
153
|
+
|
|
154
|
+
### 4.1 OAuth Management
|
|
155
|
+
|
|
156
|
+
| Method | Endpoint | Description |
|
|
157
|
+
|---|---|---|
|
|
158
|
+
| `GET` | `/api/integrations/google/auth-url` | Generate OAuth authorization URL. Query params: `account_type=personal\|workspace` |
|
|
159
|
+
| `GET` | `/api/integrations/google/callback` | Handle OAuth callback. Exchanges code for tokens, stores encrypted. Redirects to settings. |
|
|
160
|
+
| `GET` | `/api/integrations/google/status` | Returns list of connected accounts with email, type, status, expiry. No tokens exposed. |
|
|
161
|
+
| `DELETE` | `/api/integrations/google/disconnect` | Body: `{ accountId }`. Revokes token with Google, deletes from storage. |
|
|
162
|
+
|
|
163
|
+
### 4.2 Email Endpoints
|
|
164
|
+
|
|
165
|
+
| Method | Endpoint | Description |
|
|
166
|
+
|---|---|---|
|
|
167
|
+
| `GET` | `/api/integrations/google/email/inbox` | List emails. Query: `accountId`, `maxResults` (default 20), `pageToken`, `q` (search), `labelIds` |
|
|
168
|
+
| `GET` | `/api/integrations/google/email/:id` | Get full email (headers, body, attachments metadata). Query: `accountId` |
|
|
169
|
+
| `POST` | `/api/integrations/google/email/send` | Send email. Body: `{ accountId, to, cc, bcc, subject, body, inReplyTo?, threadId? }` |
|
|
170
|
+
| `POST` | `/api/integrations/google/email/:id/archive` | Remove INBOX label. Body: `{ accountId }` |
|
|
171
|
+
| `POST` | `/api/integrations/google/email/:id/mark-read` | Remove UNREAD label. Body: `{ accountId }` |
|
|
172
|
+
|
|
173
|
+
All email endpoints use the Gmail API v1 (`gmail.googleapis.com`).
|
|
174
|
+
|
|
175
|
+
### 4.3 Calendar Endpoints
|
|
176
|
+
|
|
177
|
+
| Method | Endpoint | Description |
|
|
178
|
+
|---|---|---|
|
|
179
|
+
| `GET` | `/api/integrations/google/calendar/events` | List events. Query: `accountId`, `calendarId` (default "primary"), `timeMin`, `timeMax`, `maxResults` |
|
|
180
|
+
| `POST` | `/api/integrations/google/calendar/events` | Create event. Body: `{ accountId, calendarId?, summary, start, end, description?, attendees?, location? }` |
|
|
181
|
+
| `PUT` | `/api/integrations/google/calendar/events/:id` | Update event. Body: same as create + `{ accountId, calendarId? }` |
|
|
182
|
+
| `DELETE` | `/api/integrations/google/calendar/events/:id` | Delete event. Query: `accountId`, `calendarId?` |
|
|
183
|
+
|
|
184
|
+
All calendar endpoints use the Google Calendar API v3.
|
|
185
|
+
|
|
186
|
+
### 4.4 Response Format
|
|
187
|
+
|
|
188
|
+
```json
|
|
189
|
+
{
|
|
190
|
+
"success": true,
|
|
191
|
+
"data": { ... },
|
|
192
|
+
"error": null
|
|
193
|
+
}
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
On token expiry that can't be auto-refreshed:
|
|
197
|
+
```json
|
|
198
|
+
{
|
|
199
|
+
"success": false,
|
|
200
|
+
"error": "GOOGLE_AUTH_EXPIRED",
|
|
201
|
+
"message": "Google account requires re-authentication",
|
|
202
|
+
"accountId": "..."
|
|
203
|
+
}
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
---
|
|
207
|
+
|
|
208
|
+
## 5. Migration Path
|
|
209
|
+
|
|
210
|
+
### 5.1 Transition Strategy
|
|
211
|
+
|
|
212
|
+
| Phase | Email Source | Calendar Source | GOG Status |
|
|
213
|
+
|---|---|---|---|
|
|
214
|
+
| Current | GOG CLI | GOG CLI | Required on every device |
|
|
215
|
+
| Phase 2 complete | AIVA API | GOG CLI | Required only for calendar |
|
|
216
|
+
| Phase 3 complete | AIVA API | AIVA API | Not required |
|
|
217
|
+
| Phase 4 complete | AIVA API | AIVA API | Can be uninstalled |
|
|
218
|
+
|
|
219
|
+
### 5.2 Backward Compatibility
|
|
220
|
+
|
|
221
|
+
- During rollout, **both paths remain functional** — GOG CLI continues to work alongside new API endpoints
|
|
222
|
+
- OpenClaw TOOLS.md gets updated with new API instructions but GOG fallback documented
|
|
223
|
+
- Feature flag: `GOOGLE_INTEGRATION_ENABLED=true` in env to toggle new endpoints
|
|
224
|
+
|
|
225
|
+
### 5.3 Agent Migration (Phase 4)
|
|
226
|
+
|
|
227
|
+
OpenClaw agents currently run:
|
|
228
|
+
```bash
|
|
229
|
+
GOG_KEYRING_PASSWORD=openclaw gog calendar events primary --from ... --to ... --account ...
|
|
230
|
+
GOG_KEYRING_PASSWORD=openclaw gog gmail inbox --account ...
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
After migration, agents call:
|
|
234
|
+
```bash
|
|
235
|
+
curl -H "x-aiva-internal: true" "http://localhost:PORT/api/integrations/google/calendar/events?accountId=...&timeMin=...&timeMax=..."
|
|
236
|
+
curl -H "x-aiva-internal: true" "http://localhost:PORT/api/integrations/google/email/inbox?accountId=..."
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
TOOLS.md and relevant SOPs updated to reflect new commands. Old GOG references removed after Phase 4 validation.
|
|
240
|
+
|
|
241
|
+
---
|
|
242
|
+
|
|
243
|
+
## 6. Security
|
|
244
|
+
|
|
245
|
+
### 6.1 Token Encryption at Rest
|
|
246
|
+
- **AES-256-GCM** encryption for both `access_token` and `refresh_token`
|
|
247
|
+
- Encryption key sourced from environment variable (same pattern as existing `tokens.json`)
|
|
248
|
+
- Tokens never written to disk unencrypted
|
|
249
|
+
|
|
250
|
+
### 6.2 Scope Minimization
|
|
251
|
+
- Request only the scopes listed in §2.5 — no `mail.google.com` (full access)
|
|
252
|
+
- `gmail.modify` instead of `gmail.compose` + `gmail.readonly` (single scope covers all needs)
|
|
253
|
+
- Contacts are read-only
|
|
254
|
+
|
|
255
|
+
### 6.3 Token Revocation
|
|
256
|
+
- On disconnect, call Google's revocation endpoint (`https://oauth2.googleapis.com/revoke`)
|
|
257
|
+
- Delete tokens from `oauth-tokens.json` immediately after revocation
|
|
258
|
+
- If revocation API call fails, still delete local tokens (defense in depth)
|
|
259
|
+
|
|
260
|
+
### 6.4 Git Safety
|
|
261
|
+
- `oauth-tokens.json` in `.gitignore`
|
|
262
|
+
- `GOOGLE_CLIENT_SECRET` only in env, never in code
|
|
263
|
+
- No tokens logged (even at debug level)
|
|
264
|
+
|
|
265
|
+
### 6.5 CSRF Protection
|
|
266
|
+
- `state` parameter in OAuth flow validated on callback
|
|
267
|
+
- `state` stored server-side in short-lived session/cache (5 min TTL)
|
|
268
|
+
|
|
269
|
+
---
|
|
270
|
+
|
|
271
|
+
## 7. Phases
|
|
272
|
+
|
|
273
|
+
### Phase 1: OAuth Flow + Token Management + Settings UI
|
|
274
|
+
- Google Cloud Console project setup (consent screen, credentials)
|
|
275
|
+
- `/auth-url`, `/callback`, `/status`, `/disconnect` endpoints
|
|
276
|
+
- Token encryption/storage/refresh logic
|
|
277
|
+
- Settings page UI (connect, status, disconnect)
|
|
278
|
+
- Multiple account support
|
|
279
|
+
- **Deliverable:** User can connect/disconnect Google accounts from AIVA settings
|
|
280
|
+
|
|
281
|
+
### Phase 2: Email API Endpoints + Email Tab Migration
|
|
282
|
+
- All 5 email endpoints (inbox, get, send, archive, mark-read)
|
|
283
|
+
- Update AIVA email tab to use new endpoints instead of GOG
|
|
284
|
+
- Search/filter support
|
|
285
|
+
- Thread view support
|
|
286
|
+
- **Deliverable:** Email fully works through AIVA app, GOG no longer needed for email
|
|
287
|
+
|
|
288
|
+
### Phase 3: Calendar API Endpoints + Calendar Migration
|
|
289
|
+
- All 4 calendar endpoints (list, create, update, delete)
|
|
290
|
+
- Update AIVA calendar features to use new endpoints
|
|
291
|
+
- Multi-calendar support (primary + secondary calendars)
|
|
292
|
+
- **Deliverable:** Calendar fully works through AIVA app, GOG no longer needed at all
|
|
293
|
+
|
|
294
|
+
### Phase 4: OpenClaw Agent Migration
|
|
295
|
+
- Update TOOLS.md with new API-based commands
|
|
296
|
+
- Update SOPs referencing GOG CLI
|
|
297
|
+
- Test all agent workflows (email check, send, calendar lookup, event creation)
|
|
298
|
+
- Remove GOG CLI from device requirements
|
|
299
|
+
- **Deliverable:** Zero dependency on GOG CLI across all devices and agents
|
|
300
|
+
|
|
301
|
+
---
|
|
302
|
+
|
|
303
|
+
## 8. Effort Estimates
|
|
304
|
+
|
|
305
|
+
| Phase | Scope | Estimate |
|
|
306
|
+
|---|---|---|
|
|
307
|
+
| Phase 1 | OAuth flow + tokens + settings UI | 2–3 days |
|
|
308
|
+
| Phase 2 | Email endpoints + email tab migration | 2–3 days |
|
|
309
|
+
| Phase 3 | Calendar endpoints + calendar migration | 1–2 days |
|
|
310
|
+
| Phase 4 | Agent migration + cleanup | 1 day |
|
|
311
|
+
| **Total** | | **6–9 days** |
|
|
312
|
+
|
|
313
|
+
Estimates assume one developer, familiar with the AIVA codebase. Includes basic testing but not Google OAuth app verification (see Risks).
|
|
314
|
+
|
|
315
|
+
---
|
|
316
|
+
|
|
317
|
+
## 9. Risks and Considerations
|
|
318
|
+
|
|
319
|
+
### 9.1 Google OAuth App Verification
|
|
320
|
+
- Apps requesting sensitive scopes (`gmail.modify`, `calendar`) require **Google verification**
|
|
321
|
+
- Verification process: privacy policy, homepage, demo video — can take **1–4 weeks**
|
|
322
|
+
- **Mitigation:** During development, app runs in "Testing" mode (limited to 100 test users, which is fine for our use case). If we stay under 100 users, verification may not be required. Can submit for verification later if needed.
|
|
323
|
+
|
|
324
|
+
### 9.2 API Rate Limits
|
|
325
|
+
- **Gmail API:** 250 quota units/second per user (inbox list = 5 units, send = 100 units)
|
|
326
|
+
- **Calendar API:** 1,000,000 queries/day (effectively unlimited for our use)
|
|
327
|
+
- **Mitigation:** Implement basic rate limiting on our endpoints; cache inbox listings briefly (30s TTL)
|
|
328
|
+
|
|
329
|
+
### 9.3 Consent Screen Branding
|
|
330
|
+
- Unverified apps show a "This app isn't verified" warning screen
|
|
331
|
+
- Users must click "Advanced" → "Go to AIVA (unsafe)" to proceed
|
|
332
|
+
- **Mitigation:** Acceptable for internal use. Submit for verification if it becomes user-facing beyond our team.
|
|
333
|
+
|
|
334
|
+
### 9.4 Refresh Token Expiration
|
|
335
|
+
- Google refresh tokens can expire if: unused for 6 months, user changes password, user revokes access, or app exceeds 50 refresh tokens per account
|
|
336
|
+
- **Mitigation:** Proactive refresh keeps tokens active. Status endpoint surfaces auth issues. UI shows re-auth prompt when needed.
|
|
337
|
+
|
|
338
|
+
### 9.5 Multi-Account Complexity
|
|
339
|
+
- Users with both personal and Workspace accounts need clear UX to pick which account to use for sending
|
|
340
|
+
- **Mitigation:** Default account setting + per-action account selector in email/calendar UI
|
|
341
|
+
|
|
342
|
+
---
|
|
343
|
+
|
|
344
|
+
## Approval
|
|
345
|
+
|
|
346
|
+
**Decision needed from Brandon:**
|
|
347
|
+
- [ ] Approve scope as written
|
|
348
|
+
- [ ] Approve with modifications (note changes)
|
|
349
|
+
- [ ] Defer (with reason)
|
|
350
|
+
|
|
351
|
+
Once approved, begin Phase 1.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Settings Page Changes — Scope Doc
|
|
2
|
+
|
|
3
|
+
## Task
|
|
4
|
+
Modify the AIVA app settings page and add a Getting Started page.
|
|
5
|
+
|
|
6
|
+
## Changes Required
|
|
7
|
+
|
|
8
|
+
### 1. Settings Page (`public/index.html`)
|
|
9
|
+
**Location:** Lines ~2641-2650 in the settings view
|
|
10
|
+
|
|
11
|
+
**Remove** the entire "Admin Tools" section:
|
|
12
|
+
```html
|
|
13
|
+
<div class="settings-section">
|
|
14
|
+
<h2>Admin Tools</h2>
|
|
15
|
+
<div class="settings-row">
|
|
16
|
+
<label>OpenClaw Dashboard</label>
|
|
17
|
+
<a href="https://brandons-mac-studio.tail61c982.ts.net/" target="_blank" class="btn btn-ghost">Open →</a>
|
|
18
|
+
</div>
|
|
19
|
+
</div>
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
**Replace with:**
|
|
23
|
+
```html
|
|
24
|
+
<div class="settings-section">
|
|
25
|
+
<h2>Resources</h2>
|
|
26
|
+
<div class="settings-row">
|
|
27
|
+
<label>Getting Started with AIVA</label>
|
|
28
|
+
<a href="https://aivahelpme.com/getting-started" target="_blank" class="btn btn-ghost">Open →</a>
|
|
29
|
+
</div>
|
|
30
|
+
</div>
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
### 2. Getting Started Page
|
|
34
|
+
Create a new route `/getting-started` that serves the Getting Started content.
|
|
35
|
+
|
|
36
|
+
**Content source:** `docs/getting-started.md`
|
|
37
|
+
|
|
38
|
+
This should be a clean, public-facing page (no auth required) rendered from the markdown content. Style it nicely — clean typography, readable, mobile-friendly. Use the AIVA branding (dark theme matches the app).
|
|
39
|
+
|
|
40
|
+
**Add route in `server.js`:**
|
|
41
|
+
- `GET /getting-started` → serves rendered Getting Started page
|
|
42
|
+
|
|
43
|
+
### 3. Updates Page (Future)
|
|
44
|
+
Create a stub route `/updates` that we can populate later with changelog entries. For now, a simple page saying "Check back for the latest AIVA updates" is fine.
|
|
45
|
+
|
|
46
|
+
## Important
|
|
47
|
+
- NO emojis in the UI — use custom SVGs only (existing rule)
|
|
48
|
+
- Wait — the markdown content has emojis for readability. The rendered HTML page should convert those to simple text labels or SVG icons instead.
|
|
49
|
+
- Keep the page public (no auth) so the link works for anyone with the URL
|
|
50
|
+
- Mobile-first responsive design
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
# xAI Grok Imagine Integration — Image & Video Generation
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
Add Grok Imagine as a generation backend for images and videos in the AIVA app. When Brandon (or AIVA via the main agent) requests image/video generation, Grok Imagine should be available as a provider.
|
|
5
|
+
|
|
6
|
+
## API Details
|
|
7
|
+
|
|
8
|
+
### Image Generation
|
|
9
|
+
- **Endpoint:** `POST https://api.x.ai/v1/images/generations`
|
|
10
|
+
- **Model:** `grok-imagine-image`
|
|
11
|
+
- **Auth:** `Authorization: Bearer <XAI_API_KEY>`
|
|
12
|
+
- **Capabilities:** text-to-image, image editing (with source image), style transfer
|
|
13
|
+
- **Request body:**
|
|
14
|
+
```json
|
|
15
|
+
{
|
|
16
|
+
"model": "grok-imagine-image",
|
|
17
|
+
"prompt": "description",
|
|
18
|
+
"n": 1,
|
|
19
|
+
"response_format": "url"
|
|
20
|
+
}
|
|
21
|
+
```
|
|
22
|
+
- For image editing, add `"image_url": "<url or data:image/jpeg;base64,...>"`
|
|
23
|
+
- **Response:** `{ "data": [{ "url": "..." }] }` (OpenAI-compatible format)
|
|
24
|
+
|
|
25
|
+
### Video Generation
|
|
26
|
+
- **Endpoint:** `POST https://api.x.ai/v1/videos/generations`
|
|
27
|
+
- **Model:** `grok-imagine-video`
|
|
28
|
+
- **Async:** Returns `{ "request_id": "..." }` — must poll for completion
|
|
29
|
+
- **Poll:** `GET https://api.x.ai/v1/videos/<request_id>`
|
|
30
|
+
- **Status values:** `pending`, `done`, `expired`
|
|
31
|
+
- **When done:** `{ "status": "done", "video": { "url": "...", "duration": N } }`
|
|
32
|
+
- **Options:** `duration` (1-15 sec), `aspect_ratio` (16:9, 9:16, 1:1, etc.), `resolution` (480p, 720p)
|
|
33
|
+
- For image-to-video, add `"image_url": "..."`
|
|
34
|
+
- For video editing, add `"video_url": "..."` (max 8.7 sec input)
|
|
35
|
+
|
|
36
|
+
### API Key
|
|
37
|
+
Use env var `XAI_API_KEY` or fallback to hardcoded: `xai-Gn37fuJg5ty4gvWFG2rbth34AxNORUKH8r4vTXQDtjwMGUqKZ7nYy8u2YStosGUCVBEg7VMHSqQZcKS4`
|
|
38
|
+
|
|
39
|
+
## What to Build
|
|
40
|
+
|
|
41
|
+
### 1. Server module: `grok-imagine.js`
|
|
42
|
+
Create a new module at `~/.openclaw/workspace/aiva-tasks/grok-imagine.js` with:
|
|
43
|
+
|
|
44
|
+
```javascript
|
|
45
|
+
// Functions:
|
|
46
|
+
async function generateImage(prompt, options = {})
|
|
47
|
+
// options: { imageUrl, n, aspectRatio, resolution }
|
|
48
|
+
// Returns: { url: "...", ... }
|
|
49
|
+
|
|
50
|
+
async function generateVideo(prompt, options = {})
|
|
51
|
+
// options: { imageUrl, videoUrl, duration, aspectRatio, resolution }
|
|
52
|
+
// Handles polling internally (poll every 5 sec, timeout 10 min)
|
|
53
|
+
// Returns: { url: "...", duration: N }
|
|
54
|
+
|
|
55
|
+
async function editImage(prompt, sourceImageUrl)
|
|
56
|
+
// Convenience wrapper for image editing
|
|
57
|
+
|
|
58
|
+
async function editVideo(prompt, sourceVideoUrl)
|
|
59
|
+
// Convenience wrapper for video editing
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
### 2. Server route: `POST /api/generate`
|
|
63
|
+
Add to `server.js`:
|
|
64
|
+
```
|
|
65
|
+
POST /api/generate
|
|
66
|
+
Body: {
|
|
67
|
+
"type": "image" | "video",
|
|
68
|
+
"prompt": "...",
|
|
69
|
+
"options": { ... } // optional: imageUrl, videoUrl, duration, aspectRatio, resolution
|
|
70
|
+
}
|
|
71
|
+
```
|
|
72
|
+
- For images: call `generateImage()`, return URL immediately
|
|
73
|
+
- For videos: call `generateVideo()`, which polls until done, return URL
|
|
74
|
+
- Save generated media to chat history so it shows up in the media gallery
|
|
75
|
+
- Send as an AIVA message in the chat (with imageUrl or videoUrl set)
|
|
76
|
+
|
|
77
|
+
### 3. Voice call tool: `generate_media`
|
|
78
|
+
Add to voice-call.js tools array:
|
|
79
|
+
```json
|
|
80
|
+
{
|
|
81
|
+
"type": "function",
|
|
82
|
+
"name": "generate_media",
|
|
83
|
+
"description": "Generate an image or video using AI. Use when Brandon asks you to create, generate, or make an image or video.",
|
|
84
|
+
"parameters": {
|
|
85
|
+
"type": "object",
|
|
86
|
+
"properties": {
|
|
87
|
+
"type": { "type": "string", "enum": ["image", "video"], "description": "Type of media to generate" },
|
|
88
|
+
"prompt": { "type": "string", "description": "Description of what to generate" },
|
|
89
|
+
"duration": { "type": "number", "description": "Video duration in seconds (1-15, default 5)" },
|
|
90
|
+
"aspect_ratio": { "type": "string", "description": "Aspect ratio (16:9, 9:16, 1:1, 4:3, etc.)" }
|
|
91
|
+
},
|
|
92
|
+
"required": ["type", "prompt"]
|
|
93
|
+
}
|
|
94
|
+
}
|
|
95
|
+
```
|
|
96
|
+
Implementation: call the `/api/generate` endpoint internally, return the URL to Grok so Ara can tell Brandon it's ready.
|
|
97
|
+
|
|
98
|
+
### 4. Chat integration
|
|
99
|
+
When AIVA (main agent) gets a request to generate images/videos, it can use this endpoint too. The generated media should:
|
|
100
|
+
- Be saved as a chat message with the appropriate URL field (imageUrl/videoUrl)
|
|
101
|
+
- Show up in the media gallery automatically
|
|
102
|
+
- Include the prompt as the message caption
|
|
103
|
+
|
|
104
|
+
## Files to Create/Modify
|
|
105
|
+
1. **CREATE** `grok-imagine.js` — Generation module
|
|
106
|
+
2. **MODIFY** `server.js` — Add `/api/generate` route + require grok-imagine
|
|
107
|
+
3. **MODIFY** `voice-call.js` — Add `generate_media` tool to xAI session tools array + handler in executeTool
|
|
108
|
+
|
|
109
|
+
## DO NOT TOUCH
|
|
110
|
+
- Microphone icon
|
|
111
|
+
- Media gallery (just built)
|
|
112
|
+
- Audio streaming code
|
|
113
|
+
- Any other existing features
|
|
114
|
+
|
|
115
|
+
## After changes
|
|
116
|
+
`pm2 restart aiva-app`
|
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
# xAI Voice Agent Integration — Scope Doc
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
Replace the current phone icon voice call feature in the AIVA app with xAI's Voice Agent API using the **Ara** voice. The microphone icon (voice-to-text for chat) must NOT be touched.
|
|
5
|
+
|
|
6
|
+
## Current Architecture
|
|
7
|
+
- **Client:** `public/index.html` — phone icon button triggers `toggleVoiceCall()` → push-to-talk UI
|
|
8
|
+
- **Server:** `voice-call.js` — Socket.IO events, Whisper STT → Claude → ElevenLabs TTS pipeline
|
|
9
|
+
- **Flow:** Record audio → send blob via socket → server transcribes → Claude responds → ElevenLabs generates speech → send audio back
|
|
10
|
+
|
|
11
|
+
## New Architecture
|
|
12
|
+
Replace with xAI's real-time Voice Agent API (WebSocket-based, bidirectional audio streaming).
|
|
13
|
+
|
|
14
|
+
### xAI Voice Agent API Details
|
|
15
|
+
- **Endpoint:** `wss://api.x.ai/v1/realtime`
|
|
16
|
+
- **API Key:** `xai-Gn37fuJg5ty4gvWFG2rbth34AxNORUKH8r4vTXQDtjwMGUqKZ7nYy8u2YStosGUCVBEg7VMHSqQZcKS4`
|
|
17
|
+
- **Voice:** `Ara`
|
|
18
|
+
- **Price:** $0.05/min
|
|
19
|
+
- **Docs:** https://docs.x.ai/developers/model-capabilities/audio/voice-agent
|
|
20
|
+
- **Compatible with OpenAI Realtime API spec**
|
|
21
|
+
|
|
22
|
+
### Key Changes
|
|
23
|
+
|
|
24
|
+
#### Server (`voice-call.js`)
|
|
25
|
+
1. When a voice call starts, open a WebSocket to `wss://api.x.ai/v1/realtime`
|
|
26
|
+
2. Authenticate with the API key
|
|
27
|
+
3. Send `session.update` with:
|
|
28
|
+
- `voice: "Ara"`
|
|
29
|
+
- `instructions`: Use the same system prompt builder (`buildSystemPrompt`) but adapted for Grok context
|
|
30
|
+
- Any tools we want (web_search, etc. are optional)
|
|
31
|
+
4. Stream client audio to xAI WebSocket (bidirectional)
|
|
32
|
+
5. Stream xAI audio responses back to client
|
|
33
|
+
6. Handle turn detection, interruption, etc.
|
|
34
|
+
|
|
35
|
+
#### Client (`public/index.html`)
|
|
36
|
+
1. Change from push-to-talk to **continuous streaming** (xAI handles VAD/turn detection)
|
|
37
|
+
2. When call starts: get mic stream, connect via Socket.IO to server
|
|
38
|
+
3. Server proxies audio to/from xAI WebSocket
|
|
39
|
+
4. Play incoming audio in real-time (streaming, not waiting for full response)
|
|
40
|
+
5. UI updates: remove "Hold mic to talk" — change to just "Listening..." / "Ara speaking..."
|
|
41
|
+
6. Keep the call timer, end call button, transcript display
|
|
42
|
+
|
|
43
|
+
#### Environment
|
|
44
|
+
- Add `XAI_API_KEY` env var (or use the hardcoded key for now)
|
|
45
|
+
- Keep all existing ElevenLabs/Whisper/Claude env vars (they're used by other features)
|
|
46
|
+
|
|
47
|
+
## DO NOT TOUCH
|
|
48
|
+
- The microphone icon and its voice-to-text functionality
|
|
49
|
+
- Any other part of the app
|
|
50
|
+
- The `sag` CLI or ElevenLabs integration used elsewhere
|
|
51
|
+
|
|
52
|
+
## UI Changes
|
|
53
|
+
- Phone icon behavior stays the same (tap to start/end call)
|
|
54
|
+
- Remove push-to-talk mic button — conversation is now continuous/hands-free
|
|
55
|
+
- Keep: call timer, end call button, transcript area, voice visualizer
|
|
56
|
+
- Status text: "Connecting..." → "Listening..." → "Ara speaking..." → "Listening..."
|
|
57
|
+
- NO emojis in any UI elements (use text or SVGs only)
|
|
58
|
+
|
|
59
|
+
## Audio Format
|
|
60
|
+
- xAI Realtime API uses PCM16 audio at 24kHz (same as OpenAI Realtime)
|
|
61
|
+
- Client needs to capture mic audio and send as PCM16
|
|
62
|
+
- Server receives PCM16 from xAI and sends to client for playback
|
|
63
|
+
|
|
64
|
+
## AIVA Integration Hooks (CRITICAL)
|
|
65
|
+
|
|
66
|
+
### 1. Pre-Call Context Injection
|
|
67
|
+
When a call starts, BEFORE sending `session.update` to xAI:
|
|
68
|
+
1. Fetch context from `http://localhost:3847/api/context/voice` (already exists in voice-call.js)
|
|
69
|
+
2. Build a rich system prompt that includes:
|
|
70
|
+
- Recent AIVA app chat history (last 10 messages)
|
|
71
|
+
- Active tasks from the task board
|
|
72
|
+
- User info (Brandon's preferences, current projects)
|
|
73
|
+
- Today's calendar events (if available from context API)
|
|
74
|
+
- Memory/personality context
|
|
75
|
+
3. Inject this as the `instructions` field in `session.update`
|
|
76
|
+
4. This way Grok/Ara has full context about what Brandon is working on
|
|
77
|
+
|
|
78
|
+
### 2. Post-Call Transcript Hook
|
|
79
|
+
When a call ends:
|
|
80
|
+
1. Collect the full transcript (all user + assistant turns) accumulated during the call
|
|
81
|
+
2. Save transcript to `~/.openclaw/workspace/memory/call-logs/pending_ara_<timestamp>.json` with format:
|
|
82
|
+
```json
|
|
83
|
+
{
|
|
84
|
+
"type": "ara-voice-call",
|
|
85
|
+
"timestamp": "<ISO>",
|
|
86
|
+
"duration": <seconds>,
|
|
87
|
+
"transcript": [
|
|
88
|
+
{"role": "user", "text": "..."},
|
|
89
|
+
{"role": "assistant", "text": "..."}
|
|
90
|
+
]
|
|
91
|
+
}
|
|
92
|
+
```
|
|
93
|
+
3. Also fire a wake hook to notify AIVA main agent:
|
|
94
|
+
```
|
|
95
|
+
POST http://localhost:3847/hooks/wake
|
|
96
|
+
x-aiva-internal: true
|
|
97
|
+
{"text": "[VOICE-CALL-COMPLETE] Ara voice call ended. Duration: Xm Xs. Transcript saved to memory/call-logs/pending_ara_<timestamp>.json. Process action items."}
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
This lets AIVA (Claude) review what was discussed and act on any tasks/requests.
|
|
101
|
+
|
|
102
|
+
## Testing
|
|
103
|
+
- Tap phone icon → should connect and start listening immediately
|
|
104
|
+
- Speak naturally → Ara responds in real-time with her voice
|
|
105
|
+
- Tap end call → cleanly disconnects
|
|
106
|
+
- Microphone icon (chat voice-to-text) still works exactly as before
|
|
107
|
+
|
|
108
|
+
## Files to Modify
|
|
109
|
+
1. `voice-call.js` — Replace STT/Claude/TTS pipeline with xAI WebSocket proxy
|
|
110
|
+
2. `public/index.html` — Update voice call UI (continuous mode, remove PTT)
|
|
111
|
+
3. `package.json` — Add `ws` package if not already present (for WebSocket client)
|
|
112
|
+
|
|
113
|
+
## Reference
|
|
114
|
+
- xAI Voice Agent docs: https://docs.x.ai/developers/model-capabilities/audio/voice-agent
|
|
115
|
+
- Compatible with OpenAI Realtime API format
|