dsh-agora 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (60) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +88 -0
  3. package/assets/agora/SKILL.md +113 -0
  4. package/assets/agora/references/cli/README.md +161 -0
  5. package/assets/agora/references/cli/automation.md +189 -0
  6. package/assets/agora/references/cli/doctor.md +129 -0
  7. package/assets/agora/references/cli/env.md +158 -0
  8. package/assets/agora/references/cli/install-auth.md +152 -0
  9. package/assets/agora/references/cli/projects.md +116 -0
  10. package/assets/agora/references/cli/quickstarts.md +117 -0
  11. package/assets/agora/references/cloud-recording/README.md +86 -0
  12. package/assets/agora/references/conversational-ai/README.md +285 -0
  13. package/assets/agora/references/conversational-ai/agent-client-toolkit-react.md +182 -0
  14. package/assets/agora/references/conversational-ai/agent-samples.md +101 -0
  15. package/assets/agora/references/conversational-ai/agent-toolkit-android.md +209 -0
  16. package/assets/agora/references/conversational-ai/agent-toolkit-ios.md +208 -0
  17. package/assets/agora/references/conversational-ai/agent-toolkit.md +201 -0
  18. package/assets/agora/references/conversational-ai/agent-ui-kit.md +63 -0
  19. package/assets/agora/references/conversational-ai/architecture.md +221 -0
  20. package/assets/agora/references/conversational-ai/auth-flow.md +154 -0
  21. package/assets/agora/references/conversational-ai/conversational-ai-studio.md +173 -0
  22. package/assets/agora/references/conversational-ai/go-sdk.md +184 -0
  23. package/assets/agora/references/conversational-ai/integration-from-quickstart.md +203 -0
  24. package/assets/agora/references/conversational-ai/python-sdk.md +122 -0
  25. package/assets/agora/references/conversational-ai/quickstarts.md +710 -0
  26. package/assets/agora/references/conversational-ai/server-custom-llm.md +45 -0
  27. package/assets/agora/references/conversational-ai/server-mcp.md +40 -0
  28. package/assets/agora/references/conversational-ai/server-sdk-rename.md +78 -0
  29. package/assets/agora/references/conversational-ai/server-sdks.md +128 -0
  30. package/assets/agora/references/doc-fetching.md +67 -0
  31. package/assets/agora/references/integration-patterns.md +201 -0
  32. package/assets/agora/references/mcp-tools.md +49 -0
  33. package/assets/agora/references/rtc/README.md +104 -0
  34. package/assets/agora/references/rtc/android.md +344 -0
  35. package/assets/agora/references/rtc/cross-platform-coordination.md +61 -0
  36. package/assets/agora/references/rtc/flutter.md +282 -0
  37. package/assets/agora/references/rtc/ios.md +306 -0
  38. package/assets/agora/references/rtc/nextjs.md +87 -0
  39. package/assets/agora/references/rtc/react-native.md +266 -0
  40. package/assets/agora/references/rtc/react.md +186 -0
  41. package/assets/agora/references/rtc/web.md +506 -0
  42. package/assets/agora/references/rtm/README.md +80 -0
  43. package/assets/agora/references/rtm/android.md +277 -0
  44. package/assets/agora/references/rtm/ios.md +231 -0
  45. package/assets/agora/references/rtm/web.md +348 -0
  46. package/assets/agora/references/server/README.md +22 -0
  47. package/assets/agora/references/server/tokens.md +74 -0
  48. package/assets/agora/references/server-gateway/README.md +80 -0
  49. package/assets/agora/references/server-gateway/linux-cpp.md +251 -0
  50. package/assets/agora/references/testing-guidance/SKILL.md +65 -0
  51. package/assets/agora/references/testing-guidance/completeness-gate.md +28 -0
  52. package/assets/agora/references/testing-guidance/convoai-rest.md +83 -0
  53. package/assets/agora/references/testing-guidance/mobile-rtm-and-renewal.md +109 -0
  54. package/assets/agora/references/testing-guidance/rtc-android.md +70 -0
  55. package/assets/agora/references/testing-guidance/rtc-ios.md +73 -0
  56. package/assets/agora/references/testing-guidance/rtc-react.md +51 -0
  57. package/assets/agora/references/testing-guidance/rtc-web.md +94 -0
  58. package/cordis.patch.yml +5 -0
  59. package/index.js +53 -0
  60. package/package.json +49 -0
@@ -0,0 +1,45 @@
1
+ # Custom LLM Server
2
+
3
+ OpenAI-compatible LLM proxy for Agora Conversational AI. Intercepts LLM requests for RAG, tool calling, and conversation memory.
4
+
5
+ **Repo:** <https://github.com/AgoraIO-Conversational-AI/server-custom-llm>
6
+
7
+ ## Implementations
8
+
9
+ | Language | Framework | Port | Notes |
10
+ |----------|-----------|------|-------|
11
+ | Python | FastAPI | 8100 | |
12
+ | Node.js | Express | 8101 | Advanced — exclusive features below |
13
+ | Go | Gin | 8102 | |
14
+
15
+ ## Endpoints
16
+
17
+ - `/chat/completions` — LLM proxy with server-side tool execution (up to 5 passes)
18
+ - `/rag/chat/completions` — RAG-enhanced with retrieval step
19
+ - `/audio/chat/completions` — Multimodal audio responses
20
+
21
+ > **[README — Endpoints](https://github.com/AgoraIO-Conversational-AI/server-custom-llm#endpoints)**
22
+
23
+ ## Integration with ConvoAI
24
+
25
+ Set `LLM_URL` to your server endpoint, `LLM_VENDOR=custom` in agent-samples `.env`.
26
+
27
+ > **[README — Integration](https://github.com/AgoraIO-Conversational-AI/server-custom-llm#integration-with-agora-conversational-ai)**
28
+
29
+ ## Features
30
+
31
+ - Server-side tool execution (get_weather, calculate examples)
32
+ - Conversation memory (per appId:userId:channel, 100 message limit, 24h cleanup)
33
+ - RTM integration for text messaging (Node.js)
34
+ - Streaming SSE responses
35
+
36
+ > **[README — Features](https://github.com/AgoraIO-Conversational-AI/server-custom-llm#features)**
37
+
38
+ ## Node.js-Exclusive Features
39
+
40
+ - `/register-agent` and `/unregister-agent` endpoints — agent lifecycle hooks
41
+ - Pluggable module system: `init()`, `getToolDefinitions()`, `onRequest`/`onResponse` hooks
42
+ - Dynamic RTM initialization via request headers (`X-Agora-Customllm-*`)
43
+ - Go audio subscriber for RTC audio capture (spawned as child process)
44
+
45
+ > **[node/integrations/README.md](https://github.com/AgoraIO-Conversational-AI/server-custom-llm/blob/main/node/integrations/README.md)** — Module system and integrations
@@ -0,0 +1,40 @@
1
+ # MCP Memory Server
2
+
3
+ Multi-user MCP server with persistent memory and full-text search for Agora Conversational AI agents.
4
+
5
+ **Repo:** <https://github.com/AgoraIO-Conversational-AI/server-mcp-memory>
6
+
7
+ ## Implementations
8
+
9
+ All 3 implementations are production-ready with feature parity. Test suite: `test/run_all.sh` (comprehensive happy + failure path tests).
10
+
11
+ | Language | Framework | Port | Notes |
12
+ |----------|-----------|------|-------|
13
+ | Python | Starlette | 8090 | |
14
+ | Node.js | Express | 8091 | |
15
+ | Go | Gin | 8092 | Requires `CGO_ENABLED=1` build flag (SQLite) |
16
+
17
+ ## MCP Tools
18
+
19
+ - `save_memory` — store a memory with category and tags
20
+ - `search_memory` — BM25 full-text search
21
+ - `list_memories` — list by category
22
+ - `delete_memory` — delete by ID
23
+ - `compact_memories` — merge related memories
24
+ - `log_message` — append to conversation log
25
+
26
+ > **[README — Tools](https://github.com/AgoraIO-Conversational-AI/server-mcp-memory#mcp-tools)**
27
+
28
+ ## Integration with ConvoAI
29
+
30
+ Configure `MCP_SERVERS` JSON array in agent-samples `.env`. Uses `build_mcp_servers` function.
31
+
32
+ > **[README — Integration](https://github.com/AgoraIO-Conversational-AI/server-mcp-memory#integration-with-agora-conversational-ai)**
33
+
34
+ ## Architecture
35
+
36
+ - SQLite with FTS5 full-text search
37
+ - Per-user memory isolation via URL path (`/mcp/{user_id}`)
38
+ - MCP Streamable HTTP protocol
39
+
40
+ > **[README — Architecture](https://github.com/AgoraIO-Conversational-AI/server-mcp-memory#architecture)**
@@ -0,0 +1,78 @@
1
+ ---
2
+ name: agora-server-sdk-rename
3
+ description: |
4
+ Opt-in only. Load when the user's project uses outdated ConvoAI server SDK package or
5
+ module names and they need to migrate to current names. Do not load for greenfield work.
6
+ Triggers on: agora-agent-server-sdk, agora-agent (PyPI), agora-agent-server-sdk-go,
7
+ AgoraIO-Community/agora-agent-server-sdk-go, migrate agora sdk, upgrade server sdk package.
8
+ license: MIT
9
+ metadata:
10
+ author: agora
11
+ version: '1.0.0'
12
+ ---
13
+
14
+ # ConvoAI Server SDK — package rename migration
15
+
16
+ > **Opt-in only.** Load this file when workspace detection or the user shows outdated server SDK names. Default ConvoAI routing uses [server-sdks.md](server-sdks.md), [python-sdk.md](python-sdk.md), and [go-sdk.md](go-sdk.md) with current names.
17
+
18
+ Apply the renames below in dependency manifests and import statements. The public API is unchanged — only install paths and import strings move.
19
+
20
+ ## Detection signals
21
+
22
+ Scan the user's project (read-only) for any of:
23
+
24
+ | Signal | Where to look |
25
+ |--------|----------------|
26
+ | `agora-agent-server-sdk` | `package.json`, lockfiles, `import` / `require` strings |
27
+ | `agora-agent` (not `agora-agents`) | `pyproject.toml`, `requirements.txt`, `Pipfile`, `poetry.lock` |
28
+ | `github.com/AgoraIO-Community/agora-agent-server-sdk-go` | `go.mod`, Go import paths |
29
+ | `agent-server-sdk-go` repo URL | README, CI config, docs |
30
+
31
+ If none match, do not load this file.
32
+
33
+ ## Rename map
34
+
35
+ ### TypeScript / npm
36
+
37
+ | Update | From | To |
38
+ |--------|------|-----|
39
+ | Dependency | `agora-agent-server-sdk` | `agora-agents` |
40
+ | Import | `from 'agora-agent-server-sdk'` | `from 'agora-agents'` |
41
+
42
+ The npm package `agora-agent-server-sdk` still resolves via a compat shim, but new projects and migrations should use `agora-agents`.
43
+
44
+ ### Python / PyPI
45
+
46
+ | Update | From | To |
47
+ |--------|------|-----|
48
+ | Dependency | `agora-agent` or `agora-agent-server-sdk` | `agora-agents` |
49
+ | Import | `from agora_agent import ...` | **No change** |
50
+
51
+ Only the PyPI install name changes. Python import paths stay `agora_agent`.
52
+
53
+ ### Go
54
+
55
+ | Update | From | To |
56
+ |--------|------|-----|
57
+ | Module | `github.com/AgoraIO-Community/agora-agent-server-sdk-go` | `github.com/AgoraIO/agora-agents-go` |
58
+ | Imports | `.../agora-agent-server-sdk-go/...` | `github.com/AgoraIO/agora-agents-go/...` |
59
+ | Repo | `AgoraIO-Conversational-AI/agent-server-sdk-go` | `AgoraIO/agora-agents-go` |
60
+
61
+ Go has no module alias — update `go.mod` and all import paths together, then `go mod tidy`.
62
+
63
+ ## Migration steps
64
+
65
+ 1. Update the manifest (`package.json`, `pyproject.toml` / `requirements.txt`, or `go.mod`).
66
+ 2. Replace import strings in source files per the table above.
67
+ 3. Reinstall / tidy (`npm install`, `pip install -r ...`, or `go mod tidy`).
68
+ 4. Run the project's test suite or a minimal start/stop agent smoke test.
69
+
70
+ Do not change application logic, env var names, or AgentKit builder code — the API surface is the same.
71
+
72
+ ## After migration
73
+
74
+ Route back to the current SDK references for ongoing work:
75
+
76
+ - TypeScript: [server-sdks.md](server-sdks.md)
77
+ - Python: [python-sdk.md](python-sdk.md)
78
+ - Go: [go-sdk.md](go-sdk.md)
@@ -0,0 +1,128 @@
1
+ ---
2
+ name: agora-convoai-server-sdks
3
+ description: |
4
+ Server-side SDKs for Agora Conversational AI: TypeScript, Python, and Go wrappers around the
5
+ ConvoAI REST API. Use when the user is building a backend to start/stop/manage ConvoAI agents.
6
+ Triggers on: agora-agents, AgoraClient, AgentSession, session.start, session.stop,
7
+ agent server SDK, ConvoAI backend, ConvoAI server, withStt, withLlm, withTts.
8
+ license: MIT
9
+ metadata:
10
+ author: agora
11
+ version: '1.0.0'
12
+ ---
13
+
14
+ # ConvoAI Server SDKs
15
+
16
+ > **Architecture first:** For call sequence and lifecycle overview before SDK details, read [architecture.md](architecture.md).
17
+
18
+ TypeScript, Go, and Python SDKs — convenience wrappers around the ConvoAI REST API. For any other backend language, call the REST API directly. Fetch the live OpenAPI spec for the full schema: `https://docs-md.agora.io/api/conversational-ai-api-v2.x.yaml`
19
+
20
+ ## TypeScript — `agora-agents`
21
+
22
+ ```bash
23
+ npm install agora-agents
24
+ ```
25
+
26
+ Builder pattern — configure the AI pipeline then create sessions:
27
+
28
+ ```typescript
29
+ import { AgoraClient, Agent, Area } from 'agora-agents';
30
+
31
+ const client = new AgoraClient({
32
+ area: Area.US,
33
+ appId: process.env.AGORA_APP_ID,
34
+ appCertificate: process.env.AGORA_APP_CERTIFICATE,
35
+ });
36
+
37
+ const agent = new Agent({
38
+ name: `agent_${crypto.randomUUID().slice(0, 8)}`, // must be unique per project
39
+ instructions: 'You are a helpful voice assistant.',
40
+ greeting: 'Hello! How can I help you today?',
41
+ })
42
+ .withStt(new DeepgramSTT({ apiKey: process.env.DEEPGRAM_API_KEY }))
43
+ .withLlm(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }))
44
+ .withTts(new ElevenLabsTTS({ apiKey: process.env.ELEVENLABS_API_KEY }));
45
+
46
+ // Start a session (joins the agent to a channel)
47
+ const session = agent.createSession({ channel: 'my-channel', agentUid: 0 });
48
+ const sessionId = await session.start();
49
+
50
+ // Stop from the same process
51
+ await session.stop();
52
+
53
+ // Stop from a stateless server (e.g. a different request handler)
54
+ await client.stopAgent(sessionId);
55
+ ```
56
+
57
+ Token auth is handled automatically when `appCertificate` is provided. For vendor-specific STT/LLM/TTS import paths and MLLM (OpenAI Realtime, Gemini Live) config, see the [SDK README](https://github.com/AgoraIO-Conversational-AI/agent-server-sdk-ts).
58
+
59
+ ## Auth Modes
60
+
61
+ Three modes, in order of recommendation:
62
+
63
+ - **App Credentials** (`appId` + `appCertificate`): SDK generates a fresh ConvoAI token per REST call. No token management needed. The App Certificate never leaves your server. Recommended for production.
64
+ - **Token Auth** (`authToken`): A pre-built combined RTC+RTM token you supply. This token is **reused for every request** until you replace the SDK instance — the SDK does not refresh it. You are responsible for refreshing before expiry. Max token validity is 24 hours.
65
+ - **Basic Auth** (`customerId` + `customerSecret`): Credentials never expire but are long-lived secrets that grant access to every project on your account. Use for local testing only; do not ship to production.
66
+
67
+ ## Session State Machine
68
+
69
+ Sessions follow a strict state sequence. Calling methods outside the valid states throws an error:
70
+
71
+ | State | `start()` | `stop()` | `say()` / `interrupt()` / `update()` |
72
+ |-------|-----------|----------|---------------------------------------|
73
+ | `idle` | ✅ | ❌ | ❌ |
74
+ | `starting` | ❌ | ❌ | ❌ |
75
+ | `running` | ❌ | ✅ | ✅ |
76
+ | `stopping` | ❌ | ❌ | ❌ |
77
+ | `stopped` | ✅ | ❌ | ❌ |
78
+ | `error` | ✅ | ❌ | ❌ |
79
+
80
+ `stop()` on a 404 (agent already stopped on the platform) resolves without throwing — the SDK treats it as already stopped.
81
+
82
+ ## In-Process Events (TypeScript)
83
+
84
+ `AgentSession` emits events within your Node.js process — these are not HTTP webhooks:
85
+
86
+ ```typescript
87
+ session.on('started', () => {
88
+ // Agent has connected to the RTC channel and is ready
89
+ // session.id is now set; session.status === 'running'
90
+ });
91
+
92
+ session.on('stopped', () => {
93
+ // Agent has left the channel
94
+ });
95
+
96
+ session.on('error', (err) => {
97
+ // Non-recoverable error; session.status === 'error'
98
+ // Safe to call session.start() again from this state
99
+ });
100
+ ```
101
+
102
+ ## Avatar + TTS Sample Rate
103
+
104
+ HeyGen and Akool avatars require a specific TTS sample rate. The SDK validates this at `session.start()` and throws if mismatched. The error message identifies the avatar config as the problem — the root cause is actually the TTS sample rate.
105
+
106
+ | Avatar vendor | Required TTS sample rate |
107
+ |--------------|--------------------------|
108
+ | HeyGen | **24000 Hz** |
109
+ | Akool | **16000 Hz** |
110
+
111
+ Whenever an avatar vendor is set, explicitly configure the TTS sample rate to match. Do not rely on defaults.
112
+
113
+ ```typescript
114
+ // HeyGen — must pair with 24 kHz TTS
115
+ const agent = new Agent({ ... })
116
+ .withAvatar(new HeyGen({ ... }))
117
+ .withTts(new ElevenLabsTTS({ sampleRate: 24000, ... }));
118
+
119
+ // Akool — must pair with 16 kHz TTS
120
+ const agent = new Agent({ ... })
121
+ .withAvatar(new Akool({ ... }))
122
+ .withTts(new ElevenLabsTTS({ sampleRate: 16000, ... }));
123
+ ```
124
+
125
+ ## Python and Go SDKs
126
+
127
+ - **[python-sdk.md](python-sdk.md)** — Python SDK: sync vs async, deprecation warnings, debug logging.
128
+ - **[go-sdk.md](go-sdk.md)** — Go SDK: context.Context pattern, builder syntax, SessionStatus constants, token helpers.
@@ -0,0 +1,67 @@
1
+ # Documentation Lookup
2
+
3
+ **Level 1 — Bundled references (always try first)**
4
+
5
+ Check the relevant file under `skills/agora/references/`. These are inline-stable:
6
+ RTC init patterns, RTM messaging, token generation, ConvoAI gotchas and generation
7
+ rules. If the answer is here, stop — no fetch needed.
8
+
9
+ **Level 2 — Live docs (when Level 1 is insufficient)**
10
+
11
+ When bundled references don't cover the detail needed (full request/response schemas,
12
+ vendor-specific configs, language-specific quick-start code):
13
+
14
+ If the Agora Docs MCP tool (`agora-docs-mcp`) is available in the current tool/runtime, prefer it for Level 2 documentation lookup. Otherwise use the HTTP fetch flow below. If MCP returns no useful result, fall back to HTTP fetch.
15
+
16
+ 1. Fetch the Agora docs sitemap:
17
+ ```
18
+ GET https://docs.agora.io/en/llms.txt
19
+ ```
20
+ 2. Scan the response for a URL matching the product and topic.
21
+ 3. Fetch that URL and use its content to answer.
22
+
23
+ ## Fallback
24
+
25
+ If `llms.txt` is unreachable or the fetched URL returns no useful content, try these
26
+ known markdown entry points directly:
27
+
28
+ | Product / Language | Markdown URL |
29
+ |---|---|
30
+ | RTC (Web/general) | https://docs-md.agora.io/en/video-calling/get-started/get-started-sdk.md |
31
+ | RTC (voice-only) | https://docs-md.agora.io/en/voice-calling/get-started/get-started-sdk.md |
32
+ | RTM (Web/general) | https://docs-md.agora.io/en/signaling/get-started/sdk-quickstart.md |
33
+ | RTM (iOS) | https://docs-md.agora.io/en/signaling/get-started/sdk-quickstart?platform=ios.md |
34
+ | RTM (Android) | https://docs-md.agora.io/en/signaling/get-started/sdk-quickstart?platform=android.md |
35
+ | ConvoAI | https://docs-md.agora.io/en/conversational-ai/get-started/quickstart.md |
36
+ | ConvoAI (TypeScript SDK) | https://docs-md.agora.io/en/conversational-ai/develop/integrate-sdk.md |
37
+ | ConvoAI (Python SDK) | https://docs-md.agora.io/en/conversational-ai/develop/integrate-sdk?platform=python.md |
38
+ | Cloud Recording | https://docs-md.agora.io/en/cloud-recording/get-started/getstarted.md |
39
+ | Server Gateway (C++) | https://docs-md.agora.io/en/server-gateway/get-started/integrate-sdk.md |
40
+ | Server Gateway (Java) | https://docs-md.agora.io/en/server-gateway/get-started/integrate-sdk?platform=java.md |
41
+ | Server Gateway (Python) | https://docs-md.agora.io/en/server-gateway/get-started/integrate-sdk?platform=python.md |
42
+ | Server Gateway (Go) | https://docs-md.agora.io/en/server-gateway/get-started/integrate-sdk?platform=go.md |
43
+ | Tokens | https://docs-md.agora.io/en/video-calling/token-authentication/deploy-token-server.md |
44
+
45
+ ## Agora Docs MCP Server (optional)
46
+
47
+ Agora also provides `agora-docs-mcp`, an MCP server that gives AI assistants
48
+ direct tool-call access to documentation — an alternative to the Level 2 HTTP
49
+ fetch above. It traverses docs only; it does not use Agora backend APIs, log in,
50
+ create projects, or inspect account state. If a user asks about installing or
51
+ using the Agora Docs MCP server, see [mcp-tools.md](mcp-tools.md).
52
+
53
+ ## Agora CLI
54
+
55
+ For local `agora` command-line usage, check [cli/README.md](cli/README.md) first. Treat that file as the source of truth for the currently verified CLI baseline and command surface.
56
+
57
+ If the bundled CLI references do not cover a CLI-only detail, use the canonical CLI repository instead of general Agora product docs:
58
+
59
+ - <https://github.com/AgoraIO/cli>
60
+ - `README.md` — install, first run, command model, common workflows
61
+ - `docs/commands.md` — generated command reference
62
+ - `docs/automation.md` — JSON envelopes and automation contract
63
+ - `docs/error-codes.md` — stable error codes and recovery decisions
64
+ - `docs/telemetry.md` — telemetry preferences and environment controls
65
+ - `CHANGELOG.md` and GitHub Releases — version-specific changes
66
+
67
+ Do not invent CLI commands, flags, environment variables, or JSON fields. If the installed CLI exposes a command that is not in the bundled references, prefer `agora introspect --json` or `agora --help --all`, then cross-check the CLI repository docs before answering.
@@ -0,0 +1,201 @@
1
+ # Multi-Product Integration Patterns
2
+
3
+ This file covers how RTC, RTM, and Conversational AI work together. For SDK-specific code, follow the Level 2 fetch links in each product's reference file.
4
+
5
+ ## When to Use Multiple Products
6
+
7
+ | Scenario | Products |
8
+ |---|---|
9
+ | Video/voice call only | RTC only |
10
+ | Video call + chat or presence | RTC + RTM |
11
+ | AI agent in a call | RTC + ConvoAI REST API (server-side) |
12
+ | AI agent + live transcripts or state | RTC + RTM + ConvoAI REST API |
13
+ | Text-only signaling (no media) | RTM only |
14
+
15
+ ## RTC + RTM Together
16
+
17
+ ### Initialization Order
18
+
19
+ Always initialize both SDKs independently — they share no initialization path.
20
+
21
+ RTC UID and RTM user ID do **not** need to match. Using a shared identity is a coordination convention, not a platform requirement. The right initialization order depends on which identity strategy your app uses.
22
+
23
+ #### Default pattern — auto-assigned RTC UID
24
+
25
+ Use this when the RTC client joins with `uid: 0` / `null` and the SDK assigns the final RTC UID:
26
+
27
+ 1. Create and initialize the RTC engine/client
28
+ 2. Join the RTC channel
29
+ 3. Read the assigned RTC UID from the join callback / resolved promise
30
+ 4. Create or log in to the RTM client with `String(rtcUid)`
31
+ 5. Subscribe to the RTM channel
32
+
33
+ This is the safest default because it avoids guessing the RTM identity before the RTC UID is known.
34
+
35
+ #### Advanced pattern — app-level stable identity
36
+
37
+ Use this when your app already has a stable user identity independent of the RTC UID:
38
+
39
+ 1. Create and initialize the RTC engine/client
40
+ 2. Create and log in to the RTM client with your app-level user identity
41
+ 3. Subscribe to the RTM channel
42
+ 4. Join the RTC channel
43
+
44
+ This pattern is useful when you want signaling, presence, or pre-call state to arrive before media starts. It requires an explicit RTC↔RTM mapping in your app or backend.
45
+
46
+ ### UID Strategy
47
+
48
+ RTC and RTM use different UID types:
49
+
50
+ | SDK | Platform | UID type |
51
+ |---|---|---|
52
+ | RTC | Web | `number` |
53
+ | RTC | iOS | `UInt` (unsigned 32-bit) |
54
+ | RTC | Android | `Int` (signed 32-bit) |
55
+ | RTM | All platforms | `String` |
56
+
57
+ Recommended default: use `String(rtcUid)` as the RTM `userId` when your RTC UID is auto-assigned and you want a simple 1:1 mapping between the two systems.
58
+
59
+ Alternative: use a stable app-level string identity for RTM and maintain an explicit mapping from RTC UID → RTM user ID in your app/backend.
60
+
61
+ The RTM token subject and RTM login `userId` must match exactly. If your server minted the RTM token for `String(rtcUid)`, the client must log in RTM with that same string — not a different random or app-generated identity.
62
+
63
+ UIDs greater than 2,147,483,647 wrap to negative on Android RTC — avoid them if Android clients are present.
64
+
65
+ ### Channel Name Convention
66
+
67
+ Use the **same channel name** for both RTC and RTM. The RTM channel is the coordination layer for the RTC session — same name makes routing unambiguous and is required by the ConvoAI toolkit.
68
+
69
+ ### Token Matrix
70
+
71
+ Both SDKs require separate tokens in production. Generate them independently on your server:
72
+
73
+ | Token | Scope | Renew on |
74
+ |---|---|---|
75
+ | RTC token | `appId` + `channelName` + `uid` | `token-privilege-will-expire` / `onTokenPrivilegeWillExpire` |
76
+ | RTM token | `appId` + `userId` | RTM connection state `TOKEN_EXPIRED` or equivalent |
77
+
78
+ See [server/tokens.md](server/tokens.md) for token generation details.
79
+
80
+ For Level 2 fetch: fetch `https://docs.agora.io/en/llms.txt`, find the token management guide for your platform, then fetch it.
81
+
82
+ ## RTC + RTM + ConvoAI
83
+
84
+ Before wiring ConvoAI into an existing multi-product app, read [conversational-ai/architecture.md](conversational-ai/architecture.md) for the end-to-end call sequence, then use the official quickstart as the source of truth for code shape and commands. For existing-app work, use [conversational-ai/integration-from-quickstart.md](conversational-ai/integration-from-quickstart.md) after the quickstart source is inspected; runtime proof is required before claiming the integrated flow works.
85
+
86
+ ### How the Products Connect
87
+
88
+ ```
89
+ Client Server
90
+ ──────────────────────────────────────────────────────
91
+ RTC Engine/Client ──── media ────► Agora RTC
92
+ RTM Client ────── messages ──────► Agora RTM
93
+
94
+ [ConvoAI agent joins
95
+ same RTC channel,
96
+ publishes transcripts
97
+ and state to same
98
+ RTM channel]
99
+
100
+ App Server ── REST POST /join ──────► Agora ConvoAI API
101
+ ```
102
+
103
+ ### Initialization Order for ConvoAI
104
+
105
+ 1. Initialize RTC (but do not join the channel yet)
106
+ 2. Initialize RTM and log in
107
+ 3. Subscribe to the RTM channel (to receive agent events)
108
+ 4. Call ConvoAI REST `POST /join` from your **app server** with `channelName`, `uid` (your user's RTC uid), and the RTM flags (see below)
109
+ 5. Join the RTC channel — the agent will already be there or will join shortly
110
+
111
+ If your client uses auto-assigned RTC UIDs, adapt the order slightly:
112
+
113
+ 1. Initialize RTC
114
+ 2. Join RTC and wait for the assigned user UID
115
+ 3. Log in to RTM with `String(rtcUid)` and subscribe to the RTM channel
116
+ 4. Call `POST /join` from your app server
117
+
118
+ The agent can publish transcripts/state through RTM only after the RTM client is logged in and subscribed.
119
+
120
+ If RTM was just enabled on the project, allow bounded wait/retry before treating an RTM startup failure as a persistent project misconfiguration. Control-plane state can update before the RTM runtime path is actually usable.
121
+
122
+ ### Required ConvoAI Flags for RTM Delivery
123
+
124
+ When calling `POST /join`, include both of these in the request body:
125
+
126
+ ```json
127
+ {
128
+ "advanced_features": {
129
+ "enable_rtm": true
130
+ },
131
+ "parameters": {
132
+ "data_channel": "rtm"
133
+ }
134
+ }
135
+ ```
136
+
137
+ Both flags are required. Omitting either one means the agent publishes transcripts via RTC data channel instead of RTM, and RTM message events will not fire.
138
+
139
+ ### RTM Channel Name = RTC Channel Name
140
+
141
+ The ConvoAI agent joins the RTM channel with the **same name** as the RTC channel it joined. When handling `onMessageEvent` / `didReceiveMessageEvent`, filter by `channelName` matching your RTC channel if you are subscribed to multiple channels.
142
+
143
+ ### Agent Events via RTM
144
+
145
+ The agent publishes JSON messages to the RTM channel. Parse `event.message.data` (Android) or `event.message.stringData` (iOS) as JSON. The `customType` field indicates the event type (transcript, interruption, state change, etc.).
146
+
147
+ For the full event schema and `customType` values, use Level 2 fetch: fetch `https://docs.agora.io/en/llms.txt`, locate the Conversational AI event reference, then fetch it.
148
+
149
+ ### Token Matrix for ConvoAI
150
+
151
+ ConvoAI sessions require three separate tokens:
152
+
153
+ | Token | Purpose | Who generates |
154
+ |---|---|---|
155
+ | RTC token (user) | User joins RTC channel | Your app server |
156
+ | RTM token (user) | User logs into RTM | Your app server |
157
+ | RTC token (agent) | ConvoAI agent joins RTC | Your app server, passed in POST /join |
158
+
159
+ See [conversational-ai/auth-flow.md](conversational-ai/auth-flow.md) for the full three-token flow (REST API implementors) or [conversational-ai/README.md](conversational-ai/README.md) for SDK-based flows.
160
+
161
+ ## Codec Selection for Mixed Platforms
162
+
163
+ When Web clients share a channel with iOS or Android native clients:
164
+
165
+ - Web defaults to `codec: 'vp8'` — this scales well for multi-user calls and is the recommended choice
166
+ - `'vp9'` is also a good option with better compression
167
+ - `'h264'` does not scale well beyond small groups — avoid it for multi-user scenarios
168
+ - If codecs differ between Web and native clients, Agora transcodes server-side — this works but adds latency and is billed separately
169
+
170
+ See [rtc/cross-platform-coordination.md](rtc/cross-platform-coordination.md) for full cross-platform interop notes.
171
+
172
+ ## Cleanup Order
173
+
174
+ Reverse of initialization:
175
+
176
+ 1. Leave the RTC channel
177
+ 2. Unsubscribe from RTM channels
178
+ 3. Log out of RTM
179
+ 4. Release both engines/clients
180
+
181
+ For ConvoAI: call `DELETE /leave` (or the SDK equivalent) from your app server before leaving the RTC channel, to give the agent time to exit gracefully.
182
+
183
+ ## Reference Files by Product
184
+
185
+ | Product | Platform | File |
186
+ |---|---|---|
187
+ | RTC | Web | [rtc/web.md](rtc/web.md) |
188
+ | RTC | React | [rtc/react.md](rtc/react.md) |
189
+ | RTC | Next.js | [rtc/nextjs.md](rtc/nextjs.md) |
190
+ | RTC | iOS | [rtc/ios.md](rtc/ios.md) |
191
+ | RTC | Android | [rtc/android.md](rtc/android.md) |
192
+ | RTC | React Native | [rtc/react-native.md](rtc/react-native.md) |
193
+ | RTC | Flutter | [rtc/flutter.md](rtc/flutter.md) |
194
+ | RTM | Web | [rtm/web.md](rtm/web.md) |
195
+ | RTM | iOS | [rtm/ios.md](rtm/ios.md) |
196
+ | RTM | Android | [rtm/android.md](rtm/android.md) |
197
+ | ConvoAI | All | [conversational-ai/README.md](conversational-ai/README.md) |
198
+ | ConvoAI toolkit | iOS | [conversational-ai/agent-toolkit-ios.md](conversational-ai/agent-toolkit-ios.md) |
199
+ | ConvoAI toolkit | Android | [conversational-ai/agent-toolkit-android.md](conversational-ai/agent-toolkit-android.md) |
200
+ | Tokens | Server | [server/tokens.md](server/tokens.md) |
201
+ | Cross-platform | All | [rtc/cross-platform-coordination.md](rtc/cross-platform-coordination.md) |
@@ -0,0 +1,49 @@
1
+ # Agora Docs MCP Server
2
+
3
+ <!-- applies-from: v0.2.0 -->
4
+
5
+ The Agora Docs MCP server (`agora-docs-mcp`) gives AI assistants direct tool-call access to Agora documentation. It is an
6
+ optional enhancement — the skill works without it using the two-tier fetch
7
+ approach in [doc-fetching.md](doc-fetching.md).
8
+
9
+ Agora Docs MCP is for traversing docs only. It does not operate Agora backends,
10
+ log in to accounts, create projects, write env files, or check project readiness;
11
+ the local `agora` CLI handles those workflows.
12
+
13
+ **Only use MCP when the user explicitly asks for it.** The default documentation
14
+ lookup is the two-tier fetch approach in [doc-fetching.md](doc-fetching.md) — use
15
+ that regardless of whether MCP is installed.
16
+
17
+ **MCP endpoint:** `https://mcp.agora.io`
18
+
19
+ ## Tools
20
+
21
+ | Tool | Input | Returns |
22
+ |---|---|---|
23
+ | `get-doc-content` | `{"uri": "docs://..."}` | Full markdown content |
24
+ | `search-docs` | `{"query": "keyword"}` | List of matching doc URIs |
25
+ | `list-docs` | `{"category": "...", "limit": 20}` | All docs in a category |
26
+
27
+ Use `search-docs` when the topic is known but the URI isn't. Use `get-doc-content`
28
+ directly when the URI is known.
29
+
30
+ ## Installation
31
+
32
+ **Claude Code:**
33
+ ```bash
34
+ claude mcp add agora-docs-mcp --transport http https://mcp.agora.io
35
+ ```
36
+
37
+ **Cursor / Windsurf / other MCP-compatible tools:** Add `https://mcp.agora.io` as
38
+ an HTTP MCP server in your tool's MCP settings. See your tool's documentation for
39
+ the exact configuration format.
40
+
41
+ For the latest setup instructions and any changes to the endpoint, see:
42
+ <https://docs.agora.io/en/mcp>
43
+
44
+ ## Usage Note
45
+
46
+ After fetching quick-start docs via MCP, use the content for API structure and field
47
+ names only. Do NOT copy sample code verbatim — quick-start examples typically hardcode
48
+ credentials and omit production requirements. Apply the gotchas and generation rules
49
+ in `references/conversational-ai/README.md` to any generated code.