@andreprado/agentkit 0.1.0-alpha.15 → 0.1.0-alpha.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -0
- package/docs/guides/add-channel.md +65 -0
- package/docs/guides/add-managed-composio.md +137 -0
- package/docs/guides/channel-security.md +32 -0
- package/docs/guides/channels-production-handoff.md +1 -1
- package/docs/guides/connect-telegram.md +60 -0
- package/docs/guides/connect-whatsapp-zapster.md +65 -0
- package/docs/guides/prepare-deploy.md +3 -1
- package/docs/llms-full.txt +13 -1
- package/docs/llms.txt +7 -0
- package/package.json +1 -1
- package/src/cli/commands/channels.ts +121 -7
- package/src/cli/commands/transcribe.ts +171 -0
- package/src/cli/deploy-readiness.ts +59 -3
- package/src/cli/help.ts +15 -0
- package/src/cli/index.ts +130 -0
- package/src/create-project.ts +4 -32
- package/src/index.ts +231 -0
- package/src/runtime/channel-test-harness.ts +2 -0
- package/src/runtime/channels/telegram.ts +326 -10
- package/src/runtime/channels/whatsapp-zapster.ts +319 -0
- package/src/runtime/channels.ts +47 -1
- package/src/runtime/chat.ts +2 -1
- package/src/runtime/config.ts +214 -4
- package/src/runtime/core/manifest.ts +61 -2
- package/src/runtime/deploy-readiness.ts +31 -1
- package/src/runtime/dev-server.ts +72 -4
- package/src/runtime/inspect.ts +142 -4
- package/src/runtime/integrations/composio.ts +257 -0
- package/src/runtime/skills.ts +95 -0
- package/src/runtime/targets/cloudflare/build.ts +162 -5
- package/src/runtime/tool-runner.ts +2 -1
- package/src/runtime/transcription.ts +483 -0
- package/src/templates/skills/agentkit-capsule/SKILL.md +1 -0
- package/src/templates/skills/agentkit-channels/SKILL.md +36 -1
- package/src/templates/skills/agentkit-channels/references/channel-debugging.md +16 -1
- package/src/templates/skills/agentkit-channels/references/telegram.md +34 -0
- package/src/templates/skills/agentkit-channels/references/whatsapp-zapster.md +29 -0
- package/src/templates/skills/agentkit-deploy/SKILL.md +1 -1
- package/src/templates/skills/agentkit-integrations/SKILL.md +66 -0
- package/src/templates/skills/agentkit-troubleshooting/SKILL.md +3 -2
package/README.md
CHANGED
|
@@ -37,6 +37,8 @@ agentkit tool <name> [--input <path-or-json>]
|
|
|
37
37
|
agentkit knowledge add <path-or-url>
|
|
38
38
|
agentkit knowledge sync
|
|
39
39
|
agentkit knowledge search <query> [--top-k <number>]
|
|
40
|
+
agentkit integrations status
|
|
41
|
+
agentkit integrations connect composio --toolkit gmail
|
|
40
42
|
agentkit spec init --brief <text>
|
|
41
43
|
agentkit db migrate
|
|
42
44
|
agentkit sync run
|
|
@@ -50,18 +52,24 @@ Knowledge indexes local `.md`, `.txt`, and `.csv` sources into the capsule datab
|
|
|
50
52
|
|
|
51
53
|
Every chat run receives dynamic runtime date context: current ISO timestamp, local date, weekday, local date/time, and timezone. Set `timeZone` in `agentkit.config.ts` for scheduling agents so relative dates like "today" and "next Friday" resolve in the right business/user timezone.
|
|
52
54
|
|
|
55
|
+
AgentKit-managed Composio is a paid hosted integration layer for per-agent connected apps. Configure it with `composioManaged({...})`, deploy with a `managed_composio` entitlement, then use `agentkit integrations connect composio --toolkit <slug>` to create hosted Connect Links. BYO Composio remains available through normal `defineTool` wrappers.
|
|
56
|
+
|
|
53
57
|
## Deploy Later
|
|
54
58
|
|
|
55
59
|
```sh
|
|
56
60
|
agentkit login --token <token>
|
|
57
61
|
agentkit deploy doctor
|
|
62
|
+
agentkit skills status
|
|
58
63
|
agentkit secret set OPENAI_API_KEY --from-local-env
|
|
59
64
|
agentkit deploy
|
|
60
65
|
agentkit deploy status
|
|
66
|
+
agentkit channels test support-telegram --message "hello"
|
|
67
|
+
agentkit transcribe smoke --provider groq
|
|
61
68
|
agentkit chat-ui --deploy
|
|
62
69
|
```
|
|
63
70
|
|
|
64
71
|
Production secrets are managed secrets, not committed `.env` values.
|
|
72
|
+
After upgrading AgentKit in an existing capsule, run `agentkit skills status` and `agentkit skills sync` if local agent-facing skills are stale.
|
|
65
73
|
Private hosted deploys automatically store a local chat/UI deploy access token at `.agentkit/chat-access-token.json`. `agentkit chat-ui --deploy` serves a local UI pointed at the hosted deploy without exposing that token to browser code.
|
|
66
74
|
|
|
67
75
|
The npm package contains the public CLI/runtime/client surface only. AgentKit Cloud's control-plane server, operator commands, Postgres store, and Cloudflare/Turso/R2 publisher live in the private repo workspace and are not part of the published package.
|
|
@@ -87,6 +87,62 @@ agentkit channels buffers clear support-whatsapp <conversation-id>
|
|
|
87
87
|
agentkit channels buffers retry support-whatsapp <conversation-id>
|
|
88
88
|
```
|
|
89
89
|
|
|
90
|
+
## Auto Transcribe Audio
|
|
91
|
+
|
|
92
|
+
Use `transcription` at the agent level and `audio.mode: "transcribe"` on each channel that should accept voice notes or audio files.
|
|
93
|
+
|
|
94
|
+
```ts
|
|
95
|
+
export default defineAgent({
|
|
96
|
+
name: "support-agent",
|
|
97
|
+
runtime: "edge",
|
|
98
|
+
provider: { name: "openai", model: "gpt-5.4-mini" },
|
|
99
|
+
instructions: "./prompts/instructions.md",
|
|
100
|
+
transcription: {
|
|
101
|
+
provider: "groq",
|
|
102
|
+
model: "whisper-large-v3-turbo",
|
|
103
|
+
secret: "GROQ_API_KEY",
|
|
104
|
+
language: "pt",
|
|
105
|
+
limits: {
|
|
106
|
+
maxDurationSeconds: 180,
|
|
107
|
+
maxBytes: 20_000_000,
|
|
108
|
+
},
|
|
109
|
+
},
|
|
110
|
+
channels: [
|
|
111
|
+
telegramChannel({
|
|
112
|
+
name: "support-telegram",
|
|
113
|
+
audio: { mode: "transcribe" },
|
|
114
|
+
}),
|
|
115
|
+
whatsappChannel({
|
|
116
|
+
name: "support-whatsapp",
|
|
117
|
+
provider: "zapster",
|
|
118
|
+
audio: { mode: "transcribe" },
|
|
119
|
+
}),
|
|
120
|
+
],
|
|
121
|
+
access: { mode: "public" },
|
|
122
|
+
storage: { driver: "agentkit" },
|
|
123
|
+
});
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Supported transcription providers in V1:
|
|
127
|
+
|
|
128
|
+
| Provider | Default secret | Supported models |
|
|
129
|
+
| --- | --- | --- |
|
|
130
|
+
| `openai` | `OPENAI_API_KEY` | `gpt-4o-mini-transcribe`, `gpt-4o-transcribe`, `whisper-1` |
|
|
131
|
+
| `groq` | `GROQ_API_KEY` | `whisper-large-v3-turbo`, `whisper-large-v3`, `distil-whisper-large-v3-en` |
|
|
132
|
+
|
|
133
|
+
Audio transcription is paid by the capsule owner because AgentKit only passes through the configured provider secret. Hosted channel creation automatically requires the transcription secret when a channel enables `audio.mode: "transcribe"`.
|
|
134
|
+
|
|
135
|
+
Processing order:
|
|
136
|
+
|
|
137
|
+
1. Provider webhook is validated and deduped.
|
|
138
|
+
2. Channel adapter normalizes the audio metadata.
|
|
139
|
+
3. AgentKit records `audio_received` and enqueues a channel job before acknowledging the webhook.
|
|
140
|
+
4. The retryable channel worker downloads the audio using the channel provider secret.
|
|
141
|
+
5. The transcription adapter sends the file to the configured transcription provider.
|
|
142
|
+
6. The agent receives a text message containing the transcript.
|
|
143
|
+
|
|
144
|
+
V1 keeps the raw audio in memory for the request path and delivery metadata only records redacted status/error fields. `rawAudioTtlSeconds` is part of the manifest contract for future object-storage retention, but V1 does not persist raw audio by default.
|
|
145
|
+
|
|
90
146
|
## Safety Rules
|
|
91
147
|
|
|
92
148
|
- Never put provider token values in `agentkit.config.ts`.
|
|
@@ -94,6 +150,7 @@ agentkit channels buffers retry support-whatsapp <conversation-id>
|
|
|
94
150
|
- Treat channel webhook URLs as public transport endpoints. Provider validation or the AgentKit website channel token controls authenticity.
|
|
95
151
|
- Keep channels separate from tools. Channels deliver user messages; tools let the agent call external systems.
|
|
96
152
|
- Keep `maxMessages` and `maxChars` bounded so one burst cannot create an oversized prompt or unexpected model spend.
|
|
153
|
+
- Keep `audio.limits` bounded so one voice note cannot create unexpected transcription spend.
|
|
97
154
|
|
|
98
155
|
## Verification
|
|
99
156
|
|
|
@@ -103,6 +160,8 @@ bun test
|
|
|
103
160
|
agentkit inspect
|
|
104
161
|
agentkit channels list
|
|
105
162
|
agentkit channels test support-telegram --message "hello"
|
|
163
|
+
agentkit channels test-audio support-telegram --fixture voice-note
|
|
164
|
+
agentkit transcribe smoke --provider groq
|
|
106
165
|
agentkit channels deliveries list support-telegram
|
|
107
166
|
agentkit channels buffers list support-telegram
|
|
108
167
|
```
|
|
@@ -124,4 +183,10 @@ The provider webhook secret, token, or origin header does not match the managed
|
|
|
124
183
|
`channel_limit_exceeded`:
|
|
125
184
|
The channel daily message limit was reached. Website requests return `429`; Telegram and WhatsApp are acknowledged and skipped to avoid provider retry storms.
|
|
126
185
|
|
|
186
|
+
`transcription_secret_missing`:
|
|
187
|
+
Set the managed transcription secret declared by `agentkit inspect`, for example `OPENAI_API_KEY` or `GROQ_API_KEY`.
|
|
188
|
+
|
|
189
|
+
`transcription_audio_format_unsupported`:
|
|
190
|
+
The channel delivered an audio format the configured transcription provider does not accept. Telegram voice notes are OGG/Opus and work with Groq in V1; OpenAI accepts MP3, MP4, MPEG, MPGA, M4A, WAV, and WEBM in the AgentKit adapter.
|
|
191
|
+
|
|
127
192
|
Buffered messages stay in `buffered` delivery state until the quiet window or max wait flushes them into one queued run.
|
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
# Add Managed Composio
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
|
|
5
|
+
Enable AgentKit-managed Composio for one deployed agent so the agent can use explicitly allowed external app actions without the user owning Composio credentials.
|
|
6
|
+
|
|
7
|
+
Use BYO `defineTool` wrappers instead when the user wants to use their own Composio account/API key for free.
|
|
8
|
+
|
|
9
|
+
## Contract
|
|
10
|
+
|
|
11
|
+
Managed Composio is paid hosted AgentKit infrastructure:
|
|
12
|
+
|
|
13
|
+
- It works per agent/project, not per client.
|
|
14
|
+
- It requires an AgentKit Cloud account with `managed_composio`.
|
|
15
|
+
- AgentKit Cloud injects `COMPOSIO_API_KEY`; do not put it in `.env`, `.env.schema`, or `agentkit.config.ts`.
|
|
16
|
+
- AgentKit Cloud assigns the deployed Composio `user_id` from the account, project, agent, and integration name. The local inspect/build id is only a preview.
|
|
17
|
+
- The agent gets one generated tool, `agentkit_composio_execute`, only when explicit Composio action slugs are configured.
|
|
18
|
+
- Anonymous deploys cannot use managed Composio.
|
|
19
|
+
|
|
20
|
+
## Minimal Config
|
|
21
|
+
|
|
22
|
+
Edit `agentkit.config.ts`:
|
|
23
|
+
|
|
24
|
+
```ts
|
|
25
|
+
import { composioManaged, defineAgent } from "@andreprado/agentkit";
|
|
26
|
+
|
|
27
|
+
export default defineAgent({
|
|
28
|
+
name: "acme-receptionist",
|
|
29
|
+
runtime: "edge",
|
|
30
|
+
provider: {
|
|
31
|
+
name: "test",
|
|
32
|
+
model: "fake",
|
|
33
|
+
},
|
|
34
|
+
instructions: "./prompts/instructions.md",
|
|
35
|
+
secrets: [],
|
|
36
|
+
tools: [],
|
|
37
|
+
integrations: [
|
|
38
|
+
composioManaged({
|
|
39
|
+
toolkits: ["gmail", "googlecalendar"],
|
|
40
|
+
tools: {
|
|
41
|
+
gmail: ["GMAIL_FETCH_EMAILS", "GMAIL_SEND_EMAIL"],
|
|
42
|
+
googlecalendar: ["GOOGLECALENDAR_CREATE_EVENT"],
|
|
43
|
+
},
|
|
44
|
+
}),
|
|
45
|
+
],
|
|
46
|
+
access: {
|
|
47
|
+
mode: "private",
|
|
48
|
+
},
|
|
49
|
+
storage: {
|
|
50
|
+
driver: "agentkit",
|
|
51
|
+
database: {
|
|
52
|
+
driver: "turso",
|
|
53
|
+
},
|
|
54
|
+
},
|
|
55
|
+
});
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
## Deploy
|
|
59
|
+
|
|
60
|
+
```sh
|
|
61
|
+
agentkit login --token agk_user_...
|
|
62
|
+
agentkit deploy doctor
|
|
63
|
+
agentkit deploy
|
|
64
|
+
agentkit integrations status
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Expected readiness:
|
|
68
|
+
|
|
69
|
+
- `cloudflare_deploy_alpha` is active.
|
|
70
|
+
- `managed_composio` is active.
|
|
71
|
+
- `COMPOSIO_API_KEY` appears as an AgentKit-managed secret, not a user-managed hosted secret.
|
|
72
|
+
|
|
73
|
+
## Connect Apps
|
|
74
|
+
|
|
75
|
+
After deploy, create a hosted Composio Connect Link:
|
|
76
|
+
|
|
77
|
+
```sh
|
|
78
|
+
agentkit integrations connect composio --toolkit gmail
|
|
79
|
+
agentkit integrations connect composio --toolkit googlecalendar
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
AgentKit prints a URL. Send that URL to the person who owns the app account.
|
|
83
|
+
|
|
84
|
+
If the deploy has only one toolkit, `--toolkit` can be omitted.
|
|
85
|
+
|
|
86
|
+
## Operator Setup
|
|
87
|
+
|
|
88
|
+
Trusted AgentKit Cloud operators grant access and configure Composio auth configs:
|
|
89
|
+
|
|
90
|
+
```sh
|
|
91
|
+
npm run agentkit:operator -- accounts grant user@example.com --managed-composio
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Required control-plane environment:
|
|
95
|
+
|
|
96
|
+
```txt
|
|
97
|
+
AGENTKIT_SECRET_COMPOSIO_API_KEY=<project API key>
|
|
98
|
+
AGENTKIT_COMPOSIO_AUTH_CONFIG_GMAIL=<auth config id>
|
|
99
|
+
AGENTKIT_COMPOSIO_AUTH_CONFIG_GOOGLECALENDAR=<auth config id>
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Use one `AGENTKIT_COMPOSIO_AUTH_CONFIG_<TOOLKIT>` value per toolkit slug, uppercased with non-alphanumeric characters converted to `_`.
|
|
103
|
+
|
|
104
|
+
## Verification
|
|
105
|
+
|
|
106
|
+
```sh
|
|
107
|
+
agentkit inspect
|
|
108
|
+
agentkit build --target cloudflare
|
|
109
|
+
agentkit deploy doctor
|
|
110
|
+
agentkit integrations status
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
Expected:
|
|
114
|
+
|
|
115
|
+
- `inspect.integrations[0].provider` is `composio`.
|
|
116
|
+
- `inspect.tools` includes `agentkit_composio_execute` when allowed actions are configured.
|
|
117
|
+
- Build manifest includes `integrations`.
|
|
118
|
+
- Build manifest includes `COMPOSIO_API_KEY`.
|
|
119
|
+
- `deploy doctor` does not ask the user to set `COMPOSIO_API_KEY`.
|
|
120
|
+
|
|
121
|
+
## Troubleshooting
|
|
122
|
+
|
|
123
|
+
`managed_composio_entitlement_required`:
|
|
124
|
+
|
|
125
|
+
Log in with a paid AgentKit Cloud account that has `managed_composio`.
|
|
126
|
+
|
|
127
|
+
`managed_composio_auth_config_missing`:
|
|
128
|
+
|
|
129
|
+
An operator must set `AGENTKIT_COMPOSIO_AUTH_CONFIG_<TOOLKIT>` on the control plane.
|
|
130
|
+
|
|
131
|
+
`managed_composio_not_configured`:
|
|
132
|
+
|
|
133
|
+
An operator must set `AGENTKIT_SECRET_COMPOSIO_API_KEY` on the control plane.
|
|
134
|
+
|
|
135
|
+
`integration_toolkit_required`:
|
|
136
|
+
|
|
137
|
+
The deploy has multiple configured toolkits. Re-run connect with `--toolkit <slug>`.
|
|
@@ -15,6 +15,7 @@ agentkit inspect
|
|
|
15
15
|
agentkit channels status <name>
|
|
16
16
|
agentkit channels deliveries show <delivery-id>
|
|
17
17
|
bun test packages/agentkit/src/runtime/channels/adapters.test.ts
|
|
18
|
+
bun test packages/agentkit/src/runtime/transcription.test.ts
|
|
18
19
|
bun test packages/agentkit/src/runtime/deploy.test.ts
|
|
19
20
|
```
|
|
20
21
|
|
|
@@ -41,9 +42,12 @@ Zapster smoke also requires `ZAPSTER_API_KEY`, `ZAPSTER_INSTANCE_ID`, and `AGENT
|
|
|
41
42
|
- Validate provider authenticity when the provider supports it.
|
|
42
43
|
- Do not store channel plumbing in the user's Turso database.
|
|
43
44
|
- Do not store raw webhook bodies in delivery records; store a SHA-256 hash.
|
|
45
|
+
- Do not store raw audio in delivery records. V1 audio transcription downloads provider media into memory and stores only redacted metadata and state transitions.
|
|
44
46
|
- Redact bearer tokens, bot tokens, signing secrets, provider API tokens, and phone numbers.
|
|
45
47
|
- Inject only channel-declared secrets into adapter code.
|
|
48
|
+
- Inject transcription secrets only when the deploy manifest declares transcription and the channel uses `audio.mode: "transcribe"`.
|
|
46
49
|
- Prefer acknowledging Telegram/WhatsApp over retry storms when a channel-level limit is exceeded.
|
|
50
|
+
- Keep audio duration and byte limits bounded before provider calls to avoid uncontrolled transcription spend.
|
|
47
51
|
|
|
48
52
|
## Minimal Working Example
|
|
49
53
|
|
|
@@ -54,17 +58,42 @@ telegramChannel({
|
|
|
54
58
|
});
|
|
55
59
|
```
|
|
56
60
|
|
|
61
|
+
With audio transcription:
|
|
62
|
+
|
|
63
|
+
```ts
|
|
64
|
+
export default defineAgent({
|
|
65
|
+
// ...
|
|
66
|
+
transcription: {
|
|
67
|
+
provider: "groq",
|
|
68
|
+
model: "whisper-large-v3-turbo",
|
|
69
|
+
secret: "GROQ_API_KEY",
|
|
70
|
+
limits: {
|
|
71
|
+
maxDurationSeconds: 180,
|
|
72
|
+
maxBytes: 20_000_000,
|
|
73
|
+
},
|
|
74
|
+
},
|
|
75
|
+
channels: [
|
|
76
|
+
telegramChannel({
|
|
77
|
+
name: "support-telegram",
|
|
78
|
+
audio: { mode: "transcribe" },
|
|
79
|
+
}),
|
|
80
|
+
],
|
|
81
|
+
});
|
|
82
|
+
```
|
|
83
|
+
|
|
57
84
|
The config contains secret names only. Hosted responses report:
|
|
58
85
|
|
|
59
86
|
```txt
|
|
60
87
|
TELEGRAM_BOT_TOKEN: set
|
|
61
88
|
TELEGRAM_WEBHOOK_SECRET: missing
|
|
89
|
+
GROQ_API_KEY: set
|
|
62
90
|
```
|
|
63
91
|
|
|
64
92
|
## Verification
|
|
65
93
|
|
|
66
94
|
```sh
|
|
67
95
|
bun test packages/agentkit/src/runtime/channels/adapters.test.ts
|
|
96
|
+
bun test packages/agentkit/src/runtime/transcription.test.ts
|
|
68
97
|
bun test packages/agentkit/src/runtime/deploy.test.ts
|
|
69
98
|
npm run typecheck
|
|
70
99
|
```
|
|
@@ -79,3 +108,6 @@ Redact it to a stable partial form such as `5511******9999`.
|
|
|
79
108
|
|
|
80
109
|
Webhook accepts invalid signatures or origin headers:
|
|
81
110
|
Fix `verifyWebhook` for the adapter before enabling provider setup docs.
|
|
111
|
+
|
|
112
|
+
Raw audio or transcript provider secret appears in output:
|
|
113
|
+
Stop and add a regression test before changing behavior. Delivery APIs may include transcript text in normalized agent messages after successful transcription, but must never include raw bytes or provider secret values.
|
|
@@ -67,7 +67,7 @@ Telegram channel URL: https://<deploy-host>/channels/support-telegram/telegram/t
|
|
|
67
67
|
Zapster channel URL: https://<deploy-host>/channels/support-whatsapp/whatsapp/zapster/webhook
|
|
68
68
|
Required provider webhook settings: Telegram setWebhook URL/secret token; Zapster webhook URL plus optional ZAPSTER_WEBHOOK_TOKEN query parameter
|
|
69
69
|
Required managed secrets: TELEGRAM_BOT_TOKEN, TELEGRAM_WEBHOOK_SECRET, ZAPSTER_API_KEY, ZAPSTER_INSTANCE_ID, ZAPSTER_WEBHOOK_ID
|
|
70
|
-
Smoke commands: agentkit channels doctor <name>; agentkit channels test <name> --message "hello"; agentkit channels deliveries list <name> --since 1h
|
|
70
|
+
Smoke commands: agentkit channels doctor <name>; agentkit channels test <name> --message "hello"; agentkit channels test-audio <name> --fixture voice-note for audio channels; agentkit transcribe smoke --provider groq for transcription; agentkit channels deliveries list <name> --since 1h
|
|
71
71
|
Rollback command: curl -X DELETE "$AGENTKIT_CLOUD_API_URL/v1/channels/<channel-id>" -H "Authorization: Bearer $AGENTKIT_CLOUD_API_TOKEN"; then remove provider webhook registration before rolling back the Worker
|
|
72
72
|
Known unverified items: any channel with no real provider inbound, no provider_sent outbound, stale buffers, or AGENTKIT_CHANNEL_SEND_DRY_RUN=1
|
|
73
73
|
```
|
|
@@ -27,6 +27,12 @@ TELEGRAM_BOT_TOKEN
|
|
|
27
27
|
TELEGRAM_WEBHOOK_SECRET
|
|
28
28
|
```
|
|
29
29
|
|
|
30
|
+
If Telegram audio transcription is enabled, also set the transcription provider secret declared by `agentkit inspect`, usually:
|
|
31
|
+
|
|
32
|
+
```txt
|
|
33
|
+
GROQ_API_KEY
|
|
34
|
+
```
|
|
35
|
+
|
|
30
36
|
## Files Created Or Edited
|
|
31
37
|
|
|
32
38
|
- `agentkit.config.ts`: `telegramChannel({ name: "support-telegram" })`.
|
|
@@ -66,6 +72,52 @@ telegramChannel({
|
|
|
66
72
|
})
|
|
67
73
|
```
|
|
68
74
|
|
|
75
|
+
## Auto Transcribe Telegram Audio
|
|
76
|
+
|
|
77
|
+
Telegram voice notes arrive as OGG/Opus. In V1, use Groq for the most obvious Telegram voice-note path because the AgentKit Groq adapter accepts `audio/ogg`.
|
|
78
|
+
|
|
79
|
+
```ts
|
|
80
|
+
import { defineAgent, telegramChannel } from "@andreprado/agentkit";
|
|
81
|
+
|
|
82
|
+
export default defineAgent({
|
|
83
|
+
name: "support-agent",
|
|
84
|
+
runtime: "edge",
|
|
85
|
+
provider: { name: "openai", model: "gpt-5.4-mini" },
|
|
86
|
+
instructions: "./prompts/instructions.md",
|
|
87
|
+
transcription: {
|
|
88
|
+
provider: "groq",
|
|
89
|
+
model: "whisper-large-v3-turbo",
|
|
90
|
+
secret: "GROQ_API_KEY",
|
|
91
|
+
language: "pt",
|
|
92
|
+
limits: {
|
|
93
|
+
maxDurationSeconds: 180,
|
|
94
|
+
maxBytes: 20_000_000,
|
|
95
|
+
},
|
|
96
|
+
},
|
|
97
|
+
channels: [
|
|
98
|
+
telegramChannel({
|
|
99
|
+
name: "support-telegram",
|
|
100
|
+
audio: {
|
|
101
|
+
mode: "transcribe",
|
|
102
|
+
},
|
|
103
|
+
}),
|
|
104
|
+
],
|
|
105
|
+
access: { mode: "public" },
|
|
106
|
+
storage: { driver: "agentkit" },
|
|
107
|
+
});
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Processing order:
|
|
111
|
+
|
|
112
|
+
1. AgentKit validates `X-Telegram-Bot-Api-Secret-Token`.
|
|
113
|
+
2. Telegram `voice` or `audio` payloads become normalized audio messages.
|
|
114
|
+
3. AgentKit records `audio_received` and enqueues a channel job before acknowledging Telegram.
|
|
115
|
+
4. The retryable channel worker calls Telegram `getFile`, downloads the file with `TELEGRAM_BOT_TOKEN`, and does not log the token.
|
|
116
|
+
5. AgentKit sends the audio bytes to the configured transcription provider using the user's managed secret.
|
|
117
|
+
6. The agent run receives a text message with the transcript.
|
|
118
|
+
|
|
119
|
+
OpenAI transcription can be used for Telegram files that arrive as MP3, MP4, MPEG, MPGA, M4A, WAV, or WEBM. Telegram voice notes are usually OGG/Opus, so they should use Groq in V1 unless the provider payload is converted before it reaches AgentKit.
|
|
120
|
+
|
|
69
121
|
## Setup Behavior
|
|
70
122
|
|
|
71
123
|
`agentkit channels setup support-telegram` is read-only and prints the webhook URL.
|
|
@@ -89,6 +141,8 @@ Against AgentKit Cloud, `--apply` runs through the Cloud API and uses managed ho
|
|
|
89
141
|
```sh
|
|
90
142
|
agentkit channels status support-telegram
|
|
91
143
|
agentkit channels test support-telegram --message "hello"
|
|
144
|
+
agentkit channels test-audio support-telegram --fixture voice-note
|
|
145
|
+
agentkit transcribe smoke --provider groq
|
|
92
146
|
agentkit channels deliveries show <delivery-id>
|
|
93
147
|
```
|
|
94
148
|
|
|
@@ -108,3 +162,9 @@ The incoming Telegram secret token does not match `TELEGRAM_WEBHOOK_SECRET`.
|
|
|
108
162
|
|
|
109
163
|
`channel_payload_invalid`:
|
|
110
164
|
The update is malformed or is not a supported private text message.
|
|
165
|
+
|
|
166
|
+
`transcription_secret_missing`:
|
|
167
|
+
Set `GROQ_API_KEY` or the custom secret named in `transcription.secret` as a managed hosted secret.
|
|
168
|
+
|
|
169
|
+
`transcription_audio_format_unsupported`:
|
|
170
|
+
The configured transcription provider does not accept the Telegram file format. Use Groq for OGG/Opus voice notes in V1. `agentkit inspect` warns when a Telegram channel uses `audio.mode: "transcribe"` with OpenAI transcription.
|
|
@@ -33,6 +33,18 @@ Optional hardening secret:
|
|
|
33
33
|
ZAPSTER_WEBHOOK_TOKEN
|
|
34
34
|
```
|
|
35
35
|
|
|
36
|
+
If WhatsApp audio transcription is enabled, also set the transcription provider secret declared by `agentkit inspect`, usually:
|
|
37
|
+
|
|
38
|
+
```txt
|
|
39
|
+
OPENAI_API_KEY
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
or:
|
|
43
|
+
|
|
44
|
+
```txt
|
|
45
|
+
GROQ_API_KEY
|
|
46
|
+
```
|
|
47
|
+
|
|
36
48
|
## Files Created Or Edited
|
|
37
49
|
|
|
38
50
|
- `agentkit.config.ts`: `whatsappChannel({ name: "support-whatsapp", provider: "zapster" })`.
|
|
@@ -73,6 +85,53 @@ whatsappChannel({
|
|
|
73
85
|
})
|
|
74
86
|
```
|
|
75
87
|
|
|
88
|
+
## Auto Transcribe WhatsApp Audio
|
|
89
|
+
|
|
90
|
+
Use `transcription` at the agent level and `audio.mode: "transcribe"` on the Zapster channel.
|
|
91
|
+
|
|
92
|
+
```ts
|
|
93
|
+
import { defineAgent, whatsappChannel } from "@andreprado/agentkit";
|
|
94
|
+
|
|
95
|
+
export default defineAgent({
|
|
96
|
+
name: "support-agent",
|
|
97
|
+
runtime: "edge",
|
|
98
|
+
provider: { name: "openai", model: "gpt-5.4-mini" },
|
|
99
|
+
instructions: "./prompts/instructions.md",
|
|
100
|
+
transcription: {
|
|
101
|
+
provider: "openai",
|
|
102
|
+
model: "gpt-4o-mini-transcribe",
|
|
103
|
+
secret: "OPENAI_API_KEY",
|
|
104
|
+
language: "pt",
|
|
105
|
+
limits: {
|
|
106
|
+
maxDurationSeconds: 180,
|
|
107
|
+
maxBytes: 20_000_000,
|
|
108
|
+
},
|
|
109
|
+
},
|
|
110
|
+
channels: [
|
|
111
|
+
whatsappChannel({
|
|
112
|
+
name: "support-whatsapp",
|
|
113
|
+
provider: "zapster",
|
|
114
|
+
audio: {
|
|
115
|
+
mode: "transcribe",
|
|
116
|
+
},
|
|
117
|
+
}),
|
|
118
|
+
],
|
|
119
|
+
access: { mode: "public" },
|
|
120
|
+
storage: { driver: "agentkit" },
|
|
121
|
+
});
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Processing order:
|
|
125
|
+
|
|
126
|
+
1. AgentKit validates Zapster origin headers and the optional webhook token.
|
|
127
|
+
2. Zapster audio payloads become normalized audio messages.
|
|
128
|
+
3. AgentKit records `audio_received` and enqueues a channel job before acknowledging Zapster.
|
|
129
|
+
4. The retryable channel worker downloads the media URL from a trusted Zapster HTTPS host using `ZAPSTER_API_KEY`.
|
|
130
|
+
5. AgentKit sends the audio bytes to the configured transcription provider using the user's managed secret.
|
|
131
|
+
6. The agent run receives a text message with the transcript.
|
|
132
|
+
|
|
133
|
+
Zapster payloads must include a media download URL such as `audio.downloadUrl`, `audio.url`, `audio.mediaUrl`, or the snake_case equivalents. The URL must be HTTPS and hosted by Zapster; AgentKit rejects arbitrary webhook-provided hosts before sending `ZAPSTER_API_KEY`. If Zapster sends only a media ID without a download URL, AgentKit records `channel_audio_download_unavailable` and does not create an agent run for that audio in V1.
|
|
134
|
+
|
|
76
135
|
## Setup Behavior
|
|
77
136
|
|
|
78
137
|
`agentkit channels setup support-whatsapp` prints the stable AgentKit webhook URL. Paste it into Zapster webhook settings.
|
|
@@ -125,3 +184,9 @@ Zapster is not sending the expected instance/webhook IDs, or the optional query
|
|
|
125
184
|
|
|
126
185
|
`channel_unsupported_message_type` or skipped delivery:
|
|
127
186
|
The inbound WhatsApp event was not supported text. Inspect the delivery record for provider metadata.
|
|
187
|
+
|
|
188
|
+
`channel_audio_download_unavailable`:
|
|
189
|
+
Zapster sent an audio event without a usable media download URL, or the URL was not an HTTPS Zapster media host. Configure Zapster to include a trusted Zapster media URL in webhook payloads, or add a Zapster media lookup adapter before enabling `audio.mode: "transcribe"`.
|
|
190
|
+
|
|
191
|
+
`transcription_secret_missing`:
|
|
192
|
+
Set `OPENAI_API_KEY`, `GROQ_API_KEY`, or the custom secret named in `transcription.secret` as a managed hosted secret.
|
|
@@ -192,7 +192,9 @@ Before a hosted production deploy, run:
|
|
|
192
192
|
npm run agentkit -- deploy doctor
|
|
193
193
|
```
|
|
194
194
|
|
|
195
|
-
The doctor checks AgentKit Cloud login, alpha deploy entitlement, online deploy capacity, declared hosted secrets, local `.env` names that have not been uploaded with `agentkit secret set`, and private-access runtime token handling. `agentkit deploy` runs the same readiness check automatically before building and uploading the artifact.
|
|
195
|
+
The doctor checks AgentKit Cloud login, Node version, alpha deploy entitlement, online deploy capacity, declared hosted secrets, local `.env` names that have not been uploaded with `npm run agentkit -- secret set`, and private-access runtime token handling. `agentkit deploy` runs the same readiness check automatically before building and uploading the artifact.
|
|
196
|
+
|
|
197
|
+
If the capsule configures `composioManaged({...})`, the doctor also checks the paid `managed_composio` entitlement. `COMPOSIO_API_KEY` is AgentKit-managed in this path; the user must not set it with `agentkit secret set`. Operators configure `AGENTKIT_SECRET_COMPOSIO_API_KEY` and per-toolkit `AGENTKIT_COMPOSIO_AUTH_CONFIG_<TOOLKIT>` values on the control plane.
|
|
196
198
|
|
|
197
199
|
`alpha_access_required`:
|
|
198
200
|
|
package/docs/llms-full.txt
CHANGED
|
@@ -56,8 +56,13 @@ agentkit channels add <website|telegram|whatsapp> <name> [--provider zapster|met
|
|
|
56
56
|
agentkit channels setup <name> [--apply] [--api <url>]
|
|
57
57
|
agentkit channels status <name> [--api <url>]
|
|
58
58
|
agentkit channels test <name> [--message <text>] [--fixture <path>] [--api <url>]
|
|
59
|
+
agentkit channels test-audio <name> [--fixture voice-note|audio-file|<path>] [--api <url>]
|
|
59
60
|
agentkit channels deliveries list <name> [--api <url>]
|
|
60
61
|
agentkit channels deliveries show <delivery-id> [--api <url>]
|
|
62
|
+
agentkit integrations status [--api <url>]
|
|
63
|
+
agentkit integrations connect composio [--toolkit <slug>] [--callback-url <url>] [--api <url>]
|
|
64
|
+
agentkit skills status
|
|
65
|
+
agentkit skills sync
|
|
61
66
|
agentkit inspect
|
|
62
67
|
agentkit build [--target cloudflare|container]
|
|
63
68
|
agentkit login --token <token>
|
|
@@ -68,6 +73,7 @@ agentkit deploy smoke [--message <text>] [--api <url>]
|
|
|
68
73
|
agentkit deploy status
|
|
69
74
|
agentkit deploy pause
|
|
70
75
|
agentkit deploy resume
|
|
76
|
+
agentkit transcribe smoke [--provider test|openai|groq] [--model <model>] [--fixture <audio-file>]
|
|
71
77
|
agentkit secret set <NAME> <VALUE>
|
|
72
78
|
agentkit secret set <NAME> --stdin
|
|
73
79
|
agentkit secret set <NAME> --from-env [ENV_NAME]
|
|
@@ -93,6 +99,8 @@ agentkit help commands
|
|
|
93
99
|
|
|
94
100
|
Prefer `env set --stdin` or `--from-env` for local secret values, and prefer `secret set --stdin`, `--from-env`, or `--from-local-env` for hosted secrets. Inline `<VALUE>` forms exist for simple non-sensitive values, but agents should avoid putting secrets in shell history.
|
|
95
101
|
|
|
102
|
+
Managed Composio is configured with `composioManaged({...})` in `agentkit.config.ts` and is paid hosted AgentKit infrastructure. It requires a non-anonymous AgentKit Cloud deploy with `managed_composio`, uses one Composio settings profile per agent, injects `COMPOSIO_API_KEY` as an AgentKit-managed secret, and exposes the generated `agentkit_composio_execute` tool only for explicit configured action slugs. See `docs/guides/add-managed-composio.md`.
|
|
103
|
+
|
|
96
104
|
Planned commands described by the contract but not implemented yet:
|
|
97
105
|
|
|
98
106
|
```sh
|
|
@@ -398,11 +406,13 @@ agentkit channels list
|
|
|
398
406
|
agentkit channels add telegram support-telegram
|
|
399
407
|
agentkit channels setup support-telegram
|
|
400
408
|
agentkit channels test support-telegram --message "hello"
|
|
409
|
+
agentkit channels test-audio support-telegram --fixture voice-note
|
|
410
|
+
agentkit transcribe smoke --provider groq
|
|
401
411
|
agentkit channels deliveries list support-telegram
|
|
402
412
|
agentkit channels deliveries show <delivery-id>
|
|
403
413
|
```
|
|
404
414
|
|
|
405
|
-
`channels setup` is read-only by default. `channels setup <telegram-name> --apply` calls Telegram `setWebhook` and requires `TELEGRAM_BOT_TOKEN` plus `TELEGRAM_WEBHOOK_SECRET`.
|
|
415
|
+
`channels connect` creates or refreshes the channel resource, validates secrets, runs provider setup when supported, then runs the official synthetic smoke. `channels setup` is read-only by default. `channels setup <telegram-name> --apply` calls Telegram `setWebhook` and requires `TELEGRAM_BOT_TOKEN` plus `TELEGRAM_WEBHOOK_SECRET`.
|
|
406
416
|
|
|
407
417
|
Default tests are offline. Real provider smoke tests are opt-in:
|
|
408
418
|
|
|
@@ -796,6 +806,8 @@ Account/access flow:
|
|
|
796
806
|
agentkit secret set OPENAI_API_KEY --from-local-env
|
|
797
807
|
agentkit secret sync --from-local
|
|
798
808
|
agentkit secret list
|
|
809
|
+
agentkit skills status
|
|
810
|
+
agentkit skills sync
|
|
799
811
|
agentkit access token create website-chat --out .agentkit/website-chat-access-token.json
|
|
800
812
|
agentkit access token list
|
|
801
813
|
```
|
package/docs/llms.txt
CHANGED
|
@@ -16,6 +16,7 @@ Task guides:
|
|
|
16
16
|
- Build container artifact: `agentkit build --target container`
|
|
17
17
|
- Generate VPS handoff: `agentkit deploy --target vps --host agent.example.com --dry-run`
|
|
18
18
|
- Add hosted channels: `docs/guides/add-channel.md`
|
|
19
|
+
- Add AgentKit-managed Composio: `docs/guides/add-managed-composio.md`
|
|
19
20
|
- Buffer rapid channel messages: `docs/guides/add-channel.md#buffer-bursty-messages`
|
|
20
21
|
- Connect Telegram: `docs/guides/connect-telegram.md`
|
|
21
22
|
- Connect WhatsApp through Zapster: `docs/guides/connect-whatsapp-zapster.md`
|
|
@@ -48,7 +49,13 @@ agentkit channels add telegram support-telegram
|
|
|
48
49
|
agentkit channels setup support-telegram
|
|
49
50
|
agentkit channels status support-telegram
|
|
50
51
|
agentkit channels test support-telegram --message "hello"
|
|
52
|
+
agentkit channels test-audio support-telegram --fixture voice-note
|
|
53
|
+
agentkit transcribe smoke --provider groq
|
|
51
54
|
agentkit channels deliveries list support-telegram
|
|
55
|
+
agentkit integrations status
|
|
56
|
+
agentkit integrations connect composio --toolkit gmail
|
|
57
|
+
agentkit skills status
|
|
58
|
+
agentkit skills sync
|
|
52
59
|
agentkit login --token agk_user_...
|
|
53
60
|
agentkit deploy doctor
|
|
54
61
|
agentkit deploy --smoke "hello"
|