mercury-agent 0.6.0 → 0.7.0-beta.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -41,6 +41,21 @@ Per-space overrides via `mrctl config set context.<key> <value>` always win over
41
41
 
42
42
  You may also set a top-level **`model_chain`** array as an alias for `model.chain`.
43
43
 
44
+ ## Extension config defaults (`extensions:`)
45
+
46
+ Deployment-wide defaults for extension config keys, applied to **every space** (including auto-created DM spaces) at read time:
47
+
48
+ ```yaml
49
+ extensions:
50
+ voice-transcribe:
51
+ provider: openai
52
+ model: whisper-large-v3
53
+ base_url: https://api.groq.com/openai/v1
54
+ language: he
55
+ ```
56
+
57
+ Resolution order for an extension config key: per-space value (`mrctl config set`) → `@global` scope (set from the dashboard **Features** page) → this YAML section (env: `MERCURY_EXTENSION_DEFAULTS` as flat JSON, e.g. `{"voice-transcribe.provider":"openai"}`) → the extension's registered default. Changing a global or YAML value takes effect for all existing and future spaces immediately (YAML on restart) — nothing is copied into per-space rows.
58
+
44
59
  ## Model chain
45
60
 
46
61
  In YAML, use a list of `{ provider, model }` objects under `model.chain` (max 4 legs). The same rules apply as for `MERCURY_MODEL_CHAIN` JSON.
package/docs/dashboard.md CHANGED
@@ -64,6 +64,8 @@ Manage API keys for supported model providers (Anthropic, OpenAI, Google Gemini,
64
64
 
65
65
  Extension management through the UI. Shows installed extensions with their capabilities (CLI / skill / widget) and a remove button. Lists available catalog extensions with a one-click install button. Installing or removing an extension triggers a container image rebuild and restart.
66
66
 
67
+ Also hosts the **Global config defaults** panel: set deployment-wide values for any registered extension config key (the `@global` scope). These apply to every space — including auto-created DM spaces — unless a space overrides them, and win over the `extensions:` section in `mercury.yaml`. See [configuration.md](configuration.md#extension-config-defaults-extensions).
68
+
67
69
  ## Real-time Updates
68
70
 
69
71
  The dashboard uses Server-Sent Events (SSE) for live updates:
@@ -95,10 +95,12 @@ Declare an environment variable this extension needs. Only injected into contain
95
95
  ```typescript
96
96
  mercury.env({ from: "MERCURY_GH_TOKEN" }); // injected as GH_TOKEN
97
97
  mercury.env({ from: "MERCURY_GH_TOKEN", as: "GITHUB_TOKEN" }); // custom container name
98
+ mercury.env({ from: "MERCURY_STT_API_KEY", hostOnly: true }); // host-side only
98
99
  ```
99
100
 
100
101
  - `from` — env var name as set in `.env` (e.g. `MERCURY_GH_TOKEN`)
101
102
  - `as` — (optional) name inside the container. Defaults to `from` with `MERCURY_` prefix stripped
103
+ - `hostOnly` — (optional) claim the var (kept out of the blind `MERCURY_*` passthrough) but **never inject it into any container**. Use for secrets consumed only by host-side hooks/jobs (e.g. an STT API key used in `before_container`).
102
104
 
103
105
  Can be called multiple times for multiple env vars.
104
106
 
@@ -223,6 +225,8 @@ mercury.config("enabled", {
223
225
 
224
226
  Keys are namespaced to the extension: the above registers `napkin.enabled`. Users set it via `mrctl config set napkin.enabled false`.
225
227
 
228
+ In hooks and jobs, read values with `ctx.getConfig(spaceId, "napkin.enabled")` instead of `ctx.db.getSpaceConfig(...) ?? default`. It resolves the full chain: per-space value → `@global` scope (dashboard **Features** page) → `extensions:` section in `mercury.yaml` → the registered default. This makes deployment-wide defaults work for auto-created spaces without any per-space setup.
229
+
226
230
  ### `mercury.widget(def)`
227
231
 
228
232
  Register a dashboard widget.
@@ -262,9 +266,38 @@ interface MercuryExtensionContext {
262
266
  readonly config: AppConfig; // Mercury configuration
263
267
  readonly log: Logger; // Logger scoped to the extension
264
268
  hasCallerPermission(spaceId: string, callerId: string, permission: string): boolean;
269
+ getConfig(spaceId: string, key: string): string | null;
270
+ send(opts: { to: string; text: string }): Promise<{ spaceId: string }>;
265
271
  }
266
272
  ```
267
273
 
274
+ ### `ctx.send(opts)` — deterministic direct send
275
+
276
+ Delivers a message straight to the adapter outbox — no agent run, no LLM in the path. Use it from jobs and capability handlers when the message text must arrive exactly as composed (reminders, notifications):
277
+
278
+ ```typescript
279
+ mercury.job("appointment-reminders", {
280
+ cron: "*/15 * * * *",
281
+ run: async (ctx) => {
282
+ for (const r of dueReminders(ctx)) {
283
+ try {
284
+ await ctx.send({ to: r.callerId, text: r.text });
285
+ } catch (err) {
286
+ ctx.log.warn("reminder send failed — retrying next sweep", {
287
+ error: err instanceof Error ? err.message : String(err),
288
+ });
289
+ }
290
+ }
291
+ },
292
+ });
293
+ ```
294
+
295
+ `to` resolves callerId-first: an exact space id, a raw WhatsApp caller id (phone JID or opaque LID — same normalization that keys DM auto-spaces), a platform-qualified id (`whatsapp:123@lid`), or a phone with leading `+`. It never creates spaces — sending to someone without an existing conversation fails.
296
+
297
+ Failures throw `DirectSendError` with a `reason` you can branch on: `sender_not_ready` (adapters not up yet, or the context has no delivery path — e.g. connection status probes), `unknown_recipient` (no existing space matches), `invalid_text` (empty or over 4096 chars).
298
+
299
+ The same delivery path is exposed to ops scripts as `POST /api/send` (Bearer `MERCURY_API_SECRET`, global-admin caller, body `{ "recipient": "...", "text": "..." }`). Like `/api/broadcast`, it is host-side only — never member-callable from containers.
300
+
268
301
  ## Container Integration
269
302
 
270
303
  ### Skill Files
@@ -132,8 +132,12 @@ Check out this sunset!
132
132
  Voice and audio attachments are not playable inside pi. Install the **`voice-transcribe`** extension (see dashboard catalog or `examples/extensions/voice-transcribe/`) to append a text transcript before the agent runs.
133
133
 
134
134
  - **Default (`voice-transcribe.provider=local`)**: runs Python on the Mercury host; install deps from the extension’s `requirements.txt`. Set `voice-transcribe.local_engine` to `transformers` (default, e.g. [mike249/whisper-tiny-he-2](https://huggingface.co/mike249/whisper-tiny-he-2)) or `faster_whisper` for [CTranslate2](https://github.com/OpenNMT/CTranslate2) models on the Hub (e.g. [ivrit-ai](https://huggingface.co/ivrit-ai) `*-ct2` repos). See the extension skill for `MERCURY_VOICE_FW_COMPUTE_TYPE` and `MERCURY_VOICE_LANGUAGE`.
135
+ - **OpenAI-compatible (`voice-transcribe.provider=openai`)**: cloud `POST /audio/transcriptions` with `MERCURY_STT_API_KEY` (host-only). Works with OpenAI (default) or Groq via `voice-transcribe.base_url=https://api.groq.com/openai/v1`. No Python/GPU on the host — best for small VPS deployments.
136
+ - **Gemini (`voice-transcribe.provider=gemini`)**: Google Gemini audio understanding with `MERCURY_STT_GEMINI_API_KEY` (host-only, plain API key).
135
137
  - **API (`voice-transcribe.provider=api`)**: uses the [Hugging Face Inference API](https://huggingface.co/docs/api-inference) with `MERCURY_HF_TOKEN` — choose a model that has a Hub Inference Provider.
136
138
 
139
+ Config keys resolve per space with deployment-wide fallbacks — see [Extension config defaults](../configuration.md#extension-config-defaults-extensions) for the `extensions:` YAML section and the dashboard `@global` scope.
140
+
137
141
  ## Future Enhancements
138
142
 
139
143
  - [ ] Video frame extraction
@@ -155,7 +155,7 @@ Unknown MIME types default to `.bin`.
155
155
 
156
156
  1. **No re-download** — Media is downloaded once when the message arrives. If the file is deleted, it's gone.
157
157
 
158
- 2. **No transcription** — Voice notes are saved as audio files. pi cannot play them. Future: add Whisper transcription.
158
+ 2. **No built-in transcription** — Voice notes are saved as audio files; pi cannot play them. Install the **voice-transcribe** extension (local Whisper or cloud OpenAI/Groq/Gemini) to prepend a text transcript — see [overview.md](overview.md#voice-transcription).
159
159
 
160
160
  3. **Reply context doesn't include file** — When replying to a media message, we include metadata but not the actual file path. The original attachment would need to be looked up.
161
161
 
@@ -13,6 +13,16 @@ const DEFAULT_PROVIDER = "local";
13
13
  const DEFAULT_LOCAL_ENGINE = "transformers";
14
14
  /** Classic HF Inference API (serverless). */
15
15
  const HF_INFERENCE_BASE = "https://api-inference.huggingface.co/models";
16
+ /** OpenAI-compatible /audio/transcriptions root; override via base_url for Groq etc. */
17
+ const DEFAULT_OPENAI_BASE_URL = "https://api.openai.com/v1";
18
+ /** Sensible model when provider=openai and the configured model is not an OpenAI/Groq one. */
19
+ const DEFAULT_OPENAI_MODEL = "gpt-4o-mini-transcribe";
20
+ const GEMINI_API_BASE =
21
+ "https://generativelanguage.googleapis.com/v1beta/models";
22
+ /** Sensible model when provider=gemini and the configured model is not a Gemini one. */
23
+ const DEFAULT_GEMINI_MODEL = "gemini-2.5-flash";
24
+ const PROVIDERS = ["local", "api", "openai", "gemini"] as const;
25
+ type Provider = (typeof PROVIDERS)[number];
16
26
 
17
27
  const SCRIPT_PATH = path.join(
18
28
  path.dirname(fileURLToPath(import.meta.url)),
@@ -63,6 +73,95 @@ async function transcribeWithApi(
63
73
  return parseTranscriptionJson(raw);
64
74
  }
65
75
 
76
+ /**
77
+ * OpenAI-compatible transcription (OpenAI, Groq, or any host implementing
78
+ * `POST {base}/audio/transcriptions`).
79
+ */
80
+ async function transcribeWithOpenAi(
81
+ filePath: string,
82
+ mimeType: string,
83
+ modelId: string,
84
+ baseUrl: string,
85
+ language: string,
86
+ token: string,
87
+ ): Promise<string> {
88
+ const body = await readFile(filePath);
89
+ const form = new FormData();
90
+ form.append(
91
+ "file",
92
+ new Blob([body], { type: mimeType.split(";")[0].trim() || "audio/ogg" }),
93
+ path.basename(filePath) || "audio.ogg",
94
+ );
95
+ form.append("model", modelId);
96
+ form.append("response_format", "json");
97
+ if (language) form.append("language", language);
98
+
99
+ const url = `${baseUrl.replace(/\/+$/, "")}/audio/transcriptions`;
100
+ const res = await fetch(url, {
101
+ method: "POST",
102
+ headers: { Authorization: `Bearer ${token}` },
103
+ body: form,
104
+ });
105
+ const raw = await res.text();
106
+ if (!res.ok) {
107
+ throw new Error(`OpenAI-compatible STT ${res.status}: ${raw.slice(0, 240)}`);
108
+ }
109
+ return parseTranscriptionJson(raw);
110
+ }
111
+
112
+ /** Gemini audio understanding via generateContent (API key, no GCP project). */
113
+ async function transcribeWithGemini(
114
+ filePath: string,
115
+ mimeType: string,
116
+ modelId: string,
117
+ language: string,
118
+ apiKey: string,
119
+ ): Promise<string> {
120
+ const body = await readFile(filePath);
121
+ const langHint = language ? ` The audio language is "${language}".` : "";
122
+ const url = `${GEMINI_API_BASE}/${encodeURIComponent(modelId)}:generateContent`;
123
+ const res = await fetch(url, {
124
+ method: "POST",
125
+ headers: {
126
+ "x-goog-api-key": apiKey,
127
+ "Content-Type": "application/json",
128
+ },
129
+ body: JSON.stringify({
130
+ contents: [
131
+ {
132
+ parts: [
133
+ {
134
+ text: `Transcribe this audio verbatim.${langHint} Output only the transcript text, nothing else.`,
135
+ },
136
+ {
137
+ inline_data: {
138
+ mime_type: mimeType.split(";")[0].trim() || "audio/ogg",
139
+ data: body.toString("base64"),
140
+ },
141
+ },
142
+ ],
143
+ },
144
+ ],
145
+ }),
146
+ });
147
+ const raw = await res.text();
148
+ if (!res.ok) {
149
+ throw new Error(`Gemini STT ${res.status}: ${raw.slice(0, 240)}`);
150
+ }
151
+ try {
152
+ const data = JSON.parse(raw) as {
153
+ candidates?: Array<{ content?: { parts?: Array<{ text?: string }> } }>;
154
+ };
155
+ const parts = data.candidates?.[0]?.content?.parts ?? [];
156
+ return parts
157
+ .map((p) => p.text ?? "")
158
+ .join("")
159
+ .trim();
160
+ } catch {
161
+ return "";
162
+ }
163
+ }
164
+
66
165
  function parseLastJsonLine(stdout: string): {
67
166
  text?: string;
68
167
  error?: string;
@@ -207,9 +306,33 @@ function localTimeoutMs(): number {
207
306
  return 300_000;
208
307
  }
209
308
 
309
+ type HookCtx = {
310
+ db: { getSpaceConfig: (spaceId: string, key: string) => string | null };
311
+ log: {
312
+ info: (msg: string, extra?: unknown) => void;
313
+ warn: (msg: string, extra?: unknown) => void;
314
+ error: (msg: string, extra?: unknown) => void;
315
+ };
316
+ hasCallerPermission: (
317
+ spaceId: string,
318
+ callerId: string,
319
+ permission: string,
320
+ ) => boolean;
321
+ /** Host-resolved config (space → @global → mercury.yaml → default). Optional for older hosts. */
322
+ getConfig?: (spaceId: string, key: string) => string | null;
323
+ };
324
+
325
+ /** Resolve a config value via ctx.getConfig when the host supports it. */
326
+ function readConfig(ctx: HookCtx, spaceId: string, key: string): string {
327
+ const resolved = ctx.getConfig
328
+ ? ctx.getConfig(spaceId, `${EXT}.${key}`)
329
+ : ctx.db.getSpaceConfig(spaceId, `${EXT}.${key}`);
330
+ return resolved?.trim() ?? "";
331
+ }
332
+
210
333
  export default function (mercury: {
211
334
  permission(opts: { defaultRoles: string[] }): void;
212
- env(def: { from: string }): void;
335
+ env(def: { from: string; hostOnly?: boolean }): void;
213
336
  config(
214
337
  key: string,
215
338
  def: {
@@ -230,33 +353,29 @@ export default function (mercury: {
230
353
  containerWorkspace: string;
231
354
  attachments?: Attachment[];
232
355
  },
233
- ctx: {
234
- db: { getSpaceConfig: (spaceId: string, key: string) => string | null };
235
- log: {
236
- info: (msg: string, extra?: unknown) => void;
237
- warn: (msg: string, extra?: unknown) => void;
238
- error: (msg: string, extra?: unknown) => void;
239
- };
240
- hasCallerPermission: (
241
- spaceId: string,
242
- callerId: string,
243
- permission: string,
244
- ) => boolean;
245
- },
356
+ ctx: HookCtx,
246
357
  ) => Promise<{ promptAppend?: string } | undefined>,
247
358
  ): void;
248
359
  }) {
249
360
  mercury.permission({ defaultRoles: ["admin", "member"] });
250
- mercury.env({ from: "MERCURY_HF_TOKEN" });
361
+ // All STT keys are consumed host-side in before_container — never inject
362
+ // them into agent containers (customers in DM auto-spaces hold the
363
+ // voice-transcribe permission).
364
+ mercury.env({ from: "MERCURY_HF_TOKEN", hostOnly: true });
365
+ // NOTE: MERCURY_GEMINI_API_KEY (model-chain key) is intentionally not
366
+ // reused — claiming it hostOnly would strip GEMINI_API_KEY from containers
367
+ // whose model chain runs on Gemini.
368
+ mercury.env({ from: "MERCURY_STT_API_KEY", hostOnly: true });
369
+ mercury.env({ from: "MERCURY_STT_GEMINI_API_KEY", hostOnly: true });
251
370
  mercury.config("provider", {
252
371
  description:
253
- '"local" = Python+transformers on Mercury host (see skill); "api" = Hugging Face Inference API (needs MERCURY_HF_TOKEN)',
372
+ '"local" = Python+transformers on Mercury host (see skill); "api" = Hugging Face Inference API (MERCURY_HF_TOKEN); "openai" = OpenAI-compatible /audio/transcriptions incl. Groq (MERCURY_STT_API_KEY, see base_url); "gemini" = Google Gemini audio (MERCURY_STT_GEMINI_API_KEY)',
254
373
  default: DEFAULT_PROVIDER,
255
- validate: (v) => v === "local" || v === "api",
374
+ validate: (v) => (PROVIDERS as readonly string[]).includes(v),
256
375
  });
257
376
  mercury.config("model", {
258
377
  description:
259
- "Hugging Face model id (e.g. mike249/whisper-tiny-he-2 for local Hebrew; openai/whisper-large-v3 for api)",
378
+ "Model id per provider: HF id for local/api (e.g. mike249/whisper-tiny-he-2), e.g. gpt-4o-mini-transcribe or whisper-large-v3 for openai, gemini-2.5-flash for gemini",
260
379
  default: DEFAULT_MODEL,
261
380
  });
262
381
  mercury.config("local_engine", {
@@ -265,6 +384,17 @@ export default function (mercury: {
265
384
  default: DEFAULT_LOCAL_ENGINE,
266
385
  validate: (v) => v === "transformers" || v === "faster_whisper",
267
386
  });
387
+ mercury.config("base_url", {
388
+ description:
389
+ "openai provider only: API root implementing /audio/transcriptions (default OpenAI; use https://api.groq.com/openai/v1 for Groq)",
390
+ default: DEFAULT_OPENAI_BASE_URL,
391
+ validate: (v) => /^https:\/\//.test(v),
392
+ });
393
+ mercury.config("language", {
394
+ description:
395
+ 'ISO-639-1 hint passed to cloud STT (e.g. "he"). Improves accuracy on short voice notes. Empty = auto-detect.',
396
+ default: "",
397
+ });
268
398
  mercury.skill("./skill");
269
399
 
270
400
  mercury.on("before_container", async (event, ctx) => {
@@ -277,19 +407,21 @@ export default function (mercury: {
277
407
  );
278
408
  if (!atts?.length) return undefined;
279
409
 
280
- const provider =
281
- ctx.db.getSpaceConfig(event.spaceId, `${EXT}.provider`)?.trim() ||
282
- DEFAULT_PROVIDER;
283
- const model =
284
- ctx.db.getSpaceConfig(event.spaceId, `${EXT}.model`)?.trim() ||
285
- DEFAULT_MODEL;
286
- const rawLocalEngine = ctx.db
287
- .getSpaceConfig(event.spaceId, `${EXT}.local_engine`)
288
- ?.trim();
410
+ const providerRaw = readConfig(ctx, event.spaceId, "provider");
411
+ const provider: Provider = (PROVIDERS as readonly string[]).includes(
412
+ providerRaw,
413
+ )
414
+ ? (providerRaw as Provider)
415
+ : DEFAULT_PROVIDER;
416
+ const model = readConfig(ctx, event.spaceId, "model") || DEFAULT_MODEL;
417
+ const rawLocalEngine = readConfig(ctx, event.spaceId, "local_engine");
289
418
  const localEngine =
290
419
  rawLocalEngine === "faster_whisper" || rawLocalEngine === "transformers"
291
420
  ? rawLocalEngine
292
421
  : DEFAULT_LOCAL_ENGINE;
422
+ const baseUrl =
423
+ readConfig(ctx, event.spaceId, "base_url") || DEFAULT_OPENAI_BASE_URL;
424
+ const language = readConfig(ctx, event.spaceId, "language");
293
425
 
294
426
  let token: string | undefined;
295
427
  if (provider === "api") {
@@ -301,6 +433,24 @@ export default function (mercury: {
301
433
  );
302
434
  return undefined;
303
435
  }
436
+ } else if (provider === "openai") {
437
+ token = process.env.MERCURY_STT_API_KEY?.trim();
438
+ if (!token) {
439
+ ctx.log.warn(
440
+ "MERCURY_STT_API_KEY not set; skipping voice transcription (openai provider)",
441
+ { extension: EXT },
442
+ );
443
+ return undefined;
444
+ }
445
+ } else if (provider === "gemini") {
446
+ token = process.env.MERCURY_STT_GEMINI_API_KEY?.trim();
447
+ if (!token) {
448
+ ctx.log.warn(
449
+ "MERCURY_STT_GEMINI_API_KEY not set; skipping voice transcription (gemini provider)",
450
+ { extension: EXT },
451
+ );
452
+ return undefined;
453
+ }
304
454
  } else if (provider === "local") {
305
455
  if (!existsSync(SCRIPT_PATH)) {
306
456
  ctx.log.error("Local transcribe script missing", {
@@ -311,6 +461,18 @@ export default function (mercury: {
311
461
  }
312
462
  }
313
463
 
464
+ // The registered `model` default is an HF id for local/api. When the
465
+ // provider is cloud and the model was never changed from that default,
466
+ // use a provider-native model instead of sending the HF id upstream.
467
+ const cloudModel =
468
+ model !== DEFAULT_MODEL
469
+ ? model
470
+ : provider === "openai"
471
+ ? DEFAULT_OPENAI_MODEL
472
+ : provider === "gemini"
473
+ ? DEFAULT_GEMINI_MODEL
474
+ : model;
475
+
314
476
  const lines: string[] = [];
315
477
  const timeoutMs = localTimeoutMs();
316
478
  const pythonBin = defaultPython();
@@ -326,6 +488,23 @@ export default function (mercury: {
326
488
  model,
327
489
  token as string,
328
490
  );
491
+ } else if (provider === "openai") {
492
+ text = await transcribeWithOpenAi(
493
+ hostPath,
494
+ att.mimeType,
495
+ cloudModel,
496
+ baseUrl,
497
+ language,
498
+ token as string,
499
+ );
500
+ } else if (provider === "gemini") {
501
+ text = await transcribeWithGemini(
502
+ hostPath,
503
+ att.mimeType,
504
+ cloudModel,
505
+ language,
506
+ token as string,
507
+ );
329
508
  } else {
330
509
  ctx.log.info("Voice transcription starting (local)", {
331
510
  extension: EXT,
@@ -1,19 +1,23 @@
1
1
  ---
2
2
  name: voice-transcribe
3
- description: Voice notes are transcribed to text before the agent runs (local Python or Hugging Face Inference API).
3
+ description: Voice notes are transcribed to text before the agent runs (local Python, OpenAI-compatible cloud, Gemini, or Hugging Face Inference API).
4
4
  ---
5
5
 
6
6
  # Voice transcription
7
7
 
8
8
  When a user sends a **voice note** or **audio** attachment, Mercury runs the `voice-transcribe` extension **before** the container starts. The transcript is appended to the user message as a `[Voice transcript]` block. You receive normal text — you do not need to read or play audio files for intent.
9
9
 
10
- ## Configuration (per space)
10
+ ## Configuration
11
+
12
+ Keys resolve per space, then fall back to deployment-wide defaults (`@global` scope set from the dashboard Features page, or the `extensions:` section of `mercury.yaml`), then to the extension defaults.
11
13
 
12
14
  | Key | Purpose |
13
15
  |-----|---------|
14
- | `voice-transcribe.provider` | `local` (default) or `api` |
15
- | `voice-transcribe.model` | Hugging Face model id (see below) |
16
+ | `voice-transcribe.provider` | `local` (default), `openai`, `gemini`, or `api` |
17
+ | `voice-transcribe.model` | Model id for the chosen provider (see below) |
16
18
  | `voice-transcribe.local_engine` | Local only: `transformers` (default) or `faster_whisper` (CTranslate2 Hub repos) |
19
+ | `voice-transcribe.base_url` | `openai` only: API root (default `https://api.openai.com/v1`; Groq: `https://api.groq.com/openai/v1`) |
20
+ | `voice-transcribe.language` | Cloud providers: ISO-639-1 hint (e.g. `he`). Improves accuracy on short voice notes |
17
21
 
18
22
  ### Local provider (`local`)
19
23
 
@@ -44,6 +48,20 @@ Hub messages like *Xet Storage… hf_xet* are warnings only, not the cause of fa
44
48
 
45
49
  First run downloads the model into the Hugging Face cache (size depends on the model). **Each voice note spawns a new Python process**, so the first transcription after Mercury starts can be slow while the model loads (no in-process reuse yet).
46
50
 
51
+ ### OpenAI-compatible provider (`openai`)
52
+
53
+ POSTs audio to `{base_url}/audio/transcriptions` (multipart). Requires `MERCURY_STT_API_KEY` on the host (the key never enters agent containers). Works with:
54
+
55
+ - **OpenAI** (default `base_url`) — models `gpt-4o-mini-transcribe` (default), `gpt-4o-transcribe`, `whisper-1`
56
+ - **Groq** — set `voice-transcribe.base_url=https://api.groq.com/openai/v1`, model `whisper-large-v3`
57
+ - Any other host implementing the same endpoint
58
+
59
+ No Python, ffmpeg, or GPU on the host — ideal for small VPS deployments. Set `voice-transcribe.language` (e.g. `he`) for better accuracy on short notes.
60
+
61
+ ### Gemini provider (`gemini`)
62
+
63
+ Sends audio inline to the Gemini API (`generateContent`) with a transcription instruction. Requires `MERCURY_STT_GEMINI_API_KEY` on the host (a plain API key — no GCP project setup; deliberately separate from the model-chain `MERCURY_GEMINI_API_KEY`, which must keep flowing into containers). Default model `gemini-2.5-flash`. Also host-only, no local dependencies.
64
+
47
65
  ### API provider (`api`)
48
66
 
49
67
  POSTs audio to `https://api-inference.huggingface.co/models/<model>`. Requires `MERCURY_HF_TOKEN` on the host. Pick a model that has an [Inference Provider](https://huggingface.co/docs/api-inference) on its Hub page (e.g. `openai/whisper-large-v3`). Hebrew-tuned models such as [mike249/whisper-tiny-he-2](https://huggingface.co/mike249/whisper-tiny-he-2) often have **no** hosted provider — use **`local`** for those. CT2 / Faster-Whisper Hub repos are for the **local** engine only, not this API.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "mercury-agent",
3
- "version": "0.6.0",
3
+ "version": "0.7.0-beta.1",
4
4
  "description": "Personal AI assistant for chat platforms (WhatsApp, Slack, Discord, Telegram)",
5
5
  "license": "MIT",
6
6
  "author": "Avishai Tsabari",
@@ -244,7 +244,9 @@ async function runSingleAgent(
244
244
  };
245
245
  }
246
246
 
247
- const args: string[] = ["--mode", "json", "-p", "--no-session"];
247
+ // --no-approve: the trust store dir (~/.pi/agent) is mounted read-only in
248
+ // containers and any trust-store access acquires a lock there (EROFS crash).
249
+ const args: string[] = ["--mode", "json", "-p", "--no-session", "--no-approve"];
248
250
  if (agent.model) args.push("--model", agent.model);
249
251
  if (agent.tools && agent.tools.length > 0) args.push("--tools", agent.tools.join(","));
250
252
 
@@ -2,24 +2,6 @@
2
2
 
3
3
  You are a helpful AI assistant running inside a chat platform (WhatsApp, Slack, or Discord).
4
4
 
5
- ## Destructive Operations — Confirmation Required
6
-
7
- Before deleting, trashing, or permanently removing any data, you MUST stop and confirm with the user first:
8
-
9
- 1. **List exactly what will be affected** — names, count, and location
10
- 2. **Ask explicitly** — e.g. "This will permanently delete 12 files from your Google Drive. Reply YES to confirm."
11
- 3. **Wait for an unambiguous "yes"** — do not proceed until you have it
12
-
13
- This applies to **all personal data**, regardless of where it lives:
14
- - Connected accounts: Google Drive, Gmail, Google Photos, Yahoo Mail, and any other connected service
15
- - Filesystem: any `rm`, `rmdir`, file deletion, or bulk removal from the user's files
16
-
17
- **Always prefer the reversible option** — move to trash instead of permanent delete, archive instead of delete, move to a folder instead of remove. If the user hasn't explicitly asked for permanent deletion, choose the reversible path by default.
18
-
19
- **Exception**: temp files the agent created during the current task (e.g., scratch files in `/tmp`) may be cleaned up without confirmation.
20
-
21
- This rule applies even when the request implies deletion (e.g., "clean up", "organize", "clear out", "remove duplicates"). When in doubt, ask.
22
-
23
5
  ## Guidelines
24
6
 
25
7
  1. **Be concise** — Chat messages should be readable on mobile
@@ -32,18 +14,6 @@ This rule applies even when the request implies deletion (e.g., "clean up", "org
32
14
  - Running in a container with limited resources
33
15
  - Long-running tasks may time out
34
16
 
35
- ## Presenting tool results
36
-
37
- After running any command or tool, never send raw output to the user. Always translate into plain conversational language before responding.
38
-
39
- - **Names only** — show the human-readable name; never show file IDs, message IDs, or thread IDs
40
- - **Plain types** — say "Google Doc", "spreadsheet", "folder", "PDF"; never show MIME type strings
41
- - **Simple lists** — numbered or bulleted with name + one-word type hint; no tables of raw fields
42
- - **Errors** — explain what went wrong in plain terms; never show exit codes, stack traces, or raw error strings
43
- - **Never show** — JSON blobs, bash code blocks, command flags, or API parameter objects in replies
44
-
45
- This rule applies to all tools: Google Workspace, TradeStation, web search, and any future extension.
46
-
47
17
  ## Mercury Control (mrctl)
48
18
 
49
19
  Full command reference for managing Mercury from inside the container:
@@ -155,22 +125,3 @@ You can delegate tasks to specialized sub-agents:
155
125
 
156
126
  ### Chained Workflow
157
127
  "Use a chain: first have explore find the code, then have worker implement the fix"
158
-
159
- ## Character
160
-
161
- The system prompt may include a "Bot Character" section — the owner-defined voice for
162
- all conversations. Always follow it.
163
-
164
- When a user asks you to change your personality, tone, greeting style, or character:
165
- 1. Read the current character: `mrctl character get`
166
- 2. Draft the FULL updated character text — merge their request with the existing
167
- character into one coherent text. Do not append contradictory fragments.
168
- 3. Show the draft and ask for explicit confirmation.
169
- 4. On confirmation, write the draft to a temp file and run:
170
- `mrctl character set --file <path>`
171
- 5. Relay the result. If the API returns 403, tell the user that only the bot owner
172
- can change the global character.
173
-
174
- Only global admins (configured on the host) can change the character — the API
175
- enforces this. Per-space tone adjustments go in this space's `system_prompt`,
176
- which is set from the dashboard (Spaces settings).
@@ -5,6 +5,8 @@ MERCURY_ANTHROPIC_API_KEY=
5
5
  # MERCURY_GEMINI_API_KEY=
6
6
  # MERCURY_GROQ_API_KEY=
7
7
  # MERCURY_HF_TOKEN= # Hugging Face token (voice-transcribe.provider=api only)
8
+ # MERCURY_STT_API_KEY= # OpenAI/Groq key for voice-transcribe.provider=openai (host-only, never enters containers)
9
+ # MERCURY_STT_GEMINI_API_KEY= # Gemini key for voice-transcribe.provider=gemini (host-only)
8
10
  # MERCURY_VOICE_PYTHON= # Optional: python exe for voice-transcribe.provider=local
9
11
  # MERCURY_VOICE_TRANSCRIBE_TIMEOUT_MS=300000
10
12
 
@@ -83,6 +83,16 @@
83
83
  # default_system_prompt: "" # seeded into auto-created user spaces
84
84
  # default_member_permissions: "prompt,prefs.get" # restrict users to chat only
85
85
 
86
+ # ─── Extension config defaults ──────────────────────────────────────────────
87
+ # Deployment-wide defaults for extension config keys, applied to every space
88
+ # (incl. auto-created DM spaces) unless overridden per-space or in the
89
+ # dashboard's global scope. See docs/configuration.md.
90
+ # extensions:
91
+ # voice-transcribe:
92
+ # provider: openai # local | api | openai | gemini
93
+ # model: gpt-4o-mini-transcribe
94
+ # language: he
95
+
86
96
  # ─── Applicative profile ────────────────────────────────────────────────────
87
97
  # A profile wraps raw capabilities with deterministic business logic and scopes
88
98
  # what members may do. Apply one with `mercury profile apply <name|path|git-url>`