@firstpick/pi-package-webui 0.5.9 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -1
- package/bin/pi-webui.mjs +333 -10
- package/lib/git-worktrees.mjs +5 -1
- package/package.json +2 -2
- package/public/app.js +568 -5
- package/public/index.html +10 -2
- package/public/service-worker.js +2 -1
- package/public/styles.css +158 -4
- package/public/voice-conversation.mjs +466 -0
- package/tests/fixtures/compacting-pi.mjs +0 -0
- package/tests/fixtures/fake-pi.mjs +154 -1
- package/tests/http-endpoints-harness.test.mjs +76 -3
- package/tests/mobile-static.test.mjs +54 -5
- package/tests/voice-conversation.test.mjs +361 -0
package/README.md
CHANGED
|
@@ -127,6 +127,15 @@ Environment variables:
|
|
|
127
127
|
- `PI_WEBUI_FAST_PICKS_FILE=/path/to/paths.json` overrides saved cwd fast-pick storage.
|
|
128
128
|
- `PI_WEBUI_NPM_BIN=/path/to/npm` selects the npm executable used by optional feature install/update actions.
|
|
129
129
|
|
|
130
|
+
Optional Natural Conversation server-side voice fallback variables:
|
|
131
|
+
|
|
132
|
+
- `PI_VOICE_STT_URL` enables a local STT endpoint for `POST /api/stt/transcribe`; Web UI forwards audio as multipart form data.
|
|
133
|
+
- `PI_VOICE_TTS_URL` enables a local TTS endpoint for `POST /api/tts/speech`; Web UI forwards `{ text, voice, format }` JSON.
|
|
134
|
+
- `PI_VOICE_STT_PROVIDER=groq|openai` selects hosted STT when no request provider is supplied; set `GROQ_API_KEY` or `OPENAI_API_KEY` server-side.
|
|
135
|
+
- `PI_VOICE_TTS_PROVIDER=openai` selects hosted OpenAI TTS when no request provider is supplied; set `OPENAI_API_KEY` server-side.
|
|
136
|
+
- `PI_VOICE_GROQ_STT_MODEL`, `PI_VOICE_OPENAI_STT_MODEL`, `PI_VOICE_OPENAI_TTS_MODEL`, `PI_VOICE_OPENAI_TTS_VOICE`, `PI_VOICE_OPENAI_TTS_FORMAT`, `PI_VOICE_TTS_VOICE`, and `PI_VOICE_TTS_FORMAT` tune fallback models/voices/formats.
|
|
137
|
+
- `PI_VOICE_PROVIDER_TIMEOUT_MS` controls outbound fallback-provider timeout. API keys are never sent from the browser; remote/LAN raw-audio STT uploads must include explicit per-request microphone-streaming consent.
|
|
138
|
+
|
|
130
139
|
## Main features
|
|
131
140
|
|
|
132
141
|
- Pathless `pi-webui` startup: the server opens first, then the browser prompts for the first terminal CWD.
|
|
@@ -136,7 +145,8 @@ Environment variables:
|
|
|
136
145
|
- Streaming chat transcript with Markdown, copy buttons for fenced code blocks, rendered Mermaid diagrams from fenced `mermaid`/`mmd` code blocks, thinking output, tool/bash cards, queue and compaction events, edit-and-retry from user prompts, transcript search, copy buttons, and guarded abort controls that require holding Esc or the Abort button for 3 seconds.
|
|
137
146
|
- Prompt composer with uploads, drag/drop/paste, inline image support, generated text attachments for long input or clipboard text, editable text attachments, slash-command autocomplete, and `@` file/path references with live suggestions.
|
|
138
147
|
- Leading `!` and `!!` user-bash commands from the composer, serialized per tab; `!` keeps output in the next model context and `!!` excludes it.
|
|
139
|
-
- Optional Natural Conversation Mode shell for the standalone `@firstpick/pi-package-natural-conversation` package: when `/talk` (or `/voice`/`/conversation`) is loaded in the active Pi tab, Web UI shows per-tab Start/End controls, a read-only voice-mode chip, and backend guards that keep thinking `off` while blocking unsafe Web UI actions.
|
|
148
|
+
- Optional Natural Conversation Mode shell for the standalone `@firstpick/pi-package-natural-conversation` package: when `/talk` (or `/voice`/`/conversation`) is loaded in the active Pi tab, Web UI shows per-tab Start/End controls, a read-only voice-mode chip, and backend guards that keep thinking `off` while blocking unsafe Web UI actions.
|
|
149
|
+
- Browser voice loop for Natural Conversation Mode (`public/voice-conversation.mjs`): while the mode is active in a tab, the browser's Web Speech APIs listen for speech, send final transcripts as normal prompts, and speak Pi's final answers. The microphone pauses while answers are spoken (echo prevention), speech during final-output streaming becomes a steering interruption, speech during tool execution is queued until the tool phase ends, and silence after a spoken question sends a single structured silence event. Remote (non-localhost) sessions keep the microphone off until the per-tab `Allow remote microphone streaming` consent is granted; only text transcripts ever reach the Pi host on the browser-default path. Opt-in server-side fallback routes are available for local/Groq/OpenAI STT and local/OpenAI TTS when configured with server-side env vars; remote/LAN raw-audio STT fallback uploads require explicit per-request consent.
|
|
140
150
|
- Browser-native Pi dialogs for `/model`, `/settings`, `/theme`, `/fork`, `/clone`, `/name`, `/resume`, `/tree`, `/login`, `/logout`, `/scoped-models`, `/tools`, and `/skills`, plus native-command adapter output for `/copy`, `/session`, `/new`, `/compact`, `/reload`, and `/export`.
|
|
141
151
|
- Runtime `/tools` and `/skills` selectors backed by the hidden Web UI RPC helper; skill toggles persist on the session branch, disabled skills are removed from the system prompt, and tracked `SKILL.md` files can be opened/edited from skill tags.
|
|
142
152
|
- Session resume/switch, metadata rename, and localhost-only safe delete with active/open-tab/session-directory guards.
|
package/bin/pi-webui.mjs
CHANGED
|
@@ -91,6 +91,10 @@ const OPENAI_CODEX_USAGE_ENDPOINT = process.env.PI_WEBUI_CODEX_USAGE_URL || "htt
|
|
|
91
91
|
const BODY_LIMIT_BYTES = 1024 * 1024;
|
|
92
92
|
const SKILL_FILE_BODY_LIMIT_BYTES = 2 * 1024 * 1024;
|
|
93
93
|
const PROMPT_BODY_LIMIT_BYTES = 24 * 1024 * 1024;
|
|
94
|
+
const VOICE_AUDIO_BODY_LIMIT_BYTES = 24 * 1024 * 1024;
|
|
95
|
+
const VOICE_AUDIO_JSON_BODY_LIMIT_BYTES = Math.ceil(VOICE_AUDIO_BODY_LIMIT_BYTES * 1.4) + 1024 * 1024;
|
|
96
|
+
const VOICE_PROVIDER_TIMEOUT_MS = Math.max(1000, Number.parseInt(process.env.PI_VOICE_PROVIDER_TIMEOUT_MS || "120000", 10) || 120000);
|
|
97
|
+
const VOICE_TTS_TEXT_MAX_CHARS = 12_000;
|
|
94
98
|
const UPLOAD_BODY_LIMIT_BYTES = 96 * 1024 * 1024;
|
|
95
99
|
const ATTACHMENT_UPLOAD_MAX_FILES = 12;
|
|
96
100
|
const ATTACHMENT_UPLOAD_MAX_FILE_BYTES = 64 * 1024 * 1024;
|
|
@@ -1114,7 +1118,10 @@ function runCommand(command, args, { cwd, timeoutMs = 2000, maxOutputLength = 20
|
|
|
1114
1118
|
if (stderr.length > maxOutputLength) stderr = stderr.slice(-maxOutputLength);
|
|
1115
1119
|
});
|
|
1116
1120
|
child.on("error", (error) => finish({ exitCode: undefined, stdout, stderr: sanitizeError(error), error: sanitizeError(error) }));
|
|
1117
|
-
|
|
1121
|
+
// "close", not "exit": exit can fire before the stdio pipes flush, which
|
|
1122
|
+
// intermittently truncates stdout (empty `git rev-parse --show-toplevel`
|
|
1123
|
+
// output resolves to process.cwd() and reads the wrong repository).
|
|
1124
|
+
child.on("close", (exitCode) => finish({ exitCode, stdout, stderr, timedOut: false }));
|
|
1118
1125
|
});
|
|
1119
1126
|
}
|
|
1120
1127
|
|
|
@@ -4321,7 +4328,9 @@ function runWorkflowCommand(command, args, { cwd, label = formatWorkflowCommand(
|
|
|
4321
4328
|
if (stderr.length > 100000) stderr = stderr.slice(-100000);
|
|
4322
4329
|
});
|
|
4323
4330
|
child.on("error", (error) => finish({ exitCode: undefined, stderr: stderr || sanitizeError(error), error: sanitizeError(error) }));
|
|
4324
|
-
|
|
4331
|
+
// "close", not "exit": exit can fire before the stdio pipes flush,
|
|
4332
|
+
// intermittently truncating collected stdout/stderr.
|
|
4333
|
+
child.on("close", (exitCode, signal) => finish({ exitCode, signal }));
|
|
4325
4334
|
});
|
|
4326
4335
|
}
|
|
4327
4336
|
|
|
@@ -4538,7 +4547,7 @@ async function readBundledThemes() {
|
|
|
4538
4547
|
function normalizeStaticPath(urlPath) {
|
|
4539
4548
|
if (urlPath === "/") return "index.html";
|
|
4540
4549
|
const name = urlPath.startsWith("/") ? urlPath.slice(1) : urlPath;
|
|
4541
|
-
if (!["index.html", "app.js", "styles.css", "favicon.svg", "apple-touch-icon.png", "icon-192.png", "icon-512.png", "catppuccin-mocha-background.png", "matrix-background.webp", "manifest.webmanifest", "service-worker.js"].includes(name)) return undefined;
|
|
4550
|
+
if (!["index.html", "app.js", "voice-conversation.mjs", "styles.css", "favicon.svg", "apple-touch-icon.png", "icon-192.png", "icon-512.png", "catppuccin-mocha-background.png", "matrix-background.webp", "manifest.webmanifest", "service-worker.js"].includes(name)) return undefined;
|
|
4542
4551
|
return name;
|
|
4543
4552
|
}
|
|
4544
4553
|
|
|
@@ -7107,11 +7116,12 @@ function rememberNaturalConversationCommands(tab, commands = []) {
|
|
|
7107
7116
|
return tab.conversationMode;
|
|
7108
7117
|
}
|
|
7109
7118
|
|
|
7110
|
-
async function getCommandData(tab) {
|
|
7119
|
+
async function getCommandData(tab, { annotateSkills = true } = {}) {
|
|
7111
7120
|
try {
|
|
7112
7121
|
const response = await tab.rpc.send({ type: "get_commands" });
|
|
7113
7122
|
if (response.success === false) throw makeHttpError(400, response.error || "failed to load commands");
|
|
7114
|
-
const
|
|
7123
|
+
const rawCommands = (response.data?.commands || []).filter((command) => command?.name !== WEBUI_HELPER_COMMAND);
|
|
7124
|
+
const rpcCommands = annotateSkills ? await annotateSkillCommandState(tab, rawCommands) : rawCommands;
|
|
7115
7125
|
rememberNaturalConversationCommands(tab, rpcCommands);
|
|
7116
7126
|
return { commands: [...NATIVE_SLASH_COMMANDS, ...rpcCommands], rpcRunning: true };
|
|
7117
7127
|
} catch (error) {
|
|
@@ -7135,7 +7145,7 @@ async function naturalConversationFeatureData(tab, { refreshCommands = true } =
|
|
|
7135
7145
|
let commandError = "";
|
|
7136
7146
|
if (refreshCommands) {
|
|
7137
7147
|
try {
|
|
7138
|
-
commandData = await getCommandData(tab);
|
|
7148
|
+
commandData = await getCommandData(tab, { annotateSkills: false });
|
|
7139
7149
|
} catch (error) {
|
|
7140
7150
|
commandError = sanitizeError(error);
|
|
7141
7151
|
rememberNaturalConversationCommands(tab, []);
|
|
@@ -7186,15 +7196,297 @@ async function setNaturalConversationMode(tab, body = {}) {
|
|
|
7186
7196
|
return { ...(await naturalConversationFeatureData(tab, { refreshCommands: false })), response };
|
|
7187
7197
|
}
|
|
7188
7198
|
|
|
7189
|
-
|
|
7199
|
+
// Piper voice switching for the native /talk audio loop. The WebUI never
|
|
7200
|
+
// imports the natural-conversation package; it mirrors the package's small
|
|
7201
|
+
// voice catalog for display, reads voice.json / the piper voice dir for
|
|
7202
|
+
// state, and performs the actual switch (incl. download) by sending
|
|
7203
|
+
// `/talk voice <id>` over RPC so the package stays the single writer.
|
|
7204
|
+
const CONVERSATION_VOICE_CATALOG = [
|
|
7205
|
+
{ id: "en_US-lessac-medium", file: "en_US-lessac-medium.onnx", sizeMb: 63, note: "natural US English" },
|
|
7206
|
+
{ id: "en_GB-alba-medium", file: "en_GB-alba-medium.onnx", sizeMb: 63, note: "natural British English" },
|
|
7207
|
+
{ id: "de_DE-thorsten-medium", file: "de_DE-thorsten-medium.onnx", sizeMb: 63, note: "natural German" },
|
|
7208
|
+
{ id: "de_DE-thorsten-high", file: "de_DE-thorsten-high.onnx", sizeMb: 110, note: "German, best quality" },
|
|
7209
|
+
];
|
|
7210
|
+
const CONVERSATION_VOICE_SWITCH_TIMEOUT_MS = 10 * 60 * 1000; // downloads can take a while
|
|
7211
|
+
|
|
7212
|
+
function conversationVoiceConfigPath() {
|
|
7213
|
+
const override = String(process.env.PI_VOICE_CONFIG_PATH || "").trim();
|
|
7214
|
+
return override || path.join(homedir(), ".pi", "agent", "voice.json");
|
|
7215
|
+
}
|
|
7216
|
+
|
|
7217
|
+
function conversationVoiceDirs() {
|
|
7218
|
+
const data = String(process.env.XDG_DATA_HOME || "").trim() || path.join(homedir(), ".local", "share");
|
|
7219
|
+
return [path.join(data, "piper"), path.join(data, "piper", "voices"), path.join(data, "piper-voices")];
|
|
7220
|
+
}
|
|
7221
|
+
|
|
7222
|
+
async function conversationVoicesData() {
|
|
7223
|
+
let ttsProvider = null;
|
|
7224
|
+
let currentModelPath = null;
|
|
7225
|
+
try {
|
|
7226
|
+
const config = JSON.parse(await readFile(conversationVoiceConfigPath(), "utf8"));
|
|
7227
|
+
ttsProvider = config?.native?.tts?.provider ?? null;
|
|
7228
|
+
if (ttsProvider === "piper") currentModelPath = config?.native?.tts?.modelPath ?? null;
|
|
7229
|
+
} catch {
|
|
7230
|
+
// no voice.json yet — the dropdown still lists the catalog
|
|
7231
|
+
}
|
|
7232
|
+
|
|
7233
|
+
const onDisk = new Map();
|
|
7234
|
+
for (const dir of conversationVoiceDirs()) {
|
|
7235
|
+
let entries = [];
|
|
7236
|
+
try {
|
|
7237
|
+
entries = await readdir(dir);
|
|
7238
|
+
} catch {
|
|
7239
|
+
continue;
|
|
7240
|
+
}
|
|
7241
|
+
for (const entry of entries) {
|
|
7242
|
+
if (!entry.endsWith(".onnx") || onDisk.has(entry)) continue;
|
|
7243
|
+
const modelPath = path.join(dir, entry);
|
|
7244
|
+
const sidecar = await stat(`${modelPath}.json`).catch(() => null);
|
|
7245
|
+
if (!sidecar) continue;
|
|
7246
|
+
const size = await stat(modelPath).catch(() => null);
|
|
7247
|
+
onDisk.set(entry, { path: modelPath, sizeMb: size ? Math.round(size.size / (1024 * 1024)) : 0 });
|
|
7248
|
+
}
|
|
7249
|
+
}
|
|
7250
|
+
|
|
7251
|
+
const voices = CONVERSATION_VOICE_CATALOG.map((entry) => ({
|
|
7252
|
+
id: entry.id,
|
|
7253
|
+
sizeMb: entry.sizeMb,
|
|
7254
|
+
note: entry.note,
|
|
7255
|
+
downloaded: onDisk.has(entry.file),
|
|
7256
|
+
current: Boolean(currentModelPath && currentModelPath.endsWith(`/${entry.file}`)),
|
|
7257
|
+
}));
|
|
7258
|
+
for (const [file, info] of onDisk) {
|
|
7259
|
+
if (CONVERSATION_VOICE_CATALOG.some((entry) => entry.file === file)) continue;
|
|
7260
|
+
voices.push({
|
|
7261
|
+
id: file.replace(/\.onnx$/, ""),
|
|
7262
|
+
sizeMb: info.sizeMb,
|
|
7263
|
+
note: "found on disk",
|
|
7264
|
+
downloaded: true,
|
|
7265
|
+
current: info.path === currentModelPath,
|
|
7266
|
+
});
|
|
7267
|
+
}
|
|
7268
|
+
return { voices, current: voices.find((voice) => voice.current)?.id ?? null, ttsProvider };
|
|
7269
|
+
}
|
|
7270
|
+
|
|
7271
|
+
async function setConversationVoice(tab, body = {}) {
|
|
7272
|
+
const voiceId = String(body.voice || "").trim();
|
|
7273
|
+
if (!/^[\w.-]{1,120}$/.test(voiceId)) throw makeHttpError(400, "voice must be a Piper voice id (letters, digits, dot, dash, underscore)");
|
|
7274
|
+
const feature = await naturalConversationFeatureData(tab);
|
|
7275
|
+
if (!feature.available) throw makeHttpError(404, feature.unavailableReason);
|
|
7276
|
+
const commandName = feature.commands.find((name) => name === "talk") || feature.commands[0] || "talk";
|
|
7277
|
+
const response = await tab.rpc.send({ type: "prompt", message: `/${commandName} voice ${voiceId}` }, CONVERSATION_VOICE_SWITCH_TIMEOUT_MS);
|
|
7278
|
+
if (response.success === false) throw makeHttpError(400, response.error || `Failed to switch the conversation voice to ${voiceId}`);
|
|
7279
|
+
return { ...(await conversationVoicesData()), response };
|
|
7280
|
+
}
|
|
7281
|
+
|
|
7282
|
+
const VOICE_AUDIO_MIME_TYPES = new Set(["audio/webm", "audio/wav", "audio/x-wav", "audio/mpeg", "audio/mp3", "audio/mp4", "audio/ogg", "audio/flac", "application/octet-stream"]);
|
|
7283
|
+
const VOICE_STT_PROVIDER_IDS = ["local", "groq", "openai", "cloudflare"];
|
|
7284
|
+
const VOICE_TTS_PROVIDER_IDS = ["local", "openai"];
|
|
7285
|
+
|
|
7286
|
+
function safeVoiceEndpointLabel(value) {
|
|
7287
|
+
try {
|
|
7288
|
+
const parsed = new URL(String(value || ""));
|
|
7289
|
+
return `${parsed.origin}${parsed.pathname}`;
|
|
7290
|
+
} catch {
|
|
7291
|
+
return "";
|
|
7292
|
+
}
|
|
7293
|
+
}
|
|
7294
|
+
|
|
7295
|
+
function naturalConversationVoiceProviderStatus(kind = "stt") {
|
|
7296
|
+
const stt = kind === "stt";
|
|
7297
|
+
const localUrl = stt ? process.env.PI_VOICE_STT_URL : process.env.PI_VOICE_TTS_URL;
|
|
7298
|
+
const providers = [
|
|
7299
|
+
{
|
|
7300
|
+
id: "local",
|
|
7301
|
+
label: stt ? "Local STT endpoint" : "Local TTS endpoint",
|
|
7302
|
+
configured: !!localUrl,
|
|
7303
|
+
env: stt ? ["PI_VOICE_STT_URL"] : ["PI_VOICE_TTS_URL"],
|
|
7304
|
+
endpoint: safeVoiceEndpointLabel(localUrl),
|
|
7305
|
+
},
|
|
7306
|
+
];
|
|
7307
|
+
if (stt) {
|
|
7308
|
+
providers.push(
|
|
7309
|
+
{ id: "groq", label: "Groq Whisper", configured: !!process.env.GROQ_API_KEY, env: ["GROQ_API_KEY"], defaultModel: process.env.PI_VOICE_GROQ_STT_MODEL || "whisper-large-v3-turbo" },
|
|
7310
|
+
{ id: "openai", label: "OpenAI transcription", configured: !!process.env.OPENAI_API_KEY, env: ["OPENAI_API_KEY"], defaultModel: process.env.PI_VOICE_OPENAI_STT_MODEL || "gpt-4o-mini-transcribe" },
|
|
7311
|
+
{ id: "cloudflare", label: "Cloudflare Workers AI Whisper", configured: !!(process.env.CLOUDFLARE_API_TOKEN && process.env.CLOUDFLARE_ACCOUNT_ID), env: ["CLOUDFLARE_API_TOKEN", "CLOUDFLARE_ACCOUNT_ID"], deferred: true },
|
|
7312
|
+
);
|
|
7313
|
+
} else {
|
|
7314
|
+
providers.push({ id: "openai", label: "OpenAI speech", configured: !!process.env.OPENAI_API_KEY, env: ["OPENAI_API_KEY"], defaultModel: process.env.PI_VOICE_OPENAI_TTS_MODEL || "gpt-4o-mini-tts" });
|
|
7315
|
+
}
|
|
7316
|
+
return {
|
|
7317
|
+
kind: stt ? "stt" : "tts",
|
|
7318
|
+
providers,
|
|
7319
|
+
defaultProvider: (stt ? process.env.PI_VOICE_STT_PROVIDER : process.env.PI_VOICE_TTS_PROVIDER) || (localUrl ? "local" : ""),
|
|
7320
|
+
remoteConsent: stt ? "Remote WebUI raw-audio uploads require per-request remoteMicStreamingConsentAccepted=true; browser Web Speech does not use this route." : "Server-side TTS sends answer text to the selected provider only when this route is explicitly used.",
|
|
7321
|
+
};
|
|
7322
|
+
}
|
|
7323
|
+
|
|
7324
|
+
function naturalConversationUnavailableResponse(tab, kind = "stt") {
|
|
7190
7325
|
return {
|
|
7191
7326
|
featureId: NATURAL_CONVERSATION_FEATURE_ID,
|
|
7192
7327
|
available: false,
|
|
7193
7328
|
mode: naturalConversationModeSnapshot(tab, { available: false, enabled: false, uiState: "off" }),
|
|
7194
|
-
|
|
7329
|
+
voice: naturalConversationVoiceProviderStatus(kind),
|
|
7330
|
+
message: "Natural Conversation server-side voice fallbacks are opt-in. Configure a local endpoint or explicit hosted provider env vars; browser-native audio remains the default Web UI path.",
|
|
7195
7331
|
};
|
|
7196
7332
|
}
|
|
7197
7333
|
|
|
7334
|
+
function naturalConversationProviderNotConfiguredError(kind, provider, message) {
|
|
7335
|
+
const error = makeHttpError(provider ? 424 : 501, message);
|
|
7336
|
+
error.voice = naturalConversationVoiceProviderStatus(kind);
|
|
7337
|
+
return error;
|
|
7338
|
+
}
|
|
7339
|
+
|
|
7340
|
+
function requestedNaturalConversationVoiceProvider(kind, body = {}) {
|
|
7341
|
+
const requested = String(body.provider || body.providerId || (kind === "stt" ? process.env.PI_VOICE_STT_PROVIDER : process.env.PI_VOICE_TTS_PROVIDER) || "").trim().toLowerCase();
|
|
7342
|
+
const ids = kind === "stt" ? VOICE_STT_PROVIDER_IDS : VOICE_TTS_PROVIDER_IDS;
|
|
7343
|
+
const localUrl = kind === "stt" ? process.env.PI_VOICE_STT_URL : process.env.PI_VOICE_TTS_URL;
|
|
7344
|
+
const provider = requested || (localUrl ? "local" : "");
|
|
7345
|
+
if (!provider) throw naturalConversationProviderNotConfiguredError(kind, "", `No Natural Conversation ${kind.toUpperCase()} provider is configured. Set ${kind === "stt" ? "PI_VOICE_STT_URL or PI_VOICE_STT_PROVIDER" : "PI_VOICE_TTS_URL or PI_VOICE_TTS_PROVIDER"}.`);
|
|
7346
|
+
if (!ids.includes(provider)) throw makeHttpError(400, `Unsupported Natural Conversation ${kind.toUpperCase()} provider: ${provider}`);
|
|
7347
|
+
if (provider === "local" && !localUrl) throw naturalConversationProviderNotConfiguredError(kind, provider, `Local Natural Conversation ${kind.toUpperCase()} endpoint is not configured. Set ${kind === "stt" ? "PI_VOICE_STT_URL" : "PI_VOICE_TTS_URL"}.`);
|
|
7348
|
+
if (provider === "groq" && !process.env.GROQ_API_KEY) throw naturalConversationProviderNotConfiguredError(kind, provider, "Groq STT is not configured. Set GROQ_API_KEY server-side; never send it from the browser.");
|
|
7349
|
+
if (provider === "openai" && !process.env.OPENAI_API_KEY) throw naturalConversationProviderNotConfiguredError(kind, provider, `OpenAI ${kind.toUpperCase()} is not configured. Set OPENAI_API_KEY server-side; never send it from the browser.`);
|
|
7350
|
+
if (provider === "cloudflare") throw makeHttpError(501, "Cloudflare Workers AI STT is a selected Phase 4 provider but its adapter is deferred until the exact upload contract is validated; use local, Groq, or OpenAI for this slice.");
|
|
7351
|
+
return provider;
|
|
7352
|
+
}
|
|
7353
|
+
|
|
7354
|
+
function requireRemoteMicConsentForStt(req, body = {}) {
|
|
7355
|
+
if (isLocalRequest(req)) return;
|
|
7356
|
+
if (body.remoteMicStreamingConsentAccepted === true || body.remoteMicStreamingConsent === true) return;
|
|
7357
|
+
throw makeHttpError(403, "Remote WebUI STT uploads require explicit per-tab remote microphone streaming consent before raw audio is sent to the Pi host or an STT provider.");
|
|
7358
|
+
}
|
|
7359
|
+
|
|
7360
|
+
function decodeVoiceAudioBody(body = {}) {
|
|
7361
|
+
const value = String(body.audioBase64 || body.audio || body.data || "").trim();
|
|
7362
|
+
if (!value) throw makeHttpError(400, "audioBase64 is required for Natural Conversation STT fallback uploads");
|
|
7363
|
+
const dataUrlMatch = value.match(/^data:([^;,]+);base64,(.*)$/s);
|
|
7364
|
+
const mimeType = String(body.mimeType || body.contentType || dataUrlMatch?.[1] || "audio/webm").toLowerCase();
|
|
7365
|
+
if (!VOICE_AUDIO_MIME_TYPES.has(mimeType)) throw makeHttpError(415, `Unsupported voice audio mime type: ${mimeType}`);
|
|
7366
|
+
const base64 = (dataUrlMatch?.[2] || value).replace(/\s+/g, "");
|
|
7367
|
+
if (!/^[A-Za-z0-9+/]*={0,2}$/.test(base64)) throw makeHttpError(400, "audioBase64 must be valid base64");
|
|
7368
|
+
const buffer = Buffer.from(base64, "base64");
|
|
7369
|
+
if (!buffer.length) throw makeHttpError(400, "audioBase64 decoded to an empty audio payload");
|
|
7370
|
+
if (buffer.length > VOICE_AUDIO_BODY_LIMIT_BYTES) throw makeHttpError(413, `Audio payload is too large (limit ${formatBytes(VOICE_AUDIO_BODY_LIMIT_BYTES)})`);
|
|
7371
|
+
return {
|
|
7372
|
+
buffer,
|
|
7373
|
+
mimeType,
|
|
7374
|
+
fileName: String(body.fileName || `speech.${mimeType.includes("wav") ? "wav" : mimeType.includes("mpeg") || mimeType.includes("mp3") ? "mp3" : "webm"}`).replace(/[\\/\0]/g, "_").slice(0, 120),
|
|
7375
|
+
};
|
|
7376
|
+
}
|
|
7377
|
+
|
|
7378
|
+
function voiceFetchError(provider, response, text) {
|
|
7379
|
+
const status = response.status >= 500 ? 502 : 400;
|
|
7380
|
+
return makeHttpError(status, `${provider} voice provider returned HTTP ${response.status}: ${truncateStatusText(text || response.statusText || "request failed", 500)}`);
|
|
7381
|
+
}
|
|
7382
|
+
|
|
7383
|
+
async function fetchVoiceProvider(provider, url, init = {}) {
|
|
7384
|
+
const response = await fetch(url, { ...init, signal: AbortSignal.timeout(VOICE_PROVIDER_TIMEOUT_MS) });
|
|
7385
|
+
if (!response.ok) throw voiceFetchError(provider, response, await response.text().catch(() => ""));
|
|
7386
|
+
return response;
|
|
7387
|
+
}
|
|
7388
|
+
|
|
7389
|
+
function voiceFormData(audio, body = {}, model = "") {
|
|
7390
|
+
const form = new FormData();
|
|
7391
|
+
form.set("file", new Blob([audio.buffer], { type: audio.mimeType }), audio.fileName);
|
|
7392
|
+
if (model) form.set("model", model);
|
|
7393
|
+
if (body.language) form.set("language", String(body.language));
|
|
7394
|
+
if (body.prompt) form.set("prompt", String(body.prompt));
|
|
7395
|
+
return form;
|
|
7396
|
+
}
|
|
7397
|
+
|
|
7398
|
+
function transcriptTextFromProviderJson(json) {
|
|
7399
|
+
return String(json?.text || json?.transcript || json?.data?.text || json?.result?.text || "").trim();
|
|
7400
|
+
}
|
|
7401
|
+
|
|
7402
|
+
async function parseTranscriptResponse(provider, response) {
|
|
7403
|
+
const contentType = String(response.headers.get("content-type") || "").toLowerCase();
|
|
7404
|
+
if (contentType.includes("application/json")) {
|
|
7405
|
+
const json = await response.json();
|
|
7406
|
+
const text = transcriptTextFromProviderJson(json);
|
|
7407
|
+
if (!text) throw makeHttpError(502, `${provider} STT provider returned JSON without a transcript text field`);
|
|
7408
|
+
return text;
|
|
7409
|
+
}
|
|
7410
|
+
const text = (await response.text()).trim();
|
|
7411
|
+
if (!text) throw makeHttpError(502, `${provider} STT provider returned an empty transcript`);
|
|
7412
|
+
return text;
|
|
7413
|
+
}
|
|
7414
|
+
|
|
7415
|
+
async function transcribeWithLocalProvider(audio, body = {}) {
|
|
7416
|
+
const form = voiceFormData(audio, body, body.model ? String(body.model) : "");
|
|
7417
|
+
const response = await fetchVoiceProvider("local STT", process.env.PI_VOICE_STT_URL, { method: "POST", body: form });
|
|
7418
|
+
return parseTranscriptResponse("local", response);
|
|
7419
|
+
}
|
|
7420
|
+
|
|
7421
|
+
async function transcribeWithOpenAiCompatibleProvider(provider, audio, body = {}) {
|
|
7422
|
+
const openai = provider === "openai";
|
|
7423
|
+
const url = openai ? "https://api.openai.com/v1/audio/transcriptions" : "https://api.groq.com/openai/v1/audio/transcriptions";
|
|
7424
|
+
const model = String(body.model || (openai ? process.env.PI_VOICE_OPENAI_STT_MODEL || "gpt-4o-mini-transcribe" : process.env.PI_VOICE_GROQ_STT_MODEL || "whisper-large-v3-turbo"));
|
|
7425
|
+
const form = voiceFormData(audio, body, model);
|
|
7426
|
+
form.set("response_format", "json");
|
|
7427
|
+
const response = await fetchVoiceProvider(`${provider} STT`, url, {
|
|
7428
|
+
method: "POST",
|
|
7429
|
+
headers: { authorization: `Bearer ${openai ? process.env.OPENAI_API_KEY : process.env.GROQ_API_KEY}` },
|
|
7430
|
+
body: form,
|
|
7431
|
+
});
|
|
7432
|
+
return parseTranscriptResponse(provider, response);
|
|
7433
|
+
}
|
|
7434
|
+
|
|
7435
|
+
async function handleNaturalConversationSttTranscribe(req, tab, body = {}) {
|
|
7436
|
+
requireRemoteMicConsentForStt(req, body);
|
|
7437
|
+
const provider = requestedNaturalConversationVoiceProvider("stt", body);
|
|
7438
|
+
const audio = decodeVoiceAudioBody(body);
|
|
7439
|
+
const text = provider === "local" ? await transcribeWithLocalProvider(audio, body) : await transcribeWithOpenAiCompatibleProvider(provider, audio, body);
|
|
7440
|
+
return { provider, text, mimeType: audio.mimeType, byteLength: audio.buffer.length, mode: naturalConversationModeSnapshot(tab) };
|
|
7441
|
+
}
|
|
7442
|
+
|
|
7443
|
+
function normalizeVoiceText(body = {}) {
|
|
7444
|
+
const text = String(body.text || body.input || "").trim();
|
|
7445
|
+
if (!text) throw makeHttpError(400, "text is required for Natural Conversation TTS fallback synthesis");
|
|
7446
|
+
if (text.length > VOICE_TTS_TEXT_MAX_CHARS) throw makeHttpError(413, `TTS text is too long (limit ${VOICE_TTS_TEXT_MAX_CHARS} characters)`);
|
|
7447
|
+
return text;
|
|
7448
|
+
}
|
|
7449
|
+
|
|
7450
|
+
async function audioPayloadFromResponse(provider, response, preferredFormat = "") {
|
|
7451
|
+
const contentType = String(response.headers.get("content-type") || "").toLowerCase();
|
|
7452
|
+
if (contentType.includes("application/json")) {
|
|
7453
|
+
const json = await response.json();
|
|
7454
|
+
const audioBase64 = String(json.audioBase64 || json.audio || json.data?.audioBase64 || "").trim();
|
|
7455
|
+
if (!audioBase64) throw makeHttpError(502, `${provider} TTS provider returned JSON without audioBase64`);
|
|
7456
|
+
const buffer = Buffer.from(audioBase64, "base64");
|
|
7457
|
+
return { audioBase64, contentType: json.mimeType || json.contentType || "audio/mpeg", format: json.format || preferredFormat || "mp3", byteLength: buffer.length };
|
|
7458
|
+
}
|
|
7459
|
+
const buffer = Buffer.from(await response.arrayBuffer());
|
|
7460
|
+
if (!buffer.length) throw makeHttpError(502, `${provider} TTS provider returned empty audio`);
|
|
7461
|
+
return { audioBase64: buffer.toString("base64"), contentType: contentType || "audio/mpeg", format: preferredFormat || (contentType.includes("wav") ? "wav" : "mp3"), byteLength: buffer.length };
|
|
7462
|
+
}
|
|
7463
|
+
|
|
7464
|
+
async function synthesizeWithLocalProvider(text, body = {}) {
|
|
7465
|
+
const response = await fetchVoiceProvider("local TTS", process.env.PI_VOICE_TTS_URL, {
|
|
7466
|
+
method: "POST",
|
|
7467
|
+
headers: { "content-type": "application/json" },
|
|
7468
|
+
body: JSON.stringify({ text, voice: body.voice || process.env.PI_VOICE_TTS_VOICE || undefined, format: body.format || process.env.PI_VOICE_TTS_FORMAT || "mp3" }),
|
|
7469
|
+
});
|
|
7470
|
+
return audioPayloadFromResponse("local", response, body.format || process.env.PI_VOICE_TTS_FORMAT || "mp3");
|
|
7471
|
+
}
|
|
7472
|
+
|
|
7473
|
+
async function synthesizeWithOpenAiProvider(text, body = {}) {
|
|
7474
|
+
const format = String(body.format || process.env.PI_VOICE_OPENAI_TTS_FORMAT || "mp3");
|
|
7475
|
+
const response = await fetchVoiceProvider("openai TTS", "https://api.openai.com/v1/audio/speech", {
|
|
7476
|
+
method: "POST",
|
|
7477
|
+
headers: { authorization: `Bearer ${process.env.OPENAI_API_KEY}`, "content-type": "application/json" },
|
|
7478
|
+
body: JSON.stringify({ model: body.model || process.env.PI_VOICE_OPENAI_TTS_MODEL || "gpt-4o-mini-tts", voice: body.voice || process.env.PI_VOICE_OPENAI_TTS_VOICE || "alloy", input: text, response_format: format }),
|
|
7479
|
+
});
|
|
7480
|
+
return audioPayloadFromResponse("openai", response, format);
|
|
7481
|
+
}
|
|
7482
|
+
|
|
7483
|
+
async function handleNaturalConversationTtsSpeech(_req, tab, body = {}) {
|
|
7484
|
+
const provider = requestedNaturalConversationVoiceProvider("tts", body);
|
|
7485
|
+
const text = normalizeVoiceText(body);
|
|
7486
|
+
const audio = provider === "local" ? await synthesizeWithLocalProvider(text, body) : await synthesizeWithOpenAiProvider(text, body);
|
|
7487
|
+
return { provider, ...audio, mode: naturalConversationModeSnapshot(tab) };
|
|
7488
|
+
}
|
|
7489
|
+
|
|
7198
7490
|
function resolveCliPath(value) {
|
|
7199
7491
|
const text = String(value || "").trim();
|
|
7200
7492
|
if (!text) return "";
|
|
@@ -8678,9 +8970,40 @@ const server = createServer(async (req, res) => {
|
|
|
8678
8970
|
return;
|
|
8679
8971
|
}
|
|
8680
8972
|
|
|
8681
|
-
if (
|
|
8973
|
+
if (url.pathname === "/api/conversation-voices" && req.method === "GET") {
|
|
8682
8974
|
const tab = getRequestedTab(req, url);
|
|
8683
|
-
sendJson(res,
|
|
8975
|
+
sendJson(res, 200, { ok: true, data: await conversationVoicesData(), tab: tabMeta(tab) });
|
|
8976
|
+
return;
|
|
8977
|
+
}
|
|
8978
|
+
|
|
8979
|
+
if (url.pathname === "/api/conversation-voice" && req.method === "POST") {
|
|
8980
|
+
const body = await readJsonBody(req);
|
|
8981
|
+
const tab = getRequestedTab(req, url, body);
|
|
8982
|
+
sendJson(res, 200, { ok: true, data: await setConversationVoice(tab, body), tab: tabMeta(tab) });
|
|
8983
|
+
return;
|
|
8984
|
+
}
|
|
8985
|
+
|
|
8986
|
+
if (url.pathname === "/api/stt/transcribe" && req.method === "POST") {
|
|
8987
|
+
const body = await readJsonBody(req, { limitBytes: VOICE_AUDIO_JSON_BODY_LIMIT_BYTES });
|
|
8988
|
+
const tab = getRequestedTab(req, url, body);
|
|
8989
|
+
try {
|
|
8990
|
+
sendJson(res, 200, { ok: true, data: await handleNaturalConversationSttTranscribe(req, tab, body) });
|
|
8991
|
+
} catch (error) {
|
|
8992
|
+
const statusCode = error?.statusCode || 500;
|
|
8993
|
+
sendJson(res, statusCode, { ok: false, feature_unavailable: statusCode === 501, data: naturalConversationUnavailableResponse(tab, "stt"), voice: error?.voice || naturalConversationVoiceProviderStatus("stt"), error: statusCode >= 500 ? sanitizeError(error) : formatCliError(error) });
|
|
8994
|
+
}
|
|
8995
|
+
return;
|
|
8996
|
+
}
|
|
8997
|
+
|
|
8998
|
+
if (url.pathname === "/api/tts/speech" && req.method === "POST") {
|
|
8999
|
+
const body = await readJsonBody(req);
|
|
9000
|
+
const tab = getRequestedTab(req, url, body);
|
|
9001
|
+
try {
|
|
9002
|
+
sendJson(res, 200, { ok: true, data: await handleNaturalConversationTtsSpeech(req, tab, body) });
|
|
9003
|
+
} catch (error) {
|
|
9004
|
+
const statusCode = error?.statusCode || 500;
|
|
9005
|
+
sendJson(res, statusCode, { ok: false, feature_unavailable: statusCode === 501, data: naturalConversationUnavailableResponse(tab, "tts"), voice: error?.voice || naturalConversationVoiceProviderStatus("tts"), error: statusCode >= 500 ? sanitizeError(error) : formatCliError(error) });
|
|
9006
|
+
}
|
|
8684
9007
|
return;
|
|
8685
9008
|
}
|
|
8686
9009
|
|
package/lib/git-worktrees.mjs
CHANGED
|
@@ -71,7 +71,11 @@ function runCommand(command, args, { cwd, timeoutMs = GIT_DEFAULT_TIMEOUT_MS, ma
|
|
|
71
71
|
if (stderr.length > maxOutputLength) stderr = stderr.slice(-maxOutputLength);
|
|
72
72
|
});
|
|
73
73
|
child.on("error", (error) => finish({ exitCode: undefined, stdout, stderr, error: error?.message || String(error) }));
|
|
74
|
-
|
|
74
|
+
// "close", not "exit": exit can fire before the stdio pipes flush, which
|
|
75
|
+
// intermittently yields empty/truncated stdout for a successful command
|
|
76
|
+
// (e.g. `git rev-parse --show-toplevel` → "" → path.resolve("") →
|
|
77
|
+
// process.cwd() → worktree data read from the wrong repository).
|
|
78
|
+
child.on("close", (exitCode, signal) => finish({ exitCode, signal, stdout, stderr, timedOut: false }));
|
|
75
79
|
});
|
|
76
80
|
}
|
|
77
81
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@firstpick/pi-package-webui",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.6.0",
|
|
4
4
|
"description": "Pi Web UI companion package with a local browser UI CLI plus /webui-start and /webui-status commands.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"homepage": "https://github.com/Firstp1ck/npm-packages/tree/main/pi-package-webui#readme",
|
|
@@ -49,7 +49,7 @@
|
|
|
49
49
|
"pi-webui": "./bin/pi-webui.mjs"
|
|
50
50
|
},
|
|
51
51
|
"scripts": {
|
|
52
|
-
"check": "node --check public/app.js && node --check bin/pi-webui.mjs && node --check webui-rpc-helper.mjs && node --check lib/trust-boundaries.mjs && node --check lib/native-command-adapter.mjs && node --check lib/session-actions.mjs && node --check lib/auth-actions.mjs && node --check lib/temp-artifacts.mjs && node tests/run-all.mjs",
|
|
52
|
+
"check": "node --check public/app.js && node --check public/voice-conversation.mjs && node --check bin/pi-webui.mjs && node --check webui-rpc-helper.mjs && node --check lib/trust-boundaries.mjs && node --check lib/native-command-adapter.mjs && node --check lib/session-actions.mjs && node --check lib/auth-actions.mjs && node --check lib/temp-artifacts.mjs && node tests/run-all.mjs",
|
|
53
53
|
"test": "node tests/run-all.mjs"
|
|
54
54
|
},
|
|
55
55
|
"dependencies": {
|