@fatihdogmus/opencode-voice-input 0.0.0-stage → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Muhammed Fatih Doğmuş
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,153 @@
1
- # Temporary Holding Version
1
+ # @fatihdogmus/opencode-voice-input
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ OpenCode plugin that adds voice prompts to OpenCode Desktop and the TUI, transcribed with the OpenAI account selected in
4
+ OpenCode. The desktop app:
5
+
6
+ - **Mic button** next to the send button. Click to record (it turns red with a timer), click again to transcribe. The
7
+ text is typed into the prompt at the cursor for you to review and send (nothing is sent automatically). Works in
8
+ session composers and the new-session composer.
9
+ - This placement uses the composer's internal DOM, not the plugin API: the button is inserted into the static toolbar
10
+ row before `[data-slot="composer-actions"]` and styled with the app's `icon-button-v2` styles. If an app update
11
+ moves that element, the plugin falls back to a **🎙 Dictate** pill above the composer (plugin API), so it keeps
12
+ working.
13
+ - Set `"placement": "pill"` in the options to use only the pill.
14
+ - **Shortcut** `⌘⌥M` (changeable in **Settings > Shortcuts**), also `/dictate` and the command palette.
15
+ - **Cancel voice dictation** discards a recording in progress.
16
+
17
+ ## Install
18
+
19
+ Add the package to your global `~/.config/opencode/opencode.json(c)`. OpenCode Desktop loads GUI parts only from the
20
+ global config, and the TUI loads the TUI part of every plugin its server runs:
21
+
22
+ ```jsonc
23
+ {
24
+ "$schema": "https://opencode.ai/config.json",
25
+ "plugins": ["@fatihdogmus/opencode-voice-input"],
26
+ }
27
+ ```
28
+
29
+ Connect an OpenAI account with `/connect` if you haven't. The TUI also needs `ffmpeg` (`brew install ffmpeg`).
30
+
31
+ ## For agents
32
+
33
+ Follow these steps when a user asks you to install this plugin.
34
+
35
+ 1. **Check the OpenAI account.** `opencode auth list` must show an `OpenAI` row. If it doesn't, ask the user to run
36
+ `/connect` in OpenCode and sign in; don't start an OAuth flow yourself.
37
+ 2. **Check ffmpeg** (only needed for the TUI) with `ffmpeg -version`. If it's missing, ask before installing it
38
+ (`brew install ffmpeg` on macOS, the distribution's `ffmpeg` package on Linux, `winget install Gyan.FFmpeg` on
39
+ Windows).
40
+ 3. **Install it globally** with OpenCode's plugin command. It installs the package and adds it to the user's global
41
+ config. Never add this plugin to a project config: OpenCode Desktop loads GUI parts only from the global config.
42
+ Don't edit OpenCode's database or run `npm install` yourself.
43
+
44
+ ```sh
45
+ opencode plugin add @fatihdogmus/opencode-voice-input
46
+ ```
47
+
48
+ Set [options](#options) such as `language` only if the user asks; that means replacing the entry in
49
+ `~/.config/opencode/opencode.json(c)` with the object form and keeping every other entry.
50
+ 4. **Verify** that `opencode plugin list` has a row with the ID `voice-input`. Config changes reload on their own;
51
+ don't restart OpenCode or its service. Optionally, check transcription end to end. This sends a short clip through
52
+ the user's OpenAI account; on macOS:
53
+
54
+ ```sh
55
+ say -o /tmp/voice-check.aiff "hello from voice input"
56
+ ffmpeg -loglevel error -y -i /tmp/voice-check.aiff -c:a libopus /tmp/voice-check.webm
57
+ opencode api post /api/rpc/voice-input/transcribe \
58
+ --data "{\"input\":{\"audio\":\"$(base64 < /tmp/voice-check.webm | tr -d '\n')\",\"mimeType\":\"audio/webm\"}}"
59
+ ```
60
+
61
+ The reply's `output.text` should contain the sentence, and `output.account` names the account used.
62
+ 5. **Tell the user** how to use it: the mic button next to send (or `⌘⌥M`) in OpenCode Desktop, and `ctrl+x v` (or
63
+ `/dictate`) in the TUI. macOS asks for microphone access the first time they record. A TUI that was already open may
64
+ need to be reopened before the shortcut works.
65
+
66
+ To remove it: `opencode plugin remove @fatihdogmus/opencode-voice-input`.
67
+
68
+ ## TUI
69
+
70
+ The same package has a `./tui` part (`src/tui.tsx`), which the TUI loads automatically because the plugin is in the
71
+ server's config.
72
+
73
+ - **`<leader>v`** (`ctrl+x v` by default) or `/dictate` starts recording and stops it. The prompt footer shows
74
+ nothing while idle; it shows `● 0:07 stop` while recording and `◌ transcribing…` after. **Cancel voice dictation** and
75
+ **Copy last dictation** are in the command palette. Rebind the key in `cli.json` under `keybinds` as
76
+ `"voice-input.dictate"`.
77
+ - Recording uses `ffmpeg` (`brew install ffmpeg`) with the system default microphone (macOS AVFoundation, Linux
78
+ PulseAudio). macOS asks for microphone access for your terminal app the first time.
79
+ - The TUI plugin API cannot write to the prompt, so the transcript is put on the clipboard and the TUI's own
80
+ `prompt.paste` command inserts it at the cursor. Your previous clipboard text is restored 1.5 s later (set
81
+ `"restoreClipboard": false` to keep the transcript on the clipboard). If the paste can't happen, **Copy last
82
+ dictation** copies it again.
83
+ - After adding the `./tui` export to an already running shared service, run `opencode service restart` once: Bun
84
+ caches `package.json` exports, so the running service doesn't see the new part until it restarts.
85
+ - For testing without a microphone, `OPENCODE_VOICE_INPUT_TEST_FILE=/path/to/audio opencode` records that file instead.
86
+
87
+ ## How transcription works
88
+
89
+ The window records WebM/Opus with `MediaRecorder` and sends it to the plugin's server part (`src/index.ts`) through
90
+ its RPC, `voice-input.transcribe`:
91
+
92
+ 1. `ctx.integration.connection.active("openai")` returns the OpenAI account selected in OpenCode (`/connect`,
93
+ `opencode auth switch openai <label>`).
94
+ 2. `ctx.integration.connection.resolve(connection)` returns its credential. OpenCode refreshes an OAuth token that
95
+ expires within 5 minutes and stores the rotated refresh token, so the plugin never refreshes tokens itself.
96
+ 3. The audio is uploaded depending on the account's sign-in method:
97
+ - **Codex browser / device code** (`chatgpt-browser`, `chatgpt-headless`): `POST
98
+ https://chatgpt.com/backend-api/transcribe` with the ChatGPT access token, `ChatGPT-Account-Id`,
99
+ `originator: Codex Desktop` and a Codex Desktop user agent, as [codex-asr](https://github.com/Wangnov/codex-asr)
100
+ does. This is Codex Desktop's dictation endpoint; it is not a public API and can change.
101
+ - **API key**, `OPENAI_API_KEY`, or **Sign in with ChatGPT** (token sharing): `POST
102
+ https://api.openai.com/v1/audio/transcriptions` with model `gpt-4o-mini-transcribe` (option `model`).
103
+
104
+ The token stays in the OpenCode server; the window only sends audio and receives text.
105
+
106
+ Test it without the GUI:
107
+
108
+ ```sh
109
+ opencode api post /api/rpc/voice-input/transcribe --data '{"input":{"audio":"<base64>","mimeType":"audio/webm"}}'
110
+ ```
111
+
112
+ ## Options
113
+
114
+ ```jsonc
115
+ {
116
+ "plugins": [
117
+ {
118
+ "package": "@fatihdogmus/opencode-voice-input",
119
+ "options": {
120
+ "language": "tr", // optional language hint
121
+ "model": "gpt-4o-mini-transcribe", // public-API model for API-key / token-sharing accounts
122
+ "placement": "composer", // desktop: "composer" (mic next to send, default) | "pill"
123
+ "microphone": "default", // TUI: ffmpeg input device (avfoundation index/name, pulse source, or dshow name)
124
+ "ffmpegPath": "/opt/homebrew/bin/ffmpeg", // TUI: optional
125
+ "restoreClipboard": true, // TUI: put the previous clipboard text back after pasting
126
+ },
127
+ },
128
+ ],
129
+ }
130
+ ```
131
+
132
+ ## Files
133
+
134
+ - `src/index.ts`: server part, the `voice-input.transcribe` RPC.
135
+ - `src/rpc.ts`: the RPC contract.
136
+ - `src/gui.tsx`: window part: recorder, commands, pill, inserting the text.
137
+ - `src/inject.tsx`: the mic button next to the send button.
138
+ - `src/tui.tsx`, `src/tui-audio.ts`: TUI part: shortcut and commands, recording status, ffmpeg recorder, clipboard
139
+ paste.
140
+ - `src/main.ts`, `src/contract.ts`: asks macOS for microphone access from the app's main process.
141
+
142
+ ## Develop
143
+
144
+ This package lives in the [opencode-plugins](https://github.com/fatihdogmus/opencode-plugins) monorepo. From the
145
+ repository root:
146
+
147
+ ```sh
148
+ pnpm install
149
+ pnpm --filter @fatihdogmus/opencode-voice-input check # typecheck + build the desktop bundle
150
+ pnpm --filter @fatihdogmus/opencode-voice-input dev # vite build --watch; each rebuild reloads the GUI
151
+ ```
152
+
153
+ To try a local checkout, point the plugin entry at `packages/voice-input` instead of the package name.
@@ -0,0 +1,13 @@
1
+ Object.defineProperties(exports,{__esModule:{value:!0},[Symbol.toStringTag]:{value:`Module`}});let e=require("solid-js"),t=require("@opencode/plugin/gui"),n=require("effect"),r=require("solid-js/web"),i=require("@opencode/plugin/rpc"),a=require("electron");var o=t.Ipc.define({id:`voice-input.voice`,methods:{requestMicrophone:{output:n.Schema.Boolean}}}),s=(0,r.template)(`<svg viewBox="0 0 16 16"fill=none aria-hidden=true><rect x=5.5 y=1.5 width=5 height=8.5 rx=2.5 stroke=currentColor></rect><path d="M3 7.75a5 5 0 0 0 10 0M8 12.75v1.75M5.75 14.5h4.5"stroke=currentColor stroke-linecap=round>`),c=(0,r.template)(`<svg viewBox="0 0 16 16"aria-hidden=true><rect x=4 y=4 width=8 height=8 rx=1.5 fill=currentColor>`),l=(0,r.template)(`<svg viewBox="0 0 16 16"fill=none aria-hidden=true><circle cx=8 cy=8 r=5.5 stroke=currentColor stroke-opacity=.25 stroke-width=1.5></circle><path d="M13.5 8A5.5 5.5 0 0 0 8 2.5"stroke=currentColor stroke-width=1.5 stroke-linecap=round>`),u=(0,r.template)(`<span data-slot=voice-input-dot>`),d=(0,r.template)(`<span data-slot=voice-input-time>`),f=(0,r.template)(`<button type=button data-component=icon-button-v2 data-size=normal data-variant=ghost-muted>`),p=`[data-component="composer"] [data-slot="composer-actions"]`,m=`data-voice-input`,h=`
2
+ [${m}] { display: flex; align-items: center; gap: 4px; flex-shrink: 0; margin-inline-end: 6px; }
3
+ [${m}] > button[data-component="icon-button-v2"] { width: 28px; height: 28px; border-radius: 6px; }
4
+ [${m}] > button svg { width: 16px; height: 16px; display: block; }
5
+ [${m}][data-phase="recording"] > button { color: var(--v2-state-fg-danger); background-color: color-mix(in srgb, var(--v2-state-fg-danger) 14%, transparent); }
6
+ [${m}][data-phase="recording"] > button:hover { background-color: color-mix(in srgb, var(--v2-state-fg-danger) 22%, transparent); }
7
+ [${m}][data-phase="transcribing"] > button svg, [${m}][data-phase="starting"] > button svg { animation: voice-input-spin .9s linear infinite; }
8
+ [${m}] [data-slot="voice-input-time"] { font-size: 11px; font-weight: 530; font-variant-numeric: tabular-nums; color: var(--v2-state-fg-danger); }
9
+ [${m}][data-phase="recording"] [data-slot="voice-input-dot"] { width: 6px; height: 6px; border-radius: 9999px; background: var(--v2-state-fg-danger); animation: voice-input-pulse 1.2s ease-in-out infinite; }
10
+ @keyframes voice-input-spin { to { transform: rotate(360deg); } }
11
+ @keyframes voice-input-pulse { 50% { opacity: .25; } }
12
+ :root[data-motion="reduce"] [${m}] * { animation: none !important; }
13
+ `,g=()=>s(),_=()=>c(),v=()=>l();function y(t){let n=()=>{let e=t.model.phase(),n=t.model.owner();return e===`idle`||!n||n===t.host?e:`busy`};return(0,e.createEffect)(()=>{t.host.dataset.phase=n()===`busy`?`idle`:n()}),[(0,r.createComponent)(e.Switch,{get children(){return(0,r.createComponent)(e.Match,{get when(){return n()===`recording`},get children(){return[u(),(()=>{var e=d();return(0,r.insert)(e,()=>t.model.elapsed()),e})()]}})}}),(()=>{var i=f();return i.$$click=e=>{e.preventDefault(),e.stopPropagation(),t.model.onClick(t.host)},i.$$mousedown=e=>e.preventDefault(),(0,r.insert)(i,(0,r.createComponent)(e.Switch,{get fallback(){return(0,r.createComponent)(g,{})},get children(){return[(0,r.createComponent)(e.Match,{get when(){return n()===`recording`},get children(){return(0,r.createComponent)(_,{})}}),(0,r.createComponent)(e.Match,{get when(){return n()===`transcribing`||n()===`starting`},get children(){return(0,r.createComponent)(v,{})}})]}})),(0,r.effect)(e=>{var a=t.model.title(),o=t.model.title(),s=n()===`recording`,c=n()===`busy`||n()===`transcribing`||n()===`starting`;return a!==e.e&&(0,r.setAttribute)(i,`title`,e.e=a),o!==e.t&&(0,r.setAttribute)(i,`aria-label`,e.t=o),s!==e.a&&(0,r.setAttribute)(i,`aria-pressed`,e.a=s),c!==e.o&&(i.disabled=e.o=c),e},{e:void 0,t:void 0,a:void 0,o:void 0}),i})()]}function b(t){let n=new Map,[i,a]=(0,e.createSignal)(0),o=0,s=()=>{o=0;for(let e of document.querySelectorAll(p)){let i=e.parentElement;if(!i||i.querySelector(`:scope > [${m}]`))continue;let a=document.createElement(`div`);a.setAttribute(m,``),i.insertBefore(a,e),n.set(a,(0,r.render)(()=>(0,r.createComponent)(y,{host:a,model:t}),a))}for(let[e,t]of n)e.isConnected||(t(),n.delete(e));a(n.size)},c=new MutationObserver(()=>{o||=requestAnimationFrame(s)});return c.observe(document.body,{childList:!0,subtree:!0}),s(),{count:i,dispose(){c.disconnect(),o&&cancelAnimationFrame(o);for(let[e,t]of n)t(),e.remove();n.clear()}}}function x(e){let t=e?.closest(`[data-component="composer"]`);return t?.isConnected?t:document.activeElement?.closest(`[data-component="composer"]`)||[...document.querySelectorAll(`[data-component="composer"]`)].find(e=>e.offsetParent)}function S(e,t){let n=e.querySelector(`[data-component="composer-editor"]`);return n?(n.focus(),document.execCommand(`insertText`,!1,t)):!1}(0,r.delegateEvents)([`mousedown`,`click`]);var C={type:`object`,properties:{message:{type:`string`}},required:[`message`],additionalProperties:!1},w=i.Rpc.define({id:`voice-input`,methods:{transcribe:{input:{type:`object`,properties:{audio:{type:`string`,description:`Base64-encoded recording`},mimeType:{type:`string`},language:{type:`string`}},required:[`audio`,`mimeType`],additionalProperties:!1},output:{type:`object`,properties:{text:{type:`string`},account:{type:`string`}},required:[`text`,`account`],additionalProperties:!1},errors:{no_account:C,unsupported_account:C,upstream:C}}},events:{}}),T=e=>{e.provide(o,{requestMicrophone:async()=>process.platform!==`darwin`||a.systemPreferences.getMediaAccessStatus(`microphone`)===`granted`||a.systemPreferences.askForMediaAccess(`microphone`)})},E=t.Plugin.define({i18n:{en:{group:`Voice input`,"pill.idle":`🎙 Dictate`,"pill.idle.title":`Record a voice prompt and transcribe it with your OpenAI account`,"pill.starting":`🎙 Starting…`,"pill.recording":`Stop {{time}}`,"pill.recording.title":`Stop recording and transcribe`,"pill.transcribing":`Transcribing…`,"button.idle":`Dictate (⌘⌥M)`,"button.recording":`Stop and transcribe (⌘⌥M)`,"button.busy":`Transcribing…`,"command.toggle":`Start or stop voice dictation`,"command.cancel":`Cancel voice dictation`,"error.title":`Voice input failed`,"error.microphone":`Microphone access was denied. Allow OpenCode in System Settings › Privacy & Security › Microphone.`,nospeech:`No speech was heard`,copied:`No prompt is open, so the transcript was copied to the clipboard`}}}),D=6e5,O=[`audio/webm;codecs=opus`,`audio/webm`,`audio/ogg;codecs=opus`,`audio/mp4`],k=e=>new Promise((t,n)=>{let r=new FileReader;r.onload=()=>t(String(r.result).replace(/^data:[^,]*,/,``)),r.onerror=()=>n(r.error??Error(`Could not read the recording`)),r.readAsDataURL(e)}),A=e=>{let t=e?.data;return typeof t?.message==`string`?t.message:e instanceof Error?e.message:String(e)},j=class{stream;recorder;chunks=[];async start(){this.stream=await navigator.mediaDevices.getUserMedia({audio:{echoCancellation:!0,noiseSuppression:!0,autoGainControl:!0}});let e=O.find(e=>MediaRecorder.isTypeSupported(e));this.chunks=[],this.recorder=new MediaRecorder(this.stream,e?{mimeType:e}:void 0),this.recorder.ondataavailable=e=>{e.data.size>0&&this.chunks.push(e.data)},this.recorder.start(250)}stop(){let e=this.recorder;return e?new Promise(t=>{e.onstop=()=>{let n=new Blob(this.chunks,{type:e.mimeType||`audio/webm`});this.release(),t(n)},e.stop()}):Promise.resolve(void 0)}release(){this.recorder?.state===`recording`&&this.recorder.stop(),this.stream?.getTracks().forEach(e=>e.stop()),this.stream=void 0,this.recorder=void 0,this.chunks=[]}},M=n=>{let r=n.main(o),i=new j,[a,s]=(0,e.createSignal)({kind:`idle`}),[c,l]=(0,e.createSignal)(Date.now()),u,[d,f]=(0,e.createSignal)(),p=n.options.placement===`pill`?`pill`:`composer`,m=()=>typeof n.options.language==`string`?n.options.language:``,g=()=>{u&&clearInterval(u),u=void 0},_=e=>{g(),i.release(),s({kind:`idle`}),f(void 0),n.toast.show({variant:`error`,title:n.t(`error.title`),description:A(e)})},v=e=>{if(!e){n.toast.show({title:n.t(`nospeech`)});return}let t=n.screen.current()?.composer,r=x(d());if(!t&&r&&S(r,e))return;if(!t){n.system.copy(e),n.toast.show({title:n.t(`copied`)});return}let i=t.text();t.insert(i&&!/\s$/.test(i)?` ${e}`:e),t.focus()},y=async e=>{if(a().kind===`idle`){f(e),s({kind:`starting`});try{if(!await r.requestMicrophone())throw Error(n.t(`error.microphone`));await i.start();let e=Date.now();l(e),s({kind:`recording`,since:e}),u=setInterval(()=>{l(Date.now()),Date.now()-e>D&&C()},500)}catch(e){_(e)}}},C=async()=>{if(a().kind===`recording`){g(),s({kind:`transcribing`});try{let e=await i.stop();if(!e||e.size===0)throw Error(n.t(`nospeech`));let t=await k(e),r=(await n.client.rpc(w).transcribe({audio:t,mimeType:e.type,language:m()})).text;s({kind:`idle`}),v(r),f(void 0)}catch(e){_(e)}}},T=()=>{if(a().kind!==`recording`&&a().kind!==`starting`)return!1;g(),i.release(),s({kind:`idle`}),f(void 0)},E=e=>{let t=a().kind;t===`idle`?y(e??N()):t===`recording`&&C()},O=e=>{let t=Math.max(0,Math.floor((c()-e)/1e3));return`${Math.floor(t/60)}:${String(t%60).padStart(2,`0`)}`},M=p===`composer`?b({phase:()=>a().kind,owner:d,elapsed:()=>{let e=a();return e.kind===`recording`?O(e.since):``},title:()=>{let e=a().kind;return e===`recording`?n.t(`button.recording`):e===`idle`?n.t(`button.idle`):n.t(`button.busy`)},onClick:e=>E(e)}):void 0;M&&n.add(t.Style,h);function N(){return x(void 0)?.querySelector(`[data-voice-input]`)??void 0}n.add(t.ComposerPill,{id:`mic`,show:()=>{if(M&&M.count()>0)return;let e=a();switch(e.kind){case`idle`:return{label:n.t(`pill.idle`),title:n.t(`pill.idle.title`),run:()=>E()};case`starting`:return{label:n.t(`pill.starting`)};case`recording`:return{label:n.t(`pill.recording`,{time:O(e.since)}),title:n.t(`pill.recording.title`),icon:`stop`,tone:`danger`,run:()=>E()};case`transcribing`:return{label:n.t(`pill.transcribing`),icon:`outline-wave-audio`,tone:`warning`}}}}),n.add(t.Command,()=>({id:`toggle`,title:n.t(`command.toggle`),group:n.t(`group`),keywords:`voice mic microphone dictate speech record transcribe`,section:`prompt`,bind:`mod+alt+m`,editable:!0,slash:{name:`dictate`},run:()=>E()})),n.add(t.Command,()=>({id:`cancel`,title:n.t(`command.cancel`),group:n.t(`group`),section:`prompt`,enabled:a().kind===`recording`||a().kind===`starting`,run:()=>T()})),(0,e.onCleanup)(()=>{M?.dispose(),g(),i.release()})},N=t.Plugin.define({...E,setup:M,main:T});exports.default=N;
package/package.json CHANGED
@@ -1,6 +1,62 @@
1
1
  {
2
2
  "name": "@fatihdogmus/opencode-voice-input",
3
- "version": "0.0.0-stage",
4
- "stub": true,
5
- "description": "Temporary package placeholder for staged publishing"
3
+ "version": "1.0.0",
4
+ "description": "Voice prompts for OpenCode Desktop and the TUI, transcribed with the OpenAI account selected in OpenCode",
5
+ "license": "MIT",
6
+ "type": "module",
7
+ "engines": {
8
+ "node": ">=26"
9
+ },
10
+ "repository": {
11
+ "type": "git",
12
+ "url": "git+https://github.com/fatihdogmus/opencode-plugins.git",
13
+ "directory": "packages/voice-input"
14
+ },
15
+ "homepage": "https://github.com/fatihdogmus/opencode-plugins/tree/main/packages/voice-input#readme",
16
+ "bugs": {
17
+ "url": "https://github.com/fatihdogmus/opencode-plugins/issues"
18
+ },
19
+ "keywords": [
20
+ "opencode",
21
+ "voice",
22
+ "dictation",
23
+ "speech-to-text",
24
+ "transcription",
25
+ "plugin"
26
+ ],
27
+ "publishConfig": {
28
+ "access": "public",
29
+ "registry": "https://registry.npmjs.org/"
30
+ },
31
+ "exports": {
32
+ ".": "./src/index.ts",
33
+ "./rpc": "./src/rpc.ts",
34
+ "./tui": "./src/tui.tsx",
35
+ "./gui-experimental": "./dist/gui-experimental.cjs"
36
+ },
37
+ "files": [
38
+ "src",
39
+ "dist"
40
+ ],
41
+ "dependencies": {
42
+ "@opencode/plugin": "2.2.0",
43
+ "effect": "4.0.1"
44
+ },
45
+ "devDependencies": {
46
+ "@opencode/theme": "2.2.0",
47
+ "@opentui/core": "0.7.0",
48
+ "@opentui/solid": "0.7.0",
49
+ "@types/node": "^22.0.0",
50
+ "electron": "^39.0.0",
51
+ "solid-js": "^1.9.15",
52
+ "typescript": "^7.0.2",
53
+ "vite": "^8.3.5",
54
+ "vite-plugin-solid": "^2.11.0"
55
+ },
56
+ "scripts": {
57
+ "build": "vite build",
58
+ "dev": "vite build --watch",
59
+ "typecheck": "tsc --noEmit",
60
+ "check": "pnpm typecheck && pnpm build"
61
+ }
6
62
  }
@@ -0,0 +1,9 @@
1
+ import { Schema } from "effect"
2
+ import { Ipc } from "@opencode/plugin/gui"
3
+
4
+ export const Voice = Ipc.define({
5
+ id: "voice-input.voice",
6
+ methods: {
7
+ requestMicrophone: { output: Schema.Boolean },
8
+ },
9
+ })
package/src/gui.tsx ADDED
@@ -0,0 +1,274 @@
1
+ import { createSignal, onCleanup } from "solid-js"
2
+ import { Command, ComposerPill, Plugin, Style, type ComposerPillView, type SetupContext } from "@opencode/plugin/gui"
3
+ import { Voice } from "./contract"
4
+ import { composerOf, CSS, injectComposerButtons, typeInto } from "./inject"
5
+ import { VoiceRpc } from "./rpc"
6
+ import { main } from "./main"
7
+
8
+ const definition = Plugin.define({
9
+ i18n: {
10
+ en: {
11
+ group: "Voice input",
12
+ "pill.idle": "🎙 Dictate",
13
+ "pill.idle.title": "Record a voice prompt and transcribe it with your OpenAI account",
14
+ "pill.starting": "🎙 Starting…",
15
+ "pill.recording": "Stop {{time}}",
16
+ "pill.recording.title": "Stop recording and transcribe",
17
+ "pill.transcribing": "Transcribing…",
18
+ "button.idle": "Dictate (⌘⌥M)",
19
+ "button.recording": "Stop and transcribe (⌘⌥M)",
20
+ "button.busy": "Transcribing…",
21
+ "command.toggle": "Start or stop voice dictation",
22
+ "command.cancel": "Cancel voice dictation",
23
+ "error.title": "Voice input failed",
24
+ "error.microphone":
25
+ "Microphone access was denied. Allow OpenCode in System Settings › Privacy & Security › Microphone.",
26
+ nospeech: "No speech was heard",
27
+ copied: "No prompt is open, so the transcript was copied to the clipboard",
28
+ },
29
+ },
30
+ })
31
+
32
+ type Ctx = SetupContext<typeof definition>
33
+
34
+ type Phase = { kind: "idle" } | { kind: "starting" } | { kind: "recording"; since: number } | { kind: "transcribing" }
35
+
36
+ const MAX_RECORDING_MS = 10 * 60_000
37
+ const MIME_TYPES = ["audio/webm;codecs=opus", "audio/webm", "audio/ogg;codecs=opus", "audio/mp4"]
38
+
39
+ const toBase64 = (blob: Blob) =>
40
+ new Promise<string>((resolve, reject) => {
41
+ const reader = new FileReader()
42
+ reader.onload = () => resolve(String(reader.result).replace(/^data:[^,]*,/, ""))
43
+ reader.onerror = () => reject(reader.error ?? new Error("Could not read the recording"))
44
+ reader.readAsDataURL(blob)
45
+ })
46
+
47
+ const message = (error: unknown) => {
48
+ // Plugin RPC errors carry the user-facing reason in data.message.
49
+ const data = (error as { data?: { message?: unknown } } | undefined)?.data
50
+ if (typeof data?.message === "string") return data.message
51
+ return error instanceof Error ? error.message : String(error)
52
+ }
53
+
54
+ class Recorder {
55
+ private stream?: MediaStream
56
+ private recorder?: MediaRecorder
57
+ private chunks: Blob[] = []
58
+
59
+ async start() {
60
+ this.stream = await navigator.mediaDevices.getUserMedia({
61
+ audio: { echoCancellation: true, noiseSuppression: true, autoGainControl: true },
62
+ })
63
+ const mimeType = MIME_TYPES.find((type) => MediaRecorder.isTypeSupported(type))
64
+ this.chunks = []
65
+ this.recorder = new MediaRecorder(this.stream, mimeType ? { mimeType } : undefined)
66
+ this.recorder.ondataavailable = (event) => {
67
+ if (event.data.size > 0) this.chunks.push(event.data)
68
+ }
69
+ this.recorder.start(250)
70
+ }
71
+
72
+ stop() {
73
+ const recorder = this.recorder
74
+ if (!recorder) return Promise.resolve(undefined)
75
+ return new Promise<Blob>((resolve) => {
76
+ recorder.onstop = () => {
77
+ const blob = new Blob(this.chunks, { type: recorder.mimeType || "audio/webm" })
78
+ this.release()
79
+ resolve(blob)
80
+ }
81
+ recorder.stop()
82
+ })
83
+ }
84
+
85
+ release() {
86
+ if (this.recorder?.state === "recording") this.recorder.stop()
87
+ this.stream?.getTracks().forEach((track) => track.stop())
88
+ this.stream = undefined
89
+ this.recorder = undefined
90
+ this.chunks = []
91
+ }
92
+ }
93
+
94
+ const setup = (ctx: Ctx) => {
95
+ const voice = ctx.main(Voice)
96
+ const recorder = new Recorder()
97
+ const [phase, setPhase] = createSignal<Phase>({ kind: "idle" })
98
+ const [now, setNow] = createSignal(Date.now())
99
+ let ticker: ReturnType<typeof setInterval> | undefined
100
+ const [owner, setOwner] = createSignal<HTMLElement>()
101
+ const placement = ctx.options.placement === "pill" ? "pill" : "composer"
102
+
103
+ const language = () => (typeof ctx.options.language === "string" ? ctx.options.language : "")
104
+
105
+ const stopTicker = () => {
106
+ if (ticker) clearInterval(ticker)
107
+ ticker = undefined
108
+ }
109
+
110
+ const fail = (error: unknown) => {
111
+ stopTicker()
112
+ recorder.release()
113
+ setPhase({ kind: "idle" })
114
+ setOwner(undefined)
115
+ ctx.toast.show({ variant: "error", title: ctx.t("error.title"), description: message(error) })
116
+ }
117
+
118
+ const insert = (text: string) => {
119
+ if (!text) {
120
+ ctx.toast.show({ title: ctx.t("nospeech") })
121
+ return
122
+ }
123
+ const composer = ctx.screen.current()?.composer
124
+ const target = composerOf(owner())
125
+ // The plugin API reaches the routed session's prompt; a new-session composer is only reachable through the DOM.
126
+ if (!composer && target && typeInto(target, text)) return
127
+ if (!composer) {
128
+ void ctx.system.copy(text)
129
+ ctx.toast.show({ title: ctx.t("copied") })
130
+ return
131
+ }
132
+ const current = composer.text()
133
+ composer.insert(current && !/\s$/.test(current) ? ` ${text}` : text)
134
+ composer.focus()
135
+ }
136
+
137
+ const start = async (host?: HTMLElement) => {
138
+ if (phase().kind !== "idle") return
139
+ setOwner(host)
140
+ setPhase({ kind: "starting" })
141
+ try {
142
+ if (!(await voice.requestMicrophone())) throw new Error(ctx.t("error.microphone"))
143
+ await recorder.start()
144
+ const since = Date.now()
145
+ setNow(since)
146
+ setPhase({ kind: "recording", since })
147
+ ticker = setInterval(() => {
148
+ setNow(Date.now())
149
+ if (Date.now() - since > MAX_RECORDING_MS) void stop()
150
+ }, 500)
151
+ } catch (error) {
152
+ fail(error)
153
+ }
154
+ }
155
+
156
+ const stop = async () => {
157
+ if (phase().kind !== "recording") return
158
+ stopTicker()
159
+ setPhase({ kind: "transcribing" })
160
+ try {
161
+ const blob = await recorder.stop()
162
+ if (!blob || blob.size === 0) throw new Error(ctx.t("nospeech"))
163
+ const audio = await toBase64(blob)
164
+ const result = await ctx.client.rpc(VoiceRpc).transcribe({ audio, mimeType: blob.type, language: language() })
165
+ const text = (result as { text: string }).text
166
+ setPhase({ kind: "idle" })
167
+ insert(text)
168
+ setOwner(undefined)
169
+ } catch (error) {
170
+ fail(error)
171
+ }
172
+ }
173
+
174
+ const cancel = () => {
175
+ if (phase().kind !== "recording" && phase().kind !== "starting") return false
176
+ stopTicker()
177
+ recorder.release()
178
+ setPhase({ kind: "idle" })
179
+ setOwner(undefined)
180
+ }
181
+
182
+ const toggle = (host?: HTMLElement) => {
183
+ const kind = phase().kind
184
+ if (kind === "idle") void start(host ?? hostOfFocusedComposer())
185
+ else if (kind === "recording") void stop()
186
+ }
187
+
188
+ const elapsed = (since: number) => {
189
+ const seconds = Math.max(0, Math.floor((now() - since) / 1000))
190
+ return `${Math.floor(seconds / 60)}:${String(seconds % 60).padStart(2, "0")}`
191
+ }
192
+
193
+ const buttons =
194
+ placement === "composer"
195
+ ? injectComposerButtons({
196
+ phase: () => phase().kind,
197
+ owner,
198
+ elapsed: () => {
199
+ const current = phase()
200
+ return current.kind === "recording" ? elapsed(current.since) : ""
201
+ },
202
+ title: () => {
203
+ const kind = phase().kind
204
+ if (kind === "recording") return ctx.t("button.recording")
205
+ if (kind === "idle") return ctx.t("button.idle")
206
+ return ctx.t("button.busy")
207
+ },
208
+ onClick: (host) => toggle(host),
209
+ })
210
+ : undefined
211
+ if (buttons) ctx.add(Style, CSS)
212
+
213
+ function hostOfFocusedComposer() {
214
+ return composerOf(undefined)?.querySelector<HTMLElement>("[data-voice-input]") ?? undefined
215
+ }
216
+
217
+ ctx.add(ComposerPill, {
218
+ id: "mic",
219
+ // The pill stands in when the mic can't be placed next to the send button, e.g. after an app update moves it.
220
+ show: (): ComposerPillView | undefined => {
221
+ if (buttons && buttons.count() > 0) return undefined
222
+ const current = phase()
223
+ switch (current.kind) {
224
+ case "idle":
225
+ return {
226
+ label: ctx.t("pill.idle"),
227
+ title: ctx.t("pill.idle.title"),
228
+ run: () => toggle(),
229
+ }
230
+ case "starting":
231
+ return { label: ctx.t("pill.starting") }
232
+ case "recording":
233
+ return {
234
+ label: ctx.t("pill.recording", { time: elapsed(current.since) }),
235
+ title: ctx.t("pill.recording.title"),
236
+ icon: "stop",
237
+ tone: "danger",
238
+ run: () => toggle(),
239
+ }
240
+ case "transcribing":
241
+ return { label: ctx.t("pill.transcribing"), icon: "outline-wave-audio", tone: "warning" }
242
+ }
243
+ },
244
+ })
245
+
246
+ ctx.add(Command, (): Command => ({
247
+ id: "toggle",
248
+ title: ctx.t("command.toggle"),
249
+ group: ctx.t("group"),
250
+ keywords: "voice mic microphone dictate speech record transcribe",
251
+ section: "prompt",
252
+ bind: "mod+alt+m",
253
+ editable: true,
254
+ slash: { name: "dictate" },
255
+ run: () => toggle(),
256
+ }))
257
+
258
+ ctx.add(Command, (): Command => ({
259
+ id: "cancel",
260
+ title: ctx.t("command.cancel"),
261
+ group: ctx.t("group"),
262
+ section: "prompt",
263
+ enabled: phase().kind === "recording" || phase().kind === "starting",
264
+ run: () => cancel(),
265
+ }))
266
+
267
+ onCleanup(() => {
268
+ buttons?.dispose()
269
+ stopTicker()
270
+ recorder.release()
271
+ })
272
+ }
273
+
274
+ export default Plugin.define({ ...definition, setup, main })
package/src/index.ts ADDED
@@ -0,0 +1,115 @@
1
+ import { readFileSync } from "node:fs"
2
+ import { Plugin } from "@opencode/plugin"
3
+ import { VoiceRpc, type TranscribeInput, type TranscribeOutput } from "./rpc"
4
+
5
+ // The ChatGPT backend's one-shot dictation endpoint that Codex Desktop uses (as codex-asr does). It takes the same
6
+ // ChatGPT access token as Codex logins: OpenCode's "Codex browser/device code" OpenAI sign-ins.
7
+ const CHATGPT_TRANSCRIBE = "https://chatgpt.com/backend-api/transcribe"
8
+ const CHATGPT_METHODS = new Set(["chatgpt-browser", "chatgpt-headless"])
9
+ // API keys and "Sign in with ChatGPT" (token sharing) tokens are used against the public API.
10
+ const OPENAI_TRANSCRIBE = "https://api.openai.com/v1/audio/transcriptions"
11
+ const DEFAULT_API_MODEL = "gpt-4o-mini-transcribe"
12
+ const CODEX_DESKTOP_VERSION = "26.429.30905"
13
+
14
+ const codexUserAgent = () => {
15
+ let version = CODEX_DESKTOP_VERSION
16
+ try {
17
+ const plist = readFileSync("/Applications/Codex.app/Contents/Info.plist", "utf8")
18
+ version = plist.match(/<key>CFBundleShortVersionString<\/key>\s*<string>([^<]+)<\/string>/)?.[1] ?? version
19
+ } catch {}
20
+ const os = process.platform === "darwin" ? "macos" : process.platform === "win32" ? "windows" : process.platform
21
+ const arch = process.arch === "arm64" ? "aarch64" : process.arch === "x64" ? "x86_64" : process.arch
22
+ return `Codex Desktop/${version} (${os}; ${arch})`
23
+ }
24
+
25
+ const accountFromToken = (token: string) => {
26
+ try {
27
+ const payload = JSON.parse(Buffer.from(token.split(".")[1] ?? "", "base64url").toString())
28
+ const id = payload?.["https://api.openai.com/auth"]?.chatgpt_account_id ?? payload?.chatgpt_account_id
29
+ return typeof id === "string" && id ? id : undefined
30
+ } catch {
31
+ return undefined
32
+ }
33
+ }
34
+
35
+ const extension = (mimeType: string) => {
36
+ if (mimeType.includes("ogg")) return "ogg"
37
+ if (mimeType.includes("mp4") || mimeType.includes("m4a")) return "m4a"
38
+ if (mimeType.includes("wav")) return "wav"
39
+ if (mimeType.includes("mpeg")) return "mp3"
40
+ return "webm"
41
+ }
42
+
43
+ const clip = (body: string) => body.split(/\s+/).join(" ").slice(0, 300)
44
+
45
+ class UpstreamError extends Error {}
46
+
47
+ const upload = async (url: string, headers: Record<string, string>, form: FormData, signal: AbortSignal) => {
48
+ const response = await fetch(url, { method: "POST", headers, body: form, signal })
49
+ const body = await response.text()
50
+ if (!response.ok) throw new UpstreamError(`Transcription failed with HTTP ${response.status}: ${clip(body)}`)
51
+ const parsed = JSON.parse(body) as { text?: unknown }
52
+ if (typeof parsed.text !== "string") throw new UpstreamError("Transcription response did not contain text")
53
+ return parsed.text.trim()
54
+ }
55
+
56
+ const audioForm = (input: TranscribeInput) => {
57
+ const type = input.mimeType.split(";")[0] || "audio/webm"
58
+ const form = new FormData()
59
+ form.append("file", new Blob([Buffer.from(input.audio, "base64")], { type }), `voice.${extension(type)}`)
60
+ if (input.language) form.append("language", input.language)
61
+ return form
62
+ }
63
+
64
+ export default Plugin.define({
65
+ id: "voice-input",
66
+ async setup(ctx) {
67
+ const apiModel = typeof ctx.options.model === "string" && ctx.options.model ? ctx.options.model : DEFAULT_API_MODEL
68
+
69
+ const registration = await ctx.rpc.register(VoiceRpc, {
70
+ transcribe: async (raw, call) => {
71
+ const input = raw as TranscribeInput
72
+ const connection = await ctx.integration.connection.active("openai")
73
+ if (!connection) {
74
+ return call.error("no_account", "No OpenAI account is connected", {
75
+ message: "Connect OpenAI with /connect (or `opencode auth login openai`), then try again.",
76
+ })
77
+ }
78
+ const account = connection.type === "credential" ? connection.label : connection.name
79
+ // resolve() refreshes an OAuth token that expires within 5 minutes and saves the rotated refresh token.
80
+ const credential = await ctx.integration.connection.resolve(connection)
81
+
82
+ try {
83
+ let text: string
84
+ if (credential?.type === "oauth" && CHATGPT_METHODS.has(credential.methodID)) {
85
+ const accountID = (credential.metadata?.accountID as string | undefined) ?? accountFromToken(credential.access)
86
+ const headers: Record<string, string> = {
87
+ Authorization: `Bearer ${credential.access}`,
88
+ originator: "Codex Desktop",
89
+ "User-Agent": codexUserAgent(),
90
+ }
91
+ if (accountID) headers["ChatGPT-Account-Id"] = accountID
92
+ text = await upload(CHATGPT_TRANSCRIBE, headers, audioForm(input), call.signal)
93
+ } else if (credential?.type === "key" || credential?.type === "oauth") {
94
+ const token = credential.type === "key" ? credential.key : credential.access
95
+ const form = audioForm(input)
96
+ form.append("model", apiModel)
97
+ form.append("response_format", "json")
98
+ text = await upload(OPENAI_TRANSCRIBE, { Authorization: `Bearer ${token}` }, form, call.signal)
99
+ } else {
100
+ return call.error("unsupported_account", "This OpenAI account can't transcribe", {
101
+ message: `The active OpenAI account "${account}" has no usable token. Switch accounts with /connect.`,
102
+ })
103
+ }
104
+ return { text, account } satisfies TranscribeOutput
105
+ } catch (error) {
106
+ if (call.signal.aborted) throw error
107
+ const message = error instanceof Error ? error.message : String(error)
108
+ return call.error("upstream", message, { message: `${message} (account "${account}")` })
109
+ }
110
+ },
111
+ })
112
+
113
+ return () => registration.dispose()
114
+ },
115
+ })
package/src/inject.tsx ADDED
@@ -0,0 +1,167 @@
1
+ import { createEffect, createSignal, Match, Switch, type Accessor } from "solid-js"
2
+ import { render } from "solid-js/web"
3
+
4
+ // Internal DOM of OpenCode Desktop's composer (app/src/composer/editor/editor.tsx), not plugin API:
5
+ // <form data-component="composer"> … <div class="flex h-11 …"> ← static row
6
+ // <div>+ menu</div> <div data-slot="composer-controls"/> <div data-slot="composer-actions"/> ← send button
7
+ // The mic goes into the static row right before composer-actions, so Solid never reconciles over it.
8
+ const ACTIONS = '[data-component="composer"] [data-slot="composer-actions"]'
9
+ const HOST = "data-voice-input"
10
+
11
+ export type ButtonPhase = "idle" | "starting" | "recording" | "transcribing"
12
+
13
+ export interface ButtonModel {
14
+ phase: Accessor<ButtonPhase>
15
+ owner: Accessor<HTMLElement | undefined>
16
+ elapsed: Accessor<string>
17
+ title: Accessor<string>
18
+ onClick(host: HTMLElement): void
19
+ }
20
+
21
+ export const CSS = `
22
+ [${HOST}] { display: flex; align-items: center; gap: 4px; flex-shrink: 0; margin-inline-end: 6px; }
23
+ [${HOST}] > button[data-component="icon-button-v2"] { width: 28px; height: 28px; border-radius: 6px; }
24
+ [${HOST}] > button svg { width: 16px; height: 16px; display: block; }
25
+ [${HOST}][data-phase="recording"] > button { color: var(--v2-state-fg-danger); background-color: color-mix(in srgb, var(--v2-state-fg-danger) 14%, transparent); }
26
+ [${HOST}][data-phase="recording"] > button:hover { background-color: color-mix(in srgb, var(--v2-state-fg-danger) 22%, transparent); }
27
+ [${HOST}][data-phase="transcribing"] > button svg, [${HOST}][data-phase="starting"] > button svg { animation: voice-input-spin .9s linear infinite; }
28
+ [${HOST}] [data-slot="voice-input-time"] { font-size: 11px; font-weight: 530; font-variant-numeric: tabular-nums; color: var(--v2-state-fg-danger); }
29
+ [${HOST}][data-phase="recording"] [data-slot="voice-input-dot"] { width: 6px; height: 6px; border-radius: 9999px; background: var(--v2-state-fg-danger); animation: voice-input-pulse 1.2s ease-in-out infinite; }
30
+ @keyframes voice-input-spin { to { transform: rotate(360deg); } }
31
+ @keyframes voice-input-pulse { 50% { opacity: .25; } }
32
+ :root[data-motion="reduce"] [${HOST}] * { animation: none !important; }
33
+ `
34
+
35
+ const MicIcon = () => (
36
+ <svg viewBox="0 0 16 16" fill="none" aria-hidden="true">
37
+ <rect x="5.5" y="1.5" width="5" height="8.5" rx="2.5" stroke="currentColor" />
38
+ <path d="M3 7.75a5 5 0 0 0 10 0M8 12.75v1.75M5.75 14.5h4.5" stroke="currentColor" stroke-linecap="round" />
39
+ </svg>
40
+ )
41
+
42
+ const StopIcon = () => (
43
+ <svg viewBox="0 0 16 16" aria-hidden="true">
44
+ <rect x="4" y="4" width="8" height="8" rx="1.5" fill="currentColor" />
45
+ </svg>
46
+ )
47
+
48
+ const SpinnerIcon = () => (
49
+ <svg viewBox="0 0 16 16" fill="none" aria-hidden="true">
50
+ <circle cx="8" cy="8" r="5.5" stroke="currentColor" stroke-opacity=".25" stroke-width="1.5" />
51
+ <path d="M13.5 8A5.5 5.5 0 0 0 8 2.5" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" />
52
+ </svg>
53
+ )
54
+
55
+ function MicButton(props: { host: HTMLElement; model: ButtonModel }) {
56
+ // Only the composer that started a recording shows its progress; the others stay a plain, disabled mic.
57
+ const phase = () => {
58
+ const current = props.model.phase()
59
+ const owner = props.model.owner()
60
+ return current === "idle" || !owner || owner === props.host ? current : "busy"
61
+ }
62
+
63
+ createEffect(() => {
64
+ props.host.dataset.phase = phase() === "busy" ? "idle" : phase()
65
+ })
66
+
67
+ return (
68
+ <>
69
+ <Switch>
70
+ <Match when={phase() === "recording"}>
71
+ <span data-slot="voice-input-dot" />
72
+ <span data-slot="voice-input-time">{props.model.elapsed()}</span>
73
+ </Match>
74
+ </Switch>
75
+ <button
76
+ type="button"
77
+ data-component="icon-button-v2"
78
+ data-size="normal"
79
+ data-variant="ghost-muted"
80
+ title={props.model.title()}
81
+ aria-label={props.model.title()}
82
+ aria-pressed={phase() === "recording"}
83
+ disabled={phase() === "busy" || phase() === "transcribing" || phase() === "starting"}
84
+ // Keep focus and the caret in the prompt, as the composer's own controls do.
85
+ onMouseDown={(event) => event.preventDefault()}
86
+ onClick={(event) => {
87
+ event.preventDefault()
88
+ event.stopPropagation()
89
+ props.model.onClick(props.host)
90
+ }}
91
+ >
92
+ <Switch fallback={<MicIcon />}>
93
+ <Match when={phase() === "recording"}>
94
+ <StopIcon />
95
+ </Match>
96
+ <Match when={phase() === "transcribing" || phase() === "starting"}>
97
+ <SpinnerIcon />
98
+ </Match>
99
+ </Switch>
100
+ </button>
101
+ </>
102
+ )
103
+ }
104
+
105
+ /** Keeps a mic button before the send button of every composer in the window. Returns how many are mounted. */
106
+ export function injectComposerButtons(model: ButtonModel) {
107
+ const mounted = new Map<HTMLElement, () => void>()
108
+ const [count, setCount] = createSignal(0)
109
+ let frame = 0
110
+
111
+ const scan = () => {
112
+ frame = 0
113
+ for (const actions of document.querySelectorAll<HTMLElement>(ACTIONS)) {
114
+ const row = actions.parentElement
115
+ if (!row || row.querySelector(`:scope > [${HOST}]`)) continue
116
+ const host = document.createElement("div")
117
+ host.setAttribute(HOST, "")
118
+ row.insertBefore(host, actions)
119
+ mounted.set(
120
+ host,
121
+ render(() => <MicButton host={host} model={model} />, host),
122
+ )
123
+ }
124
+ for (const [host, dispose] of mounted) {
125
+ if (host.isConnected) continue
126
+ dispose()
127
+ mounted.delete(host)
128
+ }
129
+ setCount(mounted.size)
130
+ }
131
+
132
+ const observer = new MutationObserver(() => {
133
+ if (!frame) frame = requestAnimationFrame(scan)
134
+ })
135
+ observer.observe(document.body, { childList: true, subtree: true })
136
+ scan()
137
+
138
+ return {
139
+ count,
140
+ dispose() {
141
+ observer.disconnect()
142
+ if (frame) cancelAnimationFrame(frame)
143
+ for (const [host, dispose] of mounted) {
144
+ dispose()
145
+ host.remove()
146
+ }
147
+ mounted.clear()
148
+ },
149
+ }
150
+ }
151
+
152
+ /** The composer element a mic host, or failing that the focused or first visible composer, belongs to. */
153
+ export function composerOf(host: HTMLElement | undefined) {
154
+ const from = host?.closest<HTMLElement>('[data-component="composer"]')
155
+ if (from?.isConnected) return from
156
+ const focused = document.activeElement?.closest<HTMLElement>('[data-component="composer"]')
157
+ if (focused) return focused
158
+ return [...document.querySelectorAll<HTMLElement>('[data-component="composer"]')].find((el) => el.offsetParent)
159
+ }
160
+
161
+ /** Types text into a composer's editor through the browser's own input path, for composers the plugin API can't reach. */
162
+ export function typeInto(composer: HTMLElement, text: string) {
163
+ const editor = composer.querySelector<HTMLElement>('[data-component="composer-editor"]')
164
+ if (!editor) return false
165
+ editor.focus()
166
+ return document.execCommand("insertText", false, text)
167
+ }
package/src/main.ts ADDED
@@ -0,0 +1,13 @@
1
+ import { systemPreferences } from "electron"
2
+ import type { Plugin } from "@opencode/plugin/gui"
3
+ import { Voice } from "./contract"
4
+
5
+ export const main: Plugin.MainSetup = (ctx) => {
6
+ ctx.provide(Voice, {
7
+ requestMicrophone: async () => {
8
+ if (process.platform !== "darwin") return true
9
+ if (systemPreferences.getMediaAccessStatus("microphone") === "granted") return true
10
+ return systemPreferences.askForMediaAccess("microphone")
11
+ },
12
+ })
13
+ }
package/src/rpc.ts ADDED
@@ -0,0 +1,52 @@
1
+ import { Rpc } from "@opencode/plugin/rpc"
2
+
3
+ const reason = {
4
+ type: "object",
5
+ properties: { message: { type: "string" } },
6
+ required: ["message"],
7
+ additionalProperties: false,
8
+ } as const
9
+
10
+ export const VoiceRpc = Rpc.define({
11
+ id: "voice-input",
12
+ methods: {
13
+ transcribe: {
14
+ input: {
15
+ type: "object",
16
+ properties: {
17
+ audio: { type: "string", description: "Base64-encoded recording" },
18
+ mimeType: { type: "string" },
19
+ language: { type: "string" },
20
+ },
21
+ required: ["audio", "mimeType"],
22
+ additionalProperties: false,
23
+ },
24
+ output: {
25
+ type: "object",
26
+ properties: {
27
+ text: { type: "string" },
28
+ account: { type: "string" },
29
+ },
30
+ required: ["text", "account"],
31
+ additionalProperties: false,
32
+ },
33
+ errors: {
34
+ no_account: reason,
35
+ unsupported_account: reason,
36
+ upstream: reason,
37
+ },
38
+ },
39
+ },
40
+ events: {},
41
+ })
42
+
43
+ export interface TranscribeInput {
44
+ audio: string
45
+ mimeType: string
46
+ language?: string
47
+ }
48
+
49
+ export interface TranscribeOutput {
50
+ text: string
51
+ account: string
52
+ }
@@ -0,0 +1,122 @@
1
+ import { execFile, spawn, type ChildProcess } from "node:child_process"
2
+ import { randomUUID } from "node:crypto"
3
+ import { existsSync } from "node:fs"
4
+ import { readFile, rm } from "node:fs/promises"
5
+ import { homedir, tmpdir } from "node:os"
6
+ import { delimiter, join } from "node:path"
7
+
8
+ const findBinary = (name: string, override?: unknown) => {
9
+ if (typeof override === "string" && override.trim()) return override.trim().replace(/^~(?=\/)/, homedir())
10
+ const dirs = [...(process.env.PATH ?? "").split(delimiter), "/opt/homebrew/bin", "/usr/local/bin", "/usr/bin"]
11
+ return dirs.filter(Boolean).map((dir) => join(dir, name)).find((path) => existsSync(path))
12
+ }
13
+
14
+ const run = (command: string, args: string[], input?: string) =>
15
+ new Promise<string>((resolve, reject) => {
16
+ const child = execFile(command, args, { maxBuffer: 16 * 1024 * 1024 }, (error, stdout) =>
17
+ error ? reject(error) : resolve(String(stdout)),
18
+ )
19
+ if (input !== undefined) child.stdin?.end(input)
20
+ })
21
+
22
+ // ffmpeg's microphone input for this OS: the system default device unless `device` names another.
23
+ const inputArgs = (device: string | undefined) => {
24
+ // For testing without a microphone: plays a file in real time as if it were spoken.
25
+ const testFile = process.env.OPENCODE_VOICE_INPUT_TEST_FILE
26
+ if (testFile) return ["-re", "-i", testFile]
27
+ if (process.platform === "darwin") return ["-f", "avfoundation", "-i", `:${device || "default"}`]
28
+ if (process.platform === "linux") return ["-f", "pulse", "-i", device || "default"]
29
+ if (process.platform === "win32" && device) return ["-f", "dshow", "-i", `audio=${device}`]
30
+ throw new Error("Set the plugin's `microphone` option to your DirectShow microphone name to record on Windows.")
31
+ }
32
+
33
+ export class FfmpegRecorder {
34
+ private child?: ChildProcess
35
+ private file?: string
36
+ private stderr = ""
37
+ private exited?: Promise<number | null>
38
+
39
+ constructor(private readonly options: { ffmpegPath?: unknown; microphone?: unknown }) {}
40
+
41
+ async start() {
42
+ const ffmpeg = findBinary("ffmpeg", this.options.ffmpegPath)
43
+ if (!ffmpeg) throw new Error("ffmpeg was not found. Install it (`brew install ffmpeg`) or set the `ffmpegPath` option.")
44
+ const device = typeof this.options.microphone === "string" ? this.options.microphone : undefined
45
+ this.file = join(tmpdir(), `opencode-voice-${randomUUID()}.webm`)
46
+ this.stderr = ""
47
+ const child = spawn(
48
+ ffmpeg,
49
+ ["-hide_banner", "-loglevel", "error", ...inputArgs(device), "-ac", "1", "-ar", "16000", "-c:a", "libopus",
50
+ "-b:a", "32k", "-y", this.file],
51
+ { stdio: ["pipe", "ignore", "pipe"] },
52
+ )
53
+ this.child = child
54
+ child.stdin?.on("error", () => undefined)
55
+ child.stderr?.on("data", (chunk) => (this.stderr = (this.stderr + chunk).slice(-2000)))
56
+ this.exited = new Promise((resolve) => {
57
+ child.once("exit", (code) => resolve(code))
58
+ child.once("error", () => resolve(-1))
59
+ })
60
+ // ffmpeg fails within a moment when the device or the microphone permission is missing.
61
+ const early = await Promise.race([this.exited, new Promise((resolve) => setTimeout(() => resolve("running"), 400))])
62
+ if (early !== "running") {
63
+ await this.cleanup()
64
+ throw new Error(`Could not record: ${this.stderr.trim().split("\n").pop() || `ffmpeg exited with ${early}`}`)
65
+ }
66
+ }
67
+
68
+ /** Stops recording and returns the WebM/Opus file's contents. */
69
+ async stop() {
70
+ const child = this.child
71
+ const file = this.file
72
+ if (!child || !file) throw new Error("Not recording")
73
+ // "q" makes ffmpeg finish the file properly; SIGINT does the same if it doesn't answer.
74
+ if (child.exitCode === null) child.stdin?.end("q")
75
+ const timer = setTimeout(() => child.kill("SIGINT"), 3000)
76
+ await this.exited
77
+ clearTimeout(timer)
78
+ try {
79
+ return await readFile(file)
80
+ } finally {
81
+ await this.cleanup()
82
+ }
83
+ }
84
+
85
+ async cancel() {
86
+ this.child?.kill("SIGKILL")
87
+ await this.cleanup()
88
+ }
89
+
90
+ private async cleanup() {
91
+ if (this.file) await rm(this.file, { force: true })
92
+ this.child = undefined
93
+ this.file = undefined
94
+ }
95
+ }
96
+
97
+ const clipboardCommands = () => {
98
+ if (process.platform === "darwin") return { read: ["pbpaste"], write: ["pbcopy"] }
99
+ if (process.platform === "win32")
100
+ return { read: ["powershell.exe", "-NoProfile", "-Command", "Get-Clipboard -Raw"], write: ["clip.exe"] }
101
+ if (process.env.WAYLAND_DISPLAY && findBinary("wl-copy"))
102
+ return { read: ["wl-paste", "--no-newline"], write: ["wl-copy"] }
103
+ if (findBinary("xclip"))
104
+ return { read: ["xclip", "-selection", "clipboard", "-o"], write: ["xclip", "-selection", "clipboard"] }
105
+ return undefined
106
+ }
107
+
108
+ export const clipboard = {
109
+ available: () => clipboardCommands() !== undefined,
110
+ async read() {
111
+ const commands = clipboardCommands()
112
+ if (!commands) return undefined
113
+ const [command, ...args] = commands.read
114
+ return run(command!, args).catch(() => undefined)
115
+ },
116
+ async write(text: string) {
117
+ const commands = clipboardCommands()
118
+ if (!commands) throw new Error("No clipboard tool found (pbcopy, wl-copy or xclip).")
119
+ const [command, ...args] = commands.write
120
+ await run(command!, args, text)
121
+ },
122
+ }
package/src/tui.tsx ADDED
@@ -0,0 +1,190 @@
1
+ /** @jsxImportSource @opentui/solid */
2
+ import { createSignal, Show } from "solid-js"
3
+ import { Plugin } from "@opencode/plugin/tui"
4
+ import { VoiceRpc } from "./rpc"
5
+ import { clipboard, FfmpegRecorder } from "./tui-audio"
6
+
7
+ type Phase = "idle" | "starting" | "recording" | "transcribing"
8
+
9
+ const COMMAND = "voice-input.dictate"
10
+ const MAX_RECORDING_MS = 10 * 60_000
11
+ // prompt.paste reads the clipboard asynchronously; the previous clipboard text is put back after it has.
12
+ const RESTORE_DELAY_MS = 1500
13
+
14
+ const message = (error: unknown) => {
15
+ const data = (error as { data?: { message?: unknown } } | undefined)?.data
16
+ if (typeof data?.message === "string") return data.message
17
+ return error instanceof Error ? error.message : String(error)
18
+ }
19
+
20
+ export default Plugin.define({
21
+ id: "voice-input.tui",
22
+ setup(context) {
23
+ const options = context.options
24
+ const language = typeof options.language === "string" ? options.language : ""
25
+ const restoreClipboard = options.restoreClipboard !== false
26
+ const recorder = new FfmpegRecorder({ ffmpegPath: options.ffmpegPath, microphone: options.microphone })
27
+ const [phase, setPhase] = createSignal<Phase>("idle")
28
+ const [since, setSince] = createSignal(0)
29
+ const [now, setNow] = createSignal(Date.now())
30
+ let ticker: ReturnType<typeof setInterval> | undefined
31
+ let restoreTimer: ReturnType<typeof setTimeout> | undefined
32
+ let last = ""
33
+
34
+ const stopTicker = () => {
35
+ if (ticker) clearInterval(ticker)
36
+ ticker = undefined
37
+ }
38
+
39
+ const fail = (error: unknown) => {
40
+ stopTicker()
41
+ setPhase("idle")
42
+ context.ui.toast.show({ variant: "error", title: "Voice input failed", message: message(error) })
43
+ }
44
+
45
+ // The TUI plugin API cannot write to the prompt, so the transcript goes through the clipboard and the prompt's own
46
+ // paste command, which inserts it at the cursor. The previous clipboard text is restored afterwards.
47
+ const paste = async (text: string) => {
48
+ const pasteReachable = context.keymap.commands().some((command) => command.id === "prompt.paste")
49
+ const previous = restoreClipboard && pasteReachable ? await clipboard.read() : undefined
50
+ await clipboard.write(text)
51
+ if (!pasteReachable) {
52
+ context.ui.toast.show({ variant: "info", message: "Transcript copied. Paste it into the prompt." })
53
+ return
54
+ }
55
+ context.keymap.dispatch("prompt.paste")
56
+ if (previous === undefined || previous === "") return
57
+ if (restoreTimer) clearTimeout(restoreTimer)
58
+ restoreTimer = setTimeout(() => void clipboard.write(previous).catch(() => undefined), RESTORE_DELAY_MS)
59
+ }
60
+
61
+ const start = async () => {
62
+ if (phase() !== "idle") return
63
+ setPhase("starting")
64
+ try {
65
+ await recorder.start()
66
+ const started = Date.now()
67
+ setSince(started)
68
+ setNow(started)
69
+ setPhase("recording")
70
+ ticker = setInterval(() => {
71
+ setNow(Date.now())
72
+ if (Date.now() - started > MAX_RECORDING_MS) void stop()
73
+ }, 500)
74
+ } catch (error) {
75
+ fail(error)
76
+ }
77
+ }
78
+
79
+ const stop = async () => {
80
+ if (phase() !== "recording") return
81
+ stopTicker()
82
+ setPhase("transcribing")
83
+ try {
84
+ const audio = await recorder.stop()
85
+ if (audio.byteLength === 0) throw new Error("Nothing was recorded")
86
+ const result = await context.client.rpc(VoiceRpc).transcribe({
87
+ audio: audio.toString("base64"),
88
+ mimeType: "audio/webm",
89
+ language,
90
+ })
91
+ const text = (result as { text: string }).text.trim()
92
+ setPhase("idle")
93
+ if (!text) {
94
+ context.ui.toast.show({ variant: "info", message: "No speech was heard" })
95
+ return
96
+ }
97
+ last = text
98
+ await paste(`${text} `)
99
+ } catch (error) {
100
+ fail(error)
101
+ }
102
+ }
103
+
104
+ const cancel = async () => {
105
+ if (phase() !== "recording") return false
106
+ stopTicker()
107
+ await recorder.cancel()
108
+ setPhase("idle")
109
+ }
110
+
111
+ const toggle = () => {
112
+ if (phase() === "idle") void start()
113
+ else if (phase() === "recording") void stop()
114
+ }
115
+
116
+ const elapsed = () => {
117
+ const seconds = Math.max(0, Math.floor((now() - since()) / 1000))
118
+ return `${Math.floor(seconds / 60)}:${String(seconds % 60).padStart(2, "0")}`
119
+ }
120
+
121
+ context.keymap.layer(() => ({
122
+ mode: "global",
123
+ commands: [
124
+ {
125
+ id: COMMAND,
126
+ title: "Start or stop voice dictation",
127
+ group: "Voice input",
128
+ bind: "<leader>v",
129
+ palette: true,
130
+ slash: { name: "dictate" },
131
+ run: () => toggle(),
132
+ },
133
+ {
134
+ id: "voice-input.copy-last",
135
+ title: "Copy last dictation",
136
+ group: "Voice input",
137
+ palette: true,
138
+ enabled: () => last !== "",
139
+ run: () =>
140
+ void clipboard
141
+ .write(last)
142
+ .then(() => context.ui.toast.show({ variant: "success", message: "Last dictation copied" })),
143
+ },
144
+ {
145
+ id: "voice-input.cancel",
146
+ title: "Cancel voice dictation",
147
+ group: "Voice input",
148
+ palette: true,
149
+ enabled: () => phase() === "recording",
150
+ run: () => void cancel(),
151
+ },
152
+ ],
153
+ }))
154
+
155
+ // The footer shows nothing while idle; it only shows that the mic is recording or transcribing.
156
+ const removeSlot = context.ui.slot({
157
+ append: "prompt.footer.status",
158
+ render: () => {
159
+ const theme = context.theme
160
+ const label = () => {
161
+ switch (phase()) {
162
+ case "recording":
163
+ return `● ${elapsed()} stop`
164
+ case "transcribing":
165
+ return "◌ transcribing…"
166
+ case "starting":
167
+ return "◌ mic…"
168
+ default:
169
+ return ""
170
+ }
171
+ }
172
+ const color = () => (phase() === "recording" ? theme.text.feedback.error.base : theme.text.feedback.warning.base)
173
+ return (
174
+ <Show when={phase() !== "idle"}>
175
+ <text fg={color()} onMouseDown={() => toggle()}>
176
+ {label()}
177
+ </text>
178
+ </Show>
179
+ )
180
+ },
181
+ })
182
+
183
+ return () => {
184
+ removeSlot()
185
+ stopTicker()
186
+ if (restoreTimer) clearTimeout(restoreTimer)
187
+ void recorder.cancel()
188
+ }
189
+ },
190
+ })