@allmodels/dsh-speech 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -5
- package/lib/client.js +90 -18
- package/lib/index.js +15 -8
- package/package.json +4 -1
package/README.md
CHANGED
|
@@ -33,7 +33,7 @@ Or create a portable tarball:
|
|
|
33
33
|
|
|
34
34
|
```sh
|
|
35
35
|
pnpm pack
|
|
36
|
-
dsh plugin --profile web add ./allmodels-dsh-speech-0.1.
|
|
36
|
+
dsh plugin --profile web add ./allmodels-dsh-speech-0.1.2.tgz
|
|
37
37
|
```
|
|
38
38
|
|
|
39
39
|
Restart the profile after adding or removing the bundle.
|
|
@@ -54,9 +54,9 @@ See the [AllModels agentic setup documentation](https://docs.allmodels.io/agenti
|
|
|
54
54
|
1. Click the microphone next to Send.
|
|
55
55
|
2. Speak while the amplitude bar and partial transcription update.
|
|
56
56
|
3. While recording, choose **System default** or another input from the microphone selector beside TTFT. On a new chat, the selector appears beside the mode selector in the hero controls row.
|
|
57
|
-
4. Click Stop
|
|
57
|
+
4. Click Stop to finalize and leave the text as an editable draft, or click the Send arrow to finalize and send in one action. The Send arrow becomes available after speech text first appears.
|
|
58
58
|
|
|
59
|
-
|
|
59
|
+
Existing draft text is preserved and the transcript is appended. Composer editing remains locked while the microphone is active. Stop never sends; the recording Send arrow submits exactly once after finalization.
|
|
60
60
|
|
|
61
61
|
Only one microphone may be active at a time. Leaving the originating session immediately stops capture, closes the speech socket, clears the transient composer lock, and preserves the latest draft text.
|
|
62
62
|
|
|
@@ -96,7 +96,7 @@ The bundle row may be given composition-level defaults in a Harness patch:
|
|
|
96
96
|
|
|
97
97
|
- Audio is resampled in the browser and relayed through the local plugin host only while recording. It is never written to disk or retained for retry.
|
|
98
98
|
- API keys remain host-side in the Harness credential service and are inserted only into authenticated AllModels requests.
|
|
99
|
-
- Transcripts
|
|
99
|
+
- Transcripts remain in the browser's composer draft until the user uses the normal Send arrow or the recording Send arrow.
|
|
100
100
|
- The preferred microphone device ID is kept only in plugin-namespaced browser-local storage. It is not written to Harness sessions.
|
|
101
101
|
- The host plugin does not inject Harness session services, import session packages, read session files, or write session files. A build check fails if a session/filesystem persistence dependency is introduced.
|
|
102
102
|
- The plugin's custom HTTP and WebSocket routes accept only same-origin loopback requests and enforce bounded request/frame sizes.
|
|
@@ -117,14 +117,29 @@ The only durable plugin data is its namespaced settings, credential, and browser
|
|
|
117
117
|
```sh
|
|
118
118
|
pnpm install
|
|
119
119
|
pnpm check
|
|
120
|
+
pnpm exec playwright install chromium
|
|
121
|
+
pnpm test:e2e
|
|
120
122
|
pnpm pack --dry-run
|
|
121
123
|
```
|
|
122
124
|
|
|
123
125
|
`pnpm check` runs the no-session-access boundary, TypeScript, unit/protocol tests, and both production bundles. Tests cover catalog defaults, provider capabilities, targeted promotional balances, transcript sequencing/CJK spacing, route security, credential redaction behavior, and the AllModels streaming wire mapping.
|
|
124
126
|
|
|
127
|
+
`pnpm test:e2e` builds and packs the plugin, installs that tarball into a disposable DeepSeek Harness `0.1.1-rc.2` web profile, launches a local AllModels HTTP/WebSocket mock, and runs the Chromium desktop and mobile suite with a generated virtual microphone. It covers first-run account UI, API-key connection, settings layout and persistence, balance/top-up, microphone placement, preparing/recording/finishing states, live partials and finals, waveform motion, composer locking, Stop-without-send, finish-and-send, Cancel, microphone selection, dark-mode menus, and mobile layout. The disposable profile and workspace are created below the operating system's temporary directory; the runner never opens or modifies the user's Harness profile or sessions.
|
|
128
|
+
|
|
129
|
+
For the optional live smoke test, put the following in a local `.secrets` file (which is gitignored), or export the variables normally:
|
|
130
|
+
|
|
131
|
+
```dotenv
|
|
132
|
+
ALLMODELS_API_KEY=...
|
|
133
|
+
DEEPSEEK_API_KEY=...
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
Then run `pnpm test:e2e:live`. The live test checks the environment-managed AllModels account, starts and stops a real streaming STT connection using the virtual microphone, sends a minimal DeepSeek request, and verifies microphone placement in both new-chat and existing-session layouts. It consumes a small amount of API balance. The coordinator removes both credentials from its ambient environment before starting build, package-manager, and Playwright children; only the isolated Harness process receives them. Live Harness output and Playwright traces, screenshots, videos, HTML reports, and artifact uploads are disabled so credentials cannot be reflected into CI diagnostics.
|
|
137
|
+
|
|
125
138
|
## Release
|
|
126
139
|
|
|
127
|
-
|
|
140
|
+
Every branch push and pull request to `main` runs the static/unit/package gates followed by the deterministic Playwright suite. A push to `main` additionally runs the live browser smoke test with the `ALLMODELS_API_KEY` and `DEEPSEEK_API_KEY` repository secrets. Secrets are never exposed to pull-request jobs.
|
|
141
|
+
|
|
142
|
+
After all jobs for a `main` push pass, the Release workflow checks out the exact tested commit and publishes its package version to npm with provenance if that version is not already present. It then tags the commit and creates the matching GitHub release. Before enabling this gate, configure both repository secrets and configure `allmodels-io/dsh-speech`'s `release.yml` workflow as the npm trusted publisher for `@allmodels/dsh-speech`. Increment `package.json` before merging a version intended for publication; already-published versions are safely skipped.
|
|
128
143
|
|
|
129
144
|
## License
|
|
130
145
|
|
package/lib/client.js
CHANGED
|
@@ -330,7 +330,8 @@ var SpeechController = class {
|
|
|
330
330
|
amplitude: 0,
|
|
331
331
|
metadataLoading: false,
|
|
332
332
|
microphones: [],
|
|
333
|
-
switchingMicrophone: false
|
|
333
|
+
switchingMicrophone: false,
|
|
334
|
+
hasTranscript: false
|
|
334
335
|
};
|
|
335
336
|
listeners = /* @__PURE__ */ new Set();
|
|
336
337
|
blocks;
|
|
@@ -408,6 +409,12 @@ var SpeechController = class {
|
|
|
408
409
|
}, 5e3);
|
|
409
410
|
await stoppingCapture;
|
|
410
411
|
}
|
|
412
|
+
async stopAndSend() {
|
|
413
|
+
const active = this.active;
|
|
414
|
+
if (active === void 0 || active.finished || !this.snapshot.hasTranscript) return;
|
|
415
|
+
active.sendAfterFinish = true;
|
|
416
|
+
await this.stop();
|
|
417
|
+
}
|
|
411
418
|
cancel() {
|
|
412
419
|
const active = this.active;
|
|
413
420
|
if (active === void 0 || active.finished) return;
|
|
@@ -473,7 +480,8 @@ var SpeechController = class {
|
|
|
473
480
|
microphones: [],
|
|
474
481
|
activeMicrophoneId: void 0,
|
|
475
482
|
switchingMicrophone: false,
|
|
476
|
-
microphoneError: void 0
|
|
483
|
+
microphoneError: void 0,
|
|
484
|
+
hasTranscript: false
|
|
477
485
|
});
|
|
478
486
|
try {
|
|
479
487
|
await new Promise((resolve, reject) => {
|
|
@@ -555,6 +563,7 @@ var SpeechController = class {
|
|
|
555
563
|
...typeof message.languageCode === "string" ? { languageCode: message.languageCode } : {}
|
|
556
564
|
});
|
|
557
565
|
active.inputActions.setDraft(transcriptText(active.transcript));
|
|
566
|
+
if (message.text.trim().length > 0 && !this.snapshot.hasTranscript) this.publish({ hasTranscript: true });
|
|
558
567
|
if (message.type === "final" && this.snapshot.phase === "finalizing") {
|
|
559
568
|
if (active.finalTimer !== void 0) clearTimeout(active.finalTimer);
|
|
560
569
|
active.finalTimer = setTimeout(() => {
|
|
@@ -595,8 +604,10 @@ var SpeechController = class {
|
|
|
595
604
|
activeMicrophoneId: void 0,
|
|
596
605
|
switchingMicrophone: false,
|
|
597
606
|
microphoneError: void 0,
|
|
607
|
+
hasTranscript: false,
|
|
598
608
|
...error === void 0 ? { error: void 0 } : { error }
|
|
599
609
|
});
|
|
610
|
+
if (error === void 0 && active.sendAfterFinish === true) active.inputActions.submit();
|
|
600
611
|
this.ensureMetadata(true);
|
|
601
612
|
}
|
|
602
613
|
publish(patch) {
|
|
@@ -667,7 +678,9 @@ const styles = {
|
|
|
667
678
|
recordingTrack: "dsh-speech-recording-track",
|
|
668
679
|
recordingCanvas: "dsh-speech-recording-canvas",
|
|
669
680
|
recordingCancel: "dsh-speech-recording-cancel",
|
|
681
|
+
recordingActions: "dsh-speech-recording-actions",
|
|
670
682
|
recordingStop: "dsh-speech-recording-stop",
|
|
683
|
+
recordingSend: "dsh-speech-recording-send",
|
|
671
684
|
recordingProgress: "dsh-speech-recording-progress",
|
|
672
685
|
deviceDock: "dsh-speech-device-dock",
|
|
673
686
|
deviceMarker: "dsh-speech-device-marker",
|
|
@@ -699,7 +712,7 @@ const STYLE_TEXT = String.raw`
|
|
|
699
712
|
.dsh-speech-primary,.dsh-speech-secondary,.dsh-speech-checkout{border:1px solid transparent;border-radius:8px;padding:8px 12px;font:inherit;font-size:13px;font-weight:600;cursor:pointer;text-decoration:none;text-align:center}.dsh-speech-primary{background:#e8e8e8;color:#151515}.dsh-speech-secondary,.dsh-speech-checkout{border-color:var(--color-border,#4a4a4a);background:transparent;color:inherit}.dsh-speech-primary:disabled,.dsh-speech-secondary:disabled{cursor:default;opacity:.45}.dsh-speech-good{color:#65c88a}.dsh-speech-warning{margin:0;color:#efb85c}.dsh-speech-danger,.dsh-speech-dock-error{margin:0;color:#f07878}
|
|
700
713
|
.dsh-speech-top-up{justify-content:flex-start;align-items:end;flex-wrap:wrap}.dsh-speech-top-up .dsh-speech-field{max-width:180px}
|
|
701
714
|
.dsh-speech-mic-wrap{position:relative;display:inline-flex;align-items:center;gap:5px;order:1}[data-slot="conversation.input.right"]:has(>.dsh-speech-mic-wrap)~button:last-child{order:2}.dsh-speech-mic{width:30px;height:30px;display:inline-grid;place-items:center;padding:0;border:0;border-radius:7px;background:transparent;color:var(--color-text-secondary,#a8a8a8);cursor:pointer}.dsh-speech-mic:hover:not(:disabled),.dsh-speech-mic:focus-visible{background:var(--color-background-hover,rgba(255,255,255,.08));color:var(--color-text,#eee)}.dsh-speech-mic[aria-disabled="true"]{opacity:.48}.dsh-speech-mic:disabled{cursor:default}.dsh-speech-mic svg{width:18px;height:18px;fill:currentColor}.dsh-speech-mic-active{color:#ef6666;background:rgba(239,102,102,.12)}.dsh-speech-mic-tooltip{position:absolute;right:0;bottom:calc(100% + 8px);z-index:40;width:max-content;max-width:260px;padding:6px 9px;border:1px solid var(--color-border,#343434);border-radius:7px;background:var(--color-background-elevated,#202020);color:var(--color-text,#eee);font-size:12px;line-height:1.35;box-shadow:0 6px 18px rgba(0,0,0,.24);opacity:0;visibility:hidden;transform:translateY(2px);pointer-events:none;transition:opacity .12s ease,transform .12s ease,visibility .12s}.dsh-speech-mic-wrap:hover .dsh-speech-mic-tooltip,.dsh-speech-mic-wrap:focus-within .dsh-speech-mic-tooltip,.dsh-speech-mic-wrap[data-explanation-open="true"] .dsh-speech-mic-tooltip{opacity:1;visibility:visible;transform:translateY(0)}
|
|
702
|
-
:where(div):has(>:where(div)>[data-slot="conversation.input.right"]>.dsh-speech-recording-takeover){position:relative}:where(div):has(>:where(div)>[data-slot="conversation.input.right"]>.dsh-speech-recording-takeover)>*{visibility:hidden}.dsh-speech-recording-takeover{position:absolute;inset:0;z-index:8;visibility:visible;display:flex;align-items:center;gap:10px;padding:4px 8px;box-sizing:border-box;color:var(--color-text-secondary,#a8a8a8)}.dsh-speech-recording-track{height:30px;min-width:0;flex:1;display:flex;align-items:center}.dsh-speech-recording-canvas{display:block;width:100%;height:30px;color:var(--color-text-secondary,#a8a8a8)}.dsh-speech-recording-cancel,.dsh-speech-recording-stop{flex:none;border-radius:999px}.dsh-speech-recording-cancel{color:var(--color-text-secondary,#a8a8a8);background:transparent}.dsh-speech-recording-cancel:hover:not(:disabled),.dsh-speech-recording-cancel:focus-visible{background:var(--color-background-hover,rgba(255,255,255,.08));color:var(--color-text,#eee)}.dsh-speech-recording-stop{color:#ef6666;background:rgba(239,102,102,.12)}.dsh-speech-recording-stop:hover:not(:disabled),.dsh-speech-recording-stop:focus-visible{color:#ff7777;background:rgba(239,102,102,.2)}.dsh-speech-recording-progress{display:flex;align-items:center;justify-content:flex-end;gap:8px;width:100%;color:var(--color-text-secondary,#a8a8a8);font-size:12px}.dsh-speech-recording-takeover[data-phase="starting"] .dsh-speech-recording-progress{justify-content:center}.dsh-speech-recording-progress i{width:13px;height:13px;border:2px solid currentColor;border-right-color:transparent;border-radius:50%;animation:dsh-speech-spin .8s linear infinite}.dsh-speech-sr-only{position:absolute;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip:rect(0,0,0,0);white-space:nowrap;border:0}@keyframes dsh-speech-spin{to{transform:rotate(360deg)}}
|
|
715
|
+
:where(div):has(>:where(div)>[data-slot="conversation.input.right"]>.dsh-speech-recording-takeover){position:relative}:where(div):has(>:where(div)>[data-slot="conversation.input.right"]>.dsh-speech-recording-takeover)>*{visibility:hidden}.dsh-speech-recording-takeover{position:absolute;inset:0;z-index:8;visibility:visible;display:flex;align-items:center;gap:10px;padding:4px 8px;box-sizing:border-box;color:var(--color-text-secondary,#a8a8a8)}.dsh-speech-recording-track{height:30px;min-width:0;flex:1;display:flex;align-items:center}.dsh-speech-recording-canvas{display:block;width:100%;height:30px;color:var(--color-text-secondary,#a8a8a8)}.dsh-speech-recording-cancel,.dsh-speech-recording-stop{flex:none;border-radius:999px}.dsh-speech-recording-cancel{color:var(--color-text-secondary,#a8a8a8);background:transparent}.dsh-speech-recording-cancel:hover:not(:disabled),.dsh-speech-recording-cancel:focus-visible{background:var(--color-background-hover,rgba(255,255,255,.08));color:var(--color-text,#eee)}.dsh-speech-recording-actions{display:flex;align-items:center;gap:6px;flex:none}.dsh-speech-recording-stop{color:#ef6666;background:rgba(239,102,102,.12)}.dsh-speech-recording-stop:hover:not(:disabled),.dsh-speech-recording-stop:focus-visible{color:#ff7777;background:rgba(239,102,102,.2)}.dsh-speech-recording-send{width:34px;height:34px;display:inline-grid;place-items:center;flex:none;padding:0;border:0;border-radius:999px;background:var(--dsw-alias-button-info-fill,#679efe);color:#fff;cursor:pointer}.dsh-speech-recording-send:hover:not(:disabled),.dsh-speech-recording-send:focus-visible{outline:0;background:var(--dsw-alias-button-info-hover,#4176e6)}.dsh-speech-recording-send:disabled{cursor:default;opacity:.4}.dsh-speech-recording-send svg{width:16px;height:16px}.dsh-speech-recording-progress{display:flex;align-items:center;justify-content:flex-end;gap:8px;width:100%;color:var(--color-text-secondary,#a8a8a8);font-size:12px}.dsh-speech-recording-takeover[data-phase="starting"] .dsh-speech-recording-progress{justify-content:center}.dsh-speech-recording-progress i{width:13px;height:13px;border:2px solid currentColor;border-right-color:transparent;border-radius:50%;animation:dsh-speech-spin .8s linear infinite}.dsh-speech-sr-only{position:absolute;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip:rect(0,0,0,0);white-space:nowrap;border:0}@keyframes dsh-speech-spin{to{transform:rotate(360deg)}}
|
|
703
716
|
.dsh-speech-device-marker{display:none}[data-slot="conversation.composer.dock"]>:has(>.dsh-speech-device-dock){overflow:visible}.dsh-speech-device-fallback{min-height:22px;display:flex;align-items:center;justify-content:flex-start;padding:0 8px;overflow:visible}.dsh-speech-device-fallback .dsh-speech-device-dock{margin-left:0}.dsh-speech-device-separator{margin-left:4px;color:var(--dsw-alias-label-tertiary,var(--color-text-secondary,#888))}.dsh-speech-device-dock{position:relative;display:inline-flex;vertical-align:middle;margin-left:5px;color:var(--dsw-alias-label-secondary,var(--color-text-secondary,#a8a8a8));font:inherit}.dsh-speech-device-trigger{max-width:126px;height:20px;display:inline-flex;align-items:center;gap:4px;padding:0 5px;border:0;border-radius:6px;background:transparent;color:inherit;font:inherit;font-size:11px;line-height:20px;cursor:pointer}.dsh-speech-device-trigger:hover,.dsh-speech-device-trigger:focus-visible,.dsh-speech-device-trigger[aria-expanded="true"]{outline:0;background:var(--dsw-alias-interactive-bg-hover,rgba(255,255,255,.08));color:var(--dsw-alias-label-primary,var(--color-text,#eee))}.dsh-speech-device-trigger:disabled{cursor:wait;opacity:.6}.dsh-speech-device-dock[data-error="true"] .dsh-speech-device-trigger{color:var(--dsw-alias-state-error-primary,#f07878)}.dsh-speech-device-icon,.dsh-speech-device-chevron{display:inline-flex;flex:none}.dsh-speech-device-icon svg{width:12px;height:12px}.dsh-speech-device-chevron svg{width:10px;height:10px}.dsh-speech-device-label{min-width:0;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}.dsh-speech-device-menu{position:absolute;right:0;bottom:calc(100% + 6px);z-index:80;min-width:240px;max-width:min(320px,calc(100vw - 32px));display:grid;box-sizing:border-box;padding:4px;border:1px solid var(--dsw-alias-border-l2-darkmode-thin,rgba(255,255,255,.06));border-radius:12px;background:var(--dsw-alias-bg-layer-3,#353638);color:var(--dsw-alias-label-primary,#f9fafb);box-shadow:0 0 1px rgba(0,0,0,.2),0 0 4px rgba(0,0,0,.02),0 12px 32px rgba(0,0,0,.08);font:13px/20px -apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif}.dsh-speech-device-menu-item{width:100%;min-width:0;height:38px;display:flex;align-items:center;justify-content:space-between;gap:12px;padding:0 10px;border:0;border-radius:10px;background:transparent;color:inherit;font:inherit;text-align:left;cursor:pointer}.dsh-speech-device-menu-item:hover,.dsh-speech-device-menu-item:focus-visible{outline:0;background:var(--dsw-alias-interactive-bg-hover,rgba(255,255,255,.08))}.dsh-speech-device-menu-item>span:first-child{min-width:0;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}.dsh-speech-device-check{width:14px;height:14px;display:inline-flex;flex:none;color:var(--dsw-alias-brand-primary,var(--dsw-alias-label-primary,#f9fafb))}.dsh-speech-device-check svg{width:14px;height:14px}.dsh-speech-device-dock[data-variant="hero"]{margin-left:0;color:var(--dsw-alias-label-primary,var(--color-text,#f9fafb));font:500 13px/20px -apple-system,BlinkMacSystemFont,"Segoe UI","PingFang SC","Hiragino Sans GB","Microsoft YaHei","Helvetica Neue",Helvetica,Arial,sans-serif}.dsh-speech-device-dock[data-variant="hero"] .dsh-speech-device-trigger{max-width:220px;height:28px;padding:0 8px;gap:4px;border-radius:16px;font:inherit;line-height:20px}.dsh-speech-device-dock[data-variant="hero"] .dsh-speech-device-icon svg{width:16px;height:16px}.dsh-speech-device-dock[data-variant="hero"] .dsh-speech-device-chevron svg{width:14px;height:14px}.dsh-speech-device-dock[data-variant="hero"] .dsh-speech-device-menu{left:0;right:auto;top:calc(100% + 6px);bottom:auto;min-width:218px}
|
|
704
717
|
[data-dsh-speech-nav]>svg{display:none}[data-dsh-speech-nav]::before{content:"";width:16px;height:16px;flex:none;background:currentColor;mask:url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 20 20'%3E%3Crect x='7' y='2' width='6' height='11' rx='3' fill='none' stroke='black' stroke-width='1.7'/%3E%3Cpath d='M4.5 9.5a5.5 5.5 0 0 0 11 0M10 15v3M7 18h6' fill='none' stroke='black' stroke-width='1.7' stroke-linecap='round'/%3E%3C/svg%3E") center/contain no-repeat}
|
|
705
718
|
.dsh-speech-dock,.dsh-speech-dock-error{min-height:22px;display:flex;align-items:center;gap:9px;padding:4px 8px 0;font-size:12px}.dsh-speech-dock-detail{color:var(--color-text-secondary,#8d8d8d)}
|
|
@@ -782,6 +795,16 @@ function CloseIcon() {
|
|
|
782
795
|
})
|
|
783
796
|
});
|
|
784
797
|
}
|
|
798
|
+
function SendIcon() {
|
|
799
|
+
return /* @__PURE__ */ (0, react_jsx_runtime.jsx)("svg", {
|
|
800
|
+
viewBox: "0 0 16 16",
|
|
801
|
+
"aria-hidden": "true",
|
|
802
|
+
children: /* @__PURE__ */ (0, react_jsx_runtime.jsx)("path", {
|
|
803
|
+
d: "M8.3125.980183c.35517.072917.66652.223997.9502.452147.22454.18064.46759.4256.71679.67481L14.707 6.83468l-1.414 1.41406L9 3.95577V15.0417H7V3.95577L2.70703 8.24874 1.29297 6.83468l4.72754-4.72754c.2492-.24921.49226-.49417.71679-.67481.23932-.19247.54715-.38831.9502-.452147.2098-.033177.4156-.025023.625 0Z",
|
|
804
|
+
fill: "currentColor"
|
|
805
|
+
})
|
|
806
|
+
});
|
|
807
|
+
}
|
|
785
808
|
const WAVEFORM_PITCH = 4;
|
|
786
809
|
const WAVEFORM_BASELINE = .025;
|
|
787
810
|
function RecordingWaveform({ amplitude }) {
|
|
@@ -851,7 +874,7 @@ function RecordingWaveform({ amplitude }) {
|
|
|
851
874
|
"aria-hidden": "true"
|
|
852
875
|
});
|
|
853
876
|
}
|
|
854
|
-
function RecordingTakeover({ amplitude, cancelLabel, disabled, phase, label, onCancel, onStop, stopLabel }) {
|
|
877
|
+
function RecordingTakeover({ amplitude, cancelLabel, disabled, phase, label, onCancel, onSend, onStop, sendDisabled, sendLabel, stopLabel }) {
|
|
855
878
|
const recording = phase === "recording";
|
|
856
879
|
return /* @__PURE__ */ (0, react_jsx_runtime.jsxs)("div", {
|
|
857
880
|
className: styles.recordingTakeover,
|
|
@@ -875,14 +898,25 @@ function RecordingTakeover({ amplitude, cancelLabel, disabled, phase, label, onC
|
|
|
875
898
|
children: [/* @__PURE__ */ (0, react_jsx_runtime.jsx)("i", { "aria-hidden": "true" }), label]
|
|
876
899
|
})
|
|
877
900
|
}),
|
|
878
|
-
recording ? /* @__PURE__ */ (0, react_jsx_runtime.
|
|
879
|
-
|
|
880
|
-
|
|
881
|
-
|
|
882
|
-
|
|
883
|
-
|
|
884
|
-
|
|
885
|
-
|
|
901
|
+
recording ? /* @__PURE__ */ (0, react_jsx_runtime.jsxs)("div", {
|
|
902
|
+
className: styles.recordingActions,
|
|
903
|
+
children: [/* @__PURE__ */ (0, react_jsx_runtime.jsx)("button", {
|
|
904
|
+
type: "button",
|
|
905
|
+
className: `${styles.mic} ${styles.recordingStop}`,
|
|
906
|
+
disabled,
|
|
907
|
+
"aria-label": stopLabel,
|
|
908
|
+
title: stopLabel,
|
|
909
|
+
onClick: onStop,
|
|
910
|
+
children: /* @__PURE__ */ (0, react_jsx_runtime.jsx)(MicIcon, { stopped: true })
|
|
911
|
+
}), /* @__PURE__ */ (0, react_jsx_runtime.jsx)("button", {
|
|
912
|
+
type: "button",
|
|
913
|
+
className: styles.recordingSend,
|
|
914
|
+
disabled: sendDisabled,
|
|
915
|
+
"aria-label": sendLabel,
|
|
916
|
+
title: sendLabel,
|
|
917
|
+
onClick: onSend,
|
|
918
|
+
children: /* @__PURE__ */ (0, react_jsx_runtime.jsx)(SendIcon, {})
|
|
919
|
+
})]
|
|
886
920
|
}) : null,
|
|
887
921
|
/* @__PURE__ */ (0, react_jsx_runtime.jsx)("span", {
|
|
888
922
|
className: styles.srOnly,
|
|
@@ -943,7 +977,14 @@ function SpeechMic({ controller, getLocale, sessionId, input, inputActions, t })
|
|
|
943
977
|
onCancel: () => {
|
|
944
978
|
controller.cancel();
|
|
945
979
|
},
|
|
980
|
+
onSend: () => {
|
|
981
|
+
controller.stopAndSend().catch((error) => {
|
|
982
|
+
controller.reportError(error instanceof Error ? error.message : "Unable to send voice input");
|
|
983
|
+
});
|
|
984
|
+
},
|
|
946
985
|
onStop: toggle,
|
|
986
|
+
sendDisabled: disabled || !state.hasTranscript,
|
|
987
|
+
sendLabel: t("micSend"),
|
|
947
988
|
stopLabel: title
|
|
948
989
|
});
|
|
949
990
|
}
|
|
@@ -1073,18 +1114,47 @@ function MicrophoneDockPortal({ controller, state, t }) {
|
|
|
1073
1114
|
const markerRef = (0, react.useRef)(null);
|
|
1074
1115
|
const [target, setTarget] = (0, react.useState)(null);
|
|
1075
1116
|
(0, react.useLayoutEffect)(() => {
|
|
1076
|
-
|
|
1077
|
-
|
|
1078
|
-
|
|
1117
|
+
let disposed = false;
|
|
1118
|
+
let frame;
|
|
1119
|
+
let observedSlot = null;
|
|
1120
|
+
const observer = new MutationObserver(() => {
|
|
1121
|
+
findTarget();
|
|
1122
|
+
});
|
|
1123
|
+
const retry = () => {
|
|
1124
|
+
if (disposed || frame !== void 0) return;
|
|
1125
|
+
frame = requestAnimationFrame(() => {
|
|
1126
|
+
frame = void 0;
|
|
1127
|
+
findTarget();
|
|
1128
|
+
});
|
|
1129
|
+
};
|
|
1079
1130
|
const findTarget = () => {
|
|
1131
|
+
if (disposed) return;
|
|
1132
|
+
const marker = markerRef.current;
|
|
1133
|
+
const slot = marker?.closest("[data-slot=\"conversation.composer.dock\"]");
|
|
1134
|
+
if (!(slot instanceof HTMLElement)) {
|
|
1135
|
+
setTarget(null);
|
|
1136
|
+
retry();
|
|
1137
|
+
return;
|
|
1138
|
+
}
|
|
1139
|
+
if (observedSlot !== slot) {
|
|
1140
|
+
observer.disconnect();
|
|
1141
|
+
observedSlot = slot;
|
|
1142
|
+
observer.observe(slot, {
|
|
1143
|
+
childList: true,
|
|
1144
|
+
subtree: true,
|
|
1145
|
+
characterData: true
|
|
1146
|
+
});
|
|
1147
|
+
}
|
|
1080
1148
|
const metrics = Array.from(slot.children).find((child) => child !== marker && child.textContent?.includes("TTFT"));
|
|
1081
|
-
|
|
1149
|
+
const nextTarget = metrics instanceof HTMLElement ? metrics : null;
|
|
1150
|
+
setTarget((current) => current === nextTarget ? current : nextTarget);
|
|
1151
|
+
if (nextTarget === null) retry();
|
|
1082
1152
|
};
|
|
1083
1153
|
findTarget();
|
|
1084
|
-
const observer = new MutationObserver(findTarget);
|
|
1085
|
-
observer.observe(slot, { childList: true });
|
|
1086
1154
|
return () => {
|
|
1155
|
+
disposed = true;
|
|
1087
1156
|
observer.disconnect();
|
|
1157
|
+
if (frame !== void 0) cancelAnimationFrame(frame);
|
|
1088
1158
|
};
|
|
1089
1159
|
}, []);
|
|
1090
1160
|
return /* @__PURE__ */ (0, react_jsx_runtime.jsxs)(react_jsx_runtime.Fragment, { children: [/* @__PURE__ */ (0, react_jsx_runtime.jsx)("span", {
|
|
@@ -1652,6 +1722,7 @@ const en = {
|
|
|
1652
1722
|
saved: "Saved",
|
|
1653
1723
|
micStart: "Start voice input",
|
|
1654
1724
|
micStop: "Stop voice input",
|
|
1725
|
+
micSend: "Finish voice input and send",
|
|
1655
1726
|
micCancel: "Cancel voice input",
|
|
1656
1727
|
starting: "Preparing microphone…",
|
|
1657
1728
|
listening: "Listening…",
|
|
@@ -1714,6 +1785,7 @@ const zh = {
|
|
|
1714
1785
|
saved: "已保存",
|
|
1715
1786
|
micStart: "开始语音输入",
|
|
1716
1787
|
micStop: "停止语音输入",
|
|
1788
|
+
micSend: "完成语音输入并发送",
|
|
1717
1789
|
micCancel: "取消语音输入",
|
|
1718
1790
|
starting: "正在准备麦克风…",
|
|
1719
1791
|
listening: "正在聆听…",
|
package/lib/index.js
CHANGED
|
@@ -184,7 +184,10 @@ async function requestJson(baseURL, path, init = {}, apiKey) {
|
|
|
184
184
|
signal: controller.signal
|
|
185
185
|
});
|
|
186
186
|
const body = await response.json().catch(() => void 0);
|
|
187
|
-
if (!response.ok)
|
|
187
|
+
if (!response.ok) {
|
|
188
|
+
const message = apiKey === void 0 ? errorMessage(body, "AllModels request failed") : "AllModels rejected the authenticated request";
|
|
189
|
+
throw new AllModelsError(response.status, `HTTP_${String(response.status)}`, message);
|
|
190
|
+
}
|
|
188
191
|
return body;
|
|
189
192
|
} catch (error) {
|
|
190
193
|
if (error instanceof AllModelsError) throw error;
|
|
@@ -333,7 +336,7 @@ function upstreamUrl(settings, binding) {
|
|
|
333
336
|
if (binding.contextSupported && settings.context !== void 0 && settings.context.trim() !== "") base.searchParams.set("context", settings.context.trim());
|
|
334
337
|
return base;
|
|
335
338
|
}
|
|
336
|
-
function normalizedEvent(data) {
|
|
339
|
+
function normalizedEvent(data, credential) {
|
|
337
340
|
try {
|
|
338
341
|
const text = typeof data === "string" ? data : data.toString("utf8");
|
|
339
342
|
const value = JSON.parse(text);
|
|
@@ -355,11 +358,15 @@ function normalizedEvent(data) {
|
|
|
355
358
|
text: event.text,
|
|
356
359
|
...languageCode === void 0 ? {} : { languageCode }
|
|
357
360
|
};
|
|
358
|
-
if (eventType === "stt.error")
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
361
|
+
if (eventType === "stt.error") {
|
|
362
|
+
const code = typeof event.code === "string" && /^[A-Za-z0-9_.-]{1,64}$/u.test(event.code) ? event.code : "UPSTREAM_ERROR";
|
|
363
|
+
const upstreamMessage = typeof event.message === "string" && event.message.length <= 500 ? event.message : "Speech recognition failed";
|
|
364
|
+
return {
|
|
365
|
+
type: "error",
|
|
366
|
+
code,
|
|
367
|
+
message: credential !== void 0 && credential.length > 0 && upstreamMessage.includes(credential) ? "Speech recognition failed" : upstreamMessage
|
|
368
|
+
};
|
|
369
|
+
}
|
|
363
370
|
if (eventType === "stt.session.ended" || eventType === "stt.ended") return { type: "ended" };
|
|
364
371
|
return;
|
|
365
372
|
} catch {
|
|
@@ -483,7 +490,7 @@ var SttProxy = class {
|
|
|
483
490
|
});
|
|
484
491
|
});
|
|
485
492
|
upstream.on("message", (data) => {
|
|
486
|
-
const event = normalizedEvent(data);
|
|
493
|
+
const event = normalizedEvent(data, credential.value);
|
|
487
494
|
if (event !== void 0) send(client, event);
|
|
488
495
|
});
|
|
489
496
|
upstream.on("error", () => {
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@allmodels/dsh-speech",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.2",
|
|
4
4
|
"description": "Streaming speech-to-text for DeepSeek Harness using AllModels.io.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"dsh-plugin",
|
|
@@ -61,6 +61,8 @@
|
|
|
61
61
|
"clean": "rm -rf lib coverage",
|
|
62
62
|
"test": "vitest run",
|
|
63
63
|
"test:coverage": "vitest run --coverage",
|
|
64
|
+
"test:e2e": "node scripts/run-e2e.mjs mock",
|
|
65
|
+
"test:e2e:live": "node scripts/run-e2e.mjs live",
|
|
64
66
|
"typecheck": "tsc --noEmit",
|
|
65
67
|
"check:sessions": "node scripts/check-no-session-access.mjs",
|
|
66
68
|
"check": "pnpm check:sessions && pnpm typecheck && pnpm test && pnpm build",
|
|
@@ -95,6 +97,7 @@
|
|
|
95
97
|
"@deepseek-ai/dsh-settings": "0.1.1-rc.2",
|
|
96
98
|
"@deepseek-ai/schemastery": "3.18.1",
|
|
97
99
|
"@testing-library/react": "^16.3.0",
|
|
100
|
+
"@playwright/test": "^1.62.1",
|
|
98
101
|
"@types/node": "^22.18.0",
|
|
99
102
|
"@types/react": "^18.3.24",
|
|
100
103
|
"@types/react-dom": "^18.3.7",
|