@hoanghong/speaking-core 0.1.0 → 0.3.0

package/CHANGELOG.md

@@ -0,0 +1,22 @@
+# Changelog
+
+## 0.3.0
+
+- Per-application keys with server-managed quota; appKey setup without a service URL.
+
+- Hosted public service configuration with automatic short-lived sessions and quota errors.
+- Select realtime/upload using serviceType; no URL or token setup for hosted use.
+- Completed callback includes text and audio: recorded PCM as WAV, or original uploaded file.
+- Bounded five-minute recording buffer and preserved final residual audio.
+
+
+## 0.2.0
+
+- Add createSpeakingGateway for shared HTTP/WebSocket/ticket configuration.
+- Validate gateway URLs and malformed ticket responses.
+- Add complete microphone/upload examples and gateway/asset setup instructions.
+- Preserve existing low-level 0.1 APIs.
+
+## 0.1.0
+
+- Initial batch, realtime and browser microphone SDK.

package/README.md

@@ -1,19 +1,56 @@
 # @hoanghong/speaking-core
 
-TypeScript SDK cho batch ASR và realtime WebSocket v1. Không nhúng model hay service credential.
+SDK online cho người tự làm UI. Cài `npm install @hoanghong/speaking-core@^0.3.0`. Angular dùng @hoanghong/speaking-angular với component có sẵn; Flutter dùng fsel_speaking. MIT.
 
-Exports: `BatchSpeakingClient`, `RealtimeSpeakingClient`, `SpeakingMicrophone`, contracts và errors.
-Microphone dùng AudioWorklet; copy asset `assets/pcm-worklet.js` vào app và truyền `workletUrl`.
+## Kết nối tự động, không cấu hình URL/token
 
-```bash
-npm ci
-npm test
-npm pack
+```ts
+import { createSpeakingService, RealtimeSpeakingClient, SpeakingMicrophone } from '@hoanghong/speaking-core';
+const config = createSpeakingService();
+const client = new RealtimeSpeakingClient(config.realtime);
+const mic = new SpeakingMicrophone(client, config.microphone);
+const unsubscribe = client.onEvent(event => {
+  if (event.type === 'partial') console.log(event.text);
+  if (event.type === 'clientError') console.error(event.error);
+});
+// Gọi từ handler nút bắt đầu; không tự ghi âm khi load trang:
+export async function start() { await mic.start({ language: 'vi' }); }
+export async function stop() {
+  const result = await mic.stop();
+  return { text: result.result?.text ?? '', audio: result.audio };
+}
+export async function cancel() { await mic.cancel(); }
+export async function dispose() { unsubscribe(); await mic.dispose(); }
 ```
 
-Realtime `ticketProvider` gọi backend ứng dụng để lấy ticket một lần, không gọi private ASR ticket API
-bằng service token trong browser. Batch `headers` cung cấp token người dùng cho gateway.
-`stop()` chờ ACK toàn bộ audio; `SpeakingMicrophone.stop()` flush phần PCM cuối trước finalize.
-Mất mạng/quá backlog trả lỗi rõ ràng, không tự reconnect/resume hay fallback.
+Audio là File WAV PCM16 mono 16 kHz, gồm phần audio cuối khi dừng. Dùng URL.createObjectURL để nghe/tải rồi revokeObjectURL; hoặc FormData để lưu. Cần browser AudioWorklet/getUserMedia và HTTPS/localhost. Public worklet được tải tự động; không copy asset. Gọi dispose ở lifecycle unmount.
 
-Hướng dẫn cấu hình, ví dụ và contract: `docs/realtime-and-sdk.md` ở repository root.
+## Upload
+
+```ts
+import { BatchSpeakingClient, createSpeakingService } from '@hoanghong/speaking-core';
+const batch = new BatchSpeakingClient(createSpeakingService().batch);
+export async function upload(file: File) {
+  if (file.size > 25 * 1024 * 1024) throw new Error('File tối đa 25 MiB');
+  let job = await batch.submitAudio(file, 'vi', crypto.randomUUID(), 'short_answer', file.name);
+  const deadline = Date.now() + 600000;
+  while (!['succeeded', 'no_speech', 'failed', 'cancelled'].includes(job.status)) {
+    if (Date.now() > deadline) throw new Error('Hết thời gian chờ');
+    await new Promise(resolve => setTimeout(resolve, 1500));
+    job = await batch.getResult(job.job_id);
+  }
+  if (job.error) throw new Error(job.error.message);
+  if (job.status === 'cancelled') throw new Error('Đã hủy');
+  return { text: job.result?.text ?? '', audio: file, language: 'vi', serviceType: 'upload' };
+}
+```
+
+Public client quản lý capability cho một job tại một thời điểm; tạo client/config riêng nếu xử lý nhiều job song song. Giữ client khi cần getResult/cancelJob cho job đó. Không chia sẻ capability hay serialize audio vào JSON.
+
+Ngôn ngữ en/ja/ko/zh/vi. Public service mặc định 5 phiên/IP/ngày, 100 phiên/ngày/instance (UTC); có thể được FSEL điều chỉnh. Mỗi phiên mới tính lượt kể cả khi lỗi; 429/QUOTA_EXCEEDED là hết lượt. SDK không tự retry/replay. Realtime tối đa 300 giây với buffer PCM 9,6 MB; dọn buffer khi lỗi/hủy. Upload không tự lưu vĩnh viễn file trên S3. Mạng/CSP cần cho phép truy cập dịch vụ FSEL. 404/503 có thể do server chưa triển khai public SDK/model. Cần server hỗ trợ bản 0.3 trước khi sử dụng.
+
+createSpeakingGateway và các constructor cấu hình thủ công cũ vẫn được giữ cho nhu cầu nâng cao; không cần dùng trong luồng public trên.
+
+## Key ứng dụng
+
+Dùng `createSpeakingService({ appKey: "KEY_DUOC_FSEL_CAP" })` thay cho lời gọi không tham số. SDK gửi key khi cấp phiên; quota quản lý phía server, không có URL cần cấu hình. Key này là định danh ứng dụng có thể quan sát ở frontend, không có quyền quản trị. INVALID_APP_KEY/401 nghĩa là sai hoặc đã bị khóa; không tự fallback ẩn danh.

package/dist/audio.d.ts

@@ -0,0 +1,9 @@
+/** Five minutes of mono PCM16 at 16 kHz; bounded independently of server limits. */
+export declare const MAX_RECORDING_BYTES: number;
+export declare class SpeakingAudioBuffer {
+    private chunks;
+    private size;
+    append(pcm: Uint8Array): void;
+    file(): File;
+    clear(): void;
+}

package/dist/audio.js

@@ -0,0 +1,36 @@
+import { SpeakingError } from './contracts.js';
+/** Five minutes of mono PCM16 at 16 kHz; bounded independently of server limits. */
+export const MAX_RECORDING_BYTES = 16000 * 2 * 300;
+export class SpeakingAudioBuffer {
+    chunks = [];
+    size = 0;
+    append(pcm) {
+        if (pcm.byteLength % 2)
+            throw new SpeakingError('INCOMPLETE_AUDIO', 'PCM16 requires complete samples');
+        if (this.size + pcm.byteLength > MAX_RECORDING_BYTES)
+            throw new SpeakingError('AUDIO_TOO_LONG', 'Recording exceeds five minutes');
+        this.chunks.push(pcm.slice());
+        this.size += pcm.byteLength;
+    }
+    file() {
+        const header = new ArrayBuffer(44);
+        const view = new DataView(header);
+        const text = (at, value) => { for (let i = 0; i < value.length; ++i)
+            view.setUint8(at + i, value.charCodeAt(i)); };
+        text(0, 'RIFF');
+        view.setUint32(4, 36 + this.size, true);
+        text(8, 'WAVE');
+        text(12, 'fmt ');
+        view.setUint32(16, 16, true);
+        view.setUint16(20, 1, true);
+        view.setUint16(22, 1, true);
+        view.setUint32(24, 16000, true);
+        view.setUint32(28, 32000, true);
+        view.setUint16(32, 2, true);
+        view.setUint16(34, 16, true);
+        text(36, 'data');
+        view.setUint32(40, this.size, true);
+        return new File([header, ...this.chunks], 'speaking.wav', { type: 'audio/wav' });
+    }
+    clear() { this.chunks = []; this.size = 0; }
+}

package/dist/batch.d.ts

@@ -4,6 +4,7 @@ export interface BatchOptions {
     /** User/gateway auth only. Never distribute an ASR service credential to an app. */
     headers: () => Promise<HeadersInit>;
     fetch?: typeof globalThis.fetch;
+    beforeSubmit?: (idempotencyKey: string) => Promise<void>;
 }
 export declare class BatchSpeakingClient {
     private readonly options;

package/dist/batch.js

@@ -17,6 +17,7 @@ export class BatchSpeakingClient {
     async submitAudio(audio, language, idempotencyKey, audioProfile = 'short_answer', filename = 'recording.wav', signal) {
         if (!idempotencyKey.trim())
             throw new SpeakingError('INVALID_REQUEST', 'An idempotency key is required');
+        await this.options.beforeSubmit?.(idempotencyKey);
         const form = new FormData();
         form.append('file', audio, filename);
         form.append('language', language);

package/dist/contracts.d.ts

@@ -1,4 +1,14 @@
 export type Language = 'en' | 'ja' | 'ko' | 'zh' | 'vi';
+export type SpeakingServiceType = 'realtime' | 'upload';
+export type SpeakingResult = (RealtimeFinal | Job) & {
+    text: string;
+    audio: File;
+    language: Language;
+    serviceType: SpeakingServiceType;
+};
+export type RecordedSpeakingFinal = RealtimeFinal & {
+    audio: File;
+};
 export type AudioProfile = 'short_answer' | 'passage';
 export type SpeakingState = 'idle' | 'connecting' | 'recording' | 'stopping' | 'stopped' | 'cancelled' | 'failed';
 export interface Transcript {

package/dist/gateway.d.ts

@@ -0,0 +1,16 @@
+import { BatchOptions } from './batch.js';
+import { MicrophoneOptions } from './microphone.js';
+import { RealtimeOptions } from './realtime.js';
+export interface SpeakingGatewayOptions {
+    /** Application gateway, e.g. /api/speaking. It must relay /v1 HTTP + WebSocket. */
+    baseUrl?: string;
+    /** Optional short-lived application auth. Never embed the ASR service token. */
+    headers?: () => Promise<HeadersInit>;
+    workletUrl?: string;
+    fetch?: typeof globalThis.fetch;
+}
+export declare function createSpeakingGateway(options?: SpeakingGatewayOptions): {
+    batch: BatchOptions;
+    realtime: RealtimeOptions;
+    microphone: MicrophoneOptions;
+};

package/dist/gateway.js

@@ -0,0 +1,35 @@
+import { SpeakingError } from './contracts.js';
+export function createSpeakingGateway(options = {}) {
+    const origin = typeof location === 'undefined' ? undefined : location.origin;
+    const url = new URL(options.baseUrl ?? '/api/speaking', origin);
+    if (!['http:', 'https:'].includes(url.protocol) || url.username || url.password || url.search || url.hash) {
+        throw new SpeakingError('INVALID_CONFIG', 'Use an HTTP(S) gateway URL without credentials, query or hash');
+    }
+    const baseUrl = url.href.replace(/\/$/, '');
+    const headers = options.headers ?? (async () => ({}));
+    const request = options.fetch ?? globalThis.fetch.bind(globalThis);
+    const socket = new URL(baseUrl + '/v1/realtime');
+    socket.protocol = url.protocol === 'https:' ? 'wss:' : 'ws:';
+    return {
+        batch: { baseUrl, headers, fetch: request },
+        realtime: {
+            websocketUrl: socket.href,
+            ticketProvider: async (start) => {
+                const auth = new Headers(await headers());
+                auth.set('Content-Type', 'application/json');
+                const response = await request(baseUrl + '/v1/realtime-tickets', {
+                    method: 'POST', headers: auth, credentials: 'same-origin',
+                    body: JSON.stringify({ language: start.language, audio_profile: start.audioProfile ?? 'short_answer' }),
+                });
+                const decoded = await response.json().catch(() => ({}));
+                const body = decoded && typeof decoded === 'object' ? decoded : {};
+                if (!response.ok)
+                    throw new SpeakingError(body.code ?? 'HTTP_ERROR', body.message ?? 'Cannot create speaking ticket', response.status);
+                if (typeof body.ticket !== 'string' || !body.ticket)
+                    throw new SpeakingError('INVALID_RESPONSE', 'Gateway returned no ticket');
+                return body.ticket;
+            },
+        },
+        microphone: { workletUrl: options.workletUrl ?? 'assets/speaking/pcm-worklet.js' },
+    };
+}

package/dist/index.d.ts

@@ -2,3 +2,5 @@ export * from './contracts.js';
 export * from './batch.js';
 export * from './realtime.js';
 export * from './microphone.js';
+export * from './gateway.js';
+export * from './public-service.js';

package/dist/index.js

@@ -2,3 +2,5 @@ export * from './contracts.js';
 export * from './batch.js';
 export * from './realtime.js';
 export * from './microphone.js';
+export * from './gateway.js';
+export * from './public-service.js';

package/dist/microphone.d.ts

@@ -1,5 +1,5 @@
 import { RealtimeSpeakingClient } from './realtime.js';
-import { RealtimeFinal, StartOptions } from './contracts.js';
+import { RecordedSpeakingFinal, StartOptions } from './contracts.js';
 export interface MicrophoneOptions {
     /** Host the packaged pcm-worklet.js with your app's normal CSP; no inline/blob worker. */
     workletUrl: string;
@@ -18,9 +18,10 @@ export declare class SpeakingMicrophone {
     private starting;
     private generation;
     private stopPromise?;
+    private readonly recording;
     constructor(client: RealtimeSpeakingClient, options: MicrophoneOptions);
     start(options: StartOptions): Promise<void>;
-    stop(): Promise<RealtimeFinal>;
+    stop(): Promise<RecordedSpeakingFinal>;
     private release;
     cancel(): Promise<void>;
     dispose(): Promise<void>;

package/dist/microphone.js

@@ -1,4 +1,5 @@
 import { SpeakingError } from './contracts.js';
+import { SpeakingAudioBuffer } from './audio.js';
 export class SpeakingMicrophone {
     client;
     options;
@@ -13,6 +14,7 @@ export class SpeakingMicrophone {
     starting = false;
     generation = 0;
     stopPromise;
+    recording = new SpeakingAudioBuffer();
     constructor(client, options) {
         this.client = client;
         this.options = options;
@@ -25,6 +27,7 @@ export class SpeakingMicrophone {
         this.stopPromise = undefined;
         this.audioError = undefined;
         this.sends = Promise.resolve();
+        this.recording.clear();
         try {
             const stream = await navigator.mediaDevices.getUserMedia({
                 audio: { channelCount: 1, echoCancellation: true, noiseSuppression: true },
@@ -55,6 +58,15 @@ export class SpeakingMicrophone {
                 if (generation !== this.generation)
                     return;
                 if (data.type === 'pcm') {
+                    try {
+                        this.recording.append(new Uint8Array(data.buffer));
+                    }
+                    catch (error) {
+                        this.audioError = error;
+                        this.client.audioFailed(this.audioError);
+                        void this.release();
+                        return;
+                    }
                     this.client.audioLevel(data.level);
                     // Enqueue immediately to let the transport enforce a bounded audio backlog.
                     const pending = this.client.pushAudio(new Uint8Array(data.buffer));
@@ -106,7 +118,10 @@ export class SpeakingMicrophone {
                     throw new SpeakingError('CANCELLED', 'Microphone session changed');
                 if (this.audioError)
                     throw this.audioError;
-                return await this.client.stop();
+                const result = await this.client.stop();
+                if (generation !== this.generation)
+                    throw new SpeakingError('CANCELLED', 'Microphone session changed');
+                return { ...result, audio: this.recording.file() };
             }
             catch (error) {
                 if (generation === this.generation)
@@ -121,6 +136,7 @@ export class SpeakingMicrophone {
         return this.stopPromise;
     }
     async release() {
+        this.recording.clear();
         this.unsubscribe?.();
         this.unsubscribe = undefined;
         this.source?.disconnect();

package/dist/public-service.d.ts

@@ -0,0 +1,8 @@
+/** Hosted public service. The SDK handles short-lived capabilities internally. */
+export declare function createSpeakingService(options?: {
+    appKey?: string;
+}): {
+    batch: import("./batch.js").BatchOptions;
+    realtime: import("./realtime.js").RealtimeOptions;
+    microphone: import("./microphone.js").MicrophoneOptions;
+};

package/dist/public-service.js

@@ -0,0 +1,41 @@
+import { createSpeakingGateway } from './gateway.js';
+import { SpeakingError } from './contracts.js';
+const SERVICE = 'https://ai-service.fsel.edu.vn/sdk';
+/** Hosted public service. The SDK handles short-lived capabilities internally. */
+export function createSpeakingService(options = {}) {
+    if (options.appKey !== undefined && !options.appKey.trim())
+        throw new SpeakingError('INVALID_CONFIG', 'Application key cannot be empty');
+    const request = globalThis.fetch.bind(globalThis);
+    const session = async () => {
+        const response = await request(SERVICE + '/session', {
+            method: 'POST', credentials: 'omit',
+            headers: options.appKey ? { 'X-Speaking-Key': options.appKey } : undefined,
+        });
+        const body = await response.json().catch(() => ({}));
+        if (!response.ok)
+            throw new SpeakingError(body?.code ?? 'HTTP_ERROR', body?.message ?? 'Speaking service unavailable', response.status);
+        if (typeof body?.access_token !== 'string' || !body.access_token)
+            throw new SpeakingError('INVALID_RESPONSE', 'Speaking session unavailable');
+        return body.access_token;
+    };
+    let token;
+    let previousKey;
+    const config = createSpeakingGateway({
+        baseUrl: SERVICE,
+        headers: async () => {
+            if (!token)
+                throw new SpeakingError('INVALID_STATE', 'Submit audio before reading a public job');
+            return { Authorization: 'Bearer ' + token };
+        },
+        workletUrl: SERVICE + '/pcm-worklet.js',
+    });
+    config.batch.beforeSubmit = async (key) => {
+        if (!token || key !== previousKey) {
+            token = await session();
+            previousKey = key;
+        }
+    };
+    const realtime = createSpeakingGateway({ baseUrl: SERVICE, headers: async () => ({ Authorization: 'Bearer ' + await session() }) });
+    config.realtime = realtime.realtime;
+    return config;
+}

package/dist/realtime.d.ts

@@ -1,4 +1,4 @@
-import { RealtimeFinal, SpeakingEvent, SpeakingState, Started, StartOptions } from './contracts.js';
+import { RealtimeFinal, SpeakingError, SpeakingEvent, SpeakingState, Started, StartOptions } from './contracts.js';
 export interface RealtimeOptions {
     websocketUrl: string;
     /** Ask an authenticated application backend for a fresh single-use ticket per start. */
@@ -30,6 +30,7 @@ export declare class RealtimeSpeakingClient {
     private emit;
     private setState;
     audioLevel(level: number): void;
+    audioFailed(error: SpeakingError): void;
     private wait;
     private rejectWaiters;
     private fail;

package/dist/realtime.js

@@ -35,6 +35,7 @@ export class RealtimeSpeakingClient {
         this.emit({ type: 'state', state });
     }
     audioLevel(level) { this.emit({ type: 'audioLevel', level }); }
+    audioFailed(error) { this.fail(error); }
     wait(type, timeout) {
         return new Promise((resolve, reject) => {
             const waiter = { type, resolve, reject, timer: setTimeout(() => {

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@hoanghong/speaking-core",
-  "version": "0.1.0",
-  "description": "FSEL batch and native realtime speaking SDK for browsers",
+  "version": "0.3.0",
+  "description": "Online speaking SDK: gateway configuration, realtime microphone and audio upload",
   "type": "module",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
@@ -15,7 +15,8 @@
   "files": [
     "dist",
     "assets",
-    "README.md"
+    "README.md",
+    "CHANGELOG.md"
   ],
   "scripts": {
     "build": "tsc -p tsconfig.json",