@cartanova/qgrid-ai-sdk 2.15.4 → 2.16.0

package/README.ko.md

@@ -394,7 +394,7 @@ Grok language model은 user 이미지 part(https URL 또는 inline bytes)를 버
 | Image | `n`(호출당 최대 3), `aspectRatio`, `xai.resolution`(`1k`/`1.5k`/`2k`), `xai.quality`, `xai.aspect_ratio`(`aspectRatio`가 없을 때) | 참조 `images`는 `referenceImages`(edit endpoint), `xai.output_format`은 `outputFormat`이 되며 `png`, `jpeg`, `webp`만 허용 | `size`, `seed`, `mask`, `xai.sync_mode`, `xai.user` |
 | Video | `duration`(정수 1-15), `aspectRatio`, `xai.resolution`(`480p`/`720p`/`1080p`), `xai.mode: "reference-to-video"`(지원하는 유일한 mode, `xai.referenceImageUrls` 필요), `xai.pollIntervalMs`, `xai.pollTimeoutMs` | `resolution` `WxH`는 `@ai-sdk/xai`와 같이 매핑(`1280x720` → `720p`), prompt `image`는 시작 프레임, `xai.referenceImageUrls`(7개가 아닌 최대 5개)는 `referenceImages`, `xai.lastFrame`(qgrid 확장, `grok-imagine-video-1.5` 전용) | `fps`, `seed`, `xai.keyframes`, `xai.storageOptions`, `xai.referenceVoiceIds`, `xai.user`. `xai.mode` `edit-video`/`extend-video`와 `xai.videoUrl`은 예외 |
 | Transcription | `xai.language` | 오디오 bytes는 base64로, `mediaType`은 파라미터 제거(`audio/webm;codecs=opus` → `audio/webm`) | 나머지 모든 `xai` 키(`audioFormat`, `sampleRate`, `format`, `diarize`, `keyterm`, ...) |
-| Speech | `voice`, `language` | `outputFormat`은 `format`(`mp3`, `wav`, `pcm`, `mulaw`, `alaw`, 그 외 값은 `@ai-sdk/xai`처럼 warning과 함께 `mp3`) | `instructions`, `speed`, 모든 `xai` 키(`sampleRate`, `bitRate`, `withTimestamps`, ...) |
+| Speech | `voice`, `language`, `xai.withTimestamps` | `outputFormat`은 `format`(`mp3`, `wav`, `pcm`, `mulaw`, `alaw`, 그 외 값은 `@ai-sdk/xai`처럼 warning과 함께 `mp3`) | `instructions`, `speed`, 그 밖의 모든 `xai` 키(`sampleRate`, `bitRate`, `replace`, ...) |
 
 **요청 크기.** qgrid 서버는 최대 10 MiB JSON body를 받습니다. inline 미디어는 base64로 전송되어 3분의 1만큼
 커지므로, 서버는 inline 오디오, 각 inline 이미지, 한 요청의 inline 이미지 합계(비디오: `image` + `lastFrame` +
@@ -487,6 +487,7 @@ speech.audio.uint8Array;
 
 - `transcribe`는 bytes로 오디오 형식을 감지하고 실패하면 `audio/wav`를 씁니다. 서버는 `audio/*`가 아닌 값을 거부합니다. transcript에는 segment가 없습니다(`segments: []`).
 - 음성 합성 텍스트는 최대 15,000자입니다. `voice`, `language`, `outputFormat`을 지정하지 않으면 서버 기본값 `ara`, `en`, `mp3`가 쓰이며, live 검증된 조합은 이것뿐입니다. 다른 voice, language(`@ai-sdk/xai`의 `auto` 포함), codec은 그대로 전달되지만 미검증입니다. 서버가 돌려준 오디오 MIME 타입은 `providerMetadata.qgrid.mediaType`에 있고, AI SDK의 `speech.audio.mediaType`은 bytes에서 추정한 값입니다.
+- `providerOptions: { xai: { withTimestamps: true } }`(2.16.0+, Grok 전용)를 주면 글자별 타이밍이 `providerMetadata.qgrid.audioTimestamps`에 담깁니다. `graphChars`는 입력 글자(공백·문장부호·음성 태그 포함), `graphTimes`는 같은 인덱스의 `[start, end]` 초이고, 전체 길이는 `providerMetadata.qgrid.duration`입니다. `@ai-sdk/xai`가 `providerMetadata.xai`에 담는 모양과 같습니다. 텍스트 정규화로 풀린 토큰은 첫 글자가 전체 구간을 가집니다. `qgrid.speech("openai/...")`는 타이밍 데이터가 없어 서버가 `withTimestamps`를 HTTP 400으로 거부합니다.
 
 ### OpenAI 음성 인식과 합성 (ChatGPT 구독)
 

package/README.md

@@ -404,7 +404,7 @@ Grok language models keep user image parts (https URLs or inline bytes) instead
 | Image | `n` (up to 3 per call), `aspectRatio`, `xai.resolution` (`1k`/`1.5k`/`2k`), `xai.quality`, `xai.aspect_ratio` (used when `aspectRatio` is unset) | reference `images` become `referenceImages` (edit endpoint); `xai.output_format` becomes `outputFormat` and must be `png`, `jpeg`, or `webp` | `size`, `seed`, `mask`, `xai.sync_mode`, `xai.user` |
 | Video | `duration` (integer 1-15), `aspectRatio`, `xai.resolution` (`480p`/`720p`/`1080p`), `xai.mode: "reference-to-video"` (the only supported mode; needs `xai.referenceImageUrls`), `xai.pollIntervalMs`, `xai.pollTimeoutMs` | `resolution` `WxH` is mapped as in `@ai-sdk/xai` (`1280x720` to `720p`); prompt `image` becomes the start frame; `xai.referenceImageUrls` (up to 5, not 7) become `referenceImages`; `xai.lastFrame` (qgrid extension, `grok-imagine-video-1.5` only) | `fps`, `seed`, `xai.keyframes`, `xai.storageOptions`, `xai.referenceVoiceIds`, `xai.user`. `xai.mode` `edit-video`/`extend-video` and `xai.videoUrl` throw |
 | Transcription | `xai.language` | audio bytes become base64 and `mediaType` loses its parameters (`audio/webm;codecs=opus` to `audio/webm`) | every other `xai` key (`audioFormat`, `sampleRate`, `format`, `diarize`, `keyterm`, ...) |
-| Speech | `voice`, `language` | `outputFormat` becomes `format` (`mp3`, `wav`, `pcm`, `mulaw`, `alaw`; anything else falls back to `mp3` with a warning, as in `@ai-sdk/xai`) | `instructions`, `speed`, every `xai` key (`sampleRate`, `bitRate`, `withTimestamps`, ...) |
+| Speech | `voice`, `language`, `xai.withTimestamps` | `outputFormat` becomes `format` (`mp3`, `wav`, `pcm`, `mulaw`, `alaw`; anything else falls back to `mp3` with a warning, as in `@ai-sdk/xai`) | `instructions`, `speed`, every other `xai` key (`sampleRate`, `bitRate`, `replace`, ...) |
 
 **Request size.** The qgrid server accepts JSON bodies up to 10 MiB. Inline media travels as
 base64, which grows by a third, so the server caps inline audio, each inline image, and all
@@ -499,6 +499,7 @@ speech.audio.uint8Array;
 
 - `transcribe` detects the audio type from its bytes and falls back to `audio/wav`; the server rejects anything that is not `audio/*`. Transcripts carry no segments (`segments: []`).
 - Speech text is limited to 15,000 characters. Unset `voice`, `language` and `outputFormat` use the server defaults `ara`, `en` and `mp3`, the only combination verified live. Other voices, languages (including `@ai-sdk/xai`'s `auto`) and codecs are passed through but unverified. The server's audio MIME type is in `providerMetadata.qgrid.mediaType`; AI SDK's `speech.audio.mediaType` is sniffed from the bytes.
+- `providerOptions: { xai: { withTimestamps: true } }` (2.16.0+, Grok only) returns per-character timing in `providerMetadata.qgrid.audioTimestamps`: `graphChars` (input characters, spaces, punctuation and speech tags included) and the index-aligned `graphTimes` `[start, end]` seconds, plus `providerMetadata.qgrid.duration`. The shape matches what `@ai-sdk/xai` puts under `providerMetadata.xai`. Text normalization gives an expanded token's whole span to its first character. `qgrid.speech("openai/...")` has no timing data and the server rejects `withTimestamps` with HTTP 400.
 
 ### OpenAI transcription and speech (ChatGPT subscription)
 

package/dist/index.d.ts

@@ -282,7 +282,9 @@ type QgridOpenAISpeechProviderOptions = {
 /**
  * Grok text-to-speech through `POST /api/qgrid/speak`. `voice`, `language` and `outputFormat`
  * map to the server's `voice`, `language` and `format`; when omitted the server applies
- * `ara` / `en` / `mp3` (the only combination verified live). The model id only labels the log.
+ * `ara` / `en` / `mp3`. The model id only labels the log. `providerOptions.xai.withTimestamps`
+ * returns per-character timing in `providerMetadata.qgrid.audioTimestamps` (`graphChars`,
+ * `graphTimes`), the shape `@ai-sdk/xai` puts under `providerMetadata.xai`.
  */
 declare function createQgridSpeech(modelId: QgridSpeechModel, config?: QgridMediaProviderConfig): SpeechModelV3;
 //#endregion

package/dist/index.js

@@ -1478,7 +1478,7 @@ function isSpeechFormat(value) {
 }
 /**
 * `providerOptions.xai` for Grok speech, field-for-field from `@ai-sdk/xai`
-* `xaiSpeechModelOptionsSchema`. The qgrid server takes none of them; every set key warns.
+* `xaiSpeechModelOptionsSchema`. Only `withTimestamps` is honored; every other set key warns.
 */
 const xaiSpeechOptionsSchema = z.object({
 	sampleRate: z.union([
@@ -1520,11 +1520,18 @@ const OPENAI_SPEECH_FORMATS = ["wav", "pcm"];
 function isOpenAISpeechFormat(value) {
 	return OPENAI_SPEECH_FORMATS.some((format) => format === value);
 }
+/** `providerOptions.xai` keys honored by `qgrid.speech("grok/...")`. */
+const XAI_SPEECH_HONORED = /* @__PURE__ */ new Set(["withTimestamps"]);
 const speakOutputSchema = z.object({
 	audio: z.string(),
 	mimeType: z.string(),
 	tokenName: z.string(),
-	requestLogId: z.number().optional()
+	requestLogId: z.number().optional(),
+	audioTimestamps: z.object({
+		graphChars: z.array(z.string()),
+		graphTimes: z.array(z.tuple([z.number(), z.number()]))
+	}).optional(),
+	durationSec: z.number().optional()
 });
 function base64ToBytes(data) {
 	if (typeof Buffer !== "undefined") return new Uint8Array(Buffer.from(data, "base64"));
@@ -1536,7 +1543,9 @@ function base64ToBytes(data) {
 /**
 * Grok text-to-speech through `POST /api/qgrid/speak`. `voice`, `language` and `outputFormat`
 * map to the server's `voice`, `language` and `format`; when omitted the server applies
-* `ara` / `en` / `mp3` (the only combination verified live). The model id only labels the log.
+* `ara` / `en` / `mp3`. The model id only labels the log. `providerOptions.xai.withTimestamps`
+* returns per-character timing in `providerMetadata.qgrid.audioTimestamps` (`graphChars`,
+* `graphTimes`), the shape `@ai-sdk/xai` puts under `providerMetadata.xai`.
 */
 function createQgridSpeech(modelId, config) {
 	const provider = voiceProviderOf(modelId, LABEL$2);
@@ -1550,7 +1559,7 @@ function createQgridSpeech(modelId, config) {
 			if (!options.text.trim()) throw new Error(`${LABEL$2}: text must not be empty`);
 			if (options.text.length > MAX_TEXT_CHARS) throw new Error(`${LABEL$2}: text is limited to ${MAX_TEXT_CHARS} characters`);
 			const qgridOptions = parseGrokQgridOptions(options.providerOptions?.qgrid, LABEL$2);
-			const xai = parseXaiMediaOptions(options.providerOptions?.xai, xaiSpeechOptionsSchema, /* @__PURE__ */ new Set(), LABEL$2);
+			const xai = parseXaiMediaOptions(options.providerOptions?.xai, xaiSpeechOptionsSchema, XAI_SPEECH_HONORED, LABEL$2);
 			const warnings = [];
 			let format;
 			if (options.outputFormat !== void 0) {
@@ -1580,6 +1589,7 @@ function createQgridSpeech(modelId, config) {
 			if (options.voice !== void 0) requestParams.voice = options.voice;
 			if (options.language !== void 0) requestParams.language = options.language;
 			if (format !== void 0) requestParams.format = format;
+			if (xai.options?.withTimestamps === true) requestParams.withTimestamps = true;
 			if (projectName) requestParams.projectName = projectName;
 			const params = {
 				...requestParams,
@@ -1601,6 +1611,8 @@ function createQgridSpeech(modelId, config) {
 				tokenName: data.tokenName
 			};
 			if (data.requestLogId !== void 0) qgridMetadata.requestLogId = data.requestLogId;
+			if (data.audioTimestamps) qgridMetadata.audioTimestamps = data.audioTimestamps;
+			if (data.durationSec !== void 0) qgridMetadata.duration = data.durationSec;
 			return {
 				audio: base64ToBytes(data.audio),
 				warnings,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@cartanova/qgrid-ai-sdk",
-  "version": "2.15.4",
+  "version": "2.16.0",
   "description": "AI SDK LanguageModelV3 and ImageModelV3 provider for qgrid",
   "repository": {
     "type": "git",