@henjicc/ai-sdk 0.4.1 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. package/CHANGELOG.md +36 -23
  2. package/README.md +61 -47
  3. package/dist/capabilities/media-request.d.ts +17 -0
  4. package/dist/capabilities/media-request.js +121 -0
  5. package/dist/capabilities/speech-recognition/bailian/module.js +21 -26
  6. package/dist/capabilities/speech-recognition/bailian/presets.js +5 -3
  7. package/dist/capabilities/speech-recognition/bailian/upload.js +32 -18
  8. package/dist/capabilities/speech-recognition/groq/module.js +15 -18
  9. package/dist/capabilities/speech-recognition/groq/types.d.ts +1 -1
  10. package/dist/capabilities/speech-recognition/siliconflow/module.js +7 -17
  11. package/dist/providers/provider-fetch.js +48 -14
  12. package/dist/runtime/AiRuntimeError.d.ts +2 -1
  13. package/dist/runtime/AiRuntimeError.js +3 -1
  14. package/dist/runtime/MediaReader.d.ts +24 -2
  15. package/dist/runtime/MediaReader.js +39 -1
  16. package/dist/runtime/Transport.d.ts +19 -0
  17. package/dist/runtime/index.d.ts +2 -1
  18. package/dist/runtime/index.js +1 -0
  19. package/dist/runtime/retry.js +5 -1
  20. package/docs/consumers.md +35 -18
  21. package/docs/model-adaptation/Fun-ASR/Fun-ASR_/347/231/276/347/202/274.md +5 -1
  22. package/docs/model-adaptation/Fun-ASR-Flash-2026-06-15/Fun-ASR-Flash-2026-06-15_/347/231/276/347/202/274.md +5 -1
  23. package/docs/model-adaptation/Qwen3-ASR-Flash/Qwen3-ASR-Flash_/347/231/276/347/202/274.md +5 -1
  24. package/docs/model-adaptation/Qwen3-ASR-Flash-2026-02-10/Qwen3-ASR-Flash-2026-02-10_/347/231/276/347/202/274.md +5 -1
  25. package/docs/model-adaptation/Qwen3-ASR-Flash-Filetrans/Qwen3-ASR-Flash-Filetrans_/347/231/276/347/202/274.md +5 -1
  26. package/docs/model-adaptation/README.md +22 -22
  27. package/docs/model-adaptation/SenseVoiceSmall/SenseVoiceSmall_/347/241/205/345/237/272/346/265/201/345/212/250.md +5 -1
  28. package/docs/model-adaptation/TeleSpeechASR/TeleSpeechASR_/347/241/205/345/237/272/346/265/201/345/212/250.md +5 -1
  29. package/docs/model-adaptation/Whisper-Large-v3/Whisper-Large-v3_Groq.md +5 -1
  30. package/docs/model-adaptation/Whisper-Large-v3-Turbo/Whisper-Large-v3-Turbo_Groq.md +5 -1
  31. package/docs/model-adaptation//344/276/233/345/272/224/345/225/206/APIMart.md +3 -1
  32. package/docs/model-adaptation//344/276/233/345/272/224/345/225/206//347/231/276/347/202/274.md +24 -10
  33. package/examples/form-renderer/package.json +1 -1
  34. package/examples/llm-chat/package.json +1 -1
  35. package/examples/minimal-node/package.json +1 -1
  36. package/package.json +8 -8
package/CHANGELOG.md CHANGED
@@ -1,26 +1,39 @@
1
- # Changelog
2
-
3
- ## 0.4.1 - 2026-09-11
4
-
5
- - 修复 KIE GPT Image 2 / 2.5 误带参考生视频标签;保留图片生成、编辑、多图输入与原有请求契约。
6
- - 增加全目录输出模态与视频能力标签一致性回归检查,防止图片模型再次显示视频功能。
7
-
8
- ## 0.4.0 - 2026-09-11
9
-
10
- - 新增硅基流动 LLM 按需入口与四个常用模型预设,支持流式正文、思考和工具调用,复用聊天与模型步骤运行时。
11
- - 硅基流动模型发现支持服务端 chat / embedding / reranker 分类,保留新模型;无效响应不再伪装为空列表。
12
- - DeepSeek 官方默认仍为 V4.1 Flash;硅基流动托管模型使用独立协议与能力标注。
13
-
14
- ## 0.3.0 - 2026-09-11
15
-
16
- - 新增按需文本 Embedding/Rerank 能力:硅基流动、百炼、派欧云、智谱两类入口,以及火山方舟单条文本向量入口;统一向量索引、重排分数、用量与取消,通过宿主 transport/credentials 执行。火山 VikingDB 重排不在本轮范围。
17
-
18
- - 新增 GPT Image 2.5 APIMart、KIE、Fal、Grsai 模型及按需入口,默认生成目录从 105 扩展至 109;保留 GPT Image 2 及现有参数和入口。
19
- - DeepSeek 官方默认与唯一推荐模型更新为 `deepseek-flash`(V4.1 Flash),登记图片理解、1M 上下文和 384K 输出,默认使用 Responses;聚合渠道旧模型保持独立。
20
- - 根据供应商分别支持 Flare / Sunburst、官方 / Ext 渠道、1K–4K、质量、透明背景、多图和 Fal 遮罩编辑;不支持的组合在构建请求时明确拒绝。
21
- - 价格区分按张、积分与 token:Fal/APIMart 官方渠道显示输出参考估算,输入另计;Grsai 保留基础充值档估价,并说明大额充值折扣。
22
- - 补充官方字段与响应 fixture、文生图/编辑契约和宿主参数切换回归;未执行付费生成,不代表实测质量、速度或账单。
23
-
1
+ # Changelog
2
+
3
+ ## 0.5.0 - 待发布
4
+
5
+ - 文件 ASR 支持分块 JSON/Base64 multipart,请求体不重组整文件;支持结构化大小错误、PCM 时长估算及取消清理。
6
+ - **迁移要求**:百炼异步本地文件需宿主 transport.uploadFile 原生直传 OSS,不再走媒体读取通道。其他流式路径需 describe/readChunk + fetchStream;旧 read 路径保留但不得放宽内存保护。
7
+ - 修复上传凭证字符串大小限制漏检,采用官方临时上传 1 GB 上限;Qwen 按编码后 10 MB 预检,Groq/硅基流动附件采用保守十进制上限。
8
+ - 包含原待发布 0.4.2 网络重试修复;尚未发布时,外部消费者不得从源码旁路安装。
9
+
10
+ ## 0.4.2 - 待发布
11
+
12
+ - 明确识别 TLS 建立前的断连;安全请求在端点尝试耗尽后按 1、3、8 秒退避恢复,取消可立即结束等待。
13
+ - 每次失败重新判定是否可以重试;提交状态不明的写请求停止重放,查询中断继续使用原任务。
14
+ - 网络错误附带脱敏端点、连接阶段、耗时与提交状态,便于宿主定位失败。
15
+
16
+ ## 0.4.1 - 2026-09-11
17
+
18
+ - 修复 KIE GPT Image 2 / 2.5 误带参考生视频标签;保留图片生成、编辑、多图输入与原有请求契约。
19
+ - 增加全目录输出模态与视频能力标签一致性回归检查,防止图片模型再次显示视频功能。
20
+
21
+ ## 0.4.0 - 2026-09-11
22
+
23
+ - 新增硅基流动 LLM 按需入口与四个常用模型预设,支持流式正文、思考和工具调用,复用聊天与模型步骤运行时。
24
+ - 硅基流动模型发现支持服务端 chat / embedding / reranker 分类,保留新模型;无效响应不再伪装为空列表。
25
+ - DeepSeek 官方默认仍为 V4.1 Flash;硅基流动托管模型使用独立协议与能力标注。
26
+
27
+ ## 0.3.0 - 2026-09-11
28
+
29
+ - 新增按需文本 Embedding/Rerank 能力:硅基流动、百炼、派欧云、智谱两类入口,以及火山方舟单条文本向量入口;统一向量索引、重排分数、用量与取消,通过宿主 transport/credentials 执行。火山 VikingDB 重排不在本轮范围。
30
+
31
+ - 新增 GPT Image 2.5 的 APIMart、KIE、Fal、Grsai 模型及按需入口,默认生成目录从 105 扩展至 109;保留 GPT Image 2 及现有参数和入口。
32
+ - DeepSeek 官方默认与唯一推荐模型更新为 `deepseek-flash`(V4.1 Flash),登记图片理解、1M 上下文和 384K 输出,默认使用 Responses;聚合渠道旧模型保持独立。
33
+ - 根据供应商分别支持 Flare / Sunburst、官方 / Ext 渠道、1K–4K、质量、透明背景、多图和 Fal 遮罩编辑;不支持的组合在构建请求时明确拒绝。
34
+ - 价格区分按张、积分与 token:Fal/APIMart 官方渠道显示输出参考估算,输入另计;Grsai 保留基础充值档估价,并说明大额充值折扣。
35
+ - 补充官方字段与响应 fixture、文生图/编辑契约和宿主参数切换回归;未执行付费生成,不代表实测质量、速度或账单。
36
+
24
37
  ## Unreleased
25
38
 
26
39
  ## 0.2.8 - 2026-08-31
package/README.md CHANGED
@@ -1,57 +1,57 @@
1
1
  # @henjicc/ai-sdk
2
2
 
3
3
  痕迹AI 的多供应商模型 SDK:内含 8 个生成供应商、109 个图片/视频/音频模型,以及
4
- 10 个 LLM 供应商预设和 15 个按需 ASR 模型。另有 12 个 FAL 图片工具使用独立按需入口,不进入默认 109 模型目录。预制 LLM 会按供应商与具体模型自动选择 Responses API 或 Chat Completions,宿主不需要暴露逐模型协议设置。SDK 负责目录、请求构建、媒体预处理、
4
+ 10 个 LLM 供应商预设和 15 个按需 ASR 模型。另有 12 个 FAL 图片工具使用独立按需入口,不进入默认 109 模型目录。预制 LLM 会按供应商与具体模型自动选择 Responses API 或 Chat Completions,宿主不需要暴露逐模型协议设置。SDK 负责目录、请求构建、媒体预处理、
5
5
  供应商调用、轮询、SSE 与错误归一化;宿主只需注入网络、凭据、媒体读取和日志。
6
6
 
7
- ## 5 分钟快速开始
8
-
9
- ### 硅基流动聊天与实时模型列表
10
-
11
- `llm/siliconflow` 提供四个推荐预设(DeepSeek V4 Flash、GLM-5.3、Kimi K2.7 Code、Qwen3.8-27B),并通过官方 `GET /v1/models` 获取账号当前可用模型,不受预设名单限制。
12
-
13
- ```ts
14
- import { discoverSiliconflowModels, runSiliconflowChatStream } from '@henjicc/ai-sdk/llm/siliconflow'
15
-
16
- const models = await discoverSiliconflowModels(runtime, { modelType: 'chat' })
17
- // modelType 也支持 embedding / reranker;分别读取 llm / embedding / rerank 凭据。
18
- const result = await runSiliconflowChatStream({
19
- modelId: 'deepseek-ai/DeepSeek-V4-Flash',
20
- messages: [{ role: 'user', content: '你好' }],
21
- }, 'chat-1', event => console.log(event), runtime)
22
- ```
23
-
24
- 动态列表不提供完整能力或价格,缺失的上下文与输出限制保持 `null`。新模型可直接传入 `modelId`;宿主按官方资料提供能力配置。硅基流动的 V4 Flash 与 DeepSeek 官方默认 `deepseek-flash`(V4.1)不同,全局默认不变。详见[硅基流动适配资料](docs/model-adaptation/供应商/硅基流动.md)。
25
-
26
- ### 文本向量与重排序
27
-
28
- 通过 `capabilities/embedding/<provider>` 和 `capabilities/rerank/<provider>` 按需导入。硅基流动、百炼、派欧云和智谱支持两类能力;火山目前支持单条文本向量。模型清单与限制见 [检索适配资料](docs/model-adaptation/README.md#文本-embedding--rerank2026-09-11)。输入仅支持文本;不自动切块、重试或合并多次付费请求。
29
-
30
- ```ts
31
- import { createCapabilityClient } from '@henjicc/ai-sdk/capabilities'
32
- import { createSiliconflowEmbeddingModule } from '@henjicc/ai-sdk/capabilities/embedding/siliconflow'
33
- import { createSiliconflowRerankModule } from '@henjicc/ai-sdk/capabilities/rerank/siliconflow'
34
-
35
- // runtime.credentials.get(scope, credentialId) 须支持 embedding / rerank,
36
- // 默认 credentialId 是供应商 ID(例如 siliconflow)。其余宿主能力见下文。
37
- const retrieval = createCapabilityClient({ runtime })
38
- try {
39
- const embedding = retrieval.register(createSiliconflowEmbeddingModule())
40
- const rerank = retrieval.register(createSiliconflowRerankModule())
41
- const vectors = await embedding.execute({ texts: ['向量检索', '图片编辑'] })
42
- const ranked = await rerank.execute({ query: '知识库搜索', documents: ['向量检索', '图片编辑'], topN: 1 })
43
- // vectors.embeddings: { index, vector }[];ranked.results: { index, score, document }[]
44
- } finally {
45
- await retrieval.dispose()
46
- }
47
- ```
48
-
49
- 百炼工厂必须传 `baseUrl` 为实际地域/工作空间的 API 根地址,不包含端点路径。其他供应商可以覆盖根地址与 `credentialId`;SDK 不自动切换地域或账号。不同模型的向量不能混用;切换模型通常需要重建向量索引。Rerank 分数仅在本次请求内比较。
50
-
51
- SDK `0.4.1` 的正式分发渠道为公共 npm,无需配置 registry 或访问令牌:
7
+ ## 5 分钟快速开始
8
+
9
+ ### 硅基流动聊天与实时模型列表
10
+
11
+ `llm/siliconflow` 提供四个推荐预设(DeepSeek V4 Flash、GLM-5.3、Kimi K2.7 Code、Qwen3.8-27B),并通过官方 `GET /v1/models` 获取账号当前可用模型,不受预设名单限制。
12
+
13
+ ```ts
14
+ import { discoverSiliconflowModels, runSiliconflowChatStream } from '@henjicc/ai-sdk/llm/siliconflow'
15
+
16
+ const models = await discoverSiliconflowModels(runtime, { modelType: 'chat' })
17
+ // modelType 也支持 embedding / reranker;分别读取 llm / embedding / rerank 凭据。
18
+ const result = await runSiliconflowChatStream({
19
+ modelId: 'deepseek-ai/DeepSeek-V4-Flash',
20
+ messages: [{ role: 'user', content: '你好' }],
21
+ }, 'chat-1', event => console.log(event), runtime)
22
+ ```
23
+
24
+ 动态列表不提供完整能力或价格,缺失的上下文与输出限制保持 `null`。新模型可直接传入 `modelId`;宿主按官方资料提供能力配置。硅基流动的 V4 Flash 与 DeepSeek 官方默认 `deepseek-flash`(V4.1)不同,全局默认不变。详见[硅基流动适配资料](docs/model-adaptation/供应商/硅基流动.md)。
25
+
26
+ ### 文本向量与重排序
27
+
28
+ 通过 `capabilities/embedding/<provider>` 和 `capabilities/rerank/<provider>` 按需导入。硅基流动、百炼、派欧云和智谱支持两类能力;火山目前支持单条文本向量。模型清单与限制见 [检索适配资料](docs/model-adaptation/README.md#文本-embedding--rerank2026-09-11)。输入仅支持文本;不自动切块、重试或合并多次付费请求。
29
+
30
+ ```ts
31
+ import { createCapabilityClient } from '@henjicc/ai-sdk/capabilities'
32
+ import { createSiliconflowEmbeddingModule } from '@henjicc/ai-sdk/capabilities/embedding/siliconflow'
33
+ import { createSiliconflowRerankModule } from '@henjicc/ai-sdk/capabilities/rerank/siliconflow'
34
+
35
+ // runtime.credentials.get(scope, credentialId) 须支持 embedding / rerank,
36
+ // 默认 credentialId 是供应商 ID(例如 siliconflow)。其余宿主能力见下文。
37
+ const retrieval = createCapabilityClient({ runtime })
38
+ try {
39
+ const embedding = retrieval.register(createSiliconflowEmbeddingModule())
40
+ const rerank = retrieval.register(createSiliconflowRerankModule())
41
+ const vectors = await embedding.execute({ texts: ['向量检索', '图片编辑'] })
42
+ const ranked = await rerank.execute({ query: '知识库搜索', documents: ['向量检索', '图片编辑'], topN: 1 })
43
+ // vectors.embeddings: { index, vector }[];ranked.results: { index, score, document }[]
44
+ } finally {
45
+ await retrieval.dispose()
46
+ }
47
+ ```
48
+
49
+ 百炼工厂必须传 `baseUrl` 为实际地域/工作空间的 API 根地址,不包含端点路径。其他供应商可以覆盖根地址与 `credentialId`;SDK 不自动切换地域或账号。不同模型的向量不能混用;切换模型通常需要重建向量索引。Rerank 分数仅在本次请求内比较。
50
+
51
+ SDK `0.5.0` 的正式分发渠道为公共 npm,无需配置 registry 或访问令牌:
52
52
 
53
53
  ```bash
54
- npm install @henjicc/ai-sdk@0.4.1
54
+ npm install @henjicc/ai-sdk@0.5.0
55
55
  ```
56
56
 
57
57
  然后提供 4 个宿主能力(`Transport` / `CredentialStore` / `MediaReader` / `Logger`),创建客户端:
@@ -512,3 +512,17 @@ module 只发送 Token/ReasoningToken 增量并返回最终结果。`createGroqL
512
512
  - 重要决定记录:`docs/task/模型SDK抽离/重要记录.md`
513
513
  - 本包内的调研资料索引:[docs/README.md](docs/README.md)
514
514
  - 版本记录:[CHANGELOG.md](CHANGELOG.md)
515
+
516
+ ## 0.5.0 文件 ASR 宿主迁移
517
+
518
+ - QuickJS 必须把 `media.describe(ref)` 和 `media.readChunk(ref, offset, length)` 暴露给 RuntimeContext;SDK 单次读取最多 48 KiB。原先只有 `read()` 的宿主仍使用整文件兼容路径,不能据此调大旧 10 MiB 限制。
519
+ - `transport.fetchStream(url, {body, contentLength, ...init})` 按拉取顺序消费 `AsyncIterable<Uint8Array>`,必须有背压、支持 AbortSignal、禁止整段拼接及自动重试。`contentLength` 是精确请求体字节数;宿主负责把它交给 HTTP 栈。无需 Node 或 Web Streams 全局对象。
520
+ - 百炼异步 `media-ref` 必须实现 `transport.uploadFile(url, {ref, fields, fileField, maxBytes, signal})`。宿主在受控文件作用域内 stat、校验大小、原生 multipart 直传,文件字段最后;不得回调 media API,不得把文件内容放入 QuickJS。上传策略与 oss:// 路径仍由 SDK 生成。缺失该能力返回 `native_upload_unsupported`,这是本次次版本的迁移要求。
521
+ - 保留旧 `read()` 与显式 `bytes` 输入用于兼容;它们不具有恒定内存保证。要使用新流式路径,两个媒体方法及 fetchStream 必须一起接入。异步上传的公网 URL 输入完全不读取本地媒体。
522
+ - 文件超限使用 `AiRuntimeError('media_too_large', ..., details)`,details 含 `actualBytes`、`maxBytes`、`estimatedDurationSeconds`。宿主若在 describe/readChunk/uploadFile 提前拒绝,也必须传递该结构,不能只传错误字符串。SDK 会保留跨 RPC 的同形错误。
523
+ - 可选 `MediaDescription.audio` 提供采样率、声道、位深、PCM payload 字节数或实际时长。PCM 估算公式为 payloadBytes / (sampleRateHz × channels × bitsPerSample / 8);WAV 应提供去掉容器头的 pcmBytes,压缩音频必须提供解析所得 durationSeconds,无法确定时返回 null,不冒充真实时长。宿主可复用 runtime 导出的 assertMediaSize 生成错误。
524
+ - 新宿主的媒体分块总文件上限可按最大所用模型设为 2,000,000,000 字节(Fun-ASR Flash 官方文件上限),单块仍不得超过 64 KiB;这不是所有模型都允许 2 GB。Qwen Base64 原文件 ≤7,500,000 字节、Groq 附件 ≤25,000,000、硅基流动 ≤50,000,000;百炼异步原生临时上传 ≤min(1,000,000,000, policy MB ×1,000,000)。若宿主只将媒体通道用于 Qwen/Groq/硅基流动,可把分块总文件上限收紧为 50,000,000。不得修改整个旧 read 通道为同一大上限。
525
+ - 所有 MB/GB 用十进制保守值;上限来自各模型及上传服务文档,实际可用值还受宿主作用域、账号、格式与时长限制。Fun-ASR Flash/Qwen 短音频最多5分钟、长音频最多12小时、硅基流动最多1小时;流式传输不绕过这些约束。远端 URL 的真实大小/时长需由宿主预检或供应商校验。
526
+
527
+ 验证入口:`node packages/ai-sdk/scripts/verify-asr-streaming-quickjs.cjs <临时安装的 quickjs-emscripten 绝对路径>`。
528
+ 该探针在真实 QuickJS 64 MiB 堆执行 50 MB multipart 与 7.5/12 MB JSON,禁止整文件 read,并用128 MiB分配失败验证内存限制;不代表真实供应商调用或 Tauri 原生上传已经验收。
@@ -0,0 +1,17 @@
1
+ import type { MediaDescription, ResolvedRuntimeContext } from '../runtime/index.js';
2
+ import { type CapabilityMediaSource } from './media.js';
3
+ export interface PreparedMedia {
4
+ description: MediaDescription;
5
+ chunks: AsyncIterable<Uint8Array>;
6
+ /** Only legacy read()/explicit bytes inputs have a complete buffer. */
7
+ bytes?: Uint8Array;
8
+ }
9
+ export declare function prepareMedia(source: CapabilityMediaSource, runtime: ResolvedRuntimeContext, maxBytes: number, signal: AbortSignal): Promise<PreparedMedia>;
10
+ export interface MediaRequest {
11
+ body: BodyInit | AsyncIterable<Uint8Array>;
12
+ contentType?: string;
13
+ contentLength?: number;
14
+ }
15
+ export declare function sendMediaRequest(runtime: ResolvedRuntimeContext, url: string, init: Omit<RequestInit, 'body'>, request: MediaRequest): Promise<Response>;
16
+ export declare function jsonAudioRequest(media: PreparedMedia, build: (data: string) => unknown): MediaRequest;
17
+ export declare function multipartAudioRequest(media: PreparedMedia, fields: readonly (readonly [string, string])[]): MediaRequest;
@@ -0,0 +1,121 @@
1
+ import { strToU8 } from 'fflate';
2
+ import { assertMediaSize, rethrowMediaError } from '../runtime/MediaReader.js';
3
+ import { AiRuntimeError, cancelledError } from '../runtime/AiRuntimeError.js';
4
+ import { toBase64, toDataUri } from '../upload/base64.js';
5
+ import { readCapabilityMediaSource } from './media.js';
6
+ const CHUNK_BYTES = 48 * 1024; // divisible by 3 for base64, below the host's 64 KiB ceiling
7
+ export async function prepareMedia(source, runtime, maxBytes, signal) {
8
+ const check = () => { if (signal.aborted)
9
+ throw cancelledError('media-upload'); };
10
+ check();
11
+ if (source.kind === 'media-ref' && runtime.media.describe && runtime.media.readChunk) {
12
+ const description = await runtime.media.describe(source.ref).catch(rethrowMediaError);
13
+ check();
14
+ assertMediaSize(description, maxBytes);
15
+ if (source.mediaType && source.mediaType !== description.mimeType) {
16
+ throw new AiRuntimeError('capability_media_type_mismatch', 'Host media type differs from the requested type');
17
+ }
18
+ if (!runtime.transport.fetchStream)
19
+ throw new AiRuntimeError('streaming_upload_unsupported', '宿主尚未接入流式请求体传输');
20
+ return { description, chunks: (async function* () {
21
+ let offset = 0;
22
+ while (offset < description.size) {
23
+ check();
24
+ const length = Math.min(CHUNK_BYTES, description.size - offset);
25
+ const chunk = await runtime.media.readChunk(source.ref, offset, length).catch(rethrowMediaError);
26
+ check();
27
+ if (!(chunk instanceof Uint8Array) || chunk.length === 0 || chunk.length > length) {
28
+ throw new AiRuntimeError('invalid_media_chunk', 'Host media chunk does not match the declared size');
29
+ }
30
+ offset += chunk.length;
31
+ yield chunk;
32
+ }
33
+ })() };
34
+ }
35
+ // Old hosts remain compatible. Their read() allocation limit must NOT be raised.
36
+ const media = await readCapabilityMediaSource(source, runtime.media).catch(rethrowMediaError);
37
+ check();
38
+ const description = { size: media.bytes.byteLength, mimeType: media.mimeType, filename: media.filename };
39
+ assertMediaSize(description, maxBytes);
40
+ return { description, bytes: media.bytes, chunks: (async function* () {
41
+ for (let offset = 0; offset < media.bytes.length; offset += CHUNK_BYTES) {
42
+ check();
43
+ yield media.bytes.subarray(offset, offset + CHUNK_BYTES);
44
+ }
45
+ })() };
46
+ }
47
+ export async function sendMediaRequest(runtime, url, init, request) {
48
+ const headers = new Headers(init.headers);
49
+ if (request.contentType)
50
+ headers.set('Content-Type', request.contentType);
51
+ if (request.contentLength !== undefined) {
52
+ if (!runtime.transport.fetchStream)
53
+ throw new AiRuntimeError('streaming_upload_unsupported', '宿主尚未接入流式请求体传输');
54
+ const iterator = request.body[Symbol.asyncIterator]();
55
+ try {
56
+ return await runtime.transport.fetchStream(url, {
57
+ ...init, headers, body: { [Symbol.asyncIterator]: () => iterator }, contentLength: request.contentLength,
58
+ });
59
+ }
60
+ finally {
61
+ await iterator.return?.();
62
+ }
63
+ }
64
+ return runtime.transport.fetch(url, { ...init, headers, body: request.body });
65
+ }
66
+ export function jsonAudioRequest(media, build) {
67
+ if (media.bytes)
68
+ return { body: JSON.stringify(build(toDataUri(media.bytes, media.description.mimeType))), contentType: 'application/json' };
69
+ let marker = '__sdk_audio_payload__';
70
+ let serialized = JSON.stringify(build(marker));
71
+ while (serialized.split(JSON.stringify(marker)).length !== 2) {
72
+ marker += '_';
73
+ serialized = JSON.stringify(build(marker));
74
+ }
75
+ const [before, after] = serialized.split(JSON.stringify(marker));
76
+ const prefix = strToU8(before + JSON.stringify(`data:${media.description.mimeType};base64,`).slice(0, -1));
77
+ const suffix = strToU8('"' + after);
78
+ return {
79
+ contentType: 'application/json',
80
+ contentLength: prefix.length + 4 * Math.ceil(media.description.size / 3) + suffix.length,
81
+ body: (async function* () {
82
+ yield prefix;
83
+ let remainder = new Uint8Array(0);
84
+ for await (const chunk of media.chunks) {
85
+ const bytes = new Uint8Array(remainder.length + chunk.length);
86
+ bytes.set(remainder);
87
+ bytes.set(chunk, remainder.length);
88
+ const end = bytes.length - bytes.length % 3;
89
+ if (end)
90
+ yield strToU8(toBase64(bytes.subarray(0, end)));
91
+ remainder = bytes.slice(end);
92
+ }
93
+ if (remainder.length)
94
+ yield strToU8(toBase64(remainder));
95
+ yield suffix;
96
+ })(),
97
+ };
98
+ }
99
+ export function multipartAudioRequest(media, fields) {
100
+ if (!media.description.size)
101
+ throw new AiRuntimeError('invalid_media', 'Transcription audio is empty');
102
+ if (media.bytes) {
103
+ const form = new FormData();
104
+ for (const [key, value] of fields)
105
+ form.append(key, value);
106
+ form.append('file', new Blob([new Uint8Array(media.bytes)], { type: media.description.mimeType }), media.description.filename);
107
+ return { body: form };
108
+ }
109
+ const boundary = `henji-${Date.now().toString(36)}-${Math.random().toString(36).slice(2)}`;
110
+ const quote = (value) => value.replace(/\r/g, '%0D').replace(/\n/g, '%0A').replace(/"/g, '%22');
111
+ const mime = media.description.mimeType;
112
+ if (/[\r\n]/.test(mime))
113
+ throw new AiRuntimeError('invalid_media', 'Invalid media MIME type');
114
+ const prefix = strToU8(fields.map(([key, value]) => `--${boundary}\r\nContent-Disposition: form-data; name="${quote(key)}"\r\n\r\n${value}\r\n`).join('') + `--${boundary}\r\nContent-Disposition: form-data; name="file"; filename="${quote(media.description.filename)}"\r\nContent-Type: ${mime}\r\n\r\n`);
115
+ const suffix = strToU8(`\r\n--${boundary}--\r\n`);
116
+ return {
117
+ contentType: `multipart/form-data; boundary=${boundary}`,
118
+ contentLength: prefix.length + media.description.size + suffix.length,
119
+ body: (async function* () { yield prefix; yield* media.chunks; yield suffix; })(),
120
+ };
121
+ }
@@ -1,6 +1,5 @@
1
- import { toDataUri } from '../../../upload/base64.js';
2
1
  import { AiRuntimeError, cancelledError } from '../../../runtime/AiRuntimeError.js';
3
- import { readCapabilityMediaSource } from '../../media.js';
2
+ import { prepareMedia, jsonAudioRequest, sendMediaRequest } from '../../media-request.js';
4
3
  import { parseFileTranscript, parseFunShortSse, parseQwenShortResponse, parseTaskState, transcriptionUrlFromTask, } from './parse.js';
5
4
  import { resolveAsyncAudioUrl } from './upload.js';
6
5
  const DEFAULT_API_BASE = 'https://dashscope.aliyuncs.com/api/v1';
@@ -44,7 +43,7 @@ function checkAbort(context) {
44
43
  if (context.signal.aborted)
45
44
  throw cancelledError(context.requestId);
46
45
  }
47
- async function inlineAudio(input, preset, context) {
46
+ async function inlineAudio(input, preset, context, build) {
48
47
  if (input.audio.kind === 'remote-url') {
49
48
  let url;
50
49
  try {
@@ -56,13 +55,10 @@ async function inlineAudio(input, preset, context) {
56
55
  if (url.protocol !== 'https:' && url.protocol !== 'http:') {
57
56
  throw new AiRuntimeError('invalid_media_url', 'Bailian short ASR media URL must use HTTP(S)');
58
57
  }
59
- return url.toString();
58
+ return { body: JSON.stringify(build(url.toString())), contentType: 'application/json' };
60
59
  }
61
- const media = await readCapabilityMediaSource(input.audio, context.runtime.media);
62
- if (preset.maxInlineBytes !== undefined && media.bytes.byteLength > preset.maxInlineBytes) {
63
- throw new AiRuntimeError('media_too_large', `Bailian ${preset.modelId} inline audio exceeds ${preset.maxInlineBytes} bytes`);
64
- }
65
- return toDataUri(media.bytes, media.mimeType);
60
+ const media = await prepareMedia(input.audio, context.runtime, preset.maxInlineBytes ?? 10 * 1024 * 1024, context.signal);
61
+ return jsonAudioRequest(media, build);
66
62
  }
67
63
  function formatFrom(input, options) {
68
64
  if (options.format?.trim())
@@ -82,7 +78,6 @@ async function executeFunShort(preset, input, apiKey, apiBaseUrl, context) {
82
78
  const content = [];
83
79
  if (options.context?.trim())
84
80
  content.push({ type: 'text', text: options.context.trim() });
85
- content.push({ type: 'input_audio', input_audio: { data: await inlineAudio(input, preset, context) } });
86
81
  const parameters = { format: formatFrom(input, options) };
87
82
  if (options.sampleRateHz !== undefined)
88
83
  parameters.sample_rate = options.sampleRateHz;
@@ -92,16 +87,18 @@ async function executeFunShort(preset, input, apiKey, apiBaseUrl, context) {
92
87
  .map((hint) => hint.trim()).filter(Boolean))];
93
88
  if (languageHints.length)
94
89
  parameters.language_hints = languageHints;
95
- const response = await context.runtime.transport.fetch(`${apiBaseUrl}/services/aigc/multimodal-generation/generation`, {
90
+ const request = await inlineAudio(input, preset, context, (data) => ({
91
+ model: preset.modelId, input: { messages: [{ role: 'user', content: [...content, { type: 'input_audio', input_audio: { data } }] }] }, parameters,
92
+ }));
93
+ const response = await sendMediaRequest(context.runtime, `${apiBaseUrl}/services/aigc/multimodal-generation/generation`, {
96
94
  method: 'POST',
97
95
  headers: {
98
96
  Authorization: `Bearer ${apiKey}`,
99
97
  'Content-Type': 'application/json',
100
98
  'X-DashScope-SSE': 'enable',
101
99
  },
102
- body: JSON.stringify({ model: preset.modelId, input: { messages: [{ role: 'user', content }] }, parameters }),
103
100
  signal: context.signal,
104
- });
101
+ }, request);
105
102
  if (!response.ok)
106
103
  throw new AiRuntimeError('provider_http_error', `Bailian Fun-ASR failed with HTTP ${response.status}`);
107
104
  const output = parseFunShortSse(await response.text());
@@ -112,19 +109,17 @@ async function executeFunShort(preset, input, apiKey, apiBaseUrl, context) {
112
109
  }
113
110
  async function executeQwenShort(preset, input, apiKey, compatibleBaseUrl, context) {
114
111
  const options = providerOptions(input);
115
- const response = await context.runtime.transport.fetch(`${compatibleBaseUrl}/chat/completions`, {
116
- method: 'POST',
117
- headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json' },
118
- body: JSON.stringify({
119
- model: preset.modelId,
120
- messages: [{ role: 'user', content: [{ type: 'input_audio', input_audio: { data: await inlineAudio(input, preset, context) } }] }],
121
- asr_options: {
122
- ...(input.language ? { language: input.language } : {}),
123
- ...(options.enableItn !== undefined ? { enable_itn: options.enableItn } : {}),
124
- },
125
- }),
126
- signal: context.signal,
127
- });
112
+ const request = await inlineAudio(input, preset, context, (data) => ({
113
+ model: preset.modelId,
114
+ messages: [{ role: 'user', content: [{ type: 'input_audio', input_audio: { data } }] }],
115
+ asr_options: {
116
+ ...(input.language ? { language: input.language } : {}),
117
+ ...(options.enableItn !== undefined ? { enable_itn: options.enableItn } : {}),
118
+ },
119
+ }));
120
+ const response = await sendMediaRequest(context.runtime, `${compatibleBaseUrl}/chat/completions`, {
121
+ method: 'POST', headers: { Authorization: `Bearer ${apiKey}` }, signal: context.signal,
122
+ }, request);
128
123
  const output = parseQwenShortResponse(await responseJson(response, 'Qwen short ASR'));
129
124
  await context.emit({ type: 'final', text: output.text });
130
125
  await context.emit({ type: 'completed', output });
@@ -22,17 +22,19 @@ function definePreset(input) {
22
22
  export const bailianFunAsrFlash20260615 = definePreset({
23
23
  modelId: 'fun-asr-flash-2026-06-15',
24
24
  protocol: 'fun-short-sse',
25
- maxInlineBytes: 10 * 1024 * 1024,
25
+ maxInlineBytes: 2_000_000_000,
26
26
  });
27
27
  export const bailianQwen3AsrFlash = definePreset({
28
28
  modelId: 'qwen3-asr-flash',
29
29
  protocol: 'qwen-short',
30
- maxInlineBytes: 10 * 1024 * 1024,
30
+ // Official 10 MB encoded audio limit; raw bytes expand by 4/3.
31
+ maxInlineBytes: 7_500_000,
31
32
  });
32
33
  export const bailianQwen3AsrFlash20260210 = definePreset({
33
34
  modelId: 'qwen3-asr-flash-2026-02-10',
34
35
  protocol: 'qwen-short',
35
- maxInlineBytes: 10 * 1024 * 1024,
36
+ // Official 10 MB encoded audio limit; raw bytes expand by 4/3.
37
+ maxInlineBytes: 7_500_000,
36
38
  });
37
39
  export const bailianFunAsr = definePreset({
38
40
  modelId: 'fun-asr',
@@ -1,4 +1,5 @@
1
1
  import { readCapabilityMediaSource } from '../../media.js';
2
+ import { assertMediaSize, rethrowMediaError } from '../../../runtime/MediaReader.js';
2
3
  import { AiRuntimeError } from '../../../runtime/AiRuntimeError.js';
3
4
  function asRecord(value) {
4
5
  return value !== null && typeof value === 'object' && !Array.isArray(value)
@@ -41,11 +42,9 @@ export async function resolveAsyncAudioUrl(source, modelId, apiKey, apiBaseUrl,
41
42
  }
42
43
  return { url: parsed.toString(), usesOss: false };
43
44
  }
44
- const media = await readCapabilityMediaSource(source, context.runtime.media);
45
- context.runtime.logger.info('百炼音频上传开始', {
46
- event: 'capability.bailian_asr.upload.start', requestId: context.requestId,
47
- providerId: 'bailian', modelId, context: { bytes: media.bytes.byteLength, mediaType: media.mimeType },
48
- });
45
+ if (source.kind === 'media-ref' && !context.runtime.transport.uploadFile) {
46
+ throw new AiRuntimeError('native_upload_unsupported', '长音频上传需要宿主接入原生文件直传;请升级宿主适配器');
47
+ }
49
48
  const policyResponse = await context.runtime.transport.fetch(`${apiBaseUrl}/uploads?action=getPolicy&model=${encodeURIComponent(modelId)}`, { headers: { Authorization: `Bearer ${apiKey}` }, signal: context.signal });
50
49
  const policyPayload = asRecord(await json(policyResponse, 'upload policy'));
51
50
  const policy = asRecord(policyPayload?.data);
@@ -60,15 +59,16 @@ export async function resolveAsyncAudioUrl(source, modelId, apiKey, apiBaseUrl,
60
59
  if (host.protocol !== 'https:')
61
60
  throw new AiRuntimeError('invalid_response', 'Bailian upload_host must use HTTPS');
62
61
  const uploadDir = requiredString(policy, 'upload_dir').replace(/^\/+|\/+$/g, '');
63
- const maxFileSizeMb = policy?.max_file_size_mb;
64
- if (typeof maxFileSizeMb === 'number'
65
- && Number.isFinite(maxFileSizeMb)
66
- && media.bytes.byteLength > maxFileSizeMb * 1024 * 1024) {
67
- throw new AiRuntimeError('media_too_large', `Bailian temporary upload limit is ${maxFileSizeMb} MB`);
62
+ const rawLimit = policy?.max_file_size_mb;
63
+ const maxFileSizeMb = typeof rawLimit === 'number' || typeof rawLimit === 'string' ? Number(rawLimit) : NaN;
64
+ if (!Number.isFinite(maxFileSizeMb) || maxFileSizeMb <= 0) {
65
+ throw new AiRuntimeError('invalid_response', 'Bailian upload policy has invalid max_file_size_mb');
68
66
  }
69
- const safeName = media.filename.replace(/[^A-Za-z0-9._-]/g, '_') || 'audio';
70
- const objectKey = `${uploadDir}/${context.requestId.replace(/[^A-Za-z0-9._-]/g, '_')}-${Date.now()}-${safeName}`;
71
- const form = new FormData();
67
+ // Official temporary storage ceiling is 1 GB; use decimal bytes conservatively.
68
+ const maxBytes = Math.min(1_000_000_000, Math.floor(maxFileSizeMb * 1_000_000));
69
+ const objectKey = `${uploadDir}/${context.requestId.replace(/[^A-Za-z0-9._-]/g, '_')}-${Date.now()}-audio`;
70
+ const fields = [];
71
+ const form = { append: (key, value) => { fields.push([key, value]); } };
72
72
  form.append('OSSAccessKeyId', requiredString(policy, 'oss_access_key_id'));
73
73
  form.append('Signature', requiredString(policy, 'signature'));
74
74
  form.append('policy', requiredString(policy, 'policy'));
@@ -76,12 +76,26 @@ export async function resolveAsyncAudioUrl(source, modelId, apiKey, apiBaseUrl,
76
76
  form.append('x-oss-forbid-overwrite', requiredString(policy, 'x_oss_forbid_overwrite'));
77
77
  form.append('key', objectKey);
78
78
  form.append('success_action_status', '200');
79
- const uploadBytes = new Uint8Array(media.bytes.byteLength);
80
- uploadBytes.set(media.bytes);
81
- form.append('file', new Blob([uploadBytes], { type: media.mimeType }), media.filename);
82
- const uploadResponse = await context.runtime.transport.fetch(host.toString(), {
83
- method: 'POST', body: form, signal: context.signal,
79
+ context.runtime.logger.info('百炼音频上传开始', {
80
+ event: 'capability.bailian_asr.upload.start', requestId: context.requestId, providerId: 'bailian', modelId,
84
81
  });
82
+ let uploadResponse;
83
+ if (source.kind === 'media-ref') {
84
+ uploadResponse = await context.runtime.transport.uploadFile(host.toString(), {
85
+ ref: source.ref, fields, fileField: 'file', maxBytes, signal: context.signal,
86
+ }).catch(rethrowMediaError);
87
+ }
88
+ else {
89
+ const media = await readCapabilityMediaSource(source, context.runtime.media);
90
+ assertMediaSize({ size: media.bytes.byteLength, mimeType: media.mimeType, filename: media.filename }, maxBytes);
91
+ const multipart = new FormData();
92
+ for (const [key, value] of fields)
93
+ multipart.append(key, value);
94
+ multipart.append('file', new Blob([new Uint8Array(media.bytes)], { type: media.mimeType }), media.filename);
95
+ uploadResponse = await context.runtime.transport.fetch(host.toString(), {
96
+ method: 'POST', body: multipart, signal: context.signal,
97
+ });
98
+ }
85
99
  if (!uploadResponse.ok) {
86
100
  throw new AiRuntimeError('upload_failed', `Bailian OSS upload failed with HTTP ${uploadResponse.status}`);
87
101
  }
@@ -1,8 +1,8 @@
1
- import { readCapabilityMediaSource } from '../../media.js';
1
+ import { prepareMedia, multipartAudioRequest, sendMediaRequest } from '../../media-request.js';
2
2
  import { AiRuntimeError, cancelledError } from '../../../runtime/AiRuntimeError.js';
3
3
  import { parseGroqTranscription } from './parse.js';
4
4
  const DEFAULT_API_BASE = 'https://api.groq.com/openai/v1';
5
- const DEFAULT_MAX_FILE_BYTES = 25 * 1024 * 1024;
5
+ const DEFAULT_MAX_FILE_BYTES = 25_000_000;
6
6
  function endpoint(value) {
7
7
  const normalized = (value?.trim() || DEFAULT_API_BASE).replace(/\/+$/, '');
8
8
  let parsed;
@@ -123,22 +123,12 @@ function validateOptions(provider) {
123
123
  async function formData(preset, input, maxFileBytes, context) {
124
124
  const provider = options(input);
125
125
  validateOptions(provider);
126
- const form = new FormData();
126
+ const fields = [];
127
+ const form = { append: (key, value) => { fields.push([key, value]); } };
127
128
  form.append('model', preset.modelId);
128
129
  if (input.audio.kind === 'remote-url') {
129
130
  form.append('url', remoteUrl(input.audio.url));
130
131
  }
131
- else {
132
- const media = await readCapabilityMediaSource(input.audio, context.runtime.media);
133
- if (media.bytes.byteLength === 0)
134
- throw new AiRuntimeError('invalid_media', 'Groq transcription audio is empty');
135
- if (media.bytes.byteLength > maxFileBytes) {
136
- throw new AiRuntimeError('media_too_large', `Groq transcription audio exceeds ${maxFileBytes} bytes`);
137
- }
138
- const uploadBytes = new Uint8Array(media.bytes.byteLength);
139
- uploadBytes.set(media.bytes);
140
- form.append('file', new Blob([uploadBytes], { type: media.mimeType }), media.filename);
141
- }
142
132
  if (input.language?.trim())
143
133
  form.append('language', input.language.trim());
144
134
  if (provider.prompt?.trim())
@@ -150,7 +140,14 @@ async function formData(preset, input, maxFileBytes, context) {
150
140
  for (const granularity of timestampGranularities(input, provider)) {
151
141
  form.append('timestamp_granularities[]', granularity);
152
142
  }
153
- return form;
143
+ if (input.audio.kind === 'remote-url') {
144
+ const multipart = new FormData();
145
+ for (const [key, value] of fields)
146
+ multipart.append(key, value);
147
+ return { body: multipart };
148
+ }
149
+ const media = await prepareMedia(input.audio, context.runtime, maxFileBytes, context.signal);
150
+ return multipartAudioRequest(media, fields);
154
151
  }
155
152
  function errorMessage(payload) {
156
153
  if (!payload || typeof payload !== 'object' || Array.isArray(payload))
@@ -205,12 +202,12 @@ export function createGroqAsrModule(preset, moduleOptions = {}) {
205
202
  const provider = options(input);
206
203
  validateOptions(provider);
207
204
  const format = responseFormat(input, provider);
208
- const response = await context.runtime.transport.fetch(`${apiBaseUrl}/audio/transcriptions`, {
205
+ const request = await formData(preset, input, maxFileBytes, context);
206
+ const response = await sendMediaRequest(context.runtime, `${apiBaseUrl}/audio/transcriptions`, {
209
207
  method: 'POST',
210
208
  headers: { Authorization: `Bearer ${apiKey}` },
211
- body: await formData(preset, input, maxFileBytes, context),
212
209
  signal: context.signal,
213
- });
210
+ }, request);
214
211
  const output = await parseResponse(response, format);
215
212
  if (output.segments?.length) {
216
213
  for (const segment of output.segments)
@@ -9,6 +9,6 @@ export interface GroqAsrOptions {
9
9
  export interface GroqAsrModuleOptions {
10
10
  /** Groq OpenAI-compatible API root. */
11
11
  apiBaseUrl?: string;
12
- /** Local upload guard. Defaults to the Free plan's documented 25 MB limit; paid hosts may raise it explicitly. */
12
+ /** Local upload guard. Defaults to the documented 25 MB attachment limit; larger files require a remote URL, even on paid plans. */
13
13
  maxFileBytes?: number;
14
14
  }