copilot-custom-endpoint 3.7.1 → 4.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +47 -25
- package/docs/example-config.md +89 -63
- package/docs/models/deepseek.md +122 -8
- package/docs/models/glm.md +44 -44
- package/docs/models/kimi.md +15 -6
- package/docs/models/mimo.md +34 -40
- package/docs/models/minimax.md +38 -41
- package/docs/models/qwen.md +35 -30
- package/docs/pricing.md +15 -3
- package/lib/kimi-rewrite.mjs +78 -0
- package/lib/mimo-rewrite.mjs +1 -20
- package/lib/qwen-rewrite.mjs +45 -0
- package/lib/rewrite-summary.mjs +40 -0
- package/package.json +1 -1
- package/proxy/kimi-proxy.mjs +8 -102
- package/proxy/qwen-proxy.mjs +6 -63
package/docs/models/qwen.md
CHANGED
|
@@ -1,23 +1,25 @@
|
|
|
1
1
|
# Qwen (DashScope) — VS Code Custom Endpoint Setup Guide
|
|
2
2
|
|
|
3
|
-
> **TL;DR:** `qwen3.8-max`
|
|
3
|
+
> **TL;DR:** `qwen3.8-max` works both direct and via the local proxy. The proxy forwards the upstream `model` value unchanged, so use the exact model ID supported by the endpoint you choose. The proxy gives you dynamic thinking suppression: reasoning stays ON in plain chat but turns OFF automatically when tools are invoked. The direct path is simpler if you don't need reasoning in chat. Use the **OpenAI-compatible** endpoint on your **workspace-dedicated** domain ([details](#which-endpoint-to-use)).
|
|
4
4
|
|
|
5
5
|
## At a Glance
|
|
6
6
|
|
|
7
|
-
| Field
|
|
8
|
-
|
|
|
9
|
-
| Mode
|
|
10
|
-
| Billing
|
|
11
|
-
| Vision
|
|
12
|
-
| Tool calling
|
|
13
|
-
| Context
|
|
14
|
-
| Max output
|
|
15
|
-
| Protocol
|
|
16
|
-
| Endpoint
|
|
17
|
-
| Proxy endpoint
|
|
18
|
-
| Auth
|
|
19
|
-
| `
|
|
20
|
-
| `requestBody`
|
|
7
|
+
| Field | Value |
|
|
8
|
+
| -------------- | -------------------------------------------------------------------------------------------------- |
|
|
9
|
+
| Mode | **Proxy** (local on `:3458`) **or** **Direct** (static `enable_thinking: false`) |
|
|
10
|
+
| Billing | **Pay-as-You-Go only** — 1M-token free quota for new users |
|
|
11
|
+
| Vision | ✅ Yes (`qwen3.8-max`) |
|
|
12
|
+
| Tool calling | ✅ Yes |
|
|
13
|
+
| Context | 1M |
|
|
14
|
+
| Max output | 131K |
|
|
15
|
+
| Protocol | **OpenAI-compatible** (`/compatible-mode/v1`) — not native DashScope (`/api/v1`) |
|
|
16
|
+
| Endpoint | `https://<your-workspace-id>.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions` |
|
|
17
|
+
| Proxy endpoint | `http://127.0.0.1:3458/v1/chat/completions` |
|
|
18
|
+
| Auth | `Authorization: Bearer $DASHSCOPE_API_KEY` |
|
|
19
|
+
| `modelOptions` | `temperature: 1`, `top_p: 0.95` (the only sampling control VS Code forwards) |
|
|
20
|
+
| `requestBody` | **Not forwarded on VS Code 1.146** — thinking must be controlled by the proxy |
|
|
21
|
+
|
|
22
|
+
> **Live-validated 2026-11-13** (VS Code 1.146, both harnesses): plain chat plus a multi-step tool loop returned HTTP 200 on both turns through the local proxy. Turn 1 carried 128 tools and no `tool_choice` (so thinking stayed at the DashScope default) and still succeeded; the proxy injected `enable_thinking: false` on the tool turn. Copilot sends 128 tools and Local sends 57; vision and long-context compaction were not tested. See [harness research § 9](../../research/vscode-1.146-copilot-harness.md#9-phase-3-live-provider-validation-2026-11-13).
|
|
21
23
|
|
|
22
24
|
### Models
|
|
23
25
|
|
|
@@ -25,7 +27,7 @@
|
|
|
25
27
|
| ------------- | ------ | ------- | ---------- | ------------------------------------------------------- |
|
|
26
28
|
| `qwen3.8-max` | ✅ Yes | 1M | 131K | Frontier multimodal model; reasoning enabled by default |
|
|
27
29
|
|
|
28
|
-
> The live `chatLanguageModels.json` points Qwen at the local proxy by default
|
|
30
|
+
> The live `chatLanguageModels.json` points Qwen at the local proxy by default. A direct DashScope entry cannot disable reasoning on VS Code 1.146 — `requestBody` is dropped — so the direct path below runs with DashScope's thinking default and is listed only for connectivity checks.
|
|
29
31
|
|
|
30
32
|
## Quick Start
|
|
31
33
|
|
|
@@ -78,25 +80,27 @@ API keys are bound to their region (and, on a workspace-dedicated domain, to the
|
|
|
78
80
|
{
|
|
79
81
|
"name": "Qwen",
|
|
80
82
|
"vendor": "customendpoint",
|
|
81
|
-
"apiKey": "",
|
|
83
|
+
"apiKey": "${input:qwenApiKey}",
|
|
82
84
|
"apiType": "chat-completions",
|
|
83
85
|
"models": [
|
|
84
86
|
{
|
|
85
87
|
"id": "qwen3.8-max",
|
|
86
|
-
"name": "Qwen 3.8 Max
|
|
88
|
+
"name": "Qwen 3.8 Max",
|
|
87
89
|
"url": "https://<your-workspace-id>.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions",
|
|
88
90
|
"toolCalling": true,
|
|
89
91
|
"vision": true,
|
|
90
92
|
"streaming": true,
|
|
91
|
-
"maxInputTokens":
|
|
93
|
+
"maxInputTokens": 868928,
|
|
92
94
|
"maxOutputTokens": 131072,
|
|
93
|
-
"
|
|
95
|
+
"modelOptions": { "temperature": 1, "top_p": 0.95 }
|
|
94
96
|
}
|
|
95
97
|
]
|
|
96
98
|
}
|
|
97
99
|
```
|
|
98
100
|
|
|
99
|
-
>
|
|
101
|
+
> **Context budget:** `868928 + 131072 = 1000000`, the `qwen3.8-max` window. VS Code uses that sum as the model's total context; DashScope separately caps input at 991000 tokens, so 868928 leaves room for the output budget. Raise `maxInputTokens` only by lowering `maxOutputTokens`.
|
|
102
|
+
|
|
103
|
+
> **Direct path caveat:** `enable_thinking` cannot be set from `chatLanguageModels.json` on VS Code 1.146, so reasoning stays at DashScope's default and tool loops may hit `reasoning_content` errors. Use the proxy for tool-heavy work.
|
|
100
104
|
|
|
101
105
|
#### Proxy path
|
|
102
106
|
|
|
@@ -112,23 +116,23 @@ API keys are bound to their region (and, on a workspace-dedicated domain, to the
|
|
|
112
116
|
curl http://127.0.0.1:3458/healthz
|
|
113
117
|
```
|
|
114
118
|
|
|
115
|
-
2. Use this snippet (URLs point at the proxy; no
|
|
119
|
+
2. Use this snippet (URLs point at the proxy; no thinking configuration needed):
|
|
116
120
|
|
|
117
121
|
```json
|
|
118
122
|
{
|
|
119
123
|
"name": "Qwen",
|
|
120
124
|
"vendor": "customendpoint",
|
|
121
|
-
"apiKey": "",
|
|
125
|
+
"apiKey": "${input:qwenApiKey}",
|
|
122
126
|
"apiType": "chat-completions",
|
|
123
127
|
"models": [
|
|
124
128
|
{
|
|
125
129
|
"id": "qwen3.8-max",
|
|
126
|
-
"name": "Qwen 3.8 Max
|
|
130
|
+
"name": "Qwen 3.8 Max",
|
|
127
131
|
"url": "http://127.0.0.1:3458/v1/chat/completions",
|
|
128
132
|
"toolCalling": true,
|
|
129
133
|
"vision": true,
|
|
130
134
|
"streaming": true,
|
|
131
|
-
"maxInputTokens":
|
|
135
|
+
"maxInputTokens": 868928,
|
|
132
136
|
"maxOutputTokens": 131072
|
|
133
137
|
}
|
|
134
138
|
]
|
|
@@ -146,7 +150,7 @@ DashScope is **Pay-as-You-Go only** — but new Model Studio users get **1M inpu
|
|
|
146
150
|
3. Find the **Qwen** group → **Update API Key**.
|
|
147
151
|
4. Paste your DashScope API key.
|
|
148
152
|
|
|
149
|
-
>
|
|
153
|
+
> **`apiKey` must be non-empty** — the file's schema enforces `minLength: 1`, so `"apiKey": ""` shows _String is shorter than the minimum length of 1_. Use the `${input:...}` placeholder shown above and set it through the Language Models UI, which rewrites it to a `${input:chat.lm.secret.<id>}` keychain reference. Keys are region-specific.
|
|
150
154
|
>
|
|
151
155
|
> New keys start with `sk-ws-` and are shown only once, in the **API Quick Access** dialog — copy the key before closing it. Older `sk-` keys keep working.
|
|
152
156
|
|
|
@@ -193,8 +197,9 @@ The proxy detects active tool use by examining the conversation state (a `"tool"
|
|
|
193
197
|
## Notes
|
|
194
198
|
|
|
195
199
|
- **Vision (`qwen3.8-max`)** uses OpenAI-compatible `content` array format. Base64 data URIs work reliably; external image URLs may fail if DashScope can't reach them. If a drag-and-drop image fails to load, providing the absolute file path (e.g. `c:\path\to\image.png`) in the prompt is a reliable workaround.
|
|
196
|
-
- **Qwen 3.8 reasoning:** `qwen3.8-max` enables reasoning by default and supports `reasoning_effort` values `low`, `medium`, and `xhigh` (default).
|
|
197
|
-
- **Thinking trade-off:**
|
|
200
|
+
- **Qwen 3.8 reasoning:** `qwen3.8-max` enables reasoning by default and supports `reasoning_effort` values `low`, `medium`, and `xhigh` (default). On VS Code 1.146 neither `enable_thinking` nor `reasoning_effort` can be set from `requestBody` — the proxy is what toggles thinking, leaving it on for plain chat and suppressing it when tool activity is detected.
|
|
201
|
+
- **Thinking trade-off:** Proxy = thinking on in plain chat, off in tool turns. Without the proxy, thinking stays on and tool loops are at the provider's mercy.
|
|
202
|
+
- **First tool-capable request:** detection is history-based, so an initial request that sends a `tools` array with no `tool_choice` and no tool-role history is treated as plain chat (thinking stays on). Measured on VS Code 1.146: both the Copilot and Local harnesses send their full tool set with **no** `tool_choice`, so the first turn is always plain chat. `tool_choice: "none"` is also treated as plain chat. See [the harness research](../research/vscode-1.146-copilot-harness.md).
|
|
198
203
|
- **`tool_choice` only supports `auto`** — don't override it (VS Code's default is `auto`).
|
|
199
204
|
|
|
200
205
|
## Troubleshooting
|
|
@@ -202,12 +207,12 @@ The proxy detects active tool use by examining the conversation state (a `"tool"
|
|
|
202
207
|
| Symptom | Likely cause | Fix |
|
|
203
208
|
| ----------------------------------------------- | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
|
|
204
209
|
| "Connection refused" (proxy mode) | Proxy not running | `npm run proxy:qwen` |
|
|
205
|
-
| Tool loops fail with `reasoning_content` errors | Direct path
|
|
210
|
+
| Tool loops fail with `reasoning_content` errors | Direct path with thinking on; `requestBody` cannot disable it | Use the proxy (`npm run proxy:qwen`) |
|
|
206
211
|
| Tool loops still fail with proxy | Proxy not rewriting | Check `debug_log/qwen-proxy.ndjson` — verify `hasTools: true` requests have `rewrittenEnableThinking: false` |
|
|
207
212
|
| Vision fails with external image URL | DashScope couldn't reach the URL | Use a base64 data URI instead |
|
|
208
213
|
| 401 Unauthorized | API key region or workspace mismatch | Match your key to the endpoint's region (and workspace, on a workspace-dedicated domain) |
|
|
209
214
|
| 404 Not Found | URL uses the native DashScope path (`/api/v1`) or omits the path | Use `/compatible-mode/v1/chat/completions` |
|
|
210
|
-
| Want to switch back to direct | Proxy mode active | Revert `url` to DashScope endpoint
|
|
215
|
+
| Want to switch back to direct | Proxy mode active | Revert `url` to the DashScope endpoint; reasoning stays at the provider default |
|
|
211
216
|
|
|
212
217
|
## Pricing
|
|
213
218
|
|
package/docs/pricing.md
CHANGED
|
@@ -20,7 +20,7 @@ All prices below are in **USD per 1M tokens** (non-cached). To convert to AI cre
|
|
|
20
20
|
|
|
21
21
|
All models are listed together below, sorted by Cost per intelligence ascending (lower is better). Models without a Cost per intelligence are ordered by estimated session cost ascending. Session cost assumes ~10K input + ~2K output tokens per turn, 50 turns.
|
|
22
22
|
|
|
23
|
-
**All AA Intelligence Index values in this table were re-based from v4.3.2 to v5.0 on October 31, 2026.** AA’s v5.0 composite aggregates a different evaluation set than v4.3.2, so scores are not comparable with earlier snapshots of this file.
|
|
23
|
+
**All AA Intelligence Index values in this table were re-based from v4.3.2 to v5.0 on October 31, 2026.** AA’s v5.0 composite aggregates a different evaluation set than v4.3.2, so scores are not comparable with earlier snapshots of this file. AA has since relabelled the index **v5.0.1** without changing the ordering; `Gemini 4 Carbon`'s **60** is the current v5.0.1 value (Google's October 28 launch post listed **59** under v5.0).
|
|
24
24
|
|
|
25
25
|
| Model | Provider | Cost per intelligence | Intelligence Score | Input (per 1M) | Cached input | Output (per 1M) | Est. session | Context window |
|
|
26
26
|
| ---------------------------- | --------- | --------------------- | ------------------ | ----------------------------- | ----------------------------- | ----------------------------- | ------------ | -------------- |
|
|
@@ -33,7 +33,9 @@ All models are listed together below, sorted by Cost per intelligence ascending
|
|
|
33
33
|
| **MiMo V2.6 Pro** | Xiaomi | **~$0.0102** | **29.9** | $0.435 | $0.0036 | $0.87 | ~$0.30 | 1M |
|
|
34
34
|
| **GPT-6.1 Terra** | OpenAI | **~$0.0196** | **51** | $1.00 | $0.05 | $5.00 | ~$1.00 | 1M |
|
|
35
35
|
| **Gemini 3.8 Flash** | Google | **~$0.0246** | **30.5** | $0.75 | $0.075 | $3.75 | ~$0.75 | 1M |
|
|
36
|
+
| **Gemini 4 Hydrogen** | Google | **~$0.0278** | **54** | $1.50 | $0.075 | $7.50 | ~$1.50 | 1M |
|
|
36
37
|
| **Muse Spark 1.3** | Meta | **~$0.0288** | **36.4** | $1.25 | $0.15 | $4.25 | ~$1.05 | 1M |
|
|
38
|
+
| **Gemini 4 Carbon** | Google | **~$0.0333** | **60** | $2.00 | $0.10 | $10.00 | ~$2.00 | 1M |
|
|
37
39
|
| **GLM 5.3** | Z.ai | **~$0.0360** | **31.7** | $1.40 | $0.26 | $4.40 | ~$1.14 | 1M |
|
|
38
40
|
| **GPT-6.1 Sol** | OpenAI | **~$0.0367** | **54.5** | $2.00 | $0.10 | $10.00 | ~$2.00 | 1M |
|
|
39
41
|
| **Claude Sonnet 5.5** | Anthropic | **~$0.0369** | **54.2** | $2.00 | $0.10 | $10.00 | ~$2.00 | 1M |
|
|
@@ -56,7 +58,8 @@ Cost per intelligence = estimated session cost ÷ Intelligence Index score. Sess
|
|
|
56
58
|
> - **MiMo:** the table shows cache-miss pricing; cache hits are far cheaper (see the Cached input column). A Token Plan subscription with discounted rates is also available.
|
|
57
59
|
> - **Anthropic:** cache reads and writes are billed separately. Haiku 5.5 is the only Claude model with prompt-length tiering; the table shows its ≤100K rates. Fast mode is $8 / $40 per 1M.
|
|
58
60
|
> - **OpenAI:** GPT-6 Sol, GPT-6.1 Sol, and Luna bill cache writes at 1.25× input. Requests over 272K input cost 2× input and cache rates, with 1.5× output.
|
|
59
|
-
> - **Gemini 4 Argon:**
|
|
61
|
+
> - **Gemini 4 Argon and Gemini 4 Carbon:** both bill $2 / $10 per 1M with $0.10 implicit cache through November 30, 2026, then rise to $4 / $20 with $0.20 cache; explicit caching is unsupported. Output includes thinking tokens. Batch and Flex are half price; Priority is $3.60 / $18. Carbon is the more deliberate of the two and spends more tokens per task, so its session cost runs higher than Argon's at identical rates.
|
|
62
|
+
> - **Gemini 4 Hydrogen:** flat $1.50 / $7.50 per 1M with $0.075 implicit cache across the entire 1M window; explicit caching is unsupported. Batch and Flex are half price ($0.75 / $3.75), Priority is $2.70 / $13.50, and a free tier is available in AI Studio. Hydrogen always thinks (`low` / `medium` / `high` / `xhigh`), so token spend — and this modeled session cost — rises with the thinking level; its **54** Intelligence Score is Artificial Analysis' `xhigh` measurement, while Google reports **51** at the default `high` level. It is API-only and not Copilot-native as of November 13, 2026.
|
|
60
63
|
> - **Kimi K3.1 Pro and MiniMax M3.1 Flash:** both always think, so control cost with `reasoning_effort`. See [docs/models/kimi.md](models/kimi.md) and [docs/models/minimax.md](models/minimax.md).
|
|
61
64
|
> - **GLM:** cache hits are $0.26/M for 5.3 and $0.03/M for 5.3 Flash at list.
|
|
62
65
|
> - **Qwen:** pricing is tiered by input tokens per request; the table shows non-thinking rates. DashScope offers 1M input + 1M output free tokens per model for 90 days.
|
|
@@ -65,7 +68,7 @@ Cost per intelligence = estimated session cost ÷ Intelligence Index score. Sess
|
|
|
65
68
|
|
|
66
69
|
> **How long does 7,000 credits last?** A Pro+ subscriber running 50-turn sessions can estimate monthly capacity by multiplying each modeled session cost by 100 to convert it to AI credits, then mixing models as needed.
|
|
67
70
|
|
|
68
|
-
> Prices last verified: November
|
|
71
|
+
> Prices last verified: November 13, 2026. Always check the official pages for the latest rates:
|
|
69
72
|
>
|
|
70
73
|
> - [GitHub Copilot models & pricing](https://docs.github.com/en/copilot/reference/copilot-billing/models-and-pricing)
|
|
71
74
|
> - [OpenAI pricing](https://openai.com/api/pricing/)
|
|
@@ -94,6 +97,15 @@ Cost per intelligence = estimated session cost ÷ Intelligence Index score. Sess
|
|
|
94
97
|
> - [GitHub Copilot supported models](https://docs.github.com/en/copilot/reference/ai-models/supported-models)
|
|
95
98
|
> - [Artificial Analysis GPT-6 Sol](https://artificialanalysis.ai/models/gpt-6-sol), [GPT-6 Luna](https://artificialanalysis.ai/models/gpt-6-luna), [GPT-6.1 Sol](https://artificialanalysis.ai/models/gpt-6-1-sol), [GPT-6.1 Terra](https://artificialanalysis.ai/models/gpt-6-1-terra), [Claude Opus 5.5](https://artificialanalysis.ai/models/claude-opus-5-5), [Claude Sonnet 5.5](https://artificialanalysis.ai/models/claude-sonnet-5-5), [Claude Haiku 5.5](https://artificialanalysis.ai/models/claude-haiku-5-5), [MiMo-V2.6-Flash](https://artificialanalysis.ai/models/mimo-v2-6-flash), [Gemini 4 Argon](https://artificialanalysis.ai/models/gemini-4-argon), [Muse Spark 1.3](https://artificialanalysis.ai/models/muse-spark-1-3), [Kimi K3.1 Pro](https://artificialanalysis.ai/models/kimi-k3-1-pro), and [MiniMax-M3.1-Flash](https://artificialanalysis.ai/models/minimax-m3-1-flash)
|
|
96
99
|
> - [Google Gemini pricing](https://ai.google.dev/pricing)
|
|
100
|
+
> - [Google Gemini 4 Carbon announcement](https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-carbon/)
|
|
101
|
+
> - [Google Gemini 4 Carbon availability](https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-4-carbon-now-available/)
|
|
102
|
+
> - [Google Gemini 4 Carbon model card](https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-4-Carbon-Model-Card.pdf)
|
|
103
|
+
> - [GitHub Copilot Gemini 4 Carbon release](https://github.blog/changelog/2026-11-10-gemini-4-carbon-in-github-copilot)
|
|
104
|
+
> - [Artificial Analysis Gemini 4 Carbon](https://artificialanalysis.ai/models/gemini-4-carbon)
|
|
105
|
+
> - [Google Gemini 4 Hydrogen announcement](https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-hydrogen/)
|
|
106
|
+
> - [Google Gemini 4 Hydrogen model card](https://deepmind.google/models/model-cards/gemini-4-hydrogen/)
|
|
107
|
+
> - [Google Gemini 4 Hydrogen pricing](https://ai.google.dev/gemini-api/docs/pricing#gemini-4-hydrogen)
|
|
108
|
+
> - [Artificial Analysis Gemini 4 Hydrogen](https://artificialanalysis.ai/models/gemini-4-hydrogen)
|
|
97
109
|
> - [Google Gemini 4 Argon (what's new)](https://ai.google.dev/gemini-api/docs/latest-model)
|
|
98
110
|
> - [Google Gemini 4 Argon model card](https://ai.google.dev/gemini-api/docs/models/gemini-4-argon)
|
|
99
111
|
> - [GitHub Copilot Gemini 4 Argon release](https://github.blog/changelog/2026-10-13-gemini-4-argon-in-github-copilot)
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
import { hasActiveToolCall, summarizePayload } from './rewrite-summary.mjs'
|
|
2
|
+
|
|
3
|
+
export const KIMI_REWRITE_DEFAULTS = {
|
|
4
|
+
forcedTemperature: 1,
|
|
5
|
+
forcedNonThinkingTemperature: 0.6,
|
|
6
|
+
forcedTopP: 0.95,
|
|
7
|
+
disableThinkingWithTools: true
|
|
8
|
+
}
|
|
9
|
+
|
|
10
|
+
/**
|
|
11
|
+
* Rewrite a Moonshot/Kimi chat-completions payload in place.
|
|
12
|
+
*
|
|
13
|
+
* K2.7 and K3 are always-thinking and reject `thinking: disabled`, so the
|
|
14
|
+
* thinking-disable rewrite is skipped for them. K3 uses `reasoning_effort`
|
|
15
|
+
* instead of `thinking` and rejects the `thinking` parameter entirely.
|
|
16
|
+
*
|
|
17
|
+
* @returns {{ summary: object, consoleMsg: string }}
|
|
18
|
+
*/
|
|
19
|
+
export function rewriteKimi(payload, options = {}) {
|
|
20
|
+
const {
|
|
21
|
+
forcedTemperature,
|
|
22
|
+
forcedNonThinkingTemperature,
|
|
23
|
+
forcedTopP,
|
|
24
|
+
disableThinkingWithTools
|
|
25
|
+
} = { ...KIMI_REWRITE_DEFAULTS, ...options }
|
|
26
|
+
|
|
27
|
+
const incomingTemperature = payload.temperature
|
|
28
|
+
const incomingTopP = payload.top_p
|
|
29
|
+
const incomingThinkingType = payload?.thinking?.type
|
|
30
|
+
const incomingReasoningEffort = payload?.reasoning_effort
|
|
31
|
+
const model = payload.model ?? ''
|
|
32
|
+
|
|
33
|
+
const isK27 = model.startsWith('kimi-k2.7')
|
|
34
|
+
const isK3 = model.startsWith('kimi-k3')
|
|
35
|
+
const isAlwaysThinking = isK27 || isK3
|
|
36
|
+
|
|
37
|
+
const hasTools = hasActiveToolCall(payload)
|
|
38
|
+
const useNonThinkingMode =
|
|
39
|
+
!isAlwaysThinking && disableThinkingWithTools && hasTools
|
|
40
|
+
const rewrittenTemperature = useNonThinkingMode
|
|
41
|
+
? forcedNonThinkingTemperature
|
|
42
|
+
: forcedTemperature
|
|
43
|
+
|
|
44
|
+
payload.temperature = rewrittenTemperature
|
|
45
|
+
payload.top_p = forcedTopP
|
|
46
|
+
|
|
47
|
+
if (useNonThinkingMode) {
|
|
48
|
+
payload.thinking = { type: 'disabled' }
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
// K3 does not accept `thinking` — delete it if present so it doesn't cause a 400
|
|
52
|
+
if (isK3 && payload.thinking) {
|
|
53
|
+
delete payload.thinking
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
const rewriteInfo = {
|
|
57
|
+
model,
|
|
58
|
+
isK27,
|
|
59
|
+
isK3,
|
|
60
|
+
incomingTemperature,
|
|
61
|
+
rewrittenTemperature,
|
|
62
|
+
incomingTopP,
|
|
63
|
+
rewrittenTopP: forcedTopP,
|
|
64
|
+
incomingThinkingType,
|
|
65
|
+
rewrittenThinkingType: payload.thinking?.type,
|
|
66
|
+
incomingReasoningEffort
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
const modeTag = hasTools ? '[tools]' : '[chat]'
|
|
70
|
+
const k27Tag = isK27 ? '[k2.7]' : ''
|
|
71
|
+
const k3Tag = isK3 ? '[k3]' : ''
|
|
72
|
+
const consoleMsg = `${k27Tag}${k3Tag}${modeTag} temperature ${String(incomingTemperature)} -> ${String(rewrittenTemperature)}, top_p ${String(incomingTopP)} -> ${String(forcedTopP)}, thinking ${String(incomingThinkingType)} -> ${String(payload.thinking?.type)}, reasoning_effort ${String(incomingReasoningEffort)}`
|
|
73
|
+
|
|
74
|
+
return {
|
|
75
|
+
summary: summarizePayload(payload, hasTools, rewriteInfo),
|
|
76
|
+
consoleMsg
|
|
77
|
+
}
|
|
78
|
+
}
|
package/lib/mimo-rewrite.mjs
CHANGED
|
@@ -1,23 +1,4 @@
|
|
|
1
|
-
|
|
2
|
-
const messages = Array.isArray(payload.messages) ? payload.messages : []
|
|
3
|
-
const tools = Array.isArray(payload.tools) ? payload.tools : []
|
|
4
|
-
|
|
5
|
-
return {
|
|
6
|
-
model: payload.model,
|
|
7
|
-
stream: payload.stream,
|
|
8
|
-
hasTools,
|
|
9
|
-
toolCount: tools.length,
|
|
10
|
-
toolChoice: payload.tool_choice,
|
|
11
|
-
...rewriteInfo,
|
|
12
|
-
maxTokens:
|
|
13
|
-
payload.max_tokens ??
|
|
14
|
-
payload.max_completion_tokens ??
|
|
15
|
-
payload.max_output_tokens,
|
|
16
|
-
messageCount: messages.length,
|
|
17
|
-
messageRoles: messages.map((message) => message?.role).slice(0, 16),
|
|
18
|
-
topLevelKeys: Object.keys(payload).sort()
|
|
19
|
-
}
|
|
20
|
-
}
|
|
1
|
+
import { summarizePayload } from './rewrite-summary.mjs'
|
|
21
2
|
|
|
22
3
|
export function hasMissingReasoningContent(messages) {
|
|
23
4
|
return messages.some((message) => {
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
import { hasActiveToolCall, summarizePayload } from './rewrite-summary.mjs'
|
|
2
|
+
|
|
3
|
+
export const QWEN_REWRITE_DEFAULTS = {
|
|
4
|
+
disableThinkingWithTools: true
|
|
5
|
+
}
|
|
6
|
+
|
|
7
|
+
/**
|
|
8
|
+
* Rewrite a DashScope/Qwen chat-completions payload in place.
|
|
9
|
+
*
|
|
10
|
+
* Thinking stays on for plain chat (the field is removed so DashScope uses its
|
|
11
|
+
* default) and is suppressed only for detected tool activity.
|
|
12
|
+
*
|
|
13
|
+
* @returns {{ summary: object, consoleMsg: string }}
|
|
14
|
+
*/
|
|
15
|
+
export function rewriteQwen(payload, options = {}) {
|
|
16
|
+
const { disableThinkingWithTools } = {
|
|
17
|
+
...QWEN_REWRITE_DEFAULTS,
|
|
18
|
+
...options
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
const hasTools = hasActiveToolCall(payload)
|
|
22
|
+
const incomingEnableThinking = payload.enable_thinking
|
|
23
|
+
const suppressThinking = disableThinkingWithTools && hasTools
|
|
24
|
+
|
|
25
|
+
if (suppressThinking) {
|
|
26
|
+
payload.enable_thinking = false
|
|
27
|
+
} else {
|
|
28
|
+
delete payload.enable_thinking
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
const rewriteInfo = {
|
|
32
|
+
incomingEnableThinking,
|
|
33
|
+
rewrittenEnableThinking: suppressThinking ? false : undefined
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
const modeTag = hasTools ? '[tools]' : '[chat]'
|
|
37
|
+
const consoleMsg = `${modeTag} enable_thinking=${String(incomingEnableThinking)} -> ${
|
|
38
|
+
suppressThinking ? 'false' : '<deleted>'
|
|
39
|
+
}, model=${payload.model ?? '?'}`
|
|
40
|
+
|
|
41
|
+
return {
|
|
42
|
+
summary: summarizePayload(payload, hasTools, rewriteInfo),
|
|
43
|
+
consoleMsg
|
|
44
|
+
}
|
|
45
|
+
}
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Build the NDJSON log summary shared by the provider rewrites.
|
|
3
|
+
* Keep this shape stable — it is the documented debug_log contract.
|
|
4
|
+
*/
|
|
5
|
+
export function summarizePayload(payload, hasTools, rewriteInfo) {
|
|
6
|
+
const messages = Array.isArray(payload.messages) ? payload.messages : []
|
|
7
|
+
const tools = Array.isArray(payload.tools) ? payload.tools : []
|
|
8
|
+
|
|
9
|
+
return {
|
|
10
|
+
model: payload.model,
|
|
11
|
+
stream: payload.stream,
|
|
12
|
+
...rewriteInfo,
|
|
13
|
+
maxTokens:
|
|
14
|
+
payload.max_tokens ??
|
|
15
|
+
payload.max_completion_tokens ??
|
|
16
|
+
payload.max_output_tokens,
|
|
17
|
+
toolChoice: payload.tool_choice,
|
|
18
|
+
toolCount: tools.length,
|
|
19
|
+
hasTools,
|
|
20
|
+
messageCount: messages.length,
|
|
21
|
+
messageRoles: messages.map((message) => message?.role).slice(0, 16),
|
|
22
|
+
topLevelKeys: Object.keys(payload).sort()
|
|
23
|
+
}
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
/**
|
|
27
|
+
* A request counts as tool activity when a "tool" role message is present or
|
|
28
|
+
* `tool_choice` is set to anything other than "none"/null. A nonempty `tools`
|
|
29
|
+
* array alone is not enough — see docs/models/qwen.md.
|
|
30
|
+
*/
|
|
31
|
+
export function hasActiveToolCall(payload) {
|
|
32
|
+
const messages = Array.isArray(payload.messages) ? payload.messages : []
|
|
33
|
+
const hasToolRole = messages.some((message) => message?.role === 'tool')
|
|
34
|
+
const toolChoice = payload.tool_choice
|
|
35
|
+
|
|
36
|
+
return (
|
|
37
|
+
hasToolRole ||
|
|
38
|
+
(toolChoice !== undefined && toolChoice !== 'none' && toolChoice !== null)
|
|
39
|
+
)
|
|
40
|
+
}
|
package/package.json
CHANGED
package/proxy/kimi-proxy.mjs
CHANGED
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
import 'dotenv/config'
|
|
3
3
|
import { fileURLToPath } from 'node:url'
|
|
4
4
|
import { createProxy } from '../lib/create-proxy.mjs'
|
|
5
|
+
import { rewriteKimi } from '../lib/kimi-rewrite.mjs'
|
|
5
6
|
|
|
6
7
|
/**
|
|
7
8
|
* Supported model scope for this proxy:
|
|
@@ -86,107 +87,6 @@ if (!Number.isFinite(forcedTopP)) {
|
|
|
86
87
|
)
|
|
87
88
|
}
|
|
88
89
|
|
|
89
|
-
// ---- Provider-specific rewrite logic ----
|
|
90
|
-
|
|
91
|
-
function summarizePayload(payload, hasTools, rewriteInfo) {
|
|
92
|
-
const messages = Array.isArray(payload.messages) ? payload.messages : []
|
|
93
|
-
const tools = Array.isArray(payload.tools) ? payload.tools : []
|
|
94
|
-
|
|
95
|
-
return {
|
|
96
|
-
model: payload.model,
|
|
97
|
-
stream: payload.stream,
|
|
98
|
-
...rewriteInfo,
|
|
99
|
-
maxTokens:
|
|
100
|
-
payload.max_tokens ??
|
|
101
|
-
payload.max_completion_tokens ??
|
|
102
|
-
payload.max_output_tokens,
|
|
103
|
-
toolChoice: payload.tool_choice,
|
|
104
|
-
toolCount: tools.length,
|
|
105
|
-
hasTools,
|
|
106
|
-
messageCount: messages.length,
|
|
107
|
-
messageRoles: messages.map((message) => message?.role).slice(0, 16),
|
|
108
|
-
topLevelKeys: Object.keys(payload).sort()
|
|
109
|
-
}
|
|
110
|
-
}
|
|
111
|
-
|
|
112
|
-
function rewriteKimi(payload) {
|
|
113
|
-
const incomingTemperature = payload.temperature
|
|
114
|
-
const incomingTopP = payload.top_p
|
|
115
|
-
const incomingThinkingType = payload?.thinking?.type
|
|
116
|
-
const incomingReasoningEffort = payload?.reasoning_effort
|
|
117
|
-
const model = payload.model ?? ''
|
|
118
|
-
|
|
119
|
-
// K2.7 and K3 are always-thinking and reject thinking: disabled.
|
|
120
|
-
// K2.7 uses `thinking` parameter; K3 uses `reasoning_effort` instead.
|
|
121
|
-
// Detect K2.7 variants (e.g. kimi-k2.7-code) and K3-family variants
|
|
122
|
-
// (e.g. kimi-k3, kimi-k3.1-pro, kimi-k3.1-pro-highspeed) and skip the
|
|
123
|
-
// thinking-disable rewrite while keeping temperature/top_p enforcement.
|
|
124
|
-
const isK27 = model.startsWith('kimi-k2.7')
|
|
125
|
-
const isK3 = model.startsWith('kimi-k3')
|
|
126
|
-
const isAlwaysThinking = isK27 || isK3
|
|
127
|
-
|
|
128
|
-
// Determine if a tool is actually being invoked:
|
|
129
|
-
// - tool_choice is set and not "none"
|
|
130
|
-
// - OR there is a "tool" role message in the conversation
|
|
131
|
-
const messages = Array.isArray(payload.messages) ? payload.messages : []
|
|
132
|
-
const hasToolRole = messages.some((message) => message?.role === 'tool')
|
|
133
|
-
const toolChoice = payload.tool_choice
|
|
134
|
-
const hasActiveToolCall =
|
|
135
|
-
hasToolRole ||
|
|
136
|
-
(toolChoice !== undefined && toolChoice !== 'none' && toolChoice !== null)
|
|
137
|
-
const hasTools = hasActiveToolCall
|
|
138
|
-
|
|
139
|
-
const useNonThinkingMode =
|
|
140
|
-
!isAlwaysThinking && disableThinkingWithTools && hasTools
|
|
141
|
-
const rewrittenTemperature = useNonThinkingMode
|
|
142
|
-
? forcedNonThinkingTemperature
|
|
143
|
-
: forcedTemperature
|
|
144
|
-
|
|
145
|
-
// Capture incoming state before mutation
|
|
146
|
-
payload.__incomingThinkingType = incomingThinkingType
|
|
147
|
-
payload.__incomingReasoningEffort = incomingReasoningEffort
|
|
148
|
-
|
|
149
|
-
// Apply rewrites
|
|
150
|
-
payload.temperature = rewrittenTemperature
|
|
151
|
-
payload.top_p = forcedTopP
|
|
152
|
-
|
|
153
|
-
if (useNonThinkingMode) {
|
|
154
|
-
payload.thinking = { type: 'disabled' }
|
|
155
|
-
}
|
|
156
|
-
|
|
157
|
-
// K3 does not accept `thinking` — delete it if present so it doesn't cause a 400
|
|
158
|
-
if (isK3 && payload.thinking) {
|
|
159
|
-
delete payload.thinking
|
|
160
|
-
}
|
|
161
|
-
|
|
162
|
-
const rewrittenThinkingType = payload.thinking?.type
|
|
163
|
-
const rewriteInfo = {
|
|
164
|
-
model,
|
|
165
|
-
isK27,
|
|
166
|
-
isK3,
|
|
167
|
-
incomingTemperature,
|
|
168
|
-
rewrittenTemperature,
|
|
169
|
-
incomingTopP,
|
|
170
|
-
rewrittenTopP: forcedTopP,
|
|
171
|
-
incomingThinkingType,
|
|
172
|
-
rewrittenThinkingType,
|
|
173
|
-
incomingReasoningEffort
|
|
174
|
-
}
|
|
175
|
-
|
|
176
|
-
const summary = summarizePayload(payload, hasTools, rewriteInfo)
|
|
177
|
-
|
|
178
|
-
const modeTag = hasTools ? '[tools]' : '[chat]'
|
|
179
|
-
const k27Tag = isK27 ? '[k2.7]' : ''
|
|
180
|
-
const k3Tag = isK3 ? '[k3]' : ''
|
|
181
|
-
const consoleMsg = `${k27Tag}${k3Tag}${modeTag} temperature ${String(incomingTemperature)} -> ${String(rewrittenTemperature)}, top_p ${String(incomingTopP)} -> ${String(forcedTopP)}, thinking ${String(incomingThinkingType)} -> ${String(rewrittenThinkingType)}, reasoning_effort ${String(incomingReasoningEffort)}`
|
|
182
|
-
|
|
183
|
-
// Clean up internal keys before forwarding
|
|
184
|
-
delete payload.__incomingThinkingType
|
|
185
|
-
delete payload.__incomingReasoningEffort
|
|
186
|
-
|
|
187
|
-
return { summary, consoleMsg }
|
|
188
|
-
}
|
|
189
|
-
|
|
190
90
|
// ---- Create and start ----
|
|
191
91
|
|
|
192
92
|
const { start } = createProxy({
|
|
@@ -195,7 +95,13 @@ const { start } = createProxy({
|
|
|
195
95
|
logPath,
|
|
196
96
|
label: 'kimi-proxy',
|
|
197
97
|
healthCheckExtras: { forcedTemperature, forcedTopP },
|
|
198
|
-
rewriteRequest:
|
|
98
|
+
rewriteRequest: (payload) =>
|
|
99
|
+
rewriteKimi(payload, {
|
|
100
|
+
forcedTemperature,
|
|
101
|
+
forcedNonThinkingTemperature,
|
|
102
|
+
forcedTopP,
|
|
103
|
+
disableThinkingWithTools
|
|
104
|
+
}),
|
|
199
105
|
startupMessages: (_port, _upstreamUrl) => [
|
|
200
106
|
`[kimi-proxy] listening on http://127.0.0.1:${_port}/v1/chat/completions`,
|
|
201
107
|
`[kimi-proxy] forwarding to ${_upstreamUrl}`,
|
package/proxy/qwen-proxy.mjs
CHANGED
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
import 'dotenv/config'
|
|
3
3
|
import { fileURLToPath } from 'node:url'
|
|
4
4
|
import { createProxy } from '../lib/create-proxy.mjs'
|
|
5
|
+
import { rewriteQwen } from '../lib/qwen-rewrite.mjs'
|
|
5
6
|
|
|
6
7
|
/**
|
|
7
8
|
* Supported model scope for this proxy:
|
|
@@ -33,8 +34,9 @@ if (process.argv.includes('--help')) {
|
|
|
33
34
|
console.log(`Qwen proxy
|
|
34
35
|
|
|
35
36
|
Starts a local HTTP proxy that conditionally injects enable_thinking: false
|
|
36
|
-
when the request
|
|
37
|
-
|
|
37
|
+
when the request shows tool activity (a tool-role message or a non-default
|
|
38
|
+
tool_choice), letting Qwen hybrid-thinking models show reasoning in plain
|
|
39
|
+
chat while keeping tool loops stable.
|
|
38
40
|
|
|
39
41
|
The proxy forwards the requested model ID unchanged, including the DashScope
|
|
40
42
|
ID qwen3.8-max and OpenRouter ID qwen/qwen3.8-max-0902.
|
|
@@ -57,66 +59,6 @@ Suggested VS Code model URL:
|
|
|
57
59
|
process.exit(0)
|
|
58
60
|
}
|
|
59
61
|
|
|
60
|
-
// ---- Provider-specific rewrite logic ----
|
|
61
|
-
|
|
62
|
-
function summarizePayload(payload, hasTools, rewriteInfo) {
|
|
63
|
-
const messages = Array.isArray(payload.messages) ? payload.messages : []
|
|
64
|
-
const tools = Array.isArray(payload.tools) ? payload.tools : []
|
|
65
|
-
|
|
66
|
-
return {
|
|
67
|
-
model: payload.model,
|
|
68
|
-
stream: payload.stream,
|
|
69
|
-
hasTools,
|
|
70
|
-
toolCount: tools.length,
|
|
71
|
-
toolChoice: payload.tool_choice,
|
|
72
|
-
...rewriteInfo,
|
|
73
|
-
maxTokens:
|
|
74
|
-
payload.max_tokens ??
|
|
75
|
-
payload.max_completion_tokens ??
|
|
76
|
-
payload.max_output_tokens,
|
|
77
|
-
messageCount: messages.length,
|
|
78
|
-
messageRoles: messages.map((message) => message?.role).slice(0, 16),
|
|
79
|
-
topLevelKeys: Object.keys(payload).sort()
|
|
80
|
-
}
|
|
81
|
-
}
|
|
82
|
-
|
|
83
|
-
function rewriteQwen(payload) {
|
|
84
|
-
// Determine if a tool is actually being invoked:
|
|
85
|
-
// - tool_choice is set and not "none"
|
|
86
|
-
// - OR there is a "tool" role message in the conversation
|
|
87
|
-
const messages = Array.isArray(payload.messages) ? payload.messages : []
|
|
88
|
-
const hasToolRole = messages.some((message) => message?.role === 'tool')
|
|
89
|
-
const toolChoice = payload.tool_choice
|
|
90
|
-
const hasActiveToolCall =
|
|
91
|
-
hasToolRole ||
|
|
92
|
-
(toolChoice !== undefined && toolChoice !== 'none' && toolChoice !== null)
|
|
93
|
-
const hasTools = hasActiveToolCall
|
|
94
|
-
const incomingEnableThinking = payload.enable_thinking
|
|
95
|
-
|
|
96
|
-
if (disableThinkingWithTools && hasTools) {
|
|
97
|
-
// Tool-enabled request: suppress thinking to avoid reasoning_content issues
|
|
98
|
-
payload.enable_thinking = false
|
|
99
|
-
} else {
|
|
100
|
-
// Plain chat: remove enable_thinking so the model uses its default (true)
|
|
101
|
-
delete payload.enable_thinking
|
|
102
|
-
}
|
|
103
|
-
|
|
104
|
-
const rewrittenEnableThinking =
|
|
105
|
-
disableThinkingWithTools && hasTools ? false : undefined
|
|
106
|
-
|
|
107
|
-
const summary = summarizePayload(payload, hasTools, {
|
|
108
|
-
incomingEnableThinking,
|
|
109
|
-
rewrittenEnableThinking
|
|
110
|
-
})
|
|
111
|
-
|
|
112
|
-
const modeTag = hasTools ? '[tools]' : '[chat]'
|
|
113
|
-
const consoleMsg = `${modeTag} enable_thinking=${String(incomingEnableThinking)} -> ${
|
|
114
|
-
hasTools && disableThinkingWithTools ? 'false' : '<deleted>'
|
|
115
|
-
}, model=${payload.model ?? '?'}`
|
|
116
|
-
|
|
117
|
-
return { summary, consoleMsg }
|
|
118
|
-
}
|
|
119
|
-
|
|
120
62
|
// ---- Create and start ----
|
|
121
63
|
|
|
122
64
|
const { start } = createProxy({
|
|
@@ -125,7 +67,8 @@ const { start } = createProxy({
|
|
|
125
67
|
logPath,
|
|
126
68
|
label: 'qwen-proxy',
|
|
127
69
|
healthCheckExtras: { disableThinkingWithTools },
|
|
128
|
-
rewriteRequest:
|
|
70
|
+
rewriteRequest: (payload) =>
|
|
71
|
+
rewriteQwen(payload, { disableThinkingWithTools }),
|
|
129
72
|
startupMessages: (_port, _upstreamUrl) => [
|
|
130
73
|
`[qwen-proxy] listening on http://127.0.0.1:${_port}/v1/chat/completions`,
|
|
131
74
|
`[qwen-proxy] forwarding to ${_upstreamUrl}`,
|