llm-switcher 1.3.1 → 1.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/test.yml +31 -0
- package/CHANGELOG.md +7 -0
- package/README.md +13 -3
- package/README.vi.md +13 -3
- package/blindfold/make-certs.sh +2 -1
- package/package.json +1 -1
- package/proxy.mjs +94 -1
- package/tests/agy.e2e.test.mjs +110 -7
- package/tests/cdp.mjs +21 -5
- package/tests/dashboard.e2e.test.mjs +6 -2
- package/tests/gateway.e2e.test.mjs +2 -0
- package/tests/make-certs.test.mjs +14 -0
- package/tests/update.e2e.test.mjs +7 -1
- package/tests/ws-client.mjs +46 -0
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# The shims and the blindfold certificates have one code path per platform, and a test for a
|
|
2
|
+
# platform skips itself everywhere else. Only a run on each platform proves all of them.
|
|
3
|
+
name: test
|
|
4
|
+
|
|
5
|
+
on:
|
|
6
|
+
push:
|
|
7
|
+
branches: [main]
|
|
8
|
+
pull_request:
|
|
9
|
+
|
|
10
|
+
jobs:
|
|
11
|
+
test:
|
|
12
|
+
name: Node ${{ matrix.node }} on ${{ matrix.os }}
|
|
13
|
+
runs-on: ${{ matrix.os }}
|
|
14
|
+
# A local run takes about 2 minutes. A hang must fail the job, not hold it for 6 hours.
|
|
15
|
+
timeout-minutes: 20
|
|
16
|
+
strategy:
|
|
17
|
+
fail-fast: false
|
|
18
|
+
matrix:
|
|
19
|
+
os: [ubuntu-latest, macos-latest, windows-latest]
|
|
20
|
+
node: [22]
|
|
21
|
+
include:
|
|
22
|
+
# The lowest version that package.json accepts.
|
|
23
|
+
- os: ubuntu-latest
|
|
24
|
+
node: 18
|
|
25
|
+
steps:
|
|
26
|
+
- uses: actions/checkout@v7
|
|
27
|
+
- uses: actions/setup-node@v7
|
|
28
|
+
with:
|
|
29
|
+
node-version: ${{ matrix.node }}
|
|
30
|
+
package-manager-cache: false
|
|
31
|
+
- run: npm test
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,12 @@
|
|
|
1
1
|
# Changelog — LLM Switcher
|
|
2
2
|
|
|
3
|
+
## Release 1.3.2
|
|
4
|
+
|
|
5
|
+
- **Codex over WebSocket crosses Bifrost:** Codex uses its WebSocket transport by default. The gateway now sends each WebSocket turn to intact `/v1/responses` as one HTTP request with the whole conversation, and sends each event back as one frame. Before, these turns took the convert route. A stream that stops before `response.completed` ends the turn with `response.failed`.
|
|
6
|
+
- **All Codex clients cross Bifrost:** `codex exec` sends `codex_exec/` in its `User-Agent`, not `codex_cli_rs/`. The gateway takes `codex_cli_rs`, `codex_exec`, `codex_vscode` and `codex_sdk_ts` as one client. Before, `codex exec` never crossed Bifrost.
|
|
7
|
+
- **`make-certs.sh` and a Windows path:** The script read a lone argument such as `C:\Users\me\certs` as a host name and ignored it, so it rebuilt the certificates of the checkout instead. A running interceptor then served a leaf from the old CA, and Codex failed with `workspace routing discovery failed`. A backslash now marks a path.
|
|
8
|
+
- **Tests:** `npm test` no longer rebuilds the certificates of the checkout. The suite runs on Linux, macOS and Windows in CI, and on Node 18.
|
|
9
|
+
|
|
3
10
|
## Release 1.3.1
|
|
4
11
|
|
|
5
12
|
- **agy note:** `switch plugin status` now names agy 1.2.14. This release of agy also loads `~/.gemini/antigravity-cli/hooks.json` but does not run its hooks (measured 2026-10-03), so the shim toast is still the notice for agy.
|
package/README.md
CHANGED
|
@@ -337,6 +337,10 @@ gateway for its identity proof before it gives agy the variable. If the port doe
|
|
|
337
337
|
the proof of this install within 3 seconds, the shim removes the variable, prints one line, and agy
|
|
338
338
|
uses its official endpoint.
|
|
339
339
|
|
|
340
|
+
The shim does this check only when agy starts. If the gateway stops while agy runs (`switch off`, a
|
|
341
|
+
crash, an update), the port is free. Another account on this computer can then take the port and
|
|
342
|
+
receive the token. To close this gap, close agy before you stop the gateway.
|
|
343
|
+
|
|
340
344
|
### Blindfold mode (optional)
|
|
341
345
|
|
|
342
346
|
A base URL override makes Codex print one line on its own `/model` screen:
|
|
@@ -480,11 +484,17 @@ Bifrost has no setting. The gateway turns it on for each request when two condit
|
|
|
480
484
|
| Tool | `bifrost_ua` | Path on intact |
|
|
481
485
|
| --- | --- | --- |
|
|
482
486
|
| Claude Code | `claude-cli/` | `/v1/messages` |
|
|
483
|
-
| Codex (HTTP) | `codex_cli_rs/` | `/v1/responses` |
|
|
487
|
+
| Codex (HTTP and WebSocket) | `codex_cli_rs/` | `/v1/responses` |
|
|
484
488
|
| Antigravity CLI | `antigravity/cli/` | `/v1/v1internal:streamGenerateContent`, `/v1/v1internal:generateContent` |
|
|
485
489
|
|
|
486
|
-
For agy, intact also writes the project of the chosen account into the request.
|
|
487
|
-
|
|
490
|
+
For agy, intact also writes the project of the chosen account into the request.
|
|
491
|
+
|
|
492
|
+
Codex names its client in the `User-Agent`: `codex_cli_rs` for the terminal UI, `codex_exec` for
|
|
493
|
+
`codex exec`, `codex_vscode` and `codex_sdk_ts`. The gateway takes all four as the Codex client.
|
|
494
|
+
|
|
495
|
+
Codex over its WebSocket transport crosses Bifrost too. intact keeps no WebSocket state, so the
|
|
496
|
+
gateway sends each turn to `/v1/responses` as one HTTP request with the whole conversation. Each
|
|
497
|
+
event of the answer goes back to Codex as one WebSocket frame, unchanged.
|
|
488
498
|
|
|
489
499
|
The gateway keeps the answer from intact for 10 minutes for each model. Only a 2xx answer is kept that long. If intact does not answer, or answers with an error (a 404 too), the gateway uses the normal route. It asks again after 30 seconds (`LLM_SWITCHER_BIFROST_RETRY_MS`), and it prints one line for each series of failures.
|
|
490
500
|
|
package/README.vi.md
CHANGED
|
@@ -337,6 +337,10 @@ agy gửi token này dạng rõ tới `CLOUD_CODE_URL`. Vì vậy shim `agy` h
|
|
|
337
337
|
trước khi giao biến này cho agy. Nếu cổng không trả lời đúng bằng chứng của bản cài này trong 3 giây,
|
|
338
338
|
shim gỡ biến, in một dòng thông báo, và agy dùng endpoint chính thức.
|
|
339
339
|
|
|
340
|
+
Shim chỉ kiểm tra việc này lúc agy khởi động. Nếu gateway dừng trong lúc agy đang chạy (`switch off`,
|
|
341
|
+
crash, cập nhật), cổng bị bỏ trống. Khi đó một tài khoản khác trên máy có thể chiếm cổng và nhận token.
|
|
342
|
+
Để không có khoảng hở này, hãy đóng agy trước khi dừng gateway.
|
|
343
|
+
|
|
340
344
|
### Chế độ blindfold (tùy chọn)
|
|
341
345
|
|
|
342
346
|
Khi có override base URL, Codex in một dòng ngay trên màn `/model` của nó:
|
|
@@ -475,11 +479,17 @@ Bifrost không có cấu hình. Gateway tự bật Bifrost cho từng request kh
|
|
|
475
479
|
| Công cụ | `bifrost_ua` | Path trên intact |
|
|
476
480
|
| --- | --- | --- |
|
|
477
481
|
| Claude Code | `claude-cli/` | `/v1/messages` |
|
|
478
|
-
| Codex (HTTP) | `codex_cli_rs/` | `/v1/responses` |
|
|
482
|
+
| Codex (HTTP và WebSocket) | `codex_cli_rs/` | `/v1/responses` |
|
|
479
483
|
| Antigravity CLI | `antigravity/cli/` | `/v1/v1internal:streamGenerateContent`, `/v1/v1internal:generateContent` |
|
|
480
484
|
|
|
481
|
-
Với agy, intact ghi thêm project của tài khoản được chọn vào request.
|
|
482
|
-
|
|
485
|
+
Với agy, intact ghi thêm project của tài khoản được chọn vào request.
|
|
486
|
+
|
|
487
|
+
Codex ghi tên client của nó trong `User-Agent`: `codex_cli_rs` cho giao diện terminal, `codex_exec`
|
|
488
|
+
cho `codex exec`, `codex_vscode` và `codex_sdk_ts`. Gateway coi cả bốn là client Codex.
|
|
489
|
+
|
|
490
|
+
Codex qua WebSocket cũng đi Bifrost. intact không giữ trạng thái WebSocket, nên gateway gửi mỗi lượt
|
|
491
|
+
tới `/v1/responses` thành một request HTTP chứa toàn bộ hội thoại. Mỗi event của câu trả lời quay về
|
|
492
|
+
Codex thành một frame WebSocket, giữ nguyên.
|
|
483
493
|
|
|
484
494
|
Gateway giữ câu trả lời của intact 10 phút cho mỗi model. Chỉ câu trả lời 2xx được giữ lâu như vậy. Nếu intact không trả lời, hoặc trả lỗi (kể cả 404), gateway dùng đường thường. Gateway hỏi lại sau 30 giây (`LLM_SWITCHER_BIFROST_RETRY_MS`) và in một dòng cho mỗi chuỗi lỗi.
|
|
485
495
|
|
package/blindfold/make-certs.sh
CHANGED
|
@@ -35,10 +35,11 @@ elif [ -d "$ROOT/.git" ]; then DEFAULT_OUT="$ROOT/blindfold/certs"
|
|
|
35
35
|
else DEFAULT_OUT="$HOME/.llm-switcher/blindfold/certs"; fi
|
|
36
36
|
# Older commands read `make-certs.sh <host> <out-dir>`. The host list is fixed now, so the host is
|
|
37
37
|
# ignored; two arguments keep those commands working. A lone path-like argument is the directory.
|
|
38
|
+
# A Windows path (C:\...\certs) has no forward slash, so a backslash also marks a path.
|
|
38
39
|
OUT_ARG=""
|
|
39
40
|
if [ "$#" -ge 2 ]; then OUT_ARG="$2"
|
|
40
41
|
elif [ "$#" -eq 1 ]; then
|
|
41
|
-
case "$1" in
|
|
42
|
+
case "$1" in */*|*\\*|.*) OUT_ARG="$1" ;; *) : ;; esac
|
|
42
43
|
fi
|
|
43
44
|
OUT_DIR="${OUT_ARG:-$DEFAULT_OUT}"
|
|
44
45
|
CA_DAYS=3650
|
package/package.json
CHANGED
package/proxy.mjs
CHANGED
|
@@ -548,11 +548,17 @@ async function bifrostUAFor(profile, model) {
|
|
|
548
548
|
return ua;
|
|
549
549
|
}
|
|
550
550
|
|
|
551
|
+
// Codex names its originator in the User-Agent: codex_cli_rs (TUI), codex_exec (`codex exec`),
|
|
552
|
+
// codex_vscode, codex_sdk_ts. intact names one of them; each one is the same client.
|
|
553
|
+
const BIFROST_UA_FAMILIES = [['codex_cli_rs/', 'codex_exec/', 'codex_vscode/', 'codex_sdk_ts/']];
|
|
554
|
+
|
|
551
555
|
async function crossesBifrost(req, profile, model) {
|
|
552
556
|
const clientUA = String(req.headers['user-agent'] || '');
|
|
553
557
|
if (!clientUA) return false;
|
|
554
558
|
const ua = await bifrostUAFor(profile, model);
|
|
555
|
-
|
|
559
|
+
if (!ua) return false;
|
|
560
|
+
const family = BIFROST_UA_FAMILIES.find(f => f.includes(ua)) || [ua];
|
|
561
|
+
return family.some(prefix => clientUA.startsWith(prefix));
|
|
556
562
|
}
|
|
557
563
|
|
|
558
564
|
// Every client header except its own credentials, the switcher's control headers and hop-by-hop.
|
|
@@ -1848,6 +1854,88 @@ function geminiSafeTools(tools) {
|
|
|
1848
1854
|
});
|
|
1849
1855
|
}
|
|
1850
1856
|
|
|
1857
|
+
// Bifrost on the WS transport. intact speaks HTTP and keeps no WS state, so the turn goes to
|
|
1858
|
+
// /responses as one streamed request with the whole conversation, and each SSE event goes back to
|
|
1859
|
+
// Codex as one text frame, unchanged.
|
|
1860
|
+
async function wsBifrostTurn({ socket, payload, req, ac, history, input, profile, profileKey, mappedModel }) {
|
|
1861
|
+
const started = Date.now();
|
|
1862
|
+
const log = (extra) => logInspection({
|
|
1863
|
+
clientFormat: 'responses-ws', outFormat: 'bifrost', profile: profileKey, model: mappedModel, stream: true,
|
|
1864
|
+
requestPreview: '(bifrost)', duration: Date.now() - started, tokens: { prompt: 0, completion: 0 }, ...extra
|
|
1865
|
+
});
|
|
1866
|
+
const headers = bifrostHeaders(req, profile);
|
|
1867
|
+
for (const h of Object.keys(headers)) if (h === 'upgrade' || h.startsWith('sec-websocket-')) delete headers[h];
|
|
1868
|
+
headers['content-type'] = 'application/json';
|
|
1869
|
+
headers.accept = 'text/event-stream';
|
|
1870
|
+
// `type` names the WS frame (response.create); it is not a field of a Responses request.
|
|
1871
|
+
const { type: _type, ...body } = payload;
|
|
1872
|
+
|
|
1873
|
+
let upstreamRes;
|
|
1874
|
+
try {
|
|
1875
|
+
upstreamRes = await fetch(bifrostURL(profile, 'responses', req), {
|
|
1876
|
+
method: 'POST', headers, body: JSON.stringify({ ...body, model: mappedModel, stream: true }), signal: ac.signal
|
|
1877
|
+
});
|
|
1878
|
+
} catch (err) {
|
|
1879
|
+
if (ac.signal.aborted) return log({ status: 499, error: 'client disconnected' });
|
|
1880
|
+
sendWsFailed(socket, mappedModel, `Bifrost forward error: ${err.cause?.message || err.message}`, 502);
|
|
1881
|
+
return log({ status: 502, error: err.message });
|
|
1882
|
+
}
|
|
1883
|
+
if (!upstreamRes.ok) {
|
|
1884
|
+
const errText = await upstreamRes.text().catch(() => '');
|
|
1885
|
+
sendWsFailed(socket, mappedModel, extractUpstreamMessage(errText) || `Upstream HTTP ${upstreamRes.status}`, upstreamRes.status);
|
|
1886
|
+
return log({ status: upstreamRes.status, error: errText.slice(0, 300) });
|
|
1887
|
+
}
|
|
1888
|
+
|
|
1889
|
+
let completed = null;
|
|
1890
|
+
// With store:false the backend sends response.completed with an empty output; the items of the
|
|
1891
|
+
// turn come only in response.output_item.done, and the next turn must replay them.
|
|
1892
|
+
const items = [];
|
|
1893
|
+
let ended = false; // a response.completed or response.failed reached Codex
|
|
1894
|
+
let streamError = null;
|
|
1895
|
+
const decoder = new TextDecoder();
|
|
1896
|
+
let pending = '';
|
|
1897
|
+
const reader = upstreamRes.body.getReader();
|
|
1898
|
+
try {
|
|
1899
|
+
while (true) {
|
|
1900
|
+
const { done, value } = await reader.read();
|
|
1901
|
+
if (done) break;
|
|
1902
|
+
pending += decoder.decode(value, { stream: true }).replace(/\r\n/g, '\n');
|
|
1903
|
+
let cut;
|
|
1904
|
+
while ((cut = pending.indexOf('\n\n')) >= 0) {
|
|
1905
|
+
const data = pending.slice(0, cut).split('\n').filter(l => l.startsWith('data:')).map(l => l.slice(5).trimStart()).join('\n');
|
|
1906
|
+
pending = pending.slice(cut + 2);
|
|
1907
|
+
if (!data || data === '[DONE]') continue;
|
|
1908
|
+
let event;
|
|
1909
|
+
try { event = JSON.parse(data); } catch { continue; }
|
|
1910
|
+
if (event.type === 'response.output_item.done' && event.item) items.push(event.item);
|
|
1911
|
+
if (event.type === 'response.completed') completed = event.response;
|
|
1912
|
+
if (event.type === 'response.completed' || event.type === 'response.failed') ended = true;
|
|
1913
|
+
if (!socket.writable) continue;
|
|
1914
|
+
socket.write(encodeWsFrame(data));
|
|
1915
|
+
await drained(socket, ac.signal);
|
|
1916
|
+
}
|
|
1917
|
+
}
|
|
1918
|
+
} catch (err) {
|
|
1919
|
+
if (!ac.signal.aborted) streamError = `stream interrupted: ${err.cause?.message || err.message}`;
|
|
1920
|
+
}
|
|
1921
|
+
if (ac.signal.aborted) return log({ status: 499, error: 'client disconnected mid-stream' });
|
|
1922
|
+
// Codex ends a turn only on a terminal event: a stream cut before one must still end the turn.
|
|
1923
|
+
if (!ended) {
|
|
1924
|
+
streamError ||= 'the Bifrost stream ended before response.completed';
|
|
1925
|
+
sendWsFailed(socket, mappedModel, streamError, 502);
|
|
1926
|
+
}
|
|
1927
|
+
if (completed) {
|
|
1928
|
+
const output = Array.isArray(completed.output) && completed.output.length ? completed.output : items;
|
|
1929
|
+
rememberWsTurn(history, { ...completed, output }, input);
|
|
1930
|
+
}
|
|
1931
|
+
const usage = completed?.usage || {};
|
|
1932
|
+
log({
|
|
1933
|
+
status: streamError ? 502 : 200,
|
|
1934
|
+
tokens: { prompt: usage.input_tokens || 0, completion: usage.output_tokens || 0 },
|
|
1935
|
+
...(streamError ? { error: streamError } : {})
|
|
1936
|
+
});
|
|
1937
|
+
}
|
|
1938
|
+
|
|
1851
1939
|
// One turn of the Codex WS transport. The socket loop runs turns one at a time and owns `ac`.
|
|
1852
1940
|
async function handleWsResponseCreate(socket, payload, req, ac, history = null) {
|
|
1853
1941
|
const clientFormat = 'responses';
|
|
@@ -1877,6 +1965,11 @@ async function handleWsResponseCreate(socket, payload, req, ac, history = null)
|
|
|
1877
1965
|
const { previous_response_id: _prev, ...rest } = payload;
|
|
1878
1966
|
payload = { ...rest, input };
|
|
1879
1967
|
|
|
1968
|
+
const bifrostModel = mapModel(payload.model || '', profile, clientFormat);
|
|
1969
|
+
if (await crossesBifrost(req, profile, bifrostModel)) {
|
|
1970
|
+
return wsBifrostTurn({ socket, payload, req, ac, history, input, profile, profileKey, mappedModel: bifrostModel });
|
|
1971
|
+
}
|
|
1972
|
+
|
|
1880
1973
|
let ir;
|
|
1881
1974
|
try {
|
|
1882
1975
|
ir = parseToIR('responses', payload);
|
package/tests/agy.e2e.test.mjs
CHANGED
|
@@ -9,6 +9,7 @@ import os from 'node:os';
|
|
|
9
9
|
import path from 'node:path';
|
|
10
10
|
import { spawn } from 'node:child_process';
|
|
11
11
|
import { fileURLToPath } from 'node:url';
|
|
12
|
+
import { openResponsesWs } from './ws-client.mjs';
|
|
12
13
|
|
|
13
14
|
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
|
14
15
|
const AGY_UA = 'antigravity/cli/1.2.14 (aidev_client; os_type=windows; arch=amd64; cl=990662481; auth_method=consumer)';
|
|
@@ -70,6 +71,16 @@ function startUpstream() {
|
|
|
70
71
|
{ choices: [{ index: 0, delta: {}, finish_reason: 'stop' }] }
|
|
71
72
|
]);
|
|
72
73
|
}
|
|
74
|
+
if (req.method === 'GET' && req.url.startsWith('/cbroken/v1/models/')) {
|
|
75
|
+
res.writeHead(200, { 'Content-Type': 'application/json' });
|
|
76
|
+
return res.end(JSON.stringify({ bifrost_ua: 'codex_cli_rs/' }));
|
|
77
|
+
}
|
|
78
|
+
if (req.url.startsWith('/cbroken/v1/responses')) {
|
|
79
|
+
// The first event, then the connection dies: no response.completed ever comes.
|
|
80
|
+
res.writeHead(200, { 'Content-Type': 'text/event-stream' });
|
|
81
|
+
res.write(`event: response.created\ndata: ${JSON.stringify({ type: 'response.created', response: { id: 'resp_cut', status: 'in_progress' } })}\n\n`);
|
|
82
|
+
return setTimeout(() => res.socket.destroy(), 50);
|
|
83
|
+
}
|
|
73
84
|
if (req.method === 'GET' && req.url.startsWith('/broken/v1/models/')) {
|
|
74
85
|
res.writeHead(200, { 'Content-Type': 'application/json' });
|
|
75
86
|
return res.end(JSON.stringify({ bifrost_ua: 'antigravity/cli/' }));
|
|
@@ -127,8 +138,19 @@ function startUpstream() {
|
|
|
127
138
|
]);
|
|
128
139
|
}
|
|
129
140
|
if (req.url.startsWith('/intact/v1/responses')) {
|
|
141
|
+
// A Responses stream as the Codex account on intact sends it; each turn gets its own id.
|
|
142
|
+
// With store:false the real backend sends response.completed with an empty output: the
|
|
143
|
+
// items come only in response.output_item.done (measured 2026-10-03).
|
|
144
|
+
const id = `resp_${received.length}`;
|
|
145
|
+
const item = { type: 'message', id: `msg_${id}`, role: 'assistant', content: [{ type: 'output_text', text: 'codex pool' }] };
|
|
146
|
+
const events = [
|
|
147
|
+
{ type: 'response.created', response: { id, status: 'in_progress' } },
|
|
148
|
+
{ type: 'response.output_text.delta', delta: 'codex pool' },
|
|
149
|
+
{ type: 'response.output_item.done', output_index: 0, item },
|
|
150
|
+
{ type: 'response.completed', response: { id, status: 'completed', output: [], usage: { input_tokens: 5, output_tokens: 1 } } }
|
|
151
|
+
];
|
|
130
152
|
res.writeHead(200, { 'Content-Type': 'text/event-stream' });
|
|
131
|
-
res.write(`event:
|
|
153
|
+
for (const e of events) res.write(`event: ${e.type}\ndata: ${JSON.stringify(e)}\n\n`);
|
|
132
154
|
return res.end();
|
|
133
155
|
}
|
|
134
156
|
if (req.url.startsWith('/chat/v1/chat/completions')) {
|
|
@@ -170,6 +192,7 @@ before(async () => {
|
|
|
170
192
|
agyf: { name: 'agy flaky lookup', mode: 'convert', tool: 'agy', outFormat: 'openai-chat', baseURL: `${base}/flaky/v1`, apiKey: 'sk-flaky', defaultModels: { main: 'antigravity/*' } },
|
|
171
193
|
agynf: { name: 'agy lookup 404', mode: 'convert', tool: 'agy', outFormat: 'openai-chat', baseURL: `${base}/nf/v1`, apiKey: 'sk-nf', defaultModels: { main: 'antigravity/*' } },
|
|
172
194
|
agyg: { name: 'agy broken stream', mode: 'convert', tool: 'agy', outFormat: 'openai-chat', baseURL: `${base}/broken/v1`, apiKey: 'sk-broken', defaultModels: { main: 'antigravity/*' } },
|
|
195
|
+
cdxcut: { name: 'codex cut stream', mode: 'convert', tool: 'codex', outFormat: 'openai-chat', baseURL: `${base}/cbroken/v1`, apiKey: 'sk-cut', defaultModels: { main: 'codex/gpt-5.5' } },
|
|
173
196
|
cdx: { name: 'codex on intact', mode: 'convert', tool: 'codex', outFormat: 'openai-chat', baseURL: `${base}/intact/v1`, apiKey: 'sk-intact', defaultModels: { main: 'codex/gpt-5.5' } }
|
|
174
197
|
}
|
|
175
198
|
};
|
|
@@ -237,6 +260,19 @@ const envelope = (over = {}) => ({
|
|
|
237
260
|
},
|
|
238
261
|
...over
|
|
239
262
|
});
|
|
263
|
+
// Whether the gateway ended its answer as a whole one. Read from the HTTP framing, because fetch in
|
|
264
|
+
// Node 18 takes a chunked body that a closed socket cut short as complete.
|
|
265
|
+
function answerIsComplete(target, { method = 'POST', headers = {}, body = '' } = {}) {
|
|
266
|
+
return new Promise((resolve, reject) => {
|
|
267
|
+
const req = http.request(target, { method, headers: { ...headers, 'content-length': Buffer.byteLength(body) } }, (res) => {
|
|
268
|
+
res.on('error', () => {});
|
|
269
|
+
res.on('close', () => resolve({ status: res.statusCode, complete: res.complete }));
|
|
270
|
+
res.resume();
|
|
271
|
+
});
|
|
272
|
+
req.on('error', reject);
|
|
273
|
+
req.end(body);
|
|
274
|
+
});
|
|
275
|
+
}
|
|
240
276
|
const lastTo = (prefix) => [...received].reverse().find(r => r.url.startsWith(prefix));
|
|
241
277
|
const dataLines = (text) => text.split('\n').filter(l => l.startsWith('data:')).map(l => JSON.parse(l.slice(5)));
|
|
242
278
|
|
|
@@ -383,16 +419,16 @@ test('a Code Assist host that never answers gives a 504 after the idle timeout',
|
|
|
383
419
|
|
|
384
420
|
test('an upstream that stops in the middle of a body ends the answer to agy with an error', async () => {
|
|
385
421
|
const started = Date.now();
|
|
386
|
-
const r = await
|
|
387
|
-
|
|
422
|
+
const r = await answerIsComplete(url('/v1internal:breakMidway'), { headers: agyHeaders(), body: '{}' });
|
|
423
|
+
assert.equal(r.complete, false);
|
|
388
424
|
assert.ok(Date.now() - started < 10000);
|
|
389
425
|
});
|
|
390
426
|
|
|
391
427
|
test('an upstream that stalls after its headers is cut by the idle timeout', async () => {
|
|
392
428
|
const port = await startGateway({ LLM_SWITCHER_CODE_ASSIST_URL: `${base}/google`, LLM_SWITCHER_CODE_ASSIST_TIMEOUT_MS: '1000' });
|
|
393
429
|
const started = Date.now();
|
|
394
|
-
const r = await
|
|
395
|
-
|
|
430
|
+
const r = await answerIsComplete(`http://127.0.0.1:${port}/v1internal:stallMidway`, { headers: agyHeaders(), body: '{}' });
|
|
431
|
+
assert.equal(r.complete, false);
|
|
396
432
|
assert.ok(Date.now() - started < 5000, `took ${Date.now() - started} ms`);
|
|
397
433
|
});
|
|
398
434
|
|
|
@@ -515,9 +551,76 @@ test('an agent turn that names a profile that does not exist gets a 400, never G
|
|
|
515
551
|
});
|
|
516
552
|
|
|
517
553
|
test('a Bifrost stream that breaks ends the answer to agy with an error, not a clean end', async () => {
|
|
518
|
-
const r = await
|
|
554
|
+
const r = await answerIsComplete(url('/v1internal:streamGenerateContent?alt=sse'), { headers: agyHeaders({ 'x-llm-profile': 'agyg' }), body: JSON.stringify(envelope()) });
|
|
519
555
|
assert.equal(r.status, 200);
|
|
520
|
-
|
|
556
|
+
assert.equal(r.complete, false);
|
|
557
|
+
});
|
|
558
|
+
|
|
559
|
+
test('Codex on the WebSocket transport crosses Bifrost too, and its turn history reaches intact', async () => {
|
|
560
|
+
const ws = await openResponsesWs(proxyPort, { 'User-Agent': CODEX_UA, 'x-llm-profile': 'cdx', Authorization: 'Bearer chatgpt-user-token' });
|
|
561
|
+
try {
|
|
562
|
+
assert.match(ws.reply, /^HTTP\/1\.1 101/);
|
|
563
|
+
const before = received.length;
|
|
564
|
+
const turn1 = [{ type: 'message', role: 'user', content: [{ type: 'input_text', text: 'hi' }] }];
|
|
565
|
+
ws.send({ type: 'response.create', model: 'gpt-5.5', instructions: 'x', input: turn1, store: false });
|
|
566
|
+
const done = async (n) => {
|
|
567
|
+
for (let i = 0; i < 100 && ws.messages.filter(m => m.type === 'response.completed').length < n; i++) await new Promise(r => setTimeout(r, 30));
|
|
568
|
+
return ws.messages.filter(m => m.type === 'response.completed')[n - 1];
|
|
569
|
+
};
|
|
570
|
+
const first = await done(1);
|
|
571
|
+
assert.ok(first, `no response.completed: ${JSON.stringify(ws.messages).slice(0, 300)}`);
|
|
572
|
+
assert.ok(ws.messages.some(m => m.type === 'response.output_text.delta' && m.delta === 'codex pool'), 'the events of intact reach Codex');
|
|
573
|
+
|
|
574
|
+
const up1 = received.slice(before).find(x => x.url.startsWith('/intact/v1/responses'));
|
|
575
|
+
assert.ok(up1, 'the WS turn must reach intact /responses, not the convert route');
|
|
576
|
+
assert.equal(up1.method, 'POST');
|
|
577
|
+
assert.equal(up1.headers['x-api-key'], 'sk-intact');
|
|
578
|
+
assert.equal(up1.headers.authorization, undefined, 'the ChatGPT token never reaches intact');
|
|
579
|
+
assert.equal(up1.headers['user-agent'], CODEX_UA);
|
|
580
|
+
for (const h of ['upgrade', 'sec-websocket-key', 'sec-websocket-version', 'x-llm-profile']) assert.equal(up1.headers[h], undefined, `${h} is a WebSocket or switcher header`);
|
|
581
|
+
assert.equal(up1.body.model, 'codex/gpt-5.5');
|
|
582
|
+
assert.equal(up1.body.stream, true);
|
|
583
|
+
assert.equal(up1.body.type, undefined, 'the WS frame type is not part of a Responses request');
|
|
584
|
+
assert.deepEqual(up1.body.input, turn1);
|
|
585
|
+
|
|
586
|
+
// Turn 2 names turn 1; intact has no WS state, so it gets the whole conversation.
|
|
587
|
+
const turn2 = [{ type: 'message', role: 'user', content: [{ type: 'input_text', text: 'again' }] }];
|
|
588
|
+
ws.send({ type: 'response.create', model: 'gpt-5.5', previous_response_id: first.response.id, input: turn2, store: false });
|
|
589
|
+
assert.ok(await done(2), 'no second response.completed');
|
|
590
|
+
const up2 = received.slice(before).filter(x => x.url.startsWith('/intact/v1/responses'))[1];
|
|
591
|
+
const answer1 = ws.messages.find(m => m.type === 'response.output_item.done').item;
|
|
592
|
+
assert.deepEqual(up2.body.input, [...turn1, answer1, ...turn2], 'the answer of turn 1 is replayed, though response.completed had no output');
|
|
593
|
+
assert.equal(up2.body.previous_response_id, undefined);
|
|
594
|
+
|
|
595
|
+
const token = fs.readFileSync(path.join(tmpDir, 'admin.token'), 'utf8').trim();
|
|
596
|
+
const { logs } = await (await fetch(url('/api/logs'), { headers: { 'x-llm-switcher-token': token } })).json();
|
|
597
|
+
const entry = logs.find(l => l.clientFormat === 'responses-ws' && l.outFormat === 'bifrost');
|
|
598
|
+
assert.ok(entry, 'the inspector names the WS Bifrost route');
|
|
599
|
+
assert.deepEqual(entry.tokens, { prompt: 5, completion: 1 });
|
|
600
|
+
} finally {
|
|
601
|
+
ws.close();
|
|
602
|
+
}
|
|
603
|
+
});
|
|
604
|
+
|
|
605
|
+
test('codex exec (originator codex_exec) crosses Bifrost like the Codex TUI', async () => {
|
|
606
|
+
const before = received.length;
|
|
607
|
+
const body = { model: 'gpt-5.5', instructions: 'x', input: [{ role: 'user', content: [{ type: 'input_text', text: 'hi' }] }], stream: true, store: false };
|
|
608
|
+
const r = await fetch(url('/v1/responses'), { method: 'POST', headers: { 'content-type': 'application/json', 'user-agent': 'codex_exec/0.160.0 (Windows 10.0.26300; x86_64) unknown', 'x-llm-profile': 'cdx' }, body: JSON.stringify(body) });
|
|
609
|
+
await r.text();
|
|
610
|
+
assert.ok(received.slice(before).some(x => x.url.startsWith('/intact/v1/responses')), 'codex_exec must reach intact /responses');
|
|
611
|
+
assert.equal(received.slice(before).some(x => x.url.startsWith('/intact/v1/chat/completions')), false, 'not the convert route');
|
|
612
|
+
});
|
|
613
|
+
|
|
614
|
+
test('a Codex WS Bifrost stream cut before its end still ends the turn with response.failed', async () => {
|
|
615
|
+
const ws = await openResponsesWs(proxyPort, { 'User-Agent': CODEX_UA, 'x-llm-profile': 'cdxcut' });
|
|
616
|
+
try {
|
|
617
|
+
ws.send({ type: 'response.create', model: 'gpt-5.5', input: [{ type: 'message', role: 'user', content: [{ type: 'input_text', text: 'hi' }] }] });
|
|
618
|
+
for (let i = 0; i < 150 && !ws.messages.some(m => m.type === 'response.failed'); i++) await new Promise(r => setTimeout(r, 30));
|
|
619
|
+
assert.ok(ws.messages.some(m => m.type === 'response.created' && m.response.id === 'resp_cut'), 'the events before the cut reach Codex');
|
|
620
|
+
assert.ok(ws.messages.some(m => m.type === 'response.failed'), `no terminal event: ${JSON.stringify(ws.messages).slice(0, 300)}`);
|
|
621
|
+
} finally {
|
|
622
|
+
ws.close();
|
|
623
|
+
}
|
|
521
624
|
});
|
|
522
625
|
|
|
523
626
|
test('with agy switched off, its agent turns go back to Google', async (t) => {
|
package/tests/cdp.mjs
CHANGED
|
@@ -41,16 +41,31 @@ export async function launchBrowser() {
|
|
|
41
41
|
'--headless=new', '--remote-debugging-port=0', `--user-data-dir=${dir}`,
|
|
42
42
|
'--no-first-run', '--no-default-browser-check', '--disable-extensions', '--disable-gpu',
|
|
43
43
|
'--disable-background-networking', '--window-size=1280,900',
|
|
44
|
-
|
|
44
|
+
// macOS: no keychain prompt, which would block a headless run with no one to answer it.
|
|
45
|
+
'--use-mock-keychain', '--password-store=basic',
|
|
46
|
+
// Ubuntu 24.04 blocks the user namespaces the Chrome sandbox needs. The page is a local test.
|
|
47
|
+
...(process.platform === 'linux' || process.getuid?.() === 0 ? ['--no-sandbox'] : []),
|
|
45
48
|
'about:blank'
|
|
46
49
|
];
|
|
47
|
-
const child = spawn(findChromium(), args, { stdio: 'ignore', env: { ...process.env, HOME: dir } });
|
|
50
|
+
const child = spawn(findChromium(), args, { stdio: ['ignore', 'ignore', 'pipe'], env: { ...process.env, HOME: dir } });
|
|
51
|
+
// The tail of what Chromium printed: the only clue when it starts on a machine no one can watch.
|
|
52
|
+
let stderr = '';
|
|
53
|
+
child.stderr.on('data', (d) => { stderr = (stderr + d).slice(-4000); });
|
|
54
|
+
let exited = null;
|
|
55
|
+
child.once('exit', (code, signal) => { exited = signal || code; });
|
|
48
56
|
let port = 0;
|
|
49
|
-
for (let i = 0; i <
|
|
57
|
+
for (let i = 0; i < 300 && !port && exited === null; i++) {
|
|
50
58
|
try { port = Number(fs.readFileSync(path.join(dir, 'DevToolsActivePort'), 'utf8').split('\n')[0]) || 0; } catch { /* not written yet */ }
|
|
51
59
|
if (!port) await sleep(100);
|
|
52
60
|
}
|
|
53
|
-
if (!port) {
|
|
61
|
+
if (!port) {
|
|
62
|
+
// SIGKILL and a wait: a Chromium left running holds this test process open.
|
|
63
|
+
if (exited === null) {
|
|
64
|
+
child.kill('SIGKILL');
|
|
65
|
+
await new Promise(r => { child.once('exit', r); setTimeout(r, 3000); });
|
|
66
|
+
}
|
|
67
|
+
throw new Error(`Chromium did not open a debugging port (exit: ${exited ?? 'killed'})\n${stderr}`);
|
|
68
|
+
}
|
|
54
69
|
|
|
55
70
|
const pages = [];
|
|
56
71
|
return {
|
|
@@ -65,7 +80,8 @@ export async function launchBrowser() {
|
|
|
65
80
|
for (const p of pages) p.close();
|
|
66
81
|
child.kill();
|
|
67
82
|
await new Promise(r => { child.once('exit', r); setTimeout(r, 2000); });
|
|
68
|
-
|
|
83
|
+
// On Windows a Chromium child process can hold a cache file for a moment after the exit (EBUSY).
|
|
84
|
+
fs.rmSync(dir, { recursive: true, force: true, maxRetries: 10, retryDelay: 200 });
|
|
69
85
|
}
|
|
70
86
|
};
|
|
71
87
|
}
|
|
@@ -23,8 +23,12 @@ describe('dashboard, driven in a real browser', { skip: skipReason() }, () => {
|
|
|
23
23
|
});
|
|
24
24
|
|
|
25
25
|
after(async () => {
|
|
26
|
-
|
|
27
|
-
|
|
26
|
+
// The gateway must stop even when the browser cleanup throws, or it holds this process open.
|
|
27
|
+
try {
|
|
28
|
+
await browser?.close();
|
|
29
|
+
} finally {
|
|
30
|
+
await fx?.stop();
|
|
31
|
+
}
|
|
28
32
|
});
|
|
29
33
|
|
|
30
34
|
beforeEach(async () => {
|
|
@@ -1149,6 +1149,8 @@ test('Direct passthrough waits for a slow client instead of buffering the whole
|
|
|
1149
1149
|
req.on('error', reject);
|
|
1150
1150
|
req.end(JSON.stringify({ model: 'claude-opus-4-6', max_tokens: 10, stream: true, messages: [{ role: 'user', content: 'DIRECT_BIG' }] }));
|
|
1151
1151
|
});
|
|
1152
|
+
// The end callback of the mock can run after the client's 'end' under load: 0 means not recorded yet.
|
|
1153
|
+
assert.ok(await until(() => bigState.finishedAt, 2000), 'the upstream never finished');
|
|
1152
1154
|
assert.ok(bigState.finishedAt >= resumedAt, `upstream finished ${resumedAt - bigState.finishedAt} ms before the client read anything`);
|
|
1153
1155
|
});
|
|
1154
1156
|
|
|
@@ -20,3 +20,17 @@ test('make-certs.sh builds the certificates under a path that contains a dot', {
|
|
|
20
20
|
for (const f of ['ca.pem', 'ca.key', 'leaf.pem', 'leaf.key']) assert.ok(fs.existsSync(path.join(out, f)), `${f} is missing`);
|
|
21
21
|
assert.deepEqual(fs.readdirSync(dir), ['first.last'], 'no serial file outside the output directory');
|
|
22
22
|
});
|
|
23
|
+
|
|
24
|
+
// A Windows path has no forward slash. The script took `C:\...\certs` for the host of the old
|
|
25
|
+
// two-argument form, ignored it, and rebuilt the default certificates of the checkout instead.
|
|
26
|
+
test('make-certs.sh writes to a Windows path given as its only argument', { skip: process.platform !== 'win32' && 'a Windows path' }, (t) => {
|
|
27
|
+
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'llmswcerts-win-'));
|
|
28
|
+
t.after(() => fs.rmSync(dir, { recursive: true, force: true }));
|
|
29
|
+
const out = path.join(dir, 'certs');
|
|
30
|
+
// Where the script falls back to: a decoy, so a failing run never touches the real certificates.
|
|
31
|
+
const decoy = path.join(dir, 'decoy');
|
|
32
|
+
execFileSync('bash', [path.join(ROOT, 'blindfold', 'make-certs.sh'), out], { stdio: 'pipe', env: { ...process.env, LLM_SWITCHER_BLINDFOLD_CERTS: decoy } });
|
|
33
|
+
assert.match(out, /^[A-Za-z]:\\/, 'the argument is a native Windows path');
|
|
34
|
+
for (const f of ['ca.pem', 'ca.key', 'leaf.pem', 'leaf.key']) assert.ok(fs.existsSync(path.join(out, f)), `${f} is missing in ${out}`);
|
|
35
|
+
assert.equal(fs.existsSync(decoy), false, 'nothing is written to the default directory');
|
|
36
|
+
});
|
|
@@ -192,6 +192,12 @@ test('the dashboard Update now button installs the release and reloads on the ne
|
|
|
192
192
|
await page.goto(fx.dashboardUrl);
|
|
193
193
|
await page.waitFor(`!document.getElementById('update-notice').hidden`, 'the update notice');
|
|
194
194
|
await page.click('#update-now');
|
|
195
|
-
|
|
195
|
+
try {
|
|
196
|
+
await page.waitFor(`document.getElementById('app-version')?.textContent === 'v9.9.12'`, 'the reloaded dashboard on the new version', 30000);
|
|
197
|
+
} catch (err) {
|
|
198
|
+
// A CI runner cannot be watched: put what the gateway and the page saw into the failure.
|
|
199
|
+
err.message += `\ngateway log:\n${fx.log()}\npage problems: ${JSON.stringify(page.problems)}`;
|
|
200
|
+
throw err;
|
|
201
|
+
}
|
|
196
202
|
assert.equal(await page.evaluate(`document.getElementById('update-notice').hidden`), true);
|
|
197
203
|
});
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
// A raw WebSocket client for the gateway tests. It sends the handshake headers Codex sends (a
|
|
2
|
+
// User-Agent, a profile header), which the WebSocket of Node cannot set.
|
|
3
|
+
import net from 'node:net';
|
|
4
|
+
import crypto from 'node:crypto';
|
|
5
|
+
import { createFrameReader } from '../blindfold/wsframe.mjs';
|
|
6
|
+
|
|
7
|
+
// One masked client frame, as RFC 6455 requires from a client.
|
|
8
|
+
export function clientFrame(opcode, payload) {
|
|
9
|
+
const data = Buffer.from(payload);
|
|
10
|
+
const len = data.length;
|
|
11
|
+
const head = len < 126 ? Buffer.from([0x80 | opcode, 0x80 | len])
|
|
12
|
+
: len < 65536 ? Buffer.from([0x80 | opcode, 0x80 | 126, len >> 8, len & 0xff])
|
|
13
|
+
: Buffer.concat([Buffer.from([0x80 | opcode, 0x80 | 127]), (() => { const b = Buffer.alloc(8); b.writeBigUInt64BE(BigInt(len)); return b; })()]);
|
|
14
|
+
const mask = crypto.randomBytes(4);
|
|
15
|
+
return Buffer.concat([head, mask, Buffer.from(data.map((b, i) => b ^ mask[i & 3]))]);
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
// Opens /v1/responses on the gateway. `messages` collects every JSON text frame the gateway sends.
|
|
19
|
+
export function openResponsesWs(port, headers = {}) {
|
|
20
|
+
return new Promise((resolve, reject) => {
|
|
21
|
+
const socket = net.connect(port, '127.0.0.1');
|
|
22
|
+
const extra = Object.entries(headers).map(([k, v]) => `${k}: ${v}\r\n`).join('');
|
|
23
|
+
socket.write(`GET /v1/responses HTTP/1.1\r\nHost: 127.0.0.1:${port}\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Version: 13\r\nSec-WebSocket-Key: ${crypto.randomBytes(16).toString('base64')}\r\n${extra}\r\n`);
|
|
24
|
+
let head = Buffer.alloc(0);
|
|
25
|
+
const read = createFrameReader();
|
|
26
|
+
const messages = [];
|
|
27
|
+
const onData = (chunk) => {
|
|
28
|
+
head = Buffer.concat([head, chunk]);
|
|
29
|
+
const end = head.indexOf('\r\n\r\n');
|
|
30
|
+
if (end < 0) return;
|
|
31
|
+
socket.off('data', onData);
|
|
32
|
+
const collect = (c) => { for (const f of read(c)) if (f.type === 'text') messages.push(JSON.parse(f.payload.toString())); };
|
|
33
|
+
socket.on('data', collect);
|
|
34
|
+
const rest = head.subarray(end + 4);
|
|
35
|
+
if (rest.length) collect(rest);
|
|
36
|
+
resolve({
|
|
37
|
+
reply: head.subarray(0, end).toString(),
|
|
38
|
+
messages,
|
|
39
|
+
send: (obj) => socket.write(clientFrame(1, JSON.stringify(obj))),
|
|
40
|
+
close: () => socket.destroy()
|
|
41
|
+
});
|
|
42
|
+
};
|
|
43
|
+
socket.on('data', onData);
|
|
44
|
+
socket.on('error', reject);
|
|
45
|
+
});
|
|
46
|
+
}
|