@gemus/mcp-proxy 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +56 -0
- package/package.json +42 -0
- package/src/__tests__/backfill.test.ts +225 -0
- package/src/__tests__/route.integration.test.ts +68 -0
- package/src/backfill.mjs +195 -0
- package/src/proxy.mjs +226 -0
package/README.md
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# @gemus/mcp-proxy (Issue #1751, P1 — WIP)
|
|
2
|
+
|
|
3
|
+
Local stdio↔HTTP MCP proxy that gives **Codex desktop (direct-connect / Mode B)** users canvas
|
|
4
|
+
image write-back. Bundled into the Codex plugin (`.codex-plugin/`), declared as a **command-based**
|
|
5
|
+
MCP server so codex spawns it locally; the proxy owns upstream auth to the remote gemus `/api/mcp`
|
|
6
|
+
(Codex does no OAuth for stdio servers). Full design: `docs/plans/1751-codex-desktop-companion.md`
|
|
7
|
+
and `docs/architecture/agent architecture/codex-desktop-companion.md`.
|
|
8
|
+
|
|
9
|
+
## Status: minimal skeleton, behavioral checkpoint PASSED
|
|
10
|
+
|
|
11
|
+
`src/proxy.mjs` is the validated passthrough+tap skeleton. The backfill delivery layer (Stop-hook
|
|
12
|
+
endpoint → read rollout via `transcript_path` → `ExecuteClaimTracker` → key-only `deliverImageToNode`)
|
|
13
|
+
is the next build increment (plan Steps 2–4).
|
|
14
|
+
|
|
15
|
+
## What was validated end-to-end (2026-07-06, codex-cli 0.142.2 → local dev `/api/mcp`)
|
|
16
|
+
|
|
17
|
+
- **Transparent passthrough** — initialize + tools/list relay all **12 gemus tools** (incl.
|
|
18
|
+
`execute`/`batch_execute`). SDK transport auto-captures `mcp-session-id` from the initialize
|
|
19
|
+
response; raw send/onmessage wiring works with normal sequencing.
|
|
20
|
+
- **(e) proxy-per-session** — `codex mcp add gemus -- node <proxy>` → codex spawns one proxy per
|
|
21
|
+
session (`clientInfo: codex-mcp-client 0.142.2`); reports `Auth: Unsupported` (proxy owns auth).
|
|
22
|
+
- **(d) execute-tap** — real codex `tools/call` (`workflow_list`, `canvas_read{workflowId}`) tapped;
|
|
23
|
+
`workflowId` from `arguments`, and **turn/thread identity from `_meta.x-codex-turn-metadata`
|
|
24
|
+
(`thread_id`/`turn_id`)** — so per-turn scoping needs no Stop hook. `execute` is the same path.
|
|
25
|
+
- **(c) turn-end + retrieval** — `Stop` hook fires post-flush carrying `transcript_path` → exact
|
|
26
|
+
rollout; imagegen writes `generated_images/<session>/ig_<call_id>.png` + rollout `saved_path`+base64.
|
|
27
|
+
- **Session cap** — `KEY_ONLY_SESSION_CAP=3`/user is real; the proxy MUST `DELETE` its upstream
|
|
28
|
+
session on close (implemented in `shutdown()`) to avoid leaking slots.
|
|
29
|
+
|
|
30
|
+
## Install (Codex desktop)
|
|
31
|
+
|
|
32
|
+
The key is supplied as a **literal** in the user's *local* codex config — codex does NOT interpolate
|
|
33
|
+
`${VAR}` in an MCP server's `env`, and a public plugin manifest must never carry a key (Step-0.4
|
|
34
|
+
findings). So the keyed MCP server is registered locally, not via the plugin manifest:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
# 1. Register the proxy as codex's gemus MCP server (writes a literal key into ~/.codex/config.toml).
|
|
38
|
+
codex mcp add gemus --env GEMUS_KEY=mak_<your key> -- npx -y @gemus/mcp-proxy
|
|
39
|
+
# (get a mak_ key from gemus.ai → Settings → MCP Keys; GEMUS_URL defaults to https://gemus.ai/api/mcp)
|
|
40
|
+
|
|
41
|
+
# 2. Install the plugin (Stop hook + skills) from the gemus marketplace, then TRUST the hook:
|
|
42
|
+
codex plugin add gemus@<marketplace>
|
|
43
|
+
# In interactive codex run `/hooks` and trust the gemus Stop hook — REQUIRED for canvas backfill.
|
|
44
|
+
# Without a trusted hook, images still land on the canvas (orphan node), just not the planned node.
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
## Local dev
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
pnpm --filter @gemus/codex-backfill-core build # proxy imports the built core at runtime
|
|
51
|
+
GEMUS_KEY=mak_... GEMUS_URL=http://localhost:3000/api/mcp PROXY_LOG=/tmp/proxy.log node src/proxy.mjs
|
|
52
|
+
# or register against dev: codex mcp add gemus --env GEMUS_KEY=mak_... GEMUS_URL=http://localhost:3000/api/mcp -- node <abs>/src/proxy.mjs
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Env: `GEMUS_KEY` (required), `GEMUS_URL` (default `https://gemus.ai/api/mcp`), `PROXY_LOG` (optional
|
|
56
|
+
debug log), `PROXY_BACKFILL_IDLE_MS` (default 30000 — turn-idle fallback before salvage-orphan).
|
package/package.json
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@gemus/mcp-proxy",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "Local stdio<->HTTP MCP proxy for Codex desktop: transparent passthrough + execute tap + imagegen backfill (Issue #1751, P1).",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"bin": {
|
|
7
|
+
"gemus-mcp-proxy": "src/proxy.mjs"
|
|
8
|
+
},
|
|
9
|
+
"main": "src/proxy.mjs",
|
|
10
|
+
"repository": {
|
|
11
|
+
"type": "git",
|
|
12
|
+
"url": "https://github.com/Gemus-AI/Gemus.git",
|
|
13
|
+
"directory": "packages/gemus-mcp-proxy"
|
|
14
|
+
},
|
|
15
|
+
"homepage": "https://gemus.ai",
|
|
16
|
+
"keywords": [
|
|
17
|
+
"gemus",
|
|
18
|
+
"mcp",
|
|
19
|
+
"codex",
|
|
20
|
+
"proxy",
|
|
21
|
+
"image-generation"
|
|
22
|
+
],
|
|
23
|
+
"publishConfig": {
|
|
24
|
+
"access": "public"
|
|
25
|
+
},
|
|
26
|
+
"dependencies": {
|
|
27
|
+
"@modelcontextprotocol/sdk": "^1.29.0",
|
|
28
|
+
"@gemus/codex-backfill-core": "0.1.0"
|
|
29
|
+
},
|
|
30
|
+
"engines": {
|
|
31
|
+
"node": ">=18"
|
|
32
|
+
},
|
|
33
|
+
"files": [
|
|
34
|
+
"src/",
|
|
35
|
+
"README.md"
|
|
36
|
+
],
|
|
37
|
+
"license": "MIT",
|
|
38
|
+
"private": false,
|
|
39
|
+
"scripts": {
|
|
40
|
+
"start": "node src/proxy.mjs"
|
|
41
|
+
}
|
|
42
|
+
}
|
|
@@ -0,0 +1,225 @@
|
|
|
1
|
+
import { describe, it, expect, beforeEach, vi } from 'vitest'
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* @gemus/mcp-proxy backfill orchestration (#1751, Mode B). Mocks only the core's NETWORK + rollout-IO
|
|
5
|
+
* (deliverImageToNode / addImageToCanvas / readRollout / findRolloutFile); keeps extractImageOutputs +
|
|
6
|
+
* ExecuteClaimTracker + unwrapMcpContent REAL so the tap→tracker→claim/salvage wiring is exercised
|
|
7
|
+
* end-to-end. Shares the same @modelcontextprotocol/sdk instance as bridge tests via the vitest alias.
|
|
8
|
+
*/
|
|
9
|
+
const mocks = vi.hoisted(() => ({
|
|
10
|
+
deliverImageToNode: vi.fn(),
|
|
11
|
+
addImageToCanvas: vi.fn(),
|
|
12
|
+
readRollout: vi.fn(),
|
|
13
|
+
findRolloutFile: vi.fn(),
|
|
14
|
+
}))
|
|
15
|
+
|
|
16
|
+
vi.mock('@gemus/codex-backfill-core', async (importOriginal) => {
|
|
17
|
+
const actual = await importOriginal<typeof import('@gemus/codex-backfill-core')>()
|
|
18
|
+
return {
|
|
19
|
+
...actual, // real extractImageOutputs / ExecuteClaimTracker / EXECUTE_TOOL_NAMES / unwrapMcpContent
|
|
20
|
+
deliverImageToNode: mocks.deliverImageToNode,
|
|
21
|
+
addImageToCanvas: mocks.addImageToCanvas,
|
|
22
|
+
readRollout: mocks.readRollout,
|
|
23
|
+
findRolloutFile: mocks.findRolloutFile,
|
|
24
|
+
}
|
|
25
|
+
})
|
|
26
|
+
|
|
27
|
+
import { createBackfiller, normalizeToolName, injectSteering, COMPANION_STEERING } from '../backfill.mjs'
|
|
28
|
+
import { EXECUTE_TOOL_NAMES } from '@gemus/codex-backfill-core'
|
|
29
|
+
|
|
30
|
+
/** rollout JSONL with N imagegen images (distinct call_ids, PNG-magic base64 so mime→png). */
|
|
31
|
+
function rollout(images: Array<[string, string]>): string {
|
|
32
|
+
return images
|
|
33
|
+
.map(([callId, b64]) =>
|
|
34
|
+
JSON.stringify({ payload: { type: 'image_generation_end', call_id: callId, result: b64 } }),
|
|
35
|
+
)
|
|
36
|
+
.join('\n')
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
/** MCP ContentBlock[] wrap of a structured tool result (what the wire carries; unwrapMcpContent restores it). */
|
|
40
|
+
function wrap(obj: unknown) {
|
|
41
|
+
return [{ type: 'text', text: JSON.stringify(obj) }]
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
const IMG_A: [string, string] = ['ig_a', 'iVBORw0KGgoAAAA1']
|
|
45
|
+
const IMG_B: [string, string] = ['ig_b', 'iVBORw0KGgoAAAA2']
|
|
46
|
+
|
|
47
|
+
/** Simulate one `execute` Phase-1 awaiting_client_content anchor for `nodeId` on a turn. */
|
|
48
|
+
function observeExecuteAnchor(
|
|
49
|
+
bf: ReturnType<typeof createBackfiller>,
|
|
50
|
+
jsonrpcId: number,
|
|
51
|
+
turnKey: string,
|
|
52
|
+
sessionId: string,
|
|
53
|
+
workflowId: string,
|
|
54
|
+
nodeId: string,
|
|
55
|
+
) {
|
|
56
|
+
bf.observeCall({ jsonrpcId, turnKey, sessionId, name: 'execute', args: { workflowId, nodeId, dryRun: true } })
|
|
57
|
+
bf.observeResult({
|
|
58
|
+
jsonrpcId,
|
|
59
|
+
turnKey,
|
|
60
|
+
name: 'execute',
|
|
61
|
+
resultContent: wrap({ status: 'awaiting_client_content', nodeId }),
|
|
62
|
+
})
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
function makeBackfiller() {
|
|
66
|
+
return createBackfiller({ server: 'http://localhost:3000', apiKey: 'mak_test', codexHome: '/fake/.codex' })
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
beforeEach(() => {
|
|
70
|
+
mocks.deliverImageToNode.mockReset().mockResolvedValue({ ok: true, status: 'processing' })
|
|
71
|
+
mocks.addImageToCanvas.mockReset().mockResolvedValue({ ok: true, entityId: 'agent_x' })
|
|
72
|
+
mocks.readRollout.mockReset().mockReturnValue('')
|
|
73
|
+
mocks.findRolloutFile.mockReset().mockReturnValue('/fake/rollout.jsonl')
|
|
74
|
+
})
|
|
75
|
+
|
|
76
|
+
describe('normalizeToolName (tap → tracker)', () => {
|
|
77
|
+
it('maps wire execute names to the tracker constants', () => {
|
|
78
|
+
expect(normalizeToolName('execute')).toBe('mcp__gemus__execute')
|
|
79
|
+
expect(normalizeToolName('batch_execute')).toBe('mcp__gemus__batch_execute')
|
|
80
|
+
// the whole point: the normalized name must be one the tracker actually matches.
|
|
81
|
+
expect(EXECUTE_TOOL_NAMES.has(normalizeToolName('execute'))).toBe(true)
|
|
82
|
+
expect(EXECUTE_TOOL_NAMES.has(normalizeToolName('batch_execute'))).toBe(true)
|
|
83
|
+
})
|
|
84
|
+
it('passes non-execute tool names through unchanged', () => {
|
|
85
|
+
expect(normalizeToolName('canvas_read')).toBe('canvas_read')
|
|
86
|
+
expect(normalizeToolName('workflow_list')).toBe('workflow_list')
|
|
87
|
+
})
|
|
88
|
+
})
|
|
89
|
+
|
|
90
|
+
describe('injectSteering (Mode-B initialize instructions)', () => {
|
|
91
|
+
it('appends companion steering to the initialize response instructions', () => {
|
|
92
|
+
const msg = { jsonrpc: '2.0', id: 1, result: { protocolVersion: '2024-11-05', serverInfo: { name: 'gemus' } } }
|
|
93
|
+
expect(injectSteering(msg)).toBe(true)
|
|
94
|
+
expect(msg.result.instructions).toContain('auto-backfilled')
|
|
95
|
+
expect(msg.result.instructions).toContain('NEVER') // don't self-forward
|
|
96
|
+
expect(msg.result.instructions).toBe(COMPANION_STEERING)
|
|
97
|
+
})
|
|
98
|
+
|
|
99
|
+
it('preserves any existing server instructions (appends, not replaces)', () => {
|
|
100
|
+
const msg = { id: 1, result: { protocolVersion: '1', instructions: 'server says hi' } }
|
|
101
|
+
injectSteering(msg)
|
|
102
|
+
expect(msg.result.instructions).toContain('server says hi')
|
|
103
|
+
expect(msg.result.instructions).toContain('Gemus companion')
|
|
104
|
+
})
|
|
105
|
+
|
|
106
|
+
it('is a no-op for non-initialize messages (tool results, notifications)', () => {
|
|
107
|
+
const toolResult = { id: 5, result: { content: [{ type: 'text', text: '{}' }] } }
|
|
108
|
+
expect(injectSteering(toolResult)).toBe(false)
|
|
109
|
+
expect(toolResult.result.instructions).toBeUndefined()
|
|
110
|
+
expect(injectSteering({ method: 'notifications/message' })).toBe(false)
|
|
111
|
+
})
|
|
112
|
+
})
|
|
113
|
+
|
|
114
|
+
describe('backfillTurn — claim path (Stop hook)', () => {
|
|
115
|
+
it('delivers each scavenged image to its planned node (key-only: real workflowId, no sessionId header)', async () => {
|
|
116
|
+
const bf = makeBackfiller()
|
|
117
|
+
observeExecuteAnchor(bf, 1, 't1', 's1', 'wf_real', 'nodeA')
|
|
118
|
+
observeExecuteAnchor(bf, 2, 't1', 's1', 'wf_real', 'nodeB')
|
|
119
|
+
mocks.readRollout.mockReturnValue(rollout([IMG_A, IMG_B]))
|
|
120
|
+
|
|
121
|
+
const r = await bf.backfillTurn({ turnKey: 't1', sessionId: 's1', rolloutPath: '/precise.jsonl', salvageOnly: false })
|
|
122
|
+
|
|
123
|
+
expect(r).toEqual({ delivered: 2, orphaned: 0, dropped: 0 })
|
|
124
|
+
// Stop hook path reads the PRECISE rollout (transcript_path), not findRolloutFile.
|
|
125
|
+
expect(mocks.readRollout).toHaveBeenCalledWith('/precise.jsonl')
|
|
126
|
+
expect(mocks.findRolloutFile).not.toHaveBeenCalled()
|
|
127
|
+
// N:N ordered claim: nodeA→first image, nodeB→second.
|
|
128
|
+
expect(mocks.deliverImageToNode).toHaveBeenCalledTimes(2)
|
|
129
|
+
const calls = mocks.deliverImageToNode.mock.calls.map((c) => c[0])
|
|
130
|
+
expect(calls[0]).toMatchObject({ workflowId: 'wf_real', nodeId: 'nodeA', image: { callId: 'ig_a' } })
|
|
131
|
+
expect(calls[1]).toMatchObject({ workflowId: 'wf_real', nodeId: 'nodeB', image: { callId: 'ig_b' } })
|
|
132
|
+
// key-only invariant: no X-Gemus-Session-Id (no sessionId passed to the delivery fn).
|
|
133
|
+
expect(calls[0].sessionId).toBeUndefined()
|
|
134
|
+
expect(mocks.addImageToCanvas).not.toHaveBeenCalled()
|
|
135
|
+
})
|
|
136
|
+
|
|
137
|
+
it('falls back to orphan when deliver fails (e.g. already_completed) — image not lost', async () => {
|
|
138
|
+
const bf = makeBackfiller()
|
|
139
|
+
observeExecuteAnchor(bf, 1, 't1', 's1', 'wf_real', 'nodeA')
|
|
140
|
+
mocks.readRollout.mockReturnValue(rollout([IMG_A]))
|
|
141
|
+
mocks.deliverImageToNode.mockResolvedValue({ ok: false, code: 'ALREADY_COMPLETED' })
|
|
142
|
+
|
|
143
|
+
const r = await bf.backfillTurn({ turnKey: 't1', sessionId: 's1', rolloutPath: '/x', salvageOnly: false })
|
|
144
|
+
|
|
145
|
+
expect(mocks.deliverImageToNode).toHaveBeenCalledTimes(1)
|
|
146
|
+
expect(mocks.addImageToCanvas).toHaveBeenCalledTimes(1) // orphan fallback
|
|
147
|
+
expect(r).toEqual({ delivered: 0, orphaned: 1, dropped: 0 })
|
|
148
|
+
})
|
|
149
|
+
})
|
|
150
|
+
|
|
151
|
+
describe('backfillTurn — salvageOnly fallback (untrusted hook / idle timer)', () => {
|
|
152
|
+
it('orphans every fresh image and NEVER calls claim/deliver, even with pending anchors', async () => {
|
|
153
|
+
const bf = makeBackfiller()
|
|
154
|
+
// A legit multi-image turn with pending anchors — salvage must NOT mis-claim them.
|
|
155
|
+
observeExecuteAnchor(bf, 1, 't1', 's1', 'wf_real', 'nodeA')
|
|
156
|
+
observeExecuteAnchor(bf, 2, 't1', 's1', 'wf_real', 'nodeB')
|
|
157
|
+
mocks.readRollout.mockReturnValue(rollout([IMG_A, IMG_B]))
|
|
158
|
+
|
|
159
|
+
const r = await bf.backfillTurn({ turnKey: 't1', rolloutPath: null, salvageOnly: true })
|
|
160
|
+
|
|
161
|
+
// idle fallback has no transcript_path → locates rollout via findRolloutFile(session_id).
|
|
162
|
+
expect(mocks.findRolloutFile).toHaveBeenCalledWith('/fake/.codex', 's1')
|
|
163
|
+
expect(mocks.deliverImageToNode).not.toHaveBeenCalled() // never claim in salvage
|
|
164
|
+
expect(mocks.addImageToCanvas).toHaveBeenCalledTimes(2)
|
|
165
|
+
expect(r).toEqual({ delivered: 0, orphaned: 2, dropped: 0 })
|
|
166
|
+
})
|
|
167
|
+
})
|
|
168
|
+
|
|
169
|
+
describe('backfillTurn — idempotency + guards', () => {
|
|
170
|
+
it('is idempotent across the two triggers via the seen-set (2nd pass no-ops)', async () => {
|
|
171
|
+
const bf = makeBackfiller()
|
|
172
|
+
observeExecuteAnchor(bf, 1, 't1', 's1', 'wf_real', 'nodeA')
|
|
173
|
+
mocks.readRollout.mockReturnValue(rollout([IMG_A]))
|
|
174
|
+
|
|
175
|
+
await bf.backfillTurn({ turnKey: 't1', sessionId: 's1', rolloutPath: '/x', salvageOnly: false })
|
|
176
|
+
mocks.deliverImageToNode.mockClear()
|
|
177
|
+
mocks.addImageToCanvas.mockClear()
|
|
178
|
+
// A late idle-timer fire for the same turn: all call_ids already seen → nothing to do.
|
|
179
|
+
const r = await bf.backfillTurn({ turnKey: 't1', sessionId: 's1', rolloutPath: null, salvageOnly: true })
|
|
180
|
+
|
|
181
|
+
expect(r).toEqual({ delivered: 0, orphaned: 0, dropped: 0 })
|
|
182
|
+
expect(mocks.deliverImageToNode).not.toHaveBeenCalled()
|
|
183
|
+
expect(mocks.addImageToCanvas).not.toHaveBeenCalled()
|
|
184
|
+
})
|
|
185
|
+
|
|
186
|
+
it('a FAILED salvage does not burn the image — a later Stop hook still claims it (#1751 seen-on-success)', async () => {
|
|
187
|
+
const bf = makeBackfiller()
|
|
188
|
+
observeExecuteAnchor(bf, 1, 't1', 's1', 'wf_real', 'nodeA')
|
|
189
|
+
mocks.readRollout.mockReturnValue(rollout([IMG_A]))
|
|
190
|
+
// 1) an idle salvage fires but the orphan fails (e.g. session cap) → image must NOT be marked seen.
|
|
191
|
+
mocks.addImageToCanvas.mockResolvedValueOnce({ ok: false, error: 'session cap reached' })
|
|
192
|
+
const r1 = await bf.backfillTurn({ turnKey: 't1', rolloutPath: null, salvageOnly: true })
|
|
193
|
+
expect(r1.orphaned).toBe(0)
|
|
194
|
+
// 2) the real Stop hook fires → the image is still fresh → claim to the planned node succeeds.
|
|
195
|
+
const r2 = await bf.backfillTurn({ turnKey: 't1', sessionId: 's1', rolloutPath: '/x', salvageOnly: false })
|
|
196
|
+
expect(mocks.deliverImageToNode).toHaveBeenCalledTimes(1)
|
|
197
|
+
expect(mocks.deliverImageToNode.mock.calls[0][0]).toMatchObject({ nodeId: 'nodeA' })
|
|
198
|
+
expect(r2.delivered).toBe(1)
|
|
199
|
+
})
|
|
200
|
+
|
|
201
|
+
it('an early idle tick (image not flushed yet) keeps turn state for the later hook (#1751 no-delete-on-empty)', async () => {
|
|
202
|
+
const bf = makeBackfiller()
|
|
203
|
+
observeExecuteAnchor(bf, 1, 't1', 's1', 'wf_real', 'nodeA')
|
|
204
|
+
// 1) idle fires mid-generation, before imagegen flushed to rollout → empty, must be a pure no-op.
|
|
205
|
+
mocks.readRollout.mockReturnValueOnce('')
|
|
206
|
+
const r1 = await bf.backfillTurn({ turnKey: 't1', rolloutPath: null, salvageOnly: true })
|
|
207
|
+
expect(r1).toEqual({ delivered: 0, orphaned: 0, dropped: 0 })
|
|
208
|
+
expect(mocks.addImageToCanvas).not.toHaveBeenCalled()
|
|
209
|
+
// 2) hook fires after flush → the pending anchor + workflowId survived → claim succeeds.
|
|
210
|
+
mocks.readRollout.mockReturnValue(rollout([IMG_A]))
|
|
211
|
+
const r2 = await bf.backfillTurn({ turnKey: 't1', sessionId: 's1', rolloutPath: '/x', salvageOnly: false })
|
|
212
|
+
expect(mocks.deliverImageToNode).toHaveBeenCalledTimes(1)
|
|
213
|
+
expect(r2.delivered).toBe(1)
|
|
214
|
+
})
|
|
215
|
+
|
|
216
|
+
it('does nothing (no placement) when no workflowId was ever captured', async () => {
|
|
217
|
+
const bf = makeBackfiller()
|
|
218
|
+
// No tool call carried a workflowId this turn.
|
|
219
|
+
mocks.readRollout.mockReturnValue(rollout([IMG_A]))
|
|
220
|
+
const r = await bf.backfillTurn({ turnKey: 't1', sessionId: 's1', rolloutPath: '/x', salvageOnly: false })
|
|
221
|
+
expect(r).toEqual({ delivered: 0, orphaned: 0, dropped: 0 })
|
|
222
|
+
expect(mocks.deliverImageToNode).not.toHaveBeenCalled()
|
|
223
|
+
expect(mocks.addImageToCanvas).not.toHaveBeenCalled()
|
|
224
|
+
})
|
|
225
|
+
})
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
import { describe, it, expect } from 'vitest'
|
|
2
|
+
import { Client } from '@modelcontextprotocol/sdk/client/index.js'
|
|
3
|
+
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js'
|
|
4
|
+
|
|
5
|
+
/**
|
|
6
|
+
* Opt-in integration test against a REAL running gemus `/api/mcp` route (#1751). Validates the Mode-B
|
|
7
|
+
* **key-only** contract the proxy + core delivery rely on: Bearer `mak_` auth with **no**
|
|
8
|
+
* `X-Gemus-Session-Id` header, and addressing by an explicit real `workflowId`.
|
|
9
|
+
*
|
|
10
|
+
* Excluded from the default runner (`*.integration.test.ts`) and gated on env so it never runs in CI or
|
|
11
|
+
* by accident. To run against local dev:
|
|
12
|
+
* GEMUS_IT_KEY=mak_... GEMUS_IT_WORKFLOW=<realWorkflowId> \
|
|
13
|
+
* pnpm exec vitest run packages/gemus-mcp-proxy/src/__tests__/route.integration.test.ts
|
|
14
|
+
* (GEMUS_IT_URL defaults to http://localhost:3000/api/mcp.) All calls here are read-only (no mutation).
|
|
15
|
+
*
|
|
16
|
+
* connect-per-call invariant: every case opens + closes its own client (fresh connectionNonce, #1744).
|
|
17
|
+
*/
|
|
18
|
+
const KEY = process.env.GEMUS_IT_KEY
|
|
19
|
+
const URL_ = process.env.GEMUS_IT_URL || 'http://localhost:3000/api/mcp'
|
|
20
|
+
const WORKFLOW = process.env.GEMUS_IT_WORKFLOW
|
|
21
|
+
|
|
22
|
+
/** Key-only client: Bearer only, deliberately NO X-Gemus-Session-Id (the Mode-B contract). */
|
|
23
|
+
async function connectKeyOnly() {
|
|
24
|
+
const client = new Client({ name: 'gemus-proxy-it', version: '0.1.0' })
|
|
25
|
+
const transport = new StreamableHTTPClientTransport(new URL(URL_), {
|
|
26
|
+
requestInit: { headers: { Authorization: `Bearer ${KEY}` } },
|
|
27
|
+
})
|
|
28
|
+
await client.connect(transport)
|
|
29
|
+
return client
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
describe.skipIf(!KEY)('key-only /api/mcp route (real, opt-in)', () => {
|
|
33
|
+
it('relays the full tool surface incl. execute/batch_execute (no session header)', async () => {
|
|
34
|
+
const client = await connectKeyOnly()
|
|
35
|
+
try {
|
|
36
|
+
const { tools } = await client.listTools()
|
|
37
|
+
const names = tools.map((t) => t.name)
|
|
38
|
+
expect(names).toEqual(expect.arrayContaining(['execute', 'batch_execute', 'canvas_read', 'canvas_edit']))
|
|
39
|
+
} finally {
|
|
40
|
+
await client.close()
|
|
41
|
+
}
|
|
42
|
+
})
|
|
43
|
+
|
|
44
|
+
it('serves resources (skills / knowledge) through the key-only connection', async () => {
|
|
45
|
+
const client = await connectKeyOnly()
|
|
46
|
+
try {
|
|
47
|
+
const { resources } = await client.listResources()
|
|
48
|
+
// The route exposes skill:// / knowledge:// resources; a tool-only passthrough would drop these.
|
|
49
|
+
expect(Array.isArray(resources)).toBe(true)
|
|
50
|
+
} finally {
|
|
51
|
+
await client.close()
|
|
52
|
+
}
|
|
53
|
+
})
|
|
54
|
+
|
|
55
|
+
it.skipIf(!WORKFLOW)('addresses a real workflow by explicit workflowId (key-only, no activeContext)', async () => {
|
|
56
|
+
const client = await connectKeyOnly()
|
|
57
|
+
try {
|
|
58
|
+
// canvas_read is read-only; success proves key-only addressing by explicit workflowId works
|
|
59
|
+
// without a session header (what deliverImageToNode relies on in Mode B).
|
|
60
|
+
const res = (await client.callTool({ name: 'canvas_read', arguments: { workflowId: WORKFLOW } })) as {
|
|
61
|
+
isError?: boolean
|
|
62
|
+
}
|
|
63
|
+
expect(res.isError).not.toBe(true)
|
|
64
|
+
} finally {
|
|
65
|
+
await client.close()
|
|
66
|
+
}
|
|
67
|
+
})
|
|
68
|
+
})
|
package/src/backfill.mjs
ADDED
|
@@ -0,0 +1,195 @@
|
|
|
1
|
+
// @gemus/mcp-proxy — turn-scoped imagegen backfill orchestration (Issue #1751, Mode B key-only).
|
|
2
|
+
//
|
|
3
|
+
// Consumes the shared core (@gemus/codex-backfill-core): taps proxied execute/batch_execute to feed
|
|
4
|
+
// a per-turn ExecuteClaimTracker, then at turn end reads the rollout, claims scavenged images to their
|
|
5
|
+
// planned nodes (key-only deliverImageToNode with a REAL workflowId), and orphans the rest.
|
|
6
|
+
//
|
|
7
|
+
// Turn identity (Step-0.4 real-machine capture, codex 0.142.2): both the tool-call `_meta` and the
|
|
8
|
+
// Stop-hook payload carry `turn_id` + `session_id` (NO thread_id). Turn state is keyed by `turn_id`
|
|
9
|
+
// (each turn = its own tracker); the rollout file is named `-<session_id>.jsonl`, so the idle-timer
|
|
10
|
+
// fallback locates it via findRolloutFile(codexHome, session_id).
|
|
11
|
+
//
|
|
12
|
+
// Two turn-end triggers (see proxy.mjs):
|
|
13
|
+
// • Stop hook (trusted) → backfillTurn({ rolloutPath: <transcript_path>, salvageOnly: false }) → claim
|
|
14
|
+
// • idle-timer fallback → backfillTurn({ rolloutPath: null, salvageOnly: true }) → orphan-only
|
|
15
|
+
// `seen` (session-wide call_id set) makes the two idempotent: whichever runs first marks images seen, so
|
|
16
|
+
// a later trigger no-ops. salvageOnly NEVER calls claim() — mirrors bridge salvage semantics (图不丢, 不认领).
|
|
17
|
+
import * as core from '@gemus/codex-backfill-core'
|
|
18
|
+
|
|
19
|
+
/**
|
|
20
|
+
* Normalize a raw MCP wire tool name to the tracker's expected form. The proxy sees `execute` /
|
|
21
|
+
* `batch_execute` on the wire; ExecuteClaimTracker only matches `mcp__gemus__execute` /
|
|
22
|
+
* `mcp__gemus__batch_execute` (mirrors codexEventAdapter.normalizeToolName). Without this the
|
|
23
|
+
* tracker silently no-ops → every image orphans. Non-execute names pass through unchanged.
|
|
24
|
+
*/
|
|
25
|
+
const WIRE_TO_NORMALIZED = {
|
|
26
|
+
execute: 'mcp__gemus__execute',
|
|
27
|
+
batch_execute: 'mcp__gemus__batch_execute',
|
|
28
|
+
}
|
|
29
|
+
export function normalizeToolName(name) {
|
|
30
|
+
return WIRE_TO_NORMALIZED[name] ?? name
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
// Companion steering (Mode B has no bridge to inject CLIENT_DELIVERED_GUIDANCE into the system prompt).
|
|
34
|
+
// This proxy IS the backfiller, so it tells codex — via the MCP initialize `instructions` field — not to
|
|
35
|
+
// self-forward imagegen output. Mirrors src/lib/agent/core/prompt/workflowBehavior.ts (condensed). Without
|
|
36
|
+
// it codex hunts the file / uploads it to a public host to self-deliver (#1751 real-machine observation).
|
|
37
|
+
export const COMPANION_STEERING = [
|
|
38
|
+
'# Gemus companion — codex-imagen is auto-backfilled',
|
|
39
|
+
'',
|
|
40
|
+
'A local Gemus companion backfills your own image generations onto the planned canvas nodes at turn',
|
|
41
|
+
'end. For a `codex-imagen` (gen-image) node: (1) create the node with config.model="codex-imagen",',
|
|
42
|
+
'(2) call `execute` with dryRun:true and read the returned resolvedPrompt, (3) generate the image',
|
|
43
|
+
'LOCALLY with your built-in imagegen, then (4) STOP.',
|
|
44
|
+
'',
|
|
45
|
+
'NEVER locate, read, host, upload, or submit the imagegen file yourself — you cannot access it and it',
|
|
46
|
+
'is delivered for you. NEVER pass `content` for a codex-imagen node, and NEVER spin up a local HTTP',
|
|
47
|
+
'server or upload the image to a public host to "deliver" it. A codex-imagen node showing no output /',
|
|
48
|
+
'clientDeliveredStatus:"awaiting_backfill" DURING your turn is ON TRACK, not a failure — do not re-read',
|
|
49
|
+
'it in a loop, do not switch its model to a platform provider (gemini/etc), do not self-forward. Just',
|
|
50
|
+
'end your turn; the image arrives on the node after it.',
|
|
51
|
+
'',
|
|
52
|
+
'(Text is the exception: codex-text has no backfill — self-forward with execute dryRun:false +',
|
|
53
|
+
'content:{text:"..."}.)',
|
|
54
|
+
].join('\n')
|
|
55
|
+
|
|
56
|
+
/**
|
|
57
|
+
* Mutate an MCP `initialize` RESPONSE in place to append companion steering to `result.instructions`
|
|
58
|
+
* (the one always-on channel to the model in Mode B). No-op for any other message. Returns true if it
|
|
59
|
+
* injected (for logging). Only touches the initialize result; everything else relays untouched.
|
|
60
|
+
*/
|
|
61
|
+
export function injectSteering(msg) {
|
|
62
|
+
const result = msg?.result
|
|
63
|
+
if (!result || !(result.protocolVersion || result.serverInfo)) return false
|
|
64
|
+
const existing = typeof result.instructions === 'string' ? result.instructions : ''
|
|
65
|
+
result.instructions = existing ? `${existing}\n\n${COMPANION_STEERING}` : COMPANION_STEERING
|
|
66
|
+
return true
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
/**
|
|
70
|
+
* @param {object} deps
|
|
71
|
+
* @param {string} deps.server 平台 origin(不含 /api/mcp)
|
|
72
|
+
* @param {string} deps.apiKey 用户 mak_ key
|
|
73
|
+
* @param {string} deps.codexHome ~/.codex(idle-timer fallback 用 findRolloutFile(session_id) 定位 rollout)
|
|
74
|
+
* @param {(msg: string, obj?: unknown) => void} [deps.log]
|
|
75
|
+
*/
|
|
76
|
+
export function createBackfiller({ server, apiKey, codexHome, log = () => {} }) {
|
|
77
|
+
/** @type {Map<string, { tracker: InstanceType<typeof core.ExecuteClaimTracker>, workflowId?: string, sessionId?: string }>} */
|
|
78
|
+
const turns = new Map() // keyed by turn_id; bounded (evict oldest) so a long session can't leak
|
|
79
|
+
const TURN_CAP = 64
|
|
80
|
+
/** session-wide call_ids that were **successfully placed** → 两个触发路径幂等。**只在交付成功后加入**
|
|
81
|
+
* (不在尝试前):否则一次过早/失败的 idle salvage 会把图标记 seen,真正的 Stop hook 就再也认领不到它。 */
|
|
82
|
+
const seen = new Set()
|
|
83
|
+
|
|
84
|
+
function getTurn(turnKey) {
|
|
85
|
+
let t = turns.get(turnKey)
|
|
86
|
+
if (!t) {
|
|
87
|
+
if (turns.size >= TURN_CAP) {
|
|
88
|
+
const oldest = turns.keys().next().value
|
|
89
|
+
if (oldest !== undefined) turns.delete(oldest)
|
|
90
|
+
}
|
|
91
|
+
t = { tracker: new core.ExecuteClaimTracker(), workflowId: undefined, sessionId: undefined }
|
|
92
|
+
turns.set(turnKey, t)
|
|
93
|
+
}
|
|
94
|
+
return t
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
/** tap: client→server `tools/call` request. jsonrpcId 作 toolCallId 关联 request↔response。 */
|
|
98
|
+
function observeCall({ jsonrpcId, turnKey, sessionId, name, args }) {
|
|
99
|
+
if (!turnKey) return
|
|
100
|
+
const t = getTurn(turnKey)
|
|
101
|
+
if (sessionId) t.sessionId = sessionId // idle-fallback rollout 定位用
|
|
102
|
+
// Mode B 必需:真实 workflowId 从 args 捕获(execute/canvas_read 都带;server 无 activeContext 可覆盖)。
|
|
103
|
+
if (args && typeof args.workflowId === 'string' && args.workflowId) t.workflowId = args.workflowId
|
|
104
|
+
t.tracker.observe({
|
|
105
|
+
type: 'tool-call',
|
|
106
|
+
toolName: normalizeToolName(name),
|
|
107
|
+
toolCallId: String(jsonrpcId),
|
|
108
|
+
args,
|
|
109
|
+
})
|
|
110
|
+
}
|
|
111
|
+
|
|
112
|
+
/** tap: server→client `tools/call` response. resultContent = MCP ContentBlock[](unwrap 还原结构)。 */
|
|
113
|
+
function observeResult({ jsonrpcId, turnKey, name, resultContent }) {
|
|
114
|
+
if (!turnKey) return
|
|
115
|
+
const t = getTurn(turnKey)
|
|
116
|
+
t.tracker.observe({
|
|
117
|
+
type: 'tool-result',
|
|
118
|
+
toolName: normalizeToolName(name),
|
|
119
|
+
toolCallId: String(jsonrpcId),
|
|
120
|
+
result: core.unwrapMcpContent(resultContent),
|
|
121
|
+
})
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
/** orphan 一张图(Mode B 需真实 workflowId),**只在成功后**标记 seen;返回是否成功。 */
|
|
125
|
+
async function orphanOne(workflowId, image) {
|
|
126
|
+
const r = await core.addImageToCanvas({ server, apiKey, workflowId, image })
|
|
127
|
+
if (r.ok) {
|
|
128
|
+
seen.add(image.callId)
|
|
129
|
+
return true
|
|
130
|
+
}
|
|
131
|
+
log('backfill: orphan failed', { callId: image.callId, error: r.error })
|
|
132
|
+
return false
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
/**
|
|
136
|
+
* turn 末回填。`rolloutPath` 来自 Stop hook(精确文件);缺省 → findRolloutFile(codexHome, session_id)。
|
|
137
|
+
* `salvageOnly` = true 时只 orphan(绝不 claim),镜像 bridge salvage。
|
|
138
|
+
*
|
|
139
|
+
* **不预先标记 seen、不在空/失败时删除 turn 状态**(#1751 真机 bug 修复):早期或失败的触发(如 idle
|
|
140
|
+
* timer 在生成中途误触发、或 session-cap 交付失败)绝不能烧掉图 —— 否则真正的 Stop hook 认领时
|
|
141
|
+
* fresh 为空。seen 只在成功交付后加入;turn 状态保留供后续 authoritative Stop hook 认领。
|
|
142
|
+
*/
|
|
143
|
+
async function backfillTurn({ turnKey, sessionId, rolloutPath, salvageOnly }) {
|
|
144
|
+
const t = turns.get(turnKey)
|
|
145
|
+
const sid = sessionId || t?.sessionId
|
|
146
|
+
const path = rolloutPath || core.findRolloutFile(codexHome, sid)
|
|
147
|
+
const jsonl = core.readRollout(path)
|
|
148
|
+
const all = core.extractImageOutputs(jsonl)
|
|
149
|
+
const fresh = all.filter((img) => !seen.has(img.callId))
|
|
150
|
+
|
|
151
|
+
if (fresh.length === 0) {
|
|
152
|
+
// 图还没 flush(idle 过早触发)或都已交付 —— 都不动 seen/turn 状态,留给后续 hook。
|
|
153
|
+
log('backfill: no fresh images', { turnKey, salvageOnly, path: path || '(none)' })
|
|
154
|
+
return { delivered: 0, orphaned: 0, dropped: 0 }
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
const workflowId = t?.workflowId
|
|
158
|
+
if (!workflowId) {
|
|
159
|
+
// Mode B 无 activeContext:orphan 也需真实 workflowId 寻址。turn 里任一工具调用都带 workflowId,
|
|
160
|
+
// 正常不会走到这里;防御性记录并放弃(不烧 seen,后续带 workflowId 的 hook 仍可认领)。
|
|
161
|
+
log('backfill: no workflowId captured — cannot place images', { turnKey, fresh: fresh.length })
|
|
162
|
+
return { delivered: 0, orphaned: 0, dropped: 0 }
|
|
163
|
+
}
|
|
164
|
+
|
|
165
|
+
// salvageOnly / 无 tracker → orphan-only(绝不 claim)。
|
|
166
|
+
if (salvageOnly || !t) {
|
|
167
|
+
let orphaned = 0
|
|
168
|
+
for (const image of fresh) if (await orphanOne(workflowId, image)) orphaned++
|
|
169
|
+
log('backfill: salvage', { turnKey, orphaned, of: fresh.length })
|
|
170
|
+
return { delivered: 0, orphaned, dropped: 0 }
|
|
171
|
+
}
|
|
172
|
+
|
|
173
|
+
// 认领路径:claim() 保守分配(详见 core),失败 deliver 回退 orphan(图不丢)。seen 只在成功后加。
|
|
174
|
+
const { claims, orphans, dropped } = t.tracker.claim(fresh)
|
|
175
|
+
let delivered = 0
|
|
176
|
+
let orphaned = 0
|
|
177
|
+
for (const { nodeId, image } of claims) {
|
|
178
|
+
const r = await core.deliverImageToNode({ server, apiKey, workflowId, nodeId, image })
|
|
179
|
+
if (r.ok) {
|
|
180
|
+
seen.add(image.callId)
|
|
181
|
+
delivered++
|
|
182
|
+
} else if (await orphanOne(workflowId, image)) {
|
|
183
|
+
// already_completed / 其它失败 → orphan 保底(镜像 bridge:新图未落节点则至少 image-upload)。
|
|
184
|
+
orphaned++
|
|
185
|
+
} else {
|
|
186
|
+
log('backfill: deliver+orphan both failed', { nodeId, callId: image.callId, error: r.error ?? r.code })
|
|
187
|
+
}
|
|
188
|
+
}
|
|
189
|
+
for (const image of orphans) if (await orphanOne(workflowId, image)) orphaned++
|
|
190
|
+
log('backfill: claimed', { turnKey, delivered, orphaned, dropped })
|
|
191
|
+
return { delivered, orphaned, dropped }
|
|
192
|
+
}
|
|
193
|
+
|
|
194
|
+
return { observeCall, observeResult, backfillTurn, _turns: turns, _seen: seen }
|
|
195
|
+
}
|
package/src/proxy.mjs
ADDED
|
@@ -0,0 +1,226 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// @gemus/mcp-proxy — local stdio<->HTTP MCP proxy for Codex desktop (Issue #1751, P1).
|
|
3
|
+
//
|
|
4
|
+
// Codex (stdio) <-> this proxy <-> remote gemus /api/mcp (streamable HTTP, Bearer mak_ key).
|
|
5
|
+
// The proxy is a TRANSPARENT JSON-RPC passthrough (tools + resources + notifications all relay
|
|
6
|
+
// untouched) that additionally TAPS execute/batch_execute to observe client-delivered image
|
|
7
|
+
// backfill anchors, then at turn end backfills codex imagegen output into the planned gen-* nodes.
|
|
8
|
+
// Codex does no auth for stdio MCP servers, so THIS process owns upstream auth (the user's mak_ key).
|
|
9
|
+
//
|
|
10
|
+
// Turn-end backfill has two triggers:
|
|
11
|
+
// • Stop hook (trusted) → loopback HTTP POST /backfill { transcript_path, thread_id } → claim path.
|
|
12
|
+
// • idle-timer fallback (untrusted hook) → salvageOnly (orphan-only). Both idempotent via seen-set.
|
|
13
|
+
//
|
|
14
|
+
// Validated end-to-end on 2026-07-06 (codex-cli 0.142.2, real dev /api/mcp): transparent passthrough
|
|
15
|
+
// of all 12 gemus tools; codex spawns one proxy per session; real tool-calls tapped with workflowId
|
|
16
|
+
// (from arguments) + turn/thread identity (from _meta.x-codex-turn-metadata). Step-0.4: plugin-bundled
|
|
17
|
+
// Stop hook fires (needs /hooks trust); codex does NOT interpolate ${VAR} in MCP env → key is a literal
|
|
18
|
+
// in local `codex mcp add --env`. See docs/plans/1751-codex-desktop-companion.md.
|
|
19
|
+
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'
|
|
20
|
+
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js'
|
|
21
|
+
import http from 'node:http'
|
|
22
|
+
import os from 'node:os'
|
|
23
|
+
import path from 'node:path'
|
|
24
|
+
import fs from 'node:fs'
|
|
25
|
+
import { randomBytes } from 'node:crypto'
|
|
26
|
+
import { createBackfiller, injectSteering } from './backfill.mjs'
|
|
27
|
+
|
|
28
|
+
const KEY = process.env.GEMUS_KEY || ''
|
|
29
|
+
const GEMUS_URL = process.env.GEMUS_URL || 'https://gemus.ai/api/mcp'
|
|
30
|
+
const SERVER_ORIGIN = GEMUS_URL.replace(/\/api\/mcp\/?$/, '') // core delivery appends /api/mcp itself
|
|
31
|
+
const CODEX_HOME = process.env.CODEX_HOME || path.join(os.homedir(), '.codex')
|
|
32
|
+
// turn-idle fallback (untrusted-hook path). Default 3min: imagegen alone runs 30-60s with NO MCP tool
|
|
33
|
+
// calls, so a short idle looks like "turn ended" mid-generation and fires prematurely. A trusted Stop
|
|
34
|
+
// hook fires at turn end and clears this timer, so trusted users never hit it (#1751 real-machine fix).
|
|
35
|
+
const IDLE_MS = Number(process.env.PROXY_BACKFILL_IDLE_MS) || 180_000
|
|
36
|
+
const LOG = process.env.PROXY_LOG // optional debug log path
|
|
37
|
+
const log = (dir, m) => {
|
|
38
|
+
if (!LOG) return
|
|
39
|
+
fs.appendFileSync(LOG, `[${new Date().toISOString()}] ${dir} ${typeof m === 'string' ? m : JSON.stringify(m).slice(0, 1000)}\n`)
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
const backfiller = createBackfiller({
|
|
43
|
+
server: SERVER_ORIGIN,
|
|
44
|
+
apiKey: KEY,
|
|
45
|
+
codexHome: CODEX_HOME,
|
|
46
|
+
log: (msg, obj) => log('BACKFILL', obj ? `${msg} ${JSON.stringify(obj)}` : msg),
|
|
47
|
+
})
|
|
48
|
+
|
|
49
|
+
// ── turn identity + request↔response correlation ───────────────────────────────────────────────
|
|
50
|
+
// Step-0.4 capture: tool-call `_meta` + Stop-hook payload both carry `turn_id` + `session_id` (no
|
|
51
|
+
// thread_id). Turn state keys on turn_id; session_id locates the rollout in the idle fallback.
|
|
52
|
+
/** in-flight tools/call: jsonrpc id (string) → { turnKey, name } — so the server→client response
|
|
53
|
+
* (which carries no _meta) can be attributed to its turn + tool. */
|
|
54
|
+
const inflight = new Map()
|
|
55
|
+
/** per-turn idle timer — turn-END fallback when no Stop hook fires (untrusted). Reset on activity. */
|
|
56
|
+
const idleTimers = new Map()
|
|
57
|
+
let sessionId // learned from first tool-call _meta.session_id; used for the hook discovery file
|
|
58
|
+
let discoveryFile // <tmpdir>/gemus-proxy-<sessionId>.json — written once sessionId is known
|
|
59
|
+
|
|
60
|
+
function metaOf(msg) {
|
|
61
|
+
const meta = msg?.params?._meta?.['x-codex-turn-metadata']
|
|
62
|
+
return {
|
|
63
|
+
turnId: meta?.turn_id || meta?.turnId,
|
|
64
|
+
sessionId: meta?.session_id || meta?.sessionId,
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
function armIdleTimer(turnKey) {
|
|
69
|
+
if (!turnKey) return
|
|
70
|
+
const existing = idleTimers.get(turnKey)
|
|
71
|
+
if (existing) clearTimeout(existing)
|
|
72
|
+
const timer = setTimeout(() => {
|
|
73
|
+
idleTimers.delete(turnKey)
|
|
74
|
+
// salvageOnly fallback: turn went quiet with no Stop hook. Orphan any fresh images (never claim).
|
|
75
|
+
// seen-set makes this a no-op if a hook already backfilled this turn.
|
|
76
|
+
backfiller
|
|
77
|
+
.backfillTurn({ turnKey, rolloutPath: null, salvageOnly: true })
|
|
78
|
+
.catch((e) => log('IDLE-BACKFILL-ERR', String(e)))
|
|
79
|
+
}, IDLE_MS)
|
|
80
|
+
if (typeof timer.unref === 'function') timer.unref() // don't keep the process alive on this timer alone
|
|
81
|
+
idleTimers.set(turnKey, timer)
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
function writeDiscoveryOnce() {
|
|
85
|
+
if (discoveryFile || !sessionId || !hookPort || !hookToken) return
|
|
86
|
+
discoveryFile = path.join(os.tmpdir(), `gemus-proxy-${sessionId}.json`)
|
|
87
|
+
try {
|
|
88
|
+
fs.writeFileSync(discoveryFile, JSON.stringify({ port: hookPort, token: hookToken }))
|
|
89
|
+
log('DISCOVERY-WRITE', { discoveryFile, port: hookPort })
|
|
90
|
+
} catch (e) {
|
|
91
|
+
log('DISCOVERY-ERR', String(e))
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
// injectSteering (imported) mutates the initialize response's `instructions` to tell codex not to
|
|
96
|
+
// self-forward imagegen output — the one always-on steering channel in Mode B. See backfill.mjs.
|
|
97
|
+
function steerOnInit(msg) {
|
|
98
|
+
try {
|
|
99
|
+
if (injectSteering(msg)) log('STEERING-INJECTED', { instructionsLen: msg.result.instructions.length })
|
|
100
|
+
} catch (e) {
|
|
101
|
+
log('STEERING-ERR', String(e))
|
|
102
|
+
}
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
// ── taps (do not mutate the relayed messages) ──────────────────────────────────────────────────
|
|
106
|
+
function tapClientToServer(msg) {
|
|
107
|
+
try {
|
|
108
|
+
if (msg?.method !== 'tools/call') return
|
|
109
|
+
const { turnId, sessionId: sid } = metaOf(msg)
|
|
110
|
+
if (sid && !sessionId) {
|
|
111
|
+
sessionId = sid
|
|
112
|
+
writeDiscoveryOnce()
|
|
113
|
+
}
|
|
114
|
+
if (turnId) armIdleTimer(turnId)
|
|
115
|
+
const id = msg.id
|
|
116
|
+
if (id === undefined || id === null) return // notifications: no response to correlate
|
|
117
|
+
const name = msg.params?.name || ''
|
|
118
|
+
inflight.set(String(id), { turnKey: turnId, name })
|
|
119
|
+
backfiller.observeCall({ jsonrpcId: id, turnKey: turnId, sessionId: sid, name, args: msg.params?.arguments || {} })
|
|
120
|
+
} catch (e) {
|
|
121
|
+
log('TAP-C2S-ERR', String(e))
|
|
122
|
+
}
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
function tapServerToClient(msg) {
|
|
126
|
+
try {
|
|
127
|
+
const id = msg?.id
|
|
128
|
+
if (id === undefined || id === null) return
|
|
129
|
+
const pending = inflight.get(String(id))
|
|
130
|
+
if (!pending) return
|
|
131
|
+
inflight.delete(String(id))
|
|
132
|
+
if (!msg.result) return // errors: nothing to observe for claim tracking
|
|
133
|
+
backfiller.observeResult({
|
|
134
|
+
jsonrpcId: id,
|
|
135
|
+
turnKey: pending.turnKey,
|
|
136
|
+
name: pending.name,
|
|
137
|
+
resultContent: msg.result?.content,
|
|
138
|
+
})
|
|
139
|
+
} catch (e) {
|
|
140
|
+
log('TAP-S2C-ERR', String(e))
|
|
141
|
+
}
|
|
142
|
+
}
|
|
143
|
+
|
|
144
|
+
// ── loopback Stop-hook endpoint (127.0.0.1:<random port> + per-run token) ──────────────────────
|
|
145
|
+
let hookPort
|
|
146
|
+
const hookToken = randomBytes(16).toString('hex')
|
|
147
|
+
const hookServer = http.createServer((req, res) => {
|
|
148
|
+
if (req.method !== 'POST' || !req.url?.startsWith('/backfill')) {
|
|
149
|
+
res.writeHead(404).end()
|
|
150
|
+
return
|
|
151
|
+
}
|
|
152
|
+
let body = ''
|
|
153
|
+
req.on('data', (c) => {
|
|
154
|
+
body += c
|
|
155
|
+
if (body.length > 64 * 1024) req.destroy() // trivial DoS guard; payload is tiny JSON
|
|
156
|
+
})
|
|
157
|
+
req.on('end', () => {
|
|
158
|
+
let payload
|
|
159
|
+
try {
|
|
160
|
+
payload = JSON.parse(body)
|
|
161
|
+
} catch {
|
|
162
|
+
res.writeHead(400).end()
|
|
163
|
+
return
|
|
164
|
+
}
|
|
165
|
+
if (payload?.token !== hookToken) {
|
|
166
|
+
res.writeHead(403).end()
|
|
167
|
+
return
|
|
168
|
+
}
|
|
169
|
+
// Stop-hook payload (Step-0.4 capture): { session_id, turn_id, transcript_path, ... } — no thread_id.
|
|
170
|
+
const turnKey = payload.turn_id || payload.turnId
|
|
171
|
+
const sid = payload.session_id || payload.sessionId
|
|
172
|
+
const rolloutPath = payload.transcript_path || payload.transcriptPath || null
|
|
173
|
+
log('HOOK-BACKFILL', { turnKey, sid, rolloutPath })
|
|
174
|
+
res.writeHead(200).end('ok')
|
|
175
|
+
// A Stop hook backfilled this turn → cancel its idle-timer fallback (avoid a redundant salvage pass).
|
|
176
|
+
const timer = idleTimers.get(turnKey)
|
|
177
|
+
if (timer) { clearTimeout(timer); idleTimers.delete(turnKey) }
|
|
178
|
+
// Stop hook fires post-flush → claim path (not salvage). Fire-and-forget; never throw.
|
|
179
|
+
backfiller
|
|
180
|
+
.backfillTurn({ turnKey, sessionId: sid, rolloutPath, salvageOnly: false })
|
|
181
|
+
.catch((e) => log('HOOK-BACKFILL-ERR', String(e)))
|
|
182
|
+
})
|
|
183
|
+
})
|
|
184
|
+
hookServer.on('error', (e) => log('HOOK-SERVER-ERR', String(e)))
|
|
185
|
+
hookServer.listen(0, '127.0.0.1', () => {
|
|
186
|
+
hookPort = hookServer.address().port
|
|
187
|
+
log('HOOK-SERVER', { port: hookPort })
|
|
188
|
+
writeDiscoveryOnce() // in case sessionId was already learned before listen resolved
|
|
189
|
+
})
|
|
190
|
+
if (typeof hookServer.unref === 'function') hookServer.unref()
|
|
191
|
+
|
|
192
|
+
// ── transparent JSON-RPC passthrough (raw send/onmessage) ──────────────────────────────────────
|
|
193
|
+
const upstream = new StreamableHTTPClientTransport(new URL(GEMUS_URL), {
|
|
194
|
+
requestInit: { headers: { Authorization: `Bearer ${KEY}` } },
|
|
195
|
+
})
|
|
196
|
+
const stdio = new StdioServerTransport()
|
|
197
|
+
|
|
198
|
+
stdio.onmessage = (m) => { tapClientToServer(m); log('C->G', m); upstream.send(m).catch((e) => log('ERR-http-send', String(e))) }
|
|
199
|
+
upstream.onmessage = (m) => { tapServerToClient(m); steerOnInit(m); log('G->C', m); stdio.send(m).catch((e) => log('ERR-stdio-send', String(e))) }
|
|
200
|
+
|
|
201
|
+
// ── session hygiene (spike: KEY_ONLY_SESSION_CAP=3) ────────────────────────────────────────────
|
|
202
|
+
// We must DELETE the upstream session or we leak a key-only slot when codex ends the conversation.
|
|
203
|
+
// NOTE: transport.close() does NOT issue DELETE (it only aborts + fires onclose, streamableHttp.js:280);
|
|
204
|
+
// terminateSession() sends the DELETE (:431). Call it first, then close(). Also clean up local artifacts.
|
|
205
|
+
let shuttingDown = false
|
|
206
|
+
async function shutdown() {
|
|
207
|
+
if (shuttingDown) return
|
|
208
|
+
shuttingDown = true
|
|
209
|
+
for (const t of idleTimers.values()) clearTimeout(t)
|
|
210
|
+
idleTimers.clear()
|
|
211
|
+
try { hookServer.close() } catch { /* ignore */ }
|
|
212
|
+
if (discoveryFile) { try { fs.unlinkSync(discoveryFile) } catch { /* ignore */ } }
|
|
213
|
+
try { await upstream.terminateSession() } catch { /* ignore */ }
|
|
214
|
+
try { await upstream.close() } catch { /* ignore */ }
|
|
215
|
+
process.exit(0)
|
|
216
|
+
}
|
|
217
|
+
stdio.onclose = () => { log('stdio-close', ''); void shutdown() }
|
|
218
|
+
upstream.onclose = () => { log('http-close', '') }
|
|
219
|
+
stdio.onerror = (e) => log('stdio-err', String(e))
|
|
220
|
+
upstream.onerror = (e) => log('http-err', String(e))
|
|
221
|
+
process.on('SIGTERM', shutdown)
|
|
222
|
+
process.on('SIGINT', shutdown)
|
|
223
|
+
|
|
224
|
+
await upstream.start()
|
|
225
|
+
await stdio.start()
|
|
226
|
+
log('STARTED', { GEMUS_URL, SERVER_ORIGIN, hasKey: Boolean(KEY), pid: process.pid, idleMs: IDLE_MS })
|