dsh-plugin-lcu 0.2.9 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +162 -98
- package/cordis.patch.yml +11 -6
- package/docs/README.zh.md +150 -167
- package/helper/sky-service.mjs +484 -0
- package/lib/app.js +284 -0
- package/lib/app.js.map +1 -0
- package/lib/approval.js +45 -4
- package/lib/approval.js.map +1 -1
- package/lib/connection.js +82 -35
- package/lib/connection.js.map +1 -1
- package/lib/control.js +214 -0
- package/lib/control.js.map +1 -0
- package/lib/diag.js +1 -1
- package/lib/host-guard.js +2 -2
- package/lib/index.js +100 -30
- package/lib/index.js.map +1 -1
- package/lib/session.js +137 -0
- package/lib/session.js.map +1 -0
- package/lib/tool.js +7 -4
- package/lib/tool.js.map +1 -1
- package/lib/types/app.d.ts +118 -0
- package/lib/types/app.d.ts.map +1 -0
- package/lib/types/approval.d.ts +28 -3
- package/lib/types/approval.d.ts.map +1 -1
- package/lib/types/connection.d.ts +27 -13
- package/lib/types/connection.d.ts.map +1 -1
- package/lib/types/control.d.ts +54 -0
- package/lib/types/control.d.ts.map +1 -0
- package/lib/types/diag.d.ts +1 -1
- package/lib/types/host-guard.d.ts +2 -2
- package/lib/types/index.d.ts +26 -6
- package/lib/types/index.d.ts.map +1 -1
- package/lib/types/session.d.ts +89 -0
- package/lib/types/session.d.ts.map +1 -0
- package/lib/types/tool.d.ts +6 -5
- package/lib/types/tool.d.ts.map +1 -1
- package/package.json +7 -5
- package/scripts/gen-presets.mjs +21 -2
- package/scripts/probe-lcu.mjs +40 -18
package/README.md
CHANGED
|
@@ -2,14 +2,16 @@
|
|
|
2
2
|
|
|
3
3
|
English | [中文](docs/README.zh.md)
|
|
4
4
|
|
|
5
|
-
Drive the desktop and Chrome from [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)
|
|
6
|
-
|
|
5
|
+
Drive the desktop and Chrome from [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) using the
|
|
6
|
+
computer-use runtime that ships **inside the ChatGPT desktop application**.
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
8
|
+
This plugin launches that runtime directly and speaks to it over MCP. It is the whole integration: locating and
|
|
9
|
+
validating the application, building the environment the runtime expects, the approval bridge, the turn
|
|
10
|
+
lifecycle, and the macOS service wrapper that performs per-turn cleanup. **Nothing else has to be installed** —
|
|
11
|
+
no wrapper project, no second interpreter, no Python.
|
|
12
|
+
|
|
13
|
+
**No Codex or ChatGPT authentication is involved.** The runtime comes from your local installation and is never
|
|
14
|
+
downloaded, rewritten or authenticated.
|
|
13
15
|
|
|
14
16
|
## Table of Contents
|
|
15
17
|
|
|
@@ -21,6 +23,7 @@ downloads, installs, authenticates, or rewrites it.
|
|
|
21
23
|
- [Approvals and the security model](#approvals-and-the-security-model)
|
|
22
24
|
- [Understand the implementation](#understand-the-implementation)
|
|
23
25
|
- [Troubleshooting](#troubleshooting)
|
|
26
|
+
- [Relationship to LCU](#relationship-to-lcu)
|
|
24
27
|
- [Companion tools](#companion-tools)
|
|
25
28
|
- [Known limitations and deferred work](#known-limitations-and-deferred-work)
|
|
26
29
|
- [Development](#development)
|
|
@@ -28,8 +31,6 @@ downloads, installs, authenticates, or rewrites it.
|
|
|
28
31
|
|
|
29
32
|
## What you get
|
|
30
33
|
|
|
31
|
-
Two model-facing tools, exactly as LCU defines them — this plugin does not invent a schema:
|
|
32
|
-
|
|
33
34
|
| Tool | What it does |
|
|
34
35
|
|---|---|
|
|
35
36
|
| `js` | Run one JavaScript program against the `cua` desktop/browser API. The first call returns the API documentation, and app or tab selection returns the initial UI state. |
|
|
@@ -42,7 +43,7 @@ The plugin adds exactly one tool of its own:
|
|
|
42
43
|
|
|
43
44
|
| Tool | What it does |
|
|
44
45
|
|---|---|
|
|
45
|
-
| `computer_use_stop` | With no argument, list the applications the runtime currently holds for this session. With `app` set to one of their bundle identifiers, release it.
|
|
46
|
+
| `computer_use_stop` | With no argument, list the applications the runtime currently holds for this session. With `app` set to one of their bundle identifiers, release it. It clears the host application's "computer use is active" state for one application without ending the session. |
|
|
46
47
|
|
|
47
48
|
Screenshots arrive as durable images through DSH's attachment store, so a model route that declares
|
|
48
49
|
image input can actually look at the screen.
|
|
@@ -51,45 +52,18 @@ image input can actually look at the screen.
|
|
|
51
52
|
|
|
52
53
|
| | |
|
|
53
54
|
|---|---|
|
|
54
|
-
| OS | macOS on Apple Silicon
|
|
55
|
-
| ChatGPT desktop app | Installed
|
|
56
|
-
| Python | 3.12 or newer, on `PATH` or in `/opt/homebrew/bin`, `/usr/local/bin`, `/usr/bin`. |
|
|
57
|
-
| LCU | Installed separately — see below. |
|
|
55
|
+
| OS | **macOS on Apple Silicon.** The launcher resolves a macOS application bundle and the lifecycle wrapper is a macOS service; other platforms would need both revisited. |
|
|
56
|
+
| ChatGPT desktop app | Installed at `/Applications/ChatGPT.app` (or configured with `app`). It supplies the runtime, its instructions and the signed helper. |
|
|
58
57
|
| DSH | A profile you can install a bundle into. |
|
|
59
58
|
|
|
60
|
-
|
|
61
|
-
|
|
59
|
+
There is **no** Python requirement and **no** separate runtime to install. The plugin starts the application's
|
|
60
|
+
own `node` and its own entry point.
|
|
62
61
|
|
|
63
62
|
## Install
|
|
64
63
|
|
|
65
|
-
### 1. Install
|
|
64
|
+
### 1. Install the plugin into a profile
|
|
66
65
|
|
|
67
|
-
|
|
68
|
-
register another harness: this plugin is your harness.
|
|
69
|
-
|
|
70
|
-
```sh
|
|
71
|
-
TAG=v0.9.6
|
|
72
|
-
TARGET=darwin-arm64
|
|
73
|
-
curl -fLO "https://github.com/amontlabs/lcu/releases/download/$TAG/lcu-${TAG#v}-$TARGET.tar.gz"
|
|
74
|
-
curl -fLO "https://github.com/amontlabs/lcu/releases/download/$TAG/lcu-${TAG#v}-$TARGET.tar.gz.sha256"
|
|
75
|
-
shasum -a 256 -c "lcu-${TAG#v}-$TARGET.tar.gz.sha256" # must print OK
|
|
76
|
-
tar -xzf "lcu-${TAG#v}-$TARGET.tar.gz" && cd "lcu-${TAG#v}-$TARGET"
|
|
77
|
-
./scripts/install.sh --runtime-only --yes
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
Then confirm the runtime loads:
|
|
81
|
-
|
|
82
|
-
```sh
|
|
83
|
-
~/.local/share/lcu/current/bin/lcu doctor --non-interactive
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
You want `Original Mac provider loaded; app listing and app-state methods are available`. Privacy
|
|
87
|
-
permissions are granted on first use, not here.
|
|
88
|
-
|
|
89
|
-
### 2. Install the plugin into a profile
|
|
90
|
-
|
|
91
|
-
Installing makes the plugin's one row — an LCU host — active in that profile. It opens nothing at load
|
|
92
|
-
time.
|
|
66
|
+
Installing makes the plugin's one row active in that profile. It opens nothing at load time.
|
|
93
67
|
|
|
94
68
|
```sh
|
|
95
69
|
dsh plugin --profile <profile> add dsh-plugin-lcu
|
|
@@ -98,7 +72,7 @@ dsh plugin --profile <profile> add dsh-plugin-lcu
|
|
|
98
72
|
For the **desktop** app's managed profile, the CLI refuses; install it through the app's plugin
|
|
99
73
|
manager (Settings ▸ Plugins) instead, which runs the same pnpm operation.
|
|
100
74
|
|
|
101
|
-
###
|
|
75
|
+
### 2. Generate the presets
|
|
102
76
|
|
|
103
77
|
DSH agent presets have **no inheritance**: a preset's `config.plugins` is its complete plugin list, and
|
|
104
78
|
a patch replaces a whole entry rather than merging into it. So a custom preset must restate its base.
|
|
@@ -122,10 +96,10 @@ without writing, and `--out FILE` writes somewhere else.
|
|
|
122
96
|
> sees `run_code`, so `js` would have to be nested as a JavaScript string inside another JavaScript
|
|
123
97
|
> program. `both` keeps `js` directly callable.
|
|
124
98
|
|
|
125
|
-
###
|
|
99
|
+
### 3. Configure which modes get the capability
|
|
126
100
|
|
|
127
|
-
The plugin is one root row with a `presets` allowlist. Edit the installed
|
|
128
|
-
|
|
101
|
+
The plugin is one root row with a `presets` allowlist. Edit the installed `cordis.patch.yml`
|
|
102
|
+
(or the profile patch) to match the preset ids you generated:
|
|
129
103
|
|
|
130
104
|
```yaml
|
|
131
105
|
- id: lcu
|
|
@@ -135,7 +109,7 @@ The plugin is one root row with a `presets` allowlist. Edit the installed
|
|
|
135
109
|
- heavy
|
|
136
110
|
```
|
|
137
111
|
|
|
138
|
-
###
|
|
112
|
+
### 4. Restart, then make one call
|
|
139
113
|
|
|
140
114
|
Restart the harness — plugin **code and configuration changes are not hot-reloaded**. Then start a
|
|
141
115
|
task in the `heavy` mode (displayed as **重活**) and ask it to do something harmless:
|
|
@@ -144,6 +118,16 @@ task in the `heavy` mode (displayed as **重活**) and ask it to do something ha
|
|
|
144
118
|
|
|
145
119
|
The first time an app is touched, the runtime asks for approval. See below.
|
|
146
120
|
|
|
121
|
+
On the first attach the plugin resolves the application and writes what it found to the diagnostic log:
|
|
122
|
+
|
|
123
|
+
```
|
|
124
|
+
app: /Applications/ChatGPT.app version=26.1002.52244 runtime=0.0.29/20261003001300-807782c586fc
|
|
125
|
+
attach: launching /Applications/ChatGPT.app/Contents/Resources/cua_node/bin/node [...]
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
If the application is missing or one of its files is writable by another account, attach is refused with the
|
|
129
|
+
reason instead, and the session simply runs without the capability.
|
|
130
|
+
|
|
147
131
|
### Optional: enable Chrome
|
|
148
132
|
|
|
149
133
|
```yaml
|
|
@@ -151,16 +135,12 @@ config:
|
|
|
151
135
|
chrome: true
|
|
152
136
|
```
|
|
153
137
|
|
|
154
|
-
|
|
138
|
+
This turns on the runtime's browser surface. The browser half of the runtime is OpenAI's, and driving a page
|
|
139
|
+
happens through the **official ChatGPT Chrome extension**, which talks to a native messaging host. On a machine
|
|
140
|
+
where the ChatGPT desktop application is installed that host is already registered, so nothing further is
|
|
141
|
+
needed: enable the extension in the profile you want to drive, and confirm it appears in `cua.getState()`.
|
|
155
142
|
|
|
156
|
-
|
|
157
|
-
~/.local/share/lcu/current/bin/lcu browser install
|
|
158
|
-
```
|
|
159
|
-
|
|
160
|
-
Enable the official ChatGPT extension in the Chrome profile you want to drive, and restart Chrome (or
|
|
161
|
-
toggle the extension at `chrome://extensions`) so it reconnects through LCU's relay rather than Codex's.
|
|
162
|
-
`lcu browser status` reports whether the connector points at this installation. Sites stay
|
|
163
|
-
exact-origin approvals.
|
|
143
|
+
Sites stay exact-origin approvals.
|
|
164
144
|
|
|
165
145
|
### Optional: pre-approve sites
|
|
166
146
|
|
|
@@ -192,12 +172,14 @@ Open Safari, go to example.com and read the page title.
|
|
|
192
172
|
|
|
193
173
|
| Field | Default | Meaning |
|
|
194
174
|
|---|---|---|
|
|
195
|
-
| `
|
|
196
|
-
| `
|
|
197
|
-
| `
|
|
175
|
+
| `app` | `/Applications/ChatGPT.app` | The application whose runtime provides computer use. |
|
|
176
|
+
| `command` | unset | An explicit executable that replaces the computed launch, for an application the built-in resolution cannot reach. It bypasses the runtime's own environment setup. |
|
|
177
|
+
| `chrome` | `false` | Enable the browser surface. |
|
|
178
|
+
| `audio` | `false` | Enable the runtime's computer-audio API. |
|
|
198
179
|
| `presets` | `["heavy"]` | Agent preset ids whose sessions get the tools. |
|
|
199
180
|
| `allowedOrigins` | `[]` | Exact HTTP(S) origins answered without asking. Invalid entries are dropped, never widened. |
|
|
200
|
-
| `
|
|
181
|
+
| `allowedApps` | `[]` | Bundle identifiers computer use may use without asking. The application hosting the agent is refused even when it is listed here. |
|
|
182
|
+
| `sectionOrder` | `0` | Prompt section order for the injected runtime instructions. |
|
|
201
183
|
|
|
202
184
|
## Approvals and the security model
|
|
203
185
|
|
|
@@ -215,35 +197,95 @@ The model cannot approve anything. Every decision is a person's:
|
|
|
215
197
|
- **Fail closed.** No question surface, a dismissed prompt, an unrecognized request shape, or an
|
|
216
198
|
aborted call all end as *cancel*, which the runtime treats as a refusal.
|
|
217
199
|
|
|
218
|
-
|
|
200
|
+
The plugin keeps no permission cache of its own; `Always allow` is remembered by the runtime, per app.
|
|
201
|
+
|
|
202
|
+
### Unattended operation
|
|
203
|
+
|
|
204
|
+
Every approval belongs to a person, and an unanswered question is a **refusal** —
|
|
205
|
+
the runtime does not default to granting. A run with nobody at the keyboard
|
|
206
|
+
therefore has to have both halves already decided:
|
|
207
|
+
|
|
208
|
+
```yaml
|
|
209
|
+
config:
|
|
210
|
+
allowedApps:
|
|
211
|
+
- com.google.Chrome # use this application without asking
|
|
212
|
+
allowedOrigins:
|
|
213
|
+
- https://example.com # and reach this exact origin without asking
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
- `allowedApps` is matched against the bundle identifier, case-insensitively. The
|
|
217
|
+
application hosting the agent is refused **before** the list is consulted, so no
|
|
218
|
+
entry can authorize it.
|
|
219
|
+
- `allowedOrigins` matches an exact origin only; a path, a trailing slash or
|
|
220
|
+
different case is a different origin and is still asked.
|
|
221
|
+
- Anything not listed is asked, and with nobody there it is declined.
|
|
222
|
+
|
|
223
|
+
The diagnostic log names every application and origin that was asked
|
|
224
|
+
(`approval: site https://example.com asking (add it to allowedOrigins to skip this)`),
|
|
225
|
+
which is how to discover the exact values a run needs.
|
|
219
226
|
|
|
220
227
|
## Understand the implementation
|
|
221
228
|
|
|
222
229
|
```
|
|
230
|
+
src/app.ts locate and validate the application; build the runtime environment; plan the launch
|
|
223
231
|
src/connection.ts the MCP client: handshake, tool discovery, calls, elicitation, lifecycle
|
|
224
232
|
src/approval.ts approval-shape recognition and label→value mapping
|
|
225
233
|
src/host-guard.ts the anti-self-approval guard
|
|
226
234
|
src/tool.ts tool definitions, text projection, durable screenshots
|
|
227
235
|
src/index.ts the plugin: per-Agent attach, instructions, turn_ended, approvals
|
|
236
|
+
src/control.ts the relay that carries the runtime's control API to and from the wrapper
|
|
237
|
+
src/session.ts the connection, following the application across an update
|
|
228
238
|
src/diag.ts the attach/approval diagnostic log
|
|
239
|
+
helper/sky-service.mjs the runtime's Sky service: the turn-ended hook and the control channel
|
|
229
240
|
```
|
|
230
241
|
|
|
231
|
-
**
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
242
|
+
**The runtime is launched directly.** `src/app.ts` finds `ChatGPT.app`, checks that the pieces it needs are
|
|
243
|
+
present and not writable by another account, and computes the environment the runtime expects — its own `node`
|
|
244
|
+
and `node_repl`, its module roots, its trusted code paths, the API surface, the signed helper it launches for
|
|
245
|
+
the macOS native pipe. It does not set `NODE_REPL_TRUSTED_SERVICES` to the application's default: see below.
|
|
246
|
+
|
|
247
|
+
**The turn-ended hook is the one thing the runtime lacks.** A turn that never ends is a turn whose
|
|
248
|
+
per-application Stop is never released, and the application then refuses every later turn. The runtime exposes
|
|
249
|
+
`addTurnEndedHandler` to its trusted services but installs no handler of its own, so
|
|
250
|
+
`helper/sky-service.mjs` is loaded **as** the `sky` trusted service: it forwards every request to the
|
|
251
|
+
application's own service unchanged and adds the hook, which asks the application to clean up a finished turn
|
|
252
|
+
through the application's own signed client.
|
|
253
|
+
|
|
254
|
+
That is deliberately less than the obvious implementation. Signalling a supervisor process, which then spawns
|
|
255
|
+
the signed client with a `turn-ended` argument, needs **Apple Events** — which macOS refuses to grant, and
|
|
256
|
+
refuses to even prompt for, when a hardened-runtime harness is the responsible process. That step always times
|
|
257
|
+
out. The application's own IPC is the step that performs the cleanup, it needs no Apple Events, and it settles
|
|
258
|
+
in tens of milliseconds. There is no supervisor process, and no lifetime socket.
|
|
259
|
+
|
|
260
|
+
**The control channel is a relay, and the plugin serves it.** Releasing one application early needs the
|
|
261
|
+
runtime's control API, which lives inside the runtime's process. The wrapper connects to a socket the plugin
|
|
262
|
+
serves, announces itself as the *service*, reports the turn contexts it has seen, and answers `status` and
|
|
263
|
+
`stop` requests there. The relay decides nothing: whether a session and turn are real, and whether an
|
|
264
|
+
application is actually held, are questions only the wrapper can answer, because only it has the turn
|
|
265
|
+
metadata the runtime keys that state by.
|
|
266
|
+
|
|
267
|
+
Two details of that channel are not guessable. The wrapper runs inside the runtime's JavaScript sandbox, which
|
|
268
|
+
refuses an ordinary socket connection with `EPERM` — for the per-user temporary directory and `/private/tmp`
|
|
269
|
+
alike — so it connects through the runtime's own `nativePipe` API instead. And that pipe **silently drops a
|
|
270
|
+
string write**; messages have to be written as buffers, which is easy to mistake for a peer that never
|
|
271
|
+
answered.
|
|
272
|
+
|
|
273
|
+
**No MCP SDK dependency.** The harness's own MCP bridge declares `capabilities: {}` and therefore cannot answer
|
|
274
|
+
elicitation — which is exactly how the runtime asks for approval — and pulling a second SDK into a profile
|
|
275
|
+
plugin would pin a version the host does not own. MCP over stdio is newline-delimited JSON-RPC, so the wire is
|
|
276
|
+
owned here. Together with type-only imports of the DSH packages, the plugin has **no runtime dependencies at
|
|
277
|
+
all**.
|
|
236
278
|
|
|
237
279
|
**Tools are registered per Agent, not at mount.** The server owns the tool schemas, so they can only be
|
|
238
|
-
fetched after the handshake. An Agent's connection is opened when the Agent is created or when it
|
|
239
|
-
|
|
240
|
-
|
|
280
|
+
fetched after the handshake. An Agent's connection is opened when the Agent is created or when it commits a
|
|
281
|
+
preset choice, and everything the plugin contributes is registered into that Agent's own context, so it unwinds
|
|
282
|
+
on disposal.
|
|
241
283
|
|
|
242
284
|
**Both preset timings are handled.** A new task is created with the deployment default and the picker's
|
|
243
285
|
choice is applied afterwards, so `agent/created` alone would see the wrong composition; the registry
|
|
244
286
|
re-emits `agent-preset/selected`, and the plugin reacts to that too.
|
|
245
287
|
|
|
246
|
-
**Lazy by construction.** Nothing starts at load time. No enabled session, no
|
|
288
|
+
**Lazy by construction.** Nothing starts at load time. No enabled session, no runtime process.
|
|
247
289
|
|
|
248
290
|
**Instructions are injected.** The server's `initialize.instructions` becomes a prompt section on the
|
|
249
291
|
Agent. It is short by design — the API manual lives in the `js` tool description and in the first tool
|
|
@@ -251,7 +293,7 @@ result.
|
|
|
251
293
|
|
|
252
294
|
## Troubleshooting
|
|
253
295
|
|
|
254
|
-
Everything the plugin decides about attaching and approving is appended to:
|
|
296
|
+
Everything the plugin decides about attaching, launching and approving is appended to:
|
|
255
297
|
|
|
256
298
|
```
|
|
257
299
|
~/.dsh/lcu-diag.log
|
|
@@ -264,17 +306,35 @@ otherwise swallowed silently.
|
|
|
264
306
|
| Symptom | Cause and fix |
|
|
265
307
|
|---|---|
|
|
266
308
|
| Tools never appear in an enabled mode | Check the log for `decide … composed=`. If the composed preset is not in `presets`, fix the allowlist. If there is no `agent-preset/selected` line, the mode was never committed. |
|
|
309
|
+
| `computer use is unavailable (…)` at load, or no tools | The application could not be resolved. The log names the reason and the path; check `app`. |
|
|
267
310
|
| `no userQuestions service -> cancel (fail closed)` | The approval surface is not mounted in this profile. |
|
|
268
311
|
| `refusing to approve the app hosting this agent` | Working as designed; ask for a different app. |
|
|
269
|
-
|
|
|
270
|
-
|
|
|
271
|
-
|
|
|
272
|
-
|
|
273
|
-
|
|
312
|
+
| The ChatGPT app still shows computer use on an app | Ask for `computer_use_stop`, or close the session — the connection owns the runtime process tree and releases it on the way out. |
|
|
313
|
+
| Chrome tabs never appear with `chrome: true` | The official extension is not enabled in that browser profile. Open `chrome://extensions`, enable it, and confirm it shows up in `cua.getState()`. |
|
|
314
|
+
| An application refuses every turn with "explicitly stopped by the user" | The host application's turn cleanup has not run. Check the log for a `turn cleanup` failure, and quit and relaunch the ChatGPT application to clear the state. |
|
|
315
|
+
|
|
316
|
+
`node scripts/probe-lcu.mjs` starts the runtime with no harness involved and prints the protocol version,
|
|
317
|
+
server identity, instructions length and the tool list — useful to separate a plugin problem from a runtime
|
|
318
|
+
problem.
|
|
319
|
+
|
|
320
|
+
`helper/sky-service.mjs` has three diagnostic switches, all default off, because it runs where the obvious
|
|
321
|
+
channels do not work: the runtime's JavaScript sandbox denies file writes, and it captures console output.
|
|
322
|
+
`DSH_SKY_DEBUG=1` traces the hook, `DSH_SKY_REPORT=1` makes the next call fail with the last cleanup outcome,
|
|
323
|
+
and `DSH_SKY_FORCE_ERROR=1` proves the module loaded at all.
|
|
324
|
+
|
|
325
|
+
## Relationship to LCU
|
|
326
|
+
|
|
327
|
+
[LCU](https://github.com/amontlabs/lcu) (MIT, Amont Labs) is the project that showed this was possible:
|
|
328
|
+
*Codex computer use, decoupled from the app*. It locates the same runtime and exposes it over MCP.
|
|
329
|
+
|
|
330
|
+
This plugin does that job itself, and no longer installs or invokes LCU. The differences that matter:
|
|
274
331
|
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
332
|
+
- **No second interpreter.** LCU's launcher is Python and requires 3.12+; this is TypeScript, and its only
|
|
333
|
+
requirement is the application itself. Attach is roughly an order of magnitude faster for it.
|
|
334
|
+
- **No supervisor process, and no Apple Events.** The turn cleanup goes through the application's own IPC
|
|
335
|
+
instead of a supervisor that shells out to the signed client.
|
|
336
|
+
- **Diagnostics at the moment of failure.** The application is resolved and reported at load, and every
|
|
337
|
+
attach, launch and approval decision is one line in the log.
|
|
278
338
|
|
|
279
339
|
## Companion tools
|
|
280
340
|
|
|
@@ -291,16 +351,15 @@ usually wants both:
|
|
|
291
351
|
## Known limitations and deferred work
|
|
292
352
|
|
|
293
353
|
- **A delegated child cannot be asked for approval.** DSH only accepts a human answer for a live
|
|
294
|
-
runtime root, so a subagent's
|
|
354
|
+
runtime root, so a subagent's approval fails closed. Subagents can perform read-only work that
|
|
295
355
|
needs no approval; anything that needs one must be driven from the top-level session.
|
|
296
|
-
- **A per-application Stop
|
|
297
|
-
host application's "using your computer" banner — ask the runtime to stop using one application
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
description says so, so a model does not stop an application on its own initiative.
|
|
356
|
+
- **A per-application Stop is released by the turn that asked for it.** `computer_use_stop` — and pressing
|
|
357
|
+
Esc on the host application's "using your computer" banner — ask the runtime to stop using one application
|
|
358
|
+
for the current turn. The plugin performs the host application's turn-ended cleanup through its own IPC, so
|
|
359
|
+
the next turn can use that application again, and `computer_use_stop` with no argument reports what is held.
|
|
360
|
+
If an application is still refused every turn, that cleanup failed and the log says so. **Ordinary use —
|
|
361
|
+
screenshots, clicks, typing, browser tabs — is unaffected**; the plugin's tool description says so, so a
|
|
362
|
+
model does not stop an application on its own initiative.
|
|
304
363
|
- **The `js` sandbox cannot write files.** Every write fails with `EPERM`, including in the temporary
|
|
305
364
|
directory, so a screenshot cannot be saved from inside `js`. Images are delivered to the harness as
|
|
306
365
|
attachments instead, and each stored image reports its host filesystem path in the tool result;
|
|
@@ -312,11 +371,15 @@ usually wants both:
|
|
|
312
371
|
Screen Recording and Accessibility must therefore be enabled for **DeepSeek Harness** in System
|
|
313
372
|
Settings; granting them only to ChatGPT or to "Codex Computer Use" is not enough, and macOS will not
|
|
314
373
|
prompt for the missing ones on its own.
|
|
315
|
-
- **
|
|
374
|
+
- **A browser surface without a Codex account is not wired up.** Driving a page works through the native
|
|
375
|
+
messaging host the desktop application registers. The relay that makes it work on a machine with no Codex
|
|
376
|
+
app present — forcing the extension's agent-request header — is not implemented.
|
|
377
|
+
- **The application updating under a running session is only warned about.** The runtime is executed from
|
|
378
|
+
files the application replaces when it updates, so a long session can run a mix of two generations.
|
|
379
|
+
Restart the harness after the application updates.
|
|
380
|
+
- **One connection per Agent.** The runtime's JavaScript session is per connection and its approvals are
|
|
316
381
|
bound to a real session and turn, so sharing one connection across Agents would interleave both.
|
|
317
|
-
-
|
|
318
|
-
`lcu browser install`, yields no browser surface.
|
|
319
|
-
- **Not verified on Linux.** See [Requirements](#requirements).
|
|
382
|
+
- **Not verified on Linux or Windows.** See [Requirements](#requirements).
|
|
320
383
|
|
|
321
384
|
## Development
|
|
322
385
|
|
|
@@ -327,14 +390,15 @@ npm run build # emits lib/
|
|
|
327
390
|
npm test # node --test, no build step
|
|
328
391
|
```
|
|
329
392
|
|
|
330
|
-
The connection
|
|
331
|
-
`npm test` is meaningful locally and still passes on CI. The
|
|
332
|
-
pure and always run.
|
|
393
|
+
The connection and launcher suites talk to the **real** installed computer-use runtime and skip themselves
|
|
394
|
+
when the application is not present, so `npm test` is meaningful locally and still passes on CI. The
|
|
395
|
+
approval, projection, guard and wrapper suites are pure and always run.
|
|
333
396
|
|
|
334
397
|
Plugin code and configuration are **not hot-reloaded** by the harness: a running process keeps the
|
|
335
398
|
module it loaded. Rebuild and restart to see a change.
|
|
336
399
|
|
|
337
400
|
## License
|
|
338
401
|
|
|
339
|
-
MIT.
|
|
340
|
-
terms and are used from
|
|
402
|
+
MIT. Nothing is vendored: the native half is the application's own, and this package has no runtime
|
|
403
|
+
dependencies. The ChatGPT application and its instructions remain under their own terms and are used from
|
|
404
|
+
your local installation.
|
package/cordis.patch.yml
CHANGED
|
@@ -1,16 +1,16 @@
|
|
|
1
1
|
# The dsh-plugin-lcu bundle patch.
|
|
2
2
|
#
|
|
3
|
-
# One row. The plugin owns every
|
|
4
|
-
# computer-use tools per Agent; it starts nothing at load time, so a
|
|
5
|
-
# for the
|
|
3
|
+
# One row. The plugin owns every runtime connection and registers the
|
|
4
|
+
# model-facing computer-use tools per Agent; it starts nothing at load time, so a
|
|
5
|
+
# session pays for the runtime only when its Agent preset is listed here.
|
|
6
6
|
#
|
|
7
7
|
# `presets` is an explicit allowlist rather than a capability the preset
|
|
8
8
|
# declares, because the server owns the tool schemas: they can only be fetched
|
|
9
9
|
# after the MCP handshake, which happens once a matching Agent is created.
|
|
10
10
|
#
|
|
11
|
-
# `chrome: true` enables
|
|
12
|
-
#
|
|
13
|
-
#
|
|
11
|
+
# `chrome: true` enables the runtime's browser surface. It also needs the official
|
|
12
|
+
# ChatGPT extension enabled in the browser profile you want to drive; sites stay
|
|
13
|
+
# exact-origin approvals.
|
|
14
14
|
|
|
15
15
|
- insert:
|
|
16
16
|
- id: lcu
|
|
@@ -19,5 +19,10 @@
|
|
|
19
19
|
presets:
|
|
20
20
|
- heavy
|
|
21
21
|
chrome: true
|
|
22
|
+
# Approvals are a person's, and an unanswered question is a refusal, so an
|
|
23
|
+
# unattended run needs both halves decided here. The diagnostic log names
|
|
24
|
+
# every application and origin that was asked, which is how to find them.
|
|
25
|
+
# allowedApps:
|
|
26
|
+
# - com.google.Chrome
|
|
22
27
|
# allowedOrigins:
|
|
23
28
|
# - http://localhost:3000
|