codex-chatgpt-control 0.1.0-alpha.1 → 0.2.0-alpha.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (242) hide show
  1. package/CHANGELOG.md +6 -0
  2. package/README.md +55 -0
  3. package/contracts/v1/fixtures/backend-capabilities.json +7 -0
  4. package/contracts/v1/fixtures/blocker-explanation-profiles.json +114 -0
  5. package/contracts/v1/fixtures/command-descriptors.json +192 -15
  6. package/contracts/v1/fixtures/doctor-bridge-upload.json +8 -0
  7. package/contracts/v1/fixtures/doctor-scenario-preflight.json +108 -0
  8. package/contracts/v1/fixtures/existing-tab-diagnostics-blocker.json +57 -0
  9. package/contracts/v1/fixtures/files-preflight-success.json +32 -0
  10. package/contracts/v1/fixtures/help-root.json +1 -1
  11. package/contracts/v1/fixtures/output-json-parse-success.json +17 -2
  12. package/contracts/v1/fixtures/project-sources-plan-add.json +71 -0
  13. package/contracts/v1/fixtures/read-latest-clipped.json +31 -0
  14. package/contracts/v1/fixtures/reports-create-redacted.json +15 -0
  15. package/contracts/v1/fixtures/responses-basic-success.json +11 -0
  16. package/contracts/v1/fixtures/run-basic-success.json +15 -0
  17. package/contracts/v1/fixtures/run-browser-bridge-blocker.json +2 -2
  18. package/contracts/v1/fixtures/run-timeout-partial.json +2 -2
  19. package/contracts/v1/fixtures/runner-budget-blocker.json +1 -1
  20. package/contracts/v1/fixtures/stream-blocked.ndjson +1 -1
  21. package/contracts/v1/fixtures/stream-submitted-completed.ndjson +1 -1
  22. package/contracts/v1/manifest.json +30 -0
  23. package/contracts/v1/parity-suite.json +72 -6
  24. package/contracts/v1/schemas/backend-request.schema.json +3 -0
  25. package/contracts/v1/schemas/command-result.schema.json +48 -0
  26. package/contracts/v1/schemas/response.schema.json +28 -0
  27. package/contracts/v1/schemas/run-result.schema.json +84 -0
  28. package/contracts/v1/surface-drift-policy.json +210 -0
  29. package/dist/codex-chatgpt-control-backend.mjs +5137 -488
  30. package/dist/codex-chatgpt-control.bundle.mjs +5001 -286
  31. package/dist/src/backend/client.d.ts +6 -1
  32. package/dist/src/backend/client.js +5 -0
  33. package/dist/src/backend/protocol.d.ts +1 -1
  34. package/dist/src/backend/protocol.js +7 -0
  35. package/dist/src/backend/session.js +14 -0
  36. package/dist/src/browser/attach.js +128 -20
  37. package/dist/src/browser/downloads.js +2 -1
  38. package/dist/src/browser/page-state.js +6 -3
  39. package/dist/src/client.d.ts +16 -1
  40. package/dist/src/client.js +86 -7
  41. package/dist/src/commands/artifacts.d.ts +5 -0
  42. package/dist/src/commands/artifacts.js +432 -0
  43. package/dist/src/commands/context.js +5 -2
  44. package/dist/src/commands/doctor.d.ts +10 -2
  45. package/dist/src/commands/doctor.js +242 -14
  46. package/dist/src/commands/files.d.ts +2 -1
  47. package/dist/src/commands/files.js +352 -18
  48. package/dist/src/commands/messages.d.ts +2 -1
  49. package/dist/src/commands/messages.js +232 -40
  50. package/dist/src/commands/modes.js +202 -20
  51. package/dist/src/commands/project-sources.d.ts +10 -0
  52. package/dist/src/commands/project-sources.js +643 -0
  53. package/dist/src/commands/registry.js +76 -14
  54. package/dist/src/commands/reports.d.ts +9 -0
  55. package/dist/src/commands/reports.js +83 -6
  56. package/dist/src/commands/response-actions.js +33 -0
  57. package/dist/src/commands/sequence.js +14 -0
  58. package/dist/src/commands/threads.js +2 -1
  59. package/dist/src/commands/timeouts.d.ts +2 -0
  60. package/dist/src/commands/timeouts.js +18 -0
  61. package/dist/src/diagnostics/blockers.d.ts +54 -0
  62. package/dist/src/diagnostics/blockers.js +299 -0
  63. package/dist/src/dom/artifacts.d.ts +7 -0
  64. package/dist/src/dom/artifacts.js +198 -0
  65. package/dist/src/dom/generation-state.d.ts +8 -0
  66. package/dist/src/dom/generation-state.js +103 -0
  67. package/dist/src/dom/locale/am.d.ts +28 -0
  68. package/dist/src/dom/locale/am.js +28 -0
  69. package/dist/src/dom/locale/ar.d.ts +28 -0
  70. package/dist/src/dom/locale/ar.js +28 -0
  71. package/dist/src/dom/locale/bg.d.ts +28 -0
  72. package/dist/src/dom/locale/bg.js +28 -0
  73. package/dist/src/dom/locale/bn.d.ts +28 -0
  74. package/dist/src/dom/locale/bn.js +28 -0
  75. package/dist/src/dom/locale/bs.d.ts +28 -0
  76. package/dist/src/dom/locale/bs.js +28 -0
  77. package/dist/src/dom/locale/ca.d.ts +28 -0
  78. package/dist/src/dom/locale/ca.js +28 -0
  79. package/dist/src/dom/locale/cs.d.ts +28 -0
  80. package/dist/src/dom/locale/cs.js +28 -0
  81. package/dist/src/dom/locale/da.d.ts +28 -0
  82. package/dist/src/dom/locale/da.js +28 -0
  83. package/dist/src/dom/locale/de.d.ts +30 -0
  84. package/dist/src/dom/locale/de.js +30 -0
  85. package/dist/src/dom/locale/el.d.ts +28 -0
  86. package/dist/src/dom/locale/el.js +28 -0
  87. package/dist/src/dom/locale/en.d.ts +50 -0
  88. package/dist/src/dom/locale/en.js +56 -0
  89. package/dist/src/dom/locale/es-419.d.ts +28 -0
  90. package/dist/src/dom/locale/es-419.js +28 -0
  91. package/dist/src/dom/locale/es-ES.d.ts +28 -0
  92. package/dist/src/dom/locale/es-ES.js +28 -0
  93. package/dist/src/dom/locale/et.d.ts +28 -0
  94. package/dist/src/dom/locale/et.js +28 -0
  95. package/dist/src/dom/locale/fa.d.ts +28 -0
  96. package/dist/src/dom/locale/fa.js +28 -0
  97. package/dist/src/dom/locale/fi.d.ts +28 -0
  98. package/dist/src/dom/locale/fi.js +28 -0
  99. package/dist/src/dom/locale/fr-CA.d.ts +29 -0
  100. package/dist/src/dom/locale/fr-CA.js +29 -0
  101. package/dist/src/dom/locale/fr-FR.d.ts +30 -0
  102. package/dist/src/dom/locale/fr-FR.js +30 -0
  103. package/dist/src/dom/locale/gu.d.ts +28 -0
  104. package/dist/src/dom/locale/gu.js +28 -0
  105. package/dist/src/dom/locale/hi.d.ts +28 -0
  106. package/dist/src/dom/locale/hi.js +28 -0
  107. package/dist/src/dom/locale/hr.d.ts +28 -0
  108. package/dist/src/dom/locale/hr.js +28 -0
  109. package/dist/src/dom/locale/hu.d.ts +28 -0
  110. package/dist/src/dom/locale/hu.js +28 -0
  111. package/dist/src/dom/locale/hy.d.ts +28 -0
  112. package/dist/src/dom/locale/hy.js +28 -0
  113. package/dist/src/dom/locale/id.d.ts +28 -0
  114. package/dist/src/dom/locale/id.js +28 -0
  115. package/dist/src/dom/locale/index.d.ts +79 -0
  116. package/dist/src/dom/locale/index.js +202 -0
  117. package/dist/src/dom/locale/is.d.ts +28 -0
  118. package/dist/src/dom/locale/is.js +28 -0
  119. package/dist/src/dom/locale/it.d.ts +27 -0
  120. package/dist/src/dom/locale/it.js +27 -0
  121. package/dist/src/dom/locale/ja.d.ts +27 -0
  122. package/dist/src/dom/locale/ja.js +27 -0
  123. package/dist/src/dom/locale/ka.d.ts +28 -0
  124. package/dist/src/dom/locale/ka.js +28 -0
  125. package/dist/src/dom/locale/kk.d.ts +28 -0
  126. package/dist/src/dom/locale/kk.js +28 -0
  127. package/dist/src/dom/locale/kn.d.ts +28 -0
  128. package/dist/src/dom/locale/kn.js +28 -0
  129. package/dist/src/dom/locale/ko.d.ts +28 -0
  130. package/dist/src/dom/locale/ko.js +28 -0
  131. package/dist/src/dom/locale/lt.d.ts +28 -0
  132. package/dist/src/dom/locale/lt.js +28 -0
  133. package/dist/src/dom/locale/lv.d.ts +28 -0
  134. package/dist/src/dom/locale/lv.js +28 -0
  135. package/dist/src/dom/locale/mk.d.ts +28 -0
  136. package/dist/src/dom/locale/mk.js +28 -0
  137. package/dist/src/dom/locale/ml.d.ts +28 -0
  138. package/dist/src/dom/locale/ml.js +28 -0
  139. package/dist/src/dom/locale/mn.d.ts +28 -0
  140. package/dist/src/dom/locale/mn.js +28 -0
  141. package/dist/src/dom/locale/mr.d.ts +28 -0
  142. package/dist/src/dom/locale/mr.js +28 -0
  143. package/dist/src/dom/locale/ms.d.ts +28 -0
  144. package/dist/src/dom/locale/ms.js +28 -0
  145. package/dist/src/dom/locale/my.d.ts +28 -0
  146. package/dist/src/dom/locale/my.js +28 -0
  147. package/dist/src/dom/locale/nb.d.ts +28 -0
  148. package/dist/src/dom/locale/nb.js +28 -0
  149. package/dist/src/dom/locale/nl.d.ts +28 -0
  150. package/dist/src/dom/locale/nl.js +28 -0
  151. package/dist/src/dom/locale/pa.d.ts +28 -0
  152. package/dist/src/dom/locale/pa.js +28 -0
  153. package/dist/src/dom/locale/pl.d.ts +28 -0
  154. package/dist/src/dom/locale/pl.js +28 -0
  155. package/dist/src/dom/locale/pt-BR.d.ts +28 -0
  156. package/dist/src/dom/locale/pt-BR.js +28 -0
  157. package/dist/src/dom/locale/pt-PT.d.ts +28 -0
  158. package/dist/src/dom/locale/pt-PT.js +28 -0
  159. package/dist/src/dom/locale/ro.d.ts +28 -0
  160. package/dist/src/dom/locale/ro.js +28 -0
  161. package/dist/src/dom/locale/ru.d.ts +28 -0
  162. package/dist/src/dom/locale/ru.js +28 -0
  163. package/dist/src/dom/locale/sk.d.ts +28 -0
  164. package/dist/src/dom/locale/sk.js +28 -0
  165. package/dist/src/dom/locale/sl.d.ts +28 -0
  166. package/dist/src/dom/locale/sl.js +28 -0
  167. package/dist/src/dom/locale/so.d.ts +28 -0
  168. package/dist/src/dom/locale/so.js +28 -0
  169. package/dist/src/dom/locale/sq.d.ts +28 -0
  170. package/dist/src/dom/locale/sq.js +28 -0
  171. package/dist/src/dom/locale/sr.d.ts +28 -0
  172. package/dist/src/dom/locale/sr.js +28 -0
  173. package/dist/src/dom/locale/sv.d.ts +28 -0
  174. package/dist/src/dom/locale/sv.js +28 -0
  175. package/dist/src/dom/locale/sw.d.ts +28 -0
  176. package/dist/src/dom/locale/sw.js +28 -0
  177. package/dist/src/dom/locale/ta.d.ts +28 -0
  178. package/dist/src/dom/locale/ta.js +28 -0
  179. package/dist/src/dom/locale/te.d.ts +28 -0
  180. package/dist/src/dom/locale/te.js +28 -0
  181. package/dist/src/dom/locale/th.d.ts +28 -0
  182. package/dist/src/dom/locale/th.js +28 -0
  183. package/dist/src/dom/locale/tl.d.ts +28 -0
  184. package/dist/src/dom/locale/tl.js +28 -0
  185. package/dist/src/dom/locale/tr.d.ts +28 -0
  186. package/dist/src/dom/locale/tr.js +28 -0
  187. package/dist/src/dom/locale/types.d.ts +45 -0
  188. package/dist/src/dom/locale/types.js +1 -0
  189. package/dist/src/dom/locale/uk.d.ts +28 -0
  190. package/dist/src/dom/locale/uk.js +28 -0
  191. package/dist/src/dom/locale/ur.d.ts +28 -0
  192. package/dist/src/dom/locale/ur.js +28 -0
  193. package/dist/src/dom/locale/vi.d.ts +28 -0
  194. package/dist/src/dom/locale/vi.js +28 -0
  195. package/dist/src/dom/locale/zh-HK.d.ts +28 -0
  196. package/dist/src/dom/locale/zh-HK.js +28 -0
  197. package/dist/src/dom/locale/zh-Hans.d.ts +28 -0
  198. package/dist/src/dom/locale/zh-Hans.js +28 -0
  199. package/dist/src/dom/locale/zh-TW.d.ts +29 -0
  200. package/dist/src/dom/locale/zh-TW.js +29 -0
  201. package/dist/src/dom/locale-labels.d.ts +26 -0
  202. package/dist/src/dom/locale-labels.js +26 -0
  203. package/dist/src/dom/menus.d.ts +6 -0
  204. package/dist/src/dom/menus.js +34 -8
  205. package/dist/src/dom/message-format.d.ts +2 -1
  206. package/dist/src/dom/message-format.js +67 -10
  207. package/dist/src/dom/messages.d.ts +2 -1
  208. package/dist/src/dom/messages.js +2 -4
  209. package/dist/src/dom/selectors.d.ts +1 -0
  210. package/dist/src/dom/selectors.js +42 -15
  211. package/dist/src/index.d.ts +5 -0
  212. package/dist/src/index.js +5 -0
  213. package/dist/src/platform/local-paths.d.ts +4 -0
  214. package/dist/src/platform/local-paths.js +20 -0
  215. package/dist/src/runner/interruptions.js +15 -4
  216. package/dist/src/runner/responses.js +18 -1
  217. package/dist/src/runner/result.js +15 -0
  218. package/dist/src/runner/resume.js +1 -0
  219. package/dist/src/runner/types.d.ts +3 -0
  220. package/dist/src/safety/risk.d.ts +7 -0
  221. package/dist/src/safety/risk.js +7 -0
  222. package/dist/src/safety/untrusted-output.d.ts +73 -0
  223. package/dist/src/safety/untrusted-output.js +199 -0
  224. package/dist/src/scripts/apply-intelligence-locale-captures.d.ts +1 -0
  225. package/dist/src/scripts/apply-intelligence-locale-captures.js +184 -0
  226. package/dist/src/scripts/capture-intelligence-locales.d.ts +25 -0
  227. package/dist/src/scripts/capture-intelligence-locales.js +669 -0
  228. package/dist/src/scripts/check-surface-drift.d.ts +1 -0
  229. package/dist/src/scripts/check-surface-drift.js +11 -0
  230. package/dist/src/scripts/live-smoke/scenarios.js +84 -0
  231. package/dist/src/scripts/locale-capture/language-coverage.d.ts +10 -0
  232. package/dist/src/scripts/locale-capture/language-coverage.js +39 -0
  233. package/dist/src/testing/surface-drift.d.ts +66 -0
  234. package/dist/src/testing/surface-drift.js +363 -0
  235. package/dist/src/types.d.ts +201 -2
  236. package/package.json +2 -1
  237. package/references/backend-protocol.md +93 -1
  238. package/references/language-coverage.md +142 -0
  239. package/references/localization.md +247 -0
  240. package/references/python-parity.md +63 -1
  241. package/references/safety.md +2 -0
  242. package/references/troubleshooting.md +58 -0
@@ -1,5 +1,5 @@
1
1
  export type CommandStatus = "ok" | "partial" | "timeout" | "blocked" | "needs_confirmation" | "not_found" | "unsupported" | "error";
2
- export type BlockerKind = "browser_bridge_unavailable" | "login_required" | "captcha" | "rate_limit" | "modal" | "permission" | "confirmation" | "selector_drift" | "download_unavailable" | "upload_failed" | "not_found" | "unknown";
2
+ export type BlockerKind = "browser_bridge_unavailable" | "login_required" | "captcha" | "rate_limit" | "modal" | "permission" | "confirmation" | "selector_drift" | "artifact_unavailable" | "artifact_selector_drift" | "artifact_download_unavailable" | "download_unavailable" | "upload_failed" | "not_found" | "unknown";
3
3
  export type CommandContext = {
4
4
  url?: string;
5
5
  conversationId?: string;
@@ -10,6 +10,32 @@ export type CommandContext = {
10
10
  tabId?: string;
11
11
  timestamp: string;
12
12
  };
13
+ export type ExistingTabMismatchReason = "no_candidate" | "multiple_candidates" | "non_chatgpt_tab" | "conversation_id_mismatch" | "url_mismatch" | "title_mismatch" | "selected_tab_unavailable" | "explicit_tab_id_not_open" | "user_open_tabs_unavailable";
14
+ export type ExistingTabDiagnosticTarget = {
15
+ type: string;
16
+ host?: string;
17
+ tabId?: string;
18
+ conversationId?: string;
19
+ url?: string;
20
+ title?: string;
21
+ exact?: boolean;
22
+ };
23
+ export type ExistingTabDiagnosticCandidate = {
24
+ id: string;
25
+ url?: string;
26
+ title?: string;
27
+ conversationId?: string;
28
+ lastOpened?: string;
29
+ tabGroup?: string;
30
+ };
31
+ export type ExistingTabDiagnostics = {
32
+ requestedTarget: ExistingTabDiagnosticTarget;
33
+ userOpenTabsAvailable: boolean;
34
+ chatgptTabCount: number;
35
+ mismatchReason: ExistingTabMismatchReason;
36
+ candidateTabs: ExistingTabDiagnosticCandidate[];
37
+ omittedCandidateCount?: number;
38
+ };
13
39
  export type CommandResult<T = unknown> = {
14
40
  ok: boolean;
15
41
  status: CommandStatus;
@@ -37,6 +63,9 @@ export type CommandResult<T = unknown> = {
37
63
  label: string;
38
64
  role?: string;
39
65
  }>;
66
+ diagnostics?: {
67
+ existingTab?: ExistingTabDiagnostics;
68
+ };
40
69
  resumable?: boolean;
41
70
  };
42
71
  context: CommandContext;
@@ -176,6 +205,11 @@ export type ResponseTable = {
176
205
  };
177
206
  export type ResponseCaptureSource = "semantic_dom" | "clipboard";
178
207
  export type ResponseCaptureFidelity = "clipboard_markdown" | "semantic_markdown" | "visible_text" | "normalized_text" | "html" | "blocks" | "all";
208
+ export type ResponseCaptureLimit = {
209
+ maxChars: number;
210
+ originalChars: number;
211
+ clipped: boolean;
212
+ };
179
213
  export type ResponseBranchState = {
180
214
  current?: number;
181
215
  total?: number;
@@ -225,6 +259,7 @@ export type ReadLatestData = {
225
259
  format: Exclude<ResponseFormat, "text">;
226
260
  source?: ResponseCaptureSource;
227
261
  fidelity?: ResponseCaptureFidelity;
262
+ captureLimit?: ResponseCaptureLimit;
228
263
  warnings?: string[];
229
264
  markdown?: string;
230
265
  visibleText?: string;
@@ -262,9 +297,78 @@ export type AttachedFile = {
262
297
  name: string;
263
298
  bytes: number;
264
299
  };
300
+ export type FileCategory = "text" | "document" | "spreadsheet" | "data" | "image" | "audio" | "video" | "archive" | "unknown";
301
+ export type FilePreflightArgs = {
302
+ paths: string[];
303
+ maxBytesPerFile?: number;
304
+ maxTotalBytes?: number;
305
+ };
306
+ export type FilePreflightFile = AttachedFile & {
307
+ extension: string;
308
+ mimeType: string;
309
+ category: FileCategory;
310
+ };
311
+ export type FilePreflightData = {
312
+ files: FilePreflightFile[];
313
+ totalBytes: number;
314
+ };
265
315
  export type AttachFilesData = {
266
316
  files: AttachedFile[];
267
317
  };
318
+ export type ProjectSourceStatus = "ready" | "processing" | "failed" | "unknown";
319
+ export type ProjectSource = {
320
+ name: string;
321
+ status: ProjectSourceStatus;
322
+ };
323
+ export type ProjectSourcesUrl = {
324
+ projectId: string;
325
+ projectSlug?: string;
326
+ url: string;
327
+ };
328
+ export type ProjectSourcesListArgs = {
329
+ projectUrl: string;
330
+ existingTab?: boolean | ExistingTabPolicy;
331
+ preferExistingTab?: boolean;
332
+ timeoutMs?: number;
333
+ };
334
+ export type ProjectSourcesListData = ProjectSourcesUrl & {
335
+ sources: ProjectSource[];
336
+ };
337
+ export type ProjectSourcePlanFile = FilePreflightFile & {
338
+ displayPath: string;
339
+ };
340
+ export type ProjectSourceUploadBatch = {
341
+ index: number;
342
+ files: ProjectSourcePlanFile[];
343
+ totalBytes: number;
344
+ };
345
+ export type ProjectSourcesPlanAddArgs = {
346
+ projectUrl: string;
347
+ files: string[];
348
+ batchSize?: number;
349
+ maxBytesPerFile?: number;
350
+ maxTotalBytes?: number;
351
+ };
352
+ export type ProjectSourcesAddPlanData = ProjectSourcesUrl & {
353
+ projectUrl: string;
354
+ operation: "append_add";
355
+ dryRun: true;
356
+ files: ProjectSourcePlanFile[];
357
+ batches: ProjectSourceUploadBatch[];
358
+ totalBytes: number;
359
+ };
360
+ export type ProjectSourcesAddArgs = ProjectSourcesPlanAddArgs & {
361
+ confirmMutation?: boolean;
362
+ existingTab?: boolean | ExistingTabPolicy;
363
+ preferExistingTab?: boolean;
364
+ timeoutMs?: number;
365
+ };
366
+ export type ProjectSourcesAddData = Omit<ProjectSourcesAddPlanData, "dryRun"> & {
367
+ dryRun: false;
368
+ before: ProjectSource[];
369
+ after: ProjectSource[];
370
+ added: ProjectSource[];
371
+ };
268
372
  export type DownloadLatestArgs = {
269
373
  destDir: string;
270
374
  filenamePattern?: string;
@@ -278,6 +382,47 @@ export type DownloadedFile = {
278
382
  suggestedFilename?: string;
279
383
  bytes: number;
280
384
  };
385
+ export type ArtifactKind = "image";
386
+ export type GeneratedArtifact = {
387
+ kind: ArtifactKind;
388
+ index: number;
389
+ visible: boolean;
390
+ width?: number;
391
+ height?: number;
392
+ alt?: string;
393
+ ariaLabel?: string;
394
+ src?: string;
395
+ turnId?: string;
396
+ downloadAvailable: boolean;
397
+ selectorProvenance: string;
398
+ };
399
+ export type ListArtifactsArgs = {
400
+ kind?: ArtifactKind;
401
+ max?: number;
402
+ timeoutMs?: number;
403
+ };
404
+ export type ArtifactListData = {
405
+ count: number;
406
+ artifacts: GeneratedArtifact[];
407
+ latest?: GeneratedArtifact;
408
+ };
409
+ export type ArtifactWaitArgs = ListArtifactsArgs & {
410
+ afterArtifactCount?: number;
411
+ requireDownload?: boolean;
412
+ timeoutMs?: number;
413
+ stableMs?: number;
414
+ pollMs?: number;
415
+ };
416
+ export type ArtifactWaitData = {
417
+ complete: boolean;
418
+ count: number;
419
+ latest?: GeneratedArtifact;
420
+ elapsedMs: number;
421
+ };
422
+ export type ArtifactDownloadArgs = DownloadLatestArgs & {
423
+ kind?: ArtifactKind;
424
+ prefer?: "download_control" | "visible_image_source";
425
+ };
281
426
  export type CopyResponseArgs = {
282
427
  which?: "latest" | {
283
428
  assistantIndex: number;
@@ -291,6 +436,7 @@ export type CopiedResponse = {
291
436
  source: "clipboard" | "dom";
292
437
  format: Exclude<ResponseFormat, "text">;
293
438
  fidelity?: ResponseCaptureFidelity;
439
+ captureLimit?: ResponseCaptureLimit;
294
440
  warnings?: string[];
295
441
  markdown?: string;
296
442
  visibleText?: string;
@@ -309,6 +455,9 @@ export type CopiedResponse = {
309
455
  export type SetModeArgs = {
310
456
  model?: string;
311
457
  effort?: string;
458
+ intelligence?: string;
459
+ modelVersion?: string;
460
+ version?: string;
312
461
  timeoutMs?: number;
313
462
  };
314
463
  export type SelectToolArgs = {
@@ -355,6 +504,18 @@ export type SequenceStep = {
355
504
  id: string;
356
505
  command: "messages.waitAndRead";
357
506
  args: WaitAndReadArgs;
507
+ } | {
508
+ id: string;
509
+ command: "artifacts.listLatest";
510
+ args?: ListArtifactsArgs;
511
+ } | {
512
+ id: string;
513
+ command: "artifacts.wait";
514
+ args?: ArtifactWaitArgs;
515
+ } | {
516
+ id: string;
517
+ command: "artifacts.downloadLatest";
518
+ args: ArtifactDownloadArgs;
358
519
  } | {
359
520
  id: string;
360
521
  command: "files.attach";
@@ -363,6 +524,18 @@ export type SequenceStep = {
363
524
  id: string;
364
525
  command: "files.downloadLatest";
365
526
  args: DownloadLatestArgs;
527
+ } | {
528
+ id: string;
529
+ command: "projects.sources.list";
530
+ args: ProjectSourcesListArgs;
531
+ } | {
532
+ id: string;
533
+ command: "projects.sources.planAdd";
534
+ args: ProjectSourcesPlanAddArgs;
535
+ } | {
536
+ id: string;
537
+ command: "projects.sources.add";
538
+ args: ProjectSourcesAddArgs;
366
539
  } | {
367
540
  id: string;
368
541
  command: "response.copy";
@@ -411,10 +584,13 @@ export type BrowserLike = {
411
584
  };
412
585
  tabs?: {
413
586
  create?: (url: string) => Promise<PageLike> | PageLike;
414
- new?: (url: string) => Promise<PageLike> | PageLike;
587
+ new?: (url?: string) => Promise<PageLike> | PageLike;
415
588
  selected?: () => Promise<PageLike | undefined> | PageLike | undefined;
416
589
  list?: () => Promise<PageLike[]> | PageLike[];
417
590
  get?: (id: string) => Promise<PageLike> | PageLike;
591
+ finalize?: (options: {
592
+ keep?: unknown[];
593
+ }) => Promise<void>;
418
594
  };
419
595
  newPage?: () => Promise<PageLike> | PageLike;
420
596
  };
@@ -462,10 +638,33 @@ export type PageLike = {
462
638
  keyboard?: {
463
639
  press?: (key: string) => Promise<void>;
464
640
  };
641
+ mouse?: {
642
+ move?: (x: number, y: number) => Promise<void> | void;
643
+ click?: (x: number, y: number) => Promise<void> | void;
644
+ };
645
+ cua?: {
646
+ move?: (options: {
647
+ x: number;
648
+ y: number;
649
+ }) => Promise<void> | void;
650
+ click?: (options: {
651
+ x: number;
652
+ y: number;
653
+ button?: number;
654
+ }) => Promise<void> | void;
655
+ };
465
656
  waitForTimeout?: (ms: number) => Promise<void>;
466
657
  waitForEvent?: (event: string, optionsOrCallback?: WaitForEventOptions | unknown) => Promise<unknown>;
467
658
  evaluate?: <T, A = unknown>(fn: (arg: A) => T | Promise<T>, arg?: A) => Promise<T>;
468
659
  content?: () => Promise<string>;
660
+ close?: () => Promise<void>;
661
+ capabilities?: {
662
+ get?: (id: string) => Promise<unknown> | unknown;
663
+ };
664
+ playwright?: {
665
+ waitForTimeout?: (ms: number) => Promise<void>;
666
+ [key: string]: unknown;
667
+ };
469
668
  };
470
669
  export type AskHelperArgs = AskArgs & {
471
670
  thread?: ThreadTarget;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "codex-chatgpt-control",
3
- "version": "0.1.0-alpha.1",
3
+ "version": "0.2.0-alpha.1",
4
4
  "description": "Unofficial SDK for Codex agents controlling visible ChatGPT web sessions.",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -27,6 +27,7 @@
27
27
  "bundle:backend": "esbuild src/scripts/backend-server.ts --bundle --platform=node --format=esm --outfile=dist/codex-chatgpt-control-backend.mjs",
28
28
  "bundle:live-smoke": "esbuild src/scripts/live-smoke-module.ts --bundle --platform=node --format=esm --outfile=dist/codex-chatgpt-control-live-smoke.bundle.mjs",
29
29
  "contract:validate": "node scripts/validate-contract.mjs",
30
+ "docs:drift": "tsx src/scripts/check-surface-drift.ts",
30
31
  "parity:fixtures": "node scripts/check-parity-fixtures.mjs",
31
32
  "parity:suite": "tsx src/scripts/check-parity-suite.ts",
32
33
  "test:backend-conformance": "vitest run tests/unit/backend-conformance.test.ts",
@@ -67,6 +67,8 @@ Current protocol error codes are:
67
67
 
68
68
  Browser-control blockers are not protocol errors. They are normal command or runner results with `status: "blocked"`, `status: "partial"`, or `status: "needs_confirmation"` plus blocker/interruption details.
69
69
 
70
+ `status: "partial"` is the required result for incomplete response capture. A partial result may still include `output_text` and `data.responseText`, but consumers must treat that text as incomplete until a later wait confirms completion. Common causes are wait timeout after partial assistant text, active generation controls such as `Stop answering`, stopped-generation markers such as `Stopped thinking`, or a read fallback after the wait step could not confirm completion. Intentional capture clipping uses `data.captureLimit` plus warnings; it is separate from ChatGPT generation length.
71
+
70
72
  ## Streaming
71
73
 
72
74
  Streaming commands emit backend event lines until `completed` or `error`.
@@ -107,9 +109,54 @@ The backend must support:
107
109
  - runner: `runner.run`, `runner.plan`, `runner.stream`
108
110
  - Responses adapter: `responses.create`
109
111
  - workflows: `ask`, `askInThread`, `askWithFiles`, `askAndDownload`, `runMessages`, `openThread`, `runPlan`
112
+ - diagnostics: `doctor`
110
113
  - reports: `createReport`, `reports.create`, `reports.redact`, `reports.summarize`
111
114
  - command discovery: `commands`, `describe`, `help`
112
- - primitives: `session.bootstrap`, `threads.*`, `messages.*`, `files.*`, `modes.set`, `tools.select`, `response.copy`
115
+ - primitives: `session.bootstrap`, `threads.*`, `messages.*`, `artifacts.*`, `files.preflight`, `files.attach`, `files.downloadLatest`, `projects.sources.list`, `projects.sources.planAdd`, `projects.sources.add`, `modes.set`, `tools.select`, `response.copy`
116
+
117
+ `doctor` returns a normal `CommandResult` whose `data.checks` map is extensible. Scenario checks such as `existing_tab`, `artifacts`, `file_preflight`, `localization`, and `reports` may add optional `code`, `blockerKind`, `nextCommand`, and JSON `details` fields to individual check entries while preserving the existing `status`, `message`, and `remediation` fields.
118
+
119
+ ## Host-Local Attachment Paths
120
+
121
+ Attachment paths are interpreted on the machine running the Node backend. Use an absolute path in that host operating system's native form. On macOS/Linux/WSL, use paths such as `/example/user/file.pdf`, `/home/you/file.pdf`, or `/mnt/c/example/user/file.pdf`. On Windows backend hosts, use fully qualified paths such as `C:\Users\you\file.pdf` or UNC paths such as `\\server\share\file.pdf`. Drive-relative paths like `C:Users\you\file.pdf`, root-relative paths like `\tmp\file.pdf`, and Windows-looking paths sent to a POSIX backend are rejected before filesystem access.
122
+
123
+ Use `files.preflight` for non-mutating local validation before browser upload workflows. It validates absolute paths, existence, readability, file-vs-directory status, configurable per-file and total byte limits, duplicate basenames, duplicate resolved paths, zero-byte files, and extension-based MIME/category guesses. It does not open ChatGPT, perform a live upload, or read file contents for MIME detection. `askWithFiles` and `files.attach` run the same preflight before upload attempts so obvious local file failures stop before browser interaction.
124
+
125
+ ## Project Sources
126
+
127
+ Project Sources V1 is a narrow visible-UI surface for ChatGPT Project URLs such as `https://chatgpt.com/g/g-p-.../project`.
128
+
129
+ - `projects.sources.list` opens or claims the Project Sources UI and returns source names plus coarse statuses only.
130
+ - `projects.sources.planAdd` validates explicit local file paths with the same metadata-only preflight used by `files.preflight`, batches the upload plan, and does not open ChatGPT.
131
+ - `projects.sources.add` is append-only and requires `confirmMutation: true`; without it, the command returns a `needs_confirmation` blocker with the dry-run plan.
132
+
133
+ The implementation normalizes nested project chat URLs back to the project page before operating. It does not delete, replace, sync repository trees, read cookies/storage/auth headers, call private ChatGPT endpoints, or read source file contents while planning. Confirmed upload uses visible browser file chooser primitives and diffs before/after source names to report added sources.
134
+
135
+ ## Generated Artifacts
136
+
137
+ Generated images are represented as visible artifacts, not assistant text. A
138
+ ChatGPT image-only result can be complete even when `messages.readLatest` returns
139
+ `not_found` and `assistantTurnCount` is `0`.
140
+
141
+ Use the artifact primitives for this surface:
142
+
143
+ - `artifacts.listLatest` detects visible generated artifacts.
144
+ - `artifacts.wait` waits for a generated artifact to appear and stabilize.
145
+ - `artifacts.downloadLatest` downloads via a visible artifact affordance, or
146
+ saves a visible image source when no browser download event fires.
147
+
148
+ When a claimed user-open ChatGPT tab stalls bridge page inspection, artifact
149
+ commands may recover by opening the same saved `https://chatgpt.com/c/...`
150
+ conversation in a temporary bridge-owned tab and using the bridge `pageAssets`
151
+ capability to inventory and bundle the latest non-SVG image asset. This is an
152
+ implementation detail of the TypeScript runtime; the wire command and result
153
+ shape are unchanged.
154
+
155
+ `files.downloadLatest` preserves the existing file-link behavior and falls back
156
+ to generated-artifact download only when no conventional ChatGPT file affordance
157
+ is visible. Artifact failures are reported as structured blockers such as
158
+ `artifact_unavailable`, `artifact_selector_drift`, or
159
+ `artifact_download_unavailable`, not protocol errors.
113
160
 
114
161
  `session.bootstrap` accepts `existingTab` for explicit reuse of a user-open Chrome tab before any read or prompt step. The wire shape is shared by TypeScript and Python:
115
162
 
@@ -219,6 +266,49 @@ Python SDK -> stdio relay -> bridge-hosted Node backend -> Codex Chrome bridge -
219
266
 
220
267
  The smoke covers `runner.run`, `runner.run_streamed`, `responses.create`, named `run_plan`, and redacted `reports.create`. It writes redacted JSON summaries and does not persist raw prompt/response content by default.
221
268
 
269
+ ## Untrusted Output And Integrity
270
+
271
+ Assistant output captured from ChatGPT is untrusted third-party content. Runner results with non-empty `output_text` expose `data.untrustedOutput`; Responses-shaped results expose the same object at `browser_control.untrustedOutput`.
272
+
273
+ The envelope schema is:
274
+
275
+ ```json
276
+ {
277
+ "schemaVersion": "chatgpt.browser_control.untrusted_output_return.v1",
278
+ "trusted": false,
279
+ "source": "chatgpt",
280
+ "capturedAt": "2026-06-06T00:00:00.000Z",
281
+ "contentSha256": "8f434346648f6b96df89dda901c5176b10a6d83961dd3c1ac88b59b2dc327aa4",
282
+ "contentBytes": 2,
283
+ "inline": true,
284
+ "maxInlineBytes": 12000,
285
+ "rendered": "UNTRUSTED OUTPUT RETURN ENVELOPE\n..."
286
+ }
287
+ ```
288
+
289
+ Use `rendered` when handing the captured answer to another agent, tool, or prompt. It places routing and hash metadata before the body, tells the consumer not to execute embedded instructions, and uses a markdown fence longer than any backtick run inside the content. Outputs larger than the inline byte guard are not inlined; the envelope points at the persisted output path when one is available.
290
+
291
+ Run report creation writes a sibling `*.meta.json` sidecar by default:
292
+
293
+ ```json
294
+ {
295
+ "schemaVersion": "chatgpt.browser_control.integrity.v1",
296
+ "target": {
297
+ "path": "reports/runs/run.json",
298
+ "bytes": 123,
299
+ "sha256": "..."
300
+ },
301
+ "output": {
302
+ "untrusted": true,
303
+ "bytes": 2,
304
+ "sha256": "..."
305
+ },
306
+ "inputs": []
307
+ }
308
+ ```
309
+
310
+ The sidecar hashes the report file, normalized prompt text when available, untrusted output text when available, and configured input file paths. Consumers that cross a process or trust boundary should rehash the sidecar targets before acting on the report. Report writes are atomic and refuse to overwrite an existing target path.
311
+
222
312
  ## Contract Fixtures
223
313
 
224
314
  Shared fixtures live under:
@@ -231,7 +321,9 @@ Required gates:
231
321
 
232
322
  ```bash
233
323
  npm run contract:validate
324
+ npm run docs:drift
234
325
  npm run parity:fixtures
326
+ npm run parity:suite
235
327
  npm run test:backend-conformance
236
328
  ```
237
329
 
@@ -0,0 +1,142 @@
1
+ # Language Coverage & Capture Tracker
2
+
3
+ The full set of languages ChatGPT exposes in **Settings → General → Language** (captured
4
+ 2026-06-09), with a rough total-speaker estimate and capture status. Use this to track the
5
+ localization rollout described in [`localization.md`](./localization.md). Strings are
6
+ stored per-locale under `src/dom/locale/<bcp47>.ts`.
7
+
8
+ - Speaker counts are **crude guesstimates** (native + second-language, rounded), not
9
+ researched figures. They exist only to prioritize the rollout.
10
+ - `bcp47` is the suggested per-locale file name. Regional variants ChatGPT lists separately
11
+ (Spanish, French, Portuguese, Chinese) are separate registry entries — capture the
12
+ larger-reach variant first; the others can be filled later or alias the base.
13
+ - Status: ✅ done · 🟡 in progress · ⬜ pending.
14
+
15
+ ## Capture queue — top 15 by crude speaker count
16
+
17
+ English is the canonical baseline (already in the registry), so the queue below is the top
18
+ 15 **non-English** languages by rough world speaker count. Order is a best-effort
19
+ guesstimate; the tail (≈#11–15) is close and easily reordered.
20
+
21
+ | # | Language | Native | bcp47 | ~Speakers | Status |
22
+ |---|---|---|---|---|---|
23
+ | — | English (US) | English (US) | `en-US` | ~1.5B | ✅ baseline |
24
+ | 1 | Chinese (Simplified) | 简体中文 | `zh-Hans` | ~1.1B | ✅ |
25
+ | 2 | Hindi | हिन्दी | `hi` | ~600M | ✅ |
26
+ | 3 | Spanish (Latin America) | español (Latinoamérica) | `es-419` | ~450M | ✅ |
27
+ | 4 | Arabic | العربية | `ar` | ~400M | ✅ |
28
+ | 5 | French (France) | français (France) | `fr-FR` | ~300M | ✅ |
29
+ | 6 | Bengali | বাংলা | `bn` | ~270M | ✅ |
30
+ | 7 | Portuguese (Brazil) | português (Brasil) | `pt-BR` | ~260M | ✅ |
31
+ | 8 | Russian | русский | `ru` | ~255M | ✅ |
32
+ | 9 | Urdu | اردو | `ur` | ~230M | ✅ |
33
+ | 10 | Indonesian | Indonesia | `id` | ~200M | ✅ |
34
+ | 11 | German | Deutsch | `de` | ~135M | ✅ |
35
+ | 12 | Punjabi | ਪੰਜਾਬੀ | `pa` | ~130M | ✅ |
36
+ | 13 | Japanese | 日本語 | `ja` | ~125M | ✅ |
37
+ | 14 | Marathi | मराठी | `mr` | ~95M | ✅ |
38
+ | 15 | Turkish | Türkçe | `tr` | ~85M | ✅ |
39
+
40
+ > Note: this ranks by **raw speaker count** as requested. If you'd rather weight by likely
41
+ > ChatGPT user base, the order would tilt toward European/East-Asian locales (Japanese,
42
+ > German, French, Spanish, Portuguese, Korean, Italian) and away from Punjabi/Marathi/Bengali.
43
+
44
+ ## Full language list
45
+
46
+ | Language | Native | bcp47 | ~Speakers | Status |
47
+ |---|---|---|---|---|
48
+ | English (US) | English (US) | `en-US` | ~1.5B | ✅ |
49
+ | Amharic | አማርኛ | `am` | ~60M | ✅ |
50
+ | Arabic | العربية | `ar` | ~400M | ✅ |
51
+ | Bulgarian | български | `bg` | ~8M | ✅ |
52
+ | Bengali | বাংলা | `bn` | ~270M | ✅ |
53
+ | Bosnian | bosanski | `bs` | ~3M | ✅ |
54
+ | Catalan | català | `ca` | ~10M | ✅ |
55
+ | Czech | čeština | `cs` | ~11M | ✅ |
56
+ | Danish | dansk | `da` | ~6M | ✅ |
57
+ | German | Deutsch | `de` | ~135M | ✅ |
58
+ | Greek | Ελληνικά | `el` | ~13M | ✅ |
59
+ | Spanish (Latin America) | español (Latinoamérica) | `es-419` | ~450M | ✅ |
60
+ | Spanish (Spain) | español (España) | `es-ES` | ~47M | ✅ |
61
+ | Estonian | eesti | `et` | ~1M | ✅ |
62
+ | Persian | فارسی | `fa` | ~80M | ✅ |
63
+ | Finnish | suomi | `fi` | ~5.5M | ✅ |
64
+ | French (Canada) | français (Canada) | `fr-CA` | ~10M | ✅ |
65
+ | French (France) | français (France) | `fr-FR` | ~300M | ✅ |
66
+ | Gujarati | ગુજરાતી | `gu` | ~60M | ✅ |
67
+ | Hindi | हिन्दी | `hi` | ~600M | ✅ |
68
+ | Croatian | hrvatski | `hr` | ~5M | ✅ |
69
+ | Hungarian | magyar | `hu` | ~13M | ✅ |
70
+ | Armenian | հայերեն | `hy` | ~7M | ✅ |
71
+ | Indonesian | Indonesia | `id` | ~200M | ✅ |
72
+ | Icelandic | íslenska | `is` | ~0.4M | ✅ |
73
+ | Italian | italiano | `it` | ~65M | ✅ |
74
+ | Japanese | 日本語 | `ja` | ~125M | ✅ |
75
+ | Georgian | ქართული | `ka` | ~4M | ✅ |
76
+ | Kazakh | қазақ тілі | `kk` | ~13M | ✅ |
77
+ | Kannada | ಕನ್ನಡ | `kn` | ~60M | ✅ |
78
+ | Korean | 한국어 | `ko` | ~80M | ✅ |
79
+ | Lithuanian | lietuvių | `lt` | ~3M | ✅ |
80
+ | Latvian | latviešu | `lv` | ~2M | ✅ |
81
+ | Macedonian | македонски | `mk` | ~2M | ✅ |
82
+ | Malayalam | മലയാളം | `ml` | ~38M | ✅ |
83
+ | Mongolian | монгол | `mn` | ~5M | ✅ |
84
+ | Marathi | मराठी | `mr` | ~95M | ✅ |
85
+ | Malay | Bahasa Melayu | `ms` | ~80M | ✅ |
86
+ | Burmese | မြန်မာ | `my` | ~43M | ✅ |
87
+ | Norwegian Bokmål | norsk bokmål | `nb` | ~5M | ✅ |
88
+ | Dutch | Nederlands | `nl` | ~25M | ✅ |
89
+ | Punjabi | ਪੰਜਾਬੀ | `pa` | ~130M | ✅ |
90
+ | Polish | polski | `pl` | ~45M | ✅ |
91
+ | Portuguese (Brazil) | português (Brasil) | `pt-BR` | ~210M | ✅ |
92
+ | Portuguese (Portugal) | português (Portugal) | `pt-PT` | ~10M | ✅ |
93
+ | Romanian | română | `ro` | ~24M | ✅ |
94
+ | Russian | русский | `ru` | ~255M | ✅ |
95
+ | Slovak | slovenčina | `sk` | ~5M | ✅ |
96
+ | Slovenian | slovenščina | `sl` | ~2.5M | ✅ |
97
+ | Somali | Soomaali | `so` | ~22M | ✅ |
98
+ | Albanian | shqip | `sq` | ~7.5M | ✅ |
99
+ | Serbian | српски | `sr` | ~9M | ✅ |
100
+ | Swedish | svenska | `sv` | ~10M | ✅ |
101
+ | Swahili | Kiswahili | `sw` | ~120M | ✅ |
102
+ | Tamil | தமிழ் | `ta` | ~85M | ✅ |
103
+ | Telugu | తెలుగు | `te` | ~95M | ✅ |
104
+ | Thai | ไทย | `th` | ~60M | ✅ |
105
+ | Tagalog (Filipino) | Tagalog | `tl` | ~70M | ✅ |
106
+ | Turkish | Türkçe | `tr` | ~85M | ✅ |
107
+ | Ukrainian | українська | `uk` | ~40M | ✅ |
108
+ | Urdu | اردو | `ur` | ~230M | ✅ |
109
+ | Vietnamese | Tiếng Việt | `vi` | ~85M | ✅ |
110
+ | Chinese (Simplified) | 简体中文 | `zh-Hans` | ~1.1B | ✅ |
111
+ | Chinese (Traditional, Hong Kong) | 繁體中文(香港) | `zh-HK` | ~85M | ✅ |
112
+ | Chinese (Traditional, Taiwan) | 繁體中文(台灣) | `zh-TW` | ~23M | ✅ |
113
+
114
+ ## Automated capture
115
+
116
+ Use the repository capture script to drive the visible ChatGPT UI through Settings → General
117
+ → Language, then open the composer Intelligence picker and record the localized labels:
118
+
119
+ ```bash
120
+ npm run capture:intelligence-locales -- --auto-switch --all --if-missing open
121
+ ```
122
+
123
+ The script uses the native language names from this tracker, verifies the rendered
124
+ `document.documentElement.lang`, writes JSONL evidence under
125
+ `outputs/intelligence-locale-captures/`, and restores the initially selected language by
126
+ default. It does not write cookies, storage, or private browser state directly; if ChatGPT
127
+ changes the settings or picker DOM, the run should emit a blocker record instead of guessing.
128
+
129
+ ## Capture procedure (per language)
130
+
131
+ 1. Switch ChatGPT **Settings → General → Language** to the target, reload, and confirm via
132
+ `document.documentElement.lang` + visible text.
133
+ 2. Read static controls (composer, add-files, new chat, current model label) via JS.
134
+ 3. Open the model switcher and the `+` menu with **real** clicks (Radix menus ignore
135
+ synthetic `.click()`) and read the menu items → mode + tool labels.
136
+ 4. Open search → placeholder; observe a conversation → copy-response + response-actions.
137
+ `stopControl` is only present mid-generation; login/captcha/rate-limit copy needs a
138
+ logged-out/limited state — these may lag and rely on the English fallback + the
139
+ `selector_drift` safety net until captured.
140
+ 5. Write the verified strings to `src/dom/locale/<bcp47>.ts` and register it in
141
+ `src/dom/locale/index.ts`. Build + test + bundle + sync.
142
+ 6. Restore the account language to English (US) when the run is complete.