testronaut 2.1.2 → 2.1.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/TODO.md CHANGED
@@ -12,6 +12,7 @@
12
12
  ## Browser Automation Follow-Ups
13
13
 
14
14
  - [x] Make `click_text` match exact text, prefer the innermost visible element, and fail quickly instead of clicking an ancestor or partial match.
15
+ - [ ] Complete `press_key` turn-loop integration: classify it as fire-and-forget, automatically inject the updated DOM after successful key presses, and test that Enter/Escape state changes are available to the next reasoning turn without an explicit `get_dom` call.
15
16
  - [ ] Replace framework/vendor-specific DOM class stripping with a framework-agnostic classifier that removes generated hashes, styling utilities, variants, and arbitrary selectors while preserving semantic component classes. Reconsider broad prefix rules such as `/^cl-/`, cap retained class count and length, and add representative fixtures for Tailwind, MUI, Ant Design, Chakra, Clerk, CSS Modules, and CSS-in-JS.
16
17
 
17
18
  ## Session and Reporting Follow-Ups
@@ -27,9 +28,18 @@
27
28
 
28
29
  ## Optimization and Rate-Limit Follow-Ups
29
30
 
31
+ - [ ] Reduce redundant screenshot capture in focused missions without weakening evidence: clarify or enforce when starting, transition, and final screenshots materially differ, and measure the effect on turns, tokens, and report usefulness.
30
32
  - [ ] Run a history-informed model-routing benchmark before expanding cross-provider routing. Give Jev versioned, per-model/per-mission-class reliability, token, latency, and price observations from prior runs; require a configurable reliability floor; compare against the best fixed-model-plus-guardrails control with blocked repetitions; keep selection at the mission boundary; record the evidence and reason for every choice; and define cold-start, stale-data, low-confidence, and provider-failure fallbacks. Do not retry a failed mission on another provider in the benchmark because that would confound routing quality with retry recovery.
31
33
  - [ ] Add an opt-in CLI/config policy that reroutes an imminent throttled LLM request to a compatible model as an alternative to the default backoff behavior. Keep backoff as the default; use observed TPM/RPM headroom and projected request tokens; constrain failover to explicitly configured provider/model families; preserve tool/history compatibility; prevent routing loops; fall back safely when the alternate model is unavailable; and record the original model, routed model, trigger, token usage, latency, and outcome in reports.
32
34
 
35
+ ## OpenAI Provider Follow-Ups
36
+
37
+ - [ ] Migrate the OpenAI adapter from Chat Completions to the Responses API in a dedicated change. Preserve Testronaut's provider-neutral message and tool-call contract, add response-ID/reasoning-state handling, normalize usage and raw headers, and cover multi-turn tool continuation with unit and API smoke tests. Once migrated, remove the GPT-5.6 Chat Completions compatibility override that sets `reasoning_effort` to `none` for function-tool requests.
38
+ - [ ] Add active context budgeting after its behavior is designed and tested. Use model context/output metadata to reserve response and reasoning capacity, then apply explicit history compaction or summarization rules. Do not silently truncate mission instructions, current DOM state, or unmatched tool calls.
39
+ - [ ] Learn effective OpenAI TPM/RPM limits from successful response headers and rate-limit errors. Prefer observed project/model limits over inferring a named account tier; retain the environment override and conservative startup values.
40
+ - [ ] Persist provider-observed rate limits in a separate, expiring local cache keyed by provider, API project/account, endpoint/region, model, and traffic type. Never rewrite user config or infer a named usage tier from a TPM value; expose configured, cached, and live sources distinctly and discard stale observations.
41
+ - [ ] After dynamic rate limits are established, make DOM injection quota-aware. Budget DOM tokens against both remaining TPM and the model context window, preserve mission instructions and actionable controls, and fall back to semantic sections plus on-demand DOM chunks instead of silent truncation.
42
+
33
43
  ## Automated MFA Follow-Ups
34
44
 
35
45
  - [ ] Add an integration smoke test against a local or staging API fixture for `get_mfa_code`.
package/core/agent.js CHANGED
@@ -221,6 +221,13 @@ Email invitation and magic links:
221
221
  - open_email_link enforces the operator-configured emailLinks.allowedHosts policy. If the host is not configured or navigation is blocked, report a graceful FAILURE; do not use the ordinary navigate tool as a workaround.
222
222
  `.trim();
223
223
 
224
+ systemContent += '\n\n' + `
225
+ JavaScript dialogs:
226
+ - Browser-native alert, confirm, prompt, and beforeunload dialogs block their triggering action.
227
+ - When a mission requires a particular response, call prepare_dialog_response immediately before the click, reload, or navigation that opens it. Specify the expected type, accept/dismiss decision, and an exact or partial message when known; provide input only when accepting a prompt.
228
+ - Prepared responses are one-time and expire. Unexpected dialogs are dismissed fail-closed and reported to you and the mission report. Never assume an unexpected dialog was accepted.
229
+ `.trim();
230
+
224
231
  const permittedRecoveries = Array.isArray(opts.recovery?.permitted)
225
232
  ? opts.recovery.permitted.map(value => String(value).trim().toLowerCase()).filter(Boolean)
226
233
  : null;
package/core/turnLoop.js CHANGED
@@ -653,6 +653,24 @@ export const turnLoop = async (
653
653
  hadToolIssues = true;
654
654
  }
655
655
 
656
+ const dialogEvents = browser?.drainDialogEvents?.() || [];
657
+ if (dialogEvents.length) {
658
+ step.dialogs = step.dialogs || [];
659
+ step.dialogs.push(...dialogEvents);
660
+ for (const dialogEvent of dialogEvents) {
661
+ if (dialogEvent.event === 'dialog_rule_expired') {
662
+ step.events.push(`⏱️ Dialog response ${dialogEvent.ruleId} expired before a matching ${dialogEvent.expectedType} appeared.`);
663
+ } else {
664
+ const expectation = dialogEvent.matched ? 'expected' : 'unexpected';
665
+ step.events.push(`💬 ${expectation} ${dialogEvent.type} dialog: ${JSON.stringify(dialogEvent.message)} → ${dialogEvent.outcome}`);
666
+ }
667
+ }
668
+ result = JSON.stringify({
669
+ result: (() => { try { return JSON.parse(result); } catch { return result; } })(),
670
+ dialogs: dialogEvents,
671
+ });
672
+ }
673
+
656
674
  if (fnName === 'request_human_input') {
657
675
  step.humanInput = step.humanInput || {};
658
676
  step.humanInput.requested = true;
@@ -936,7 +954,9 @@ export const turnLoop = async (
936
954
  // bloating context with large payloads (file JSON, etc.).
937
955
  // - For semantic tools (get_dom, check_text, etc.), send full result.
938
956
  let contentForModel;
939
- if (FIRE_AND_FORGET_TOOLS.has(fnName)) {
957
+ if (dialogEvents.length) {
958
+ contentForModel = result;
959
+ } else if (FIRE_AND_FORGET_TOOLS.has(fnName)) {
940
960
  contentForModel = errorMessage || 'OK';
941
961
  } else {
942
962
  contentForModel = result;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "testronaut",
3
- "version": "2.1.2",
3
+ "version": "2.1.4",
4
4
  "main": "./runner/testronaut.js",
5
5
  "bin": {
6
6
  "testronaut": "bin/cli.js"
@@ -182,6 +182,56 @@ describe('turnLoop', () => {
182
182
  expect(shared.chromeToolSpies.get_dom).not.toHaveBeenCalled();
183
183
  });
184
184
 
185
+ it('reports browser-native dialog appearance and response to the model and mission step', async () => {
186
+ const dialog = {
187
+ event: 'javascript_dialog',
188
+ type: 'confirm',
189
+ message: 'Emergency meeting! Attend?',
190
+ defaultValue: '',
191
+ pageUrl: 'https://example.test/dashboard',
192
+ matched: true,
193
+ expected: true,
194
+ action: 'accept',
195
+ outcome: 'accept',
196
+ };
197
+ browser.drainDialogEvents = vi.fn()
198
+ .mockReturnValueOnce([dialog])
199
+ .mockReturnValue([]);
200
+ shared.chatMock
201
+ .mockResolvedValueOnce({
202
+ message: {
203
+ role: 'assistant',
204
+ content: '',
205
+ tool_calls: [{
206
+ id: 'dialog_click',
207
+ type: 'function',
208
+ function: { name: 'click_text', arguments: '{"text":"Request attendance"}' },
209
+ }],
210
+ },
211
+ usage: { total_tokens: 10 },
212
+ headers: {},
213
+ })
214
+ .mockResolvedValueOnce({
215
+ message: { role: 'assistant', content: 'FINAL: success' },
216
+ usage: { total_tokens: 5 },
217
+ headers: {},
218
+ });
219
+
220
+ const result = await turnLoop(browser, baseMessages(), 2, 0, 0, {}, {
221
+ steps: [],
222
+ missionName: 'dialogs',
223
+ });
224
+
225
+ expect(result.success).toBe(true);
226
+ expect(result.steps[0].dialogs).toEqual([dialog]);
227
+ expect(result.steps[0].events).toContain(
228
+ '💬 expected confirm dialog: "Emergency meeting! Attend?" → accept',
229
+ );
230
+ const secondRequest = shared.chatMock.mock.calls[1][0].messages;
231
+ const response = secondRequest.find(message => message.tool_call_id === 'dialog_click');
232
+ expect(JSON.parse(response.content).dialogs).toEqual([dialog]);
233
+ });
234
+
185
235
  it('uses one tool-free closure turn after the working-turn budget is exhausted', async () => {
186
236
  shared.chatMock
187
237
  .mockResolvedValueOnce({
@@ -0,0 +1,102 @@
1
+ import { EventEmitter } from 'node:events';
2
+ import { describe, expect, it, vi } from 'vitest';
3
+ import { CHROME_TOOL_MAP, ChromeBrowser } from '../../tools/chromeBrowser.js';
4
+
5
+ class FakePage extends EventEmitter {
6
+ url() { return 'https://example.test/dialogs'; }
7
+ }
8
+
9
+ function fakeDialog(type, message, defaultValue = '') {
10
+ return {
11
+ type: () => type,
12
+ message: () => message,
13
+ defaultValue: () => defaultValue,
14
+ accept: vi.fn().mockResolvedValue(undefined),
15
+ dismiss: vi.fn().mockResolvedValue(undefined),
16
+ };
17
+ }
18
+
19
+ const flush = () => new Promise(resolve => setTimeout(resolve, 0));
20
+
21
+ describe('ChromeBrowser JavaScript dialogs', () => {
22
+ it.each([
23
+ ['alert', 'accept', undefined],
24
+ ['confirm', 'dismiss', undefined],
25
+ ['prompt', 'accept', 'red crewmate'],
26
+ ['beforeunload', 'accept', undefined],
27
+ ])('handles an expected %s without blocking', async (type, action, input) => {
28
+ const browser = new ChromeBrowser();
29
+ const page = new FakePage();
30
+ browser._addAndFocus(page);
31
+ browser.prepare_dialog_response({
32
+ type,
33
+ action,
34
+ input,
35
+ messageIncludes: 'Emergency',
36
+ });
37
+ const dialog = fakeDialog(type, 'Emergency meeting!', input ? 'anonymous' : '');
38
+
39
+ page.emit('dialog', dialog);
40
+ await flush();
41
+
42
+ expect(dialog[action]).toHaveBeenCalledOnce();
43
+ if (input !== undefined) expect(dialog.accept).toHaveBeenCalledWith(input);
44
+ const [event] = browser.drainDialogEvents();
45
+ expect(event).toMatchObject({
46
+ event: 'javascript_dialog',
47
+ type,
48
+ message: 'Emergency meeting!',
49
+ action,
50
+ matched: true,
51
+ outcome: action,
52
+ });
53
+ });
54
+
55
+ it('dismisses and reports an unexpected dialog while retaining a nonmatching rule', async () => {
56
+ const browser = new ChromeBrowser();
57
+ const page = new FakePage();
58
+ browser._addAndFocus(page);
59
+ browser.prepare_dialog_response({ type: 'confirm', action: 'accept', message: 'Expected' });
60
+ const dialog = fakeDialog('confirm', 'Suspicious impostor dialog');
61
+
62
+ page.emit('dialog', dialog);
63
+ await flush();
64
+
65
+ expect(dialog.dismiss).toHaveBeenCalledOnce();
66
+ expect(dialog.accept).not.toHaveBeenCalled();
67
+ expect(browser.dialogRules).toHaveLength(1);
68
+ expect(browser.drainDialogEvents()[0]).toMatchObject({
69
+ matched: false,
70
+ action: 'dismiss',
71
+ outcome: 'unexpected_dialog_dismissed',
72
+ });
73
+ });
74
+
75
+ it('reports an armed response that times out', () => {
76
+ vi.useFakeTimers();
77
+ try {
78
+ const browser = new ChromeBrowser();
79
+ browser.prepare_dialog_response({ type: 'alert', action: 'accept', timeoutMs: 250 });
80
+ vi.advanceTimersByTime(251);
81
+ expect(browser.drainDialogEvents()[0]).toMatchObject({
82
+ event: 'dialog_rule_expired',
83
+ expectedType: 'alert',
84
+ outcome: 'timeout',
85
+ });
86
+ } finally {
87
+ vi.useRealTimers();
88
+ }
89
+ });
90
+
91
+ it('rejects unscoped prompt input and exposes the dispatcher', async () => {
92
+ const browser = new ChromeBrowser();
93
+ expect(() => browser.prepare_dialog_response({ type: 'confirm', action: 'accept', input: 'secret' }))
94
+ .toThrow(/only supported.*prompt/i);
95
+
96
+ const prepare_dialog_response = vi.fn().mockReturnValue({ armed: true });
97
+ expect(CHROME_TOOL_MAP.prepare_dialog_response(
98
+ { prepare_dialog_response },
99
+ { type: 'alert', action: 'accept' },
100
+ )).toEqual({ armed: true });
101
+ });
102
+ });
@@ -0,0 +1,39 @@
1
+ import { describe, expect, it, vi } from 'vitest';
2
+ import { CHROME_TOOL_MAP, ChromeBrowser } from '../../tools/chromeBrowser.js';
3
+
4
+ describe('ChromeBrowser.press_key', () => {
5
+ it('presses a key on the active page', async () => {
6
+ const browser = new ChromeBrowser();
7
+ browser.page = { keyboard: { press: vi.fn().mockResolvedValue(undefined) } };
8
+
9
+ await expect(browser.press_key({ key: 'Escape' })).resolves.toBe('pressed Escape');
10
+ expect(browser.page.keyboard.press).toHaveBeenCalledWith('Escape');
11
+ });
12
+
13
+ it('focuses a selected element before pressing Enter', async () => {
14
+ const browser = new ChromeBrowser();
15
+ const focus = vi.fn().mockResolvedValue(undefined);
16
+ const first = vi.fn().mockReturnValue({ focus });
17
+ browser.page = {
18
+ locator: vi.fn().mockReturnValue({ first }),
19
+ keyboard: { press: vi.fn().mockResolvedValue(undefined) },
20
+ };
21
+
22
+ await expect(browser.press_key({ key: 'Enter', selector: '#email' }))
23
+ .resolves.toBe('pressed Enter after focusing #email');
24
+
25
+ expect(browser.page.locator).toHaveBeenCalledWith('#email');
26
+ expect(first).toHaveBeenCalledOnce();
27
+ expect(focus).toHaveBeenCalledWith({ timeout: 3000 });
28
+ expect(browser.page.keyboard.press).toHaveBeenCalledWith('Enter');
29
+ expect(focus.mock.invocationCallOrder[0])
30
+ .toBeLessThan(browser.page.keyboard.press.mock.invocationCallOrder[0]);
31
+ });
32
+
33
+ it('is exposed through the browser tool dispatcher', async () => {
34
+ const press_key = vi.fn().mockResolvedValue('pressed Control+B');
35
+ await expect(CHROME_TOOL_MAP.press_key({ press_key }, { key: 'Control+B' }))
36
+ .resolves.toBe('pressed Control+B');
37
+ expect(press_key).toHaveBeenCalledWith({ key: 'Control+B' });
38
+ });
39
+ });
@@ -165,6 +165,36 @@ describe('generateHtmlReport', () => {
165
165
  expect(html).not.toContain('<strong>LLM:</strong>');
166
166
  });
167
167
 
168
+ it('renders native dialog evidence as a clearly labeled report visualization', () => {
169
+ const outPath = path.join(tmpDir, 'dialogs.html');
170
+ generateHtmlReport({ missions: [{
171
+ missionName: 'Dialogs', status: 'passed', steps: [{
172
+ turn: 0,
173
+ result: '✅ Passed',
174
+ events: ['dialog handled'],
175
+ dialogs: [{
176
+ event: 'javascript_dialog',
177
+ type: 'prompt',
178
+ message: 'Name the suspicious crewmate:',
179
+ defaultValue: 'Red',
180
+ inputProvided: true,
181
+ action: 'accept',
182
+ matched: true,
183
+ outcome: 'accept',
184
+ pageUrl: 'https://example.test/dashboard',
185
+ }],
186
+ }],
187
+ }] }, outPath);
188
+
189
+ const html = fs.readFileSync(outPath, 'utf8');
190
+ expect(html).toContain('Report visualization · not a literal browser screenshot');
191
+ expect(html).toContain('Browser-native prompt');
192
+ expect(html).toContain('Name the suspicious crewmate:');
193
+ expect(html).toContain('•••••••• · input provided');
194
+ expect(html).toContain('dialog-button chosen">Accept');
195
+ expect(html).not.toContain('>Red</div>');
196
+ });
197
+
168
198
  it('renders run, mission, and step Jev token spend', () => {
169
199
  const steps = [{
170
200
  turn: 0,
@@ -21,6 +21,20 @@ describe('tools/toolSchema', () => {
21
21
  const entry = toolsSchema.find(tool => tool.function.name === 'get_dom');
22
22
  expect(entry.function.parameters.properties.selector.type).toBe('string');
23
23
  });
24
+ it('includes press_key with a required key and optional selector', () => {
25
+ const entry = toolsSchema.find(tool => tool.function.name === 'press_key');
26
+ expect(entry?.function.parameters.required).toEqual(['key']);
27
+ expect(entry.function.parameters.properties.key.type).toBe('string');
28
+ expect(entry.function.parameters.properties.selector.type).toBe('string');
29
+ });
30
+ it('includes a scoped one-shot JavaScript dialog response tool', () => {
31
+ const entry = toolsSchema.find(tool => tool.function.name === 'prepare_dialog_response');
32
+ expect(entry?.function.parameters.required).toEqual(['type', 'action']);
33
+ expect(entry.function.parameters.properties.type.enum)
34
+ .toEqual(['alert', 'confirm', 'prompt', 'beforeunload']);
35
+ expect(entry.function.parameters.properties.action.enum).toEqual(['accept', 'dismiss']);
36
+ expect(entry.function.parameters.properties.input.type).toBe('string');
37
+ });
24
38
  it('exposes iframe targeting and iframe text search', () => {
25
39
  const fill = toolsSchema.find(tool => tool.function.name === 'fill');
26
40
  const click = toolsSchema.find(tool => tool.function.name === 'click');
@@ -59,6 +59,9 @@ export class ChromeBrowser {
59
59
  this.debugEnabled = !!options.debug;
60
60
  this._domLimitLogged = false;
61
61
  this.storageStatePath = options.storageStatePath;
62
+ this.dialogRules = [];
63
+ this.dialogEvents = [];
64
+ this._dialogRuleSequence = 0;
62
65
 
63
66
  const defaultResourceGuard = {
64
67
  enabled: true,
@@ -98,6 +101,13 @@ export class ChromeBrowser {
98
101
  this.currentId = id;
99
102
  this.page = page;
100
103
 
104
+ // Playwright blocks the action that opened a JavaScript dialog until the
105
+ // dialog is handled. Keep this listener synchronous and always resolve the
106
+ // dialog so an unexpected alert can never hang a mission.
107
+ page.on('dialog', dialog => {
108
+ void this._handleDialog(dialog, page);
109
+ });
110
+
101
111
  page.on('close', () => {
102
112
  // remove from registry
103
113
  this.pages = this.pages.filter(p => p.id !== id);
@@ -115,6 +125,115 @@ export class ChromeBrowser {
115
125
  return id;
116
126
  }
117
127
 
128
+ _expireDialogRules(now = Date.now()) {
129
+ const active = [];
130
+ for (const rule of this.dialogRules) {
131
+ if (rule.expiresAt > now) {
132
+ active.push(rule);
133
+ continue;
134
+ }
135
+ this.dialogEvents.push({
136
+ event: 'dialog_rule_expired',
137
+ ruleId: rule.id,
138
+ expectedType: rule.type,
139
+ expectedMessage: rule.message,
140
+ messageIncludes: rule.messageIncludes,
141
+ action: rule.action,
142
+ armedAt: new Date(rule.armedAt).toISOString(),
143
+ expiredAt: new Date(rule.expiresAt).toISOString(),
144
+ outcome: 'timeout',
145
+ });
146
+ }
147
+ this.dialogRules = active;
148
+ }
149
+
150
+ async _handleDialog(dialog, page) {
151
+ const now = Date.now();
152
+ this._expireDialogRules(now);
153
+ const type = dialog.type();
154
+ const message = dialog.message();
155
+ const matchingIndex = this.dialogRules.findIndex(rule =>
156
+ rule.type === type
157
+ && (rule.message === undefined || rule.message === message)
158
+ && (rule.messageIncludes === undefined || message.includes(rule.messageIncludes))
159
+ );
160
+ const rule = matchingIndex >= 0 ? this.dialogRules.splice(matchingIndex, 1)[0] : null;
161
+ const action = rule?.action || 'dismiss';
162
+ const record = {
163
+ event: 'javascript_dialog',
164
+ type,
165
+ message,
166
+ defaultValue: dialog.defaultValue(),
167
+ pageUrl: page?.url?.() || null,
168
+ appearedAt: new Date(now).toISOString(),
169
+ ruleId: rule?.id || null,
170
+ matched: Boolean(rule),
171
+ expected: Boolean(rule),
172
+ action,
173
+ inputProvided: rule?.input !== undefined,
174
+ outcome: 'pending',
175
+ };
176
+
177
+ try {
178
+ if (action === 'accept') await dialog.accept(rule?.input);
179
+ else await dialog.dismiss();
180
+ record.outcome = rule ? action : 'unexpected_dialog_dismissed';
181
+ } catch (error) {
182
+ record.outcome = 'response_failed';
183
+ record.error = error?.message || String(error);
184
+ // A second response can fail if the page closed. Never mask that fact.
185
+ }
186
+ record.respondedAt = new Date().toISOString();
187
+ this.dialogEvents.push(record);
188
+ }
189
+
190
+ prepare_dialog_response({
191
+ action,
192
+ type,
193
+ message,
194
+ messageIncludes,
195
+ input,
196
+ timeoutMs = 10_000,
197
+ } = {}) {
198
+ const supportedTypes = new Set(['alert', 'confirm', 'prompt', 'beforeunload']);
199
+ if (!supportedTypes.has(type)) throw new Error(`Unsupported dialog type: ${type || '(missing)'}`);
200
+ if (!['accept', 'dismiss'].includes(action)) throw new Error('Dialog action must be "accept" or "dismiss".');
201
+ if (message !== undefined && messageIncludes !== undefined) {
202
+ throw new Error('Use either message or messageIncludes, not both.');
203
+ }
204
+ if (input !== undefined && (type !== 'prompt' || action !== 'accept')) {
205
+ throw new Error('Dialog input is only supported when accepting a prompt.');
206
+ }
207
+ const boundedTimeout = Math.min(30_000, Math.max(250, Number(timeoutMs) || 10_000));
208
+ const armedAt = Date.now();
209
+ const rule = {
210
+ id: `dialog_${++this._dialogRuleSequence}`,
211
+ action,
212
+ type,
213
+ message,
214
+ messageIncludes,
215
+ input,
216
+ armedAt,
217
+ expiresAt: armedAt + boundedTimeout,
218
+ };
219
+ this.dialogRules.push(rule);
220
+ return {
221
+ armed: true,
222
+ ruleId: rule.id,
223
+ action,
224
+ type,
225
+ message,
226
+ messageIncludes,
227
+ inputProvided: input !== undefined,
228
+ timeoutMs: boundedTimeout,
229
+ };
230
+ }
231
+
232
+ drainDialogEvents() {
233
+ this._expireDialogRules();
234
+ return this.dialogEvents.splice(0);
235
+ }
236
+
118
237
  _switchTo(target) {
119
238
  if (target === 'main') {
120
239
  this.currentId = this.pages[0]?.id ?? '';
@@ -771,6 +890,18 @@ export class ChromeBrowser {
771
890
  }, { attempts: 3, baseMs: 250 });
772
891
  }
773
892
 
893
+ async press_key({ key, selector }) {
894
+ if (selector) {
895
+ const target = this.page.locator(selector).first();
896
+ await target.focus({ timeout: 3000 });
897
+ }
898
+
899
+ await this.page.keyboard.press(key);
900
+ return selector
901
+ ? `pressed ${key} after focusing ${selector}`
902
+ : `pressed ${key}`;
903
+ }
904
+
774
905
 
775
906
 
776
907
  /**
@@ -1525,6 +1656,8 @@ export const CHROME_TOOL_MAP = {
1525
1656
  reload: (b) => b.reload(),
1526
1657
  fill: (b, args) => b.fill(args),
1527
1658
  click: (b, args) => b.click(args),
1659
+ press_key: (b, args) => b.press_key(args),
1660
+ prepare_dialog_response: (b, args) => b.prepare_dialog_response(args),
1528
1661
  get_dom: (b, args, agentMemory) => b.get_dom(args, agentMemory),
1529
1662
  check_text: (b, args) => b.check_text(args),
1530
1663
  click_text: async (browser, args, agentMemory) => {
@@ -104,6 +104,7 @@ export function generateHtmlReport(report, outputPath) {
104
104
  const missionModelUsage = summarizeModelTokens(steps);
105
105
  const stepItems = steps.map((step, idx) => {
106
106
  const events = Array.isArray(step.events) ? step.events : [];
107
+ const dialogs = Array.isArray(step.dialogs) ? step.dialogs : [];
107
108
  const ok = /✅|Passed|Mission Success/i.test(step.result || '');
108
109
  const resultRaw = step.result || '—';
109
110
  const resultTooltip = resultRaw.includes('⚠️ Turn Issues')
@@ -122,6 +123,36 @@ export function generateHtmlReport(report, outputPath) {
122
123
  const imgTag = step.screenshotPath
123
124
  ? `<img src="${esc(step.screenshotPath)}" alt="screenshot turn ${esc(step.turn ?? idx)}">`
124
125
  : '';
126
+ const dialogCards = dialogs.map(dialog => {
127
+ if (dialog.event === 'dialog_rule_expired') {
128
+ return `<article class="dialog-card dialog-timeout">
129
+ <div class="dialog-card-label">Report visualization · response timeout</div>
130
+ <strong>${esc(dialog.expectedType || 'dialog')} rule expired</strong>
131
+ <span>No matching dialog appeared before ${esc(dialog.ruleId || 'the prepared response')} expired.</span>
132
+ </article>`;
133
+ }
134
+ const type = String(dialog.type || 'dialog');
135
+ const accepted = dialog.action === 'accept';
136
+ const acceptLabel = type === 'beforeunload' ? 'Leave' : 'Accept';
137
+ const dismissLabel = type === 'beforeunload' ? 'Stay' : 'Dismiss';
138
+ return `<article class="dialog-card ${dialog.matched ? 'dialog-expected' : 'dialog-unexpected'}">
139
+ <div class="dialog-card-label">Report visualization · not a literal browser screenshot</div>
140
+ <div class="dialog-window">
141
+ <div class="dialog-title"><span class="dialog-icon">!</span><strong>Browser-native ${esc(type)}</strong></div>
142
+ <div class="dialog-message">${esc(dialog.message || '(browser supplied no message)')}</div>
143
+ ${type === 'prompt' ? `<div class="dialog-input">${dialog.inputProvided ? '•••••••• · input provided' : esc(dialog.defaultValue || 'No input provided')}</div>` : ''}
144
+ <div class="dialog-actions">
145
+ <span class="dialog-button ${accepted ? 'chosen' : ''}">${esc(acceptLabel)}</span>
146
+ <span class="dialog-button ${!accepted ? 'chosen' : ''}">${esc(dismissLabel)}</span>
147
+ </div>
148
+ </div>
149
+ <div class="dialog-meta">
150
+ <span>${dialog.matched ? '✓ Expected response matched' : '⚠ Unexpected dialog; fail-closed response'}</span>
151
+ <span>Outcome: ${esc(dialog.outcome || 'unknown')}</span>
152
+ ${dialog.pageUrl ? `<span>Page: ${esc(dialog.pageUrl)}</span>` : ''}
153
+ </div>
154
+ </article>`;
155
+ }).join('');
125
156
  const stepJev = step.jevShadow?.status === 'ok'
126
157
  ? ` • Jev: ${formatTokens(step.jevShadow.usage?.input_tokens)} in / ${formatTokens(step.jevShadow.usage?.output_tokens)} out`
127
158
  : step.jevShadow
@@ -152,6 +183,7 @@ export function generateHtmlReport(report, outputPath) {
152
183
  <span class="step-result ${ok ? 'ok' : 'bad'}" ${resultTooltip ? `title="${esc(resultTooltip)}"` : ''}>${esc(resultRaw)}</span>
153
184
  <span class="tokens">${esc(step.model || llm.model || 'Model')} tokens: ${esc(step.tokensUsed ?? '—')} (${esc(step.inputTokens ?? '—')} in / ${esc(step.outputTokens ?? '—')} out)${routeLabel}${stepJev}</span>
154
185
  </summary>
186
+ ${dialogCards ? `<div class="dialog-evidence">${dialogCards}</div>` : ''}
155
187
  <pre class="events">${esc(events.join('\n')) || '(no events)'}</pre>
156
188
  ${imgTag}
157
189
  </details>
@@ -428,6 +460,27 @@ export function generateHtmlReport(report, outputPath) {
428
460
  border: 1px solid var(--hairline-strong);
429
461
  }
430
462
 
463
+ .dialog-evidence{display:grid;gap:12px;margin:12px 0 0;}
464
+ .dialog-card{
465
+ display:grid;gap:10px;padding:12px;border:1px solid rgba(96,165,250,.34);
466
+ border-radius:12px;background:linear-gradient(145deg,rgba(15,23,42,.96),rgba(30,41,59,.92));
467
+ }
468
+ .dialog-card.dialog-unexpected,.dialog-card.dialog-timeout{border-color:rgba(251,191,36,.48);}
469
+ .dialog-card-label{font-size:10px;font-weight:800;letter-spacing:.12em;text-transform:uppercase;color:#93c5fd;}
470
+ .dialog-window{
471
+ width:min(100%,520px);margin:auto;padding:18px;border-radius:13px;color:#e5e7eb;
472
+ background:#343540;border:1px solid rgba(255,255,255,.14);box-shadow:0 18px 40px rgba(0,0,0,.35);
473
+ }
474
+ .dialog-title{display:flex;align-items:center;gap:9px;font-size:14px;}
475
+ .dialog-icon{display:grid;place-items:center;width:22px;height:22px;border-radius:50%;background:#f59e0b;color:#111827;font-weight:900;}
476
+ .dialog-message{padding:18px 0;font-size:15px;line-height:1.45;white-space:pre-wrap;}
477
+ .dialog-input{padding:9px 11px;border:2px solid #22d3ee;border-radius:8px;background:#171821;color:#cbd5e1;font:12px ui-monospace,monospace;}
478
+ .dialog-actions{display:flex;justify-content:flex-end;gap:9px;margin-top:16px;}
479
+ .dialog-button{padding:7px 15px;border-radius:8px;background:#454753;border:1px solid rgba(255,255,255,.13);font-size:12px;font-weight:750;}
480
+ .dialog-button.chosen{color:#07111e;background:#22d3ee;border-color:#67e8f9;box-shadow:0 0 0 2px rgba(34,211,238,.18);}
481
+ .dialog-meta{display:flex;gap:8px;flex-wrap:wrap;color:var(--text-muted);font-size:11px;}
482
+ .dialog-meta span{padding:4px 7px;border:1px solid var(--hairline);border-radius:999px;background:rgba(255,255,255,.035);}
483
+
431
484
  img{
432
485
  display:block; max-width:100%;
433
486
  border:1px solid var(--hairline); border-radius:12px; margin-top:10px;
@@ -75,6 +75,47 @@ const toolsSchema = [
75
75
  },
76
76
  },
77
77
  },
78
+ {
79
+ type: 'function',
80
+ function: {
81
+ name: 'press_key',
82
+ description: 'Press a keyboard key or shortcut, optionally focusing an element by CSS selector first',
83
+ parameters: {
84
+ type: 'object',
85
+ properties: {
86
+ key: {
87
+ type: 'string',
88
+ description: 'Playwright key or shortcut, such as Enter, Escape, Control+B, or Meta+B',
89
+ },
90
+ selector: {
91
+ type: 'string',
92
+ description: 'Optional CSS selector of the element to focus before pressing the key',
93
+ },
94
+ },
95
+ required: ['key'],
96
+ },
97
+ },
98
+ },
99
+ {
100
+ type: 'function',
101
+ function: {
102
+ name: 'prepare_dialog_response',
103
+ description: 'Arm a one-time response for the next matching JavaScript alert, confirm, prompt, or beforeunload dialog. Call immediately before the action that opens it. Unexpected dialogs are dismissed and reported.',
104
+ parameters: {
105
+ type: 'object',
106
+ properties: {
107
+ type: { type: 'string', enum: ['alert', 'confirm', 'prompt', 'beforeunload'] },
108
+ action: { type: 'string', enum: ['accept', 'dismiss'] },
109
+ message: { type: 'string', description: 'Optional exact message match.' },
110
+ messageIncludes: { type: 'string', description: 'Optional substring message match. Cannot be combined with message.' },
111
+ input: { type: 'string', description: 'Prompt text; valid only when accepting a prompt.' },
112
+ timeoutMs: { type: 'number', default: 10000, minimum: 250, maximum: 30000 },
113
+ },
114
+ required: ['type', 'action'],
115
+ additionalProperties: false,
116
+ },
117
+ },
118
+ },
78
119
  {
79
120
  type: 'function',
80
121
  function: {
package/todo.md DELETED
@@ -1,25 +0,0 @@
1
- # Testronaut CLI TODO
2
-
3
- ## OpenAI provider follow-ups
4
-
5
- - Migrate the OpenAI adapter from Chat Completions to the Responses API in a
6
- dedicated change. Preserve Testronaut's provider-neutral message and tool-call
7
- contract, add response-ID/reasoning-state handling, normalize usage and raw
8
- headers, and cover multi-turn tool continuation with unit and API smoke tests.
9
- Once migrated, remove the GPT-5.6 Chat Completions compatibility override that
10
- sets `reasoning_effort` to `none` for function-tool requests.
11
- - Add active context budgeting after its behavior is designed and tested. Use
12
- model context/output metadata to reserve response and reasoning capacity, then
13
- apply explicit history compaction or summarization rules. Do not silently
14
- truncate mission instructions, current DOM state, or unmatched tool calls.
15
- - Learn effective OpenAI TPM/RPM limits from successful response headers and
16
- rate-limit errors. Prefer observed project/model limits over inferring a named
17
- account tier; retain the environment override and conservative startup values.
18
- - Persist provider-observed rate limits in a separate, expiring local cache keyed
19
- by provider, API project/account, endpoint/region, model, and traffic type.
20
- Never rewrite user config or infer a named usage tier from a TPM value; expose
21
- configured, cached, and live sources distinctly and discard stale observations.
22
- - After dynamic rate limits are established, make DOM injection quota-aware.
23
- Budget DOM tokens against both remaining TPM and the model context window,
24
- preserve mission instructions and actionable controls, and fall back to
25
- semantic sections plus on-demand DOM chunks instead of silent truncation.