testronaut 2.1.1 → 2.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/TODO.md CHANGED
@@ -12,6 +12,7 @@
12
12
  ## Browser Automation Follow-Ups
13
13
 
14
14
  - [x] Make `click_text` match exact text, prefer the innermost visible element, and fail quickly instead of clicking an ancestor or partial match.
15
+ - [ ] Complete `press_key` turn-loop integration: classify it as fire-and-forget, automatically inject the updated DOM after successful key presses, and test that Enter/Escape state changes are available to the next reasoning turn without an explicit `get_dom` call.
15
16
  - [ ] Replace framework/vendor-specific DOM class stripping with a framework-agnostic classifier that removes generated hashes, styling utilities, variants, and arbitrary selectors while preserving semantic component classes. Reconsider broad prefix rules such as `/^cl-/`, cap retained class count and length, and add representative fixtures for Tailwind, MUI, Ant Design, Chakra, Clerk, CSS Modules, and CSS-in-JS.
16
17
 
17
18
  ## Session and Reporting Follow-Ups
@@ -27,9 +28,18 @@
27
28
 
28
29
  ## Optimization and Rate-Limit Follow-Ups
29
30
 
31
+ - [ ] Reduce redundant screenshot capture in focused missions without weakening evidence: clarify or enforce when starting, transition, and final screenshots materially differ, and measure the effect on turns, tokens, and report usefulness.
30
32
  - [ ] Run a history-informed model-routing benchmark before expanding cross-provider routing. Give Jev versioned, per-model/per-mission-class reliability, token, latency, and price observations from prior runs; require a configurable reliability floor; compare against the best fixed-model-plus-guardrails control with blocked repetitions; keep selection at the mission boundary; record the evidence and reason for every choice; and define cold-start, stale-data, low-confidence, and provider-failure fallbacks. Do not retry a failed mission on another provider in the benchmark because that would confound routing quality with retry recovery.
31
33
  - [ ] Add an opt-in CLI/config policy that reroutes an imminent throttled LLM request to a compatible model as an alternative to the default backoff behavior. Keep backoff as the default; use observed TPM/RPM headroom and projected request tokens; constrain failover to explicitly configured provider/model families; preserve tool/history compatibility; prevent routing loops; fall back safely when the alternate model is unavailable; and record the original model, routed model, trigger, token usage, latency, and outcome in reports.
32
34
 
35
+ ## OpenAI Provider Follow-Ups
36
+
37
+ - [ ] Migrate the OpenAI adapter from Chat Completions to the Responses API in a dedicated change. Preserve Testronaut's provider-neutral message and tool-call contract, add response-ID/reasoning-state handling, normalize usage and raw headers, and cover multi-turn tool continuation with unit and API smoke tests. Once migrated, remove the GPT-5.6 Chat Completions compatibility override that sets `reasoning_effort` to `none` for function-tool requests.
38
+ - [ ] Add active context budgeting after its behavior is designed and tested. Use model context/output metadata to reserve response and reasoning capacity, then apply explicit history compaction or summarization rules. Do not silently truncate mission instructions, current DOM state, or unmatched tool calls.
39
+ - [ ] Learn effective OpenAI TPM/RPM limits from successful response headers and rate-limit errors. Prefer observed project/model limits over inferring a named account tier; retain the environment override and conservative startup values.
40
+ - [ ] Persist provider-observed rate limits in a separate, expiring local cache keyed by provider, API project/account, endpoint/region, model, and traffic type. Never rewrite user config or infer a named usage tier from a TPM value; expose configured, cached, and live sources distinctly and discard stale observations.
41
+ - [ ] After dynamic rate limits are established, make DOM injection quota-aware. Budget DOM tokens against both remaining TPM and the model context window, preserve mission instructions and actionable controls, and fall back to semantic sections plus on-demand DOM chunks instead of silent truncation.
42
+
33
43
  ## Automated MFA Follow-Ups
34
44
 
35
45
  - [ ] Add an integration smoke test against a local or staging API fixture for `get_mfa_code`.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "testronaut",
3
- "version": "2.1.1",
3
+ "version": "2.1.3",
4
4
  "main": "./runner/testronaut.js",
5
5
  "bin": {
6
6
  "testronaut": "bin/cli.js"
@@ -69,4 +69,44 @@ describe('ChromeBrowser.check_text', () => {
69
69
 
70
70
  await expect(browser.check_text({ text: 'Hidden message' })).resolves.toBe('NOT FOUND: Hidden message');
71
71
  });
72
+
73
+ it('reports the iframe containing a visible match', async () => {
74
+ const noMatches = { count: vi.fn().mockResolvedValue(0) };
75
+ const frameMatches = {
76
+ count: vi.fn().mockResolvedValue(1),
77
+ nth: vi.fn().mockReturnValue({ isVisible: vi.fn().mockResolvedValue(true) }),
78
+ };
79
+ const mainFrame = { getByText: vi.fn().mockReturnValue(noMatches) };
80
+ const childFrame = {
81
+ getByText: vi.fn().mockReturnValue(frameMatches),
82
+ name: vi.fn().mockReturnValue(''),
83
+ frameElement: vi.fn().mockResolvedValue({ getAttribute: vi.fn().mockResolvedValue('webviewer-1') }),
84
+ url: vi.fn().mockReturnValue('https://example.test/viewer'),
85
+ };
86
+ const browser = Object.create(ChromeBrowser.prototype);
87
+ browser.page = {
88
+ mainFrame: vi.fn().mockReturnValue(mainFrame),
89
+ frames: vi.fn().mockReturnValue([mainFrame, childFrame]),
90
+ };
91
+
92
+ await expect(browser.check_text({ text: 'Quarterly report' }))
93
+ .resolves.toBe('FOUND in frame webviewer-1: Quarterly report');
94
+ });
95
+ });
96
+
97
+ describe('ChromeBrowser iframe interaction targeting', () => {
98
+ it('builds an explicit frame-qualified click locator', () => {
99
+ const locator = { first: vi.fn().mockReturnThis() };
100
+ const frameContext = { locator: vi.fn().mockReturnValue(locator) };
101
+ const browser = Object.create(ChromeBrowser.prototype);
102
+ browser.page = { frameLocator: vi.fn().mockReturnValue(frameContext) };
103
+
104
+ const [make] = browser._buildClickCandidates({
105
+ frame: 'iframe#webviewer-1', selector: 'button[aria-label="Settings"]', searchAllFrames: true,
106
+ });
107
+
108
+ expect(make()).toBe(locator);
109
+ expect(browser.page.frameLocator).toHaveBeenCalledWith('iframe#webviewer-1');
110
+ expect(frameContext.locator).toHaveBeenCalledWith('button[aria-label="Settings"]');
111
+ });
72
112
  });
@@ -0,0 +1,33 @@
1
+ import { describe, expect, it, vi } from 'vitest';
2
+ import { ChromeBrowser } from '../../tools/chromeBrowser.js';
3
+
4
+ describe('ChromeBrowser.get_dom iframe serialization', () => {
5
+ it('keeps serialized same-origin content and inaccessible-frame markers', async () => {
6
+ const browser = Object.create(ChromeBrowser.prototype);
7
+ browser.domListLimit = 3;
8
+ browser.debugEnabled = false;
9
+ browser.context = null;
10
+ browser.page = {
11
+ isClosed: vi.fn().mockReturnValue(false),
12
+ waitForLoadState: vi.fn().mockResolvedValue(undefined),
13
+ content: vi.fn().mockResolvedValue('<html><body></body></html>'),
14
+ evaluate: vi.fn().mockResolvedValue(`
15
+ <html><body>
16
+ <testronaut-frame id="viewer" data-testronaut-frame="viewer" data-testronaut-frame-status="same-origin">
17
+ <html><body><button aria-label="Settings">Settings</button><p>Quarterly report</p></body></html>
18
+ </testronaut-frame>
19
+ <testronaut-frame id="payment" data-testronaut-frame="payment" data-testronaut-frame-status="cross-origin">
20
+ [cross-origin frame, content not accessible]
21
+ </testronaut-frame>
22
+ </body></html>`),
23
+ };
24
+
25
+ const result = await browser.get_dom();
26
+
27
+ expect(result).toContain('data-testronaut-frame="viewer"');
28
+ expect(result).toContain('<iframe');
29
+ expect(result).toContain('Quarterly report');
30
+ expect(result).toContain('aria-label="Settings"');
31
+ expect(result).toContain('[cross-origin frame, content not accessible]');
32
+ });
33
+ });
@@ -0,0 +1,39 @@
1
+ import { describe, expect, it, vi } from 'vitest';
2
+ import { CHROME_TOOL_MAP, ChromeBrowser } from '../../tools/chromeBrowser.js';
3
+
4
+ describe('ChromeBrowser.press_key', () => {
5
+ it('presses a key on the active page', async () => {
6
+ const browser = new ChromeBrowser();
7
+ browser.page = { keyboard: { press: vi.fn().mockResolvedValue(undefined) } };
8
+
9
+ await expect(browser.press_key({ key: 'Escape' })).resolves.toBe('pressed Escape');
10
+ expect(browser.page.keyboard.press).toHaveBeenCalledWith('Escape');
11
+ });
12
+
13
+ it('focuses a selected element before pressing Enter', async () => {
14
+ const browser = new ChromeBrowser();
15
+ const focus = vi.fn().mockResolvedValue(undefined);
16
+ const first = vi.fn().mockReturnValue({ focus });
17
+ browser.page = {
18
+ locator: vi.fn().mockReturnValue({ first }),
19
+ keyboard: { press: vi.fn().mockResolvedValue(undefined) },
20
+ };
21
+
22
+ await expect(browser.press_key({ key: 'Enter', selector: '#email' }))
23
+ .resolves.toBe('pressed Enter after focusing #email');
24
+
25
+ expect(browser.page.locator).toHaveBeenCalledWith('#email');
26
+ expect(first).toHaveBeenCalledOnce();
27
+ expect(focus).toHaveBeenCalledWith({ timeout: 3000 });
28
+ expect(browser.page.keyboard.press).toHaveBeenCalledWith('Enter');
29
+ expect(focus.mock.invocationCallOrder[0])
30
+ .toBeLessThan(browser.page.keyboard.press.mock.invocationCallOrder[0]);
31
+ });
32
+
33
+ it('is exposed through the browser tool dispatcher', async () => {
34
+ const press_key = vi.fn().mockResolvedValue('pressed Control+B');
35
+ await expect(CHROME_TOOL_MAP.press_key({ press_key }, { key: 'Control+B' }))
36
+ .resolves.toBe('pressed Control+B');
37
+ expect(press_key).toHaveBeenCalledWith({ key: 'Control+B' });
38
+ });
39
+ });
@@ -56,7 +56,7 @@ describe('domControl helpers', () => {
56
56
  expect(result).not.toContain('<style');
57
57
  expect(result).not.toContain('<meta');
58
58
  expect(result).not.toContain('<link');
59
- expect(result).not.toContain('<iframe');
59
+ expect(result).toContain('<iframe src="frame.html"></iframe>');
60
60
  expect(result).not.toContain('<canvas');
61
61
  expect(result).not.toContain('<svg');
62
62
  expect(result).not.toContain('<!-- comment -->');
@@ -21,6 +21,20 @@ describe('tools/toolSchema', () => {
21
21
  const entry = toolsSchema.find(tool => tool.function.name === 'get_dom');
22
22
  expect(entry.function.parameters.properties.selector.type).toBe('string');
23
23
  });
24
+ it('includes press_key with a required key and optional selector', () => {
25
+ const entry = toolsSchema.find(tool => tool.function.name === 'press_key');
26
+ expect(entry?.function.parameters.required).toEqual(['key']);
27
+ expect(entry.function.parameters.properties.key.type).toBe('string');
28
+ expect(entry.function.parameters.properties.selector.type).toBe('string');
29
+ });
30
+ it('exposes iframe targeting and iframe text search', () => {
31
+ const fill = toolsSchema.find(tool => tool.function.name === 'fill');
32
+ const click = toolsSchema.find(tool => tool.function.name === 'click');
33
+ const checkText = toolsSchema.find(tool => tool.function.name === 'check_text');
34
+ expect(fill.function.parameters.properties.frame.type).toBe('string');
35
+ expect(click.function.parameters.properties.frame.type).toBe('string');
36
+ expect(checkText.function.parameters.properties.searchAllFrames.default).toBe(true);
37
+ });
24
38
  it('includes resource_progress tool with empty params', () => {
25
39
  const entry = toolsSchema.find(t => t.function?.name === 'resource_progress');
26
40
  expect(entry?.type).toBe('function');
@@ -180,8 +180,13 @@ export class ChromeBrowser {
180
180
  }
181
181
 
182
182
  // Build candidate locator MAKERS (Page|Frame-aware)
183
- _buildClickCandidates({ selector, text, role, attrs, scope, searchAllFrames }) {
184
- const frames = searchAllFrames ? this._allFrames({ includeMain: true }) : [this.page.mainFrame()];
183
+ _interactionContexts(frame, searchAllFrames) {
184
+ if (frame) return [this.page.frameLocator(frame)];
185
+ return searchAllFrames ? this._allFrames({ includeMain: true }) : [this.page.mainFrame()];
186
+ }
187
+
188
+ _buildClickCandidates({ selector, text, role, attrs, scope, searchAllFrames, frame }) {
189
+ const frames = this._interactionContexts(frame, searchAllFrames);
185
190
  const makers = [];
186
191
  const add = (fn) => fn && makers.push(fn);
187
192
 
@@ -211,8 +216,9 @@ export class ChromeBrowser {
211
216
  }
212
217
  }
213
218
 
214
- const seen = new Set();
215
- return makers.filter(fn => { const k = fn.toString(); if (seen.has(k)) return false; seen.add(k); return true; });
219
+ // Do not de-duplicate by function source: closures for different frames have
220
+ // identical source text but resolve against different documents.
221
+ return makers;
216
222
  }
217
223
 
218
224
 
@@ -459,8 +465,8 @@ export class ChromeBrowser {
459
465
  }
460
466
 
461
467
  // Build candidates for text entry (Page|Frame-aware)
462
- _buildFillCandidates({ selector, placeholder, role = 'textbox', attrs, scope, searchAllFrames }) {
463
- const frames = searchAllFrames ? this._allFrames({ includeMain: true }) : [this.page.mainFrame()];
468
+ _buildFillCandidates({ selector, placeholder, role = 'textbox', attrs, scope, searchAllFrames, frame }) {
469
+ const frames = this._interactionContexts(frame, searchAllFrames);
464
470
  const makers = [];
465
471
  const add = (fn) => fn && makers.push(fn);
466
472
 
@@ -499,9 +505,9 @@ export class ChromeBrowser {
499
505
  }
500
506
  }
501
507
 
502
- // de-dup
503
- const seen = new Set();
504
- return makers.filter(fn => { const k = fn.toString(); if (seen.has(k)) return false; seen.add(k); return true; });
508
+ // Do not de-duplicate by function source: closures for different frames have
509
+ // identical source text but resolve against different documents.
510
+ return makers;
505
511
  }
506
512
 
507
513
 
@@ -714,6 +720,7 @@ export class ChromeBrowser {
714
720
  async fill({
715
721
  selector,
716
722
  text,
723
+ frame,
717
724
  placeholder, // string | RegExp
718
725
  role = 'textbox', // generic default
719
726
  attrs,
@@ -729,9 +736,9 @@ export class ChromeBrowser {
729
736
 
730
737
  const makeCandidates = () => {
731
738
  if (strategy === 'strict') {
732
- return this._buildFillCandidates({ selector, scope, searchAllFrames });
739
+ return this._buildFillCandidates({ selector, scope, searchAllFrames, frame });
733
740
  }
734
- return this._buildFillCandidates({ selector, placeholder, role, attrs, scope, searchAllFrames });
741
+ return this._buildFillCandidates({ selector, placeholder, role, attrs, scope, searchAllFrames, frame });
735
742
  };
736
743
 
737
744
  return this._withRetries(async () => {
@@ -764,6 +771,18 @@ export class ChromeBrowser {
764
771
  }, { attempts: 3, baseMs: 250 });
765
772
  }
766
773
 
774
+ async press_key({ key, selector }) {
775
+ if (selector) {
776
+ const target = this.page.locator(selector).first();
777
+ await target.focus({ timeout: 3000 });
778
+ }
779
+
780
+ await this.page.keyboard.press(key);
781
+ return selector
782
+ ? `pressed ${key} after focusing ${selector}`
783
+ : `pressed ${key}`;
784
+ }
785
+
767
786
 
768
787
 
769
788
  /**
@@ -778,7 +797,7 @@ export class ChromeBrowser {
778
797
  * @param {number} [opts.delayMs=200] // micro settle
779
798
  */
780
799
  async click({
781
- selector, text, role, attrs, scope,
800
+ selector, text, role, attrs, scope, frame,
782
801
  searchAllFrames = true,
783
802
  strategy = 'auto',
784
803
  waitFor,
@@ -794,8 +813,8 @@ export class ChromeBrowser {
794
813
 
795
814
  // Build your candidates the way you already do (generic/hint-driven)
796
815
  const makers = (strategy === 'strict')
797
- ? this._buildClickCandidates({ selector, scope, searchAllFrames })
798
- : this._buildClickCandidates({ selector, text, role, attrs, scope, searchAllFrames });
816
+ ? this._buildClickCandidates({ selector, scope, searchAllFrames, frame })
817
+ : this._buildClickCandidates({ selector, text, role, attrs, scope, searchAllFrames, frame });
799
818
 
800
819
  // choose & click (same as your generic version)
801
820
  let chosen = null;
@@ -1014,9 +1033,57 @@ export class ChromeBrowser {
1014
1033
  // Fallback to empty string if still nothing (keeps caller logic simple)
1015
1034
  if (!html) html = '';
1016
1035
 
1036
+ // page.content() only serializes iframe elements, not their documents. Build a
1037
+ // read-only clone that embeds accessible frame documents so selectors and the
1038
+ // global output limit apply to the complete snapshot.
1039
+ try {
1040
+ html = await this.page.evaluate(() => {
1041
+ const serializeDocument = (doc) => {
1042
+ const clone = doc.documentElement.cloneNode(true);
1043
+ const sourceFrames = Array.from(doc.querySelectorAll('iframe'));
1044
+ const clonedFrames = Array.from(clone.querySelectorAll('iframe'));
1045
+
1046
+ sourceFrames.forEach((sourceFrame, index) => {
1047
+ const clonedFrame = clonedFrames[index];
1048
+ if (!clonedFrame) return;
1049
+ // iframe is a raw-text element in HTML parsers. Use a temporary
1050
+ // custom element so Cheerio parses the embedded controls as markup.
1051
+ const serializedFrame = doc.createElement('testronaut-frame');
1052
+ for (const attribute of Array.from(clonedFrame.attributes)) {
1053
+ serializedFrame.setAttribute(attribute.name, attribute.value);
1054
+ }
1055
+ clonedFrame.replaceWith(serializedFrame);
1056
+ const label = sourceFrame.id || sourceFrame.name || sourceFrame.getAttribute('src') || `frame-${index + 1}`;
1057
+ serializedFrame.setAttribute('data-testronaut-frame', label);
1058
+ try {
1059
+ const childDocument = sourceFrame.contentDocument;
1060
+ if (!childDocument?.documentElement) throw new Error('inaccessible');
1061
+ serializedFrame.setAttribute('data-testronaut-frame-status', 'same-origin');
1062
+ serializedFrame.innerHTML = serializeDocument(childDocument);
1063
+ } catch {
1064
+ serializedFrame.setAttribute('data-testronaut-frame-status', 'cross-origin');
1065
+ serializedFrame.textContent = '[cross-origin frame, content not accessible]';
1066
+ }
1067
+ });
1068
+
1069
+ return clone.outerHTML;
1070
+ };
1071
+ return serializeDocument(document);
1072
+ });
1073
+ } catch {
1074
+ // Preserve the main-document snapshot if serialization races a navigation.
1075
+ }
1076
+
1017
1077
  // --- 4) Optional DOM reduction/cleanup (your existing steps) ---
1018
1078
  let $ = cheerio.load(html);
1019
1079
 
1080
+ // Restore the public marker shape after the HTML parser has materialized
1081
+ // the temporary element's descendants.
1082
+ $('testronaut-frame').each((_, element) => {
1083
+ element.name = 'iframe';
1084
+ element.tagName = 'iframe';
1085
+ });
1086
+
1020
1087
  if (selector) {
1021
1088
  let selection;
1022
1089
  try {
@@ -1141,12 +1208,27 @@ export class ChromeBrowser {
1141
1208
  }
1142
1209
 
1143
1210
 
1144
- async check_text({ text }) {
1145
- const matches = this.page.getByText(text);
1146
- const count = await matches.count();
1147
- for (let index = 0; index < count; index++) {
1148
- if (await matches.nth(index).isVisible().catch(() => false)) {
1149
- return `FOUND: ${text}`;
1211
+ async check_text({ text, searchAllFrames = true }) {
1212
+ const mainFrame = this.page.mainFrame?.();
1213
+ const frames = searchAllFrames && this.page.frames
1214
+ ? [mainFrame, ...this.page.frames().filter(frame => frame !== mainFrame)].filter(Boolean)
1215
+ : [mainFrame || this.page];
1216
+
1217
+ for (const frame of frames) {
1218
+ const matches = frame.getByText(text);
1219
+ const count = await matches.count();
1220
+ for (let index = 0; index < count; index++) {
1221
+ if (!await matches.nth(index).isVisible().catch(() => false)) continue;
1222
+ if (frame === mainFrame || frame === this.page) return `FOUND: ${text}`;
1223
+
1224
+ let label = frame.name?.() || '';
1225
+ if (!label) {
1226
+ label = await frame.frameElement?.()
1227
+ .then(element => element.getAttribute('id'))
1228
+ .catch(() => '') || '';
1229
+ }
1230
+ label ||= frame.url?.() || 'unnamed';
1231
+ return `FOUND in frame ${label}: ${text}`;
1150
1232
  }
1151
1233
  }
1152
1234
  return `NOT FOUND: ${text}`;
@@ -1455,6 +1537,7 @@ export const CHROME_TOOL_MAP = {
1455
1537
  reload: (b) => b.reload(),
1456
1538
  fill: (b, args) => b.fill(args),
1457
1539
  click: (b, args) => b.click(args),
1540
+ press_key: (b, args) => b.press_key(args),
1458
1541
  get_dom: (b, args, agentMemory) => b.get_dom(args, agentMemory),
1459
1542
  check_text: (b, args) => b.check_text(args),
1460
1543
  click_text: async (browser, args, agentMemory) => {
@@ -7,7 +7,7 @@
7
7
  * normalize, and compress HTML so it’s cheaper and clearer for LLMs.
8
8
  *
9
9
  * Responsibilities:
10
- * - Remove non-content tags (scripts, styles, iframes, etc.) and noisy attributes.
10
+ * - Remove non-content tags (scripts, styles, etc.) and noisy attributes.
11
11
  * - Collapse overly long lists/sections into truncated markers.
12
12
  * - Reduce repeated div blocks to a representative sample.
13
13
  * - Strip obfuscated/utility CSS classes that don’t help reasoning.
@@ -23,7 +23,8 @@
23
23
 
24
24
  /**
25
25
  * Remove tags and attributes that are unlikely to help LLM reasoning.
26
- * - Strips: script, style, meta, link, noscript, iframe, canvas, svg
26
+ * - Strips: script, style, meta, link, noscript, canvas, svg
27
+ * - Keeps iframe shells and any same-origin content injected by get_dom.
27
28
  * - Keeps only a safe whitelist of attributes on remaining elements.
28
29
  * - Removes HTML comments.
29
30
  *
@@ -31,7 +32,7 @@
31
32
  * @returns {$} same Cheerio instance for chaining
32
33
  */
33
34
  export const removeNonContentTags = async ($) => {
34
- $('script, style, meta, link, noscript, iframe, canvas, svg').remove();
35
+ $('script, style, meta, link, noscript, canvas, svg').remove();
35
36
 
36
37
  $('*').each((_, el) => {
37
38
  const $el = $(el);
@@ -39,6 +40,7 @@ export const removeNonContentTags = async ($) => {
39
40
  'href', 'src', 'type', 'alt', 'title',
40
41
  'placeholder', 'name', 'id', 'class', 'value',
41
42
  'data-testid', 'disabled', 'role', 'data-state',
43
+ 'data-testronaut-frame', 'data-testronaut-frame-status',
42
44
  // keep key data-* attributes that help identify rows/items without bloating too much
43
45
  'data-type', 'data-id', 'data-title', 'data-parentid', 'data-order'
44
46
  ];
@@ -44,6 +44,7 @@ const toolsSchema = [
44
44
  properties: {
45
45
  selector: { type: 'string' },
46
46
  text: { type: 'string' },
47
+ frame: { type: 'string', description: 'Optional CSS selector for the containing iframe.' },
47
48
  },
48
49
  required: ['selector', 'text'],
49
50
  },
@@ -58,6 +59,7 @@ const toolsSchema = [
58
59
  type: 'object',
59
60
  properties: {
60
61
  selector: { type: 'string' },
62
+ frame: { type: 'string', description: 'Optional CSS selector for the containing iframe.' },
61
63
  waitFor: {
62
64
  type: 'string',
63
65
  enum: ['load', 'domcontentloaded', 'networkidle'],
@@ -73,6 +75,27 @@ const toolsSchema = [
73
75
  },
74
76
  },
75
77
  },
78
+ {
79
+ type: 'function',
80
+ function: {
81
+ name: 'press_key',
82
+ description: 'Press a keyboard key or shortcut, optionally focusing an element by CSS selector first',
83
+ parameters: {
84
+ type: 'object',
85
+ properties: {
86
+ key: {
87
+ type: 'string',
88
+ description: 'Playwright key or shortcut, such as Enter, Escape, Control+B, or Meta+B',
89
+ },
90
+ selector: {
91
+ type: 'string',
92
+ description: 'Optional CSS selector of the element to focus before pressing the key',
93
+ },
94
+ },
95
+ required: ['key'],
96
+ },
97
+ },
98
+ },
76
99
  {
77
100
  type: 'function',
78
101
  function: {
@@ -108,6 +131,11 @@ const toolsSchema = [
108
131
  type: 'object',
109
132
  properties: {
110
133
  text: { type: 'string' },
134
+ searchAllFrames: {
135
+ type: 'boolean',
136
+ default: true,
137
+ description: 'Search iframe documents as well as the main document.',
138
+ },
111
139
  },
112
140
  required: ['text'],
113
141
  },
package/todo.md DELETED
@@ -1,25 +0,0 @@
1
- # Testronaut CLI TODO
2
-
3
- ## OpenAI provider follow-ups
4
-
5
- - Migrate the OpenAI adapter from Chat Completions to the Responses API in a
6
- dedicated change. Preserve Testronaut's provider-neutral message and tool-call
7
- contract, add response-ID/reasoning-state handling, normalize usage and raw
8
- headers, and cover multi-turn tool continuation with unit and API smoke tests.
9
- Once migrated, remove the GPT-5.6 Chat Completions compatibility override that
10
- sets `reasoning_effort` to `none` for function-tool requests.
11
- - Add active context budgeting after its behavior is designed and tested. Use
12
- model context/output metadata to reserve response and reasoning capacity, then
13
- apply explicit history compaction or summarization rules. Do not silently
14
- truncate mission instructions, current DOM state, or unmatched tool calls.
15
- - Learn effective OpenAI TPM/RPM limits from successful response headers and
16
- rate-limit errors. Prefer observed project/model limits over inferring a named
17
- account tier; retain the environment override and conservative startup values.
18
- - Persist provider-observed rate limits in a separate, expiring local cache keyed
19
- by provider, API project/account, endpoint/region, model, and traffic type.
20
- Never rewrite user config or infer a named usage tier from a TPM value; expose
21
- configured, cached, and live sources distinctly and discard stale observations.
22
- - After dynamic rate limits are established, make DOM injection quota-aware.
23
- Budget DOM tokens against both remaining TPM and the model context window,
24
- preserve mission instructions and actionable controls, and fall back to
25
- semantic sections plus on-demand DOM chunks instead of silent truncation.