explorbot 0.4.5 → 0.4.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/boat/api-tester/src/ai/curler.ts +70 -66
- package/boat/api-tester/src/apibot.ts +19 -2
- package/boat/api-tester/src/cli.ts +87 -274
- package/boat/api-tester/src/commands/api-command.ts +10 -0
- package/boat/api-tester/src/commands/explore-command.ts +52 -0
- package/boat/api-tester/src/commands/init-command.ts +119 -0
- package/boat/api-tester/src/commands/know-command.ts +44 -0
- package/boat/api-tester/src/commands/plan-command.ts +42 -0
- package/boat/api-tester/src/commands/test-command.ts +54 -0
- package/boat/api-tester/src/config.ts +18 -1
- package/dist/boat/api-tester/src/ai/curler.js +55 -56
- package/dist/boat/api-tester/src/apibot.js +15 -1
- package/dist/boat/api-tester/src/cli.js +89 -243
- package/dist/boat/api-tester/src/commands/api-command.js +7 -0
- package/dist/boat/api-tester/src/commands/explore-command.js +41 -0
- package/dist/boat/api-tester/src/commands/init-command.js +88 -0
- package/dist/boat/api-tester/src/commands/know-command.js +39 -0
- package/dist/boat/api-tester/src/commands/plan-command.js +37 -0
- package/dist/boat/api-tester/src/commands/test-command.js +45 -0
- package/dist/boat/api-tester/src/config.js +3 -1
- package/dist/package.json +4 -4
- package/dist/rules/researcher/pagination.md +6 -0
- package/dist/src/action-result.d.ts +6 -0
- package/dist/src/action-result.js +12 -0
- package/dist/src/ai/planner.js +4 -0
- package/dist/src/ai/researcher/deep-analysis.d.ts +1 -1
- package/dist/src/ai/researcher/deep-analysis.js +14 -6
- package/dist/src/ai/researcher/locators.js +1 -1
- package/dist/src/ai/researcher/pagination.d.ts +16 -0
- package/dist/src/ai/researcher/pagination.js +62 -0
- package/dist/src/ai/researcher/parser.d.ts +3 -0
- package/dist/src/ai/researcher/parser.js +22 -6
- package/dist/src/ai/researcher/sections.js +1 -1
- package/dist/src/ai/researcher.js +7 -2
- package/dist/src/ai/rules.js +16 -0
- package/dist/src/ai/scout.js +8 -2
- package/dist/src/ai/tools.d.ts +1 -1
- package/dist/src/ai/tools.js +25 -13
- package/dist/src/api/spec-reader.d.ts +1 -0
- package/dist/src/api/spec-reader.js +93 -1
- package/dist/src/commands/base-command.d.ts +3 -3
- package/dist/src/commands/init-command.d.ts +3 -0
- package/dist/src/commands/init-command.js +6 -3
- package/dist/src/commands/options/ws-option.d.ts +7 -0
- package/dist/src/commands/options/ws-option.js +14 -0
- package/dist/src/config.d.ts +1 -0
- package/dist/src/config.js +14 -11
- package/dist/src/explorer.d.ts +1 -1
- package/dist/src/explorer.js +1 -1
- package/dist/src/remote.d.ts +2 -0
- package/dist/src/remote.js +23 -16
- package/dist/src/utils/aria.d.ts +2 -0
- package/dist/src/utils/aria.js +6 -1
- package/dist/src/utils/html-diff.js +4 -1
- package/dist/src/utils/markdown-query.d.ts +2 -0
- package/dist/src/utils/markdown-query.js +39 -0
- package/dist/src/utils/pagination.d.ts +16 -0
- package/dist/src/utils/pagination.js +20 -0
- package/docs/api-testing/basics.md +26 -2
- package/docs/superpowers/plans/2026-09-10-pagination.md +1420 -0
- package/docs/superpowers/specs/2026-09-09-pagination-rule-design.md +345 -0
- package/package.json +4 -4
- package/rules/researcher/pagination.md +6 -0
- package/src/action-result.ts +16 -0
- package/src/ai/planner.ts +4 -0
- package/src/ai/researcher/deep-analysis.ts +13 -6
- package/src/ai/researcher/locators.ts +1 -1
- package/src/ai/researcher/pagination.ts +68 -0
- package/src/ai/researcher/parser.ts +23 -5
- package/src/ai/researcher/sections.ts +1 -1
- package/src/ai/researcher.ts +9 -3
- package/src/ai/rules.ts +16 -0
- package/src/ai/scout.ts +9 -2
- package/src/ai/tools.ts +22 -14
- package/src/api/spec-reader.ts +106 -1
- package/src/commands/base-command.ts +3 -3
- package/src/commands/init-command.ts +6 -3
- package/src/commands/options/ws-option.ts +14 -0
- package/src/config.ts +15 -11
- package/src/explorer.ts +1 -1
- package/src/remote.ts +22 -15
- package/src/utils/aria.ts +8 -1
- package/src/utils/html-diff.ts +3 -1
- package/src/utils/markdown-query.ts +39 -0
- package/src/utils/pagination.ts +36 -0
|
@@ -0,0 +1,1420 @@
|
|
|
1
|
+
# Pagination Implementation Plan
|
|
2
|
+
|
|
3
|
+
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
4
|
+
|
|
5
|
+
**Goal:** Let Explorbot reach list content that is not loaded yet, whether the list pages through controls or grows as it is scrolled, including lists that scroll inside their own container.
|
|
6
|
+
|
|
7
|
+
**Architecture:** Detection is layered cheapest-first — HTML markers (no AI), then Researcher's own judgment while it describes a section, then a deterministic scroll gate, then a probe that scrolls and measures. The result is recorded as a `> Pagination:` line in the section's container blockquote. Tester and Navigator read that line plus a live marker scan, and inject one of two short rule fragments only when a strategy was detected. No new AI tool: `I.scrollTo(locator)` already scrolls every scrollable ancestor of its target.
|
|
8
|
+
|
|
9
|
+
**Tech Stack:** Bun (never Node), TypeScript, CodeceptJS + Playwright, `bun:test`, `mdq()` for all markdown, Biome for format/lint.
|
|
10
|
+
|
|
11
|
+
**Spec:** `docs/superpowers/specs/2026-09-09-pagination-rule-design.md`
|
|
12
|
+
|
|
13
|
+
## Global Constraints
|
|
14
|
+
|
|
15
|
+
- **Bun only.** `bun test`, `bun run format`, `bun run lint:fix`. Never `npm`/`node`.
|
|
16
|
+
- **No comments in code** unless a step's code block shows one.
|
|
17
|
+
- **No `try`/`catch` inside `try`/`catch`.** Prefer early return over `if`/`else`. No ternaries. No `...(cond ? {k:v} : {})` — use a plain `if`.
|
|
18
|
+
- **All markdown manipulation goes through `mdq()`** (`src/utils/markdown-query.ts`). Never split lines or regex over markdown structure. Regex over an *attribute value* or a *single extracted line* is fine.
|
|
19
|
+
- **Prompts must be general.** No site names, no CSS selectors, no example lifted from a debug session. 1–3 lines per bullet.
|
|
20
|
+
- **Never hardcode locators.** Markers in Task 1 are W3C-spec attributes, which is the one allowed kind of structural constant.
|
|
21
|
+
- **Types go at the end of the file.**
|
|
22
|
+
- **Private methods after public methods.**
|
|
23
|
+
- Run `bun run format` after each code change; `bun run lint:fix` before the final commit.
|
|
24
|
+
- Do **not** start a regression run, add the `regression` label, or `gh workflow run regression.yml`.
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
### Task 1: HTML marker scan
|
|
29
|
+
|
|
30
|
+
**Files:**
|
|
31
|
+
- Create: `src/utils/pagination.ts`
|
|
32
|
+
- Test: `tests/unit/pagination-markers.test.ts`
|
|
33
|
+
|
|
34
|
+
**Interfaces:**
|
|
35
|
+
- Consumes: nothing.
|
|
36
|
+
- Produces: `detectPaginationMarkers(html: string): PaginationStrategy | null` and `type PaginationStrategy = 'controls' | 'infinite'`. Tasks 4, 6 and 7 import both.
|
|
37
|
+
|
|
38
|
+
Uses `jsdom`, already a dependency and used the same way elsewhere in `src/utils/html.ts`.
|
|
39
|
+
|
|
40
|
+
**IMPORTANT — jsdom import order:** a test file that touches jsdom must have `import 'parse5';` as its **first** line. jsdom is CJS and `require()`s ESM-only parse5; without the leading import the file silently registers zero tests.
|
|
41
|
+
|
|
42
|
+
- [ ] **Step 1: Write the failing test**
|
|
43
|
+
|
|
44
|
+
Create `tests/unit/pagination-markers.test.ts`:
|
|
45
|
+
|
|
46
|
+
```ts
|
|
47
|
+
import 'parse5';
|
|
48
|
+
import { describe, expect, it } from 'bun:test';
|
|
49
|
+
import { detectPaginationMarkers } from '../../src/utils/pagination.ts';
|
|
50
|
+
|
|
51
|
+
describe('pagination markers', () => {
|
|
52
|
+
it('reads rel=next and rel=prev as controls', () => {
|
|
53
|
+
const html = '<nav><a href="?p=1" rel="prev">Back</a><a href="?p=3" rel="next">On</a></nav>';
|
|
54
|
+
|
|
55
|
+
expect(detectPaginationMarkers(html)).toBe('controls');
|
|
56
|
+
});
|
|
57
|
+
|
|
58
|
+
it('reads role=feed as infinite', () => {
|
|
59
|
+
const html = '<div role="feed"><article>One</article></div>';
|
|
60
|
+
|
|
61
|
+
expect(detectPaginationMarkers(html)).toBe('infinite');
|
|
62
|
+
});
|
|
63
|
+
|
|
64
|
+
it('reads aria-setsize=-1 as infinite', () => {
|
|
65
|
+
const html = '<ul><li aria-setsize="-1" aria-posinset="1">One</li></ul>';
|
|
66
|
+
|
|
67
|
+
expect(detectPaginationMarkers(html)).toBe('infinite');
|
|
68
|
+
});
|
|
69
|
+
|
|
70
|
+
it('ignores aria-current in every value', () => {
|
|
71
|
+
const html = '<nav><a href="/a" aria-current="page">A</a><a href="/b" aria-current="true">B</a></nav>';
|
|
72
|
+
|
|
73
|
+
expect(detectPaginationMarkers(html)).toBeNull();
|
|
74
|
+
});
|
|
75
|
+
|
|
76
|
+
it('prefers controls when both kinds of marker are present', () => {
|
|
77
|
+
const html = '<div role="feed"></div><a href="?p=2" rel="next">On</a>';
|
|
78
|
+
|
|
79
|
+
expect(detectPaginationMarkers(html)).toBe('controls');
|
|
80
|
+
});
|
|
81
|
+
|
|
82
|
+
it('returns null for a plain list', () => {
|
|
83
|
+
const html = '<ul><li>One</li><li>Two</li></ul>';
|
|
84
|
+
|
|
85
|
+
expect(detectPaginationMarkers(html)).toBeNull();
|
|
86
|
+
});
|
|
87
|
+
});
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
- [ ] **Step 2: Run test to verify it fails**
|
|
91
|
+
|
|
92
|
+
Run: `bun test tests/unit/pagination-markers.test.ts`
|
|
93
|
+
Expected: FAIL — cannot resolve `../../src/utils/pagination.ts`.
|
|
94
|
+
|
|
95
|
+
- [ ] **Step 3: Write minimal implementation**
|
|
96
|
+
|
|
97
|
+
Create `src/utils/pagination.ts`:
|
|
98
|
+
|
|
99
|
+
```ts
|
|
100
|
+
import { JSDOM } from 'jsdom';
|
|
101
|
+
|
|
102
|
+
const CONTROL_MARKERS = 'a[rel="next"], a[rel="prev"]';
|
|
103
|
+
const INFINITE_MARKERS = '[role="feed"], [aria-setsize="-1"]';
|
|
104
|
+
|
|
105
|
+
export function detectPaginationMarkers(html: string): PaginationStrategy | null {
|
|
106
|
+
if (!html) return null;
|
|
107
|
+
const { document } = new JSDOM(html).window;
|
|
108
|
+
if (document.querySelector(CONTROL_MARKERS)) return 'controls';
|
|
109
|
+
if (document.querySelector(INFINITE_MARKERS)) return 'infinite';
|
|
110
|
+
return null;
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
export type PaginationStrategy = 'controls' | 'infinite';
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
- [ ] **Step 4: Run test to verify it passes**
|
|
117
|
+
|
|
118
|
+
Run: `bun test tests/unit/pagination-markers.test.ts`
|
|
119
|
+
Expected: PASS, 6 tests.
|
|
120
|
+
|
|
121
|
+
- [ ] **Step 5: Format and commit**
|
|
122
|
+
|
|
123
|
+
```bash
|
|
124
|
+
bun run format
|
|
125
|
+
git add src/utils/pagination.ts tests/unit/pagination-markers.test.ts
|
|
126
|
+
git commit -m "Read pagination strategy from spec-defined HTML markers"
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
---
|
|
130
|
+
|
|
131
|
+
### Task 2: Parse `Data:` sections and the `Pagination:` line
|
|
132
|
+
|
|
133
|
+
**Files:**
|
|
134
|
+
- Modify: `src/ai/researcher/parser.ts`
|
|
135
|
+
- Test: `tests/unit/research-parser-pagination.test.ts`
|
|
136
|
+
|
|
137
|
+
**Interfaces:**
|
|
138
|
+
- Consumes: `PaginationStrategy` from Task 1.
|
|
139
|
+
- Produces: `parseDataSections(markdown: string): ResearchSection[]` and `extractPaginationFromBlockquote(sectionMarkdown: string): PaginationStrategy | null`. Tasks 4, 5 and 6 import both.
|
|
140
|
+
|
|
141
|
+
**Why this task exists:** Researcher emits a list of data items as a `## Data: <name>` section (`src/ai/researcher.ts:502-509`), and `parseResearchSections` filters exactly those out (`src/ai/researcher/parser.ts:100`). Probing only its output would skip every list on the page.
|
|
142
|
+
|
|
143
|
+
- [ ] **Step 1: Write the failing test**
|
|
144
|
+
|
|
145
|
+
Create `tests/unit/research-parser-pagination.test.ts`:
|
|
146
|
+
|
|
147
|
+
```ts
|
|
148
|
+
import { describe, expect, it } from 'bun:test';
|
|
149
|
+
import { extractPaginationFromBlockquote, parseDataSections, parseResearchSections } from '../../src/ai/researcher/parser.ts';
|
|
150
|
+
|
|
151
|
+
const RESEARCH = `## Menu
|
|
152
|
+
|
|
153
|
+
> Container: '.toolbar'
|
|
154
|
+
|
|
155
|
+
| Element | ARIA | CSS | eidx |
|
|
156
|
+
| Filter | button "Filter" | .filter-btn | 3 |
|
|
157
|
+
|
|
158
|
+
## Data: Suites List
|
|
159
|
+
|
|
160
|
+
> Container: '.suites-list-content'
|
|
161
|
+
> Pagination: infinite
|
|
162
|
+
|
|
163
|
+
Suite items, 13 items.
|
|
164
|
+
`;
|
|
165
|
+
|
|
166
|
+
describe('data sections', () => {
|
|
167
|
+
it('returns Data sections with their container', () => {
|
|
168
|
+
const sections = parseDataSections(RESEARCH);
|
|
169
|
+
|
|
170
|
+
expect(sections).toHaveLength(1);
|
|
171
|
+
expect(sections[0].name).toBe('Data: Suites List');
|
|
172
|
+
expect(sections[0].containerCss).toBe('.suites-list-content');
|
|
173
|
+
});
|
|
174
|
+
|
|
175
|
+
it('leaves Data sections out of parseResearchSections', () => {
|
|
176
|
+
const names = parseResearchSections(RESEARCH).map((s) => s.name);
|
|
177
|
+
|
|
178
|
+
expect(names).toEqual(['Menu']);
|
|
179
|
+
});
|
|
180
|
+
});
|
|
181
|
+
|
|
182
|
+
describe('pagination line', () => {
|
|
183
|
+
it('reads infinite', () => {
|
|
184
|
+
expect(extractPaginationFromBlockquote(parseDataSections(RESEARCH)[0].rawMarkdown)).toBe('infinite');
|
|
185
|
+
});
|
|
186
|
+
|
|
187
|
+
it('reads controls', () => {
|
|
188
|
+
const markdown = `## List\n\n> Container: '.rows'\n> Pagination: controls\n`;
|
|
189
|
+
|
|
190
|
+
expect(extractPaginationFromBlockquote(markdown)).toBe('controls');
|
|
191
|
+
});
|
|
192
|
+
|
|
193
|
+
it('returns null when the line is absent', () => {
|
|
194
|
+
const markdown = `## List\n\n> Container: '.rows'\n`;
|
|
195
|
+
|
|
196
|
+
expect(extractPaginationFromBlockquote(markdown)).toBeNull();
|
|
197
|
+
});
|
|
198
|
+
|
|
199
|
+
it('returns null for a value outside the vocabulary', () => {
|
|
200
|
+
const markdown = `## List\n\n> Container: '.rows'\n> Pagination: maybe\n`;
|
|
201
|
+
|
|
202
|
+
expect(extractPaginationFromBlockquote(markdown)).toBeNull();
|
|
203
|
+
});
|
|
204
|
+
});
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
- [ ] **Step 2: Run test to verify it fails**
|
|
208
|
+
|
|
209
|
+
Run: `bun test tests/unit/research-parser-pagination.test.ts`
|
|
210
|
+
Expected: FAIL — `parseDataSections` is not exported.
|
|
211
|
+
|
|
212
|
+
- [ ] **Step 3: Write minimal implementation**
|
|
213
|
+
|
|
214
|
+
In `src/ai/researcher/parser.ts`, add to the `PaginationStrategy` import line at the top:
|
|
215
|
+
|
|
216
|
+
```ts
|
|
217
|
+
import type { PaginationStrategy } from '../../utils/pagination.ts';
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
Add these two exports after `parseResearchSections`:
|
|
221
|
+
|
|
222
|
+
```ts
|
|
223
|
+
export function parseDataSections(markdown: string): ResearchSection[] {
|
|
224
|
+
return parseSections(markdown)
|
|
225
|
+
.filter((s) => s.name.toLowerCase().startsWith('data:'))
|
|
226
|
+
.map((section) => ({
|
|
227
|
+
name: section.name,
|
|
228
|
+
containerCss: extractContainerFromBlockquote(section.rawMarkdown),
|
|
229
|
+
elements: [],
|
|
230
|
+
rawMarkdown: section.rawMarkdown,
|
|
231
|
+
isExtended: false,
|
|
232
|
+
}));
|
|
233
|
+
}
|
|
234
|
+
|
|
235
|
+
export function extractPaginationFromBlockquote(sectionMarkdown: string): PaginationStrategy | null {
|
|
236
|
+
const bq = mdq(sectionMarkdown).query('blockquote[0]').text().trim();
|
|
237
|
+
if (!bq) return null;
|
|
238
|
+
const match = bq.match(/Pagination:\s*(\w+)/i);
|
|
239
|
+
if (!match) return null;
|
|
240
|
+
const value = match[1].toLowerCase();
|
|
241
|
+
if (value === 'controls') return 'controls';
|
|
242
|
+
if (value === 'infinite') return 'infinite';
|
|
243
|
+
return null;
|
|
244
|
+
}
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
- [ ] **Step 4: Run test to verify it passes**
|
|
248
|
+
|
|
249
|
+
Run: `bun test tests/unit/research-parser-pagination.test.ts`
|
|
250
|
+
Expected: PASS, 6 tests.
|
|
251
|
+
|
|
252
|
+
- [ ] **Step 5: Run the parser's existing tests for regressions**
|
|
253
|
+
|
|
254
|
+
Run: `bun test tests/unit/ --bail`
|
|
255
|
+
Expected: PASS. If a researcher test fails, `parseDataSections` has changed shared behaviour — it must not.
|
|
256
|
+
|
|
257
|
+
- [ ] **Step 6: Format and commit**
|
|
258
|
+
|
|
259
|
+
```bash
|
|
260
|
+
bun run format
|
|
261
|
+
git add src/ai/researcher/parser.ts tests/unit/research-parser-pagination.test.ts
|
|
262
|
+
git commit -m "Parse Data sections and the Pagination line"
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
---
|
|
266
|
+
|
|
267
|
+
### Task 3: Scroll gate and probe measurement helpers
|
|
268
|
+
|
|
269
|
+
**Files:**
|
|
270
|
+
- Modify: `src/utils/pagination.ts`
|
|
271
|
+
- Test: `tests/integration/pagination-browser.test.ts`
|
|
272
|
+
|
|
273
|
+
**Interfaces:**
|
|
274
|
+
- Consumes: nothing from earlier tasks.
|
|
275
|
+
- Produces: two in-page functions passed to `page.evaluate`, plus their result types:
|
|
276
|
+
- `measureScroll(css: string): ScrollMeasure | null` — `{ ownScroller, belowFold, scrollTop, windowScrollY, rowCount }`
|
|
277
|
+
- `restoreScroll({ css, scrollTop, windowScrollY }: ScrollRestore): void`
|
|
278
|
+
|
|
279
|
+
Task 4 calls both through `explorer.withPage`.
|
|
280
|
+
|
|
281
|
+
**`scrollIntoViewIfNeeded` moves the window too.** Measured during design on a page with both a scrollable `div` and a scrollable window: `before {box:0, win:0}` → `after {box:2200, win:1821}`. Restoring only the container's `scrollTop` would leave the page scrolled 1821px down for every screenshot, coordinate lookup and research pass that follows. Both offsets must be captured and both restored.
|
|
282
|
+
|
|
283
|
+
**Rows are direct children, not all descendants.** `querySelectorAll('*')` counts nested markup, so a row that expands or an unrelated widget rendering inside the list registers as growth. `:scope > *` counts the list's own items.
|
|
284
|
+
|
|
285
|
+
Both must be **self-contained** — passed to `page.evaluate`, so no outer-scope references, no imports used inside the body.
|
|
286
|
+
|
|
287
|
+
- [ ] **Step 1: Write the failing test**
|
|
288
|
+
|
|
289
|
+
Create `tests/integration/pagination-browser.test.ts`:
|
|
290
|
+
|
|
291
|
+
```ts
|
|
292
|
+
import { afterAll, beforeAll, describe, expect, it } from 'bun:test';
|
|
293
|
+
import { type Browser, chromium } from 'playwright';
|
|
294
|
+
import { measureScroll, restoreScroll } from '../../src/utils/pagination.ts';
|
|
295
|
+
|
|
296
|
+
let browser: Browser;
|
|
297
|
+
|
|
298
|
+
beforeAll(async () => {
|
|
299
|
+
browser = await chromium.launch();
|
|
300
|
+
});
|
|
301
|
+
|
|
302
|
+
afterAll(async () => {
|
|
303
|
+
await browser?.close();
|
|
304
|
+
});
|
|
305
|
+
|
|
306
|
+
const rows = (count: number, prefix = 'row') => Array.from({ length: count }, (_, i) => `<div class="row">${prefix} ${i}</div>`).join('');
|
|
307
|
+
|
|
308
|
+
const pageWith = async (body: string) => {
|
|
309
|
+
const page = await browser.newPage();
|
|
310
|
+
await page.setContent(`<!doctype html><html><body style="margin:0">${body}</body></html>`);
|
|
311
|
+
return page;
|
|
312
|
+
};
|
|
313
|
+
|
|
314
|
+
describe('measureScroll', () => {
|
|
315
|
+
it('reports a container that scrolls inside itself', async () => {
|
|
316
|
+
const page = await pageWith(`<div id="box" style="height:200px;overflow-y:auto">${rows(60)}</div>`);
|
|
317
|
+
|
|
318
|
+
const measure = await page.evaluate(measureScroll, '#box');
|
|
319
|
+
|
|
320
|
+
expect(measure?.ownScroller).toBe(true);
|
|
321
|
+
expect(measure?.rowCount).toBe(60);
|
|
322
|
+
await page.close();
|
|
323
|
+
});
|
|
324
|
+
|
|
325
|
+
it('reports a list that continues below the fold', async () => {
|
|
326
|
+
const page = await pageWith(`<div id="box">${rows(400)}</div>`);
|
|
327
|
+
|
|
328
|
+
const measure = await page.evaluate(measureScroll, '#box');
|
|
329
|
+
|
|
330
|
+
expect(measure?.ownScroller).toBe(false);
|
|
331
|
+
expect(measure?.belowFold).toBe(true);
|
|
332
|
+
await page.close();
|
|
333
|
+
});
|
|
334
|
+
|
|
335
|
+
it('reports neither for a short list', async () => {
|
|
336
|
+
const page = await pageWith('<div id="box"><div class="row">only</div></div>');
|
|
337
|
+
|
|
338
|
+
const measure = await page.evaluate(measureScroll, '#box');
|
|
339
|
+
|
|
340
|
+
expect(measure?.ownScroller).toBe(false);
|
|
341
|
+
expect(measure?.belowFold).toBe(false);
|
|
342
|
+
await page.close();
|
|
343
|
+
});
|
|
344
|
+
|
|
345
|
+
it('returns null for a selector that matches nothing', async () => {
|
|
346
|
+
const page = await pageWith('<div id="box"></div>');
|
|
347
|
+
|
|
348
|
+
expect(await page.evaluate(measureScroll, '#missing')).toBeNull();
|
|
349
|
+
await page.close();
|
|
350
|
+
});
|
|
351
|
+
});
|
|
352
|
+
|
|
353
|
+
describe('restoreScroll', () => {
|
|
354
|
+
it('puts a container scroller back where it was', async () => {
|
|
355
|
+
const page = await pageWith(`<div id="box" style="height:200px;overflow-y:auto">${rows(60)}</div>`);
|
|
356
|
+
|
|
357
|
+
await page.evaluate(() => {
|
|
358
|
+
document.getElementById('box')!.scrollTop = 900;
|
|
359
|
+
});
|
|
360
|
+
await page.evaluate(restoreScroll, { css: '#box', scrollTop: 0, windowScrollY: 0 });
|
|
361
|
+
|
|
362
|
+
expect((await page.evaluate(measureScroll, '#box'))?.scrollTop).toBe(0);
|
|
363
|
+
await page.close();
|
|
364
|
+
});
|
|
365
|
+
|
|
366
|
+
it('puts the window back too, since scrollIntoViewIfNeeded moves both', async () => {
|
|
367
|
+
const page = await pageWith(`<div style="height:2000px">filler</div><div id="box" style="height:200px;overflow-y:auto">${rows(60)}</div><div style="height:2000px">filler</div>`);
|
|
368
|
+
const before = await page.evaluate(measureScroll, '#box');
|
|
369
|
+
|
|
370
|
+
await page.locator('#box > *:last-child').scrollIntoViewIfNeeded();
|
|
371
|
+
const moved = await page.evaluate(measureScroll, '#box');
|
|
372
|
+
await page.evaluate(restoreScroll, { css: '#box', scrollTop: before!.scrollTop, windowScrollY: before!.windowScrollY });
|
|
373
|
+
const restored = await page.evaluate(measureScroll, '#box');
|
|
374
|
+
|
|
375
|
+
expect(moved!.windowScrollY).toBeGreaterThan(0);
|
|
376
|
+
expect(restored!.windowScrollY).toBe(before!.windowScrollY);
|
|
377
|
+
expect(restored!.scrollTop).toBe(before!.scrollTop);
|
|
378
|
+
await page.close();
|
|
379
|
+
});
|
|
380
|
+
});
|
|
381
|
+
```
|
|
382
|
+
|
|
383
|
+
- [ ] **Step 2: Run test to verify it fails**
|
|
384
|
+
|
|
385
|
+
Run: `bun test tests/integration/pagination-browser.test.ts`
|
|
386
|
+
Expected: FAIL — `measureScroll` is not exported.
|
|
387
|
+
|
|
388
|
+
- [ ] **Step 3: Write minimal implementation**
|
|
389
|
+
|
|
390
|
+
Append to `src/utils/pagination.ts`, before the types:
|
|
391
|
+
|
|
392
|
+
```ts
|
|
393
|
+
export function measureScroll(css: string): ScrollMeasure | null {
|
|
394
|
+
const element = document.querySelector(css);
|
|
395
|
+
if (!element) return null;
|
|
396
|
+
const rect = element.getBoundingClientRect();
|
|
397
|
+
return {
|
|
398
|
+
ownScroller: element.scrollHeight > element.clientHeight + 1,
|
|
399
|
+
belowFold: rect.bottom > window.innerHeight,
|
|
400
|
+
scrollTop: element.scrollTop,
|
|
401
|
+
windowScrollY: window.scrollY,
|
|
402
|
+
rowCount: element.querySelectorAll(':scope > *').length,
|
|
403
|
+
};
|
|
404
|
+
}
|
|
405
|
+
|
|
406
|
+
export function restoreScroll({ css, scrollTop, windowScrollY }: ScrollRestore): void {
|
|
407
|
+
const element = document.querySelector(css);
|
|
408
|
+
if (element) element.scrollTop = scrollTop;
|
|
409
|
+
window.scrollTo(0, windowScrollY);
|
|
410
|
+
}
|
|
411
|
+
```
|
|
412
|
+
|
|
413
|
+
Add to the types at the end of the file:
|
|
414
|
+
|
|
415
|
+
```ts
|
|
416
|
+
export interface ScrollMeasure {
|
|
417
|
+
ownScroller: boolean;
|
|
418
|
+
belowFold: boolean;
|
|
419
|
+
scrollTop: number;
|
|
420
|
+
windowScrollY: number;
|
|
421
|
+
rowCount: number;
|
|
422
|
+
}
|
|
423
|
+
|
|
424
|
+
export interface ScrollRestore {
|
|
425
|
+
css: string;
|
|
426
|
+
scrollTop: number;
|
|
427
|
+
windowScrollY: number;
|
|
428
|
+
}
|
|
429
|
+
```
|
|
430
|
+
|
|
431
|
+
`+ 1` in `ownScroller` absorbs sub-pixel layout rounding, which otherwise reports a
|
|
432
|
+
non-scrolling container as scrollable.
|
|
433
|
+
|
|
434
|
+
- [ ] **Step 4: Run test to verify it passes**
|
|
435
|
+
|
|
436
|
+
Run: `bun test tests/integration/pagination-browser.test.ts`
|
|
437
|
+
Expected: PASS, 5 tests.
|
|
438
|
+
|
|
439
|
+
- [ ] **Step 5: Format and commit**
|
|
440
|
+
|
|
441
|
+
```bash
|
|
442
|
+
bun run format
|
|
443
|
+
git add src/utils/pagination.ts tests/integration/pagination-browser.test.ts
|
|
444
|
+
git commit -m "Measure and restore scroll position for the pagination probe"
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
---
|
|
448
|
+
|
|
449
|
+
### Task 4: Researcher pagination mixin
|
|
450
|
+
|
|
451
|
+
**Files:**
|
|
452
|
+
- Create: `src/ai/researcher/pagination.ts`
|
|
453
|
+
- Modify: `src/ai/researcher.ts` (line 47, the `ResearcherBase` composition; line 49, the interface; and the call site near line 277)
|
|
454
|
+
- Modify: `src/ai/researcher/locators.ts` (extract one blockquote writer)
|
|
455
|
+
- Test: `tests/unit/researcher-pagination.test.ts`
|
|
456
|
+
|
|
457
|
+
**Interfaces:**
|
|
458
|
+
- Consumes: `PaginationStrategy`, `measureScroll`, `restoreScroll` (Tasks 1, 3); `parseDataSections`, `parseResearchSections`, `extractPaginationFromBlockquote` (Task 2).
|
|
459
|
+
- Produces: `WithPagination<T>` mixin exposing `detectPagination(result: ResearchResult): Promise<void>`, and `PaginationMethods` for the interface merge.
|
|
460
|
+
|
|
461
|
+
**One writer for the blockquote.** `updateSectionContainer` (`src/ai/researcher/locators.ts:300`) currently composes that blockquote. Extract the composition into a shared helper so `Container:` and `Pagination:` are always emitted by the same code.
|
|
462
|
+
|
|
463
|
+
**This also fixes a live bug.** `mdq(...).query('blockquote[0]').replace(text)` writes `text` verbatim — it does **not** re-add the `>` markers. So today's call:
|
|
464
|
+
|
|
465
|
+
```ts
|
|
466
|
+
result.text = sectionQuery.query('blockquote[0]').replace(`Container: '${newCss}'`);
|
|
467
|
+
```
|
|
468
|
+
|
|
469
|
+
turns the blockquote into a plain paragraph, after which `extractContainerFromBlockquote` (which queries `blockquote[0]`) returns **null** — the container is silently lost for every consumer. Verified against the real `mdq` and `parseResearchSections`:
|
|
470
|
+
|
|
471
|
+
```
|
|
472
|
+
before rewrite, container = .old-toolbar
|
|
473
|
+
after rewrite, container = null
|
|
474
|
+
```
|
|
475
|
+
|
|
476
|
+
Every line the helper emits must therefore carry its own `> ` prefix.
|
|
477
|
+
|
|
478
|
+
- [ ] **Step 1: Extract the shared blockquote writer**
|
|
479
|
+
|
|
480
|
+
In `src/ai/researcher/locators.ts`, export a helper and use it from `updateSectionContainer`:
|
|
481
|
+
|
|
482
|
+
```ts
|
|
483
|
+
export function composeContainerBlockquote(css: string, pagination: PaginationStrategy | null): string {
|
|
484
|
+
let text = `> Container: '${css}'`;
|
|
485
|
+
if (pagination) text += `\n> Pagination: ${pagination}`;
|
|
486
|
+
return text;
|
|
487
|
+
}
|
|
488
|
+
```
|
|
489
|
+
|
|
490
|
+
Add the import at the top of `locators.ts`:
|
|
491
|
+
|
|
492
|
+
```ts
|
|
493
|
+
import type { PaginationStrategy } from '../../utils/pagination.ts';
|
|
494
|
+
```
|
|
495
|
+
|
|
496
|
+
Then in `updateSectionContainer`, replace the `newCss` branch body:
|
|
497
|
+
|
|
498
|
+
```ts
|
|
499
|
+
if (newCss) {
|
|
500
|
+
const pagination = extractPaginationFromBlockquote(section.rawMarkdown);
|
|
501
|
+
result.text = sectionQuery.query('blockquote[0]').replace(composeContainerBlockquote(newCss, pagination));
|
|
502
|
+
} else {
|
|
503
|
+
```
|
|
504
|
+
|
|
505
|
+
and add `extractPaginationFromBlockquote` to the existing `./parser.ts` import in that file.
|
|
506
|
+
|
|
507
|
+
This preserves a recorded `Pagination:` line when a container is later simplified or recovered, and restores the `>` markers the current code drops.
|
|
508
|
+
|
|
509
|
+
- [ ] **Step 1b: Cover the blockquote-survives-rewrite bug**
|
|
510
|
+
|
|
511
|
+
Add to `tests/unit/research-parser-pagination.test.ts`:
|
|
512
|
+
|
|
513
|
+
```ts
|
|
514
|
+
import { composeContainerBlockquote } from '../../src/ai/researcher/locators.ts';
|
|
515
|
+
import { mdq } from '../../src/utils/markdown-query.ts';
|
|
516
|
+
|
|
517
|
+
describe('rewriting a container', () => {
|
|
518
|
+
it('leaves the blockquote readable', () => {
|
|
519
|
+
const markdown = `## Menu\n\n> Container: '.old'\n\n| Element | ARIA | CSS | eidx |\n`;
|
|
520
|
+
|
|
521
|
+
const rewritten = mdq(markdown).query('section2(~"Menu")').query('blockquote[0]').replace(composeContainerBlockquote('.new', null));
|
|
522
|
+
|
|
523
|
+
expect(parseResearchSections(rewritten)[0].containerCss).toBe('.new');
|
|
524
|
+
});
|
|
525
|
+
|
|
526
|
+
it('keeps a recorded strategy through a rewrite', () => {
|
|
527
|
+
const markdown = `## Menu\n\n> Container: '.old'\n> Pagination: controls\n\n| Element | ARIA | CSS | eidx |\n`;
|
|
528
|
+
|
|
529
|
+
const rewritten = mdq(markdown).query('section2(~"Menu")').query('blockquote[0]').replace(composeContainerBlockquote('.new', 'controls'));
|
|
530
|
+
|
|
531
|
+
expect(extractPaginationFromBlockquote(rewritten)).toBe('controls');
|
|
532
|
+
});
|
|
533
|
+
});
|
|
534
|
+
```
|
|
535
|
+
|
|
536
|
+
Run it and watch the first case fail against the current single-line `replace`, then pass with the helper.
|
|
537
|
+
|
|
538
|
+
- [ ] **Step 2: Write the failing test**
|
|
539
|
+
|
|
540
|
+
Create `tests/unit/researcher-pagination.test.ts`. It duck-types Explorer, following `tests/unit/driller.test.ts`:
|
|
541
|
+
|
|
542
|
+
```ts
|
|
543
|
+
import { describe, expect, it } from 'bun:test';
|
|
544
|
+
import { WithPagination } from '../../src/ai/researcher/pagination.ts';
|
|
545
|
+
import { ResearchResult } from '../../src/ai/researcher/research-result.ts';
|
|
546
|
+
|
|
547
|
+
const RESEARCH = `## Menu
|
|
548
|
+
|
|
549
|
+
> Container: '.toolbar'
|
|
550
|
+
|
|
551
|
+
| Element | ARIA | CSS | eidx |
|
|
552
|
+
| Filter | button "Filter" | .filter-btn | 3 |
|
|
553
|
+
|
|
554
|
+
## Data: Suites List
|
|
555
|
+
|
|
556
|
+
> Container: '.suites-list-content'
|
|
557
|
+
|
|
558
|
+
Suite items, 13 items.
|
|
559
|
+
`;
|
|
560
|
+
|
|
561
|
+
class Base {
|
|
562
|
+
explorer: any;
|
|
563
|
+
}
|
|
564
|
+
|
|
565
|
+
const agentWith = (measures: Record<string, any>, afterScrollRows: number) => {
|
|
566
|
+
const Agent = WithPagination(Base as any);
|
|
567
|
+
const agent = new Agent();
|
|
568
|
+
let scrolled = false;
|
|
569
|
+
agent.explorer = {
|
|
570
|
+
withPage: async (fn: any) =>
|
|
571
|
+
fn({
|
|
572
|
+
evaluate: async (_fn: any, arg: any) => {
|
|
573
|
+
const css = typeof arg === 'string' ? arg : arg?.css;
|
|
574
|
+
const measure = measures[css];
|
|
575
|
+
if (!measure) return null;
|
|
576
|
+
if (scrolled) return { ...measure, rowCount: afterScrollRows };
|
|
577
|
+
return measure;
|
|
578
|
+
},
|
|
579
|
+
}),
|
|
580
|
+
action: () => ({
|
|
581
|
+
attempt: async () => {
|
|
582
|
+
scrolled = true;
|
|
583
|
+
return true;
|
|
584
|
+
},
|
|
585
|
+
}),
|
|
586
|
+
};
|
|
587
|
+
return agent;
|
|
588
|
+
};
|
|
589
|
+
|
|
590
|
+
describe('detectPagination', () => {
|
|
591
|
+
it('records infinite when scrolling adds rows', async () => {
|
|
592
|
+
const result = new ResearchResult(RESEARCH, '/suites');
|
|
593
|
+
const agent = agentWith({ '.suites-list-content': { ownScroller: true, belowFold: false, scrollTop: 0, windowScrollY: 0, rowCount: 20 } }, 40);
|
|
594
|
+
|
|
595
|
+
await agent.detectPagination(result);
|
|
596
|
+
|
|
597
|
+
expect(result.text).toContain('Pagination: infinite');
|
|
598
|
+
});
|
|
599
|
+
|
|
600
|
+
it('records nothing when scrolling adds no rows', async () => {
|
|
601
|
+
const result = new ResearchResult(RESEARCH, '/suites');
|
|
602
|
+
const agent = agentWith({ '.suites-list-content': { ownScroller: true, belowFold: false, scrollTop: 0, windowScrollY: 0, rowCount: 20 } }, 20);
|
|
603
|
+
|
|
604
|
+
await agent.detectPagination(result);
|
|
605
|
+
|
|
606
|
+
expect(result.text).not.toContain('Pagination:');
|
|
607
|
+
});
|
|
608
|
+
|
|
609
|
+
it('does not probe a section that cannot scroll', async () => {
|
|
610
|
+
const result = new ResearchResult(RESEARCH, '/suites');
|
|
611
|
+
const agent = agentWith({ '.suites-list-content': { ownScroller: false, belowFold: false, scrollTop: 0, windowScrollY: 0, rowCount: 5 } }, 99);
|
|
612
|
+
|
|
613
|
+
await agent.detectPagination(result);
|
|
614
|
+
|
|
615
|
+
expect(result.text).not.toContain('Pagination:');
|
|
616
|
+
});
|
|
617
|
+
|
|
618
|
+
it('leaves an already recorded strategy alone', async () => {
|
|
619
|
+
const recorded = RESEARCH.replace("> Container: '.suites-list-content'", "> Container: '.suites-list-content'\n> Pagination: controls");
|
|
620
|
+
const result = new ResearchResult(recorded, '/suites');
|
|
621
|
+
const agent = agentWith({ '.suites-list-content': { ownScroller: true, belowFold: false, scrollTop: 0, windowScrollY: 0, rowCount: 20 } }, 40);
|
|
622
|
+
|
|
623
|
+
await agent.detectPagination(result);
|
|
624
|
+
|
|
625
|
+
expect(result.text).toContain('Pagination: controls');
|
|
626
|
+
expect(result.text).not.toContain('Pagination: infinite');
|
|
627
|
+
});
|
|
628
|
+
});
|
|
629
|
+
```
|
|
630
|
+
|
|
631
|
+
- [ ] **Step 3: Run test to verify it fails**
|
|
632
|
+
|
|
633
|
+
Run: `bun test tests/unit/researcher-pagination.test.ts`
|
|
634
|
+
Expected: FAIL — cannot resolve `src/ai/researcher/pagination.ts`.
|
|
635
|
+
|
|
636
|
+
- [ ] **Step 4: Write the mixin**
|
|
637
|
+
|
|
638
|
+
Create `src/ai/researcher/pagination.ts`:
|
|
639
|
+
|
|
640
|
+
```ts
|
|
641
|
+
import type Explorer from '../../explorer.ts';
|
|
642
|
+
import { mdq } from '../../utils/markdown-query.ts';
|
|
643
|
+
import { type PaginationStrategy, measureScroll, restoreScroll } from '../../utils/pagination.ts';
|
|
644
|
+
import { composeContainerBlockquote } from './locators.ts';
|
|
645
|
+
import { type Constructor, debugLog } from './mixin.ts';
|
|
646
|
+
import { extractPaginationFromBlockquote, parseDataSections, parseResearchSections } from './parser.ts';
|
|
647
|
+
import type { ResearchResult } from './research-result.ts';
|
|
648
|
+
|
|
649
|
+
export function WithPagination<T extends Constructor>(Base: T) {
|
|
650
|
+
return class extends Base {
|
|
651
|
+
declare explorer: Explorer;
|
|
652
|
+
|
|
653
|
+
async detectPagination(result: ResearchResult): Promise<void> {
|
|
654
|
+
const sections = [...parseResearchSections(result.text), ...parseDataSections(result.text)];
|
|
655
|
+
|
|
656
|
+
for (const section of sections) {
|
|
657
|
+
const css = section.containerCss;
|
|
658
|
+
if (!css) continue;
|
|
659
|
+
if (extractPaginationFromBlockquote(section.rawMarkdown)) continue;
|
|
660
|
+
|
|
661
|
+
const strategy = await this.probeSection(css);
|
|
662
|
+
if (!strategy) continue;
|
|
663
|
+
|
|
664
|
+
this.recordPagination(result, section.name, css, strategy);
|
|
665
|
+
debugLog(`Pagination in "${section.name}": ${strategy}`);
|
|
666
|
+
}
|
|
667
|
+
}
|
|
668
|
+
|
|
669
|
+
private async probeSection(css: string): Promise<PaginationStrategy | null> {
|
|
670
|
+
const before = await this.explorer.withPage((page) => page.evaluate(measureScroll, css));
|
|
671
|
+
if (!before) return null;
|
|
672
|
+
if (!before.ownScroller && !before.belowFold) return null;
|
|
673
|
+
|
|
674
|
+
const action = this.explorer.action();
|
|
675
|
+
const scrolled = await action.attempt(`I.scrollTo('${css} > *:last-child')`);
|
|
676
|
+
if (!scrolled) return null;
|
|
677
|
+
|
|
678
|
+
const after = await this.explorer.withPage((page) => page.evaluate(measureScroll, css));
|
|
679
|
+
await this.explorer.withPage((page) => page.evaluate(restoreScroll, { css, scrollTop: before.scrollTop, windowScrollY: before.windowScrollY }));
|
|
680
|
+
|
|
681
|
+
if (!after) return null;
|
|
682
|
+
if (after.rowCount > before.rowCount) return 'infinite';
|
|
683
|
+
return null;
|
|
684
|
+
}
|
|
685
|
+
|
|
686
|
+
private recordPagination(result: ResearchResult, name: string, css: string, strategy: PaginationStrategy): void {
|
|
687
|
+
const escaped = name.replace(/"/g, '\\"');
|
|
688
|
+
let sectionQuery = mdq(result.text).query(`section2(~"${escaped}")`);
|
|
689
|
+
if (sectionQuery.count() === 0) sectionQuery = mdq(result.text).query(`section3(~"${escaped}")`);
|
|
690
|
+
if (sectionQuery.count() === 0) return;
|
|
691
|
+
result.text = sectionQuery.query('blockquote[0]').replace(composeContainerBlockquote(css, strategy));
|
|
692
|
+
}
|
|
693
|
+
};
|
|
694
|
+
}
|
|
695
|
+
|
|
696
|
+
export interface PaginationMethods {
|
|
697
|
+
detectPagination(result: ResearchResult): Promise<void>;
|
|
698
|
+
}
|
|
699
|
+
```
|
|
700
|
+
|
|
701
|
+
**Note on the scroll target:** `'<css> > *:last-child'` targets the list's last child, and
|
|
702
|
+
`scrollIntoViewIfNeeded` walks every scrollable ancestor from there — which is what reaches a
|
|
703
|
+
container with its own scrollbar. Verified against Chromium during design.
|
|
704
|
+
|
|
705
|
+
**Rows are the evidence, not requests.** `Action.networkRequests` is private (`src/action.ts:46`)
|
|
706
|
+
and must stay so. A page fires telemetry and prefetches while scrolling, so a request count
|
|
707
|
+
would report growth where there is none — the same reason the tester's rule says a request with
|
|
708
|
+
no rows means nothing arrived.
|
|
709
|
+
|
|
710
|
+
- [ ] **Step 5: Run test to verify it passes**
|
|
711
|
+
|
|
712
|
+
Run: `bun test tests/unit/researcher-pagination.test.ts`
|
|
713
|
+
Expected: PASS, 4 tests.
|
|
714
|
+
|
|
715
|
+
A colon in the heading is fine — `mdq(...).query('section2(~"Data: Suites List")')` was verified against the real `mdq` to return exactly 1 match.
|
|
716
|
+
|
|
717
|
+
- [ ] **Step 6: Wire the mixin into Researcher**
|
|
718
|
+
|
|
719
|
+
In `src/ai/researcher.ts`, add the import beside the other mixin imports:
|
|
720
|
+
|
|
721
|
+
```ts
|
|
722
|
+
import { type PaginationMethods, WithPagination } from './researcher/pagination.ts';
|
|
723
|
+
```
|
|
724
|
+
|
|
725
|
+
Change line 47:
|
|
726
|
+
|
|
727
|
+
```ts
|
|
728
|
+
const ResearcherBase = WithSections(WithPagination(WithDeepAnalysis(WithCoordinates(WithLocators(TaskAgent as unknown as new (...args: any[]) => TaskAgent)))));
|
|
729
|
+
```
|
|
730
|
+
|
|
731
|
+
Change line 49:
|
|
732
|
+
|
|
733
|
+
```ts
|
|
734
|
+
export interface Researcher extends LocatorMethods, CoordinateMethods, DeepAnalysisMethods, SectionMethods, PaginationMethods {}
|
|
735
|
+
```
|
|
736
|
+
|
|
737
|
+
Call it after container validation and before deep analysis — replace the `backfillBrokenLocators` block's tail near line 277 so the order is: backfill, then pagination, then the focused-section fallback:
|
|
738
|
+
|
|
739
|
+
```ts
|
|
740
|
+
if (!interrupted()) {
|
|
741
|
+
await this.detectPagination(result);
|
|
742
|
+
}
|
|
743
|
+
```
|
|
744
|
+
|
|
745
|
+
- [ ] **Step 7: Verify nothing regressed**
|
|
746
|
+
|
|
747
|
+
Run: `bun test tests/unit/ tests/integration/researcher-sections.test.ts`
|
|
748
|
+
Expected: PASS.
|
|
749
|
+
|
|
750
|
+
- [ ] **Step 8: Format and commit**
|
|
751
|
+
|
|
752
|
+
```bash
|
|
753
|
+
bun run format
|
|
754
|
+
git add src/ai/researcher/pagination.ts src/ai/researcher/locators.ts src/ai/researcher.ts tests/unit/researcher-pagination.test.ts
|
|
755
|
+
git commit -m "Probe list containers for infinite scroll during research"
|
|
756
|
+
```
|
|
757
|
+
|
|
758
|
+
---
|
|
759
|
+
|
|
760
|
+
### Task 5: Researcher rule for pagination controls
|
|
761
|
+
|
|
762
|
+
**Files:**
|
|
763
|
+
- Create: `rules/researcher/pagination.md`
|
|
764
|
+
- Modify: `src/ai/researcher.ts:429` (the **primary** research prompt)
|
|
765
|
+
- Modify: `src/ai/researcher/sections.ts:81` (the per-section fallback)
|
|
766
|
+
- Modify: `src/ai/researcher.ts` (the `<output_rules>` Data-section instructions, near line 502)
|
|
767
|
+
|
|
768
|
+
**Both rule-loading sites, or this step almost never runs.** `researchBySections()` is reached only from the `ContextLengthError` catch (`src/ai/researcher.ts:193`) — it is the fallback for an over-long response. Ordinary research goes through `researchRules()`, which loads its rules at `src/ai/researcher.ts:429`. Adding `'pagination'` to only the fallback would leave step 1 dead on almost every run.
|
|
769
|
+
|
|
770
|
+
**Interfaces:**
|
|
771
|
+
- Consumes: the `> Pagination:` vocabulary from Task 2.
|
|
772
|
+
- Produces: research text that may carry `> Pagination: controls`, which Task 4 leaves alone and Task 6 reads.
|
|
773
|
+
|
|
774
|
+
- [ ] **Step 1: Write the rule file**
|
|
775
|
+
|
|
776
|
+
Create `rules/researcher/pagination.md`:
|
|
777
|
+
|
|
778
|
+
```markdown
|
|
779
|
+
<pagination_controls>
|
|
780
|
+
When a section holds controls that move between pages of the same collection — previous, next,
|
|
781
|
+
a page number, or a control that loads the following batch — add a second line to the section's
|
|
782
|
+
container blockquote:
|
|
783
|
+
|
|
784
|
+
> Pagination: controls
|
|
785
|
+
|
|
786
|
+
Judge by what the control does to the collection, not by its wording or its icon. Controls that
|
|
787
|
+
sort, filter, or switch between different collections are not pagination.
|
|
788
|
+
Omit the line when the section has no such control.
|
|
789
|
+
</pagination_controls>
|
|
790
|
+
```
|
|
791
|
+
|
|
792
|
+
- [ ] **Step 2: Load it in both research paths**
|
|
793
|
+
|
|
794
|
+
In `src/ai/researcher.ts:429` — the primary path:
|
|
795
|
+
|
|
796
|
+
```ts
|
|
797
|
+
${RulesLoader.loadRules('researcher', ['ui-map-table', 'list-element', 'container-rules', 'pagination'], currentUrl)}
|
|
798
|
+
```
|
|
799
|
+
|
|
800
|
+
And in `src/ai/researcher/sections.ts:81` — the fallback:
|
|
801
|
+
|
|
802
|
+
```ts
|
|
803
|
+
const rules = RulesLoader.loadRules('researcher', ['ui-map-table', 'list-element', 'container-rules', 'pagination'], currentUrl);
|
|
804
|
+
```
|
|
805
|
+
|
|
806
|
+
- [ ] **Step 3: Extend the Data-section instructions**
|
|
807
|
+
|
|
808
|
+
In `src/ai/researcher.ts`, in `<output_rules>` after the line reading
|
|
809
|
+
`- Data sections must NOT include a UI map table. Only include the container and a brief summary line.`, add:
|
|
810
|
+
|
|
811
|
+
```
|
|
812
|
+
- When the data list has controls that move between pages of the collection, add "> Pagination: controls" under its container.
|
|
813
|
+
```
|
|
814
|
+
|
|
815
|
+
- [ ] **Step 4: Verify the rule reaches both prompts**
|
|
816
|
+
|
|
817
|
+
Run: `bun test tests/integration/researcher-sections.test.ts tests/integration/researcher.test.ts`
|
|
818
|
+
Expected: PASS. Add an assertion in each that the prompt sent to the model contains `pagination_controls`, using aimock's Journal (`mock.getLastRequest()`) the way `tests/integration/planner.test.ts` does.
|
|
819
|
+
|
|
820
|
+
- [ ] **Step 5: Format and commit**
|
|
821
|
+
|
|
822
|
+
```bash
|
|
823
|
+
bun run format
|
|
824
|
+
git add rules/researcher/pagination.md src/ai/researcher/sections.ts src/ai/researcher.ts
|
|
825
|
+
git commit -m "Ask research to note pagination controls it can see"
|
|
826
|
+
```
|
|
827
|
+
|
|
828
|
+
---
|
|
829
|
+
|
|
830
|
+
### Task 6: Conditional rule injection into Tester and Navigator
|
|
831
|
+
|
|
832
|
+
**Files:**
|
|
833
|
+
- Modify: `src/ai/rules.ts` (add two exports)
|
|
834
|
+
- Modify: `src/ai/tester.ts` (`reinjectContextIfNeeded`, from line 554)
|
|
835
|
+
- Modify: `src/ai/navigator.ts` (its per-state context near line 414)
|
|
836
|
+
- Test: `tests/unit/pagination-rule-injection.test.ts`
|
|
837
|
+
|
|
838
|
+
**Interfaces:**
|
|
839
|
+
- Consumes: `detectPaginationMarkers` (Task 1), `extractPaginationFromBlockquote`, `parseDataSections`, `parseResearchSections` (Task 2).
|
|
840
|
+
- Produces: `paginationRuleFor(strategy: PaginationStrategy): string` and `paginationFromResearch(researchText: string): PaginationStrategy | null`.
|
|
841
|
+
|
|
842
|
+
- [ ] **Step 1: Write the failing test**
|
|
843
|
+
|
|
844
|
+
Create `tests/unit/pagination-rule-injection.test.ts`:
|
|
845
|
+
|
|
846
|
+
```ts
|
|
847
|
+
import { describe, expect, it } from 'bun:test';
|
|
848
|
+
import { paginationFromResearch, paginationRuleFor } from '../../src/ai/rules.ts';
|
|
849
|
+
|
|
850
|
+
describe('paginationRuleFor', () => {
|
|
851
|
+
it('tells the model to click through pages for controls', () => {
|
|
852
|
+
const rule = paginationRuleFor('controls');
|
|
853
|
+
|
|
854
|
+
expect(rule).toContain('<pagination>');
|
|
855
|
+
expect(rule).toContain('next');
|
|
856
|
+
expect(rule).not.toContain('scroll');
|
|
857
|
+
});
|
|
858
|
+
|
|
859
|
+
it('tells the model to scroll for infinite', () => {
|
|
860
|
+
const rule = paginationRuleFor('infinite');
|
|
861
|
+
|
|
862
|
+
expect(rule).toContain('<pagination>');
|
|
863
|
+
expect(rule).toContain('scroll');
|
|
864
|
+
});
|
|
865
|
+
});
|
|
866
|
+
|
|
867
|
+
describe('paginationFromResearch', () => {
|
|
868
|
+
it('reads a strategy recorded in a Data section', () => {
|
|
869
|
+
const research = `## Data: Rows\n\n> Container: '.rows'\n> Pagination: infinite\n\nRow items.\n`;
|
|
870
|
+
|
|
871
|
+
expect(paginationFromResearch(research)).toBe('infinite');
|
|
872
|
+
});
|
|
873
|
+
|
|
874
|
+
it('reads a strategy recorded in a normal section', () => {
|
|
875
|
+
const research = `## Menu\n\n> Container: '.toolbar'\n> Pagination: controls\n\n| Element | ARIA | CSS | eidx |\n`;
|
|
876
|
+
|
|
877
|
+
expect(paginationFromResearch(research)).toBe('controls');
|
|
878
|
+
});
|
|
879
|
+
|
|
880
|
+
it('returns null when no section records one', () => {
|
|
881
|
+
const research = `## Menu\n\n> Container: '.toolbar'\n\n| Element | ARIA | CSS | eidx |\n`;
|
|
882
|
+
|
|
883
|
+
expect(paginationFromResearch(research)).toBeNull();
|
|
884
|
+
});
|
|
885
|
+
|
|
886
|
+
it('returns null for empty research', () => {
|
|
887
|
+
expect(paginationFromResearch('')).toBeNull();
|
|
888
|
+
});
|
|
889
|
+
});
|
|
890
|
+
```
|
|
891
|
+
|
|
892
|
+
- [ ] **Step 2: Run test to verify it fails**
|
|
893
|
+
|
|
894
|
+
Run: `bun test tests/unit/pagination-rule-injection.test.ts`
|
|
895
|
+
Expected: FAIL — `paginationRuleFor` is not exported from `src/ai/rules.ts`.
|
|
896
|
+
|
|
897
|
+
- [ ] **Step 3: Write the rule fragments**
|
|
898
|
+
|
|
899
|
+
In `src/ai/rules.ts`, add the imports at the top:
|
|
900
|
+
|
|
901
|
+
```ts
|
|
902
|
+
import { extractPaginationFromBlockquote, parseDataSections, parseResearchSections } from './researcher/parser.ts';
|
|
903
|
+
import type { PaginationStrategy } from '../utils/pagination.ts';
|
|
904
|
+
```
|
|
905
|
+
|
|
906
|
+
Add these exports:
|
|
907
|
+
|
|
908
|
+
```ts
|
|
909
|
+
const paginationControlsRule = dedent`
|
|
910
|
+
<pagination>
|
|
911
|
+
This list pages through a larger collection. If what you need is not on screen,
|
|
912
|
+
click next or the page number you need before concluding it is absent.
|
|
913
|
+
</pagination>
|
|
914
|
+
`;
|
|
915
|
+
|
|
916
|
+
const infiniteScrollRule = dedent`
|
|
917
|
+
<pagination>
|
|
918
|
+
This list grows as it is scrolled. If what you need is not on screen, scroll to
|
|
919
|
+
the last item in the list — every scrollable ancestor of that item scrolls, so
|
|
920
|
+
this reaches a list with its own scrollbar.
|
|
921
|
+
|
|
922
|
+
New rows in the aria changes mean more arrived; a request with none means nothing
|
|
923
|
+
was left. Stop on the first attempt that adds no rows: the end of a collection is
|
|
924
|
+
an answer, not a failure.
|
|
925
|
+
</pagination>
|
|
926
|
+
`;
|
|
927
|
+
|
|
928
|
+
export function paginationRuleFor(strategy: PaginationStrategy): string {
|
|
929
|
+
if (strategy === 'controls') return paginationControlsRule;
|
|
930
|
+
return infiniteScrollRule;
|
|
931
|
+
}
|
|
932
|
+
|
|
933
|
+
export function paginationFromResearch(researchText: string): PaginationStrategy | null {
|
|
934
|
+
if (!researchText) return null;
|
|
935
|
+
const sections = [...parseResearchSections(researchText), ...parseDataSections(researchText)];
|
|
936
|
+
for (const section of sections) {
|
|
937
|
+
const strategy = extractPaginationFromBlockquote(section.rawMarkdown);
|
|
938
|
+
if (strategy) return strategy;
|
|
939
|
+
}
|
|
940
|
+
return null;
|
|
941
|
+
}
|
|
942
|
+
```
|
|
943
|
+
|
|
944
|
+
- [ ] **Step 4: Run test to verify it passes**
|
|
945
|
+
|
|
946
|
+
Run: `bun test tests/unit/pagination-rule-injection.test.ts`
|
|
947
|
+
Expected: PASS, 6 tests.
|
|
948
|
+
|
|
949
|
+
- [ ] **Step 5: Inject into Tester**
|
|
950
|
+
|
|
951
|
+
In `src/ai/tester.ts`, add to the existing `./rules.ts` import: `paginationFromResearch`, `paginationRuleFor`. Add to the `./utils/pagination.ts` import: `detectPaginationMarkers`, and `type PaginationStrategy`.
|
|
952
|
+
|
|
953
|
+
Add a field beside `seenUiMapUrls`:
|
|
954
|
+
|
|
955
|
+
```ts
|
|
956
|
+
private paginationByUrl = new Map<string, PaginationStrategy>();
|
|
957
|
+
```
|
|
958
|
+
|
|
959
|
+
Inside `reinjectContextIfNeeded`, in the `if (isNewUrl)` block, right after `this.seenUiMapUrls.add(currentUrl);`:
|
|
960
|
+
|
|
961
|
+
```ts
|
|
962
|
+
const recorded = paginationFromResearch(research);
|
|
963
|
+
if (recorded) this.paginationByUrl.set(currentUrl, recorded);
|
|
964
|
+
```
|
|
965
|
+
|
|
966
|
+
**Mind the early returns.** This method returns from three places: `return context;` at the end of the `isNewUrl` block (`src/ai/tester.ts:676`), `if (context) return context;` further down, and a final `dedent` fallback. Appending "at the end of the method" would never reach a new URL — which is the case that matters most.
|
|
967
|
+
|
|
968
|
+
Add a private method after the existing public methods:
|
|
969
|
+
|
|
970
|
+
```ts
|
|
971
|
+
private paginationContext(currentState: ActionResult, currentUrl: string): string {
|
|
972
|
+
let strategy = detectPaginationMarkers(currentState.html);
|
|
973
|
+
if (!strategy) strategy = this.paginationByUrl.get(currentUrl) ?? null;
|
|
974
|
+
if (!strategy) return '';
|
|
975
|
+
return `\n${paginationRuleFor(strategy)}\n`;
|
|
976
|
+
}
|
|
977
|
+
```
|
|
978
|
+
|
|
979
|
+
Call it immediately before **both** of the first two returns:
|
|
980
|
+
|
|
981
|
+
```ts
|
|
982
|
+
context += this.paginationContext(currentState, currentUrl);
|
|
983
|
+
return context;
|
|
984
|
+
}
|
|
985
|
+
```
|
|
986
|
+
|
|
987
|
+
and
|
|
988
|
+
|
|
989
|
+
```ts
|
|
990
|
+
context += this.paginationContext(currentState, currentUrl);
|
|
991
|
+
if (context) return context;
|
|
992
|
+
```
|
|
993
|
+
|
|
994
|
+
The third return is the every-fifth-iteration ARIA refresh, which carries no rules — leave it alone.
|
|
995
|
+
|
|
996
|
+
- [ ] **Step 6: Inject into Navigator**
|
|
997
|
+
|
|
998
|
+
In `src/ai/navigator.ts`, add `paginationRuleFor` to the existing `./rules.js` import and `detectPaginationMarkers` from `../utils/pagination.ts`.
|
|
999
|
+
|
|
1000
|
+
At the prompt built near line 414, add a local before the `dedent` and a slot inside it:
|
|
1001
|
+
|
|
1002
|
+
```ts
|
|
1003
|
+
let paginationHint = '';
|
|
1004
|
+
const strategy = detectPaginationMarkers(actionResult.html);
|
|
1005
|
+
if (strategy) paginationHint = paginationRuleFor(strategy);
|
|
1006
|
+
```
|
|
1007
|
+
|
|
1008
|
+
Place `${paginationHint}` directly after `${actionRule}` in that template.
|
|
1009
|
+
|
|
1010
|
+
Navigator gets the marker path only — it has no research text in hand at that point.
|
|
1011
|
+
|
|
1012
|
+
- [ ] **Step 7: Verify**
|
|
1013
|
+
|
|
1014
|
+
Run: `bun test tests/unit/ tests/integration/`
|
|
1015
|
+
Expected: PASS.
|
|
1016
|
+
|
|
1017
|
+
- [ ] **Step 8: Format and commit**
|
|
1018
|
+
|
|
1019
|
+
```bash
|
|
1020
|
+
bun run format
|
|
1021
|
+
git add src/ai/rules.ts src/ai/tester.ts src/ai/navigator.ts tests/unit/pagination-rule-injection.test.ts
|
|
1022
|
+
git commit -m "Inject the pagination rule only where a strategy was detected"
|
|
1023
|
+
```
|
|
1024
|
+
|
|
1025
|
+
---
|
|
1026
|
+
|
|
1027
|
+
### Task 7: `actionRule` documents scrolling
|
|
1028
|
+
|
|
1029
|
+
**Files:**
|
|
1030
|
+
- Modify: `src/ai/rules.ts` (`actionRule`, from line 307)
|
|
1031
|
+
- Modify: `src/ai/tools.ts` (the `form` tool description, from line 354)
|
|
1032
|
+
|
|
1033
|
+
**Interfaces:**
|
|
1034
|
+
- Consumes: nothing.
|
|
1035
|
+
- Produces: nothing other tasks import.
|
|
1036
|
+
|
|
1037
|
+
- [ ] **Step 1: Add the scroll commands to `<actions>`**
|
|
1038
|
+
|
|
1039
|
+
In `src/ai/rules.ts`, inside the `actionRule` dedent block, after the `I.click` subsection, add:
|
|
1040
|
+
|
|
1041
|
+
```
|
|
1042
|
+
### I.scrollTo
|
|
1043
|
+
|
|
1044
|
+
scrolls until the element is in view
|
|
1045
|
+
|
|
1046
|
+
I.scrollTo(<locator>)
|
|
1047
|
+
|
|
1048
|
+
Scrolls every scrollable ancestor of the target, so it reaches an element inside a container
|
|
1049
|
+
that has its own scrollbar. I.scrollPageToBottom() moves only the page itself.
|
|
1050
|
+
|
|
1051
|
+
<example>
|
|
1052
|
+
I.scrollTo('.rows > *:last-child');
|
|
1053
|
+
I.scrollTo({ role: 'listitem', text: 'Last entry' });
|
|
1054
|
+
I.scrollPageToBottom();
|
|
1055
|
+
</example>
|
|
1056
|
+
```
|
|
1057
|
+
|
|
1058
|
+
- [ ] **Step 2: Add the use case to the `form` tool description**
|
|
1059
|
+
|
|
1060
|
+
In `src/ai/tools.ts`, in the `form` tool's `Use cases:` list, add:
|
|
1061
|
+
|
|
1062
|
+
```
|
|
1063
|
+
- Reaching items further down a list (I.scrollTo)
|
|
1064
|
+
```
|
|
1065
|
+
|
|
1066
|
+
- [ ] **Step 3: Verify**
|
|
1067
|
+
|
|
1068
|
+
Run: `bun test tests/integration/`
|
|
1069
|
+
Expected: PASS.
|
|
1070
|
+
|
|
1071
|
+
- [ ] **Step 4: Format and commit**
|
|
1072
|
+
|
|
1073
|
+
```bash
|
|
1074
|
+
bun run format
|
|
1075
|
+
git add src/ai/rules.ts src/ai/tools.ts
|
|
1076
|
+
git commit -m "Document the scroll commands the model can use"
|
|
1077
|
+
```
|
|
1078
|
+
|
|
1079
|
+
---
|
|
1080
|
+
|
|
1081
|
+
### Task 8: Thread the ARIA added/removed split
|
|
1082
|
+
|
|
1083
|
+
**Files:**
|
|
1084
|
+
- Modify: `src/utils/aria.ts` (`AriaDiff` at 587, `diffAriaSnapshots` at 503)
|
|
1085
|
+
- Modify: `src/action-result.ts` (`PageDiff` at 46, `Diff` at 650, the populate site at 552, the calculate site at 740)
|
|
1086
|
+
- Test: `tests/unit/aria.test.ts` (extend)
|
|
1087
|
+
|
|
1088
|
+
**Interfaces:**
|
|
1089
|
+
- Consumes: nothing.
|
|
1090
|
+
- Produces: `AriaDiff.added: number`, `AriaDiff.removed: number`; `PageDiff.ariaAdded?: number`, `PageDiff.ariaRemoved?: number`. Task 9 reads the `PageDiff` fields.
|
|
1091
|
+
|
|
1092
|
+
- [ ] **Step 1: Write the failing test**
|
|
1093
|
+
|
|
1094
|
+
Add to `tests/unit/aria.test.ts` inside the existing `describe('aria', ...)`:
|
|
1095
|
+
|
|
1096
|
+
```ts
|
|
1097
|
+
it('counts additions and removals separately', () => {
|
|
1098
|
+
const before = `- list:\n - listitem "One"`;
|
|
1099
|
+
const after = `- list:\n - listitem "One"\n - listitem "Two"\n - listitem "Three"`;
|
|
1100
|
+
|
|
1101
|
+
const diff = diffAriaSnapshots(before, after);
|
|
1102
|
+
|
|
1103
|
+
expect(diff.added).toBe(2);
|
|
1104
|
+
expect(diff.removed).toBe(0);
|
|
1105
|
+
});
|
|
1106
|
+
|
|
1107
|
+
it('counts a replacement as both', () => {
|
|
1108
|
+
const before = `- list:\n - listitem "One"`;
|
|
1109
|
+
const after = `- list:\n - listitem "Two"`;
|
|
1110
|
+
|
|
1111
|
+
const diff = diffAriaSnapshots(before, after);
|
|
1112
|
+
|
|
1113
|
+
expect(diff.added).toBe(1);
|
|
1114
|
+
expect(diff.removed).toBe(1);
|
|
1115
|
+
});
|
|
1116
|
+
```
|
|
1117
|
+
|
|
1118
|
+
- [ ] **Step 2: Run test to verify it fails**
|
|
1119
|
+
|
|
1120
|
+
Run: `bun test tests/unit/aria.test.ts`
|
|
1121
|
+
Expected: FAIL — `diff.added` is undefined.
|
|
1122
|
+
|
|
1123
|
+
- [ ] **Step 3: Widen `AriaDiff`**
|
|
1124
|
+
|
|
1125
|
+
In `src/utils/aria.ts`, change the interface at line 587:
|
|
1126
|
+
|
|
1127
|
+
```ts
|
|
1128
|
+
export interface AriaDiff {
|
|
1129
|
+
text: string | null;
|
|
1130
|
+
count: number;
|
|
1131
|
+
added: number;
|
|
1132
|
+
removed: number;
|
|
1133
|
+
}
|
|
1134
|
+
```
|
|
1135
|
+
|
|
1136
|
+
And the return at the end of `diffAriaSnapshots` (line 524):
|
|
1137
|
+
|
|
1138
|
+
```ts
|
|
1139
|
+
return {
|
|
1140
|
+
text: formatDiff(added, removed, toggled, typed),
|
|
1141
|
+
count: added.length + removed.length + toggled.length + typed.length,
|
|
1142
|
+
added: added.length,
|
|
1143
|
+
removed: removed.length,
|
|
1144
|
+
};
|
|
1145
|
+
```
|
|
1146
|
+
|
|
1147
|
+
- [ ] **Step 4: Run test to verify it passes**
|
|
1148
|
+
|
|
1149
|
+
Run: `bun test tests/unit/aria.test.ts`
|
|
1150
|
+
Expected: PASS.
|
|
1151
|
+
|
|
1152
|
+
- [ ] **Step 5: Carry the split into `PageDiff`**
|
|
1153
|
+
|
|
1154
|
+
In `src/action-result.ts`, add to the `PageDiff` interface after `ariaChangeCount?: number;`:
|
|
1155
|
+
|
|
1156
|
+
```ts
|
|
1157
|
+
ariaAdded?: number;
|
|
1158
|
+
ariaRemoved?: number;
|
|
1159
|
+
```
|
|
1160
|
+
|
|
1161
|
+
In `Diff`, add the fields beside `_ariaChangeCount`:
|
|
1162
|
+
|
|
1163
|
+
```ts
|
|
1164
|
+
private _ariaAdded = 0;
|
|
1165
|
+
private _ariaRemoved = 0;
|
|
1166
|
+
```
|
|
1167
|
+
|
|
1168
|
+
Set them where `diffAriaSnapshots` is called (line 740):
|
|
1169
|
+
|
|
1170
|
+
```ts
|
|
1171
|
+
this._ariaAdded = ariaDiff.added;
|
|
1172
|
+
this._ariaRemoved = ariaDiff.removed;
|
|
1173
|
+
```
|
|
1174
|
+
|
|
1175
|
+
Expose them beside the existing `ariaChangeCount` getter at line 708, matching its style:
|
|
1176
|
+
|
|
1177
|
+
```ts
|
|
1178
|
+
get ariaAdded(): number {
|
|
1179
|
+
return this._ariaAdded;
|
|
1180
|
+
}
|
|
1181
|
+
|
|
1182
|
+
get ariaRemoved(): number {
|
|
1183
|
+
return this._ariaRemoved;
|
|
1184
|
+
}
|
|
1185
|
+
```
|
|
1186
|
+
|
|
1187
|
+
Populate them where `ariaChangeCount` is set (line 553):
|
|
1188
|
+
|
|
1189
|
+
```ts
|
|
1190
|
+
pageDiff.ariaAdded = diff.ariaAdded;
|
|
1191
|
+
pageDiff.ariaRemoved = diff.ariaRemoved;
|
|
1192
|
+
```
|
|
1193
|
+
|
|
1194
|
+
**Leave line 572 alone.** The other `LARGE_ARIA_CHANGE_THRESHOLD` comparison decides whether to attach iframe snapshots on a big change. That is unrelated to mode-change semantics and must keep using the total.
|
|
1195
|
+
|
|
1196
|
+
- [ ] **Step 6: Verify**
|
|
1197
|
+
|
|
1198
|
+
Run: `bun test tests/unit/`
|
|
1199
|
+
Expected: PASS.
|
|
1200
|
+
|
|
1201
|
+
- [ ] **Step 7: Format and commit**
|
|
1202
|
+
|
|
1203
|
+
```bash
|
|
1204
|
+
bun run format
|
|
1205
|
+
git add src/utils/aria.ts src/action-result.ts tests/unit/aria.test.ts
|
|
1206
|
+
git commit -m "Carry the ARIA added and removed counts into PageDiff"
|
|
1207
|
+
```
|
|
1208
|
+
|
|
1209
|
+
---
|
|
1210
|
+
|
|
1211
|
+
### Task 9: Correct the two contradicting tool messages
|
|
1212
|
+
|
|
1213
|
+
**Files:**
|
|
1214
|
+
- Modify: `src/ai/tools.ts` (`hasObservablePageChange` at 1236, `isMajorPageChange` at 1220, the `form` no-change message at 417)
|
|
1215
|
+
- Test: `tests/unit/pagination-tool-results.test.ts`
|
|
1216
|
+
|
|
1217
|
+
**Interfaces:**
|
|
1218
|
+
- Consumes: `PageDiff.ariaAdded` / `ariaRemoved` (Task 8).
|
|
1219
|
+
- Produces: nothing other tasks import.
|
|
1220
|
+
|
|
1221
|
+
**Why:** an end-of-list scroll currently returns `failedToolResult` and commits `TestResult.FAILED` into the notes read by final review and Historian; and a batch of appended rows trips "MAJOR PAGE CHANGE. Page entered a different mode."
|
|
1222
|
+
|
|
1223
|
+
- [ ] **Step 1: Write the failing test**
|
|
1224
|
+
|
|
1225
|
+
Create `tests/unit/pagination-tool-results.test.ts`:
|
|
1226
|
+
|
|
1227
|
+
```ts
|
|
1228
|
+
import { describe, expect, it } from 'bun:test';
|
|
1229
|
+
import { isMajorPageChange } from '../../src/ai/tools.ts';
|
|
1230
|
+
import type { PageDiff } from '../../src/action-result.ts';
|
|
1231
|
+
|
|
1232
|
+
const diff = (over: Partial<PageDiff>): PageDiff => ({ urlChanged: false, currentUrl: '/list', ...over }) as PageDiff;
|
|
1233
|
+
|
|
1234
|
+
describe('isMajorPageChange', () => {
|
|
1235
|
+
it('treats a large batch of pure additions as growth', () => {
|
|
1236
|
+
expect(isMajorPageChange(diff({ ariaChangeCount: 120, ariaAdded: 120, ariaRemoved: 0 }))).toBe(false);
|
|
1237
|
+
});
|
|
1238
|
+
|
|
1239
|
+
it('still flags a churning diff as a mode change', () => {
|
|
1240
|
+
expect(isMajorPageChange(diff({ ariaChangeCount: 120, ariaAdded: 60, ariaRemoved: 60 }))).toBe(true);
|
|
1241
|
+
});
|
|
1242
|
+
|
|
1243
|
+
it('ignores a small diff either way', () => {
|
|
1244
|
+
expect(isMajorPageChange(diff({ ariaChangeCount: 4, ariaAdded: 2, ariaRemoved: 2 }))).toBe(false);
|
|
1245
|
+
});
|
|
1246
|
+
|
|
1247
|
+
it('falls back to the total when the split is absent', () => {
|
|
1248
|
+
expect(isMajorPageChange(diff({ ariaChangeCount: 120 }))).toBe(true);
|
|
1249
|
+
});
|
|
1250
|
+
});
|
|
1251
|
+
```
|
|
1252
|
+
|
|
1253
|
+
- [ ] **Step 2: Run test to verify it fails**
|
|
1254
|
+
|
|
1255
|
+
Run: `bun test tests/unit/pagination-tool-results.test.ts`
|
|
1256
|
+
Expected: FAIL — the first case returns `true`.
|
|
1257
|
+
|
|
1258
|
+
- [ ] **Step 3: Fix `isMajorPageChange`**
|
|
1259
|
+
|
|
1260
|
+
In `src/ai/tools.ts`:
|
|
1261
|
+
|
|
1262
|
+
```ts
|
|
1263
|
+
export function isMajorPageChange(pageDiff: PageDiff): boolean {
|
|
1264
|
+
if (pageDiff.urlChanged === true) return false;
|
|
1265
|
+
if ((pageDiff.ariaChangeCount ?? 0) < LARGE_ARIA_CHANGE_THRESHOLD) return false;
|
|
1266
|
+
if (pageDiff.ariaRemoved === 0 && (pageDiff.ariaAdded ?? 0) > 0) return false;
|
|
1267
|
+
return true;
|
|
1268
|
+
}
|
|
1269
|
+
```
|
|
1270
|
+
|
|
1271
|
+
- [ ] **Step 4: Run test to verify it passes**
|
|
1272
|
+
|
|
1273
|
+
Run: `bun test tests/unit/pagination-tool-results.test.ts`
|
|
1274
|
+
Expected: PASS, 4 tests.
|
|
1275
|
+
|
|
1276
|
+
- [ ] **Step 5: Count a request as an observable change**
|
|
1277
|
+
|
|
1278
|
+
In `hasObservablePageChange`, add before the final return:
|
|
1279
|
+
|
|
1280
|
+
```ts
|
|
1281
|
+
if (data.pageDiff.requests?.length) return true;
|
|
1282
|
+
```
|
|
1283
|
+
|
|
1284
|
+
- [ ] **Step 6: Rewrite the `form` no-change message**
|
|
1285
|
+
|
|
1286
|
+
At `src/ai/tools.ts:417`, replace the message and suggestion so they report what was observed instead of prescribing a form recovery:
|
|
1287
|
+
|
|
1288
|
+
```ts
|
|
1289
|
+
return failedToolResult('form', 'Command executed, but nothing on the page changed: no navigation, no ARIA change, no HTML change and no request.', {
|
|
1290
|
+
...toolResult,
|
|
1291
|
+
code: codeBlock,
|
|
1292
|
+
suggestion: 'The command ran without reaching anything. Re-locate the target, check whether another UI layer is active, then retry. If the goal was to load more of a list, no change means the collection has ended.',
|
|
1293
|
+
});
|
|
1294
|
+
```
|
|
1295
|
+
|
|
1296
|
+
- [ ] **Step 7: Verify**
|
|
1297
|
+
|
|
1298
|
+
Run: `bun test tests/unit/ tests/integration/`
|
|
1299
|
+
Expected: PASS.
|
|
1300
|
+
|
|
1301
|
+
- [ ] **Step 8: Format, lint, commit**
|
|
1302
|
+
|
|
1303
|
+
```bash
|
|
1304
|
+
bun run format
|
|
1305
|
+
bun run lint:fix
|
|
1306
|
+
git add src/ai/tools.ts tests/unit/pagination-tool-results.test.ts
|
|
1307
|
+
git commit -m "Stop reporting an end-of-list check as a failed step"
|
|
1308
|
+
```
|
|
1309
|
+
|
|
1310
|
+
---
|
|
1311
|
+
|
|
1312
|
+
### Task 10: Full-stack browser test and changelog
|
|
1313
|
+
|
|
1314
|
+
**Files:**
|
|
1315
|
+
- Modify: `tests/integration/pagination-browser.test.ts` (extend)
|
|
1316
|
+
- Create: `test-data/infinite-list.html`
|
|
1317
|
+
- Modify: `CHANGELOG.md`
|
|
1318
|
+
|
|
1319
|
+
**Interfaces:**
|
|
1320
|
+
- Consumes: everything above.
|
|
1321
|
+
- Produces: nothing.
|
|
1322
|
+
|
|
1323
|
+
- [ ] **Step 1: Create the fixture**
|
|
1324
|
+
|
|
1325
|
+
Create `test-data/infinite-list.html` — a container that scrolls inside itself and appends rows when scrolled near its end:
|
|
1326
|
+
|
|
1327
|
+
```html
|
|
1328
|
+
<!doctype html>
|
|
1329
|
+
<html>
|
|
1330
|
+
<body style="margin: 0">
|
|
1331
|
+
<div id="feed" style="height: 240px; overflow-y: auto"></div>
|
|
1332
|
+
<script>
|
|
1333
|
+
const feed = document.getElementById('feed');
|
|
1334
|
+
let next = 0;
|
|
1335
|
+
const addBatch = () => {
|
|
1336
|
+
for (let i = 0; i < 20; i++) {
|
|
1337
|
+
const row = document.createElement('div');
|
|
1338
|
+
row.className = 'row';
|
|
1339
|
+
row.style.height = '40px';
|
|
1340
|
+
row.textContent = `row ${next++}`;
|
|
1341
|
+
feed.appendChild(row);
|
|
1342
|
+
}
|
|
1343
|
+
};
|
|
1344
|
+
addBatch();
|
|
1345
|
+
feed.addEventListener('scroll', () => {
|
|
1346
|
+
if (feed.scrollTop + feed.clientHeight < feed.scrollHeight - 20) return;
|
|
1347
|
+
if (next >= 100) return;
|
|
1348
|
+
addBatch();
|
|
1349
|
+
});
|
|
1350
|
+
</script>
|
|
1351
|
+
</body>
|
|
1352
|
+
</html>
|
|
1353
|
+
```
|
|
1354
|
+
|
|
1355
|
+
- [ ] **Step 2: Write the failing test**
|
|
1356
|
+
|
|
1357
|
+
Append to `tests/integration/pagination-browser.test.ts`:
|
|
1358
|
+
|
|
1359
|
+
```ts
|
|
1360
|
+
describe('scrolling a container that appends', () => {
|
|
1361
|
+
it('adds rows and can be put back', async () => {
|
|
1362
|
+
const page = await browser.newPage();
|
|
1363
|
+
await page.goto(`file://${join(process.cwd(), 'test-data', 'infinite-list.html')}`, { waitUntil: 'domcontentloaded' });
|
|
1364
|
+
|
|
1365
|
+
const before = await page.evaluate(measureScroll, '#feed');
|
|
1366
|
+
await page.locator('#feed > *:last-child').scrollIntoViewIfNeeded();
|
|
1367
|
+
await page.waitForTimeout(200);
|
|
1368
|
+
const after = await page.evaluate(measureScroll, '#feed');
|
|
1369
|
+
await page.evaluate(restoreScroll, { css: '#feed', scrollTop: before!.scrollTop, windowScrollY: before!.windowScrollY });
|
|
1370
|
+
const restored = await page.evaluate(measureScroll, '#feed');
|
|
1371
|
+
|
|
1372
|
+
expect(before?.ownScroller).toBe(true);
|
|
1373
|
+
expect(after!.rowCount).toBeGreaterThan(before!.rowCount);
|
|
1374
|
+
expect(restored?.scrollTop).toBe(before!.scrollTop);
|
|
1375
|
+
expect(restored?.windowScrollY).toBe(before!.windowScrollY);
|
|
1376
|
+
await page.close();
|
|
1377
|
+
});
|
|
1378
|
+
});
|
|
1379
|
+
```
|
|
1380
|
+
|
|
1381
|
+
Add `import { join } from 'node:path';` at the top of that file if Task 3 did not.
|
|
1382
|
+
|
|
1383
|
+
- [ ] **Step 3: Run it**
|
|
1384
|
+
|
|
1385
|
+
Run: `bun test tests/integration/pagination-browser.test.ts`
|
|
1386
|
+
Expected: PASS. This is the end-to-end proof that `scrollIntoViewIfNeeded` reaches a container's own scroller and that the append fires.
|
|
1387
|
+
|
|
1388
|
+
- [ ] **Step 4: Run the whole suite**
|
|
1389
|
+
|
|
1390
|
+
Run: `bun test`
|
|
1391
|
+
Expected: PASS. Note that `codeceptjs` and `just-bash` mocks leak process-wide between test files, so an unrelated failure may be ordering, not this work — re-run the failing file alone to confirm.
|
|
1392
|
+
|
|
1393
|
+
- [ ] **Step 5: Update the changelog**
|
|
1394
|
+
|
|
1395
|
+
Invoke the `/changelog` skill to add the entry. Do not hand-write it.
|
|
1396
|
+
|
|
1397
|
+
- [ ] **Step 6: Lint and commit**
|
|
1398
|
+
|
|
1399
|
+
```bash
|
|
1400
|
+
bun run format
|
|
1401
|
+
bun run lint:fix
|
|
1402
|
+
git add test-data/infinite-list.html tests/integration/pagination-browser.test.ts CHANGELOG.md
|
|
1403
|
+
git commit -m "Prove container scrolling end to end and update the changelog"
|
|
1404
|
+
```
|
|
1405
|
+
|
|
1406
|
+
---
|
|
1407
|
+
|
|
1408
|
+
## Verification
|
|
1409
|
+
|
|
1410
|
+
Before opening a PR:
|
|
1411
|
+
|
|
1412
|
+
```bash
|
|
1413
|
+
bun test
|
|
1414
|
+
bun run lint:fix
|
|
1415
|
+
bunx tsc --noEmit -p tsconfig.json
|
|
1416
|
+
```
|
|
1417
|
+
|
|
1418
|
+
`tsc` matters: CI runs with `--noCheck`, so strict type errors pass every green check. Run it yourself on the changed files.
|
|
1419
|
+
|
|
1420
|
+
**Do not** add the `regression` label or trigger `regression.yml`. If this needs regression coverage, say so and let the user decide.
|