@sudobility/testomniac_runner 0.0.128

package/docs/superpowers/specs/2026-04-20-smarter-scanner-navigation-design.md

@@ -0,0 +1,121 @@
+# Smarter Scanner Navigation Design
+
+## Problem
+
+The current mouse scanner has several navigation shortcomings:
+
+1. **No state restoration for non-navigate actions.** If the scanner needs to execute a mouseover/click on a page it's not currently on, it proceeds anyway and fails silently.
+2. **Navigation actions are only seeded once** (to baseUrl at startup). New pages discovered via clicks don't get their own navigation actions, so they can't be revisited.
+3. **Navigate always goes to baseUrl.** There's no concept of navigating to a specific page — all navigate actions go to the same URL.
+4. **Generic selectors** cause duplicate work — but that's a separate extractor issue, not addressed here.
+
+## Design
+
+### Navigator Helper (`src/scanner/navigator.ts`)
+
+A new `Navigator` class that encapsulates all page-routing logic:
+
+```typescript
+class Navigator {
+  private pagesWithNavAction: Set<number>; // local dedup tracker
+  private stateManager: StateManager;
+  private api: ApiClient;
+  private appId: number;
+  private runId: number;
+  private sizeClass: string;
+
+  async ensureOnPage(page: Page, startingPageStateId: number | null): Promise<void>
+  async ensureNavigationActionExists(pageId: number, url: string): Promise<void>
+  private async resolvePageUrl(pageStateId: number): Promise<{ pageId: number; url: string }>
+}
+```
+
+#### `ensureOnPage(page, startingPageStateId)`
+
+Called before executing any action:
+
+1. If `startingPageStateId` is null/undefined — no-op (navigate actions don't require a starting page).
+2. If `stateManager.currentPageStateId === startingPageStateId` — no-op (already there).
+3. Otherwise:
+   - Resolve URL via API: get page state → get page → get URL.
+   - Call `page.goto(url, { waitUntil: "networkidle0", timeout: 30_000 })`.
+   - Update `stateManager` with the new state.
+   - Log the navigation at `info` level.
+
+#### `ensureNavigationActionExists(pageId, url)`
+
+Called whenever an action lands on a new page (different URL, same origin):
+
+1. Check `pagesWithNavAction.has(pageId)`.
+2. If already tracked, return.
+3. Create a navigate action: `api.createAction({ runId, type: "navigate", targetPageId: pageId, sizeClass })`.
+4. Add pageId to the set.
+
+Deduplication is local (in-memory `Set<number>`). Since only one scanner instance processes a given run, this is safe.
+
+### Action Loop Changes (`src/scanner/mouse-scanner.ts`)
+
+**Before each action:**
+Replace the current state-mismatch block (lines 218-229) with:
+```typescript
+await navigator.ensureOnPage(page, nextAction.startingPageStateId);
+```
+
+**Navigate action execution:**
+Currently always goes to `baseUrl`. Change to:
+- If `nextAction.targetPageId` is set, look up the page URL and navigate there.
+- If not set (the initial seed action), navigate to `baseUrl`.
+
+**After any action that changes the page URL:**
+```typescript
+if (result.isNew) {
+  await navigator.ensureNavigationActionExists(result.pageId, afterUrl);
+  await createMouseoverActions(result.pageStateId); // continue scanning inline
+}
+```
+
+This applies to both click actions (when a click navigates to a new page) and navigate actions (when we land on a page for the first time).
+
+### API Client Addition
+
+Add one method to resolve a page state's URL:
+
+```typescript
+// Already available via existing endpoints:
+// 1. getPageState(id) → { pageId } — needs new endpoint or use findMatchingPageState
+// 2. getPage(id) → { url }
+
+// Simplest: add a GET /scanner/page-states/:id endpoint to the API
+```
+
+The navigator needs to go from `startingPageStateId → pageId → url`. The page lookup (`getPage(id)`) doesn't exist yet in the scanner API. Need to add:
+- `GET /scanner/pages/:id` — get a single page by ID (returns `PageResponse`)
+
+### Schema/Type Changes
+
+None. The existing `CreateActionRequest.targetPageId` field is already available but unused for navigate actions. We'll start setting it.
+
+### Session Continuity
+
+No changes needed. The scanner already uses a single `page` object throughout the scan. Navigating via `page.goto(url)` preserves cookies and session state.
+
+## Files to Modify
+
+| File | Change |
+|------|--------|
+| `scanner/src/scanner/navigator.ts` | New file — Navigator class |
+| `scanner/src/scanner/mouse-scanner.ts` | Replace state-mismatch block with Navigator, update navigate action to use targetPageId URL, call ensureNavigationActionExists after discovering new pages |
+| `scanner/src/scanner/state-manager.ts` | No changes (Navigator wraps it) |
+| `scanner/src/api/client.ts` | Add `getPage(id)` method |
+| `api/src/routes/scanner.ts` | Add `GET /scanner/pages/:id` endpoint |
+| `types/src/index.ts` | No changes (PageResponse already exists) |
+
+## Verification
+
+1. Start a scan against a multi-page site.
+2. Verify logs show:
+   - Navigation actions automatically created when clicks discover new pages.
+   - `ensureOnPage` navigating to correct URLs before executing actions on other pages.
+   - No duplicate navigation actions for the same page.
+3. Verify the scanner explores all same-origin pages it discovers (not just the landing page).
+4. Verify session/cookies persist across navigations (no re-login prompts on protected pages).

package/eslint.config.js

@@ -0,0 +1,80 @@
+import js from '@eslint/js';
+import typescript from '@typescript-eslint/eslint-plugin';
+import typescriptParser from '@typescript-eslint/parser';
+import prettier from 'eslint-plugin-prettier';
+import prettierConfig from 'eslint-config-prettier';
+
+export default [
+  js.configs.recommended,
+  prettierConfig,
+  {
+    files: ['**/*.{ts,tsx}'],
+    languageOptions: {
+      parser: typescriptParser,
+      parserOptions: {
+        ecmaVersion: 'latest',
+        sourceType: 'module',
+      },
+      globals: {
+        console: 'readonly',
+        process: 'readonly',
+        Buffer: 'readonly',
+        __dirname: 'readonly',
+        __filename: 'readonly',
+        exports: 'writable',
+        global: 'readonly',
+        module: 'readonly',
+        require: 'readonly',
+        setTimeout: 'readonly',
+        setInterval: 'readonly',
+        clearTimeout: 'readonly',
+        clearInterval: 'readonly',
+        NodeJS: 'readonly',
+        fetch: 'readonly',
+        Response: 'readonly',
+        Request: 'readonly',
+        AbortController: 'readonly',
+        URL: 'readonly',
+        URLSearchParams: 'readonly',
+        Headers: 'readonly',
+        Bun: 'readonly',
+      },
+    },
+    plugins: {
+      '@typescript-eslint': typescript,
+      prettier: prettier,
+    },
+    rules: {
+      'no-unused-vars': 'off',
+      '@typescript-eslint/no-unused-vars': [
+        'error',
+        { argsIgnorePattern: '^_', varsIgnorePattern: '^_' },
+      ],
+      'no-console': ['warn', { allow: ['warn', 'error'] }],
+      'prefer-const': 'error',
+      'no-constant-binary-expression': 'off',
+      'no-undef': 'off',
+      'no-empty': ['error', { allowEmptyCatch: true }],
+      'prettier/prettier': 'error',
+    },
+  },
+  {
+    files: ['**/__tests__/**/*', '**/*.test.*', '**/*.spec.*'],
+    languageOptions: {
+      globals: {
+        describe: 'readonly',
+        it: 'readonly',
+        expect: 'readonly',
+        beforeEach: 'readonly',
+        afterEach: 'readonly',
+        vi: 'readonly',
+        test: 'readonly',
+        beforeAll: 'readonly',
+        afterAll: 'readonly',
+      },
+    },
+  },
+  {
+    ignores: ['dist/**', '**/*.example.*', 'node_modules/**', 'vendor/**'],
+  },
+];

package/package.json

@@ -0,0 +1,55 @@
+{
+  "name": "@sudobility/testomniac_runner",
+  "version": "0.0.128",
+  "publishConfig": {
+    "access": "public"
+  },
+  "type": "module",
+  "description": "Testomniac scanner worker service",
+  "scripts": {
+    "dev": "bun --watch src/index.ts",
+    "build": "bun build src/index.ts --outdir dist --target bun",
+    "start": "bun dist/index.js",
+    "test": "bun test",
+    "test:unit": "bun test --test-name-pattern '^(?!scanner service)' ",
+    "test:watch": "bun test --watch",
+    "typecheck": "bunx tsc --noEmit",
+    "lint": "bunx eslint src/",
+    "lint:fix": "bunx eslint src/ --fix",
+    "format": "bunx prettier --write \"src/**/*.ts\"",
+    "format:check": "bunx prettier --check \"src/**/*.ts\"",
+    "verify": "bun run typecheck && bun run lint && bun run test && bun run build"
+  },
+  "dependencies": {
+    "@noble/curves": "^1.0.0",
+    "@noble/hashes": "^1.0.0",
+    "@sudobility/signic_sdk": "^0.1.7",
+    "@sudobility/testomniac_runner_service": "^0.1.127",
+    "@sudobility/testomniac_types": "^0.0.66",
+    "hono": "^4.7.0",
+    "jose": "^6.1.2",
+    "openai": "^6.7.0",
+    "pino": "^10.1.0",
+    "postmark": "^4.0.7",
+    "puppeteer-core": "^24.31.0"
+  },
+  "devDependencies": {
+    "@eslint/js": "^9.38.0",
+    "@types/node": "^24.10.1",
+    "@typescript-eslint/eslint-plugin": "^8.46.2",
+    "@typescript-eslint/parser": "^8.46.2",
+    "eslint": "^9.39.2",
+    "eslint-config-prettier": "^10.1.8",
+    "eslint-plugin-prettier": "^5.5.5",
+    "globals": "^16.4.0",
+    "prettier": "^3.7.4",
+    "tsx": "^4.21.0",
+    "typescript": "~5.9.3",
+    "vitest": "^4.0.15"
+  },
+  "license": "BUSL-1.1",
+  "repository": {
+    "type": "git",
+    "url": "git@github.com:johnqh/testomniac_runner.git"
+  }
+}