@sudobility/testomniac_runner 0.0.128

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (60) hide show
  1. package/.dockerignore +75 -0
  2. package/.env.example +67 -0
  3. package/.github/workflows/ci-cd.yml +30 -0
  4. package/.prettierignore +62 -0
  5. package/.prettierrc +11 -0
  6. package/.vscode/settings.json +29 -0
  7. package/CLAUDE.md +170 -0
  8. package/Dockerfile +76 -0
  9. package/README.md +22 -0
  10. package/bun.lock +707 -0
  11. package/docs/superpowers/specs/2026-04-20-smarter-scanner-navigation-design.md +121 -0
  12. package/eslint.config.js +80 -0
  13. package/package.json +55 -0
  14. package/plans/DATA.md +703 -0
  15. package/plans/POLLING.md +569 -0
  16. package/plans/RUNNER.md +288 -0
  17. package/src/adapters/PuppeteerAdapter.ts +394 -0
  18. package/src/auth/credential-manager.ts +17 -0
  19. package/src/auth/form-identifier.test.ts +136 -0
  20. package/src/auth/form-identifier.ts +54 -0
  21. package/src/auth/login-executor.ts +112 -0
  22. package/src/auth/password-detector.test.ts +61 -0
  23. package/src/auth/password-detector.ts +119 -0
  24. package/src/auth/signic-registrar.ts +186 -0
  25. package/src/browser/chromium.ts +35 -0
  26. package/src/config/index.test.ts +23 -0
  27. package/src/config/index.ts +35 -0
  28. package/src/email/deep-link.test.ts +17 -0
  29. package/src/email/deep-link.ts +23 -0
  30. package/src/email/sender.ts +35 -0
  31. package/src/email/templates.ts +34 -0
  32. package/src/index.test.ts +17 -0
  33. package/src/index.ts +110 -0
  34. package/src/orchestrator.ts +220 -0
  35. package/src/plugins/content/ai-checks.ts +115 -0
  36. package/src/plugins/content/checks.test.ts +49 -0
  37. package/src/plugins/content/checks.ts +141 -0
  38. package/src/plugins/content/index.ts +73 -0
  39. package/src/plugins/registry.test.ts +49 -0
  40. package/src/plugins/registry.ts +21 -0
  41. package/src/plugins/security/header-checks.ts +56 -0
  42. package/src/plugins/security/html-checks.ts +93 -0
  43. package/src/plugins/security/index.ts +58 -0
  44. package/src/plugins/security/network-checks.test.ts +74 -0
  45. package/src/plugins/security/network-checks.ts +136 -0
  46. package/src/plugins/seo/checks.test.ts +70 -0
  47. package/src/plugins/seo/checks.ts +173 -0
  48. package/src/plugins/seo/index.ts +85 -0
  49. package/src/plugins/types.ts +43 -0
  50. package/src/plugins/ui-consistency/comparator.test.ts +108 -0
  51. package/src/plugins/ui-consistency/comparator.ts +58 -0
  52. package/src/plugins/ui-consistency/index.ts +36 -0
  53. package/src/plugins/ui-consistency/style-extractor.ts +79 -0
  54. package/src/runner/executor.test.ts +37 -0
  55. package/src/runner/executor.ts +167 -0
  56. package/src/runner/reporter.ts +19 -0
  57. package/src/runner/worker-pool.ts +106 -0
  58. package/src/runner-manager.ts +163 -0
  59. package/src/scanner/email-checker.ts +106 -0
  60. package/tsconfig.json +21 -0
package/plans/DATA.md ADDED
@@ -0,0 +1,703 @@
1
+ # Testomniac Data Model
2
+
3
+ ## Entity Hierarchy
4
+
5
+ ```text
6
+ Entity (organization)
7
+ └── Product
8
+ ├── Runner
9
+ │ ├── Test Suite Bundle ── uid?
10
+ │ │ └── Test Suite Bundle ↔ Test Suite (junction)
11
+ │ ├── Test Suite ── uid?
12
+ │ │ └── Test Case ── uid?
13
+ │ ├── Scaffold
14
+ │ ├── Page
15
+ │ │ └── Page State
16
+ │ │ ├── Page State Scaffold (junction)
17
+ │ │ └── Page State Pattern
18
+ │ └── Element Identity
19
+ ├── Persona
20
+ │ └── Use Case
21
+ │ └── Input Value
22
+ ├── Test Credential
23
+ ├── Test Environment
24
+ ├── Product Settings
25
+ ├── API Key
26
+ └── Test File
27
+ ```
28
+
29
+ ## Execution Hierarchy
30
+
31
+ ```text
32
+ Test Run ── runner, discovery, scanUrl, sizeClass, runnerInstanceId
33
+ ├── → Test Suite Bundle Run
34
+ │ └── Test Suite Run
35
+ │ └── Test Case Run ── expectedOutcome, observedOutcome
36
+ │ └── Test Run Finding
37
+ ├── → Test Suite Run
38
+ │ └── Test Case Run
39
+ │ └── Test Run Finding
40
+ └── → Test Case Run
41
+ └── Test Run Finding
42
+ ```
43
+
44
+ A Test Run points to exactly one of: Test Case Run, Test Suite Run, or Test Suite Bundle Run.
45
+
46
+ ---
47
+
48
+ ## Persisted Objects
49
+
50
+ ### Entity
51
+
52
+ Managed by `@sudobility/entity_service`. Represents an organization.
53
+
54
+ | Field | Type | Notes |
55
+ |-------|------|-------|
56
+ | id | UUID | PK |
57
+
58
+ ### Product
59
+
60
+ A software product owned by an entity.
61
+
62
+ | Field | Type | Notes |
63
+ |-------|------|-------|
64
+ | id | BIGSERIAL | PK |
65
+ | entityId | TEXT | Owning entity |
66
+ | title | TEXT | NOT NULL |
67
+ | description | TEXT | |
68
+ | contactEmail | TEXT | |
69
+ | claimedByUserId | TEXT | Firebase uid of claimer |
70
+ | claimedAt | TIMESTAMPTZ | |
71
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
72
+ | updatedAt | TIMESTAMPTZ | DEFAULT NOW() |
73
+
74
+ ### Runner
75
+
76
+ A target application under a product. Each runner represents a specific app to test.
77
+
78
+ | Field | Type | Notes |
79
+ |-------|------|-------|
80
+ | id | BIGSERIAL | PK |
81
+ | productId | BIGSERIAL | FK → products, NOT NULL |
82
+ | title | TEXT | NOT NULL |
83
+ | type | TEXT | NOT NULL, DEFAULT 'worker'. Values: 'worker', 'extension' |
84
+ | ownerEntityId | TEXT | |
85
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
86
+
87
+ ---
88
+
89
+ ### Test Suite Bundle
90
+
91
+ A user-created grouping of test suites. Links to test suites via a junction table.
92
+
93
+ | Field | Type | Notes |
94
+ |-------|------|-------|
95
+ | id | BIGSERIAL | PK |
96
+ | runnerId | BIGSERIAL | FK → runners, NOT NULL |
97
+ | title | TEXT | NOT NULL |
98
+ | description | TEXT | |
99
+ | uid | TEXT | **NEW** — Firebase uid of the developer who created it. NULL if created from website. |
100
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
101
+ | updatedAt | TIMESTAMPTZ | DEFAULT NOW() |
102
+
103
+ **Junction: test_suite_bundle_suites**
104
+
105
+ | Field | Type | Notes |
106
+ |-------|------|-------|
107
+ | id | BIGSERIAL | PK |
108
+ | testSuiteBundleId | BIGSERIAL | FK → test_suite_bundles, NOT NULL, CASCADE DELETE |
109
+ | testSuiteId | BIGSERIAL | FK → test_suites, NOT NULL, CASCADE DELETE |
110
+
111
+ Unique index on (testSuiteBundleId, testSuiteId).
112
+
113
+ ### Test Suite
114
+
115
+ A collection of test cases for a specific area of the application (a page, a scaffold, a pattern, etc.).
116
+
117
+ | Field | Type | Notes |
118
+ |-------|------|-------|
119
+ | id | BIGSERIAL | PK |
120
+ | runnerId | BIGSERIAL | FK → runners, NOT NULL |
121
+ | decompositionJobId | BIGSERIAL | Links to the AI job that created it |
122
+ | title | TEXT | NOT NULL |
123
+ | description | TEXT | NOT NULL |
124
+ | startingPageStateId | BIGSERIAL | FK → page_states |
125
+ | startingPath | TEXT | NOT NULL — relative URL path |
126
+ | sizeClass | TEXT | NOT NULL, DEFAULT 'desktop'. Values: 'desktop', 'mobile' |
127
+ | dependencyTestCaseId | BIGSERIAL | Test case that must pass before this suite runs |
128
+ | personaIdsJson | JSONB | Array of persona IDs |
129
+ | scaffoldId | BIGSERIAL | FK → scaffolds. Set if this suite tests a specific scaffold |
130
+ | scaffoldType | TEXT | HtmlComponentType value (e.g., 'topMenu', 'footer') |
131
+ | patternType | TEXT | UiPatternType value (e.g., 'form', 'table'). Set if testing a pattern |
132
+ | priority | INTEGER | NOT NULL, DEFAULT 3 |
133
+ | suiteTags | TEXT[] | NOT NULL, DEFAULT [] |
134
+ | estimatedDurationMs | INTEGER | |
135
+ | uid | TEXT | **NEW** — Firebase uid. NULL if created from website. |
136
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
137
+
138
+ ### Test Case
139
+
140
+ An individual test scenario with steps and expectations.
141
+
142
+ | Field | Type | Notes |
143
+ |-------|------|-------|
144
+ | id | BIGSERIAL | PK |
145
+ | runnerId | BIGSERIAL | FK → runners |
146
+ | testSuiteId | BIGSERIAL | FK → test_suites |
147
+ | title | TEXT | NOT NULL |
148
+ | testType | TEXT | NOT NULL. Values: 'render', 'interaction', 'form', 'form_negative', 'password', 'navigation', 'e2e' |
149
+ | sizeClass | TEXT | NOT NULL, DEFAULT 'desktop' |
150
+ | suiteTags | TEXT[] | NOT NULL, DEFAULT [] |
151
+ | priority | INTEGER | NOT NULL, DEFAULT 3 |
152
+ | scaffoldId | BIGSERIAL | FK → scaffolds |
153
+ | patternType | TEXT | |
154
+ | dependencyTestCaseId | BIGSERIAL | Must pass before this case runs |
155
+ | pageId | BIGSERIAL | FK → pages |
156
+ | personaId | BIGSERIAL | FK → personas |
157
+ | useCaseId | BIGSERIAL | FK → use_cases |
158
+ | startingPageStateId | BIGSERIAL | FK → page_states |
159
+ | startingPath | TEXT | Relative URL |
160
+ | stepsJson | JSONB | Array of TestStep objects |
161
+ | globalExpectationsJson | JSONB | Array of Expectation objects |
162
+ | estimatedDurationMs | INTEGER | |
163
+ | uid | TEXT | **NEW** — Firebase uid. NULL if created from website. |
164
+ | generatedAt | TIMESTAMPTZ | DEFAULT NOW() |
165
+
166
+ ### TestStep (embedded in stepsJson)
167
+
168
+ ```typescript
169
+ {
170
+ action: TestAction;
171
+ expectations: Expectation[];
172
+ description: string;
173
+ continueOnFailure: boolean;
174
+ }
175
+ ```
176
+
177
+ ### TestAction (embedded in TestStep)
178
+
179
+ ```typescript
180
+ {
181
+ actionType: PlaywrightAction; // 'goto', 'click', 'fill', 'hover', etc.
182
+ pageStateId?: number;
183
+ elementIdentityId?: number;
184
+ containerType?: HtmlComponentType;
185
+ containerElementIdentityId?: number;
186
+ value?: string;
187
+ path?: string;
188
+ playwrightCode: string; // Actual Playwright code to execute
189
+ description: string;
190
+ }
191
+ ```
192
+
193
+ ### Expectation (embedded in TestStep and globalExpectations)
194
+
195
+ ```typescript
196
+ {
197
+ expectationType: ExpectationType; // 'page_loaded', 'element_visible', 'no_console_errors', etc.
198
+ elementIdentityId?: number;
199
+ expectedValue?: string;
200
+ attributeName?: string;
201
+ severity: ExpectationSeverity; // 'must_pass', 'should_pass', 'info'
202
+ description: string;
203
+ playwrightCode: string;
204
+ }
205
+ ```
206
+
207
+ ---
208
+
209
+ ### Test Suite Bundle Run
210
+
211
+ Execution record for a test suite bundle.
212
+
213
+ | Field | Type | Notes |
214
+ |-------|------|-------|
215
+ | id | BIGSERIAL | PK |
216
+ | testSuiteBundleId | BIGSERIAL | FK → test_suite_bundles, NOT NULL |
217
+ | status | TEXT | NOT NULL, DEFAULT 'pending'. Values: 'pending', 'running', 'completed', 'failed' |
218
+ | startedAt | TIMESTAMPTZ | |
219
+ | completedAt | TIMESTAMPTZ | |
220
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
221
+
222
+ ### Test Suite Run
223
+
224
+ Execution record for a test suite.
225
+
226
+ | Field | Type | Notes |
227
+ |-------|------|-------|
228
+ | id | BIGSERIAL | PK |
229
+ | testSuiteId | BIGSERIAL | FK → test_suites, NOT NULL |
230
+ | testSuiteBundleRunId | BIGSERIAL | FK → test_suite_bundle_runs. Set if this suite run is part of a bundle run. |
231
+ | status | TEXT | NOT NULL, DEFAULT 'pending' |
232
+ | startedAt | TIMESTAMPTZ | |
233
+ | completedAt | TIMESTAMPTZ | |
234
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
235
+
236
+ ### Test Case Run
237
+
238
+ Execution record for a single test case.
239
+
240
+ | Field | Type | Notes |
241
+ |-------|------|-------|
242
+ | id | BIGSERIAL | PK |
243
+ | testCaseId | BIGSERIAL | FK → test_cases, NOT NULL |
244
+ | testSuiteRunId | BIGSERIAL | FK → test_suite_runs. Set if this case run is part of a suite run. |
245
+ | status | TEXT | NOT NULL, DEFAULT 'pending'. Values: 'pending', 'running', 'completed', 'failed' |
246
+ | durationMs | INTEGER | |
247
+ | errorMessage | TEXT | Error details if failed |
248
+ | expectedOutcome | TEXT | **NEW** — Aggregated expected outcomes from expertises |
249
+ | observedOutcome | TEXT | **NEW** — Aggregated observed outcomes from expertises |
250
+ | screenshotPath | TEXT | Captured on warning/error |
251
+ | consoleLog | TEXT | Captured on warning/error |
252
+ | networkLog | TEXT | Captured on warning/error |
253
+ | startedAt | TIMESTAMPTZ | |
254
+ | completedAt | TIMESTAMPTZ | |
255
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
256
+
257
+ ### Test Run
258
+
259
+ Top-level run object. Represents a single invocation of the runner.
260
+
261
+ | Field | Type | Notes |
262
+ |-------|------|-------|
263
+ | id | BIGSERIAL | PK |
264
+ | runnerId | BIGSERIAL | FK → runners, NOT NULL |
265
+ | testCaseRunId | BIGSERIAL | FK → test_case_runs. Set if running a single test case. |
266
+ | testSuiteRunId | BIGSERIAL | FK → test_suite_runs. Set if running a test suite. |
267
+ | testSuiteBundleRunId | BIGSERIAL | FK → test_suite_bundle_runs. Set if running a bundle. |
268
+ | testEnvironmentId | BIGSERIAL | **NEW** — FK → test_environments. Required. Specifies which environment this run targets. |
269
+ | discovery | BOOLEAN | NOT NULL, DEFAULT FALSE. If true, auto-discover new test suites/cases. |
270
+ | parentTestRunId | BIGSERIAL | FK → test_runs. For child runs. |
271
+ | rootTestRunId | BIGSERIAL | FK → test_runs. Root of the run tree. |
272
+ | sizeClass | TEXT | NOT NULL, DEFAULT 'desktop' |
273
+ | status | TEXT | NOT NULL, DEFAULT 'pending'. Values: 'pending', 'planned', 'running', 'completed', 'failed' |
274
+ | createdByUserId | TEXT | Firebase uid of the user who started the run |
275
+ | ownedByUserId | TEXT | Firebase uid of the run owner |
276
+ | runnerInstanceId | TEXT | **NEW** — UUID of the runner instance executing this run |
277
+ | runnerInstanceName | TEXT | **NEW** — Human-readable name of the runner instance |
278
+ | scanUrl | TEXT | Starting URL for the scan |
279
+ | pagesFound | INTEGER | Stats |
280
+ | pageStatesFound | INTEGER | Stats |
281
+ | testRunsCompleted | INTEGER | Stats |
282
+ | aiSummary | TEXT | AI-generated summary |
283
+ | totalDurationMs | INTEGER | |
284
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
285
+ | startedAt | TIMESTAMPTZ | |
286
+ | completedAt | TIMESTAMPTZ | |
287
+
288
+ ### Test Run Finding
289
+
290
+ An issue discovered during a test case run.
291
+
292
+ | Field | Type | Notes |
293
+ |-------|------|-------|
294
+ | id | BIGSERIAL | PK |
295
+ | testCaseRunId | BIGSERIAL | FK → test_case_runs, NOT NULL |
296
+ | expertiseRuleId | BIGSERIAL | FK → expertise_rules. Links to the rule that flagged it. |
297
+ | type | TEXT | NOT NULL. Values: 'error', 'warning' |
298
+ | title | TEXT | NOT NULL |
299
+ | description | TEXT | NOT NULL |
300
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
301
+
302
+ ---
303
+
304
+ ### Page
305
+
306
+ A URL path discovered under a runner.
307
+
308
+ | Field | Type | Notes |
309
+ |-------|------|-------|
310
+ | id | BIGSERIAL | PK |
311
+ | runnerId | BIGSERIAL | FK → runners |
312
+ | relativePath | TEXT | NOT NULL — URL path relative to base URL |
313
+ | routeKey | TEXT | Normalized route key (e.g., `/users/:id`) |
314
+ | requiresLogin | BOOLEAN | DEFAULT FALSE |
315
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
316
+
317
+ ### Page State
318
+
319
+ A snapshot of a page at a point in time. Multiple states can exist per page (different viewports, different content).
320
+
321
+ | Field | Type | Notes |
322
+ |-------|------|-------|
323
+ | id | BIGSERIAL | PK |
324
+ | pageId | BIGSERIAL | FK → pages |
325
+ | sizeClass | TEXT | NOT NULL, DEFAULT 'desktop' |
326
+ | htmlHash | CHAR(64) | SHA-256 of raw HTML |
327
+ | normalizedHtmlHash | CHAR(64) | SHA-256 of normalized HTML |
328
+ | textHash | CHAR(64) | SHA-256 of visible text |
329
+ | actionableHash | CHAR(64) | SHA-256 of actionable items |
330
+ | fixedBodyHash | CHAR(64) | SHA-256 of body minus scaffolds and patterns |
331
+ | scaffoldsHash | CHAR(64) | SHA-256 of scaffold type:hash pairs |
332
+ | patternsHash | CHAR(64) | SHA-256 of pattern type:count pairs |
333
+ | bodyHtmlElementId | BIGSERIAL | FK → html_elements |
334
+ | contentHtmlElementId | BIGSERIAL | FK → html_elements |
335
+ | fixedBodyHtmlElementId | BIGSERIAL | FK → html_elements |
336
+ | screenshotPath | TEXT | |
337
+ | rawHtmlPath | TEXT | |
338
+ | contentText | TEXT | Extracted visible text |
339
+ | createdByTestRunId | BIGSERIAL | Which test run captured this state |
340
+ | capturedAt | TIMESTAMPTZ | DEFAULT NOW() |
341
+
342
+ ### Scaffold
343
+
344
+ A detected reusable UI region (header, footer, nav, sidebar, etc.) that appears across multiple pages.
345
+
346
+ | Field | Type | Notes |
347
+ |-------|------|-------|
348
+ | id | BIGSERIAL | PK |
349
+ | runnerId | BIGSERIAL | FK → runners, NOT NULL |
350
+ | type | TEXT | NOT NULL. HtmlComponentType: 'topMenu', 'footer', 'breadcrumb', 'leftMenu', 'hamburgerMenu', 'rightSidebar', 'searchBar', 'userMenu', 'cookieBanner', 'chatWidget', 'socialLinks', 'skipNav', 'languageSwitcher', 'announcementBar', 'backToTop' |
351
+ | htmlElementId | BIGSERIAL | FK → html_elements, NOT NULL |
352
+ | htmlHash | CHAR(64) | |
353
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
354
+
355
+ Unique index on (runnerId, type, htmlElementId).
356
+
357
+ **Junction: page_state_scaffolds**
358
+
359
+ | Field | Type | Notes |
360
+ |-------|------|-------|
361
+ | id | BIGSERIAL | PK |
362
+ | pageStateId | BIGSERIAL | FK → page_states, NOT NULL |
363
+ | scaffoldId | BIGSERIAL | FK → scaffolds, NOT NULL |
364
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
365
+
366
+ Unique index on (pageStateId, scaffoldId).
367
+
368
+ ### Page State Pattern
369
+
370
+ A detected UI pattern instance on a page state.
371
+
372
+ | Field | Type | Notes |
373
+ |-------|------|-------|
374
+ | id | BIGSERIAL | PK |
375
+ | pageStateId | BIGSERIAL | FK → page_states, NOT NULL |
376
+ | patternType | TEXT | NOT NULL. UiPatternType: 'card', 'table', 'form', 'modal', 'toast', 'alert', 'tabs', 'accordion', 'carousel', 'dropdown', 'pagination', 'skeleton', 'emptyState', 'errorMessage', 'progressBar', 'tooltip', 'badge', 'avatar', 'tag', 'stepper' |
377
+ | selector | TEXT | NOT NULL — CSS selector for the pattern |
378
+ | count | INTEGER | NOT NULL, DEFAULT 0 — instance count on the page |
379
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
380
+
381
+ ### HTML Element
382
+
383
+ Deduplicated HTML content storage, referenced by scaffolds and page states.
384
+
385
+ | Field | Type | Notes |
386
+ |-------|------|-------|
387
+ | id | BIGSERIAL | PK |
388
+ | html | TEXT | NOT NULL — raw HTML markup |
389
+ | hash | CHAR(64) | NOT NULL, UNIQUE — SHA-256 for deduplication |
390
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
391
+
392
+ ### Element Identity
393
+
394
+ Persistent identity for an interactive element across scans. Allows stable references even when selectors change.
395
+
396
+ | Field | Type | Notes |
397
+ |-------|------|-------|
398
+ | id | BIGSERIAL | PK |
399
+ | runnerId | BIGSERIAL | FK → runners, NOT NULL |
400
+ | role | TEXT | NOT NULL — ARIA role |
401
+ | computedName | TEXT | ARIA computed name |
402
+ | tagName | TEXT | NOT NULL |
403
+ | labelText | TEXT | |
404
+ | groupName | TEXT | |
405
+ | placeholder | TEXT | |
406
+ | altText | TEXT | |
407
+ | testId | TEXT | data-testid value |
408
+ | inputType | TEXT | |
409
+ | nthInGroup | INTEGER | Position in group |
410
+ | formContext | TEXT | Parent form context |
411
+ | headingContext | TEXT | Nearest heading |
412
+ | landmarkAncestor | TEXT | Parent ARIA landmark |
413
+ | playwrightLocator | TEXT | NOT NULL — primary locator string |
414
+ | playwrightScopeChain | TEXT | Scoped locator chain |
415
+ | isUniqueOnPage | BOOLEAN | NOT NULL, DEFAULT TRUE |
416
+ | cssSelector | TEXT | NOT NULL — fallback CSS selector |
417
+ | locators | JSONB | NOT NULL, DEFAULT [] — array of ElementLocator |
418
+ | firstSeenTestRunId | BIGSERIAL | |
419
+ | lastSeenTestRunId | BIGSERIAL | |
420
+ | timesSeen | INTEGER | NOT NULL, DEFAULT 1 |
421
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
422
+ | updatedAt | TIMESTAMPTZ | DEFAULT NOW() |
423
+
424
+ ### Actionable Item
425
+
426
+ An interactive element on a page state (button, input, link, etc.).
427
+
428
+ | Field | Type | Notes |
429
+ |-------|------|-------|
430
+ | id | BIGSERIAL | PK |
431
+ | htmlElementId | BIGSERIAL | FK → html_elements |
432
+ | stableKey | TEXT | Deterministic key for deduplication |
433
+ | selector | TEXT | CSS selector |
434
+ | tagName | TEXT | |
435
+ | role | TEXT | ARIA role |
436
+ | actionKind | TEXT | 'click', 'fill', 'select', 'navigate', 'radio_select' |
437
+ | accessibleName | TEXT | |
438
+ | disabled | BOOLEAN | |
439
+ | visible | BOOLEAN | |
440
+ | attributesJson | JSONB | All HTML attributes |
441
+ | scaffoldId | BIGSERIAL | If this item belongs to a scaffold |
442
+ | elementIdentityId | BIGSERIAL | FK → element_identities |
443
+
444
+ ---
445
+
446
+ ### Expertise
447
+
448
+ A registered expertise category (e.g., SEO, Security, Performance).
449
+
450
+ | Field | Type | Notes |
451
+ |-------|------|-------|
452
+ | id | BIGSERIAL | PK |
453
+ | slug | TEXT | NOT NULL, UNIQUE — e.g., 'seo', 'security' |
454
+ | title | TEXT | NOT NULL |
455
+ | description | TEXT | NOT NULL |
456
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
457
+
458
+ ### Expertise Rule
459
+
460
+ A specific rule within an expertise (e.g., "Title tag present" under SEO).
461
+
462
+ | Field | Type | Notes |
463
+ |-------|------|-------|
464
+ | id | BIGSERIAL | PK |
465
+ | expertiseId | BIGSERIAL | FK → expertises, NOT NULL |
466
+ | title | TEXT | NOT NULL |
467
+ | description | TEXT | NOT NULL |
468
+ | aiEndpointUrl | TEXT | Optional AI evaluation endpoint |
469
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
470
+
471
+ ---
472
+
473
+ ### Persona
474
+
475
+ An AI-generated user persona for a product.
476
+
477
+ | Field | Type | Notes |
478
+ |-------|------|-------|
479
+ | id | BIGSERIAL | PK |
480
+ | productId | BIGSERIAL | FK → products |
481
+ | title | TEXT | NOT NULL |
482
+ | description | TEXT | |
483
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
484
+
485
+ ### Use Case
486
+
487
+ A use case for a persona with specific input values.
488
+
489
+ | Field | Type | Notes |
490
+ |-------|------|-------|
491
+ | id | BIGSERIAL | PK |
492
+ | personaId | BIGSERIAL | FK → personas |
493
+ | title | TEXT | NOT NULL |
494
+ | description | TEXT | |
495
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
496
+
497
+ ### Input Value
498
+
499
+ A stored form field value for a use case.
500
+
501
+ | Field | Type | Notes |
502
+ |-------|------|-------|
503
+ | id | BIGSERIAL | PK |
504
+ | useCaseId | BIGSERIAL | FK → use_cases |
505
+ | fieldSelector | TEXT | NOT NULL |
506
+ | fieldName | TEXT | |
507
+ | value | TEXT | NOT NULL |
508
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
509
+
510
+ ### Test Credential
511
+
512
+ Stored login credentials for a product, optionally tied to a persona.
513
+
514
+ | Field | Type | Notes |
515
+ |-------|------|-------|
516
+ | id | BIGSERIAL | PK |
517
+ | productId | BIGSERIAL | FK → products, NOT NULL |
518
+ | personaId | BIGSERIAL | FK → personas |
519
+ | signic | BOOLEAN | NOT NULL, DEFAULT FALSE — whether to use Signic email service |
520
+ | email | TEXT | |
521
+ | password | TEXT | |
522
+ | verificationCode | TEXT | |
523
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
524
+
525
+ ### Test Environment
526
+
527
+ A deployment environment for a product (e.g., dev, staging, production).
528
+
529
+ | Field | Type | Notes |
530
+ |-------|------|-------|
531
+ | id | BIGSERIAL | PK |
532
+ | productId | BIGSERIAL | FK → products, NOT NULL |
533
+ | title | TEXT | NOT NULL |
534
+ | baseUrl | TEXT | NOT NULL |
535
+ | githubBranch | TEXT | |
536
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
537
+ | updatedAt | TIMESTAMPTZ | DEFAULT NOW() |
538
+
539
+ ### Test Schedule
540
+
541
+ A recurring schedule for automated test runs.
542
+
543
+ | Field | Type | Notes |
544
+ |-------|------|-------|
545
+ | id | BIGSERIAL | PK |
546
+ | runnerId | BIGSERIAL | FK → runners, NOT NULL |
547
+ | title | TEXT | NOT NULL |
548
+ | testSuiteId | BIGSERIAL | FK → test_suites |
549
+ | testCaseId | BIGSERIAL | FK → test_cases |
550
+ | testSuiteBundleId | BIGSERIAL | FK → test_suite_bundles |
551
+ | discovery | BOOLEAN | NOT NULL, DEFAULT FALSE |
552
+ | recurrenceType | TEXT | NOT NULL. Values: 'one_time', 'weekday', 'daily', 'weekly' |
553
+ | timeOfDay | TEXT | NOT NULL — e.g., '09:00' |
554
+ | dayOfWeek | INTEGER | 0-6 for weekly |
555
+ | timezone | TEXT | NOT NULL |
556
+ | enabled | BOOLEAN | NOT NULL, DEFAULT TRUE |
557
+ | sizeClass | TEXT | NOT NULL, DEFAULT 'desktop' |
558
+ | createdByUserId | TEXT | NOT NULL — Firebase uid |
559
+ | lastRunAt | TIMESTAMPTZ | |
560
+ | nextRunAt | TIMESTAMPTZ | |
561
+ | createdAt | TIMESTAMPTZ | DEFAULT NOW() |
562
+ | updatedAt | TIMESTAMPTZ | DEFAULT NOW() |
563
+
564
+ ---
565
+
566
+ ## In-Memory Objects (Runner Service)
567
+
568
+ ### Outcome
569
+
570
+ Produced by each expertise during test case evaluation.
571
+
572
+ ```typescript
573
+ interface Outcome {
574
+ expected: string; // What was expected
575
+ observed: string; // What was actually observed
576
+ result: 'pass' | 'warning' | 'error';
577
+ }
578
+ ```
579
+
580
+ ### ExpertiseContext
581
+
582
+ Passed to each expertise for evaluation.
583
+
584
+ ```typescript
585
+ interface ExpertiseContext {
586
+ html: string; // Full page HTML including <head>
587
+ scaffolds: DetectedScaffoldRegion[]; // Decomposed scaffold list
588
+ patterns: DetectedPatternWithInstances[]; // Decomposed pattern list
589
+ consoleLogs: string[]; // Console output during test case
590
+ networkLogs: NetworkLogEntry[]; // Network requests/responses
591
+ expectations: Expectation[]; // Test case expectations (for Tester)
592
+ }
593
+ ```
594
+
595
+ ### Expertise (interface)
596
+
597
+ ```typescript
598
+ interface Expertise {
599
+ name: string;
600
+ evaluate(context: ExpertiseContext): Outcome[];
601
+ }
602
+ ```
603
+
604
+ Implementations:
605
+
606
+ | Name | Behavior |
607
+ |------|----------|
608
+ | Tester | Checks each test case expectation is met. Creates error findings for failures. |
609
+ | SEO | Checks meta tags: title, description, keywords, canonical, og:* |
610
+ | Security | Checks network calls for insecure practices (API keys in URLs, HTTP links) |
611
+ | Performance | Checks network timing — all render-blocking content returned within threshold (500ms) |
612
+ | Content | No-op. Returns empty list. |
613
+ | UI | No-op. Returns empty list. |
614
+ | Accessibility | No-op. Returns empty list. |
615
+
616
+ ### PageAnalyzer (singleton)
617
+
618
+ Used only when `discovery = true`.
619
+
620
+ ```typescript
621
+ interface PageAnalyzer {
622
+ // Called BEFORE expertises — generates baseline expectations
623
+ generateExpectations(testCase: TestCase): Expectation[];
624
+
625
+ // Called AFTER expertises — generates new test cases for scaffolds/patterns
626
+ generateTestCases(...): void;
627
+ }
628
+ ```
629
+
630
+ ### Runner Instance
631
+
632
+ ```typescript
633
+ interface RunnerInstance {
634
+ id: string; // Random UUID, generated at startup
635
+ name: string; // Human-readable name
636
+ }
637
+ ```
638
+
639
+ Set on the Test Run object to claim it and prevent concurrent execution by multiple runners.
640
+
641
+ ---
642
+
643
+ ## Enums
644
+
645
+ ### SizeClass
646
+
647
+ `'desktop' | 'mobile'`
648
+
649
+ ### TestType
650
+
651
+ `'render' | 'interaction' | 'form' | 'form_negative' | 'password' | 'navigation' | 'e2e'`
652
+
653
+ ### TestRunStatus
654
+
655
+ `'pending' | 'planned' | 'running' | 'completed' | 'failed'`
656
+
657
+ ### FindingType
658
+
659
+ `'error' | 'warning'`
660
+
661
+ ### PlaywrightAction
662
+
663
+ `'goto' | 'goBack' | 'goForward' | 'reload' | 'click' | 'dblclick' | 'fill' | 'clear' | 'type' | 'press' | 'selectOption' | 'check' | 'uncheck' | 'hover' | 'focus' | 'scrollIntoView' | 'uploadFile' | 'screenshot' | 'waitForLoadState' | 'waitForURL' | 'waitForTimeout'`
664
+
665
+ ### ExpectationType
666
+
667
+ 60+ values. Key ones:
668
+
669
+ - **Element**: `element_visible`, `element_hidden`, `element_exists`, `element_not_exists`, `element_count`, `element_enabled`, `element_disabled`, `element_focused`, `element_checked`, `element_editable`
670
+ - **Text**: `text_content`, `text_contains`, `text_matches`, `text_on_page`, `text_not_on_page`
671
+ - **Input**: `input_value`
672
+ - **Attribute**: `attribute_equals`, `attribute_contains`, `has_class`, `not_has_class`
673
+ - **URL**: `url_equals`, `url_contains`, `url_matches`, `url_changed`, `url_unchanged`
674
+ - **Page**: `page_loaded`, `title_equals`, `title_contains`
675
+ - **Console**: `no_console_errors`, `no_console_warnings`, `console_contains`, `console_not_contains`
676
+ - **Network**: `no_network_errors`, `no_server_errors`, `network_request_made`, `network_response_status`, `no_pending_requests`
677
+ - **Accessibility**: `no_aria_violations`, `element_has_role`, `element_has_label`
678
+ - **Performance**: `load_time_within`, `no_layout_shift`
679
+ - **Form**: `validation_message_visible`, `validation_message_hidden`, `form_submitted_successfully`, `error_state_visible`, `error_state_cleared`
680
+ - **Navigation**: `redirected_to`, `history_length_changed`
681
+ - **Storage**: `cookie_exists`, `cookie_not_exists`, `cookie_value`, `local_storage_value`, `session_storage_value`
682
+ - **Security**: `no_mixed_content`, `sensitive_data_not_in_url`
683
+ - **Visual**: `screenshot_match`
684
+
685
+ ### ExpectationSeverity
686
+
687
+ `'must_pass' | 'should_pass' | 'info'`
688
+
689
+ ### HtmlComponentType (Scaffold Types)
690
+
691
+ `'topMenu' | 'footer' | 'breadcrumb' | 'leftMenu' | 'hamburgerMenu' | 'rightSidebar' | 'searchBar' | 'userMenu' | 'cookieBanner' | 'chatWidget' | 'socialLinks' | 'skipNav' | 'languageSwitcher' | 'announcementBar' | 'backToTop'`
692
+
693
+ ### UiPatternType
694
+
695
+ `'card' | 'table' | 'form' | 'modal' | 'toast' | 'alert' | 'tabs' | 'accordion' | 'carousel' | 'dropdown' | 'pagination' | 'skeleton' | 'emptyState' | 'errorMessage' | 'progressBar' | 'tooltip' | 'badge' | 'avatar' | 'tag' | 'stepper'`
696
+
697
+ ### RunnerType
698
+
699
+ `'worker' | 'extension'`
700
+
701
+ ### RecurrenceType
702
+
703
+ `'one_time' | 'weekday' | 'daily' | 'weekly'`