@sudobility/testomniac_runner 0.0.128
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.dockerignore +75 -0
- package/.env.example +67 -0
- package/.github/workflows/ci-cd.yml +30 -0
- package/.prettierignore +62 -0
- package/.prettierrc +11 -0
- package/.vscode/settings.json +29 -0
- package/CLAUDE.md +170 -0
- package/Dockerfile +76 -0
- package/README.md +22 -0
- package/bun.lock +707 -0
- package/docs/superpowers/specs/2026-04-20-smarter-scanner-navigation-design.md +121 -0
- package/eslint.config.js +80 -0
- package/package.json +55 -0
- package/plans/DATA.md +703 -0
- package/plans/POLLING.md +569 -0
- package/plans/RUNNER.md +288 -0
- package/src/adapters/PuppeteerAdapter.ts +394 -0
- package/src/auth/credential-manager.ts +17 -0
- package/src/auth/form-identifier.test.ts +136 -0
- package/src/auth/form-identifier.ts +54 -0
- package/src/auth/login-executor.ts +112 -0
- package/src/auth/password-detector.test.ts +61 -0
- package/src/auth/password-detector.ts +119 -0
- package/src/auth/signic-registrar.ts +186 -0
- package/src/browser/chromium.ts +35 -0
- package/src/config/index.test.ts +23 -0
- package/src/config/index.ts +35 -0
- package/src/email/deep-link.test.ts +17 -0
- package/src/email/deep-link.ts +23 -0
- package/src/email/sender.ts +35 -0
- package/src/email/templates.ts +34 -0
- package/src/index.test.ts +17 -0
- package/src/index.ts +110 -0
- package/src/orchestrator.ts +220 -0
- package/src/plugins/content/ai-checks.ts +115 -0
- package/src/plugins/content/checks.test.ts +49 -0
- package/src/plugins/content/checks.ts +141 -0
- package/src/plugins/content/index.ts +73 -0
- package/src/plugins/registry.test.ts +49 -0
- package/src/plugins/registry.ts +21 -0
- package/src/plugins/security/header-checks.ts +56 -0
- package/src/plugins/security/html-checks.ts +93 -0
- package/src/plugins/security/index.ts +58 -0
- package/src/plugins/security/network-checks.test.ts +74 -0
- package/src/plugins/security/network-checks.ts +136 -0
- package/src/plugins/seo/checks.test.ts +70 -0
- package/src/plugins/seo/checks.ts +173 -0
- package/src/plugins/seo/index.ts +85 -0
- package/src/plugins/types.ts +43 -0
- package/src/plugins/ui-consistency/comparator.test.ts +108 -0
- package/src/plugins/ui-consistency/comparator.ts +58 -0
- package/src/plugins/ui-consistency/index.ts +36 -0
- package/src/plugins/ui-consistency/style-extractor.ts +79 -0
- package/src/runner/executor.test.ts +37 -0
- package/src/runner/executor.ts +167 -0
- package/src/runner/reporter.ts +19 -0
- package/src/runner/worker-pool.ts +106 -0
- package/src/runner-manager.ts +163 -0
- package/src/scanner/email-checker.ts +106 -0
- package/tsconfig.json +21 -0
package/plans/DATA.md
ADDED
|
@@ -0,0 +1,703 @@
|
|
|
1
|
+
# Testomniac Data Model
|
|
2
|
+
|
|
3
|
+
## Entity Hierarchy
|
|
4
|
+
|
|
5
|
+
```text
|
|
6
|
+
Entity (organization)
|
|
7
|
+
└── Product
|
|
8
|
+
├── Runner
|
|
9
|
+
│ ├── Test Suite Bundle ── uid?
|
|
10
|
+
│ │ └── Test Suite Bundle ↔ Test Suite (junction)
|
|
11
|
+
│ ├── Test Suite ── uid?
|
|
12
|
+
│ │ └── Test Case ── uid?
|
|
13
|
+
│ ├── Scaffold
|
|
14
|
+
│ ├── Page
|
|
15
|
+
│ │ └── Page State
|
|
16
|
+
│ │ ├── Page State Scaffold (junction)
|
|
17
|
+
│ │ └── Page State Pattern
|
|
18
|
+
│ └── Element Identity
|
|
19
|
+
├── Persona
|
|
20
|
+
│ └── Use Case
|
|
21
|
+
│ └── Input Value
|
|
22
|
+
├── Test Credential
|
|
23
|
+
├── Test Environment
|
|
24
|
+
├── Product Settings
|
|
25
|
+
├── API Key
|
|
26
|
+
└── Test File
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
## Execution Hierarchy
|
|
30
|
+
|
|
31
|
+
```text
|
|
32
|
+
Test Run ── runner, discovery, scanUrl, sizeClass, runnerInstanceId
|
|
33
|
+
├── → Test Suite Bundle Run
|
|
34
|
+
│ └── Test Suite Run
|
|
35
|
+
│ └── Test Case Run ── expectedOutcome, observedOutcome
|
|
36
|
+
│ └── Test Run Finding
|
|
37
|
+
├── → Test Suite Run
|
|
38
|
+
│ └── Test Case Run
|
|
39
|
+
│ └── Test Run Finding
|
|
40
|
+
└── → Test Case Run
|
|
41
|
+
└── Test Run Finding
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
A Test Run points to exactly one of: Test Case Run, Test Suite Run, or Test Suite Bundle Run.
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## Persisted Objects
|
|
49
|
+
|
|
50
|
+
### Entity
|
|
51
|
+
|
|
52
|
+
Managed by `@sudobility/entity_service`. Represents an organization.
|
|
53
|
+
|
|
54
|
+
| Field | Type | Notes |
|
|
55
|
+
|-------|------|-------|
|
|
56
|
+
| id | UUID | PK |
|
|
57
|
+
|
|
58
|
+
### Product
|
|
59
|
+
|
|
60
|
+
A software product owned by an entity.
|
|
61
|
+
|
|
62
|
+
| Field | Type | Notes |
|
|
63
|
+
|-------|------|-------|
|
|
64
|
+
| id | BIGSERIAL | PK |
|
|
65
|
+
| entityId | TEXT | Owning entity |
|
|
66
|
+
| title | TEXT | NOT NULL |
|
|
67
|
+
| description | TEXT | |
|
|
68
|
+
| contactEmail | TEXT | |
|
|
69
|
+
| claimedByUserId | TEXT | Firebase uid of claimer |
|
|
70
|
+
| claimedAt | TIMESTAMPTZ | |
|
|
71
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
72
|
+
| updatedAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
73
|
+
|
|
74
|
+
### Runner
|
|
75
|
+
|
|
76
|
+
A target application under a product. Each runner represents a specific app to test.
|
|
77
|
+
|
|
78
|
+
| Field | Type | Notes |
|
|
79
|
+
|-------|------|-------|
|
|
80
|
+
| id | BIGSERIAL | PK |
|
|
81
|
+
| productId | BIGSERIAL | FK → products, NOT NULL |
|
|
82
|
+
| title | TEXT | NOT NULL |
|
|
83
|
+
| type | TEXT | NOT NULL, DEFAULT 'worker'. Values: 'worker', 'extension' |
|
|
84
|
+
| ownerEntityId | TEXT | |
|
|
85
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
86
|
+
|
|
87
|
+
---
|
|
88
|
+
|
|
89
|
+
### Test Suite Bundle
|
|
90
|
+
|
|
91
|
+
A user-created grouping of test suites. Links to test suites via a junction table.
|
|
92
|
+
|
|
93
|
+
| Field | Type | Notes |
|
|
94
|
+
|-------|------|-------|
|
|
95
|
+
| id | BIGSERIAL | PK |
|
|
96
|
+
| runnerId | BIGSERIAL | FK → runners, NOT NULL |
|
|
97
|
+
| title | TEXT | NOT NULL |
|
|
98
|
+
| description | TEXT | |
|
|
99
|
+
| uid | TEXT | **NEW** — Firebase uid of the developer who created it. NULL if created from website. |
|
|
100
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
101
|
+
| updatedAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
102
|
+
|
|
103
|
+
**Junction: test_suite_bundle_suites**
|
|
104
|
+
|
|
105
|
+
| Field | Type | Notes |
|
|
106
|
+
|-------|------|-------|
|
|
107
|
+
| id | BIGSERIAL | PK |
|
|
108
|
+
| testSuiteBundleId | BIGSERIAL | FK → test_suite_bundles, NOT NULL, CASCADE DELETE |
|
|
109
|
+
| testSuiteId | BIGSERIAL | FK → test_suites, NOT NULL, CASCADE DELETE |
|
|
110
|
+
|
|
111
|
+
Unique index on (testSuiteBundleId, testSuiteId).
|
|
112
|
+
|
|
113
|
+
### Test Suite
|
|
114
|
+
|
|
115
|
+
A collection of test cases for a specific area of the application (a page, a scaffold, a pattern, etc.).
|
|
116
|
+
|
|
117
|
+
| Field | Type | Notes |
|
|
118
|
+
|-------|------|-------|
|
|
119
|
+
| id | BIGSERIAL | PK |
|
|
120
|
+
| runnerId | BIGSERIAL | FK → runners, NOT NULL |
|
|
121
|
+
| decompositionJobId | BIGSERIAL | Links to the AI job that created it |
|
|
122
|
+
| title | TEXT | NOT NULL |
|
|
123
|
+
| description | TEXT | NOT NULL |
|
|
124
|
+
| startingPageStateId | BIGSERIAL | FK → page_states |
|
|
125
|
+
| startingPath | TEXT | NOT NULL — relative URL path |
|
|
126
|
+
| sizeClass | TEXT | NOT NULL, DEFAULT 'desktop'. Values: 'desktop', 'mobile' |
|
|
127
|
+
| dependencyTestCaseId | BIGSERIAL | Test case that must pass before this suite runs |
|
|
128
|
+
| personaIdsJson | JSONB | Array of persona IDs |
|
|
129
|
+
| scaffoldId | BIGSERIAL | FK → scaffolds. Set if this suite tests a specific scaffold |
|
|
130
|
+
| scaffoldType | TEXT | HtmlComponentType value (e.g., 'topMenu', 'footer') |
|
|
131
|
+
| patternType | TEXT | UiPatternType value (e.g., 'form', 'table'). Set if testing a pattern |
|
|
132
|
+
| priority | INTEGER | NOT NULL, DEFAULT 3 |
|
|
133
|
+
| suiteTags | TEXT[] | NOT NULL, DEFAULT [] |
|
|
134
|
+
| estimatedDurationMs | INTEGER | |
|
|
135
|
+
| uid | TEXT | **NEW** — Firebase uid. NULL if created from website. |
|
|
136
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
137
|
+
|
|
138
|
+
### Test Case
|
|
139
|
+
|
|
140
|
+
An individual test scenario with steps and expectations.
|
|
141
|
+
|
|
142
|
+
| Field | Type | Notes |
|
|
143
|
+
|-------|------|-------|
|
|
144
|
+
| id | BIGSERIAL | PK |
|
|
145
|
+
| runnerId | BIGSERIAL | FK → runners |
|
|
146
|
+
| testSuiteId | BIGSERIAL | FK → test_suites |
|
|
147
|
+
| title | TEXT | NOT NULL |
|
|
148
|
+
| testType | TEXT | NOT NULL. Values: 'render', 'interaction', 'form', 'form_negative', 'password', 'navigation', 'e2e' |
|
|
149
|
+
| sizeClass | TEXT | NOT NULL, DEFAULT 'desktop' |
|
|
150
|
+
| suiteTags | TEXT[] | NOT NULL, DEFAULT [] |
|
|
151
|
+
| priority | INTEGER | NOT NULL, DEFAULT 3 |
|
|
152
|
+
| scaffoldId | BIGSERIAL | FK → scaffolds |
|
|
153
|
+
| patternType | TEXT | |
|
|
154
|
+
| dependencyTestCaseId | BIGSERIAL | Must pass before this case runs |
|
|
155
|
+
| pageId | BIGSERIAL | FK → pages |
|
|
156
|
+
| personaId | BIGSERIAL | FK → personas |
|
|
157
|
+
| useCaseId | BIGSERIAL | FK → use_cases |
|
|
158
|
+
| startingPageStateId | BIGSERIAL | FK → page_states |
|
|
159
|
+
| startingPath | TEXT | Relative URL |
|
|
160
|
+
| stepsJson | JSONB | Array of TestStep objects |
|
|
161
|
+
| globalExpectationsJson | JSONB | Array of Expectation objects |
|
|
162
|
+
| estimatedDurationMs | INTEGER | |
|
|
163
|
+
| uid | TEXT | **NEW** — Firebase uid. NULL if created from website. |
|
|
164
|
+
| generatedAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
165
|
+
|
|
166
|
+
### TestStep (embedded in stepsJson)
|
|
167
|
+
|
|
168
|
+
```typescript
|
|
169
|
+
{
|
|
170
|
+
action: TestAction;
|
|
171
|
+
expectations: Expectation[];
|
|
172
|
+
description: string;
|
|
173
|
+
continueOnFailure: boolean;
|
|
174
|
+
}
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
### TestAction (embedded in TestStep)
|
|
178
|
+
|
|
179
|
+
```typescript
|
|
180
|
+
{
|
|
181
|
+
actionType: PlaywrightAction; // 'goto', 'click', 'fill', 'hover', etc.
|
|
182
|
+
pageStateId?: number;
|
|
183
|
+
elementIdentityId?: number;
|
|
184
|
+
containerType?: HtmlComponentType;
|
|
185
|
+
containerElementIdentityId?: number;
|
|
186
|
+
value?: string;
|
|
187
|
+
path?: string;
|
|
188
|
+
playwrightCode: string; // Actual Playwright code to execute
|
|
189
|
+
description: string;
|
|
190
|
+
}
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
### Expectation (embedded in TestStep and globalExpectations)
|
|
194
|
+
|
|
195
|
+
```typescript
|
|
196
|
+
{
|
|
197
|
+
expectationType: ExpectationType; // 'page_loaded', 'element_visible', 'no_console_errors', etc.
|
|
198
|
+
elementIdentityId?: number;
|
|
199
|
+
expectedValue?: string;
|
|
200
|
+
attributeName?: string;
|
|
201
|
+
severity: ExpectationSeverity; // 'must_pass', 'should_pass', 'info'
|
|
202
|
+
description: string;
|
|
203
|
+
playwrightCode: string;
|
|
204
|
+
}
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
---
|
|
208
|
+
|
|
209
|
+
### Test Suite Bundle Run
|
|
210
|
+
|
|
211
|
+
Execution record for a test suite bundle.
|
|
212
|
+
|
|
213
|
+
| Field | Type | Notes |
|
|
214
|
+
|-------|------|-------|
|
|
215
|
+
| id | BIGSERIAL | PK |
|
|
216
|
+
| testSuiteBundleId | BIGSERIAL | FK → test_suite_bundles, NOT NULL |
|
|
217
|
+
| status | TEXT | NOT NULL, DEFAULT 'pending'. Values: 'pending', 'running', 'completed', 'failed' |
|
|
218
|
+
| startedAt | TIMESTAMPTZ | |
|
|
219
|
+
| completedAt | TIMESTAMPTZ | |
|
|
220
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
221
|
+
|
|
222
|
+
### Test Suite Run
|
|
223
|
+
|
|
224
|
+
Execution record for a test suite.
|
|
225
|
+
|
|
226
|
+
| Field | Type | Notes |
|
|
227
|
+
|-------|------|-------|
|
|
228
|
+
| id | BIGSERIAL | PK |
|
|
229
|
+
| testSuiteId | BIGSERIAL | FK → test_suites, NOT NULL |
|
|
230
|
+
| testSuiteBundleRunId | BIGSERIAL | FK → test_suite_bundle_runs. Set if this suite run is part of a bundle run. |
|
|
231
|
+
| status | TEXT | NOT NULL, DEFAULT 'pending' |
|
|
232
|
+
| startedAt | TIMESTAMPTZ | |
|
|
233
|
+
| completedAt | TIMESTAMPTZ | |
|
|
234
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
235
|
+
|
|
236
|
+
### Test Case Run
|
|
237
|
+
|
|
238
|
+
Execution record for a single test case.
|
|
239
|
+
|
|
240
|
+
| Field | Type | Notes |
|
|
241
|
+
|-------|------|-------|
|
|
242
|
+
| id | BIGSERIAL | PK |
|
|
243
|
+
| testCaseId | BIGSERIAL | FK → test_cases, NOT NULL |
|
|
244
|
+
| testSuiteRunId | BIGSERIAL | FK → test_suite_runs. Set if this case run is part of a suite run. |
|
|
245
|
+
| status | TEXT | NOT NULL, DEFAULT 'pending'. Values: 'pending', 'running', 'completed', 'failed' |
|
|
246
|
+
| durationMs | INTEGER | |
|
|
247
|
+
| errorMessage | TEXT | Error details if failed |
|
|
248
|
+
| expectedOutcome | TEXT | **NEW** — Aggregated expected outcomes from expertises |
|
|
249
|
+
| observedOutcome | TEXT | **NEW** — Aggregated observed outcomes from expertises |
|
|
250
|
+
| screenshotPath | TEXT | Captured on warning/error |
|
|
251
|
+
| consoleLog | TEXT | Captured on warning/error |
|
|
252
|
+
| networkLog | TEXT | Captured on warning/error |
|
|
253
|
+
| startedAt | TIMESTAMPTZ | |
|
|
254
|
+
| completedAt | TIMESTAMPTZ | |
|
|
255
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
256
|
+
|
|
257
|
+
### Test Run
|
|
258
|
+
|
|
259
|
+
Top-level run object. Represents a single invocation of the runner.
|
|
260
|
+
|
|
261
|
+
| Field | Type | Notes |
|
|
262
|
+
|-------|------|-------|
|
|
263
|
+
| id | BIGSERIAL | PK |
|
|
264
|
+
| runnerId | BIGSERIAL | FK → runners, NOT NULL |
|
|
265
|
+
| testCaseRunId | BIGSERIAL | FK → test_case_runs. Set if running a single test case. |
|
|
266
|
+
| testSuiteRunId | BIGSERIAL | FK → test_suite_runs. Set if running a test suite. |
|
|
267
|
+
| testSuiteBundleRunId | BIGSERIAL | FK → test_suite_bundle_runs. Set if running a bundle. |
|
|
268
|
+
| testEnvironmentId | BIGSERIAL | **NEW** — FK → test_environments. Required. Specifies which environment this run targets. |
|
|
269
|
+
| discovery | BOOLEAN | NOT NULL, DEFAULT FALSE. If true, auto-discover new test suites/cases. |
|
|
270
|
+
| parentTestRunId | BIGSERIAL | FK → test_runs. For child runs. |
|
|
271
|
+
| rootTestRunId | BIGSERIAL | FK → test_runs. Root of the run tree. |
|
|
272
|
+
| sizeClass | TEXT | NOT NULL, DEFAULT 'desktop' |
|
|
273
|
+
| status | TEXT | NOT NULL, DEFAULT 'pending'. Values: 'pending', 'planned', 'running', 'completed', 'failed' |
|
|
274
|
+
| createdByUserId | TEXT | Firebase uid of the user who started the run |
|
|
275
|
+
| ownedByUserId | TEXT | Firebase uid of the run owner |
|
|
276
|
+
| runnerInstanceId | TEXT | **NEW** — UUID of the runner instance executing this run |
|
|
277
|
+
| runnerInstanceName | TEXT | **NEW** — Human-readable name of the runner instance |
|
|
278
|
+
| scanUrl | TEXT | Starting URL for the scan |
|
|
279
|
+
| pagesFound | INTEGER | Stats |
|
|
280
|
+
| pageStatesFound | INTEGER | Stats |
|
|
281
|
+
| testRunsCompleted | INTEGER | Stats |
|
|
282
|
+
| aiSummary | TEXT | AI-generated summary |
|
|
283
|
+
| totalDurationMs | INTEGER | |
|
|
284
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
285
|
+
| startedAt | TIMESTAMPTZ | |
|
|
286
|
+
| completedAt | TIMESTAMPTZ | |
|
|
287
|
+
|
|
288
|
+
### Test Run Finding
|
|
289
|
+
|
|
290
|
+
An issue discovered during a test case run.
|
|
291
|
+
|
|
292
|
+
| Field | Type | Notes |
|
|
293
|
+
|-------|------|-------|
|
|
294
|
+
| id | BIGSERIAL | PK |
|
|
295
|
+
| testCaseRunId | BIGSERIAL | FK → test_case_runs, NOT NULL |
|
|
296
|
+
| expertiseRuleId | BIGSERIAL | FK → expertise_rules. Links to the rule that flagged it. |
|
|
297
|
+
| type | TEXT | NOT NULL. Values: 'error', 'warning' |
|
|
298
|
+
| title | TEXT | NOT NULL |
|
|
299
|
+
| description | TEXT | NOT NULL |
|
|
300
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
301
|
+
|
|
302
|
+
---
|
|
303
|
+
|
|
304
|
+
### Page
|
|
305
|
+
|
|
306
|
+
A URL path discovered under a runner.
|
|
307
|
+
|
|
308
|
+
| Field | Type | Notes |
|
|
309
|
+
|-------|------|-------|
|
|
310
|
+
| id | BIGSERIAL | PK |
|
|
311
|
+
| runnerId | BIGSERIAL | FK → runners |
|
|
312
|
+
| relativePath | TEXT | NOT NULL — URL path relative to base URL |
|
|
313
|
+
| routeKey | TEXT | Normalized route key (e.g., `/users/:id`) |
|
|
314
|
+
| requiresLogin | BOOLEAN | DEFAULT FALSE |
|
|
315
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
316
|
+
|
|
317
|
+
### Page State
|
|
318
|
+
|
|
319
|
+
A snapshot of a page at a point in time. Multiple states can exist per page (different viewports, different content).
|
|
320
|
+
|
|
321
|
+
| Field | Type | Notes |
|
|
322
|
+
|-------|------|-------|
|
|
323
|
+
| id | BIGSERIAL | PK |
|
|
324
|
+
| pageId | BIGSERIAL | FK → pages |
|
|
325
|
+
| sizeClass | TEXT | NOT NULL, DEFAULT 'desktop' |
|
|
326
|
+
| htmlHash | CHAR(64) | SHA-256 of raw HTML |
|
|
327
|
+
| normalizedHtmlHash | CHAR(64) | SHA-256 of normalized HTML |
|
|
328
|
+
| textHash | CHAR(64) | SHA-256 of visible text |
|
|
329
|
+
| actionableHash | CHAR(64) | SHA-256 of actionable items |
|
|
330
|
+
| fixedBodyHash | CHAR(64) | SHA-256 of body minus scaffolds and patterns |
|
|
331
|
+
| scaffoldsHash | CHAR(64) | SHA-256 of scaffold type:hash pairs |
|
|
332
|
+
| patternsHash | CHAR(64) | SHA-256 of pattern type:count pairs |
|
|
333
|
+
| bodyHtmlElementId | BIGSERIAL | FK → html_elements |
|
|
334
|
+
| contentHtmlElementId | BIGSERIAL | FK → html_elements |
|
|
335
|
+
| fixedBodyHtmlElementId | BIGSERIAL | FK → html_elements |
|
|
336
|
+
| screenshotPath | TEXT | |
|
|
337
|
+
| rawHtmlPath | TEXT | |
|
|
338
|
+
| contentText | TEXT | Extracted visible text |
|
|
339
|
+
| createdByTestRunId | BIGSERIAL | Which test run captured this state |
|
|
340
|
+
| capturedAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
341
|
+
|
|
342
|
+
### Scaffold
|
|
343
|
+
|
|
344
|
+
A detected reusable UI region (header, footer, nav, sidebar, etc.) that appears across multiple pages.
|
|
345
|
+
|
|
346
|
+
| Field | Type | Notes |
|
|
347
|
+
|-------|------|-------|
|
|
348
|
+
| id | BIGSERIAL | PK |
|
|
349
|
+
| runnerId | BIGSERIAL | FK → runners, NOT NULL |
|
|
350
|
+
| type | TEXT | NOT NULL. HtmlComponentType: 'topMenu', 'footer', 'breadcrumb', 'leftMenu', 'hamburgerMenu', 'rightSidebar', 'searchBar', 'userMenu', 'cookieBanner', 'chatWidget', 'socialLinks', 'skipNav', 'languageSwitcher', 'announcementBar', 'backToTop' |
|
|
351
|
+
| htmlElementId | BIGSERIAL | FK → html_elements, NOT NULL |
|
|
352
|
+
| htmlHash | CHAR(64) | |
|
|
353
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
354
|
+
|
|
355
|
+
Unique index on (runnerId, type, htmlElementId).
|
|
356
|
+
|
|
357
|
+
**Junction: page_state_scaffolds**
|
|
358
|
+
|
|
359
|
+
| Field | Type | Notes |
|
|
360
|
+
|-------|------|-------|
|
|
361
|
+
| id | BIGSERIAL | PK |
|
|
362
|
+
| pageStateId | BIGSERIAL | FK → page_states, NOT NULL |
|
|
363
|
+
| scaffoldId | BIGSERIAL | FK → scaffolds, NOT NULL |
|
|
364
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
365
|
+
|
|
366
|
+
Unique index on (pageStateId, scaffoldId).
|
|
367
|
+
|
|
368
|
+
### Page State Pattern
|
|
369
|
+
|
|
370
|
+
A detected UI pattern instance on a page state.
|
|
371
|
+
|
|
372
|
+
| Field | Type | Notes |
|
|
373
|
+
|-------|------|-------|
|
|
374
|
+
| id | BIGSERIAL | PK |
|
|
375
|
+
| pageStateId | BIGSERIAL | FK → page_states, NOT NULL |
|
|
376
|
+
| patternType | TEXT | NOT NULL. UiPatternType: 'card', 'table', 'form', 'modal', 'toast', 'alert', 'tabs', 'accordion', 'carousel', 'dropdown', 'pagination', 'skeleton', 'emptyState', 'errorMessage', 'progressBar', 'tooltip', 'badge', 'avatar', 'tag', 'stepper' |
|
|
377
|
+
| selector | TEXT | NOT NULL — CSS selector for the pattern |
|
|
378
|
+
| count | INTEGER | NOT NULL, DEFAULT 0 — instance count on the page |
|
|
379
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
380
|
+
|
|
381
|
+
### HTML Element
|
|
382
|
+
|
|
383
|
+
Deduplicated HTML content storage, referenced by scaffolds and page states.
|
|
384
|
+
|
|
385
|
+
| Field | Type | Notes |
|
|
386
|
+
|-------|------|-------|
|
|
387
|
+
| id | BIGSERIAL | PK |
|
|
388
|
+
| html | TEXT | NOT NULL — raw HTML markup |
|
|
389
|
+
| hash | CHAR(64) | NOT NULL, UNIQUE — SHA-256 for deduplication |
|
|
390
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
391
|
+
|
|
392
|
+
### Element Identity
|
|
393
|
+
|
|
394
|
+
Persistent identity for an interactive element across scans. Allows stable references even when selectors change.
|
|
395
|
+
|
|
396
|
+
| Field | Type | Notes |
|
|
397
|
+
|-------|------|-------|
|
|
398
|
+
| id | BIGSERIAL | PK |
|
|
399
|
+
| runnerId | BIGSERIAL | FK → runners, NOT NULL |
|
|
400
|
+
| role | TEXT | NOT NULL — ARIA role |
|
|
401
|
+
| computedName | TEXT | ARIA computed name |
|
|
402
|
+
| tagName | TEXT | NOT NULL |
|
|
403
|
+
| labelText | TEXT | |
|
|
404
|
+
| groupName | TEXT | |
|
|
405
|
+
| placeholder | TEXT | |
|
|
406
|
+
| altText | TEXT | |
|
|
407
|
+
| testId | TEXT | data-testid value |
|
|
408
|
+
| inputType | TEXT | |
|
|
409
|
+
| nthInGroup | INTEGER | Position in group |
|
|
410
|
+
| formContext | TEXT | Parent form context |
|
|
411
|
+
| headingContext | TEXT | Nearest heading |
|
|
412
|
+
| landmarkAncestor | TEXT | Parent ARIA landmark |
|
|
413
|
+
| playwrightLocator | TEXT | NOT NULL — primary locator string |
|
|
414
|
+
| playwrightScopeChain | TEXT | Scoped locator chain |
|
|
415
|
+
| isUniqueOnPage | BOOLEAN | NOT NULL, DEFAULT TRUE |
|
|
416
|
+
| cssSelector | TEXT | NOT NULL — fallback CSS selector |
|
|
417
|
+
| locators | JSONB | NOT NULL, DEFAULT [] — array of ElementLocator |
|
|
418
|
+
| firstSeenTestRunId | BIGSERIAL | |
|
|
419
|
+
| lastSeenTestRunId | BIGSERIAL | |
|
|
420
|
+
| timesSeen | INTEGER | NOT NULL, DEFAULT 1 |
|
|
421
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
422
|
+
| updatedAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
423
|
+
|
|
424
|
+
### Actionable Item
|
|
425
|
+
|
|
426
|
+
An interactive element on a page state (button, input, link, etc.).
|
|
427
|
+
|
|
428
|
+
| Field | Type | Notes |
|
|
429
|
+
|-------|------|-------|
|
|
430
|
+
| id | BIGSERIAL | PK |
|
|
431
|
+
| htmlElementId | BIGSERIAL | FK → html_elements |
|
|
432
|
+
| stableKey | TEXT | Deterministic key for deduplication |
|
|
433
|
+
| selector | TEXT | CSS selector |
|
|
434
|
+
| tagName | TEXT | |
|
|
435
|
+
| role | TEXT | ARIA role |
|
|
436
|
+
| actionKind | TEXT | 'click', 'fill', 'select', 'navigate', 'radio_select' |
|
|
437
|
+
| accessibleName | TEXT | |
|
|
438
|
+
| disabled | BOOLEAN | |
|
|
439
|
+
| visible | BOOLEAN | |
|
|
440
|
+
| attributesJson | JSONB | All HTML attributes |
|
|
441
|
+
| scaffoldId | BIGSERIAL | If this item belongs to a scaffold |
|
|
442
|
+
| elementIdentityId | BIGSERIAL | FK → element_identities |
|
|
443
|
+
|
|
444
|
+
---
|
|
445
|
+
|
|
446
|
+
### Expertise
|
|
447
|
+
|
|
448
|
+
A registered expertise category (e.g., SEO, Security, Performance).
|
|
449
|
+
|
|
450
|
+
| Field | Type | Notes |
|
|
451
|
+
|-------|------|-------|
|
|
452
|
+
| id | BIGSERIAL | PK |
|
|
453
|
+
| slug | TEXT | NOT NULL, UNIQUE — e.g., 'seo', 'security' |
|
|
454
|
+
| title | TEXT | NOT NULL |
|
|
455
|
+
| description | TEXT | NOT NULL |
|
|
456
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
457
|
+
|
|
458
|
+
### Expertise Rule
|
|
459
|
+
|
|
460
|
+
A specific rule within an expertise (e.g., "Title tag present" under SEO).
|
|
461
|
+
|
|
462
|
+
| Field | Type | Notes |
|
|
463
|
+
|-------|------|-------|
|
|
464
|
+
| id | BIGSERIAL | PK |
|
|
465
|
+
| expertiseId | BIGSERIAL | FK → expertises, NOT NULL |
|
|
466
|
+
| title | TEXT | NOT NULL |
|
|
467
|
+
| description | TEXT | NOT NULL |
|
|
468
|
+
| aiEndpointUrl | TEXT | Optional AI evaluation endpoint |
|
|
469
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
470
|
+
|
|
471
|
+
---
|
|
472
|
+
|
|
473
|
+
### Persona
|
|
474
|
+
|
|
475
|
+
An AI-generated user persona for a product.
|
|
476
|
+
|
|
477
|
+
| Field | Type | Notes |
|
|
478
|
+
|-------|------|-------|
|
|
479
|
+
| id | BIGSERIAL | PK |
|
|
480
|
+
| productId | BIGSERIAL | FK → products |
|
|
481
|
+
| title | TEXT | NOT NULL |
|
|
482
|
+
| description | TEXT | |
|
|
483
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
484
|
+
|
|
485
|
+
### Use Case
|
|
486
|
+
|
|
487
|
+
A use case for a persona with specific input values.
|
|
488
|
+
|
|
489
|
+
| Field | Type | Notes |
|
|
490
|
+
|-------|------|-------|
|
|
491
|
+
| id | BIGSERIAL | PK |
|
|
492
|
+
| personaId | BIGSERIAL | FK → personas |
|
|
493
|
+
| title | TEXT | NOT NULL |
|
|
494
|
+
| description | TEXT | |
|
|
495
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
496
|
+
|
|
497
|
+
### Input Value
|
|
498
|
+
|
|
499
|
+
A stored form field value for a use case.
|
|
500
|
+
|
|
501
|
+
| Field | Type | Notes |
|
|
502
|
+
|-------|------|-------|
|
|
503
|
+
| id | BIGSERIAL | PK |
|
|
504
|
+
| useCaseId | BIGSERIAL | FK → use_cases |
|
|
505
|
+
| fieldSelector | TEXT | NOT NULL |
|
|
506
|
+
| fieldName | TEXT | |
|
|
507
|
+
| value | TEXT | NOT NULL |
|
|
508
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
509
|
+
|
|
510
|
+
### Test Credential
|
|
511
|
+
|
|
512
|
+
Stored login credentials for a product, optionally tied to a persona.
|
|
513
|
+
|
|
514
|
+
| Field | Type | Notes |
|
|
515
|
+
|-------|------|-------|
|
|
516
|
+
| id | BIGSERIAL | PK |
|
|
517
|
+
| productId | BIGSERIAL | FK → products, NOT NULL |
|
|
518
|
+
| personaId | BIGSERIAL | FK → personas |
|
|
519
|
+
| signic | BOOLEAN | NOT NULL, DEFAULT FALSE — whether to use Signic email service |
|
|
520
|
+
| email | TEXT | |
|
|
521
|
+
| password | TEXT | |
|
|
522
|
+
| verificationCode | TEXT | |
|
|
523
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
524
|
+
|
|
525
|
+
### Test Environment
|
|
526
|
+
|
|
527
|
+
A deployment environment for a product (e.g., dev, staging, production).
|
|
528
|
+
|
|
529
|
+
| Field | Type | Notes |
|
|
530
|
+
|-------|------|-------|
|
|
531
|
+
| id | BIGSERIAL | PK |
|
|
532
|
+
| productId | BIGSERIAL | FK → products, NOT NULL |
|
|
533
|
+
| title | TEXT | NOT NULL |
|
|
534
|
+
| baseUrl | TEXT | NOT NULL |
|
|
535
|
+
| githubBranch | TEXT | |
|
|
536
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
537
|
+
| updatedAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
538
|
+
|
|
539
|
+
### Test Schedule
|
|
540
|
+
|
|
541
|
+
A recurring schedule for automated test runs.
|
|
542
|
+
|
|
543
|
+
| Field | Type | Notes |
|
|
544
|
+
|-------|------|-------|
|
|
545
|
+
| id | BIGSERIAL | PK |
|
|
546
|
+
| runnerId | BIGSERIAL | FK → runners, NOT NULL |
|
|
547
|
+
| title | TEXT | NOT NULL |
|
|
548
|
+
| testSuiteId | BIGSERIAL | FK → test_suites |
|
|
549
|
+
| testCaseId | BIGSERIAL | FK → test_cases |
|
|
550
|
+
| testSuiteBundleId | BIGSERIAL | FK → test_suite_bundles |
|
|
551
|
+
| discovery | BOOLEAN | NOT NULL, DEFAULT FALSE |
|
|
552
|
+
| recurrenceType | TEXT | NOT NULL. Values: 'one_time', 'weekday', 'daily', 'weekly' |
|
|
553
|
+
| timeOfDay | TEXT | NOT NULL — e.g., '09:00' |
|
|
554
|
+
| dayOfWeek | INTEGER | 0-6 for weekly |
|
|
555
|
+
| timezone | TEXT | NOT NULL |
|
|
556
|
+
| enabled | BOOLEAN | NOT NULL, DEFAULT TRUE |
|
|
557
|
+
| sizeClass | TEXT | NOT NULL, DEFAULT 'desktop' |
|
|
558
|
+
| createdByUserId | TEXT | NOT NULL — Firebase uid |
|
|
559
|
+
| lastRunAt | TIMESTAMPTZ | |
|
|
560
|
+
| nextRunAt | TIMESTAMPTZ | |
|
|
561
|
+
| createdAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
562
|
+
| updatedAt | TIMESTAMPTZ | DEFAULT NOW() |
|
|
563
|
+
|
|
564
|
+
---
|
|
565
|
+
|
|
566
|
+
## In-Memory Objects (Runner Service)
|
|
567
|
+
|
|
568
|
+
### Outcome
|
|
569
|
+
|
|
570
|
+
Produced by each expertise during test case evaluation.
|
|
571
|
+
|
|
572
|
+
```typescript
|
|
573
|
+
interface Outcome {
|
|
574
|
+
expected: string; // What was expected
|
|
575
|
+
observed: string; // What was actually observed
|
|
576
|
+
result: 'pass' | 'warning' | 'error';
|
|
577
|
+
}
|
|
578
|
+
```
|
|
579
|
+
|
|
580
|
+
### ExpertiseContext
|
|
581
|
+
|
|
582
|
+
Passed to each expertise for evaluation.
|
|
583
|
+
|
|
584
|
+
```typescript
|
|
585
|
+
interface ExpertiseContext {
|
|
586
|
+
html: string; // Full page HTML including <head>
|
|
587
|
+
scaffolds: DetectedScaffoldRegion[]; // Decomposed scaffold list
|
|
588
|
+
patterns: DetectedPatternWithInstances[]; // Decomposed pattern list
|
|
589
|
+
consoleLogs: string[]; // Console output during test case
|
|
590
|
+
networkLogs: NetworkLogEntry[]; // Network requests/responses
|
|
591
|
+
expectations: Expectation[]; // Test case expectations (for Tester)
|
|
592
|
+
}
|
|
593
|
+
```
|
|
594
|
+
|
|
595
|
+
### Expertise (interface)
|
|
596
|
+
|
|
597
|
+
```typescript
|
|
598
|
+
interface Expertise {
|
|
599
|
+
name: string;
|
|
600
|
+
evaluate(context: ExpertiseContext): Outcome[];
|
|
601
|
+
}
|
|
602
|
+
```
|
|
603
|
+
|
|
604
|
+
Implementations:
|
|
605
|
+
|
|
606
|
+
| Name | Behavior |
|
|
607
|
+
|------|----------|
|
|
608
|
+
| Tester | Checks each test case expectation is met. Creates error findings for failures. |
|
|
609
|
+
| SEO | Checks meta tags: title, description, keywords, canonical, og:* |
|
|
610
|
+
| Security | Checks network calls for insecure practices (API keys in URLs, HTTP links) |
|
|
611
|
+
| Performance | Checks network timing — all render-blocking content returned within threshold (500ms) |
|
|
612
|
+
| Content | No-op. Returns empty list. |
|
|
613
|
+
| UI | No-op. Returns empty list. |
|
|
614
|
+
| Accessibility | No-op. Returns empty list. |
|
|
615
|
+
|
|
616
|
+
### PageAnalyzer (singleton)
|
|
617
|
+
|
|
618
|
+
Used only when `discovery = true`.
|
|
619
|
+
|
|
620
|
+
```typescript
|
|
621
|
+
interface PageAnalyzer {
|
|
622
|
+
// Called BEFORE expertises — generates baseline expectations
|
|
623
|
+
generateExpectations(testCase: TestCase): Expectation[];
|
|
624
|
+
|
|
625
|
+
// Called AFTER expertises — generates new test cases for scaffolds/patterns
|
|
626
|
+
generateTestCases(...): void;
|
|
627
|
+
}
|
|
628
|
+
```
|
|
629
|
+
|
|
630
|
+
### Runner Instance
|
|
631
|
+
|
|
632
|
+
```typescript
|
|
633
|
+
interface RunnerInstance {
|
|
634
|
+
id: string; // Random UUID, generated at startup
|
|
635
|
+
name: string; // Human-readable name
|
|
636
|
+
}
|
|
637
|
+
```
|
|
638
|
+
|
|
639
|
+
Set on the Test Run object to claim it and prevent concurrent execution by multiple runners.
|
|
640
|
+
|
|
641
|
+
---
|
|
642
|
+
|
|
643
|
+
## Enums
|
|
644
|
+
|
|
645
|
+
### SizeClass
|
|
646
|
+
|
|
647
|
+
`'desktop' | 'mobile'`
|
|
648
|
+
|
|
649
|
+
### TestType
|
|
650
|
+
|
|
651
|
+
`'render' | 'interaction' | 'form' | 'form_negative' | 'password' | 'navigation' | 'e2e'`
|
|
652
|
+
|
|
653
|
+
### TestRunStatus
|
|
654
|
+
|
|
655
|
+
`'pending' | 'planned' | 'running' | 'completed' | 'failed'`
|
|
656
|
+
|
|
657
|
+
### FindingType
|
|
658
|
+
|
|
659
|
+
`'error' | 'warning'`
|
|
660
|
+
|
|
661
|
+
### PlaywrightAction
|
|
662
|
+
|
|
663
|
+
`'goto' | 'goBack' | 'goForward' | 'reload' | 'click' | 'dblclick' | 'fill' | 'clear' | 'type' | 'press' | 'selectOption' | 'check' | 'uncheck' | 'hover' | 'focus' | 'scrollIntoView' | 'uploadFile' | 'screenshot' | 'waitForLoadState' | 'waitForURL' | 'waitForTimeout'`
|
|
664
|
+
|
|
665
|
+
### ExpectationType
|
|
666
|
+
|
|
667
|
+
60+ values. Key ones:
|
|
668
|
+
|
|
669
|
+
- **Element**: `element_visible`, `element_hidden`, `element_exists`, `element_not_exists`, `element_count`, `element_enabled`, `element_disabled`, `element_focused`, `element_checked`, `element_editable`
|
|
670
|
+
- **Text**: `text_content`, `text_contains`, `text_matches`, `text_on_page`, `text_not_on_page`
|
|
671
|
+
- **Input**: `input_value`
|
|
672
|
+
- **Attribute**: `attribute_equals`, `attribute_contains`, `has_class`, `not_has_class`
|
|
673
|
+
- **URL**: `url_equals`, `url_contains`, `url_matches`, `url_changed`, `url_unchanged`
|
|
674
|
+
- **Page**: `page_loaded`, `title_equals`, `title_contains`
|
|
675
|
+
- **Console**: `no_console_errors`, `no_console_warnings`, `console_contains`, `console_not_contains`
|
|
676
|
+
- **Network**: `no_network_errors`, `no_server_errors`, `network_request_made`, `network_response_status`, `no_pending_requests`
|
|
677
|
+
- **Accessibility**: `no_aria_violations`, `element_has_role`, `element_has_label`
|
|
678
|
+
- **Performance**: `load_time_within`, `no_layout_shift`
|
|
679
|
+
- **Form**: `validation_message_visible`, `validation_message_hidden`, `form_submitted_successfully`, `error_state_visible`, `error_state_cleared`
|
|
680
|
+
- **Navigation**: `redirected_to`, `history_length_changed`
|
|
681
|
+
- **Storage**: `cookie_exists`, `cookie_not_exists`, `cookie_value`, `local_storage_value`, `session_storage_value`
|
|
682
|
+
- **Security**: `no_mixed_content`, `sensitive_data_not_in_url`
|
|
683
|
+
- **Visual**: `screenshot_match`
|
|
684
|
+
|
|
685
|
+
### ExpectationSeverity
|
|
686
|
+
|
|
687
|
+
`'must_pass' | 'should_pass' | 'info'`
|
|
688
|
+
|
|
689
|
+
### HtmlComponentType (Scaffold Types)
|
|
690
|
+
|
|
691
|
+
`'topMenu' | 'footer' | 'breadcrumb' | 'leftMenu' | 'hamburgerMenu' | 'rightSidebar' | 'searchBar' | 'userMenu' | 'cookieBanner' | 'chatWidget' | 'socialLinks' | 'skipNav' | 'languageSwitcher' | 'announcementBar' | 'backToTop'`
|
|
692
|
+
|
|
693
|
+
### UiPatternType
|
|
694
|
+
|
|
695
|
+
`'card' | 'table' | 'form' | 'modal' | 'toast' | 'alert' | 'tabs' | 'accordion' | 'carousel' | 'dropdown' | 'pagination' | 'skeleton' | 'emptyState' | 'errorMessage' | 'progressBar' | 'tooltip' | 'badge' | 'avatar' | 'tag' | 'stepper'`
|
|
696
|
+
|
|
697
|
+
### RunnerType
|
|
698
|
+
|
|
699
|
+
`'worker' | 'extension'`
|
|
700
|
+
|
|
701
|
+
### RecurrenceType
|
|
702
|
+
|
|
703
|
+
`'one_time' | 'weekday' | 'daily' | 'weekly'`
|