qualiow-exploratory-testing 1.3.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +22 -0
- package/.env.example +27 -0
- package/CHANGELOG.md +194 -0
- package/README.md +362 -205
- package/agents/qa-gather-agent.md +24 -0
- package/{scripts → bin}/doctor-mobile.sh +57 -18
- package/bin/mcli +15 -9
- package/bin/mobile-cli.mjs +574 -286
- package/{scripts → bin}/setup-mobile.sh +56 -19
- package/bin/wadb +10 -4
- package/bin/wk-ios +29 -17
- package/bin/wkeval.mjs +7 -1
- package/data/domains/_default.yml +1 -1
- package/data/knowledge/changelog.yml +30 -12
- package/data/knowledge/manifest.yml +333 -127
- package/data/knowledge/releases/v0.1.0/entries/technique-business-logic-race-conditions.yml +1 -1
- package/data/knowledge/releases/v0.1.0/release.yml +8 -1
- package/data/knowledge/releases/v0.6.0/entries/technique-async-callback-contracts.yml +98 -0
- package/data/knowledge/releases/v0.6.0/entries/technique-exactly-once-verification.yml +116 -0
- package/data/knowledge/releases/v0.6.0/release.yml +30 -0
- package/data/security/SECURITY-POLICY.md +273 -0
- package/data/targets/_default.yml +1 -1
- package/data/targets/_example-api-only.yml +80 -0
- package/data/targets/_example-mobile-emulation.yml +2 -2
- package/data/targets/_example-native-mobile.yml +2 -2
- package/data/targets/_example-sim-android-chrome.yml +1 -1
- package/data/targets/_example-sim-ios-safari.yml +1 -1
- package/data/templates/api-probe-matrix.md +3 -3
- package/data/templates/bug-report.md +29 -151
- package/data/templates/coverage-map.md +9 -6
- package/data/templates/expected-behaviour.md +3 -2
- package/data/templates/session-charter.md +3 -0
- package/data/templates/session-report.md +57 -80
- package/dist/cli/index.js +1452 -700
- package/dist/index.d.ts +589 -81
- package/dist/index.js +1612 -227
- package/docs/BACKEND-VERIFICATION.md +39 -9
- package/docs/GETTING-STARTED.md +188 -128
- package/docs/MOBILE-SETUP.md +149 -18
- package/package.json +32 -18
- package/skills/qa-explore/SKILL.md +31 -25
- package/skills/qa-explore/phases/00-setup.md +67 -41
- package/skills/qa-explore/phases/01-auth.md +32 -15
- package/skills/qa-explore/phases/02-charter.md +16 -5
- package/skills/qa-explore/phases/03-discovery.md +50 -26
- package/skills/qa-explore/phases/04-journeys.md +2 -2
- package/skills/qa-explore/phases/05-features.md +39 -24
- package/skills/qa-explore/phases/06-edge-cases.md +24 -13
- package/skills/qa-explore/phases/07-reporting.md +79 -140
- package/skills/qa-explore/references/context-integration.md +1 -1
- package/skills/qa-explore/references/output-contract.md +187 -0
- package/skills/qa-explore/references/paths.md +84 -0
- package/skills/qa-explore/references/playwright-agents-integration.md +61 -84
- package/skills/qa-explore/references/security-rules.md +66 -25
- package/skills/qa-explore/references/severity-guide.md +31 -4
- package/skills/qa-explore-cleanup/SKILL.md +17 -6
- package/skills/qa-explore-feedback/SKILL.md +18 -5
- package/skills/qa-explore-mobile/SKILL.md +86 -64
- package/skills/qa-explore-mobile/phases/00-setup.md +71 -62
- package/skills/qa-explore-mobile/phases/01-auth.md +28 -31
- package/skills/qa-explore-mobile/phases/02-charter.md +8 -4
- package/skills/qa-explore-mobile/phases/03-discovery.md +17 -11
- package/skills/qa-explore-mobile/phases/04-journeys.md +6 -6
- package/skills/qa-explore-mobile/phases/05-features.md +9 -9
- package/skills/qa-explore-mobile/phases/06-edge-cases.md +35 -40
- package/skills/qa-explore-mobile/phases/07-reporting.md +82 -56
- package/skills/qa-explore-mobile/references/mobile-edges.md +26 -26
- package/skills/qa-explore-quick/SKILL.md +60 -25
- package/skills/qa-explore-report/SKILL.md +40 -34
- package/skills/qa-gather/SKILL.md +17 -2
- package/skills/qa-knowledge-add/SKILL.md +20 -9
- package/skills/qa-knowledge-list/SKILL.md +12 -3
- package/skills/qa-target-setup/SKILL.md +63 -41
- package/skills/qa-verify-backend/SKILL.md +26 -17
- package/skills/qa-verify-backend/phases/00-setup.md +57 -12
- package/skills/qa-verify-backend/phases/01-ac-decomposition.md +2 -2
- package/skills/qa-verify-backend/phases/03-live-verification.md +3 -3
- package/skills/qa-verify-backend/phases/03b-api-verification.md +22 -9
- package/skills/qa-verify-backend/phases/04-e2e-trigger.md +18 -2
- package/skills/qa-verify-backend/phases/05-reporting.md +38 -10
- package/skills/qa-verify-backend/references/api-probes.md +28 -1
- package/skills/qa-verify-backend/references/payload-verification.md +1 -1
- package/skills/qa-verify-backend/references/release-readiness.md +1 -1
- package/skills/qa-verify-backend/references/safety-rules.md +13 -8
- package/data/domains/_default.md +0 -181
- package/data/domains/ecommerce.md +0 -192
- package/data/domains/fintech.md +0 -200
- package/data/domains/identity.md +0 -88
- package/data/domains/marketing.md +0 -194
- package/data/domains/saas.md +0 -197
- package/dist/cli/index.d.ts +0 -1
- package/dist/cli/index.js.map +0 -1
- package/dist/index.js.map +0 -1
package/data/domains/fintech.md
DELETED
|
@@ -1,200 +0,0 @@
|
|
|
1
|
-
# Domain: Fintech (Financial Technology)
|
|
2
|
-
|
|
3
|
-
## Priority Areas
|
|
4
|
-
|
|
5
|
-
1. **Calculation Accuracy** - Interest, fees, exchange rates, and balances must be exact
|
|
6
|
-
2. **Transaction Flows** - Transfers, payments, and settlements complete reliably
|
|
7
|
-
3. **Security** - Authentication, encryption, session management, fraud prevention
|
|
8
|
-
4. **Regulatory Compliance** - KYC, AML, PCI DSS, and jurisdiction-specific rules
|
|
9
|
-
5. **Audit Trails** - Every action logged with timestamp, user, and context
|
|
10
|
-
6. **Data Precision** - Decimal handling, rounding rules, currency formatting
|
|
11
|
-
7. **Error Handling** - Financial operations fail safely with clear recovery paths
|
|
12
|
-
|
|
13
|
-
## Feature Risk Ranking
|
|
14
|
-
|
|
15
|
-
| Tier | Features | Rationale |
|
|
16
|
-
|------|-------------------------------------------------------------------|------------------------------------------------------------------|
|
|
17
|
-
| P0 | Transfers, payments, balance calculations, authentication/security | Money movement and security are existential. A bug here = financial loss, fraud, or regulatory action. |
|
|
18
|
-
| P1 | Account management, transaction history, loan/credit features | Core daily workflows. Users depend on accurate history and account control. |
|
|
19
|
-
| P2 | Bill pay, statements, contact/profile info | Important but secondary. Failure is inconvenient, not catastrophic. |
|
|
20
|
-
| P3 | News/market updates, about page, general settings | Informational. Does not impact financial operations or security. |
|
|
21
|
-
|
|
22
|
-
## Completeness Checklist
|
|
23
|
-
|
|
24
|
-
> Minimum features a fintech application should have. Verify these exist and function before deep-testing.
|
|
25
|
-
|
|
26
|
-
- [ ] Transaction receipt generated for every completed transaction
|
|
27
|
-
- [ ] Unique transaction ID assigned and displayed for every transaction
|
|
28
|
-
- [ ] Audit trail present -- every financial action logged with timestamp, user, amount, and result
|
|
29
|
-
- [ ] Balance reconciliation -- displayed balance matches the sum of all transactions
|
|
30
|
-
- [ ] Two-factor authentication available and enforced for sensitive operations
|
|
31
|
-
- [ ] Session timeout configured (shorter than general web apps -- typically 5-15 minutes of inactivity)
|
|
32
|
-
- [ ] Account lockout after excessive failed login or PIN attempts
|
|
33
|
-
- [ ] PII masking -- account numbers, SSN, and other sensitive data partially masked in UI
|
|
34
|
-
- [ ] Transaction history with search, filter, and date range
|
|
35
|
-
- [ ] Statement generation (monthly or custom date range)
|
|
36
|
-
|
|
37
|
-
## Data Integrity Checks
|
|
38
|
-
|
|
39
|
-
> Financial data integrity is non-negotiable. After every state-changing action, verify these to the penny.
|
|
40
|
-
|
|
41
|
-
1. **Source account debited = destination account credited** -- For every transfer, the exact amount debited from the source must equal the exact amount credited to the destination (accounting for any documented fees). Check both accounts. No rounding differences.
|
|
42
|
-
2. **Transaction appears in BOTH accounts' history** -- After a transfer between two accounts, the transaction must appear in the history of both the sending and receiving account with correct amounts, timestamps, and reference IDs.
|
|
43
|
-
3. **Running balance matches after every transaction** -- After any transaction (deposit, withdrawal, transfer, fee, interest), recalculate: previous balance +/- transaction amount = new balance. This must hold for every single transaction in the history.
|
|
44
|
-
4. **Loan terms calculate correctly** -- Monthly payment, total interest, and total cost should match the amortization schedule. Verify with an independent calculator for at least one scenario.
|
|
45
|
-
5. **Fees applied correctly** -- Any fees (transfer fees, maintenance fees, overdraft fees) should match the published fee schedule and appear as distinct line items in the transaction history.
|
|
46
|
-
6. **Currency conversions are traceable** -- For cross-currency transactions, the exchange rate used, the source amount, the destination amount, and any conversion fee should all be visible and mathematically consistent.
|
|
47
|
-
|
|
48
|
-
## Cross-Feature Journeys
|
|
49
|
-
|
|
50
|
-
> End-to-end flows that cross feature boundaries. Financial bugs at the seams can cause real monetary loss.
|
|
51
|
-
|
|
52
|
-
### Journey 1: Full Banking Workflow
|
|
53
|
-
`Register -> Login -> Open Account -> Transfer Funds -> Verify Both Balances -> Pay Bill -> Check Transaction History`
|
|
54
|
-
- Verify: registration completes identity verification, account shows correct opening balance, transfer debits and credits exact amounts, both account balances are correct, bill payment appears in history with correct payee and amount.
|
|
55
|
-
|
|
56
|
-
### Journey 2: Transaction Verification and Export
|
|
57
|
-
`Login -> Transfer Funds -> View Transaction History -> Verify Amounts Match -> Export Statement -> Verify Statement Matches History`
|
|
58
|
-
- Verify: transfer completes with confirmation and receipt, transaction history shows the transfer with correct details, amounts in history match the transfer confirmation, exported statement contains the same data as the on-screen history.
|
|
59
|
-
|
|
60
|
-
### Journey 3: Security and Session Management
|
|
61
|
-
`Login -> Start Transfer -> Session Times Out -> Re-authenticate -> Complete Transfer -> Verify No Duplicate`
|
|
62
|
-
- Verify: session timeout interrupts the flow gracefully, re-authentication does not lose the in-progress transfer (or clearly notifies the user), no duplicate transaction is created, final balance is correct.
|
|
63
|
-
|
|
64
|
-
## Must-Test Patterns
|
|
65
|
-
|
|
66
|
-
### Calculations
|
|
67
|
-
- Interest calculation accuracy (simple, compound, daily accrual)
|
|
68
|
-
- Fee computation (flat, percentage, tiered, capped)
|
|
69
|
-
- Exchange rate conversion with correct rounding
|
|
70
|
-
- Loan amortization schedule accuracy
|
|
71
|
-
- APR and APY display correctness
|
|
72
|
-
- Tax withholding calculations
|
|
73
|
-
- Dividend and yield computations
|
|
74
|
-
- Split transactions and partial payments
|
|
75
|
-
- Rounding rules match regulatory requirements (banker's rounding)
|
|
76
|
-
- Calculations with very large amounts (millions, billions)
|
|
77
|
-
- Calculations with very small amounts (fractions of a cent)
|
|
78
|
-
- Multi-currency conversions with intermediary currency
|
|
79
|
-
|
|
80
|
-
### Transaction Flows
|
|
81
|
-
- Successful money transfer between accounts
|
|
82
|
-
- Transfer with insufficient funds (rejected gracefully)
|
|
83
|
-
- Transfer at exact balance amount (edge case)
|
|
84
|
-
- Scheduled and recurring transaction execution
|
|
85
|
-
- Transaction cancellation before processing
|
|
86
|
-
- Transaction reversal after completion
|
|
87
|
-
- Pending transaction status and timeline
|
|
88
|
-
- Failed transaction with automatic retry
|
|
89
|
-
- Duplicate transaction prevention (idempotency)
|
|
90
|
-
- Concurrent transactions on same account (race conditions)
|
|
91
|
-
- Cross-border transactions with currency conversion
|
|
92
|
-
- Transaction timeout handling (what happens to funds)
|
|
93
|
-
- Batch transaction processing
|
|
94
|
-
- Transaction receipt and confirmation generation
|
|
95
|
-
- Real-time balance update after transaction
|
|
96
|
-
|
|
97
|
-
### Account Management
|
|
98
|
-
- Account creation with identity verification
|
|
99
|
-
- Multiple account types (checking, savings, investment)
|
|
100
|
-
- Account linking (external bank connections)
|
|
101
|
-
- Account statements (monthly, custom date range)
|
|
102
|
-
- Account closure process and final balance handling
|
|
103
|
-
- Joint account management
|
|
104
|
-
- Beneficiary management (add, edit, remove)
|
|
105
|
-
- Account freeze and unfreeze
|
|
106
|
-
- Balance inquiry accuracy (available vs ledger balance)
|
|
107
|
-
- Account limits (daily, monthly, per-transaction)
|
|
108
|
-
|
|
109
|
-
### Security
|
|
110
|
-
- Two-factor authentication setup (TOTP, SMS, hardware key)
|
|
111
|
-
- 2FA required for high-value transactions
|
|
112
|
-
- Session timeout appropriate for financial context (shorter than general apps)
|
|
113
|
-
- Session invalidation on password change
|
|
114
|
-
- Device fingerprinting and trusted device management
|
|
115
|
-
- IP-based anomaly detection alerts
|
|
116
|
-
- Brute force protection on login and PIN entry
|
|
117
|
-
- Encryption of sensitive data at rest and in transit
|
|
118
|
-
- Secure password requirements (length, complexity)
|
|
119
|
-
- Biometric authentication (fingerprint, face ID) on mobile
|
|
120
|
-
- PIN entry for transaction confirmation
|
|
121
|
-
- Security questions as backup recovery method
|
|
122
|
-
- Account lockout after excessive failed attempts
|
|
123
|
-
- Unlock procedure (identity reverification)
|
|
124
|
-
- Phishing detection warnings on suspicious links
|
|
125
|
-
|
|
126
|
-
### Regulatory Compliance
|
|
127
|
-
- KYC (Know Your Customer) identity verification flow
|
|
128
|
-
- Document upload for ID verification (passport, driver license)
|
|
129
|
-
- Address verification (utility bill, bank statement)
|
|
130
|
-
- AML (Anti-Money Laundering) transaction monitoring
|
|
131
|
-
- Suspicious Activity Report (SAR) generation triggers
|
|
132
|
-
- OFAC sanctions list screening
|
|
133
|
-
- Transaction reporting thresholds (CTR for $10,000+)
|
|
134
|
-
- Consumer disclosure documents (Truth in Lending, fee schedules)
|
|
135
|
-
- Regulation E compliance (electronic fund transfer rights)
|
|
136
|
-
- PSD2 Strong Customer Authentication (EU)
|
|
137
|
-
- Open Banking API compliance (if applicable)
|
|
138
|
-
|
|
139
|
-
### Audit Trails
|
|
140
|
-
- Login and logout events logged
|
|
141
|
-
- Every transaction logged with full details
|
|
142
|
-
- Permission changes logged
|
|
143
|
-
- Account modification events logged
|
|
144
|
-
- Failed authentication attempts logged
|
|
145
|
-
- Export of audit log for compliance review
|
|
146
|
-
- Audit log tamper protection (immutable entries)
|
|
147
|
-
- Audit log retention meets regulatory minimums
|
|
148
|
-
- Audit log search and filter by date, user, action type
|
|
149
|
-
- Administrative actions logged separately
|
|
150
|
-
|
|
151
|
-
### Data Precision
|
|
152
|
-
- Currency amounts stored with correct decimal places (2 for USD, 0 for JPY, 8 for crypto)
|
|
153
|
-
- No floating-point arithmetic for currency (use decimal or integer cents)
|
|
154
|
-
- Rounding direction consistent (round half even for financial)
|
|
155
|
-
- Negative amounts handled correctly (debits, fees, reversals)
|
|
156
|
-
- Currency formatting matches locale (symbol, separator, decimal)
|
|
157
|
-
- Percentage displays with appropriate precision
|
|
158
|
-
- Large number formatting (thousand separators)
|
|
159
|
-
- Zero amount transactions handled correctly
|
|
160
|
-
- Display precision matches calculation precision (no hidden rounding)
|
|
161
|
-
|
|
162
|
-
### Error Handling for Financial Operations
|
|
163
|
-
- Insufficient funds returns clear error with current balance
|
|
164
|
-
- Daily limit exceeded shows limit and reset time
|
|
165
|
-
- Invalid recipient account returns specific error
|
|
166
|
-
- Network failure during transaction shows clear status (not "unknown")
|
|
167
|
-
- Timeout during payment does not result in lost funds
|
|
168
|
-
- Partial failure in batch operations identifies which items failed
|
|
169
|
-
- Concurrent modification conflict resolution
|
|
170
|
-
- Expired session during transaction saves draft or clearly notifies
|
|
171
|
-
- Payment gateway errors mapped to user-friendly messages
|
|
172
|
-
- Recovery flow after failed transaction (retry, contact support)
|
|
173
|
-
|
|
174
|
-
## Common Bugs
|
|
175
|
-
|
|
176
|
-
- **Floating-point arithmetic errors** - Using float instead of decimal for currency results in rounding drift, totals off by 0.01 after many operations, percentage fee calculates to incorrect amount
|
|
177
|
-
- **Race conditions on balance** - Two concurrent withdrawals both succeed when only one should, balance goes negative briefly before correction, pending transactions not considered in available balance
|
|
178
|
-
- **Transaction state inconsistencies** - Sender debited but receiver not credited (half-completed transfer), transaction status shows "completed" while still pending at bank, cancelled transaction still processes
|
|
179
|
-
- **Rounding discrepancy** - Statement total does not match sum of individual transactions, currency conversion rounds differently on different screens, tax calculation rounding differs from display
|
|
180
|
-
- **Security session issues** - Session not invalidated on 2FA change, token replay attack possible, device trust persists after account compromise, PIN attempt counter resets on app restart
|
|
181
|
-
- **Timezone-related bugs** - Interest accrual calculated wrong across timezone boundaries, scheduled payment executes on wrong day, statement date range off by one day
|
|
182
|
-
- **Currency display errors** - Negative amounts shown without minus sign, wrong currency symbol after locale change, thousand separator used as decimal in some locales
|
|
183
|
-
- **Compliance gaps** - KYC check can be bypassed by direct API call, transaction below threshold split into multiple to avoid reporting, audit log missing entries for bulk operations
|
|
184
|
-
- **Notification failures** - Large transaction alert not sent, fraud alert delayed, duplicate notifications on retry
|
|
185
|
-
|
|
186
|
-
## Compliance Requirements
|
|
187
|
-
|
|
188
|
-
- **PCI DSS** - Payment card data encryption, no card numbers in logs, network segmentation, quarterly vulnerability scans
|
|
189
|
-
- **SOX (Sarbanes-Oxley)** - Financial reporting accuracy, internal controls, audit trail integrity
|
|
190
|
-
- **KYC / AML** - Customer identification program, ongoing monitoring, suspicious activity reporting
|
|
191
|
-
- **GLBA (Gramm-Leach-Bliley)** - Financial privacy, safeguards rule, pretexting protection
|
|
192
|
-
- **Regulation E** - Electronic fund transfer error resolution, unauthorized transaction liability limits
|
|
193
|
-
- **PSD2 (EU)** - Strong Customer Authentication, open banking API access, transaction risk analysis
|
|
194
|
-
- **GDPR** - Data minimization, right to erasure (balanced with record retention requirements), consent management
|
|
195
|
-
- **CCPA** - Consumer data access and deletion rights
|
|
196
|
-
- **BSA (Bank Secrecy Act)** - Currency Transaction Reports, recordkeeping requirements
|
|
197
|
-
- **OFAC Sanctions** - Screening all parties against sanctions lists before transactions
|
|
198
|
-
- **SOC 2 Type II** - Security, availability, and confidentiality controls audited
|
|
199
|
-
- **Accessibility (WCAG 2.1 AA)** - All financial workflows accessible, including transaction flows and statements
|
|
200
|
-
- **Data Residency** - Financial data stored in jurisdictions meeting regulatory requirements
|
package/data/domains/identity.md
DELETED
|
@@ -1,88 +0,0 @@
|
|
|
1
|
-
# Domain: Identity & Access Platforms
|
|
2
|
-
|
|
3
|
-
Covers identity and account platforms **and** the internal admin consoles that
|
|
4
|
-
configure them (tenant/scope configuration, consent definitions, audit history).
|
|
5
|
-
|
|
6
|
-
## Priority Areas
|
|
7
|
-
|
|
8
|
-
1. **Audit Trail Integrity** - every change attributable, complete, ordered, immutable
|
|
9
|
-
2. **Identity Propagation** - the real human reaches the persisted record
|
|
10
|
-
3. **Consent & Privacy** - capture, versioning, data minimisation
|
|
11
|
-
4. **Configuration Correctness** - tenant isolation, validation, promotion
|
|
12
|
-
5. **Admin RBAC** - write gating enforced at the API, not just the UI
|
|
13
|
-
6. **Log & Data Hygiene** - no personal data leaking into logs or downstream platforms
|
|
14
|
-
|
|
15
|
-
## Feature Risk Ranking
|
|
16
|
-
|
|
17
|
-
| Tier | Features | Rationale |
|
|
18
|
-
|------|----------|-----------|
|
|
19
|
-
| P0 | Authentication, consent & privacy, audit trail integrity, tenant configuration, personal data handling | A gap here is a regulatory and incident-response failure, not a UX annoyance. |
|
|
20
|
-
| P1 | Registration journeys, account recovery, admin RBAC, config promotion | Core operator and end-user workflows; failure blocks the business. |
|
|
21
|
-
| P2 | Config search & filtering, notifications, reporting views | Supporting features; failure is friction. |
|
|
22
|
-
| P3 | Static help content, about/version pages | Informational. |
|
|
23
|
-
|
|
24
|
-
## Completeness Checklist
|
|
25
|
-
|
|
26
|
-
- [ ] Every state-changing action on identity or configuration data produces an audit record (who, what, when)
|
|
27
|
-
- [ ] Audit records are immutable — no update or delete path exists for the actor who wrote them
|
|
28
|
-
- [ ] Actor identity on an audit record is the authenticated user, never a service account or `unknown`
|
|
29
|
-
- [ ] Consent capture records purpose, version, timestamp and locale for every tenant
|
|
30
|
-
- [ ] Personal data is never written to application logs, traces or error payloads
|
|
31
|
-
- [ ] Admin writes are gated by role — a read-only operator cannot save configuration
|
|
32
|
-
- [ ] Configuration changes are scoped per tenant
|
|
33
|
-
- [ ] Config promotion between environments preserves the audit chain and does not rewrite authorship
|
|
34
|
-
- [ ] Deletion of a configuration entity is auditable and reversible via the audit trail
|
|
35
|
-
- [ ] Every environment has the same audit guarantees — ephemeral envs are not a special case
|
|
36
|
-
|
|
37
|
-
## Data Integrity Checks
|
|
38
|
-
|
|
39
|
-
> After every state-changing action in an identity system, verify these.
|
|
40
|
-
|
|
41
|
-
1. **Audit entry exists per change** — exactly one record per modified item. Not zero, not duplicated.
|
|
42
|
-
2. **Actor is correct** — equals the logged-in user's identity; not `unknown`, not the previous editor, not a service role.
|
|
43
|
-
3. **Timestamp ordering holds** — two rapid changes produce two distinct, ordered records; neither overwrites the other.
|
|
44
|
-
4. **Snapshot fidelity** — the stored post-change snapshot matches the config store field for field.
|
|
45
|
-
5. **Delete attribution** — a deletion names the deleter, not the last editor.
|
|
46
|
-
6. **Tenant isolation** — changing tenant A produces records only for tenant A.
|
|
47
|
-
7. **No silent loss** — a downstream failure surfaces (error, DLQ, alarm); it never disappears behind a success response.
|
|
48
|
-
8. **Personal data containment** — audit record, log stream and error path carry no personal data beyond what the design authorises.
|
|
49
|
-
|
|
50
|
-
## Cross-Feature Journeys
|
|
51
|
-
|
|
52
|
-
### Journey 1: Config Change to Audit Trail
|
|
53
|
-
`Login → Open tenant config → Change one field → Save → Verify config store → Verify audit record → Verify actor/timestamp/snapshot/version`
|
|
54
|
-
- Verify: attributable, chronologically sortable, matches the config store exactly.
|
|
55
|
-
|
|
56
|
-
### Journey 2: Rapid Successive Edits
|
|
57
|
-
`Open config → Save change 1 → Immediately save change 2 → Query audit store`
|
|
58
|
-
- Verify: **two** distinct records with distinct sort keys. Same-second collisions are a classic audit-loss bug.
|
|
59
|
-
|
|
60
|
-
### Journey 3: Delete and Attribute
|
|
61
|
-
`Login as A → Edit and save → Logout → Login as B → Delete → Query audit store`
|
|
62
|
-
- Verify: the deletion record names **B**.
|
|
63
|
-
|
|
64
|
-
### Journey 4: Degraded Identity
|
|
65
|
-
`Trigger a write where the token carries no identity claim`
|
|
66
|
-
- Verify: rejected, or recorded with an explicit flag and an alert. `unknown` actors must never be silently accepted.
|
|
67
|
-
|
|
68
|
-
## Testing Guidance
|
|
69
|
-
|
|
70
|
-
The strongest bugs in this domain are **absences**: the audit record that was never
|
|
71
|
-
written, the permission that was never checked, the cleanup that was never run.
|
|
72
|
-
|
|
73
|
-
Never accept "the happy path wrote a record" as evidence the audit trail works.
|
|
74
|
-
Test the failure path, the same-second path, the delete path, and the
|
|
75
|
-
missing-identity path — that is where audit trails actually break.
|
|
76
|
-
|
|
77
|
-
**Heuristic focus:**
|
|
78
|
-
- **SFDIPOT** — Data (stored vs should-be), Operations (who may write), Time (ordering, collisions, skew)
|
|
79
|
-
- **FEW HICCUPPS** — Claims (does it do what the ticket claims?), Standards (data protection law, least privilege), Product (does the producer match the consumer's contract?)
|
|
80
|
-
|
|
81
|
-
## Compliance
|
|
82
|
-
|
|
83
|
-
- **Data protection law** — lawful basis, consent records, right to erasure, data minimisation on audit payloads
|
|
84
|
-
- **Audit Trail Integrity** — attributable, tamper-evident, retained for the stated period
|
|
85
|
-
- **Least Privilege** — producers hold write-only; readers are separate principals
|
|
86
|
-
- **Encryption** — at rest with managed keys, in transit end to end
|
|
87
|
-
- **Log Hygiene** — no personal data or credentials in logs; production log levels constrained
|
|
88
|
-
- **Separation of Environments** — ephemeral/dev data never mixed with production
|
|
@@ -1,194 +0,0 @@
|
|
|
1
|
-
# Domain: Marketing Website
|
|
2
|
-
|
|
3
|
-
## Priority Areas
|
|
4
|
-
|
|
5
|
-
1. **Landing Pages & CTAs** - Conversion-critical elements must work flawlessly
|
|
6
|
-
2. **Forms** - Contact, signup, and newsletter forms capture and submit correctly
|
|
7
|
-
3. **SEO** - Meta tags, structured data, canonical URLs, crawlability
|
|
8
|
-
4. **Analytics & Tracking** - Events fire correctly, attribution works
|
|
9
|
-
5. **Mobile Responsiveness** - Layout and interactions work on all devices
|
|
10
|
-
6. **Page Load Speed** - Fast load times directly impact conversion rates
|
|
11
|
-
7. **Content Quality** - No broken links, typos, or placeholder content
|
|
12
|
-
|
|
13
|
-
## Feature Risk Ranking
|
|
14
|
-
|
|
15
|
-
| Tier | Features | Rationale |
|
|
16
|
-
|------|---------------------------------------------------|------------------------------------------------------------------|
|
|
17
|
-
| P0 | Forms/lead capture, CTAs, page load speed | Direct revenue pipeline. Broken forms = lost leads. Broken CTAs = lost conversions. Slow pages = high bounce rate. |
|
|
18
|
-
| P1 | SEO, analytics tracking, responsive design | Discoverability and measurement. Without SEO, no traffic. Without analytics, no insight. Without responsive, lost mobile users. |
|
|
19
|
-
| P2 | Content pages, images/media, social sharing | Supports brand and engagement but does not directly block conversion. |
|
|
20
|
-
| P3 | Blog, careers page, press/media section | Informational. Rarely impacts lead generation or conversion. |
|
|
21
|
-
|
|
22
|
-
## Completeness Checklist
|
|
23
|
-
|
|
24
|
-
> Minimum features a marketing website should have. Verify these exist and function before deep-testing.
|
|
25
|
-
|
|
26
|
-
- [ ] All CTAs lead to a valid destination (no dead links, no 404s, no "#" hrefs)
|
|
27
|
-
- [ ] All forms submit successfully and display a confirmation message or redirect to a thank-you page
|
|
28
|
-
- [ ] Meta tags present on every page (title, description, og:title, og:description, og:image)
|
|
29
|
-
- [ ] Structured data valid (JSON-LD for Organization, BreadcrumbList, FAQ, etc.)
|
|
30
|
-
- [ ] Cookie consent banner present and functional (blocks non-essential cookies until accepted)
|
|
31
|
-
- [ ] Privacy policy page linked from footer and accessible
|
|
32
|
-
- [ ] Mobile responsive -- layout does not break at 320px, 375px, or 768px
|
|
33
|
-
- [ ] No placeholder content (lorem ipsum, "TODO", sample images)
|
|
34
|
-
- [ ] Favicon present
|
|
35
|
-
- [ ] 404 page is custom and helpful (not a raw server error)
|
|
36
|
-
|
|
37
|
-
## Data Integrity Checks
|
|
38
|
-
|
|
39
|
-
> Marketing sites are lighter on data, but lead capture and tracking integrity are critical.
|
|
40
|
-
|
|
41
|
-
1. **Form submission arrives in CRM/inbox** -- Submit every form with test data and verify the submission appears in the destination system (CRM, email inbox, marketing automation platform). Check all fields map correctly.
|
|
42
|
-
2. **Analytics events fire correctly** -- Open browser dev tools or a tag debugger. Verify page view events fire on load, CTA click events fire on click (with correct labels), and form submission events fire on successful submit. No duplicate events.
|
|
43
|
-
3. **UTM parameters preserved through redirects** -- Navigate to the site with UTM parameters (e.g., `?utm_source=test&utm_medium=cpc&utm_campaign=spring`). Click through CTAs and form submissions. Verify UTM values are captured in analytics and/or CRM fields. Check that redirects (301, 302) do not strip the parameters.
|
|
44
|
-
4. **Cookie consent respected** -- Before accepting cookies, non-essential scripts (analytics, ad pixels) should not load. After accepting, they should load. After rejecting, they should remain blocked.
|
|
45
|
-
|
|
46
|
-
## Cross-Feature Journeys
|
|
47
|
-
|
|
48
|
-
> End-to-end flows that cross feature boundaries. Conversion path bugs are costly.
|
|
49
|
-
|
|
50
|
-
### Journey 1: Landing Page to Lead Capture
|
|
51
|
-
`Land on Page (with UTM params) -> Scroll to CTA -> Click CTA -> Fill Form -> Submit -> See Thank-You Page -> Verify Lead Captured in CRM`
|
|
52
|
-
- Verify: UTM parameters preserved, CTA links to correct form, form submits without error, thank-you page loads, lead appears in CRM with correct source/campaign data.
|
|
53
|
-
|
|
54
|
-
### Journey 2: Content Discovery to Social Sharing
|
|
55
|
-
`Homepage -> Navigate to Blog -> Read Article -> Click Share Button -> Verify OG Tags in Share Preview`
|
|
56
|
-
- Verify: blog navigation works, article loads fully, share buttons function, Open Graph tags produce correct title, description, and image in the share preview. Check og:image dimensions meet platform minimums (1200x630).
|
|
57
|
-
|
|
58
|
-
### Journey 3: Organic Search Entry
|
|
59
|
-
`Enter via Search (simulate with direct URL + UTM) -> Browse Multiple Pages -> Click CTA -> Convert`
|
|
60
|
-
- Verify: page has correct canonical URL, meta tags match page content, internal links work, CTA conversion tracked as organic in analytics.
|
|
61
|
-
|
|
62
|
-
## Must-Test Patterns
|
|
63
|
-
|
|
64
|
-
### Landing Pages
|
|
65
|
-
- Hero section renders correctly with image, headline, subheading
|
|
66
|
-
- Primary CTA button is visible above the fold
|
|
67
|
-
- CTA links to correct destination (signup, demo, pricing)
|
|
68
|
-
- Multiple CTAs on the page all function correctly
|
|
69
|
-
- Social proof elements display (testimonials, logos, stats)
|
|
70
|
-
- Video embeds play without error
|
|
71
|
-
- Responsive layout does not break the visual hierarchy
|
|
72
|
-
- A/B test variants load correctly when configured
|
|
73
|
-
- UTM parameters preserved through CTA clicks
|
|
74
|
-
- Page loads correctly when accessed from paid ad link
|
|
75
|
-
|
|
76
|
-
### Forms
|
|
77
|
-
- Contact form submits successfully with valid data
|
|
78
|
-
- Required field validation on all mandatory fields
|
|
79
|
-
- Email format validation
|
|
80
|
-
- Phone number format handling (international formats)
|
|
81
|
-
- Form submission sends confirmation email
|
|
82
|
-
- Form data reaches CRM or email marketing system
|
|
83
|
-
- Honeypot or reCAPTCHA spam prevention works
|
|
84
|
-
- Form does not submit with empty honeypot field triggered
|
|
85
|
-
- Newsletter signup with double opt-in flow
|
|
86
|
-
- Unsubscribe link works in marketing emails
|
|
87
|
-
- Form error messages are clear and positioned near the field
|
|
88
|
-
- Form submit button shows loading state and prevents double click
|
|
89
|
-
|
|
90
|
-
### SEO
|
|
91
|
-
- Unique title tag on every page (under 60 characters)
|
|
92
|
-
- Meta description present and relevant (under 160 characters)
|
|
93
|
-
- Only one h1 per page
|
|
94
|
-
- Heading hierarchy is logical (h1 > h2 > h3, no skips)
|
|
95
|
-
- Canonical URL set on all pages
|
|
96
|
-
- Canonical URL is self-referencing or points to correct primary page
|
|
97
|
-
- Hreflang tags for multi-language sites
|
|
98
|
-
- Structured data (JSON-LD) for organization, breadcrumbs, FAQ, articles
|
|
99
|
-
- Structured data validates in Google Rich Results Test
|
|
100
|
-
- XML sitemap includes all indexable pages
|
|
101
|
-
- XML sitemap excludes noindex pages
|
|
102
|
-
- Robots.txt does not block critical resources
|
|
103
|
-
- Image alt text present and descriptive
|
|
104
|
-
- Clean URL slugs (no query parameters for main content)
|
|
105
|
-
- 301 redirects in place for moved or renamed pages
|
|
106
|
-
- No broken internal links (404 errors)
|
|
107
|
-
- Open Graph tags (og:title, og:description, og:image) present
|
|
108
|
-
- Twitter Card tags present
|
|
109
|
-
- og:image dimensions meet minimum (1200x630)
|
|
110
|
-
|
|
111
|
-
### Analytics & Tracking
|
|
112
|
-
- Google Analytics (GA4) or equivalent loads on all pages
|
|
113
|
-
- Page view events fire on every page load
|
|
114
|
-
- CTA click events tracked with correct labels
|
|
115
|
-
- Form submission events fire on successful submit
|
|
116
|
-
- Scroll depth tracking active
|
|
117
|
-
- UTM parameters captured in analytics
|
|
118
|
-
- Conversion goals or events configured for key actions
|
|
119
|
-
- Cross-domain tracking works if multi-domain setup
|
|
120
|
-
- Cookie consent does not block analytics before acceptance
|
|
121
|
-
- Analytics data matches actual behavior (no duplicate events)
|
|
122
|
-
- Facebook Pixel, LinkedIn Insight, or other ad pixels fire correctly
|
|
123
|
-
- Event parameters contain correct values (page name, CTA label)
|
|
124
|
-
|
|
125
|
-
### A/B Testing
|
|
126
|
-
- Control and variant pages render correctly
|
|
127
|
-
- Traffic split is approximately correct
|
|
128
|
-
- Variant does not flash original before changing (no flicker)
|
|
129
|
-
- A/B test tool does not significantly impact page load time
|
|
130
|
-
- Conversion tracking attributes to correct variant
|
|
131
|
-
- Test preview mode works for QA
|
|
132
|
-
|
|
133
|
-
### Social Sharing
|
|
134
|
-
- Share buttons for major platforms (Facebook, Twitter, LinkedIn, email)
|
|
135
|
-
- Share URL includes correct page link
|
|
136
|
-
- Shared content preview shows correct title, description, image
|
|
137
|
-
- Share count displays (if applicable)
|
|
138
|
-
- Copy link button works and shows confirmation
|
|
139
|
-
|
|
140
|
-
### Mobile Responsiveness
|
|
141
|
-
- Layout at 320px (small phone), 375px (standard phone), 768px (tablet)
|
|
142
|
-
- Navigation hamburger menu opens and closes
|
|
143
|
-
- Touch targets at least 44x44px
|
|
144
|
-
- No horizontal scrolling
|
|
145
|
-
- Images scale without overflow or distortion
|
|
146
|
-
- Text is readable without zooming (minimum 16px body text)
|
|
147
|
-
- Fixed headers do not cover content
|
|
148
|
-
- Phone numbers are clickable (tel: links)
|
|
149
|
-
- Maps and embeds resize correctly
|
|
150
|
-
- Popup and modal overlays work on mobile
|
|
151
|
-
|
|
152
|
-
### Page Load Speed
|
|
153
|
-
- Largest Contentful Paint (LCP) under 2.5 seconds
|
|
154
|
-
- First Input Delay (FID) under 100ms
|
|
155
|
-
- Cumulative Layout Shift (CLS) under 0.1
|
|
156
|
-
- Images use modern formats (WebP, AVIF) with fallbacks
|
|
157
|
-
- CSS and JS are minified
|
|
158
|
-
- No render-blocking resources above the fold
|
|
159
|
-
- Fonts load without causing layout shift (font-display: swap)
|
|
160
|
-
- Third-party scripts do not block main thread significantly
|
|
161
|
-
- Lazy loading for images below the fold
|
|
162
|
-
- CDN caching configured for static assets
|
|
163
|
-
|
|
164
|
-
### Content Quality
|
|
165
|
-
- No lorem ipsum or placeholder text
|
|
166
|
-
- No broken images (missing files or wrong paths)
|
|
167
|
-
- No typos or grammatical errors in headlines and CTAs
|
|
168
|
-
- Legal pages accessible (Privacy Policy, Terms of Service, Cookie Policy)
|
|
169
|
-
- Copyright year is current
|
|
170
|
-
- External links open in new tab and have rel="noopener"
|
|
171
|
-
- Downloads work (PDF brochures, whitepapers)
|
|
172
|
-
- Pricing page is accurate and up to date
|
|
173
|
-
- Team photos and bios are current
|
|
174
|
-
|
|
175
|
-
## Common Bugs
|
|
176
|
-
|
|
177
|
-
- **CTA link errors** - Primary CTA links to wrong page or staging URL, UTM parameters stripped on redirect, anchor link targets missing element
|
|
178
|
-
- **Form delivery failures** - Form submits with success message but data never reaches inbox or CRM, email integration token expired silently, spam filter catches legitimate submissions
|
|
179
|
-
- **SEO regressions** - New deploy removes meta tags, sitemap not updated after adding pages, canonical URL points to staging, noindex left on from staging environment
|
|
180
|
-
- **Analytics gaps** - Cookie consent blocks all tracking including essential analytics, duplicate event firing on single-page app navigation, ad blocker breaks conversion tracking
|
|
181
|
-
- **Mobile layout breaks** - Hero image pushes CTA below fold on mobile, navigation menu overlaps content, horizontal scroll caused by uncontained element, video embed breaks layout
|
|
182
|
-
- **Performance regressions** - New hero video slows LCP, unoptimized image added to homepage, third-party chat widget blocks interactivity, font loading causes visible text shift
|
|
183
|
-
- **Social sharing preview** - og:image URL broken, description shows default text instead of page-specific, Twitter Card validator fails
|
|
184
|
-
- **A/B test interference** - Test JavaScript causes page flicker, wrong variant shown to returning users, test not stopped after reaching significance
|
|
185
|
-
|
|
186
|
-
## Compliance Requirements
|
|
187
|
-
|
|
188
|
-
- **GDPR** - Cookie consent banner before non-essential cookies, privacy policy link, data processing disclosure
|
|
189
|
-
- **CCPA** - "Do Not Sell My Personal Information" link for California visitors
|
|
190
|
-
- **CAN-SPAM** - Unsubscribe link in all marketing emails, physical address in email footer
|
|
191
|
-
- **ADA / WCAG 2.1 AA** - Accessible forms, sufficient color contrast, keyboard navigable
|
|
192
|
-
- **ePrivacy Directive** - Cookie consent specific to EU visitors
|
|
193
|
-
- **FTC Guidelines** - Truthful advertising, clear disclosure of endorsements and sponsorships
|
|
194
|
-
- **COPPA** - Age verification if content could attract children under 13
|