@ryuenn3123/agentic-senior-core 6.15.1 → 6.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/plugins/agentic-senior-core/.codex-plugin/plugin.json +1 -1
- package/.agents/plugins/agentic-senior-core/commands/asc-reference.md +1 -1
- package/.agents/plugins/agentic-senior-core/commands/asc-reference.toml +2 -3
- package/.agents/plugins/agentic-senior-core/commands/asc-test.md +2 -0
- package/.agents/plugins/agentic-senior-core/commands/asc-test.toml +3 -0
- package/.agents/plugins/agentic-senior-core/hooks/lib/known-ui-slop-patterns.json +81 -1
- package/.agents/plugins/agentic-senior-core/plugin.json +1 -1
- package/.agents/plugins/agentic-senior-core/rules/agentic-senior-core.md +3 -2
- package/.agents/plugins/agentic-senior-core/skills/asc/SKILL.md +15 -6
- package/.agents/plugins/agentic-senior-core/skills/asc/references/writing-skills.md +72 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-develop/SKILL.md +3 -4
- package/.agents/plugins/agentic-senior-core/skills/asc-reference/SKILL.md +55 -67
- package/.agents/plugins/agentic-senior-core/skills/asc-reference/references/api.md +108 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-reference/references/database.md +97 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-reference/references/infrastructure.md +102 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-reference/references/resilience.md +87 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-review/SKILL.md +20 -7
- package/.agents/plugins/agentic-senior-core/skills/asc-test/SKILL.md +104 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-test/references/e2e-playwright.md +89 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-test/references/flaky-tests.md +96 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-test/references/regression-proof.md +58 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-test/references/tdd-cycle.md +62 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-test/references/test-design.md +75 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-test/references/uat.md +62 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/SKILL.md +61 -174
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/accessibility.md +102 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/copy-i18n.md +103 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/craft/browser-surfaces.md +69 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/craft/color-surfaces.md +46 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/craft/motion.md +55 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/craft/typography.md +49 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/divergence.md +127 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/mobile-responsive.md +115 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/modes/experience.md +27 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/modes/operate.md +53 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/modes/persuade.md +72 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/modes/read.md +43 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/redesign-audit.md +71 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/reflexes.md +59 -0
- package/.agents/plugins/agentic-senior-core/skills/asc-ui/references/states-feedback.md +148 -0
- package/README.md +1 -1
- package/gemini-extension.json +1 -1
- package/package.json +1 -1
- package/plugin.yaml +3 -1
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
# Infrastructure, Containers & CI/CD Engineering
|
|
2
|
+
|
|
3
|
+
Robust infrastructure isolates workloads, prevents credential leakage, scales predictably, and fails gracefully.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. Container Architecture & Docker Hygiene
|
|
8
|
+
|
|
9
|
+
Containers must be minimal, stateless, secure, and reproducible.
|
|
10
|
+
|
|
11
|
+
### 1.1 Multi-Stage Build Pattern
|
|
12
|
+
Separate compilation dependencies from runtime artifacts to produce lightweight, hardened production images:
|
|
13
|
+
|
|
14
|
+
```dockerfile
|
|
15
|
+
# Stage 1: Build & Dependencies
|
|
16
|
+
FROM node:20-alpine AS builder
|
|
17
|
+
WORKDIR /app
|
|
18
|
+
COPY package*.json ./
|
|
19
|
+
RUN npm ci --ignore-scripts
|
|
20
|
+
COPY . .
|
|
21
|
+
RUN npm run build && npm prune --production
|
|
22
|
+
|
|
23
|
+
# Stage 2: Hardened Minimal Runtime
|
|
24
|
+
FROM node:20-alpine AS runner
|
|
25
|
+
WORKDIR /app
|
|
26
|
+
ENV NODE_ENV=production
|
|
27
|
+
|
|
28
|
+
# Security: Run as unprivileged non-root user
|
|
29
|
+
USER node
|
|
30
|
+
|
|
31
|
+
# Copy only production artifacts
|
|
32
|
+
COPY --chown=node:node --from=builder /app/package.json ./
|
|
33
|
+
COPY --chown=node:node --from=builder /app/node_modules ./node_modules
|
|
34
|
+
COPY --chown=node:node --from=builder /app/dist ./dist
|
|
35
|
+
|
|
36
|
+
EXPOSE 3000
|
|
37
|
+
CMD ["node", "dist/server.js"]
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
### 1.2 Container Security Rules
|
|
41
|
+
1. **Never run as root:** Always specify `USER node`, `USER 10001:10001`, or an unprivileged system user. Root access inside a container increases the severity of container-escape vulnerabilities.
|
|
42
|
+
2. **Zero baked secrets:** Never copy `.env`, private SSH keys, or cloud access tokens into image layers. Pass secrets at runtime via environment variables or secret managers.
|
|
43
|
+
3. **Comprehensive `.dockerignore`:** Always ignore `.git`, `node_modules`, `.env*`, test coverage reports, and scratch directories.
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## 2. 12-Factor Configuration & Startup Validation
|
|
48
|
+
|
|
49
|
+
Store all configuration in environment variables. Applications must validate configuration at boot time and fail immediately if required values are missing.
|
|
50
|
+
|
|
51
|
+
### 2.1 Startup Validation Schema
|
|
52
|
+
```typescript
|
|
53
|
+
import { z } from 'zod';
|
|
54
|
+
|
|
55
|
+
const EnvSchema = z.object({
|
|
56
|
+
NODE_ENV: z.enum(['development', 'test', 'production']),
|
|
57
|
+
PORT: z.coerce.number().default(3000),
|
|
58
|
+
DATABASE_URL: z.string().url(),
|
|
59
|
+
REDIS_URL: z.string().url(),
|
|
60
|
+
API_SECRET_KEY: z.string().min(32),
|
|
61
|
+
});
|
|
62
|
+
|
|
63
|
+
// Fail fast before opening ports or listening to queues
|
|
64
|
+
export const env = EnvSchema.parse(process.env);
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
- **Fail Fast:** If `DATABASE_URL` is missing, the application must crash with exit code 1 immediately with a descriptive error. Never start partially and fail on the first incoming user request.
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## 3. Observability & Health Probes
|
|
72
|
+
|
|
73
|
+
### 3.1 Separate Liveness and Readiness Probes
|
|
74
|
+
Kubernetes and container orchestrators need distinct signals:
|
|
75
|
+
- **Liveness Probe (`/healthz`):** Is the process running and not deadlocked? Returns `200 OK`. If failing, the orchestrator restarts the container.
|
|
76
|
+
- **Readiness Probe (`/readyz`):** Can the container accept external traffic? Checks database connectivity, cache warming, and migration status. If failing, traffic is redirected without restarting the process.
|
|
77
|
+
|
|
78
|
+
### 3.2 Structured JSON Logging
|
|
79
|
+
Production logs must be formatted as single-line JSON objects to enable ingestion and querying by log aggregators:
|
|
80
|
+
|
|
81
|
+
```json
|
|
82
|
+
{
|
|
83
|
+
"timestamp": "2026-11-04T10:14:02.184Z",
|
|
84
|
+
"level": "info",
|
|
85
|
+
"message": "Deployment completed",
|
|
86
|
+
"trace_id": "c4b9-482a-912b",
|
|
87
|
+
"project_id": "proj_9021",
|
|
88
|
+
"duration_ms": 342
|
|
89
|
+
}
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
- Always inject correlation/trace IDs to trace requests across microservices.
|
|
93
|
+
- **Never log PII or credentials:** Sanitize passwords, tokens, API keys, and credit card numbers before logging.
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## 4. Deterministic CI/CD Pipelines
|
|
98
|
+
|
|
99
|
+
1. **Deterministic Lockfiles:** Always use lockfile-enforced installation (`npm ci`, `poetry install --no-root`, `cargo check --locked`).
|
|
100
|
+
2. **Pipeline Caching:** Cache package manager directories (`~/.npm`, `~/.cache/pip`, `~/.cargo`) across pipeline runs.
|
|
101
|
+
3. **Fail-Closed Pipelines:** CI must halt immediately upon test failures, linter errors, or type errors. Never permit broken builds to deploy to staging or production.
|
|
102
|
+
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# Service Resilience: Timeouts, Backoff & Circuit Breakers
|
|
2
|
+
|
|
3
|
+
Distributed systems fail continuously. Resilience engineering ensures failures in one dependency do not cause cascading outages across the entire application ecosystem.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. Network Timeouts: The Zero-Hang Rule
|
|
8
|
+
|
|
9
|
+
Every outbound network call (HTTP, RPC, database, cache) must specify an explicit timeout. A network call without a timeout will eventually hang indefinitely under socket partition, exhausting application worker threads.
|
|
10
|
+
|
|
11
|
+
- **Connection Timeout:** Time allowed to establish the TCP connection (e.g., 2,000ms).
|
|
12
|
+
- **Read Timeout:** Time allowed for the remote service to return the response body (e.g., 5,000ms).
|
|
13
|
+
- **Total Request Timeout:** Upper bound on the entire lifecycle (e.g., 8,000ms).
|
|
14
|
+
|
|
15
|
+
```typescript
|
|
16
|
+
// Enforced: Explicit timeout with AbortSignal
|
|
17
|
+
const controller = new AbortController();
|
|
18
|
+
const timeoutId = setTimeout(() => controller.abort(), 5000);
|
|
19
|
+
|
|
20
|
+
try {
|
|
21
|
+
const response = await fetch('https://api.external.com/v1/rates', {
|
|
22
|
+
signal: controller.signal,
|
|
23
|
+
});
|
|
24
|
+
return await response.json();
|
|
25
|
+
} finally {
|
|
26
|
+
clearTimeout(timeoutId);
|
|
27
|
+
}
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## 2. Exponential Backoff with Full Jitter
|
|
33
|
+
|
|
34
|
+
When a network operation fails transiently, retrying immediately floods the struggling service (the "thundering herd" problem).
|
|
35
|
+
|
|
36
|
+
### 2.1 The Full Jitter Formula
|
|
37
|
+
$$T = \text{random}(0, \min(T_{\max}, T_{\text{base}} \times 2^{\text{attempt}}))$$
|
|
38
|
+
|
|
39
|
+
```typescript
|
|
40
|
+
function calculateJitteredBackoff(attempt: number, baseMs = 100, maxMs = 5000): number {
|
|
41
|
+
const exponential = Math.min(maxMs, baseMs * Math.pow(2, attempt));
|
|
42
|
+
return Math.floor(Math.random() * exponential);
|
|
43
|
+
}
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
### 2.2 Retry Conditions
|
|
47
|
+
- **Only retry idempotent operations:** Safe to retry `GET`, `PUT`, `DELETE`, or requests carrying a verified `Idempotency-Key`.
|
|
48
|
+
- **Never retry non-idempotent `POST` requests without idempotency keys:** Retrying a non-idempotent payment charge will charge the customer twice.
|
|
49
|
+
- **Retry transient status codes only:** Retry `503 Service Unavailable`, `504 Gateway Timeout`, or network connection reset (`ECONNRESET`). Never retry client validation errors (`400`, `401`, `403`, `422`).
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
## 3. Circuit Breaker State Machine
|
|
54
|
+
|
|
55
|
+
Prevent a failing downstream dependency from bringing down upstream services by short-circuiting calls when the failure rate exceeds a threshold:
|
|
56
|
+
|
|
57
|
+
```
|
|
58
|
+
┌────────────────────────────────────────┐
|
|
59
|
+
│ │
|
|
60
|
+
▼ │
|
|
61
|
+
┌───────────┐ Failure threshold reached │ Probe succeeds
|
|
62
|
+
│ CLOSED │ ───────────────────────────────► │
|
|
63
|
+
└───────────┘ │
|
|
64
|
+
▲ ▼
|
|
65
|
+
│ Cooldown timer elapsed ┌───────────┐
|
|
66
|
+
│ ┌───────────────────────── │ OPEN │
|
|
67
|
+
│ ▼ └───────────┘
|
|
68
|
+
┌───────────┐
|
|
69
|
+
│ HALF-OPEN │ ───────────────────────────────► Trip on probe failure
|
|
70
|
+
└───────────┘
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
1. **Closed (Normal):** Requests pass through. Metrics track consecutive failures.
|
|
74
|
+
2. **Open (Tripped):** When error rate exceeds threshold (e.g. 5 failures within 10s), the circuit trips. All calls fail fast immediately without making network calls.
|
|
75
|
+
3. **Half-Open (Trial):** After a cooldown period (e.g. 30s), a limited number of probe requests are allowed through. If successful, the circuit resets to Closed; if failing, it trips back to Open.
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
## 4. Hard vs. Soft Dependency Classification
|
|
80
|
+
|
|
81
|
+
Classify external dependencies into two categories before writing fallback code:
|
|
82
|
+
|
|
83
|
+
| Category | Definition | Operational Response |
|
|
84
|
+
|---|---|---|
|
|
85
|
+
| **Hard Dependency** | Core system cannot complete the transaction without it (e.g., Primary database for auth). | Fail fast, roll back local transactions, return structured RFC 9457 error with actionable explanation. |
|
|
86
|
+
| **Soft Dependency** | Secondary feature that enriches experience but is not critical to the core transaction (e.g., Recommendations, avatar image fetch). | Fall back to static safe defaults, stale cache (`stale-while-revalidate`), or degraded mode with zero disruption to the primary journey. |
|
|
87
|
+
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: asc-review
|
|
3
3
|
description: >
|
|
4
|
-
Trigger this skill when the user is: reviewing pull requests, checking recent changes, auditing security, scanning for vulnerabilities, threat modeling, or detecting duplicate code. Trigger when the user says: "review this code", "check this PR", "what's wrong with this", "look at my changes", "is this production-ready", "find bugs", "audit this", "security check", "find vulnerabilities", "check for XSS", "check for SQL injection", "OWASP check", "check auth", "find duplicate code", "check for clones", "audit for duplication", "review PR ini", "cek kodingan saya", "cari celah keamanan", "cari kode duplikat".
|
|
4
|
+
Trigger this skill when the user is: reviewing pull requests, checking recent changes, auditing security, scanning for vulnerabilities, threat modeling, or detecting duplicate code. Trigger when the user says: "review this code", "check this PR", "what's wrong with this", "look at my changes", "is this production-ready", "find bugs", "audit this", "security check", "find vulnerabilities", "check for XSS", "check for SQL injection", "OWASP check", "check auth", "find duplicate code", "check for clones", "audit for duplication", "review PR ini", "cek kodingan saya", "cari celah keamanan", "cari kode duplikat".
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
# Quality & Security Assurance Playbook (`asc-review`)
|
|
@@ -12,14 +12,27 @@ Grounded in: OWASP Top 10 (2025), OWASP ASVS v5.0, OWASP Agentic Applications (A
|
|
|
12
12
|
|
|
13
13
|
---
|
|
14
14
|
|
|
15
|
-
##
|
|
15
|
+
## 1. Claim Verification Gate
|
|
16
|
+
|
|
17
|
+
Never accept claims of correctness or test coverage without fresh execution evidence:
|
|
18
|
+
|
|
19
|
+
| Claim | Requires (Sufficient Evidence) | NOT Sufficient (Invalid Evidence) |
|
|
20
|
+
|---|---|---|
|
|
21
|
+
| *"Code works / Bug fixed."* | A reproduction test failed before the fix, passes with it, and full test suite passes. | *"The code looks correct."*<br>*"It compiled without warnings."* |
|
|
22
|
+
| *"All tests pass."* | Fresh test suite run in the current session with exit code 0 and 0 failures. | *"Tests passed in a previous turn."*<br>*"I only modified a comment."* |
|
|
23
|
+
| *"Zero security vulnerabilities."* | Parameterized query checks, auth checks, and dependency audit tool output. | *"I used an ORM so it's safe."* |
|
|
24
|
+
| *"No duplicate code."* | Lexical or token dedup scan with Rule of Three enforcement. | *"I don't recall copying this."* |
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## 2. Mode 1: Production Code Review
|
|
16
29
|
|
|
17
30
|
Use when reviewing PRs, diffs, commits, or newly written feature code.
|
|
18
31
|
|
|
19
32
|
### Review Procedure
|
|
20
33
|
1. **Scope Inspection**: Run `git diff --stat` (or `git status`) to inspect modified files and line deltas.
|
|
21
|
-
2. **Test-Change Correlation**: For every modified application logic file, verify that a corresponding test was updated. If logic changed without tests,
|
|
22
|
-
3. **Deterministic Verification**: Run project linter and test suite (`ascx test` / `npm test`).
|
|
34
|
+
2. **Test-Change Correlation**: For every modified application logic file, verify that a corresponding test was updated. If logic changed without tests, flag as missing test coverage.
|
|
35
|
+
3. **Deterministic Verification**: Run project linter and test suite (`ascx test` / `npm test`). Confirm exit code 0 fresh.
|
|
23
36
|
4. **Severity Priority Order**:
|
|
24
37
|
- **Severity 1 (Critical)**: Correctness, data corruption, auth/permission bypass, security sinks, PII leakage.
|
|
25
38
|
- **Severity 2 (High)**: Action feedback failures (hanging spinners, unhandled mutations, missing error boundaries), unhandled async rejections, missing remote timeouts.
|
|
@@ -28,7 +41,7 @@ Use when reviewing PRs, diffs, commits, or newly written feature code.
|
|
|
28
41
|
|
|
29
42
|
---
|
|
30
43
|
|
|
31
|
-
## Mode 2: Security & Vulnerability Audit
|
|
44
|
+
## 3. Mode 2: Security & Vulnerability Audit
|
|
32
45
|
|
|
33
46
|
Use when assessing trust boundaries, reviewing auth systems, running threat models, or auditing external dependencies.
|
|
34
47
|
|
|
@@ -48,7 +61,7 @@ Use when assessing trust boundaries, reviewing auth systems, running threat mode
|
|
|
48
61
|
|
|
49
62
|
---
|
|
50
63
|
|
|
51
|
-
## Mode 3: Clone & Code Duplication Audit
|
|
64
|
+
## 4. Mode 3: Clone & Code Duplication Audit
|
|
52
65
|
|
|
53
66
|
Use when evaluating copy-paste repetition or consolidating duplicated logic across a repository.
|
|
54
67
|
|
|
@@ -61,7 +74,7 @@ Use when evaluating copy-paste repetition or consolidating duplicated logic acro
|
|
|
61
74
|
|
|
62
75
|
---
|
|
63
76
|
|
|
64
|
-
## Output Format for Findings
|
|
77
|
+
## 5. Output Format for Findings
|
|
65
78
|
|
|
66
79
|
Report findings ordered strictly by severity with file and line references:
|
|
67
80
|
|
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: asc-test
|
|
3
|
+
description: >
|
|
4
|
+
Trigger this skill when the user is: writing unit tests, integration tests, E2E tests, UAT checklists, reproducing bugs, fixing flaky tests, or verifying test coverage. Also trigger when the user says: "write a test", "TDD", "add tests", "verify this bug", "flaky test", "test this endpoint", "E2E test", "playwright", "UAT", "regression test", "bikin tes", "cara ngetes", "tes gagal terus". Also trigger when creating or editing files matching: `*.test.*`, `*.spec.*`, `tests/*`, `__tests__/*`, or `e2e/*`.
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Testing, TDD, Verification & UAT Router
|
|
8
|
+
|
|
9
|
+
Staff-grade testing playbook and verification gates. Enforces test-first discipline and routes deep testing guidance on demand.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## 1. The Iron Law of TDD
|
|
14
|
+
|
|
15
|
+
> **THE IRON LAW:** No production code shall be written without a failing test first.
|
|
16
|
+
|
|
17
|
+
Every feature, bugfix, or refactor must follow the Red-Green-Refactor sequence:
|
|
18
|
+
1. **RED:** Write a minimal failing test. Execute it and confirm it fails for the expected reason.
|
|
19
|
+
2. **GREEN:** Write the minimum production code necessary to make the test pass.
|
|
20
|
+
3. **REFACTOR:** Clean up code structure while keeping the test suite green.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. The Verification Gate
|
|
25
|
+
|
|
26
|
+
Never make a claim regarding test status without fresh execution evidence:
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
|
|
30
|
+
│ 1. IDENTIFY CMD │ ───► │ 2. RUN FRESH │ ───► │ 3. CONFIRM CLAIM │
|
|
31
|
+
└──────────────────┘ └──────────────────┘ └──────────────────┘
|
|
32
|
+
Locate exact test Execute runner, read State pass/fail with
|
|
33
|
+
command in project exit code directly raw exit code & logs
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
1. **Identify the Command:** Determine the exact test command from `package.json`, `Makefile`, `pytest.ini`, or project manifests.
|
|
37
|
+
2. **Execute Fresh:** Run the command in the current turn. Past execution results or assumptions are invalid.
|
|
38
|
+
3. **Confirm Claim:** Read the raw exit code and failed assertion count before reporting results.
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
## 3. Rationalization Prevention Table
|
|
43
|
+
|
|
44
|
+
| Rationalization | Fallacy | Operational Rule |
|
|
45
|
+
|---|---|---|
|
|
46
|
+
| *"The change is too trivial to test."* | Trivial changes cause silent regressions. | Write the test. If it takes 2 minutes to write the fix, it takes 2 minutes to write the test. |
|
|
47
|
+
| *"I'll write tests after the code works."* | Tests written after mirror bugs. | Write the failing assertion before implementing the behavior. |
|
|
48
|
+
| *"It's just UI / frontend wiring."* | Visual wiring has state logic. | Form validation, button states, and error handling are pure logic and testable. |
|
|
49
|
+
| *"Mocking this dependency is too hard."* | Architecture smells. | Difficulty in testing indicates tight coupling. Extract pure functions or use in-memory fakes. |
|
|
50
|
+
| *"Tests take too long to run."* | Slowness masks bugs. | Run the focused sub-suite during development; run the full suite before handoff. |
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## 4. Testing Decision Tree
|
|
55
|
+
|
|
56
|
+
Select the lowest testing tier that validates the requirement:
|
|
57
|
+
|
|
58
|
+
```
|
|
59
|
+
Is this pure business logic or algorithm?
|
|
60
|
+
│
|
|
61
|
+
┌───────────────┴───────────────┐
|
|
62
|
+
YES NO
|
|
63
|
+
│ │
|
|
64
|
+
┌───────────────┐ Does it involve database, cache, or HTTP APIs?
|
|
65
|
+
│ 1. UNIT TEST │ │
|
|
66
|
+
└───────────────┘ ┌───────┴───────┐
|
|
67
|
+
YES NO
|
|
68
|
+
│ │
|
|
69
|
+
┌───────────────┐ Does it involve browser UI or
|
|
70
|
+
│2. INTEGRATION │ multi-step user journeys?
|
|
71
|
+
└───────────────┘ │
|
|
72
|
+
┌───────┴───────┐
|
|
73
|
+
YES NO
|
|
74
|
+
│ │
|
|
75
|
+
┌───────────────┐ ┌───────────┐
|
|
76
|
+
│ 3. E2E (TEST) │ │ 4. UAT │
|
|
77
|
+
└───────────────┘ └───────────┘
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
---
|
|
81
|
+
|
|
82
|
+
## 5. On-Demand Reference Library
|
|
83
|
+
|
|
84
|
+
Load these references only when working on that specific domain:
|
|
85
|
+
|
|
86
|
+
| Topic | Reference Path |
|
|
87
|
+
|---|---|
|
|
88
|
+
| Red-Green-Refactor mechanics & whole-suite checks | [references/tdd-cycle.md](references/tdd-cycle.md) |
|
|
89
|
+
| Behavior testing, boundary isolation & mocking | [references/test-design.md](references/test-design.md) |
|
|
90
|
+
| Playwright E2E, reconnaissance & locator hierarchy | [references/e2e-playwright.md](references/e2e-playwright.md) |
|
|
91
|
+
| Given/When/Then acceptance criteria & edge case quad | [references/uat.md](references/uat.md) |
|
|
92
|
+
| 3-step bugfix verification & regression proof | [references/regression-proof.md](references/regression-proof.md) |
|
|
93
|
+
| Flakiness root causes & 50-run stress loop | [references/flaky-tests.md](references/flaky-tests.md) |
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## 6. Verification & Handoff Sweep
|
|
98
|
+
|
|
99
|
+
Before declaring any coding or testing task complete:
|
|
100
|
+
1. **Regression Proof:** For bugfixes, confirm the test failed without the fix and passes with it.
|
|
101
|
+
2. **Whole-Suite Run:** Execute the full test suite (`ascx npm test` or project test runner); confirm exit code 0.
|
|
102
|
+
3. **No Assertions Weakened:** Verify no existing test assertions were deleted or commented out to simulate success.
|
|
103
|
+
4. **Hermetic State:** Ensure tests clean up temporary files, database records, and environment variables.
|
|
104
|
+
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
# End-to-End (E2E) & Playwright Testing
|
|
2
|
+
|
|
3
|
+
E2E browser tests verify integrated user journeys across the full application stack (HTML, CSS, JS, network, database).
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. Reconnaissance Before Action
|
|
8
|
+
|
|
9
|
+
Never write browser test steps by guessing selectors or UI timing.
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
|
|
13
|
+
│ 1. RECONNAISSANCE│ ───► │ 2. VERIFY DOM │ ───► │ 3. EXECUTE │
|
|
14
|
+
└──────────────────┘ └──────────────────┘ └──────────────────┘
|
|
15
|
+
Navigate to route Capture rendered DOM Target resilient
|
|
16
|
+
Wait for networkidle or test snapshot user-facing locator
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
### 1.1 The Inspection Protocol
|
|
20
|
+
1. **Navigate and Settle:** Navigate to the target URL and wait for asynchronous network activity to stabilize (`networkidle` or specific locator hydration).
|
|
21
|
+
2. **Inspect Rendered State:** Extract the accessibility tree or screenshot to inspect the actual DOM elements currently visible to users.
|
|
22
|
+
3. **Select Resilient Locators:** Use Playwright's user-facing locator hierarchy:
|
|
23
|
+
- Priority 1: `page.getByRole('button', { name: 'Deploy' })`
|
|
24
|
+
- Priority 2: `page.getByLabel('Work email')`
|
|
25
|
+
- Priority 3: `page.getByPlaceholder('Search repositories...')`
|
|
26
|
+
- Priority 4: `page.getByTestId('billing-summary-table')`
|
|
27
|
+
- **Banned:** Brittle internal CSS class selectors: `page.locator('.sc-bdVaJa > div:nth-child(2)')` or compiled Tailwind classes `page.locator('.p-4.border.rounded-xl')`.
|
|
28
|
+
|
|
29
|
+
---
|
|
30
|
+
|
|
31
|
+
## 2. The Ban on Arbitrary Sleep Delays
|
|
32
|
+
|
|
33
|
+
Hardcoded time delays are the primary cause of flaky CI builds and slow test runs.
|
|
34
|
+
|
|
35
|
+
- **Strictly Banned:**
|
|
36
|
+
```typescript
|
|
37
|
+
// BANNED: Flaky, slow, and non-deterministic
|
|
38
|
+
await page.waitForTimeout(5000);
|
|
39
|
+
await new Promise(r => setTimeout(r, 3000));
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
- **Enforced: Auto-waiting and State Assertions:**
|
|
43
|
+
Playwright automatically waits for elements to be actionable (visible, stable, enabled) before clicking or filling.
|
|
44
|
+
```typescript
|
|
45
|
+
// ENFORCED: Waits dynamically up to timeout, proceeds immediately upon readiness
|
|
46
|
+
await expect(page.getByRole('dialog')).toBeVisible();
|
|
47
|
+
await page.getByRole('button', { name: 'Confirm deletion' }).click();
|
|
48
|
+
await expect(page.getByText('Repository deleted')).toBeVisible();
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
## 3. Web Server Lifecycle Management
|
|
54
|
+
|
|
55
|
+
E2E tests require a running backend or frontend server.
|
|
56
|
+
|
|
57
|
+
### 3.1 Playwright `webServer` Configuration
|
|
58
|
+
Configure the web server within `playwright.config.ts` so Playwright manages process startup and teardown automatically:
|
|
59
|
+
|
|
60
|
+
```typescript
|
|
61
|
+
import { defineConfig } from '@playwright/test';
|
|
62
|
+
|
|
63
|
+
export default defineConfig({
|
|
64
|
+
webServer: {
|
|
65
|
+
command: 'npm run start',
|
|
66
|
+
url: 'http://localhost:3000',
|
|
67
|
+
reuseExistingServer: !process.env.CI,
|
|
68
|
+
timeout: 120 * 1000,
|
|
69
|
+
},
|
|
70
|
+
use: {
|
|
71
|
+
baseURL: 'http://localhost:3000',
|
|
72
|
+
trace: 'on-first-retry',
|
|
73
|
+
},
|
|
74
|
+
});
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
### 3.2 Ephemeral Test Ports
|
|
78
|
+
When running tests in parallel or in multi-branch environments, avoid hardcoded port conflicts:
|
|
79
|
+
- Bind dev/test servers to port `0` or assign dynamic test ports via environment variables (`PORT=3100`).
|
|
80
|
+
|
|
81
|
+
---
|
|
82
|
+
|
|
83
|
+
## 4. Test State Setup: API Seeding Over UI Form Clicking
|
|
84
|
+
|
|
85
|
+
Do not use browser UI steps to set up prerequisite data for every test.
|
|
86
|
+
|
|
87
|
+
- **Slow & Brittle:** Logging in via UI form $\to$ clicking through 5 wizards $\to$ creating a project $\to$ finally testing the billing page.
|
|
88
|
+
- **Fast & Resilient:** Seed prerequisite users, workspaces, or API keys directly via database fixtures or HTTP API calls (`request.post('/api/test/seed')`), then navigate straight to the billing page.
|
|
89
|
+
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
# Flaky Tests Diagnosis & Remediation
|
|
2
|
+
|
|
3
|
+
A flaky test is a test that exhibits both passing and failing results under identical code and execution configurations. Flaky tests erode team trust in CI and mask real production defects.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. The Five Root Causes of Flakiness
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
┌────────────────────────────────────────────────────────┐
|
|
11
|
+
│ THE 5 FLAKINESS ROOT CAUSES │
|
|
12
|
+
├────────────────────────────────────────────────────────┤
|
|
13
|
+
│ 1. Asynchronous Race Conditions (Unawaited Promises) │
|
|
14
|
+
│ 2. Shared Mutable State (Polluted Databases or Globals)│
|
|
15
|
+
│ 3. Time, Date & Timezone Leaks │
|
|
16
|
+
│ 4. Non-Deterministic Collection Ordering │
|
|
17
|
+
│ 5. Network Latency & Port Collisions │
|
|
18
|
+
└────────────────────────────────────────────────────────┘
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## 2. Root Cause Diagnostics & Remediation
|
|
24
|
+
|
|
25
|
+
### 2.1 Asynchronous Race Conditions
|
|
26
|
+
- **Symptom:** Test occasionally fails with `Cannot read property of undefined` or misses an assertion before background work finishes.
|
|
27
|
+
- **Root Cause:** A fire-and-forget Promise or unawaited async operation.
|
|
28
|
+
- **Remediation:** Always `await` every asynchronous call:
|
|
29
|
+
```typescript
|
|
30
|
+
// FLAKY: Background work runs concurrently with test teardown
|
|
31
|
+
service.processQueue();
|
|
32
|
+
expect(queue.isEmpty()).toBe(true);
|
|
33
|
+
|
|
34
|
+
// DETERMINISTIC: Await completion before asserting
|
|
35
|
+
await service.processQueue();
|
|
36
|
+
expect(queue.isEmpty()).toBe(true);
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
### 2.2 Shared Mutable State
|
|
40
|
+
- **Symptom:** Test passes when run alone, but fails when run as part of the full suite.
|
|
41
|
+
- **Root Cause:** A preceding test inserted records into a database, set a global variable, or modified an in-memory cache without cleanup.
|
|
42
|
+
- **Remediation:** Ensure hermetic isolation in `beforeEach` and `afterEach`:
|
|
43
|
+
```typescript
|
|
44
|
+
afterEach(async () => {
|
|
45
|
+
await db.clean();
|
|
46
|
+
sinon.restore();
|
|
47
|
+
delete process.env.FEATURE_FLAG;
|
|
48
|
+
});
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
### 2.3 Time, Date & Timezone Sensitivity
|
|
52
|
+
- **Symptom:** Tests fail at midnight, on month boundaries, during Daylight Saving Time (DST) transitions, or in CI runners located in UTC.
|
|
53
|
+
- **Root Cause:** Direct invocation of `new Date()` or `Date.now()` without frozen timers.
|
|
54
|
+
- **Remediation:** Freeze time deterministically:
|
|
55
|
+
```typescript
|
|
56
|
+
// Use fake timers locked to a fixed UTC epoch
|
|
57
|
+
beforeEach(() => {
|
|
58
|
+
vi.useFakeTimers();
|
|
59
|
+
vi.setSystemTime(new Date('2026-01-15T12:00:00Z'));
|
|
60
|
+
});
|
|
61
|
+
afterEach(() => {
|
|
62
|
+
vi.useRealTimers();
|
|
63
|
+
});
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
### 2.4 Non-Deterministic Collection Ordering
|
|
67
|
+
- **Symptom:** Array equality assertions fail intermittently: `Expected [1, 2], Received [2, 1]`.
|
|
68
|
+
- **Root Cause:** SQL `SELECT` queries without an explicit `ORDER BY`, or iterating over JavaScript `Set` / Go `map`.
|
|
69
|
+
- **Remediation:** Sort collections or compare using set equivalence:
|
|
70
|
+
```typescript
|
|
71
|
+
// FLAKY: Relies on non-guaranteed database return order
|
|
72
|
+
expect(results.map(r => r.id)).toEqual(['id-1', 'id-2']);
|
|
73
|
+
|
|
74
|
+
// DETERMINISTIC: Explicit sorting before comparison
|
|
75
|
+
expect(results.map(r => r.id).sort()).toEqual(['id-1', 'id-2']);
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## 3. The 50-Run Stress Loop Protocol
|
|
81
|
+
|
|
82
|
+
When investigating a suspected flaky test, stress-test it in isolation:
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
# Bash / PowerShell stress loop
|
|
86
|
+
for ($i=1; $i -le 50; $i++) {
|
|
87
|
+
ascx npm test -- -t "SuspectedFlakySuite"
|
|
88
|
+
if ($LASTEXITCODE -ne 0) {
|
|
89
|
+
Write-Host "Flake detected on iteration $i"
|
|
90
|
+
break
|
|
91
|
+
}
|
|
92
|
+
}
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
A test is verified fixed only when it passes 50 consecutive runs under randomized order without failure.
|
|
96
|
+
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
# Regression Proof & Bugfix Verification
|
|
2
|
+
|
|
3
|
+
A bugfix is unproven until an automated test fails without the fix and passes with it.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. The Three-Step Bugfix Verification Protocol
|
|
8
|
+
|
|
9
|
+
Never fix a bug by immediately editing production code. Follow this deterministic sequence:
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
┌─────────────────────────┐
|
|
13
|
+
│ 1. REPRODUCTION TEST │ ──► Run fresh: MUST FAIL (RED)
|
|
14
|
+
└───────────┬─────────────┘
|
|
15
|
+
│
|
|
16
|
+
▼
|
|
17
|
+
┌─────────────────────────┐
|
|
18
|
+
│ 2. IMPLEMENT FIX │ ──► Run fresh: MUST PASS (GREEN)
|
|
19
|
+
└───────────┬─────────────┘
|
|
20
|
+
│
|
|
21
|
+
▼
|
|
22
|
+
┌─────────────────────────┐
|
|
23
|
+
│ 3. REGRESSION PROOF │ ──► Revert fix: MUST FAIL
|
|
24
|
+
└─────────────────────────┘ Re-apply fix: MUST PASS
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
### Step 1: Write the Reproduction Test
|
|
28
|
+
- Author a test that isolates the reported failure condition.
|
|
29
|
+
- Run the test runner:
|
|
30
|
+
```bash
|
|
31
|
+
ascx npm test -- -t "reproduces issue #42"
|
|
32
|
+
```
|
|
33
|
+
- **Verification:** The test MUST fail. If the test passes before you touch production code, you have either failed to reproduce the bug or written an assertion that tests the wrong condition.
|
|
34
|
+
|
|
35
|
+
### Step 2: Implement the Minimal Fix
|
|
36
|
+
- Edit production code to eliminate the bug.
|
|
37
|
+
- Run the reproduction test fresh: it must pass with exit code 0.
|
|
38
|
+
|
|
39
|
+
### Step 3: The Regression Proof Gate (Revert & Confirm)
|
|
40
|
+
To prove that the test actually guards against the bug:
|
|
41
|
+
1. Temporarily comment out or revert the fix in production code.
|
|
42
|
+
2. Run the test fresh: **The test must fail.**
|
|
43
|
+
3. Re-apply the fix.
|
|
44
|
+
4. Run the full test suite fresh: **All tests must pass.**
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## 2. Claim Verification Gate
|
|
49
|
+
|
|
50
|
+
Every claim regarding test results or bug fixes requires concrete execution evidence:
|
|
51
|
+
|
|
52
|
+
| Claim | Requires (Sufficient Evidence) | NOT Sufficient (Invalid Evidence) |
|
|
53
|
+
|---|---|---|
|
|
54
|
+
| *"Bug is fixed."* | A reproduction test failed before the fix, passes with the fix, fails when fix is temporarily reverted, and full suite passes. | *"The code logic looks sound."*<br>*"The compiler reports 0 syntax errors."*<br>*"I refreshed the browser page once."* |
|
|
55
|
+
| *"All tests pass."* | Fresh execution of the entire test suite showing 0 failures and exit code 0 in the current session. | *"Tests passed in an earlier turn."*<br>*"I only modified a small utility function."*<br>*"The linter reported no warnings."* |
|
|
56
|
+
| *"Refactoring didn't break anything."* | Full test suite passed before refactoring and passes after refactoring with zero modified assertions. | *"No imports were changed."*<br>*"The types align in the IDE."* |
|
|
57
|
+
| *"Test is hermetic."* | Test passes when run in isolation and when run alongside other tests in randomized order. | *"Test passes when run as the first test in the suite."* |
|
|
58
|
+
|