code-ai-installer 4.0.1-a → 4.0.1-c
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +1 -1
- package/README.md +5 -5
- package/dist/catalog.js +1 -1
- package/dist/contentTransformer.d.ts +1 -1
- package/dist/contentTransformer.js +39 -0
- package/dist/index.js +10 -5
- package/dist/mcp/cli.js +4 -4
- package/dist/mcp/config.js +8 -6
- package/dist/mcp/scorecard.d.ts +2 -2
- package/dist/mcp/task_state.d.ts +2 -2
- package/dist/mcp/tools/advance_gate.js +1 -1
- package/dist/mcp/tools/classify_gate.d.ts +2 -2
- package/dist/mcp/tools/classify_gate.js +2 -2
- package/dist/mcp/tools/load_role.d.ts +2 -2
- package/dist/mcp/tools/load_role.js +2 -2
- package/dist/mcp/tools/report_exception.d.ts +3 -3
- package/dist/mcp/tools/report_exception.js +4 -4
- package/dist/mcp/tools/request_decision.d.ts +3 -3
- package/dist/mcp/tools/request_decision.js +5 -5
- package/dist/mcp/tools/review_proposal.d.ts +1 -1
- package/dist/mcp/tools/review_proposal.js +6 -6
- package/dist/mcp/tools/sign_off.d.ts +2 -2
- package/dist/mcp/tools/sign_off.js +7 -7
- package/dist/mcp/tools/verify_claim.d.ts +1 -1
- package/dist/mcp/tools/verify_claim.js +1 -1
- package/dist/mcp_setup.d.ts +85 -29
- package/dist/mcp_setup.js +184 -62
- package/dist/platforms/adapters.js +54 -19
- package/dist/shared/frontmatter.js +1 -1
- package/dist/shared/persona.d.ts +1 -1
- package/dist/shared/persona.js +1 -1
- package/dist/shared/pipeline.d.ts +10 -10
- package/dist/shared/pipeline.js +7 -7
- package/dist/shared/tools.d.ts +15 -15
- package/dist/shared/tools.js +3 -3
- package/dist/shared/vocabulary.d.ts +4 -4
- package/dist/shared/vocabulary.js +4 -4
- package/dist/types.d.ts +1 -1
- package/domains/analytics/.agents/workflows/analytics-pipeline-rules.md +13 -3
- package/domains/analytics/.agents/workflows/analyze.md +1 -0
- package/domains/analytics/.agents/workflows/quick-insight.md +1 -0
- package/domains/analytics/locales/en/.agents/workflows/analytics-pipeline-rules.md +13 -3
- package/domains/analytics/locales/en/.agents/workflows/analyze.md +1 -0
- package/domains/analytics/locales/en/.agents/workflows/quick-insight.md +1 -0
- package/domains/analytics/locales/en/agents/interviewer.md +2 -1
- package/domains/analytics/locales/en/agents/layouter.md +2 -1
- package/domains/analytics/locales/en/agents/mediator.md +2 -1
- package/domains/analytics/locales/en/agents/researcher.md +2 -1
- package/domains/analytics/locales/en/agents/strategist.md +2 -1
- package/domains/analytics/pipeline.yaml +10 -10
- package/domains/content/.agents/skills/content-release-gate/SKILL.md +3 -5
- package/domains/content/.agents/workflows/content-pipeline-rules.md +14 -11
- package/domains/content/.agents/workflows/edit-content.md +0 -1
- package/domains/content/.agents/workflows/quick-post.md +0 -1
- package/domains/content/.agents/workflows/start-content.md +0 -1
- package/domains/content/agents/conductor.md +1 -2
- package/domains/content/locales/en/.agents/skills/content-release-gate/SKILL.md +3 -5
- package/domains/content/locales/en/.agents/workflows/content-pipeline-rules.md +14 -11
- package/domains/content/locales/en/.agents/workflows/edit-content.md +0 -1
- package/domains/content/locales/en/.agents/workflows/quick-post.md +0 -1
- package/domains/content/locales/en/.agents/workflows/start-content.md +0 -1
- package/domains/content/locales/en/agents/conductor.md +1 -2
- package/domains/content/pipeline.yaml +8 -8
- package/domains/development/.agents/skills/handoff/SKILL.md +276 -276
- package/domains/development/.agents/skills/lava-flow-legacy-detection/SKILL.md +197 -197
- package/domains/development/.agents/skills/mcp-integration/SKILL.md +211 -211
- package/domains/development/.agents/skills/qa-test-data-management/SKILL.md +250 -250
- package/domains/development/.agents/workflows/bugfix.md +16 -82
- package/domains/development/.agents/workflows/hotfix.md +16 -66
- package/domains/development/.agents/workflows/pipeline-rules.md +49 -132
- package/domains/development/.agents/workflows/start-task.md +17 -121
- package/domains/development/AGENTS.md +8 -3
- package/domains/development/agents/architect.md +247 -247
- package/domains/development/agents/conductor.md +363 -363
- package/domains/development/agents/devops.md +297 -297
- package/domains/development/agents/reviewer.md +293 -293
- package/domains/development/agents/senior_full_stack.md +295 -295
- package/domains/development/agents/tester.md +395 -395
- package/domains/development/locales/en/.agents/skills/handoff/SKILL.md +276 -276
- package/domains/development/locales/en/.agents/skills/lava-flow-legacy-detection/SKILL.md +197 -197
- package/domains/development/locales/en/.agents/skills/mcp-integration/SKILL.md +211 -211
- package/domains/development/locales/en/.agents/skills/qa-test-data-management/SKILL.md +250 -250
- package/domains/development/locales/en/.agents/workflows/bugfix.md +16 -82
- package/domains/development/locales/en/.agents/workflows/hotfix.md +15 -65
- package/domains/development/locales/en/.agents/workflows/pipeline-rules.md +48 -131
- package/domains/development/locales/en/.agents/workflows/start-task.md +17 -121
- package/domains/development/locales/en/AGENTS.md +15 -0
- package/domains/development/locales/en/agents/architect.md +247 -247
- package/domains/development/locales/en/agents/conductor.md +363 -363
- package/domains/development/locales/en/agents/devops.md +297 -297
- package/domains/development/locales/en/agents/reviewer.md +293 -293
- package/domains/development/locales/en/agents/senior_full_stack.md +295 -295
- package/domains/development/locales/en/agents/tester.md +395 -395
- package/domains/development/locales/en/prompt-examples.md +34 -120
- package/domains/development/pipeline.yaml +150 -135
- package/domains/development/prompt-examples.md +33 -119
- package/domains/product/.agents/workflows/product-pipeline-rules.md +13 -2
- package/domains/product/.agents/workflows/quick-pm.md +1 -1
- package/domains/product/.agents/workflows/shape-prioritize.md +1 -0
- package/domains/product/.agents/workflows/ship-right-thing.md +1 -0
- package/domains/product/.agents/workflows/spec.md +1 -0
- package/domains/product/agents/tech_lead.md +1 -1
- package/domains/product/locales/en/.agents/workflows/product-pipeline-rules.md +13 -2
- package/domains/product/locales/en/.agents/workflows/quick-pm.md +1 -1
- package/domains/product/locales/en/.agents/workflows/shape-prioritize.md +1 -0
- package/domains/product/locales/en/.agents/workflows/ship-right-thing.md +1 -0
- package/domains/product/locales/en/.agents/workflows/spec.md +1 -0
- package/domains/product/locales/en/agents/conductor.md +2 -2
- package/domains/product/locales/en/agents/data_analyst.md +2 -1
- package/domains/product/locales/en/agents/designer.md +2 -1
- package/domains/product/locales/en/agents/discovery.md +2 -1
- package/domains/product/locales/en/agents/layouter.md +2 -1
- package/domains/product/locales/en/agents/mediator.md +2 -1
- package/domains/product/locales/en/agents/pm.md +2 -1
- package/domains/product/locales/en/agents/product_strategist.md +2 -1
- package/domains/product/locales/en/agents/tech_lead.md +3 -2
- package/domains/product/locales/en/agents/ux_designer.md +2 -1
- package/domains/product/pipeline.yaml +12 -12
- package/package.json +5 -5
- package/domains/analytics/CONTEXT.md +0 -25
- package/domains/analytics/locales/en/CONTEXT.md +0 -25
- package/domains/content/CONTEXT.md +0 -19
- package/domains/content/locales/en/CONTEXT.md +0 -19
- package/domains/development/.agents/workflows/auto-restart-containers.md +0 -56
- package/domains/development/CONTEXT.md +0 -62
- package/domains/development/locales/en/.agents/workflows/auto-restart-containers.md +0 -24
- package/domains/development/locales/en/CONTEXT.md +0 -62
- package/domains/product/CONTEXT.md +0 -40
- package/domains/product/locales/en/CONTEXT.md +0 -40
|
@@ -1,250 +1,250 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: qa-test-data-management
|
|
3
|
-
description: Test data management — fixtures generated from real schemas (TS types, DB schema, OpenAPI), PII hygiene (faker/factory_boy for synthetics), prod-like masking when copying prod data, environment isolation (testcontainers, transactional rollback, tempdir), fixture lifecycle. Defense against Mode 1 (mock obsession) — fixture does not drift from reality.
|
|
4
|
-
type: discretionary
|
|
5
|
-
domain: development
|
|
6
|
-
owners:
|
|
7
|
-
- tester
|
|
8
|
-
gates:
|
|
9
|
-
- TEST
|
|
10
|
-
tech:
|
|
11
|
-
- docker
|
|
12
|
-
- faker
|
|
13
|
-
- factory_boy
|
|
14
|
-
topic:
|
|
15
|
-
- qa
|
|
16
|
-
triggers:
|
|
17
|
-
- test data management
|
|
18
|
-
- test fixtures
|
|
19
|
-
- PII in tests
|
|
20
|
-
- faker
|
|
21
|
-
- factory_boy
|
|
22
|
-
- testcontainers
|
|
23
|
-
- prod data masking
|
|
24
|
-
- fixture drift
|
|
25
|
-
- environment isolation
|
|
26
|
-
related:
|
|
27
|
-
- qa-test-plan
|
|
28
|
-
- qa-regression-baseline
|
|
29
|
-
- qa-mutation-testing
|
|
30
|
-
- tests-quality-review
|
|
31
|
-
budget_lines: 250
|
|
32
|
-
schema_version: 1
|
|
33
|
-
---
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
# Skill: QA Test Data Management
|
|
37
|
-
|
|
38
|
-
Test data management so fixtures **do not drift from reality**, **do not contain PII**, and **do not pollute** environment between runs. Defense against Mode 1 of Test Integrity Defense (mock obsession) — because mock obsession often starts with a "hand-crafted" fixture that diverges from real schema.
|
|
39
|
-
|
|
40
|
-
**Sections:**
|
|
41
|
-
1. [When to apply](#1-when)
|
|
42
|
-
2. [Fixture derivation from real schemas](#2-derivation)
|
|
43
|
-
3. [PII hygiene](#3-pii)
|
|
44
|
-
4. [Prod-like data masking](#4-masking)
|
|
45
|
-
5. [Environment isolation](#5-isolation)
|
|
46
|
-
6. [Fixture lifecycle](#6-lifecycle)
|
|
47
|
-
7. [Anti-patterns](#7-anti)
|
|
48
|
-
8. [Output template](#8-output)
|
|
49
|
-
9. [DoD](#9-dod)
|
|
50
|
-
|
|
51
|
-
---
|
|
52
|
-
|
|
53
|
-
## <a id="1-when"></a>1. When to apply
|
|
54
|
-
|
|
55
|
-
**Triggers:**
|
|
56
|
-
- First integration test on new module / new DB table / new API endpoint
|
|
57
|
-
- Security alert: PII detected in test fixtures (auto-scan, see `$qa-test-integrity-audit`)
|
|
58
|
-
- DB schema migration — fixtures of all affected tests need re-generation
|
|
59
|
-
- Stabilization sprint includes test data quality audit
|
|
60
|
-
|
|
61
|
-
**Outputs:**
|
|
62
|
-
- Fixture generation script / decorator / factory class
|
|
63
|
-
- Mask config for prod-data clone (if applicable)
|
|
64
|
-
- Cleanup hooks for testcontainers / tempdir
|
|
65
|
-
- PII audit report
|
|
66
|
-
|
|
67
|
-
---
|
|
68
|
-
|
|
69
|
-
## <a id="2-derivation"></a>2. Fixture derivation from real schemas
|
|
70
|
-
|
|
71
|
-
**Rule:** fixture generated from source-of-truth schema, not "hand-crafted".
|
|
72
|
-
|
|
73
|
-
**JS/TS — from TypeScript types via faker:**
|
|
74
|
-
```ts
|
|
75
|
-
import { faker } from "@faker-js/faker";
|
|
76
|
-
import type { User } from "@/types/user";
|
|
77
|
-
|
|
78
|
-
export function makeUser(overrides?: Partial<User>): User {
|
|
79
|
-
return {
|
|
80
|
-
id: faker.string.uuid(),
|
|
81
|
-
email: faker.internet.email(),
|
|
82
|
-
name: faker.person.fullName(),
|
|
83
|
-
role: "user",
|
|
84
|
-
createdAt: faker.date.recent(),
|
|
85
|
-
...overrides,
|
|
86
|
-
};
|
|
87
|
-
}
|
|
88
|
-
```
|
|
89
|
-
|
|
90
|
-
**Python — from dataclass via factory_boy:**
|
|
91
|
-
```python
|
|
92
|
-
import factory
|
|
93
|
-
from app.models import User
|
|
94
|
-
|
|
95
|
-
class UserFactory(factory.Factory):
|
|
96
|
-
class Meta:
|
|
97
|
-
model = User
|
|
98
|
-
id = factory.Faker("uuid4")
|
|
99
|
-
email = factory.Faker("email")
|
|
100
|
-
name = factory.Faker("name")
|
|
101
|
-
role = "user"
|
|
102
|
-
```
|
|
103
|
-
|
|
104
|
-
**From OpenAPI / JSON Schema:** use `openapi-typescript-codegen` (TS) or `datamodel-code-generator` (Python) for type derivation, then factory as above.
|
|
105
|
-
|
|
106
|
-
**Why:** schema changes — type changes — factory fails at compile — tests rewritten to new schema. Without this, fixture silently grows stale, tests green, prod fails.
|
|
107
|
-
|
|
108
|
-
---
|
|
109
|
-
|
|
110
|
-
## <a id="3-pii"></a>3. PII hygiene
|
|
111
|
-
|
|
112
|
-
**What must NOT appear in test fixtures (ever):**
|
|
113
|
-
- Real emails / phones / addresses (even your own)
|
|
114
|
-
- Real payment data (CC numbers, IBAN, SWIFT)
|
|
115
|
-
- Real names of real people
|
|
116
|
-
- Real API keys / tokens / passwords
|
|
117
|
-
- Real customer IDs from prod
|
|
118
|
-
|
|
119
|
-
**What to use instead:**
|
|
120
|
-
- **faker** (JS/TS): `faker.internet.email()`, `faker.phone.number()`, `faker.location.streetAddress()`
|
|
121
|
-
- **factory_boy + faker** (Python): `factory.Faker("email")`, `factory.Faker("phone_number")`
|
|
122
|
-
|
|
123
|
-
**Detection:** ESLint rule `no-restricted-syntax` + custom regex on email/phone patterns in test files. Pre-commit hook blocks if anything resembling real PII found (see `$qa-test-integrity-audit` §1).
|
|
124
|
-
|
|
125
|
-
**If PII accidentally committed:**
|
|
126
|
-
1. Rotate compromised credentials immediately
|
|
127
|
-
2. Filter git history (`git filter-repo`) — NOT rebase, so original commits become inaccessible
|
|
128
|
-
3. Notify owners if real customer data was involved
|
|
129
|
-
4. Document incident in security-baseline-dev runbook
|
|
130
|
-
|
|
131
|
-
---
|
|
132
|
-
|
|
133
|
-
## <a id="4-masking"></a>4. Prod-like data masking
|
|
134
|
-
|
|
135
|
-
Sometimes you need testing on realistic distribution (volume, edge cases) — copy prod data into staging/test, but **mask PII**.
|
|
136
|
-
|
|
137
|
-
**Masking rules:**
|
|
138
|
-
- **Emails:** replace local part with hash, keep domain category (`user-a3f9@example.com`)
|
|
139
|
-
- **Names:** replace with faker.person.fullName() seeded by hash of original (consistent within run)
|
|
140
|
-
- **Phones:** randomize last 4 digits keeping country code
|
|
141
|
-
- **Dates:** shift by random Δ ±30 days (preserve weekday patterns)
|
|
142
|
-
- **IDs:** preserve referential integrity through deterministic mapping (`user_123 → user_anon_xyz` consistently)
|
|
143
|
-
- **Payment:** zeroize CC, randomize amounts within ±20% range
|
|
144
|
-
|
|
145
|
-
**Tools:** `pgmasq` (Postgres), `Faker.unique` + custom transformers, in-house masking script. Run AS PART OF prod→staging sync, not in test runtime.
|
|
146
|
-
|
|
147
|
-
**Audit:** after mask sample 100 rows, manual check for no leakage. Document mask config in repo so reviewer audits.
|
|
148
|
-
|
|
149
|
-
---
|
|
150
|
-
|
|
151
|
-
## <a id="5-isolation"></a>5. Environment isolation
|
|
152
|
-
|
|
153
|
-
Every test run must start with **clean state** and **not leave** artifacts for the next.
|
|
154
|
-
|
|
155
|
-
**DB isolation patterns:**
|
|
156
|
-
- **Testcontainers** (Docker-based fresh DB per test suite): `@testcontainers/postgresql` (JS), `testcontainers` (Python). Spin up real Postgres/MongoDB in Docker container, teardown after.
|
|
157
|
-
- **Transactional rollback** (faster than testcontainers): wrap test in transaction, rollback at end. Works for SQL DBs, not for MongoDB.
|
|
158
|
-
- **In-memory adapters** (fastest, but less real): SQLite for Postgres-like, mongo-memory-server.
|
|
159
|
-
|
|
160
|
-
**File system isolation:**
|
|
161
|
-
- `tmp` directories per test (`os.tmpdir()` + uuid)
|
|
162
|
-
- Cleanup in `afterEach` / pytest fixture teardown
|
|
163
|
-
- Do NOT use shared `tests/fixtures/output/` directory
|
|
164
|
-
|
|
165
|
-
**Network isolation:**
|
|
166
|
-
- MSW (JS) / responses (Python) for mock external HTTP at boundary
|
|
167
|
-
- Never hit real API in test — flaky + slow + cost
|
|
168
|
-
|
|
169
|
-
**State between tests:**
|
|
170
|
-
- Module-level state NOT shared (avoid singletons in production code or wipe in `beforeEach`)
|
|
171
|
-
- DB indexes/sequences reset between suites
|
|
172
|
-
|
|
173
|
-
---
|
|
174
|
-
|
|
175
|
-
## <a id="6-lifecycle"></a>6. Fixture lifecycle
|
|
176
|
-
|
|
177
|
-
**Create:**
|
|
178
|
-
- Generation via factory (not hardcode)
|
|
179
|
-
- Minimum data needed for test (no "kitchen sink" fixtures)
|
|
180
|
-
- Specific test → specific fixture builder
|
|
181
|
-
|
|
182
|
-
**Use:**
|
|
183
|
-
- One test → one fixture instance (not shared)
|
|
184
|
-
- If related entities needed — factory composition (`UserFactory.subFactory(OrderFactory)`)
|
|
185
|
-
- Overrides only for test-specific variations
|
|
186
|
-
|
|
187
|
-
**Cleanup:**
|
|
188
|
-
- Testcontainers — auto-teardown on suite exit
|
|
189
|
-
- DB transactional rollback — auto on test exit
|
|
190
|
-
- File system — `afterEach` cleanup hook
|
|
191
|
-
- Tempdir — registered for OS cleanup
|
|
192
|
-
|
|
193
|
-
**Storage convention:**
|
|
194
|
-
- Factory definitions: `tests/factories/` (JS) or `tests/conftest.py` + factory module (Python)
|
|
195
|
-
- Test data files: do NOT commit large fixture JSON files into git; generate on demand
|
|
196
|
-
- Snapshot files OK in git (Playwright traces, vitest snapshots) — but without PII
|
|
197
|
-
|
|
198
|
-
---
|
|
199
|
-
|
|
200
|
-
## <a id="7-anti"></a>7. Anti-patterns
|
|
201
|
-
|
|
202
|
-
- 🔴 **Hardcoded real PII** — `expect(user.email).toBe("
|
|
203
|
-
- 🔴 **Shared mutable fixture** — `beforeAll(() => fixture = makeUser())` + tests mutate fixture — order dependence, flaky.
|
|
204
|
-
- 🔴 **Fixture drift** — handcrafted `{id: 1, name: "test"}` that doesn't use TS type / schema. Schema changes, fixture doesn't fail, prod fails.
|
|
205
|
-
- 🟠 **No cleanup** — testcontainers started, teardown forgotten — CI runner clogged with zombie containers within hours.
|
|
206
|
-
- 🟠 **Kitchen sink fixture** — `makeUserWithEverything()` returns user + orders + payments + sessions — each test gets more than needed, an update breaks all tests.
|
|
207
|
-
- 🟡 **Faker without seed** — randomness good, but without seed cannot reproduce failing test. Pin seed for reproducible builds (`faker.seed(42)` per test).
|
|
208
|
-
|
|
209
|
-
---
|
|
210
|
-
|
|
211
|
-
## <a id="8-output"></a>8. Output template
|
|
212
|
-
|
|
213
|
-
Tester includes in TEST report (when applicable):
|
|
214
|
-
|
|
215
|
-
```
|
|
216
|
-
### Test Data Management
|
|
217
|
-
- Factories used: N (list)
|
|
218
|
-
- PII audit: pass / N findings
|
|
219
|
-
- Masking config: configured / N/A
|
|
220
|
-
- Environment isolation:
|
|
221
|
-
- testcontainers: N suites
|
|
222
|
-
- transactional rollback: N suites
|
|
223
|
-
- tempdir cleanup: configured
|
|
224
|
-
- Fixture seeding: seeded (reproducible) / random
|
|
225
|
-
- Action items: [fixture refactors / PII findings to fix]
|
|
226
|
-
```
|
|
227
|
-
|
|
228
|
-
---
|
|
229
|
-
|
|
230
|
-
## <a id="9-dod"></a>9. DoD
|
|
231
|
-
|
|
232
|
-
- [ ] All fixtures generated via factory (no hardcoded entities)
|
|
233
|
-
- [ ] PII audit pre-commit hook configured (see `$qa-test-integrity-audit`)
|
|
234
|
-
- [ ] Environment isolation: testcontainers or transactional rollback per integration suite
|
|
235
|
-
- [ ] Mask config documented for any prod→staging sync
|
|
236
|
-
- [ ] Cleanup hooks in every test suite (no zombie state)
|
|
237
|
-
- [ ] Faker seed pinned for reproducibility
|
|
238
|
-
- [ ] Output template included in TEST report
|
|
239
|
-
|
|
240
|
-
---
|
|
241
|
-
|
|
242
|
-
## See also
|
|
243
|
-
|
|
244
|
-
- `$qa-test-plan` — test plan accounts for test data requirements per flow
|
|
245
|
-
- `$qa-regression-baseline` — baseline tracks fixture changes between releases
|
|
246
|
-
- `$qa-mutation-testing` — mutation depends strongly on fixture quality
|
|
247
|
-
- `$qa-test-integrity-audit` — PII detection rules + fixture drift detection
|
|
248
|
-
- `$qa-flaky-test-protocol` — env isolation issues → flaky cause
|
|
249
|
-
- `$tests-quality-review` — Reviewer-side audit including test data quality
|
|
250
|
-
- `$security-baseline-dev` — PII handling rules consistent with production code
|
|
1
|
+
---
|
|
2
|
+
name: qa-test-data-management
|
|
3
|
+
description: Test data management — fixtures generated from real schemas (TS types, DB schema, OpenAPI), PII hygiene (faker/factory_boy for synthetics), prod-like masking when copying prod data, environment isolation (testcontainers, transactional rollback, tempdir), fixture lifecycle. Defense against Mode 1 (mock obsession) — fixture does not drift from reality.
|
|
4
|
+
type: discretionary
|
|
5
|
+
domain: development
|
|
6
|
+
owners:
|
|
7
|
+
- tester
|
|
8
|
+
gates:
|
|
9
|
+
- TEST
|
|
10
|
+
tech:
|
|
11
|
+
- docker
|
|
12
|
+
- faker
|
|
13
|
+
- factory_boy
|
|
14
|
+
topic:
|
|
15
|
+
- qa
|
|
16
|
+
triggers:
|
|
17
|
+
- test data management
|
|
18
|
+
- test fixtures
|
|
19
|
+
- PII in tests
|
|
20
|
+
- faker
|
|
21
|
+
- factory_boy
|
|
22
|
+
- testcontainers
|
|
23
|
+
- prod data masking
|
|
24
|
+
- fixture drift
|
|
25
|
+
- environment isolation
|
|
26
|
+
related:
|
|
27
|
+
- qa-test-plan
|
|
28
|
+
- qa-regression-baseline
|
|
29
|
+
- qa-mutation-testing
|
|
30
|
+
- tests-quality-review
|
|
31
|
+
budget_lines: 250
|
|
32
|
+
schema_version: 1
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
|
|
36
|
+
# Skill: QA Test Data Management
|
|
37
|
+
|
|
38
|
+
Test data management so fixtures **do not drift from reality**, **do not contain PII**, and **do not pollute** environment between runs. Defense against Mode 1 of Test Integrity Defense (mock obsession) — because mock obsession often starts with a "hand-crafted" fixture that diverges from real schema.
|
|
39
|
+
|
|
40
|
+
**Sections:**
|
|
41
|
+
1. [When to apply](#1-when)
|
|
42
|
+
2. [Fixture derivation from real schemas](#2-derivation)
|
|
43
|
+
3. [PII hygiene](#3-pii)
|
|
44
|
+
4. [Prod-like data masking](#4-masking)
|
|
45
|
+
5. [Environment isolation](#5-isolation)
|
|
46
|
+
6. [Fixture lifecycle](#6-lifecycle)
|
|
47
|
+
7. [Anti-patterns](#7-anti)
|
|
48
|
+
8. [Output template](#8-output)
|
|
49
|
+
9. [DoD](#9-dod)
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
## <a id="1-when"></a>1. When to apply
|
|
54
|
+
|
|
55
|
+
**Triggers:**
|
|
56
|
+
- First integration test on new module / new DB table / new API endpoint
|
|
57
|
+
- Security alert: PII detected in test fixtures (auto-scan, see `$qa-test-integrity-audit`)
|
|
58
|
+
- DB schema migration — fixtures of all affected tests need re-generation
|
|
59
|
+
- Stabilization sprint includes test data quality audit
|
|
60
|
+
|
|
61
|
+
**Outputs:**
|
|
62
|
+
- Fixture generation script / decorator / factory class
|
|
63
|
+
- Mask config for prod-data clone (if applicable)
|
|
64
|
+
- Cleanup hooks for testcontainers / tempdir
|
|
65
|
+
- PII audit report
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## <a id="2-derivation"></a>2. Fixture derivation from real schemas
|
|
70
|
+
|
|
71
|
+
**Rule:** fixture generated from source-of-truth schema, not "hand-crafted".
|
|
72
|
+
|
|
73
|
+
**JS/TS — from TypeScript types via faker:**
|
|
74
|
+
```ts
|
|
75
|
+
import { faker } from "@faker-js/faker";
|
|
76
|
+
import type { User } from "@/types/user";
|
|
77
|
+
|
|
78
|
+
export function makeUser(overrides?: Partial<User>): User {
|
|
79
|
+
return {
|
|
80
|
+
id: faker.string.uuid(),
|
|
81
|
+
email: faker.internet.email(),
|
|
82
|
+
name: faker.person.fullName(),
|
|
83
|
+
role: "user",
|
|
84
|
+
createdAt: faker.date.recent(),
|
|
85
|
+
...overrides,
|
|
86
|
+
};
|
|
87
|
+
}
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
**Python — from dataclass via factory_boy:**
|
|
91
|
+
```python
|
|
92
|
+
import factory
|
|
93
|
+
from app.models import User
|
|
94
|
+
|
|
95
|
+
class UserFactory(factory.Factory):
|
|
96
|
+
class Meta:
|
|
97
|
+
model = User
|
|
98
|
+
id = factory.Faker("uuid4")
|
|
99
|
+
email = factory.Faker("email")
|
|
100
|
+
name = factory.Faker("name")
|
|
101
|
+
role = "user"
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
**From OpenAPI / JSON Schema:** use `openapi-typescript-codegen` (TS) or `datamodel-code-generator` (Python) for type derivation, then factory as above.
|
|
105
|
+
|
|
106
|
+
**Why:** schema changes — type changes — factory fails at compile — tests rewritten to new schema. Without this, fixture silently grows stale, tests green, prod fails.
|
|
107
|
+
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## <a id="3-pii"></a>3. PII hygiene
|
|
111
|
+
|
|
112
|
+
**What must NOT appear in test fixtures (ever):**
|
|
113
|
+
- Real emails / phones / addresses (even your own)
|
|
114
|
+
- Real payment data (CC numbers, IBAN, SWIFT)
|
|
115
|
+
- Real names of real people
|
|
116
|
+
- Real API keys / tokens / passwords
|
|
117
|
+
- Real customer IDs from prod
|
|
118
|
+
|
|
119
|
+
**What to use instead:**
|
|
120
|
+
- **faker** (JS/TS): `faker.internet.email()`, `faker.phone.number()`, `faker.location.streetAddress()`
|
|
121
|
+
- **factory_boy + faker** (Python): `factory.Faker("email")`, `factory.Faker("phone_number")`
|
|
122
|
+
|
|
123
|
+
**Detection:** ESLint rule `no-restricted-syntax` + custom regex on email/phone patterns in test files. Pre-commit hook blocks if anything resembling real PII found (see `$qa-test-integrity-audit` §1).
|
|
124
|
+
|
|
125
|
+
**If PII accidentally committed:**
|
|
126
|
+
1. Rotate compromised credentials immediately
|
|
127
|
+
2. Filter git history (`git filter-repo`) — NOT rebase, so original commits become inaccessible
|
|
128
|
+
3. Notify owners if real customer data was involved
|
|
129
|
+
4. Document incident in security-baseline-dev runbook
|
|
130
|
+
|
|
131
|
+
---
|
|
132
|
+
|
|
133
|
+
## <a id="4-masking"></a>4. Prod-like data masking
|
|
134
|
+
|
|
135
|
+
Sometimes you need testing on realistic distribution (volume, edge cases) — copy prod data into staging/test, but **mask PII**.
|
|
136
|
+
|
|
137
|
+
**Masking rules:**
|
|
138
|
+
- **Emails:** replace local part with hash, keep domain category (`user-a3f9@example.com`)
|
|
139
|
+
- **Names:** replace with faker.person.fullName() seeded by hash of original (consistent within run)
|
|
140
|
+
- **Phones:** randomize last 4 digits keeping country code
|
|
141
|
+
- **Dates:** shift by random Δ ±30 days (preserve weekday patterns)
|
|
142
|
+
- **IDs:** preserve referential integrity through deterministic mapping (`user_123 → user_anon_xyz` consistently)
|
|
143
|
+
- **Payment:** zeroize CC, randomize amounts within ±20% range
|
|
144
|
+
|
|
145
|
+
**Tools:** `pgmasq` (Postgres), `Faker.unique` + custom transformers, in-house masking script. Run AS PART OF prod→staging sync, not in test runtime.
|
|
146
|
+
|
|
147
|
+
**Audit:** after mask sample 100 rows, manual check for no leakage. Document mask config in repo so reviewer audits.
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
## <a id="5-isolation"></a>5. Environment isolation
|
|
152
|
+
|
|
153
|
+
Every test run must start with **clean state** and **not leave** artifacts for the next.
|
|
154
|
+
|
|
155
|
+
**DB isolation patterns:**
|
|
156
|
+
- **Testcontainers** (Docker-based fresh DB per test suite): `@testcontainers/postgresql` (JS), `testcontainers` (Python). Spin up real Postgres/MongoDB in Docker container, teardown after.
|
|
157
|
+
- **Transactional rollback** (faster than testcontainers): wrap test in transaction, rollback at end. Works for SQL DBs, not for MongoDB.
|
|
158
|
+
- **In-memory adapters** (fastest, but less real): SQLite for Postgres-like, mongo-memory-server.
|
|
159
|
+
|
|
160
|
+
**File system isolation:**
|
|
161
|
+
- `tmp` directories per test (`os.tmpdir()` + uuid)
|
|
162
|
+
- Cleanup in `afterEach` / pytest fixture teardown
|
|
163
|
+
- Do NOT use shared `tests/fixtures/output/` directory
|
|
164
|
+
|
|
165
|
+
**Network isolation:**
|
|
166
|
+
- MSW (JS) / responses (Python) for mock external HTTP at boundary
|
|
167
|
+
- Never hit real API in test — flaky + slow + cost
|
|
168
|
+
|
|
169
|
+
**State between tests:**
|
|
170
|
+
- Module-level state NOT shared (avoid singletons in production code or wipe in `beforeEach`)
|
|
171
|
+
- DB indexes/sequences reset between suites
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
## <a id="6-lifecycle"></a>6. Fixture lifecycle
|
|
176
|
+
|
|
177
|
+
**Create:**
|
|
178
|
+
- Generation via factory (not hardcode)
|
|
179
|
+
- Minimum data needed for test (no "kitchen sink" fixtures)
|
|
180
|
+
- Specific test → specific fixture builder
|
|
181
|
+
|
|
182
|
+
**Use:**
|
|
183
|
+
- One test → one fixture instance (not shared)
|
|
184
|
+
- If related entities needed — factory composition (`UserFactory.subFactory(OrderFactory)`)
|
|
185
|
+
- Overrides only for test-specific variations
|
|
186
|
+
|
|
187
|
+
**Cleanup:**
|
|
188
|
+
- Testcontainers — auto-teardown on suite exit
|
|
189
|
+
- DB transactional rollback — auto on test exit
|
|
190
|
+
- File system — `afterEach` cleanup hook
|
|
191
|
+
- Tempdir — registered for OS cleanup
|
|
192
|
+
|
|
193
|
+
**Storage convention:**
|
|
194
|
+
- Factory definitions: `tests/factories/` (JS) or `tests/conftest.py` + factory module (Python)
|
|
195
|
+
- Test data files: do NOT commit large fixture JSON files into git; generate on demand
|
|
196
|
+
- Snapshot files OK in git (Playwright traces, vitest snapshots) — but without PII
|
|
197
|
+
|
|
198
|
+
---
|
|
199
|
+
|
|
200
|
+
## <a id="7-anti"></a>7. Anti-patterns
|
|
201
|
+
|
|
202
|
+
- 🔴 **Hardcoded real PII** — `expect(user.email).toBe("user@example.com")` — security leak + flaky when the user changes email.
|
|
203
|
+
- 🔴 **Shared mutable fixture** — `beforeAll(() => fixture = makeUser())` + tests mutate fixture — order dependence, flaky.
|
|
204
|
+
- 🔴 **Fixture drift** — handcrafted `{id: 1, name: "test"}` that doesn't use TS type / schema. Schema changes, fixture doesn't fail, prod fails.
|
|
205
|
+
- 🟠 **No cleanup** — testcontainers started, teardown forgotten — CI runner clogged with zombie containers within hours.
|
|
206
|
+
- 🟠 **Kitchen sink fixture** — `makeUserWithEverything()` returns user + orders + payments + sessions — each test gets more than needed, an update breaks all tests.
|
|
207
|
+
- 🟡 **Faker without seed** — randomness good, but without seed cannot reproduce failing test. Pin seed for reproducible builds (`faker.seed(42)` per test).
|
|
208
|
+
|
|
209
|
+
---
|
|
210
|
+
|
|
211
|
+
## <a id="8-output"></a>8. Output template
|
|
212
|
+
|
|
213
|
+
Tester includes in TEST report (when applicable):
|
|
214
|
+
|
|
215
|
+
```
|
|
216
|
+
### Test Data Management
|
|
217
|
+
- Factories used: N (list)
|
|
218
|
+
- PII audit: pass / N findings
|
|
219
|
+
- Masking config: configured / N/A
|
|
220
|
+
- Environment isolation:
|
|
221
|
+
- testcontainers: N suites
|
|
222
|
+
- transactional rollback: N suites
|
|
223
|
+
- tempdir cleanup: configured
|
|
224
|
+
- Fixture seeding: seeded (reproducible) / random
|
|
225
|
+
- Action items: [fixture refactors / PII findings to fix]
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
---
|
|
229
|
+
|
|
230
|
+
## <a id="9-dod"></a>9. DoD
|
|
231
|
+
|
|
232
|
+
- [ ] All fixtures generated via factory (no hardcoded entities)
|
|
233
|
+
- [ ] PII audit pre-commit hook configured (see `$qa-test-integrity-audit`)
|
|
234
|
+
- [ ] Environment isolation: testcontainers or transactional rollback per integration suite
|
|
235
|
+
- [ ] Mask config documented for any prod→staging sync
|
|
236
|
+
- [ ] Cleanup hooks in every test suite (no zombie state)
|
|
237
|
+
- [ ] Faker seed pinned for reproducibility
|
|
238
|
+
- [ ] Output template included in TEST report
|
|
239
|
+
|
|
240
|
+
---
|
|
241
|
+
|
|
242
|
+
## See also
|
|
243
|
+
|
|
244
|
+
- `$qa-test-plan` — test plan accounts for test data requirements per flow
|
|
245
|
+
- `$qa-regression-baseline` — baseline tracks fixture changes between releases
|
|
246
|
+
- `$qa-mutation-testing` — mutation depends strongly on fixture quality
|
|
247
|
+
- `$qa-test-integrity-audit` — PII detection rules + fixture drift detection
|
|
248
|
+
- `$qa-flaky-test-protocol` — env isolation issues → flaky cause
|
|
249
|
+
- `$tests-quality-review` — Reviewer-side audit including test data quality
|
|
250
|
+
- `$security-baseline-dev` — PII handling rules consistent with production code
|
|
@@ -1,90 +1,24 @@
|
|
|
1
1
|
---
|
|
2
|
-
description: Launch the shortened
|
|
2
|
+
description: Launch the shortened bugfix pipeline (3 gates: DEV → REV → TEST).
|
|
3
3
|
---
|
|
4
4
|
|
|
5
|
-
# /bugfix —
|
|
5
|
+
# /bugfix — bugfix pipeline (3 gates)
|
|
6
6
|
|
|
7
|
-
> 🟢
|
|
8
|
-
>
|
|
7
|
+
> 🟢 For bugs in existing functionality. Full mode — `/start-task`, trivial — `/hotfix`.
|
|
8
|
+
> Absolute rules — in `pipeline-rules`.
|
|
9
9
|
|
|
10
|
-
## When
|
|
10
|
+
## When
|
|
11
|
+
- Bug in logic, API errors, broken validation, data issues.
|
|
12
|
+
- Affects > 2 files or a non-trivial blast radius.
|
|
13
|
+
- Does NOT change UI layout, add an API, or change the data schema (otherwise → `/start-task`).
|
|
11
14
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
+
## Flow
|
|
16
|
+
The Conductor confirms the mode (bugfix) and initializes the board. Then, gate by gate via the MCP flow (`classify_gate → load_role → submit_artifact → sign_off → advance_gate`):
|
|
17
|
+
1. **DEV** — TDD fix: RED (a test reproducing the bug) → GREEN → REFACTOR; JSDoc. Signed `mcp`.
|
|
18
|
+
2. **REV** — review: does the test actually reproduce the bug? side effects? regression risk? Signed `mcp`.
|
|
19
|
+
3. **TEST** — verification: bug fixed, no regressions; auto-pass on a green `run_tests`. Signed `mcp`.
|
|
15
20
|
|
|
16
|
-
|
|
21
|
+
The user is not involved in the routine — only on a `fork` via `request_decision`.
|
|
17
22
|
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
## Step 1: CONDUCTOR — Classification
|
|
21
|
-
|
|
22
|
-
1. Execute `view_file` on `agents/conductor.md`
|
|
23
|
-
2. Confirm the task = bugfix (per Decision Tree)
|
|
24
|
-
3. Create Mini Checklist:
|
|
25
|
-
```
|
|
26
|
-
[ ] BF-DEV-01 Fix + TDD + Handoff Envelope
|
|
27
|
-
[ ] BF-REV-01 Code review + regression check + Handoff Envelope
|
|
28
|
-
[ ] BF-TEST-01 Verification + regression smoke + GO/NO-GO
|
|
29
|
-
```
|
|
30
|
-
4. `notify_user` → wait for **Approved**
|
|
31
|
-
|
|
32
|
-
## Step 2: DEV — TDD Fix
|
|
33
|
-
|
|
34
|
-
1. Execute `view_file` on `agents/senior_full_stack.md`
|
|
35
|
-
2. Follow the protocol (skipping §1 UX review and §6 DEMO):
|
|
36
|
-
- §0: Context + read skills
|
|
37
|
-
- §2: Analyze bug + root cause
|
|
38
|
-
- §3: RED — write a failing test reproducing the bug
|
|
39
|
-
- §4: GREEN — minimal code to make the test pass
|
|
40
|
-
- §5: REFACTOR — improve without changing behavior
|
|
41
|
-
- §7: JSDoc on modified functions
|
|
42
|
-
3. Restart affected services if applicable
|
|
43
|
-
4. Form Handoff Envelope → REV
|
|
44
|
-
5. `notify_user` → wait for **Approved**
|
|
45
|
-
|
|
46
|
-
## Step 3: REV — Code Review
|
|
47
|
-
|
|
48
|
-
1. Execute `view_file` on `agents/reviewer.md`
|
|
49
|
-
2. Review focus:
|
|
50
|
-
- Does the test actually reproduce the bug (RED phase)?
|
|
51
|
-
- Does the fix create side effects?
|
|
52
|
-
- Is regression risk assessed?
|
|
53
|
-
- Is JSDoc in place?
|
|
54
|
-
3. Form Handoff Envelope → TEST
|
|
55
|
-
4. `notify_user` → wait for **Approved**
|
|
56
|
-
|
|
57
|
-
## Step 4: TEST — Verification
|
|
58
|
-
|
|
59
|
-
1. Execute `view_file` on `agents/tester.md`
|
|
60
|
-
2. Verify:
|
|
61
|
-
- Bug is fixed (per reproduction steps)
|
|
62
|
-
- No regression (smoke on affected modules)
|
|
63
|
-
- Tests pass (CI green)
|
|
64
|
-
3. Issue verdict: **GO ✅ / NO-GO ❌**
|
|
65
|
-
4. `notify_user` → wait for **Approved**
|
|
66
|
-
|
|
67
|
-
---
|
|
68
|
-
|
|
69
|
-
## On FAIL at REV or TEST
|
|
70
|
-
|
|
71
|
-
1. Agent produces FAIL Report + Handoff → DEV
|
|
72
|
-
2. DEV fixes → Handoff → REV → TEST (cycle repeats)
|
|
73
|
-
3. Approved is required at every gate
|
|
74
|
-
|
|
75
|
-
---
|
|
76
|
-
|
|
77
|
-
## Prompt template
|
|
78
|
-
|
|
79
|
-
```
|
|
80
|
-
@AGENTS.md /bugfix
|
|
81
|
-
|
|
82
|
-
Bug: [bug description, 1-2 sentences].
|
|
83
|
-
Reproduction: [steps, if known].
|
|
84
|
-
Files: [affected files, if known].
|
|
85
|
-
|
|
86
|
-
Rules:
|
|
87
|
-
1. Bugfix Pipeline: CONDUCTOR → DEV → REV → TEST
|
|
88
|
-
2. TDD is mandatory (RED → GREEN → REFACTOR)
|
|
89
|
-
3. Approved at every gate
|
|
90
|
-
```
|
|
23
|
+
## Fix Cycle
|
|
24
|
+
A FAIL at REV/TEST or a red `run_tests` → rollback to DEV. 2 rollbacks across the band → an architect deep audit in a fresh session (`pipeline-rules`).
|