@qball-inc/the-bulwark 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +43 -0
- package/agents/bulwark-fix-validator.md +633 -0
- package/agents/bulwark-implementer.md +391 -0
- package/agents/bulwark-issue-analyzer.md +308 -0
- package/agents/bulwark-standards-reviewer.md +221 -0
- package/agents/plan-creation-architect.md +323 -0
- package/agents/plan-creation-eng-lead.md +352 -0
- package/agents/plan-creation-po.md +300 -0
- package/agents/plan-creation-qa-critic.md +334 -0
- package/agents/product-ideation-competitive-analyzer.md +298 -0
- package/agents/product-ideation-idea-validator.md +268 -0
- package/agents/product-ideation-market-researcher.md +292 -0
- package/agents/product-ideation-pattern-documenter.md +308 -0
- package/agents/product-ideation-segment-analyzer.md +303 -0
- package/agents/product-ideation-strategist.md +259 -0
- package/agents/statusline-setup.md +97 -0
- package/hooks/hooks.json +59 -0
- package/package.json +45 -0
- package/scripts/hooks/cleanup-stale.sh +13 -0
- package/scripts/hooks/enforce-quality.sh +166 -0
- package/scripts/hooks/implementer-quality.sh +256 -0
- package/scripts/hooks/inject-protocol.sh +52 -0
- package/scripts/hooks/suggest-pipeline.sh +175 -0
- package/scripts/hooks/track-pipeline-start.sh +37 -0
- package/scripts/hooks/track-pipeline-stop.sh +52 -0
- package/scripts/init-rules.sh +35 -0
- package/scripts/init.sh +151 -0
- package/skills/anthropic-validator/SKILL.md +607 -0
- package/skills/anthropic-validator/references/agents-checklist.md +131 -0
- package/skills/anthropic-validator/references/commands-checklist.md +102 -0
- package/skills/anthropic-validator/references/hooks-checklist.md +151 -0
- package/skills/anthropic-validator/references/mcp-checklist.md +136 -0
- package/skills/anthropic-validator/references/plugins-checklist.md +148 -0
- package/skills/anthropic-validator/references/skills-checklist.md +85 -0
- package/skills/assertion-patterns/SKILL.md +296 -0
- package/skills/bug-magnet-data/SKILL.md +284 -0
- package/skills/bug-magnet-data/context/cli-args.md +91 -0
- package/skills/bug-magnet-data/context/db-query.md +104 -0
- package/skills/bug-magnet-data/context/file-contents.md +103 -0
- package/skills/bug-magnet-data/context/http-body.md +91 -0
- package/skills/bug-magnet-data/context/process-spawn.md +123 -0
- package/skills/bug-magnet-data/data/booleans/boundaries.yaml +143 -0
- package/skills/bug-magnet-data/data/collections/arrays.yaml +114 -0
- package/skills/bug-magnet-data/data/collections/objects.yaml +123 -0
- package/skills/bug-magnet-data/data/concurrency/race-conditions.yaml +118 -0
- package/skills/bug-magnet-data/data/concurrency/state-machines.yaml +115 -0
- package/skills/bug-magnet-data/data/dates/boundaries.yaml +137 -0
- package/skills/bug-magnet-data/data/dates/invalid.yaml +132 -0
- package/skills/bug-magnet-data/data/dates/timezone.yaml +118 -0
- package/skills/bug-magnet-data/data/encoding/charset.yaml +79 -0
- package/skills/bug-magnet-data/data/encoding/normalization.yaml +105 -0
- package/skills/bug-magnet-data/data/formats/email.yaml +154 -0
- package/skills/bug-magnet-data/data/formats/json.yaml +187 -0
- package/skills/bug-magnet-data/data/formats/url.yaml +165 -0
- package/skills/bug-magnet-data/data/language-specific/javascript.yaml +182 -0
- package/skills/bug-magnet-data/data/language-specific/python.yaml +174 -0
- package/skills/bug-magnet-data/data/language-specific/rust.yaml +148 -0
- package/skills/bug-magnet-data/data/numbers/boundaries.yaml +161 -0
- package/skills/bug-magnet-data/data/numbers/precision.yaml +89 -0
- package/skills/bug-magnet-data/data/numbers/special.yaml +69 -0
- package/skills/bug-magnet-data/data/strings/boundaries.yaml +109 -0
- package/skills/bug-magnet-data/data/strings/injection.yaml +208 -0
- package/skills/bug-magnet-data/data/strings/special-chars.yaml +190 -0
- package/skills/bug-magnet-data/data/strings/unicode.yaml +139 -0
- package/skills/bug-magnet-data/references/external-lists.md +115 -0
- package/skills/bulwark-brainstorm/SKILL.md +563 -0
- package/skills/bulwark-brainstorm/references/at-teammate-prompts.md +60 -0
- package/skills/bulwark-brainstorm/references/role-critical-analyst.md +78 -0
- package/skills/bulwark-brainstorm/references/role-development-lead.md +66 -0
- package/skills/bulwark-brainstorm/references/role-product-delivery-lead.md +79 -0
- package/skills/bulwark-brainstorm/references/role-product-manager.md +62 -0
- package/skills/bulwark-brainstorm/references/role-project-sme.md +59 -0
- package/skills/bulwark-brainstorm/references/role-technical-architect.md +66 -0
- package/skills/bulwark-research/SKILL.md +298 -0
- package/skills/bulwark-research/references/viewpoint-contrarian.md +63 -0
- package/skills/bulwark-research/references/viewpoint-direct-investigation.md +62 -0
- package/skills/bulwark-research/references/viewpoint-first-principles.md +65 -0
- package/skills/bulwark-research/references/viewpoint-practitioner.md +62 -0
- package/skills/bulwark-research/references/viewpoint-prior-art.md +66 -0
- package/skills/bulwark-scaffold/SKILL.md +330 -0
- package/skills/bulwark-statusline/SKILL.md +161 -0
- package/skills/bulwark-statusline/scripts/statusline.sh +144 -0
- package/skills/bulwark-verify/SKILL.md +519 -0
- package/skills/code-review/SKILL.md +428 -0
- package/skills/code-review/examples/anti-patterns/linting.ts +181 -0
- package/skills/code-review/examples/anti-patterns/security.ts +91 -0
- package/skills/code-review/examples/anti-patterns/standards.ts +195 -0
- package/skills/code-review/examples/anti-patterns/type-safety.ts +108 -0
- package/skills/code-review/examples/recommended/linting.ts +195 -0
- package/skills/code-review/examples/recommended/security.ts +154 -0
- package/skills/code-review/examples/recommended/standards.ts +231 -0
- package/skills/code-review/examples/recommended/type-safety.ts +181 -0
- package/skills/code-review/frameworks/angular.md +218 -0
- package/skills/code-review/frameworks/django.md +235 -0
- package/skills/code-review/frameworks/express.md +207 -0
- package/skills/code-review/frameworks/flask.md +298 -0
- package/skills/code-review/frameworks/generic.md +146 -0
- package/skills/code-review/frameworks/react.md +152 -0
- package/skills/code-review/frameworks/vue.md +244 -0
- package/skills/code-review/references/linting-patterns.md +221 -0
- package/skills/code-review/references/security-patterns.md +125 -0
- package/skills/code-review/references/standards-patterns.md +246 -0
- package/skills/code-review/references/type-safety-patterns.md +130 -0
- package/skills/component-patterns/SKILL.md +131 -0
- package/skills/component-patterns/references/pattern-cli-command.md +118 -0
- package/skills/component-patterns/references/pattern-database.md +166 -0
- package/skills/component-patterns/references/pattern-external-api.md +139 -0
- package/skills/component-patterns/references/pattern-file-parser.md +168 -0
- package/skills/component-patterns/references/pattern-http-server.md +162 -0
- package/skills/component-patterns/references/pattern-process-spawner.md +133 -0
- package/skills/continuous-feedback/SKILL.md +327 -0
- package/skills/continuous-feedback/references/collect-instructions.md +81 -0
- package/skills/continuous-feedback/references/specialize-code-review.md +82 -0
- package/skills/continuous-feedback/references/specialize-general.md +98 -0
- package/skills/continuous-feedback/references/specialize-test-audit.md +81 -0
- package/skills/create-skill/SKILL.md +359 -0
- package/skills/create-skill/references/agent-conventions.md +194 -0
- package/skills/create-skill/references/agent-template.md +195 -0
- package/skills/create-skill/references/content-guidance.md +291 -0
- package/skills/create-skill/references/decision-framework.md +124 -0
- package/skills/create-skill/references/template-pipeline.md +217 -0
- package/skills/create-skill/references/template-reference-heavy.md +111 -0
- package/skills/create-skill/references/template-research.md +210 -0
- package/skills/create-skill/references/template-script-driven.md +172 -0
- package/skills/create-skill/references/template-simple.md +80 -0
- package/skills/create-subagent/SKILL.md +353 -0
- package/skills/create-subagent/references/agent-conventions.md +268 -0
- package/skills/create-subagent/references/content-guidance.md +232 -0
- package/skills/create-subagent/references/decision-framework.md +134 -0
- package/skills/create-subagent/references/template-single-agent.md +192 -0
- package/skills/fix-bug/SKILL.md +241 -0
- package/skills/governance-protocol/SKILL.md +116 -0
- package/skills/init/SKILL.md +341 -0
- package/skills/issue-debugging/SKILL.md +385 -0
- package/skills/issue-debugging/references/anti-patterns.md +245 -0
- package/skills/issue-debugging/references/debug-report-schema.md +227 -0
- package/skills/mock-detection/SKILL.md +511 -0
- package/skills/mock-detection/references/false-positive-prevention.md +402 -0
- package/skills/mock-detection/references/stub-patterns.md +236 -0
- package/skills/pipeline-templates/SKILL.md +215 -0
- package/skills/pipeline-templates/references/code-change-workflow.md +277 -0
- package/skills/pipeline-templates/references/code-review.md +336 -0
- package/skills/pipeline-templates/references/fix-validation.md +421 -0
- package/skills/pipeline-templates/references/new-feature.md +335 -0
- package/skills/pipeline-templates/references/research-brainstorm.md +161 -0
- package/skills/pipeline-templates/references/research-planning.md +257 -0
- package/skills/pipeline-templates/references/test-audit.md +389 -0
- package/skills/pipeline-templates/references/test-execution-fix.md +238 -0
- package/skills/plan-creation/SKILL.md +497 -0
- package/skills/product-ideation/SKILL.md +372 -0
- package/skills/product-ideation/references/analysis-frameworks.md +161 -0
- package/skills/session-handoff/SKILL.md +139 -0
- package/skills/session-handoff/references/examples.md +223 -0
- package/skills/setup-lsp/SKILL.md +312 -0
- package/skills/setup-lsp/references/server-registry.md +85 -0
- package/skills/setup-lsp/references/troubleshooting.md +135 -0
- package/skills/subagent-output-templating/SKILL.md +415 -0
- package/skills/subagent-output-templating/references/examples.md +440 -0
- package/skills/subagent-prompting/SKILL.md +364 -0
- package/skills/subagent-prompting/references/examples.md +342 -0
- package/skills/test-audit/SKILL.md +531 -0
- package/skills/test-audit/references/known-limitations.md +41 -0
- package/skills/test-audit/references/priority-classification.md +30 -0
- package/skills/test-audit/references/prompts/deep-mode-detection.md +83 -0
- package/skills/test-audit/references/prompts/synthesis.md +57 -0
- package/skills/test-audit/references/rewrite-instructions.md +46 -0
- package/skills/test-audit/references/schemas/audit-output.yaml +100 -0
- package/skills/test-audit/references/schemas/diagnostic-output.yaml +49 -0
- package/skills/test-audit/scripts/data-flow-analyzer.ts +509 -0
- package/skills/test-audit/scripts/integration-mock-detector.ts +462 -0
- package/skills/test-audit/scripts/package.json +20 -0
- package/skills/test-audit/scripts/skip-detector.ts +211 -0
- package/skills/test-audit/scripts/verification-counter.ts +295 -0
- package/skills/test-classification/SKILL.md +310 -0
- package/skills/test-fixture-creation/SKILL.md +295 -0
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Skills Validation Checklist (Fallback)
|
|
2
|
+
|
|
3
|
+
This checklist is used when dynamic documentation fetch fails. May be outdated - prefer fetched standards.
|
|
4
|
+
|
|
5
|
+
**Last Updated**: 2026-01-17
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Frontmatter Requirements
|
|
10
|
+
|
|
11
|
+
### Required Fields
|
|
12
|
+
|
|
13
|
+
| Field | Type | Description |
|
|
14
|
+
|-------|------|-------------|
|
|
15
|
+
| `name` | string | Skill name, must match directory name |
|
|
16
|
+
| `description` | string | Concise explanation of skill purpose |
|
|
17
|
+
|
|
18
|
+
### Optional Fields
|
|
19
|
+
|
|
20
|
+
| Field | Type | Description |
|
|
21
|
+
|-------|------|-------------|
|
|
22
|
+
| `user-invocable` | boolean | `true` to show in `/` menu, `false` to hide |
|
|
23
|
+
| `agent` | string | Model selection: `haiku`, `sonnet`, `opus` |
|
|
24
|
+
| `context` | string | `fork` for isolated execution |
|
|
25
|
+
| `skills` | array | Skills to load when this skill runs |
|
|
26
|
+
| `tools` | array | Tools allowed for this skill |
|
|
27
|
+
| `hooks` | object | Skill-scoped hooks |
|
|
28
|
+
|
|
29
|
+
---
|
|
30
|
+
|
|
31
|
+
## File Structure
|
|
32
|
+
|
|
33
|
+
### Required
|
|
34
|
+
|
|
35
|
+
- `SKILL.md` in `skills/{skill-name}/` directory
|
|
36
|
+
- Name in frontmatter matches directory name
|
|
37
|
+
|
|
38
|
+
### Optional
|
|
39
|
+
|
|
40
|
+
- `references/` subdirectory for supporting files
|
|
41
|
+
- Additional markdown files for sections
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Content Guidelines
|
|
46
|
+
|
|
47
|
+
### Critical Rules
|
|
48
|
+
|
|
49
|
+
- [ ] Frontmatter is valid YAML between `---` markers
|
|
50
|
+
- [ ] `name` field matches directory name exactly
|
|
51
|
+
- [ ] `description` field is present and non-empty
|
|
52
|
+
- [ ] SKILL.md is under 500 lines (recommended)
|
|
53
|
+
|
|
54
|
+
### High Priority
|
|
55
|
+
|
|
56
|
+
- [ ] `user-invocable` is boolean if present
|
|
57
|
+
- [ ] `agent` is one of: `haiku`, `sonnet`, `opus` if present
|
|
58
|
+
- [ ] `context` is `fork` if present (no other values)
|
|
59
|
+
- [ ] `skills` is array of strings if present
|
|
60
|
+
- [ ] `tools` is array of valid tool names if present
|
|
61
|
+
|
|
62
|
+
### Medium Priority
|
|
63
|
+
|
|
64
|
+
- [ ] Description explains when to use the skill
|
|
65
|
+
- [ ] Clear section structure with headers
|
|
66
|
+
- [ ] Examples provided where appropriate
|
|
67
|
+
|
|
68
|
+
### Low Priority
|
|
69
|
+
|
|
70
|
+
- [ ] Consistent formatting
|
|
71
|
+
- [ ] No dead links in references
|
|
72
|
+
- [ ] Related skills section included
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## Common Violations
|
|
77
|
+
|
|
78
|
+
| Violation | Severity | Remediation |
|
|
79
|
+
|-----------|----------|-------------|
|
|
80
|
+
| Missing `name` | Critical | Add `name: skill-name` to frontmatter |
|
|
81
|
+
| Missing `description` | Critical | Add `description: ...` to frontmatter |
|
|
82
|
+
| Name mismatch | Critical | Ensure name matches directory |
|
|
83
|
+
| Invalid `agent` value | High | Use `haiku`, `sonnet`, or `opus` |
|
|
84
|
+
| Non-boolean `user-invocable` | High | Use `true` or `false` |
|
|
85
|
+
| Missing frontmatter | Critical | Add `---` markers with YAML |
|
|
@@ -0,0 +1,296 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: assertion-patterns
|
|
3
|
+
description: Real output verification vs mock calls. Use when transforming T1-T4 violating tests to verify observable behavior.
|
|
4
|
+
user-invocable: false
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Assertion Patterns
|
|
8
|
+
|
|
9
|
+
## Purpose
|
|
10
|
+
|
|
11
|
+
Transform T1-T4 violating assertions into real behavior verification. This skill provides
|
|
12
|
+
pattern libraries for converting mock-based tests to tests that verify observable output.
|
|
13
|
+
|
|
14
|
+
## When to Use
|
|
15
|
+
|
|
16
|
+
Load this skill when:
|
|
17
|
+
- Rewriting tests flagged by test-audit
|
|
18
|
+
- Generating verification scripts via bulwark-verify skill
|
|
19
|
+
- Implementing test-audit Step 7 rewrites
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Prerequisite Checks (T0) - CRITICAL
|
|
24
|
+
|
|
25
|
+
**Before checking T1-T4 violations, verify these prerequisites. Tests failing these checks
|
|
26
|
+
are "testing nothing real" - they pass but provide zero confidence.**
|
|
27
|
+
|
|
28
|
+
### T0.1: Production Module Imports
|
|
29
|
+
|
|
30
|
+
Test files MUST import functions from actual production modules.
|
|
31
|
+
|
|
32
|
+
| Valid | Invalid |
|
|
33
|
+
|-------|---------|
|
|
34
|
+
| `import { calculate } from '../src/calculator'` | Function defined within test file |
|
|
35
|
+
| `from calculator import add` | `def add(a, b): return a + b` in test |
|
|
36
|
+
|
|
37
|
+
**Detection:**
|
|
38
|
+
- Scan test file for function definitions
|
|
39
|
+
- Check if tested functions are imported vs defined inline
|
|
40
|
+
- Flag if test calls functions not imported from production code
|
|
41
|
+
|
|
42
|
+
**Violation Response:**
|
|
43
|
+
> "Test defines production logic inline. Move `{function_name}` to production module
|
|
44
|
+
> and import it. Tests should verify production code, not self-defined code."
|
|
45
|
+
|
|
46
|
+
### T0.2: Separation of Concerns
|
|
47
|
+
|
|
48
|
+
Test files MUST NOT contain functions representing production logic.
|
|
49
|
+
|
|
50
|
+
**Allowed in test files:**
|
|
51
|
+
- Test functions (`test_*`, `it()`, `describe()`)
|
|
52
|
+
- Test helpers/fixtures (`make_*`, `create_*`, `setup_*`)
|
|
53
|
+
- Mock factories (`mock_*`, `fake_*`, `stub_*`)
|
|
54
|
+
- Pytest fixtures (`@pytest.fixture`)
|
|
55
|
+
|
|
56
|
+
**Not allowed:**
|
|
57
|
+
- Business logic functions
|
|
58
|
+
- Utility functions that should be in production
|
|
59
|
+
- Any function that would make sense in `src/`
|
|
60
|
+
|
|
61
|
+
**Detection:**
|
|
62
|
+
- Identify all function definitions in test file
|
|
63
|
+
- Check naming patterns against allowed prefixes
|
|
64
|
+
- Flag functions that don't match test/helper patterns
|
|
65
|
+
|
|
66
|
+
### T0.3: Function Naming Conventions
|
|
67
|
+
|
|
68
|
+
Functions in test files should follow specific naming patterns.
|
|
69
|
+
|
|
70
|
+
| Legitimate Prefixes | Purpose |
|
|
71
|
+
|---------------------|---------|
|
|
72
|
+
| `test_*` | Test functions (pytest) |
|
|
73
|
+
| `_*` (underscore) | Private helpers |
|
|
74
|
+
| `pytest_*` | Pytest hooks |
|
|
75
|
+
| `make_*`, `create_*` | Factory helpers |
|
|
76
|
+
| `mock_*`, `fake_*`, `stub_*` | Mock factories |
|
|
77
|
+
| `setup_*`, `teardown_*` | Lifecycle helpers |
|
|
78
|
+
|
|
79
|
+
**Any other function definition raises suspicion** - likely production code incorrectly
|
|
80
|
+
placed in test files.
|
|
81
|
+
|
|
82
|
+
---
|
|
83
|
+
|
|
84
|
+
## Pattern Categories
|
|
85
|
+
|
|
86
|
+
### 1. Function Call Verification
|
|
87
|
+
|
|
88
|
+
| Anti-Pattern (Mock) | Real Pattern |
|
|
89
|
+
|---------------------|--------------|
|
|
90
|
+
| `expect(fn).toHaveBeenCalled()` | `const result = fn(); expect(result).toBe(expected)` |
|
|
91
|
+
| `expect(fn).toHaveBeenCalledWith(arg)` | `const result = fn(arg); expect(result.field).toBeDefined()` |
|
|
92
|
+
| `jest.spyOn(module, 'fn').mockReturnValue(x)` | `const result = module.fn(); expect(result).toBe(expected)` |
|
|
93
|
+
|
|
94
|
+
**Transformation example:**
|
|
95
|
+
```javascript
|
|
96
|
+
// BEFORE (T1 violation - mocking system under test)
|
|
97
|
+
jest.spyOn(calculator, 'add').mockReturnValue(5);
|
|
98
|
+
expect(calculator.add(2, 3)).toBe(5);
|
|
99
|
+
|
|
100
|
+
// AFTER (real verification)
|
|
101
|
+
expect(calculator.add(2, 3)).toBe(5); // Actually runs add()
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
### 2. Process Spawn Verification
|
|
105
|
+
|
|
106
|
+
| Anti-Pattern (Mock) | Real Pattern |
|
|
107
|
+
|---------------------|--------------|
|
|
108
|
+
| `jest.spyOn(cp, 'spawn').mockReturnValue(mockProc)` | `const proc = spawn(...); await waitForReady(proc)` |
|
|
109
|
+
| `expect(spawn).toHaveBeenCalled()` | `expect(await isPortOpen(PORT)).toBe(true)` |
|
|
110
|
+
| `expect(spawn).toHaveBeenCalledWith(cmd, args)` | `const output = execSync(cmd); expect(output).toContain(expected)` |
|
|
111
|
+
|
|
112
|
+
**Transformation example:**
|
|
113
|
+
```javascript
|
|
114
|
+
// BEFORE (T1 violation - mocking spawn)
|
|
115
|
+
const mockProcess = { on: jest.fn(), stdout: mockStream };
|
|
116
|
+
jest.spyOn(child_process, 'spawn').mockReturnValue(mockProcess);
|
|
117
|
+
await startProxy();
|
|
118
|
+
expect(child_process.spawn).toHaveBeenCalledWith('proxy', ['--port', '8080']);
|
|
119
|
+
|
|
120
|
+
// AFTER (real verification)
|
|
121
|
+
await startProxy();
|
|
122
|
+
expect(await checkPort(8080)).toBe(true); // Port actually open
|
|
123
|
+
const response = await fetch('http://localhost:8080/health');
|
|
124
|
+
expect(response.status).toBe(200); // Proxy actually responds
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
### 3. File Operation Verification
|
|
128
|
+
|
|
129
|
+
| Anti-Pattern (Mock) | Real Pattern |
|
|
130
|
+
|---------------------|--------------|
|
|
131
|
+
| `jest.mock('fs')` | Use real fs with temp directory |
|
|
132
|
+
| `expect(fs.writeFile).toHaveBeenCalled()` | `expect(fs.existsSync(path)).toBe(true)` |
|
|
133
|
+
| `expect(fs.readFile).toHaveBeenCalledWith(path)` | `const content = fs.readFileSync(path); expect(content).toContain('expected')` |
|
|
134
|
+
|
|
135
|
+
**Transformation example:**
|
|
136
|
+
```javascript
|
|
137
|
+
// BEFORE (T2 violation - call-only assertion)
|
|
138
|
+
await saveConfig(config);
|
|
139
|
+
expect(fs.writeFile).toHaveBeenCalled();
|
|
140
|
+
|
|
141
|
+
// AFTER (result verification)
|
|
142
|
+
await saveConfig(config);
|
|
143
|
+
expect(fs.existsSync(configPath)).toBe(true);
|
|
144
|
+
const saved = JSON.parse(fs.readFileSync(configPath, 'utf8'));
|
|
145
|
+
expect(saved.setting).toBe(config.setting);
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
### 4. HTTP Request Verification
|
|
149
|
+
|
|
150
|
+
| Anti-Pattern (Mock) | Real Pattern |
|
|
151
|
+
|---------------------|--------------|
|
|
152
|
+
| `jest.mock('node-fetch')` | Use MSW or test server |
|
|
153
|
+
| `expect(fetch).toHaveBeenCalledWith(url)` | `const resp = await fetch(url); expect(resp.status).toBe(200)` |
|
|
154
|
+
| Mock response data | Actual response from test server |
|
|
155
|
+
|
|
156
|
+
**Transformation example:**
|
|
157
|
+
```javascript
|
|
158
|
+
// BEFORE (T3 violation - mocking integration boundary)
|
|
159
|
+
jest.mock('node-fetch');
|
|
160
|
+
fetch.mockResolvedValue({ json: () => ({ id: 1 }) });
|
|
161
|
+
const user = await fetchUser(1);
|
|
162
|
+
expect(fetch).toHaveBeenCalledWith('/api/users/1');
|
|
163
|
+
|
|
164
|
+
// AFTER (real integration with MSW)
|
|
165
|
+
import { setupServer } from 'msw/node';
|
|
166
|
+
import { rest } from 'msw';
|
|
167
|
+
|
|
168
|
+
const server = setupServer(
|
|
169
|
+
rest.get('/api/users/:id', (req, res, ctx) => {
|
|
170
|
+
return res(ctx.json({ id: req.params.id, name: 'Test User' }));
|
|
171
|
+
})
|
|
172
|
+
);
|
|
173
|
+
|
|
174
|
+
beforeAll(() => server.listen());
|
|
175
|
+
afterAll(() => server.close());
|
|
176
|
+
|
|
177
|
+
const user = await fetchUser(1); // Real fetch, intercepted at network level
|
|
178
|
+
expect(user.name).toBe('Test User');
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
### 5. Database Verification
|
|
182
|
+
|
|
183
|
+
| Anti-Pattern (Mock) | Real Pattern |
|
|
184
|
+
|---------------------|--------------|
|
|
185
|
+
| `expect(db.save).toHaveBeenCalled()` | `await db.save(data); const found = await db.find(id); expect(found).toBeDefined()` |
|
|
186
|
+
| `expect(db.delete).toHaveBeenCalledWith(id)` | `await db.delete(id); expect(await db.find(id)).toBeNull()` |
|
|
187
|
+
| `jest.mock('./database')` | Use test database instance |
|
|
188
|
+
|
|
189
|
+
**Transformation example:**
|
|
190
|
+
```javascript
|
|
191
|
+
// BEFORE (T2 violation - call-only)
|
|
192
|
+
await saveUser(user);
|
|
193
|
+
expect(db.insert).toHaveBeenCalledWith('users', user);
|
|
194
|
+
|
|
195
|
+
// AFTER (result verification)
|
|
196
|
+
await saveUser(user);
|
|
197
|
+
const saved = await db.findOne('users', { id: user.id });
|
|
198
|
+
expect(saved).toBeDefined();
|
|
199
|
+
expect(saved.email).toBe(user.email);
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
---
|
|
203
|
+
|
|
204
|
+
## T1-T4 Transformation Rules
|
|
205
|
+
|
|
206
|
+
### T1: Mock System Under Test -> Remove mock, verify real output
|
|
207
|
+
|
|
208
|
+
The system under test should NEVER be mocked. Remove the mock and verify actual behavior.
|
|
209
|
+
|
|
210
|
+
```javascript
|
|
211
|
+
// T1 violation: Mocking the function being tested
|
|
212
|
+
jest.spyOn(calculator, 'add').mockReturnValue(5);
|
|
213
|
+
expect(calculator.add(2, 3)).toBe(5); // Always passes regardless of implementation
|
|
214
|
+
|
|
215
|
+
// Fixed: Test real implementation
|
|
216
|
+
expect(calculator.add(2, 3)).toBe(5); // Fails if add() is broken
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
### T2: Call-Only Assertion -> Add result assertion
|
|
220
|
+
|
|
221
|
+
Verifying a function was called is insufficient. Verify the RESULT of that call.
|
|
222
|
+
|
|
223
|
+
```javascript
|
|
224
|
+
// T2 violation: Only checks call happened
|
|
225
|
+
await saveConfig(config);
|
|
226
|
+
expect(db.save).toHaveBeenCalled(); // Passes even if wrong data saved
|
|
227
|
+
|
|
228
|
+
// Fixed: Verify the actual saved data
|
|
229
|
+
await saveConfig(config);
|
|
230
|
+
const saved = await db.find(config.id);
|
|
231
|
+
expect(saved.value).toBe(config.value); // Fails if wrong data saved
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
### T3: Mock Integration Boundary -> Use test infrastructure
|
|
235
|
+
|
|
236
|
+
Integration tests should test real integration. Use MSW, test servers, or in-memory DBs.
|
|
237
|
+
|
|
238
|
+
```javascript
|
|
239
|
+
// T3 violation: Mocking HTTP in integration test
|
|
240
|
+
jest.mock('node-fetch');
|
|
241
|
+
const data = await fetchUserData(id); // Not testing real HTTP
|
|
242
|
+
|
|
243
|
+
// Fixed: Use MSW to intercept at network level
|
|
244
|
+
const server = setupServer(rest.get('/api/user/:id', handler));
|
|
245
|
+
const data = await fetchUserData(id); // Real fetch, real HTTP, controlled response
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
### T3+: Broken Integration Chain -> Chain real function outputs
|
|
249
|
+
|
|
250
|
+
Integration tests should chain real outputs through the pipeline.
|
|
251
|
+
|
|
252
|
+
```javascript
|
|
253
|
+
// T3+ violation: Using mock data instead of real output
|
|
254
|
+
const mockOrder = { id: 1, items: [{ sku: 'ABC', qty: 2 }] };
|
|
255
|
+
await processOrder(mockOrder); // Not testing real order creation
|
|
256
|
+
|
|
257
|
+
// Fixed: Chain real function outputs
|
|
258
|
+
const order = await createOrder({ sku: 'ABC', qty: 2 }); // Real order
|
|
259
|
+
await processOrder(order); // Processes real order data
|
|
260
|
+
const result = await getOrderStatus(order.id);
|
|
261
|
+
expect(result.status).toBe('processed');
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
---
|
|
265
|
+
|
|
266
|
+
## Quick Reference: Violation to Pattern
|
|
267
|
+
|
|
268
|
+
| Violation | Pattern Category | Fix Strategy |
|
|
269
|
+
|-----------|------------------|--------------|
|
|
270
|
+
| T0.1 | Prerequisites | Move function to production, import it |
|
|
271
|
+
| T0.2 | Prerequisites | Extract non-test functions to src/ |
|
|
272
|
+
| T0.3 | Prerequisites | Rename or move functions |
|
|
273
|
+
| T1 | Function Call | Remove mock, call real function |
|
|
274
|
+
| T2 | Any category | Add result assertion after call |
|
|
275
|
+
| T3 | HTTP/DB/Process | Use MSW, test DB, or spawn real process |
|
|
276
|
+
| T3+ | Integration Chain | Chain real outputs, don't pass mocks |
|
|
277
|
+
|
|
278
|
+
---
|
|
279
|
+
|
|
280
|
+
## Diagnostic Output
|
|
281
|
+
|
|
282
|
+
Write diagnostic output to `logs/diagnostics/assertion-patterns-{YYYYMMDD-HHMMSS}.yaml`:
|
|
283
|
+
|
|
284
|
+
```yaml
|
|
285
|
+
skill: assertion-patterns
|
|
286
|
+
timestamp: {ISO-8601}
|
|
287
|
+
diagnostics:
|
|
288
|
+
t0_checks:
|
|
289
|
+
t0_1_imports: pass|fail
|
|
290
|
+
t0_2_separation: pass|fail
|
|
291
|
+
t0_3_naming: pass|fail
|
|
292
|
+
patterns_applied: [T1, T2, T3]
|
|
293
|
+
transformations_suggested: 3
|
|
294
|
+
files_analyzed: 1
|
|
295
|
+
completion_status: success
|
|
296
|
+
```
|
|
@@ -0,0 +1,284 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: bug-magnet-data
|
|
3
|
+
description: Curated edge case test data for boundary testing, verification scripts, and test generation. Provides pre-curated reference data organized by data type with context-specific loading guidance.
|
|
4
|
+
user-invocable: false
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Bug Magnet Data
|
|
8
|
+
|
|
9
|
+
Curated edge case test data for boundary testing, verification scripts, and test generation. 50+ years of testing wisdom distilled into small, high-signal collections organized by data type.
|
|
10
|
+
|
|
11
|
+
**Core Principle**: Curation beats generation. 50 well-chosen edge cases find more bugs than 10,000 random inputs.
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## When to Use This Skill
|
|
16
|
+
|
|
17
|
+
**Load this skill when the consumer request matches ANY of these patterns:**
|
|
18
|
+
|
|
19
|
+
| Consumer | Trigger | Usage |
|
|
20
|
+
|----------|---------|-------|
|
|
21
|
+
| test-audit | Step 7 (edge case gap detection) | Identify missing boundary test coverage |
|
|
22
|
+
| bulwark-verify | Generating verification scripts | Inject edge cases into test scenarios |
|
|
23
|
+
| bulwark-fix-validator | Validating a fix | Test fix against boundary conditions |
|
|
24
|
+
|
|
25
|
+
**DO NOT use for:**
|
|
26
|
+
- Encrypted/compressed data (edge cases won't penetrate wrapping)
|
|
27
|
+
- Pure unit tests with fully mocked dependencies (edge cases need real execution)
|
|
28
|
+
- Performance/load testing (use dedicated load testing tools)
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## Pre-Flight Gate (BLOCKING)
|
|
33
|
+
|
|
34
|
+
**STOP. Before providing ANY edge case data, you MUST follow the three-phase workflow.**
|
|
35
|
+
|
|
36
|
+
This skill provides **curated data** through a **deterministic workflow**. You must execute all phases.
|
|
37
|
+
|
|
38
|
+
### What You MUST Do
|
|
39
|
+
|
|
40
|
+
1. **Phase 1: Component Detection** - Identify component type and load context file
|
|
41
|
+
2. **Phase 2: Data Loading** - Load T0 + T1 data files (REQUIRED), T2 if specified by context
|
|
42
|
+
3. **Phase 3: Edge Case Application** - Apply edge cases and report what was loaded
|
|
43
|
+
|
|
44
|
+
### What You MUST NOT Do
|
|
45
|
+
|
|
46
|
+
- **Do NOT generate edge cases from your own knowledge** - use the curated data files
|
|
47
|
+
- **Do NOT skip loading context files** - they determine which categories apply
|
|
48
|
+
- **Do NOT skip the safety filter** - patterns marked `safe_for_automation: false` must be excluded
|
|
49
|
+
- **Do NOT return partial data** - all applicable tiers must be loaded
|
|
50
|
+
|
|
51
|
+
### Why This Matters
|
|
52
|
+
|
|
53
|
+
The curated data exists because:
|
|
54
|
+
- **Curation beats generation** - 50 well-chosen edge cases find more bugs than 10,000 random inputs
|
|
55
|
+
- **Reproducibility** - Same component type = same edge cases every time
|
|
56
|
+
- **Safety** - Destructive patterns are explicitly marked and filtered
|
|
57
|
+
|
|
58
|
+
**If you find yourself thinking "I know some good edge cases" - STOP. Use the data files.**
|
|
59
|
+
|
|
60
|
+
### Completion Checklist
|
|
61
|
+
|
|
62
|
+
Before returning to consumer, verify ALL items:
|
|
63
|
+
|
|
64
|
+
- [ ] Phase 1: Component type detected
|
|
65
|
+
- [ ] Phase 1: Context file loaded for component type
|
|
66
|
+
- [ ] Phase 2: T0 data files loaded (boundaries, booleans, collections)
|
|
67
|
+
- [ ] Phase 2: T1 data files loaded (unicode, special-chars, injection, special numbers)
|
|
68
|
+
- [ ] Phase 2: T2 data files loaded (if specified by context file)
|
|
69
|
+
- [ ] Phase 2: Safety filter applied (excluded manual_only and safe_for_automation: false)
|
|
70
|
+
- [ ] Phase 3: Edge cases applied to test/verification scenario
|
|
71
|
+
- [ ] Phase 3: Report includes categories loaded and patterns excluded
|
|
72
|
+
|
|
73
|
+
**Do NOT return to consumer until all checkboxes can be marked complete.**
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## Dependencies
|
|
78
|
+
|
|
79
|
+
This skill provides data files and context guidance. Understanding what to load ensures deterministic execution.
|
|
80
|
+
|
|
81
|
+
| Category | Files | Requirement | When to Load |
|
|
82
|
+
|----------|-------|-------------|--------------|
|
|
83
|
+
| **Context files** | `context/{component-type}.md` | REQUIRED | Always load for detected component type |
|
|
84
|
+
| **T0 data (boundaries)** | `data/strings/boundaries.yaml`, `data/numbers/boundaries.yaml`, `data/booleans/boundaries.yaml`, `data/collections/arrays.yaml` | REQUIRED | Every edge case injection |
|
|
85
|
+
| **T1 data (common)** | `data/strings/unicode.yaml`, `data/strings/special-chars.yaml`, `data/strings/injection.yaml`, `data/numbers/special.yaml` | REQUIRED | Most edge case injections |
|
|
86
|
+
| **T2 data (context-specific)** | `data/dates/*.yaml`, `data/encoding/*.yaml`, `data/formats/*.yaml`, `data/concurrency/*.yaml` | CONDITIONALLY REQUIRED | If context file specifies → MUST load |
|
|
87
|
+
| **Language-specific** | `data/language-specific/{language}.yaml` | CONDITIONALLY REQUIRED | If testing language-specific behavior → MUST load |
|
|
88
|
+
| **External references** | `references/external-lists.md` | REQUIRED | For source attribution and update checking |
|
|
89
|
+
|
|
90
|
+
**Fallback behavior:**
|
|
91
|
+
- If component type detected → Loading `context/{type}.md` is REQUIRED
|
|
92
|
+
- If context file specifies a category → Loading that category is REQUIRED
|
|
93
|
+
- If a referenced file is missing → Note in output, continue with available data
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Data Tiers
|
|
98
|
+
|
|
99
|
+
| Tier | Categories | When to Load |
|
|
100
|
+
|------|------------|--------------|
|
|
101
|
+
| **T0 (Always)** | Boundaries (empty/single/max), Null handling | Every edge case injection |
|
|
102
|
+
| **T1 (Common)** | Basic injection, Unicode basics, Numeric edges | Every edge case injection |
|
|
103
|
+
| **T2 (Context)** | Date/time, Encoding, Formats, Concurrency | When context file specifies |
|
|
104
|
+
| **T3 (Manual)** | Patterns marked `manual_only: true` | NEVER for automated runs |
|
|
105
|
+
|
|
106
|
+
**Safety Filtering**: Patterns with `safe_for_automation: false` or `manual_only: true` MUST be excluded from automated test runs.
|
|
107
|
+
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## Three-Phase Workflow
|
|
111
|
+
|
|
112
|
+
**CRITICAL**: All three phases are REQUIRED. Do not skip any phase.
|
|
113
|
+
|
|
114
|
+
```
|
|
115
|
+
Phase 1: Component Detection (Deterministic)
|
|
116
|
+
├── Identify component type from code under test
|
|
117
|
+
├── Map to context file: context/{cli-args|http-body|file-contents|db-query|process-spawn}.md
|
|
118
|
+
└── Load context file → get applicable categories list
|
|
119
|
+
|
|
120
|
+
Phase 2: Data Loading (Deterministic)
|
|
121
|
+
├── Load T0 data files (REQUIRED - always)
|
|
122
|
+
├── Load T1 data files (REQUIRED - always)
|
|
123
|
+
├── Load T2 data files specified by context file (CONDITIONALLY REQUIRED)
|
|
124
|
+
├── Load language-specific file if applicable (CONDITIONALLY REQUIRED)
|
|
125
|
+
└── Apply safety filter: exclude patterns with safe_for_automation: false
|
|
126
|
+
|
|
127
|
+
Phase 3: Edge Case Application
|
|
128
|
+
├── Inject loaded edge cases into test scenarios
|
|
129
|
+
├── Report which categories were loaded
|
|
130
|
+
└── Report any patterns excluded due to safety filtering
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
---
|
|
134
|
+
|
|
135
|
+
## Component Type Detection
|
|
136
|
+
|
|
137
|
+
Map code under test to component type. **Detection determines which context file to load.**
|
|
138
|
+
|
|
139
|
+
| Code Pattern | Component Type | Context File |
|
|
140
|
+
|--------------|----------------|--------------|
|
|
141
|
+
| CLI argument parsing, process.argv, argparse | CLI | `context/cli-args.md` |
|
|
142
|
+
| HTTP request/response, req.body, fetch, axios | HTTP | `context/http-body.md` |
|
|
143
|
+
| File I/O, fs.read, open(), file parsing | File | `context/file-contents.md` |
|
|
144
|
+
| Database queries, SQL, ORM operations | Database | `context/db-query.md` |
|
|
145
|
+
| Child process, spawn, exec, subprocess | Process | `context/process-spawn.md` |
|
|
146
|
+
|
|
147
|
+
**If multiple types apply**: Load context files for each applicable type.
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
## Category Reference
|
|
152
|
+
|
|
153
|
+
### Strings (T0/T1)
|
|
154
|
+
|
|
155
|
+
| File | Contents | Bugs Caught |
|
|
156
|
+
|------|----------|-------------|
|
|
157
|
+
| `strings/boundaries.yaml` | Empty, single char, long strings, whitespace | NullPointerException, buffer overflow, off-by-one |
|
|
158
|
+
| `strings/unicode.yaml` | Multi-byte, normalization, emoji, RTL | Encoding errors, length calculation bugs |
|
|
159
|
+
| `strings/special-chars.yaml` | Quotes, escapes, control characters | Escape sequence handling, delimiter confusion |
|
|
160
|
+
| `strings/injection.yaml` | SQL, XSS, command injection, path traversal | Security vulnerabilities |
|
|
161
|
+
|
|
162
|
+
### Numbers (T0/T1)
|
|
163
|
+
|
|
164
|
+
| File | Contents | Bugs Caught |
|
|
165
|
+
|------|----------|-------------|
|
|
166
|
+
| `numbers/boundaries.yaml` | 0, -1, 1, MAX_INT, MIN_INT | Integer overflow/underflow, off-by-one |
|
|
167
|
+
| `numbers/special.yaml` | NaN, Infinity, -0 | Special value handling, NaN propagation |
|
|
168
|
+
| `numbers/precision.yaml` | 0.1+0.2, large/small floats | Floating point comparison failures |
|
|
169
|
+
|
|
170
|
+
### Booleans (T0)
|
|
171
|
+
|
|
172
|
+
| File | Contents | Bugs Caught |
|
|
173
|
+
|------|----------|-------------|
|
|
174
|
+
| `booleans/boundaries.yaml` | true, false, null, truthy/falsy | Null reference, truthy/falsy confusion |
|
|
175
|
+
|
|
176
|
+
### Collections (T0)
|
|
177
|
+
|
|
178
|
+
| File | Contents | Bugs Caught |
|
|
179
|
+
|------|----------|-------------|
|
|
180
|
+
| `collections/arrays.yaml` | Empty, single, large, nested, sparse | Index out of bounds, empty collection crashes |
|
|
181
|
+
| `collections/objects.yaml` | Empty, nested, circular, prototype pollution | Null reference, prototype pollution |
|
|
182
|
+
|
|
183
|
+
### Dates (T2)
|
|
184
|
+
|
|
185
|
+
| File | Contents | Bugs Caught |
|
|
186
|
+
|------|----------|-------------|
|
|
187
|
+
| `dates/boundaries.yaml` | Epoch, Y2K38, leap year | Y2K38 overflow, leap year bugs |
|
|
188
|
+
| `dates/timezone.yaml` | DST transitions, UTC offsets | DST errors, timezone conversion |
|
|
189
|
+
| `dates/invalid.yaml` | Feb 30, invalid formats | Date parsing failures |
|
|
190
|
+
|
|
191
|
+
### Encoding (T2)
|
|
192
|
+
|
|
193
|
+
| File | Contents | Bugs Caught |
|
|
194
|
+
|------|----------|-------------|
|
|
195
|
+
| `encoding/charset.yaml` | ASCII, UTF-8, BOM | Encoding detection, mojibake |
|
|
196
|
+
| `encoding/normalization.yaml` | NFC, NFD, overlong | Normalization mismatches |
|
|
197
|
+
|
|
198
|
+
### Formats (T2)
|
|
199
|
+
|
|
200
|
+
| File | Contents | Bugs Caught |
|
|
201
|
+
|------|----------|-------------|
|
|
202
|
+
| `formats/email.yaml` | Valid/invalid patterns | Overly strict/lenient validation |
|
|
203
|
+
| `formats/url.yaml` | Valid/invalid patterns | URL parsing errors |
|
|
204
|
+
| `formats/json.yaml` | Valid/invalid patterns | JSON parsing errors |
|
|
205
|
+
|
|
206
|
+
### Concurrency (T2)
|
|
207
|
+
|
|
208
|
+
| File | Contents | Bugs Caught |
|
|
209
|
+
|------|----------|-------------|
|
|
210
|
+
| `concurrency/race-conditions.yaml` | Double submit, concurrent edit | Race conditions, lost updates |
|
|
211
|
+
| `concurrency/state-machines.yaml` | Invalid transitions | State corruption |
|
|
212
|
+
|
|
213
|
+
### Language-Specific (Conditional)
|
|
214
|
+
|
|
215
|
+
| File | Contents | Bugs Caught |
|
|
216
|
+
|------|----------|-------------|
|
|
217
|
+
| `language-specific/javascript.yaml` | == vs ===, truthy/falsy | Type coercion bugs |
|
|
218
|
+
| `language-specific/python.yaml` | None vs False, mutable defaults | Python-specific gotchas |
|
|
219
|
+
| `language-specific/rust.yaml` | Ownership, borrowing | Memory safety issues |
|
|
220
|
+
|
|
221
|
+
---
|
|
222
|
+
|
|
223
|
+
## Data File Format
|
|
224
|
+
|
|
225
|
+
All data files use this YAML structure:
|
|
226
|
+
|
|
227
|
+
```yaml
|
|
228
|
+
metadata:
|
|
229
|
+
version: "1.0.0"
|
|
230
|
+
last_updated: "2026-02-01"
|
|
231
|
+
source_urls: []
|
|
232
|
+
|
|
233
|
+
category: strings
|
|
234
|
+
subcategory: boundaries
|
|
235
|
+
tier: T0
|
|
236
|
+
|
|
237
|
+
values:
|
|
238
|
+
identifier:
|
|
239
|
+
value: "actual value"
|
|
240
|
+
bugs_caught: ["Bug type 1", "Bug type 2"]
|
|
241
|
+
safe_for_automation: true
|
|
242
|
+
manual_only: false
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
**Safety flags to check:**
|
|
246
|
+
- `safe_for_automation: false` → Exclude from automated runs
|
|
247
|
+
- `manual_only: true` → NEVER include in automated runs
|
|
248
|
+
|
|
249
|
+
---
|
|
250
|
+
|
|
251
|
+
## Integration Examples
|
|
252
|
+
|
|
253
|
+
### test-audit (Step 7)
|
|
254
|
+
|
|
255
|
+
```
|
|
256
|
+
1. Detect component type from test file
|
|
257
|
+
2. Load context file for component type
|
|
258
|
+
3. Load T0 + T1 data files
|
|
259
|
+
4. Load T2 files specified by context
|
|
260
|
+
5. Compare test coverage against loaded edge cases
|
|
261
|
+
6. Report missing edge case coverage
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
### bulwark-verify
|
|
265
|
+
|
|
266
|
+
```
|
|
267
|
+
1. Detect component type from code under verification
|
|
268
|
+
2. Load context file for component type
|
|
269
|
+
3. Load T0 + T1 data files
|
|
270
|
+
4. Load T2 files specified by context
|
|
271
|
+
5. Filter out patterns with safe_for_automation: false
|
|
272
|
+
6. Inject applicable edge cases into verification script
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
### bulwark-fix-validator
|
|
276
|
+
|
|
277
|
+
```
|
|
278
|
+
1. Detect component type from fix
|
|
279
|
+
2. Load context file for component type
|
|
280
|
+
3. Load T0 + T1 data files
|
|
281
|
+
4. Test fix against loaded edge cases
|
|
282
|
+
5. Report any edge cases that break the fix
|
|
283
|
+
```
|
|
284
|
+
|