@agent-ix/quoin 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/plugins/marketplace.json +14 -0
- package/.claude-plugin/marketplace.json +16 -0
- package/.claude-plugin/plugin.json +8 -0
- package/.codex-plugin/plugin.json +27 -0
- package/.github/plugin/marketplace.json +16 -0
- package/LICENSE +21 -0
- package/README.md +266 -0
- package/bin/quoin.js +14 -0
- package/default-modules.yaml +82 -0
- package/dist/base-DrUMkAh1.js +137 -0
- package/dist/base.d.ts +23 -0
- package/dist/base.d.ts.map +1 -0
- package/dist/catalog.d.ts +32 -0
- package/dist/catalog.d.ts.map +1 -0
- package/dist/cli.d.ts +18 -0
- package/dist/cli.d.ts.map +1 -0
- package/dist/cli.js +20 -0
- package/dist/commands/catalog/index.d.ts +2 -0
- package/dist/commands/catalog/index.d.ts.map +1 -0
- package/dist/commands/catalog/index.js +2 -0
- package/dist/commands/catalog/list.d.ts +11 -0
- package/dist/commands/catalog/list.d.ts.map +1 -0
- package/dist/commands/catalog/list.js +22 -0
- package/dist/commands/catalog/show.d.ts +13 -0
- package/dist/commands/catalog/show.d.ts.map +1 -0
- package/dist/commands/catalog/show.js +25 -0
- package/dist/commands/catalog/validate.d.ts +10 -0
- package/dist/commands/catalog/validate.d.ts.map +1 -0
- package/dist/commands/catalog/validate.js +27 -0
- package/dist/commands/config/doctor.d.ts +7 -0
- package/dist/commands/config/doctor.d.ts.map +1 -0
- package/dist/commands/config/doctor.js +15 -0
- package/dist/commands/config/edit.d.ts +7 -0
- package/dist/commands/config/edit.d.ts.map +1 -0
- package/dist/commands/config/edit.js +13 -0
- package/dist/commands/config/get.d.ts +11 -0
- package/dist/commands/config/get.d.ts.map +1 -0
- package/dist/commands/config/get.js +20 -0
- package/dist/commands/config/index.d.ts +2 -0
- package/dist/commands/config/index.d.ts.map +1 -0
- package/dist/commands/config/index.js +2 -0
- package/dist/commands/config/set.d.ts +12 -0
- package/dist/commands/config/set.d.ts.map +1 -0
- package/dist/commands/config/set.js +26 -0
- package/dist/commands/matrix.d.ts +14 -0
- package/dist/commands/matrix.d.ts.map +1 -0
- package/dist/commands/matrix.js +15 -0
- package/dist/commands/module/ensure-defaults.d.ts +8 -0
- package/dist/commands/module/ensure-defaults.d.ts.map +1 -0
- package/dist/commands/module/ensure-defaults.js +17 -0
- package/dist/commands/module/index.d.ts +2 -0
- package/dist/commands/module/index.d.ts.map +1 -0
- package/dist/commands/module/index.js +2 -0
- package/dist/commands/module/install.d.ts +11 -0
- package/dist/commands/module/install.d.ts.map +1 -0
- package/dist/commands/module/install.js +22 -0
- package/dist/commands/module/list.d.ts +8 -0
- package/dist/commands/module/list.d.ts.map +1 -0
- package/dist/commands/module/list.js +13 -0
- package/dist/commands/module/remove.d.ts +10 -0
- package/dist/commands/module/remove.d.ts.map +1 -0
- package/dist/commands/module/remove.js +16 -0
- package/dist/commands/plugin/deprecation.d.ts +10 -0
- package/dist/commands/plugin/deprecation.d.ts.map +1 -0
- package/dist/commands/plugin/ensure-defaults.d.ts +8 -0
- package/dist/commands/plugin/ensure-defaults.d.ts.map +1 -0
- package/dist/commands/plugin/ensure-defaults.js +12 -0
- package/dist/commands/plugin/index.d.ts +2 -0
- package/dist/commands/plugin/index.d.ts.map +1 -0
- package/dist/commands/plugin/index.js +2 -0
- package/dist/commands/plugin/install.d.ts +8 -0
- package/dist/commands/plugin/install.d.ts.map +1 -0
- package/dist/commands/plugin/install.js +12 -0
- package/dist/commands/plugin/list.d.ts +8 -0
- package/dist/commands/plugin/list.d.ts.map +1 -0
- package/dist/commands/plugin/list.js +12 -0
- package/dist/commands/plugin/remove.d.ts +8 -0
- package/dist/commands/plugin/remove.d.ts.map +1 -0
- package/dist/commands/plugin/remove.js +12 -0
- package/dist/commands/review.d.ts +14 -0
- package/dist/commands/review.d.ts.map +1 -0
- package/dist/commands/review.js +15 -0
- package/dist/commands/to-plan.d.ts +14 -0
- package/dist/commands/to-plan.d.ts.map +1 -0
- package/dist/commands/to-plan.js +15 -0
- package/dist/commands/update.d.ts +13 -0
- package/dist/commands/update.d.ts.map +1 -0
- package/dist/commands/update.js +34 -0
- package/dist/commands/write.d.ts +16 -0
- package/dist/commands/write.d.ts.map +1 -0
- package/dist/commands/write.js +33 -0
- package/dist/config-schema-Dp-vlxuR.js +13 -0
- package/dist/config-schema.d.ts +52 -0
- package/dist/config-schema.d.ts.map +1 -0
- package/dist/deprecation-fRxUHoxv.js +4 -0
- package/dist/flow-command-SMjCSOFg.js +88 -0
- package/dist/flow-command.d.ts +25 -0
- package/dist/flow-command.d.ts.map +1 -0
- package/dist/flows.d.ts +7 -0
- package/dist/flows.d.ts.map +1 -0
- package/dist/index.d.ts +11 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +9 -0
- package/dist/modules-CW8pBK41.js +22 -0
- package/dist/modules.d.ts +10 -0
- package/dist/modules.d.ts.map +1 -0
- package/dist/org.d.ts +61 -0
- package/dist/org.d.ts.map +1 -0
- package/dist/plugins-GSxafyD3.js +77 -0
- package/dist/plugins.d.ts +13 -0
- package/dist/plugins.d.ts.map +1 -0
- package/dist/version-_1nY4KyJ.js +17 -0
- package/dist/version.d.ts +3 -0
- package/dist/version.d.ts.map +1 -0
- package/dist/write-CQAitcI5.js +156 -0
- package/dist/write.d.ts +28 -0
- package/dist/write.d.ts.map +1 -0
- package/package.json +119 -0
- package/skills/gap-analysis/SKILL.md +97 -0
- package/skills/gap-analysis/references/step-1-target-selection.md +53 -0
- package/skills/gap-analysis/references/step-2-plan-completion.md +49 -0
- package/skills/gap-analysis/references/step-3-matrix-verification.md +54 -0
- package/skills/gap-analysis/references/step-4-underspecified-code.md +63 -0
- package/skills/gap-analysis/references/step-5-semantic-review.md +51 -0
- package/skills/gap-analysis/references/step-6-specreview-artifact.md +95 -0
- package/skills/spec-app-review/SKILL.md +51 -0
- package/skills/spec-app-review/references/checklist.md +197 -0
- package/skills/spec-dependency-analysis/SKILL.md +75 -0
- package/skills/spec-ears-analysis/SKILL.md +89 -0
- package/skills/spec-evidence-analysis/SKILL.md +78 -0
- package/skills/spec-failure-domain-analysis/SKILL.md +44 -0
- package/skills/spec-ideation/SKILL.md +46 -0
- package/skills/spec-integrity-analysis/SKILL.md +47 -0
- package/skills/spec-matrix/SKILL.md +168 -0
- package/skills/spec-matrix/assets/test-cases/TC-001.md +35 -0
- package/skills/spec-matrix/assets/test-cases/TC-002.md +37 -0
- package/skills/spec-matrix/assets/test-cases/TC-006.md +50 -0
- package/skills/spec-matrix/assets/test-matrix-example.md +104 -0
- package/skills/spec-matrix/assets/test-matrix-template.md +123 -0
- package/skills/spec-matrix/workflow-assets/skills/matrix/SKILL.md +35 -0
- package/skills/spec-matrix/workflow-assets/skills/matrix/scripts/invariants.js +1 -0
- package/skills/spec-matrix/workflow-assets/skills/matrix/templates/report/default.md +30 -0
- package/skills/spec-matrix/workflow-assets/skills/matrix/workflows/matrix/def.yaml +82 -0
- package/skills/spec-object-review/SKILL.md +125 -0
- package/skills/spec-object-review/references/object-type-guide.md +118 -0
- package/skills/spec-review/SKILL.md +71 -0
- package/skills/spec-review/references/checklist.md +50 -0
- package/skills/spec-review/workflow-assets/skills/review/SKILL.md +96 -0
- package/skills/spec-review/workflow-assets/skills/review/scripts/invariants.js +60 -0
- package/skills/spec-review/workflow-assets/skills/review/workflows/review/def.yaml +68 -0
- package/skills/spec-risk-complexity-analysis/SKILL.md +77 -0
- package/skills/spec-scope-boundary-analysis/SKILL.md +86 -0
- package/skills/spec-security-analysis/SKILL.md +84 -0
- package/skills/spec-security-analysis/assets/security-analysis-example.md +89 -0
- package/skills/spec-security-analysis/assets/security-analysis-template.md +73 -0
- package/skills/spec-security-analysis/references/component-type-mapping.md +161 -0
- package/skills/spec-security-analysis/references/gdpr/ChII-Principles.md +58 -0
- package/skills/spec-security-analysis/references/gdpr/ChIII-DataSubjectRights.md +84 -0
- package/skills/spec-security-analysis/references/gdpr/ChIV-ControllerProcessor.md +83 -0
- package/skills/spec-security-analysis/references/gdpr/ChV-DataTransfers.md +62 -0
- package/skills/spec-security-analysis/references/gdpr/index.md +54 -0
- package/skills/spec-security-analysis/references/iso27001/A5-OrganizationalControls.md +64 -0
- package/skills/spec-security-analysis/references/iso27001/A6-PeopleControls.md +30 -0
- package/skills/spec-security-analysis/references/iso27001/A7-PhysicalControls.md +35 -0
- package/skills/spec-security-analysis/references/iso27001/A8-TechnologicalControls.md +66 -0
- package/skills/spec-security-analysis/references/iso27001/index.md +43 -0
- package/skills/spec-security-analysis/references/nist-800-53/AC-AccessControl.md +57 -0
- package/skills/spec-security-analysis/references/nist-800-53/AU-AuditAccountability.md +49 -0
- package/skills/spec-security-analysis/references/nist-800-53/CM-ConfigurationManagement.md +36 -0
- package/skills/spec-security-analysis/references/nist-800-53/IA-IdentificationAuthentication.md +44 -0
- package/skills/spec-security-analysis/references/nist-800-53/SA-SystemServicesAcquisition.md +50 -0
- package/skills/spec-security-analysis/references/nist-800-53/SC-SystemCommunications.md +53 -0
- package/skills/spec-security-analysis/references/nist-800-53/SI-SystemInformationIntegrity.md +52 -0
- package/skills/spec-security-analysis/references/nist-800-53/index.md +58 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V1-Architecture.md +33 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V10-MaliciousCode.md +33 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V11-BusinessLogic.md +32 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V12-Files.md +34 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V13-API.md +34 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V14-Configuration.md +35 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V2-Authentication.md +36 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V3-SessionManagement.md +35 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V4-AccessControl.md +39 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V5-Validation.md +37 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V6-Cryptography.md +34 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V7-ErrorLogging.md +37 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V8-DataProtection.md +34 -0
- package/skills/spec-security-analysis/references/owasp-asvs/V9-Communication.md +32 -0
- package/skills/spec-security-analysis/references/owasp-asvs/index.md +59 -0
- package/skills/spec-security-analysis/references/soc2/CC1-ControlEnvironment.md +31 -0
- package/skills/spec-security-analysis/references/soc2/CC2-CommunicationInformation.md +27 -0
- package/skills/spec-security-analysis/references/soc2/CC3-RiskAssessment.md +29 -0
- package/skills/spec-security-analysis/references/soc2/CC4-MonitoringActivities.md +27 -0
- package/skills/spec-security-analysis/references/soc2/CC5-ControlActivities.md +28 -0
- package/skills/spec-security-analysis/references/soc2/CC6-LogicalPhysicalAccess.md +45 -0
- package/skills/spec-security-analysis/references/soc2/CC7-SystemOperations.md +42 -0
- package/skills/spec-security-analysis/references/soc2/CC8-ChangeManagement.md +32 -0
- package/skills/spec-security-analysis/references/soc2/CC9-RiskMitigation.md +34 -0
- package/skills/spec-security-analysis/references/soc2/index.md +46 -0
- package/skills/spec-security-analysis/references/standards-index.md +46 -0
- package/skills/spec-to-plan/SKILL.md +75 -0
- package/skills/spec-to-plan/references/step-0-plan-selection.md +121 -0
- package/skills/spec-to-plan/references/step-1-analysis.md +88 -0
- package/skills/spec-to-plan/references/step-2-test-plan.md +43 -0
- package/skills/spec-to-plan/references/step-3-execution-plan.md +218 -0
- package/skills/spec-to-plan/workflow-assets/skills/to-plan/SKILL.md +35 -0
- package/skills/spec-to-plan/workflow-assets/skills/to-plan/scripts/invariants.js +1 -0
- package/skills/spec-to-plan/workflow-assets/skills/to-plan/templates/implementation-plan/default.md +44 -0
- package/skills/spec-to-plan/workflow-assets/skills/to-plan/workflows/to-plan/def.yaml +85 -0
- package/skills/specify/SKILL.md +154 -0
- package/skills/specify/assets/app-spec-template.md +143 -0
- package/skills/specify/assets/spec-template.md +299 -0
- package/skills/specify/references/integration-tests.md +72 -0
- package/skills/specify/references/us-to-fr.md +40 -0
- package/skills/specify/references/writing-good-requirements.md +61 -0
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: spec-evidence-analysis
|
|
3
|
+
description: Define verification methods and evidence artifacts for requirements.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Verification & Evidence Strategy
|
|
7
|
+
|
|
8
|
+
Make sure every requirement has a defined verification method and a concrete artifact that proves it was verified.
|
|
9
|
+
|
|
10
|
+
## When to use
|
|
11
|
+
|
|
12
|
+
- After requirements are written but before `spec-matrix` builds the test matrix.
|
|
13
|
+
- During `spec-review` when "how do we know this is satisfied?" is unclear.
|
|
14
|
+
- Whenever a new NFR is added (NFRs are most prone to missing evidence).
|
|
15
|
+
|
|
16
|
+
## Verification methods (ISO/IEC/IEEE 29148)
|
|
17
|
+
|
|
18
|
+
| Method | Use when |
|
|
19
|
+
|--------|----------|
|
|
20
|
+
| `test` | The requirement is exercisable by automated or manual test execution. Default. |
|
|
21
|
+
| `analysis` | The requirement is proven by calculation, modeling, or static reasoning (e.g. complexity bound, cryptographic argument). |
|
|
22
|
+
| `inspection` | The requirement is satisfied by visual or structural review of the artifact (e.g. license headers, doc presence). |
|
|
23
|
+
| `demonstration` | The requirement is satisfied by observing operation in a realistic environment without instrumentation (e.g. UX flow walkthrough). |
|
|
24
|
+
|
|
25
|
+
## Evidence artifacts
|
|
26
|
+
|
|
27
|
+
| Artifact | Pairs with |
|
|
28
|
+
|----------|------------|
|
|
29
|
+
| `test_case` | `test` — points to a TC-XXX file or test path |
|
|
30
|
+
| `analysis_report` | `analysis` — points to a doc/notebook |
|
|
31
|
+
| `inspection_checklist` | `inspection` — points to a checklist file |
|
|
32
|
+
| `demo_recording` | `demonstration` — points to a recording or scripted walkthrough |
|
|
33
|
+
| `metric` | any — observability metric that proves runtime behavior |
|
|
34
|
+
| `log` | any — log event proving the path executed |
|
|
35
|
+
|
|
36
|
+
**Rule**: Every implementation task MUST cite at least one evidence artifact. A task with no evidence is unfinishable.
|
|
37
|
+
|
|
38
|
+
## Process
|
|
39
|
+
|
|
40
|
+
1. **Tag every FR/StR/NFR** with `verification_method` and `evidence` fields in its frontmatter.
|
|
41
|
+
2. **Resolve "test" defaults**: if an FR is tagged `test`, the matching test case must already exist or be queued in `spec-matrix`.
|
|
42
|
+
3. **Resolve non-test methods**: for `analysis`/`inspection`/`demonstration`, name the artifact path explicitly. No "TBD".
|
|
43
|
+
|
|
44
|
+
## Deliverable template
|
|
45
|
+
|
|
46
|
+
Per-requirement frontmatter (add to existing FR/StR/NFR files):
|
|
47
|
+
|
|
48
|
+
```yaml
|
|
49
|
+
verification_method: test # test | analysis | inspection | demonstration
|
|
50
|
+
evidence:
|
|
51
|
+
- kind: test_case
|
|
52
|
+
ref: tests/integration/test_login.py::test_invalid_credentials
|
|
53
|
+
- kind: metric
|
|
54
|
+
ref: auth_failure_total
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Summary deliverable in `spec/analysis/evidence.md`:
|
|
58
|
+
|
|
59
|
+
```markdown
|
|
60
|
+
# Verification & Evidence Summary
|
|
61
|
+
|
|
62
|
+
| Req | Method | Artifact |
|
|
63
|
+
|-----|--------|----------|
|
|
64
|
+
| FR-003 | test | tests/integration/test_login.py::test_login_happy_path |
|
|
65
|
+
| FR-010 | analysis | docs/analysis/storage-bound.md |
|
|
66
|
+
| NFR-002 | demonstration | docs/demos/uptime-walkthrough.md |
|
|
67
|
+
| NFR-005 | test + metric | tests/perf/test_p99_latency.py, metric `http_request_duration_seconds` |
|
|
68
|
+
|
|
69
|
+
## Open gaps
|
|
70
|
+
|
|
71
|
+
None.
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
## Acceptance Criteria
|
|
75
|
+
|
|
76
|
+
- Every FR/StR/NFR has both `verification_method` and at least one `evidence` entry.
|
|
77
|
+
- No `evidence: TBD`, no missing refs.
|
|
78
|
+
- The summary table has zero open gaps before `spec-to-plan` is invoked.
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: spec-failure-domain-analysis
|
|
3
|
+
description: Identify unstated failure modes, identity confusion, purity gaps, and topological edge cases in ISO-style specifications.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Spec Failure Domain Analysis
|
|
7
|
+
|
|
8
|
+
Run this skill during spec review to catch common omissions before implementation.
|
|
9
|
+
|
|
10
|
+
## When to Use
|
|
11
|
+
- After drafting FR/NFR/StR requirements
|
|
12
|
+
- Before `spec-to-plan` (part of its readiness gate)
|
|
13
|
+
- During peer review of specifications
|
|
14
|
+
|
|
15
|
+
## Checklist
|
|
16
|
+
|
|
17
|
+
### 1. Extension Points (Trust Boundaries)
|
|
18
|
+
For each callback, hook, observer, or plugin interface:
|
|
19
|
+
- [ ] Is failure behavior defined? Options:
|
|
20
|
+
- **Strict**: Failure aborts operation (data integrity)
|
|
21
|
+
- **Resilient**: Failure is logged/suppressed (availability)
|
|
22
|
+
- **Configurable**: User chooses policy
|
|
23
|
+
|
|
24
|
+
### 2. Entity Identity
|
|
25
|
+
For each core domain entity (State, Event, User, etc.):
|
|
26
|
+
- [ ] Is the uniqueness key explicit?
|
|
27
|
+
- Bad: "A set of states"
|
|
28
|
+
- Good: "States unique by `name` property"
|
|
29
|
+
|
|
30
|
+
### 3. Evaluation Purity
|
|
31
|
+
For each user-supplied logic (guards, filters, validators):
|
|
32
|
+
- [ ] Are side-effects constrained?
|
|
33
|
+
- Document if read-only is required for simulation/reasoning
|
|
34
|
+
|
|
35
|
+
### 4. Topological Robustness
|
|
36
|
+
For each graph/tree algorithm (search, path-finding):
|
|
37
|
+
- [ ] Is termination guaranteed on cycles?
|
|
38
|
+
- [ ] Are worst-case structures considered? (deep nesting, disconnected nodes)
|
|
39
|
+
|
|
40
|
+
## Output
|
|
41
|
+
Document any missing constraints as proposed additions:
|
|
42
|
+
- **NFR**: Non-functional (resilience, performance)
|
|
43
|
+
- **StR**: Structural (integrity, purity)
|
|
44
|
+
- **FR**: Functional (if behavior is undefined)
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: spec-ideation
|
|
3
|
+
description: Use for loose, exploratory specification drafting before formal authoring, review, matrix, or planning work.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Spec Ideation
|
|
7
|
+
|
|
8
|
+
## Mode
|
|
9
|
+
|
|
10
|
+
This is a config-only skill. Do not create workflow state, call workflow
|
|
11
|
+
commands, or enforce phase gates while using it.
|
|
12
|
+
|
|
13
|
+
## Framing Moves
|
|
14
|
+
|
|
15
|
+
- State the problem in concrete user-facing terms.
|
|
16
|
+
- Separate users, operators, and system actors.
|
|
17
|
+
- Draft candidate features as options, not commitments.
|
|
18
|
+
- Name non-goals early when they protect scope.
|
|
19
|
+
- Convert uncertainty into open questions instead of blocking progress.
|
|
20
|
+
- Define success criteria that can later become evidence targets.
|
|
21
|
+
|
|
22
|
+
## Draft Outline
|
|
23
|
+
|
|
24
|
+
```markdown
|
|
25
|
+
# Draft Spec
|
|
26
|
+
|
|
27
|
+
## Problem
|
|
28
|
+
|
|
29
|
+
## Users And Actors
|
|
30
|
+
|
|
31
|
+
## Goals
|
|
32
|
+
|
|
33
|
+
## Non-Goals
|
|
34
|
+
|
|
35
|
+
## Candidate Features
|
|
36
|
+
|
|
37
|
+
## Open Questions
|
|
38
|
+
|
|
39
|
+
## Success Criteria
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## Promotion Signal
|
|
43
|
+
|
|
44
|
+
Recommend promotion to `specify` when the draft has a clear problem, named
|
|
45
|
+
users or actors, scoped goals and non-goals, and at least one success criterion
|
|
46
|
+
that can be verified.
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: spec-integrity-analysis
|
|
3
|
+
description: Perform quality gate analysis on ISO-style specifications to ensure completeness, consistency, and atomicity.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Specification Integrity Analysis
|
|
7
|
+
|
|
8
|
+
This skill performs a quality gate analysis on specifications (User Stories, FR/SR, StR, NFR) before task generation.
|
|
9
|
+
|
|
10
|
+
## Process
|
|
11
|
+
|
|
12
|
+
1. **Completeness Analysis**:
|
|
13
|
+
- Verify every User Story (US) maps to >= 1 Functional/System Requirement (FR/SR).
|
|
14
|
+
- Verify every FR/SR maps to >= 1 Stakeholder Requirement (StR).
|
|
15
|
+
- Verify every FR/SR maps to >= 1 verification method.
|
|
16
|
+
- Verify every NFR is explicitly scoped and referenced by affected FR/SR.
|
|
17
|
+
- **Deliverable**: Traceability matrix (US -> FR/SR -> StR -> Verification).
|
|
18
|
+
|
|
19
|
+
2. **Consistency & Conflict Analysis**:
|
|
20
|
+
- Identify contradictory constraints (e.g., latency vs security).
|
|
21
|
+
- Identify duplicated requirements with different wording.
|
|
22
|
+
- Identify hidden assumptions embedded in prose.
|
|
23
|
+
- **Rule**: No requirement may have more than one valid interpretation.
|
|
24
|
+
|
|
25
|
+
### Hidden Assumption Probes
|
|
26
|
+
|
|
27
|
+
For every FR, explicitly check for these patterns:
|
|
28
|
+
|
|
29
|
+
| Pattern | Probe Question |
|
|
30
|
+
|---------|---------------|
|
|
31
|
+
| FR delegates to an external CLI (git, gh, cookiecutter, etc.) | Is there an NFR declaring the tool's minimum version, detection method, and user-facing error when absent? |
|
|
32
|
+
| FR performs a lookup over multiple sources (registries, plugins, taps) | Is there an explicit tie-breaking policy when two sources return the same key? (first-wins, last-wins, error, merge — must be a stated decision) |
|
|
33
|
+
| FR calls an external API that returns paginated results | Does an NFR state whether full enumeration is required? If a cap is acceptable, is it documented as a known limitation? |
|
|
34
|
+
| FR calls an external API in a loop or concurrently | Does an NFR declare max concurrency, rate limit awareness, retry policy, and that errors must surface (not be silently swallowed)? |
|
|
35
|
+
| FR involves a scaffolding or generation command | Are both an interactive mode (human at terminal) and a scripted/CI mode explicitly specified with their flags? |
|
|
36
|
+
| FR calls an authenticated external API | Does an NFR declare the required auth scopes and mandate a clear user-facing error when scope is insufficient (not silent empty results)? |
|
|
37
|
+
| FR depends on a package or service not yet implemented | Is the interim fallback and eventual resolution chain documented? Is the hardcoded default traceable to a known stub? |
|
|
38
|
+
|
|
39
|
+
3. **Atomicity & Testability Analysis**:
|
|
40
|
+
- Ensure each FR/SR/StR defines a *single* behavior/obligation.
|
|
41
|
+
- Ensure behavior is externally observable.
|
|
42
|
+
- Ensure behavior is verifiable by test, inspection, analysis, or demonstration.
|
|
43
|
+
- **Action**: Flag non-testable requirements for rewriting or splitting.
|
|
44
|
+
|
|
45
|
+
## Failure Domain Check
|
|
46
|
+
Before finalizing, verify against `spec-failure-domain-analysis`:
|
|
47
|
+
- Extension failures, Identity keys, Evaluation purity, Topological robustness.
|
|
@@ -0,0 +1,168 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: spec-matrix
|
|
3
|
+
description: Build and maintain the requirements Test Matrix ensuring 100% coverage.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Build Test Matrix
|
|
7
|
+
|
|
8
|
+
Use this skill to create the requirements Test Matrix and ensure comprehensive coverage.
|
|
9
|
+
|
|
10
|
+
## Rules (The Six Requirements)
|
|
11
|
+
|
|
12
|
+
1. Coverage: Every Acceptance Criterion (AC) must have ≥1 Test Case (TC).
|
|
13
|
+
2. Option Permutation: Test all valid option combinations.
|
|
14
|
+
3. Constraint Boundary: Test min, max, below-min (fail), above-max (fail).
|
|
15
|
+
4. Error Path: Test all documented error conditions.
|
|
16
|
+
5. State Transition: Test all valid state transitions (if applicable).
|
|
17
|
+
6. Edge Case: Identify and test extreme/unusual scenarios.
|
|
18
|
+
|
|
19
|
+
## EARS precondition (atomic, mappable requirements)
|
|
20
|
+
|
|
21
|
+
Coverage (Rule 1) is only meaningful when each requirement states **one**
|
|
22
|
+
obligation — a statement packing two `shall` clauses cannot map cleanly to one
|
|
23
|
+
Acceptance Criterion, so its matrix row is ambiguous. Before building the
|
|
24
|
+
matrix, run the EARS requirement-grammar check:
|
|
25
|
+
|
|
26
|
+
```bash
|
|
27
|
+
quire validate --scope . "spec/**/*.md" --summary
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Treat these `[ears:…]` findings as **un-mappable requirements to fix first**,
|
|
31
|
+
not coverage you can claim:
|
|
32
|
+
|
|
33
|
+
- `non-singular` — split the statement into one requirement per `shall`, then
|
|
34
|
+
map each to its own AC/TC.
|
|
35
|
+
- `unclassifiable` / `missing-subject` — the obligation is unclear; clarify the
|
|
36
|
+
requirement before authoring a TC against it.
|
|
37
|
+
|
|
38
|
+
Do not mark the matrix ✅ Complete while non-singular or unclassifiable
|
|
39
|
+
requirements remain — the coverage they appear to have is illusory. (The check
|
|
40
|
+
is advisory; this skill treats it as a quality gate for the matrix.)
|
|
41
|
+
|
|
42
|
+
## Process
|
|
43
|
+
|
|
44
|
+
1. Initialize:
|
|
45
|
+
- Use template `assets/test-matrix-template.md`.
|
|
46
|
+
- Target: Correct location.
|
|
47
|
+
- Single Repo: `spec/tests.md`
|
|
48
|
+
- Multi-Repo: `specs/<category>/<component>/spec/tests.md`
|
|
49
|
+
2. Traceability:
|
|
50
|
+
- Map `StR` -> `TC`.
|
|
51
|
+
- Map `US` -> `TC`.
|
|
52
|
+
- Map `FR` -> `TC`.
|
|
53
|
+
- Map `NFR` -> `TC`.
|
|
54
|
+
- Map `C` (Constraints) -> `TC`.
|
|
55
|
+
3. Enumerate: List all Test Cases with ID, Title, Type, Priority, Status.
|
|
56
|
+
See `Test Case Summary` below for the `Type` and `Priority` vocabularies —
|
|
57
|
+
both are validated, so a value outside the set fails `quire validate`.
|
|
58
|
+
4. Define Detailed TCs (Optional):
|
|
59
|
+
- For complex tests, create at:
|
|
60
|
+
- Single Repo: `spec/test-cases/TC-XXX.md`
|
|
61
|
+
- Multi-Repo: `specs/<category>/<component>/spec/test-cases/TC-XXX.md`
|
|
62
|
+
5. Verify: Ensure all 6 rules are satisfied.
|
|
63
|
+
6. Status: Mark as ✅ Complete only when all rules are met.
|
|
64
|
+
|
|
65
|
+
## Test Case Summary
|
|
66
|
+
|
|
67
|
+
The `## Test Case Summary` table is structurally validated by `quire validate`
|
|
68
|
+
against the `TestMatrix` archetype (`spec-artifacts-process` FR-003). Columns are
|
|
69
|
+
exactly `Test ID | Title | Type | Priority | Traces To | Status`.
|
|
70
|
+
|
|
71
|
+
The vocabularies below are **not owned by this skill**. The single source is
|
|
72
|
+
`spec_artifacts_process/manifest.yaml` (`traceability.vocabularies.test_type`,
|
|
73
|
+
mirrored into `column_choices`); if this list and the manifest ever disagree, the
|
|
74
|
+
manifest wins and this section is the bug.
|
|
75
|
+
|
|
76
|
+
### `Type` — exactly one value per row
|
|
77
|
+
|
|
78
|
+
Pick by *what makes the test fail*, not by which framework runs it.
|
|
79
|
+
|
|
80
|
+
| Value | Use when |
|
|
81
|
+
|---|---|
|
|
82
|
+
| `Unit` | One function/module in isolation, example-based. The default. |
|
|
83
|
+
| `Integration` | Two or more components, or a real dependency (DB, service, browser). |
|
|
84
|
+
| `E2E` | The whole system through its outermost interface, as a user drives it. |
|
|
85
|
+
| `Property` | The criterion holds **over a domain of inputs**, and the test generates them. See below. |
|
|
86
|
+
| `Fuzz` | Untargeted/mutational input generation looking for crashes, not a stated property. |
|
|
87
|
+
| `Benchmark` | The criterion is a latency/throughput number; the test measures it. |
|
|
88
|
+
| `Static` | Proved by analysis over the source — a lint, an audit script, an architecture check. No runtime. |
|
|
89
|
+
| `Compile` | The failure mode *is* a compile error (type-level guarantee, `forbid(unsafe_code)`). |
|
|
90
|
+
| `Snapshot` | Byte/DOM-identity against a stored artifact. |
|
|
91
|
+
| `Manual` | Verified by a human procedure; no automated gate. |
|
|
92
|
+
|
|
93
|
+
**When `Property` rather than `Unit`.** Read the acceptance criterion and ask
|
|
94
|
+
whether it quantifies. A criterion that names one input and one expected output
|
|
95
|
+
is a `Unit` example. A criterion that asserts something for *every* input, or
|
|
96
|
+
relates two runs to each other, is property-shaped:
|
|
97
|
+
|
|
98
|
+
- **Invariant** — "output is always sorted", "the count never exceeds `max_size`"
|
|
99
|
+
- **Round-trip** — "parse then render returns the original bytes"
|
|
100
|
+
- **Idempotence** — "applying it twice equals applying it once"
|
|
101
|
+
- **Metamorphic** — "reordering the inputs does not change the result"
|
|
102
|
+
- **Oracle** — "the fast path agrees with the reference implementation"
|
|
103
|
+
|
|
104
|
+
Words like *always*, *never*, *for any*, *regardless of*, *preserves*,
|
|
105
|
+
*round-trips*, *deterministic*, *order-independent* are the signal. Where a
|
|
106
|
+
criterion is property-shaped, prefer `Property` — a generated test discriminates
|
|
107
|
+
the claim, a single example only witnesses it.
|
|
108
|
+
|
|
109
|
+
`Property` is a first-class value: do not downgrade a property-shaped criterion
|
|
110
|
+
to `Unit` because the repo has no generator yet. Author the row, and let the
|
|
111
|
+
missing generator show up as a gap.
|
|
112
|
+
|
|
113
|
+
### `Priority`
|
|
114
|
+
|
|
115
|
+
`P0` | `P1` | `P2` | `P3` | `P4`.
|
|
116
|
+
|
|
117
|
+
### `Status`
|
|
118
|
+
|
|
119
|
+
`✅` complete · `⚠️` partial · `❌` failed · `🚧` in progress · `⛔` retired.
|
|
120
|
+
A marker may be followed by text (`✅ Complete`); the marker must come first.
|
|
121
|
+
|
|
122
|
+
### `Traces To`
|
|
123
|
+
|
|
124
|
+
Comma-separated requirement ids (`FR-012-AC-3`, `US-004-AC-1`), ranges allowed
|
|
125
|
+
(`FR-012-AC-1..3`), optional trailing ` (note)`. Trace to the **criterion**, not
|
|
126
|
+
the requirement, and never to another `TC`.
|
|
127
|
+
|
|
128
|
+
## React Components (Conditional)
|
|
129
|
+
|
|
130
|
+
For React projects, add a Storybook Test Matrix section.
|
|
131
|
+
|
|
132
|
+
See `skills/review-react/SKILL.md` for:
|
|
133
|
+
- Story file requirements per component
|
|
134
|
+
- AC trace code format in story JSDoc
|
|
135
|
+
- Coverage matrix template (FR → AC → Story)
|
|
136
|
+
|
|
137
|
+
## Integration Test Matrix (Conditional)
|
|
138
|
+
|
|
139
|
+
If ANY requirement specifies integration testing, include this section.
|
|
140
|
+
|
|
141
|
+
### When Required
|
|
142
|
+
|
|
143
|
+
- FR/NFR annotation: `integration_test: true` or mentions integration verification
|
|
144
|
+
- Cross-project interactions (other services in your org)
|
|
145
|
+
- Interactions with local-provided services (Redis, Postgres, message bus)
|
|
146
|
+
- UI component browser testing (React → Storybook)
|
|
147
|
+
|
|
148
|
+
### Section Contents
|
|
149
|
+
|
|
150
|
+
Use template section `## Integration Test Matrix` with:
|
|
151
|
+
1. **Purpose column**: What integration is being tested
|
|
152
|
+
2. **Target column**: External system/service/project
|
|
153
|
+
3. **Type column**: `service` | `browser` | `event` | `database`
|
|
154
|
+
4. **Test Cases**: Extensive functional coverage, not just smoke tests
|
|
155
|
+
|
|
156
|
+
### Coverage Expectations
|
|
157
|
+
|
|
158
|
+
- Extensively test functionality (multiple scenarios per integration)
|
|
159
|
+
- Include happy path + error conditions + edge cases
|
|
160
|
+
- For browser tests: visual verification + interaction flows
|
|
161
|
+
|
|
162
|
+
## Markers
|
|
163
|
+
|
|
164
|
+
- ✅ Complete
|
|
165
|
+
- ⚠️ Partial
|
|
166
|
+
- ❌ Missing
|
|
167
|
+
- 🚧 In Progress
|
|
168
|
+
- ⛔ Retired
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# [TC-001] Clone Public GitHub Repository Successfully
|
|
2
|
+
|
|
3
|
+
## Test Information
|
|
4
|
+
- **Type**: Integration
|
|
5
|
+
- **Traces To**: US-001-AC-1, FR-001-AC-1
|
|
6
|
+
- **Priority**: P0
|
|
7
|
+
- **Status**: ✅ Pass
|
|
8
|
+
|
|
9
|
+
## Preconditions
|
|
10
|
+
- Public GitHub repository exists with valid Kubernetes configs
|
|
11
|
+
- Test data: `https://github.com/example/sample-k8s-project`
|
|
12
|
+
|
|
13
|
+
## Test Steps
|
|
14
|
+
1. Submit ingestion request with public repository URL
|
|
15
|
+
2. Wait for ingestion to complete
|
|
16
|
+
3. Query catalog for project components
|
|
17
|
+
|
|
18
|
+
## Expected Result
|
|
19
|
+
Project is ingested with all components discovered
|
|
20
|
+
|
|
21
|
+
## Verification
|
|
22
|
+
- **Options Tested**: FR-001-OPT-A (shallow clone)
|
|
23
|
+
- **Constraints Verified**: FR-001-CON-1 (size limit), FR-001-CON-3 (read-only)
|
|
24
|
+
|
|
25
|
+
## Test Data
|
|
26
|
+
```json
|
|
27
|
+
{
|
|
28
|
+
"repository_url": "https://github.com/example/sample-k8s-project",
|
|
29
|
+
"branch": "main"
|
|
30
|
+
}
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## Notes
|
|
34
|
+
- Verifies basic public repository ingestion flow
|
|
35
|
+
- Tests the most common use case
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# [TC-002] Clone Private GitHub Repository with Auth Token
|
|
2
|
+
|
|
3
|
+
## Test Information
|
|
4
|
+
- **Type**: Integration
|
|
5
|
+
- **Traces To**: US-001-AC-1, FR-001-AC-2
|
|
6
|
+
- **Priority**: P0
|
|
7
|
+
- **Status**: ✅ Pass
|
|
8
|
+
|
|
9
|
+
## Preconditions
|
|
10
|
+
- Private GitHub repository exists
|
|
11
|
+
- Valid Personal Access Token (PAT) available
|
|
12
|
+
- Test data: `https://github.com/example/private-k8s-project`
|
|
13
|
+
|
|
14
|
+
## Test Steps
|
|
15
|
+
1. Submit ingestion request with private repository URL and auth token
|
|
16
|
+
2. Wait for ingestion to complete
|
|
17
|
+
3. Query catalog for project components
|
|
18
|
+
|
|
19
|
+
## Expected Result
|
|
20
|
+
Project is ingested with all components discovered
|
|
21
|
+
|
|
22
|
+
## Verification
|
|
23
|
+
- **Options Tested**: FR-001-OPT-A (shallow clone)
|
|
24
|
+
- **Constraints Verified**: FR-001-CON-3 (read-only)
|
|
25
|
+
|
|
26
|
+
## Test Data
|
|
27
|
+
```json
|
|
28
|
+
{
|
|
29
|
+
"repository_url": "https://github.com/example/private-k8s-project",
|
|
30
|
+
"branch": "main",
|
|
31
|
+
"auth_token": "ghp_xxxxxxxxxxxxxxxxxxxx"
|
|
32
|
+
}
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
## Notes
|
|
36
|
+
- Verifies authentication flow for private repositories
|
|
37
|
+
- Token should have read-only access
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# [TC-006] Identify Deployment Resources
|
|
2
|
+
|
|
3
|
+
## Test Information
|
|
4
|
+
- **Type**: Unit
|
|
5
|
+
- **Traces To**: US-001-AC-3, FR-001-AC-3
|
|
6
|
+
- **Priority**: P0
|
|
7
|
+
- **Status**: ✅ Pass
|
|
8
|
+
|
|
9
|
+
## Preconditions
|
|
10
|
+
- Sample Deployment YAML file available
|
|
11
|
+
|
|
12
|
+
## Test Steps
|
|
13
|
+
1. Parse YAML file containing Deployment resource
|
|
14
|
+
2. Identify resource kind
|
|
15
|
+
3. Extract component metadata (name, image, env vars)
|
|
16
|
+
|
|
17
|
+
## Expected Result
|
|
18
|
+
Resource correctly identified as Deployment with all metadata extracted
|
|
19
|
+
|
|
20
|
+
## Verification
|
|
21
|
+
- **Options Tested**: N/A
|
|
22
|
+
- **Constraints Verified**: None
|
|
23
|
+
|
|
24
|
+
## Test Data
|
|
25
|
+
```yaml
|
|
26
|
+
apiVersion: apps/v1
|
|
27
|
+
kind: Deployment
|
|
28
|
+
metadata:
|
|
29
|
+
name: catalog-service
|
|
30
|
+
spec:
|
|
31
|
+
replicas: 3
|
|
32
|
+
selector:
|
|
33
|
+
matchLabels:
|
|
34
|
+
app: catalog
|
|
35
|
+
template:
|
|
36
|
+
metadata:
|
|
37
|
+
labels:
|
|
38
|
+
app: catalog
|
|
39
|
+
spec:
|
|
40
|
+
containers:
|
|
41
|
+
- name: catalog
|
|
42
|
+
image: catalog-service:v1.0.0
|
|
43
|
+
env:
|
|
44
|
+
- name: DATABASE_URL
|
|
45
|
+
value: "postgresql://db:5432/catalog"
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## Notes
|
|
49
|
+
- Tests core YAML parsing logic
|
|
50
|
+
- Validates metadata extraction accuracy
|
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
# Test Matrix - Example
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
This is an example test matrix demonstrating comprehensive test coverage for the Catalog Project Ingestion feature.
|
|
5
|
+
|
|
6
|
+
## Test Matrix Rules
|
|
7
|
+
1. **Coverage Rule**: Every acceptance criterion (AC) must have at least one test case
|
|
8
|
+
2. **Option Permutation Rule**: Test all valid combinations of options
|
|
9
|
+
3. **Constraint Boundary Rule**: Test at boundaries of each constraint
|
|
10
|
+
4. **Error Path Rule**: Test all documented error conditions
|
|
11
|
+
5. **State Transition Rule**: Test all valid state transitions (if applicable)
|
|
12
|
+
6. **Edge Case Rule**: Identify and test edge cases for each requirement
|
|
13
|
+
|
|
14
|
+
## Requirements Traceability
|
|
15
|
+
|
|
16
|
+
### User Story Coverage
|
|
17
|
+
| User Story | Acceptance Criteria | Test Cases | Coverage Status |
|
|
18
|
+
|------------|---------------------|------------|-----------------|
|
|
19
|
+
| US-001 | US-001-AC-1 | TC-001, TC-002, TC-003 | ✅ Complete |
|
|
20
|
+
| US-001 | US-001-AC-2 | TC-004, TC-005 | ✅ Complete |
|
|
21
|
+
| US-001 | US-001-AC-3 | TC-006, TC-007, TC-008 | ✅ Complete |
|
|
22
|
+
|
|
23
|
+
### Functional Requirement Coverage
|
|
24
|
+
| Functional Req | Acceptance Criteria | Test Cases | Coverage Status |
|
|
25
|
+
|----------------|---------------------|------------|-----------------|
|
|
26
|
+
| FR-001 | FR-001-AC-1 | TC-001 | ✅ Complete |
|
|
27
|
+
| FR-001 | FR-001-AC-2 | TC-002 | ✅ Complete |
|
|
28
|
+
| FR-001 | FR-001-AC-3 | TC-006, TC-022 | ✅ Complete |
|
|
29
|
+
| FR-001 | FR-001-AC-4 | TC-007 | ✅ Complete |
|
|
30
|
+
| FR-001 | FR-001-AC-5 | TC-008 | ✅ Complete |
|
|
31
|
+
| FR-001 | FR-001-AC-6 | TC-009 | ✅ Complete |
|
|
32
|
+
| FR-001 | FR-001-AC-7 | TC-010 | ✅ Complete |
|
|
33
|
+
| FR-001 | FR-001-AC-8 | TC-011 | ✅ Complete |
|
|
34
|
+
|
|
35
|
+
## Test Case Summary
|
|
36
|
+
|
|
37
|
+
| Test ID | Title | Type | Priority | Traces To | Status |
|
|
38
|
+
|---------|-------|------|----------|-----------|--------|
|
|
39
|
+
| TC-001 | Clone Public GitHub Repository Successfully | Integration | P0 | US-001-AC-1, FR-001-AC-1 | ✅ Pass |
|
|
40
|
+
| TC-002 | Clone Private GitHub Repository with Auth Token | Integration | P0 | US-001-AC-1, FR-001-AC-2 | ✅ Pass |
|
|
41
|
+
| TC-003 | Authentication Failure on Private Repository | Integration | P1 | US-001-AC-1 | ✅ Pass |
|
|
42
|
+
| TC-004 | Ingest from Local Filesystem Path | Integration | P0 | US-001-AC-2 | ✅ Pass |
|
|
43
|
+
| TC-005 | Local Path Does Not Exist | Integration | P1 | US-001-AC-2 | ✅ Pass |
|
|
44
|
+
| TC-006 | Identify Deployment Resources | Unit | P0 | US-001-AC-3, FR-001-AC-3 | ✅ Pass |
|
|
45
|
+
| TC-007 | Identify Service Resources | Unit | P0 | US-001-AC-3, FR-001-AC-4 | ✅ Pass |
|
|
46
|
+
| TC-008 | Extract Environment Variables from Deployment | Unit | P0 | US-001-AC-3, FR-001-AC-5 | ✅ Pass |
|
|
47
|
+
| TC-009 | Handle Repository with No Kubernetes Configs | Integration | P1 | FR-001-AC-6 | ✅ Pass |
|
|
48
|
+
| TC-010 | Reject Repository Exceeding Size Limit | Integration | P1 | FR-001-AC-7 | ✅ Pass |
|
|
49
|
+
| TC-011 | Handle Clone Timeout | Integration | P2 | FR-001-AC-8 | ✅ Pass |
|
|
50
|
+
| TC-012 | Full Clone Option | Integration | P2 | FR-001-OPT-B | 🚧 In Progress |
|
|
51
|
+
| TC-013 | Repository at Size Limit | Integration | P2 | FR-001-CON-1 | 🚧 In Progress |
|
|
52
|
+
| TC-014 | Clone at Timeout Boundary | Integration | P2 | FR-001-CON-2 | 🚧 In Progress |
|
|
53
|
+
| TC-015 | Monorepo with Multiple K8s Projects | Integration | P2 | EC-001 | ⚠️ Partial |
|
|
54
|
+
| TC-016 | Mixed K8s and Non-K8s YAML Files | Integration | P2 | EC-002 | ⚠️ Partial |
|
|
55
|
+
| TC-017 | Malformed YAML Handling | Integration | P1 | EC-003 | ❌ Missing |
|
|
56
|
+
| TC-018 | Symlinks to K8s Configs | Integration | P2 | EC-004 | ❌ Missing |
|
|
57
|
+
| TC-019 | Repository Changes During Clone | Integration | P2 | EC-005 | ❌ Missing |
|
|
58
|
+
| TC-022 | Resource identification is order-independent and idempotent over any generated manifest set | Property | P1 | FR-001-AC-3 | ✅ Pass |
|
|
59
|
+
|
|
60
|
+
**Note**: Detailed test case definitions are in the `test-cases/` directory.
|
|
61
|
+
|
|
62
|
+
**Note on `Type`**: TC-022 is `Property`, not `Unit`, because FR-001-AC-3 is
|
|
63
|
+
quantified — it holds for *any* manifest set, not for one worked example. TC-006
|
|
64
|
+
witnesses the criterion with a specific input; TC-022 discriminates it across a
|
|
65
|
+
generated domain. Both are legitimate and they trace to the same AC.
|
|
66
|
+
|
|
67
|
+
## Option Permutation Matrix
|
|
68
|
+
| Test Case | OPT-A (Shallow) | OPT-B (Full) | Expected Behavior |
|
|
69
|
+
|-----------|-----------------|--------------|-------------------|
|
|
70
|
+
| TC-001 | Y | N | Fast clone, only latest commit |
|
|
71
|
+
| TC-012 | N | Y | Slower clone, full history available |
|
|
72
|
+
|
|
73
|
+
## Constraint Boundary Tests
|
|
74
|
+
| Constraint | Boundary Type | Test Value | Test Case | Expected |
|
|
75
|
+
|------------|---------------|------------|-----------|----------|
|
|
76
|
+
| FR-001-CON-1 | Max Size | 1GB | TC-013 | Pass (at limit) |
|
|
77
|
+
| FR-001-CON-1 | Above Max | 1.1GB | TC-010 | Error CATALOG-004 |
|
|
78
|
+
| FR-001-CON-2 | At Timeout | 5 minutes | TC-014 | Pass (just under) |
|
|
79
|
+
| FR-001-CON-2 | Exceed Timeout | 5.5 minutes | TC-011 | Error CATALOG-005 |
|
|
80
|
+
|
|
81
|
+
## Edge Cases
|
|
82
|
+
| ID | Description | Related Req | Test Case | Risk if Untested |
|
|
83
|
+
|----|-------------|-------------|-----------|------------------|
|
|
84
|
+
| EC-001 | Monorepo with multiple K8s projects | US-001 | TC-015 | May miss components or create duplicate entries |
|
|
85
|
+
| EC-002 | Repository with mixed K8s and non-K8s YAML files | FR-001 | TC-016 | May incorrectly parse non-K8s YAML as components |
|
|
86
|
+
| EC-003 | Malformed YAML that crashes parser | FR-001 | TC-017 | Service crash or incomplete ingestion |
|
|
87
|
+
| EC-004 | Repository with symlinks to K8s configs | FR-001 | TC-018 | May not follow symlinks, miss components |
|
|
88
|
+
| EC-005 | Repository changes during clone/scan | FR-001 | TC-019 | Inconsistent state or race conditions |
|
|
89
|
+
|
|
90
|
+
## Coverage Gaps
|
|
91
|
+
| Gap ID | Description | Risk Level | Mitigation |
|
|
92
|
+
|--------|-------------|------------|------------|
|
|
93
|
+
| GAP-001 | No tests for GitLab/Bitbucket | Low | Not in scope for v1, document for future |
|
|
94
|
+
| GAP-002 | No performance tests for concurrent ingestions | Medium | Add TC-020 to test 10 concurrent ingestions |
|
|
95
|
+
| GAP-003 | No tests for webhook-triggered re-ingestion | High | Add TC-021 for GitHub webhook integration |
|
|
96
|
+
|
|
97
|
+
## Test Execution Summary
|
|
98
|
+
| Category | Total | Passed | Failed | Blocked | Coverage |
|
|
99
|
+
|----------|-------|--------|--------|---------|----------|
|
|
100
|
+
| Unit | 3 | 3 | 0 | 0 | 100% |
|
|
101
|
+
| Integration | 8 | 8 | 0 | 0 | 100% |
|
|
102
|
+
| E2E | 0 | 0 | 0 | 0 | N/A |
|
|
103
|
+
| Property | 1 | 1 | 0 | 0 | 100% |
|
|
104
|
+
| Edge Cases | 5 | 3 | 0 | 2 | 60% |
|