ultra-builder-pro-cli 0.10.0 → 0.12.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.ultra-template/specs/architecture.md +54 -223
- package/.ultra-template/specs/discovery.md +34 -231
- package/.ultra-template/specs/product.md +44 -200
- package/.ultra-template/tasks/contexts/TEMPLATE.md +36 -0
- package/.ultra-template/tasks/tasks.json +6 -0
- package/CHANGELOG.md +86 -1
- package/README.md +21 -9
- package/adapters/_shared/codex-assets.cjs +24 -342
- package/adapters/_shared/handbook.cjs +4 -3
- package/adapters/_shared/runtime-assets.cjs +24 -0
- package/adapters/_shared/tests/handbook.test.cjs +4 -0
- package/adapters/_shared/tests/runtime-assets.test.cjs +11 -4
- package/adapters/claude.js +28 -5
- package/adapters/kimi.js +15 -138
- package/adapters/opencode.js +22 -49
- package/adapters/tests/claude.test.cjs +22 -8
- package/adapters/tests/codex.test.cjs +21 -8
- package/adapters/tests/kimi-hook.test.cjs +1 -1
- package/adapters/tests/kimi.test.cjs +11 -11
- package/adapters/tests/opencode.test.cjs +47 -8
- package/agents/code-reviewer.md +24 -85
- package/agents/debugger.md +30 -132
- package/agents/review-code.md +24 -104
- package/agents/review-comments.md +20 -83
- package/agents/review-coordinator.md +40 -137
- package/agents/review-design.md +21 -118
- package/agents/review-errors.md +22 -70
- package/agents/review-spec.md +36 -0
- package/agents/review-tests.md +20 -76
- package/agents/tdd-runner.md +26 -84
- package/commands/learn.md +5 -36
- package/commands/ultra-change.md +5 -43
- package/commands/ultra-deliver.md +4 -46
- package/commands/ultra-dev.md +4 -48
- package/commands/ultra-doctor.md +4 -30
- package/commands/ultra-init.md +5 -47
- package/commands/ultra-plan.md +5 -48
- package/commands/ultra-research.md +5 -54
- package/commands/ultra-status.md +5 -44
- package/commands/ultra-test.md +5 -45
- package/commands/ultra-think.md +5 -40
- package/docs/AGENT-CONTEXT.md +16 -8
- package/docs/ARCHITECTURE.md +32 -5
- package/docs/PLAN.zh-CN.md +8 -6
- package/docs/ROADMAP.md +5 -3
- package/docs/RUNTIME-COMPAT-MATRIX.md +8 -5
- package/docs/SKILL-AUTHORING.md +75 -0
- package/docs/STATE-DB-ACCESS-POLICY.md +17 -11
- package/docs/USER-HANDBOOK-CONTRACT.md +8 -5
- package/hooks/README.md +4 -3
- package/hooks/adapters/codex.py +1 -1
- package/hooks/adapters/kimi.py +1 -1
- package/hooks/context_spine.py +120 -16
- package/hooks/health_check.py +11 -0
- package/hooks/pre_stop_check.py +6 -5
- package/mcp-server/lib/baseline-workflow.cjs +415 -0
- package/mcp-server/lib/baseline-workflow.test.cjs +225 -0
- package/mcp-server/lib/change-workflow.cjs +24 -27
- package/mcp-server/lib/change-workflow.test.cjs +202 -6
- package/mcp-server/lib/context-spine.cjs +74 -9
- package/mcp-server/lib/context-spine.test.cjs +15 -0
- package/mcp-server/lib/doctor.cjs +14 -0
- package/mcp-server/lib/doctor.test.cjs +25 -1
- package/mcp-server/lib/init-project.cjs +49 -14
- package/mcp-server/lib/provider-refs.cjs +42 -0
- package/mcp-server/lib/spec-learning.test.cjs +5 -0
- package/mcp-server/lib/state-db.cjs +63 -4
- package/mcp-server/lib/state-db.test.cjs +8 -3
- package/mcp-server/server.cjs +48 -3
- package/mcp-server/tests/init-project.test.cjs +54 -8
- package/mcp-server/tests/server-contract.test.cjs +99 -2
- package/mcp-server/tests/state-db.test.cjs +11 -2
- package/package.json +3 -2
- package/skills/cc-collab/SKILL.md +16 -18
- package/skills/code-review-expert/SKILL.md +39 -114
- package/skills/code-review-expert/references/correctness-reliability.md +30 -0
- package/skills/code-review-expert/references/design-boundaries.md +26 -0
- package/skills/code-review-expert/references/removal-plan.md +17 -45
- package/skills/codex-collab/SKILL.md +23 -28
- package/skills/codex-collab/evals/evals.json +18 -23
- package/skills/integration-rules/SKILL.md +28 -67
- package/skills/learn/SKILL.md +46 -106
- package/skills/security-rules/SKILL.md +34 -56
- package/skills/testing-rules/SKILL.md +28 -52
- package/skills/ultra-change/SKILL.md +68 -114
- package/skills/ultra-deliver/SKILL.md +58 -86
- package/skills/ultra-dev/SKILL.md +46 -105
- package/skills/ultra-doctor/SKILL.md +45 -72
- package/skills/ultra-init/SKILL.md +62 -147
- package/skills/ultra-init/references/brownfield-adoption.md +65 -0
- package/skills/ultra-plan/SKILL.md +56 -89
- package/skills/ultra-research/SKILL.md +64 -152
- package/skills/ultra-research/references/architecture.md +38 -0
- package/skills/ultra-research/references/problem-product.md +39 -0
- package/skills/ultra-research/references/quality-delivery.md +34 -0
- package/skills/ultra-research/references/synthesis.md +38 -0
- package/skills/ultra-research/references/users-scope.md +40 -0
- package/skills/ultra-review/SKILL.md +54 -98
- package/skills/ultra-review/references/unified-schema.md +76 -101
- package/skills/ultra-review/scripts/review_wait.py +242 -77
- package/skills/ultra-status/SKILL.md +41 -56
- package/skills/ultra-test/SKILL.md +37 -72
- package/skills/ultra-think/SKILL.md +43 -102
- package/skills/ultra-verify/SKILL.md +44 -84
- package/skills/ultra-verify/evals/evals.json +24 -27
- package/spec/cli-protocol.md +4 -0
- package/spec/mcp-tools.yaml +223 -4
- package/spec/schemas/command-manifest.schema.json +7 -25
- package/spec/schemas/mcp-tools.schema.json +1 -1
- package/spec/schemas/skill-manifest.schema.json +9 -79
- package/spec/schemas/state-db.sql +33 -2
- package/ultra-tools/commands/status.cjs +20 -2
- package/ultra-tools/commands/task.cjs +4 -1
- package/ultra-tools/status-command.test.cjs +20 -0
- package/ultra-tools/system-command.test.cjs +5 -0
- package/skills/code-review-expert/references/code-quality-checklist.md +0 -112
- package/skills/code-review-expert/references/integration-checklist.md +0 -46
- package/skills/code-review-expert/references/security-checklist.md +0 -100
- package/skills/code-review-expert/references/solid-checklist.md +0 -65
- package/skills/ultra-research/steps/step-00-problem-validation.md +0 -154
- package/skills/ultra-research/steps/step-01-opportunity-discovery.md +0 -172
- package/skills/ultra-research/steps/step-02-market-assessment.md +0 -195
- package/skills/ultra-research/steps/step-03-competitive-landscape.md +0 -197
- package/skills/ultra-research/steps/step-04-product-strategy.md +0 -196
- package/skills/ultra-research/steps/step-05-assumptions-validation.md +0 -234
- package/skills/ultra-research/steps/step-10-user-personas.md +0 -169
- package/skills/ultra-research/steps/step-11-user-scenarios.md +0 -181
- package/skills/ultra-research/steps/step-20-user-stories.md +0 -156
- package/skills/ultra-research/steps/step-21-features-scope.md +0 -157
- package/skills/ultra-research/steps/step-22-success-metrics.md +0 -195
- package/skills/ultra-research/steps/step-30-architecture-context.md +0 -154
- package/skills/ultra-research/steps/step-31-solution-strategy.md +0 -172
- package/skills/ultra-research/steps/step-32-building-blocks.md +0 -188
- package/skills/ultra-research/steps/step-40-deployment.md +0 -123
- package/skills/ultra-research/steps/step-41-quality-risks.md +0 -135
- package/skills/ultra-research/steps/step-99-synthesis.md +0 -188
- package/skills/ultra-review/scripts/review_verdict_update.py +0 -99
|
@@ -1,71 +1,32 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: integration-rules
|
|
3
|
-
description:
|
|
4
|
-
user-invocable: false
|
|
3
|
+
description: Verify that changed components are reachable, contract-compatible, and tested across the boundaries required by the accepted behavior. Use only when an assigned review or implementation slice crosses modules, services, persistence, protocols, or UI seams.
|
|
5
4
|
---
|
|
6
5
|
|
|
7
|
-
#
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
##
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|------|--------|----------|
|
|
35
|
-
| 1 | Define interface before implementation | TypeScript interface, OpenAPI schema, or protobuf |
|
|
36
|
-
| 2 | Both sides code against the contract | Import shared types or generated clients |
|
|
37
|
-
| 3 | Contract test validates both sides | Test that producer output matches consumer expectation |
|
|
38
|
-
|
|
39
|
-
## Integration Test Requirements
|
|
40
|
-
|
|
41
|
-
| Boundary | Required Test |
|
|
42
|
-
|----------|---------------|
|
|
43
|
-
| HTTP API endpoint | Request with real HTTP client, validate response shape + status |
|
|
44
|
-
| Database operation | Testcontainers with real queries, validate data persisted |
|
|
45
|
-
| Message queue | Real producer + consumer, validate message delivered |
|
|
46
|
-
| External service | Test Double with `// Test Double rationale:` (only exception) |
|
|
47
|
-
| In-process module boundary | Real function call, validate input/output contract |
|
|
48
|
-
|
|
49
|
-
## Orphan Detection
|
|
50
|
-
|
|
51
|
-
Code without a live entry point is dead-on-arrival.
|
|
52
|
-
|
|
53
|
-
**Valid entry points** (at least one required per new module):
|
|
54
|
-
- HTTP/WebSocket handler
|
|
55
|
-
- CLI command handler
|
|
56
|
-
- Event/message listener
|
|
57
|
-
- Scheduled job/cron handler
|
|
58
|
-
- Exported function called by a module that has an entry point
|
|
59
|
-
|
|
60
|
-
**Detection**: Trace from new code upward — if no path reaches an entry point, the code is an orphan.
|
|
61
|
-
|
|
62
|
-
## Detection Checklist
|
|
63
|
-
|
|
64
|
-
When reviewing code, flag:
|
|
65
|
-
1. New module/service without any caller (orphan code)
|
|
66
|
-
2. Task structured as horizontal layer instead of vertical slice
|
|
67
|
-
3. Two components communicating without shared interface/contract
|
|
68
|
-
4. New boundary crossing without integration test
|
|
69
|
-
5. Use case with DB/API dependency but only unit tests
|
|
70
|
-
6. Component "works in isolation" but not wired to any entry point
|
|
71
|
-
7. Interface defined but no contract test validating compatibility
|
|
6
|
+
# Verify integration and reachability
|
|
7
|
+
|
|
8
|
+
Prove that new behavior is connected to a real consumer and that data and failures
|
|
9
|
+
cross each boundary according to one contract.
|
|
10
|
+
|
|
11
|
+
## Procedure
|
|
12
|
+
|
|
13
|
+
1. Start at the declared public seam or entry point and trace the changed path through
|
|
14
|
+
domain behavior, state, side effects, and the final consumer.
|
|
15
|
+
2. At each boundary, identify the producer, consumer, data shape, ownership, error
|
|
16
|
+
semantics, compatibility expectation, and recovery behavior.
|
|
17
|
+
3. Confirm both sides use the same contract or a deliberate adapter. Inspect runtime
|
|
18
|
+
configuration and registration, not only exported types.
|
|
19
|
+
4. Verify the implementation is reachable in the production path. A module, handler,
|
|
20
|
+
schema, or component with no caller or registration is incomplete.
|
|
21
|
+
5. Check that tests cross the boundary whose semantics matter and would fail on a
|
|
22
|
+
producer-consumer mismatch.
|
|
23
|
+
6. For a larger feature, confirm an executable vertical slice exists before broad
|
|
24
|
+
expansion. The slice may be narrow, but it must use real wiring and observable data.
|
|
25
|
+
7. Inspect partial failure, retry, idempotency, ordering, rollback, and observability
|
|
26
|
+
when the boundary can leave inconsistent state.
|
|
27
|
+
|
|
28
|
+
## Finding contract
|
|
29
|
+
|
|
30
|
+
Report the broken path, mismatched contract, missing consumer, or unverified failure
|
|
31
|
+
mode with exact source evidence and observable impact. Do not flag a horizontal task
|
|
32
|
+
solely by its name when another completed slice already proves the integration.
|
package/skills/learn/SKILL.md
CHANGED
|
@@ -1,118 +1,58 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: learn
|
|
3
|
-
description:
|
|
4
|
-
user-invocable: true
|
|
3
|
+
description: Extract one verified reusable workflow from completed work into a portable user skill after explicit approval. Use when the user asks to preserve a repeatable method that does not belong in project guidance or ordinary documentation.
|
|
5
4
|
---
|
|
6
5
|
|
|
7
|
-
#
|
|
6
|
+
# Capture a reusable skill
|
|
8
7
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
confirmation is mandatory before the write — never silently persist.
|
|
8
|
+
Create a focused workflow from evidence in the current task. Do not capture prompts,
|
|
9
|
+
transcripts, private memory, or one-off incident details.
|
|
12
10
|
|
|
13
|
-
##
|
|
11
|
+
## Placement gate
|
|
14
12
|
|
|
15
|
-
|
|
16
|
-
generalizable (not a one-off typo or outage-specific fix).
|
|
13
|
+
Before drafting, decide whether the material belongs in a skill:
|
|
17
14
|
|
|
18
|
-
|
|
15
|
+
- durable repository convention: update the nearest project guidance instead;
|
|
16
|
+
- product or architecture fact: update project documentation instead;
|
|
17
|
+
- external live data or action: use an MCP integration instead;
|
|
18
|
+
- repeatable procedure with non-obvious steps, references, or scripts: create a skill.
|
|
19
19
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
| **Error resolution** | Root cause + fix that applies to similar errors |
|
|
23
|
-
| **Debugging technique** | Non-obvious steps, effective tool combos |
|
|
24
|
-
| **Workaround** | Library quirk / API limitation / version-specific fix |
|
|
25
|
-
| **Project-specific pattern** | Codebase convention / integration pattern discovered this session |
|
|
20
|
+
Stop when the candidate is a typo fix, transient outage, generic advice, unsupported
|
|
21
|
+
hypothesis, or a duplicate of an existing skill.
|
|
26
22
|
|
|
27
23
|
## Workflow
|
|
28
24
|
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
---
|
|
64
|
-
|
|
65
|
-
# <Descriptive Pattern Name>
|
|
66
|
-
|
|
67
|
-
**Extracted**: <YYYY-MM-DD>
|
|
68
|
-
**Confidence**: Speculation (unverified)
|
|
69
|
-
**Context**: <one-sentence applicability>
|
|
70
|
-
|
|
71
|
-
## Problem
|
|
72
|
-
<what problem does this pattern solve — be specific>
|
|
73
|
-
|
|
74
|
-
## Solution
|
|
75
|
-
<the pattern / technique / workaround>
|
|
76
|
-
|
|
77
|
-
## Example
|
|
78
|
-
<code example if applicable — else a transcribed command sequence>
|
|
79
|
-
|
|
80
|
-
## Trigger Conditions
|
|
81
|
-
<what situations should activate this skill>
|
|
82
|
-
|
|
83
|
-
## Verification Status
|
|
84
|
-
- [ ] Human review passed
|
|
85
|
-
- [ ] Multiple successful uses
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
## Verification upgrade path
|
|
89
|
-
|
|
90
|
-
1. **Speculation** (this skill writes this level) — freshly extracted, unverified
|
|
91
|
-
2. **Inference** — after human review passes, rename the directory to remove
|
|
92
|
-
`-unverified` and update the frontmatter name
|
|
93
|
-
3. **Fact** — after multiple successful uses confirm the pattern
|
|
94
|
-
|
|
95
|
-
## What NOT to extract
|
|
96
|
-
|
|
97
|
-
- Simple typo fixes
|
|
98
|
-
- One-time issues (API outage, transient CI flake)
|
|
99
|
-
- Patterns too project-specific to reuse (belongs in project docs, not a skill)
|
|
100
|
-
- Anything already captured in CLAUDE.md or equivalent project instructions
|
|
101
|
-
|
|
102
|
-
## Approval boundary
|
|
103
|
-
|
|
104
|
-
Ask the user directly before writing. No Ultra MCP or legacy interaction wrapper is involved.
|
|
105
|
-
|
|
106
|
-
## What this skill DOES NOT do
|
|
107
|
-
|
|
108
|
-
- Does NOT modify state.db
|
|
109
|
-
- Does NOT save without user approval
|
|
110
|
-
- Does NOT extract more than one pattern per run
|
|
111
|
-
|
|
112
|
-
## Integration
|
|
113
|
-
|
|
114
|
-
| | |
|
|
115
|
-
|---|---|
|
|
116
|
-
| **Input** | current session transcript / recent tool outputs |
|
|
117
|
-
| **Output** | `~/.claude/skills/learned-<slug>-unverified/SKILL.md` |
|
|
118
|
-
| **Next** | user may promote after review; `/learn` again later when another pattern appears |
|
|
25
|
+
1. Identify one coherent procedure that succeeded in the current work. Preserve the
|
|
26
|
+
trigger, decisive steps, non-obvious failure modes, inputs, outputs, and validation
|
|
27
|
+
command; remove task-specific names, dates, branches, and conclusions.
|
|
28
|
+
2. Search the current host's user skill directory for overlap. Extend an existing
|
|
29
|
+
skill only when its job and trigger remain coherent.
|
|
30
|
+
3. Draft a portable skill directory:
|
|
31
|
+
|
|
32
|
+
```text
|
|
33
|
+
<skill-name>/
|
|
34
|
+
SKILL.md
|
|
35
|
+
scripts/ # only deterministic reusable helpers
|
|
36
|
+
references/ # only conditional detailed guidance
|
|
37
|
+
assets/ # only reusable templates or resources
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
4. Keep `SKILL.md` focused on the procedure. Its frontmatter contains only `name` and
|
|
41
|
+
`description`; the description states both what the skill does and when to use it.
|
|
42
|
+
Move detailed schemas, long examples, and optional branches into focused resources
|
|
43
|
+
and say exactly when to load them.
|
|
44
|
+
5. Present the proposed name, trigger, file inventory, and concise draft to the user.
|
|
45
|
+
Write nothing until the user approves.
|
|
46
|
+
6. Write to the current host's user skill directory without overwriting an unrelated
|
|
47
|
+
skill. Add host-specific UI or invocation metadata outside `SKILL.md` when the host
|
|
48
|
+
supports it.
|
|
49
|
+
7. Run the available Agent Skills validator and at least one positive and one negative
|
|
50
|
+
trigger check. Report any behavior that remains unverified.
|
|
51
|
+
|
|
52
|
+
## Quality gate
|
|
53
|
+
|
|
54
|
+
The captured skill must be reusable without the original conversation, grounded in
|
|
55
|
+
observed evidence, concise enough to compete safely for context, and explicit about
|
|
56
|
+
validation. Create one skill per coherent job, not one skill per fact.
|
|
57
|
+
|
|
58
|
+
This workflow never modifies Ultra state and never runs automatically at task close.
|
|
@@ -1,60 +1,38 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: security-rules
|
|
3
|
-
description:
|
|
4
|
-
user-invocable: false
|
|
3
|
+
description: Evaluate changed trust boundaries for exploitable input, authorization, secret, data-exposure, and supply-chain failures. Use only when an assigned review scope contains a security-relevant boundary.
|
|
5
4
|
---
|
|
6
5
|
|
|
7
|
-
#
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
## Error Handling Security
|
|
42
|
-
|
|
43
|
-
- Never expose stack traces to end users
|
|
44
|
-
- Never include sensitive data in error messages
|
|
45
|
-
- Log security events with sufficient context for investigation
|
|
46
|
-
- Use typed errors, not generic messages
|
|
47
|
-
|
|
48
|
-
## Review Checklist
|
|
49
|
-
|
|
50
|
-
When reviewing code, check for:
|
|
51
|
-
1. SQL injection vectors (string concatenation in queries)
|
|
52
|
-
2. XSS vectors (unescaped user input in HTML/templates)
|
|
53
|
-
3. Hardcoded credentials, API keys, or secrets
|
|
54
|
-
4. Missing authentication/authorization checks
|
|
55
|
-
5. Missing input validation on external boundaries
|
|
56
|
-
6. Insecure direct object references (IDOR)
|
|
57
|
-
7. Missing rate limiting on sensitive endpoints
|
|
58
|
-
8. Sensitive data in logs or error messages
|
|
59
|
-
9. Missing CSRF protection on state-changing operations
|
|
60
|
-
10. Insecure deserialization
|
|
6
|
+
# Review security-relevant changes
|
|
7
|
+
|
|
8
|
+
Follow data from an attacker-controlled or less-trusted source to a protected action,
|
|
9
|
+
secret, identity, or output. Report plausible exploit paths, not isolated pattern
|
|
10
|
+
matches.
|
|
11
|
+
|
|
12
|
+
## Review procedure
|
|
13
|
+
|
|
14
|
+
1. Identify trust boundaries, protected assets, principals, and state-changing actions
|
|
15
|
+
in the diff.
|
|
16
|
+
2. Trace input validation, canonicalization, authorization, and output handling through
|
|
17
|
+
the real call path.
|
|
18
|
+
3. Verify authorization is derived from trusted server state and applies to the
|
|
19
|
+
requested resource, tenant, or object.
|
|
20
|
+
4. Inspect persistence and command boundaries for injection, unsafe deserialization,
|
|
21
|
+
path traversal, and unbounded resource use.
|
|
22
|
+
5. Inspect logs, errors, telemetry, and client responses for secret or sensitive-data
|
|
23
|
+
disclosure.
|
|
24
|
+
6. Inspect changed authentication, cryptography, dependency, file-upload, webhook, and
|
|
25
|
+
cross-origin behavior when present.
|
|
26
|
+
7. Check the failure path: rate limits, replay or idempotency, partial writes, rollback,
|
|
27
|
+
and audit evidence where the risk requires them.
|
|
28
|
+
|
|
29
|
+
## Finding threshold
|
|
30
|
+
|
|
31
|
+
Report only when the current code supports a triggering input and a meaningful impact.
|
|
32
|
+
Include the source-to-sink path, affected asset, exploit preconditions, and a bounded
|
|
33
|
+
fix. Distinguish a missing defense from a defense implemented elsewhere in the live
|
|
34
|
+
path.
|
|
35
|
+
|
|
36
|
+
Treat confirmed secret exposure, authorization bypass, injection with material impact,
|
|
37
|
+
and destructive cross-tenant access as blocking. Calibrate other issues from their
|
|
38
|
+
actual reachability and blast radius rather than a fixed pattern-to-severity table.
|
|
@@ -1,56 +1,32 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: testing-rules
|
|
3
|
-
description:
|
|
4
|
-
user-invocable: false
|
|
3
|
+
description: Review whether changed behavior has meaningful, trustworthy, and proportionate test evidence across its real boundaries. Use only when assigned to a test review or delivery verification scope.
|
|
5
4
|
---
|
|
6
5
|
|
|
7
|
-
#
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
## Coverage Requirements
|
|
36
|
-
|
|
37
|
-
- 80% overall minimum
|
|
38
|
-
- 100% Functional Core (pure logic must be fully tested)
|
|
39
|
-
- Critical paths for Imperative Shell
|
|
40
|
-
|
|
41
|
-
## Dev/Prod Parity
|
|
42
|
-
|
|
43
|
-
Tests must use real dependencies:
|
|
44
|
-
- Real database via Testcontainers (not in-memory substitutes)
|
|
45
|
-
- Config via environment variables
|
|
46
|
-
- Mock tests passing ≠ production working
|
|
47
|
-
|
|
48
|
-
## Detection Checklist
|
|
49
|
-
|
|
50
|
-
When analyzing test files, flag:
|
|
51
|
-
1. Any `jest.fn()` usage on Repository, Service, or Domain classes
|
|
52
|
-
2. Any `InMemory*` or `Mock*` or `Fake*` class definitions
|
|
53
|
-
3. Any `jest.mock()` calls on internal modules
|
|
54
|
-
4. Any skipped tests with database/slow excuses
|
|
55
|
-
5. Missing error case coverage
|
|
56
|
-
6. Missing edge case coverage
|
|
6
|
+
# Review test evidence
|
|
7
|
+
|
|
8
|
+
Evaluate what the tests prove about the accepted behavior. Line coverage and the mere
|
|
9
|
+
presence of a test file are supporting signals, not the contract.
|
|
10
|
+
|
|
11
|
+
## Procedure
|
|
12
|
+
|
|
13
|
+
1. Map each changed behavior and acceptance claim to a test or other executable check.
|
|
14
|
+
2. Confirm the test can fail for the intended defect and observes a public or stable
|
|
15
|
+
contract rather than implementation trivia.
|
|
16
|
+
3. Inspect happy, error, boundary, state-transition, and concurrency cases according to
|
|
17
|
+
the risk introduced by the diff.
|
|
18
|
+
4. Verify integration seams with the most realistic practical boundary. Prefer real
|
|
19
|
+
persistence and protocol behavior when their semantics matter.
|
|
20
|
+
5. Accept test doubles at costly, unavailable, or nondeterministic external boundaries
|
|
21
|
+
when the double preserves the documented contract and the reason is clear.
|
|
22
|
+
6. Flag mocks or fakes only when they bypass the behavior under review, encode a false
|
|
23
|
+
contract, or make a passing test unrelated to production behavior.
|
|
24
|
+
7. Detect skipped checks, tautological assertions, swallowed failures, non-determinism,
|
|
25
|
+
weakened expectations, and tests that never reach the changed code.
|
|
26
|
+
|
|
27
|
+
## Finding contract
|
|
28
|
+
|
|
29
|
+
Name the unproven behavior, why current evidence is insufficient, the failure that can
|
|
30
|
+
escape, and the smallest test or boundary check that closes the gap. Calibrate severity
|
|
31
|
+
to product risk and reachability; do not impose a universal coverage percentage or a
|
|
32
|
+
single test technology on every repository.
|