@dtlabs/ai-toolkit 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/assessment-manager.md +218 -0
- package/.claude/agents/concurrency-safety-assessment.md +190 -0
- package/.claude/agents/define-feature.md +265 -0
- package/.claude/agents/dependency-injection-refactoring.md +237 -0
- package/.claude/agents/dependency-supply-chain-security.md +262 -0
- package/.claude/agents/developer-backend.md +105 -0
- package/.claude/agents/developer-frontend.md +111 -0
- package/.claude/agents/developer-testing.md +121 -0
- package/.claude/agents/domain-model-refactoring.md +208 -0
- package/.claude/agents/generate-requirements.md +168 -0
- package/.claude/agents/generate-tech-spec.md +259 -0
- package/.claude/agents/generate-work-breakdown.md +284 -0
- package/.claude/agents/generic-software-assessment.md +165 -0
- package/.claude/agents/god-class-decomposition.md +198 -0
- package/.claude/agents/init-agents-md.md +322 -0
- package/.claude/agents/install-toolkit.md +196 -0
- package/.claude/agents/intervention-documentation-standard.md +131 -0
- package/.claude/agents/layered-architecture-assessment.md +205 -0
- package/.claude/agents/project-manager.md +309 -0
- package/.claude/agents/review-solution.md +138 -0
- package/.claude/agents/security-hardening.md +240 -0
- package/.claude/agents/validate-feature-docs.md +171 -0
- package/.claude/commands/assessment-status.md +116 -0
- package/.claude/commands/check-docs.md +18 -0
- package/.claude/commands/feature-status.md +20 -0
- package/.claude/commands/next-task.md +37 -0
- package/.claude/commands/pr-description.md +34 -0
- package/.claude/skills/assess-codebase/SKILL.md +16 -0
- package/.claude/skills/define-feature/SKILL.md +13 -0
- package/.claude/skills/hi-gaia/SKILL.md +261 -0
- package/.claude/skills/implement-feature/SKILL.md +69 -0
- package/.claude/skills/init-agents/SKILL.md +14 -0
- package/.claude/skills/install-toolkit/SKILL.md +14 -0
- package/CLAUDE.md +135 -0
- package/README.md +77 -0
- package/bin/cli.js +350 -0
- package/docs/installation.md +80 -0
- package/docs/pricing.md +99 -0
- package/docs/procedures/approval-gates.md +142 -0
- package/docs/procedures/assessment-approval-gate.md +60 -0
- package/docs/procedures/code-generation.md +48 -0
- package/docs/procedures/code-review.md +63 -0
- package/docs/procedures/effort-estimation.md +86 -0
- package/docs/procedures/feature-registry.md +50 -0
- package/docs/procedures/issues-register.md +24 -0
- package/docs/procedures/process-log.md +28 -0
- package/docs/procedures/secure-coding.md +50 -0
- package/docs/procedures/testing.md +63 -0
- package/docs/procedures/token-estimation.md +123 -0
- package/docs/reference.md +228 -0
- package/package.json +23 -0
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# Procedure: Assessment Remediation Gate
|
|
2
|
+
|
|
3
|
+
After intervention documents are generated, present findings to the user and obtain explicit approval before any remediation code is written.
|
|
4
|
+
|
|
5
|
+
## Gate presentation
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
π Assessment Complete β ASSESS-001
|
|
9
|
+
βββββββββββββββββββββββββββββββββββββββ
|
|
10
|
+
Assessment results:
|
|
11
|
+
Generic quality: [KPI summary scores]
|
|
12
|
+
Layer violations: N CRITICAL, N HIGH, N MEDIUM
|
|
13
|
+
Concurrency hazards: N CRITICAL, N HIGH
|
|
14
|
+
[other assessments]
|
|
15
|
+
|
|
16
|
+
Interventions proposed: N total
|
|
17
|
+
CRITICAL: N (immediate attention)
|
|
18
|
+
HIGH: N (next sprint)
|
|
19
|
+
MEDIUM: N (planned cycle)
|
|
20
|
+
LOW: N (backlog)
|
|
21
|
+
|
|
22
|
+
Interventions Index: docs/assessments/ASSESS-001/ASSESS-001-Interventions-Index.md
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Use `AskUserQuestion`:
|
|
26
|
+
- **"Approve all"** β all proposed interventions approved
|
|
27
|
+
- **"Approve CRITICAL only"** β only CRITICAL severity approved
|
|
28
|
+
- **"Select interventions"** β user specifies INT-NNN to approve (comma-separated)
|
|
29
|
+
- **"Assessment only"** β stop here, no remediation
|
|
30
|
+
|
|
31
|
+
If the user selects **"Assessment only"**, skip to the Summary phase.
|
|
32
|
+
|
|
33
|
+
## Approvals file format
|
|
34
|
+
|
|
35
|
+
Write `{PREFIX}-Approvals.md` in the assessment directory:
|
|
36
|
+
|
|
37
|
+
```markdown
|
|
38
|
+
# Approval Record β {PREFIX}
|
|
39
|
+
|
|
40
|
+
## Assessment Approvals
|
|
41
|
+
|
|
42
|
+
| Document | Status | Date | Notes |
|
|
43
|
+
|----------|--------|------|-------|
|
|
44
|
+
| {PREFIX}-Generic-Assessment.md | β
Reviewed | {date} | β |
|
|
45
|
+
| {PREFIX}-Interventions-Index.md | β
Reviewed | {date} | β |
|
|
46
|
+
|
|
47
|
+
## Remediation Scope
|
|
48
|
+
|
|
49
|
+
| Intervention | Approved | Date | Notes |
|
|
50
|
+
|---|---|---|---|
|
|
51
|
+
| INT-001 β sql-injection-hardening | β
Approved | {date} | β |
|
|
52
|
+
| INT-002 β god-class-decomposition | β
Approved | {date} | β |
|
|
53
|
+
| INT-003 β di-refactoring | β Deferred | {date} | Out of current sprint scope |
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## Rules
|
|
57
|
+
|
|
58
|
+
- **Never skip this gate** β remediation cannot start without explicit approval
|
|
59
|
+
- **Log the approval outcome** in the process log
|
|
60
|
+
- **Record every decision** including deferred interventions and the reason
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# Code Generation Procedure
|
|
2
|
+
|
|
3
|
+
Step-by-step workflow for generating or modifying production code.
|
|
4
|
+
|
|
5
|
+
## When to use
|
|
6
|
+
|
|
7
|
+
Before writing any new code or modifying existing code. This ensures pattern compliance and avoids introducing inconsistencies.
|
|
8
|
+
|
|
9
|
+
## Steps
|
|
10
|
+
|
|
11
|
+
### 1. Read conventions
|
|
12
|
+
|
|
13
|
+
- Read `AGENTS.md` in the project root β understand tech stack, patterns, constraints
|
|
14
|
+
- If ADR files are referenced, read those too (they contain implementation guardrails)
|
|
15
|
+
- Note the build/verify commands
|
|
16
|
+
|
|
17
|
+
### 2. Search for existing patterns
|
|
18
|
+
|
|
19
|
+
- Before writing new code, search the codebase for similar implementations
|
|
20
|
+
- Find at least one existing file that does something similar to your task
|
|
21
|
+
- Match its structure, imports, naming, and style exactly
|
|
22
|
+
- If no similar code exists, follow the patterns described in AGENTS.md
|
|
23
|
+
|
|
24
|
+
### 3. Write code
|
|
25
|
+
|
|
26
|
+
- Write in small, focused units β one function, one class, one component at a time
|
|
27
|
+
- Follow the conventions from Step 1 precisely β no creative deviations
|
|
28
|
+
- Use existing utilities, helpers, and abstractions rather than creating new ones
|
|
29
|
+
- Name things consistently with the existing codebase
|
|
30
|
+
|
|
31
|
+
### 4. Verify compilation
|
|
32
|
+
|
|
33
|
+
- Run the build/type-check command defined in AGENTS.md
|
|
34
|
+
- Fix any errors or warnings before proceeding
|
|
35
|
+
- If you introduced a new file, verify it's properly imported/registered
|
|
36
|
+
|
|
37
|
+
### 5. Communicate assumptions
|
|
38
|
+
|
|
39
|
+
- If you made any assumptions during implementation, state them explicitly
|
|
40
|
+
- If you deviated from the spec for a technical reason, explain why
|
|
41
|
+
- If you identified potential issues or future work, note them
|
|
42
|
+
|
|
43
|
+
## Anti-patterns
|
|
44
|
+
|
|
45
|
+
- Writing code without first reading existing patterns
|
|
46
|
+
- Introducing new abstractions when existing ones work
|
|
47
|
+
- Guessing at business logic when the spec is ambiguous (ask instead)
|
|
48
|
+
- Skipping the verification step
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# Code Review Procedure
|
|
2
|
+
|
|
3
|
+
Process for reviewing code or preparing a pull request.
|
|
4
|
+
|
|
5
|
+
## When to use
|
|
6
|
+
|
|
7
|
+
Before committing code, when reviewing a PR, or when the architect review agent runs.
|
|
8
|
+
|
|
9
|
+
## Steps
|
|
10
|
+
|
|
11
|
+
### 1. Understand the change
|
|
12
|
+
|
|
13
|
+
- Read the diff (or the files in scope)
|
|
14
|
+
- Understand what the change is trying to accomplish
|
|
15
|
+
- Cross-reference with the Tech-Spec or task description if available
|
|
16
|
+
|
|
17
|
+
### 2. Verify convention adherence
|
|
18
|
+
|
|
19
|
+
- Read `AGENTS.md` β check every rule against the change
|
|
20
|
+
- Pay special attention to:
|
|
21
|
+
- Directory structure and file naming
|
|
22
|
+
- Import/export patterns
|
|
23
|
+
- Styling approach (no forbidden CSS patterns)
|
|
24
|
+
- Responsive design requirements
|
|
25
|
+
- i18n completeness (all locale files updated)
|
|
26
|
+
- Type safety (no `any`, proper nullability)
|
|
27
|
+
- Authorization patterns (all endpoints/pages protected)
|
|
28
|
+
|
|
29
|
+
### 3. Apply checklists
|
|
30
|
+
|
|
31
|
+
**General checklist (all projects):**
|
|
32
|
+
- [ ] No hardcoded secrets, tokens, or API keys
|
|
33
|
+
- [ ] No `console.log` or debug statements left in
|
|
34
|
+
- [ ] Error handling is present and meaningful
|
|
35
|
+
- [ ] Loading/empty states handled
|
|
36
|
+
- [ ] No unused imports or dead code
|
|
37
|
+
- [ ] Types/interfaces match between layers (DTO β frontend interface)
|
|
38
|
+
|
|
39
|
+
**Project-specific checklist:**
|
|
40
|
+
- Derive from AGENTS.md's constraints and "do NOT" list
|
|
41
|
+
- Check any ADR guardrails that apply
|
|
42
|
+
|
|
43
|
+
### 4. Report findings
|
|
44
|
+
|
|
45
|
+
For each issue, provide:
|
|
46
|
+
- Severity: CRITICAL / WARNING / INFO
|
|
47
|
+
- Location: file path + line number
|
|
48
|
+
- Description: what's wrong
|
|
49
|
+
- Fix suggestion: what to do about it
|
|
50
|
+
|
|
51
|
+
## PR Description Template
|
|
52
|
+
|
|
53
|
+
When preparing a PR:
|
|
54
|
+
|
|
55
|
+
```
|
|
56
|
+
## Summary
|
|
57
|
+
- [1-3 bullet points describing what changed and why]
|
|
58
|
+
|
|
59
|
+
## Test plan
|
|
60
|
+
- [ ] Build/type-check passes
|
|
61
|
+
- [ ] [Specific test scenarios for this change]
|
|
62
|
+
- [ ] No regressions in related features
|
|
63
|
+
```
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
# Procedure: Effort Estimation
|
|
2
|
+
|
|
3
|
+
Write `{PREFIX}-Effort-Estimate.md` in the feature directory after Work Breakdown is generated (before the WB approval gate). Append actuals at the end of the pipeline.
|
|
4
|
+
|
|
5
|
+
## Estimate template
|
|
6
|
+
|
|
7
|
+
```markdown
|
|
8
|
+
# Effort Estimate β {PREFIX} β {Feature Title}
|
|
9
|
+
|
|
10
|
+
| Metric | Value |
|
|
11
|
+
|--------|-------|
|
|
12
|
+
| User Stories | N (US-01 Γ· US-NN) |
|
|
13
|
+
| Total tasks | N (DB:N, BE:N, FE:N, INFRA:N, TEST:N) |
|
|
14
|
+
| Implementation phases | N |
|
|
15
|
+
| Human estimate | ~Nh (sequential, no parallelism) |
|
|
16
|
+
| Agent estimate | ~Nh Nmin (parallel dispatch, critical path only) |
|
|
17
|
+
|
|
18
|
+
## Domain breakdown
|
|
19
|
+
|
|
20
|
+
| Domain | Tasks | Notes |
|
|
21
|
+
|--------|-------|-------|
|
|
22
|
+
| DB | N | ... |
|
|
23
|
+
| BE | N | ... |
|
|
24
|
+
| FE | N | ... |
|
|
25
|
+
| INFRA | N | ... |
|
|
26
|
+
| TEST | N | ... |
|
|
27
|
+
|
|
28
|
+
## Implementation phases
|
|
29
|
+
|
|
30
|
+
| Phase | Tasks | Parallelism |
|
|
31
|
+
|-------|-------|-------------|
|
|
32
|
+
| Phase 1 β {name} | N tasks | N agents in parallel |
|
|
33
|
+
| Phase 2 β {name} | N tasks | N agents in parallel |
|
|
34
|
+
|
|
35
|
+
## Notes
|
|
36
|
+
|
|
37
|
+
Estimation assumptions (average task duration, agent concurrency limits, etc.).
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Estimation rules
|
|
41
|
+
|
|
42
|
+
- **Human estimate**: sum all task durations sequentially. Complexity: simple=2h, medium=4h, complex=8h
|
|
43
|
+
- **Agent estimate**: critical path only β max task duration per phase, summed across phases (parallel dispatch within phase)
|
|
44
|
+
|
|
45
|
+
## Actuals template (appended at pipeline end)
|
|
46
|
+
|
|
47
|
+
```markdown
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## Actuals vs Estimate
|
|
51
|
+
|
|
52
|
+
| Metric | Estimated | Actual | Delta |
|
|
53
|
+
|--------|-----------|--------|-------|
|
|
54
|
+
| Total wall-clock (agent) | ~Xh Ymin | Xh Ymin | Β±Zmin |
|
|
55
|
+
| Phase 1 β {name} | ~Xmin | Xmin | Β±Zmin |
|
|
56
|
+
| Phase 2 β {name} | ~Xmin | Xmin | Β±Zmin |
|
|
57
|
+
|
|
58
|
+
## Task-level actuals
|
|
59
|
+
|
|
60
|
+
| Task ID | Domain | Agent estimate | Actual | Delta |
|
|
61
|
+
|---------|--------|---------------|--------|-------|
|
|
62
|
+
| INFRA-T01 | INFRA | ~Xmin | Ymin | Β±Zmin |
|
|
63
|
+
| US-01-T01 | BE | ~Xmin | Ymin | Β±Zmin |
|
|
64
|
+
|
|
65
|
+
## Estimation accuracy
|
|
66
|
+
|
|
67
|
+
| Category | Tasks | Avg delta | Trend |
|
|
68
|
+
|----------|-------|-----------|-------|
|
|
69
|
+
| DB | N | Β±Xmin | over/under/on-target |
|
|
70
|
+
| BE | N | Β±Xmin | over/under/on-target |
|
|
71
|
+
| FE | N | Β±Xmin | over/under/on-target |
|
|
72
|
+
| INFRA | N | Β±Xmin | over/under/on-target |
|
|
73
|
+
| TEST | N | Β±Xmin | over/under/on-target |
|
|
74
|
+
|
|
75
|
+
## Notes
|
|
76
|
+
|
|
77
|
+
Which tasks took longer than expected and why.
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
## Actuals rules
|
|
81
|
+
|
|
82
|
+
- Round times to the nearest minute
|
|
83
|
+
- Delta: negative = faster than estimated, positive = slower
|
|
84
|
+
- Trend: over-target if avg delta > +15%, under-target if < -15%, otherwise on-target
|
|
85
|
+
- Include total time across all rework attempts for reworked tasks
|
|
86
|
+
- If the process log has gaps, note them and exclude affected tasks from accuracy stats
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Procedure: Feature Registry
|
|
2
|
+
|
|
3
|
+
`docs/features/REGISTRY.md` is a cross-feature index maintained by the Project Manager.
|
|
4
|
+
|
|
5
|
+
## Entry format
|
|
6
|
+
|
|
7
|
+
```markdown
|
|
8
|
+
## {PREFIX} β {Title}
|
|
9
|
+
**Keywords:** keyword1, keyword2, keyword3, ...
|
|
10
|
+
**Status:** in-progress | completed
|
|
11
|
+
**Summary:** 4β5 lines. What was built, which entities/endpoints were introduced,
|
|
12
|
+
which shared infrastructure was added (e.g. auth middleware, base components),
|
|
13
|
+
and any constraints or decisions that affect future features.
|
|
14
|
+
β [Detail]({PREFIX}-{slug}/feature.md)
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
## File structure
|
|
18
|
+
|
|
19
|
+
```markdown
|
|
20
|
+
# Feature Registry
|
|
21
|
+
|
|
22
|
+
This file is maintained automatically by the Project Manager.
|
|
23
|
+
Each entry summarises a feature for cross-reference by future features.
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## FTR-001 β ...
|
|
28
|
+
...
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## FTR-002 β ...
|
|
33
|
+
...
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
## Cross-reference analysis
|
|
37
|
+
|
|
38
|
+
When starting a new feature, check each registry entry for:
|
|
39
|
+
- **OVERLAP** β same domain, entities, or user flows
|
|
40
|
+
- **DEPENDENCY** β requires something delivered by an existing feature
|
|
41
|
+
- **CONFLICT** β contradicts existing business rules or data models
|
|
42
|
+
|
|
43
|
+
Show findings to the user before executing the plan. If conflicts are found, use `AskUserQuestion` before continuing.
|
|
44
|
+
|
|
45
|
+
## Rules
|
|
46
|
+
|
|
47
|
+
- **Max 5 lines for Summary** β brevity is the point
|
|
48
|
+
- **Keywords must be specific** β avoid generic terms like "feature", "user", "data"
|
|
49
|
+
- **Append-only** β never delete existing entries; only update `Status` from `in-progress` to `completed`
|
|
50
|
+
- **Set Status to `completed`** when the PR is successfully created
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# Procedure: Issues Register
|
|
2
|
+
|
|
3
|
+
The Issues Register tracks all non-CRITICAL review findings throughout implementation.
|
|
4
|
+
|
|
5
|
+
**File**: `{PREFIX}-Issues.md` in the same directory as `feature.md`.
|
|
6
|
+
|
|
7
|
+
## Format
|
|
8
|
+
|
|
9
|
+
```markdown
|
|
10
|
+
# Issues Register β {PREFIX}
|
|
11
|
+
|
|
12
|
+
| # | Severity | US / Scope | File(s) | Description | Status | Resolved by |
|
|
13
|
+
|---|----------|-----------|---------|-------------|--------|-------------|
|
|
14
|
+
| 1 | WARNING | US-01 | file.tsx:45 | Description | OPEN | β |
|
|
15
|
+
| 2 | INFO | US-02 | file.tsx | Description | OPEN | β |
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
## Rules
|
|
19
|
+
|
|
20
|
+
- **CRITICAL findings** are NOT tracked here β they trigger immediate rework during US review
|
|
21
|
+
- **WARNING and INFO findings** are logged here and do NOT block the current phase
|
|
22
|
+
- Every review pass appends new findings β never delete rows
|
|
23
|
+
- Final statuses after remediation: `FIXED`, `DEFERRED`
|
|
24
|
+
- The register is the input for the Remediation phase
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Procedure: Process Log
|
|
2
|
+
|
|
3
|
+
The Project Manager maintains `{PREFIX}-process-log.txt` in the same directory as `feature.md`.
|
|
4
|
+
|
|
5
|
+
## Rules
|
|
6
|
+
|
|
7
|
+
- **Create or append** at the start of every run β never overwrite previous runs
|
|
8
|
+
- **Log every significant event** with an ISO timestamp
|
|
9
|
+
- **Token tracking**: after every agent call, extract `subagent_tokens` and `duration_ms` from the `<usage>` block. Log immediately and accumulate in an internal token ledger keyed by `agent-name + scope`
|
|
10
|
+
|
|
11
|
+
## Format
|
|
12
|
+
|
|
13
|
+
```
|
|
14
|
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
15
|
+
RUN STARTED β 2026-06-12T09:01:23
|
|
16
|
+
Feature: FTR-001 β [Feature Title]
|
|
17
|
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
18
|
+
[2026-06-12T09:01:23] Discovery complete β N agents found, prefix: FTR-001
|
|
19
|
+
[2026-06-12T09:01:24] State: FTR-001-Requirements.md (fresh), FTR-001-Tech-Spec.md (missing)
|
|
20
|
+
[2026-06-12T09:01:25] Agent START: generate-tech-spec (FTR-001)
|
|
21
|
+
[2026-06-12T09:03:41] Agent DONE: generate-tech-spec (FTR-001) β tokens: 21330, duration: 3m 28s
|
|
22
|
+
[2026-06-12T09:03:42] Agent START: validate-feature-docs (FTR-001)
|
|
23
|
+
[2026-06-12T09:05:10] Agent DONE: validate-feature-docs (FTR-001) β tokens: 9210, duration: 1m 54s
|
|
24
|
+
[2026-06-12T09:05:11] APPROVAL REQUESTED
|
|
25
|
+
[2026-06-12T09:06:02] APPROVAL GRANTED by user
|
|
26
|
+
[2026-06-12T09:08:15] RUN COMPLETE
|
|
27
|
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
28
|
+
```
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Secure Coding Procedure
|
|
2
|
+
|
|
3
|
+
Security checklist for modifying authentication, authorization, or sensitive data handling.
|
|
4
|
+
|
|
5
|
+
## When to use
|
|
6
|
+
|
|
7
|
+
When modifying code that handles: authentication, authorization, user input, tokens, secrets, file uploads, or external API calls.
|
|
8
|
+
|
|
9
|
+
## Checklist
|
|
10
|
+
|
|
11
|
+
### Authentication & Authorization
|
|
12
|
+
|
|
13
|
+
- [ ] All endpoints/pages require authentication (unless explicitly public)
|
|
14
|
+
- [ ] Authorization checks are performed server-side, not just in UI
|
|
15
|
+
- [ ] Role/permission checks use the project's authorization framework (from AGENTS.md)
|
|
16
|
+
- [ ] No endpoint allows privilege escalation
|
|
17
|
+
|
|
18
|
+
### Secrets & Tokens
|
|
19
|
+
|
|
20
|
+
- [ ] No secrets hardcoded in source (use environment variables or secret managers)
|
|
21
|
+
- [ ] Tokens are never logged, even at debug level
|
|
22
|
+
- [ ] Tokens are stored securely (follow project's token storage approach from AGENTS.md)
|
|
23
|
+
- [ ] Token validation is performed server-side on every request
|
|
24
|
+
|
|
25
|
+
### Input Validation
|
|
26
|
+
|
|
27
|
+
- [ ] All user input is validated at the boundary (controller/endpoint level)
|
|
28
|
+
- [ ] Validation uses the project's validation framework (from AGENTS.md)
|
|
29
|
+
- [ ] No SQL injection vectors (use parameterized queries or ORM)
|
|
30
|
+
- [ ] No XSS vectors (no unsafe HTML rendering, sanitize user content)
|
|
31
|
+
- [ ] No command injection (no unsanitized input in shell commands)
|
|
32
|
+
|
|
33
|
+
### API Security
|
|
34
|
+
|
|
35
|
+
- [ ] CORS is configured restrictively (only allowed origins)
|
|
36
|
+
- [ ] Rate limiting is considered for public-facing endpoints
|
|
37
|
+
- [ ] Error responses don't leak internal details (stack traces, SQL errors)
|
|
38
|
+
- [ ] File uploads validate size, type, and content
|
|
39
|
+
|
|
40
|
+
### Data Protection
|
|
41
|
+
|
|
42
|
+
- [ ] PII is not logged
|
|
43
|
+
- [ ] Sensitive data is not exposed in API responses beyond what's needed
|
|
44
|
+
- [ ] Database queries don't over-fetch (select only needed fields)
|
|
45
|
+
|
|
46
|
+
## Response to violations
|
|
47
|
+
|
|
48
|
+
- If you find a security issue in existing code while implementing a feature: note it in the Issues Register as a WARNING
|
|
49
|
+
- If your own code introduces a security issue: fix it immediately before proceeding
|
|
50
|
+
- When in doubt about a security decision: ask the user, don't guess
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# Testing Procedure
|
|
2
|
+
|
|
3
|
+
Strategy and guidelines for writing and validating tests.
|
|
4
|
+
|
|
5
|
+
## When to use
|
|
6
|
+
|
|
7
|
+
When adding tests to the project, either as part of a feature implementation or as standalone test coverage improvement.
|
|
8
|
+
|
|
9
|
+
## Steps
|
|
10
|
+
|
|
11
|
+
### 1. Identify what to test
|
|
12
|
+
|
|
13
|
+
Prioritized test targets:
|
|
14
|
+
1. **Business logic** β core domain operations, calculations, state transitions
|
|
15
|
+
2. **Security** β authorization checks, input validation
|
|
16
|
+
3. **Edge cases** β null/empty inputs, boundaries, error conditions
|
|
17
|
+
4. **API contracts** β request/response shapes, status codes
|
|
18
|
+
5. **Integration points** β database operations, external API calls
|
|
19
|
+
|
|
20
|
+
### 2. Determine test type
|
|
21
|
+
|
|
22
|
+
| Type | When | Characteristics |
|
|
23
|
+
|------|------|-----------------|
|
|
24
|
+
| Unit | Isolated logic, pure functions, services | Fast, no I/O, mocked dependencies |
|
|
25
|
+
| Integration | Multi-layer operations, DB queries | Real (or in-memory) DB, slower |
|
|
26
|
+
| API/Controller | HTTP endpoint behavior | Full request pipeline, auth headers |
|
|
27
|
+
| Component | UI component rendering and interaction | DOM rendering, user events |
|
|
28
|
+
|
|
29
|
+
### 3. Check test infrastructure
|
|
30
|
+
|
|
31
|
+
- Read existing tests to understand the established patterns
|
|
32
|
+
- Check for test helpers, fixtures, and factories already available
|
|
33
|
+
- Use the project's test framework and libraries (defined in AGENTS.md)
|
|
34
|
+
|
|
35
|
+
### 4. Write tests
|
|
36
|
+
|
|
37
|
+
- **Naming**: describe the scenario and expected outcome (e.g., `CreateUser_WithInvalidEmail_Returns422`)
|
|
38
|
+
- **AAA pattern**: Arrange (setup), Act (execute), Assert (verify)
|
|
39
|
+
- **One logical assertion per test** β test one behavior at a time
|
|
40
|
+
- **Isolation**: each test creates its own state, never depends on others
|
|
41
|
+
- **Readability**: a test should be understandable without reading the implementation
|
|
42
|
+
|
|
43
|
+
### 5. Verify
|
|
44
|
+
|
|
45
|
+
- Run the test command from AGENTS.md
|
|
46
|
+
- All tests must pass (new and existing)
|
|
47
|
+
- Check for flaky tests (tests that sometimes pass, sometimes fail)
|
|
48
|
+
|
|
49
|
+
## What NOT to test
|
|
50
|
+
|
|
51
|
+
- Framework/library behavior (e.g., "does the ORM save correctly?")
|
|
52
|
+
- Trivial code (getters, setters, data classes)
|
|
53
|
+
- Static configuration
|
|
54
|
+
- Third-party code behavior
|
|
55
|
+
- Implementation details that may change during refactoring
|
|
56
|
+
|
|
57
|
+
## Anti-patterns
|
|
58
|
+
|
|
59
|
+
- Testing implementation instead of behavior
|
|
60
|
+
- Tests that break when internal code is refactored
|
|
61
|
+
- Tests with hidden dependencies (shared state, execution order)
|
|
62
|
+
- Overly specific mocks that mirror the implementation
|
|
63
|
+
- Missing edge case coverage for the happy path
|
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
# Procedure: Token Estimation
|
|
2
|
+
|
|
3
|
+
Write `{PREFIX}-Token-Estimate.md` in the feature directory after Work Breakdown is generated. Doc-gen agent actuals are filled in immediately (they have already completed). Implementation agent actuals are filled in at pipeline end. The orchestrator row is appended by the `/implement-feature` skill.
|
|
4
|
+
|
|
5
|
+
## Estimation model
|
|
6
|
+
|
|
7
|
+
| Parameter | Value |
|
|
8
|
+
|-----------|-------|
|
|
9
|
+
| Avg chars per token | 4 |
|
|
10
|
+
| Agent system prompt weight β haiku | ~2,000 tokens |
|
|
11
|
+
| Agent system prompt weight β sonnet | ~3,000 tokens |
|
|
12
|
+
| Base overhead per agent call | ~5,000 tokens |
|
|
13
|
+
|
|
14
|
+
## Cost estimation
|
|
15
|
+
|
|
16
|
+
Read pricing data from `docs/pricing.md`. Use the **Blended unit cost reference** table for the model in use.
|
|
17
|
+
|
|
18
|
+
Cost formula per agent call:
|
|
19
|
+
```
|
|
20
|
+
cost ($) = tokens Γ blended_cost_per_token
|
|
21
|
+
= tokens Γ (input_price Γ 0.80 + output_price Γ 0.20) / 1,000,000
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
The 80/20 input/output split is defined in `docs/pricing.md` β update there if the ratio changes.
|
|
25
|
+
|
|
26
|
+
## Input token estimation per agent type
|
|
27
|
+
|
|
28
|
+
- **Doc-gen agents**: system prompt + files to read (bytes Γ· 4) + base overhead
|
|
29
|
+
- **Developer agents**: tech-spec section + WB task size + relevant codebase files + base overhead
|
|
30
|
+
- **Review agents**: all files modified in the US scope + base overhead; output β 1,500 tokens fixed
|
|
31
|
+
- **Orchestrator**: sum of all child result sizes (as input) + process log writes (as output) + base overhead
|
|
32
|
+
- **Rework contingency**: add 1 extra developer + 1 review per US (30% rework probability)
|
|
33
|
+
|
|
34
|
+
## Estimate template
|
|
35
|
+
|
|
36
|
+
```markdown
|
|
37
|
+
# Token Estimate β {PREFIX} β {Feature Title}
|
|
38
|
+
|
|
39
|
+
> Estimates computed before execution. Doc-gen actuals filled on completion of each agent.
|
|
40
|
+
> Implementation actuals filled at pipeline end. Orchestrator row added by /implement-feature.
|
|
41
|
+
> Pricing model: docs/pricing.md (80% input / 20% output split).
|
|
42
|
+
|
|
43
|
+
## Estimation model
|
|
44
|
+
|
|
45
|
+
| Parameter | Value |
|
|
46
|
+
|-----------|-------|
|
|
47
|
+
| Avg chars per token | 4 |
|
|
48
|
+
| Haiku system prompt | ~2,000 tokens |
|
|
49
|
+
| Sonnet system prompt | ~3,000 tokens |
|
|
50
|
+
| Base overhead per call | ~5,000 tokens |
|
|
51
|
+
| Input/output split | 80% / 20% |
|
|
52
|
+
|
|
53
|
+
## Agent token estimates and early actuals
|
|
54
|
+
|
|
55
|
+
| Agent | Model | Est. tokens | Est. cost ($) | Actual tokens | Actual cost ($) | Status |
|
|
56
|
+
|-------|-------|------------|--------------|---------------|----------------|--------|
|
|
57
|
+
| generate-requirements | haiku | N | $N.NNNN | N (actual) | $N.NNNN | β
complete |
|
|
58
|
+
| generate-tech-spec | haiku | N | $N.NNNN | N (actual) | $N.NNNN | β
complete |
|
|
59
|
+
| validate-feature-docs | haiku | N | $N.NNNN | N (actual) | $N.NNNN | β
complete |
|
|
60
|
+
| generate-work-breakdown | haiku | N | $N.NNNN | N (actual) | $N.NNNN | β
complete |
|
|
61
|
+
| developer-backend (ΓN) | sonnet | N | $N.NNNN | β | β | β³ pending |
|
|
62
|
+
| developer-frontend (ΓN) | sonnet | N | $N.NNNN | β | β | β³ pending |
|
|
63
|
+
| developer-testing (ΓN) | sonnet | N | $N.NNNN | β | β | β³ pending |
|
|
64
|
+
| review-solution (ΓN) | sonnet | N | $N.NNNN | β | β | β³ pending |
|
|
65
|
+
| project-manager (orchestrator) | sonnet | N | $N.NNNN | β | β | β³ pending |
|
|
66
|
+
|
|
67
|
+
## Phase subtotals
|
|
68
|
+
|
|
69
|
+
| Phase | Est. tokens | Est. cost ($) | Actual tokens | Actual cost ($) |
|
|
70
|
+
|-------|------------|--------------|---------------|----------------|
|
|
71
|
+
| Doc generation | N | $N.NNNN | N (actual) | $N.NNNN |
|
|
72
|
+
| Phase 1 β {name} | N | $N.NNNN | β³ pending | β³ pending |
|
|
73
|
+
|
|
74
|
+
## Grand total
|
|
75
|
+
|
|
76
|
+
| Metric | Estimated | Actual |
|
|
77
|
+
|--------|-----------|--------|
|
|
78
|
+
| Total tokens | N | partial β updated at pipeline end |
|
|
79
|
+
| Total cost ($) | $N.NN | partial β updated at pipeline end |
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
## Actuals template (appended at pipeline end)
|
|
83
|
+
|
|
84
|
+
```markdown
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## Actuals vs Estimate
|
|
88
|
+
|
|
89
|
+
| Agent | Task / Scope | Model | Est. tokens | Actual tokens | Delta | Est. cost ($) | Actual cost ($) | Duration |
|
|
90
|
+
|-------|-------------|-------|------------|---------------|-------|--------------|----------------|----------|
|
|
91
|
+
| generate-requirements | {PREFIX} | haiku | N | N | Β±N | $N.NNNN | $N.NNNN | Xmin |
|
|
92
|
+
| generate-tech-spec | {PREFIX} | haiku | N | N | Β±N | $N.NNNN | $N.NNNN | Xmin |
|
|
93
|
+
| validate-feature-docs | {PREFIX} | haiku | N | N | Β±N | $N.NNNN | $N.NNNN | Xmin |
|
|
94
|
+
| generate-work-breakdown | {PREFIX} | haiku | N | N | Β±N | $N.NNNN | $N.NNNN | Xmin |
|
|
95
|
+
| developer-backend | US-01-T01, T02 | sonnet | N | N | Β±N | $N.NNNN | $N.NNNN | Xmin |
|
|
96
|
+
| review-solution | US-01 | sonnet | N | N | Β±N | $N.NNNN | $N.NNNN | Xmin |
|
|
97
|
+
| project-manager (orchestrator) | β | sonnet | N | N | Β±N | $N.NNNN | $N.NNNN | Xh Ymin |
|
|
98
|
+
|
|
99
|
+
## Estimation accuracy by agent type
|
|
100
|
+
|
|
101
|
+
| Agent type | Invocations | Avg est. tokens | Avg actual tokens | Avg delta | Trend |
|
|
102
|
+
|-----------|------------|----------------|------------------|-----------|-------|
|
|
103
|
+
| haiku agents | N | N | N | Β±N | over/under/on-target |
|
|
104
|
+
| sonnet agents | N | N | N | Β±N | over/under/on-target |
|
|
105
|
+
|
|
106
|
+
## Grand Total
|
|
107
|
+
|
|
108
|
+
| Metric | Estimated | Actual | Delta |
|
|
109
|
+
|--------|-----------|--------|-------|
|
|
110
|
+
| Total tokens (all agents) | N | N | Β±N |
|
|
111
|
+
| Total cost ($) | $N.NN | $N.NN | Β±$N.NN |
|
|
112
|
+
| Total wall-clock | Xh Ymin | Xh Ymin | Β±Zmin |
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
## Actuals rules
|
|
116
|
+
|
|
117
|
+
- One row per invocation β rework invocations get a "(rework)" suffix
|
|
118
|
+
- Delta (tokens): negative = fewer tokens than estimated, positive = more
|
|
119
|
+
- Cost: computed with blended formula from `docs/pricing.md`; use 4 decimal places for per-agent rows, 2 decimal places for totals
|
|
120
|
+
- Trend: over-target if avg delta > +20%, under-target if < -20%, otherwise on-target
|
|
121
|
+
- Duration: convert `duration_ms` to minutes/seconds
|
|
122
|
+
- Model: read from agent frontmatter, default to `sonnet` if not set
|
|
123
|
+
- Missing `<usage>` block β write `N/A`, exclude from accuracy stats and cost totals
|