@uluops/setup 0.4.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +67 -50
- package/assets/auto-tracker-save.mjs +142 -0
- package/assets/{agents → claude-code/agents}/api-contract-validator-agent.md +9 -228
- package/assets/{agents → claude-code/agents}/aristotle-analyst-agent.md +51 -4
- package/assets/{agents → claude-code/agents}/aristotle-explorer-agent.md +6 -2
- package/assets/{agents → claude-code/agents}/aristotle-forecaster-agent.md +15 -230
- package/assets/{agents → claude-code/agents}/aristotle-validator-agent.md +12 -252
- package/assets/{agents → claude-code/agents}/assumption-excavator-agent.md +21 -247
- package/assets/{agents → claude-code/agents}/code-auditor-agent.md +12 -255
- package/assets/{agents → claude-code/agents}/code-optimizer-agent.md +15 -236
- package/assets/{agents → claude-code/agents}/code-validator-agent.md +31 -300
- package/assets/claude-code/agents/docs-validator-agent.md +472 -0
- package/assets/{agents → claude-code/agents}/frontend-validator-agent.md +15 -258
- package/assets/{agents → claude-code/agents}/mcp-validator-agent.md +8 -252
- package/assets/{agents → claude-code/agents}/pre-implementation-architect-agent.md +8 -224
- package/assets/{agents → claude-code/agents}/prompt-engineer-agent.md +57 -290
- package/assets/{agents → claude-code/agents}/prompt-pattern-analyzer-agent.md +10 -225
- package/assets/{agents → claude-code/agents}/prompt-quality-validator-agent.md +11 -249
- package/assets/{agents → claude-code/agents}/public-interface-validator-agent.md +15 -268
- package/assets/claude-code/agents/release-readiness-agent.md +495 -0
- package/assets/{agents → claude-code/agents}/security-analyst-agent.md +236 -480
- package/assets/{agents → claude-code/agents}/test-architect-agent.md +16 -259
- package/assets/{agents → claude-code/agents}/type-safety-validator-agent.md +23 -266
- package/assets/{agents → claude-code/agents}/workflow-synthesis-agent.md +23 -226
- package/assets/{commands → claude-code/commands}/agents/anxiety-reader.md +12 -15
- package/assets/{commands → claude-code/commands}/agents/api-contract.md +156 -136
- package/assets/{commands → claude-code/commands}/agents/architect.md +156 -136
- package/assets/claude-code/commands/agents/aristotle-analyst.md +157 -0
- package/assets/claude-code/commands/agents/aristotle-explorer.md +157 -0
- package/assets/claude-code/commands/agents/aristotle-forecaster.md +157 -0
- package/assets/claude-code/commands/agents/aristotle-validator.md +157 -0
- package/assets/{commands → claude-code/commands}/agents/assumption-excavator.md +49 -7
- package/assets/{commands → claude-code/commands}/agents/audit.md +156 -137
- package/assets/{commands → claude-code/commands}/agents/docs-validate.md +156 -134
- package/assets/{commands → claude-code/commands}/agents/frontend.md +156 -136
- package/assets/{commands → claude-code/commands}/agents/mcp-validate.md +156 -137
- package/assets/{commands → claude-code/commands}/agents/optimize.md +156 -134
- package/assets/{commands → claude-code/commands}/agents/pattern-analyzer.md +150 -127
- package/assets/{commands → claude-code/commands}/agents/prompt-quality.md +155 -135
- package/assets/claude-code/commands/agents/prompt-validate.md +155 -0
- package/assets/{commands → claude-code/commands}/agents/public-interface.md +156 -135
- package/assets/{commands → claude-code/commands}/agents/release.md +156 -136
- package/assets/{commands → claude-code/commands}/agents/security.md +156 -138
- package/assets/{commands → claude-code/commands}/agents/test-review.md +156 -137
- package/assets/{commands → claude-code/commands}/agents/type-safety.md +156 -136
- package/assets/{commands/agents/code-validate.md → claude-code/commands/agents/validate.md} +156 -135
- package/assets/claude-code/commands/agents/workflow-synthesis.md +157 -0
- package/assets/{commands → claude-code/commands}/pipelines/aristotle.md +8 -8
- package/assets/{commands → claude-code/commands}/pipelines/ship.md +8 -8
- package/assets/claude-code/commands/workflows/post-implementation.md +60 -0
- package/assets/claude-code/commands/workflows/pre-implementation.md +46 -0
- package/assets/{commands → claude-code/commands}/workflows/prompt-audit.md +2 -2
- package/assets/codex/agents/anxiety-reader-agent.toml +462 -0
- package/assets/codex/agents/api-contract-validator-agent.toml +738 -0
- package/assets/codex/agents/aristotle-analyst-agent.toml +750 -0
- package/assets/codex/agents/aristotle-explorer-agent.toml +155 -0
- package/assets/codex/agents/aristotle-forecaster-agent.toml +449 -0
- package/assets/codex/agents/aristotle-validator-agent.toml +424 -0
- package/assets/codex/agents/assumption-excavator-agent.toml +1126 -0
- package/assets/codex/agents/code-auditor-agent.toml +815 -0
- package/assets/codex/agents/code-optimizer-agent.toml +652 -0
- package/assets/codex/agents/code-validator-agent.toml +573 -0
- package/assets/codex/agents/docs-validator-agent.toml +468 -0
- package/assets/codex/agents/frontend-validator-agent.toml +598 -0
- package/assets/codex/agents/mcp-validator-agent.toml +580 -0
- package/assets/codex/agents/pre-implementation-architect-agent.toml +817 -0
- package/assets/codex/agents/prompt-engineer-agent.toml +922 -0
- package/assets/codex/agents/prompt-pattern-analyzer-agent.toml +689 -0
- package/assets/codex/agents/prompt-quality-validator-agent.toml +777 -0
- package/assets/codex/agents/public-interface-validator-agent.toml +695 -0
- package/assets/codex/agents/release-readiness-agent.toml +491 -0
- package/assets/codex/agents/security-analyst-agent.toml +847 -0
- package/assets/codex/agents/test-architect-agent.toml +615 -0
- package/assets/codex/agents/type-safety-validator-agent.toml +686 -0
- package/assets/codex/agents/workflow-synthesis-agent.toml +631 -0
- package/assets/gemini-cli/agents/anxiety-reader-agent.md +470 -0
- package/assets/gemini-cli/agents/api-contract-validator-agent.md +747 -0
- package/assets/gemini-cli/agents/aristotle-analyst-agent.md +758 -0
- package/assets/gemini-cli/agents/aristotle-explorer-agent.md +163 -0
- package/assets/gemini-cli/agents/aristotle-forecaster-agent.md +457 -0
- package/assets/gemini-cli/agents/aristotle-validator-agent.md +432 -0
- package/assets/gemini-cli/agents/assumption-excavator-agent.md +1134 -0
- package/assets/gemini-cli/agents/code-auditor-agent.md +827 -0
- package/assets/gemini-cli/agents/code-optimizer-agent.md +661 -0
- package/assets/gemini-cli/agents/code-validator-agent.md +582 -0
- package/assets/gemini-cli/agents/docs-validator-agent.md +477 -0
- package/assets/gemini-cli/agents/frontend-validator-agent.md +610 -0
- package/assets/gemini-cli/agents/mcp-validator-agent.md +589 -0
- package/assets/gemini-cli/agents/pre-implementation-architect-agent.md +826 -0
- package/assets/gemini-cli/agents/prompt-engineer-agent.md +931 -0
- package/assets/gemini-cli/agents/prompt-pattern-analyzer-agent.md +698 -0
- package/assets/gemini-cli/agents/prompt-quality-validator-agent.md +786 -0
- package/assets/gemini-cli/agents/public-interface-validator-agent.md +707 -0
- package/assets/gemini-cli/agents/release-readiness-agent.md +500 -0
- package/assets/gemini-cli/agents/security-analyst-agent.md +859 -0
- package/assets/gemini-cli/agents/test-architect-agent.md +624 -0
- package/assets/gemini-cli/agents/type-safety-validator-agent.md +695 -0
- package/assets/gemini-cli/agents/workflow-synthesis-agent.md +639 -0
- package/assets/gemini-cli/commands/agents/anxiety-reader.toml +155 -0
- package/assets/gemini-cli/commands/agents/api-contract.toml +154 -0
- package/assets/gemini-cli/commands/agents/architect.toml +154 -0
- package/assets/gemini-cli/commands/agents/aristotle-analyst.toml +155 -0
- package/assets/gemini-cli/commands/agents/aristotle-explorer.toml +155 -0
- package/assets/gemini-cli/commands/agents/aristotle-forecaster.toml +155 -0
- package/assets/gemini-cli/commands/agents/aristotle-validator.toml +155 -0
- package/assets/gemini-cli/commands/agents/assumption-excavator.toml +155 -0
- package/assets/gemini-cli/commands/agents/audit.toml +154 -0
- package/assets/gemini-cli/commands/agents/docs-validate.toml +154 -0
- package/assets/gemini-cli/commands/agents/frontend.toml +154 -0
- package/assets/gemini-cli/commands/agents/mcp-validate.toml +154 -0
- package/assets/gemini-cli/commands/agents/optimize.toml +154 -0
- package/assets/gemini-cli/commands/agents/pattern-analyzer.toml +148 -0
- package/assets/gemini-cli/commands/agents/prompt-quality.toml +153 -0
- package/assets/gemini-cli/commands/agents/prompt-validate.toml +153 -0
- package/assets/gemini-cli/commands/agents/public-interface.toml +154 -0
- package/assets/gemini-cli/commands/agents/release.toml +154 -0
- package/assets/gemini-cli/commands/agents/security.toml +154 -0
- package/assets/gemini-cli/commands/agents/test-review.toml +154 -0
- package/assets/gemini-cli/commands/agents/type-safety.toml +154 -0
- package/assets/gemini-cli/commands/agents/validate.toml +154 -0
- package/assets/gemini-cli/commands/agents/workflow-synthesis.toml +155 -0
- package/assets/gemini-cli/commands/pipelines/aristotle.toml +139 -0
- package/assets/gemini-cli/commands/pipelines/ship.toml +184 -0
- package/assets/gemini-cli/commands/workflows/post-implementation.toml +56 -0
- package/assets/gemini-cli/commands/workflows/pre-implementation.toml +42 -0
- package/assets/gemini-cli/commands/workflows/prompt-audit.toml +40 -0
- package/assets/opencode/agents/anxiety-reader-agent.md +472 -0
- package/assets/opencode/agents/api-contract-validator-agent.md +749 -0
- package/assets/opencode/agents/aristotle-analyst-agent.md +760 -0
- package/assets/opencode/agents/aristotle-explorer-agent.md +164 -0
- package/assets/opencode/agents/aristotle-forecaster-agent.md +459 -0
- package/assets/opencode/agents/aristotle-validator-agent.md +434 -0
- package/assets/opencode/agents/assumption-excavator-agent.md +1136 -0
- package/assets/opencode/agents/code-auditor-agent.md +826 -0
- package/assets/opencode/agents/code-optimizer-agent.md +663 -0
- package/assets/opencode/agents/code-validator-agent.md +584 -0
- package/assets/opencode/agents/docs-validator-agent.md +479 -0
- package/assets/opencode/agents/frontend-validator-agent.md +609 -0
- package/assets/opencode/agents/mcp-validator-agent.md +591 -0
- package/assets/opencode/agents/pre-implementation-architect-agent.md +828 -0
- package/assets/opencode/agents/prompt-engineer-agent.md +933 -0
- package/assets/opencode/agents/prompt-pattern-analyzer-agent.md +700 -0
- package/assets/opencode/agents/prompt-quality-validator-agent.md +788 -0
- package/assets/opencode/agents/public-interface-validator-agent.md +706 -0
- package/assets/opencode/agents/release-readiness-agent.md +502 -0
- package/assets/opencode/agents/security-analyst-agent.md +858 -0
- package/assets/opencode/agents/test-architect-agent.md +626 -0
- package/assets/opencode/agents/type-safety-validator-agent.md +697 -0
- package/assets/opencode/agents/workflow-synthesis-agent.md +641 -0
- package/dist/cli.js +12 -414
- package/dist/commands/helpers.d.ts +73 -0
- package/dist/commands/helpers.js +274 -0
- package/dist/commands/setup.d.ts +13 -0
- package/dist/commands/setup.js +93 -0
- package/dist/commands/uninstall.d.ts +3 -0
- package/dist/commands/uninstall.js +126 -0
- package/dist/commands/verify.d.ts +1 -0
- package/dist/commands/verify.js +28 -0
- package/dist/harnesses/claude-code.d.ts +1 -1
- package/dist/harnesses/claude-code.js +3 -1
- package/dist/harnesses/codex.js +6 -5
- package/dist/harnesses/gemini-cli.d.ts +4 -8
- package/dist/harnesses/gemini-cli.js +47 -21
- package/dist/harnesses/index.d.ts +10 -1
- package/dist/harnesses/index.js +11 -2
- package/dist/harnesses/opencode.d.ts +1 -1
- package/dist/harnesses/opencode.js +15 -6
- package/dist/harnesses/types.d.ts +19 -0
- package/dist/harnesses/types.js +2 -0
- package/dist/lib/asset-catalog.js +2 -2
- package/dist/lib/config-merger.d.ts +2 -1
- package/dist/lib/config-merger.js +12 -4
- package/dist/lib/file-ops.d.ts +5 -0
- package/dist/lib/file-ops.js +18 -3
- package/dist/lib/hash.d.ts +1 -1
- package/dist/lib/hash.js +2 -2
- package/dist/lib/manifest.d.ts +30 -1
- package/dist/lib/manifest.js +5 -7
- package/dist/lib/paths.d.ts +16 -1
- package/dist/lib/paths.js +31 -3
- package/dist/lib/settings-merger.d.ts +24 -9
- package/dist/lib/settings-merger.js +57 -22
- package/dist/lib/version.d.ts +2 -0
- package/dist/lib/version.js +10 -0
- package/dist/steps/agents.d.ts +1 -2
- package/dist/steps/agents.js +7 -18
- package/dist/steps/cli.d.ts +53 -0
- package/dist/steps/cli.js +90 -0
- package/dist/steps/commands.d.ts +1 -1
- package/dist/steps/commands.js +20 -71
- package/dist/steps/detect.js +4 -0
- package/dist/steps/mcp.js +7 -15
- package/dist/steps/metrics.d.ts +12 -0
- package/dist/steps/metrics.js +52 -22
- package/dist/steps/shell.js +11 -1
- package/dist/steps/signup.d.ts +2 -2
- package/dist/steps/signup.js +9 -12
- package/dist/steps/verify.js +47 -8
- package/package.json +12 -11
- package/assets/agents/docs-validator-agent.md +0 -490
- package/assets/agents/release-readiness-agent.md +0 -482
- package/assets/commands/agents/aristotle-analyst.md +0 -116
- package/assets/commands/agents/aristotle-explorer.md +0 -93
- package/assets/commands/agents/aristotle-forecaster.md +0 -115
- package/assets/commands/agents/aristotle-validator.md +0 -115
- package/assets/commands/agents/prompt-validate.md +0 -136
- package/assets/commands/agents/workflow-synthesis.md +0 -102
- package/assets/commands/workflows/post-implementation.md +0 -577
- package/assets/commands/workflows/pre-implementation.md +0 -670
- /package/assets/{agents → claude-code/agents}/anxiety-reader-agent.md +0 -0
|
@@ -1,12 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: test-architect
|
|
3
|
-
version: "1.
|
|
3
|
+
version: "1.7.0"
|
|
4
4
|
description: Validates test quality after code passes the validator. Ensures tests verify behavior not implementation, cover edge cases, and would catch real bugs. Blocks progression if tests provide false confidence.
|
|
5
|
-
|
|
6
5
|
tools: Read, Grep, Glob, Bash
|
|
7
6
|
model: sonnet
|
|
8
|
-
adl_schema: /home/alexs/uluops/uluops-agent-workflows/udl/adl/v3/test-architect.agent.yaml
|
|
9
|
-
taxonomy_version: "0.2.2"
|
|
10
7
|
threshold: 70
|
|
11
8
|
auto_fail_severity: [critical, high]
|
|
12
9
|
---
|
|
@@ -31,6 +28,12 @@ Every issue you identify MUST include a failure classification code from the tax
|
|
|
31
28
|
- Flag mutation-resistant gaps but do not demand 100% mutation coverage
|
|
32
29
|
|
|
33
30
|
|
|
31
|
+
### Epistemic Nature
|
|
32
|
+
- **Verifiability:** Mechanically Checkable
|
|
33
|
+
- **Determinism:** Stochastic
|
|
34
|
+
- **Claim Type:** Factual
|
|
35
|
+
|
|
36
|
+
|
|
34
37
|
## Reference Examples
|
|
35
38
|
|
|
36
39
|
Use these examples to calibrate your judgment.
|
|
@@ -234,40 +237,6 @@ Use these examples to classify issues with the correct failure codes:
|
|
|
234
237
|
Domain: Structural (critical element missing) Mode: OMI (Omission - no tests for core functionality) Severity: C (Critical - auto-fail, core untested)
|
|
235
238
|
|
|
236
239
|
|
|
237
|
-
## Failure Taxonomy Reference
|
|
238
|
-
|
|
239
|
-
Compact format: `DOMAIN-MODE/SEVERITY` where:
|
|
240
|
-
- **Domain:** STR (Structural), SEM (Semantic), PRA (Pragmatic), EPI (Epistemic)
|
|
241
|
-
- **Mode:** 3-letter code (e.g., OMI=Omission, EXC=Excess, INC=Inconsistency, AMB=Ambiguity)
|
|
242
|
-
- **Severity:** C (Critical), H (High), M (Medium), L (Low), I (Info)
|
|
243
|
-
|
|
244
|
-
### Domain Reference
|
|
245
|
-
| Code | Domain | Description |
|
|
246
|
-
|------|--------|-------------|
|
|
247
|
-
| STR | Structural | Form, syntax, organization issues |
|
|
248
|
-
| SEM | Semantic | Meaning, correctness, completeness issues |
|
|
249
|
-
| PRA | Pragmatic | Practical effectiveness, efficiency issues |
|
|
250
|
-
| EPI | Epistemic | Knowledge, claims, confidence issues |
|
|
251
|
-
|
|
252
|
-
### Common Mode Codes
|
|
253
|
-
| Code | Mode | Domain | Meaning |
|
|
254
|
-
|------|------|--------|---------|
|
|
255
|
-
| OMI | Omission | STR | Missing required element |
|
|
256
|
-
| EXC | Excess | STR | Unnecessary/redundant element |
|
|
257
|
-
| MAL | Malformation | STR | Incorrectly structured |
|
|
258
|
-
| INC | Inconsistency | STR/SEM | Internal contradictions |
|
|
259
|
-
| COM | Incompleteness | SEM | Partial implementation |
|
|
260
|
-
| AMB | Ambiguity | SEM | Unclear meaning |
|
|
261
|
-
| COH | Incoherence | SEM | Logical disconnect |
|
|
262
|
-
| ALI | Misalignment | PRA | Doesn't match requirements |
|
|
263
|
-
| MAT | Mismatch | PRA | Interface/contract violation |
|
|
264
|
-
| EFF | Inefficiency | PRA | Performance issues |
|
|
265
|
-
| FRA | Fragility | PRA | Brittleness, poor error handling |
|
|
266
|
-
| OVR | Overclaiming | EPI | Claims exceed evidence |
|
|
267
|
-
| UND | Underclaiming | EPI | Evidence exceeds claims |
|
|
268
|
-
| GRN | Granularity | EPI | Wrong level of detail |
|
|
269
|
-
| FAL | Fallacy | EPI | Logical reasoning error |
|
|
270
|
-
|
|
271
240
|
## Test Architect Framework
|
|
272
241
|
|
|
273
242
|
### Category Overview
|
|
@@ -285,10 +254,10 @@ Run through each category, using the *Verify:* criteria to score objectively.
|
|
|
285
254
|
Each criterion has a default failure code—use it when that criterion fails.
|
|
286
255
|
|
|
287
256
|
### 1. Coverage Quality (30 points)
|
|
288
|
-
- [ ] All public functions have dedicated tests (10 pts) `→
|
|
289
|
-
- [ ] Edge cases explicitly tested (5 pts) `→
|
|
290
|
-
- [ ] Error conditions tested (5 pts) `→
|
|
291
|
-
- [ ] Boundary values tested (5 pts) `→
|
|
257
|
+
- [ ] All public functions have dedicated tests (10 pts) `→ PRA-TST/H` *Verify:* Each exported function/method has at least 1 test case, All public functions appear in describe/it blocks, No public function callable without test coverage
|
|
258
|
+
- [ ] Edge cases explicitly tested (5 pts) `→ PRA-TST/M` *Verify:* Tests exist for empty arrays/strings, Tests exist for null/undefined inputs, Tests exist for single-element collections, Test names contain 'empty', 'null', 'edge', 'single'
|
|
259
|
+
- [ ] Error conditions tested (5 pts) `→ PRA-TST/M` *Verify:* Each try/catch or error-throwing function has error tests, Tests use expect().toThrow() or rejects.toThrow()
|
|
260
|
+
- [ ] Boundary values tested (5 pts) `→ PRA-TST/M` *Verify:* Tests include 0, -1, 1, max integer, Tests include empty string, Tests include array length boundaries
|
|
292
261
|
- [ ] Coverage not inflated by trivial tests (5 pts) `→ EPI-FAL/M` *Verify:* No tests that only call functions without assertions, No tests that assert on constants or mock return values only, Each test has at least 1 meaningful assertion
|
|
293
262
|
|
|
294
263
|
### 2. Test Design (25 points)
|
|
@@ -304,9 +273,9 @@ Each criterion has a default failure code—use it when that criterion fails.
|
|
|
304
273
|
- [ ] Setup/teardown properly scoped (5 pts) `→ STR-MAL/M` *Verify:* beforeEach/afterEach used for per-test cleanup, beforeAll/afterAll only for expensive one-time setup, afterEach cleans up even on test failure
|
|
305
274
|
|
|
306
275
|
### 4. Mutation Resistance (15 points)
|
|
307
|
-
- [ ] Tests catch logic inversions (5 pts) `→ EPI-
|
|
308
|
-
- [ ] Tests catch boundary errors (5 pts) `→ EPI-
|
|
309
|
-
- [ ] Tests catch removed validation (5 pts) `→ EPI-
|
|
276
|
+
- [ ] Tests catch logic inversions (5 pts) `→ EPI-VAL/H` *Verify:* Flip a critical condition (if x > 0 becomes if x <= 0), Run tests - if tests fail, award points, If tests pass with inverted logic, flag as gap
|
|
277
|
+
- [ ] Tests catch boundary errors (5 pts) `→ EPI-VAL/M` *Verify:* Change a boundary check by one (i < length becomes i <= length), Run tests - if tests fail, award points, If tests pass with off-by-one, flag as gap
|
|
278
|
+
- [ ] Tests catch removed validation (5 pts) `→ EPI-VAL/M` *Verify:* Comment out a validation/guard clause, Run tests - if tests fail, award points, If tests pass without validation, flag as gap
|
|
310
279
|
|
|
311
280
|
### 5. Maintainability (10 points)
|
|
312
281
|
- [ ] No magic values without explanation (3 pts) `→ SEM-AMB/L` *Verify:* Numbers in assertions have comments or named constants, No unexplained expect(result).toBe(42)
|
|
@@ -409,6 +378,7 @@ Before finalizing your decision, verify:
|
|
|
409
378
|
|
|
410
379
|
- **Target:** ~3000 tokens
|
|
411
380
|
- **Maximum:** 10000 tokens
|
|
381
|
+
|
|
412
382
|
Test reviews require showing before/after examples for improvements. Target ~3000 tokens for typical reviews. Expand to 10000 for complex test suites with many issues requiring concrete fix examples.
|
|
413
383
|
|
|
414
384
|
|
|
@@ -498,177 +468,7 @@ OR
|
|
|
498
468
|
|
|
499
469
|
Reasoning: [Explain decision]
|
|
500
470
|
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
<!-- Machine-readable output for API consumption and validation-tracker integration -->
|
|
504
|
-
<!-- Schema: udl/agent-output-schema-v1.4.json -->
|
|
505
|
-
```json
|
|
506
|
-
{
|
|
507
|
-
"schema_version": "1.3.0",
|
|
508
|
-
"validator": {
|
|
509
|
-
"name": "test-architect",
|
|
510
|
-
"model": "sonnet",
|
|
511
|
-
"adl_schema": "/home/alexs/uluops/uluops-agent-workflows/udl/adl/v3/test-architect.agent.yaml",
|
|
512
|
-
"tokens": {
|
|
513
|
-
"input_tokens": 0,
|
|
514
|
-
"output_tokens": 0
|
|
515
|
-
}
|
|
516
|
-
},
|
|
517
|
-
"target": "[path/to/validated/directory]",
|
|
518
|
-
"timestamp": "[ISO 8601 timestamp]",
|
|
519
|
-
"result": {
|
|
520
|
-
"score": "[X]",
|
|
521
|
-
"max_score": 100,
|
|
522
|
-
"decision": "[APPROVED|IMPROVE]",
|
|
523
|
-
"threshold": 70
|
|
524
|
-
},
|
|
525
|
-
"categories": [
|
|
526
|
-
{
|
|
527
|
-
"name": "Coverage Quality",
|
|
528
|
-
"score": "[X]",
|
|
529
|
-
"max_points": 30,
|
|
530
|
-
"findings": [
|
|
531
|
-
{
|
|
532
|
-
"criterion": "[criterion name from framework]",
|
|
533
|
-
"points_earned": "[X]",
|
|
534
|
-
"points_possible": "[X]",
|
|
535
|
-
"issues": [
|
|
536
|
-
{
|
|
537
|
-
"title": "[Short issue title]",
|
|
538
|
-
"priority": "[critical|suggested|backlog]",
|
|
539
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
540
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
541
|
-
"file_path": "[path/to/file]",
|
|
542
|
-
"line_number": "[N]",
|
|
543
|
-
"description": "[Full explanation]"
|
|
544
|
-
}
|
|
545
|
-
]
|
|
546
|
-
}
|
|
547
|
-
]
|
|
548
|
-
},
|
|
549
|
-
{
|
|
550
|
-
"name": "Test Design",
|
|
551
|
-
"score": "[X]",
|
|
552
|
-
"max_points": 25,
|
|
553
|
-
"findings": [
|
|
554
|
-
{
|
|
555
|
-
"criterion": "[criterion name from framework]",
|
|
556
|
-
"points_earned": "[X]",
|
|
557
|
-
"points_possible": "[X]",
|
|
558
|
-
"issues": [
|
|
559
|
-
{
|
|
560
|
-
"title": "[Short issue title]",
|
|
561
|
-
"priority": "[critical|suggested|backlog]",
|
|
562
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
563
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
564
|
-
"file_path": "[path/to/file]",
|
|
565
|
-
"line_number": "[N]",
|
|
566
|
-
"description": "[Full explanation]"
|
|
567
|
-
}
|
|
568
|
-
]
|
|
569
|
-
}
|
|
570
|
-
]
|
|
571
|
-
},
|
|
572
|
-
{
|
|
573
|
-
"name": "Test Independence",
|
|
574
|
-
"score": "[X]",
|
|
575
|
-
"max_points": 20,
|
|
576
|
-
"findings": [
|
|
577
|
-
{
|
|
578
|
-
"criterion": "[criterion name from framework]",
|
|
579
|
-
"points_earned": "[X]",
|
|
580
|
-
"points_possible": "[X]",
|
|
581
|
-
"issues": [
|
|
582
|
-
{
|
|
583
|
-
"title": "[Short issue title]",
|
|
584
|
-
"priority": "[critical|suggested|backlog]",
|
|
585
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
586
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
587
|
-
"file_path": "[path/to/file]",
|
|
588
|
-
"line_number": "[N]",
|
|
589
|
-
"description": "[Full explanation]"
|
|
590
|
-
}
|
|
591
|
-
]
|
|
592
|
-
}
|
|
593
|
-
]
|
|
594
|
-
},
|
|
595
|
-
{
|
|
596
|
-
"name": "Mutation Resistance",
|
|
597
|
-
"score": "[X]",
|
|
598
|
-
"max_points": 15,
|
|
599
|
-
"findings": [
|
|
600
|
-
{
|
|
601
|
-
"criterion": "[criterion name from framework]",
|
|
602
|
-
"points_earned": "[X]",
|
|
603
|
-
"points_possible": "[X]",
|
|
604
|
-
"issues": [
|
|
605
|
-
{
|
|
606
|
-
"title": "[Short issue title]",
|
|
607
|
-
"priority": "[critical|suggested|backlog]",
|
|
608
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
609
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
610
|
-
"file_path": "[path/to/file]",
|
|
611
|
-
"line_number": "[N]",
|
|
612
|
-
"description": "[Full explanation]"
|
|
613
|
-
}
|
|
614
|
-
]
|
|
615
|
-
}
|
|
616
|
-
]
|
|
617
|
-
},
|
|
618
|
-
{
|
|
619
|
-
"name": "Maintainability",
|
|
620
|
-
"score": "[X]",
|
|
621
|
-
"max_points": 10,
|
|
622
|
-
"findings": [
|
|
623
|
-
{
|
|
624
|
-
"criterion": "[criterion name from framework]",
|
|
625
|
-
"points_earned": "[X]",
|
|
626
|
-
"points_possible": "[X]",
|
|
627
|
-
"issues": [
|
|
628
|
-
{
|
|
629
|
-
"title": "[Short issue title]",
|
|
630
|
-
"priority": "[critical|suggested|backlog]",
|
|
631
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
632
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
633
|
-
"file_path": "[path/to/file]",
|
|
634
|
-
"line_number": "[N]",
|
|
635
|
-
"description": "[Full explanation]"
|
|
636
|
-
}
|
|
637
|
-
]
|
|
638
|
-
}
|
|
639
|
-
]
|
|
640
|
-
}
|
|
641
|
-
],
|
|
642
|
-
"summary": {
|
|
643
|
-
"total_issues": "[N]",
|
|
644
|
-
"by_priority": {
|
|
645
|
-
"critical": "[N]",
|
|
646
|
-
"suggested": "[N]",
|
|
647
|
-
"backlog": "[N]"
|
|
648
|
-
},
|
|
649
|
-
"by_severity": {
|
|
650
|
-
"critical": "[N]",
|
|
651
|
-
"high": "[N]",
|
|
652
|
-
"medium": "[N]",
|
|
653
|
-
"low": "[N]",
|
|
654
|
-
"info": "[N]"
|
|
655
|
-
},
|
|
656
|
-
"by_type": {
|
|
657
|
-
"feature": "[N]",
|
|
658
|
-
"bug": "[N]",
|
|
659
|
-
"refactor": "[N]",
|
|
660
|
-
"config": "[N]",
|
|
661
|
-
"docs": "[N]",
|
|
662
|
-
"infra": "[N]",
|
|
663
|
-
"security": "[N]",
|
|
664
|
-
"test": "[N]",
|
|
665
|
-
"observation": "[N]",
|
|
666
|
-
"deficiency": "[N]",
|
|
667
|
-
"ambiguity": "[N]"
|
|
668
|
-
}
|
|
669
|
-
}
|
|
670
|
-
}
|
|
671
|
-
```
|
|
471
|
+
|
|
672
472
|
```
|
|
673
473
|
|
|
674
474
|
## Output Examples
|
|
@@ -752,45 +552,6 @@ Critical issues include:
|
|
|
752
552
|
- **AF-006** Error paths completely untested
|
|
753
553
|
|
|
754
554
|
|
|
755
|
-
## Priority & Severity Mapping
|
|
756
|
-
|
|
757
|
-
When generating the JSON OUTPUT section, map issues as follows:
|
|
758
|
-
|
|
759
|
-
**Priority (for triage):**
|
|
760
|
-
| Severity | Priority | Meaning |
|
|
761
|
-
|----------|----------|---------|
|
|
762
|
-
| Critical | `critical` | Blocks progression, must fix now |
|
|
763
|
-
| High | `critical` | Should fix before next phase |
|
|
764
|
-
| Medium | `suggested` | Should fix soon |
|
|
765
|
-
| Low | `backlog` | Optional improvement |
|
|
766
|
-
| Info | `backlog` | Informational only |
|
|
767
|
-
|
|
768
|
-
**Severity is derived from failure_code suffix:**
|
|
769
|
-
| Suffix | Severity | Priority |
|
|
770
|
-
|--------|----------|----------|
|
|
771
|
-
| `/C` | critical | critical |
|
|
772
|
-
| `/H` | high | critical |
|
|
773
|
-
| `/M` | medium | suggested |
|
|
774
|
-
| `/L` | low | backlog |
|
|
775
|
-
| `/I` | info | backlog |
|
|
776
|
-
|
|
777
|
-
## Failure Code Selection
|
|
778
|
-
|
|
779
|
-
**1. Use the default code from the criterion that failed** (e.g., `→ SEM-COM/H`)
|
|
780
|
-
|
|
781
|
-
**2. Adjust severity letter based on actual impact:**
|
|
782
|
-
- `/C` - Security vulnerabilities, data loss risk, crashes, blocks all functionality
|
|
783
|
-
- `/H` - Broken functionality, missing critical tests, significant user impact
|
|
784
|
-
- `/M` - Code quality issues, maintainability concerns, moderate impact
|
|
785
|
-
- `/L` - Style issues, minor improvements, low impact
|
|
786
|
-
- `/I` - Suggestions, informational, no functional impact
|
|
787
|
-
|
|
788
|
-
**3. Consider context when adjusting:**
|
|
789
|
-
- A naming issue in a public API → elevate to `/M` or `/H`
|
|
790
|
-
- A complexity issue in rarely-used code → may stay at `/L`
|
|
791
|
-
- Missing error handling in user-facing code → `/H` or `/C`
|
|
792
|
-
- Missing error handling in internal utility → `/M`
|
|
793
|
-
|
|
794
555
|
## Edge Case Handling
|
|
795
556
|
|
|
796
557
|
### No test files
|
|
@@ -841,10 +602,6 @@ When generating the JSON OUTPUT section, map issues as follows:
|
|
|
841
602
|
### Position in Pipeline
|
|
842
603
|
**Runs after:** code-validator
|
|
843
604
|
|
|
844
|
-
### Handoff: What This Agent Passes Downstream
|
|
845
|
-
|
|
846
|
-
### Handoff: What This Agent Expects From Predecessors
|
|
847
|
-
**From code-validator:** Validation results from code-validator
|
|
848
605
|
|
|
849
606
|
---
|
|
850
607
|
|