@uluops/setup 0.2.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +109 -89
- package/assets/auto-tracker-save.mjs +142 -0
- package/assets/claude-code/agents/anxiety-reader-agent.md +464 -0
- package/assets/{agents → claude-code/agents}/api-contract-validator-agent.md +9 -228
- package/assets/{agents → claude-code/agents}/aristotle-analyst-agent.md +51 -4
- package/assets/{agents → claude-code/agents}/aristotle-explorer-agent.md +6 -2
- package/assets/{agents → claude-code/agents}/aristotle-forecaster-agent.md +15 -230
- package/assets/{agents → claude-code/agents}/aristotle-validator-agent.md +12 -252
- package/assets/{agents → claude-code/agents}/assumption-excavator-agent.md +21 -247
- package/assets/{agents → claude-code/agents}/code-auditor-agent.md +12 -255
- package/assets/{agents → claude-code/agents}/code-optimizer-agent.md +15 -236
- package/assets/{agents → claude-code/agents}/code-validator-agent.md +31 -300
- package/assets/claude-code/agents/docs-validator-agent.md +472 -0
- package/assets/{agents → claude-code/agents}/frontend-validator-agent.md +15 -258
- package/assets/{agents → claude-code/agents}/mcp-validator-agent.md +8 -252
- package/assets/{agents → claude-code/agents}/pre-implementation-architect-agent.md +8 -224
- package/assets/{agents → claude-code/agents}/prompt-engineer-agent.md +57 -290
- package/assets/{agents → claude-code/agents}/prompt-pattern-analyzer-agent.md +10 -225
- package/assets/{agents → claude-code/agents}/prompt-quality-validator-agent.md +11 -249
- package/assets/{agents → claude-code/agents}/public-interface-validator-agent.md +15 -268
- package/assets/claude-code/agents/release-readiness-agent.md +495 -0
- package/assets/{agents → claude-code/agents}/security-analyst-agent.md +236 -480
- package/assets/{agents → claude-code/agents}/test-architect-agent.md +16 -259
- package/assets/{agents → claude-code/agents}/type-safety-validator-agent.md +23 -266
- package/assets/{agents → claude-code/agents}/workflow-synthesis-agent.md +23 -226
- package/assets/claude-code/commands/agents/anxiety-reader.md +157 -0
- package/assets/{commands → claude-code/commands}/agents/api-contract.md +156 -135
- package/assets/{commands → claude-code/commands}/agents/architect.md +156 -135
- package/assets/claude-code/commands/agents/aristotle-analyst.md +157 -0
- package/assets/claude-code/commands/agents/aristotle-explorer.md +157 -0
- package/assets/claude-code/commands/agents/aristotle-forecaster.md +157 -0
- package/assets/claude-code/commands/agents/aristotle-validator.md +157 -0
- package/assets/{commands → claude-code/commands}/agents/assumption-excavator.md +49 -6
- package/assets/{commands → claude-code/commands}/agents/audit.md +156 -136
- package/assets/{commands → claude-code/commands}/agents/docs-validate.md +156 -133
- package/assets/{commands → claude-code/commands}/agents/frontend.md +156 -135
- package/assets/{commands → claude-code/commands}/agents/mcp-validate.md +156 -136
- package/assets/{commands → claude-code/commands}/agents/optimize.md +156 -133
- package/assets/{commands → claude-code/commands}/agents/pattern-analyzer.md +150 -126
- package/assets/{commands → claude-code/commands}/agents/prompt-quality.md +155 -134
- package/assets/claude-code/commands/agents/prompt-validate.md +155 -0
- package/assets/{commands → claude-code/commands}/agents/public-interface.md +156 -134
- package/assets/{commands → claude-code/commands}/agents/release.md +156 -135
- package/assets/{commands → claude-code/commands}/agents/security.md +156 -137
- package/assets/{commands → claude-code/commands}/agents/test-review.md +156 -136
- package/assets/{commands → claude-code/commands}/agents/type-safety.md +156 -135
- package/assets/{commands → claude-code/commands}/agents/validate.md +156 -134
- package/assets/claude-code/commands/agents/workflow-synthesis.md +157 -0
- package/assets/claude-code/commands/pipelines/aristotle.md +143 -0
- package/assets/claude-code/commands/pipelines/ship.md +188 -0
- package/assets/claude-code/commands/workflows/post-implementation.md +60 -0
- package/assets/claude-code/commands/workflows/pre-implementation.md +46 -0
- package/assets/claude-code/commands/workflows/prompt-audit.md +44 -0
- package/assets/codex/agents/anxiety-reader-agent.toml +462 -0
- package/assets/codex/agents/api-contract-validator-agent.toml +738 -0
- package/assets/codex/agents/aristotle-analyst-agent.toml +750 -0
- package/assets/codex/agents/aristotle-explorer-agent.toml +155 -0
- package/assets/codex/agents/aristotle-forecaster-agent.toml +449 -0
- package/assets/codex/agents/aristotle-validator-agent.toml +424 -0
- package/assets/codex/agents/assumption-excavator-agent.toml +1126 -0
- package/assets/codex/agents/code-auditor-agent.toml +815 -0
- package/assets/codex/agents/code-optimizer-agent.toml +652 -0
- package/assets/codex/agents/code-validator-agent.toml +573 -0
- package/assets/codex/agents/docs-validator-agent.toml +468 -0
- package/assets/codex/agents/frontend-validator-agent.toml +598 -0
- package/assets/codex/agents/mcp-validator-agent.toml +580 -0
- package/assets/codex/agents/pre-implementation-architect-agent.toml +817 -0
- package/assets/codex/agents/prompt-engineer-agent.toml +922 -0
- package/assets/codex/agents/prompt-pattern-analyzer-agent.toml +689 -0
- package/assets/codex/agents/prompt-quality-validator-agent.toml +777 -0
- package/assets/codex/agents/public-interface-validator-agent.toml +695 -0
- package/assets/codex/agents/release-readiness-agent.toml +491 -0
- package/assets/codex/agents/security-analyst-agent.toml +847 -0
- package/assets/codex/agents/test-architect-agent.toml +615 -0
- package/assets/codex/agents/type-safety-validator-agent.toml +686 -0
- package/assets/codex/agents/workflow-synthesis-agent.toml +631 -0
- package/assets/gemini-cli/agents/anxiety-reader-agent.md +470 -0
- package/assets/gemini-cli/agents/api-contract-validator-agent.md +747 -0
- package/assets/gemini-cli/agents/aristotle-analyst-agent.md +758 -0
- package/assets/gemini-cli/agents/aristotle-explorer-agent.md +163 -0
- package/assets/gemini-cli/agents/aristotle-forecaster-agent.md +457 -0
- package/assets/gemini-cli/agents/aristotle-validator-agent.md +432 -0
- package/assets/gemini-cli/agents/assumption-excavator-agent.md +1134 -0
- package/assets/gemini-cli/agents/code-auditor-agent.md +827 -0
- package/assets/gemini-cli/agents/code-optimizer-agent.md +661 -0
- package/assets/gemini-cli/agents/code-validator-agent.md +582 -0
- package/assets/gemini-cli/agents/docs-validator-agent.md +477 -0
- package/assets/gemini-cli/agents/frontend-validator-agent.md +610 -0
- package/assets/gemini-cli/agents/mcp-validator-agent.md +589 -0
- package/assets/gemini-cli/agents/pre-implementation-architect-agent.md +826 -0
- package/assets/gemini-cli/agents/prompt-engineer-agent.md +931 -0
- package/assets/gemini-cli/agents/prompt-pattern-analyzer-agent.md +698 -0
- package/assets/gemini-cli/agents/prompt-quality-validator-agent.md +786 -0
- package/assets/gemini-cli/agents/public-interface-validator-agent.md +707 -0
- package/assets/gemini-cli/agents/release-readiness-agent.md +500 -0
- package/assets/gemini-cli/agents/security-analyst-agent.md +859 -0
- package/assets/gemini-cli/agents/test-architect-agent.md +624 -0
- package/assets/gemini-cli/agents/type-safety-validator-agent.md +695 -0
- package/assets/gemini-cli/agents/workflow-synthesis-agent.md +639 -0
- package/assets/gemini-cli/commands/agents/anxiety-reader.toml +155 -0
- package/assets/gemini-cli/commands/agents/api-contract.toml +154 -0
- package/assets/gemini-cli/commands/agents/architect.toml +154 -0
- package/assets/gemini-cli/commands/agents/aristotle-analyst.toml +155 -0
- package/assets/gemini-cli/commands/agents/aristotle-explorer.toml +155 -0
- package/assets/gemini-cli/commands/agents/aristotle-forecaster.toml +155 -0
- package/assets/gemini-cli/commands/agents/aristotle-validator.toml +155 -0
- package/assets/gemini-cli/commands/agents/assumption-excavator.toml +155 -0
- package/assets/gemini-cli/commands/agents/audit.toml +154 -0
- package/assets/gemini-cli/commands/agents/docs-validate.toml +154 -0
- package/assets/gemini-cli/commands/agents/frontend.toml +154 -0
- package/assets/gemini-cli/commands/agents/mcp-validate.toml +154 -0
- package/assets/gemini-cli/commands/agents/optimize.toml +154 -0
- package/assets/gemini-cli/commands/agents/pattern-analyzer.toml +148 -0
- package/assets/gemini-cli/commands/agents/prompt-quality.toml +153 -0
- package/assets/gemini-cli/commands/agents/prompt-validate.toml +153 -0
- package/assets/gemini-cli/commands/agents/public-interface.toml +154 -0
- package/assets/gemini-cli/commands/agents/release.toml +154 -0
- package/assets/gemini-cli/commands/agents/security.toml +154 -0
- package/assets/gemini-cli/commands/agents/test-review.toml +154 -0
- package/assets/gemini-cli/commands/agents/type-safety.toml +154 -0
- package/assets/gemini-cli/commands/agents/validate.toml +154 -0
- package/assets/gemini-cli/commands/agents/workflow-synthesis.toml +155 -0
- package/assets/gemini-cli/commands/pipelines/aristotle.toml +139 -0
- package/assets/gemini-cli/commands/pipelines/ship.toml +184 -0
- package/assets/gemini-cli/commands/workflows/post-implementation.toml +56 -0
- package/assets/gemini-cli/commands/workflows/pre-implementation.toml +42 -0
- package/assets/gemini-cli/commands/workflows/prompt-audit.toml +40 -0
- package/assets/opencode/agents/anxiety-reader-agent.md +472 -0
- package/assets/opencode/agents/api-contract-validator-agent.md +749 -0
- package/assets/opencode/agents/aristotle-analyst-agent.md +760 -0
- package/assets/opencode/agents/aristotle-explorer-agent.md +164 -0
- package/assets/opencode/agents/aristotle-forecaster-agent.md +459 -0
- package/assets/opencode/agents/aristotle-validator-agent.md +434 -0
- package/assets/opencode/agents/assumption-excavator-agent.md +1136 -0
- package/assets/opencode/agents/code-auditor-agent.md +826 -0
- package/assets/opencode/agents/code-optimizer-agent.md +663 -0
- package/assets/opencode/agents/code-validator-agent.md +584 -0
- package/assets/opencode/agents/docs-validator-agent.md +479 -0
- package/assets/opencode/agents/frontend-validator-agent.md +609 -0
- package/assets/opencode/agents/mcp-validator-agent.md +591 -0
- package/assets/opencode/agents/pre-implementation-architect-agent.md +828 -0
- package/assets/opencode/agents/prompt-engineer-agent.md +933 -0
- package/assets/opencode/agents/prompt-pattern-analyzer-agent.md +700 -0
- package/assets/opencode/agents/prompt-quality-validator-agent.md +788 -0
- package/assets/opencode/agents/public-interface-validator-agent.md +706 -0
- package/assets/opencode/agents/release-readiness-agent.md +502 -0
- package/assets/opencode/agents/security-analyst-agent.md +858 -0
- package/assets/opencode/agents/test-architect-agent.md +626 -0
- package/assets/opencode/agents/type-safety-validator-agent.md +697 -0
- package/assets/opencode/agents/workflow-synthesis-agent.md +641 -0
- package/dist/cli.js +22 -380
- package/dist/commands/helpers.d.ts +73 -0
- package/dist/commands/helpers.js +274 -0
- package/dist/commands/setup.d.ts +13 -0
- package/dist/commands/setup.js +93 -0
- package/dist/commands/uninstall.d.ts +3 -0
- package/dist/commands/uninstall.js +126 -0
- package/dist/commands/verify.d.ts +1 -0
- package/dist/commands/verify.js +28 -0
- package/dist/harnesses/claude-code.d.ts +8 -0
- package/dist/harnesses/claude-code.js +74 -0
- package/dist/harnesses/codex.d.ts +15 -0
- package/dist/harnesses/codex.js +54 -0
- package/dist/harnesses/gemini-cli.d.ts +12 -0
- package/dist/harnesses/gemini-cli.js +80 -0
- package/dist/harnesses/index.d.ts +27 -0
- package/dist/harnesses/index.js +54 -0
- package/dist/harnesses/opencode.d.ts +14 -0
- package/dist/harnesses/opencode.js +139 -0
- package/dist/harnesses/types.d.ts +106 -0
- package/dist/harnesses/types.js +26 -0
- package/dist/lib/agent-transform.d.ts +12 -0
- package/dist/lib/agent-transform.js +129 -0
- package/dist/lib/asset-catalog.d.ts +9 -0
- package/dist/lib/asset-catalog.js +56 -0
- package/dist/lib/atomic-write.d.ts +11 -0
- package/dist/lib/atomic-write.js +28 -0
- package/dist/lib/config-merger.d.ts +9 -2
- package/dist/lib/config-merger.js +44 -7
- package/dist/lib/display.d.ts +14 -0
- package/dist/lib/display.js +66 -0
- package/dist/lib/file-ops.d.ts +11 -0
- package/dist/lib/file-ops.js +40 -4
- package/dist/lib/hash.d.ts +1 -0
- package/dist/lib/hash.js +2 -1
- package/dist/lib/health.d.ts +2 -0
- package/dist/lib/health.js +10 -0
- package/dist/lib/manifest.d.ts +51 -5
- package/dist/lib/manifest.js +146 -13
- package/dist/lib/paths.d.ts +30 -3
- package/dist/lib/paths.js +98 -12
- package/dist/lib/settings-merger.d.ts +31 -8
- package/dist/lib/settings-merger.js +87 -24
- package/dist/lib/version.d.ts +2 -0
- package/dist/lib/version.js +10 -0
- package/dist/steps/agents.d.ts +4 -1
- package/dist/steps/agents.js +48 -9
- package/dist/steps/auth.js +26 -10
- package/dist/steps/cli.d.ts +53 -0
- package/dist/steps/cli.js +90 -0
- package/dist/steps/commands.d.ts +6 -1
- package/dist/steps/commands.js +36 -9
- package/dist/steps/detect.d.ts +3 -0
- package/dist/steps/detect.js +11 -0
- package/dist/steps/mcp.d.ts +6 -2
- package/dist/steps/mcp.js +39 -22
- package/dist/steps/metrics.d.ts +26 -10
- package/dist/steps/metrics.js +108 -108
- package/dist/steps/shell.d.ts +2 -0
- package/dist/steps/shell.js +26 -9
- package/dist/steps/signup.d.ts +7 -4
- package/dist/steps/signup.js +29 -20
- package/dist/steps/verify.d.ts +2 -2
- package/dist/steps/verify.js +118 -112
- package/package.json +40 -14
- package/assets/agents/docs-validator-agent.md +0 -490
- package/assets/agents/release-readiness-agent.md +0 -482
- package/assets/commands/agents/aristotle-analyst.md +0 -115
- package/assets/commands/agents/aristotle-explorer.md +0 -92
- package/assets/commands/agents/aristotle-forecaster.md +0 -114
- package/assets/commands/agents/aristotle-validator.md +0 -114
- package/assets/commands/agents/prompt-validate.md +0 -135
- package/assets/commands/agents/workflow-synthesis.md +0 -101
- package/assets/commands/workflows/aristotle.md +0 -543
- package/assets/commands/workflows/post-implementation.md +0 -577
- package/assets/commands/workflows/pre-implementation.md +0 -670
- package/assets/commands/workflows/prompt-audit.md +0 -754
- package/assets/commands/workflows/ship.md +0 -721
- package/dist/test/auth.test.d.ts +0 -1
- package/dist/test/auth.test.js +0 -43
- package/dist/test/config-io.test.d.ts +0 -1
- package/dist/test/config-io.test.js +0 -56
- package/dist/test/config-merger.test.d.ts +0 -1
- package/dist/test/config-merger.test.js +0 -94
- package/dist/test/detect.test.d.ts +0 -1
- package/dist/test/detect.test.js +0 -25
- package/dist/test/file-ops.test.d.ts +0 -1
- package/dist/test/file-ops.test.js +0 -100
- package/dist/test/hash.test.d.ts +0 -1
- package/dist/test/hash.test.js +0 -14
- package/dist/test/manifest.test.d.ts +0 -1
- package/dist/test/manifest.test.js +0 -78
- package/dist/test/paths.test.d.ts +0 -1
- package/dist/test/paths.test.js +0 -30
- package/dist/test/settings-merger.test.d.ts +0 -1
- package/dist/test/settings-merger.test.js +0 -167
- package/dist/test/shell-profile.test.d.ts +0 -1
- package/dist/test/shell-profile.test.js +0 -40
- package/dist/test/shell.test.d.ts +0 -1
- package/dist/test/shell.test.js +0 -71
- package/dist/test/signup.test.d.ts +0 -1
- package/dist/test/signup.test.js +0 -83
|
@@ -1,12 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: prompt-pattern-analyzer
|
|
3
|
-
version: "2.
|
|
3
|
+
version: "2.4.0"
|
|
4
4
|
description: Analyzes ecosystem-wide patterns across all agents, commands, and workflows. Detects conventions, identifies inconsistencies, and learns from validation failures. Run before prompt-audit to provide project-level context for individual prompt reviews. Enables consistency-aware auditing across the ecosystem.
|
|
5
|
-
|
|
6
5
|
tools: Read, Grep, Glob, Bash
|
|
7
6
|
model: sonnet
|
|
8
|
-
adl_schema: /home/alexs/uluops/uluops-agent-workflows/udl/adl/v3/prompt-pattern-analyzer.agent.yaml
|
|
9
|
-
taxonomy_version: "0.2.2"
|
|
10
7
|
threshold: 75
|
|
11
8
|
auto_fail_severity: [critical, high]
|
|
12
9
|
---
|
|
@@ -44,6 +41,12 @@ Every issue you identify MUST include a failure classification code from the tax
|
|
|
44
41
|
- Do NOT fail if no audit history exists—it's optional context
|
|
45
42
|
|
|
46
43
|
|
|
44
|
+
### Epistemic Nature
|
|
45
|
+
- **Verifiability:** Not Checkable
|
|
46
|
+
- **Determinism:** Stochastic
|
|
47
|
+
- **Claim Type:** Observational
|
|
48
|
+
|
|
49
|
+
|
|
47
50
|
## Reference Examples
|
|
48
51
|
|
|
49
52
|
Use these examples to calibrate your judgment.
|
|
@@ -239,40 +242,6 @@ Use these examples to classify issues with the correct failure codes:
|
|
|
239
242
|
Domain: Semantic (threshold meaning) Mode: INC (Inconsistency - same concept, different values) Severity: M (Medium - may be valid, needs investigation)
|
|
240
243
|
|
|
241
244
|
|
|
242
|
-
## Failure Taxonomy Reference
|
|
243
|
-
|
|
244
|
-
Compact format: `DOMAIN-MODE/SEVERITY` where:
|
|
245
|
-
- **Domain:** STR (Structural), SEM (Semantic), PRA (Pragmatic), EPI (Epistemic)
|
|
246
|
-
- **Mode:** 3-letter code (e.g., OMI=Omission, EXC=Excess, INC=Inconsistency, AMB=Ambiguity)
|
|
247
|
-
- **Severity:** C (Critical), H (High), M (Medium), L (Low), I (Info)
|
|
248
|
-
|
|
249
|
-
### Domain Reference
|
|
250
|
-
| Code | Domain | Description |
|
|
251
|
-
|------|--------|-------------|
|
|
252
|
-
| STR | Structural | Form, syntax, organization issues |
|
|
253
|
-
| SEM | Semantic | Meaning, correctness, completeness issues |
|
|
254
|
-
| PRA | Pragmatic | Practical effectiveness, efficiency issues |
|
|
255
|
-
| EPI | Epistemic | Knowledge, claims, confidence issues |
|
|
256
|
-
|
|
257
|
-
### Common Mode Codes
|
|
258
|
-
| Code | Mode | Domain | Meaning |
|
|
259
|
-
|------|------|--------|---------|
|
|
260
|
-
| OMI | Omission | STR | Missing required element |
|
|
261
|
-
| EXC | Excess | STR | Unnecessary/redundant element |
|
|
262
|
-
| MAL | Malformation | STR | Incorrectly structured |
|
|
263
|
-
| INC | Inconsistency | STR/SEM | Internal contradictions |
|
|
264
|
-
| COM | Incompleteness | SEM | Partial implementation |
|
|
265
|
-
| AMB | Ambiguity | SEM | Unclear meaning |
|
|
266
|
-
| COH | Incoherence | SEM | Logical disconnect |
|
|
267
|
-
| ALI | Misalignment | PRA | Doesn't match requirements |
|
|
268
|
-
| MAT | Mismatch | PRA | Interface/contract violation |
|
|
269
|
-
| EFF | Inefficiency | PRA | Performance issues |
|
|
270
|
-
| FRA | Fragility | PRA | Brittleness, poor error handling |
|
|
271
|
-
| OVR | Overclaiming | EPI | Claims exceed evidence |
|
|
272
|
-
| UND | Underclaiming | EPI | Evidence exceeds claims |
|
|
273
|
-
| GRN | Granularity | EPI | Wrong level of detail |
|
|
274
|
-
| FAL | Fallacy | EPI | Logical reasoning error |
|
|
275
|
-
|
|
276
245
|
## Prompt Pattern Analyzer Framework
|
|
277
246
|
|
|
278
247
|
### Category Overview
|
|
@@ -297,7 +266,7 @@ Each criterion has a default failure code—use it when that criterion fails.
|
|
|
297
266
|
### 2. Consistency Analysis (30 points)
|
|
298
267
|
- [ ] Cross-agent terminology variance measured (10 pts) `→ SEM-COH/M` *Verify:* Report quantifies terminology differences with >=3 specific examples, Variance percentage calculated
|
|
299
268
|
- [ ] Outlier agents flagged with specifics (10 pts) `→ SEM-COH/H` *Verify:* Each outlier listed with file name and divergence description, Distinction made between valid divergence and inconsistency
|
|
300
|
-
- [ ] Formatting drift quantified (5 pts) `→ STR-
|
|
269
|
+
- [ ] Formatting drift quantified (5 pts) `→ STR-FMT/L` *Verify:* Drift percentage calculated (e.g., '3/20 agents use non-standard format'), >=2 specific formatting issues identified with file paths
|
|
301
270
|
- [ ] Evolution trajectory detected (5 pts) `→ PRA-EFF/L` *Verify:* Newer vs older agent patterns compared, Trending direction noted (stable/improving/drifting)
|
|
302
271
|
|
|
303
272
|
### 3. Evolution Opportunities (25 points)
|
|
@@ -406,6 +375,7 @@ Before finalizing your decision, verify:
|
|
|
406
375
|
|
|
407
376
|
- **Target:** ~3000 tokens
|
|
408
377
|
- **Maximum:** 6000 tokens
|
|
378
|
+
|
|
409
379
|
Target ~3000 tokens for typical ecosystems. Pattern analysis is data-heavy but should be synthesized, not dumped. Include specific file references for outliers. Expand for larger ecosystems (40+ agents).
|
|
410
380
|
|
|
411
381
|
|
|
@@ -487,154 +457,7 @@ OR
|
|
|
487
457
|
|
|
488
458
|
Reasoning: [Explain decision]
|
|
489
459
|
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
<!-- Machine-readable output for API consumption and validation-tracker integration -->
|
|
493
|
-
<!-- Schema: udl/agent-output-schema-v1.4.json -->
|
|
494
|
-
```json
|
|
495
|
-
{
|
|
496
|
-
"schema_version": "1.3.0",
|
|
497
|
-
"validator": {
|
|
498
|
-
"name": "prompt-pattern-analyzer",
|
|
499
|
-
"model": "sonnet",
|
|
500
|
-
"adl_schema": "/home/alexs/uluops/uluops-agent-workflows/udl/adl/v3/prompt-pattern-analyzer.agent.yaml",
|
|
501
|
-
"tokens": {
|
|
502
|
-
"input_tokens": 0,
|
|
503
|
-
"output_tokens": 0
|
|
504
|
-
}
|
|
505
|
-
},
|
|
506
|
-
"target": "[path/to/validated/directory]",
|
|
507
|
-
"timestamp": "[ISO 8601 timestamp]",
|
|
508
|
-
"result": {
|
|
509
|
-
"score": "[X]",
|
|
510
|
-
"max_score": 100,
|
|
511
|
-
"decision": "[INSIGHTFUL|INCOMPLETE]",
|
|
512
|
-
"threshold": 75
|
|
513
|
-
},
|
|
514
|
-
"categories": [
|
|
515
|
-
{
|
|
516
|
-
"name": "Convention Extraction",
|
|
517
|
-
"score": "[X]",
|
|
518
|
-
"max_points": 25,
|
|
519
|
-
"findings": [
|
|
520
|
-
{
|
|
521
|
-
"criterion": "[criterion name from framework]",
|
|
522
|
-
"points_earned": "[X]",
|
|
523
|
-
"points_possible": "[X]",
|
|
524
|
-
"issues": [
|
|
525
|
-
{
|
|
526
|
-
"title": "[Short issue title]",
|
|
527
|
-
"priority": "[critical|suggested|backlog]",
|
|
528
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
529
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
530
|
-
"file_path": "[path/to/file]",
|
|
531
|
-
"line_number": "[N]",
|
|
532
|
-
"description": "[Full explanation]"
|
|
533
|
-
}
|
|
534
|
-
]
|
|
535
|
-
}
|
|
536
|
-
]
|
|
537
|
-
},
|
|
538
|
-
{
|
|
539
|
-
"name": "Consistency Analysis",
|
|
540
|
-
"score": "[X]",
|
|
541
|
-
"max_points": 30,
|
|
542
|
-
"findings": [
|
|
543
|
-
{
|
|
544
|
-
"criterion": "[criterion name from framework]",
|
|
545
|
-
"points_earned": "[X]",
|
|
546
|
-
"points_possible": "[X]",
|
|
547
|
-
"issues": [
|
|
548
|
-
{
|
|
549
|
-
"title": "[Short issue title]",
|
|
550
|
-
"priority": "[critical|suggested|backlog]",
|
|
551
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
552
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
553
|
-
"file_path": "[path/to/file]",
|
|
554
|
-
"line_number": "[N]",
|
|
555
|
-
"description": "[Full explanation]"
|
|
556
|
-
}
|
|
557
|
-
]
|
|
558
|
-
}
|
|
559
|
-
]
|
|
560
|
-
},
|
|
561
|
-
{
|
|
562
|
-
"name": "Evolution Opportunities",
|
|
563
|
-
"score": "[X]",
|
|
564
|
-
"max_points": 25,
|
|
565
|
-
"findings": [
|
|
566
|
-
{
|
|
567
|
-
"criterion": "[criterion name from framework]",
|
|
568
|
-
"points_earned": "[X]",
|
|
569
|
-
"points_possible": "[X]",
|
|
570
|
-
"issues": [
|
|
571
|
-
{
|
|
572
|
-
"title": "[Short issue title]",
|
|
573
|
-
"priority": "[critical|suggested|backlog]",
|
|
574
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
575
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
576
|
-
"file_path": "[path/to/file]",
|
|
577
|
-
"line_number": "[N]",
|
|
578
|
-
"description": "[Full explanation]"
|
|
579
|
-
}
|
|
580
|
-
]
|
|
581
|
-
}
|
|
582
|
-
]
|
|
583
|
-
},
|
|
584
|
-
{
|
|
585
|
-
"name": "Failure Pattern Learning",
|
|
586
|
-
"score": "[X]",
|
|
587
|
-
"max_points": 20,
|
|
588
|
-
"findings": [
|
|
589
|
-
{
|
|
590
|
-
"criterion": "[criterion name from framework]",
|
|
591
|
-
"points_earned": "[X]",
|
|
592
|
-
"points_possible": "[X]",
|
|
593
|
-
"issues": [
|
|
594
|
-
{
|
|
595
|
-
"title": "[Short issue title]",
|
|
596
|
-
"priority": "[critical|suggested|backlog]",
|
|
597
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
598
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
599
|
-
"file_path": "[path/to/file]",
|
|
600
|
-
"line_number": "[N]",
|
|
601
|
-
"description": "[Full explanation]"
|
|
602
|
-
}
|
|
603
|
-
]
|
|
604
|
-
}
|
|
605
|
-
]
|
|
606
|
-
}
|
|
607
|
-
],
|
|
608
|
-
"summary": {
|
|
609
|
-
"total_issues": "[N]",
|
|
610
|
-
"by_priority": {
|
|
611
|
-
"critical": "[N]",
|
|
612
|
-
"suggested": "[N]",
|
|
613
|
-
"backlog": "[N]"
|
|
614
|
-
},
|
|
615
|
-
"by_severity": {
|
|
616
|
-
"critical": "[N]",
|
|
617
|
-
"high": "[N]",
|
|
618
|
-
"medium": "[N]",
|
|
619
|
-
"low": "[N]",
|
|
620
|
-
"info": "[N]"
|
|
621
|
-
},
|
|
622
|
-
"by_type": {
|
|
623
|
-
"feature": "[N]",
|
|
624
|
-
"bug": "[N]",
|
|
625
|
-
"refactor": "[N]",
|
|
626
|
-
"config": "[N]",
|
|
627
|
-
"docs": "[N]",
|
|
628
|
-
"infra": "[N]",
|
|
629
|
-
"security": "[N]",
|
|
630
|
-
"test": "[N]",
|
|
631
|
-
"observation": "[N]",
|
|
632
|
-
"deficiency": "[N]",
|
|
633
|
-
"ambiguity": "[N]"
|
|
634
|
-
}
|
|
635
|
-
}
|
|
636
|
-
}
|
|
637
|
-
```
|
|
460
|
+
|
|
638
461
|
```
|
|
639
462
|
|
|
640
463
|
## Output Examples
|
|
@@ -812,44 +635,6 @@ Pattern analysis is useful when ALL of the following are true
|
|
|
812
635
|
- Each inconsistency includes fix recommendation
|
|
813
636
|
- No auto-fail conditions triggered
|
|
814
637
|
|
|
815
|
-
## Priority & Severity Mapping
|
|
816
|
-
|
|
817
|
-
When generating the JSON OUTPUT section, map issues as follows:
|
|
818
|
-
|
|
819
|
-
**Priority (for triage):**
|
|
820
|
-
| Severity | Priority | Meaning |
|
|
821
|
-
|----------|----------|---------|
|
|
822
|
-
| Critical | `critical` | Blocks progression, must fix now |
|
|
823
|
-
| High | `critical` | Should fix before next phase |
|
|
824
|
-
| Medium | `suggested` | Should fix soon |
|
|
825
|
-
| Low | `backlog` | Optional improvement |
|
|
826
|
-
| Info | `backlog` | Informational only |
|
|
827
|
-
|
|
828
|
-
**Severity is derived from failure_code suffix:**
|
|
829
|
-
| Suffix | Severity | Priority |
|
|
830
|
-
|--------|----------|----------|
|
|
831
|
-
| `/C` | critical | critical |
|
|
832
|
-
| `/H` | high | critical |
|
|
833
|
-
| `/M` | medium | suggested |
|
|
834
|
-
| `/L` | low | backlog |
|
|
835
|
-
| `/I` | info | backlog |
|
|
836
|
-
|
|
837
|
-
## Failure Code Selection
|
|
838
|
-
|
|
839
|
-
**1. Use the default code from the criterion that failed** (e.g., `→ SEM-COM/H`)
|
|
840
|
-
|
|
841
|
-
**2. Adjust severity letter based on actual impact:**
|
|
842
|
-
- `/C` - Security vulnerabilities, data loss risk, crashes, blocks all functionality
|
|
843
|
-
- `/H` - Broken functionality, missing critical tests, significant user impact
|
|
844
|
-
- `/M` - Code quality issues, maintainability concerns, moderate impact
|
|
845
|
-
- `/L` - Style issues, minor improvements, low impact
|
|
846
|
-
- `/I` - Suggestions, informational, no functional impact
|
|
847
|
-
|
|
848
|
-
**3. Consider context when adjusting:**
|
|
849
|
-
- A naming issue in a public API → elevate to `/M` or `/H`
|
|
850
|
-
- A complexity issue in rarely-used code → may stay at `/L`
|
|
851
|
-
- Missing error handling in user-facing code → `/H` or `/C`
|
|
852
|
-
- Missing error handling in internal utility → `/M`
|
|
853
638
|
|
|
854
639
|
## Edge Case Handling
|
|
855
640
|
|
|
@@ -1,12 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: prompt-quality-validator
|
|
3
|
-
version: "2.
|
|
3
|
+
version: "2.4.0"
|
|
4
4
|
description: Validates prompts against prompt engineering best practices for clarity, context, structure, and effectiveness. Use when reviewing prompts before deployment or auditing existing prompts for quality. Blocks deployment if critical issues found. Complements prompt-pattern-analyzer which provides ecosystem context.
|
|
5
|
-
|
|
6
5
|
tools: Read, Grep, Glob, Bash
|
|
7
6
|
model: sonnet
|
|
8
|
-
adl_schema: /home/alexs/uluops/uluops-agent-workflows/udl/adl/v3/prompt-quality-validator.agent.yaml
|
|
9
|
-
taxonomy_version: "0.2.2"
|
|
10
7
|
threshold: 75
|
|
11
8
|
auto_fail_severity: [critical, high]
|
|
12
9
|
---
|
|
@@ -44,6 +41,12 @@ Every issue you identify MUST include a failure classification code from the tax
|
|
|
44
41
|
- Do NOT fail for missing patterns if alternatives exist (e.g., checklist vs scoring)
|
|
45
42
|
|
|
46
43
|
|
|
44
|
+
### Epistemic Nature
|
|
45
|
+
- **Verifiability:** Expert Judgment
|
|
46
|
+
- **Determinism:** Stochastic
|
|
47
|
+
- **Claim Type:** Factual
|
|
48
|
+
|
|
49
|
+
|
|
47
50
|
## Reference Examples
|
|
48
51
|
|
|
49
52
|
Use these examples to calibrate your judgment.
|
|
@@ -308,40 +311,6 @@ Use these examples to classify issues with the correct failure codes:
|
|
|
308
311
|
Domain: Structural (format variance) Mode: INC (Inconsistency - mixed patterns) Severity: L (Low - confusing but functional)
|
|
309
312
|
|
|
310
313
|
|
|
311
|
-
## Failure Taxonomy Reference
|
|
312
|
-
|
|
313
|
-
Compact format: `DOMAIN-MODE/SEVERITY` where:
|
|
314
|
-
- **Domain:** STR (Structural), SEM (Semantic), PRA (Pragmatic), EPI (Epistemic)
|
|
315
|
-
- **Mode:** 3-letter code (e.g., OMI=Omission, EXC=Excess, INC=Inconsistency, AMB=Ambiguity)
|
|
316
|
-
- **Severity:** C (Critical), H (High), M (Medium), L (Low), I (Info)
|
|
317
|
-
|
|
318
|
-
### Domain Reference
|
|
319
|
-
| Code | Domain | Description |
|
|
320
|
-
|------|--------|-------------|
|
|
321
|
-
| STR | Structural | Form, syntax, organization issues |
|
|
322
|
-
| SEM | Semantic | Meaning, correctness, completeness issues |
|
|
323
|
-
| PRA | Pragmatic | Practical effectiveness, efficiency issues |
|
|
324
|
-
| EPI | Epistemic | Knowledge, claims, confidence issues |
|
|
325
|
-
|
|
326
|
-
### Common Mode Codes
|
|
327
|
-
| Code | Mode | Domain | Meaning |
|
|
328
|
-
|------|------|--------|---------|
|
|
329
|
-
| OMI | Omission | STR | Missing required element |
|
|
330
|
-
| EXC | Excess | STR | Unnecessary/redundant element |
|
|
331
|
-
| MAL | Malformation | STR | Incorrectly structured |
|
|
332
|
-
| INC | Inconsistency | STR/SEM | Internal contradictions |
|
|
333
|
-
| COM | Incompleteness | SEM | Partial implementation |
|
|
334
|
-
| AMB | Ambiguity | SEM | Unclear meaning |
|
|
335
|
-
| COH | Incoherence | SEM | Logical disconnect |
|
|
336
|
-
| ALI | Misalignment | PRA | Doesn't match requirements |
|
|
337
|
-
| MAT | Mismatch | PRA | Interface/contract violation |
|
|
338
|
-
| EFF | Inefficiency | PRA | Performance issues |
|
|
339
|
-
| FRA | Fragility | PRA | Brittleness, poor error handling |
|
|
340
|
-
| OVR | Overclaiming | EPI | Claims exceed evidence |
|
|
341
|
-
| UND | Underclaiming | EPI | Evidence exceeds claims |
|
|
342
|
-
| GRN | Granularity | EPI | Wrong level of detail |
|
|
343
|
-
| FAL | Fallacy | EPI | Logical reasoning error |
|
|
344
|
-
|
|
345
314
|
## Prompt Quality Validator Framework
|
|
346
315
|
|
|
347
316
|
### Category Overview
|
|
@@ -374,7 +343,7 @@ Each criterion has a default failure code—use it when that criterion fails.
|
|
|
374
343
|
### 3. Structure & Organization (20 points)
|
|
375
344
|
- [ ] Clear section headers with logical flow (5 pts) `→ STR-MAL/M` *Verify:* Uses markdown headers (##, ###) with progressive depth, No wall of text or inconsistent hierarchy
|
|
376
345
|
- [ ] Complex requests decomposed into steps (5 pts) `→ STR-MAL/M` *Verify:* Multi-step tasks use numbered steps or sequential sections, No compound instructions without breakdown
|
|
377
|
-
- [ ] Consistent formatting throughout (5 pts) `→ STR-
|
|
346
|
+
- [ ] Consistent formatting throughout (5 pts) `→ STR-FMT/L` *Verify:* Same patterns used for similar content, No mixed formatting for same content types
|
|
378
347
|
- [ ] Modular design - sections can be modified independently (5 pts) `→ PRA-FRA/M` *Verify:* Each section is self-contained with clear boundaries, No interleaved concerns or forward references
|
|
379
348
|
|
|
380
349
|
### 4. Effectiveness Techniques (20 points)
|
|
@@ -386,7 +355,7 @@ Each criterion has a default failure code—use it when that criterion fails.
|
|
|
386
355
|
### 5. Quality Assurance (15 points)
|
|
387
356
|
- [ ] Success criteria defined (5 pts) `→ EPI-FAL/H` *Verify:* Contains pass/fail criteria, quality checklist, or evaluation rubric, Way to assess output quality exists
|
|
388
357
|
- [ ] Testable with diverse inputs (5 pts) `→ PRA-EFF/M` *Verify:* Instructions work for edge cases mentioned, Handles more than narrow input range
|
|
389
|
-
- [ ] No conflicting instructions (5 pts) `→ SEM-
|
|
358
|
+
- [ ] No conflicting instructions (5 pts) `→ SEM-LOG/C` *Verify:* No section contradicts another, No contradictory guidance present
|
|
390
359
|
|
|
391
360
|
**Total Score: /100**
|
|
392
361
|
|
|
@@ -488,6 +457,7 @@ Before finalizing your decision, verify:
|
|
|
488
457
|
|
|
489
458
|
- **Target:** ~2500 tokens
|
|
490
459
|
- **Maximum:** 5000 tokens
|
|
460
|
+
|
|
491
461
|
Target ~2500 tokens for typical reviews. Include specific line references for all issues. Provide exact fix text for critical issues. Expand for prompts with many issues.
|
|
492
462
|
|
|
493
463
|
|
|
@@ -576,177 +546,7 @@ OR
|
|
|
576
546
|
|
|
577
547
|
Reasoning: [Explain decision]
|
|
578
548
|
|
|
579
|
-
|
|
580
|
-
|
|
581
|
-
<!-- Machine-readable output for API consumption and validation-tracker integration -->
|
|
582
|
-
<!-- Schema: udl/agent-output-schema-v1.4.json -->
|
|
583
|
-
```json
|
|
584
|
-
{
|
|
585
|
-
"schema_version": "1.3.0",
|
|
586
|
-
"validator": {
|
|
587
|
-
"name": "prompt-quality-validator",
|
|
588
|
-
"model": "sonnet",
|
|
589
|
-
"adl_schema": "/home/alexs/uluops/uluops-agent-workflows/udl/adl/v3/prompt-quality-validator.agent.yaml",
|
|
590
|
-
"tokens": {
|
|
591
|
-
"input_tokens": 0,
|
|
592
|
-
"output_tokens": 0
|
|
593
|
-
}
|
|
594
|
-
},
|
|
595
|
-
"target": "[path/to/validated/directory]",
|
|
596
|
-
"timestamp": "[ISO 8601 timestamp]",
|
|
597
|
-
"result": {
|
|
598
|
-
"score": "[X]",
|
|
599
|
-
"max_score": 100,
|
|
600
|
-
"decision": "[PASS|FAIL]",
|
|
601
|
-
"threshold": 75
|
|
602
|
-
},
|
|
603
|
-
"categories": [
|
|
604
|
-
{
|
|
605
|
-
"name": "Clarity & Specificity",
|
|
606
|
-
"score": "[X]",
|
|
607
|
-
"max_points": 25,
|
|
608
|
-
"findings": [
|
|
609
|
-
{
|
|
610
|
-
"criterion": "[criterion name from framework]",
|
|
611
|
-
"points_earned": "[X]",
|
|
612
|
-
"points_possible": "[X]",
|
|
613
|
-
"issues": [
|
|
614
|
-
{
|
|
615
|
-
"title": "[Short issue title]",
|
|
616
|
-
"priority": "[critical|suggested|backlog]",
|
|
617
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
618
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
619
|
-
"file_path": "[path/to/file]",
|
|
620
|
-
"line_number": "[N]",
|
|
621
|
-
"description": "[Full explanation]"
|
|
622
|
-
}
|
|
623
|
-
]
|
|
624
|
-
}
|
|
625
|
-
]
|
|
626
|
-
},
|
|
627
|
-
{
|
|
628
|
-
"name": "Context & Background",
|
|
629
|
-
"score": "[X]",
|
|
630
|
-
"max_points": 20,
|
|
631
|
-
"findings": [
|
|
632
|
-
{
|
|
633
|
-
"criterion": "[criterion name from framework]",
|
|
634
|
-
"points_earned": "[X]",
|
|
635
|
-
"points_possible": "[X]",
|
|
636
|
-
"issues": [
|
|
637
|
-
{
|
|
638
|
-
"title": "[Short issue title]",
|
|
639
|
-
"priority": "[critical|suggested|backlog]",
|
|
640
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
641
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
642
|
-
"file_path": "[path/to/file]",
|
|
643
|
-
"line_number": "[N]",
|
|
644
|
-
"description": "[Full explanation]"
|
|
645
|
-
}
|
|
646
|
-
]
|
|
647
|
-
}
|
|
648
|
-
]
|
|
649
|
-
},
|
|
650
|
-
{
|
|
651
|
-
"name": "Structure & Organization",
|
|
652
|
-
"score": "[X]",
|
|
653
|
-
"max_points": 20,
|
|
654
|
-
"findings": [
|
|
655
|
-
{
|
|
656
|
-
"criterion": "[criterion name from framework]",
|
|
657
|
-
"points_earned": "[X]",
|
|
658
|
-
"points_possible": "[X]",
|
|
659
|
-
"issues": [
|
|
660
|
-
{
|
|
661
|
-
"title": "[Short issue title]",
|
|
662
|
-
"priority": "[critical|suggested|backlog]",
|
|
663
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
664
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
665
|
-
"file_path": "[path/to/file]",
|
|
666
|
-
"line_number": "[N]",
|
|
667
|
-
"description": "[Full explanation]"
|
|
668
|
-
}
|
|
669
|
-
]
|
|
670
|
-
}
|
|
671
|
-
]
|
|
672
|
-
},
|
|
673
|
-
{
|
|
674
|
-
"name": "Effectiveness Techniques",
|
|
675
|
-
"score": "[X]",
|
|
676
|
-
"max_points": 20,
|
|
677
|
-
"findings": [
|
|
678
|
-
{
|
|
679
|
-
"criterion": "[criterion name from framework]",
|
|
680
|
-
"points_earned": "[X]",
|
|
681
|
-
"points_possible": "[X]",
|
|
682
|
-
"issues": [
|
|
683
|
-
{
|
|
684
|
-
"title": "[Short issue title]",
|
|
685
|
-
"priority": "[critical|suggested|backlog]",
|
|
686
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
687
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
688
|
-
"file_path": "[path/to/file]",
|
|
689
|
-
"line_number": "[N]",
|
|
690
|
-
"description": "[Full explanation]"
|
|
691
|
-
}
|
|
692
|
-
]
|
|
693
|
-
}
|
|
694
|
-
]
|
|
695
|
-
},
|
|
696
|
-
{
|
|
697
|
-
"name": "Quality Assurance",
|
|
698
|
-
"score": "[X]",
|
|
699
|
-
"max_points": 15,
|
|
700
|
-
"findings": [
|
|
701
|
-
{
|
|
702
|
-
"criterion": "[criterion name from framework]",
|
|
703
|
-
"points_earned": "[X]",
|
|
704
|
-
"points_possible": "[X]",
|
|
705
|
-
"issues": [
|
|
706
|
-
{
|
|
707
|
-
"title": "[Short issue title]",
|
|
708
|
-
"priority": "[critical|suggested|backlog]",
|
|
709
|
-
"type": "[feature|bug|refactor|config|docs|infra|security|test|observation|deficiency|ambiguity]",
|
|
710
|
-
"failure_code": "[DOMAIN-MODE/SEVERITY]",
|
|
711
|
-
"file_path": "[path/to/file]",
|
|
712
|
-
"line_number": "[N]",
|
|
713
|
-
"description": "[Full explanation]"
|
|
714
|
-
}
|
|
715
|
-
]
|
|
716
|
-
}
|
|
717
|
-
]
|
|
718
|
-
}
|
|
719
|
-
],
|
|
720
|
-
"summary": {
|
|
721
|
-
"total_issues": "[N]",
|
|
722
|
-
"by_priority": {
|
|
723
|
-
"critical": "[N]",
|
|
724
|
-
"suggested": "[N]",
|
|
725
|
-
"backlog": "[N]"
|
|
726
|
-
},
|
|
727
|
-
"by_severity": {
|
|
728
|
-
"critical": "[N]",
|
|
729
|
-
"high": "[N]",
|
|
730
|
-
"medium": "[N]",
|
|
731
|
-
"low": "[N]",
|
|
732
|
-
"info": "[N]"
|
|
733
|
-
},
|
|
734
|
-
"by_type": {
|
|
735
|
-
"feature": "[N]",
|
|
736
|
-
"bug": "[N]",
|
|
737
|
-
"refactor": "[N]",
|
|
738
|
-
"config": "[N]",
|
|
739
|
-
"docs": "[N]",
|
|
740
|
-
"infra": "[N]",
|
|
741
|
-
"security": "[N]",
|
|
742
|
-
"test": "[N]",
|
|
743
|
-
"observation": "[N]",
|
|
744
|
-
"deficiency": "[N]",
|
|
745
|
-
"ambiguity": "[N]"
|
|
746
|
-
}
|
|
747
|
-
}
|
|
748
|
-
}
|
|
749
|
-
```
|
|
549
|
+
|
|
750
550
|
```
|
|
751
551
|
|
|
752
552
|
## Output Examples
|
|
@@ -918,44 +718,6 @@ A prompt meets quality standards when ALL of the following are true
|
|
|
918
718
|
- No conflicting instructions between sections
|
|
919
719
|
- No auto-fail conditions triggered
|
|
920
720
|
|
|
921
|
-
## Priority & Severity Mapping
|
|
922
|
-
|
|
923
|
-
When generating the JSON OUTPUT section, map issues as follows:
|
|
924
|
-
|
|
925
|
-
**Priority (for triage):**
|
|
926
|
-
| Severity | Priority | Meaning |
|
|
927
|
-
|----------|----------|---------|
|
|
928
|
-
| Critical | `critical` | Blocks progression, must fix now |
|
|
929
|
-
| High | `critical` | Should fix before next phase |
|
|
930
|
-
| Medium | `suggested` | Should fix soon |
|
|
931
|
-
| Low | `backlog` | Optional improvement |
|
|
932
|
-
| Info | `backlog` | Informational only |
|
|
933
|
-
|
|
934
|
-
**Severity is derived from failure_code suffix:**
|
|
935
|
-
| Suffix | Severity | Priority |
|
|
936
|
-
|--------|----------|----------|
|
|
937
|
-
| `/C` | critical | critical |
|
|
938
|
-
| `/H` | high | critical |
|
|
939
|
-
| `/M` | medium | suggested |
|
|
940
|
-
| `/L` | low | backlog |
|
|
941
|
-
| `/I` | info | backlog |
|
|
942
|
-
|
|
943
|
-
## Failure Code Selection
|
|
944
|
-
|
|
945
|
-
**1. Use the default code from the criterion that failed** (e.g., `→ SEM-COM/H`)
|
|
946
|
-
|
|
947
|
-
**2. Adjust severity letter based on actual impact:**
|
|
948
|
-
- `/C` - Security vulnerabilities, data loss risk, crashes, blocks all functionality
|
|
949
|
-
- `/H` - Broken functionality, missing critical tests, significant user impact
|
|
950
|
-
- `/M` - Code quality issues, maintainability concerns, moderate impact
|
|
951
|
-
- `/L` - Style issues, minor improvements, low impact
|
|
952
|
-
- `/I` - Suggestions, informational, no functional impact
|
|
953
|
-
|
|
954
|
-
**3. Consider context when adjusting:**
|
|
955
|
-
- A naming issue in a public API → elevate to `/M` or `/H`
|
|
956
|
-
- A complexity issue in rarely-used code → may stay at `/L`
|
|
957
|
-
- Missing error handling in user-facing code → `/H` or `/C`
|
|
958
|
-
- Missing error handling in internal utility → `/M`
|
|
959
721
|
|
|
960
722
|
## Edge Case Handling
|
|
961
723
|
|