create-byan-agent 1.1.3 → 1.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +273 -202
- package/LICENSE +21 -21
- package/README.md +1251 -421
- package/bin/create-byan-agent-backup.js +220 -220
- package/bin/create-byan-agent-fixed.js +301 -301
- package/bin/create-byan-agent.js +155 -322
- package/lib/errors.js +61 -0
- package/lib/exit-codes.js +54 -0
- package/lib/platforms/claude-code.js +113 -0
- package/lib/platforms/codex.js +92 -0
- package/lib/platforms/copilot-cli.js +123 -0
- package/lib/platforms/index.js +14 -0
- package/lib/platforms/vscode.js +51 -0
- package/lib/utils/config-loader.js +79 -0
- package/lib/utils/file-utils.js +104 -0
- package/lib/utils/git-detector.js +35 -0
- package/lib/utils/logger.js +64 -0
- package/lib/utils/node-detector.js +58 -0
- package/lib/utils/os-detector.js +74 -0
- package/lib/utils/yaml-utils.js +87 -0
- package/lib/yanstaller/backuper.js +308 -0
- package/lib/yanstaller/detector.js +141 -0
- package/lib/yanstaller/index.js +93 -0
- package/lib/yanstaller/installer.js +225 -0
- package/lib/yanstaller/interviewer.js +250 -0
- package/lib/yanstaller/recommender.js +298 -0
- package/lib/yanstaller/troubleshooter.js +498 -0
- package/lib/yanstaller/validator.js +578 -0
- package/lib/yanstaller/wizard.js +211 -0
- package/package.json +61 -55
- package/templates/.github/agents/bmad-agent-bmad-master.md +15 -15
- package/templates/.github/agents/bmad-agent-bmb-agent-builder.md +15 -15
- package/templates/.github/agents/bmad-agent-bmb-module-builder.md +15 -15
- package/templates/.github/agents/bmad-agent-bmb-workflow-builder.md +15 -15
- package/templates/.github/agents/bmad-agent-bmm-analyst.md +15 -15
- package/templates/.github/agents/bmad-agent-bmm-architect.md +15 -15
- package/templates/.github/agents/bmad-agent-bmm-dev.md +15 -15
- package/templates/.github/agents/bmad-agent-bmm-pm.md +15 -15
- package/templates/.github/agents/bmad-agent-bmm-quick-flow-solo-dev.md +15 -15
- package/templates/.github/agents/bmad-agent-bmm-quinn.md +15 -15
- package/templates/.github/agents/bmad-agent-bmm-sm.md +15 -15
- package/templates/.github/agents/bmad-agent-bmm-tech-writer.md +15 -15
- package/templates/.github/agents/bmad-agent-bmm-ux-designer.md +15 -15
- package/templates/.github/agents/bmad-agent-byan-test.md +32 -0
- package/templates/.github/agents/bmad-agent-byan.md +224 -224
- package/templates/.github/agents/bmad-agent-carmack.md +18 -0
- package/templates/.github/agents/bmad-agent-cis-brainstorming-coach.md +15 -15
- package/templates/.github/agents/bmad-agent-cis-creative-problem-solver.md +15 -15
- package/templates/.github/agents/bmad-agent-cis-design-thinking-coach.md +15 -15
- package/templates/.github/agents/bmad-agent-cis-innovation-strategist.md +15 -15
- package/templates/.github/agents/bmad-agent-cis-presentation-master.md +15 -15
- package/templates/.github/agents/bmad-agent-cis-storyteller.md +15 -15
- package/templates/.github/agents/bmad-agent-marc.md +48 -48
- package/templates/.github/agents/bmad-agent-patnote.md +48 -0
- package/templates/.github/agents/bmad-agent-rachid.md +47 -47
- package/templates/.github/agents/bmad-agent-tea-tea.md +15 -15
- package/templates/.github/agents/bmad-agent-test-dynamic.md +21 -0
- package/templates/.github/agents/expert-merise-agile.md +1 -0
- package/templates/.github/agents/franck.md +379 -0
- package/templates/_bmad/bmb/agents/agent-builder.md +59 -59
- package/templates/_bmad/bmb/agents/byan-test.md +116 -116
- package/templates/_bmad/bmb/agents/byan.md +215 -215
- package/templates/_bmad/bmb/agents/marc.md +303 -303
- package/templates/_bmad/bmb/agents/module-builder.md +60 -60
- package/templates/_bmad/bmb/agents/patnote.md +495 -495
- package/templates/_bmad/bmb/agents/rachid.md +184 -184
- package/templates/_bmad/bmb/agents/workflow-builder.md +61 -61
- package/templates/_bmad/bmb/workflows/byan/data/mantras.yaml +272 -272
- package/templates/_bmad/bmb/workflows/byan/data/templates.yaml +59 -59
- package/templates/_bmad/bmb/workflows/byan/delete-agent-workflow.md +657 -657
- package/templates/_bmad/bmb/workflows/byan/edit-agent-workflow.md +688 -688
- package/templates/_bmad/bmb/workflows/byan/interview-workflow.md +753 -753
- package/templates/_bmad/bmb/workflows/byan/quick-create-workflow.md +450 -450
- package/templates/_bmad/bmb/workflows/byan/templates/base-agent-template.md +79 -79
- package/templates/_bmad/bmb/workflows/byan/validate-agent-workflow.md +676 -676
- package/templates/_bmad/core/agents/carmack.md +238 -238
|
@@ -1,676 +1,676 @@
|
|
|
1
|
-
# BYAN Validate Agent Workflow
|
|
2
|
-
|
|
3
|
-
**Workflow:** Comprehensive Agent Validation Against 64 Mantras
|
|
4
|
-
**Duration:** 5-10 minutes
|
|
5
|
-
**Methodology:** Merise Agile + TDD + Full Mantra Compliance
|
|
6
|
-
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
## OVERVIEW
|
|
10
|
-
|
|
11
|
-
Validate Agent checks agent specifications against:
|
|
12
|
-
- 64 Mantras (39 Conception + 25 AI Agents)
|
|
13
|
-
- BMAD compliance standards
|
|
14
|
-
- Merise Agile best practices
|
|
15
|
-
- All business rules (RG-*)
|
|
16
|
-
|
|
17
|
-
**When to Use:**
|
|
18
|
-
- After creating new agent (Quick Create or Full Interview)
|
|
19
|
-
- After editing existing agent
|
|
20
|
-
- Before deploying agent
|
|
21
|
-
- Regular quality audits
|
|
22
|
-
- Troubleshooting agent issues
|
|
23
|
-
|
|
24
|
-
**Output:**
|
|
25
|
-
- Validation report with pass/fail/warnings
|
|
26
|
-
- Compliance score (0-100%)
|
|
27
|
-
- Actionable recommendations
|
|
28
|
-
- Auto-fix suggestions where possible
|
|
29
|
-
|
|
30
|
-
---
|
|
31
|
-
|
|
32
|
-
## VALIDATION LEVELS
|
|
33
|
-
|
|
34
|
-
### Level 1: CRITICAL (Must Pass)
|
|
35
|
-
- Business rules violations (RG-*)
|
|
36
|
-
- BMAD format compliance
|
|
37
|
-
- Structural requirements
|
|
38
|
-
- Security issues
|
|
39
|
-
|
|
40
|
-
**Failure = Agent CANNOT be deployed**
|
|
41
|
-
|
|
42
|
-
### Level 2: IMPORTANT (Should Pass)
|
|
43
|
-
- Mantra compliance
|
|
44
|
-
- Best practices
|
|
45
|
-
- Documentation completeness
|
|
46
|
-
- Test coverage
|
|
47
|
-
|
|
48
|
-
**Failure = Agent CAN deploy but with warnings**
|
|
49
|
-
|
|
50
|
-
### Level 3: SUGGESTIONS (Nice to Have)
|
|
51
|
-
- Optimization opportunities
|
|
52
|
-
- Enhanced capabilities
|
|
53
|
-
- Additional use cases
|
|
54
|
-
- Improved documentation
|
|
55
|
-
|
|
56
|
-
**Failure = Recommendations only**
|
|
57
|
-
|
|
58
|
-
---
|
|
59
|
-
|
|
60
|
-
## WORKFLOW STEPS
|
|
61
|
-
|
|
62
|
-
### Step 1: Select Agent to Validate
|
|
63
|
-
|
|
64
|
-
```
|
|
65
|
-
"Which agent should I validate?
|
|
66
|
-
|
|
67
|
-
Available agents:
|
|
68
|
-
1. {agent_1} - [{status}] - Last validated: {date}
|
|
69
|
-
2. {agent_2} - [{status}] - Last validated: {date}
|
|
70
|
-
3. {agent_3} - [{status}] - Never validated
|
|
71
|
-
...
|
|
72
|
-
|
|
73
|
-
Agent number or name: ____
|
|
74
|
-
"
|
|
75
|
-
```
|
|
76
|
-
|
|
77
|
-
**Load agent:**
|
|
78
|
-
```python
|
|
79
|
-
agent_spec = AgentSpec.load(agent_id_or_name)
|
|
80
|
-
context = ProjectContext.load(agent_spec.context_id)
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
---
|
|
84
|
-
|
|
85
|
-
### Step 2: Run Validation Checks
|
|
86
|
-
|
|
87
|
-
**Progress indicator:**
|
|
88
|
-
```
|
|
89
|
-
"Validating agent: {agent_name}
|
|
90
|
-
|
|
91
|
-
Running validation checks:
|
|
92
|
-
[▓▓▓▓▓▓▓▓░░] 80% - Checking mantra compliance...
|
|
93
|
-
|
|
94
|
-
Checks completed:
|
|
95
|
-
✅ Business rules (RG-*)
|
|
96
|
-
✅ BMAD format compliance
|
|
97
|
-
✅ Structural requirements
|
|
98
|
-
⏳ Mantra compliance (39/39)
|
|
99
|
-
⏳ AI Agent mantras (25/25)
|
|
100
|
-
⏳ Best practices
|
|
101
|
-
⏳ Documentation
|
|
102
|
-
"
|
|
103
|
-
```
|
|
104
|
-
|
|
105
|
-
---
|
|
106
|
-
|
|
107
|
-
### Step 3: Business Rules Validation (CRITICAL)
|
|
108
|
-
|
|
109
|
-
#### RG-AGT-001: Unique Agent Name
|
|
110
|
-
|
|
111
|
-
```python
|
|
112
|
-
def validate_RG_AGT_001(agent_spec):
|
|
113
|
-
existing = AgentSpec.find_by_name(agent_spec.agent_name)
|
|
114
|
-
if existing and existing.id != agent_spec.id:
|
|
115
|
-
return ValidationError(
|
|
116
|
-
rule="RG-AGT-001",
|
|
117
|
-
level="CRITICAL",
|
|
118
|
-
message=f"Agent name '{agent_spec.agent_name}' already exists",
|
|
119
|
-
fix="Choose a unique name"
|
|
120
|
-
)
|
|
121
|
-
|
|
122
|
-
# Validate kebab-case
|
|
123
|
-
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', agent_spec.agent_name):
|
|
124
|
-
return ValidationError(
|
|
125
|
-
rule="RG-AGT-001",
|
|
126
|
-
level="CRITICAL",
|
|
127
|
-
message=f"Agent name must be kebab-case",
|
|
128
|
-
current=agent_spec.agent_name,
|
|
129
|
-
fix="Convert to kebab-case, e.g., 'backend-expert'"
|
|
130
|
-
)
|
|
131
|
-
|
|
132
|
-
return ValidationSuccess()
|
|
133
|
-
```
|
|
134
|
-
|
|
135
|
-
#### RG-AGT-002: Minimum 3 Capabilities
|
|
136
|
-
|
|
137
|
-
```python
|
|
138
|
-
def validate_RG_AGT_002(agent_spec):
|
|
139
|
-
cap_count = len(agent_spec.capabilities)
|
|
140
|
-
if cap_count < 3:
|
|
141
|
-
return ValidationError(
|
|
142
|
-
rule="RG-AGT-002",
|
|
143
|
-
level="CRITICAL",
|
|
144
|
-
message=f"Agent has only {cap_count} capabilities, minimum is 3",
|
|
145
|
-
fix=f"Add {3 - cap_count} more capabilities"
|
|
146
|
-
)
|
|
147
|
-
return ValidationSuccess()
|
|
148
|
-
```
|
|
149
|
-
|
|
150
|
-
#### RG-AGT-003: Minimum 5 Mantras
|
|
151
|
-
|
|
152
|
-
```python
|
|
153
|
-
def validate_RG_AGT_003(agent_spec):
|
|
154
|
-
mantra_count = len(agent_spec.mantras_applied)
|
|
155
|
-
if mantra_count < 5:
|
|
156
|
-
return ValidationWarning(
|
|
157
|
-
rule="RG-AGT-003",
|
|
158
|
-
level="IMPORTANT",
|
|
159
|
-
message=f"Agent applies only {mantra_count} mantras, recommended minimum is 5",
|
|
160
|
-
fix="Add more relevant mantras",
|
|
161
|
-
suggestions=suggest_mantras(agent_spec)
|
|
162
|
-
)
|
|
163
|
-
return ValidationSuccess()
|
|
164
|
-
```
|
|
165
|
-
|
|
166
|
-
#### RG-AGT-004: Minimum 3 Use Cases
|
|
167
|
-
|
|
168
|
-
```python
|
|
169
|
-
def validate_RG_AGT_004(agent_spec):
|
|
170
|
-
use_case_count = len(agent_spec.use_cases)
|
|
171
|
-
if use_case_count < 3:
|
|
172
|
-
return ValidationWarning(
|
|
173
|
-
rule="RG-AGT-004",
|
|
174
|
-
level="IMPORTANT",
|
|
175
|
-
message=f"Agent has only {use_case_count} use cases, recommended minimum is 3",
|
|
176
|
-
fix="Document more use cases to clarify agent value"
|
|
177
|
-
)
|
|
178
|
-
return ValidationSuccess()
|
|
179
|
-
```
|
|
180
|
-
|
|
181
|
-
#### RG-AGT-005: Valid Status Workflow
|
|
182
|
-
|
|
183
|
-
```python
|
|
184
|
-
def validate_RG_AGT_005(agent_spec):
|
|
185
|
-
valid_statuses = ["draft", "validated", "deployed", "deprecated"]
|
|
186
|
-
if agent_spec.status not in valid_statuses:
|
|
187
|
-
return ValidationError(
|
|
188
|
-
rule="RG-AGT-005",
|
|
189
|
-
level="CRITICAL",
|
|
190
|
-
message=f"Invalid status: {agent_spec.status}",
|
|
191
|
-
valid_values=valid_statuses,
|
|
192
|
-
fix="Set to valid status"
|
|
193
|
-
)
|
|
194
|
-
return ValidationSuccess()
|
|
195
|
-
```
|
|
196
|
-
|
|
197
|
-
---
|
|
198
|
-
|
|
199
|
-
### Step 4: Mantra Compliance Validation
|
|
200
|
-
|
|
201
|
-
**For each of 64 mantras, check if applied correctly:**
|
|
202
|
-
|
|
203
|
-
#### Conception Mantras (39)
|
|
204
|
-
|
|
205
|
-
**Mantra #1: Le Modèle Sert le Métier**
|
|
206
|
-
```python
|
|
207
|
-
def validate_mantra_01(agent_spec, context):
|
|
208
|
-
# Check if agent knowledge includes business concepts
|
|
209
|
-
business_concepts = context.glossaire.keys()
|
|
210
|
-
agent_business = agent_spec.knowledge_business
|
|
211
|
-
|
|
212
|
-
overlap = set(business_concepts) & set(agent_business)
|
|
213
|
-
coverage = len(overlap) / len(business_concepts) if business_concepts else 0
|
|
214
|
-
|
|
215
|
-
if coverage < 0.3: # Less than 30% coverage
|
|
216
|
-
return ValidationWarning(
|
|
217
|
-
mantra=1,
|
|
218
|
-
message="Agent has weak connection to business domain",
|
|
219
|
-
coverage=f"{coverage*100:.0f}%",
|
|
220
|
-
recommendation="Add more business concepts to knowledge_business"
|
|
221
|
-
)
|
|
222
|
-
return ValidationSuccess()
|
|
223
|
-
```
|
|
224
|
-
|
|
225
|
-
**Mantra #4: Fail Fast, Fail Visible**
|
|
226
|
-
```python
|
|
227
|
-
def validate_mantra_04(agent_spec):
|
|
228
|
-
# Check if agent has error handling capabilities
|
|
229
|
-
capabilities = [c['description'].lower() for c in agent_spec.capabilities]
|
|
230
|
-
|
|
231
|
-
error_keywords = ['error', 'fail', 'exception', 'validate', 'check']
|
|
232
|
-
has_error_handling = any(kw in cap for cap in capabilities for kw in error_keywords)
|
|
233
|
-
|
|
234
|
-
if not has_error_handling and 4 in agent_spec.mantras_applied:
|
|
235
|
-
return ValidationWarning(
|
|
236
|
-
mantra=4,
|
|
237
|
-
message="Agent claims Mantra #4 but has no error handling capabilities",
|
|
238
|
-
recommendation="Add capability for error detection/handling"
|
|
239
|
-
)
|
|
240
|
-
return ValidationSuccess()
|
|
241
|
-
```
|
|
242
|
-
|
|
243
|
-
**Mantra #7: KISS (Keep It Simple)**
|
|
244
|
-
```python
|
|
245
|
-
def validate_mantra_07(agent_spec):
|
|
246
|
-
# Check if agent is over-complicated
|
|
247
|
-
cap_count = len(agent_spec.capabilities)
|
|
248
|
-
|
|
249
|
-
if cap_count > 10:
|
|
250
|
-
return ValidationWarning(
|
|
251
|
-
mantra=7,
|
|
252
|
-
message=f"Agent has {cap_count} capabilities - may violate KISS principle",
|
|
253
|
-
recommendation="Consider splitting into multiple focused agents"
|
|
254
|
-
)
|
|
255
|
-
|
|
256
|
-
# Check role clarity
|
|
257
|
-
role_words = len(agent_spec.role.split())
|
|
258
|
-
if role_words > 20:
|
|
259
|
-
return ValidationWarning(
|
|
260
|
-
mantra=7,
|
|
261
|
-
message="Agent role description is too complex",
|
|
262
|
-
recommendation="Simplify role to one clear sentence"
|
|
263
|
-
)
|
|
264
|
-
|
|
265
|
-
return ValidationSuccess()
|
|
266
|
-
```
|
|
267
|
-
|
|
268
|
-
**Mantra #18: TDD is Not Optional**
|
|
269
|
-
```python
|
|
270
|
-
def validate_mantra_18(agent_spec):
|
|
271
|
-
if 18 in agent_spec.mantras_applied:
|
|
272
|
-
# Check if agent has test-related capabilities
|
|
273
|
-
capabilities = [c['description'].lower() for c in agent_spec.capabilities]
|
|
274
|
-
test_keywords = ['test', 'tdd', 'coverage', 'assertion']
|
|
275
|
-
|
|
276
|
-
has_test_cap = any(kw in cap for cap in capabilities for kw in test_keywords)
|
|
277
|
-
|
|
278
|
-
if not has_test_cap:
|
|
279
|
-
return ValidationWarning(
|
|
280
|
-
mantra=18,
|
|
281
|
-
message="Agent claims TDD mantra but has no testing capabilities",
|
|
282
|
-
recommendation="Add test generation or verification capability"
|
|
283
|
-
)
|
|
284
|
-
return ValidationSuccess()
|
|
285
|
-
```
|
|
286
|
-
|
|
287
|
-
**Mantra #37: Rasoir d'Ockham**
|
|
288
|
-
```python
|
|
289
|
-
def validate_mantra_37(agent_spec):
|
|
290
|
-
# Always check for simplicity
|
|
291
|
-
issues = []
|
|
292
|
-
|
|
293
|
-
# Check capabilities count
|
|
294
|
-
if len(agent_spec.capabilities) > 7:
|
|
295
|
-
issues.append("Too many capabilities (>7) - prefer focused agent")
|
|
296
|
-
|
|
297
|
-
# Check for overlapping capabilities
|
|
298
|
-
caps = [c['description'] for c in agent_spec.capabilities]
|
|
299
|
-
similar_pairs = find_similar_capabilities(caps)
|
|
300
|
-
if similar_pairs:
|
|
301
|
-
issues.append(f"Overlapping capabilities detected: {similar_pairs}")
|
|
302
|
-
|
|
303
|
-
# Check for unnecessary complexity
|
|
304
|
-
if len(agent_spec.mantras_applied) > 15:
|
|
305
|
-
issues.append("Too many mantras (>15) - focus on most critical ones")
|
|
306
|
-
|
|
307
|
-
if issues:
|
|
308
|
-
return ValidationWarning(
|
|
309
|
-
mantra=37,
|
|
310
|
-
message="Agent violates Ockham's Razor (simplicity principle)",
|
|
311
|
-
issues=issues,
|
|
312
|
-
recommendation="Simplify agent by removing non-essential elements"
|
|
313
|
-
)
|
|
314
|
-
|
|
315
|
-
return ValidationSuccess()
|
|
316
|
-
```
|
|
317
|
-
|
|
318
|
-
**Mantra #39: Évaluation des Conséquences**
|
|
319
|
-
```python
|
|
320
|
-
def validate_mantra_39(agent_spec):
|
|
321
|
-
# Check if agent is deployed without consequence evaluation
|
|
322
|
-
if agent_spec.status == "deployed":
|
|
323
|
-
if not agent_spec.consequences_evaluated:
|
|
324
|
-
return ValidationError(
|
|
325
|
-
mantra=39,
|
|
326
|
-
level="CRITICAL",
|
|
327
|
-
message="Agent deployed without consequences evaluation",
|
|
328
|
-
fix="Run Edit Agent workflow to evaluate consequences"
|
|
329
|
-
)
|
|
330
|
-
return ValidationSuccess()
|
|
331
|
-
```
|
|
332
|
-
|
|
333
|
-
#### AI Agent Mantras (25)
|
|
334
|
-
|
|
335
|
-
**Mantra IA-1: Trust But Verify**
|
|
336
|
-
```python
|
|
337
|
-
def validate_mantra_IA_01(agent_spec):
|
|
338
|
-
if 'IA-1' in agent_spec.mantras_applied:
|
|
339
|
-
# Check if agent has validation capabilities
|
|
340
|
-
capabilities = [c['description'].lower() for c in agent_spec.capabilities]
|
|
341
|
-
validation_keywords = ['verify', 'validate', 'check', 'review', 'audit']
|
|
342
|
-
|
|
343
|
-
has_validation = any(kw in cap for cap in capabilities for kw in validation_keywords)
|
|
344
|
-
|
|
345
|
-
if not has_validation:
|
|
346
|
-
return ValidationWarning(
|
|
347
|
-
mantra='IA-1',
|
|
348
|
-
message="Agent claims Trust But Verify but lacks validation capabilities",
|
|
349
|
-
recommendation="Add verification/validation capability"
|
|
350
|
-
)
|
|
351
|
-
return ValidationSuccess()
|
|
352
|
-
```
|
|
353
|
-
|
|
354
|
-
**Mantra IA-16: Challenge Before Confirm**
|
|
355
|
-
```python
|
|
356
|
-
def validate_mantra_IA_16(agent_spec):
|
|
357
|
-
if 'IA-16' in agent_spec.mantras_applied:
|
|
358
|
-
# Check persona for challenging behavior
|
|
359
|
-
persona_text = agent_spec.persona.lower()
|
|
360
|
-
challenge_keywords = ['challenge', 'question', 'verify', 'validate', 'devil\'s advocate']
|
|
361
|
-
|
|
362
|
-
has_challenge_behavior = any(kw in persona_text for kw in challenge_keywords)
|
|
363
|
-
|
|
364
|
-
if not has_challenge_behavior:
|
|
365
|
-
return ValidationWarning(
|
|
366
|
-
mantra='IA-16',
|
|
367
|
-
message="Agent claims Challenge Before Confirm but persona doesn't reflect challenging behavior",
|
|
368
|
-
recommendation="Update persona to include questioning/challenging approach"
|
|
369
|
-
)
|
|
370
|
-
return ValidationSuccess()
|
|
371
|
-
```
|
|
372
|
-
|
|
373
|
-
**Mantra IA-23: No Emoji Pollution**
|
|
374
|
-
```python
|
|
375
|
-
def validate_mantra_IA_23(agent_spec):
|
|
376
|
-
# Check for emojis in technical content
|
|
377
|
-
emoji_pattern = re.compile("["
|
|
378
|
-
u"\U0001F600-\U0001F64F" # emoticons
|
|
379
|
-
u"\U0001F300-\U0001F5FF" # symbols & pictographs
|
|
380
|
-
u"\U0001F680-\U0001F6FF" # transport & map symbols
|
|
381
|
-
"]+", flags=re.UNICODE)
|
|
382
|
-
|
|
383
|
-
issues = []
|
|
384
|
-
|
|
385
|
-
# Check capabilities
|
|
386
|
-
for cap in agent_spec.capabilities:
|
|
387
|
-
if emoji_pattern.search(cap['description']):
|
|
388
|
-
issues.append(f"Emoji in capability: {cap['description']}")
|
|
389
|
-
|
|
390
|
-
# Check use cases
|
|
391
|
-
for uc in agent_spec.use_cases:
|
|
392
|
-
if emoji_pattern.search(uc['scenario']):
|
|
393
|
-
issues.append(f"Emoji in use case: {uc['scenario']}")
|
|
394
|
-
|
|
395
|
-
# Check persona (emojis allowed in communication, NOT in code examples)
|
|
396
|
-
# This would require parsing persona for code blocks
|
|
397
|
-
|
|
398
|
-
if issues:
|
|
399
|
-
return ValidationError(
|
|
400
|
-
mantra='IA-23',
|
|
401
|
-
level="CRITICAL",
|
|
402
|
-
message="Emoji pollution detected",
|
|
403
|
-
violations=issues,
|
|
404
|
-
fix="Remove all emojis from technical content"
|
|
405
|
-
)
|
|
406
|
-
|
|
407
|
-
return ValidationSuccess()
|
|
408
|
-
```
|
|
409
|
-
|
|
410
|
-
**Mantra IA-24: Clean Code = No Useless Comments**
|
|
411
|
-
```python
|
|
412
|
-
def validate_mantra_IA_24(agent_spec):
|
|
413
|
-
# Check if agent generates clean code
|
|
414
|
-
if 'IA-24' in agent_spec.mantras_applied:
|
|
415
|
-
capabilities = [c['description'].lower() for c in agent_spec.capabilities]
|
|
416
|
-
code_gen_keywords = ['generate', 'create', 'write', 'scaffold']
|
|
417
|
-
|
|
418
|
-
generates_code = any(kw in cap and 'code' in cap
|
|
419
|
-
for cap in capabilities
|
|
420
|
-
for kw in code_gen_keywords)
|
|
421
|
-
|
|
422
|
-
if generates_code:
|
|
423
|
-
# Check if persona mentions clean code principles
|
|
424
|
-
persona_text = agent_spec.persona.lower()
|
|
425
|
-
if 'self-document' not in persona_text and 'clean code' not in persona_text:
|
|
426
|
-
return ValidationWarning(
|
|
427
|
-
mantra='IA-24',
|
|
428
|
-
message="Agent generates code but doesn't emphasize clean code principles",
|
|
429
|
-
recommendation="Add clean code guidance to persona"
|
|
430
|
-
)
|
|
431
|
-
|
|
432
|
-
return ValidationSuccess()
|
|
433
|
-
```
|
|
434
|
-
|
|
435
|
-
---
|
|
436
|
-
|
|
437
|
-
### Step 5: BMAD Format Compliance
|
|
438
|
-
|
|
439
|
-
```python
|
|
440
|
-
def validate_bmad_format(agent_spec):
|
|
441
|
-
issues = []
|
|
442
|
-
|
|
443
|
-
# Check required sections
|
|
444
|
-
required_sections = [
|
|
445
|
-
'agent',
|
|
446
|
-
'activation',
|
|
447
|
-
'persona',
|
|
448
|
-
'menu'
|
|
449
|
-
]
|
|
450
|
-
|
|
451
|
-
for section in required_sections:
|
|
452
|
-
if not has_section(agent_spec.file_content, section):
|
|
453
|
-
issues.append(f"Missing required section: {section}")
|
|
454
|
-
|
|
455
|
-
# Check XML structure
|
|
456
|
-
if not is_valid_xml_structure(agent_spec.file_content):
|
|
457
|
-
issues.append("Invalid XML structure in agent definition")
|
|
458
|
-
|
|
459
|
-
# Check menu format
|
|
460
|
-
menu = extract_menu(agent_spec.file_content)
|
|
461
|
-
if len(menu.items) < 3:
|
|
462
|
-
issues.append("Menu must have at least 3 items (MH, CH, EXIT minimum)")
|
|
463
|
-
|
|
464
|
-
# Check activation steps
|
|
465
|
-
activation = extract_activation(agent_spec.file_content)
|
|
466
|
-
if not activation.has_config_load:
|
|
467
|
-
issues.append("Activation missing config load step (critical)")
|
|
468
|
-
|
|
469
|
-
if issues:
|
|
470
|
-
return ValidationError(
|
|
471
|
-
category="BMAD Format",
|
|
472
|
-
level="CRITICAL",
|
|
473
|
-
message="BMAD format violations detected",
|
|
474
|
-
violations=issues,
|
|
475
|
-
fix="Review BMAD format standards"
|
|
476
|
-
)
|
|
477
|
-
|
|
478
|
-
return ValidationSuccess()
|
|
479
|
-
```
|
|
480
|
-
|
|
481
|
-
---
|
|
482
|
-
|
|
483
|
-
### Step 6: Generate Validation Report
|
|
484
|
-
|
|
485
|
-
```
|
|
486
|
-
"════════════════════════════════════════════════════
|
|
487
|
-
VALIDATION REPORT: {agent_name}
|
|
488
|
-
════════════════════════════════════════════════════
|
|
489
|
-
|
|
490
|
-
Agent: {agent_name}
|
|
491
|
-
Role: {role}
|
|
492
|
-
Status: {status}
|
|
493
|
-
Validation Date: {timestamp}
|
|
494
|
-
|
|
495
|
-
────────────────────────────────────────────────────
|
|
496
|
-
OVERALL SCORE: {score}/100 [{grade}]
|
|
497
|
-
────────────────────────────────────────────────────
|
|
498
|
-
|
|
499
|
-
Grade Legend:
|
|
500
|
-
A+ (95-100): Exemplary - No issues
|
|
501
|
-
A (90-94): Excellent - Minor suggestions only
|
|
502
|
-
B (80-89): Good - Few warnings
|
|
503
|
-
C (70-79): Acceptable - Multiple warnings
|
|
504
|
-
D (60-69): Needs improvement - Some errors
|
|
505
|
-
F (<60): Failing - Critical errors
|
|
506
|
-
|
|
507
|
-
────────────────────────────────────────────────────
|
|
508
|
-
CRITICAL ISSUES: {critical_count}
|
|
509
|
-
────────────────────────────────────────────────────
|
|
510
|
-
{list_critical_issues_with_fixes}
|
|
511
|
-
|
|
512
|
-
────────────────────────────────────────────────────
|
|
513
|
-
IMPORTANT WARNINGS: {warning_count}
|
|
514
|
-
────────────────────────────────────────────────────
|
|
515
|
-
{list_warnings_with_recommendations}
|
|
516
|
-
|
|
517
|
-
────────────────────────────────────────────────────
|
|
518
|
-
SUGGESTIONS: {suggestion_count}
|
|
519
|
-
────────────────────────────────────────────────────
|
|
520
|
-
{list_suggestions}
|
|
521
|
-
|
|
522
|
-
────────────────────────────────────────────────────
|
|
523
|
-
MANTRA COMPLIANCE: {mantra_score}%
|
|
524
|
-
────────────────────────────────────────────────────
|
|
525
|
-
|
|
526
|
-
Mantras Applied: {mantras_applied_count}
|
|
527
|
-
Mantras Validated: ✅ {pass_count} | ⚠️ {warning_count} | ❌ {fail_count}
|
|
528
|
-
|
|
529
|
-
Failed Mantras:
|
|
530
|
-
{list_failed_mantras_with_details}
|
|
531
|
-
|
|
532
|
-
────────────────────────────────────────────────────
|
|
533
|
-
BUSINESS RULES: {rules_status}
|
|
534
|
-
────────────────────────────────────────────────────
|
|
535
|
-
|
|
536
|
-
✅ RG-AGT-001: Agent name unique and valid
|
|
537
|
-
✅ RG-AGT-002: Capabilities count >= 3
|
|
538
|
-
⚠️ RG-AGT-003: Mantras count = 4 (recommend >= 5)
|
|
539
|
-
✅ RG-AGT-004: Use cases >= 3
|
|
540
|
-
✅ RG-AGT-005: Valid status workflow
|
|
541
|
-
|
|
542
|
-
────────────────────────────────────────────────────
|
|
543
|
-
RECOMMENDATIONS:
|
|
544
|
-
────────────────────────────────────────────────────
|
|
545
|
-
|
|
546
|
-
Priority 1 (Must Fix):
|
|
547
|
-
{must_fix_list}
|
|
548
|
-
|
|
549
|
-
Priority 2 (Should Fix):
|
|
550
|
-
{should_fix_list}
|
|
551
|
-
|
|
552
|
-
Priority 3 (Nice to Have):
|
|
553
|
-
{nice_to_have_list}
|
|
554
|
-
|
|
555
|
-
────────────────────────────────────────────────────
|
|
556
|
-
AUTO-FIX AVAILABLE: {autofix_available}
|
|
557
|
-
────────────────────────────────────────────────────
|
|
558
|
-
|
|
559
|
-
The following issues can be auto-fixed:
|
|
560
|
-
{list_autofix_candidates}
|
|
561
|
-
|
|
562
|
-
Run auto-fix? (yes/no): ____
|
|
563
|
-
|
|
564
|
-
════════════════════════════════════════════════════
|
|
565
|
-
"
|
|
566
|
-
```
|
|
567
|
-
|
|
568
|
-
---
|
|
569
|
-
|
|
570
|
-
### Step 7: Auto-Fix (Optional)
|
|
571
|
-
|
|
572
|
-
```
|
|
573
|
-
"Auto-fixing issues...
|
|
574
|
-
|
|
575
|
-
[▓▓▓▓░░░░░░] 40% - Fixing RG-AGT-003 (adding recommended mantras)...
|
|
576
|
-
|
|
577
|
-
Auto-fixes applied:
|
|
578
|
-
✅ Added 1 mantra to reach minimum (RG-AGT-003)
|
|
579
|
-
✅ Fixed agent name casing (RG-AGT-001)
|
|
580
|
-
✅ Removed emoji from capability description (IA-23)
|
|
581
|
-
✅ Added missing use case (RG-AGT-004)
|
|
582
|
-
|
|
583
|
-
Unable to auto-fix (manual review needed):
|
|
584
|
-
⚠️ Capability overlap detected - requires human decision
|
|
585
|
-
⚠️ Persona missing challenging behavior - requires rewrite
|
|
586
|
-
|
|
587
|
-
Re-running validation...
|
|
588
|
-
"
|
|
589
|
-
```
|
|
590
|
-
|
|
591
|
-
---
|
|
592
|
-
|
|
593
|
-
### Step 8: Export Validation Report
|
|
594
|
-
|
|
595
|
-
```
|
|
596
|
-
"Validation report saved to:
|
|
597
|
-
{output_folder}/validation-{agent_name}-{timestamp}.md
|
|
598
|
-
|
|
599
|
-
Would you like to:
|
|
600
|
-
1. Fix issues now (guided)
|
|
601
|
-
2. Deploy anyway (if no critical issues)
|
|
602
|
-
3. Export report and fix later
|
|
603
|
-
4. Return to menu
|
|
604
|
-
|
|
605
|
-
Choice: ____
|
|
606
|
-
"
|
|
607
|
-
```
|
|
608
|
-
|
|
609
|
-
---
|
|
610
|
-
|
|
611
|
-
## VALIDATION SCORING
|
|
612
|
-
|
|
613
|
-
```python
|
|
614
|
-
def calculate_score(validation_results):
|
|
615
|
-
score = 100
|
|
616
|
-
|
|
617
|
-
# Critical issues: -20 points each
|
|
618
|
-
score -= len(validation_results.critical) * 20
|
|
619
|
-
|
|
620
|
-
# Warnings: -5 points each
|
|
621
|
-
score -= len(validation_results.warnings) * 5
|
|
622
|
-
|
|
623
|
-
# Suggestions: -1 point each
|
|
624
|
-
score -= len(validation_results.suggestions) * 1
|
|
625
|
-
|
|
626
|
-
# Bonus for excellent compliance
|
|
627
|
-
if validation_results.mantra_compliance >= 90:
|
|
628
|
-
score += 5
|
|
629
|
-
|
|
630
|
-
# Cap at 0-100
|
|
631
|
-
return max(0, min(100, score))
|
|
632
|
-
```
|
|
633
|
-
|
|
634
|
-
**Grading:**
|
|
635
|
-
- A+ (95-100): Exemplary
|
|
636
|
-
- A (90-94): Excellent
|
|
637
|
-
- B (80-89): Good
|
|
638
|
-
- C (70-79): Acceptable
|
|
639
|
-
- D (60-69): Needs improvement
|
|
640
|
-
- F (<60): Failing
|
|
641
|
-
|
|
642
|
-
---
|
|
643
|
-
|
|
644
|
-
## SUCCESS CRITERIA
|
|
645
|
-
|
|
646
|
-
✅ All critical issues resolved
|
|
647
|
-
✅ Score >= 70% (Grade C or better)
|
|
648
|
-
✅ All business rules pass
|
|
649
|
-
✅ BMAD format valid
|
|
650
|
-
✅ No emoji pollution
|
|
651
|
-
✅ Validation report generated
|
|
652
|
-
✅ User aware of remaining issues
|
|
653
|
-
|
|
654
|
-
**For deployment:** Score >= 80% recommended (Grade B or better)
|
|
655
|
-
|
|
656
|
-
---
|
|
657
|
-
|
|
658
|
-
## COMPLETION
|
|
659
|
-
|
|
660
|
-
```
|
|
661
|
-
"Validation complete!
|
|
662
|
-
|
|
663
|
-
FINAL SCORE: {score}/100 [{grade}]
|
|
664
|
-
STATUS: {PASS/FAIL}
|
|
665
|
-
|
|
666
|
-
{if PASS:}
|
|
667
|
-
Agent is ready for deployment.
|
|
668
|
-
Generate files now? (yes/later): ____
|
|
669
|
-
|
|
670
|
-
{if FAIL:}
|
|
671
|
-
Agent has critical issues and CANNOT be deployed.
|
|
672
|
-
Fix issues with [EA] Edit Agent workflow.
|
|
673
|
-
|
|
674
|
-
Report saved to: {report_path}
|
|
675
|
-
"
|
|
676
|
-
```
|
|
1
|
+
# BYAN Validate Agent Workflow
|
|
2
|
+
|
|
3
|
+
**Workflow:** Comprehensive Agent Validation Against 64 Mantras
|
|
4
|
+
**Duration:** 5-10 minutes
|
|
5
|
+
**Methodology:** Merise Agile + TDD + Full Mantra Compliance
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## OVERVIEW
|
|
10
|
+
|
|
11
|
+
Validate Agent checks agent specifications against:
|
|
12
|
+
- 64 Mantras (39 Conception + 25 AI Agents)
|
|
13
|
+
- BMAD compliance standards
|
|
14
|
+
- Merise Agile best practices
|
|
15
|
+
- All business rules (RG-*)
|
|
16
|
+
|
|
17
|
+
**When to Use:**
|
|
18
|
+
- After creating new agent (Quick Create or Full Interview)
|
|
19
|
+
- After editing existing agent
|
|
20
|
+
- Before deploying agent
|
|
21
|
+
- Regular quality audits
|
|
22
|
+
- Troubleshooting agent issues
|
|
23
|
+
|
|
24
|
+
**Output:**
|
|
25
|
+
- Validation report with pass/fail/warnings
|
|
26
|
+
- Compliance score (0-100%)
|
|
27
|
+
- Actionable recommendations
|
|
28
|
+
- Auto-fix suggestions where possible
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## VALIDATION LEVELS
|
|
33
|
+
|
|
34
|
+
### Level 1: CRITICAL (Must Pass)
|
|
35
|
+
- Business rules violations (RG-*)
|
|
36
|
+
- BMAD format compliance
|
|
37
|
+
- Structural requirements
|
|
38
|
+
- Security issues
|
|
39
|
+
|
|
40
|
+
**Failure = Agent CANNOT be deployed**
|
|
41
|
+
|
|
42
|
+
### Level 2: IMPORTANT (Should Pass)
|
|
43
|
+
- Mantra compliance
|
|
44
|
+
- Best practices
|
|
45
|
+
- Documentation completeness
|
|
46
|
+
- Test coverage
|
|
47
|
+
|
|
48
|
+
**Failure = Agent CAN deploy but with warnings**
|
|
49
|
+
|
|
50
|
+
### Level 3: SUGGESTIONS (Nice to Have)
|
|
51
|
+
- Optimization opportunities
|
|
52
|
+
- Enhanced capabilities
|
|
53
|
+
- Additional use cases
|
|
54
|
+
- Improved documentation
|
|
55
|
+
|
|
56
|
+
**Failure = Recommendations only**
|
|
57
|
+
|
|
58
|
+
---
|
|
59
|
+
|
|
60
|
+
## WORKFLOW STEPS
|
|
61
|
+
|
|
62
|
+
### Step 1: Select Agent to Validate
|
|
63
|
+
|
|
64
|
+
```
|
|
65
|
+
"Which agent should I validate?
|
|
66
|
+
|
|
67
|
+
Available agents:
|
|
68
|
+
1. {agent_1} - [{status}] - Last validated: {date}
|
|
69
|
+
2. {agent_2} - [{status}] - Last validated: {date}
|
|
70
|
+
3. {agent_3} - [{status}] - Never validated
|
|
71
|
+
...
|
|
72
|
+
|
|
73
|
+
Agent number or name: ____
|
|
74
|
+
"
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
**Load agent:**
|
|
78
|
+
```python
|
|
79
|
+
agent_spec = AgentSpec.load(agent_id_or_name)
|
|
80
|
+
context = ProjectContext.load(agent_spec.context_id)
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
### Step 2: Run Validation Checks
|
|
86
|
+
|
|
87
|
+
**Progress indicator:**
|
|
88
|
+
```
|
|
89
|
+
"Validating agent: {agent_name}
|
|
90
|
+
|
|
91
|
+
Running validation checks:
|
|
92
|
+
[▓▓▓▓▓▓▓▓░░] 80% - Checking mantra compliance...
|
|
93
|
+
|
|
94
|
+
Checks completed:
|
|
95
|
+
✅ Business rules (RG-*)
|
|
96
|
+
✅ BMAD format compliance
|
|
97
|
+
✅ Structural requirements
|
|
98
|
+
⏳ Mantra compliance (39/39)
|
|
99
|
+
⏳ AI Agent mantras (25/25)
|
|
100
|
+
⏳ Best practices
|
|
101
|
+
⏳ Documentation
|
|
102
|
+
"
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
### Step 3: Business Rules Validation (CRITICAL)
|
|
108
|
+
|
|
109
|
+
#### RG-AGT-001: Unique Agent Name
|
|
110
|
+
|
|
111
|
+
```python
|
|
112
|
+
def validate_RG_AGT_001(agent_spec):
|
|
113
|
+
existing = AgentSpec.find_by_name(agent_spec.agent_name)
|
|
114
|
+
if existing and existing.id != agent_spec.id:
|
|
115
|
+
return ValidationError(
|
|
116
|
+
rule="RG-AGT-001",
|
|
117
|
+
level="CRITICAL",
|
|
118
|
+
message=f"Agent name '{agent_spec.agent_name}' already exists",
|
|
119
|
+
fix="Choose a unique name"
|
|
120
|
+
)
|
|
121
|
+
|
|
122
|
+
# Validate kebab-case
|
|
123
|
+
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', agent_spec.agent_name):
|
|
124
|
+
return ValidationError(
|
|
125
|
+
rule="RG-AGT-001",
|
|
126
|
+
level="CRITICAL",
|
|
127
|
+
message=f"Agent name must be kebab-case",
|
|
128
|
+
current=agent_spec.agent_name,
|
|
129
|
+
fix="Convert to kebab-case, e.g., 'backend-expert'"
|
|
130
|
+
)
|
|
131
|
+
|
|
132
|
+
return ValidationSuccess()
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
#### RG-AGT-002: Minimum 3 Capabilities
|
|
136
|
+
|
|
137
|
+
```python
|
|
138
|
+
def validate_RG_AGT_002(agent_spec):
|
|
139
|
+
cap_count = len(agent_spec.capabilities)
|
|
140
|
+
if cap_count < 3:
|
|
141
|
+
return ValidationError(
|
|
142
|
+
rule="RG-AGT-002",
|
|
143
|
+
level="CRITICAL",
|
|
144
|
+
message=f"Agent has only {cap_count} capabilities, minimum is 3",
|
|
145
|
+
fix=f"Add {3 - cap_count} more capabilities"
|
|
146
|
+
)
|
|
147
|
+
return ValidationSuccess()
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
#### RG-AGT-003: Minimum 5 Mantras
|
|
151
|
+
|
|
152
|
+
```python
|
|
153
|
+
def validate_RG_AGT_003(agent_spec):
|
|
154
|
+
mantra_count = len(agent_spec.mantras_applied)
|
|
155
|
+
if mantra_count < 5:
|
|
156
|
+
return ValidationWarning(
|
|
157
|
+
rule="RG-AGT-003",
|
|
158
|
+
level="IMPORTANT",
|
|
159
|
+
message=f"Agent applies only {mantra_count} mantras, recommended minimum is 5",
|
|
160
|
+
fix="Add more relevant mantras",
|
|
161
|
+
suggestions=suggest_mantras(agent_spec)
|
|
162
|
+
)
|
|
163
|
+
return ValidationSuccess()
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
#### RG-AGT-004: Minimum 3 Use Cases
|
|
167
|
+
|
|
168
|
+
```python
|
|
169
|
+
def validate_RG_AGT_004(agent_spec):
|
|
170
|
+
use_case_count = len(agent_spec.use_cases)
|
|
171
|
+
if use_case_count < 3:
|
|
172
|
+
return ValidationWarning(
|
|
173
|
+
rule="RG-AGT-004",
|
|
174
|
+
level="IMPORTANT",
|
|
175
|
+
message=f"Agent has only {use_case_count} use cases, recommended minimum is 3",
|
|
176
|
+
fix="Document more use cases to clarify agent value"
|
|
177
|
+
)
|
|
178
|
+
return ValidationSuccess()
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
#### RG-AGT-005: Valid Status Workflow
|
|
182
|
+
|
|
183
|
+
```python
|
|
184
|
+
def validate_RG_AGT_005(agent_spec):
|
|
185
|
+
valid_statuses = ["draft", "validated", "deployed", "deprecated"]
|
|
186
|
+
if agent_spec.status not in valid_statuses:
|
|
187
|
+
return ValidationError(
|
|
188
|
+
rule="RG-AGT-005",
|
|
189
|
+
level="CRITICAL",
|
|
190
|
+
message=f"Invalid status: {agent_spec.status}",
|
|
191
|
+
valid_values=valid_statuses,
|
|
192
|
+
fix="Set to valid status"
|
|
193
|
+
)
|
|
194
|
+
return ValidationSuccess()
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
---
|
|
198
|
+
|
|
199
|
+
### Step 4: Mantra Compliance Validation
|
|
200
|
+
|
|
201
|
+
**For each of 64 mantras, check if applied correctly:**
|
|
202
|
+
|
|
203
|
+
#### Conception Mantras (39)
|
|
204
|
+
|
|
205
|
+
**Mantra #1: Le Modèle Sert le Métier**
|
|
206
|
+
```python
|
|
207
|
+
def validate_mantra_01(agent_spec, context):
|
|
208
|
+
# Check if agent knowledge includes business concepts
|
|
209
|
+
business_concepts = context.glossaire.keys()
|
|
210
|
+
agent_business = agent_spec.knowledge_business
|
|
211
|
+
|
|
212
|
+
overlap = set(business_concepts) & set(agent_business)
|
|
213
|
+
coverage = len(overlap) / len(business_concepts) if business_concepts else 0
|
|
214
|
+
|
|
215
|
+
if coverage < 0.3: # Less than 30% coverage
|
|
216
|
+
return ValidationWarning(
|
|
217
|
+
mantra=1,
|
|
218
|
+
message="Agent has weak connection to business domain",
|
|
219
|
+
coverage=f"{coverage*100:.0f}%",
|
|
220
|
+
recommendation="Add more business concepts to knowledge_business"
|
|
221
|
+
)
|
|
222
|
+
return ValidationSuccess()
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
**Mantra #4: Fail Fast, Fail Visible**
|
|
226
|
+
```python
|
|
227
|
+
def validate_mantra_04(agent_spec):
|
|
228
|
+
# Check if agent has error handling capabilities
|
|
229
|
+
capabilities = [c['description'].lower() for c in agent_spec.capabilities]
|
|
230
|
+
|
|
231
|
+
error_keywords = ['error', 'fail', 'exception', 'validate', 'check']
|
|
232
|
+
has_error_handling = any(kw in cap for cap in capabilities for kw in error_keywords)
|
|
233
|
+
|
|
234
|
+
if not has_error_handling and 4 in agent_spec.mantras_applied:
|
|
235
|
+
return ValidationWarning(
|
|
236
|
+
mantra=4,
|
|
237
|
+
message="Agent claims Mantra #4 but has no error handling capabilities",
|
|
238
|
+
recommendation="Add capability for error detection/handling"
|
|
239
|
+
)
|
|
240
|
+
return ValidationSuccess()
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
**Mantra #7: KISS (Keep It Simple)**
|
|
244
|
+
```python
|
|
245
|
+
def validate_mantra_07(agent_spec):
|
|
246
|
+
# Check if agent is over-complicated
|
|
247
|
+
cap_count = len(agent_spec.capabilities)
|
|
248
|
+
|
|
249
|
+
if cap_count > 10:
|
|
250
|
+
return ValidationWarning(
|
|
251
|
+
mantra=7,
|
|
252
|
+
message=f"Agent has {cap_count} capabilities - may violate KISS principle",
|
|
253
|
+
recommendation="Consider splitting into multiple focused agents"
|
|
254
|
+
)
|
|
255
|
+
|
|
256
|
+
# Check role clarity
|
|
257
|
+
role_words = len(agent_spec.role.split())
|
|
258
|
+
if role_words > 20:
|
|
259
|
+
return ValidationWarning(
|
|
260
|
+
mantra=7,
|
|
261
|
+
message="Agent role description is too complex",
|
|
262
|
+
recommendation="Simplify role to one clear sentence"
|
|
263
|
+
)
|
|
264
|
+
|
|
265
|
+
return ValidationSuccess()
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
**Mantra #18: TDD is Not Optional**
|
|
269
|
+
```python
|
|
270
|
+
def validate_mantra_18(agent_spec):
|
|
271
|
+
if 18 in agent_spec.mantras_applied:
|
|
272
|
+
# Check if agent has test-related capabilities
|
|
273
|
+
capabilities = [c['description'].lower() for c in agent_spec.capabilities]
|
|
274
|
+
test_keywords = ['test', 'tdd', 'coverage', 'assertion']
|
|
275
|
+
|
|
276
|
+
has_test_cap = any(kw in cap for cap in capabilities for kw in test_keywords)
|
|
277
|
+
|
|
278
|
+
if not has_test_cap:
|
|
279
|
+
return ValidationWarning(
|
|
280
|
+
mantra=18,
|
|
281
|
+
message="Agent claims TDD mantra but has no testing capabilities",
|
|
282
|
+
recommendation="Add test generation or verification capability"
|
|
283
|
+
)
|
|
284
|
+
return ValidationSuccess()
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
**Mantra #37: Rasoir d'Ockham**
|
|
288
|
+
```python
|
|
289
|
+
def validate_mantra_37(agent_spec):
|
|
290
|
+
# Always check for simplicity
|
|
291
|
+
issues = []
|
|
292
|
+
|
|
293
|
+
# Check capabilities count
|
|
294
|
+
if len(agent_spec.capabilities) > 7:
|
|
295
|
+
issues.append("Too many capabilities (>7) - prefer focused agent")
|
|
296
|
+
|
|
297
|
+
# Check for overlapping capabilities
|
|
298
|
+
caps = [c['description'] for c in agent_spec.capabilities]
|
|
299
|
+
similar_pairs = find_similar_capabilities(caps)
|
|
300
|
+
if similar_pairs:
|
|
301
|
+
issues.append(f"Overlapping capabilities detected: {similar_pairs}")
|
|
302
|
+
|
|
303
|
+
# Check for unnecessary complexity
|
|
304
|
+
if len(agent_spec.mantras_applied) > 15:
|
|
305
|
+
issues.append("Too many mantras (>15) - focus on most critical ones")
|
|
306
|
+
|
|
307
|
+
if issues:
|
|
308
|
+
return ValidationWarning(
|
|
309
|
+
mantra=37,
|
|
310
|
+
message="Agent violates Ockham's Razor (simplicity principle)",
|
|
311
|
+
issues=issues,
|
|
312
|
+
recommendation="Simplify agent by removing non-essential elements"
|
|
313
|
+
)
|
|
314
|
+
|
|
315
|
+
return ValidationSuccess()
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
**Mantra #39: Évaluation des Conséquences**
|
|
319
|
+
```python
|
|
320
|
+
def validate_mantra_39(agent_spec):
|
|
321
|
+
# Check if agent is deployed without consequence evaluation
|
|
322
|
+
if agent_spec.status == "deployed":
|
|
323
|
+
if not agent_spec.consequences_evaluated:
|
|
324
|
+
return ValidationError(
|
|
325
|
+
mantra=39,
|
|
326
|
+
level="CRITICAL",
|
|
327
|
+
message="Agent deployed without consequences evaluation",
|
|
328
|
+
fix="Run Edit Agent workflow to evaluate consequences"
|
|
329
|
+
)
|
|
330
|
+
return ValidationSuccess()
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
#### AI Agent Mantras (25)
|
|
334
|
+
|
|
335
|
+
**Mantra IA-1: Trust But Verify**
|
|
336
|
+
```python
|
|
337
|
+
def validate_mantra_IA_01(agent_spec):
|
|
338
|
+
if 'IA-1' in agent_spec.mantras_applied:
|
|
339
|
+
# Check if agent has validation capabilities
|
|
340
|
+
capabilities = [c['description'].lower() for c in agent_spec.capabilities]
|
|
341
|
+
validation_keywords = ['verify', 'validate', 'check', 'review', 'audit']
|
|
342
|
+
|
|
343
|
+
has_validation = any(kw in cap for cap in capabilities for kw in validation_keywords)
|
|
344
|
+
|
|
345
|
+
if not has_validation:
|
|
346
|
+
return ValidationWarning(
|
|
347
|
+
mantra='IA-1',
|
|
348
|
+
message="Agent claims Trust But Verify but lacks validation capabilities",
|
|
349
|
+
recommendation="Add verification/validation capability"
|
|
350
|
+
)
|
|
351
|
+
return ValidationSuccess()
|
|
352
|
+
```
|
|
353
|
+
|
|
354
|
+
**Mantra IA-16: Challenge Before Confirm**
|
|
355
|
+
```python
|
|
356
|
+
def validate_mantra_IA_16(agent_spec):
|
|
357
|
+
if 'IA-16' in agent_spec.mantras_applied:
|
|
358
|
+
# Check persona for challenging behavior
|
|
359
|
+
persona_text = agent_spec.persona.lower()
|
|
360
|
+
challenge_keywords = ['challenge', 'question', 'verify', 'validate', 'devil\'s advocate']
|
|
361
|
+
|
|
362
|
+
has_challenge_behavior = any(kw in persona_text for kw in challenge_keywords)
|
|
363
|
+
|
|
364
|
+
if not has_challenge_behavior:
|
|
365
|
+
return ValidationWarning(
|
|
366
|
+
mantra='IA-16',
|
|
367
|
+
message="Agent claims Challenge Before Confirm but persona doesn't reflect challenging behavior",
|
|
368
|
+
recommendation="Update persona to include questioning/challenging approach"
|
|
369
|
+
)
|
|
370
|
+
return ValidationSuccess()
|
|
371
|
+
```
|
|
372
|
+
|
|
373
|
+
**Mantra IA-23: No Emoji Pollution**
|
|
374
|
+
```python
|
|
375
|
+
def validate_mantra_IA_23(agent_spec):
|
|
376
|
+
# Check for emojis in technical content
|
|
377
|
+
emoji_pattern = re.compile("["
|
|
378
|
+
u"\U0001F600-\U0001F64F" # emoticons
|
|
379
|
+
u"\U0001F300-\U0001F5FF" # symbols & pictographs
|
|
380
|
+
u"\U0001F680-\U0001F6FF" # transport & map symbols
|
|
381
|
+
"]+", flags=re.UNICODE)
|
|
382
|
+
|
|
383
|
+
issues = []
|
|
384
|
+
|
|
385
|
+
# Check capabilities
|
|
386
|
+
for cap in agent_spec.capabilities:
|
|
387
|
+
if emoji_pattern.search(cap['description']):
|
|
388
|
+
issues.append(f"Emoji in capability: {cap['description']}")
|
|
389
|
+
|
|
390
|
+
# Check use cases
|
|
391
|
+
for uc in agent_spec.use_cases:
|
|
392
|
+
if emoji_pattern.search(uc['scenario']):
|
|
393
|
+
issues.append(f"Emoji in use case: {uc['scenario']}")
|
|
394
|
+
|
|
395
|
+
# Check persona (emojis allowed in communication, NOT in code examples)
|
|
396
|
+
# This would require parsing persona for code blocks
|
|
397
|
+
|
|
398
|
+
if issues:
|
|
399
|
+
return ValidationError(
|
|
400
|
+
mantra='IA-23',
|
|
401
|
+
level="CRITICAL",
|
|
402
|
+
message="Emoji pollution detected",
|
|
403
|
+
violations=issues,
|
|
404
|
+
fix="Remove all emojis from technical content"
|
|
405
|
+
)
|
|
406
|
+
|
|
407
|
+
return ValidationSuccess()
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
**Mantra IA-24: Clean Code = No Useless Comments**
|
|
411
|
+
```python
|
|
412
|
+
def validate_mantra_IA_24(agent_spec):
|
|
413
|
+
# Check if agent generates clean code
|
|
414
|
+
if 'IA-24' in agent_spec.mantras_applied:
|
|
415
|
+
capabilities = [c['description'].lower() for c in agent_spec.capabilities]
|
|
416
|
+
code_gen_keywords = ['generate', 'create', 'write', 'scaffold']
|
|
417
|
+
|
|
418
|
+
generates_code = any(kw in cap and 'code' in cap
|
|
419
|
+
for cap in capabilities
|
|
420
|
+
for kw in code_gen_keywords)
|
|
421
|
+
|
|
422
|
+
if generates_code:
|
|
423
|
+
# Check if persona mentions clean code principles
|
|
424
|
+
persona_text = agent_spec.persona.lower()
|
|
425
|
+
if 'self-document' not in persona_text and 'clean code' not in persona_text:
|
|
426
|
+
return ValidationWarning(
|
|
427
|
+
mantra='IA-24',
|
|
428
|
+
message="Agent generates code but doesn't emphasize clean code principles",
|
|
429
|
+
recommendation="Add clean code guidance to persona"
|
|
430
|
+
)
|
|
431
|
+
|
|
432
|
+
return ValidationSuccess()
|
|
433
|
+
```
|
|
434
|
+
|
|
435
|
+
---
|
|
436
|
+
|
|
437
|
+
### Step 5: BMAD Format Compliance
|
|
438
|
+
|
|
439
|
+
```python
|
|
440
|
+
def validate_bmad_format(agent_spec):
|
|
441
|
+
issues = []
|
|
442
|
+
|
|
443
|
+
# Check required sections
|
|
444
|
+
required_sections = [
|
|
445
|
+
'agent',
|
|
446
|
+
'activation',
|
|
447
|
+
'persona',
|
|
448
|
+
'menu'
|
|
449
|
+
]
|
|
450
|
+
|
|
451
|
+
for section in required_sections:
|
|
452
|
+
if not has_section(agent_spec.file_content, section):
|
|
453
|
+
issues.append(f"Missing required section: {section}")
|
|
454
|
+
|
|
455
|
+
# Check XML structure
|
|
456
|
+
if not is_valid_xml_structure(agent_spec.file_content):
|
|
457
|
+
issues.append("Invalid XML structure in agent definition")
|
|
458
|
+
|
|
459
|
+
# Check menu format
|
|
460
|
+
menu = extract_menu(agent_spec.file_content)
|
|
461
|
+
if len(menu.items) < 3:
|
|
462
|
+
issues.append("Menu must have at least 3 items (MH, CH, EXIT minimum)")
|
|
463
|
+
|
|
464
|
+
# Check activation steps
|
|
465
|
+
activation = extract_activation(agent_spec.file_content)
|
|
466
|
+
if not activation.has_config_load:
|
|
467
|
+
issues.append("Activation missing config load step (critical)")
|
|
468
|
+
|
|
469
|
+
if issues:
|
|
470
|
+
return ValidationError(
|
|
471
|
+
category="BMAD Format",
|
|
472
|
+
level="CRITICAL",
|
|
473
|
+
message="BMAD format violations detected",
|
|
474
|
+
violations=issues,
|
|
475
|
+
fix="Review BMAD format standards"
|
|
476
|
+
)
|
|
477
|
+
|
|
478
|
+
return ValidationSuccess()
|
|
479
|
+
```
|
|
480
|
+
|
|
481
|
+
---
|
|
482
|
+
|
|
483
|
+
### Step 6: Generate Validation Report
|
|
484
|
+
|
|
485
|
+
```
|
|
486
|
+
"════════════════════════════════════════════════════
|
|
487
|
+
VALIDATION REPORT: {agent_name}
|
|
488
|
+
════════════════════════════════════════════════════
|
|
489
|
+
|
|
490
|
+
Agent: {agent_name}
|
|
491
|
+
Role: {role}
|
|
492
|
+
Status: {status}
|
|
493
|
+
Validation Date: {timestamp}
|
|
494
|
+
|
|
495
|
+
────────────────────────────────────────────────────
|
|
496
|
+
OVERALL SCORE: {score}/100 [{grade}]
|
|
497
|
+
────────────────────────────────────────────────────
|
|
498
|
+
|
|
499
|
+
Grade Legend:
|
|
500
|
+
A+ (95-100): Exemplary - No issues
|
|
501
|
+
A (90-94): Excellent - Minor suggestions only
|
|
502
|
+
B (80-89): Good - Few warnings
|
|
503
|
+
C (70-79): Acceptable - Multiple warnings
|
|
504
|
+
D (60-69): Needs improvement - Some errors
|
|
505
|
+
F (<60): Failing - Critical errors
|
|
506
|
+
|
|
507
|
+
────────────────────────────────────────────────────
|
|
508
|
+
CRITICAL ISSUES: {critical_count}
|
|
509
|
+
────────────────────────────────────────────────────
|
|
510
|
+
{list_critical_issues_with_fixes}
|
|
511
|
+
|
|
512
|
+
────────────────────────────────────────────────────
|
|
513
|
+
IMPORTANT WARNINGS: {warning_count}
|
|
514
|
+
────────────────────────────────────────────────────
|
|
515
|
+
{list_warnings_with_recommendations}
|
|
516
|
+
|
|
517
|
+
────────────────────────────────────────────────────
|
|
518
|
+
SUGGESTIONS: {suggestion_count}
|
|
519
|
+
────────────────────────────────────────────────────
|
|
520
|
+
{list_suggestions}
|
|
521
|
+
|
|
522
|
+
────────────────────────────────────────────────────
|
|
523
|
+
MANTRA COMPLIANCE: {mantra_score}%
|
|
524
|
+
────────────────────────────────────────────────────
|
|
525
|
+
|
|
526
|
+
Mantras Applied: {mantras_applied_count}
|
|
527
|
+
Mantras Validated: ✅ {pass_count} | ⚠️ {warning_count} | ❌ {fail_count}
|
|
528
|
+
|
|
529
|
+
Failed Mantras:
|
|
530
|
+
{list_failed_mantras_with_details}
|
|
531
|
+
|
|
532
|
+
────────────────────────────────────────────────────
|
|
533
|
+
BUSINESS RULES: {rules_status}
|
|
534
|
+
────────────────────────────────────────────────────
|
|
535
|
+
|
|
536
|
+
✅ RG-AGT-001: Agent name unique and valid
|
|
537
|
+
✅ RG-AGT-002: Capabilities count >= 3
|
|
538
|
+
⚠️ RG-AGT-003: Mantras count = 4 (recommend >= 5)
|
|
539
|
+
✅ RG-AGT-004: Use cases >= 3
|
|
540
|
+
✅ RG-AGT-005: Valid status workflow
|
|
541
|
+
|
|
542
|
+
────────────────────────────────────────────────────
|
|
543
|
+
RECOMMENDATIONS:
|
|
544
|
+
────────────────────────────────────────────────────
|
|
545
|
+
|
|
546
|
+
Priority 1 (Must Fix):
|
|
547
|
+
{must_fix_list}
|
|
548
|
+
|
|
549
|
+
Priority 2 (Should Fix):
|
|
550
|
+
{should_fix_list}
|
|
551
|
+
|
|
552
|
+
Priority 3 (Nice to Have):
|
|
553
|
+
{nice_to_have_list}
|
|
554
|
+
|
|
555
|
+
────────────────────────────────────────────────────
|
|
556
|
+
AUTO-FIX AVAILABLE: {autofix_available}
|
|
557
|
+
────────────────────────────────────────────────────
|
|
558
|
+
|
|
559
|
+
The following issues can be auto-fixed:
|
|
560
|
+
{list_autofix_candidates}
|
|
561
|
+
|
|
562
|
+
Run auto-fix? (yes/no): ____
|
|
563
|
+
|
|
564
|
+
════════════════════════════════════════════════════
|
|
565
|
+
"
|
|
566
|
+
```
|
|
567
|
+
|
|
568
|
+
---
|
|
569
|
+
|
|
570
|
+
### Step 7: Auto-Fix (Optional)
|
|
571
|
+
|
|
572
|
+
```
|
|
573
|
+
"Auto-fixing issues...
|
|
574
|
+
|
|
575
|
+
[▓▓▓▓░░░░░░] 40% - Fixing RG-AGT-003 (adding recommended mantras)...
|
|
576
|
+
|
|
577
|
+
Auto-fixes applied:
|
|
578
|
+
✅ Added 1 mantra to reach minimum (RG-AGT-003)
|
|
579
|
+
✅ Fixed agent name casing (RG-AGT-001)
|
|
580
|
+
✅ Removed emoji from capability description (IA-23)
|
|
581
|
+
✅ Added missing use case (RG-AGT-004)
|
|
582
|
+
|
|
583
|
+
Unable to auto-fix (manual review needed):
|
|
584
|
+
⚠️ Capability overlap detected - requires human decision
|
|
585
|
+
⚠️ Persona missing challenging behavior - requires rewrite
|
|
586
|
+
|
|
587
|
+
Re-running validation...
|
|
588
|
+
"
|
|
589
|
+
```
|
|
590
|
+
|
|
591
|
+
---
|
|
592
|
+
|
|
593
|
+
### Step 8: Export Validation Report
|
|
594
|
+
|
|
595
|
+
```
|
|
596
|
+
"Validation report saved to:
|
|
597
|
+
{output_folder}/validation-{agent_name}-{timestamp}.md
|
|
598
|
+
|
|
599
|
+
Would you like to:
|
|
600
|
+
1. Fix issues now (guided)
|
|
601
|
+
2. Deploy anyway (if no critical issues)
|
|
602
|
+
3. Export report and fix later
|
|
603
|
+
4. Return to menu
|
|
604
|
+
|
|
605
|
+
Choice: ____
|
|
606
|
+
"
|
|
607
|
+
```
|
|
608
|
+
|
|
609
|
+
---
|
|
610
|
+
|
|
611
|
+
## VALIDATION SCORING
|
|
612
|
+
|
|
613
|
+
```python
|
|
614
|
+
def calculate_score(validation_results):
|
|
615
|
+
score = 100
|
|
616
|
+
|
|
617
|
+
# Critical issues: -20 points each
|
|
618
|
+
score -= len(validation_results.critical) * 20
|
|
619
|
+
|
|
620
|
+
# Warnings: -5 points each
|
|
621
|
+
score -= len(validation_results.warnings) * 5
|
|
622
|
+
|
|
623
|
+
# Suggestions: -1 point each
|
|
624
|
+
score -= len(validation_results.suggestions) * 1
|
|
625
|
+
|
|
626
|
+
# Bonus for excellent compliance
|
|
627
|
+
if validation_results.mantra_compliance >= 90:
|
|
628
|
+
score += 5
|
|
629
|
+
|
|
630
|
+
# Cap at 0-100
|
|
631
|
+
return max(0, min(100, score))
|
|
632
|
+
```
|
|
633
|
+
|
|
634
|
+
**Grading:**
|
|
635
|
+
- A+ (95-100): Exemplary
|
|
636
|
+
- A (90-94): Excellent
|
|
637
|
+
- B (80-89): Good
|
|
638
|
+
- C (70-79): Acceptable
|
|
639
|
+
- D (60-69): Needs improvement
|
|
640
|
+
- F (<60): Failing
|
|
641
|
+
|
|
642
|
+
---
|
|
643
|
+
|
|
644
|
+
## SUCCESS CRITERIA
|
|
645
|
+
|
|
646
|
+
✅ All critical issues resolved
|
|
647
|
+
✅ Score >= 70% (Grade C or better)
|
|
648
|
+
✅ All business rules pass
|
|
649
|
+
✅ BMAD format valid
|
|
650
|
+
✅ No emoji pollution
|
|
651
|
+
✅ Validation report generated
|
|
652
|
+
✅ User aware of remaining issues
|
|
653
|
+
|
|
654
|
+
**For deployment:** Score >= 80% recommended (Grade B or better)
|
|
655
|
+
|
|
656
|
+
---
|
|
657
|
+
|
|
658
|
+
## COMPLETION
|
|
659
|
+
|
|
660
|
+
```
|
|
661
|
+
"Validation complete!
|
|
662
|
+
|
|
663
|
+
FINAL SCORE: {score}/100 [{grade}]
|
|
664
|
+
STATUS: {PASS/FAIL}
|
|
665
|
+
|
|
666
|
+
{if PASS:}
|
|
667
|
+
Agent is ready for deployment.
|
|
668
|
+
Generate files now? (yes/later): ____
|
|
669
|
+
|
|
670
|
+
{if FAIL:}
|
|
671
|
+
Agent has critical issues and CANNOT be deployed.
|
|
672
|
+
Fix issues with [EA] Edit Agent workflow.
|
|
673
|
+
|
|
674
|
+
Report saved to: {report_path}
|
|
675
|
+
"
|
|
676
|
+
```
|