ma-agents 2.19.2 → 2.20.0

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-nfr/steps-c/step-02-define-thresholds.md

@@ -0,0 +1,107 @@
+---
+name: 'step-02-define-thresholds'
+description: 'Identify NFR categories and thresholds'
+nextStepFile: './step-03-gather-evidence.md'
+outputFile: '{test_artifacts}/nfr-assessment.md'
+---
+
+# Step 2: Define NFR Categories & Thresholds
+
+## STEP GOAL
+
+Establish the NFR categories to assess and the thresholds used for validation.
+
+## MANDATORY EXECUTION RULES
+
+- 📖 Read the entire step file before acting
+- ✅ Speak in `{communication_language}`
+- 🚫 Never guess thresholds
+
+---
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow the MANDATORY SEQUENCE exactly
+- 💾 Record outputs before proceeding
+- 📖 Load the next step only when instructed
+
+## CONTEXT BOUNDARIES:
+
+- Available context: config, loaded artifacts, and knowledge fragments
+- Focus: this step's goal only
+- Limits: do not execute future steps
+- Dependencies: prior steps' outputs (if any)
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
+
+## 1. Select Categories
+
+Use the ADR Quality Readiness Checklist (8 categories):
+
+1. Testability & Automation
+2. Test Data Strategy
+3. Scalability & Availability
+4. Disaster Recovery
+5. Security
+6. Monitorability/Debuggability/Manageability
+7. QoS/QoE
+8. Deployability
+
+Add any `custom_nfr_categories` if provided.
+
+---
+
+## 2. Define Thresholds
+
+For each category, extract thresholds from:
+
+- tech-spec (primary)
+- PRD (secondary)
+- story or test-design (feature-specific)
+
+If a threshold is unknown, mark it **UNKNOWN** and plan to report **CONCERNS**.
+
+---
+
+## 3. Confirm NFR Matrix
+
+List each NFR category with its threshold or UNKNOWN status.
+
+---
+
+## 4. Save Progress
+
+**Save this step's accumulated work to `{outputFile}`.**
+
+- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
+
+  ```yaml
+  ---
+  stepsCompleted: ['step-02-define-thresholds']
+  lastStep: 'step-02-define-thresholds'
+  lastSaved: '{date}'
+  ---
+  ```
+
+  Then write this step's output below the frontmatter.
+
+- **If `{outputFile}` already exists**, update:
+  - Add `'step-02-define-thresholds'` to `stepsCompleted` array (only if not already present)
+  - Set `lastStep: 'step-02-define-thresholds'`
+  - Set `lastSaved: '{date}'`
+  - Append this step's output to the appropriate section of the document.
+
+Load next step: `{nextStepFile}`
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
+
+### ✅ SUCCESS:
+
+- Step completed in full with required outputs
+
+### ❌ SYSTEM FAILURE:
+
+- Skipped sequence steps or missing outputs
+  **Master Rule:** Skipping steps is FORBIDDEN.

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-nfr/steps-c/step-03-gather-evidence.md

@@ -0,0 +1,108 @@
+---
+name: 'step-03-gather-evidence'
+description: 'Collect evidence for each NFR category'
+nextStepFile: './step-04-evaluate-and-score.md'
+outputFile: '{test_artifacts}/nfr-assessment.md'
+---
+
+# Step 3: Gather Evidence
+
+## STEP GOAL
+
+Collect measurable evidence to evaluate each NFR category.
+
+## MANDATORY EXECUTION RULES
+
+- 📖 Read the entire step file before acting
+- ✅ Speak in `{communication_language}`
+
+---
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow the MANDATORY SEQUENCE exactly
+- 💾 Record outputs before proceeding
+- 📖 Load the next step only when instructed
+
+## CONTEXT BOUNDARIES:
+
+- Available context: config, loaded artifacts, and knowledge fragments
+- Focus: this step's goal only
+- Limits: do not execute future steps
+- Dependencies: prior steps' outputs (if any)
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
+
+## 1. Evidence Sources
+
+Collect evidence for:
+
+- **Performance**: load tests, metrics, response time data
+- **Security**: scans, auth tests, vuln reports
+- **Reliability**: error rates, burn-in runs, failover tests
+- **Maintainability**: test quality, code health signals
+- **Other categories**: logs, monitoring, DR drills, deployability checks
+
+---
+
+## 2. Browser-Based Evidence Collection (if `tea_browser_automation` is `cli` or `auto`)
+
+> **Fallback:** If CLI is not installed, fall back to MCP (if available) or skip browser-based evidence collection.
+
+For performance and security categories, CLI can gather live evidence:
+
+**Performance evidence (page load, response times):**
+
+1. `playwright-cli -s=tea-nfr open <target_url>`
+2. `playwright-cli -s=tea-nfr network` → capture response times and payload sizes
+3. `playwright-cli -s=tea-nfr screenshot --filename={test_artifacts}/nfr/perf-<page>.png`
+4. `playwright-cli -s=tea-nfr close`
+
+> **Session Hygiene:** Always close sessions using `playwright-cli -s=tea-nfr close`. Do NOT use `close-all` — it kills every session on the machine and breaks parallel execution.
+
+Store artifacts under `{test_artifacts}/nfr/`
+
+---
+
+## 3. Evidence Gaps
+
+If evidence is missing for a category, mark that category as **CONCERNS**.
+
+---
+
+## 4. Save Progress
+
+**Save this step's accumulated work to `{outputFile}`.**
+
+- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
+
+  ```yaml
+  ---
+  stepsCompleted: ['step-03-gather-evidence']
+  lastStep: 'step-03-gather-evidence'
+  lastSaved: '{date}'
+  ---
+  ```
+
+  Then write this step's output below the frontmatter.
+
+- **If `{outputFile}` already exists**, update:
+  - Add `'step-03-gather-evidence'` to `stepsCompleted` array (only if not already present)
+  - Set `lastStep: 'step-03-gather-evidence'`
+  - Set `lastSaved: '{date}'`
+  - Append this step's output to the appropriate section of the document.
+
+Load next step: `{nextStepFile}`
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
+
+### ✅ SUCCESS:
+
+- Step completed in full with required outputs
+
+### ❌ SYSTEM FAILURE:
+
+- Skipped sequence steps or missing outputs
+  **Master Rule:** Skipping steps is FORBIDDEN.

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-nfr/steps-c/step-04-evaluate-and-score.md

@@ -0,0 +1,254 @@
+---
+name: 'step-04-evaluate-and-score'
+description: 'Orchestrate adaptive NFR domain assessments (agent-team, subagent, or sequential)'
+nextStepFile: './step-04e-aggregate-nfr.md'
+---
+
+# Step 4: Orchestrate Adaptive NFR Assessment
+
+## STEP GOAL
+
+Select execution mode deterministically, then assess NFR domains using agent-team, subagent, or sequential execution while preserving output contracts.
+
+## MANDATORY EXECUTION RULES
+
+- 📖 Read the entire step file before acting
+- ✅ Speak in `{communication_language}`
+- ✅ Resolve execution mode from config (`tea_execution_mode`, `tea_capability_probe`)
+- ✅ Apply fallback rules deterministically when requested mode is unsupported
+- ✅ Wait for required worker steps to complete
+- ❌ Do NOT skip capability checks when probing is enabled
+
+---
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow the MANDATORY SEQUENCE exactly
+- 💾 Wait for subagent outputs
+- 📖 Load the next step only when instructed
+
+---
+
+## MANDATORY SEQUENCE
+
+### 1. Prepare Execution Context
+
+**Generate unique timestamp:**
+
+```javascript
+const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
+```
+
+**Prepare context:**
+
+```javascript
+const parseBooleanFlag = (value, defaultValue = true) => {
+  if (typeof value === 'string') {
+    const normalized = value.trim().toLowerCase();
+    if (['false', '0', 'off', 'no'].includes(normalized)) return false;
+    if (['true', '1', 'on', 'yes'].includes(normalized)) return true;
+  }
+  if (value === undefined || value === null) return defaultValue;
+  return Boolean(value);
+};
+
+const subagentContext = {
+  system_context: /* from Step 1 */,
+  nfr_thresholds: /* from Step 2 */,
+  evidence_gathered: /* from Step 3 */,
+  config: {
+    execution_mode: config.tea_execution_mode || 'auto',  // "auto" | "subagent" | "agent-team" | "sequential"
+    capability_probe: parseBooleanFlag(config.tea_capability_probe, true),  // supports booleans and "false"/"true" strings
+  },
+  timestamp: timestamp
+};
+```
+
+---
+
+### 2. Resolve Execution Mode with Capability Probe
+
+```javascript
+const normalizeUserExecutionMode = (mode) => {
+  if (typeof mode !== 'string') return null;
+  const normalized = mode.trim().toLowerCase().replace(/[-_]/g, ' ').replace(/\s+/g, ' ');
+
+  if (normalized === 'auto') return 'auto';
+  if (normalized === 'sequential') return 'sequential';
+  if (normalized === 'subagent' || normalized === 'sub agent' || normalized === 'subagents' || normalized === 'sub agents') {
+    return 'subagent';
+  }
+  if (normalized === 'agent team' || normalized === 'agent teams' || normalized === 'agentteam') {
+    return 'agent-team';
+  }
+
+  return null;
+};
+
+const normalizeConfigExecutionMode = (mode) => {
+  if (mode === 'subagent') return 'subagent';
+  if (mode === 'auto' || mode === 'sequential' || mode === 'subagent' || mode === 'agent-team') {
+    return mode;
+  }
+  return null;
+};
+
+// Explicit user instruction in the active run takes priority over config.
+const explicitModeFromUser = normalizeUserExecutionMode(runtime.getExplicitExecutionModeHint?.() || null);
+
+const requestedMode = explicitModeFromUser || normalizeConfigExecutionMode(subagentContext.config.execution_mode) || 'auto';
+const probeEnabled = subagentContext.config.capability_probe;
+
+const supports = {
+  subagent: false,
+  agentTeam: false,
+};
+
+if (probeEnabled) {
+  supports.subagent = runtime.canLaunchSubagents?.() === true;
+  supports.agentTeam = runtime.canLaunchAgentTeams?.() === true;
+}
+
+let resolvedMode = requestedMode;
+
+if (requestedMode === 'auto') {
+  if (supports.agentTeam) resolvedMode = 'agent-team';
+  else if (supports.subagent) resolvedMode = 'subagent';
+  else resolvedMode = 'sequential';
+} else if (probeEnabled && requestedMode === 'agent-team' && !supports.agentTeam) {
+  resolvedMode = supports.subagent ? 'subagent' : 'sequential';
+} else if (probeEnabled && requestedMode === 'subagent' && !supports.subagent) {
+  resolvedMode = 'sequential';
+}
+
+subagentContext.execution = {
+  requestedMode,
+  resolvedMode,
+  probeEnabled,
+  supports,
+};
+```
+
+Resolution precedence:
+
+1. Explicit user request in this run (`agent team` => `agent-team`; `subagent` => `subagent`; `sequential`; `auto`)
+2. `tea_execution_mode` from config
+3. Runtime capability fallback (when probing enabled)
+
+If probing is disabled, honor the requested mode strictly. If that mode cannot be executed at runtime, fail with explicit error instead of silent fallback.
+
+---
+
+### 3. Dispatch 4 NFR Workers
+
+**Subagent A: Security Assessment**
+
+- File: `./step-04a-subagent-security.md`
+- Output: `/tmp/tea-nfr-security-${timestamp}.json`
+- Execution:
+  - `agent-team` or `subagent`: launch non-blocking
+  - `sequential`: run blocking and wait
+- Status: Running... ⟳
+
+**Subagent B: Performance Assessment**
+
+- File: `./step-04b-subagent-performance.md`
+- Output: `/tmp/tea-nfr-performance-${timestamp}.json`
+- Status: Running... ⟳
+
+**Subagent C: Reliability Assessment**
+
+- File: `./step-04c-subagent-reliability.md`
+- Output: `/tmp/tea-nfr-reliability-${timestamp}.json`
+- Status: Running... ⟳
+
+**Subagent D: Scalability Assessment**
+
+- File: `./step-04d-subagent-scalability.md`
+- Output: `/tmp/tea-nfr-scalability-${timestamp}.json`
+- Status: Running... ⟳
+
+In `agent-team` and `subagent` modes, runtime decides worker scheduling and concurrency.
+
+---
+
+### 4. Wait for Expected Worker Completion
+
+**If `resolvedMode` is `agent-team` or `subagent`:**
+
+```
+⏳ Waiting for 4 NFR subagents to complete...
+  ├── Subagent A (Security): Running... ⟳
+  ├── Subagent B (Performance): Running... ⟳
+  ├── Subagent C (Reliability): Running... ⟳
+  └── Subagent D (Scalability): Running... ⟳
+
+[... time passes ...]
+
+✅ All 4 NFR subagents completed!
+```
+
+**If `resolvedMode` is `sequential`:**
+
+```
+✅ Sequential mode: each worker already completed during dispatch.
+```
+
+---
+
+### 5. Verify All Outputs Exist
+
+```javascript
+const outputs = ['security', 'performance', 'reliability', 'scalability'].map((domain) => `/tmp/tea-nfr-${domain}-${timestamp}.json`);
+
+outputs.forEach((output) => {
+  if (!fs.existsSync(output)) {
+    throw new Error(`Subagent output missing: ${output}`);
+  }
+});
+```
+
+---
+
+### 6. Execution Report
+
+```
+🚀 Performance Report:
+- Execution Mode: {resolvedMode}
+- Total Elapsed: ~mode-dependent
+- Parallel Gain: ~67% faster when mode is subagent/agent-team
+```
+
+---
+
+### 7. Proceed to Aggregation
+
+Load next step: `{nextStepFile}`
+
+The aggregation step will:
+
+- Read all 4 NFR domain outputs
+- Calculate overall risk level
+- Aggregate compliance status
+- Identify cross-domain risks
+- Generate executive summary
+
+---
+
+## EXIT CONDITION
+
+Proceed when all 4 required worker steps completed and outputs exist.
+
+---
+
+## 🚨 SYSTEM SUCCESS METRICS
+
+### ✅ SUCCESS:
+
+- All required worker steps completed
+- Fallback behavior respected configuration and capability probe rules
+
+### ❌ FAILURE:
+
+- One or more subagents failed
+- Unsupported requested mode with probing disabled

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-nfr/steps-c/step-04a-subagent-security.md

@@ -0,0 +1,138 @@
+---
+name: 'step-04a-subagent-security'
+description: 'Subagent: Security NFR assessment'
+subagent: true
+outputFile: '/tmp/tea-nfr-security-{{timestamp}}.json'
+---
+
+# Subagent 4A: Security NFR Assessment
+
+## SUBAGENT CONTEXT
+
+This is an **isolated subagent** running in parallel with other NFR domain assessments.
+
+**Your task:** Assess SECURITY NFR domain only.
+
+---
+
+## MANDATORY EXECUTION RULES
+
+- ✅ Assess SECURITY only (not performance, reliability, scalability)
+- ✅ Output structured JSON to temp file
+- ❌ Do NOT assess other NFR domains
+
+---
+
+## SUBAGENT TASK
+
+### 1. Security Assessment Categories
+
+**Assess the following security dimensions:**
+
+**A) Authentication & Authorization:**
+
+- OAuth2/JWT implementation
+- Session management
+- Multi-factor authentication
+- Role-based access control (RBAC)
+
+**B) Data Protection:**
+
+- Encryption at rest
+- Encryption in transit (HTTPS/TLS)
+- Sensitive data handling (PII, passwords)
+- Database encryption
+
+**C) Input Validation:**
+
+- SQL injection prevention
+- XSS prevention
+- CSRF protection
+- Input sanitization
+
+**D) API Security:**
+
+- Rate limiting
+- API authentication
+- CORS configuration
+- Security headers
+
+**E) Secrets Management:**
+
+- Environment variables for secrets
+- No hardcoded credentials
+- Secret rotation policies
+- Key management systems
+
+### 2. Risk Assessment
+
+For each category, determine status:
+
+- **PASS**: Properly implemented
+- **CONCERN**: Partially implemented or weak
+- **FAIL**: Not implemented or critical vulnerability
+- **N/A**: Not applicable to this system
+
+### 3. Compliance Check
+
+**Common compliance standards:**
+
+- SOC2
+- GDPR
+- HIPAA
+- PCI-DSS
+- ISO 27001
+
+---
+
+## OUTPUT FORMAT
+
+```json
+{
+  "domain": "security",
+  "risk_level": "MEDIUM",
+  "findings": [
+    {
+      "category": "Authentication",
+      "status": "PASS",
+      "description": "OAuth2 with JWT tokens implemented",
+      "evidence": ["src/auth/oauth.ts", "JWT refresh token rotation"],
+      "recommendations": []
+    },
+    {
+      "category": "Data Encryption",
+      "status": "CONCERN",
+      "description": "Database encryption at rest not enabled",
+      "evidence": ["Database config shows no encryption"],
+      "recommendations": ["Enable database encryption at rest", "Use AWS RDS encryption or equivalent", "Implement key rotation policy"]
+    },
+    {
+      "category": "Input Validation",
+      "status": "FAIL",
+      "description": "SQL injection vulnerability in search endpoint",
+      "evidence": ["src/api/search.ts:42 - direct SQL concatenation"],
+      "recommendations": ["URGENT: Use parameterized queries", "Add input sanitization library", "Implement WAF rules"]
+    }
+  ],
+  "compliance": {
+    "SOC2": "PARTIAL",
+    "GDPR": "PASS",
+    "HIPAA": "N/A",
+    "PCI-DSS": "FAIL"
+  },
+  "priority_actions": [
+    "Fix SQL injection vulnerability (URGENT)",
+    "Enable database encryption within 30 days",
+    "Implement rate limiting for all APIs"
+  ],
+  "summary": "Security posture is MEDIUM risk with 1 critical vulnerability requiring immediate attention"
+}
+```
+
+---
+
+## EXIT CONDITION
+
+Subagent completes when JSON output written to temp file.
+
+**Subagent terminates here.**

package/lib/bmad-cache/tea/src/workflows/testarch/bmad-testarch-nfr/steps-c/step-04b-subagent-performance.md

@@ -0,0 +1,84 @@
+---
+name: 'step-04b-subagent-performance'
+description: 'Subagent: Performance NFR assessment'
+subagent: true
+outputFile: '/tmp/tea-nfr-performance-{{timestamp}}.json'
+---
+
+# Subagent 4B: Performance NFR Assessment
+
+## SUBAGENT CONTEXT
+
+This is an **isolated subagent** running in parallel with other NFR domain assessments.
+
+**Your task:** Assess PERFORMANCE NFR domain only.
+
+---
+
+## SUBAGENT TASK
+
+### 1. Performance Assessment Categories
+
+**A) Response Times:**
+
+- API response times (<200ms target)
+- Page load times (<2s target)
+- Time to interactive (<3s target)
+
+**B) Throughput:**
+
+- Requests per second capacity
+- Concurrent user support
+- Database query performance
+
+**C) Resource Usage:**
+
+- Memory consumption
+- CPU utilization
+- Database connection pooling
+
+**D) Optimization:**
+
+- Caching strategies
+- CDN usage
+- Code splitting/lazy loading
+- Database indexing
+
+---
+
+## OUTPUT FORMAT
+
+```json
+{
+  "domain": "performance",
+  "risk_level": "LOW",
+  "findings": [
+    {
+      "category": "Response Times",
+      "status": "PASS",
+      "description": "API endpoints respond in <150ms (P95)",
+      "evidence": ["Load testing results show 140ms P95"],
+      "recommendations": []
+    },
+    {
+      "category": "Caching",
+      "status": "CONCERN",
+      "description": "No CDN for static assets",
+      "evidence": ["Static files served from origin"],
+      "recommendations": ["Implement CDN (CloudFront/Cloudflare)", "Cache static assets for 1 year"]
+    }
+  ],
+  "compliance": {
+    "SLA_99.9": "PASS",
+    "SLA_99.99": "CONCERN"
+  },
+  "priority_actions": ["Implement CDN for static assets", "Add database query caching for frequent reads"],
+  "summary": "Performance is acceptable with minor optimization opportunities"
+}
+```
+
+---
+
+## EXIT CONDITION
+
+Subagent completes when JSON output written to temp file.