@agentic-qe/v3 3.0.0-alpha.6 → 3.0.0-alpha.8

package/assets/skills/security-testing/SKILL.md

@@ -0,0 +1,306 @@
+---
+name: security-testing
+description: "Test for security vulnerabilities using OWASP principles. Use when conducting security audits, testing auth, or implementing security practices."
+category: specialized-testing
+priority: critical
+tokenEstimate: 1200
+agents: [qe-security-scanner, qe-api-contract-validator, qe-quality-analyzer]
+implementation_status: optimized
+optimization_version: 1.0
+last_optimized: 2025-12-02
+dependencies: []
+quick_reference_card: true
+tags: [security, owasp, sast, dast, vulnerabilities, auth, injection]
+---
+
+# Security Testing
+
+<default_to_action>
+When testing security or conducting audits:
+1. TEST OWASP Top 10 vulnerabilities systematically
+2. VALIDATE authentication and authorization on every endpoint
+3. SCAN dependencies for known vulnerabilities (npm audit)
+4. CHECK for injection attacks (SQL, XSS, command)
+5. VERIFY secrets aren't exposed in code/logs
+
+**Quick Security Checks:**
+- Access control → Test horizontal/vertical privilege escalation
+- Crypto → Verify password hashing, HTTPS, no sensitive data exposed
+- Injection → Test SQL injection, XSS, command injection
+- Auth → Test weak passwords, session fixation, MFA enforcement
+- Config → Check error messages don't leak info
+
+**Critical Success Factors:**
+- Think like an attacker, build like a defender
+- Security is built in, not added at the end
+- Test continuously in CI/CD, not just before release
+</default_to_action>
+
+## Quick Reference Card
+
+### When to Use
+- Security audits and penetration testing
+- Testing authentication/authorization
+- Validating input sanitization
+- Reviewing security configuration
+
+### OWASP Top 10 (2021)
+| # | Vulnerability | Key Test |
+|---|---------------|----------|
+| 1 | Broken Access Control | User A accessing User B's data |
+| 2 | Cryptographic Failures | Plaintext passwords, HTTP |
+| 3 | Injection | SQL/XSS/command injection |
+| 4 | Insecure Design | Rate limiting, session timeout |
+| 5 | Security Misconfiguration | Verbose errors, exposed /admin |
+| 6 | Vulnerable Components | npm audit, outdated packages |
+| 7 | Auth Failures | Weak passwords, no MFA |
+| 8 | Integrity Failures | Unsigned updates, malware |
+| 9 | Logging Failures | No audit trail for breaches |
+| 10 | SSRF | Server fetching internal URLs |
+
+### Tools
+| Type | Tool | Purpose |
+|------|------|---------|
+| SAST | SonarQube, Semgrep | Static code analysis |
+| DAST | OWASP ZAP, Burp | Dynamic scanning |
+| Deps | npm audit, Snyk | Dependency vulnerabilities |
+| Secrets | git-secrets, TruffleHog | Secret scanning |
+
+### Agent Coordination
+- `qe-security-scanner`: Multi-layer SAST/DAST scanning
+- `qe-api-contract-validator`: API security testing
+- `qe-quality-analyzer`: Security code review
+
+---
+
+## Key Vulnerability Tests
+
+### 1. Broken Access Control
+```javascript
+// Horizontal escalation - User A accessing User B's data
+test('user cannot access another user\'s order', async () => {
+  const userAToken = await login('userA');
+  const userBOrder = await createOrder('userB');
+
+  const response = await api.get(`/orders/${userBOrder.id}`, {
+    headers: { Authorization: `Bearer ${userAToken}` }
+  });
+  expect(response.status).toBe(403);
+});
+
+// Vertical escalation - Regular user accessing admin
+test('regular user cannot access admin', async () => {
+  const userToken = await login('regularUser');
+  expect((await api.get('/admin/users', {
+    headers: { Authorization: `Bearer ${userToken}` }
+  })).status).toBe(403);
+});
+```
+
+### 2. Injection Attacks
+```javascript
+// SQL Injection
+test('prevents SQL injection', async () => {
+  const malicious = "' OR '1'='1";
+  const response = await api.get(`/products?search=${malicious}`);
+  expect(response.body.length).toBeLessThan(100); // Not all products
+});
+
+// XSS
+test('sanitizes HTML output', async () => {
+  const xss = '<script>alert("XSS")</script>';
+  await api.post('/comments', { text: xss });
+
+  const html = (await api.get('/comments')).body;
+  expect(html).toContain('&lt;script&gt;');
+  expect(html).not.toContain('<script>');
+});
+```
+
+### 3. Cryptographic Failures
+```javascript
+test('passwords are hashed', async () => {
+  await db.users.create({ email: 'test@example.com', password: 'MyPassword123' });
+  const user = await db.users.findByEmail('test@example.com');
+
+  expect(user.password).not.toBe('MyPassword123');
+  expect(user.password).toMatch(/^\$2[aby]\$\d{2}\$/); // bcrypt
+});
+
+test('no sensitive data in API response', async () => {
+  const response = await api.get('/users/me');
+  expect(response.body).not.toHaveProperty('password');
+  expect(response.body).not.toHaveProperty('ssn');
+});
+```
+
+### 4. Security Misconfiguration
+```javascript
+test('errors don\'t leak sensitive info', async () => {
+  const response = await api.post('/login', { email: 'nonexistent@test.com', password: 'wrong' });
+  expect(response.body.error).toBe('Invalid credentials'); // Generic message
+});
+
+test('sensitive endpoints not exposed', async () => {
+  const endpoints = ['/debug', '/.env', '/.git', '/admin'];
+  for (let ep of endpoints) {
+    expect((await fetch(`https://example.com${ep}`)).status).not.toBe(200);
+  }
+});
+```
+
+### 5. Rate Limiting
+```javascript
+test('rate limiting prevents brute force', async () => {
+  const responses = [];
+  for (let i = 0; i < 20; i++) {
+    responses.push(await api.post('/login', { email: 'test@example.com', password: 'wrong' }));
+  }
+  expect(responses.filter(r => r.status === 429).length).toBeGreaterThan(0);
+});
+```
+
+---
+
+## Security Checklist
+
+### Authentication
+- [ ] Strong password requirements (12+ chars)
+- [ ] Password hashing (bcrypt, scrypt, Argon2)
+- [ ] MFA for sensitive operations
+- [ ] Account lockout after failed attempts
+- [ ] Session ID changes after login
+- [ ] Session timeout
+
+### Authorization
+- [ ] Check authorization on every request
+- [ ] Least privilege principle
+- [ ] No horizontal escalation
+- [ ] No vertical escalation
+
+### Data Protection
+- [ ] HTTPS everywhere
+- [ ] Encrypted at rest
+- [ ] Secrets not in code/logs
+- [ ] PII compliance (GDPR)
+
+### Input Validation
+- [ ] Server-side validation
+- [ ] Parameterized queries (no SQL injection)
+- [ ] Output encoding (no XSS)
+- [ ] Rate limiting
+
+---
+
+## CI/CD Integration
+
+```yaml
+# GitHub Actions
+security-checks:
+  steps:
+    - name: Dependency audit
+      run: npm audit --audit-level=high
+
+    - name: SAST scan
+      run: npm run sast
+
+    - name: Secret scan
+      uses: trufflesecurity/trufflehog@main
+
+    - name: DAST scan
+      if: github.ref == 'refs/heads/main'
+      run: docker run owasp/zap2docker-stable zap-baseline.py -t https://staging.example.com
+```
+
+**Pre-commit hooks:**
+```bash
+#!/bin/sh
+git-secrets --scan
+npm run lint:security
+```
+
+---
+
+## Agent-Assisted Security Testing
+
+```typescript
+// Comprehensive multi-layer scan
+await Task("Security Scan", {
+  target: 'src/',
+  layers: { sast: true, dast: true, dependencies: true, secrets: true },
+  severity: ['critical', 'high', 'medium']
+}, "qe-security-scanner");
+
+// OWASP Top 10 testing
+await Task("OWASP Scan", {
+  categories: ['broken-access-control', 'injection', 'cryptographic-failures'],
+  depth: 'comprehensive'
+}, "qe-security-scanner");
+
+// Validate fix
+await Task("Validate Fix", {
+  vulnerability: 'CVE-2024-12345',
+  expectedResolution: 'upgrade package to v2.0.0',
+  retestAfterFix: true
+}, "qe-security-scanner");
+```
+
+---
+
+## Agent Coordination Hints
+
+### Memory Namespace
+```
+aqe/security/
+├── scans/*           - Scan results
+├── vulnerabilities/* - Found vulnerabilities
+├── fixes/*           - Remediation tracking
+└── compliance/*      - Compliance status
+```
+
+### Fleet Coordination
+```typescript
+const securityFleet = await FleetManager.coordinate({
+  strategy: 'security-testing',
+  agents: [
+    'qe-security-scanner',
+    'qe-api-contract-validator',
+    'qe-quality-analyzer',
+    'qe-deployment-readiness'
+  ],
+  topology: 'parallel'
+});
+```
+
+---
+
+## Common Mistakes
+
+### ❌ Security by Obscurity
+Hiding admin at `/super-secret-admin` → **Use proper auth**
+
+### ❌ Client-Side Validation Only
+JavaScript validation can be bypassed → **Always validate server-side**
+
+### ❌ Trusting User Input
+Assuming input is safe → **Sanitize, validate, escape all input**
+
+### ❌ Hardcoded Secrets
+API keys in code → **Environment variables, secret management**
+
+---
+
+## Related Skills
+- [agentic-quality-engineering](../agentic-quality-engineering/) - Security with agents
+- [api-testing-patterns](../api-testing-patterns/) - API security testing
+- [compliance-testing](../compliance-testing/) - GDPR, HIPAA, SOC2
+
+---
+
+## Remember
+
+**Think like an attacker:** What would you try to break? Test that.
+**Build like a defender:** Assume input is malicious until proven otherwise.
+**Test continuously:** Security testing is ongoing, not one-time.
+
+**With Agents:** Agents automate vulnerability scanning, track remediation, and validate fixes. Use agents to maintain security posture at scale.

package/assets/skills/sherlock-review/SKILL.md

@@ -0,0 +1,250 @@
+---
+name: sherlock-review
+description: "Evidence-based investigative code review using deductive reasoning to determine what actually happened versus what was claimed. Use when verifying implementation claims, investigating bugs, validating fixes, or conducting root cause analysis. Elementary approach to finding truth through systematic observation."
+category: quality-review
+priority: high
+tokenEstimate: 1100
+agents: [qe-code-reviewer, qe-security-auditor, qe-performance-validator]
+implementation_status: optimized
+optimization_version: 1.0
+last_optimized: 2025-12-03
+dependencies: []
+quick_reference_card: true
+tags: [investigation, evidence-based, code-review, root-cause, deduction]
+---
+
+# Sherlock Review
+
+<default_to_action>
+When investigating code claims:
+1. OBSERVE: Gather all evidence (code, tests, history, behavior)
+2. DEDUCE: What does evidence actually show vs. what was claimed?
+3. ELIMINATE: Rule out what cannot be true
+4. CONCLUDE: Does evidence support the claim?
+5. DOCUMENT: Findings with proof, not assumptions
+
+**The 3-Step Investigation:**
+```bash
+# 1. OBSERVE: Gather evidence
+git diff <commit>
+npm test -- --coverage
+
+# 2. DEDUCE: Compare claim vs reality
+# Does code match description?
+# Do tests prove the fix/feature?
+
+# 3. CONCLUDE: Verdict with evidence
+# SUPPORTED / PARTIALLY SUPPORTED / NOT SUPPORTED
+```
+
+**Holmesian Principles:**
+- "Data! Data! Data!" - Collect before concluding
+- "Eliminate the impossible" - What cannot be true?
+- "You see, but do not observe" - Run code, don't just read
+- Trust only reproducible evidence
+</default_to_action>
+
+## Quick Reference Card
+
+### Evidence Collection Checklist
+
+| Category | What to Check | How |
+|----------|---------------|-----|
+| **Claim** | PR description, commit messages | Read thoroughly |
+| **Code** | Actual file changes | `git diff` |
+| **Tests** | Coverage, assertions | Run independently |
+| **Behavior** | Runtime output | Execute locally |
+| **Timeline** | When things happened | `git log`, `git blame` |
+
+### Verdict Levels
+
+| Verdict | Meaning |
+|---------|---------|
+| ✓ **TRUE** | Evidence fully supports claim |
+| ⚠ **PARTIALLY TRUE** | Claim accurate but incomplete |
+| ✗ **FALSE** | Evidence contradicts claim |
+| ? **NONSENSICAL** | Claim doesn't apply to context |
+
+---
+
+## Investigation Template
+
+```markdown
+## Sherlock Investigation: [Claim]
+
+### The Claim
+"[What PR/commit claims to do]"
+
+### Evidence Examined
+- Code changes: [files, lines]
+- Tests added: [count, coverage]
+- Behavior observed: [what actually happens]
+
+### Deductive Analysis
+
+**Claim**: [specific assertion]
+**Evidence**: [what you found]
+**Deduction**: [logical conclusion]
+**Verdict**: ✓/⚠/✗
+
+### Findings
+- What works: [with evidence]
+- What doesn't: [with evidence]
+- What's missing: [gaps in implementation/testing]
+
+### Recommendations
+1. [Action based on findings]
+```
+
+---
+
+## Investigation Scenarios
+
+### Scenario 1: "This Fixed the Bug"
+
+**Steps:**
+1. Reproduce bug on commit before fix
+2. Verify bug is gone on commit with fix
+3. Check if fix addresses root cause or symptom
+4. Test edge cases not in original report
+
+**Red Flags:**
+- Fix that just removes error logging
+- Works only for specific test case
+- Workarounds instead of root cause fix
+- No regression test added
+
+### Scenario 2: "Improved Performance by 50%"
+
+**Steps:**
+1. Run benchmark on baseline commit
+2. Run same benchmark on optimized commit
+3. Compare in identical conditions
+4. Verify measurement methodology
+
+**Red Flags:**
+- Tested only on toy data
+- Different comparison conditions
+- Trade-offs not mentioned
+
+### Scenario 3: "Handles All Edge Cases"
+
+**Steps:**
+1. List all edge cases in code path
+2. Check each has test coverage
+3. Test boundary conditions
+4. Verify error handling paths
+
+**Red Flags:**
+- `catch {}` swallowing errors
+- Generic error messages
+- No logging of critical errors
+
+---
+
+## Example Investigation
+
+```markdown
+## Case: PR #123 "Fix race condition in async handler"
+
+### Claims Examined:
+1. "Eliminates race condition"
+2. "Adds mutex locking"
+3. "100% thread safe"
+
+### Evidence:
+- File: src/handlers/async-handler.js
+- Changes: Added `async/await`, removed callbacks
+- Tests: 2 new tests for async flow
+- Coverage: 85% (was 75%)
+
+### Analysis:
+
+**Claim 1: "Eliminates race condition"**
+Evidence: Added `await` to sequential operations. No actual mutex.
+Deduction: Race avoided by removing concurrency, not synchronization.
+Verdict: ⚠ PARTIALLY TRUE (solved differently than claimed)
+
+**Claim 2: "Adds mutex locking"**
+Evidence: No mutex library, no lock variables, no sync primitives.
+Verdict: ✗ FALSE
+
+**Claim 3: "100% thread safe"**
+Evidence: JavaScript is single-threaded. No worker threads used.
+Verdict: ? NONSENSICAL (meaningless in this context)
+
+### Conclusion:
+Fix works but not for reasons claimed. Race condition avoided by
+making operations sequential, not by adding synchronization.
+
+### Recommendations:
+1. Update PR description to accurately reflect solution
+2. Add test for concurrent request handling
+3. Remove incorrect technical claims
+```
+
+---
+
+## Agent Integration
+
+```typescript
+// Evidence-based code review
+await Task("Sherlock Review", {
+  prNumber: 123,
+  claims: [
+    "Fixes memory leak",
+    "Improves performance 30%"
+  ],
+  verifyReproduction: true,
+  testEdgeCases: true
+}, "qe-code-reviewer");
+
+// Bug fix verification
+await Task("Verify Fix", {
+  bugCommit: 'abc123',
+  fixCommit: 'def456',
+  reproductionSteps: steps,
+  testBoundaryConditions: true
+}, "qe-code-reviewer");
+```
+
+---
+
+## Agent Coordination Hints
+
+### Memory Namespace
+```
+aqe/sherlock/
+├── investigations/*   - Investigation reports
+├── evidence/*         - Collected evidence
+├── verdicts/*         - Claim verdicts
+└── patterns/*         - Common deception patterns
+```
+
+### Fleet Coordination
+```typescript
+const investigationFleet = await FleetManager.coordinate({
+  strategy: 'evidence-investigation',
+  agents: [
+    'qe-code-reviewer',        // Code analysis
+    'qe-security-auditor',     // Security claim verification
+    'qe-performance-validator' // Performance claim verification
+  ],
+  topology: 'parallel'
+});
+```
+
+---
+
+## Related Skills
+- [brutal-honesty-review](../brutal-honesty-review/) - Direct technical criticism
+- [context-driven-testing](../context-driven-testing/) - Adapt to context
+- [bug-reporting-excellence](../bug-reporting-excellence/) - Document findings
+
+---
+
+## Remember
+
+**"It is a capital mistake to theorize before one has data."** Trust only reproducible evidence. Don't trust commit messages, documentation, or "works on my machine."
+
+**The Sherlock Standard:** Every claim must be verified empirically. What does the evidence actually show?