@northbridge-security/secureai 0.1.13

package/docs/guides/agents.tdd.md

@@ -0,0 +1,838 @@
+# Test-Driven Development for AI Agents
+
+This guide establishes test-driven development (TDD) principles for AI agents working with codebases. These patterns apply across all languages and frameworks, ensuring testable, maintainable, and reliable code.
+
+## Target Audience
+
+AI agents (Claude Code, Cursor, GitHub Copilot, etc.) writing production code that requires automated testing, regression prevention, and quality assurance.
+
+## Core Principles
+
+### The TDD Cycle
+
+**Red → Green → Refactor** is the fundamental TDD workflow:
+
+```text
+1. RED: Write a failing test
+   ↓
+2. GREEN: Write minimal code to pass the test
+   ↓
+3. REFACTOR: Improve code without changing behavior
+   ↓
+(Repeat)
+```
+
+**Why this order matters:**
+
+- **Red first** - Proves the test can fail (validates test correctness)
+- **Green quickly** - Gets to working state fast (validates implementation)
+- **Refactor safely** - Tests catch regressions (enables improvement)
+
+### Test First, Code Second
+
+**Always write tests before implementation:**
+
+```text
+❌ WRONG:
+1. Write function implementation
+2. Write tests to verify it works
+3. Find bugs, fix, repeat
+
+✓ CORRECT:
+1. Write test describing expected behavior
+2. Run test (should fail - RED)
+3. Write minimal code to pass test (GREEN)
+4. Refactor for quality (REFACTOR)
+```
+
+**Benefits:**
+
+- **Better design** - Writing tests first forces you to think about API design
+- **Complete coverage** - Every line of code has a corresponding test
+- **No dead code** - Only write code needed to pass tests
+- **Living documentation** - Tests document how code should be used
+
+### Small Steps
+
+**Make incremental progress with small, focused tests:**
+
+```text
+Testing a validator function:
+
+Step 1: Test empty input
+Step 2: Test valid input
+Step 3: Test invalid format
+Step 4: Test boundary conditions
+Step 5: Test error messages
+```
+
+**Why small steps:**
+
+- Easier to identify what broke when tests fail
+- Faster feedback loop (run tests every few minutes)
+- Reduced cognitive load (focus on one behavior at a time)
+- Natural progression toward complete implementation
+
+## Test Types
+
+### Unit Tests
+
+**Test individual functions/classes in isolation:**
+
+**Characteristics:**
+
+- Fast (< 10ms per test)
+- No external dependencies (filesystem, network, database)
+- Use mocks/stubs for dependencies
+- Test one behavior per test
+
+**Example scenarios:**
+
+- Pure functions (input → output)
+- Business logic calculations
+- Data transformations
+- Validation rules
+- String parsing
+- Math operations
+
+**When to use:**
+
+- Testing business logic
+- Validating calculations
+- Checking edge cases
+- Regression prevention
+
+### Integration Tests
+
+**Test multiple components working together:**
+
+**Characteristics:**
+
+- Slower (100ms - 5 seconds per test)
+- May use real dependencies (files, databases, APIs)
+- Test interaction between components
+- Verify end-to-end workflows
+
+**Example scenarios:**
+
+- Reading/writing files
+- Database queries
+- API calls
+- Command execution
+- Configuration loading
+- Multi-step processes
+
+**When to use:**
+
+- Testing system boundaries (filesystem, network)
+- Verifying component integration
+- End-to-end workflow validation
+- Infrastructure verification
+
+### Functional/End-to-End Tests
+
+**Test complete user workflows:**
+
+**Characteristics:**
+
+- Slowest (seconds to minutes per test)
+- Full system deployment
+- Real environment (staging/production-like)
+- User-centric scenarios
+
+**Example scenarios:**
+
+- CLI command workflows
+- Web application user flows
+- API endpoint chains
+- Installation processes
+- Update/migration procedures
+
+**When to use:**
+
+- Critical user workflows
+- Release verification
+- Smoke testing deployments
+- Regression testing major features
+
+## Test Organization
+
+### Folder Structure
+
+**Organize tests parallel to source code:**
+
+```text
+project/
+├── src/
+│   ├── auth/
+│   │   ├── authenticate.ts
+│   │   ├── session.ts
+│   │   └── tokens.ts
+│   └── users/
+│       ├── repository.ts
+│       └── service.ts
+├── tests/
+│   ├── unit/
+│   │   ├── auth/
+│   │   │   ├── authenticate.test.ts
+│   │   │   ├── session.test.ts
+│   │   │   └── tokens.test.test.ts
+│   │   └── users/
+│   │       ├── repository.test.ts
+│   │       └── service.test.ts
+│   ├── integration/
+│   │   ├── auth-flow.test.ts
+│   │   └── user-management.test.ts
+│   └── mocks/
+│       ├── auth/
+│       │   └── mock-session.ts
+│       └── users/
+│           └── mock-repository.ts
+```
+
+**Benefits:**
+
+- Easy to find related tests
+- Clear separation of test types
+- Parallel structure to source code
+- Shared mocks in dedicated folder
+
+### Naming Conventions
+
+**Test file names:**
+
+| Pattern                | Example                    | Purpose            |
+| ---------------------- | -------------------------- | ------------------ |
+| `<module>.test.<ext>`  | `authenticate.test.ts`     | Unit tests         |
+| `<feature>.test.<ext>` | `auth-flow.test.ts`        | Integration tests  |
+| `<workflow>.e2e.<ext>` | `user-registration.e2e.ts` | End-to-end tests   |
+| `mock-<module>.<ext>`  | `mock-repository.ts`       | Test doubles/mocks |
+
+**Test case names:**
+
+Use descriptive names that explain behavior:
+
+```text
+✓ GOOD:
+- "should return user when valid credentials provided"
+- "should throw error when password is too short"
+- "should hash password before storing in database"
+
+✗ BAD:
+- "test1"
+- "authentication"
+- "it works"
+```
+
+## Writing Effective Tests
+
+### Arrange-Act-Assert Pattern
+
+**Structure every test with three sections:**
+
+```text
+// Arrange: Set up test data and preconditions
+const input = "test@example.com";
+const expected = { email: "test@example.com", valid: true };
+
+// Act: Execute the code being tested
+const result = validateEmail(input);
+
+// Assert: Verify the outcome
+expect(result).toEqual(expected);
+```
+
+**Why this structure:**
+
+- Clear separation of setup, execution, and verification
+- Easy to understand what's being tested
+- Simple to debug when tests fail
+- Consistent pattern across all tests
+
+### One Assertion Per Concept
+
+**Test one behavior at a time:**
+
+```text
+✓ GOOD - Single concept:
+test("should validate email format") {
+  const result = validateEmail("test@example.com");
+  expect(result.valid).toBe(true);
+}
+
+test("should extract email domain") {
+  const result = validateEmail("test@example.com");
+  expect(result.domain).toBe("example.com");
+}
+
+✗ BAD - Multiple concepts:
+test("should validate email") {
+  const result = validateEmail("test@example.com");
+  expect(result.valid).toBe(true);
+  expect(result.domain).toBe("example.com");
+  expect(result.username).toBe("test");
+}
+```
+
+**Exception:** Multiple assertions are acceptable when testing the same concept:
+
+```text
+✓ ACCEPTABLE - Same concept (object shape):
+test("should return complete user object") {
+  const user = createUser("John", "john@example.com");
+
+  expect(user.name).toBe("John");
+  expect(user.email).toBe("john@example.com");
+  expect(user.id).toBeDefined();
+  expect(user.createdAt).toBeInstanceOf(Date);
+}
+```
+
+### Test Edge Cases
+
+**Cover boundary conditions and error scenarios:**
+
+**Input validation example:**
+
+```text
+Function: validateAge(age: number): boolean
+
+Test cases:
+1. Valid age (18-120): expect true
+2. Minimum boundary (18): expect true
+3. Below minimum (17): expect false
+4. Maximum boundary (120): expect true
+5. Above maximum (121): expect false
+6. Zero: expect false
+7. Negative: expect false
+8. Decimal: expect false
+9. NaN: expect false
+10. Infinity: expect false
+```
+
+**Common edge cases:**
+
+- Empty inputs (null, undefined, empty string, empty array)
+- Boundary values (min, max, zero, one)
+- Invalid types (wrong type, NaN, Infinity)
+- Special characters (Unicode, emojis, control characters)
+- Large inputs (performance, memory limits)
+- Concurrent operations (race conditions)
+
+### Avoid Test Interdependence
+
+**Each test should be independent:**
+
+```text
+✓ GOOD - Independent tests:
+test("should add user") {
+  const db = createTestDatabase();
+  db.addUser({ name: "Alice" });
+  expect(db.count()).toBe(1);
+}
+
+test("should remove user") {
+  const db = createTestDatabase();
+  db.addUser({ name: "Bob" });
+  db.removeUser("Bob");
+  expect(db.count()).toBe(0);
+}
+
+✗ BAD - Dependent tests:
+let db;
+
+test("should add user") {
+  db = createTestDatabase();
+  db.addUser({ name: "Alice" });
+  expect(db.count()).toBe(1);
+}
+
+test("should remove user") {
+  // DEPENDS ON PREVIOUS TEST
+  db.removeUser("Alice");
+  expect(db.count()).toBe(0);
+}
+```
+
+**Why independence matters:**
+
+- Tests can run in any order
+- Tests can run in parallel
+- Failures are isolated (one failure doesn't cascade)
+- Tests can be run individually for debugging
+
+## Mocking and Test Doubles
+
+### When to Use Mocks
+
+**Use mocks for external dependencies:**
+
+**Mock these:**
+
+- File system operations (read, write, delete)
+- Network requests (HTTP, WebSocket, database)
+- System commands (exec, spawn)
+- Time-dependent code (Date.now(), timers)
+- Random number generation
+- External APIs
+
+**Don't mock these:**
+
+- Pure functions (no side effects)
+- Data structures (objects, arrays)
+- Simple utilities (string manipulation, math)
+- Code you're testing directly
+
+### Types of Test Doubles
+
+**Different patterns for different needs:**
+
+**Stub** - Returns canned responses:
+
+```text
+mockDatabase.getUser() → returns { id: 1, name: "Test User" }
+```
+
+**Spy** - Records how it was called:
+
+```text
+mockLogger.log("message")
+→ Verify: called once with "message"
+```
+
+**Mock** - Programmable behavior with expectations:
+
+```text
+mockAPI
+  .expect("POST", "/users")
+  .withBody({ name: "Alice" })
+  .respond({ id: 1 })
+```
+
+**Fake** - Working implementation (lightweight):
+
+```text
+InMemoryDatabase - Real database logic, but in-memory storage
+```
+
+### Dependency Injection for Testability
+
+**Pass dependencies as parameters:**
+
+```text
+✓ GOOD - Injectable dependency:
+function saveUser(user, database) {
+  return database.insert(user);
+}
+
+// In tests:
+const mockDB = createMockDatabase();
+saveUser({ name: "Alice" }, mockDB);
+
+✗ BAD - Hard-coded dependency:
+import { realDatabase } from './database';
+
+function saveUser(user) {
+  return realDatabase.insert(user);
+  // Cannot test without real database
+}
+```
+
+**For class-based code, use constructor injection:**
+
+```text
+✓ GOOD - Constructor injection:
+class UserService {
+  constructor(database, emailService) {
+    this.database = database;
+    this.emailService = emailService;
+  }
+}
+
+// In tests:
+const service = new UserService(mockDB, mockEmail);
+
+✗ BAD - Hard-coded dependencies:
+class UserService {
+  constructor() {
+    this.database = new RealDatabase();
+    this.emailService = new RealEmailService();
+  }
+}
+```
+
+## Test Coverage
+
+### Coverage Metrics
+
+**Understand what coverage measures:**
+
+| Metric             | Meaning                      | Target |
+| ------------------ | ---------------------------- | ------ |
+| Line coverage      | % of code lines executed     | 80%+   |
+| Branch coverage    | % of if/else branches tested | 80%+   |
+| Function coverage  | % of functions called        | 90%+   |
+| Statement coverage | % of statements executed     | 80%+   |
+
+**Coverage is not quality:**
+
+- 100% coverage doesn't mean bug-free code
+- Focus on meaningful tests, not coverage numbers
+- Cover critical paths and edge cases thoroughly
+- Low-value code (getters/setters) can have lower coverage
+
+### What to Prioritize
+
+**Test these thoroughly (aim for 100%):**
+
+- Business logic and algorithms
+- Security-critical code (authentication, authorization)
+- Data validation and sanitization
+- Error handling and edge cases
+- Public APIs and interfaces
+
+**Lower priority (aim for 60-80%):**
+
+- Simple getters/setters
+- Configuration loading
+- Logging statements
+- UI layout code
+- Trivial utilities
+
+### Excluding Code from Coverage
+
+**Mark code that shouldn't be covered:**
+
+```text
+// Language-specific examples:
+
+// TypeScript/JavaScript
+/* istanbul ignore next */
+function developmentOnlyHelper() { ... }
+
+// Python
+def debug_helper():  # pragma: no cover
+    ...
+
+// Go
+// +build !test
+```
+
+**What to exclude:**
+
+- Development/debug utilities
+- Platform-specific code on other platforms
+- Defensive assertions that should never happen
+- Generated code
+
+## Anti-Patterns to Avoid
+
+### Testing Implementation Details
+
+**Test behavior, not implementation:**
+
+```text
+✗ BAD - Tests internal state:
+test("should increment counter") {
+  const obj = new Counter();
+  obj.increment();
+  expect(obj._internalCounter).toBe(1); // Testing private state
+}
+
+✓ GOOD - Tests public behavior:
+test("should return incremented value") {
+  const counter = new Counter();
+  counter.increment();
+  expect(counter.getValue()).toBe(1);
+}
+```
+
+**Why this matters:**
+
+- Internal refactoring shouldn't break tests
+- Tests document public contract, not implementation
+- Enables changing internals without test changes
+
+### Brittle Tests
+
+**Avoid tests that break on unrelated changes:**
+
+```text
+✗ BAD - Hardcoded values:
+test("should format date") {
+  const result = formatDate(new Date());
+  expect(result).toBe("2024-11-22 14:30:45"); // Breaks constantly
+}
+
+✓ GOOD - Flexible matching:
+test("should format date") {
+  const result = formatDate(new Date());
+  expect(result).toMatch(/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}$/);
+}
+```
+
+### Testing Multiple Things
+
+**One test, one responsibility:**
+
+```text
+✗ BAD - Tests entire workflow:
+test("user workflow") {
+  const user = createUser();
+  user.login();
+  user.updateProfile();
+  user.changePassword();
+  user.logout();
+  // If any step fails, which one?
+}
+
+✓ GOOD - Separate tests:
+test("should create user")
+test("should login user")
+test("should update profile")
+test("should change password")
+test("should logout user")
+```
+
+### Slow Tests
+
+**Keep tests fast:**
+
+**Performance targets:**
+
+- Unit test: < 10ms
+- Integration test: < 1 second
+- E2E test: < 30 seconds
+
+**Optimization strategies:**
+
+- Use in-memory databases instead of real ones
+- Mock slow external dependencies
+- Parallelize test execution
+- Use test data factories (don't recreate fixtures)
+- Share expensive setup across tests (carefully)
+
+## TDD in Practice
+
+### Starting a New Feature
+
+**TDD workflow for new features:**
+
+```text
+1. Write first test for simplest case
+   → Test fails (RED)
+
+2. Write minimal code to pass
+   → Test passes (GREEN)
+
+3. Write test for next case
+   → Test fails (RED)
+
+4. Extend code to pass new test
+   → All tests pass (GREEN)
+
+5. Refactor if needed
+   → Tests still pass (GREEN)
+
+6. Repeat until feature complete
+```
+
+**Example: Building an email validator**
+
+```text
+Step 1: Test empty string
+  Test: validateEmail("") → {valid: false}
+  Code: function validateEmail(email) { return {valid: false}; }
+
+Step 2: Test simple valid email
+  Test: validateEmail("a@b.c") → {valid: true}
+  Code: Add check for @ and .
+
+Step 3: Test invalid format (no @)
+  Test: validateEmail("invalid") → {valid: false}
+  Code: Already passes
+
+Step 4: Test invalid format (no domain)
+  Test: validateEmail("test@") → {valid: false}
+  Code: Add domain check
+
+Step 5: Test complex valid email
+  Test: validateEmail("user.name+tag@example.co.uk") → {valid: true}
+  Code: Improve regex pattern
+
+(Continue for all edge cases...)
+```
+
+### Fixing Bugs
+
+**TDD workflow for bug fixes:**
+
+```text
+1. Write test that reproduces the bug
+   → Test fails (confirms bug exists)
+
+2. Fix the bug
+   → Test passes (bug is fixed)
+
+3. Ensure all other tests still pass
+   → Regression test in place forever
+```
+
+**Example: Bug report: "App crashes on empty input"**
+
+```text
+1. Write failing test:
+   test("should handle empty input") {
+     expect(() => processInput("")).not.toThrow();
+   }
+   → Test fails: TypeError: Cannot read property 'length' of undefined
+
+2. Fix code:
+   function processInput(input) {
+     if (!input) return null;  // Add null check
+     return input.length;
+   }
+   → Test passes
+
+3. Run all tests:
+   → All pass, regression prevented
+```
+
+### Refactoring
+
+**TDD enables safe refactoring:**
+
+```text
+1. Ensure comprehensive test coverage
+   → All tests pass (GREEN)
+
+2. Refactor code
+   → Change structure, not behavior
+
+3. Run tests frequently
+   → Tests catch regressions immediately
+
+4. If tests fail:
+   → Either fix code or fix test (if test was wrong)
+
+5. Repeat until refactoring complete
+   → All tests still pass
+```
+
+**Refactoring example:**
+
+```text
+BEFORE:
+function calculateTotal(items) {
+  let total = 0;
+  for (let i = 0; i < items.length; i++) {
+    total += items[i].price * items[i].quantity;
+  }
+  return total;
+}
+
+Tests: ✓ All passing
+
+AFTER (refactored):
+function calculateTotal(items) {
+  return items.reduce((sum, item) =>
+    sum + (item.price * item.quantity), 0
+  );
+}
+
+Tests: ✓ All still passing (behavior unchanged)
+```
+
+## Best Practices for AI Agents
+
+### Always Run Tests Before Coding
+
+**Workflow for AI agents:**
+
+```text
+1. Read existing tests
+2. Understand expected behavior
+3. Write new test for feature/fix
+4. Run tests (should fail)
+5. Write code to pass test
+6. Run tests (should pass)
+7. Refactor if needed
+8. Run tests (should still pass)
+```
+
+**Never skip step 4** - Confirming the test fails proves it's valid.
+
+### Communicate Test Results
+
+**Report test status to users:**
+
+```text
+✓ GOOD:
+"I've written a test for email validation. Running tests..."
+[test output]
+"Test failed as expected (RED). Now implementing the validator..."
+[writes code]
+"Running tests again..."
+[test output]
+"Test passes (GREEN). Email validation is working correctly."
+
+✗ BAD:
+"I've implemented email validation."
+[no tests mentioned, no verification shown]
+```
+
+### Use Test Output for Debugging
+
+**When tests fail, analyze output:**
+
+```text
+Test failure output:
+Expected: { valid: true, domain: "example.com" }
+Received: { valid: true, domain: undefined }
+
+Analysis:
+- valid flag is correct
+- domain extraction is broken
+- Focus debugging on domain parsing logic
+```
+
+### Maintain Test Quality
+
+**Treat tests as production code:**
+
+- Use descriptive names
+- Keep tests simple and readable
+- Refactor duplicate test code
+- Delete obsolete tests
+- Update tests when requirements change
+
+## Language-Specific Guides
+
+For implementation details in specific languages:
+
+- **TypeScript/JavaScript**: See [agents.tdd.ts.md](./agents.tdd.ts.md)
+- **Python**: See [agents.tdd.py.md](./agents.tdd.py.md) (coming soon)
+- **Go**: See [agents.tdd.go.md](./agents.tdd.go.md) (coming soon)
+- **Ruby**: See [agents.tdd.rb.md](./agents.tdd.rb.md) (coming soon)
+
+## Summary
+
+**TDD fundamentals:**
+
+- Write tests first (RED → GREEN → REFACTOR)
+- Test behavior, not implementation
+- Keep tests fast and independent
+- Use mocks for external dependencies
+- Aim for 80%+ coverage on critical code
+- One test, one behavior
+- Run tests frequently
+
+**For AI agents:**
+
+- Always write tests before implementation
+- Verify tests fail before writing code (RED)
+- Show test output to users
+- Use test failures for debugging
+- Maintain test quality like production code
+
+**Result**: Reliable, maintainable code with regression protection and living documentation.