aiox-core 5.0.7 → 5.0.8

package/.claude/skills/architect-first/references/testing-strategy-guide.md

@@ -0,0 +1,477 @@
+# Testing Strategy Guide
+
+Guide for implementing the "Quality Escape Hatch" philosophy: tests as safety net for temporary imperfection.
+
+## Core Philosophy
+
+**Tests permit temporary imperfection**
+
+- Code quality is negotiable IF backed by comprehensive tests
+- "Ugly" code WITH tests is acceptable
+- "Ugly" code WITHOUT tests is rejected
+- Tests are the safety net that enables pragmatic execution
+
+## Testing Hierarchy
+
+### 1. Unit Tests
+
+**Purpose**: Validate individual components in isolation
+
+**When to write**:
+- Before or immediately after writing the component
+- For all business logic
+- For all data transformations
+- For all utility functions
+
+**Coverage targets**:
+- Minimum 80% coverage for new code
+- 100% coverage for critical paths
+- All edge cases tested
+
+**Example structure**:
+```python
+def test_component_happy_path():
+    """Test the primary use case"""
+    # Arrange
+    input_data = create_test_data()
+
+    # Act
+    result = my_component(input_data)
+
+    # Assert
+    assert result.is_valid()
+    assert result.output == expected_output
+
+def test_component_edge_case_empty_input():
+    """Test edge case: empty input"""
+    result = my_component([])
+    assert result.is_empty()
+
+def test_component_error_invalid_input():
+    """Test error handling: invalid input"""
+    with pytest.raises(ValidationError):
+        my_component(invalid_data)
+```
+
+### 2. Integration Tests
+
+**Purpose**: Validate component interactions and workflows
+
+**When to write**:
+- After multiple components are implemented
+- For cross-module interactions
+- For data flow validations
+- For configuration loading
+
+**Coverage targets**:
+- All major workflows tested end-to-end
+- All integration points validated
+- Configuration variations tested
+
+**Example structure**:
+```python
+def test_workflow_user_registration():
+    """Test complete user registration workflow"""
+    # Setup
+    config = load_test_config()
+    system = SystemUnderTest(config)
+
+    # Execute workflow
+    user_data = create_test_user()
+    result = system.register_user(user_data)
+
+    # Validate workflow steps
+    assert result.validation_passed
+    assert result.user_created
+    assert result.notification_sent
+
+    # Validate integration
+    assert database.user_exists(user_data.email)
+    assert email_service.sent_welcome_email(user_data.email)
+```
+
+### 3. End-to-End Tests
+
+**Purpose**: Validate complete system behavior from user perspective
+
+**When to write**:
+- After core use case is implemented
+- For acceptance criteria validation
+- For user story validation
+
+**Coverage targets**:
+- All user stories have E2E test
+- All acceptance criteria validated
+- Core use case thoroughly tested
+
+**Example structure**:
+```python
+def test_e2e_complete_user_journey():
+    """Test complete user journey from registration to usage"""
+    # User registers
+    user = register_new_user()
+
+    # User logs in
+    session = login_user(user.credentials)
+
+    # User performs core action
+    result = perform_core_action(session)
+
+    # Validate complete journey
+    assert result.success
+    assert user_activity_logged()
+    assert metrics_updated()
+```
+
+## Test-Driven Development (TDD) Pattern
+
+### Standard TDD Cycle
+
+1. **Red**: Write failing test first
+2. **Green**: Write minimal code to pass test
+3. **Refactor**: Improve code while keeping tests green
+
+### Architect-First TDD Adaptation
+
+1. **Architect**: Design and document first
+2. **Red**: Write tests based on architecture
+3. **Green**: Implement to pass tests (code can be "ugly")
+4. **Refactor**: Improve code quality (optional, tests enable this)
+
+**Key difference**: Architecture and documentation precede test writing.
+
+## Quality Escape Hatch: When "Ugly" Code is Acceptable
+
+### Conditions for Acceptance
+
+"Ugly" code is acceptable when ALL of these are true:
+
+1. ✅ **Comprehensive tests exist**
+   - Unit tests cover all logic paths
+   - Integration tests validate workflows
+   - Tests actually run and pass
+
+2. ✅ **Logging/observation points added**
+   - Key decision points logged
+   - Error context captured
+   - Debugging hooks available
+
+3. ✅ **Core use case works**
+   - Primary workflow functional
+   - Acceptance criteria met
+   - User value delivered
+
+4. ✅ **Technical debt documented**
+   - TODO comments with context
+   - Refactoring plan outlined
+   - Known limitations documented
+
+### Examples of Acceptable "Ugly" Code
+
+```python
+# ACCEPTABLE: Nested ifs, but fully tested
+def process_data(data):
+    """Process data through validation pipeline.
+
+    TODO: Refactor nested ifs to strategy pattern
+    See: docs/refactoring/data-processing.md
+    """
+    if data.type == "A":
+        if data.valid:
+            if data.priority == "high":
+                return fast_process_a(data)
+            else:
+                return slow_process_a(data)
+        else:
+            raise ValidationError("Invalid A")
+    elif data.type == "B":
+        # Similar nesting...
+        pass
+    # ... comprehensive tests exist for all paths
+```
+
+```python
+# ACCEPTABLE: Quick implementation, but tested
+def calculate_metrics(records):
+    """Calculate metrics from records.
+
+    TODO: Optimize query - currently loads all to memory
+    Performance: ~500ms for 10k records (acceptable for MVP)
+    """
+    # Load everything (inefficient but works)
+    all_data = list(records)
+
+    # Calculate (could be vectorized)
+    result = sum(r.value for r in all_data) / len(all_data)
+
+    logging.info(f"Calculated metric: {result} from {len(all_data)} records")
+    return result
+    # Unit tests verify correctness
+    # Integration tests verify performance acceptable
+```
+
+### Examples of UNACCEPTABLE Code
+
+```python
+# REJECTED: No tests, no logging
+def process_important_data(data):
+    # Complex logic with no tests
+    result = data.field1 + data.field2 * 3.14
+    if result > threshold:  # hardcoded threshold
+        return do_something(result)
+    return None
+```
+
+```python
+# REJECTED: Tests exist but don't actually validate behavior
+def calculate_revenue(orders):
+    # Complex calculation
+    total = sum(o.amount for o in orders)
+    return total * 1.1  # Why 1.1? No comment, no doc
+
+def test_calculate_revenue():
+    # Useless test - doesn't validate logic
+    result = calculate_revenue([])
+    assert result >= 0  # Always passes, validates nothing
+```
+
+## Logging Strategy for Test Support
+
+### Strategic Logging Points
+
+1. **Entry/Exit of major functions**
+   ```python
+   logging.info(f"Starting process_workflow with {len(items)} items")
+   result = process_workflow(items)
+   logging.info(f"Completed process_workflow: {result.summary()}")
+   ```
+
+2. **Decision points**
+   ```python
+   if should_use_fast_path(data):
+       logging.debug("Using fast path: criteria met")
+       return fast_path(data)
+   else:
+       logging.debug(f"Using slow path: {data.reason}")
+       return slow_path(data)
+   ```
+
+3. **Error conditions**
+   ```python
+   try:
+       result = risky_operation()
+   except ValidationError as e:
+       logging.error(f"Validation failed: {e}", extra={
+           "input": data,
+           "context": current_context
+       })
+       raise
+   ```
+
+4. **Performance monitoring**
+   ```python
+   import time
+   start = time.time()
+   result = expensive_operation()
+   duration = time.time() - start
+   logging.info(f"Operation completed in {duration:.2f}s")
+   if duration > THRESHOLD:
+       logging.warning(f"Operation slow: {duration:.2f}s > {THRESHOLD}s")
+   ```
+
+## Test Organization
+
+### Directory Structure
+
+```
+project/
+├── src/
+│   └── module/
+│       ├── __init__.py
+│       └── component.py
+├── tests/
+│   ├── unit/
+│   │   └── test_component.py
+│   ├── integration/
+│   │   └── test_module_integration.py
+│   └── e2e/
+│       └── test_user_workflows.py
+└── conftest.py  # Shared fixtures
+```
+
+### Naming Conventions
+
+- Test files: `test_*.py`
+- Test functions: `test_[component]_[scenario]`
+- Test classes: `Test[Component]`
+
+**Examples**:
+- `test_parser_valid_input()`
+- `test_parser_empty_input()`
+- `test_parser_invalid_format_raises_error()`
+
+## Test Fixtures and Helpers
+
+### Configuration for Tests
+
+```python
+# conftest.py
+import pytest
+
+@pytest.fixture
+def test_config():
+    """Load test configuration"""
+    return {
+        "database": "sqlite:///:memory:",
+        "log_level": "DEBUG",
+        "timeout": 1.0
+    }
+
+@pytest.fixture
+def sample_data():
+    """Create sample test data"""
+    return [
+        {"id": 1, "value": 100},
+        {"id": 2, "value": 200},
+    ]
+```
+
+### Mock External Dependencies
+
+```python
+from unittest.mock import Mock, patch
+
+def test_with_external_api():
+    """Test component that calls external API"""
+    with patch('module.external_api') as mock_api:
+        mock_api.fetch_data.return_value = {"status": "ok"}
+
+        result = my_component.process()
+
+        assert result.success
+        mock_api.fetch_data.assert_called_once()
+```
+
+## Running Tests
+
+### Local Development
+
+```bash
+# Run all tests
+pytest
+
+# Run with coverage
+pytest --cov=src --cov-report=html
+
+# Run specific test file
+pytest tests/unit/test_component.py
+
+# Run specific test
+pytest tests/unit/test_component.py::test_parser_valid_input
+
+# Run with verbose output
+pytest -v
+
+# Run with logging output
+pytest -s --log-cli-level=DEBUG
+```
+
+### CI/CD Integration
+
+```yaml
+# .github/workflows/test.yml
+name: Tests
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.9
+      - name: Install dependencies
+        run: pip install -r requirements-test.txt
+      - name: Run tests
+        run: pytest --cov=src --cov-fail-under=80
+```
+
+## Coverage Targets
+
+### Minimum Coverage Requirements
+
+- **New code**: 80% minimum
+- **Critical paths**: 100% required
+- **Existing code**: No decrease allowed
+
+### Measuring Coverage
+
+```bash
+# Generate coverage report
+pytest --cov=src --cov-report=html
+
+# View in browser
+open htmlcov/index.html
+
+# Check specific module
+pytest --cov=src.module --cov-report=term-missing
+```
+
+### Exceptions to Coverage
+
+Acceptable to exclude from coverage:
+- Logging statements
+- Debug code
+- Type checking code (`if TYPE_CHECKING:`)
+- Abstract base class definitions
+- Explicit `# pragma: no cover` with justification
+
+## Test Quality Checklist
+
+- [ ] Tests are independent (can run in any order)
+- [ ] Tests are repeatable (same result every time)
+- [ ] Tests are fast (< 1s for unit tests)
+- [ ] Tests have clear names describing what they test
+- [ ] Tests follow Arrange-Act-Assert pattern
+- [ ] Edge cases are tested
+- [ ] Error conditions are tested
+- [ ] Tests don't depend on external state
+- [ ] Mocks are used for external dependencies
+- [ ] Test data is clearly defined
+
+## Refactoring with Test Safety Net
+
+**Process**:
+
+1. Ensure comprehensive tests exist
+2. Run tests → all green
+3. Refactor code incrementally
+4. Run tests after each change
+5. If tests fail → fix or revert
+6. Continue refactoring
+7. Final test run → all green
+8. Commit
+
+**Rules**:
+- NEVER refactor without tests
+- NEVER change tests and code simultaneously
+- ALWAYS keep tests passing
+- COMMIT frequently with passing tests
+
+---
+
+## Summary
+
+**Quality Escape Hatch in Practice**:
+
+1. **Before coding**: Define test plan
+2. **While coding**: Write tests (can be concurrent or after)
+3. **Code quality**: Can be "ugly" IF tests comprehensive
+4. **Validation**: Tests + logs + manual inspection
+5. **Refactoring**: Optional, enabled by tests
+6. **Deployment**: Core case working + tests passing
+
+**Remember**: Tests are your license to write imperfect code. Without tests, perfection is required.