devforgeai 1.0.4 → 1.0.6

package/src/claude/scripts/devforgeai_cli/feedback/README_TEST_SPECS.md

@@ -0,0 +1,486 @@
+# Test Specifications for STORY-011 - Complete Test Suite
+
+**Generated:** 2025-11-10
+**Story:** STORY-011 Configuration Management System
+**Status:** Test Specifications Complete - Ready for Implementation
+
+---
+
+## Overview
+
+This directory contains comprehensive test specifications for 6 untested Python modules in the DevForgeAI feedback configuration system:
+
+1. **config_manager.py** (161 statements) → 68 tests
+2. **hot_reload.py** (99 statements) → 55 tests
+3. **config_models.py** (85 statements) → 59 tests
+4. **skip_tracker.py** (78 statements) → 51 tests
+5. **config_schema.py** (4 statements) → 15 tests
+6. **config_defaults.py** (8 statements) → 15 tests
+
+**Total:** 263+ tests targeting 435 untested statements (95%+ coverage)
+
+---
+
+## Documents in This Directory
+
+### 1. TEST_SPECIFICATIONS.md (Complete)
+**Size:** 11,000+ lines
+**Purpose:** Detailed test method specifications for all 263 tests
+**Organization:** By module with test class, fixture, assertion details
+
+**Sections:**
+- Module-by-module breakdown
+- Test class definitions
+- Test method names and descriptions
+- Fixture requirements
+- Key assertions
+- Coverage targets
+- Edge cases covered
+
+**Use This When:**
+- Implementing individual test methods
+- Need detailed assertion patterns
+- Looking up specific test requirements
+- Verifying coverage targets
+
+---
+
+### 2. TEST_IMPLEMENTATION_GUIDE.md (Quick Reference)
+**Size:** 800+ lines
+**Purpose:** Implementation roadmap, timeline, and quick reference
+**Organization:** By implementation phase with time estimates
+
+**Sections:**
+- Summary statistics (263 tests, 435 statements, 95%+ target)
+- Module-by-module breakdown with priorities
+- Implementation roadmap (4 phases, 35-41 hours)
+- Pytest configuration
+- Testing patterns (thread safety, file I/O, parametrization)
+- Running tests
+- AC-to-test mapping table
+- Success criteria checklist
+
+**Use This When:**
+- Planning implementation sprints
+- Estimating time and effort
+- Need quick reference for phase breakdown
+- Want to see testing patterns
+- Creating pytest configuration
+
+---
+
+### 3. AC_TO_TEST_MAPPING.md (Requirements Traceability)
+**Size:** 800+ lines
+**Purpose:** Map acceptance criteria (AC-1 through AC-9) to test methods
+**Organization:** By acceptance criterion with requirement, tests, assertions
+
+**Sections:**
+- AC-1: YAML loading (7 tests)
+- AC-2: Master enable/disable (5 tests)
+- AC-3: Trigger modes (8 tests)
+- AC-4: Conversation settings (9 tests)
+- AC-5: Skip tracking (14 tests)
+- AC-6: Template preferences (11 tests)
+- AC-7: Invalid config errors (16 tests)
+- AC-8: Default config (15 tests)
+- AC-9: Hot-reload (14 tests)
+- Summary matrix
+- Test execution strategy
+- Validation checklist
+
+**Use This When:**
+- Verifying AC compliance
+- Running tests for specific feature
+- Creating validation checklists
+- Ensuring traceability
+- Documentation/audit purposes
+
+---
+
+## Quick Start
+
+### 1. Read Overview (15 min)
+```bash
+# Review key documents
+- This README_TEST_SPECS.md (5 min)
+- TEST_IMPLEMENTATION_GUIDE.md (10 min)
+```
+
+### 2. Review Detailed Specs (1 hour)
+```bash
+# Read the detailed test specifications
+- TEST_SPECIFICATIONS.md sections for your module
+```
+
+### 3. Setup Environment (30 min)
+```bash
+# Create conftest.py with shared fixtures
+# Setup pytest.ini configuration
+# Install dependencies: pytest, pytest-cov
+```
+
+### 4. Implement Tests (16-20 hours)
+```bash
+# Phase 1: Data models (4-5 hours)
+pytest test_config_models.py -v
+pytest test_config_defaults.py -v
+pytest test_config_schema.py -v
+
+# Phase 2: Configuration management (10-12 hours)
+pytest test_config_manager.py -v
+
+# Phase 3: Hot-reload (11-13 hours)
+pytest test_hot_reload.py -v
+
+# Phase 4: Skip tracking (10-11 hours)
+pytest test_skip_tracker.py -v
+```
+
+### 5. Validate Coverage (1 hour)
+```bash
+# Run all tests with coverage
+pytest --cov --cov-report=html
+
+# Check coverage is >95% per module
+# Generate coverage report: htmlcov/index.html
+```
+
+---
+
+## Test Specification Format
+
+### Typical Test Method Entry
+
+```markdown
+#### test_method_name
+- **Purpose:** What behavior is being tested
+- **Setup:** How test is initialized
+- **Assertions:** What is verified (bullet list)
+- **Coverage:** Which lines of code tested (e.g., Lines 34-52)
+```
+
+### Example Test Method (from TEST_SPECIFICATIONS.md)
+
+```markdown
+#### test_load_valid_yaml_file
+- **Purpose:** Load valid YAML configuration file
+- **Setup:** Create valid YAML with all sections
+- **Assertions:**
+  - load_configuration() returns FeedbackConfiguration
+  - All sections populated correctly
+  - No errors logged
+- **Coverage:** Lines 109-136, 226-248
+```
+
+---
+
+## Module Coverage Targets
+
+| Module | Statements | Target Tests | Coverage % |
+|--------|-----------|-------------|-----------|
+| config_manager.py | 161 | 68 | 95%+ |
+| hot_reload.py | 99 | 55 | 95%+ |
+| config_models.py | 85 | 59 | 95%+ |
+| skip_tracker.py | 78 | 51 | 95%+ |
+| config_schema.py | 4 | 15 | 100% |
+| config_defaults.py | 8 | 15 | 100% |
+| **TOTAL** | **435** | **263+** | **95%+** |
+
+---
+
+## Acceptance Criteria Coverage
+
+All 9 acceptance criteria fully specified with tests:
+
+| AC | Feature | Tests | Status |
+|----|---------|-------|--------|
+| 1 | YAML loads | 7 | ✓ Specified |
+| 2 | Enable/disable | 5 | ✓ Specified |
+| 3 | Trigger modes | 8 | ✓ Specified |
+| 4 | Conversation settings | 9 | ✓ Specified |
+| 5 | Skip tracking | 14 | ✓ Specified |
+| 6 | Template preferences | 11 | ✓ Specified |
+| 7 | Invalid config errors | 16 | ✓ Specified |
+| 8 | Default config | 15 | ✓ Specified |
+| 9 | Hot-reload | 14 | ✓ Specified |
+
+---
+
+## Key Features of Specifications
+
+### 1. Complete Coverage
+- Every acceptance criterion has corresponding tests
+- Every statement of untested code targeted
+- Edge cases and error paths included
+- Thread safety validated for concurrent operations
+- File I/O and persistence covered
+
+### 2. Implementation-Ready
+- Each test method has clear purpose statement
+- Setup instructions provided
+- Assertion examples shown
+- Fixture requirements listed
+- Coverage targets identified
+
+### 3. Quality Standards
+- AAA pattern (Arrange, Act, Assert)
+- Descriptive test names
+- Independent tests (no ordering dependencies)
+- Proper fixture cleanup
+- Parametrized tests where applicable
+
+### 4. Traceability
+- AC-to-test mapping document
+- Line-of-code coverage targets
+- Module-specific test classes
+- Integration tests for workflows
+
+---
+
+## Pytest Configuration
+
+### Basic pytest.ini
+```ini
+[pytest]
+testpaths = .claude/scripts/devforgeai_cli/feedback
+python_files = test_*.py
+python_classes = Test*
+python_functions = test_*
+addopts = -v --tb=short --cov --cov-report=html
+```
+
+### Recommended conftest.py Fixtures
+```python
+@pytest.fixture
+def temp_config_dir(tmp_path):
+    """Temporary config directory"""
+    config_dir = tmp_path / "devforgeai" / "config"
+    config_dir.mkdir(parents=True)
+    return config_dir
+
+@pytest.fixture
+def temp_logs_dir(tmp_path):
+    """Temporary logs directory"""
+    logs_dir = tmp_path / "devforgeai" / "logs"
+    logs_dir.mkdir(parents=True)
+    return logs_dir
+
+@pytest.fixture
+def config_manager_instance(temp_config_dir, temp_logs_dir):
+    """ConfigurationManager test instance"""
+    reset_config_manager()
+    manager = ConfigurationManager(
+        config_file_path=temp_config_dir / "feedback.yaml",
+        logs_dir=temp_logs_dir,
+        enable_hot_reload=False
+    )
+    yield manager
+    manager.shutdown()
+    reset_config_manager()
+```
+
+---
+
+## Implementation Phases
+
+### Phase 1: Data Models (4-5 hours)
+**Priority:** HIGH (foundational)
+```
+test_config_models.py     59 tests
+test_config_defaults.py   15 tests
+test_config_schema.py     15 tests
+```
+
+### Phase 2: Configuration Management (10-12 hours)
+**Priority:** CRITICAL (core functionality)
+```
+test_config_manager.py    68 tests
+```
+
+### Phase 3: Hot-Reload System (11-13 hours)
+**Priority:** HIGH (AC-9)
+```
+test_hot_reload.py        55 tests
+```
+
+### Phase 4: Skip Tracking (10-11 hours)
+**Priority:** MEDIUM (AC-5)
+```
+test_skip_tracker.py      51 tests
+```
+
+---
+
+## Running Tests
+
+### All Tests
+```bash
+pytest -v --cov
+```
+
+### Single Module
+```bash
+pytest test_config_models.py -v
+```
+
+### Specific Test Class
+```bash
+pytest test_config_manager.py::TestConfigurationValidation -v
+```
+
+### With Coverage Report
+```bash
+pytest --cov --cov-report=html
+open htmlcov/index.html
+```
+
+### By Acceptance Criterion
+```bash
+# AC-5: Skip tracking
+pytest test_skip_tracker.py -v
+
+# AC-9: Hot-reload
+pytest test_hot_reload.py -v
+```
+
+---
+
+## Success Criteria
+
+After implementation, verify:
+
+1. **All Tests Pass**
+   - [ ] 263+ tests implemented
+   - [ ] 100% test pass rate
+   - [ ] No failures or errors
+
+2. **Coverage Targets**
+   - [ ] config_manager.py: 95%+ (153/161)
+   - [ ] hot_reload.py: 95%+ (94/99)
+   - [ ] config_models.py: 95%+ (81/85)
+   - [ ] skip_tracker.py: 95%+ (74/78)
+   - [ ] config_schema.py: 100% (4/4)
+   - [ ] config_defaults.py: 100% (8/8)
+   - [ ] **Overall: 95%+ (414/435)**
+
+3. **Acceptance Criteria**
+   - [ ] AC-1: YAML loading ✓
+   - [ ] AC-2: Enable/disable ✓
+   - [ ] AC-3: Trigger modes ✓
+   - [ ] AC-4: Conversation settings ✓
+   - [ ] AC-5: Skip tracking ✓
+   - [ ] AC-6: Template preferences ✓
+   - [ ] AC-7: Invalid config errors ✓
+   - [ ] AC-8: Default config ✓
+   - [ ] AC-9: Hot-reload ✓
+
+4. **Quality Standards**
+   - [ ] AAA pattern applied
+   - [ ] Tests independent (no ordering)
+   - [ ] Thread safety validated
+   - [ ] Error paths tested
+   - [ ] Edge cases covered
+   - [ ] Fixtures cleaned up
+
+---
+
+## References
+
+### In This Directory
+- **TEST_SPECIFICATIONS.md** - Complete test method specifications (11,000+ lines)
+- **TEST_IMPLEMENTATION_GUIDE.md** - Implementation roadmap and quick reference
+- **AC_TO_TEST_MAPPING.md** - Acceptance criteria to test traceability
+
+### In DevForgeAI Repository
+- **devforgeai/specs/context/** - Architecture constraints
+- **.claude/skills/devforgeai-qa/** - QA automation patterns
+- **devforgeai/specs/Stories/STORY-011.story.md** - Original story file
+
+### External Resources
+- **Pytest Documentation:** https://docs.pytest.org/
+- **Python Threading:** https://docs.python.org/3/library/threading.html
+- **Pytest Fixtures:** https://docs.pytest.org/en/stable/fixture.html
+
+---
+
+## FAQ
+
+### Q: How many tests should I write per day?
+**A:** Aim for 30-40 tests/day for experienced developers (4-5 hours). Start with Phase 1 data model tests for quick wins.
+
+### Q: Can I run tests incrementally?
+**A:** Yes! Start with `test_config_models.py` (lowest complexity, fastest feedback). Run after each test class to verify progress.
+
+### Q: What if a test is too complex?
+**A:** Break it into smaller test methods. Each test should focus on one behavior. Look for parametrization opportunities.
+
+### Q: How do I handle timing issues in hot-reload tests?
+**A:** Use polling loops with small waits (0.1s intervals) and reasonable timeouts (5s max). See examples in TEST_SPECIFICATIONS.md.
+
+### Q: Should I implement conftest.py first?
+**A:** Yes. Create shared fixtures early (temp directories, mock callbacks). This makes test implementation much faster.
+
+---
+
+## Support & Troubleshooting
+
+### Test Isolation Issues
+**Problem:** Tests affect each other
+**Solution:** Ensure proper fixture cleanup (use `yield`, cleanup after)
+
+### Timing Issues
+**Problem:** Tests flaky due to timing
+**Solution:** Use `pytest-timeout` plugin, polling loops instead of fixed sleep
+
+### Import Errors
+**Problem:** Cannot import modules under test
+**Solution:** Ensure devforgeai_cli is in Python path, use PYTHONPATH environment variable
+
+### Coverage Not Increasing
+**Problem:** Tests not hitting expected lines
+**Solution:** Review test assertions, add print statements during development, use pytest -s flag
+
+---
+
+## Next Steps
+
+1. **Review this README** (15 min) ✓
+2. **Read TEST_IMPLEMENTATION_GUIDE.md** (15 min) ✓
+3. **Setup pytest environment** (30 min)
+4. **Create conftest.py** (30 min)
+5. **Implement Phase 1 tests** (4-5 hours)
+6. **Validate coverage** (1 hour)
+7. **Continue to Phase 2** (repeat)
+
+---
+
+## Document History
+
+| Version | Date | Author | Changes |
+|---------|------|--------|---------|
+| 1.0 | 2025-11-10 | Test Automator | Initial complete specifications |
+
+---
+
+## Summary
+
+This test specification package provides everything needed to implement 263+ tests for STORY-011 Configuration Management system:
+
+- ✅ **Complete specifications** for every test method
+- ✅ **Implementation roadmap** with time estimates
+- ✅ **Acceptance criteria traceability** to every test
+- ✅ **Code coverage targets** (95%+ per module)
+- ✅ **Testing patterns** (thread safety, file I/O, etc.)
+- ✅ **Pytest configuration** examples
+- ✅ **Success criteria** checklist
+
+**Status:** Ready for implementation
+**Estimated Effort:** 35-41 hours
+**Expected Coverage Gain:** 435 → 0 untested statements (435 statements → 95%+ coverage)
+**Target Completion:** Complete within 1 sprint (2 weeks)
+
+---
+
+**Generated by:** Test Automator (TDD-driven test generation)
+**For:** STORY-011 Configuration Management System
+**Date:** 2025-11-10