gitflow-analytics 3.3.0__tar.gz → 3.5.2__tar.gz

{gitflow_analytics-3.3.0 → gitflow_analytics-3.5.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gitflow-analytics
-Version: 3.3.0
+Version: 3.5.2
 Summary: Analyze Git repositories for developer productivity insights
 Author-email: Bob Matyas <bobmatnyc@gmail.com>
 License: MIT

gitflow_analytics-3.5.2/docs/REFACTOR.md

@@ -0,0 +1,470 @@
+# GitFlow Analytics Refactoring Guide
+
+**Last Updated**: 2025-10-06
+**Current Code Quality**: B- (improving from C+)
+**Target Code Quality**: A-
+
+## Executive Summary
+
+This document tracks ongoing refactoring efforts to improve code quality, maintainability, and safety in the GitFlow Analytics project. We follow an incremental, risk-managed approach with comprehensive testing at each phase.
+
+---
+
+## Refactoring Phases
+
+### Phase 1: Critical Safety Fixes ✅ COMPLETED
+
+**Objective**: Eliminate critical safety issues and add foundational type hints
+
+**Completed Items:**
+1. **Fixed 5 Bare Exception Handlers** (CRITICAL)
+   - `subprocess_git.py`: 2 occurrences → specific TimeoutExpired, OSError handlers
+   - `data_fetcher.py`: 2 occurrences → proper exception types with logging
+   - `progress_display.py`: 1 occurrence → ImportError handling
+   - **Impact**: Prevents silent failures, allows Ctrl+C interruption
+
+2. **Added Type Hints to Critical Paths** (HIGH)
+   - `GitDataFetcher.__init__`: Full parameter type annotations
+   - Helper functions: Return type annotations (fetch_branch_commits, get_diff_output)
+   - CLI helpers: format_option_help with proper types
+   - **Impact**: Better IDE support, early type error detection
+
+3. **Enhanced Error Logging** (MEDIUM)
+   - Include repository paths in error messages
+   - Log cleanup failures at appropriate levels (debug/warning)
+   - Provide actionable debugging information
+   - **Impact**: Easier troubleshooting, better debugging experience
+
+**Commits:**
+- `bbfb375` - refactor: fix bare exception handlers and add type hints
+
+**Testing:**
+- ✅ 201/201 tests passing
+- ✅ Black formatting applied
+- ✅ No new linting issues
+- ✅ Zero breaking changes
+
+---
+
+### Phase 2: Constants Extraction ✅ COMPLETED
+
+**Objective**: Eliminate magic numbers and centralize configuration values
+
+**Completed Items:**
+1. **Created `src/gitflow_analytics/constants.py`** (NEW FILE)
+   - `Timeouts`: 11 timeout constants (GIT_FETCH=30, GIT_BRANCH_ITERATION=15, etc.)
+   - `BatchSizes`: 5 batch size constants (COMMIT_STORAGE=1000, TICKET_FETCH=50, etc.)
+   - `CacheTTL`: 2 TTL constants (ONE_WEEK_HOURS=168)
+   - `Thresholds`: 2 threshold constants (CACHE_HIT_RATE_GOOD=50)
+   - `Estimations`: 2 estimation constants
+
+2. **Updated 3 Core Files**
+   - `data_fetcher.py`: 13 magic numbers replaced
+   - `git_timeout_wrapper.py`: 4 timeout values now use Timeouts class
+   - `cache.py`: 3 values replaced (TTL, batch size, threshold)
+
+**Commits:**
+- `f83a6bd` - refactor: extract magic numbers to centralized constants module
+
+**Benefits:**
+- ✅ All config values in one location
+- ✅ Descriptive names explain purpose
+- ✅ Easy global adjustments
+- ✅ Type safety for all constants
+
+**Testing:**
+- ✅ All tests passing
+- ✅ Constants import correctly
+- ✅ No behavioral changes
+
+---
+
+### Phase 3: Type System Enhancement 🔄 IN PROGRESS
+
+**Objective**: Add comprehensive type hints and create typed data structures
+
+**Planned Items:**
+
+1. **Create TypedDict for CommitData** (HIGH PRIORITY)
+   ```python
+   from typing import TypedDict
+   from datetime import datetime
+
+   class CommitData(TypedDict, total=False):
+       """Structure for commit data dictionaries."""
+       hash: str
+       commit_hash_short: str
+       message: str
+       author_name: str
+       author_email: str
+       timestamp: datetime
+       branch: str
+       project_key: str
+       repo_path: str
+       is_merge: bool
+       files_changed: list[str]
+       files_changed_count: int
+       lines_added: int
+       lines_deleted: int
+       ticket_references: list[str]
+       story_points: Optional[int]
+   ```
+
+2. **Add Type Hints to Cache Methods** (MEDIUM PRIORITY)
+   - `cache.py::get_cached_commit()` → `Optional[CachedCommit]`
+   - `cache.py::cache_commit()` → `None`
+   - `cache.py::get_cache_stats()` → `dict[str, Any]`
+
+3. **Add Type Hints to Remaining Public APIs** (MEDIUM PRIORITY)
+   - Focus on public methods first
+   - Add return types to all `__init__` methods
+   - Use `from __future__ import annotations` for forward references
+
+**Estimated Effort**: 2-3 days
+**Risk Level**: LOW (additive changes only)
+
+---
+
+### Phase 4: Architecture Improvements 📋 PLANNED
+
+**Objective**: Reduce complexity and improve code organization
+
+**High Priority Items:**
+
+1. **Split `cli.py` into Modules** (CRITICAL - DEFERRED)
+   - **Current**: 5,365 lines in single file
+   - **Target**: Modular structure with command modules
+   ```
+   src/gitflow_analytics/cli/
+     __init__.py           # Main CLI group
+     commands/
+       __init__.py
+       analyze_command.py  # analyze subcommand
+       fetch_command.py    # fetch subcommand
+       identity_commands.py # identity management
+       cache_commands.py   # cache operations
+       training_commands.py # ML training
+       tui_command.py      # TUI launcher
+     error_handlers.py     # ImprovedErrorHandler class
+     formatters.py         # RichHelpFormatter class
+     utils.py              # Helper functions
+   ```
+   - **Estimated Effort**: 5-8 days
+   - **Risk Level**: HIGH (requires comprehensive testing)
+
+2. **Extract `analyze()` into AnalysisPipeline** (CRITICAL - DEFERRED)
+   - **Current**: 3,446-line mega-function with complexity 525
+   - **Target**: Pipeline with discrete stages
+   ```python
+   class AnalysisPipeline:
+       def run(self, weeks, options):
+           self._validate_configuration()
+           self._authenticate_services()
+           if options.clear_cache:
+               self._clear_cache()
+           raw_data = self._fetch_data(weeks)
+           analysis_results = self._analyze_data(raw_data)
+           self._generate_reports(analysis_results, options)
+   ```
+   - **Estimated Effort**: 5-8 days
+   - **Risk Level**: HIGH (core business logic)
+
+3. **Consolidate Progress Reporting** (HIGH PRIORITY)
+   - **Current**: 3 different progress systems
+   - **Target**: Single unified `ProgressReporter` interface
+   - **Estimated Effort**: 1-2 days
+   - **Risk Level**: MEDIUM
+
+**Medium Priority Items:**
+
+4. **Simplify Glob Pattern Matching** (161 lines → use pathspec library)
+5. **Centralize Environment Variables** (Use EnvironmentConfig dataclass)
+6. **Fix Thread-Local Storage Pattern** (Use explicit context managers)
+7. **Improve Cache Configuration Hash** (Deterministic serialization)
+
+---
+
+## Code Quality Metrics
+
+### Current Status (as of 2025-10-06)
+
+| Metric | Current | Target | Status |
+|--------|---------|--------|--------|
+| **Code Quality Grade** | B- | A- | 🟡 Improving |
+| **Lines of Code** | 68,175 | <60,000 | 🔴 Needs work |
+| **Largest File** | 5,365 (cli.py) | <2,500 | 🔴 Critical |
+| **Bare Exceptions** | 0 | 0 | 🟢 Excellent |
+| **High Complexity Functions** | 124 (8.1%) | <5% | 🟡 Improving |
+| **Long Functions (>50 lines)** | 314 (20.5%) | <15% | 🟡 Improving |
+| **Large Files (>1000 lines)** | 15 (11.4%) | <5% | 🔴 Needs work |
+| **Functions with >5 params** | 61 (4.0%) | <3% | 🟡 Improving |
+| **Large Classes (>500 lines)** | 33 (13.1%) | <10% | 🟡 Improving |
+| **Type Hint Coverage** | ~50% | >90% | 🟡 Improving |
+| **Test Coverage** | 30% | >80% | 🔴 Needs work |
+
+### Complexity Distribution
+
+**Top 5 Most Complex Functions:**
+1. `cli.py::analyze()` - Complexity: 525 (Target: <10)
+2. `data_fetcher.py::fetch_repository_data()` - Complexity: ~20
+3. `cache.py::bulk_store_commits()` - Complexity: ~15
+4. Multiple functions: 10-15 complexity range
+
+---
+
+## Critical Issues Remaining
+
+### Priority 1: God Classes
+
+1. **cli.py** (5,365 lines)
+   - Contains 19+ distinct functions
+   - Mixed concerns: commands, business logic, error handling
+   - Single `analyze()` function is 3,446 lines
+   - **Recommendation**: DEFER until Phase 4
+
+2. **GitAnalysisCache** (1,672 lines in cache.py)
+   - Too many responsibilities
+   - **Recommendation**: Extract query builders, validators
+
+3. **GitDataFetcher** (2,224 lines in data_fetcher.py)
+   - Complex fetch/store/analyze logic
+   - **Recommendation**: Extract separate classes for fetch/store
+
+### Priority 2: Missing Abstractions
+
+1. **No Service Layer**
+   - Business logic mixed with presentation
+   - **Recommendation**: Extract service objects
+
+2. **No Repository Pattern**
+   - Direct database access everywhere
+   - **Recommendation**: Create repository classes
+
+3. **No Domain Events**
+   - Tight coupling between modules
+   - **Recommendation**: Event-driven architecture for decoupling
+
+---
+
+## Testing Strategy
+
+### For Each Refactoring Phase:
+
+1. **Characterization Tests**
+   - Add tests to lock in current behavior BEFORE changes
+   - Document expected behavior explicitly
+
+2. **Mutation Testing**
+   - Verify test coverage is adequate
+   - Use `mutmut` or similar tools
+
+3. **Integration Tests**
+   - Run full test suite after each phase
+   - Test with real EWTN dataset (97 repos)
+
+4. **Performance Regression Testing**
+   - Benchmark cache operations
+   - Monitor analysis time for large repos
+
+### Test Coverage Goals
+
+| Component | Current | Target | Priority |
+|-----------|---------|--------|----------|
+| Core (cache, fetcher, analyzer) | 30% | 80% | HIGH |
+| CLI | ~10% | 60% | MEDIUM |
+| Integrations | 25% | 70% | HIGH |
+| Reports | 35% | 70% | MEDIUM |
+| Utils | 40% | 80% | LOW |
+
+---
+
+## Refactoring Principles
+
+### Do's ✅
+
+1. **Make Small, Incremental Changes**
+   - Each commit should be independently reviewable
+   - Each phase should be independently testable
+
+2. **Test Before and After**
+   - Ensure all tests pass before starting
+   - Verify tests still pass after changes
+
+3. **Maintain Backward Compatibility**
+   - Don't break public APIs
+   - Don't change database schemas without migrations
+
+4. **Document Changes**
+   - Update docstrings with type hints
+   - Add comments explaining complex logic
+   - Update CHANGELOG.md
+
+5. **Review Impact**
+   - Check for breaking changes
+   - Update dependent code
+   - Test edge cases
+
+### Don'ts ❌
+
+1. **Don't Mix Refactoring with Features**
+   - Refactoring commits should be pure refactorings
+   - Feature commits should be separate
+
+2. **Don't Skip Tests**
+   - Every refactoring must maintain test coverage
+   - Add tests if coverage decreases
+
+3. **Don't Rush Large Refactorings**
+   - High-risk changes need extensive testing
+   - Consider feature flags for gradual rollout
+
+4. **Don't Ignore Performance**
+   - Benchmark before and after
+   - Watch for N+1 queries or memory leaks
+
+5. **Don't Optimize Prematurely**
+   - Focus on correctness and maintainability first
+   - Optimize only when profiling shows bottlenecks
+
+---
+
+## Session History
+
+### 2025-10-06: Initial Refactoring Session
+
+**Completed:**
+- ✅ Phase 1: Critical safety fixes (bare exceptions, type hints)
+- ✅ Phase 2: Constants extraction
+- ✅ Fixed remote branch analysis bug
+- ✅ Added pre-flight git authentication
+
+**Commits:**
+1. `78e9c2d` - feat: add pre-flight git authentication and enhanced error reporting
+2. `4625cc6` - style: apply Black formatting and auto-fix Ruff linting issues
+3. `5ccca16` - fix: resolve remote branch analysis by preserving full branch references
+4. `bbfb375` - refactor: fix bare exception handlers and add type hints
+5. `f83a6bd` - refactor: extract magic numbers to centralized constants module
+
+**Impact:**
+- Code quality: C+ → B-
+- Type coverage: ~40% → ~50% (critical paths)
+- Magic numbers: Reduced by 20+
+- Bare exceptions: 5 → 0
+- Safety: Significantly improved
+
+---
+
+## Next Steps
+
+### Immediate (This Week)
+1. ✅ Complete Phase 2 constants extraction
+2. 🔄 Start Phase 3 type system enhancement
+3. 📋 Create TypedDict for CommitData
+4. 📋 Add type hints to cache methods
+
+### Short Term (This Month)
+1. 📋 Complete Phase 3 type system
+2. 📋 Consolidate progress reporting
+3. 📋 Extract environment configuration
+4. 📋 Improve glob pattern matching
+
+### Long Term (Next Quarter)
+1. 📋 Split cli.py into modules (Phase 4)
+2. 📋 Extract analyze() mega-function
+3. 📋 Implement service layer pattern
+4. 📋 Increase test coverage to 80%
+
+---
+
+## Resources
+
+### Tools Used
+- **Black**: Code formatting
+- **Ruff**: Linting and style checking
+- **mypy**: Type checking
+- **pytest**: Testing framework
+- **Code Analyzer Agent**: Pattern detection
+- **Python Engineer Agent**: Refactoring implementation
+
+### References
+- [Code Analysis Report](./code_analysis_report.md) - Comprehensive analysis findings
+- [CONTRIBUTING.md](../CONTRIBUTING.md) - Development guidelines
+- [STRUCTURE.md](./STRUCTURE.md) - Project structure documentation
+
+### Related Issues
+- Track refactoring progress via GitHub issues
+- Link TODOs to specific issues (e.g., `TODO(#123): Extract this class`)
+
+---
+
+## Appendix: Common Refactoring Patterns
+
+### Extract Method
+```python
+# Before
+def long_function():
+    # 100 lines of code
+    x = complex_calculation()
+    # more code
+
+# After
+def long_function():
+    x = _calculate_value()
+    # rest of code
+
+def _calculate_value():
+    return complex_calculation()
+```
+
+### Replace Magic Number with Constant
+```python
+# Before
+timeout = 30
+
+# After
+from ..constants import Timeouts
+timeout = Timeouts.GIT_FETCH
+```
+
+### Introduce Parameter Object
+```python
+# Before
+def func(param1, param2, param3, param4, param5):
+    pass
+
+# After
+@dataclass
+class FuncParams:
+    param1: str
+    param2: int
+    param3: bool
+    param4: Optional[str]
+    param5: list[str]
+
+def func(params: FuncParams):
+    pass
+```
+
+### Extract Class
+```python
+# Before
+class GodClass:
+    def method1(self): pass
+    def method2(self): pass
+    def method3(self): pass
+    # 50+ more methods
+
+# After
+class GodClass:
+    def __init__(self):
+        self.feature1 = Feature1Service()
+        self.feature2 = Feature2Service()
+
+class Feature1Service:
+    def method1(self): pass
+    def method2(self): pass
+```
+
+---
+
+**End of Document**