ado-git-repo-insights 3.3.0__tar.gz → 3.6.0__tar.gz

{ado_git_repo_insights-3.3.0 → ado_git_repo_insights-3.6.0}/.github/workflows/ci.yml

@@ -95,6 +95,18 @@ jobs:
 
           echo "✓ No CRLF detected in Unix-executed files"
 
+  # Guard: Verify UI bundle is in sync with extension/ui
+  ui-bundle-sync:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Check UI Bundle Synchronization
+        run: |
+          chmod +x scripts/check-ui-bundle-sync.sh
+          ./scripts/check-ui-bundle-sync.sh
+        shell: bash
+
   # Guard: Block task Major changes without explicit marker
   task-major-guard:
     runs-on: ubuntu-latest
@@ -218,7 +230,35 @@ jobs:
       - name: Validate Test Results (Python)
         if: always() && matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11'
         run: |
-          # Robust test validation using JUnit XML parsing
+          # Two-phase validation for robust CI diagnostics
+          # Phase 1: Check if pytest produced output at all
+          if [ ! -f test-results.xml ]; then
+            echo "::error::test-results.xml not found - pytest likely failed to start"
+            echo ""
+            echo "================================================================================"
+            echo "DIAGNOSTIC: pytest did not produce JUnit XML output"
+            echo "================================================================================"
+            echo ""
+            echo "Common causes:"
+            echo "  1. Import error in test file or source code"
+            echo "  2. Syntax error in Python files"
+            echo "  3. Missing dependency"
+            echo "  4. Test collection failed"
+            echo ""
+            echo "Check the 'Run Tests with Coverage' step above for the actual error."
+            echo ""
+            if [ -f pytest_output.txt ]; then
+              echo "================================================================================"
+              echo "Last 50 lines of pytest output:"
+              echo "================================================================================"
+              tail -50 pytest_output.txt
+            fi
+            echo ""
+            echo "================================================================================"
+            exit 1
+          fi
+
+          # Phase 2: Validate test results using JUnit XML parsing
           # MIN_COLLECTED: Minimum expected tests (update when adding/removing tests)
           # MAX_SKIPS: Maximum allowed skipped tests (0 = no skips allowed in CI)
           python .github/scripts/validate-test-results.py \
@@ -384,7 +424,35 @@ jobs:
       - name: Validate Test Results (Extension)
         if: always()
         run: |
-          # Robust test validation using JUnit XML parsing
+          # Two-phase validation for robust CI diagnostics
+          # Phase 1: Check if Jest produced output at all
+          if [ ! -f extension/test-results.xml ]; then
+            echo "::error::extension/test-results.xml not found - Jest likely failed to start"
+            echo ""
+            echo "================================================================================"
+            echo "DIAGNOSTIC: Jest did not produce JUnit XML output"
+            echo "================================================================================"
+            echo ""
+            echo "Common causes:"
+            echo "  1. Syntax error in test file or source code"
+            echo "  2. Missing npm dependency"
+            echo "  3. Jest configuration issue"
+            echo "  4. Test collection failed"
+            echo ""
+            echo "Check the 'Run Extension UI Tests' step above for the actual error."
+            echo ""
+            if [ -f extension/npm_test_output.txt ]; then
+              echo "================================================================================"
+              echo "Last 50 lines of Jest output:"
+              echo "================================================================================"
+              tail -50 extension/npm_test_output.txt
+            fi
+            echo ""
+            echo "================================================================================"
+            exit 1
+          fi
+
+          # Phase 2: Validate test results using JUnit XML parsing
           # MIN_COLLECTED: Minimum expected tests (update when adding/removing tests)
           # MAX_SKIPS: Maximum allowed skipped tests (0 = no skips allowed in CI)
           python .github/scripts/validate-test-results.py \

{ado_git_repo_insights-3.3.0 → ado_git_repo_insights-3.6.0}/.gitignore

@@ -41,6 +41,7 @@ htmlcov/
 .cache
 nosetests.xml
 coverage.xml
+test-results.xml
 *.cover
 *.py,cover
 .hypothesis/

{ado_git_repo_insights-3.3.0 → ado_git_repo_insights-3.6.0}/CHANGELOG.md

@@ -1,3 +1,43 @@
+# [3.6.0](https://github.com/oddessentials/ado-git-repo-insights/compare/v3.5.1...v3.6.0) (2026-01-19)
+
+
+### Bug Fixes
+
+* **ci:** add two-phase test validation for robust diagnostics ([ea1585a](https://github.com/oddessentials/ado-git-repo-insights/commit/ea1585ae173b8aa68640e6bc70036cf128e50b36))
+* **tests:** add last_updated column to teams test fixtures ([f376e6f](https://github.com/oddessentials/ado-git-repo-insights/commit/f376e6f63fa2952f49bb45e6adacf0c483ebb792))
+
+
+### Features
+
+* **aggregators:** implement by_team dimension slices (Phase 7.2) ([02c0728](https://github.com/oddessentials/ado-git-repo-insights/commit/02c07284a50c150c99fb5591a03da25771c484fd))
+* **ci:** add UI bundle sync verification (Phase 7.1) ([0309a5b](https://github.com/oddessentials/ado-git-repo-insights/commit/0309a5b9ac36435f4679892555e04d544b8a3fd2))
+
+## [3.5.1](https://github.com/oddessentials/ado-git-repo-insights/compare/v3.5.0...v3.5.1) (2026-01-19)
+
+
+### Bug Fixes
+
+* replace ui_bundle symlink with actual files for pip packaging ([6cb3504](https://github.com/oddessentials/ado-git-repo-insights/commit/6cb35048f21805e9cb28bbc47007faeb4eb0bc62))
+
+# [3.5.0](https://github.com/oddessentials/ado-git-repo-insights/compare/v3.4.0...v3.5.0) (2026-01-19)
+
+
+### Bug Fixes
+
+* use relative symlink for ui_bundle (CI compatibility) ([2d27fcd](https://github.com/oddessentials/ado-git-repo-insights/commit/2d27fcd832b45ffc2f74282a9da2453b05d35a76))
+
+
+### Features
+
+* **phase6:** add local dashboard and build-aggregates commands ([3233dcd](https://github.com/oddessentials/ado-git-repo-insights/commit/3233dcde7447ec48f71fa7c19b903ca1501e58be))
+
+# [3.4.0](https://github.com/oddessentials/ado-git-repo-insights/compare/v3.3.0...v3.4.0) (2026-01-19)
+
+
+### Features
+
+* **dashboard:** fix reviewer count bug and implement client-side filtering ([4f0526e](https://github.com/oddessentials/ado-git-repo-insights/commit/4f0526e62ae6ea18372ff7a0adb58fb9cd6b8fd5))
+
 # [3.3.0](https://github.com/oddessentials/ado-git-repo-insights/compare/v3.2.0...v3.3.0) (2026-01-18)
 
 

ado_git_repo_insights-3.6.0/CONTRIBUTING.md

@@ -0,0 +1,156 @@
+# Contributing to ADO Git Repo Insights
+
+Thank you for your interest in contributing! This document covers the essential guidelines for contributing to this project.
+
+## Development Setup
+
+1. Clone the repository
+2. Install Python dependencies:
+   ```bash
+   pip install -e .[dev]
+   ```
+3. Install Node.js dependencies for extension development:
+   ```bash
+   cd extension && npm ci
+   ```
+4. Install pre-commit hooks:
+   ```bash
+   pip install pre-commit
+   pre-commit install
+   ```
+
+## Running Tests
+
+### Python Tests
+```bash
+pytest
+```
+
+### Extension Tests
+```bash
+cd extension && npm test
+```
+
+---
+
+## UI Bundle Synchronization (IMPORTANT)
+
+The dashboard UI files exist in **two locations** that must stay synchronized:
+
+| Location | Purpose |
+|----------|---------|
+| `extension/ui/` | Source of truth for Azure DevOps extension |
+| `src/ado_git_repo_insights/ui_bundle/` | Copy for Python pip package |
+
+### Why Two Locations?
+
+- **Symlinks don't work with pip packages**: When building Python wheels with setuptools, symlinks are not preserved. The wheel would contain broken symlinks instead of actual files.
+- **The `ado-insights dashboard` command requires bundled UI files**: When users install via `pip install ado-git-repo-insights`, the UI files must be physically present in the package.
+
+### Synchronization Process
+
+**After modifying any files in `extension/ui/`**, you must sync to `ui_bundle/`:
+
+```bash
+cp -r extension/ui/* src/ado_git_repo_insights/ui_bundle/
+```
+
+Then commit both locations together.
+
+### CI Enforcement
+
+The `ui-bundle-sync` CI job automatically verifies synchronization on every PR. If the directories are out of sync, the job will fail and display:
+- A patch-format diff showing the differences
+- Instructions on how to fix the issue
+
+### Files Ignored During Sync Check
+
+The following file patterns are ignored during synchronization checks:
+- `*.map` (source maps)
+- `.DS_Store` (macOS metadata)
+- `*.swp`, `*~`, `*.bak` (editor backup files)
+
+---
+
+## Commit Message Format
+
+This project uses [Conventional Commits](https://www.conventionalcommits.org/) for semantic versioning:
+
+```
+<type>(<scope>): <description>
+
+[optional body]
+
+[optional footer(s)]
+```
+
+### Types
+- `feat`: New feature (triggers minor version bump)
+- `fix`: Bug fix (triggers patch version bump)
+- `docs`: Documentation only
+- `test`: Adding or updating tests
+- `chore`: Maintenance tasks
+- `refactor`: Code changes that neither fix bugs nor add features
+- `perf`: Performance improvements
+- `ci`: CI/CD changes
+
+### Breaking Changes
+
+For breaking changes, add `BREAKING CHANGE:` in the commit body or append `!` after the type:
+
+```
+feat(api)!: change response format
+
+BREAKING CHANGE: The API now returns dates in ISO 8601 format.
+```
+
+### Task Version Changes
+
+Changes to the Azure DevOps task Major version require special approval. Include `BREAKING TASK CHANGE:` in the PR title or commit message to acknowledge the breaking change.
+
+---
+
+## Pull Request Guidelines
+
+1. **Create a feature branch** from `main`
+2. **Write tests** for new functionality
+3. **Run the full test suite** before submitting
+4. **Keep PRs focused** - one feature or fix per PR
+5. **Update documentation** if behavior changes
+
+### CI Checks
+
+All PRs must pass:
+- Secret scanning (gitleaks)
+- Line ending checks (no CRLF in Unix-executed files)
+- UI bundle synchronization check
+- Python tests (matrix: 3 OSes × 3 Python versions)
+- Extension tests
+- Pre-commit hooks (ruff, formatting)
+
+---
+
+## Architecture Notes
+
+### Dataset Contract
+
+The dataset format is documented in `docs/dataset-contract.md`. Changes to the dataset schema require:
+1. Version bump in manifest
+2. Update to schema documentation
+3. Backward compatibility consideration
+
+### ML Features
+
+ML features are optional and gated behind the `[ml]` extra:
+```bash
+pip install ado-git-repo-insights[ml]
+```
+
+The base package must function without ML dependencies. See `tests/unit/test_ml_cli_flags.py` for isolation tests.
+
+---
+
+## Questions?
+
+- Check existing issues for similar questions
+- Open a new issue with the `question` label

{ado_git_repo_insights-3.3.0/src/ado_git_repo_insights.egg-info → ado_git_repo_insights-3.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ado-git-repo-insights
-Version: 3.3.0
+Version: 3.6.0
 Summary: Extract Azure DevOps Pull Request metrics to SQLite and generate PowerBI-compatible CSVs.
 Author-email: "Odd Essentials, LLC" <admin@oddessentials.com>
 License: MIT

ado_git_repo_insights-3.6.0/VERSION

@@ -0,0 +1 @@
+3.6.0

ado_git_repo_insights-3.6.0/docs/DASHBOARD-HARDENING-PLAN.md

@@ -0,0 +1,280 @@
+# Dashboard Hardening & Enhancement Plan
+
+This document outlines the implementation plan for hardening existing dashboard features, fixing bugs, and preparing for future enhancements.
+
+---
+
+## Executive Summary
+
+### Critical Issues Identified
+
+1. **Reviewer Activity Bug (P0)**: `reviewers_count` is hardcoded to `0` in `aggregators.py:501`
+   - Data exists in the `reviewers` table but is never counted
+   - UI correctly renders the value but always shows "0"
+
+2. **Filters Not Working (P0)**: Dimension filters are populated but never applied
+   - Filter dropdowns work (populated from `dimensions.json`)
+   - Filter state is tracked (`currentFilters` object)
+   - **Missing**: Client-side filtering logic in `refreshMetrics()`
+   - **Missing**: Pre-computed dimension slices in weekly rollups
+
+3. **Test Coverage Gaps (P1)**: No tests for filtering or reviewer aggregation
+
+---
+
+## Bug Analysis
+
+### Bug #1: Reviewer Count Always Zero
+
+**Location**: `src/ado_git_repo_insights/transform/aggregators.py:501`
+
+```python
+reviewers_count=0,  # TODO: Add reviewer counting
+```
+
+**Root Cause**: The weekly rollup generation never joins with the `reviewers` table.
+
+**Fix Required**: Query unique reviewers per PR, then aggregate per week:
+```sql
+SELECT COUNT(DISTINCT r.user_id) as reviewers_count
+FROM reviewers r
+JOIN pull_requests pr ON r.pull_request_uid = pr.pull_request_uid
+WHERE pr.closed_date >= ? AND pr.closed_date <= ?
+```
+
+**Test Data Verification**: The `reviewers` table schema exists and is validated:
+- `database.py:117` validates `reviewers` table on connect
+- Row counts are tracked in `_get_row_counts()` (line 665)
+
+---
+
+### Bug #2: Filters Not Applied
+
+**Location**: `extension/ui/dashboard.js`
+
+**Current Behavior**:
+1. `populateFilterDropdowns()` correctly loads dimensions (line 1522)
+2. `handleFilterChange()` updates `currentFilters` state (line 1560)
+3. `refreshMetrics()` loads data **ignoring filters** (line 809)
+
+**Missing Logic**:
+1. **Client-side filtering**: After loading rollups, filter by selected repos/teams
+2. **Dimension-sliced data**: Aggregator should generate `by_repository` and `by_team` slices
+
+**Test Fixture Shows Expected Format** (`2026-W02.json`):
+```json
+{
+  "by_repository": {
+    "main-repo": { "pr_count": 22, "cycle_time_p50": 200.0 },
+    "secondary-repo": { "pr_count": 8, "cycle_time_p50": 360.0 }
+  },
+  "by_team": {
+    "Backend Team": { "pr_count": 18, "cycle_time_p50": 180.0 },
+    "Frontend Team": { "pr_count": 12, "cycle_time_p50": 300.0 }
+  }
+}
+```
+
+---
+
+## Implementation Plan
+
+### Phase 1: Fix Reviewer Count Bug (P0)
+
+**Tasks**:
+1. Modify `_generate_weekly_rollups()` in `aggregators.py`
+2. Add query to count distinct reviewers per week
+3. Add unit test for reviewer aggregation
+4. Verify in integration test
+
+**SQL Query**:
+```sql
+SELECT
+    strftime('%G-W%W', pr.closed_date) as week,
+    COUNT(DISTINCT r.user_id) as reviewers_count
+FROM reviewers r
+JOIN pull_requests pr ON r.pull_request_uid = pr.pull_request_uid
+WHERE pr.closed_date IS NOT NULL AND pr.status = 'completed'
+GROUP BY week
+```
+
+**Estimated Effort**: 1-2 hours
+
+---
+
+### Phase 2: Implement Client-Side Filtering (P0)
+
+**Tasks**:
+1. Add `filterRollups()` function to `dashboard.js`
+2. Modify `refreshMetrics()` to apply filters after loading
+3. Add dimension slicing to `aggregators.py` (optional for MVP)
+4. Add filter tests
+
+**Implementation Approach** (MVP - Client-Side):
+```javascript
+function filterRollups(rollups, filters) {
+    if (!filters.repos.length && !filters.teams.length) {
+        return rollups; // No filters, return all
+    }
+
+    // If rollups have by_repository/by_team slices, use those
+    // Otherwise, filter at aggregate level (less granular)
+    return rollups.map(rollup => {
+        let filtered = { ...rollup };
+        // Apply repo filter
+        if (filters.repos.length && rollup.by_repository) {
+            const repoData = filters.repos.map(r => rollup.by_repository[r]).filter(Boolean);
+            filtered.pr_count = repoData.reduce((sum, d) => sum + d.pr_count, 0);
+            // ... aggregate other metrics
+        }
+        return filtered;
+    });
+}
+```
+
+**Estimated Effort**: 2-3 hours
+
+---
+
+### Phase 3: Add Dimension-Sliced Data Generation (P1)
+
+**Tasks**:
+1. Modify `_generate_weekly_rollups()` to include `by_repository` slices
+2. Add `by_team` slices (requires team membership join)
+3. Update test fixtures
+4. Add contract tests for new fields
+
+**Query for Repository Slices**:
+```sql
+SELECT
+    strftime('%G-W%W', pr.closed_date) as week,
+    r.repository_name,
+    COUNT(*) as pr_count,
+    PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY pr.cycle_time_minutes) as cycle_time_p50
+FROM pull_requests pr
+JOIN repositories r ON pr.repository_id = r.repository_id
+WHERE pr.closed_date IS NOT NULL AND pr.status = 'completed'
+GROUP BY week, r.repository_name
+```
+
+**Estimated Effort**: 3-4 hours
+
+---
+
+### Phase 4: Enhanced Testing (P1)
+
+**New Tests Required**:
+
+1. **Unit Test**: `test_aggregator_reviewer_count.py`
+   - Verify reviewers are counted per week
+   - Verify unique counting (no duplicates)
+
+2. **Unit Test**: `test_aggregator_dimension_slices.py`
+   - Verify `by_repository` slices generated
+   - Verify `by_team` slices generated
+
+3. **Integration Test**: `test_dashboard_filtering.js`
+   - Verify filter state updates URL
+   - Verify metrics change when filters applied
+   - Verify filter chips display
+
+4. **Contract Test**: `test_weekly_rollup_schema.py`
+   - Add schema validation for new fields
+   - Backward compatibility check
+
+**Estimated Effort**: 2-3 hours
+
+---
+
+## Enhancement Ideas for Future Phases
+
+### Visual Enhancements
+
+1. **Interactive Chart Tooltips**: Already partially implemented, extend to all charts
+2. **Responsive Sparklines**: Already implemented, ensure consistent rendering
+3. **Loading States per Chart**: Show skeleton loaders during filter changes
+
+### Performance Optimizations
+
+1. **Lazy Loading Dimension Slices**: Only load repo/team data when filter selected
+2. **Caching Strategy**: Cache filtered results by filter combination
+3. **Virtual Scrolling**: For large dimension lists in filter dropdowns
+
+### Future Features (Phase 6+ Integration)
+
+1. **Local Dashboard Testing**:
+   - Download aggregates artifact from pipeline
+   - Serve locally for rapid iteration
+   - Validate compatibility between extension and local versions
+
+2. **Schema Compatibility Layer**:
+   ```javascript
+   // version-adapter.js
+   function adaptRollup(rollup, targetVersion) {
+       if (rollup.schema_version < targetVersion) {
+           // Add missing fields with defaults
+           return { ...rollup, by_repository: {}, by_team: {} };
+       }
+       return rollup;
+   }
+   ```
+
+3. **Dashboard Experimentation Framework**:
+   - A/B test new visualizations locally
+   - Validate with real data before deploying
+   - Share test configurations
+
+---
+
+## Success Criteria
+
+After implementation:
+
+- [ ] Reviewer Activity chart shows real data (not 0)
+- [ ] Repository filter changes displayed metrics
+- [ ] Team filter changes displayed metrics
+- [ ] Filter combinations persist in URL
+- [ ] All existing tests pass
+- [ ] New tests cover filtering and reviewer aggregation
+- [ ] No regressions in cycle time or throughput calculations
+
+---
+
+## Risk Assessment
+
+| Risk | Impact | Mitigation |
+|------|--------|------------|
+| Breaking existing data contract | High | Add backward compat tests, version bump |
+| Performance degradation with slices | Medium | Lazy load slices, cache aggressively |
+| Filter state sync issues | Low | Use URL as source of truth |
+
+---
+
+## Appendix: Current Data Flow
+
+```
+┌─────────────────┐     ┌──────────────────┐     ┌────────────────┐
+│ ADO API         │ ──▶ │ SQLite Database  │ ──▶ │ Aggregates     │
+│ (PRs, Reviewers)│     │ (pull_requests,  │     │ (weekly_rollups│
+│                 │     │  reviewers)      │     │  dimensions)   │
+└─────────────────┘     └──────────────────┘     └───────┬────────┘
+                                                        │
+                        ┌───────────────────────────────▼────────┐
+                        │             Dashboard UI               │
+                        │  ┌─────────────┐  ┌─────────────────┐  │
+                        │  │ Filters     │  │ Metrics Display │  │
+                        │  │ (NOT WORKING)│ │ (reviewers=0)   │  │
+                        │  └─────────────┘  └─────────────────┘  │
+                        └────────────────────────────────────────┘
+```
+
+**After Fix**:
+```
+                        ┌────────────────────────────────────────┐
+                        │             Dashboard UI               │
+                        │  ┌─────────────┐  ┌─────────────────┐  │
+                        │  │ Filters     │──│ Metrics Display │  │
+                        │  │ (WORKING)   │  │ (reviewers=N)   │  │
+                        │  └─────────────┘  └─────────────────┘  │
+                        └────────────────────────────────────────┘
+```